class RubyLLM::Batch

A Batch is a provider-side batch of chat completions: chats awaiting a response go in together, answers come back at batch prices, typically within hours. Persist the id, pick the batch back up from any process, and collect the messages once processing ends.

chats = documents.map do |doc|
  RubyLLM.chat(model: "claude-haiku-4-5").ask_later(doc.text)
end
batch = RubyLLM.batch(chats)
batch.id                # => "msgbatch_01EhcDuvb5XfWqcdJArbsfNX"
batch.refresh.complete? # => false, check back later
batch.messages          # the responses, in submission order

Attributes

chats [R]

The submitted Chat objects in order, or nil when the batch was loaded by id via ::find.

id [R]

The provider’s batch id. Persist it to load the batch again later from any process with ::find.

request_counts [R]

The provider-reported request tallies by state, or nil when the provider does not report them.

status [R]

The provider-reported status string, such as “in_progress”. Refreshed by refresh.

Public Class Methods

find (id, provider:, context: nil)

Source

# File lib/ruby_llm/batch.rb, line 81
def find(id, provider:, context: nil)
  raise ArgumentError, 'Provider must be specified to find a batch' unless provider

  config = context&.config || RubyLLM.config
  provider = Provider.resolve!(provider).new(config)
  raise Error, "#{provider.slug} doesn't support batch requests" unless provider.batches?

  new(provider:, **provider.find_batch(id))
end

Returns a Batch reflecting the provider’s current state for id. Use it to pick a batch back up from any process.

batch = RubyLLM::Batch.find("msgbatch_01EhcDuvb5XfWqcdJArbsfNX",
                            provider: :anthropic)
batch.complete? # => true

Pass context: to use a Context in place of the global configuration. Raises ArgumentError if provider is not given.

submit (chats)

Source

# File lib/ruby_llm/batch.rb, line 50
def submit(chats)
  chats = chats.is_a?(Chat) ? [chats] : Array(chats)
  chats = chats.map { |chat| chat.respond_to?(:to_llm) ? chat.to_llm : chat }
  raise ArgumentError, 'Cannot submit an empty batch' if chats.empty?

  unless chats.all? { |chat| awaiting_model?(chat) }
    raise ArgumentError,
          'Every chat in a batch must be awaiting the model; stage one with ask_later, or run_tools first'
  end

  provider = shared_provider(chats)
  payload = { provider: provider.slug, provider_class: provider.class.display_name, requests: chats.size }
  RubyLLM.instrument('batch.ruby_llm', payload, config: provider.config) do |event|
    requests = chats.each_with_index.map do |chat, index|
      { custom_id: index.to_s, model: chat.model.id, payload: chat.render }
    end
    batch = new(provider:, chats:, **provider.create_batch(requests))
    event[:batch_id] = batch.id
    batch
  end
end

Submits chats to their shared provider as a batch and returns a new Batch. Accepts a single Chat or an array. Every chat must be awaiting the model (see Chat#ask_later), and all must use the same provider.

chats = tickets.map do |ticket|
  RubyLLM.chat(model: "claude-haiku-4-5").ask_later(ticket.body)
end
batch = RubyLLM::Batch.submit(chats)
batch.status # => "in_progress"

Raises ArgumentError if chats is empty, mixes providers, or includes a chat that is not awaiting the model.

Public Instance Methods

cancel ()

Source

# File lib/ruby_llm/batch.rb, line 134
def cancel
  apply(@provider.cancel_batch(id))
  self
end

Asks the provider to cancel the batch and applies the new state. Requests already processed still return results. Returns self.

complete? ()

Source

# File lib/ruby_llm/batch.rb, line 121
def complete?
  @completed
end

Returns whether the batch has finished processing, as of the last state fetched from the provider. Never contacts the provider; poll with refresh.

sleep 60 until batch.refresh.complete?

messages ()

Source

# File lib/ruby_llm/batch.rb, line 148
def messages
  return @messages if @messages

  collected = collect_messages
  @messages = collected if @completed
  collected
end

Returns the answers in submission order, nil where a request failed, each also appended to its chat. Fetches results from the provider; cached once complete? is true, so collecting early keeps reading fresh.

batch.messages.each do |message|
  puts message.content
end

refresh ()

Source

# File lib/ruby_llm/batch.rb, line 127
def refresh
  apply(@provider.find_batch(id))
  self
end

Re-fetches the batch from the provider, updating status, request_counts, and complete?. Returns self.