class RubyLLM::Batch

A Batch is a provider-side batch of requests: chats awaiting a response (or texts awaiting embeddings) go in together, answers come back at batch prices, typically within hours. Persist the id, pick the batch back up from any process, and collect the results once processing ends.

chats = documents.map do |doc|
  RubyLLM.chat(model: "claude-haiku-4-5").ask_later(doc.text)
end
batch = RubyLLM.batch(chats)
batch.id                # => "msgbatch_01EhcDuvb5XfWqcdJArbsfNX"
batch.refresh.complete? # => false, check back later
batch.messages          # the responses, in submission order

Attributes

chats [R]

The submitted Chat objects in order, or nil when the batch was loaded by id via ::find or holds embedding requests.

id [R]

The provider’s batch id. Persist it to load the batch again later from any process with ::find.

raw_status [R]

The provider-reported status string, such as “in_progress”. Refreshed by refresh.

request_counts [R]

The provider-reported request tallies by state, or nil when the provider does not report them.

requests [R]

The submitted EmbeddingRequest objects in order, or nil when the batch was loaded by id via ::find or holds chats.

status [R]

The provider-neutral lifecycle status: :pending, :succeeded, :failed, or :cancelled. Refreshed by refresh.

statuses [R]

The normalized outcome of each collected request, in submission order. Values are :succeeded, :failed, or :cancelled.

Public Class Methods

find (id, provider: nil, context: nil)

Source

# File lib/ruby_llm/batch.rb, line 83
def find(id, provider: nil, context: nil)
  config = context&.config || RubyLLM.config
  persisted = config.batch_store&.fetch(id, provider:, context:)
  return persisted if persisted

  unless provider
    raise ArgumentError, 'Provider must be specified to find a batch that is not persisted by RubyLLM'
  end

  provider = Provider.resolve!(provider).new(config)
  raise Error, "#{provider.slug} doesn't support batch requests" unless provider.batches?

  new(provider:, store: config.batch_store, **provider.find_batch(id))
end

Returns a Batch reflecting the provider’s current state for id. Use it to pick a batch back up from any process.

batch = RubyLLM::Batch.find("msgbatch_01EhcDuvb5XfWqcdJArbsfNX",
                            provider: :anthropic)
batch.complete? # => true

Pass context: to use a Context in place of the global configuration. Raises ArgumentError if provider is not given.

submit (chats)

Source

# File lib/ruby_llm/batch.rb, line 67
def submit(chats)
  records = wrap_records(chats)
  return submit_embeddings(records) if records.any?(EmbeddingRequest)

  submit_chats(records)
end

Submits chats or embedding requests to their shared provider as a batch and returns a new Batch. Accepts a single Chat or an array. Every chat must be awaiting the model (see Chat#ask_later), and all requests must use the same provider.

chats = tickets.map do |ticket|
  RubyLLM.chat(model: "claude-haiku-4-5").ask_later(ticket.body)
end
batch = RubyLLM::Batch.submit(chats)
batch.status     # => :pending
batch.raw_status # => "in_progress"

Raises ArgumentError if the batch is empty, mixes providers, mixes chats with embedding requests, or includes a chat that is not awaiting the model.

Public Instance Methods

cancel ()

Source

# File lib/ruby_llm/batch.rb, line 223
def cancel
  apply(@provider.cancel_batch(id))
  persist_state
  self
end

Asks the provider to cancel the batch and applies the new state. Requests already processed still return results. Returns self.

cancelled? ()

Source

# File lib/ruby_llm/batch.rb, line 209
def cancelled?
  status == :cancelled
end

Returns whether the provider cancelled the batch.

complete? ()

Source

# File lib/ruby_llm/batch.rb, line 194
def complete?
  @completed
end

Returns whether the batch has finished processing, as of the last state fetched from the provider. Never contacts the provider; poll with refresh.

sleep 60 until batch.refresh.complete?

cost ()

Source

# File lib/ruby_llm/batch.rb, line 258
def cost
  return Cost.aggregate([reported_cost], complete: complete?) if reported_cost
  return Cost.aggregate([], complete: false) unless complete?

  Cost.aggregate(messages.compact.map(&:cost))
end

Returns a Cost for the batch. Uses the provider’s reported total when available, otherwise aggregates collected response costs at batch rates. The total is nil until the batch ends or when pricing is unknown.

failed? ()

Source

# File lib/ruby_llm/batch.rb, line 204
def failed?
  status == :failed
end

Returns whether the provider failed or expired the batch.

messages ()

Source

# File lib/ruby_llm/batch.rb, line 240
def messages
  return @messages if @messages

  collected = collect_results
  @messages = collected if @completed
  collected
end

Returns the answers in submission order, nil where a request failed. In a chat batch the answers are Messages, each also appended to its chat; in an embeddings batch they are Embeddings, each also hydrated into its request’s EmbeddingRequest#result. Fetches results from the provider; cached once complete? is true, so collecting early keeps reading fresh.

batch.messages.each do |message|
  puts message.content
end

Also aliased as: results

provider ()

Source

# File lib/ruby_llm/batch.rb, line 182
def provider
  @provider.slug
end

The slug of the provider running the batch, as a String.

refresh ()

Source

# File lib/ruby_llm/batch.rb, line 215
def refresh
  apply(@provider.find_batch(id))
  persist_state
  self
end

Re-fetches the batch from the provider, updating status, raw_status, request_counts, and complete?. Returns self.

results ()

Alias for: messages

succeeded? ()

Source

# File lib/ruby_llm/batch.rb, line 199
def succeeded?
  status == :succeeded
end

Returns whether the provider completed the batch successfully.

tokens ()

Source

# File lib/ruby_llm/batch.rb, line 251
def tokens
  Tokens.aggregate(messages.compact.map(&:tokens))
end

Returns token usage aggregated across the batch’s collected responses.