class RubyLLM::Chat

Creates a chat with model:, or with the configured default model when model: is nil. Most code calls RubyLLM.chat instead.

A model is identified by its name, an optional provider:, and an optional protocol:. Pass provider: to disambiguate models available from several providers, and protocol: to override the wire protocol the provider would otherwise pick for the model. With assume_model_exists: true the registry lookup is skipped, which requires provider:. Pass a Context as context: to use its configuration instead of the global one.

Appends a message to the conversation and returns it as a Message. Accepts a Message, an attribute Hash, or a record responding to to_llm.

chat.add_message(role: :user, content: "What's the capital of France?")

Registers a callback that receives the Fallback attempt once it has succeeded or failed. Returns self.

Registers a callback that receives each assistant response and each tool result message once it has been appended. Returns self.

chat.after_message { |message| puts message.content }

Registers a callback that receives each local tool’s result after execution. Returns self.

Records approval for tool_call, a ToolCall or its id, so the next complete or run_tools executes a local tool or records permission for the provider to execute a remote tool on the next request. Returns self.

chat.approve(tool_call)
chat.complete

Adds message as a user message and runs the conversation loop, executing tools until the model answers or a call needs approval. Returns the latest assistant Message; check awaiting_approval? before treating it as a final answer. Attach files with with:. A given block receives streamed Chunk objects as they arrive.

chat.ask "What's the best way to learn Ruby?"
chat.ask "What's in this image?", with: "ruby_conf.jpg"
chat.ask "Analyze these files", with: ["diagram.png", "report.pdf"]
chat.ask("Tell me a story") { |chunk| print chunk.content }

Stages message as a user message without requesting a completion, leaving the chat ready for complete, a single step, or a provider-side batch via RubyLLM.batch. Accepts attachments with with: like ask. Returns self.

chats = tickets.map { |t| RubyLLM.chat.ask_later(t.body) }
RubyLLM.batch(chats)

Raises PendingToolCallsError while the last response has unanswered tool calls: finish the round first, recording approve or deny decisions for calls that require approval.

Returns whether the conversation can make no progress without an approval decision: every remaining pending tool call requires approval and has none recorded. While true, complete returns without executing them; record decisions with approve or deny, then call complete again. Tool calls that need no approval still execute before the loop pauses.

Consults each pending tool’s approval resolver when one is declared, so resolvers must be idempotent reads.

Registers a callback that receives the Fallback attempt after the current model fails and before the fallback model is tried. Returns self.

Registers a callback that runs before each assistant response or tool result is appended to the conversation. Callbacks are additive: every registered block runs. Returns self.

Registers a callback that receives the fully rendered request payload before it is sent and may mutate it in place. Runs after all RubyLLM formatting and with_provider_options merging. Returns self.

chat.before_request { |payload| logger.debug payload }

Registers a callback that receives each local ToolCall before the tool executes. Returns self.

chat.before_tool_call { |tool_call| puts tool_call.name }

Marks the latest message as an explicit prompt cache boundary, asking the provider to cache everything up to this point. Returns self.

Raises ArgumentError if the chat has no messages.

Cancels the current in-flight chat operation. The next cancellation checkpoint raises CancelledError and clears the flag so the chat can be reused.

Returns whether this in-memory chat has been marked for cancellation.

Compacts the conversation’s model context and returns an assistant Message. The message can have empty text and carries the provider’s compacted context internally. Every earlier message remains in messages, including on persisted Rails chats.

chat.ask "Remember these project requirements..."
chat.compact
chat.ask "Which requirement should we implement first?"

Uses the current instructions, headers, and request hooks. Records reported usage and runs the normal message callbacks. Raises Error when the provider has no manual compaction endpoint, and PendingToolCallsError until pending tool calls have been answered.

Runs the conversation loop until complete? or awaiting_approval? is true. Returns the last conversation Message, or nil for an empty chat. Used after ask_later; ask calls complete for you.

When a pending tool call requires approval and no decision has been recorded, the loop pauses. Record approve or deny decisions, then call complete again to continue.

Returns whether the chat has no pending response or tool execution: nothing is staged, or the model answered without requesting tools.

Returns a Cost aggregating every provider attempt this chat has made, including retries and attempts that produced no message.

chat.cost.total

Counts input tokens for the conversation, including instructions, function tools, structured output, thinking, and attachments. Pass message to include it as a staged user message without mutating the chat. Returns an Integer.

chat.with_instructions("Be terse.").with_tools(Weather)
chat.count_tokens("What's the weather in Berlin?")

Server tools, provider_options, compaction, and before_request hooks are not included. Raises Error when the provider has no token counting endpoint.

Records denial for tool_call, a ToolCall or its id. The next complete or run_tools appends a structured denial result instead of executing a local tool, or sends a refusal for a remote tool on the next request. The model continues from there. Returns self.

Yields each Message in the conversation. Returns an Enumerator when no block is given. Chat includes Enumerable, so the usual collection methods are available.

Requests one completion from the model, appends the response to the conversation, and returns it as a Message. Honors the fallbacks configured with with_fallbacks. A given block receives streamed Chunk objects. Tool calls in the response are not executed; that is run_tools.

Replaces the conversation with new_messages, coercing each element into a Message. Accepts Message objects, attribute Hashes, and records responding to to_llm.

Returns the tool calls from the latest response that require approval and have no recorded decision, as an array of ToolCall objects. Pairs with approve and deny. ToolCall#remote? identifies provider-executed calls.

chat.pending_approvals.each { |tool_call| puts tool_call.name }
chat.approve(chat.pending_approvals.first)

Returns the request payload this chat would send to the provider for its next completion, with before_request hooks applied. Useful for inspecting and testing request output.

Executes the tool calls pending in the latest response and appends their result messages, without asking the model to respond. Tool calls that already have results are skipped, so a chat reloaded mid-round resumes with only the remaining tools. Calls whose tool was declared with Tool.requires_approval only execute once approve records a decision; denied calls receive a structured denial result, and undecided calls stay pending. Does nothing when no tool calls are pending. The chat is then ready for the next generate, or the next batch round. Returns self.

Advances the conversation by one move: runs the pending tool calls if any are unanswered, otherwise generates the next response. Returns the Message that move produced, and nil once there is nothing left to do or the loop is parked on an approval.

Returns the thinking options resolved for the current model, or nil when thinking was not configured or needs no provider control.

Returns token usage aggregated across every provider attempt this chat has made, including retries and attempts that produced no message.

chat.tokens.input

Returns the choice, calls, and concurrency set with with_tool_options, with nil for anything left at the default.

Enables provider prompt caching. With no arguments the provider’s default behavior applies; options such as ttl: apply where supported. Pass id: with a CachedContent (or its name) from RubyLLM.cache to attach an explicit content cache. Pass false to stop RubyLLM from sending cache controls or rendering explicit cache boundaries. A provider may still cache prompts implicitly. Passing nil raises ArgumentError. Returns self.

chat.with_caching
chat.with_caching(ttl: "1h")
chat.with_caching(id: cache)
chat.with_caching(false)

Enables document citations, so the model backs its claims with quotes from attached files. Pass false to disable. Passing nil raises ArgumentError. Returns self.

chat.with_citations
response = chat.ask "Who created Ruby?", with: "facts.txt"
response.citations.each { |citation| puts citation.cited_text }

Enables provider-side context compaction, so a long conversation keeps going instead of overflowing the context window. The provider condenses the earlier turns itself and returns a block that RubyLLM replays on later requests. With no arguments the provider’s own defaults apply. The options are provider-neutral:

at

the input-token count that triggers compaction.

instructions

a custom prompt for the summary the provider writes.

pause_after

end the turn once compaction runs, instead of continuing straight into the answer.

Each provider applies the options it supports. Unsupported options are ignored with a debug log. Pass false to disable; passing nil raises ArgumentError. Returns self.

chat.with_compaction
chat.with_compaction(at: 50_000)
chat.with_compaction(at: 100_000, instructions: "Keep every decision.")
chat.with_compaction(false)

What a provider does when the threshold is crossed differs. Anthropic and OpenAI summarize the compacted span into an opaque block that replaces it; OpenRouter drops messages from the middle of the conversation instead, and has no threshold of its own.

Rebinds the chat to context, a Context built with RubyLLM.context, so subsequent requests use its configuration. Pass nil to return to the global RubyLLM.config. Returns self.

Identifies the end user behind the conversation for the provider’s abuse monitoring. Providers without an equivalent field omit it. Pass nil to remove it. Returns self.

chat.with_end_user("user-123").ask "Hello"

The value is sent as given, so use an opaque id such as a hash of your user id, never personal data.

Sets fallback models to try, in order, when generation fails. on: selects the error classes that trigger a fallback; the default covers transient provider and network errors. Pass nil to remove all fallbacks and restore the default error classes. Returns self.

chat.with_fallbacks("gpt-4.1-mini", "claude-haiku-4-5")
chat.with_fallbacks(nil)

Sets extra HTTP headers sent with completion requests, replacing any previously set headers; nil clears them. Returns self.

chat.with_headers('anthropic-beta' => 'fine-grained-tool-streaming-2025-05-14')

Sets the system instructions for the conversation, replacing any existing system messages. With append: true the instructions are added alongside the existing ones. With cache_until_here: true the instruction becomes an explicit prompt cache boundary. Pass nil to remove all system instructions. Returns self.

chat.with_instructions "You are a helpful Ruby tutor."
chat.with_instructions "Use exactly one short paragraph.", append: true
chat.with_instructions nil

Caps the number of tokens the model may generate. Pass nil to remove the limit. Returns self.

chat.with_max_output_tokens(1000)

Switches the chat to model_id and its provider. Pass provider: to disambiguate, and assume_model_exists: true to skip registry validation for custom or private models. Pass nil to return to the configured default model. Returns self.

protocol: overrides the wire protocol the provider would pick for the model, such as :responses or :chat_completions for OpenAI. It stays nil by default, meaning the provider chooses the protocol for each request. A bare with_model resets the override to nil, just as it re-resolves the provider from the model.

Raises ModelNotFoundError if model_id is not in the registry and assume_model_exists: is false.

chat.with_model('claude-sonnet-5')
chat.with_model('gpt-5.6', protocol: :chat_completions)

Sets options in the provider’s request vocabulary, merged into the request payload as-is and overriding RubyLLM’s defaults. Replaces any previously set provider options; nil clears them. Returns self.

chat.with_provider_options(service_tier: "flex")

Sets the schema for structured output. Accepts a JSON Schema Hash, a Schematist::Schema class or instance, or any object responding to to_json_schema. Returns self.

class PersonSchema < Schematist::Schema
  string :name
  integer :age
end

chat.with_schema(PersonSchema)
response = chat.ask("Generate a person named Alice who is 30 years old")
response.parsed # => {"name" => "Alice", "age" => 30}

Pass nil to remove the schema, returning the chat to plain text responses.

Enables tools that run on the provider’s servers, such as web search or code execution. Accepts portable alias Symbols, alias-with-options keywords whose options use the provider’s own vocabulary, and raw Hashes passed to the provider verbatim, so provider tools RubyLLM has no alias for yet work without a gem update. Entries add to any tools enabled earlier; pass nil to clear them all. Returns self.

chat.with_server_tools(:web_search)
chat.with_server_tools(:web_search, :code_execution)
chat.with_server_tools(web_search: { allowed_domains: ["ruby-lang.org"] })
chat.with_server_tools({ type: "web_search_20260318", name: "web_search" })

The tool steps the model ran come back on Message#server_tool_calls, citations from search tools on Message#citations, and per-use billing counters on message.tokens.server_tool_use.

Raises UnsupportedServerToolError at request time when the provider has no server-tool support or does not define a requested alias.

Sets the sampling temperature for subsequent requests. Pass nil to return to the model’s default sampling behavior. Returns self.

chat.with_temperature(0.2)

Configures extended thinking for models that support it. With no arguments, RubyLLM uses the current model’s registered default. Pass false to disable thinking, or tune it with effort: (:low, :medium, :high, :none, or a provider-specific tier such as :minimal, :xhigh, or :max, passed through as-is), budget: (a token count), and display: (:summarized or :omitted, controlling whether providers that support it return readable thinking text). Accepts keywords or an options Hash. Passing nil raises ArgumentError. Returns self.

chat.with_thinking
chat.with_thinking(false)
chat.with_thinking(effort: :high)
chat.with_thinking(budget: 10_000)
chat.with_thinking(display: :summarized)

Configures how the model uses the registered tools. choice: constrains tool use to :auto, :none, :required, a tool name, or a Tool class. calls: limits how many tool calls one response may contain (:many or :one). concurrency: runs tool calls concurrently: true or :threads for threads, :fibers for fibers. An omitted option is left unchanged; passing nil explicitly resets that option (+concurrency: nil+ returns to the configured default). Returns self.

chat.with_tools(Weather, Search).with_tool_options(choice: :required)
chat.with_tool_options(calls: :one, concurrency: :threads)
chat.with_tool_options(choice: nil)

Registers tools, each a Tool class or instance, for the model to call. Configure how the model uses them with with_tool_options. Pass nil to remove all registered tools. Returns self.

chat.with_tools(Weather, Search)
chat.with_tools(Weather).with_tool_options(choice: :required)

To replace the registered tools, clear them first:

chat.with_tools(nil).with_tools(NewTool)