Chat

Start a conversation, give it instructions, and switch models while keeping the same Ruby API.

After reading this guide, you will know:

  • How to start and continue conversations with AI models.
  • How to guide AI behavior with system prompts.
  • How to select and work with different models and providers.
  • How to control response creativity with temperature.
  • How to read the raw provider response.
  • Where to go for file attachments, request control, token tracking, and event handlers.

Starting a Conversation

RubyLLM.chat starts a conversation with your configured default model:

chat = RubyLLM.chat

response = chat.ask "Explain the concept of 'Convention over Configuration' in Rails."

puts response.content
# => "Convention over Configuration (CoC) is a core principle of Ruby on Rails..."

The ask method adds your message to the conversation history with the :user role, sends the entire conversation history to the AI provider, and returns a RubyLLM::Message object containing the assistant’s response.

The say method is an alias for ask, so you can use whichever feels more natural in your code.

When the model uses tools, ask runs the conversation to completion: it calls the model, runs any tools the model asks for, and calls the model again until it answers without a tool. When you need to control that loop yourself, see Driving the Loop Yourself.

Continuing the Conversation

Ask a follow-up question on the same chat:

response = chat.ask "Can you give a specific example in Rails?"
puts response.content
# => "Certainly! A classic example is database table naming..."

chat.messages.each do |message|
  puts "[#{message.role.to_s.upcase}] #{message.content.lines.first.strip}"
end
# => [USER] Explain the concept of 'Convention over Configuration' in Rails.
# => [ASSISTANT] Convention over Configuration (CoC) is a core principle...
# => [USER] Can you give a specific example in Rails?
# => [ASSISTANT] Certainly! A classic example is database table naming...

chat.messages holds the conversation. RubyLLM includes that history in each request, so you do not need to assemble it yourself.

Guiding AI Behavior with System Prompts

System prompts, also called instructions, allow you to set the overall behavior, personality, and constraints for the AI assistant. These instructions persist throughout the conversation and help ensure consistent responses.

chat = RubyLLM.chat

chat.with_instructions "Explain Ruby concepts to a beginner. Use short, runnable examples."

response = chat.ask "What is a variable?"
puts response.content
# => "Imagine you have a special box, and you can put things in it..."

# By default, with_instructions replaces the active system instruction
chat.with_instructions "Always end your response with 'Got it?'"

response = chat.ask "What is a loop?"
puts response.content
# => "A loop is like singing your favorite song over and over again... Got it?"

# Append an additional system instruction only when needed
chat.with_instructions "Use exactly one short paragraph.", append: true

# Clear system instructions
chat.with_instructions(nil)

Instructions are :system messages included in each request.

For reusable instructions stored in app/prompts, render a template with Prompt Rendering and pass the result to with_instructions.

When using the Rails Integration, system messages are persisted in your database along with user and assistant messages, maintaining the full conversation context.

Working with Different Models

RubyLLM’s published registry covers more than 1,400 models, and local models are discovered when you refresh it. While RubyLLM.chat uses your configured default model, you can specify a different model:

chat_claude = RubyLLM.chat(model: 'claude-sonnet-5')
chat_gemini = RubyLLM.chat(model: 'gemini-3.7-flash')

chat = RubyLLM.chat(model: 'gpt-5.6-luna')
response1 = chat.ask "Initial question..."

chat.with_model('claude-sonnet-5')
response2 = chat.ask "Follow-up question..."

Pass nil to return to your configured default model:

chat.with_model(nil)

For detailed information about model selection, capabilities, aliases, and working with custom models, see Model Registry. For exactly how a name becomes a model and provider, see Model Resolution.

Controlling Responses

Temperature and Creativity

The temperature parameter controls the randomness of the model’s responses. Understanding temperature helps you get the right balance between creativity and consistency for your use case.

  • Low temperature (0.0 - 0.3): More deterministic and focused responses. Use for factual queries, technical explanations, or when consistency is important.
  • Medium temperature (0.4 - 0.7): Balanced creativity and coherence. Good for general conversation and most applications.
  • High temperature (0.8 - 1.0): More creative and varied responses. Use for brainstorming, creative writing, or when you want diverse outputs.
factual_chat = RubyLLM.chat.with_temperature(0.2)
response1 = factual_chat.ask "What is the boiling point of water at sea level in Celsius?"
puts response1.content

creative_chat = RubyLLM.chat.with_temperature(0.9)
response2 = creative_chat.ask "Write a short poem about the color blue."
puts response2.content

The with_temperature method returns the chat instance, allowing you to chain multiple configuration calls together. Pass nil to clear a temperature override:

chat.with_temperature(nil)

RubyLLM sends the value you set, unchanged. Until you set one, no temperature goes on the wire at all and the model uses its own default.

Not every model accepts one. Reasoning models such as OpenAI’s o series and gpt-5-nano, the OpenAI search preview models, and newer Claude models such as claude-sonnet-5 reject the parameter or restrict it to a single value, and the list differs from model to model even inside one family. Set a temperature a model rejects and the provider answers with an error, which RubyLLM raises as RubyLLM::BadRequestError. RubyLLM does not rewrite or drop your value to hide that error. When a model rejects the parameter, leave it unset and let the model use its default.

For provider-specific request options, wire protocols, request hooks, and custom HTTP headers, see Advanced Request Control. For local ERB prompt templates, see Prompt Rendering. For provider-side prompt reuse, see Prompt Caching.

Raw Responses

You can access the raw response from the API provider with response.raw.

response = chat.ask("What is the capital of France?")
puts response.raw.body

The raw response is a Faraday::Response object, which you can use to access the headers, body, and status code.

Finish Reasons

finish_reason tells you why the model stopped. RubyLLM maps each provider’s values onto four symbols: :stop, :max_tokens, :tool_calls, and :content_filter. Anything else, such as Anthropic’s :pause_turn, comes through as the provider spelled it.

response = chat.ask("Summarize this in one paragraph")

if response.max_tokens?
  puts "The response hit a token limit."
elsif response.content_filtered?
  puts "The provider filtered the response."
elsif response.tool_call_stop?
  puts "The model requested a tool call."
elsif response.stopped?
  puts "The model finished normally."
end

response.finish_reason # => :stop, :max_tokens, :tool_calls, or :content_filter

Anthropic’s end_turn, Gemini’s MAX_TOKENS, Cohere’s COMPLETE, and the Responses API’s completed all arrive as :stop, so the same code reads every provider. A response that carries tool calls answers tool_call_stop?, not stopped?. A finish reason describes a response the provider completed. A failed request raises an error instead. RubyLLM never sends finish_reason back to a provider.

Advanced: Replacing the LLM Transcript

For advanced context management, chat.messages is whatever you show the LLM. Your application can keep and render a different user-visible transcript if needed. This is useful for chat compaction, moderation, redaction, or any workflow where the LLM should see a different transcript from the user.

messages_for_model = chat.messages.last(4)
chat.messages = messages_for_model

For persisted Rails chats, see Separate User and LLM Transcripts.

Going Further

This page covers the core Chat interface. Each facet of a conversation has its own focused guide:

  • Attachments - attach images, video, audio, text files, and PDFs to a message.
  • Streaming - display responses in real time as they are generated.
  • Structured Output - get responses that match an exact JSON schema.
  • Extended Thinking - give reasoning models room to deliberate and read their thinking.
  • Citations - get verifiable answers backed by your documents and web sources.
  • Prompt Rendering - render reusable ERB templates from app/prompts.
  • Prompt Caching - reuse stable prompt prefixes automatically or with explicit boundaries.
  • Advanced Request Control - provider-specific parameters, wire protocols, request hooks, and custom headers.
  • Tokens and Costs - read per-turn and per-conversation token counts and costs, including retries, cancellations, and provider attempts.
  • Chat Event Handlers - hook into the chat lifecycle for UI updates, logging, and analytics.

Next Steps