Chat

Learn how to have conversations with AI models, work with different providers, and attach files like images and PDFs

Table of contents

  1. Starting a Conversation
  2. Continuing the Conversation
  3. Guiding AI Behavior with System Prompts
  4. Working with Different Models
  5. Controlling Responses
    1. Temperature and Creativity
  6. Raw Responses
  7. Finish Reasons
  8. Advanced: Replacing the LLM Transcript
  9. Going Further
  10. Next Steps

After reading this guide, you will know:

  • How to start and continue conversations with AI models.
  • How to guide AI behavior with system prompts.
  • How to render reusable prompt templates.
  • How to select and work with different models and providers.
  • How to control response creativity with temperature.
  • How to read the raw provider response.
  • Where to go for file attachments, request control, token tracking, and event handlers.

Starting a Conversation

When you want to interact with an AI model, you create a chat instance. The simplest approach uses RubyLLM.chat, which creates a new conversation with your configured default model.

chat = RubyLLM.chat

response = chat.ask "Explain the concept of 'Convention over Configuration' in Rails."

puts response.content
# => "Convention over Configuration (CoC) is a core principle of Ruby on Rails..."

puts "Model Used: #{response.model}"
puts "Tokens Used: #{response.tokens.input} input, #{response.tokens.output} output"
puts "Cache Reads: #{response.tokens.cache_read}" # v1.15+
puts "Cache Writes: #{response.tokens.cache_write}" # v1.15+

The ask method adds your message to the conversation history with the :user role, sends the entire conversation history to the AI provider, and returns a RubyLLM::Message object containing the assistant’s response.

The say method is an alias for ask, so you can use whichever feels more natural in your code.

When the model uses tools, ask runs the conversation to completion: it calls the model, runs any tools the model asks for, and calls the model again until it answers without a tool. When you need to control that loop yourself, see Driving the Loop Yourself.

Continuing the Conversation

One of the key features of chat-based AI models is their ability to maintain context across multiple exchanges. The Chat object automatically manages this conversation history for you.

response = chat.ask "Can you give a specific example in Rails?"
puts response.content
# => "Certainly! A classic example is database table naming..."

chat.messages.each do |message|
  puts "[#{message.role.to_s.upcase}] #{message.content.lines.first.strip}"
end
# => [USER] Explain the concept of 'Convention over Configuration' in Rails.
# => [ASSISTANT] Convention over Configuration (CoC) is a core principle...
# => [USER] Can you give a specific example in Rails?
# => [ASSISTANT] Certainly! A classic example is database table naming...

Each time you call ask, RubyLLM sends the entire conversation history to the AI provider. This allows the model to understand the full context of your conversation, enabling natural follow-up questions and maintaining coherent dialogue.

Guiding AI Behavior with System Prompts

System prompts, also called instructions, allow you to set the overall behavior, personality, and constraints for the AI assistant. These instructions persist throughout the conversation and help ensure consistent responses.

chat = RubyLLM.chat

chat.with_instructions "You are a helpful assistant that explains Ruby concepts simply, like explaining to a five-year-old."

response = chat.ask "What is a variable?"
puts response.content
# => "Imagine you have a special box, and you can put things in it..."

# By default, with_instructions replaces the active system instruction
chat.with_instructions "Always end your response with 'Got it?'"

response = chat.ask "What is a loop?"
puts response.content
# => "A loop is like singing your favorite song over and over again... Got it?"

# Append an additional system instruction only when needed
chat.with_instructions "Use exactly one short paragraph.", append: true

# Clear system instructions
chat.without_instructions

System prompts are added to the conversation as messages with the :system role and are sent with every request to the AI provider. This ensures the model always considers your instructions when generating responses.

Plain chats do not have a class name, so they do not infer a prompt path. For reusable instructions stored in app/prompts, render a template with Prompt Rendering and pass the result to with_instructions.

When using the Rails Integration, system messages are persisted in your database along with user and assistant messages, maintaining the full conversation context.

Working with Different Models

RubyLLM supports over 600 models from various providers. While RubyLLM.chat uses your configured default model, you can specify different models:

chat_claude = RubyLLM.chat(model: 'claude-sonnet-4-6')
chat_gemini = RubyLLM.chat(model: 'gemini-3.1-pro-preview')

chat = RubyLLM.chat(model: 'gpt-5-nano')
response1 = chat.ask "Initial question..."

chat.with_model('claude-sonnet-4-6')
response2 = chat.ask "Follow-up question..."

Pass nil to return to your configured default model:

chat.with_model(nil)

For detailed information about model selection, capabilities, aliases, and working with custom models, see Working with Models. For exactly how a name becomes a model and provider, see Model Resolution.

Controlling Responses

Temperature and Creativity

The temperature parameter controls the randomness of the model’s responses. Understanding temperature helps you get the right balance between creativity and consistency for your use case.

  • Low temperature (0.0 - 0.3): More deterministic and focused responses. Use for factual queries, technical explanations, or when consistency is important.
  • Medium temperature (0.4 - 0.7): Balanced creativity and coherence. Good for general conversation and most applications.
  • High temperature (0.8 - 1.0): More creative and varied responses. Use for brainstorming, creative writing, or when you want diverse outputs.
factual_chat = RubyLLM.chat.with_temperature(0.2)
response1 = factual_chat.ask "What is the boiling point of water at sea level in Celsius?"
puts response1.content

creative_chat = RubyLLM.chat.with_temperature(0.9)
response2 = creative_chat.ask "Write a short poem about the color blue."
puts response2.content

The with_temperature method returns the chat instance, allowing you to chain multiple configuration calls together. Call without_temperature to clear a temperature override:

chat.without_temperature

For provider-specific request options, wire protocols, raw content blocks, and custom HTTP headers, see Advanced Request Control. For local ERB prompt templates, see Prompt Rendering. For provider-side prompt reuse, see Prompt Caching.

Raw Responses

You can access the raw response from the API provider with response.raw.

response = chat.ask("What is the capital of France?")
puts response.raw.body

The raw response is a Faraday::Response object, which you can use to access the headers, body, and status code.

Finish Reasons

Responses expose finish_reason when the provider reports why the model stopped. RubyLLM preserves the provider value as-is and provides predicate methods for common cases.

response = chat.ask("Summarize this in one paragraph")

if response.max_tokens?
  puts "The response hit a token limit."
elsif response.content_filtered?
  puts "The provider filtered the response."
elsif response.tool_call_stop?
  puts "The model requested a tool call."
elsif response.stopped?
  puts "The model finished normally."
end

response.finish_reason # Raw provider value, such as "stop", "max_tokens", or "MAX_TOKENS"

Provider names differ: OpenAI Chat Completions may return tool_calls, Anthropic may return max_tokens, Gemini may return MAX_TOKENS, and Bedrock may return stopReason values such as end_turn. Finish reasons describe completed provider responses; failed requests still raise RubyLLM error classes. RubyLLM does not send finish_reason back to providers when replaying conversation history.

Advanced: Replacing the LLM Transcript

For advanced context management, chat.messages is whatever you show the LLM. Your application can keep and render a different user-visible transcript if needed. This is useful for chat compaction, moderation, redaction, or any workflow where the LLM should see a different transcript from the user.

messages_for_model = chat.messages.last(4)
chat.messages = messages_for_model

For persisted Rails chats, see Separate User and LLM Transcripts.

Going Further

This page covers the core Chat interface. Each facet of a conversation has its own focused guide:

  • Attachments - attach images, video, audio, text files, and PDFs to a message.
  • Streaming - display responses in real time as they are generated.
  • Structured Output - get responses that match an exact JSON schema.
  • Extended Thinking - give reasoning models room to deliberate and read their thinking.
  • Citations - get verifiable answers backed by your documents and web sources.
  • Prompt Rendering - render reusable ERB templates from app/prompts.
  • Prompt Caching - reuse stable prompt prefixes automatically or with explicit boundaries.
  • Advanced Request Control - provider-specific parameters, wire protocols, raw content blocks, and custom headers.
  • Token Usage and Cost - read per-turn and per-conversation token counts and costs.
  • Chat Event Handlers - hook into the chat lifecycle for UI updates, logging, and analytics.

Next Steps

Table of contents