Agents

Give agents instructions, tools, and schemas in a Ruby class, then use them in scripts, services, and Rails jobs.

After reading this guide, you will know:

  • How to define agents with a class-based DSL
  • How to use agents with plain Ruby chats and Rails-backed chats
  • How runtime context works (chat, inputs, and lazy evaluation)
  • How prompt conventions work in app/prompts
  • Which methods are available on agent instances
  • How to handle errors for a whole agent with rescue_from

What Are Agents?

Agents are a class-based way to define a chat setup once and reuse it everywhere.

For example, instead of re-adding the same instructions and tools in every controller, job, or service, you define them once in an agent class and call that agent wherever you need it.

class SupportAgent < RubyLLM::Agent
  model "gpt-6.1-sol"
  instructions "You are a concise support assistant."
  tools SearchDocs, LookupAccount
end

response = SupportAgent.new.ask "How do I reset my API key?"

In other words, an agent is a named wrapper around the same configuration you would otherwise apply progressively with chat.with_* calls (with_instructions, with_tools, with_provider_options, and so on).

Agents work in two modes:

  • Plain Ruby mode via .chat (returns RubyLLM::Chat)
  • Rails mode via .create/.create!/.find when chat_class is configured (returns your ActiveRecord chat model)

Example of Rails mode:

class WorkAssistant < RubyLLM::Agent
  chat_class Chat  # this activates the Rails integration
  model "gpt-6.1-sol"
  instructions "You are a helpful assistant."
  tools SearchDocs, LookupAccount
end

chat = WorkAssistant.create!(user: current_user)
same_chat = WorkAssistant.find(chat.id)

Defining an Agent

Create a class that inherits from RubyLLM::Agent and declare its configuration:

# app/agents/work_assistant.rb
class WorkAssistant < RubyLLM::Agent
  model "gpt-6.1-sol"
  instructions "You are a helpful assistant."
  tools SearchDocs, LookupAccount
  temperature 0.2
  max_output_tokens 256
end

Supported class macros:

These macros use the same arguments you already know from RubyLLM.chat(...) and Chat#with_* methods. For example, model maps to RubyLLM.chat(model:, provider:, ...), tools maps to with_tools, tool_options maps to with_tool_options, instructions maps to with_instructions, and so on.

Call caching without options to use the provider’s default prompt cache policy:

class WorkAssistant < RubyLLM::Agent
  caching
end

This is the agent equivalent of calling chat.with_caching. Pass provider-specific options when you need them, such as caching ttl: "1h".

Feature macros follow the same shape as their with_* methods: call them bare to enable the registered default, pass false to disable them, or pass options to tune them.

class WorkAssistant < RubyLLM::Agent
  thinking
  caching
  compaction
  citations
end

tools sets which tools the agent’s chats may call. Use tool_options for choice, calls, and concurrency:

class WorkAssistant < RubyLLM::Agent
  tools SearchDocs, LookupAccount
  tool_options choice: :auto, calls: :one
end

schema supports:

  • A schema class (for example PersonSchema) - same as with_schema
  • A JSON schema hash - same as with_schema
  • An inline DSL block with schema do ... end - agent-specific convenience

Inline DSL example:

class CriticAgent < RubyLLM::Agent
  schema do
    string :verdict, enum: ["pass", "revise"]
    string :feedback
  end
end

Model Fallbacks

Use fallbacks to give every chat created by the agent the same ordered fallback models:

class WorkAssistant < RubyLLM::Agent
  model "gpt-4.1"
  fallbacks "gpt-4.1-mini", "claude-haiku-4-5"
end

You can also customize which errors trigger fallback:

class WorkAssistant < RubyLLM::Agent
  model "gpt-4.1"
  fallbacks "gpt-4.1-mini",
            on: [RubyLLM::RateLimitError, RubyLLM::ServiceUnavailableError]
end

Fallbacks can be model IDs or RubyLLM::Model objects:

class WorkAssistant < RubyLLM::Agent
  model "gpt-4.1"
  fallbacks RubyLLM.models.find("claude-haiku-4-5", provider: :anthropic)
end

This works for both WorkAssistant.chat and Rails-backed agents configured with chat_class.

Configuration Contexts

Use a configuration context to give an agent its own credentials, endpoint, or connection settings without changing global configuration.

You can define a context elsewhere in your application and pass it to the agent:

# config/initializers/ai_context.rb
SupportAIContext = RubyLLM.context do |config|
  config.openai_api_key = ENV.fetch('SUPPORT_OPENAI_API_KEY')
  config.request_timeout = 180
end

# app/agents/support_agent.rb
class SupportAgent < RubyLLM::Agent
  model "gpt-6.1-sol", provider: :openai
  context SupportAIContext
end

Or configure the context directly with a block:

class SupportAgent < RubyLLM::Agent
  model "gpt-6.1-sol", provider: :openai

  context do |config|
    config.openai_api_key = ENV.fetch('SUPPORT_OPENAI_API_KEY')
    config.request_timeout = 180
  end
end

The block runs when the agent class is defined and receives a copy of the global configuration. Pass either a context object or a block. Both forms work with SupportAgent.chat and Rails-backed agents configured with chat_class. Call SupportAgent.context to read the configured context.

See Contexts: Isolated Configurations for more configuration examples.

Runtime Context and Inputs

Agents support runtime-evaluated values using blocks and lambdas.

Declare additional runtime inputs with inputs:

class WorkAssistant < RubyLLM::Agent
  chat_class Chat
  inputs :workspace

  instructions { "You are helping #{workspace.name}" }
end

Most runtime blocks can use chat in their execution context:

  • In .chat mode, chat is a RubyLLM::Chat
  • In .create/.create!/.find mode, chat is your chat_class record

The model block runs before the chat exists, so it can read inputs but not chat. A deferred context block follows the same timing in plain Ruby, so chat is nil there. In Rails mode, the context block runs while the chat_class record is being configured, so chat is available.

This enables Rails-style usage:

class WorkAssistant < RubyLLM::Agent
  chat_class Chat

  instructions current_date_time: -> { Time.current.strftime("%B %d, %Y") },
    display_name: -> { chat.user.display_name_or_email },
    full_name: -> { chat.user.full_name.presence || chat.user.display_name_or_email }

  tools do
    [
      TodoTool.new(chat: chat),
      GoogleDriveListTool.new(user: chat.user),
      GoogleDriveSearchTool.new(user: chat.user),
      GoogleDriveReadTool.new(user: chat.user)
    ]
  end
end

Values that depend on runtime chat must be lazy (blocks or lambdas), not eager class-load expressions.

Choosing the model at runtime

model takes a block too, so an agent can route work to a cheaper or larger model based on its inputs:

class CardAgent < RubyLLM::Agent
  inputs :card
  model { card.special_type? ? "gpt-4.1-mini" : "gpt-4.1-nano" }
  instructions "You are helpful."
end

CardAgent.chat(card: card)

Options stay alongside the block: model(provider: :openai) { ... }.

Prompt Management and Conventions

Agents have prompt conventions built in. They use the same app/prompts templates as Prompt Rendering, with class-based lookup layered on top.

Default instructions prompt

Named agents automatically use their conventional instructions prompt when it exists:

class WorkAssistant < RubyLLM::Agent
  chat_class Chat
end

RubyLLM looks for:

  • app/prompts/work_assistant/instructions.txt.erb

Instructions are selected in this order:

  1. instructions declarations on the agent class.
  2. The agent’s conventional instructions.txt.erb template.
  3. Inherited instructions declarations, including their persistence and cache settings.

Child declarations and templates replace inherited instructions. An empty child template or an explicit instructions "" means no instructions. If none of these sources exists, the agent starts without system instructions.

To require a prompt and fail loudly when it is missing, reference it explicitly:

class WorkAssistant < RubyLLM::Agent
  chat_class Chat
  instructions { prompt("instructions") }
end

If that file does not exist, RubyLLM raises RubyLLM::PromptNotFoundError.

Prompt shorthand with locals

You can pass locals directly:

class WorkAssistant < RubyLLM::Agent
  chat_class Chat
  instructions display_name: -> { chat.user.display_name_or_email }
end

This also renders instructions.txt.erb for that agent path.

Prompt helper in runtime blocks

Within execution context you can call:

instructions { prompt("instructions", display_name: chat.user.display_name_or_email) }

Naming conventions

Agent prompt path is derived from class name:

  • WorkAssistant -> app/prompts/work_assistant/...
  • Admin::SupportAgent -> app/prompts/admin/support_agent/...

Prompt extension defaults to .txt.erb.

For rendering a prompt directly outside an agent, use RubyLLM.render_prompt. See Prompt Rendering.

Using an Agent

Plain Ruby chat

chat = WorkAssistant.chat
response = chat.ask("Hello")

puts response.content

WorkAssistant.chat(...) returns a configured RubyLLM::Chat.

Instance API

You can still instantiate and use an agent instance directly:

agent = WorkAssistant.new
response = agent.ask("Hello")

response.cost.total
agent.cost.total

Agent instances delegate the conversation API from RubyLLM::Chat to the wrapped chat object. Direct transcript replacement with messages= stays on the wrapped chat because Rails-backed chat models own their message association.

Delegated methods include:

  • model, model_id, provider, messages, tools, provider_tools, provider_options, headers, schema
  • concurrency, caching, compaction, end_user, fallbacks
  • tokens, cost, count_tokens, render
  • ask, say, complete, complete?, ask_later, generate, run_tools, step
  • cancel, cancelled?, waiting?, approve, deny, awaiting_approval?, pending_approvals
  • queue, queued_messages, unqueue, join_queued_messages
  • add_message, each
  • cache_until_here, with_tools, with_provider_tools, with_tool_options
  • with_model, with_instructions, with_temperature, with_max_output_tokens, with_thinking, with_citations, with_end_user, with_compaction, with_context
  • with_caching, with_provider_options, with_headers, with_schema, with_fallbacks
  • before_request, before_message, after_message, before_tool_call, after_tool_result, after_tool_progress, before_fallback, after_fallback, after_queue, around_join, after_join, after_unqueue

You can always access the wrapped chat object directly via agent.chat.

Handling Errors with rescue_from

rescue_from declares how an agent handles exceptions raised by its chat operations: ask, say, ask_later, complete, generate, run_tools, step, count_tokens, and compact.

class ApplicationAgent < RubyLLM::Agent
  rescue_from RubyLLM::RateLimitError, RubyLLM::ServerError, with: :handle_transient
  rescue_from RubyLLM::BadRequestError, with: :handle_bad_request

  private

  def handle_transient(error)
    StatsD.increment("llm.api_error", tags: ["type:transient"])
    Rails.logger.error("#{error.class}: #{error.message}")
    raise # re-raise after instrumenting
  end

  def handle_bad_request(error)
    StatsD.increment("llm.api_error", tags: ["type:bad_request"])
    Sentry.capture_exception(error) # this one is a bug in our pipeline
    raise
  end
end

Handlers apply however you reach the chat: SummaryAgent.new, .chat, .create, .create!, .find, and calls chained from a setter such as agent.with_temperature(0.2).ask("Hi"). The find in a generated ChatResponseJob is covered too:

chat = ChatAgent.find(chat_id)
chat.complete(&:broadcast) # handle_transient runs on a rate limit

Handlers run on an agent instance, so self, the agent’s inputs, and chat are available for instrumentation. With find and create, pass the inputs to that call. Pass a block instead of with: when the handler is a one-liner:

class SummaryAgent < ApplicationAgent
  rescue_from RubyLLM::RateLimitError do |error|
    Rails.logger.warn("#{self.class.name} rate limited: #{error.message}")
    nil
  end
end

The semantics match ActiveSupport::Rescuable:

  • Handlers are searched in reverse declaration order, so the last matching handler wins and a subclass can override an inherited one.
  • Exception classes may be given as strings (rescue_from "MyGem::Error"), which defers constant lookup until an exception is raised.
  • Handlers catch StandardErrors. Declaring a class that can never match one, such as Interrupt, raises ArgumentError. A module matches the errors that include it.
  • Re-raise inside the handler to let the caller see the exception. Without a raise, the exception is swallowed and the handler’s return value becomes the operation’s return value.
  • Exceptions that no handler matches are re-raised with their original backtrace.

Subclasses inherit the handlers declared when they are defined, so an ApplicationAgent base class is the usual place for shared error policy.

Rails-Backed Agents

Set chat_class to use your ActiveRecord chat model. bin/rails generate ruby_llm:install puts it in ApplicationAgent, the place for settings every agent in your app shares:

class ApplicationAgent < RubyLLM::Agent
  chat_class Chat
end

Agents inherit from it, the way models inherit from ApplicationRecord, and bin/rails generate ruby_llm:agent Support writes class SupportAgent < ApplicationAgent for you:

class WorkAssistant < ApplicationAgent
  model "gpt-6.1-sol"
  instructions "You are a helpful assistant."
  tools SearchDocs, LookupAccount
end

Then you can:

chat = WorkAssistant.create!(user: current_user)

chat = WorkAssistant.find(params[:id])

WorkAssistant.sync_instructions(chat)

create/create!/find require chat_class. Calling them without it raises an error.

Instruction persistence contract in Rails mode:

  • create/create! applies and persists instructions
  • find applies instructions at runtime only (no persistence side effects)
  • sync_instructions explicitly persists the current agent instructions

Using an Existing Chat Record

If you already have a Chat record, pass it to Agent.new(chat:) instead of calling Agent.find. This applies all agent configuration (instructions, tools, etc.) without an extra database query:

chat_record = Chat.find(params[:id])
chat = WorkAssistant.new(chat: chat_record)
chat.ask("Hello")

When to Use an Agent

These two styles are equivalent in capability, but optimized for different contexts.

Use progressive Chat#with_* when configuration is local and one-off:

chat = RubyLLM.chat(model: "gpt-6.1-sol")
chat.with_instructions("You are a helpful assistant.")
chat.with_tools(SearchDocs, LookupAccount)
chat.ask("Help me find docs about callbacks.")

Use agents when that setup should be centralized and reused:

class WorkAssistant < RubyLLM::Agent
  model "gpt-6.1-sol"
  instructions "You are a helpful assistant."
  tools SearchDocs, LookupAccount
end

WorkAssistant.new.ask("Help me find docs about callbacks.")

Next Steps