How RubyLLM Works

Understand RubyLLM’s public API, its providers and protocols, and the services they share.

After reading this guide, you will know:

• How the RubyLLM API covers conversations and individual AI operations.
• What providers do and what protocols do.
• How model selection, configuration, and usage tracking work across the API.
• How Rails integration adds persistence, attachments, streaming, and jobs.

RubyLLM is an AI framework for Ruby and Rails. You build with its Ruby API: text generation and conversations on one side, individual AI operations on the other. Providers and protocols connect that API to AI services. Rails integration brings it into your application with the conventions you already use.

The RubyLLM API

Text Generation and Conversations

A Chat holds the conversation and its settings. Calling ask adds your question, generates a response, runs any tool calls, and returns a Message:

chat = RubyLLM.chat(model: "gpt-5.6-luna")
response = chat.ask "Help me plan a Ruby study group."
puts response.content

The conversation API provides:

• Messages and attachments for text, images, audio, and documents the model supports.
• Streaming to display a response as it arrives.
• Tools that let the model call your Ruby code, with optional human approval before execution.
• Structured output for results you read as a Hash through response.parsed.
• Agents that put a model, instructions, tools, and other settings in a reusable Ruby class.
• Loop control to generate a response, run tools, or advance one step at a time.

An agent uses the same conversation API:

class StudyPlanner < RubyLLM::Agent
  model "gpt-5.6-luna"
  instructions "Help organize practical Ruby study sessions."
end

response = StudyPlanner.new.ask "Plan a session about Ruby blocks."

Use ordinary Ruby methods and jobs to coordinate agents. Agentic Workflows covers handoffs and explicit loop control; Durable Agents covers resuming work across jobs and deploys.

Individual AI Operations

These operations take an input and return a result without maintaining a conversation. Use them directly:

image = RubyLLM.paint "A watercolor illustration of a Ruby study group"
image.save "study_group.png"

Use paint for images, animate for video, and speak for speech. In the other direction, transcribe turns audio into text and ocr extracts document text.

For search, embed creates vectors and rerank orders candidate documents by relevance. Use moderate to check content flags and categories.

Each result has readers for its output, such as transcription.text, document.markdown, or embedding.vectors. Generated images, video, and speech have a save method.

An individual operation may involve several requests. animate submits a job and waits for the video; animate_later returns a VideoJob immediately. Transcription can stream. You use the same operation API while RubyLLM handles that lifecycle.

Providers and Protocols

Your Ruby code describes the work. Providers and protocols translate it into calls to a particular service.

Providers

A provider represents the service you use. It supplies the base URL, authentication, configuration, and model catalog. It also selects the protocol for a model and supplies service-specific settings or routing rules.

For example, OpenAI and Ollama have different hosts and authentication requirements, but both can use the Chat Completions protocol. One provider can also expose several protocols: OpenAI supports Responses and Chat Completions, while Vertex AI routes models to several API formats.

Protocols

A protocol implements an API format. It renders requests, parses responses, handles streaming events, and turns service errors into RubyLLM errors. Examples include Chat Completions, Responses, Anthropic, Gemini, and Converse.

Services sometimes differ from the protocol they implement. A provider can select a small dialect of an existing protocol for those differences. Request and response format changes stay in the protocol; service URLs, credentials, and catalog rules stay in the provider. Your chat, tool, or agent does not need to know those details.

A service that implements an existing protocol can be supported without writing its request format again. See Custom Providers and Protocols for extending this layer.

What They Share

Model and Provider Selection

The Model Registry records which providers offer each model, its capabilities, limits, and pricing. RubyLLM uses it to resolve a model name to a provider:

chat = RubyLLM.chat(model: "gpt-5.6-luna")
chat.model.provider
# => "openai"

Pass provider: when you want to choose the service explicitly. You still need its credentials configured. Model Resolution explains aliases and selection when several providers offer the same model.

Capabilities depend on the model. Query them through the registry, or browse the Models page:

model = RubyLLM.models.find("gpt-5.6")
model.supports?(:vision)
model.supports?(:function_calling)

Protocol Selection

The provider chooses a protocol for the model and operation. For conversations, you can override its default:

chat = RubyLLM.chat(
  model: "gpt-5.6-luna",
  provider: :openai,
  protocol: :chat_completions
)

The chat still returns Message objects and uses the same tools and callbacks. See Choosing the Wire Protocol.

Configuration

Set application-wide defaults with RubyLLM.configure. Use a Context for an isolated set of credentials or defaults, such as one tenant’s configuration. Contexts expose the same entry points, including chat, paint, and embed:

context = RubyLLM.context do |config|
  config.openai_api_key = tenant.api_key
end

chat = context.chat

Set conversation-specific options with chainable methods such as with_instructions and with_tools. Use shared RubyLLM options where available; provider_options carries fields specific to a service. See Configuration.

Usage, Costs, and Instrumentation

Read usage from a response:

response = chat.ask "Explain Ruby blocks."
response.tokens.input
response.tokens.output
response.cost.total

The accounting follows provider attempts, so a successful answer can include the cost of earlier failed attempts. Unknown costs stay unknown. In Rails, the usage ledger keeps those attempts separately from messages. See Cost and Usage Tracking.

Instrumentation lets you observe calls across chats, tools, and individual operations. RubyLLM.workflow groups events from ordinary Ruby code into a named workflow.

Background Work and Provider Resources

Batches submit chats or embedding requests for provider-side processing. Stage them with ask_later or embed_later and submit them with RubyLLM.batch.

File Storage handles provider uploads and downloads with RubyLLM.upload and RubyLLM.download. Prompt Caching supports reusing prompt content, including provider-managed caches through RubyLLM.cache.

Error Handling provides common exceptions, retries, and model fallbacks. These services support the public API without requiring you to manage each provider’s request format.

Rails Integration

Rails integration keeps the Ruby API and adds the parts a Rails application needs. acts_as_chat and acts_as_message give your Active Record models the same conversation methods as the plain-Ruby objects:

class Chat < ApplicationRecord
  acts_as_chat
end

class Message < ApplicationRecord
  acts_as_message
end

chat = Chat.create!(model: "gpt-5.6-luna")
response = chat.ask "Help me plan a Ruby study group."

RubyLLM owns the supporting model-registry, tool-call, usage, and batch tables. Your application keeps its users, permissions, and other relationships on its own records. The framework can evolve its supporting data without asking you to maintain those models.

With Active Record, you can reload conversations, restore agents, and read usage through associations. Pass Active Storage attachments with with:, just as you pass a file in plain Ruby.

Use Hotwire to stream replies into a page and Active Job to run agents in your existing job backend. Persisted agents can resume their work in another process.

The generators set up these pieces in conventional Rails directories. They create persistence, agents, tools, schemas, and an optional chat UI with streaming already connected.

Individual operations such as paint, transcribe, and ocr also work directly in your Rails services and jobs. See Rails Integration to get started.

How the Parts Fit Together

Next Steps

Try the examples in Getting Started, then follow the guide for the feature you want to build.