# RubyLLM Documentation

> Developer documentation for RubyLLM 2.0.0.rc4 (prerelease), the Ruby-native AI framework for chats, tools, agents, RAG, multimodal workflows, and Rails applications.

The canonical docs cover installation, configuration, providers, core features, agents, Rails integration, and the complete reference. Also available: the [API reference Markdown index](https://rubyllm.com/api/index.md), the live [model registry as JSON](https://rubyllm.com/models.json), the [source](https://github.com/crmne/ruby_llm), and the [RubyGems package](https://rubygems.org/gems/ruby_llm).

## Full Documentation

- [Complete documentation](https://rubyllm.com/llms-full.txt): All canonical pages in one Markdown document.

## Overview

- [RubyLLM](https://rubyllm.com/): Build AI features in Ruby and Rails with chats, tools, agents, structured output, images, audio, and video across 17 providers.

## Start Here

- [Getting Started](https://rubyllm.com/getting-started/): Install RubyLLM and build with chats, tools, agents, images, video, audio, and document processing in Ruby and Rails.
- [How RubyLLM Works](https://rubyllm.com/overview/): Understand RubyLLM's public API, its providers and protocols, and the services they share.
- [Configuration](https://rubyllm.com/configuration/): Set provider credentials, choose default models, and share configuration across the RubyLLM API.
- [Provider Setup and Custom Endpoints](https://rubyllm.com/configuration-providers/): Per-provider API keys, organization headers, Bedrock and Vertex AI authentication, and OpenAI-compatible custom endpoints.
- [Connection, Logging and Contexts](https://rubyllm.com/configuration-connection/): Timeouts, retries, proxies, logging, the model registry file, and isolated multi-tenant contexts.
- [What's New in 2.0](https://rubyllm.com/whats-new-in-2-0/): Explore the expanded provider coverage, new AI operations, conversation controls, and Rails integration in RubyLLM 2.0.
- [AI Coding Assistants](https://rubyllm.com/ai-coding-assistants/): Install the RubyLLM skill to help your coding assistant use the API and documentation that match your application.

## Core Features

- [Chat](https://rubyllm.com/chat/): Start a conversation, give it instructions, and switch models while keeping the same Ruby API.
- [Attachments](https://rubyllm.com/attachments/): Ask questions about images, recordings, videos, and documents through one attachment API.
- [Stream Responses](https://rubyllm.com/streaming/): Learn how to display AI responses in real-time as they're generated
- [Structured Output](https://rubyllm.com/structured-output/): Get AI responses that match an exact JSON schema with required fields and types
- [Extended Thinking](https://rubyllm.com/thinking/): Give reasoning models more time and budget to deliberate, with optional access to thinking output
- [Citations](https://rubyllm.com/citations/): Get verifiable answers with normalized citations pointing at documents and web sources, on every provider that supports them
- [Prompt Rendering](https://rubyllm.com/prompt-rendering/): Render reusable ERB prompt templates from app/prompts with RubyLLM.render_prompt
- [Prompt Caching](https://rubyllm.com/prompt-caching/): Reuse stable prompt prefixes with automatic or explicit prompt caching
- [Advanced Request Control](https://rubyllm.com/chat-request-control/): Reach provider-specific features with custom parameters, wire protocols, request hooks, and HTTP headers
- [Tokens and Costs](https://rubyllm.com/cost-and-usage-tracking/): Read normalized token counts and costs for every response, chat, and provider attempt, including retries and cancellations.
- [Chat Event Handlers](https://rubyllm.com/chat-callbacks/): Hook into the chat lifecycle with additive callbacks for UI updates, logging, and analytics
- [Tools](https://rubyllm.com/tools/): Let AI call your Ruby code. Connect to databases, APIs, or any external system with function calling.
- [Tool Parameters](https://rubyllm.com/tool-parameters/): Declare tool arguments - from inferred signatures to full JSON Schema, rich return values, and provider-specific metadata.
- [Controlling Tool Execution](https://rubyllm.com/tool-execution/): Choose tools, require approval, run calls concurrently, and observe their results
- [Server Tools](https://rubyllm.com/server-tools/): Search the web, run code, and connect remote MCP tools through the same chat API
- [Embeddings](https://rubyllm.com/embeddings/): Create vectors from text, images, audio, video, and documents for search and similarity
- [Reranking](https://rubyllm.com/rerank/): Order candidate documents by how well they answer a query, the second stage of a retrieval pipeline
- [Image Generation](https://rubyllm.com/image-generation/): Generate and edit images from text prompts, reference images, and masks
- [Video Generation](https://rubyllm.com/video-generation/): Generate, edit, and extend videos from prompts, images, and audio
- [Audio Transcription](https://rubyllm.com/audio-transcription/): Convert speech to text with support for multiple languages and speaker diarization
- [Text to Speech](https://rubyllm.com/text-to-speech/): Convert text into spoken audio
- [Document OCR](https://rubyllm.com/ocr/): Extract text, tables, and images from PDFs and scans as clean markdown
- [Moderation](https://rubyllm.com/moderation/): Screen text and images for harmful content before it reaches your models
- [Files](https://rubyllm.com/files/): Upload files once, reuse them in chats and batches, and download provider results.
- [Tokenization](https://rubyllm.com/tokenization/): Inspect token IDs and count a conversation's input tokens before generating a response.

## Building Real Apps

- [Rails Integration](https://rubyllm.com/rails/): Use the RubyLLM API with Active Record, Active Storage, Hotwire, and your existing Rails jobs.
- [Persistence with acts_as](https://rubyllm.com/rails-persistence/): Persist application-owned chats and messages while RubyLLM manages its supporting records.
- [Streaming with Hotwire/Turbo](https://rubyllm.com/rails-streaming/): Broadcast AI responses token by token to the browser with Turbo Streams and background jobs.
- [Generators and App Conventions](https://rubyllm.com/rails-generators/): Scaffold chats, a chat UI, agents, tools, and schemas with RubyLLM's Rails generators and conventions.
- [Advanced Rails Configuration](https://rubyllm.com/rails-advanced-config/): Route models through different providers, use per-tenant contexts, persist cache boundaries, adjust provider payloads per request, and run fiber-safe.
- [Agents](https://rubyllm.com/agents/): Give agents instructions, tools, and schemas in a Ruby class, then use them in scripts, services, and Rails jobs.
- [Agentic Workflows](https://rubyllm.com/agentic-workflows/): Compose agents and AI operations with ordinary Ruby methods, from sequential steps to routing and parallel work.
- [Retrieval-Augmented Generation (RAG)](https://rubyllm.com/rag/): Retrieve relevant context from your own documents, then answer with that context
- [Durable Agents](https://rubyllm.com/durable-agents/): Run agent turns as background jobs that survive crashes, deploys, and the wait for a human decision.
- [Memory](https://rubyllm.com/memory/): Give agents short-term memory through the transcript and long-term memory your app owns, recalled by meaning.
- [Batches](https://rubyllm.com/batches/): Process chats and embeddings in batches, then collect their results when they are ready.
- [Scale with Async](https://rubyllm.com/async/): Run AI jobs with Solid Queue fiber workers, use Async for concurrent Ruby calls, and choose Async::Job for higher throughput.
- [Instrumentation and Observability](https://rubyllm.com/instrumentation/): Observe RubyLLM model operations, provider attempts, tool calls, workflows, batches, and model refreshes
- [Error Handling](https://rubyllm.com/error-handling/): Rescue provider errors, fall back to other models, and let automatic retries absorb transient failures
- [Hosted Research](https://rubyllm.com/hosted-research/): Run a hosted research agent with remote tools and collect its report through a durable job ID

## Reference

- [API Reference](https://rubyllm.com/api-reference/): RDoc documentation for every public RubyLLM class and method
- [Model Registry](https://rubyllm.com/models/): Find models by provider or capability, inspect their pricing, and refresh the registry in plain Ruby and Rails.
- [Model Resolution](https://rubyllm.com/model-resolution/): How RubyLLM turns a model name into a concrete model and provider, step by step, covering aliases, the registry, provider preference, and unlisted models.
- [Custom Endpoints and Unlisted Models](https://rubyllm.com/custom-endpoints/): Target OpenAI-compatible endpoints and use model IDs the registry doesn't list
- [Custom Providers and Protocols](https://rubyllm.com/custom-providers/): Teach RubyLLM to talk to a new AI service by writing a provider, a protocol, or both, and ship it as a gem.
- [Upgrading](https://rubyllm.com/upgrading/): Move from RubyLLM 1.16 to 2.0, update your Ruby code, and migrate your Rails data.
- [RubyLLM Ecosystem](https://rubyllm.com/ecosystem/): Extend RubyLLM with MCP servers, structured schemas, instrumentation, monitoring and community-built tools for production AI apps.

## Pages

- [About RubyLLM](https://rubyllm.com/about/): RubyLLM is an open source Ruby gem for building AI applications across major AI providers with one consistent Ruby framework.