RubyLLM

bundle add ruby_llm --git https://github.com/crmne/ruby_llm --branch main

You don't need Python or JavaScript for AI. Use Ruby.

19 providers. One API.

Build with the models you want. Move between hosted and local providers without rewriting your application, or connect an OpenAI-compatible endpoint.

chat = RubyLLM.chat(model: "claude-opus-5")
chat.ask "Hello!"

Hetzner

Ollama

Ollama Cloud

TypeSafe AI

Browse models and pricing · Track usage and costs

Missing a provider?

Scaffold a provider gem with the same infrastructure as RubyLLM: configuration, tests, release setup, and a model registry that plugs into RubyLLM at runtime. Reuse a supported protocol without writing protocol code.

Provider gem guide

gem install ruby_llm --pre
ruby_llm provider-gem Acme \
  --api-base https://api.acme.example/v1

A complete AI framework for Ruby

Build agents, run code safely, evaluate results, and track usage with shared Ruby and Rails conventions.

Let agents do real work, safely

Run code on your own servers

The model writes Ruby, Python, JavaScript, or shell to calculate and make files. Each run happens in a sandbox on your servers that cannot read your application or its secrets. The files it makes come back as attachments.

Code execution guide

chat = RubyLLM.chat
  .with_code_execution(python: %w[matplotlib])
chat.ask "Chart revenue by month", with: "revenue.csv"

chat.messages.find(&:tool_result?).attachments

Ask before reaching further

Describe wider sandboxes as Ruby classes and list the ones the model may ask for. A call that needs one waits for a person to approve it, runs once, and never turns into standing access.

Sandboxes guide

class WorkspaceSandbox < MXC::Sandbox
  writable Rails.root.join("storage/workspace")
end

chat = RubyLLM.chat
  .with_code_execution(escalate_to: [WorkspaceSandbox])
chat.ask "Add a changelog entry for the export"

chat.approve(chat.pending_approvals.first)

Teach them your workflows

Write skills in the open Agent Skills format: instructions, scripts, and reference files in a folder. The model loads a skill only when a task needs it and runs its scripts in the sandbox.

Skills guide

app/skills/invoices/
├── SKILL.md
├── references/TAX_RULES.md
└── scripts/build_invoice.py

chat = RubyLLM.chat
  .with_skills(:invoices)
  .with_code_execution(python: %w[reportlab])
chat.ask "Bill order 1042 to ACME", with: "order.json"

Connect them to your systems

Describe each MCP server as a Ruby class in app/mcp. Inputs pass in the current user, only picks the tools the model sees, and OAuth credentials are stored encrypted in your database.

MCP client guide

class Linear < RubyLLM::MCP
  url "https://mcp.linear.app/mcp"
  inputs :user
  oauth owner: :user
  only :list_issues, :get_issue
end

chat.with_mcp(Linear.new(user: current_user))
    .ask "What's blocking the release?"

Resume across jobs and deploys

A saved conversation is the agent's whole state. Any job can pick it up and continue, after a crash, a deploy, or a wait for approval.

Durable agents guide

class AgentJob < ApplicationJob
  def perform(chat_id)
    SupportAgent.find(chat_id).complete
  end
end

chat = SupportAgent.find(chat_id)
chat.approve(chat.pending_approvals.first)
AgentJob.perform_later(chat.id)

Know they work

Keep questions and expected answers in a dataset next to an evaluation class in app/evals. RubyLLM checks each answer for correctness. Run the same evaluation from RSpec, Minitest, or Rake, and fail CI when a case fails.

Evaluations guide

class SupportEvaluation < RubyLLM::Evaluation
  def perform(input)
    SupportAgent.new.ask(input)
  end
end

cases:
  - name: within_return_window
    inputs: Can I return an unopened item after 14 days?
    expected_output: Yes, within 30 days if it is unopened.

bin/rails "ruby_llm:eval[SupportEvaluation]"

Run them in production

class IssueRefund < RubyLLM::Tool
  description "Issues a refund for an order"
  requires_approval

  def execute(order_id:) = Refunds.issue!(order_id)
end

RubyLLM.chat(model: "gpt-6-luna")
  .with_fallbacks("claude-sonnet-5")
  .ask("Explain Ruby blocks")

chat = RubyLLM.chat(model: "claude-sonnet-5")
  .with_caching
chat.with_instructions(File.read("review.md"))
  .cache_until_here
chat.ask "Review this diff", with: "changes.patch"

RubyLLM.with_usage_owner(current_account) do
  SupportAgent.new.ask "Where is order 42?"
end

current_account.ruby_llm_usages.sum(:total_cost)

OpenTelemetry::SDK.configure
RubyLLM::OpenTelemetry.enable

SupportAgent.new.ask "Where is order 42?"
# chat → execute_tool lookup_order → chat

RubyLLM.workflow("Write article") do |workflow|
  notes = workflow.step("Research") do
    ResearchAgent.new.ask(topic).content
  end
  workflow.step("Draft") do
    WriterAgent.new.ask(notes).content
  end
end

Work with your data

documents = ["Ruby is expressive", "Go is fast"]
embeddings = RubyLLM.embed(documents)
ranked = RubyLLM.rerank("Ruby language", documents,
                        model: "rerank-v3.5")

response = RubyLLM.chat
  .with_provider_tools(:web_search)
  .ask "What's the latest Ruby release?"

response.citations

class Urgency < RubyLLM::Judge
  model "jev-latest"
  probability :urgent, "Needs action today?"
end

judgment = Urgency.judge("My account is locked!")
judgment.urgent.probability

chats = documents.map do |doc|
  RubyLLM.chat(model: "claude-sonnet-5")
    .with_instructions("Summarize in one line.")
    .ask_later(doc.text)
end

batch = RubyLLM.batch(chats)

Read, hear, and make media

document = RubyLLM.ocr "scanned-contract.pdf"
puts document.markdown

transcript = RubyLLM.transcribe "meeting.wav"
RubyLLM.speak(transcript.text).save "transcript.mp3"

RubyLLM.paint "a watercolor sunset over mountains"
RubyLLM.animate "a paper boat in a rainy gutter"

RubyLLM.moderate(comment.body).flagged?

Give agents memory. Use separate configurations for each tenant and retries when requests fail, or explore community gems for monitoring dashboards.

Feels at home in Rails

Save conversations with Active Record and stream replies with Hotwire. The generators give you a working chat UI. Watch the two-minute demo.

$ bundle add ruby_llm --git https://github.com/crmne/ruby_llm --branch main
$ bin/rails generate ruby_llm:install
$ bin/rails db:migrate
$ bin/rails generate ruby_llm:chat_ui

create  app/models/chat.rb
create  app/models/message.rb
create  app/controllers/chats_controller.rb
create  app/views/chats/show.html.erb
create  app/jobs/chat_response_job.rb

$ bin/rails server
=> Booting Puma
=> Rails application starting on http://localhost:3000
~

class Chat < ApplicationRecord
  acts_as_chat
end

chat = Chat.create! model: "claude-opus-5"
chat.ask "What's in this file?", with: "report.pdf"
chat.messages.last.content

app/
├── agents/     support_agent.rb
├── tools/      issue_refund.rb
├── schemas/    product_schema.rb
├── prompts/    support_agent/
├── skills/     invoices/SKILL.md
├── sandboxes/  workspace_sandbox.rb
├── mcp/        linear.rb
├── evals/      support_evaluation.rb
└── judges/     urgency_judge.rb

Generators create each piece where Rails expects it. Store files with Active Storage and write your prompts in ERB templates. Use Turbo Streams to show replies and Active Job to run agents in the background, one process per chat, with approvals that survive restarts.

Read the Rails guide

Start with one line.
Add files, tools, and agents

Everything above is built on one chat API. Follow it from your first question to your first agent.

Just ask

Ask a question, then ask again. The chat keeps the conversation history, so every follow-up has context.

Chat guide

chat = RubyLLM.chat
chat.ask "What's the best way to learn Ruby?"
# Build small things you care about...

chat.ask "What should I build first?"
# A command-line tool for something you do every day...

Stream responses

Pass a block to display the response as it arrives, in your terminal or a Rails view.

Streaming guide

chat = RubyLLM.chat
chat.ask "Tell me a story about Ruby" do |chunk|
  print chunk.content
end

Send files

Ask about an image, a recording, or a PDF. Pass local paths, URLs, or several files at once with with: and RubyLLM prepares them for the model.

Attachments guide

chat = RubyLLM.chat(model: "gemini-3.7-flash")
chat.ask "What's in this image?", with: "ruby_conf.jpg"
chat.ask "Summarize", with: "https://example.com/q3.pdf"
chat.ask "Recap this call", with: ["call.wav", "notes.pdf"]

Give the model tools

Describe a tool in a Ruby class and implement execute. RubyLLM runs the tool calls and sends the results back to the model.

Tools guide

class Weather < RubyLLM::Tool
  description "Get current weather"

  def execute(latitude:, longitude:)
    url = "https://api.open-meteo.com/v1/forecast"
    query = { latitude:, longitude:, current_weather: true }
    JSON.parse(Faraday.get(url, query).body)
  end
end

chat.with_tools(Weather).ask "What's the weather in Berlin?"

Get structured output

Define the fields you want in a Ruby schema. Read the result as a Hash with response.parsed.

Structured output guide

class ProductSchema < Schematist::Schema
  string :name
  number :price
  array :features do
    string
  end
end

response = chat.with_schema(ProductSchema)
               .ask "Analyze this", with: "product.txt"
response.parsed["features"]

Define an agent

Give an agent its model, instructions, and tools in a Ruby class. In Rails, it saves its conversation, and find picks it up again in any request or job.

Agents guide

class SupportAgent < RubyLLM::Agent
  chat_model Chat
  model "gpt-6.1-sol"
  instructions "Refund unopened items within 30 days."
  tools LookupOrder, IssueRefund
end

SupportAgent.find(chat_id).ask "Where's my refund?"

Build your first feature

Ready for your coding agent

The RubyLLM gem includes a skill for your coding agent. Give it the API, examples, and Rails conventions that match your application.

npx skills add "$(bundle show ruby_llm)" --skill rubyllm

Run this from your application after installing RubyLLM. Choose your coding assistant when prompted. Skill setup and updates

Built with RubyLLM

Used in production by the teams behind these products.

Using RubyLLM? Get featured or Sponsor us

Gold sponsors

Their support helps fund RubyLLM's development. Thank you for investing in AI for Ruby.

Become a sponsor

Why Rubyists choose RubyLLM

Jorge ManrubiaPrincipal Programmer, 37signals

We are using OpenAI API using the fantastic RubyLLM gem from @paolino.

Nick WarwickFounding Engineer, Nodal Networks

Our Langgraph agent was failing. I took a gamble and rebuilt it using RubyLLM. Not only was it far simpler, it performed better.

Brendan SamekFounding Software Engineer, Build Canada

It feels natural. At Yuma, serving over 100,000 end users, our unified AI interface had accumulated so much cruft. RubyLLM is so much nicer than all of that.