RubyLLM

bundle add ruby_llm --git https://github.com/crmne/ruby_llm --branch main

You don't need Python or JavaScript for AI. Use Ruby.

19 providers. One Ruby API.

Build with the models you want. Move between hosted and local providers without rewriting your application, or connect an OpenAI-compatible endpoint.

chat = RubyLLM.chat(model: "claude-opus-5")
chat.ask "Hello!"

Hetzner

Ollama

Ollama Cloud

TypeSafe AI

Browse models and pricing · Track usage and costs

Missing a provider?

Scaffold a provider gem with the same infrastructure as RubyLLM: configuration, tests, release setup, and a model registry that plugs into RubyLLM at runtime. Reuse a supported protocol without writing protocol code.

Provider gem guide

gem install ruby_llm --pre
ruby_llm provider-gem Acme \
  --api-base https://api.acme.example/v1

Start with one line.
Add files, tools, and agents

Just ask

Ask a question, then ask again. The chat keeps the conversation history, so every follow-up has context.

Chat guide

chat = RubyLLM.chat
chat.ask "What's the best way to learn Ruby?"
# Build small things you care about and read good Ruby code...

chat.ask "What should I build first?"
# A command-line tool for something you do every day...

Stream responses

Pass a block to display the response as it arrives, in your terminal or a Rails view.

Streaming guide

chat = RubyLLM.chat
chat.ask "Tell me a story about Ruby" do |chunk|
  print chunk.content
end

Send files

Ask about an image, a recording, or a PDF. Pass local paths, URLs, or several files at once with with: and RubyLLM prepares them for the model.

Attachments guide

chat = RubyLLM.chat(model: "gemini-3.7-flash")
chat.ask "What's in this image?", with: "ruby_conf.jpg"
chat.ask "Summarize this", with: "https://example.com/q3.pdf"
chat.ask "Recap this call", with: ["call.wav", "notes.pdf"]

Give the model tools

Describe a tool in a Ruby class and implement execute. RubyLLM runs the tool calls and sends the results back to the model.

Tools guide

class Weather < RubyLLM::Tool
  description "Get current weather"

  def execute(latitude:, longitude:)
    url = "https://api.open-meteo.com/v1/forecast?latitude=#{latitude}&longitude=#{longitude}&current=temperature_2m,wind_speed_10m"
    JSON.parse(Faraday.get(url).body)
  end
end

chat.with_tools(Weather).ask "What's the weather in Berlin?"

Connect to MCP servers

Describe a Model Context Protocol server in a Ruby class. Its tools become the model's tools, and you choose which ones it sees.

MCP client guide

class Linear < RubyLLM::MCP
  url "https://mcp.linear.app/mcp"
  inputs :user
  bearer_token { user.linear_token }
  only :list_issues, :get_issue
end

chat.with_mcp(Linear.new(user: current_user))
    .ask "What's blocking the release?"

Get structured output

Define the fields you want in a Ruby schema. Read the result as a Hash with response.parsed.

Structured output guide

class ProductSchema < Schematist::Schema
  string :name
  number :price
  array :features do
    string
  end
end

response = chat.with_schema(ProductSchema).ask "Analyze this product", with: "product.txt"

Define an agent

Give an agent its model and instructions in a Ruby class. Create an instance whenever you need it.

Agents guide

class SupportAgent < RubyLLM::Agent
  model "gpt-5.6-luna"
  instructions "Accept returns of unopened items within 30 days."
end

Evaluate your agents

Run your agent against a dataset of questions and expected answers. RubyLLM checks correctness by default. Use the same evaluation from Ruby, RSpec, Minitest, or Rake.

Evaluations guide

class SupportEvaluation < RubyLLM::Evaluation
  def perform(input)
    SupportAgent.new.ask(input)
  end
end

cases:
  - name: within_return_window
    inputs: Can I return an unopened item after 14 days?
    expected_output: Yes, unopened items can be returned within 30 days.

A complete AI framework for Ruby

Agents, evaluations, workflows, RAG, images, audio, and video. Built in, with usage tracking, OpenTelemetry tracing, and Rails integration to bring them into your app.

class IssueRefund < RubyLLM::Tool
  description "Issues a refund for an order"
  requires_approval

  def execute(order_id:) = Refunds.issue!(order_id)
end

chats = documents.map do |doc|
  RubyLLM.chat(model: "claude-sonnet-5")
    .with_instructions("Summarize in one paragraph.")
    .ask_later(doc.text)
end

batch = RubyLLM.batch(chats)

chat = RubyLLM.chat(model: "claude-sonnet-5")
  .with_caching
chat.with_instructions(File.read("review.md"))
  .cache_until_here
chat.ask "Review this diff", with: "large_diff.patch"

response = RubyLLM.chat
  .with_provider_tools(:web_search)
  .ask "What's the latest stable Ruby? Cite sources."

response.citations

documents = ["Ruby is expressive", "Python uses indentation"]
embeddings = RubyLLM.embed(documents)
ranked = RubyLLM.rerank("Ruby language", documents, model: "rerank-v3.5")

class Urgency < RubyLLM::Judge
  model "jev-latest"
  probability :urgent, "Does this need action today?"
end

Urgency.judge("My account is locked!").urgent.probability

response = chat.ask "Explain embeddings"
response.tokens.output
response.cost.total

transcript = RubyLLM.transcribe "meeting.wav"
RubyLLM.speak(transcript.text).save "transcript.mp3"

document = RubyLLM.ocr "scanned-contract.pdf"
puts document.markdown

RubyLLM.paint "a sunset over mountains in watercolor style"
RubyLLM.animate "a paper boat sailing down a rainy gutter"

RubyLLM.moderate("Some user-generated content").flagged?

RubyLLM.chat(model: "gpt-5.6-luna")
  .with_fallbacks("claude-sonnet-5")
  .ask("Explain Ruby blocks")

Coordinate agents with workflows, give them memory, and resume their work across jobs and deploys. Use separate configurations for each tenant and retries when requests fail. Follow requests with instrumentation, or explore community gems for monitoring dashboards.

Build your first feature

Feels at home in Rails

Save conversations with Active Record and stream replies with Hotwire. The generators give you a working chat UI. Watch the two-minute demo.

$ bundle add ruby_llm --git https://github.com/crmne/ruby_llm --branch main
$ bin/rails generate ruby_llm:install
$ bin/rails db:migrate
$ bin/rails generate ruby_llm:chat_ui

create  app/models/chat.rb
create  app/models/message.rb
create  app/controllers/chats_controller.rb
create  app/views/chats/show.html.erb
create  app/jobs/chat_response_job.rb

$ bin/rails server
=> Booting Puma
=> Rails application starting on http://localhost:3000
~

chat = Chat.create! model: "claude-opus-5"
chat.ask "What's in this file?", with: "report.pdf"

bin/rails generate ruby_llm:agent Support
bin/rails generate ruby_llm:tool Weather
bin/rails generate ruby_llm:schema Product

Keep your agents, tools, and schemas in app/. Store files with Active Storage and write your prompts in ERB templates. Use Turbo Streams to show replies and Active Job to run agents in the background.

Read the Rails guide

Ready for your coding agent

The RubyLLM gem includes a skill for your coding agent. Give it the API, examples, and Rails conventions that match your application.

npx skills add "$(bundle show ruby_llm)" --skill rubyllm

Run this from your application after installing RubyLLM. Choose your coding assistant when prompted. Skill setup and updates

Built with RubyLLM

Used in production by the teams behind these products.

Using RubyLLM? Get featured or Sponsor us

Gold sponsors

Their support helps fund RubyLLM's development. Thank you for investing in AI for Ruby.

Become a sponsor

Why Rubyists choose RubyLLM

Jorge ManrubiaPrincipal Programmer, 37signals

We are using OpenAI API using the fantastic RubyLLM gem from @paolino.

Nick WarwickFounding Engineer, Nodal Networks

Our Langgraph agent was failing. I took a gamble and rebuilt it using RubyLLM. Not only was it far simpler, it performed better.

Brendan SamekFounding Software Engineer, Build Canada

It feels natural. At Yuma, serving over 100,000 end users, our unified AI interface had accumulated so much cruft. RubyLLM is so much nicer than all of that.