# Error Handling

Rescue provider errors, fall back to other models, and let automatic retries absorb transient failures

After reading this guide, you will know:

*   RubyLLM's error hierarchy.
*   How to rescue specific types of errors.
*   How to access details from the original API response.
*   How to keep what providers quote back out of error messages.
*   How to see the shape of a request a provider rejected.
*   How errors are handled during streaming.
*   How to fall back to another model when a provider has a transient failure.
*   Best practices for handling errors within Tools.
*   RubyLLM's automatic retry behavior.
*   How to enable debug logging.

## RubyLLM Error Hierarchy

Provider/API operation errors inherit from `RubyLLM::Error`. Local setup and programming errors inherit from `StandardError` directly, so rescue them separately when you need to handle them.

```ruby
RubyLLM::Error                    # Base error class for provider-operation issues
    RubyLLM::BadRequestError      # 400: Invalid request parameters
    RubyLLM::ConnectionError      # The provider or an MCP server could not be reached or timed out
    RubyLLM::UnauthorizedError    # 401: API key issues
    RubyLLM::PaymentRequiredError # 402: Billing issues, such as an account out of credit
    RubyLLM::ForbiddenError       # 403: Permission issues
    RubyLLM::ContextLengthExceededError # Context/token limits exceeded (provider-specific)
    RubyLLM::ContentFilterError   # Provider content filters blocked a transcription
    RubyLLM::ToolCallParseError   # Provider returned malformed tool-call arguments
        RubyLLM::ToolCallTruncatedError # The output token limit cut off a tool call
    RubyLLM::UnsupportedAttachmentError # Attachment cannot be sent to this provider/model
    RubyLLM::UnsupportedServerToolError # Provider/protocol does not define this server tool
    RubyLLM::RateLimitError       # 429: Rate limit exceeded
    RubyLLM::ServerError          # 500: Provider server error
    RubyLLM::ServiceUnavailableError # 502/503/504: Service unavailable
    RubyLLM::OverloadedError      # 529: Service overloaded (Specific providers)

# Local Errors (inherit from StandardError)
RubyLLM::ConfigurationError   # Missing required configuration (e.g., API key)
RubyLLM::PromptNotFoundError  # Prompt file not found
RubyLLM::ModelNotFoundError   # Requested model ID not found in registry
RubyLLM::InvalidRoleError     # Invalid role symbol used for a message
RubyLLM::InvalidToolChoiceError # Invalid tool choice option
RubyLLM::PendingToolCallsError # A new question was staged before pending tools finished
RubyLLM::ModelRegistryError   # Model registry could not be fetched, parsed, or persisted
RubyLLM::CancelledError       # An in-flight chat operation was cancelled
```

## Basic Error Handling

Catch the base `RubyLLM::Error` to handle most provider-operation issues.

```ruby
begin
  chat = RubyLLM.chat
  response = chat.ask "Translate 'hello' to French."
  puts response.content
rescue RubyLLM::ConfigurationError => e
  puts "Configuration missing: #{e.message}"
  # Abort or prompt for configuration
rescue RubyLLM::Error => e
  puts "An API error occurred: #{e.message}"
  # logger.error "RubyLLM API Error: #{e.class} - #{e.message}"
end
```

## Handling Specific Errors

For more granular control, rescue specific error classes. This allows you to implement different recovery strategies based on the error type.

```ruby
begin
  chat = RubyLLM.chat
  response = chat.ask "Generate a complex report."
rescue RubyLLM::UnauthorizedError
  puts "Authentication failed. Please check your API key configuration."
  # Maybe exit or redirect to config settings
rescue RubyLLM::PaymentRequiredError
  puts "Payment required. Please check your provider account balance or plan."
  # Notify admin or user
rescue RubyLLM::RateLimitError
  puts "Rate limit hit. Please wait a moment before trying again."
  # Implement backoff/retry logic (though RubyLLM has some built-in retries)
rescue RubyLLM::ContextLengthExceededError
  puts "Your prompt/conversation is too large for this model."
  # Reduce prompt size or use a model with a larger context window
rescue RubyLLM::ServiceUnavailableError
  puts "The AI service is temporarily unavailable. Please try again later."
  # Maybe offer a fallback or notify user
rescue RubyLLM::ConnectionError => e
  puts e.timeout? ? "The request timed out." : "Could not reach the provider."
  # e.cause holds the original Faraday error
rescue RubyLLM::BadRequestError => e
  puts "Invalid request sent to the API: #{e.message}"
  # Check the data being sent
rescue RubyLLM::ModelNotFoundError => e
  puts "Error: #{e.message}. Check available models with RubyLLM.models.all"
rescue RubyLLM::Error => e
  puts "An unexpected API error occurred: #{e.message}"
end
```

## Accessing API Response Details

Instances of `RubyLLM::Error` (and its subclasses related to API responses) hold the original `Faraday::Response` object in the `response` attribute. This can be useful for debugging or extracting provider-specific error codes.

```ruby
begin
  chat = RubyLLM.chat(model: 'gpt-6.1-sol') # Assume this requires a specific org sometimes
  response = chat.ask "Some specific query"
rescue RubyLLM::ForbiddenError => e
  puts "Access forbidden: #{e.message}"
  if e.response&.body&.include?('invalid_organization')
    puts "Hint: Check if your API key is enabled for the correct OpenAI organization."
  end
  puts "Status Code: #{e.response&.status}"
  # puts "Full Response Body: #{e.response&.body}" # For deep debugging
end
```

## Keeping Prompts Out of Error Messages

Providers often quote the request back when they refuse it: a fragment of the prompt, a file name, or a tool argument ends up in `e.message`, and from there in your logs and your error tracker. Turn on `redact_error_messages` and the message names the provider, the HTTP status, and the provider's error code, and nothing the provider wrote:

```ruby
RubyLLM.configure do |config|
  config.redact_error_messages = true
end

begin
  chat.ask "Summarize the attached contract"
rescue RubyLLM::BadRequestError => e
  e.message          # => "openai returned HTTP 400 (invalid_request_error)"
  e.provider         # => "openai"
  e.error_code       # => "invalid_request_error"
  e.provider_message # => what the provider wrote, quotes and all
  e.response_body    # => the raw body
end
```

The error code comes from the fields providers keep it in, such as OpenAI's `code` or `type`, Gemini's `status`, or Bedrock's error type, and only when it reads like a code: a word of letters, digits, `_`, `.`, `:`, or `-`, at most 64 characters. RubyLLM still picks the error class from what the provider wrote, so a context window error is still a `ContextLengthExceededError`. Messages RubyLLM writes itself, such as `Provider returned no completion message`, stay as they are, and an MCP server's errors read like `The MCP server returned an error (-32602)`.

`provider_message` and `response_body` are there when you want the provider's words, so reading them is a choice your code makes. The setting is off by default. Set it on the global configuration: every context follows it.

## Describing a Rejected Request

Some providers reject a conversation without saying what is wrong with it, answering with little more than "Request contains an invalid argument." `request_shape` shows you the shape of what the provider received, and the problems RubyLLM finds in it:

```ruby
begin
  chat.ask "What's due today?"
rescue RubyLLM::BadRequestError => e
  Rails.logger.error("#{e.message}\n#{e.request_shape}")
  raise
end
```

```text
Request contains an invalid argument.
gemini, gemini-2.5-flash, model call 1 of the turn
payload keys: contents, systemInstruction, generationConfig, tools
thinking: includeThoughts true, thinkingBudget -1
instructions: text (240 chars)
#0 user: text (17 chars)
#1 model: call find_tasks (args 40 chars), signed
#2 user: result find_tasks (6859 chars)
#3 model: thinking (no text), signed, text (1075 chars)
#4 user: text (31 chars)
tools: find_tasks, add_task
tool round at #1: 1 call, 1 result, paired
problem at #3, part 0: thinking part carries no data
```

The first lines name the provider and model, say which model call of the turn the request was, and list the payload's top-level keys and its thinking settings. Then comes one line per turn, numbered the way providers number turns in their error messages and named with the payload's own roles. Each part shows its size in characters, or in bytes for inline media. A tool result names the call it answers, and `signed` marks a part that carries a thinking signature.

The last lines are what RubyLLM found: each tool round with its calls and results, and the problems providers are known to refuse, such as a part with no data, calls and results that do not pair up, or a step without the signature its provider requires. Here, turn 3 sends a thinking part that holds a signature and nothing else. A tool result that answers no call shows the id of the call it names, the only id a shape ever shows.

The shape holds no text, tool arguments, tool results, signatures, URLs, or file data, so you can send it to your error tracker. `to_h` has the same keys every time:

```ruby
Rails.error.report(e, context: { request_shape: e.request_shape&.to_h })
```

A long conversation keeps its first 10 and last 50 turns, and `omitted_turns` counts the rest, while problems cover every turn. An error raised for a conversation request has a `request_shape`, including token counts and compaction. It is `nil` for errors raised before anything is sent, such as an unsupported attachment, and for operations that send no conversation, such as embeddings.

## Error Handling During Streaming

When using streaming with a block, errors can occur *during* the stream after some chunks have already been processed. The `ask` method will raise the error *after* the block execution finishes or is interrupted by the error.

```ruby
begin
  chat = RubyLLM.chat
  accumulated_content = ""
  chat.ask "Generate a very long story..." do |chunk|
    print chunk.content
    accumulated_content << chunk.content
  end
  puts "\nStream completed successfully."
rescue RubyLLM::RateLimitError
  puts "\nStream interrupted by rate limit. Partial content received:"
  puts accumulated_content
rescue RubyLLM::Error => e
  puts "\nStream failed: #{e.message}. Partial content received:"
  puts accumulated_content
end
```

Your block will execute for chunks received *before* the error. Because `ask` raises, it does not return a final response in this case; handle the exception and any partial content you accumulated.

## Model Fallbacks

Use `with_fallbacks` when you want RubyLLM to try another model after the current model fails with a transient provider or network error.

```ruby
chat = RubyLLM.chat(model: "gpt-4.1")
              .with_fallbacks("gpt-4.1-mini", "claude-haiku-4-5")

response = chat.ask("Summarize this incident report.")
```

Fallbacks are tried in order. The fallback only applies to that generation attempt; after the response finishes or the error bubbles up, the chat returns to its original model.

By default, fallbacks handle rate limits, server errors, service unavailable errors, overload errors, timeouts, and connection failures. Pass `on:` to choose the errors yourself:

```ruby
chat.with_fallbacks(
  "gpt-4.1-mini",
  on: [RubyLLM::RateLimitError, RubyLLM::ServiceUnavailableError]
)
```

Cancelling a chat never triggers a fallback, whatever `on:` lists. `chat.cancel` raises `RubyLLM::CancelledError` and no fallback model runs. The same holds for a persisted chat that another process takes over mid-run: this process steps aside and no fallback model runs. An error your streaming block raises is yours to handle too: it propagates as it is, and the block does not run again on a fallback model.

Fallbacks can be model IDs or `RubyLLM::Model` objects:

```ruby
chat.with_fallbacks(
  RubyLLM.models.find("claude-haiku-4-5", provider: :anthropic)
)
```

### Fallback Callbacks

Use `before_fallback` and `after_fallback` to observe each fallback attempt:

```ruby
chat.before_fallback do |fallback|
  Rails.logger.info(
    "Falling back from #{fallback.from.id} to #{fallback.to.id}: #{fallback.error.class}"
  )
end

chat.after_fallback do |fallback|
  if fallback.succeeded?
    Rails.logger.info("Fallback succeeded with #{fallback.to.id}")
  else
    Rails.logger.warn("Fallback failed with #{fallback.fallback_error.class}")
  end
end
```

The callback receives a `RubyLLM::Fallback` with the configured target (`id`, `provider`, `model`) and the runtime attempt details (`from`, `to`, `error`, `attempt`, `response`, `fallback_error`, `streaming?`, and `chunks_yielded?`).

When streaming has already yielded chunks before a fallback-worthy error, RubyLLM cannot take those chunks back. It starts a new assistant message lifecycle for the fallback response, and `fallback.chunks_yielded?` lets your UI or logs distinguish that case.

## Handling Errors Within Tools

When building [Tools](/next/tools/), you need to decide how errors within the tool's `execute` method should be handled:

1.  **Return Error to LLM:** If the error is something the LLM might be able to recover from (e.g., invalid parameters provided by the LLM, temporary lookup failure), return a Hash containing an `:error` key. The LLM will see this error message as the tool's output and may try again or use a different approach.

```ruby
    class Weather < RubyLLM::Tool
      # ... params ...
      def execute(location:)
        if location.blank?
          return { error: "Location cannot be blank. Please provide a city name." }
        end
        # ... perform API call ...
      rescue Faraday::TimeoutError
        { error: "Weather API timed out. Please try again later." }
      end
    end
    ```

2.  **Raise Error for Application:** If the error indicates a problem with the tool itself or the application's state (e.g., database connection lost, configuration error, unrecoverable external API failure), `raise` an exception as normal. This will halt the RubyLLM interaction and bubble up to your application's main error handling (`begin/rescue`).

```ruby
    class DatabaseQueryTool < RubyLLM::Tool
      # ... params ...
      def execute(query:)
        User.find_by_sql(query) # Example query
      rescue ActiveRecord::ConnectionNotEstablished => e
        raise e # Let the application's error handling take over.
      rescue StandardError => e
        # Maybe return less critical errors to the LLM
        { error: "Database query failed: #{e.message}" }
      end
    end
    ```

## Agent-Level Handlers

When you use [Agents](/next/agents/), `rescue_from` moves error handling out of every call site and into the agent class:

```ruby
class ApplicationAgent < RubyLLM::Agent
  rescue_from RubyLLM::RateLimitError, RubyLLM::ConnectionError, with: :handle_transient

private

def handle_transient(error)
    StatsD.increment("llm.api_error", tags: ["type:transient"])
    raise
  end
end
```

Handlers cover the agent's chat operations (`ask`, `say`, `ask_later`, `complete`, `generate`, `run_tools`, `step`), run on the agent instance, and follow `ActiveSupport::Rescuable` semantics. See [Handling Errors with `rescue_from`](/next/agents/#handling-errors-with-rescue_from).

## Automatic Retries

RubyLLM automatically retries requests that fail due to transient network or server issues using Faraday's retry middleware.
Retries are driven by error classification (exception types), not raw HTTP status codes alone.

Retries are attempted for:

*   Network failures (`RubyLLM::ConnectionError`: timeouts, refused or dropped connections, and TLS failures)
*   Rate limit errors (`RubyLLM::RateLimitError`, often HTTP 429)
*   Server-side errors (`RubyLLM::ServerError`, `RubyLLM::ServiceUnavailableError`, `RubyLLM::OverloadedError` / HTTP 500, 502, 503, 504, 529)

`RubyLLM::ContextLengthExceededError` is not retried.

RubyLLM honors `Retry-After` and `retry-after-ms` headers across providers. `Retry-After` takes precedence when both are present. Invalid millisecond delays fall back to provider-specific timing or the configured backoff. If the requested delay exceeds `retry_max_interval`, RubyLLM raises the error without retrying early.

Requests that create something on the provider's side are never retried: batch submissions, video generation jobs, file uploads, and content caches. When such a request reaches the provider but its response is lost, retrying would create a second job you still pay for, so RubyLLM raises the error and lets you decide whether to submit again.

You can configure retry behavior via `RubyLLM.configure`:

```ruby
RubyLLM.configure do |config|
  config.max_retries = 5 # Default: 3
  config.retry_interval = 0.5 # Default: 0.1
  # config.retry_backoff_factor = 2 # Default: 2
  # config.retry_interval_randomness = 0.5 # Default: 0.5
end
```

## Debugging

If you encounter unexpected errors or behavior, enable debug logging by setting the `RUBYLLM_DEBUG` environment variable:

```bash
export RUBYLLM_DEBUG=true
```

Debug logs show request and response headers and bodies, with API keys filtered. Use them to inspect the call that failed.

## Next Steps

*   [Using Tools](/next/tools/)
*   [Streaming Responses](/next/streaming/)
*   [Rails Integration](/next/rails/)

Error Handling

Rescue provider errors, fall back to other models, and let automatic retries absorb transient failures

After reading this guide, you will know:

RubyLLM’s error hierarchy.
How to rescue specific types of errors.
How to access details from the original API response.
How to keep what providers quote back out of error messages.
How to see the shape of a request a provider rejected.
How errors are handled during streaming.
How to fall back to another model when a provider has a transient failure.
Best practices for handling errors within Tools.
RubyLLM’s automatic retry behavior.
How to enable debug logging.

RubyLLM Error Hierarchy

Provider/API operation errors inherit from RubyLLM::Error. Local setup and programming errors inherit from StandardError directly, so rescue them separately when you need to handle them.

RubyLLM::Error                    # Base error class for provider-operation issues
    RubyLLM::BadRequestError      # 400: Invalid request parameters
    RubyLLM::ConnectionError      # The provider or an MCP server could not be reached or timed out
    RubyLLM::UnauthorizedError    # 401: API key issues
    RubyLLM::PaymentRequiredError # 402: Billing issues, such as an account out of credit
    RubyLLM::ForbiddenError       # 403: Permission issues
    RubyLLM::ContextLengthExceededError # Context/token limits exceeded (provider-specific)
    RubyLLM::ContentFilterError   # Provider content filters blocked a transcription
    RubyLLM::ToolCallParseError   # Provider returned malformed tool-call arguments
        RubyLLM::ToolCallTruncatedError # The output token limit cut off a tool call
    RubyLLM::UnsupportedAttachmentError # Attachment cannot be sent to this provider/model
    RubyLLM::UnsupportedServerToolError # Provider/protocol does not define this server tool
    RubyLLM::RateLimitError       # 429: Rate limit exceeded
    RubyLLM::ServerError          # 500: Provider server error
    RubyLLM::ServiceUnavailableError # 502/503/504: Service unavailable
    RubyLLM::OverloadedError      # 529: Service overloaded (Specific providers)

# Local Errors (inherit from StandardError)
RubyLLM::ConfigurationError   # Missing required configuration (e.g., API key)
RubyLLM::PromptNotFoundError  # Prompt file not found
RubyLLM::ModelNotFoundError   # Requested model ID not found in registry
RubyLLM::InvalidRoleError     # Invalid role symbol used for a message
RubyLLM::InvalidToolChoiceError # Invalid tool choice option
RubyLLM::PendingToolCallsError # A new question was staged before pending tools finished
RubyLLM::ModelRegistryError   # Model registry could not be fetched, parsed, or persisted
RubyLLM::CancelledError       # An in-flight chat operation was cancelled

Basic Error Handling

Catch the base RubyLLM::Error to handle most provider-operation issues.

begin
  chat = RubyLLM.chat
  response = chat.ask "Translate 'hello' to French."
  puts response.content
rescue RubyLLM::ConfigurationError => e
  puts "Configuration missing: #{e.message}"
  # Abort or prompt for configuration
rescue RubyLLM::Error => e
  puts "An API error occurred: #{e.message}"
  # logger.error "RubyLLM API Error: #{e.class} - #{e.message}"
end

Handling Specific Errors

For more granular control, rescue specific error classes. This allows you to implement different recovery strategies based on the error type.

begin
  chat = RubyLLM.chat
  response = chat.ask "Generate a complex report."
rescue RubyLLM::UnauthorizedError
  puts "Authentication failed. Please check your API key configuration."
  # Maybe exit or redirect to config settings
rescue RubyLLM::PaymentRequiredError
  puts "Payment required. Please check your provider account balance or plan."
  # Notify admin or user
rescue RubyLLM::RateLimitError
  puts "Rate limit hit. Please wait a moment before trying again."
  # Implement backoff/retry logic (though RubyLLM has some built-in retries)
rescue RubyLLM::ContextLengthExceededError
  puts "Your prompt/conversation is too large for this model."
  # Reduce prompt size or use a model with a larger context window
rescue RubyLLM::ServiceUnavailableError
  puts "The AI service is temporarily unavailable. Please try again later."
  # Maybe offer a fallback or notify user
rescue RubyLLM::ConnectionError => e
  puts e.timeout? ? "The request timed out." : "Could not reach the provider."
  # e.cause holds the original Faraday error
rescue RubyLLM::BadRequestError => e
  puts "Invalid request sent to the API: #{e.message}"
  # Check the data being sent
rescue RubyLLM::ModelNotFoundError => e
  puts "Error: #{e.message}. Check available models with RubyLLM.models.all"
rescue RubyLLM::Error => e
  puts "An unexpected API error occurred: #{e.message}"
end

Accessing API Response Details

Instances of RubyLLM::Error (and its subclasses related to API responses) hold the original Faraday::Response object in the response attribute. This can be useful for debugging or extracting provider-specific error codes.

begin
  chat = RubyLLM.chat(model: 'gpt-6.1-sol') # Assume this requires a specific org sometimes
  response = chat.ask "Some specific query"
rescue RubyLLM::ForbiddenError => e
  puts "Access forbidden: #{e.message}"
  if e.response&.body&.include?('invalid_organization')
    puts "Hint: Check if your API key is enabled for the correct OpenAI organization."
  end
  puts "Status Code: #{e.response&.status}"
  # puts "Full Response Body: #{e.response&.body}" # For deep debugging
end

Keeping Prompts Out of Error Messages

Providers often quote the request back when they refuse it: a fragment of the prompt, a file name, or a tool argument ends up in e.message, and from there in your logs and your error tracker. Turn on redact_error_messages and the message names the provider, the HTTP status, and the provider’s error code, and nothing the provider wrote:

RubyLLM.configure do |config|
  config.redact_error_messages = true
end

begin
  chat.ask "Summarize the attached contract"
rescue RubyLLM::BadRequestError => e
  e.message          # => "openai returned HTTP 400 (invalid_request_error)"
  e.provider         # => "openai"
  e.error_code       # => "invalid_request_error"
  e.provider_message # => what the provider wrote, quotes and all
  e.response_body    # => the raw body
end

The error code comes from the fields providers keep it in, such as OpenAI’s code or type, Gemini’s status, or Bedrock’s error type, and only when it reads like a code: a word of letters, digits, _, ., :, or -, at most 64 characters. RubyLLM still picks the error class from what the provider wrote, so a context window error is still a ContextLengthExceededError. Messages RubyLLM writes itself, such as Provider returned no completion message, stay as they are, and an MCP server’s errors read like The MCP server returned an error (-32602).

provider_message and response_body are there when you want the provider’s words, so reading them is a choice your code makes. The setting is off by default. Set it on the global configuration: every context follows it.

Describing a Rejected Request

Some providers reject a conversation without saying what is wrong with it, answering with little more than “Request contains an invalid argument.” request_shape shows you the shape of what the provider received, and the problems RubyLLM finds in it:

begin
  chat.ask "What's due today?"
rescue RubyLLM::BadRequestError => e
  Rails.logger.error("#{e.message}\n#{e.request_shape}")
  raise
end

Request contains an invalid argument.
gemini, gemini-2.5-flash, model call 1 of the turn
payload keys: contents, systemInstruction, generationConfig, tools
thinking: includeThoughts true, thinkingBudget -1
instructions: text (240 chars)
#0 user: text (17 chars)
#1 model: call find_tasks (args 40 chars), signed
#2 user: result find_tasks (6859 chars)
#3 model: thinking (no text), signed, text (1075 chars)
#4 user: text (31 chars)
tools: find_tasks, add_task
tool round at #1: 1 call, 1 result, paired
problem at #3, part 0: thinking part carries no data

The first lines name the provider and model, say which model call of the turn the request was, and list the payload’s top-level keys and its thinking settings. Then comes one line per turn, numbered the way providers number turns in their error messages and named with the payload’s own roles. Each part shows its size in characters, or in bytes for inline media. A tool result names the call it answers, and signed marks a part that carries a thinking signature.

The shape holds no text, tool arguments, tool results, signatures, URLs, or file data, so you can send it to your error tracker. to_h has the same keys every time:

Rails.error.report(e, context: { request_shape: e.request_shape&.to_h })

A long conversation keeps its first 10 and last 50 turns, and omitted_turns counts the rest, while problems cover every turn. An error raised for a conversation request has a request_shape, including token counts and compaction. It is nil for errors raised before anything is sent, such as an unsupported attachment, and for operations that send no conversation, such as embeddings.

Error Handling During Streaming

When using streaming with a block, errors can occur during the stream after some chunks have already been processed. The ask method will raise the error after the block execution finishes or is interrupted by the error.

begin
  chat = RubyLLM.chat
  accumulated_content = ""
  chat.ask "Generate a very long story..." do |chunk|
    print chunk.content
    accumulated_content << chunk.content
  end
  puts "\nStream completed successfully."
rescue RubyLLM::RateLimitError
  puts "\nStream interrupted by rate limit. Partial content received:"
  puts accumulated_content
rescue RubyLLM::Error => e
  puts "\nStream failed: #{e.message}. Partial content received:"
  puts accumulated_content
end

Your block will execute for chunks received before the error. Because ask raises, it does not return a final response in this case; handle the exception and any partial content you accumulated.

Model Fallbacks

Use with_fallbacks when you want RubyLLM to try another model after the current model fails with a transient provider or network error.

chat = RubyLLM.chat(model: "gpt-4.1")
              .with_fallbacks("gpt-4.1-mini", "claude-haiku-4-5")

response = chat.ask("Summarize this incident report.")

Fallbacks are tried in order. The fallback only applies to that generation attempt; after the response finishes or the error bubbles up, the chat returns to its original model.

By default, fallbacks handle rate limits, server errors, service unavailable errors, overload errors, timeouts, and connection failures. Pass on: to choose the errors yourself:

chat.with_fallbacks(
  "gpt-4.1-mini",
  on: [RubyLLM::RateLimitError, RubyLLM::ServiceUnavailableError]
)

Cancelling a chat never triggers a fallback, whatever on: lists. chat.cancel raises RubyLLM::CancelledError and no fallback model runs. The same holds for a persisted chat that another process takes over mid-run: this process steps aside and no fallback model runs. An error your streaming block raises is yours to handle too: it propagates as it is, and the block does not run again on a fallback model.

Fallbacks can be model IDs or RubyLLM::Model objects:

chat.with_fallbacks(
  RubyLLM.models.find("claude-haiku-4-5", provider: :anthropic)
)

Fallback Callbacks

Use before_fallback and after_fallback to observe each fallback attempt:

chat.before_fallback do |fallback|
  Rails.logger.info(
    "Falling back from #{fallback.from.id} to #{fallback.to.id}: #{fallback.error.class}"
  )
end

chat.after_fallback do |fallback|
  if fallback.succeeded?
    Rails.logger.info("Fallback succeeded with #{fallback.to.id}")
  else
    Rails.logger.warn("Fallback failed with #{fallback.fallback_error.class}")
  end
end

The callback receives a RubyLLM::Fallback with the configured target (id, provider, model) and the runtime attempt details (from, to, error, attempt, response, fallback_error, streaming?, and chunks_yielded?).

When streaming has already yielded chunks before a fallback-worthy error, RubyLLM cannot take those chunks back. It starts a new assistant message lifecycle for the fallback response, and fallback.chunks_yielded? lets your UI or logs distinguish that case.

Handling Errors Within Tools

When building Tools, you need to decide how errors within the tool’s execute method should be handled:

Return Error to LLM: If the error is something the LLM might be able to recover from (e.g., invalid parameters provided by the LLM, temporary lookup failure), return a Hash containing an :error key. The LLM will see this error message as the tool’s output and may try again or use a different approach.

class Weather < RubyLLM::Tool
  # ... params ...
  def execute(location:)
    if location.blank?
      return { error: "Location cannot be blank. Please provide a city name." }
    end
    # ... perform API call ...
  rescue Faraday::TimeoutError
    { error: "Weather API timed out. Please try again later." }
  end
end

Raise Error for Application: If the error indicates a problem with the tool itself or the application’s state (e.g., database connection lost, configuration error, unrecoverable external API failure), raise an exception as normal. This will halt the RubyLLM interaction and bubble up to your application’s main error handling (begin/rescue).

class DatabaseQueryTool < RubyLLM::Tool
  # ... params ...
  def execute(query:)
    User.find_by_sql(query) # Example query
  rescue ActiveRecord::ConnectionNotEstablished => e
    raise e # Let the application's error handling take over.
  rescue StandardError => e
    # Maybe return less critical errors to the LLM
    { error: "Database query failed: #{e.message}" }
  end
end

Agent-Level Handlers

When you use Agents, rescue_from moves error handling out of every call site and into the agent class:

class ApplicationAgent < RubyLLM::Agent
  rescue_from RubyLLM::RateLimitError, RubyLLM::ConnectionError, with: :handle_transient

  private

  def handle_transient(error)
    StatsD.increment("llm.api_error", tags: ["type:transient"])
    raise
  end
end

Handlers cover the agent’s chat operations (ask, say, ask_later, complete, generate, run_tools, step), run on the agent instance, and follow ActiveSupport::Rescuable semantics. See Handling Errors with rescue_from.

Automatic Retries

RubyLLM automatically retries requests that fail due to transient network or server issues using Faraday’s retry middleware. Retries are driven by error classification (exception types), not raw HTTP status codes alone.

Retries are attempted for:

Network failures (RubyLLM::ConnectionError: timeouts, refused or dropped connections, and TLS failures)
Rate limit errors (RubyLLM::RateLimitError, often HTTP 429)
Server-side errors (RubyLLM::ServerError, RubyLLM::ServiceUnavailableError, RubyLLM::OverloadedError / HTTP 500, 502, 503, 504, 529)

RubyLLM::ContextLengthExceededError is not retried.

RubyLLM honors Retry-After and retry-after-ms headers across providers. Retry-After takes precedence when both are present. Invalid millisecond delays fall back to provider-specific timing or the configured backoff. If the requested delay exceeds retry_max_interval, RubyLLM raises the error without retrying early.

Requests that create something on the provider’s side are never retried: batch submissions, video generation jobs, file uploads, and content caches. When such a request reaches the provider but its response is lost, retrying would create a second job you still pay for, so RubyLLM raises the error and lets you decide whether to submit again.

You can configure retry behavior via RubyLLM.configure:

RubyLLM.configure do |config|
  config.max_retries = 5 # Default: 3
  config.retry_interval = 0.5 # Default: 0.1
  # config.retry_backoff_factor = 2 # Default: 2
  # config.retry_interval_randomness = 0.5 # Default: 0.5
end

Debugging

If you encounter unexpected errors or behavior, enable debug logging by setting the RUBYLLM_DEBUG environment variable:

export RUBYLLM_DEBUG=true

Debug logs show request and response headers and bodies, with API keys filtered. Use them to inspect the call that failed.