Advanced Rails Configuration

Route models through different providers, use per-tenant contexts, persist cache boundaries and raw payloads, and run fiber-safe.

Table of contents

  1. Provider Overrides
  2. Custom Contexts and Dynamic Models
    1. Using Custom Contexts
    2. Dynamic Model Creation
  3. Working with Prompt Caching
  4. Working with Provider-Specific Payloads
  5. Fiber-Safe ActiveRecord Connections for Async/Fiber Workloads
  6. Instrumentation
  7. Next Steps

After reading this guide, you will know:

  • How to route a model through a different provider per chat.
  • How to use per-tenant API keys with custom contexts.
  • How to create chats for models that aren’t in the registry.
  • How to persist cache boundaries and raw provider payloads.
  • How to run ActiveRecord safely inside fiber-based async workloads.

Once the basics are in place, RubyLLM gives you fine-grained control over how persisted chats reach providers. This guide covers the configuration you reach for in production: routing models through alternate providers, isolating credentials per tenant, persisting cache boundaries and provider-specific payloads, and keeping ActiveRecord connections correct under async workloads.

Provider Overrides

Route models through different providers dynamically:

chat = Chat.create!(
  model: 'claude-sonnet-4-6',
  provider: 'bedrock'  # Route this model through AWS Bedrock
)

chat.ask("Hello!")

Custom Contexts and Dynamic Models

Using Custom Contexts

Use different API keys per chat in multi-tenant applications:

With DB-backed model registry (default in v1.7.0+):

custom_context = RubyLLM.context do |config|
  config.openai_api_key = 'sk-customer-specific-key'
end

chat = Chat.create!(
  model: 'gpt-5.4',
  context: custom_context
)

Context is not persisted. Set it after reloading chats.

# Later, in a different request or after restart
chat = Chat.find(chat_id)
chat.context = custom_context  # Must set this!
chat.ask("Continue our conversation")

For multi-tenant apps, consider using an after_find callback:

class Chat < ApplicationRecord
  acts_as_chat
  belongs_to :tenant

  after_find :set_tenant_context

  private

  def set_tenant_context
    self.context = RubyLLM.context do |config|
      config.openai_api_key = tenant.openai_api_key
    end
  end
end

Dynamic Model Creation

When using models not in the registry (e.g., new OpenRouter models), pass assume_model_exists: true to skip the registry lookup. See Model Resolution for exactly how this bypasses the registry:

chat = Chat.create!(
  model: 'experimental-llm-v2',
  provider: 'openrouter',
  assume_model_exists: true  # Creates Model record automatically
)

Like context, assume_model_exists is not persisted.

# When switching to another dynamic model later
chat = Chat.find(chat_id)
chat.assume_model_exists = true
chat.with_model('another-experimental-model', provider: 'openrouter')

Working with Prompt Caching

Prompt caching configuration is applied to the underlying LLM chat, and explicit boundaries are persisted on messages. Mark the stable part of the conversation, then continue normally:

chat = Chat.create!(model: 'claude-sonnet-4-5')
chat.with_caching(ttl: "1h")
chat.with_instructions('Reusable analysis prompt').cache_until_here!
chat.add_message(role: :user, content: long_context).cache_until_here!
chat.ask("Today's request: #{summary}")

Existing apps: run the latest upgrade generator after updating RubyLLM so message tables include cache_until_here and the other current persistence columns. New apps get the proper columns from the install generator.

See Prompt Caching for provider behavior.

Working with Provider-Specific Payloads

Message content is always text: what you persist is the conversation, not a provider’s wire format. When a request needs provider-specific blocks RubyLLM has not wrapped, use a before_request hook; it adjusts the rendered payload per request and stores nothing.

Fiber-Safe ActiveRecord Connections for Async/Fiber Workloads

Rails 7.2.1+ / 8.x

If your app performs database work inside Fibers (for example with async-based workflow stacks), use fiber-safe connection isolation:

# config/application.rb
config.active_support.isolation_level = :fiber

Why: Rails defaults to thread-based connection isolation. In fiber-heavy flows, that can cause intermittent connection-state issues. :fiber scopes ActiveRecord connections per Fiber instead of per Thread.

If you use this setting, prefer Rails versions with fiber isolation fixes (Rails 7.2.1+ / 8.x).

Instrumentation

Rails apps automatically emit RubyLLM events through ActiveSupport::Notifications. See Instrumentation and Observability for events, payloads, and non-Rails instrumenters.

Next Steps