Advanced Rails Configuration

Route models through different providers, use per-tenant contexts, persist cache boundaries, adjust provider payloads per request, and run fiber-safe.

After reading this guide, you will know:

  • How to route a model through a different provider per chat.
  • How to use per-tenant API keys with custom contexts.
  • How to create chats for models that aren’t in the registry.
  • How to persist cache boundaries and adjust provider payloads per request.
  • How to run ActiveRecord safely inside fiber-based async workloads.

Persisted chats use the same configuration methods as plain Ruby. Use them to select a provider, isolate tenant credentials, or set cache boundaries on a conversation.

Provider Overrides

Route models through different providers dynamically:

chat = Chat.create!(
  model: 'claude-sonnet-5',
  provider: 'bedrock'  # Route this model through AWS Bedrock
)

chat.ask("Hello!")

Custom Contexts and Dynamic Models

Using Custom Contexts

Use different API keys per chat in multi-tenant applications:

custom_context = RubyLLM.context do |config|
  config.openai_api_key = 'sk-customer-specific-key'
end

chat = Chat.create!(
  model: 'gpt-5.6',
  context: custom_context
)

Context is not persisted. Set it after reloading chats.

# Later, in a different request or after restart
chat = Chat.find(chat_id)
chat.context = custom_context  # Must set this!
chat.ask("Continue our conversation")

For multi-tenant apps, consider using an after_find callback:

class Chat < ApplicationRecord
  acts_as_chat
  belongs_to :tenant

  after_find :set_tenant_context

  private

  def set_tenant_context
    self.context = RubyLLM.context do |config|
      config.openai_api_key = tenant.openai_api_key
    end
  end
end

Dynamic Model Creation

When using models not in the registry (e.g., new OpenRouter models), pass assume_model_exists: true to skip the registry lookup. See Model Resolution for exactly how this bypasses the registry:

chat = Chat.create!(
  model: ENV.fetch("OPENROUTER_MODEL"),
  provider: 'openrouter',
  assume_model_exists: true  # Skips registry validation
)

Like context, assume_model_exists is not persisted.

# When switching to another dynamic model later
chat = Chat.find(chat_id)
chat.with_model(ENV.fetch("OPENROUTER_FALLBACK_MODEL"), provider: 'openrouter', assume_model_exists: true)

Working with Prompt Caching

Prompt caching configuration is applied to the underlying LLM chat, and explicit boundaries are persisted on messages. Mark the stable part of the conversation, then continue normally:

chat = Chat.create!(model: 'claude-sonnet-4-5')
chat.with_caching(ttl: "1h")
chat.with_instructions('Reusable analysis prompt').cache_until_here
chat.add_message(role: :user, content: long_context).cache_until_here
chat.ask("Today's request: #{summary}")

Existing apps: run the latest upgrade generator after updating RubyLLM so message tables include cache_until_here and the other current persistence columns. New apps get the proper columns from the install generator.

When the stable prefix should not be stored with the transcript, disable persistence. This is useful for an application-wide policy followed by tenant or request context:

chat.with_caching
chat.with_instructions(stable_policy, persist: false, cache_until_here: true)
chat.with_instructions(current_context, append: true, persist: false)

Unpersisted instructions are not written to the message table. They and their cache boundary remain configured when you call reload on the same record instance. Reapply them after finding the record in another process, or declare them with persist: false on a RubyLLM agent so Agent.find does that for you.

See Prompt Caching for provider behavior.

Working with Provider-Specific Payloads

Message content is always text: what you persist is the conversation, not a provider’s wire format. When a request needs provider-specific blocks RubyLLM has not wrapped, use a before_request hook; it adjusts the rendered payload per request and stores nothing.

Fiber-Safe ActiveRecord Connections for Async/Fiber Workloads

Rails 7.2.1+ / 8.x

If your app uses Solid Queue fiber workers or runs database work inside Async tasks, enable fiber isolation:

# config/application.rb
config.active_support.isolation_level = :fiber

Rails defaults to thread-scoped execution state. :fiber keeps that state, including Active Record connections, separate for each fiber. This setting applies to the whole application; Solid Queue requires it before starting fiber workers.

If you use this setting, prefer Rails versions with fiber isolation fixes (Rails 7.2.1+ / 8.x).

Instrumentation

Rails apps automatically emit RubyLLM events through ActiveSupport::Notifications. See Instrumentation and Observability for events, payloads, and non-Rails instrumenters.

Next Steps