Connection, Logging and Contexts

Timeouts, retries, proxies, logging, the model registry file, and isolated multi-tenant contexts.

Table of contents

  1. Model Registry File
  2. Connection Settings
    1. Timeouts & Retries
    2. HTTP Proxy Support
  3. Logging & Debugging
    1. Basic Logging
    2. Advanced Logging Options
  4. Contexts: Isolated Configurations
    1. Basic Context Usage
    2. Multi-Tenant Applications
    3. Key Context Behaviors
  5. Next Steps

After reading this guide, you will know:

  • Where RubyLLM reads the model registry and how to relocate it.
  • How to tune timeouts, retries, and HTTP proxies.
  • How to configure logging and debug streaming responses.
  • How to create isolated configurations with contexts for multi-tenancy.

Model Registry File

RubyLLM ships with a registry snapshot, so a new installation works without a network request. Plain Ruby applications use the registry in the operating system’s user cache when one exists:

  • Linux: $XDG_CACHE_HOME/ruby_llm/models.json, or ~/.cache/ruby_llm/models.json
  • macOS: ~/Library/Caches/RubyLLM/models.json
  • Windows: %LOCALAPPDATA%\RubyLLM\Cache\models.json

RubyLLM.models.refresh! fetches the latest published catalog, merges models discovered from your configured providers, and atomically updates this cache. If the cache cannot be written, it raises RubyLLM::ModelRegistryError and leaves the in-memory registry unchanged.

Set model_registry_file only when your application needs a specific persistent location:

RubyLLM.configure do |config|
  config.model_registry_file = '/var/app/models.json'
end

RubyLLM.models.refresh!

RubyLLM falls back to its bundled snapshot until the configured file exists. The first successful refresh creates the file, and later refreshes replace it. Use RubyLLM.models.save_to_json('/another/path/models.json') only when you want to export the currently loaded registry elsewhere.

With the Active Record integration, the models table is the registry. RubyLLM.models.refresh! updates it automatically. While the table is empty, RubyLLM falls back to the registry file, then to the bundled snapshot.

Connection Settings

Timeouts & Retries

Fine-tune how RubyLLM handles network connections:

RubyLLM.configure do |config|
  # Basic settings
  config.request_timeout = 120        # Seconds to wait for response (default: 300)
  config.max_retries = 3              # Retry attempts on failure (default: 3)

  # Advanced retry behavior
  config.retry_interval = 0.1         # Initial retry delay in seconds (default: 0.1)
  config.retry_backoff_factor = 2     # Exponential backoff multiplier (default: 2)
  config.retry_interval_randomness = 0.5  # Jitter to prevent thundering herd (default: 0.5)
end

Example for high-latency connections:

RubyLLM.configure do |config|
  config.request_timeout = 300        # 5 minutes for complex tasks
  config.max_retries = 5              # More retry attempts
  config.retry_interval = 1.0         # Start with 1 second delay
  config.retry_backoff_factor = 1.5   # Less aggressive backoff
end

HTTP Proxy Support

Route requests through a proxy:

RubyLLM.configure do |config|
  # Basic proxy
  config.http_proxy = "http://proxy.company.com:8080"

  # Authenticated proxy
  config.http_proxy = "http://user:pass@proxy.company.com:8080"

  # SOCKS5 proxy
  config.http_proxy = "socks5://proxy.company.com:1080"
end

Logging & Debugging

Basic Logging 

RubyLLM.configure do |config|
  config.log_file = '/var/log/ruby_llm.log'
  config.log_level = :info  # :debug, :info, :warn

  # Or use Rails logger
  config.logger = Rails.logger  # Overrides log_file and log_level
end

Log levels:

  • :debug - Detailed request/response information
  • :info - General operational information
  • :warn - Non-critical issues

You can also enable debug logging conditionally from an environment variable:

RubyLLM.configure do |config|
  config.log_level = :debug if ENV['RUBYLLM_DEBUG'] == 'true'
end

Setting config.logger overrides log_file and log_level settings.

Advanced Logging Options

Use these options when you need deeper troubleshooting or safer handling of large debug payloads.

RubyLLM.configure do |config|
  config.log_stream_debug = true

  # Available in v1.13.0+
  config.log_regexp_timeout = 1.5
end

log_stream_debug notes:

  • Shows chunk-by-chunk streaming internals (accumulator state, parsing, tool chunks)
  • Invaluable for diagnosing streaming/provider parsing issues
  • Can also be enabled with RUBYLLM_STREAM_DEBUG=true

log_regexp_timeout notes:

  • Available in v1.13.0+
  • Applies to regex filters used in request/response debug logging
  • Supported on Ruby 3.2+ (uses Regexp.timeout)
  • On Ruby <3.2, RubyLLM warns if set and continues without timeout
  • Helps bound regex execution time when debug logs contain very large payloads

Built-in debug log redaction:

  • Large base64-like blobs are redacted as [BASE64 DATA]
  • Large embedding arrays are redacted as [EMBEDDINGS ARRAY]

Contexts: Isolated Configurations

Create temporary configuration scopes without affecting global settings. Perfect for multi-tenancy, testing, or specific task requirements.

Basic Context Usage 

# Global config uses production OpenAI
RubyLLM.configure do |config|
  config.openai_api_key = ENV['OPENAI_PROD_KEY']
end

ctx = RubyLLM.context do |config|
  config.openai_api_key = ENV['ANOTHER_PROVIDER_KEY']
  config.openai_api_base = "https://another-provider.com"
  config.request_timeout = 180
end

ctx_chat = ctx.chat(model: 'gpt-5.4')
response = ctx_chat.ask("Process this with another provider...")

regular_chat = RubyLLM.chat  # Still uses production OpenAI

Multi-Tenant Applications 

class TenantService
  def initialize(tenant)
    @context = RubyLLM.context do |config|
      config.openai_api_key = tenant.openai_key
      config.default_model = tenant.preferred_model
      config.request_timeout = tenant.timeout_seconds
    end
  end

  def chat
    @context.chat
  end
end

# Each tenant gets isolated configuration
tenant_a_service = TenantService.new(tenant_a)
tenant_b_service = TenantService.new(tenant_b)

Key Context Behaviors

  • Inheritance: Contexts start with a copy of global configuration
  • Isolation: Changes don’t affect global RubyLLM.config
  • Thread Safety: Each context is independent and thread-safe

Next Steps