Connection, Logging and Contexts

Timeouts, retries, proxies, logging, the model registry file, and isolated multi-tenant contexts.

After reading this guide, you will know:

  • Where RubyLLM reads the model registry and how to relocate it.
  • How to tune timeouts, retries, and HTTP proxies.
  • How to configure logging and debug streaming responses.
  • How to create isolated configurations with contexts for multi-tenancy.

Model Registry File

RubyLLM ships with a registry snapshot, so a new installation works without a network request. Plain Ruby applications use the registry in the operating system’s user cache when one exists:

  • Linux: $XDG_CACHE_HOME/ruby_llm/models.json, or ~/.cache/ruby_llm/models.json
  • macOS: ~/Library/Caches/RubyLLM/models.json
  • Windows: %LOCALAPPDATA%\RubyLLM\Cache\models.json

RubyLLM.models.refresh fetches the latest published catalog, merges models discovered from your configured providers, and atomically updates this cache. If the cache cannot be written, it raises RubyLLM::ModelRegistryError and leaves the in-memory registry unchanged.

Set model_registry_file only when your application needs a specific persistent location:

RubyLLM.configure do |config|
  config.model_registry_file = '/var/app/models.json'
end

RubyLLM.models.refresh

RubyLLM falls back to its bundled snapshot until the configured file exists. The first successful refresh creates the file, and later refreshes replace it. Use RubyLLM.models.save_to_json('/another/path/models.json') only when you want to export the currently loaded registry elsewhere.

With the Active Record integration, RubyLLM’s internal ruby_llm_models table is the registry. RubyLLM.models.refresh updates it automatically. While the table is empty, RubyLLM falls back to the registry file, then to the bundled snapshot.

Connection Settings

Timeouts & Retries

Fine-tune how RubyLLM handles network connections:

RubyLLM.configure do |config|
  # Basic settings
  config.request_timeout = 120        # Seconds to wait for response (default: 300)
  config.max_retries = 3              # Retry attempts on failure (default: 3)

  # Advanced retry behavior
  config.retry_interval = 0.1         # Initial retry delay in seconds (default: 0.1)
  config.retry_backoff_factor = 2     # Exponential backoff multiplier (default: 2)
  config.retry_interval_randomness = 0.5  # Jitter to prevent thundering herd (default: 0.5)
  config.retry_max_interval = 30      # Longest delay to honor between retries (default: 30)
end

When a provider reports how long to wait through rate-limit headers (the standard Retry-After, or OpenAI’s x-ratelimit-reset-requests and x-ratelimit-reset-tokens), retries wait that long instead of the backoff interval. If the requested wait exceeds retry_max_interval, the request fails immediately with RubyLLM::RateLimitError rather than sleeping.

These settings cover requests you can safely repeat. Requests that create something on the provider’s side, such as submitting a batch, starting a video generation job, uploading a file, or creating a content cache, are sent once and raise on failure, because a retry after a lost response would create a second job.

Example for high-latency connections:

RubyLLM.configure do |config|
  config.request_timeout = 300        # 5 minutes for complex tasks
  config.max_retries = 5              # More retry attempts
  config.retry_interval = 1.0         # Start with 1 second delay
  config.retry_backoff_factor = 1.5   # Less aggressive backoff
end

HTTP Proxy Support

Route requests through a proxy:

RubyLLM.configure do |config|
  # Basic proxy
  config.http_proxy = "http://proxy.company.com:8080"

  # Authenticated proxy
  config.http_proxy = "http://user:pass@proxy.company.com:8080"

  # SOCKS5 proxy
  config.http_proxy = "socks5://proxy.company.com:1080"
end

Logging & Debugging

Basic Logging

RubyLLM.configure do |config|
  config.log_file = '/var/log/ruby_llm.log'  # Or set RUBYLLM_LOG_FILE
  config.log_level = :info  # :debug, :info, :warn

  # Or use Rails logger
  config.logger = Rails.logger  # Overrides log_file and log_level
end

Log levels:

  • :debug - Detailed request/response information
  • :info - General operational information
  • :warn - Non-critical issues

log_file notes:

  • Defaults to $stdout
  • Can also be set with RUBYLLM_LOG_FILE=/path/to/file.log

Debug logging is also enabled automatically when the RUBYLLM_DEBUG=true environment variable is set. No configuration change is needed.

Setting config.logger overrides log_file and log_level settings.

Advanced Logging Options

Use these options when you need deeper troubleshooting or safer handling of large debug payloads.

RubyLLM.configure do |config|
  config.log_stream_debug = true
  config.log_regexp_timeout = 1.5
end

log_stream_debug notes:

  • Shows chunk-by-chunk streaming internals (accumulator state, parsing, tool chunks)
  • Helps diagnose streaming and response parsing issues
  • Can also be enabled with RUBYLLM_STREAM_DEBUG=true

log_regexp_timeout notes:

  • Applies to regex filters used in request/response debug logging
  • Supported on Ruby 3.2+ (uses Regexp.timeout)
  • On Ruby <3.2, RubyLLM warns if set and continues without timeout
  • Helps bound regex execution time when debug logs contain very large payloads

Built-in debug log redaction:

  • Large base64-like blobs are redacted as [BASE64 DATA]
  • Large embedding arrays are redacted as [EMBEDDINGS ARRAY]

Contexts: Isolated Configurations

Use a context for a tenant’s credentials or a task’s defaults without changing global configuration.

Basic Context Usage

# Global config uses production OpenAI
RubyLLM.configure do |config|
  config.openai_api_key = ENV['OPENAI_PROD_KEY']
end

ctx = RubyLLM.context do |config|
  config.openai_api_key = ENV['ANOTHER_PROVIDER_KEY']
  config.openai_api_base = "https://another-provider.com"
  config.request_timeout = 180
end

ctx_chat = ctx.chat(model: 'gpt-5.6')
response = ctx_chat.ask("Process this with another provider...")

regular_chat = RubyLLM.chat  # Still uses production OpenAI

Individual Operations

A context offers the same entry points as the top-level RubyLLM module, so the rest of the API runs under its configuration too:

image = ctx.paint("a paper boat sailing down a rainy gutter")
image.save("boat.png")

embedding = ctx.embed("Ruby is a joy to write")
transcript = ctx.transcribe("interview.mp3")

paint, animate, animate_later, embed, embed_later, moderate, speak, transcribe, ocr, rerank, upload, download, and cache all use the context’s keys, endpoints, and connection settings. So do the downloads RubyLLM performs on your behalf: fetching a URL attachment for a chat, or reading image.to_blob and video.to_blob from a provider’s hosted file, goes through the context’s http_proxy and request_timeout rather than the global ones.

Multi-Tenant Applications

class TenantService
  def initialize(tenant)
    @context = RubyLLM.context do |config|
      config.openai_api_key = tenant.openai_key
      config.default_model = tenant.preferred_model
      config.request_timeout = tenant.timeout_seconds
    end
  end

  def chat
    @context.chat
  end
end

# Each tenant gets isolated configuration
tenant_a_service = TenantService.new(tenant_a)
tenant_b_service = TenantService.new(tenant_b)

Key Context Behaviors

  • Inheritance: Contexts start with a copy of global configuration
  • Isolation: Changes don’t affect global RubyLLM.config
  • Coverage: Every entry point the context exposes uses it, chat and non-chat alike, including the file downloads those calls make
  • Thread Safety: Each context is independent and thread-safe

Next Steps