class RubyLLM::Provider

A Provider connects RubyLLM to one AI service. It knows where to talk (host, authentication headers, configuration) and which protocol to speak for a given model and request. The wire formats themselves live under RubyLLM::Protocols.

Subclass Provider to support a new service, then make it available with ::register:

class Acme < RubyLLM::Provider
  protocol :chat_completions, RubyLLM::Protocols::ChatCompletions

  def self.configuration_options
    %i[acme_api_key]
  end

  def api_base
    'https://api.acme.ai/v1'
  end

  def headers
    { 'Authorization' => "Bearer #{@config.acme_api_key}" }
  end
end

RubyLLM::Provider.register :acme, Acme

See the custom providers guide for the full walkthrough.

Attributes

config [R]

The Configuration the provider was built with.

Public Class Methods

assume_models_exist? ()

Source

# File lib/ruby_llm/provider.rb, line 557
def assume_models_exist?
  false
end

Returns whether the provider accepts model ids missing from the model registry. The base implementation returns false.

capabilities ()

Source

# File lib/ruby_llm/provider.rb, line 492
def capabilities
  nil
end

Returns the provider’s narrow model capability augmenter, or nil when models.dev and the provider listing are sufficient.

configuration_options ()

Source

# File lib/ruby_llm/provider.rb, line 540
def configuration_options
  []
end

Returns every configuration key the provider contributes. ::register defines a Configuration accessor for each one. The base implementation returns an empty array.

def self.configuration_options
  %i[acme_api_key acme_api_base]
end

configuration_requirements ()

Source

# File lib/ruby_llm/provider.rb, line 528
def configuration_requirements
  []
end

Returns the configuration keys that must be set before the provider is usable. The base implementation returns an empty array.

def self.configuration_requirements
  %i[acme_api_key]
end

deployed_model_id (_model_id, _config = nil)

Source

# File lib/ruby_llm/provider.rb, line 623
def deployed_model_id(_model_id, _config = nil)
  nil
end

Returns the id of the model that the deployment named model_id runs, or nil when model_id is not a declared deployment. Providers whose requests name deployments override it.

display_name ()

Source

# File lib/ruby_llm/provider.rb, line 486
def display_name
  to_s.split('::').last
end

Returns the human-readable provider name, derived from the class name. Override for custom branding.

local? ()

Source

# File lib/ruby_llm/provider.rb, line 547
def local?
  false
end

Returns whether the provider talks to a locally hosted service. The base implementation returns false. Local providers such as Ollama return true.

model_required? (**)

Source

# File lib/ruby_llm/provider.rb, line 563
def model_required?(**)
  true
end

Returns whether operation requires an inference model. Override for endpoints that operate on an explicitly configured resource.

protocol (name, protocol_class, batches: nil)

Source

# File lib/ruby_llm/provider.rb, line 578
def protocol(name, protocol_class, batches: nil)
  @default_protocol = name.to_sym if protocols.empty?
  protocols[name.to_sym] = batches ? Class.new(protocol_class) { include batches } : protocol_class
end

Registers protocol_class under name. The first registered protocol becomes the provider’s default. Pass batches: to compose batch operations into the registered protocol.

protocol :chat_completions, ChatCompletions
protocol :responses, Protocols::Responses, batches: Protocols::Responses::Batches

providers ()

Source

# File lib/ruby_llm/provider.rb, line 629
def providers
  @providers ||= {}
end

Returns the global registry of providers, a hash mapping slug symbols to provider classes.

Source

# File lib/ruby_llm/provider.rb, line 597
def register(name, provider_class, models: nil)
  provider_class.slug = name.to_s
  providers[name.to_sym] = provider_class
  models ? model_registry_files[name.to_sym] = models : model_registry_files.delete(name.to_sym)
  RubyLLM::Configuration.register_provider_options(provider_class.configuration_options + [:"#{name}_protocol"])
end

Registers provider_class under the slug name, making it available to RubyLLM.chat and the other top-level helpers. Stamps the class’s slug, adds it to ::providers, and defines a Configuration accessor for each of its configuration options. A provider gem may pass the path to its bundled model catalog.

RubyLLM::Provider.register :acme, RubyLLM::Providers::Acme
RubyLLM::Provider.register :acme, RubyLLM::Providers::Acme,
                           models: File.expand_path('../../../models.json', __dir__)

resolve_registry_id (model_id, _models, _config = nil)

Source

# File lib/ruby_llm/provider.rb, line 616
def resolve_registry_id(model_id, _models, _config = nil)
  model_id
end

Resolves model_id to the id the registry stores it under for this provider. Defaults to the id unchanged; providers whose catalog ids differ from their request ids (Bedrock’s region prefixes) override it.

slug ()

Source

# File lib/ruby_llm/provider.rb, line 480
def slug
  @slug ||= to_s.split('::').last.downcase
end

Returns the provider slug, a short lowercase string that identifies the provider and prefixes its configuration keys. Set by ::register, or derived from the class name.

Public Instance Methods

account_identity ()

Source

# File lib/ruby_llm/provider.rb, line 309
def account_identity
  nil
end

Returns a String naming the account that owns the files this provider uploads, or nil to keep each upload within the process that made it. The base implementation returns nil. Rails chats reuse an upload in later processes only under the same identity, so build it from what decides where uploaded files live, such as the endpoint and API key, an organization or project, or a storage bucket. Digest anything secret, and leave out credentials that rotate, such as session tokens:

def account_identity
  Digest::SHA256.hexdigest([api_base, @config.acme_api_key].join("\0"))
end

api_base ()

Source

# File lib/ruby_llm/provider.rb, line 66
def api_base
  raise NotImplementedError
end

Returns the base URL that relative endpoint paths resolve against. The base implementation raises NotImplementedError, so every subclass must define it.

def api_base
  @config.acme_api_base || 'https://api.acme.ai/v1'
end

headers ()

Source

# File lib/ruby_llm/provider.rb, line 77
def headers
  {}
end

Returns the headers merged into every request. The default is an empty hash. Override to supply authentication.

def headers
  { 'Authorization' => "Bearer #{@config.acme_api_key}" }
end

protocol_for (_model, **)

Source

# File lib/ruby_llm/provider.rb, line 132
def protocol_for(_model, **)
  default_protocol
end

Returns the protocol class to use for model. Override to route between registered protocols per model or request operation. An explicit protocol: override on the chat or the provider’s <slug>_protocol configuration option takes precedence over this hook.

def protocol_for(model, **)
  model.id.match?(/audio|realtime/) ? protocols[:chat_completions] : super
end

retry_delay (_response)

Source

# File lib/ruby_llm/provider.rb, line 91
def retry_delay(_response)
  nil
end

Returns how many seconds the service asked us to wait before retrying a rate-limited request, or nil when the response carries no timing information. The retry middleware already honors the standard Retry-After header; override this to read provider-specific rate-limit headers.

def retry_delay(response)
  response.response_headers['x-acme-ratelimit-reset']&.to_f
end

slug ()

Source

# File lib/ruby_llm/provider.rb, line 96
def slug
  self.class.slug
end

Returns the provider slug, delegating to ::slug.