Custom Endpoints and Unlisted Models

Target OpenAI-compatible endpoints and use model IDs the registry doesn’t list

After reading this guide, you will know:

  • When you need to step outside the standard model registry.
  • How to point provider: :openai requests at a custom URL with openai_api_base.
  • How to use model IDs the registry doesn’t list with assume_model_exists.
  • What capability checks RubyLLM skips when you assume a model exists.
  • What you stay responsible for when bypassing registry validation.

Sometimes you need to interact with models or endpoints not covered by the standard registry, such as:

  • API Proxies & Gateways (LiteLLM, Fastly AI Accelerator).
  • Self-hosted or local models (LM Studio, vLLM).
  • Brand-new model releases.
  • Custom fine-tunes or deployments with unique names.

Azure has a first-class provider: configure azure_api_base and use provider: :azure instead. See the Provider Setup and Custom Endpoints Guide.

RubyLLM offers two mechanisms for these cases.

Custom OpenAI API Base URL (openai_api_base)

If you need to target an endpoint that uses the OpenAI API format but has a different URL, configure openai_api_base in RubyLLM.configure.

# config/initializers/ruby_llm.rb
RubyLLM.configure do |config|
  config.openai_api_key = ENV['GATEWAY_API_KEY'] # Key for your endpoint
  config.openai_api_base = "https://ai-gateway.internal.example.com/v1" # Your endpoint
end
  • This setting only affects requests made with provider: :openai.
  • It directs those requests to your specified URL instead of https://api.openai.com/v1.
  • See the Provider Setup and Custom Endpoints Guide for the full api_base setup details.

Assuming Model Existence (assume_model_exists)

To use a model identifier not listed in RubyLLM’s registry, use the assume_model_exists: true flag. This tells RubyLLM to bypass its validation check.

# Assumes openai_api_base points at your gateway
chat = RubyLLM.chat(
  model: ENV.fetch("CUSTOM_CHAT_MODEL"), # Your custom deployment name
  provider: :openai,                # MUST specify provider
  assume_model_exists: true         # Bypass registry check
)
response = chat.ask("Internal knowledge query...")
puts response.content

# You can also use it in .with_model
chat.with_model(
  ENV.fetch("CUSTOM_FALLBACK_MODEL"),
  provider: :openai,                # MUST specify provider
  assume_model_exists: true
)

The assume_model_exists flag also works with RubyLLM.embed and RubyLLM.paint for embedding and image generation models:

embedding = RubyLLM.embed(
  "Test text",
  model: ENV.fetch("CUSTOM_EMBEDDING_MODEL"),
  provider: :openai,
  assume_model_exists: true
)

image = RubyLLM.paint(
  "A beautiful landscape",
  model: ENV.fetch("CUSTOM_IMAGE_MODEL"),
  provider: :openai,
  assume_model_exists: true
)

Key Points when Assuming Existence:

  • provider: is Mandatory: You must tell RubyLLM which API format to use (ArgumentError otherwise).
  • No Validation: RubyLLM won’t check the registry for the model ID.
  • Capability Assumptions: The model is assumed to support function_calling, streaming, vision, and structured_output; checks outside that set return false. You are responsible for ensuring the model supports the features you use.
  • Your Responsibility: Ensure the model ID is correct for the target endpoint.
  • Warning Log: A warning is logged indicating validation was skipped.

Use these features when the standard registry doesn’t cover your specific model or endpoint needs. For standard models, rely on the registry for validation and capability awareness.

Next Steps

  • Chat - start a conversation once your custom endpoint or unlisted model is wired up.
  • Model Resolution - the exact procedure behind assume_model_exists and provider selection.
  • Model Registry - return to the registry for validated, capability-aware models.