Tool Search
Give a model hundreds of tools without sending them all, and let it search for the ones it needs.
Table of Contents
- Why Defer Tools
- Deferring an MCP Server’s Tools
- Deferring Tools in a Chat
- Agents and Rails
- Reading Deferred Tools
- Choosing the Search Tool
- Provider Support
- Next Steps
After reading this guide, you will know:
- Why large tool sets need tool search.
- How to defer the tools of an MCP server, a chat, an agent, or a tool class.
- How to see which tools are deferred and which ones the model found.
- Which providers search for tools, and what happens on the others.
Why Defer Tools
Every tool a chat registers sends its name, description, and schema with every request. An MCP server like GitHub’s offers dozens of tools, and connecting a few servers puts hundreds of definitions in front of the model before it reads your question. That costs input tokens on every turn, and models pick the right tool less often when they have to choose from a long list.
A deferred tool stays out of the model’s context. The model gets a search tool instead, and searches for what it needs: “weather” finds weather_lookup. The matching definitions are then loaded, and the model calls them like any other tool. Your tools run the same way they always do.
Deferring an MCP Server’s Tools
MCP servers are where tool lists grow longest. Pass defer: true to with_mcp:
chat = RubyLLM.chat(model: "claude-sonnet-5")
chat.with_mcp(GitHub.new(user: current_user), Linear.new(user: current_user), defer: true)
chat.ask "Which open issues mention the flaky login spec?"
To defer some of a server’s tools every time you connect it, declare them in the class. Without names, defer defers every tool of the server:
class GitHub < RubyLLM::MCP
url "https://api.githubcopilot.com/mcp/"
bearer_token ENV.fetch("GITHUB_TOKEN")
defer :search_code, :list_workflows, :get_workflow_run
end
defer: false on with_mcp offers every tool of the server up front in one chat, whatever the class declares. Tool classes you add to a server with tool keep their own defer.
Deferring Tools in a Chat
with_tools takes the same option:
chat.with_tools(CurrentTime)
chat.with_tools(*Catalog.tools, defer: true)
CurrentTime stays in front of the model on every turn, and the catalog’s tools wait to be found. Keep the tools a conversation almost always needs undeferred, so the model can call them without searching first.
A tool that should always wait to be found declares it in its class:
class DeepResearch < RubyLLM::Tool
description "Runs a multi-step web investigation"
defer
parameter :query, description: "What to investigate"
def execute(query:)
Research.run(query)
end
end
chat.with_tools(DeepResearch)
chat.with_tools(DeepResearch, defer: false)
The second call registers it as an ordinary tool for that chat.
To defer one tool and not others of its class, call defer on it:
chat.with_tools(CurrentTime.new, *connector_tools.map(&:defer))
Agents and Rails
Agents pass defer: from their tools and mcp macros:
class SupportAgent < RubyLLM::Agent
model "claude-sonnet-5"
tools LookupAccount
mcp Zendesk, Stripe, defer: true
end
A lambda decides when the chat is built, so a chat with few tools can keep them all in front of the model:
class SupportAgent < RubyLLM::Agent
inputs :account
mcp(defer: -> { account.connectors.many? }) { account.connectors.map(&:server) }
end
Chat records respond to with_tools and with_mcp with defer:, and agents with a chat_class apply their deferred tools to the records they create and find. The search a model ran is part of its reply, so a persisted chat that keeps raw_content on its messages replays it on the next turn like any other reply.
Reading Deferred Tools
deferred_tools returns the deferred tools, keyed by name like tools:
chat.tools.keys # => [:current_time, :search_issues, :create_issue, ...]
chat.deferred_tools.keys # => [:search_issues, :create_issue, ...]
The search appears on the reply that ran it, in server_tool_calls, before the tool calls it made possible:
response = chat.messages.find(&:tool_call?)
response.server_tool_calls.any? # => true
response.tool_calls.map(&:name) # => ["search_issues"]
A tool the model found stays found for the rest of the conversation. RubyLLM sends the same tool list on every turn, so a prompt cache that covers the tools keeps hitting.
Choosing the Search Tool
RubyLLM adds the provider’s default search tool. Anthropic also offers a regular-expression search. Enable it with with_provider_tools, and RubyLLM uses it instead of adding the default:
chat.with_provider_tools({ type: "tool_search_tool_regex_20251119", name: "tool_search_tool_regex" })
See Provider Tools for how raw tool definitions work.
Provider Support
RubyLLM uses the provider’s own tool search wherever it speaks Anthropic’s Messages API or OpenAI’s Responses API, including Claude on Bedrock and Vertex AI and GPT on Azure. It does not check the model first: a model without tool search answers with the provider’s error. Check the provider’s documentation for the models that support it. xAI is the exception: it speaks the Responses API, but limits its tool search to alpha accounts, so xAI chats get RubyLLM’s own search.
Every other provider and protocol, such as Gemini, xAI, Bedrock’s Converse API (which Claude uses only with protocol: :converse), and OpenAI’s Chat Completions API, gets a tool_search tool from RubyLLM instead. Its description lists the deferred tools by name, and the model calls it with a few words, or with select: and tool names:
chat = RubyLLM.chat(model: "gemini-3.7-flash")
chat.with_tools(CurrentTime).with_tools(*Catalog.tools, defer: true)
chat.tools.keys # => [:current_time, :search_issues, ..., :tool_search]
The tools it finds are sent with every request after that, after the tools that are not deferred, in the order the conversation first loaded or called them. A deferred tool the conversation called stays loaded too. Each request repeats the one before it and adds to its end, so a prompt cache that covers the tools keeps hitting until the model loads another one. The search and its result are an ordinary tool call in messages, so a persisted chat keeps what it loaded.
When a chat switches to another model of the same provider, the tools the earlier model found stay found. Switching to another provider starts over: a model with tool search searches again with the provider’s search, and one without it loads tools with tool_search.