Instrumentation Events

Every event RubyLLM emits, when it fires, and the payload fields your subscribers can read.

Subscribe to these events as shown in Instrumentation and Observability.

Events

The embedding, image, judgment, moderation, ocr, rerank, speech, transcription, and video_job events also carry provider, provider_class, model, model_info, provider_options, and metadata. All of them except video_job add result and response_model when the call succeeds.

See Progress and Monitoring in the Evaluations guide for evaluation progress, run IDs, and report payloads.

Agent Events

agent.ruby_llm wraps one run of a chat an agent configured: a turn of ask or complete, or a single step, generate, or run_tools. It fires however you reach the chat: through the agent, the chat that Agent.chat returns, or a Rails chat record from create, create!, or find, including its to_llm chat. The model calls and tool executions of the run happen inside it, so their events come first.

A run opens one event, however many moves it takes. A turn that goes on to answer a steering message queued while it ran stays in the same event, and two ask calls on the same chat give two events. A subagent that runs inside a tool opens its own event, nested in its parent’s.

The workflow fields are added as usual. chat.ruby_llm, compaction.ruby_llm, and tool_call.ruby_llm also carry agent_name when the chat belongs to a named agent class, so subscribers can attribute model calls and tools without tracking the enclosing event.

MCP Events

mcp_unreachable.ruby_llm fires when a chat finds an MCP server down and goes on without its tools. It never carries the error’s message, since a connection error’s message can name the server’s host or URL and a server’s answer can quote anything.

Tokens and Cost

Operations that expose normalized usage (chat, embedding, image, judgment, moderation, rerank, speech, and transcription) include payload[:tokens] and payload[:cost]. Both value objects are always present. Their individual fields are nil when the provider did not report usage or RubyLLM could not price it. Batch, OCR, and video events expose their operation-specific fields instead.

chat.ruby_llm also includes response_tokens, the usage for that model call before earlier fallback attempts are attached to the returned message. On failure it contains any usage recorded for that call. The tokens field keeps its response-level meaning.

Usage Events

usage.ruby_llm fires once for every finished physical attempt of a tracked operation. Tokenization is not tracked and emits no usage event. Jobs and batches report differently, so don’t treat the event count as a request count:

• A video or research job reports when a job object first sees it finished, so another object that loads the finished job reports it again.
• A batch reports when its results are collected, under the workflow active then: one event per successful result, and one for a failed slot only when the provider billed it. A batch picked up with Batch.find without its chats or requests reports once per Batch object, so collecting it in two places reports it twice.

The payload contains operation, provider, model, status, tokens, and cost. The values use the same objects as the rest of RubyLLM; the internal usage record is not part of the public API. Both tokens and cost are always present. Their individual fields are nil when an attempt may have been billed and the provider supplied no defensible figure. When an operation has a resolved model, an attempt that cannot have been billed, such as one that never reached the provider, reports zero input and output tokens. Without a resolved model, those fields stay nil. cost uses the provider’s reported amount when there is one. Otherwise token-priced operations calculate it from the tokens and registry pricing, and a hosted research job, billed per task, leaves it unknown.

model is nil for operations that do not select a model, so subscribers should accept a missing model. It can also hold the provider’s own resource identifier rather than a registry model ID: the hosted agent’s ID for a research job, or the guardrail’s id:version for Bedrock moderation. See Cost and Usage Tracking for status semantics and Rails persistence.

Workflow Fields

Every event inside a workflow carries these fields when they apply. A usage.ruby_llm event reports the workflow its attempt started in, even when it finishes outside that block, such as a polled video job. An attempt that started outside any workflow reports the workflow it finishes in, if any. To attribute a batch collected later to its original workflow, re-enter that workflow when you collect it.

Other Fields

Non-Rails instrumenters control their own error payload behavior.

Context Across Concurrent Tools

When a chat runs tools concurrently, each tool call runs in its own thread or fiber. An instrumenter that keeps its current span in thread or fiber state would lose it there, so a tool_call.ruby_llm event would start without its parent.

An instrumenter can carry that state across by also responding to two optional methods:

• capture_context returns whatever identifies the current span. RubyLLM calls it in the chat’s thread before it starts the tools.
• with_context(value) { ... } makes that value current for the block. RubyLLM wraps each tool call with it, in both :threads and :fibers mode.

RubyLLM uses the instrumenter of the chat’s own configuration, so this works for a context with its own instrumenter too. An instrumenter that wraps another one passes both methods through, keeping the inner one’s context:

class TracingInstrumenter
  def initialize(inner)
    @inner = inner
  end

  def instrument(name, payload, &block)
    Tracing.span(name) do
      @inner.respond_to?(:instrument) ? @inner.instrument(name, payload, &block) : block&.call
    end
  end

  def capture_context
    [Tracing.current_span, @inner.respond_to?(:capture_context) && @inner.capture_context]
  end

  def with_context((span, inner))
    Tracing.with_span(span) do
      @inner.respond_to?(:with_context) ? @inner.with_context(inner) { yield } : yield
    end
  end
end

RubyLLM.configure do |config|
  config.instrumenter = TracingInstrumenter.new(config.instrumenter)
end

Both methods are optional. Without them, events from concurrent tools still arrive, with no parent from your instrumenter.

Next Steps

• Instrumentation: subscribe to these events.
• OpenTelemetry: send them to your tracing backend.