# Instrumentation and Observability

Observe RubyLLM model operations, provider attempts, tool calls, workflows, batches, and model refreshes

After reading this guide, you will know:

*   How to subscribe to RubyLLM events in Rails.
*   How to connect RubyLLM instrumentation outside Rails.
*   How to enable optional OpenTelemetry tracing.
*   Which events RubyLLM emits.
*   Which payload fields may contain sensitive application data.

## Rails

Rails apps automatically emit RubyLLM events through `ActiveSupport::Notifications`. Subscribe to them the same way you would subscribe to Rails framework events:

```ruby
# config/initializers/ruby_llm_instrumentation.rb
ActiveSupport::Notifications.subscribe('chat.ruby_llm') do |event|
  payload = event.payload
  Rails.logger.info(
    provider: payload[:provider],
    model: payload[:model],
    duration_ms: event.duration,
    cost: payload[:cost].total
  )
end
```

When an instrumented block raises, Rails adds the standard `:exception` and `:exception_object` payload keys.

## Outside Rails

You can use the same subscribers outside Rails. Add `activesupport` to your bundle, then configure its notifications module:

```ruby
require 'active_support'
require 'active_support/notifications'

RubyLLM.configure do |config|
  config.instrumenter = ActiveSupport::Notifications
end
```

Use your application's logger in subscribers instead of `Rails.logger`.

### Custom Instrumenters

To connect another observability system, set `config.instrumenter` to an object that responds to `instrument(name, payload)` with an optional block. This example passes timing and exception details to your application's `Observability.record` method:

```ruby
class AppInstrumenter
  def instrument(name, payload)
    started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)

yield if block_given?
  rescue StandardError => error
    payload = payload.merge(
      exception: [error.class.name, error.message],
      exception_object: error
    )
    raise
  ensure
    duration = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at
    Observability.record(name, payload.merge(duration: duration))
  end
end

RubyLLM.configure do |config|
  config.instrumenter = AppInstrumenter.new
end
```

You can also set `instrumenter` on a [context](/configuration-connection/#contexts-isolated-configurations) when you only want instrumentation around a specific operation.

## OpenTelemetry

You can trace model calls, tool execution, and workflows with OpenTelemetry. Configure your application's SDK and exporters, then enable tracing:

```ruby
RubyLLM::OpenTelemetry.enable
```

See the [OpenTelemetry guide](/opentelemetry/) for setup, span coverage, context propagation, and exported data. Existing subscribers continue to receive their events and payloads.

## Workflows and Steps

Use `RubyLLM.workflow` to group events from a piece of work. Each step adds its own timing and name. The code inside can call chats, agents, or individual operations:

```ruby
RubyLLM.workflow("Summarize meeting", id: "meeting-42") do |workflow|
  transcript = workflow.step("Transcribe") do
    RubyLLM.transcribe("meeting.wav").text
  end

workflow.step("Summarize") do
    RubyLLM.chat.with_instructions("Summarize the decisions and action items.")
      .ask(transcript).content
  end
end
```

The ID is optional. RubyLLM generates a UUID when it is omitted. Pass an application ID when you need to correlate the same logical workflow across logs, records, or background jobs.

Pass `metadata:` to attach application data to the whole workflow:

```ruby
RubyLLM.workflow("Write article", metadata: { account_id: account.id }) do |workflow|
  # ...
end
```

Nested events receive it as `workflow_metadata`, a separate key from the per-call `metadata` payload described below, so the two never collide.

`Context#workflow` provides the same API while using that context's instrumenter:

```ruby
tenant_llm.workflow("Answer support request", id: ticket.id) do |workflow|
  workflow.step("Answer") { tenant_llm.chat.ask(ticket.question) }
end
```

Every event inside the workflow receives `workflow_id` and `workflow_name`. Events inside a step also receive `workflow_step_id` and `workflow_step_name`. Nested steps include `workflow_step_parent_id`, allowing an instrumentation adapter to render the observed execution tree.

The wrappers also emit `workflow.ruby_llm` and `workflow_step.ruby_llm` events around their blocks. Their timings and exception behavior therefore come from the configured instrumenter in exactly the same way as other RubyLLM block events.

Branches, loops, fan-out, and fan-in stay as normal Ruby. For concurrent application code, start the step inside each task so the task establishes its own context:

```ruby
RubyLLM.workflow("Review code") do |workflow|
  Async do |task|
    security = task.async do
      workflow.step("Security review") { SecurityAgent.new.ask(code) }
    end
    style = task.async do
      workflow.step("Style review") { StyleAgent.new.ask(code) }
    end

[security.wait, style.wait]
  end.wait
end
```

RubyLLM carries the current workflow and step through its own concurrent tool execution. A dashboard can reconstruct the work that ran from these events. Your application controls scheduling, retries, and persistence.

Workflows can nest. A `RubyLLM.workflow` call inside another workflow keeps its own identity: its events carry the inner `workflow_id` and `workflow_name`, plus `workflow_parent_id` and, when the nesting happened inside a step, `workflow_parent_step_id`. The outer workflow's context is restored when the inner block returns. A service object that declares its own workflow therefore appears as a sub-tree when a larger workflow calls it, the same way nested spans appear in OpenTelemetry.

Workflow context lasts exactly as long as the block. Work that resumes elsewhere, such as polling a [batch](/batches/) for results in a later job, is not tagged automatically; re-enter `RubyLLM.workflow` with the persisted ID to correlate it:

```ruby
RubyLLM.workflow("Nightly summaries", id: batch_record.workflow_id) do |workflow|
  workflow.step("Collect results") { RubyLLM::Batch.find(batch_record.batch_id, provider: :anthropic).messages }
end
```

## Per-Call Metadata

One-shot APIs accept `metadata:` for application observability data:

```ruby
RubyLLM.embed(
  "A short document",
  metadata: { account_id: current_account.id, feature: "search" }
)
```

RubyLLM includes that value as `payload[:metadata]` on the emitted event. It is not sent to the provider. Use `provider_options:` for provider request fields, and `metadata:` for values your own instrumentation subscribers need.

## Events

RubyLLM emits these events:

*   `workflow.ruby_llm` - one named workflow block and its correlation ID
*   `workflow_step.ruby_llm` - one named code region within a workflow
*   `evaluation.ruby_llm` - one evaluation run, its progress counts, and completed report
*   `evaluation_trial.ruby_llm` - one completed evaluation case and repetition, including its trial result
*   `request.ruby_llm` - HTTP request metadata such as provider, method, URL, and status
*   `usage.ruby_llm` - one finished provider attempt, including retries and cancellations, with status, tokens, cost, and owner
*   `batch.ruby_llm` - one batch submission operation
*   `chat.ruby_llm` - chat completion metadata including model, provider, messages, response, and token usage
*   `tool_call.ruby_llm` - tool name, arguments, and result
*   `embedding.ruby_llm` - embedding model, input, result, token usage, and vector dimensions
*   `image.ruby_llm` - image generation model, prompt, size, and result
*   `video.ruby_llm` - one blocking video generation, including the wait for the job and the resulting video
*   `video_job.ruby_llm` - one video job submission with model, prompt, and the provider's job id
*   `research_job.ruby_llm` - one hosted research job submission with the provider's job id
*   `moderation.ruby_llm` - moderation model, input, result, and flagged status
*   `ocr.ruby_llm` - OCR model, provider options, and extracted document result
*   `rerank.ruby_llm` - reranking model, query, document count, result, token usage, and cost
*   `judgment.ruby_llm` - judgment model, question count, attachment count, result, token usage, cost, and application metadata
*   `speech.ruby_llm` - speech generation model, input, voice, format, and audio byte size
*   `transcription.ruby_llm` - transcription model, language, result, and token usage
*   `tokenization.ruby_llm` - tokenization model and result
*   `compaction.ruby_llm` - one provider-side compaction of a chat's history
*   `models.refresh.ruby_llm` - model registry refresh metadata

Operations that expose normalized usage (`chat`, `embedding`, `image`, `moderation`, `rerank`, `speech`, and `transcription`) include `payload[:tokens]` and `payload[:cost]`. Both value objects are always present; their individual fields may be `nil` when the provider did not report usage or RubyLLM could not price it. Batch, OCR, and video events expose their operation-specific result and lifecycle fields instead.

See [Evaluation Progress and Monitoring](/evaluation-progress/) for evaluation progress, run IDs, and report payloads.

### Usage Events

Subscribe to `usage.ruby_llm` for cost and usage tracking that survives transport retries and chat cancellation:

```ruby
ActiveSupport::Notifications.subscribe('usage.ruby_llm') do |event|
  payload = event.payload

Rails.logger.info(
    operation: payload[:operation],
    provider: payload[:provider],
    model: payload[:model],
    status: payload[:status],
    input_tokens: payload[:tokens].input,
    output_tokens: payload[:tokens].output,
    cost: payload[:cost].total
  )
end
```

This event fires once for every finished physical attempt. Its payload contains `operation`, `provider`, `model`, `status`, `tokens`, and `cost`. The values use the same objects as the rest of RubyLLM; the internal usage record is not part of the public API. Both objects are always present, while individual fields are `nil` when the provider supplied no defensible figure. See [Cost and Usage Tracking](/cost-and-usage-tracking/) for status semantics and Rails persistence.

`model` is `nil` for operations that do not select a model, so event subscribers should accept a missing model.

## Payloads

Payloads include the Ruby objects needed by observability adapters, but message content, tool arguments, and provider responses may be sensitive. Only export or log those fields when your application policy allows it.

`chat.ruby_llm` also includes `response_tokens`, the usage for that model call before earlier fallback attempts are attached to the returned message. On failure it contains any usage recorded for that call. The existing `tokens` field retains its response-level meaning.

Non-Rails instrumenters control their own error payload behavior. If your instrumenter records exceptions, keep those payloads consistent with the rest of your observability stack.

Instrumentation and Observability

Observe RubyLLM model operations, provider attempts, tool calls, workflows, batches, and model refreshes

After reading this guide, you will know:

How to subscribe to RubyLLM events in Rails.
How to connect RubyLLM instrumentation outside Rails.
How to enable optional OpenTelemetry tracing.
Which events RubyLLM emits.
Which payload fields may contain sensitive application data.

Rails

Rails apps automatically emit RubyLLM events through ActiveSupport::Notifications. Subscribe to them the same way you would subscribe to Rails framework events:

# config/initializers/ruby_llm_instrumentation.rb
ActiveSupport::Notifications.subscribe('chat.ruby_llm') do |event|
  payload = event.payload
  Rails.logger.info(
    provider: payload[:provider],
    model: payload[:model],
    duration_ms: event.duration,
    cost: payload[:cost].total
  )
end

When an instrumented block raises, Rails adds the standard :exception and :exception_object payload keys.

Outside Rails

You can use the same subscribers outside Rails. Add activesupport to your bundle, then configure its notifications module:

require 'active_support'
require 'active_support/notifications'

RubyLLM.configure do |config|
  config.instrumenter = ActiveSupport::Notifications
end

Use your application’s logger in subscribers instead of Rails.logger.

Custom Instrumenters

To connect another observability system, set config.instrumenter to an object that responds to instrument(name, payload) with an optional block. This example passes timing and exception details to your application’s Observability.record method:

class AppInstrumenter
  def instrument(name, payload)
    started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)

    yield if block_given?
  rescue StandardError => error
    payload = payload.merge(
      exception: [error.class.name, error.message],
      exception_object: error
    )
    raise
  ensure
    duration = Process.clock_gettime(Process::CLOCK_MONOTONIC) - started_at
    Observability.record(name, payload.merge(duration: duration))
  end
end

RubyLLM.configure do |config|
  config.instrumenter = AppInstrumenter.new
end

You can also set instrumenter on a context when you only want instrumentation around a specific operation.

OpenTelemetry

You can trace model calls, tool execution, and workflows with OpenTelemetry. Configure your application’s SDK and exporters, then enable tracing:

RubyLLM::OpenTelemetry.enable

See the OpenTelemetry guide for setup, span coverage, context propagation, and exported data. Existing subscribers continue to receive their events and payloads.

Workflows and Steps

Use RubyLLM.workflow to group events from a piece of work. Each step adds its own timing and name. The code inside can call chats, agents, or individual operations:

RubyLLM.workflow("Summarize meeting", id: "meeting-42") do |workflow|
  transcript = workflow.step("Transcribe") do
    RubyLLM.transcribe("meeting.wav").text
  end

  workflow.step("Summarize") do
    RubyLLM.chat.with_instructions("Summarize the decisions and action items.")
      .ask(transcript).content
  end
end

The ID is optional. RubyLLM generates a UUID when it is omitted. Pass an application ID when you need to correlate the same logical workflow across logs, records, or background jobs.

Pass metadata: to attach application data to the whole workflow:

RubyLLM.workflow("Write article", metadata: { account_id: account.id }) do |workflow|
  # ...
end

Nested events receive it as workflow_metadata, a separate key from the per-call metadata payload described below, so the two never collide.

Context#workflow provides the same API while using that context’s instrumenter:

tenant_llm.workflow("Answer support request", id: ticket.id) do |workflow|
  workflow.step("Answer") { tenant_llm.chat.ask(ticket.question) }
end

Every event inside the workflow receives workflow_id and workflow_name. Events inside a step also receive workflow_step_id and workflow_step_name. Nested steps include workflow_step_parent_id, allowing an instrumentation adapter to render the observed execution tree.

The wrappers also emit workflow.ruby_llm and workflow_step.ruby_llm events around their blocks. Their timings and exception behavior therefore come from the configured instrumenter in exactly the same way as other RubyLLM block events.

Branches, loops, fan-out, and fan-in stay as normal Ruby. For concurrent application code, start the step inside each task so the task establishes its own context:

RubyLLM.workflow("Review code") do |workflow|
  Async do |task|
    security = task.async do
      workflow.step("Security review") { SecurityAgent.new.ask(code) }
    end
    style = task.async do
      workflow.step("Style review") { StyleAgent.new.ask(code) }
    end

    [security.wait, style.wait]
  end.wait
end

Workflows can nest. A RubyLLM.workflow call inside another workflow keeps its own identity: its events carry the inner workflow_id and workflow_name, plus workflow_parent_id and, when the nesting happened inside a step, workflow_parent_step_id. The outer workflow’s context is restored when the inner block returns. A service object that declares its own workflow therefore appears as a sub-tree when a larger workflow calls it, the same way nested spans appear in OpenTelemetry.

Workflow context lasts exactly as long as the block. Work that resumes elsewhere, such as polling a batch for results in a later job, is not tagged automatically; re-enter RubyLLM.workflow with the persisted ID to correlate it:

RubyLLM.workflow("Nightly summaries", id: batch_record.workflow_id) do |workflow|
  workflow.step("Collect results") { RubyLLM::Batch.find(batch_record.batch_id, provider: :anthropic).messages }
end

Per-Call Metadata

One-shot APIs accept metadata: for application observability data:

RubyLLM.embed(
  "A short document",
  metadata: { account_id: current_account.id, feature: "search" }
)

RubyLLM includes that value as payload[:metadata] on the emitted event. It is not sent to the provider. Use provider_options: for provider request fields, and metadata: for values your own instrumentation subscribers need.

Events

RubyLLM emits these events:

workflow.ruby_llm - one named workflow block and its correlation ID
workflow_step.ruby_llm - one named code region within a workflow
evaluation.ruby_llm - one evaluation run, its progress counts, and completed report
evaluation_trial.ruby_llm - one completed evaluation case and repetition, including its trial result
request.ruby_llm - HTTP request metadata such as provider, method, URL, and status
usage.ruby_llm - one finished provider attempt, including retries and cancellations, with status, tokens, cost, and owner
batch.ruby_llm - one batch submission operation
chat.ruby_llm - chat completion metadata including model, provider, messages, response, and token usage
tool_call.ruby_llm - tool name, arguments, and result
embedding.ruby_llm - embedding model, input, result, token usage, and vector dimensions
image.ruby_llm - image generation model, prompt, size, and result
video.ruby_llm - one blocking video generation, including the wait for the job and the resulting video
video_job.ruby_llm - one video job submission with model, prompt, and the provider’s job id
research_job.ruby_llm - one hosted research job submission with the provider’s job id
moderation.ruby_llm - moderation model, input, result, and flagged status
ocr.ruby_llm - OCR model, provider options, and extracted document result
rerank.ruby_llm - reranking model, query, document count, result, token usage, and cost
judgment.ruby_llm - judgment model, question count, attachment count, result, token usage, cost, and application metadata
speech.ruby_llm - speech generation model, input, voice, format, and audio byte size
transcription.ruby_llm - transcription model, language, result, and token usage
tokenization.ruby_llm - tokenization model and result
compaction.ruby_llm - one provider-side compaction of a chat’s history
models.refresh.ruby_llm - model registry refresh metadata

Operations that expose normalized usage (chat, embedding, image, moderation, rerank, speech, and transcription) include payload[:tokens] and payload[:cost]. Both value objects are always present; their individual fields may be nil when the provider did not report usage or RubyLLM could not price it. Batch, OCR, and video events expose their operation-specific result and lifecycle fields instead.

See Evaluation Progress and Monitoring for evaluation progress, run IDs, and report payloads.

Usage Events

Subscribe to usage.ruby_llm for cost and usage tracking that survives transport retries and chat cancellation:

ActiveSupport::Notifications.subscribe('usage.ruby_llm') do |event|
  payload = event.payload

  Rails.logger.info(
    operation: payload[:operation],
    provider: payload[:provider],
    model: payload[:model],
    status: payload[:status],
    input_tokens: payload[:tokens].input,
    output_tokens: payload[:tokens].output,
    cost: payload[:cost].total
  )
end

This event fires once for every finished physical attempt. Its payload contains operation, provider, model, status, tokens, and cost. The values use the same objects as the rest of RubyLLM; the internal usage record is not part of the public API. Both objects are always present, while individual fields are nil when the provider supplied no defensible figure. See Cost and Usage Tracking for status semantics and Rails persistence.

model is nil for operations that do not select a model, so event subscribers should accept a missing model.

Payloads

chat.ruby_llm also includes response_tokens, the usage for that model call before earlier fallback attempts are attached to the returned message. On failure it contains any usage recorded for that call. The existing tokens field retains its response-level meaning.

Non-Rails instrumenters control their own error payload behavior. If your instrumenter records exceptions, keep those payloads consistent with the rest of your observability stack.