Tokens and Costs

Read normalized token counts and costs for every response, chat, and provider attempt, including retries and cancellations.

After reading this guide, you will know:

• How to read input, output, cache, and thinking token counts from a response.
• How to read per-turn and per-conversation costs.
• How to count a request’s tokens before you send it.
• How RubyLLM normalizes token buckets across providers.
• How the internal usage ledger accounts for retries, fallbacks, and cancellations.
• How to price token usage yourself with cost_for and Cost.aggregate.
• How costs are recorded in Rails and how to keep registry pricing fresh.

Reading Tokens and Costs

Token counts always live on a RubyLLM::Tokens value, and costs always live on a RubyLLM::Cost value. Both objects are present even when a provider does not report every field; unknown fields are nil:

response = chat.ask "Explain the Ruby Global Interpreter Lock (GIL)."

input_tokens = response.tokens.input
output_tokens = response.tokens.output
cache_read_tokens = response.tokens.cache_read
cache_write_tokens = response.tokens.cache_write
thinking_tokens = response.tokens.thinking

puts "Input Tokens: #{input_tokens}"
puts "Output Tokens: #{output_tokens}"

puts "Input Cost: $#{format('%.6f', response.cost.input)}" if response.cost.input
puts "Output Cost: $#{format('%.6f', response.cost.output)}" if response.cost.output
puts "Total Cost: $#{format('%.6f', response.cost.total)}" if response.cost.total

puts "Total Conversation Cost: $#{format('%.6f', chat.cost.total)}" if chat.cost.total

response.tokens aggregates every transport attempt associated with the response. chat.tokens and chat.cost aggregate the chat’s whole internal ledger, including failed retries and cancelled attempts (see The Usage Ledger below).

The same pair of value objects is used by one-shot results. Their fields remain nil when the provider does not report enough information:

embedding = RubyLLM.embed("Ruby")
embedding.tokens.input
embedding.cost.total

transcription = RubyLLM.transcribe("meeting.wav")
transcription.tokens.input
transcription.tokens.output
transcription.cost.total

RubyLLM uses token usage from the provider and pricing from the model registry. If the registry is missing pricing for tokens that were used, the affected cost and cost.total return nil instead of pretending the cost was zero. These helpers cover token-priced conversation usage; provider-specific add-ons such as search-query charges are left to the provider’s raw usage payload.

Provider-Reported Costs

Some providers report the exact amount a request cost. OpenRouter includes it on every chat, embedding, image, and transcription response. When a reported cost is present, tokens.reported_cost carries it and cost.total returns it instead of a registry-price estimate, so totals stay exact even for models the registry has no pricing for. Component costs such as cost.input are still estimated from registry pricing.

chat = RubyLLM.chat(model: "claude-haiku-4-5", provider: :openrouter)
response = chat.ask "Explain Ractors."

response.tokens.reported_cost # exact request cost in USD, nil on providers that report none
response.cost.total           # equals the reported cost when one is present

Counting Tokens Before You Send

Everything above reports usage after a request ran. chat.count_tokens asks the provider how many input tokens the next request would carry before sending it, which is what you need to enforce a context budget or meter user input before submission:

chat = RubyLLM.chat(model: "claude-sonnet-5")
           .with_instructions(system_prompt)
           .with_tools(Weather)

chat.count_tokens("Summarize the attached contract.") # => 9412

The count covers the whole request the chat would send as currently configured: instructions, tools, thinking configuration, and attachments, not only the message. Passing a message stages it for the count without adding it to the conversation, so the chat is unchanged afterwards:

chat.count_tokens("What's the weather in Berlin?")
chat.messages.size # => 0

Call it with no argument to count the conversation as it stands:

chat.ask "Tell me about Ruby's history."
chat.count_tokens # => 1204, everything the next request would carry

When you want to measure a bare string rather than a configured chat, RubyLLM.count_tokens builds a throwaway chat for you:

RubyLLM.count_tokens("What is the capital of France?", model: "claude-haiku-4-5") # => 14

This is the provider’s own tokenizer over the real payload, not an estimate, which is why it needs a provider that offers a counting endpoint. Anthropic, Bedrock, Gemini, and Vertex AI do. The rest raise rather than guess:

RubyLLM.chat(model: "deepseek-chat").count_tokens("hi")
# => RubyLLM::Error: DeepSeek doesn't support token counting

A guessed token count is worse than no count, because you would size a budget against it. Rescue RubyLLM::Error if your code runs across providers that differ here.

How Providers Are Normalized

RubyLLM handles provider token differences for you. tokens.input means the standard input bucket used for pricing. Cache activity is exposed separately as tokens.cache_read and tokens.cache_write, even when the provider includes those tokens in a raw prompt total.

This means the same RubyLLM code works across providers: tokens.input for standard input, tokens.output for output, tokens.cache_read for prompt cache reads, and tokens.cache_write for prompt cache writes. To display the full request-side input activity, add tokens.input + tokens.cache_read + tokens.cache_write.

Thinking token usage is available via response.tokens.thinking when providers report it. For most providers, thinking/reasoning tokens are a breakdown of output work, not an extra bucket to add yourself. RubyLLM keeps tokens.output as the billable output bucket: OpenAI-style providers that include reasoning in completion tokens stay as-is, while OpenAI-compatible providers that report reasoning outside completion tokens are normalized so tokens.output includes the billable generated total.

When a model has distinct reasoning-token pricing, response.cost.thinking prices that bucket separately. Otherwise, thinking tokens are treated as part of response.cost.output and response.cost.thinking stays nil.

The Usage Ledger

An LLM transcript is not an accounting ledger. A request can be retried before one response arrives, and a cancelled stream can consume tokens without producing a completed assistant message. Counting messages therefore undercounts real provider work.

RubyLLM records each physical provider attempt internally. This changes the implementation of token and cost helpers without adding a tracking object to the public API.

One response can require several physical attempts. If a transport request fails twice and the third attempt succeeds, the response’s tokens and cost aggregate the reported usage from all three attempts.

Fallbacks work the same way. When an initial model fails and a fallback produces the message, every attempt involved in that logical generation contributes to the resulting message’s aggregates.

A cancelled attempt may have no message to attach to. It still contributes to chat.tokens and chat.cost when RubyLLM has enough reported usage. If an attempt may have been billed but the provider supplied no defensible usage, the token fields for that attempt and chat.cost.total are nil rather than misleading zeroes. An attempt that provably never reached the provider, such as a refused connection or a failed TLS handshake, records zero tokens instead: it cannot have been billed, so it does not blank the total.

Pricing Usage Yourself

Once a model is in the registry, RubyLLM can turn token usage into a RubyLLM::Cost object so you can attach a dollar figure to any usage payload:

model = RubyLLM.models.find('gpt-5.6-luna')
response = RubyLLM.chat(model: model.id, provider: model.provider).ask("Summarize Ruby's object model.")

cost = model.cost_for(response.tokens)
puts cost.input
puts cost.output
puts cost.cache_read
puts cost.cache_write
puts cost.thinking
puts cost.total

When a model publishes a long-context tier, cost_for uses those rates once the prompt exceeds the registry threshold. Prompt size is input + cache_read + cache_write. Below that threshold, or when the model has no long-context tier, standard rates apply. Batch rates remain unused by cost_for; see Batches.

Most applications use the shorter helpers on messages, chats, and agents: response.cost.total, chat.cost.total, agent.cost.total.

To combine several cost objects yourself, use RubyLLM::Cost.aggregate:

cost = RubyLLM::Cost.aggregate(messages.map(&:cost))
cost.total

If pricing is incomplete for tokens that were used, the affected cost and cost.total return nil.

Rails Persistence

With acts_as_chat, RubyLLM writes finished attempts to ruby_llm_usages immediately. This happens before the message callback, so cancellation cannot erase usage merely because no assistant message was saved.

chat = Chat.find(params[:id])
message = chat.messages.last

message.tokens.input
message.cost.total
chat.tokens.input
chat.cost.total

The ledger is internal to RubyLLM; your application still owns only its Chat and Message models. Each usage row records the operation, provider, model, status, token buckets, cost components, and timestamps in normalized numeric columns, so ordinary SQL sums and period queries work directly against ruby_llm_usages. Costs are frozen at completion, so a later RubyLLM.models.refresh that changes registry pricing leaves recorded usage untouched. The upgrade migration moves token counts and frozen costs from pre-2.0 message columns into the ledger and removes those columns, so old and new rows read through the same path.

Keeping Registry Pricing Fresh

Because recorded costs are frozen at completion, stale registry pricing only affects new attempts, never usage you have already saved. To keep new costs accurate, refresh the registry on a schedule. A daily job is a good default; providers change prices infrequently.

In a Rails app with the database-backed registry, the same refresh call writes the new pricing to RubyLLM’s internal model table:

# lib/tasks/ruby_llm.rake
namespace :ruby_llm do
  desc 'Refresh the model registry pricing and capabilities'
  task refresh_models: :environment do
    RubyLLM.models.refresh
  end
end

Run bin/rails ruby_llm:refresh_models from cron, or schedule the call with your background job framework (GoodJob, Sidekiq, or similar). RubyLLM does not refresh the registry on its own; you choose the cadence.

Instrumentation

Every finished physical attempt emits usage.ruby_llm. Use it when you need per-attempt detail for billing exports, metrics, or observability:

ActiveSupport::Notifications.subscribe("usage.ruby_llm") do |event|
  payload = event.payload

  Metrics.increment("llm.attempt", tags: {
    provider: payload[:provider],
    model: payload[:model],
    status: payload[:status]
  })

  Metrics.distribution("llm.input_tokens", payload[:tokens].input) if payload[:tokens].input
  Metrics.distribution("llm.cost", payload[:cost].total) if payload[:cost].total
end

The payload contains operation, provider, model, status, tokens, and cost. It does not expose RubyLLM’s internal persistence record.

status is succeeded, failed, or cancelled. tokens and cost use the same value objects returned by responses. They are always present; individual fields are nil when the provider did not report enough information. A failed or cancelled attempt may contain the token counts observed before it stopped.

Instrumentation is an observer, not the source of truth for a persisted Rails chat. RubyLLM maintains the internal ledger and emits the same normalized facts for applications that need them elsewhere.

Next Steps

• Chat - the core conversation interface these counts come from.
• Working with Models - explore the registry and the RubyLLM::Model pricing fields behind cost_for.
• Extended Thinking - work with reasoning-capable models.
• Instrumentation and Observability - configure subscribers and custom instrumenters.
• Persistence with acts_as - persist chats while RubyLLM manages the internal ledger.
• Error Handling - recover from provider failures and configure fallbacks.