Audio Transcription

Convert speech to text with support for multiple languages and speaker diarization

After reading this guide, you will know:

• How to transcribe audio files to text.
• How to identify different speakers with diarization.
• How to improve accuracy with language hints and prompts.
• How to access segments and timestamps.
• How to stream a transcript as it is produced.
• How to handle a transcription the provider blocks.

Basic Transcription

Turn a recording into text:

transcription = RubyLLM.transcribe("meeting.wav")
puts transcription.text

Pass a local path, URL, IO object, or Active Storage attachment. Supported audio formats depend on the model.

Choosing Models

Pass model: to choose a transcription model:

RubyLLM.transcribe("meeting.wav", model: "scribe_v2")

Set default_transcription_model in Configuration to change the default. Browse the Models page for audio models. Pass provider: to select a hosted deployment explicitly.

For Azure, pass your deployment name. A diarization deployment with a custom name also needs format: "diarized_json".

Language and Vocabulary Hints

Give the model the recording’s language and unfamiliar terms:

RubyLLM.transcribe("entrevista.mp3", language: "es")

RubyLLM.transcribe(
  "developer-talk.mp3",
  prompt: "Discussion about Ruby, Rails, PostgreSQL, and Redis."
)

Use ISO 639-1 language codes such as en, es, and fr. Google’s dedicated transcription models also accept BCP-47 hints such as en-US.

For provider-specific vocabulary and formatting controls, use provider_options::

RubyLLM.transcribe(
  "developer-talk.mp3",
  model: "scribe_v2",
  provider_options: { keyterms: ["RubyLLM", "Zeitwerk"], tag_audio_events: true }
)

Speaker Diarization

Pass speaker_names: [] to request speaker labels:

transcription = RubyLLM.transcribe(
  "meeting.wav",
  model: "gemini-3.5-transcribe",
  speaker_names: [],
  timestamps: :word
)

transcription.words.each do |word|
  puts "#{word['speaker']}: #{word['word']} (#{word['start']}s)"
end

Speaker and timing fields retain the provider’s names. Depending on the model, speaker labels appear on words or segments. OpenAI’s diarization model returns segments:

transcription = RubyLLM.transcribe("meeting.wav", model: "gpt-4o-transcribe-diarize")

transcription.segments.each do |segment|
  puts "#{segment['speaker']}: #{segment['text']}"
  puts "#{segment['start']}s - #{segment['end']}s"
end

For Google’s dedicated transcription model on Vertex AI, use model: "gemini-3.5-transcribe-preview", provider: :vertexai, and set vertexai_location to "global" in Configuration.

Identifying Known Speakers

OpenAI’s diarization models can match speakers against 2–10 second reference clips:

transcription = RubyLLM.transcribe(
  "team-meeting.wav",
  model: "gpt-4o-transcribe-diarize",
  speaker_names: ["Alice", "Bob"],
  speaker_references: ["alice-voice.wav", "bob-voice.wav"]
)

References accept file paths, URLs, IO objects, or Active Storage attachments. Other diarization providers assign speaker labels without matching known identities. ElevenLabs uses the number of supplied names as a speaker-count limit.

Segments and Timestamps

Use timestamps: to request timing information:

transcription = RubyLLM.transcribe(
  "interview.mp3",
  model: "whisper-1",
  timestamps: [:word, :segment]
)

puts "Duration: #{transcription.duration} seconds"

transcription.words.each do |word|
  puts "#{word['start']}s - #{word['end']}s: #{word['word']}"
end

Available granularities depend on the model. Whisper accepts word and segment timestamps without streaming; Mistral accepts segment timestamps, including streaming. Deepgram, xAI, and ElevenLabs return word timing by default. ElevenLabs also accepts timestamps: :character.

Output Formats

For models that produce subtitles, pass format::

transcription = RubyLLM.transcribe(
  "interview.mp3",
  model: "whisper-1",
  format: "srt"
)
File.write("interview.srt", transcription.text)

Formats use the provider’s names. Leave format: unset to get the model’s default transcript.

Streaming Transcripts

Pass a block to receive text as it is transcribed. The call returns the completed Transcription:

transcription = RubyLLM.transcribe("meeting.wav", model: "gpt-4o-transcribe") do |chunk|
  print chunk.delta if chunk.delta?
end

puts transcription.text

Read the final transcript from transcription.text. chunk.raw holds the original event when you need additional fields.

Diarization models can stream speaker segments:

RubyLLM.transcribe("meeting.wav", model: "gpt-4o-transcribe-diarize") do |chunk|
  next unless chunk.segment?

  puts "#{chunk.segment['speaker']}: #{chunk.segment['text']}"
end

WebSocket Transcription

For Deepgram, ElevenLabs Scribe Realtime, xAI, and Google Live transcription, add the optional dependency:

gem "websocket-driver"

Use the same block API with the appropriate streaming model:

RubyLLM.transcribe(
  "meeting.wav",
  model: "scribe_v2_realtime",
  provider: :elevenlabs,
  assume_model_exists: true
) do |chunk|
  print chunk.delta if chunk.delta?
end

These models have different input requirements:

Google Live returns text without speaker labels or word timestamps; use its dedicated file transcription model for those. Scribe Realtime returns word timing but does not diarize. These WebSocket connections do not support HTTP proxies.

The block processes an existing recording.

Turning a Recording into Meeting Notes

Use the transcript as input to a chat:

transcript = RubyLLM.transcribe "meeting.wav"
notes = RubyLLM.chat.ask "Summarize the decisions and action items:\n#{transcript.text}"
puts notes.content

For fields your application can process, add a schema with Structured Output. To make an audio summary, pass the notes to RubyLLM.speak and save the result.

Blocked Transcriptions

Gemini’s safety filters can block a transcription, even of ordinary speech. Rescue RubyLLM::ContentFilterError to try another model:

begin
  transcription = RubyLLM.transcribe("interview.mp3", model: "gemini-3.7-flash")
rescue RubyLLM::ContentFilterError => e
  Rails.logger.warn(e.message)
  transcription = RubyLLM.transcribe("interview.mp3", model: "gpt-transcribe")
end

The message names the reason the provider gave, as in Gemini blocked the transcription: SAFETY. RubyLLM raises instead of returning an empty transcript, so a blocked recording never looks like a silent one. Silent audio still returns an empty transcription.text.

Google still bills the audio it read before blocking it. The failed attempt keeps those tokens in usage.ruby_llm and, in Rails, in the usage ledger.

Longer Recordings and Errors

Long recordings may need a longer request timeout. See Error Handling for retries and provider failures.

Next Steps

• Chat - ask questions about a transcript.
• Text to Speech - turn a summary into audio.
• Structured Output - extract decisions, speakers, and action items.