Attachments

Ask questions about images, recordings, videos, and documents through one attachment API.

After reading this guide, you will know:

  • How to attach images, video, audio, text files, and PDFs to a message.
  • How to send local files and remote URLs through the with: parameter.
  • How RubyLLM detects file types automatically.
  • How to trade tokens for detail with resolution:.
  • When to use in-prompt attachments versus provider-managed file IDs.

Attaching Files

Send files alongside your question with with:. Choose a model that accepts the file’s input type; the Models page lets you filter by input modality.

chat = RubyLLM.chat(model: "gemini-3.7-flash")
response = chat.ask "What happens in this video?", with: "demo.mp4"
puts response.content

RubyLLM can upload large local attachments automatically. To upload once and reuse a file across requests, see Files.

Attachment Security

String sources are instructions to read a local file or fetch a URL. Only pass paths and URLs that your application trusts and the current user is authorized to access. RubyLLM does not restrict them to a directory or public network addresses. Reading can begin during attachment construction to detect the file type.

Passing an unchecked upload parameter to with: or RubyLLM::Attachment.new lets a user supply a URL or server-side path instead of a file. This can expose internal services through server-side request forgery (SSRF), or disclose local files when their contents are sent to a model. The same restriction applies to file inputs for standalone operations such as RubyLLM.upload and RubyLLM.transcribe.

For browser uploads, validate that each value is an actual uploaded file before passing it to RubyLLM. The Rails persistence example shows this check. For stored files, load attachments through records the current user is authorized to access.

Working with Images

Vision-capable models can analyze images, answer questions about visual content, and even compare multiple images.

chat = RubyLLM.chat(model: 'gpt-5.6')

response = chat.ask "Describe this logo.", with: "path/to/ruby_logo.png"
puts response.content

response = chat.ask "What kind of architecture is shown here?", with: "https://example.com/eiffel_tower.jpg"
puts response.content

response = chat.ask "Compare the user interfaces in these two screenshots.", with: ["screenshot_v1.png", "screenshot_v2.png"]
puts response.content

Working with Videos

Ask a video-capable model about local videos or URLs:

chat = RubyLLM.chat(model: 'gemini-3.7-flash')
response = chat.ask "What happens in this video?", with: "path/to/demo.mp4"
puts response.content

response = chat.ask "Summarize the main events in this video.", with: "https://example.com/demo_video.mp4"
puts response.content

response = chat.ask "Analyze these files for visual content.", with: ["diagram.png", "demo.mp4", "notes.txt"]
puts response.content

Working with Audio

Audio-capable models can answer questions about a recording:

chat = RubyLLM.chat(model: 'gpt-audio-1.5')

response = chat.ask "Please transcribe this meeting recording.", with: "path/to/meeting.mp3"
puts response.content

response = chat.ask "What were the main action items discussed?"
puts response.content

For a transcript without a conversation, use Audio Transcription.

Working with Text Files

You can provide text files directly to models for analysis, summarization, or question answering. This works with any text-based format including plain text, code files, CSV, JSON, and more.

chat = RubyLLM.chat(model: 'claude-sonnet-5')

response = chat.ask "Summarize the key points in this document.", with: "path/to/document.txt"
puts response.content

response = chat.ask "Explain what this Ruby file does.", with: "app/models/user.rb"
puts response.content

Working with PDFs

Ask about a report, manual, or research paper with a model that accepts PDFs:

chat = RubyLLM.chat(model: 'claude-sonnet-5')

response = chat.ask "Summarize the key findings in this research paper.", with: "path/to/paper.pdf"
puts response.content

response = chat.ask "What are the terms and conditions outlined here?", with: "https://example.com/terms.pdf"
puts response.content

response = chat.ask "Based on section 3 of this document, what is the warranty period?", with: "manual.pdf"
puts response.content

Documents still have to fit the model’s context limit, even when uploaded separately.

Working with Office Documents

Word documents, presentations, and spreadsheets attach the same way when the selected model and provider support them:

chat = RubyLLM.chat(model: 'gpt-5.6-luna')

response = chat.ask "What is the project codename in this document?", with: "brief.docx"
response = chat.ask "Which region had the highest revenue?", with: "sales.xlsx"

Providers without native document support raise RubyLLM::UnsupportedAttachmentError. Convert those files to PDF or text first.

Converting Unsupported Attachments

Register a converter when you want the same conversation to work across protocols with different document formats:

require "stringio"

chat.convert_unsupported_attachments do |attachment|
  text = TextExtraction.call(attachment)
  RubyLLM::Attachment.new(StringIO.new(text), filename: "extracted.txt") if text
end

chat.ask "Summarize this report.", with: "report.docx"

TextExtraction is your application’s converter. The callback receives the original RubyLLM::Attachment and returns a replacement attachment. Return nil when you cannot convert it; RubyLLM keeps the protocol’s existing error. A replacement must itself be supported. Errors from your converter propagate to the caller.

Callbacks run in registration order until one returns a replacement. A chat converts each attachment once and reuses the replacement, including its provider upload, on later requests. A persisted chat loaded in another process converts again, so cache expensive conversions in your application. The conversation keeps the original attachment. Switching to a protocol that supports the original file sends that file without calling the converter. Persisted chats and agents use the same callback.

If the provider deletes an uploaded replacement, RubyLLM uploads the cached replacement again and retries the request once. It does not call your converter again or retry after streaming has reached your application.

This handles formats the protocol cannot send. A model can still reject an attachment the protocol can represent; that API error does not trigger the callback.

Choosing the Media Resolution

Small print and dense tables need more detail than a photo of a cat. Build the attachment yourself and set resolution: to control how many tokens the model spends on it:

page = RubyLLM::Attachment.new("page-3.png", resolution: :original)
chat.ask "Where is the revenue figure?", with: page

The values are :low, :medium, :high, :ultra_high, and :original. Each attachment keeps its own setting, so one message can mix a high-detail page with low-detail thumbnails. Leave it unset to use the provider’s default. Persisted chats keep the setting, so later turns and background jobs send the same detail.

Gemini applies the setting to images, videos, and PDFs, sending :high for videos and PDFs when you ask for :ultra_high. OpenAI, Azure, OpenRouter, and xAI apply it to images, sending :low as low detail and :medium, :high, and :ultra_high as high detail. Other providers ignore it, since it is a quality hint rather than a requirement.

For :original, OpenAI and Azure send detail: "original". RubyLLM passes the request through without checking the model; a model that does not support original detail returns an API error. Gemini treats :original like :ultra_high, using its highest setting for the attachment type. OpenRouter and xAI use high detail, and other providers ignore it.

Automatic File Type Detection

RubyLLM automatically detects file types based on extensions and content, so you can pass files directly without specifying the type:

chat = RubyLLM.chat(model: 'gemini-3.7-flash')

response = chat.ask "What's in this file?", with: "path/to/document.pdf"

# Multiple files of different types
response = chat.ask "Analyze these files", with: [
  "diagram.png",
  "report.pdf",
  "meeting_notes.txt",
  "recording.mp3"
]

Recognized file types:

  • Images: .jpg, .jpeg, .png, .gif, .webp, .bmp
  • Videos: .mp4, .mov, .avi, .webm
  • Audio: .mp3, .wav, .m4a, .ogg, .flac
  • Documents: .pdf, .txt, .md, .csv, .json, .xml
  • Code: .rb, .py, .js, .html, .css (and many others)

Next Steps