fix Claude relay to handle file content blocks to support document types

[sevenjay]

fix Claude relay to handle file content blocks as document blocks (PDF + plain text) and to guard against nil image/file content so it fails gracefully instead of panicking.

In the new-api/relay/channel/claude/relay-claude.go, around line 360, if it's not text, it treats it as an image. It doesn't support document types. When you send a non-image file, the imageUrl object can't be extracted, which causes a panic.

Summary by CodeRabbit

• New Features
  • Extended Claude integration to support multiple content types including text, images (with URL and base64-encoded support), and file attachments.
  • Added automatic handling for different file types including PDFs and text documents with proper format validation and conversion.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Enhanced Claude message construction in the relay service to support multiple content types: text, image URLs, and files. Added base64 encoding/decoding capabilities, MIME type normalization, and special processing logic for PDF and plain text files.

Changes

Cohort / File(s)	Summary
Claude Content Type Handling relay/channel/claude/relay-claude.go	Extended message construction with multi-content-type support: added ContentTypeImageURL handling with validation and base64/URL support; introduced ContentTypeFile handling with MIME type normalization, special cases for application/pdf (base64 transmission) and text/plain (decoded text transmission), and error handling for unsupported types. Added base64 import.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

• seefs001

Poem

🐰 A hop and a bound through content types new,
Images dancing and files breaking through,
Base64 whispers, MIME types align,
PDFs and text in perfect design!
The relay hops forward, so clever and bright!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: adding support for file content blocks (including PDF and plain text documents) to the Claude relay, which directly addresses the panic issue when non-image files were sent.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Fix all issues with AI agents

In `@relay/channel/claude/relay-claude.go`:
- Around line 389-441: The code misinterprets raw base64 as images because
DecodeBase64FileData expects a data: URI; update the file handling in the
dto.ContentTypeFile branch to explicitly handle "data:" URIs and fail fast for
raw base64: check file.FileData for a "data:" prefix (in addition to the
existing "http" check), pass that through DecodeBase64FileData to obtain
mimeType and base64Data, and if file.FileData is raw base64 (no "data:" and not
an http URL) return a clear error saying the caller must provide a
data:<mime>;base64,<data> URI (or alternatively add a mime field to MessageFile
if you prefer supporting raw base64), preserving the existing logic that maps
"application/pdf" and "text/plain" into dto.ClaudeMessageSource and rejecting
unsupported mime types.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# First, find the FileData definition and its usage patterns
echo "=== Searching for FileData definitions ==="
rg -n -B2 -A2 'FileData\s*string' --type go

echo -e "\n=== Searching for file_data mentions in comments/docs ==="
rg -n -C2 'file_data|file.FileData' --type go | head -100

echo -e "\n=== Looking for ContentTypeFile usage ==="
rg -n -C3 'ContentTypeFile' --type go

Repository: QuantumNous/new-api

Length of output: 8001

---

🏁 Script executed:

#!/bin/bash
# Find the DecodeBase64FileData implementation
echo "=== Finding DecodeBase64FileData implementation ==="
rg -n -B5 -A15 'func.*DecodeBase64FileData' --type go

echo -e "\n=== Finding GetFileBase64FromUrl implementation ==="
rg -n -B5 -A15 'func.*GetFileBase64FromUrl' --type go

Repository: QuantumNous/new-api

Length of output: 2503

---

🏁 Script executed:

#!/bin/bash
# Search for test files related to file handling
echo "=== Finding test files with file/base64 tests ==="
fd -e _test.go | xargs rg -l 'FileData|DecodeBase64FileData|GetFileBase64FromUrl' 2>/dev/null

echo -e "\n=== Searching for example usages or data preparation ==="
rg -n 'FileData.*=' --type go | head -30

Repository: QuantumNous/new-api

Length of output: 890

---

🏁 Script executed:

#!/bin/bash
# Get full DecodeBase64FileData and DecodeBase64ImageData implementations
echo "=== Full DecodeBase64FileData implementation ==="
sed -n '43,80p' service/image.go

echo -e "\n=== Full DecodeBase64ImageData implementation ==="
rg -n -A30 'func DecodeBase64ImageData' service/image.go

Repository: QuantumNous/new-api

Length of output: 2455

---

🏁 Script executed:

#!/bin/bash
# Look for tests or documentation about FileData format expectations
echo "=== Test files mentioning DecodeBase64FileData ==="
fd -e _test.go | xargs rg -l 'DecodeBase64FileData' 2>/dev/null

echo -e "\n=== Example or documentation files ==="
rg -n 'FileData.*format|file_data.*format|base64.*PDF|base64.*text' --type go -i

Repository: QuantumNous/new-api

Length of output: 1217

---

🏁 Script executed:

#!/bin/bash
# Check if data: prefix checking exists elsewhere, per the learning
echo "=== Checking for data: prefix handling patterns ==="
rg -n 'strings.HasPrefix.*"data:"' --type go

echo -e "\n=== Check task_video.go from the learning ==="
rg -n -B5 -A5 'FailReason.*data:' --type go

Repository: QuantumNous/new-api

Length of output: 1465

---

Clarify expected file_data format and add data: URI handling.
DecodeBase64FileData only extracts MIME types from proper data-URIs (data:<mime>;base64,<data>); raw base64 triggers image detection, returning "image/<format>" instead of the expected "application/pdf" or "text/plain". The code checks for "http" prefix at line 406 but doesn't check for "data:" prefix (which the codebase already handles elsewhere). Either require and validate data-URI format with explicit MIME type, add a MIME type parameter to MessageFile, or fail fast with a clear error message about expected format.

🤖 Prompt for AI Agents

In `@relay/channel/claude/relay-claude.go` around lines 389 - 441, The code
misinterprets raw base64 as images because DecodeBase64FileData expects a data:
URI; update the file handling in the dto.ContentTypeFile branch to explicitly
handle "data:" URIs and fail fast for raw base64: check file.FileData for a
"data:" prefix (in addition to the existing "http" check), pass that through
DecodeBase64FileData to obtain mimeType and base64Data, and if file.FileData is
raw base64 (no "data:" and not an http URL) return a clear error saying the
caller must provide a data:<mime>;base64,<data> URI (or alternatively add a mime
field to MessageFile if you prefer supporting raw base64), preserving the
existing logic that maps "application/pdf" and "text/plain" into
dto.ClaudeMessageSource and rejecting unsupported mime types.