Repository navigation
fix(antigravity): add image passthrough for Claude models - #898
AndrewDragonIV wants to merge 2 commits into
Conversation
The wrapInCloudCodeEnvelopeForClaude function converts Claude message blocks to Antigravity/Gemini parts format but only handles text, tool_use, and tool_result types. Image blocks (type: 'image' with base64 source) are silently dropped, causing Claude models routed through Antigravity to never receive image content. This adds handling for image blocks, converting them to inlineData format (the same format the Gemini path already uses successfully). Tested: Claude via Antigravity now correctly receives and describes image content that was previously invisible.
There was a problem hiding this comment.
Code Review
This pull request adds support for image content blocks when translating Claude requests to Gemini format within the wrapInCloudCodeEnvelopeForClaude function. A review comment points out that the property name 'mimeType' should be changed to 'mime_type' to ensure compatibility with the Gemini API and consistency with existing helper functions in the codebase.
| } else if (block.type === "image" && block.source) { | ||
| parts.push({ | ||
| inlineData: { | ||
| mimeType: block.source.media_type, |
There was a problem hiding this comment.
The Gemini API and the existing helper functions in this repository (see convertOpenAIContentToParts in geminiHelper.ts) use mime_type (snake_case) rather than mimeType (camelCase) for image parts. To ensure compatibility and consistency with the rest of the translation logic, this should be changed to mime_type.
| mimeType: block.source.media_type, | |
| mime_type: block.source.media_type, |
Code Review SummaryStatus: 1 Issue Found | Recommendation: Address before merge Overview
Issue Details (click to expand)
Files Reviewed (1 file)
NotesThe PR correctly adds image block handling for Claude models routed through Antigravity. The code uses The identified issue is a minor defensive programming concern. The code path typically receives base64 images from the upstream translator, but adding an explicit type check would align with the pattern used in |
|
Merged manually into |
Problem
wrapInCloudCodeEnvelopeForClaude()inopenai-to-gemini.tsconverts Claude message content blocks to Antigravity/Geminipartsformat. It handlestext,tool_use, andtool_resultblock types — but silently dropsimageblocks.This means Claude models routed through the Antigravity provider never receive image content. The model responds with "I don't see any image" even though the image was correctly converted from OpenAI
image_urlformat to Claudeimageformat in the upstream translator.Affected path
Not affected
The Gemini path through Antigravity works correctly —
openaiToGeminiRequest()properly convertsimage_url→inlineData, andwrapInCloudCodeEnvelope()preserves it.Fix
Add an
else ifbranch forblock.type === "image"that converts Claude's image format to Gemini'sinlineDataformat (the same format the Gemini path already uses successfully):Testing
Tested locally on v3.4.1:
curlwith base64 image → Claude responds "I don't see any image"