Skip to content

fix(antigravity): add image passthrough for Claude models - #898

Closed
AndrewDragonIV wants to merge 2 commits into
cheaper-inference:mainfrom
AndrewDragonIV:fix/antigravity-claude-image-passthrough
Closed

AndrewDragonIV wants to merge 2 commits into
cheaper-inference:mainfrom
AndrewDragonIV:fix/antigravity-claude-image-passthrough

Conversation

@AndrewDragonIV

Copy link
Copy Markdown
Contributor

Problem

wrapInCloudCodeEnvelopeForClaude() in openai-to-gemini.ts converts Claude message content blocks to Antigravity/Gemini parts format. It handles text, tool_use, and tool_result block types — but silently drops image blocks.

This means Claude models routed through the Antigravity provider never receive image content. The model responds with "I don't see any image" even though the image was correctly converted from OpenAI image_url format to Claude image format in the upstream translator.

Affected path

OpenAI format (image_url) 
  → openaiToClaudeRequest (correctly converts to {type: "image", source: {base64}})
  → openaiToClaudeRequestForAntigravity (passes through)
  → wrapInCloudCodeEnvelopeForClaude (BUG: drops image blocks)
  → Antigravity API (never sees the image)

Not affected

The Gemini path through Antigravity works correctly — openaiToGeminiRequest() properly converts image_url → inlineData, and wrapInCloudCodeEnvelope() preserves it.

Fix

Add an else if branch for block.type === "image" that converts Claude's image format to Gemini's inlineData format (the same format the Gemini path already uses successfully):

} else if (block.type === "image" && block.source) {
  parts.push({
    inlineData: {
      mimeType: block.source.media_type,
      data: block.source.data,
    },
  });
}

Testing

Tested locally on v3.4.1:

  • Before patch: curl with base64 image → Claude responds "I don't see any image"
  • After patch: Claude correctly describes image content
  • Gemini models: Unaffected (already working via separate path)
  • Text/tool requests: Unaffected

The wrapInCloudCodeEnvelopeForClaude function converts Claude message
blocks to Antigravity/Gemini parts format but only handles text,
tool_use, and tool_result types. Image blocks (type: 'image' with
base64 source) are silently dropped, causing Claude models routed
through Antigravity to never receive image content.

This adds handling for image blocks, converting them to inlineData
format (the same format the Gemini path already uses successfully).

Tested: Claude via Antigravity now correctly receives and describes
image content that was previously invisible.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for image content blocks when translating Claude requests to Gemini format within the wrapInCloudCodeEnvelopeForClaude function. A review comment points out that the property name 'mimeType' should be changed to 'mime_type' to ensure compatibility with the Gemini API and consistency with existing helper functions in the codebase.

} else if (block.type === "image" && block.source) {
parts.push({
inlineData: {
mimeType: block.source.media_type,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The Gemini API and the existing helper functions in this repository (see convertOpenAIContentToParts in geminiHelper.ts) use mime_type (snake_case) rather than mimeType (camelCase) for image parts. To ensure compatibility and consistency with the rest of the translation logic, this should be changed to mime_type.

Suggested change
mimeType: block.source.media_type,
mime_type: block.source.media_type,

@kilo-code-bot

kilo-code-bot Bot commented Apr 1, 2026 •

Copy link
Copy Markdown

Code Review Summary

Status: 1 Issue Found | Recommendation: Address before merge

Overview

Severity Count
WARNING 1
Issue Details (click to expand)
File Line Issue
open-sse/translator/request/openai-to-gemini.ts 435 Missing defensive check for block.source.type - Unlike claude-to-gemini.ts (line 132) which checks block.source?.type === "base64", this code only checks block.source existence. While the typical flow through openai-to-claude.ts always produces base64 images, adding explicit type validation would be more defensive.
Files Reviewed (1 file)
  • open-sse/translator/request/openai-to-gemini.ts - 1 issue

Notes

The PR correctly adds image block handling for Claude models routed through Antigravity. The code uses mime_type (snake_case) which is consistent with geminiHelper.ts. The existing comment about mimeType vs mime_type appears to be incorrect - the code in the diff already uses mime_type (snake_case).

The identified issue is a minor defensive programming concern. The code path typically receives base64 images from the upstream translator, but adding an explicit type check would align with the pattern used in claude-to-gemini.ts.

diegosouzapw added a commit that referenced this pull request Apr 1, 2026
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Merged manually into release/v3.4.2. Thanks for the contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants