Skip to content

fix(translator): convert OpenAI tool-result images to Claude image blocks (#9692) - #10791

Merged
diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.50from
Prudhvivuda:fix/9692-openai-to-claude-tool-images
Aug 20, 2026
Merged

diegosouzapw merged 1 commit into
cheaper-inference:release/v3.8.50from
Prudhvivuda:fix/9692-openai-to-claude-tool-images

Conversation

@Prudhvivuda

@Prudhvivuda Prudhvivuda commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Convert OpenAI image_url (and AI-SDK image) parts nested in role: "tool" / tool_result content into Claude Messages image source blocks during OpenAI→Claude translation.
  • User-message vision parts already converted; tool results were forwarded raw, so the next Claude-format upstream turn 400'd before inference (Kimi Code CLI ReadMediaFile and any OpenAI-compatible client that returns media from a tool).
  • Scope is defect A only (payload conversion). The K3 protocol-override sibling in fix(kimi/translator): tool-result images break OpenAI→Claude routing and K3 protocol override is ineffective #9692 is not changed here.

Closes #9692

⚠️ base-red inherited: #9985

Test plan

  • node --import tsx/esm --test tests/unit/openai-to-claude-tool-result-images-9692.test.ts
  • node --import tsx/esm --test tests/unit/openai-to-claude-strip-empty.test.ts
  • node --import tsx/esm --test tests/unit/openai-to-claude-file-attachments.test.ts
  • node --import tsx/esm --test tests/unit/claude-to-openai-image-toolresult.test.ts
  • node --import tsx/esm --test tests/unit/openai-to-claude-bare-tool.test.ts
  • Optional live: OpenAI-compatible client → OmniRoute → Claude-format Kimi/Anthropic with a tool that returns a PNG; next turn should be 200, not 400 Invalid request

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Verified this against the actual translator code path — solid fix, scoped correctly.

What I checked:

  • Confirmed the base code at getContentBlocksFromMessage really did forward raw image_url/AI-SDK image parts unchanged for role: "tool" and nested tool_result content, while the sibling role: "user" array path already converted them — matches the described defect exactly.
  • imageBlocks.ts cleanly extracts the shared conversion logic (extractOpenAiImageUrl, urlToClaudeImageBlock, openAiImagePartToClaudeBlock, normalizeToolResultImages) and both call sites (role: "tool" and nested tool_result) now route through it.
  • Nice side-effect catch: the old inline code did part.image_url.url unconditionally, which throws TypeError when a client sends a bare-string image_url (no {url} wrapper) instead of an object. extractOpenAiImageUrl now handles both shapes, and your test suite ("mixed text + image_url...") exercises exactly that case. That's a real latent-crash fix beyond the reported issue.
  • Ran the new test file: 7/7 pass. Ran the 5 adjacent openai-to-claude test files (bare-tool, strip-empty, file-attachments, translator-openai-to-claude, empty-messages, glm-split-tool-name) — 32/32 pass, no regressions. ESLint clean on the 3 changed files.
  • On the CI reds: I reproduced mutation-test-coverage, dead-code, and open-sse-typecheck failures identically on the unmodified release/v3.8.50 tip (checked out the exact same scripts against the base commit) — all three are pre-existing base-red (🔴 Release branch not green: release/v3.8.50 #9985) and unrelated to openai-to-claude.ts/imageBlocks.ts. The ⚠️ base-red inherited: #9985 note in the PR body is accurate and appreciated.

No blocking issues found. Good scoping decision keeping the K3 protocol-override sibling out of this PR.

@diegosouzapw
diegosouzapw merged commit 71f858f into cheaper-inference:release/v3.8.50 Aug 20, 2026
5 checks passed
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…ocks (cheaper-inference#9692) (cheaper-inference#10791)

Merged via merge-train (release/v3.8.50, batch1 2026-08-20) — static gates (typecheck/file-size/complexity/cognitive/changelog) green on the combined tree; test:unit reds observed in the boarded run were verified pre-existing on the pure release tip (unrelated flake), not caused by this PR. Thanks for the contribution!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(kimi/translator): tool-result images break OpenAI→Claude routing and K3 protocol override is ineffective

2 participants