Skip to content

fix(openai-shim): handle image tool results per provider - #1276

Closed
stamsam wants to merge 2 commits into
Twigpine:mainfrom
stamsam:codex/fix-425-tool-result-images
Closed

stamsam wants to merge 2 commits into
Twigpine:mainfrom
stamsam:codex/fix-425-tool-result-images

Conversation

@stamsam

@stamsam stamsam commented May 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add provider-gated handling for image content returned inside tool_result blocks
  • keep OpenAI default behavior unchanged
  • make DeepSeek tool-result images text-only so sessions do not 400 on image_url
  • split Xiaomi MiMo/OpenGateway image tool results into tool text plus a user image message

Fixes #425.

Testing

  • bun test src/services/api/openaiShim.test.ts -t "DeepSeek image tool results"
  • bun test src/services/api/openaiShim.test.ts -t "Xiaomi MiMo image tool results"
  • bun test src/services/api/openaiShim.test.ts -t "preserves image tool results"
  • bun test src/services/api/openaiShim.test.ts -t "preserves mixed text and image tool results"
  • git diff --check

@jatmn

jatmn commented May 20, 2026

Copy link
Copy Markdown
Collaborator

Closing this as a duplicate/competing follow-up to the earlier OpenAI-shim image tool-result work in #237, with related overlap in #268.

#237 predates this PR and already covers preserving images in tool_result handling in src/services/api/openaiShim.ts with focused tests. That existing PR is already under maintainer review, including a current requested-change thread about how image tool results should be represented safely for provider compatibility. #268 also overlaps the same tool-result image handling area while addressing third-party request leakage.

Please continue the fix on the existing reviewed PR thread rather than opening a parallel branch for the same shim behavior.

@jatmn jatmn closed this May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Image reading tool appears to be broken

2 participants