fix(xai): GIF attachments reach Grok as PNG instead of being silently stripped (port opencode#53787)

[teknium1]

GIF images attached to an xAI (Grok) turn now reach the model as PNG. Before this, they were silently dropped and the model answered as if nothing had been attached.

What changes

• agent/codex_responses_adapter.py: on the xai_responses issuer, image/gif joins the inline subtypes the converter rejects, and a rejected inline raster is re-encoded to PNG with the Pillow transcoder native attachments already use (agent.image_routing._transcode_to_png). SVG keeps its rasterizer. If an image can't be re-encoded, it gets the existing text placeholder.
• The rejected set is threaded through _chat_content_to_responses_parts / _tool_output_items, so user turns, replayed history, vision_analyze tool results and auxiliary Responses calls all get the same handling.
• tools/vision_tools_image_prep.unsupported_inline_image_media_type takes an optional also_rejected set for per-backend refusals. The default behavior is unchanged.
• Other backends (Codex, OpenAI, Anthropic, custom Responses relays) still receive the original GIF.

Root cause

xAI decodes only JPG/PNG/WebP/ICO, and a GIF input_image gets HTTP 400 invalid_image. Hermes classifies that 400 as image_corrupt (_IMAGE_CORRUPT_PATTERNS, #69078) and strips the image from the retry. The retried turn succeeds, so the user sees an answer with no error, but the model never saw the image.

Validation

Check	origin/main	this PR
Direct API, grok-4.7 /v1/responses (stream + non-stream) and grok-4.20 /v1/chat/completions, GIF part	400 Downloaded response does not contain a valid JPG, PNG, WebP, or ICO image.	n/a (vendor behavior)
Live hermes chat -q … --image solid-green.gif (provider xai, model grok-4.7), neutral filename, "do not use tools"	NOIMAGE ×3	green ×3
Wire capture (HERMES_DUMP_REQUESTS=1) of the user turn	attempt 1 input_image data:image/gif, then attempt 2 a text-only string (image stripped)	a single attempt carrying input_image data:image/png
Same harness, real PNG control	green ×2	green ×2
test_xai_gif_is_reencoded_to_png_and_other_backends_keep_it	red (sources swapped to git show origin/main: copies)	green
tests/agent/test_codex_responses_adapter.py, test_anthropic_adapter.py, test_image_routing.py, tests/tools/test_vision_tools.py	—	green

tests/agent/transports/ shows the same 70 failures on a pristine origin/main archive as on this branch in this host's environment (provider-plugin snapshot/guard environment), so they are unrelated to this diff and CI is the judge.

Live probe cost: ~12 xAI calls on grok-4.7 / grok-4.20 at ~16k input tokens each, well under $0.50.

Source

Ported from anomalyco/opencode#53787 ("only send png, jpeg and webp images to xAI"). OpenCode replaces the GIF with an ERROR: Cannot read … text part at its model-request layer. Hermes re-encodes it to PNG at the Responses wire seam instead, so the model still sees the image. Hermes also already owns the transcoder and the per-backend placeholder seam (#29711), so the port only extends that seam to accept a per-issuer rejected set.

Infographic

[github-actions]

૮ >ﻌ< ა ci review

_{ran on a2f5a68 — fix(xai): GIF attachments reach Grok as PNG instead of being}

debug info

CI timings

CI timings · View report · View job

Wall time 6m49s vs 45m13s (-84.9%). 8 job(s) slower, 7 faster, 1 unchanged.

• OS-specific tests / macOS-only tests (1/2): -88.0s
• Python tests / Run tests (2/2): -64.0s
• OS-specific tests / Windows-only tests (arm64): -60.0s
• OS-specific tests / Windows-only tests: -54.0s
• Check no committed infographics / check-no-committed-infographics: +34.0s

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with no blocking behavior or structural issues found.

Summary

xAI Responses requests now send rejected inline GIFs as PNG instead of sending a format Grok rejects.

• Grok turns receive attached GIFs as PNG images.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[User image or tool-result image] --> B[Responses image conversion]
  B --> C{Backend rejects inline format?}
  C -->|No| D[Send original image]
  C -->|Yes| E[Convert to PNG]
  E --> F{Conversion succeeds?}
  F -->|Yes| G[Send PNG]
  F -->|No| H[Send text placeholder]

Loading

_{Reviews (1) · Last reviewed commit: "fix(xai): GIF attachments reach Grok as ..." · Reviewed by Greptile}