Skip to content

feat(image-gen): support Codex image inputs (salvage #55828) - #824

Merged
hashbender merged 1 commit into
mainfrom
mirror/pr-57017
Jul 2, 2026
Merged

feat(image-gen): support Codex image inputs (salvage #55828)#824
hashbender merged 1 commit into
mainfrom
mirror/pr-57017

Conversation

@hashbender

Copy link
Copy Markdown
Owner

Summary

The openai-codex (Codex OAuth) image-gen backend can now do image-to-image / reference-image editing — it previously hard-rejected any image_url/reference_image_urls and advertised text-only.

Salvage of @CrazyBoyM's NousResearch#55828 onto current main, with one follow-up commit that reuses the shared magic-byte sniffer and gates it to the raster formats gpt-image-2 actually accepts.

Changes

  • plugins/image_gen/openai-codex/__init__.py:
    • Accept image_url + reference_image_urls; attach them as Responses input_image content parts alongside the prompt.
    • Normalize refs via the shared normalize_reference_images(...), clamp to 16 (same cap as the API-key OpenAI path).
    • Validate local/data-URL inputs by magic bytes (not extensions), 25MB cap, and route local paths through the agent.file_safety read guard before inlining.
    • Advertise modalities: ["text","image"] / max_reference_images: 16 only now that the path is wired.
    • Follow-up (ours): delegate magic-byte detection to the shared agent.image_routing._sniff_mime_from_bytes (drops a duplicated format table) and gate the result to png/jpeg/gif/webp — the raster set gpt-image-2's input_image accepts — so SVG/TIFF/ICO fail locally with a clear invalid_image_input error instead of an opaque server-side 400.
  • tests/plugins/image_gen/test_openai_codex_provider.py: payload shape with source+reference images, capability metadata, non-image + SVG rejection, 16-image clamp.
  • Docs: image-generation + tools-reference updated for Codex OAuth image editing.

Validation

Result
tests/plugins/image_gen/test_openai_codex_provider.py + dispatch 26 passed
Full tests/plugins/image_gen/ 172 passed
ruff (diff vs main) clean
E2E (real file I/O, temp HERMES_HOME) local PNG inlined + saved; .txt/nonexistent/fake-magic-byte/SVG/TIFF rejected invalid_image_input; refs clamp 20→16; text-to-image regression clean; .env/auth.json blocked by read guard (no credential exfil)

Fixes NousResearch#21661.
Fixes NousResearch#36737.

Closes NousResearch#55828. Commits cherry-picked from @CrazyBoyM with authorship preserved.


Mirror-of: NousResearch#57017
NousResearch#57017

@hashbender
hashbender merged commit e25848f into main Jul 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant