feat(image_gen): enable image-to-image on the Codex backend - #51838
Closed
iainlane wants to merge 1 commit into
Closed
feat(image_gen): enable image-to-image on the Codex backend#51838iainlane wants to merge 1 commit into
iainlane wants to merge 1 commit into
Conversation
19 tasks
The ChatGPT/Codex OAuth image-gen backend advertised text-to-image only and rejected any source image with a "not capable of image-to-image" error, even though the ChatGPT app itself edits uploaded images happily. The limitation was an implementation gap, not an API one. The backend drives gpt-image-2 through the Responses image_generation tool, and that tool conditions on input_image content parts — the same mechanism the rest of the codebase already uses to send images to /backend-api/codex/responses (see agent/codex_responses_adapter.py). The plugin simply never attached the source image to the request. Attach each source as an input_image part alongside the prompt text. A source may be a public URL (passed through), a data: URI, or a local file path — what a user-attached image looks like once the gateway has saved it to disk and surfaced it to the model. Local files and data: URIs are validated as real images by magic-byte sniffing rather than trusting the extension or the data: label, and local files additionally pass the read denylist so a credential store can't be shipped to the image API just because its bytes happen to look like an image. capabilities() now reports image support, so the dynamic tool schema offers image_url when this backend is active.
iainlane
force-pushed
the
feat/codex-image-to-image
branch
from
June 24, 2026 11:18
baa83e5 to
095aff2
Compare
iainlane
marked this pull request as ready for review
June 24, 2026 11:49
Collaborator
|
Closing as a duplicate — Codex OAuth image-to-image / reference-image input is now on That PR wires source/reference images through the Codex Responses If you spot a gap in the merged implementation, please open a follow-up. 🙏 |
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Enable image-to-image / editing on the ChatGPT/Codex OAuth image-gen backend (
openai-codex). Until now it advertised text-to-image only and hard-rejected any source image with a "not capable of image-to-image" error — even though the ChatGPT app itself edits uploaded images.Why it was rejected, and why that was wrong
The limitation was an implementation gap, not an API one. The backend drives
gpt-image-2through the Responsesimage_generationtool, and that tool conditions oninput_imagecontent parts. The codebase already sends images to/backend-api/codex/responsesthis way for the main agent's vision (agent/codex_responses_adapter.py). The plugin simply never attached the source image.What changed
input_imagecontent part alongside the prompt text in the Responses payload.data:URI, or a local file path — what a user-attached image looks like once the gateway has saved it to disk. Local files anddata:URIs are validated as real images by magic-byte sniffing (not the extension ordata:label), and local files additionally pass the read denylist so a credential file can't be shipped to the image API just because its bytes look like an image.capabilities()now reports["text", "image"], so the dynamic tool schema offersimage_urlwhen this backend is active.This commit deliberately uses a small in-plugin resolver built on the
get_read_block_error+_sniff_mime_from_bytesprimitives already onmain, so it has no new dependencies and merges independently.Verification
input_image, remote passthrough, text-only has noinput_image), the denylist, and non-image rejection.