Skip to content

feat(sse): Codex CLI image_generation + DALL-E-style image route - #1544

Merged
diegosouzapw merged 6 commits into
cheaper-inference:release/v3.7.0from
payne0420:feat/codex-image-generation
Apr 24, 2026
Merged

diegosouzapw merged 6 commits into
cheaper-inference:release/v3.7.0from
payne0420:feat/codex-image-generation

Conversation

@payne0420

@payne0420 payne0420 commented Apr 23, 2026 •

Copy link
Copy Markdown
Contributor

Adds end-to-end Codex image generation support — both the chat-flow hosted tool (what Codex CLI uses) and the legacy /v1/images/generations endpoint (what OpenWebUI and similar clients use).

Problem

Codex CLI ships with the default-enabled feature flag:

[features]
image_generation = true

When enabled, Codex CLI injects an OpenAI Responses-API hosted tool into every POST /v1/responses request:

{ "type": "image_generation", "output_format": "png" }

OpenAI's Responses API then executes gpt-image-1 server-side and streams back response.output_item.done events where item.type === "image_generation_call" carrying the base64 PNG in item.result. Codex CLI saves that PNG to $CODEX_HOME/generated_images/<session>/<call_id>.png and renders a TUI artifact panel.

Through OmniRoute this feature silently no-oped. Two distinct breakages:

  1. /v1/responses path — normalizeCodexTools() in open-sse/executors/codex.ts dropped every tool whose type !== "function". The hosted image_generation tool was stripped before the request reached OpenAI, so upstream never produced an image_generation_call output item. The same break affected other hosted tools (web_search, file_search, …).
  2. /v1/images/generations path — OmniRoute's image handler had no Codex branch, so a DALL-E-style request with model: codex/gpt-5.4 fell through to a generic OpenAI-compatible code path that built a bogus URL (chatgpt.com/backend-api/codex/responses/images/generations), which Cloudflare 403'd.
  3. store: true default — once the hosted tool was preserved, the Codex backend rejected requests with "Store must be set to false" because OmniRoute's executor defaults store to true for the OpenClaw chaining scenario.

Fix

Two commits on this branch:

fix(sse): preserve Responses API hosted tools in Codex executor

  • Add a whitelist of Responses-API hosted tool types (image_generation, web_search, web_search_preview, file_search, computer, computer_use_preview, code_interpreter, mcp, local_shell).
  • Teach normalizeCodexTools to preserve hosted tools (shape: type set, no .function, no .name, type in whitelist) alongside the existing function-tool handling. Unknown non-function types are still dropped locally with a console.debug.
  • tool_choice handling is unchanged — it only inspects type === "function", so hosted tool_choice values pass through.

feat(sse): route DALL-E-style image generation through Codex hosted tool

  • Register codex (alias cx) in the image provider registry with format: "codex-responses".
  • Add handleCodexImageGeneration to the image handler: builds a /v1/responses body with tools: [{ type: "image_generation", output_format: "png" }], POSTs to the Codex endpoint with proper ChatGPT-OAuth headers (chatgpt-account-id, session_id, User-Agent), reads the SSE stream, extracts image_generation_call items, and returns OpenAI image response shape ({ data: [{ b64_json }] } or { data: [{ url: "data:image/png;base64,..." }] } depending on response_format).
  • Export a shared extractImageGenerationCalls() helper for testability.
  • Default store: false in the executor when a request contains an image_generation tool — avoids the "Store must be set to false" upstream 400 for the chat-flow path. Explicit client store values are respected.

Scope

  • Only open-sse/executors/codex.ts, open-sse/config/imageRegistry.ts, and open-sse/handlers/imageGeneration.ts are modified on the production side. Other executors and providers are untouched.
  • No new settings, feature flags, or DB columns.
  • n > 1 for the DALL-E-style endpoint is handled sequentially (the hosted tool generates one image per call); clients that need parallel generation should use multiple requests.

Test plan

  • node --import tsx/esm --test tests/unit/executor-codex.test.ts — 19/19 pass (7 new cases covering hosted-tool preservation, tool_choice regression, and the image-gen store default).
  • node --import tsx/esm --test tests/unit/image-generation-handler.test.ts — 36/36 pass (5 new cases covering extractImageGenerationCalls, /responses round-trip, b64_json vs url response formats, upstream error propagation, and the no-image-emitted error path).
  • node --import tsx/esm --test tests/unit/tools-filter-anthropic-format.test.ts tests/unit/codex-stream-false.test.ts tests/unit/t19-codex-responses-empty-content.test.ts — 17/17 pass, no adjacent regressions.
  • npm run typecheck:core — clean.
  • ESLint on changed files — clean.
  • Manual end-to-end smoke test (chat flow): POST /v1/responses with tools: [{ "type": "image_generation", "output_format": "png" }] and "store": false against a Codex ChatGPT-OAuth connection — SSE stream contains response.output_item.done with item.type === "image_generation_call" and a non-empty base64 item.result. Verified in user testing.
  • Manual end-to-end smoke test (DALL-E shim): POST /v1/images/generations with { "model": "codex/gpt-5.4", "prompt": "...", "response_format": "b64_json" } against the same connection — returns { data: [{ b64_json: "..." }] }.
  • tests/manual/image-generation.http has both request blocks appended.

normalizeCodexTools was dropping every non-function tool, which stripped
Codex CLI's built-in image_generation tool (and other hosted tools like
web_search / file_search) before they reached OpenAI. Add a whitelist
and structural check so they pass through, while keeping unknown types
filtered locally with a debug log.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@payne0420
payne0420 requested a review from diegosouzapw as a code owner April 23, 2026 19:55

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for OpenAI-hosted tool types, such as image_generation, web_search, and file_search, within the Codex executor. It adds a set of recognized hosted tool types and updates the normalizeCodexTools function to correctly identify and preserve these tools while filtering out malformed or unknown ones. Comprehensive unit tests and a manual test case have been included to verify the transformation logic and ensure proper handling of tool choices. I have no feedback to provide.

Adds a Codex branch to the /v1/images/generations handler so DALL-E-style
requests with `model: codex/*` (or `cx/*`) are translated into /responses
calls with the `image_generation` hosted tool, then unpacked back into
OpenAI image response shape. Enables OpenWebUI and other clients that hit
the legacy images endpoint to drive Codex image generation.

Also defaults `store: false` in the Codex executor whenever an
`image_generation` tool is present — the Codex backend rejects store=true
with hosted image generation ("Store must be set to false").

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@payne0420 payne0420 changed the title fix(sse): preserve Responses API hosted tools in Codex executor feat(sse): Codex CLI image_generation + DALL-E-style image route Apr 23, 2026
@payne0420

Copy link
Copy Markdown
Contributor Author

This fixes stuff like Open WebUI image generation:
image

…ation tool

The Codex image-gen shim was ignoring `size` and `quality` from the incoming
/v1/images/generations body, so OpenWebUI's size and quality selectors had
no effect. Forward both into the hosted tool config, mapping DALL-E's
`standard`/`hd` to the image_generation tool's `medium`/`high` so legacy
clients keep working.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@diegosouzapw
diegosouzapw changed the base branch from main to release/v3.7.0 April 24, 2026 11:11
Register Nous Research as an OpenAI-compatible gateway with remote
model discovery and validation against chat completions.

Add Petals provider metadata, default config, validation, and a
specialized executor that maps OpenAI-style requests to the public
generate endpoint. Also allow optional API keys and configurable base
URLs for Petals in the dashboard and provider schemas.

Expand provider model and catalog tests to cover both integrations.
Await runtime request queue updates so limiter settings and auto-enabled
API key protections are recomputed when resilience settings change.

Preserve cancelled batch state for in-flight work by marking input files
processed without generating output artifacts, and replace cached synced
models with an empty set when remote discovery returns no models so the
providers route falls back to the local catalog instead of stale cache.
@diegosouzapw
diegosouzapw merged commit 35c29fa into cheaper-inference:release/v3.7.0 Apr 24, 2026
1 check passed
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Awesome feature! 🚀 This has been successfully merged into release/v3.7.0. Supporting Codex image generation and DALL-E shimming is a huge plus. Thanks for the contribution!

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3449aa1daf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

});
}

const requestedCount = Number.isInteger(body.n) && (body.n as number) > 0 ? (body.n as number) : 1;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Clamp Codex image n before sequential upstream loop

This handler uses body.n directly to control a per-image for loop, issuing one upstream /responses request per iteration. Because the request schema does not enforce an upper bound for n, a client can send a very large value and trigger excessive sequential upstream calls in a single API request, causing runaway latency/cost and tying up server capacity. Add a strict max (or reject oversized n) before entering the loop.

Useful? React with 👍 / 👎.

body.store = false;
} else if (body.store === undefined) {
body.store = true;
body.store = hasImageGenerationTool(body) ? false : true;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Compute store default after Codex tool normalization

store is defaulted from hasImageGenerationTool(body) before normalizeCodexTools(body) removes invalid hosted tools. If an image_generation entry is later filtered out (for example because it also has a name or function field), store remains forced to false even though no image tool is sent upstream, which can unexpectedly disable response persistence and break previous_response_id chaining for the remaining valid tools. Derive this default from the normalized tool list instead.

Useful? React with 👍 / 👎.

@diegosouzapw diegosouzapw mentioned this pull request Apr 25, 2026
Poid-ZA pushed a commit to Poid-ZA/OmniRoute that referenced this pull request Aug 5, 2026
…aper-inference#1544)

* fix(sse): preserve Responses API hosted tools in Codex executor

normalizeCodexTools was dropping every non-function tool, which stripped
Codex CLI's built-in image_generation tool (and other hosted tools like
web_search / file_search) before they reached OpenAI. Add a whitelist
and structural check so they pass through, while keeping unknown types
filtered locally with a debug log.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(sse): route DALL-E-style image generation through Codex hosted tool

Adds a Codex branch to the /v1/images/generations handler so DALL-E-style
requests with `model: codex/*` (or `cx/*`) are translated into /responses
calls with the `image_generation` hosted tool, then unpacked back into
OpenAI image response shape. Enables OpenWebUI and other clients that hit
the legacy images endpoint to drive Codex image generation.

Also defaults `store: false` in the Codex executor whenever an
`image_generation` tool is present — the Codex backend rejects store=true
with hosted image generation ("Store must be set to false").

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(sse): forward size/quality from DALL-E body to Codex image_generation tool

The Codex image-gen shim was ignoring `size` and `quality` from the incoming
/v1/images/generations body, so OpenWebUI's size and quality selectors had
no effect. Forward both into the hosted tool config, mapping DALL-E's
`standard`/`hd` to the image_generation tool's `medium`/`high` so legacy
clients keep working.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(providers): add Petals and Nous Research provider support

Register Nous Research as an OpenAI-compatible gateway with remote
model discovery and validation against chat completions.

Add Petals provider metadata, default config, validation, and a
specialized executor that maps OpenAI-style requests to the public
generate endpoint. Also allow optional API keys and configurable base
URLs for Petals in the dashboard and provider schemas.

Expand provider model and catalog tests to cover both integrations.

* fix(resilience): sync queue updates and clear stale discovery caches

Await runtime request queue updates so limiter settings and auto-enabled
API key protections are recomputed when resilience settings change.

Preserve cancelled batch state for in-flight work by marking input files
processed without generating output artifacts, and replace cached synced
models with an empty set when remote discovery returns no models so the
providers route falls back to the local catalog instead of stale cache.

---------

Co-authored-by: Payne <trader-payne@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…aper-inference#1544)

* fix(sse): preserve Responses API hosted tools in Codex executor

normalizeCodexTools was dropping every non-function tool, which stripped
Codex CLI's built-in image_generation tool (and other hosted tools like
web_search / file_search) before they reached OpenAI. Add a whitelist
and structural check so they pass through, while keeping unknown types
filtered locally with a debug log.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(sse): route DALL-E-style image generation through Codex hosted tool

Adds a Codex branch to the /v1/images/generations handler so DALL-E-style
requests with `model: codex/*` (or `cx/*`) are translated into /responses
calls with the `image_generation` hosted tool, then unpacked back into
OpenAI image response shape. Enables OpenWebUI and other clients that hit
the legacy images endpoint to drive Codex image generation.

Also defaults `store: false` in the Codex executor whenever an
`image_generation` tool is present — the Codex backend rejects store=true
with hosted image generation ("Store must be set to false").

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(sse): forward size/quality from DALL-E body to Codex image_generation tool

The Codex image-gen shim was ignoring `size` and `quality` from the incoming
/v1/images/generations body, so OpenWebUI's size and quality selectors had
no effect. Forward both into the hosted tool config, mapping DALL-E's
`standard`/`hd` to the image_generation tool's `medium`/`high` so legacy
clients keep working.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(providers): add Petals and Nous Research provider support

Register Nous Research as an OpenAI-compatible gateway with remote
model discovery and validation against chat completions.

Add Petals provider metadata, default config, validation, and a
specialized executor that maps OpenAI-style requests to the public
generate endpoint. Also allow optional API keys and configurable base
URLs for Petals in the dashboard and provider schemas.

Expand provider model and catalog tests to cover both integrations.

* fix(resilience): sync queue updates and clear stale discovery caches

Await runtime request queue updates so limiter settings and auto-enabled
API key protections are recomputed when resilience settings change.

Preserve cancelled batch state for in-flight work by marking input files
processed without generating output artifacts, and replace cached synced
models with an empty set when remote discovery returns no models so the
providers route falls back to the local catalog instead of stale cache.

---------

Co-authored-by: Payne <trader-payne@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants