Repository navigation
feat(sse): Codex CLI image_generation + DALL-E-style image route - #1544
diegosouzapw merged 6 commits into
Conversation
normalizeCodexTools was dropping every non-function tool, which stripped Codex CLI's built-in image_generation tool (and other hosted tools like web_search / file_search) before they reached OpenAI. Add a whitelist and structural check so they pass through, while keeping unknown types filtered locally with a debug log. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces support for OpenAI-hosted tool types, such as image_generation, web_search, and file_search, within the Codex executor. It adds a set of recognized hosted tool types and updates the normalizeCodexTools function to correctly identify and preserve these tools while filtering out malformed or unknown ones. Comprehensive unit tests and a manual test case have been included to verify the transformation logic and ensure proper handling of tool choices. I have no feedback to provide.
Adds a Codex branch to the /v1/images/generations handler so DALL-E-style
requests with `model: codex/*` (or `cx/*`) are translated into /responses
calls with the `image_generation` hosted tool, then unpacked back into
OpenAI image response shape. Enables OpenWebUI and other clients that hit
the legacy images endpoint to drive Codex image generation.
Also defaults `store: false` in the Codex executor whenever an
`image_generation` tool is present — the Codex backend rejects store=true
with hosted image generation ("Store must be set to false").
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ation tool The Codex image-gen shim was ignoring `size` and `quality` from the incoming /v1/images/generations body, so OpenWebUI's size and quality selectors had no effect. Forward both into the hosted tool config, mapping DALL-E's `standard`/`hd` to the image_generation tool's `medium`/`high` so legacy clients keep working. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Register Nous Research as an OpenAI-compatible gateway with remote model discovery and validation against chat completions. Add Petals provider metadata, default config, validation, and a specialized executor that maps OpenAI-style requests to the public generate endpoint. Also allow optional API keys and configurable base URLs for Petals in the dashboard and provider schemas. Expand provider model and catalog tests to cover both integrations.
Await runtime request queue updates so limiter settings and auto-enabled API key protections are recomputed when resilience settings change. Preserve cancelled batch state for in-flight work by marking input files processed without generating output artifacts, and replace cached synced models with an empty set when remote discovery returns no models so the providers route falls back to the local catalog instead of stale cache.
35c29fa
into
cheaper-inference:release/v3.7.0
|
Awesome feature! 🚀 This has been successfully merged into |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3449aa1daf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }); | ||
| } | ||
|
|
||
| const requestedCount = Number.isInteger(body.n) && (body.n as number) > 0 ? (body.n as number) : 1; |
There was a problem hiding this comment.
Clamp Codex image n before sequential upstream loop
This handler uses body.n directly to control a per-image for loop, issuing one upstream /responses request per iteration. Because the request schema does not enforce an upper bound for n, a client can send a very large value and trigger excessive sequential upstream calls in a single API request, causing runaway latency/cost and tying up server capacity. Add a strict max (or reject oversized n) before entering the loop.
Useful? React with 👍 / 👎.
| body.store = false; | ||
| } else if (body.store === undefined) { | ||
| body.store = true; | ||
| body.store = hasImageGenerationTool(body) ? false : true; |
There was a problem hiding this comment.
Compute store default after Codex tool normalization
store is defaulted from hasImageGenerationTool(body) before normalizeCodexTools(body) removes invalid hosted tools. If an image_generation entry is later filtered out (for example because it also has a name or function field), store remains forced to false even though no image tool is sent upstream, which can unexpectedly disable response persistence and break previous_response_id chaining for the remaining valid tools. Derive this default from the normalized tool list instead.
Useful? React with 👍 / 👎.
…aper-inference#1544) * fix(sse): preserve Responses API hosted tools in Codex executor normalizeCodexTools was dropping every non-function tool, which stripped Codex CLI's built-in image_generation tool (and other hosted tools like web_search / file_search) before they reached OpenAI. Add a whitelist and structural check so they pass through, while keeping unknown types filtered locally with a debug log. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sse): route DALL-E-style image generation through Codex hosted tool Adds a Codex branch to the /v1/images/generations handler so DALL-E-style requests with `model: codex/*` (or `cx/*`) are translated into /responses calls with the `image_generation` hosted tool, then unpacked back into OpenAI image response shape. Enables OpenWebUI and other clients that hit the legacy images endpoint to drive Codex image generation. Also defaults `store: false` in the Codex executor whenever an `image_generation` tool is present — the Codex backend rejects store=true with hosted image generation ("Store must be set to false"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sse): forward size/quality from DALL-E body to Codex image_generation tool The Codex image-gen shim was ignoring `size` and `quality` from the incoming /v1/images/generations body, so OpenWebUI's size and quality selectors had no effect. Forward both into the hosted tool config, mapping DALL-E's `standard`/`hd` to the image_generation tool's `medium`/`high` so legacy clients keep working. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(providers): add Petals and Nous Research provider support Register Nous Research as an OpenAI-compatible gateway with remote model discovery and validation against chat completions. Add Petals provider metadata, default config, validation, and a specialized executor that maps OpenAI-style requests to the public generate endpoint. Also allow optional API keys and configurable base URLs for Petals in the dashboard and provider schemas. Expand provider model and catalog tests to cover both integrations. * fix(resilience): sync queue updates and clear stale discovery caches Await runtime request queue updates so limiter settings and auto-enabled API key protections are recomputed when resilience settings change. Preserve cancelled batch state for in-flight work by marking input files processed without generating output artifacts, and replace cached synced models with an empty set when remote discovery returns no models so the providers route falls back to the local catalog instead of stale cache. --------- Co-authored-by: Payne <trader-payne@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
…aper-inference#1544) * fix(sse): preserve Responses API hosted tools in Codex executor normalizeCodexTools was dropping every non-function tool, which stripped Codex CLI's built-in image_generation tool (and other hosted tools like web_search / file_search) before they reached OpenAI. Add a whitelist and structural check so they pass through, while keeping unknown types filtered locally with a debug log. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sse): route DALL-E-style image generation through Codex hosted tool Adds a Codex branch to the /v1/images/generations handler so DALL-E-style requests with `model: codex/*` (or `cx/*`) are translated into /responses calls with the `image_generation` hosted tool, then unpacked back into OpenAI image response shape. Enables OpenWebUI and other clients that hit the legacy images endpoint to drive Codex image generation. Also defaults `store: false` in the Codex executor whenever an `image_generation` tool is present — the Codex backend rejects store=true with hosted image generation ("Store must be set to false"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(sse): forward size/quality from DALL-E body to Codex image_generation tool The Codex image-gen shim was ignoring `size` and `quality` from the incoming /v1/images/generations body, so OpenWebUI's size and quality selectors had no effect. Forward both into the hosted tool config, mapping DALL-E's `standard`/`hd` to the image_generation tool's `medium`/`high` so legacy clients keep working. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(providers): add Petals and Nous Research provider support Register Nous Research as an OpenAI-compatible gateway with remote model discovery and validation against chat completions. Add Petals provider metadata, default config, validation, and a specialized executor that maps OpenAI-style requests to the public generate endpoint. Also allow optional API keys and configurable base URLs for Petals in the dashboard and provider schemas. Expand provider model and catalog tests to cover both integrations. * fix(resilience): sync queue updates and clear stale discovery caches Await runtime request queue updates so limiter settings and auto-enabled API key protections are recomputed when resilience settings change. Preserve cancelled batch state for in-flight work by marking input files processed without generating output artifacts, and replace cached synced models with an empty set when remote discovery returns no models so the providers route falls back to the local catalog instead of stale cache. --------- Co-authored-by: Payne <trader-payne@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>

Adds end-to-end Codex image generation support — both the chat-flow hosted tool (what Codex CLI uses) and the legacy
/v1/images/generationsendpoint (what OpenWebUI and similar clients use).Problem
Codex CLI ships with the default-enabled feature flag:
When enabled, Codex CLI injects an OpenAI Responses-API hosted tool into every
POST /v1/responsesrequest:{ "type": "image_generation", "output_format": "png" }OpenAI's Responses API then executes
gpt-image-1server-side and streams backresponse.output_item.doneevents whereitem.type === "image_generation_call"carrying the base64 PNG initem.result. Codex CLI saves that PNG to$CODEX_HOME/generated_images/<session>/<call_id>.pngand renders a TUI artifact panel.Through OmniRoute this feature silently no-oped. Two distinct breakages:
/v1/responsespath —normalizeCodexTools()inopen-sse/executors/codex.tsdropped every tool whosetype !== "function". The hostedimage_generationtool was stripped before the request reached OpenAI, so upstream never produced animage_generation_calloutput item. The same break affected other hosted tools (web_search,file_search, …)./v1/images/generationspath — OmniRoute's image handler had no Codex branch, so a DALL-E-style request withmodel: codex/gpt-5.4fell through to a generic OpenAI-compatible code path that built a bogus URL (chatgpt.com/backend-api/codex/responses/images/generations), which Cloudflare 403'd.store: truedefault — once the hosted tool was preserved, the Codex backend rejected requests with"Store must be set to false"because OmniRoute's executor defaultsstoretotruefor the OpenClaw chaining scenario.Fix
Two commits on this branch:
fix(sse): preserve Responses API hosted tools in Codex executorimage_generation,web_search,web_search_preview,file_search,computer,computer_use_preview,code_interpreter,mcp,local_shell).normalizeCodexToolsto preserve hosted tools (shape:typeset, no.function, no.name, type in whitelist) alongside the existing function-tool handling. Unknown non-function types are still dropped locally with aconsole.debug.tool_choicehandling is unchanged — it only inspectstype === "function", so hosted tool_choice values pass through.feat(sse): route DALL-E-style image generation through Codex hosted toolcodex(aliascx) in the image provider registry withformat: "codex-responses".handleCodexImageGenerationto the image handler: builds a/v1/responsesbody withtools: [{ type: "image_generation", output_format: "png" }], POSTs to the Codex endpoint with proper ChatGPT-OAuth headers (chatgpt-account-id,session_id,User-Agent), reads the SSE stream, extractsimage_generation_callitems, and returns OpenAI image response shape ({ data: [{ b64_json }] }or{ data: [{ url: "data:image/png;base64,..." }] }depending onresponse_format).extractImageGenerationCalls()helper for testability.store: falsein the executor when a request contains animage_generationtool — avoids the"Store must be set to false"upstream 400 for the chat-flow path. Explicit clientstorevalues are respected.Scope
open-sse/executors/codex.ts,open-sse/config/imageRegistry.ts, andopen-sse/handlers/imageGeneration.tsare modified on the production side. Other executors and providers are untouched.n > 1for the DALL-E-style endpoint is handled sequentially (the hosted tool generates one image per call); clients that need parallel generation should use multiple requests.Test plan
node --import tsx/esm --test tests/unit/executor-codex.test.ts— 19/19 pass (7 new cases covering hosted-tool preservation,tool_choiceregression, and the image-gen store default).node --import tsx/esm --test tests/unit/image-generation-handler.test.ts— 36/36 pass (5 new cases coveringextractImageGenerationCalls,/responsesround-trip,b64_jsonvsurlresponse formats, upstream error propagation, and the no-image-emitted error path).node --import tsx/esm --test tests/unit/tools-filter-anthropic-format.test.ts tests/unit/codex-stream-false.test.ts tests/unit/t19-codex-responses-empty-content.test.ts— 17/17 pass, no adjacent regressions.npm run typecheck:core— clean.POST /v1/responseswithtools: [{ "type": "image_generation", "output_format": "png" }]and"store": falseagainst a Codex ChatGPT-OAuth connection — SSE stream containsresponse.output_item.donewithitem.type === "image_generation_call"and a non-empty base64item.result. Verified in user testing.POST /v1/images/generationswith{ "model": "codex/gpt-5.4", "prompt": "...", "response_format": "b64_json" }against the same connection — returns{ data: [{ b64_json: "..." }] }.tests/manual/image-generation.httphas both request blocks appended.