[codex] Add native audio input support

[Schwartz10]

Summary

Adds the smallest native-audio path through Hermes agent/API code:

• accept and validate OpenAI-style input_audio / audio content parts in the API server
• pass normalized audio through Codex Responses input conversion
• summarize audio parts without persisting raw base64 in logs/session serialization

This intentionally leaves CLI, Telegram/gateway voice behavior, desktop microphone UX, and STT fallback routing untouched.

Validation

• .venv/bin/python -m compileall agent/audio_routing.py agent/codex_responses_adapter.py gateway/platforms/api_server.py run_agent.py
• .venv/bin/python -m pytest tests/agent/test_audio_routing.py tests/run_agent/test_run_agent_multimodal_prologue.py -q
• .venv/bin/python -m pytest tests/gateway/test_api_server_multimodal.py tests/gateway/test_api_server.py -q -k 'audio or multimodal or capabilities_advertises_plugin_safe_contract or capabilities_requires_auth_when_key_configured or capabilities_advertises_session_key_header'

[Schwartz10]

@claude can you review this PR for any bugs, security issues, or areas for improvement? we dont want to over engineer here, but we want to make sure we're solid supporting transcription through the subscription model provider when the model provider supports it natively.

[claude]

Claude finished @Schwartz10's task in 6m 29s —— View job

---

Claude finished reviewing this PR

Todo list:

• Gather context (read AGENTS.md, diff, changed files)
• Review agent/audio_routing.py
• Review agent/codex_responses_adapter.py changes
• Review gateway/platforms/api_server.py changes
• Review run_agent.py changes
• Review test coverage
• Post final review feedback

Summary

This is a small, well-scoped PR — validation logic is centralized in agent/audio_routing.py and reused consistently across the API server and the Codex Responses adapter. No raw base64 audio gets logged (matches the PR's stated goal), there's no SSRF surface (audio must be inline base64/data: URLs, never a fetched remote URL), and the new tests cover the core normalization paths well.

One gap stands out given your stated goal of "solid support... when the model provider supports it natively" — see #1 below.

Findings

1. No model-capability gating for native audio input (the main thing worth fixing before merge)

Unlike vision, which has a real fallback gate — AIAgent._model_supports_vision() (run_agent.py:4554) is checked before sending images, and non-vision models get images converted to text via vision_analyze (run_agent.py:4670-4733, agent/image_routing.py) — audio has no equivalent check. ModelCapabilities.supports_audio_input() already exists in agent/models_dev.py:93-94 but has zero call sites anywhere in the codebase. The /v1/capabilities endpoint unconditionally advertises "audio_api": True (gateway/platforms/api_server.py:1288) regardless of which model/provider is actually configured for the session.

Practically: if a client sends input_audio while the active model doesn't natively accept audio, the part is normalized and forwarded straight through to the provider, which will presumably reject it with an opaque 400 deep in the request pipeline rather than a fast, clear error — and there's no fallback to the existing transcription/STT pipeline (tools/transcription_tools.py, tools/voice_mode.py) the way images fall back to a text description.

Given you don't want to over-engineer this: I wouldn't suggest building an STT fallback path here. But wiring the already-existing supports_audio_input() into a lightweight pre-check (mirroring the existing _model_supports_vision() pattern) so unsupported models get a clear, immediate error instead of a deep provider failure seems like the right-sized fix.

Fix this →

2. Overly broad exception swallowing in _chat_content_to_responses_parts

agent/codex_responses_adapter.py:128-134:

if ptype in {"input_audio", "audio"} and role != "assistant":
    try:
        from agent.audio_routing import normalize_input_audio_part
        converted.append(normalize_input_audio_part(part, validate_data=False))
    except Exception:
        continue

This catches any exception — including bugs like TypeError/AttributeError from a future refactor, not just expected ValueError validation failures — and silently drops the entire audio turn with no log/warning. Contrast with _preflight_codex_input_items (same file, lines 789-796), which raises a clear ValueError for the same kind of malformed audio. Since this function converts already-stored conversation history on replay, a corrupted/edge-case audio part would just vanish from a user's turn with no trace. Suggest narrowing to except ValueError (or at least logging a warning before continue).

3. Minor: MAX_AUDIO_BYTES (25MB) is effectively unreachable through the API server

agent/audio_routing.py:10 sets MAX_AUDIO_BYTES = 25 * 1024 * 1024, but gateway/platforms/api_server.py:92 caps the entire HTTP request body at MAX_REQUEST_BYTES = 10_000_000, enforced via aiohttp's client_max_size (line ~4510). Since base64 inflates size ~33%, the largest audio payload that can ever reach normalize_input_audio_part via this endpoint is ~7.5MB — the audio_too_large / HTTP 413 branch in _multimodal_validation_error can never actually fire here. The capabilities endpoint itself already accounts for this correctly (min(MAX_AUDIO_BYTES, MAX_REQUEST_BYTES * 3 // 4) at line 1244), so this isn't a functional bug, just a misleading constant — worth a comment or deriving it from MAX_REQUEST_BYTES directly so the dead code path isn't confusing later.

4. Minor: inconsistent recognition of the "audio" type spelling

_summarize_user_message_for_log (agent/codex_responses_adapter.py:176) only matches ptype == "input_audio", while content_has_audio_parts() and the rest of the new code treat both "input_audio" and "audio" as audio (_AUDIO_PART_TYPES = frozenset({"input_audio", "audio"})). Harmless today because _normalize_multimodal_content always normalizes to "input_audio" before persisting, but if an unnormalized "audio"-typed part ever reaches this function directly, it silently disappears from the summary instead of showing up as [N audio].

5. Nit: video/webm is accepted as a valid audio MIME for data URLs (agent/audio_routing.py:27,81) — reasonable since browser voice recordings are sometimes labeled video/webm, but a one-line comment would help a future reader not mistake it for a typo.

Test coverage

Good coverage of the validation/normalization core (tests/agent/test_audio_routing.py, tests/gateway/test_api_server_multimodal.py, tests/run_agent/test_run_agent_multimodal_prologue.py). Gaps: no test exercises the audio_too_large/413 path, no test covers an assistant-role message carrying audio being rejected (agent/codex_responses_adapter.py:785-788), and (per #1) no test asserts behavior when the configured model lacks audio support.

Note: I wasn't able to execute the test suite in this sandboxed review environment (command execution required approval that wasn't available), so I relied on static review — the commands listed in the PR description look correct and should be run in CI.
· branch: codex/native-audio-input

[Schwartz10]

Pushed update 1095fee for the native audio honesty/safety work.

What changed:

• models.dev capabilities now track supports_audio_input from modalities.input.
• Added audio capability lookup following the vision override pattern, with unknown models staying unsupported.
• /v1/capabilities keeps features.audio_api=true but now reports audio.native_model from the active provider/model, plus fallback_transcription=false, max_bytes, formats, and unsupported_audio_input.
• Gateway preflight rejects unsupported input_audio before provider dispatch or session/run mutation across session chat, chat completions, responses, and runs.
• Tightened Codex Responses audio conversion/log summaries so invalid audio is handled narrowly and raw base64 audio is not logged.
• Added regression coverage for supported forwarding, unsupported 400s, malformed audio validation, assistant-role audio, and stored previous_response_id history.

Validation:

• .venv/bin/python -m pytest tests/agent/test_models_dev.py tests/agent/test_audio_routing.py tests/gateway/test_api_server.py tests/gateway/test_api_server_multimodal.py tests/gateway/test_session_api.py tests/run_agent/test_run_agent_multimodal_prologue.py -> 287 passed.
• python3 -m py_compile and git diff --check passed.

@claude please review this PR again, especially the native audio capability resolution and gateway preflight ordering.

[claude]

Claude finished @Schwartz10's task in 4m 45s —— View job

---

Claude finished re-reviewing this PR

Todo list:

• Gather context (diff of latest commit 1095fee against prior state, read updated files)
• Review models.dev audio capability resolution (agent/models_dev.py)
• Review /v1/capabilities audio reporting (gateway/platforms/api_server.py)
• Review gateway preflight ordering (session chat, chat completions, responses, runs)
• Review agent/codex_responses_adapter.py audio conversion/log summary changes
• Review new/updated tests
• Post final review feedback

Summary

This update lands the fix I flagged as the main gap last round, and does it well. ModelCapabilities.supports_audio_input (agent/models_dev.py:404,484-486) now has real call sites, and every audio-accepting endpoint (session chat, chat completions, responses, runs) rejects input_audio/audio parts with a fast, clear unsupported_audio_input 400 before any session/run state is mutated. All three previous findings (#2 broad exception swallowing, #3 misleading MAX_AUDIO_BYTES, #4 inconsistent "audio" spelling) are also addressed. Nice work.

Capability resolution (agent/audio_routing.py:124-158, agent/models_dev.py:481-486)

• lookup_supports_audio_input() mirrors image_routing._supports_vision_override() closely: config override (model.supports_audio_input → providers.<provider>.models.<model>.supports_audio_input) checked first, then falls through to models.dev via get_model_capabilities(), returning None for genuinely unknown models. get_model_capabilities() correctly derives supports_audio_input from modalities.input containing "audio" (case-normalized), defaulting False when modalities are absent — "unknown stays unsupported" holds up.
• One asymmetry vs. the vision path: _supports_vision_override() also checks the legacy list-style custom_providers config format (image_routing.py:226-249), but lookup_supports_audio_input() doesn't. Low-impact — anyone relying on that legacy format for a custom vision-capable provider won't get an equivalent audio override — but worth a one-line note or a follow-up if that config style is still supported elsewhere.
• _active_model_supports_audio_input() (gateway/platforms/api_server.py:1199-1214) fails closed (return False) on any exception from config/provider resolution, so a transient provider-resolution error blocks audio rather than accidentally letting it through — the right default here.

Preflight ordering (the other thing you asked me to look at)

Traced all four call sites end to end:

• Session chat (_handle_session_chat / _handle_session_chat_stream, lines 1766-1822): audio check on user_message and on loaded history both happen before _run_agent/_run_and_signal — no session mutation precedes rejection.
• Chat completions (_handle_chat_completions, lines 2023-2101): per-message role check inline while building conversation_messages, then a full-content capability check (line 2040) before X-Hermes-Session-Id continuation logic runs, and again after loading DB history for continued sessions (line 2099) — before _run_agent.
• Responses (_handle_responses, lines 3172-3243): role + capability checks on input, conversation_history, and previous_response_id-sourced history all resolve before session_id = stored_session_id or str(uuid.uuid4()) (line 3251) and before the response is persisted to the store.
• Runs (_handle_runs, lines 4137-4209): same pattern, resolved before run_id/session_id assignment and before _set_run_status/_run_streams are populated.

This is corroborated by the new tests in tests/gateway/test_api_server.py (TestRunsAudioPreflight), which explicitly assert adapter._run_streams == {} and adapter._run_statuses == {} after a rejected request — good regression coverage for exactly the ordering concern you asked about.

Minor observations (non-blocking)

1. _content_has_audio_parts_deep recurses on client-controlled JSON with no depth guard (gateway/platforms/api_server.py:358-369, new in this commit). A deeply nested content structure could exhaust Python's recursion limit. In practice request.json() parsing will itself hit RecursionError on comparably deep JSON before this function is reached, so this doesn't introduce a materially new DoS surface — just flagging since it's a new recursive function over untrusted input, in case that assumption ever changes (e.g. if a custom JSON loader with a higher effective depth were introduced).
2. _active_model_supports_audio_input() re-resolves config/provider on every call — some request paths call it 2+ times (e.g. once for user_message, once for history). Each call does _load_gateway_config() (file read) + _resolve_runtime_agent_kwargs() (provider/credential resolution, which can itself attempt fallback-provider resolution on auth errors). Not a correctness issue, but worth caching per-request if this ever shows up as latency in practice.
3. Nice touch: MAX_REQUEST_BYTES now has a comment (line 90-91) explaining why MAX_AUDIO_BYTES is effectively capped lower via base64 inflation — resolves the "misleading constant" note from the previous review.

Test coverage

Good — new tests directly exercise the scenarios that matter most: capability advertised via /v1/capabilities (both supported and unknown-model cases), unsupported audio rejected with no run/session-state mutation, assistant-role audio rejected in both live messages and previous_response_id-restored history, and malformed audio failing validation before the capability check (correct precedence — a malformed payload shouldn't get a capability-related error code). I wasn't able to execute the test suite in this sandboxed review environment (test-running commands require approval that wasn't available here), so this is a static review — the 287 passed result you reported should be authoritative, and CI should confirm.
· branch: codex/native-audio-input

[Schwartz10]

@claude please review the latest commit (5a0786c1d).

I changed this PR from native-audio chat support to an STT utility endpoint only.

What changed:

• Added only POST /v1/audio/transcriptions with required multipart file and model.
• Codex mode resolves through resolve_codex_runtime_credentials() and does not fall back to API keys or the gateway chat fallback chain.
• Non-Codex mode forwards to the configured OpenAI-compatible runtime /audio/transcriptions, which is the WilliamOS custom proxy path.
• Chat/session/responses/runs reject raw input_audio and tell callers to transcribe first.
• Removed the branch-added native-audio routing/tests while preserving pre-existing upstream STT/voice code.

Why: William should transcribe audio first, then send transcript text into normal Hermes chat. That keeps Hermes chat text/image-based and keeps ChatGPT subscription credentials behind the Codex auth boundary.

Verification: focused pytest suite passed (101 passed), plus py_compile and git diff --check.

[claude]

Claude finished @Schwartz10's task in 5m 19s —— View job

---

Reviewing latest commit 5a0786c1d

Todo list:

• Gather context (diff of 5a0786c1d and full branch diff vs origin/williamos/main)
• Review new POST /v1/audio/transcriptions endpoint (gateway/platforms/api_server.py)
• Review Codex-mode credential resolution path
• Review non-Codex proxy forwarding path
• Review chat/session/responses/runs rejection of raw input_audio
• Review removed native-audio routing code for stray references
• Review new tests (tests/gateway/test_api_server_audio_transcriptions.py)
• Post final review feedback

Summary

This is a clean pivot from native-audio chat to an STT-only utility endpoint. The removal of agent/audio_routing.py and all its call sites is complete — no dangling imports/references remain (grep for audio_routing, normalize_input_audio_part, lookup_supports_audio_input, MAX_AUDIO_BYTES turns up nothing outside tests). /v1/audio/transcriptions requires auth, never touches _session_db/_run_agent (confirmed by test_transcription_succeeds_with_codex_auth), and validates format + size before resolving any runtime credentials — good fail-fast ordering. Chat/session/responses/runs now unconditionally reject input_audio/audio parts with a clear pointer to the new endpoint, including for audio persisted in previous_response_id history from before this pivot.

Findings

1. The "no fallback for Codex" guarantee only holds when the provider is explicitly configured — not on the (likely common) "auto" path

gateway/platforms/api_server.py:1239-1290 (_resolve_audio_transcription_runtime):

requested_provider = resolve_requested_provider()
...
if provider == "openai-codex":
    creds = resolve_codex_runtime_credentials()   # no fallback — AuthError propagates as 401/429
    ...
else:
    runtime_kwargs = resolve_runtime_provider(requested=requested_provider)  # general resolver

resolve_requested_provider() (hermes_cli/runtime_provider.py:480-494) only returns "openai-codex" when model.provider is explicitly set in config or HERMES_INFERENCE_PROVIDER is set. If neither is set, it returns "auto" — which takes the else branch straight into resolve_runtime_provider(requested="auto").

But resolve_runtime_provider itself can still land on Codex via resolve_provider()'s auto-detection (hermes_cli/auth.py:1529, priority NousResearch#6: "auth.json active_provider — last-resort fallback"), which is exactly the state for a ChatGPT-subscription-only user with no other provider configured. When it does, and resolve_codex_runtime_credentials() raises AuthError, resolve_runtime_provider catches it and silently falls through to other providers (hermes_cli/runtime_provider.py:1664-1669, "Auto-detected Codex provider but credentials failed; falling through to next provider.") instead of surfacing the auth error — the exact "gateway chat fallback chain" behavior the PR description says this endpoint should avoid ("does not fall back to API keys or the gateway chat fallback chain... keeps ChatGPT subscription credentials behind the Codex auth boundary").

Net effect: on a config where model.provider is unset/"auto" and Codex is the active OAuth login, a stale/expired Codex token wouldn't surface as a clear 401 codex_auth_missing — it could silently transcribe (or fail) against a completely different provider/base_url than the one backing the ChatGPT subscription, with the client-supplied model field (e.g. "gpt-4o-transcribe") now being sent to that unrelated provider. No test currently exercises the "auto"-provider-resolves-to-codex case (all tests explicitly patch resolve_requested_provider to return "openai-codex" or "custom").

Suggested fix: after resolve_runtime_provider() returns in the else branch, check if the resolved provider came back as "openai-codex" despite requested_provider != "openai-codex" and reject/re-resolve without the auto-fallback (or have _resolve_audio_transcription_runtime call resolve_provider(requested_provider) first to see what it would resolve to, and route into the strict Codex branch whenever that's "openai-codex", regardless of whether it came from explicit config or auto-detection).

[Fix this →](https://claude.ai/code?q=In%20gateway%2Fplatforms%2Fapi_server.py%3A1239-1290%20%28_resolve_audio_transcription_runtime%29%20on%20branch%20codex%2Fnative-audio-input%3A%20when%20resolve_requested_provider%28%29%20returns%20%22auto%22%2C%20the%20else%20branch%20calls%20resolve_runtime_provider%28requested%3D%22auto%22%29%2C%20which%20can%20still%20auto-detect%20openai-codex%20%28via%20resolve_provider%28%29%27s%20last-resort%20active_provider%20fallback%20in%20hermes_cli%2Fauth.py%3A1529%29%20but%20with%20its%20built-in%20behavior%20of%20silently%20falling%20through%20to%20other%20providers%20on%20AuthError%20%28hermes_cli%2Fruntime_provider.py%3A1664-1669%29.%20This%20defeats%20the%20PR%27s%20stated%20goal%20of%20never%20falling%20back%20off%20the%20Codex%20auth%20boundary.%20Fix%3A%20detect%20when%20auto-resolution%20would%20land%20on%20openai-codex%20%28e.g.%20call%20resolve_provider%28requested_provider%29%20first%29%20and%20route%20into%20the%20strict%20resolve_codex_runtime_credentials%28%29-only%20path%20in%20that%20case%20too%2C%20not%20just%20when%20the%20provider%20is%20explicitly%20configured.%20Add%20a%20regression%20test%20for%20the%20auto-resolves-to-codex-then-fails case.&repo=Schwartz10/hermes-agent)

2. Minor: global client_max_size bump weakens the request-body ceiling for non-audio endpoints when Content-Length is absent

gateway/platforms/api_server.py:4907 now sets client_max_size=AUDIO_TRANSCRIPTION_REQUEST_MAX_BYTES (~26 MB) on the whole web.Application, up from the previous MAX_REQUEST_BYTES (10 MB). The per-request 10 MB ceiling for non-audio routes is now enforced only by body_limit_middleware (gateway/platforms/api_server.py:677-691), which reads the Content-Length header — a request without that header (e.g. chunked Transfer-Encoding) skips the middleware check entirely and falls through to aiohttp's own hard cap, which is now ~2.6x larger for every endpoint, not just /v1/audio/transcriptions. Not exploitable for anything beyond extra memory/CPU (still bounded at ~26 MB), but it's a real, if small, regression in defense-in-depth for /v1/chat/completions and friends. Scoping the larger cap to just the audio route (e.g. via request.clone(client_max_size=...) inside _handle_audio_transcriptions, or a per-route client_max_size if aiohttp's routing supports it) would restore the original 10 MB hard ceiling for everything else.

3. Nit: dead audio_too_large branch

_multimodal_validation_error (gateway/platforms/api_server.py:395-405) still special-cases code == "audio_too_large" → HTTP 413, but _normalize_multimodal_content now always raises unsupported_audio_input for any audio part before any size check runs, so that branch is unreachable through this function. Harmless, but confusing for a future reader; worth deleting now that agent/audio_routing.py (the only place that used to raise audio_too_large) is gone.

Test coverage

Solid — tests/gateway/test_api_server_audio_transcriptions.py covers Codex success, custom-runtime success, missing/expired Codex auth (with an explicit assertion that the runtime-provider fallback is not called), missing runtime config, missing model/file, empty file, oversized file, unsupported format, and upstream failure propagation. The multimodal/session/runs suites were updated consistently to assert the new unsupported_audio_input rejection everywhere audio used to be accepted. The one gap is finding #1 above — no test drives the "auto"-provider-resolves-to-Codex path.

Note: command execution requires approval that isn't available in this sandboxed review environment, so I wasn't able to run pytest myself — this is a static review. The 101 passed result you reported should be authoritative; CI should confirm.
· branch: codex/native-audio-input

[Schwartz10]

@claude Final review request for follow-up commit 589a93695.

I addressed the three findings from your review:

1. Strict auto-Codex auth boundary for STT: /v1/audio/transcriptions now calls resolve_runtime_provider(..., allow_auto_codex_fallback=False) on non-explicit provider paths. The shared resolver keeps the default chat behavior, but when this flag is false an auto-detected Codex auth failure raises instead of falling through to another provider. Added API-level and resolver-level regression tests.
2. Request size scoping: the app-wide client_max_size is back to MAX_REQUEST_BYTES (10 MB). The transcription handler clones only that request with AUDIO_TRANSCRIPTION_REQUEST_MAX_BYTES before reading multipart data.
3. Dead branch cleanup: removed the stale audio_too_large special case from _multimodal_validation_error; chat/session multimodal validation now always returns 400 for its validation errors.

Self-review against your original comment:

• Finding 1: fixed and covered by test_auto_codex_auth_error_does_not_use_runtime_fallback plus test_auto_detected_codex_auth_failure_can_raise_in_strict_mode.
• Finding 2: fixed by restoring the global aiohttp limit and moving the larger limit to the audio handler clone.
• Finding 3: fixed by deleting the unreachable status branch.

Validation run:

• ./.venv/bin/python -m pytest tests/gateway/test_api_server_audio_transcriptions.py tests/gateway/test_api_server_multimodal.py tests/gateway/test_api_server.py::TestCapabilitiesEndpoint tests/gateway/test_api_server.py::TestRunsAudioPreflight tests/gateway/test_session_api.py::test_session_chat_rejects_unsupported_audio_before_run tests/gateway/test_session_api.py::test_session_chat_stream_rejects_unsupported_audio_before_sse tests/run_agent/test_run_agent_multimodal_prologue.py tests/agent/test_models_dev.py tests/hermes_cli/test_runtime_provider_resolution.py::test_auto_detected_codex_auth_failure_falls_through_to_openrouter tests/hermes_cli/test_runtime_provider_resolution.py::test_auto_detected_codex_auth_failure_can_raise_in_strict_mode -> 104 passed, 18 warnings
• ./.venv/bin/python -m py_compile gateway/platforms/api_server.py hermes_cli/runtime_provider.py tests/gateway/test_api_server_audio_transcriptions.py tests/hermes_cli/test_runtime_provider_resolution.py
• git diff --check

[claude]

Claude finished @Schwartz10's task in 2m 55s —— View job

---

Reviewing latest commit 589a93695

Todo list:

• Gather context (diff of 589a93695 against prior reviewed state)
• Review Finding 1 fix: strict auto-Codex auth boundary (allow_auto_codex_fallback flag)
• Review Finding 2 fix: request size scoping (client_max_size restore + per-route clone)
• Review Finding 3 fix: dead audio_too_large branch removal
• Review new/updated regression tests
• Post final review feedback

Summary

All three findings from the last round are correctly and cleanly fixed. This is a solid, well-scoped tightening commit — no new issues found.

Finding 1 — Strict auto-Codex auth boundary ✅

resolve_runtime_provider() (hermes_cli/runtime_provider.py:1434,1667-1673) gets a new allow_auto_codex_fallback: bool = True param, defaulting to the historical behavior for the normal chat path. The Codex-specific except AuthError: block now checks if requested_provider != "auto" or not allow_auto_codex_fallback: raise, so only the Codex fallback is tightened — Nous/xAI auto-fallback (lines 1649, 1687) are untouched, which is correct scope (the concern was specifically about leaking off the ChatGPT-subscription auth boundary).

_resolve_audio_transcription_runtime() (gateway/platforms/api_server.py:1287-1290) now calls resolve_runtime_provider(requested=requested_provider, allow_auto_codex_fallback=False) in the non-explicit-Codex branch, and the new except AuthError (lines 1291-1301) routes any error carrying provider == "openai-codex" or a codex_-prefixed code to the shared _codex_audio_auth_error_response() helper (401/429), falling back to a generic provider_config_error 503 for genuinely unrelated provider failures. This correctly closes the gap: an "auto" config that resolves to a stale/expired Codex session now surfaces as a clear Codex auth error instead of silently transcribing against (or failing against) an unrelated fallback provider.

Traced the scope check (AuthError import moved into the outer try, still valid in the else branch below since Python has no block scoping) — no issue there. New tests (test_auto_codex_auth_error_does_not_use_runtime_fallback in the API test file, test_auto_detected_codex_auth_failure_can_raise_in_strict_mode in the resolver test file) directly exercise this path and assert the resolver is called with allow_auto_codex_fallback=False and that AuthError propagates instead of falling through.

Finding 2 — Request size scoping ✅

web.Application (gateway/platforms/api_server.py:4929) is back to the app-wide client_max_size=MAX_REQUEST_BYTES (10 MB), and _handle_audio_transcriptions (line 1437) clones just that request with the larger client_max_size=AUDIO_TRANSCRIPTION_REQUEST_MAX_BYTES before calling request.multipart() — this is exactly the idiomatic aiohttp pattern for per-route body size overrides, called right after the multipart content-type check and before any parsing. body_limit_middleware (line 690-693) already special-cased this path by Content-Length, so both the header-based fast-path and the aiohttp-level ceiling are now consistently scoped to just this one route. Non-audio endpoints are back to their original 10 MB defense-in-depth ceiling.

Finding 3 — Dead branch cleanup ✅

_multimodal_validation_error (line 395-405) no longer special-cases audio_too_large → 413; it's a flat 400 now, matching the fact that chat/session/responses/runs always reject audio via unsupported_audio_input before any size check runs. Confirmed the real audio_too_large → 413 response for the transcription endpoint's own oversized-file check (line 1476, inside _handle_audio_transcriptions's multipart read loop) is untouched and independent — that one's still reachable and correctly returns 413. Grepped the test suite; no test asserts 413 through _multimodal_validation_error anymore, so nothing was left stale.

Test coverage

New tests directly cover the scenario each finding fixes: test_auto_codex_auth_error_does_not_use_runtime_fallback (API-level, asserts resolve_runtime_provider called with allow_auto_codex_fallback=False, 401 codex_auth_missing, no fallback POST) and test_auto_detected_codex_auth_failure_can_raise_in_strict_mode (resolver-level, asserts AuthError propagates in strict mode) both look correct and match the fix precisely.

Note: command execution requires approval that wasn't available in this sandboxed review environment, so I wasn't able to run pytest myself — this is a static review based on reading the diff and surrounding code. CI on this commit (589a936) is green (run 28617095114), which corroborates the reported 104 passed.

No blocking issues. This looks ready to merge from a correctness/security standpoint.

[Schwartz10]

@claude please do a final review of the new Codex transcription fix in commit f63da06.

What changed:

• openai-codex transcription now calls the ChatGPT transcribe endpoint derived from the Codex base URL, e.g. https://chatgpt.com/backend-api/transcribe, instead of appending /audio/transcriptions under /backend-api/codex.
• Codex transcription requests send multipart field file only, with no model field, and include Authorization, Accept, Origin, Referer, originator: codex_cli_rs, and ChatGPT-Account-ID when the JWT exposes it.
• Non-Codex providers still use the OpenAI-compatible /audio/transcriptions path and include the requested model.
• Codex error handling now sanitizes HTML/non-JSON failures and redacts asset_pointer-style values and bearer tokens from structured error messages.

Verification:

• ./.venv/bin/python -m py_compile gateway/platforms/api_server.py tests/gateway/test_api_server_audio_transcriptions.py
• git diff --check
• ./.venv/bin/python -m pytest tests/gateway/test_api_server_audio_transcriptions.py tests/gateway/test_api_server_multimodal.py tests/gateway/test_api_server.py::TestCapabilitiesEndpoint tests/gateway/test_api_server.py::TestRunsAudioPreflight tests/gateway/test_session_api.py::test_session_chat_rejects_unsupported_audio_before_run tests/gateway/test_session_api.py::test_session_chat_stream_rejects_unsupported_audio_before_sse

Focused self-review found one issue before push: structured Codex JSON errors could have echoed asset_pointer metadata. I fixed that and added coverage.

[claude]

Claude finished @Schwartz10's task in 2m 45s —— View job

---

Reviewing latest commit f63da0609

Todo list:

• Gather context (diff of f63da0609 against prior reviewed state)
• Review Codex transcribe endpoint URL derivation
• Review Codex multipart request construction (headers, fields)
• Review Codex error sanitization / redaction logic
• Review non-Codex path is unaffected
• Review new/updated tests
• Post final review feedback

Summary

This is a clean, correctly-scoped fix. I traced each of the four changes end to end and didn't find any new issues — the self-caught asset_pointer leak is properly closed, and the redaction/URL-derivation logic holds up against the edge cases I checked.

URL derivation (_codex_transcription_upstream_url, gateway/platforms/api_server.py:436-447)

Traced all the branches: empty base_url falls back to the documented DEFAULT_CODEX_BASE_URL (https://chatgpt.com/backend-api/codex, hermes_cli/auth.py:81) before transformation, a value already ending in /transcribe passes through unchanged (idempotent if ever called twice), .../backend-api/codex → .../backend-api/transcribe (matches the two direct unit tests), and any other custom HERMES_CODEX_BASE_URL override falls through to <normalized>/backend-api/transcribe, which is a reasonable heuristic given the sibling-endpoint relationship described in the PR.

Header/field construction (_post_audio_transcription, lines ~1403-1432)

• Accept/Origin/Referer are added before merging _codex_cloudflare_headers() (agent/auxiliary_client.py:556), so there's no key collision — User-Agent, originator: codex_cli_rs, and the conditional ChatGPT-Account-ID (extracted from the JWT's chatgpt_account_id claim) all layer in cleanly. Malformed/unsigned JWTs degrade gracefully (header helper swallows its own parse errors and just omits the account-ID header) rather than crashing the request.
• The previous silent except Exception: pass around header construction is now a real 500 with a clear codex_transcription_header_error code — strictly better: the only way this branch throws now is a genuine import/programming error, not a runtime data issue, so surfacing it beats silently sending unauthenticated-looking requests upstream.
• model field is correctly omitted from the multipart body only for provider == "openai-codex" (if provider != "openai-codex": form.add_field("model", model)) while the non-Codex path is untouched — confirmed the else OpenAI-compatible branch (_transcription_upstream_url, model field, generic error message formatting) is byte-for-byte unchanged from before this commit.

Error sanitization (_codex_transcription_error_message, _redact_codex_transcription_error_text, lines 459-497)

• Non-JSON bodies (the Cloudflare-HTML-challenge case that motivated this fix) never reach the client for any status code, not just 403: json.loads failure leaves payload as a raw string, isinstance(payload, dict) is False, so the function always falls through to the generic "ChatGPT transcription request failed with HTTP {status}." fallback — the 403 branch just gives an even friendlier fixed string on top of that, but the HTML-leak protection isn't actually 403-specific, which is a nice generalization beyond the reported bug.
• For structured JSON errors, only error.message / error (str) / message are ever extracted (_transcription_error_message_from_payload) — nested fields (e.g. an asset_pointer buried under error.metadata) are never even looked at, so the extraction itself is conservative before redaction even runs.
• Verified the redaction pipeline against the test payload ("failed for asset_pointer ptr_123 using {jwt}"): _redact_api_error_text (→ redact_sensitive_text(force=True)) masks the JWT via the eyJ... prefix pattern first, then the two asset_pointer-keyword regexes and the ptr_... regex catch the pointer reference regardless of whether it appeared in key: value form or bare prose — three overlapping patterns here is a bit belt-and-suspenders but not wrong, and matches the two new redaction tests.

Test coverage

New tests directly cover the fix: test_codex_transcription_url_uses_backend_api_sibling (URL derivation), test_post_audio_transcription_uses_codex_transcribe_request (headers/fields/no-model-field, using a real base64-JWT fixture with an embedded chatgpt_account_id claim), test_codex_transcription_403_html_error_is_sanitized (the original bug — HTML body no longer leaks), and test_codex_transcription_structured_error_redacts_sensitive_fields (the self-caught asset_pointer/bearer-token leak). All four assert on the right things and match the implementation.

No blocking issues found. Nothing to add beyond what's already covered by the new tests.

Note: command execution requires approval that wasn't available in this sandboxed review environment, so this is a static review — I wasn't able to run pytest myself. CI for this commit was still in_progress at review time (run 28637067653); worth confirming it goes green before merging. The 104 passed result you reported for the underlying suite (unaffected by this commit) plus the 4 new tests above should give solid coverage.
· branch: codex/native-audio-input