fix(codex): recover from response.completed with output=null on chatgpt.com backend

[brennonatal]

Summary

The chatgpt.com/backend-api/codex endpoint intermittently sends response.completed SSE events whose response.output field is null (observed on gpt-5.5, May 2026). The OpenAI SDK 2.24.0's parse_response() then does for output in response.output: at openai/lib/_parsing/_responses.py:61 and raises:

TypeError: 'NoneType' object is not iterable

The outer conversation loop classifies that as a non-retryable local-validation error and aborts the turn. The user sees a generic "I encountered an error" message instead of the model's actual answer — even though every text delta was already streamed to us and accumulated in agent._codex_streamed_text_parts.

Existing protection in run_codex_stream for empty-output responses only runs after stream.get_final_response() returns. The TypeError fires inside for event in stream: (during the response.completed accumulation), so the post-stream backfill never gets a chance.

Changes

• agent/codex_runtime.py:run_codex_stream — catch the specific TypeError: 'NoneType' object is not iterable from parse_response() and recover by synthesizing a response from the deltas/items already collected (same shape as the existing post-stream backfill). When nothing was collected, fall back to _run_codex_create_stream_fallback — mirrors how the existing RuntimeError(\"Expected to have received \response.completed`")` branch handles broken Responses backends like xAI OAuth, codex-lb relays, and custom Responses relays.
• agent/conversation_loop.py — when an API call fails with TypeError/ValueError (excluding UnicodeEncodeError / json.JSONDecodeError, which are handled elsewhere), log the full traceback via exc_info=. Without this the original bug was undiagnosable from the logs — only the one-line summary 'NoneType' object is not iterable reached errors.log, with no stack frame to point at parse_response.
• tests/run_agent/test_codex_stream_null_output_recovery.py — three regression tests:
  • Recovery from streamed deltas synthesizes a completed response with the assembled text.
  • When no content was collected, fall back to _run_codex_create_stream_fallback.
  • Unrelated TypeError messages still propagate (so we don't mask real bugs).

Test plan

• scripts/run_tests.sh tests/run_agent/test_codex_stream_null_output_recovery.py — 3 passed
• scripts/run_tests.sh tests/run_agent/test_codex_xai_oauth_recovery.py tests/agent/test_codex_ttfb_watchdog.py tests/run_agent/test_run_agent_codex_responses.py — 105 passed, no regressions in adjacent Codex paths
• Manual: gateway running with patch applied, the message that previously failed with 'NoneType' object is not iterable now succeeds and emits the new Codex response.completed had output=null; recovered from N collected items / M text deltas warning in agent.log.
• Platform: tested on macOS (Darwin 25.2.0, Python 3.11)

Notes

• The recovery is a true fix for the user-visible symptom (an aborted turn becomes a successful response carrying the streamed text), but the upstream Codex backend should not be sending response.completed with output=null in the first place — worth reporting to OpenAI separately. The OpenAI SDK could also defensively handle this case in parse_response().
• The diagnostic exc_info change is small but worth keeping — it catches an entire class of failures (any future SDK-side iteration bug, request-building bug, or backend payload-shape regression) that would otherwise reach errors.log with no traceback.

[chokolademilch7]

Can confirm this fixes my local repro with gpt-5.5/openai-codex. Thanks for the patch.

[YoungjaeDev]

Confirmed on another Hermes gateway setup.

Environment:

• Hermes v0.14.0
• OpenAI SDK 2.24.0
• provider=openai-codex
• model=gpt-5.5
• endpoint=https://chatgpt.com/backend-api/codex

Before patch:

• Slack/gateway turns failed with TypeError: 'NoneType' object is not iterable
• The failure was logged as a non-retryable client error after codex_stream_request

Validation:

• Applying the SDK null-output guard (for output in (response.output or []):) restored hermes chat with openai-codex/gpt-5.5
• CLI smoke test completed successfully with a normal text_response
• After restarting hermes-gateway.service, Slack Socket Mode reconnected and the gateway stayed running
• No new NoneType / non-retryable client error appeared in the recent agent logs after the restart

This matches the root cause described here: the Codex backend can send terminal response.output = null, and Hermes needs a compatibility guard until the upstream SDK/backend behavior is fixed.

[teknium1]

Closing as duplicate — the Codex null-output fix has been merged via #32963 (cherry-picked from @carltonawong's PR #32890, the one Gille reviewed). Thanks for jumping on the outage so quickly; appreciate the help. Closes #11179.