Skip to content

fix: recover Codex streams with null output - #32910

Closed
bender-the-bot1k wants to merge 1 commit into
NousResearch:mainfrom
bender-the-bot1k:fix/codex-null-output-robust-recovery
Closed

fix: recover Codex streams with null output#32910
bender-the-bot1k wants to merge 1 commit into
NousResearch:mainfrom
bender-the-bot1k:fix/codex-null-output-robust-recovery

Conversation

@bender-the-bot1k

Copy link
Copy Markdown

What does this PR do?

Fixes two Codex/Responses API edge cases that both surface as:

TypeError: 'NoneType' object is not iterable

Observed in gateway logs with provider=openai-codex, base_url=https://chatgpt.com/backend-api/codex, model=gpt-5.5.

Root causes covered here:

  1. chatgpt.com/backend-api/codex can stream usable output, then send a terminal response.completed payload with response.output = null. openai-python parses that terminal event with for output in response.output, so the typed stream raises before Hermes can call stream.get_final_response() and backfill output.
  2. The Responses transport was passing tools=None when there were no tools. openai-python treats that as an explicitly provided value and iterates it inside _make_tools, causing the same NoneType error before the request is sent.

This PR keeps the recovery local to the Codex Responses path:

  • retry typed Responses streaming once on the exact null-output TypeError
  • fall back to raw responses.create(stream=True), which lets Hermes manually consume events
  • preserve already streamed response.output_item.done items or text deltas
  • coerce unrecoverable output=None to [] so downstream response.output_text access does not crash
  • guard diagnostic response.output_text access in the conversation loop
  • omit the tools kwarg entirely when there are no tools
  • log full tracebacks for unexpected API-shape errors so future provider regressions are diagnosable from agent.log

This intentionally goes a little beyond the smaller duplicate fixes: it covers both the terminal null-output stream parser failure and the request-building tools=None failure, with regression tests for each.

Related Issue

No issue number found. Related/overlapping open PRs found while checking for duplicates: #11182, #32884, #32888, #32890, #32891.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • agent/codex_runtime.py
    • handle typed stream TypeError: 'NoneType' object is not iterable caused by terminal response.output = null
    • retry once, then fall back to raw responses.create(stream=True)
    • backfill terminal responses from streamed output items/text deltas
    • coerce unrecoverable null output to []
  • agent/conversation_loop.py
    • guard computed response.output_text access during Codex invalid-response diagnostics
    • log full tracebacks for unexpected API-shape exceptions
  • agent/transports/codex.py
    • omit tools when converted Responses tools are empty/None
  • tests/run_agent/test_run_agent_codex_responses.py
    • add regressions for null-output typed stream fallback and unrecoverable null-output coercion
  • tests/agent/transports/test_codex_transport.py
    • add regression coverage for omitting tools/tool_choice/parallel_tool_calls when no tools exist

How to Test

  1. Run the focused regression suite:
python -m pytest tests/run_agent/test_run_agent_codex_responses.py tests/agent/transports/test_codex_transport.py -q -o 'addopts='
  1. Expected result:
113 passed
  1. Reproduce mentally from the logged crash shape:
openai/lib/_parsing/_responses.py:61
  for output in response.output
TypeError: 'NoneType' object is not iterable

The new tests simulate that exact typed-stream parser failure, verify fallback recovery, and verify tools=None is no longer sent.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Ubuntu 24.04 / Linux 6.8.0

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Original observed traceback:

agent/codex_runtime.py:196
  for event in stream

openai/lib/streaming/responses/_responses.py:360
  self._completed_response = parse_response(...)

openai/lib/_parsing/_responses.py:61
  for output in response.output

TypeError: 'NoneType' object is not iterable

Focused test result:

113 passed in 22.36s

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API codex duplicate This issue or pull request already exists labels May 27, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #32884. Combined fix for both Codex null output and tools=None. For the tools=None portion, see also #32911. Root cause: #11179.

@teknium1

Copy link
Copy Markdown
Contributor

Closing as duplicate — the Codex null-output fix has been merged via #32963 (cherry-picked from @carltonawong's PR #32890, the one Gille reviewed). Thanks for jumping on the outage so quickly; appreciate the help. Closes #11179.

@teknium1 teknium1 closed this May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants