Skip to content

[codex] fix(agent): preserve Codex recovered stream usage - #33025

Closed
xyang199 wants to merge 1 commit into
NousResearch:mainfrom
xyang199:codex/preserve-codex-usage
Closed

[codex] fix(agent): preserve Codex recovered stream usage#33025
xyang199 wants to merge 1 commit into
NousResearch:mainfrom
xyang199:codex/preserve-codex-usage

Conversation

@xyang199

Copy link
Copy Markdown

Summary

  • preserve terminal response.completed.response.usage when recovering Codex Responses streams after response.output = null
  • carry recovered usage through both the main Codex stream path and auxiliary Codex adapter
  • add regressions that reproduce iterator-time SDK TypeError recovery while asserting exact usage is retained

Root Cause

#32963 recovers already-streamed output when the OpenAI SDK raises TypeError: 'NoneType' object is not iterable while parsing a terminal Codex Responses event with output = null.

That terminal event can still contain accurate usage, but the synthesized recovered response used usage=None, so token accounting state such as context_compressor.last_prompt_tokens, session counters, and footer/context display could remain at 0.

Validation

python -m pytest tests/run_agent/test_run_agent_codex_responses.py::test_run_codex_stream_falls_back_when_stream_iteration_parses_null_output tests/agent/test_auxiliary_client.py::TestCodexAuxiliaryAdapterNullOutputRecovery::test_recovers_output_item_when_sdk_raises_during_iteration -q
# 2 passed

python -m pytest tests/run_agent/test_run_agent_codex_responses.py tests/agent/test_auxiliary_client.py -q
# 243 passed

Fixes #33019

@xyang199
xyang199 force-pushed the codex/preserve-codex-usage branch from 7627b40 to ea6b538 Compare May 27, 2026 04:14
@xyang199
xyang199 marked this pull request as ready for review May 27, 2026 04:15
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/openai OpenAI / Codex Responses API codex labels May 27, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Competing with #33017 — both preserve Codex recovered stream usage from null-output recovery (#32963). Both fix #33019. Recommend consolidating into a single PR.

@teknium1

Copy link
Copy Markdown
Contributor

Closing as obsolete. PR #33042 (merged commit cb38ce28c) removes the OpenAI SDK's client.responses.stream(...) helper from both Codex call sites entirely — we now use client.responses.create(stream=True) raw event iteration that assembles the final response from response.output_item.done events as they arrive. The terminal event's output field is never read for content reconstruction, so the SDK's TypeError: 'NoneType' object is not iterable parser crash on output=null is structurally impossible — no code path can produce it. Same defensive strategy OpenClaw uses for the same backend.

The symptom-patch + recovery logic this PR added on top of the SDK helper is now redundant. Thanks for the work during the outage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have provider/openai OpenAI / Codex Responses API type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex null-output stream recovery drops terminal response usage

3 participants