fix(codex): handle None response.output in OpenAI SDK stream parser - #33173
Closed
blazing-mj wants to merge 11 commits into
Closed
blazing-mj wants to merge 11 commits into
blazing-mj wants to merge 11 commits into
Conversation
Pin default gateway argv with --profile, warn on sticky active_profile redirects, log gateway boot profile drift, add profile gateway status visibility, and harden related fallback/config paths. Verified: targeted pytest slices passed; py_compile passed; static scan clean; Claude Code Max review PASS; dashboard HTTP 200.
Background — the existing tool-result persistence layer (Layer 2: maybe_persist_tool_result) only kicks in past 100,000 chars (~25K tokens). A mid-sized search_files dump at 56K chars is ~14K tokens — under the persistence threshold yet still large enough to push the session past the compression trigger after a few calls. Add cap_tool_result_tokens as a defensive Layer 2a that runs AFTER persistence but BEFORE the result is appended to the message list: when a single tool result exceeds ~8K estimated tokens, slice the head to ~6K tokens and append a "[truncated NNNN tokens]" marker so the model knows information was dropped. Multimodal (list) content is passed through untouched — image budgeting is handled elsewhere. Wired into both the concurrent and sequential paths in tool_executor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Forensics — a session showed:
Preflight compression: ~67,042 tokens >= 64,000 threshold
(model gpt-5.5, ctx 272,000)
64,000 is 0.20 * 320,000 (the old default), not 0.30 * 272,000 = 81,600
that the user's current config.yaml dictates. The threshold resolution
path is correct (agent_init reads compression.threshold from config) so
the live runtime was simply stale relative to the on-disk config and
needed a gateway restart.
Add an INFO log line at AIAgent boot (and a small helper in
conversation_compression to format it) that surfaces every knob —
threshold %, target %, protect_first/_last, context_len, and the
absolute trigger_at token count. Future stale-config / stale-runtime
mismatches show up immediately by diffing this line against config.yaml.
Compression configured: threshold=30% target=10% protect_first=3
protect_last=8 context_len=272000 trigger_at=81600 tokens
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Forensics — observed in agent.log:
context compression done: messages=30->30 tokens=~66721->57363
context compression done: messages=62->62 tokens=~65646->50434
...
Same message count in and out: when protect_first_n + protect_last_n
covers (or nearly covers) the whole transcript, the compressor has
nothing to drop and just re-summarises inline. Each pass burns Haiku
tokens, only nibbles a few % off the window, and never crosses back
below the trigger threshold — so the next tool result re-triggers
preflight on the next iteration and the cycle repeats.
Guard:
1. Add compression_made_progress() — returns False when the message
count is unchanged AND token reduction is below 15%.
2. After each preflight pass, run the predicate; if it says no
progress, log "compression-no-progress, aborting compress loop
for this turn" and latch _compression_no_progress_turn on the
agent.
3. Reset the latch at the top of each turn (alongside the other
per-turn flag resets).
4. At the post-tool-call compression site, gate the call on
should_skip_compression_for_turn(agent) so the same turn does
not retry compression on every subsequent tool batch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ChatGPT's Codex backend (chatgpt.com/backend-api/codex) emits response.completed events with response.output set to null instead of an empty list when output items were produced via output_item.done but the final accumulation came back empty. The OpenAI SDK's parse_response() blindly iterates response.output (openai/lib/_parsing/_responses.py:61), raising: TypeError: 'NoneType' object is not iterable This kills the entire turn before Hermes' existing empty-output backfill (codex_runtime.py:~247) can synthesize output from streamed deltas. Solution: - Install a defensive monkey-patch wrapper around parse_response() that coerces response.output = None → [] before iteration - Patch both _sdk_parse and _sdk_stream modules (streaming module imports parse_response by name at import time) - Idempotent guard prevents double-patching - Graceful fallback if SDK internals change Also adds full traceback logging for TypeError/AttributeError/ValueError in API call paths (conversation_loop.py) to aid future debugging. Fixes: openai-codex provider failing with 'NoneType' object is not iterable Tested: Verified on ChatGPT Codex backend (gpt-5.5 model)
alt-glitch
added
type/bug
|
The core Codex null-output fix in this PR duplicates merged #32963 which already landed the same |
Collaborator
|
Closing as obsolete. PR #33042 (merged commit If you're still seeing this on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ChatGPT's Codex backend (
chatgpt.com/backend-api/codex) returnsresponse.output = nullin completion events, causing the OpenAI SDK to crash with:This kills every turn before Hermes' empty-output backfill can run.
Solution
parse_response()that coercesNone→[]_sdk_parseand_sdk_streammodules (streaming imports by-name at import time)Also adds full traceback logging for unexpected errors (TypeError/AttributeError/etc) in API call paths.
Testing
✅ Verified on ChatGPT Codex backend with
gpt-5.5model✅ Zero errors after fix deployed
✅ Previously: TypeError on every API call
Impact
Fixes
openai-codexprovider for all Hermes users running against ChatGPT backend.