fix(agent): count codex_reasoning_items in tail-protection token estimate - #55756
Closed
ms-alan wants to merge 1 commit into
Closed
fix(agent): count codex_reasoning_items in tail-protection token estimate#55756ms-alan wants to merge 1 commit into
ms-alan wants to merge 1 commit into
Conversation
…mate Closes NousResearch#55572 The tail-protection estimator _estimate_msg_budget_tokens() was counting message content and tool_calls but not codex_reasoning_items. For sessions running on the OpenAI Responses/Codex backend, those items carry the model's encrypted_content reasoning blobs which are real wire payload replayed to the provider. The tail estimator was blind to them, so the protected tail under-measured Responses sessions by 100K+ tokens, causing compaction to trigger too late. The fix adds iteration over codex_reasoning_items using the same len(str(it)) // _CHARS_PER_TOKEN formula used for tool_calls.
Collaborator
Related to #45485 (the broader OPEN fix in the same |
Contributor
|
Thanks for the focused regression fix. This is already implemented on current
Closing as implemented on main. |
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #55572
Summary
The tail-protection estimator
_estimate_msg_budget_tokens()was counting message content andtool_callsbut notcodex_reasoning_items. For sessions running on the OpenAI Responses/Codex backend, those items carry the model'sencrypted_contentreasoning blobs — real wire payload replayed to the provider. The estimator was blind to them, so the protected tail under-measured Responses sessions by 100K+ tokens (27% of session size in observed case), causing compaction to trigger too late.Root cause
codex_reasoning_itemswas simply absent from the token-counting loop in_estimate_msg_budget_tokens, despite being counted indirectly via full-dict stringify in the preflight estimatorestimate_messages_tokens_rough.Fix
Add iteration over
codex_reasoning_itemsusing the samelen(str(it)) // _CHARS_PER_TOKENformula already used fortool_calls:Observed impact
A real 214-turn session carried 162 reasoning items totalling ~459 KB (~115K tokens) — none of which the old tail estimator counted. The protected tail (~20K-token budget) silently held ~100K+ extra wire tokens.
Note
Privacy is unaffected:
_serialize_for_summaryonly readscontent,tool_calls, andtool_call_id, so encrypted blobs are never sent to the auxiliary summarizer.