fix(compaction): let tail token budget override the message-count floor on pathological tails - #67108
Closed
Kenmege wants to merge 1 commit into
Closed
fix(compaction): let tail token budget override the message-count floor on pathological tails#67108Kenmege wants to merge 1 commit into
Kenmege wants to merge 1 commit into
Conversation
…or on pathological tails The protected recent tail is bounded by a message-count floor (`protect_last_n`, capped at `_MAX_TAIL_MESSAGE_FLOOR`) so a short run of recent turns survives compaction verbatim. That floor is applied regardless of the tail's token mass. When the tail holds a few *enormous* messages — e.g. giant tool results from a file read — the floor forces all of them to be kept, producing an incompressible tail that can pin the request over the model's context window indefinitely. Failure signature: compression exhaustion. The compressor runs its full attempt budget making sub-5% no-op passes (the protected tail is simply incompressible), then aborts with "max compression attempts reached" even though the estimate still fits the window — a few huge tool results are pinning the count-floored tail. Fix: add `hard_ceiling = token_budget * 3`. In the backward tail walk, once the tail is already viable — it holds the absolute minimum of 3 messages and the most recent user message is captured — the token ceiling overrides the count floor and stops the tail from growing further. The override is gated on the next message being individually oversized (larger than the soft ceiling), so it only engages on the genuine few-but-huge pathology and never on a long run of small turns under a tiny budget (which must still honour the count floor, NousResearch#9413). The fallback floor drops from `min_tail` to 3 when the override fires, so `min()` cannot silently re-expand the tail back to the count floor. The existing `_ensure_last_user_message_in_tail` / `_ensure_last_assistant_message_in_tail` anchors still run afterwards and can only grow the tail, so the most recent user/assistant turn is never lost, and tool_call/result groups are still never split. Adds estimator-agnostic regression tests: message sizes are measured with the compressor's own token estimator and the tail budget is derived from those measurements, so the tests hold regardless of the estimator's chars-per-token calibration. They cover: the tail shrinking below the count floor but never below 3, the giant message being excluded from the tail where the unfixed floor would have kept it, the most recent user message staying in the tail, tool groups never being split, and a normal small-message tail still honouring the count floor. Fixes NousResearch#21916 Co-Authored-By: Claude <noreply@anthropic.com>
alt-glitch
added
type/bug
Collaborator
teknium1
reviewed
Jul 19, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for isolating a real count-floor exhaustion path. Current main still retains an oversized message before min_tail is met (agent/context_compressor.py:3045) and re-applies the floor at agent/context_compressor.py:3079-3081.
Problems
- The new hard-ceiling break does not work for the stated giant-tool-result case. If it breaks immediately after a
role="tool"message, the existing_align_boundary_backward()walks back over that tool result and its parent assistant tool-call (agent/context_compressor.py:2765-2772), placing the boundary before the group. The oversized tool body is therefore retained in the tail. - The new test fixture covers a huge non-tool assistant message, so it cannot expose that alignment path.
Suggested changes
- Make the ceiling decision group-aware and cut before a complete oversized assistant/tool-result group.
- Add an end-to-end
compress()regression with a giant paired tool result immediately before the viable tail; verify the group is summarized while the user/assistant anchors and tool-pair validity hold.
This is an automated hermes-sweeper review.
| msg_tokens > soft_ceiling | ||
| and accumulated + msg_tokens > hard_ceiling | ||
| and (n - cut_idx) >= 3 | ||
| and last_user_idx >= 0 |
Contributor
There was a problem hiding this comment.
This break can leave cut_idx immediately after a giant role="tool" result, but the unchanged _align_boundary_backward() then walks back over that result and its parent assistant.tool_calls group (agent/context_compressor.py:2765-2772). The group, including the giant body this PR targets, is re-added to the tail. Make this decision group-aware and add a paired-tool-result regression.
teknium1
added
sweeper:risk-compatibility
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes a compression-exhaustion failure mode: when the protected recent tail holds a few enormous messages (e.g. giant tool results from file reads), the message-count floor (
protect_last_n, capped by_MAX_TAIL_MESSAGE_FLOOR) forces all of them to be kept verbatim regardless of their token mass. The tail becomes incompressible, every compression pass is a sub-5% no-op, the attempt budget burns down, and the turn dies withmax compression attempts reached— even when the middle of the transcript is already fully summarized.Observed in production on a long-running coding session (17 h, 1000+ messages, tool-heavy):
The transcript middle was already a compaction summary; ~all of the token mass sat in 8 count-floor-protected giant tool results that no pass was allowed to touch.
The fix: in the backward tail walk of
_find_tail_cut_by_tokens, addhard_ceiling = token_budget * 3. Once the tail is already viable — it holds the absolute minimum of 3 messages and the most recent user message — the count floor yields to the token ceiling if the next message is individually oversized (> soft_ceiling) and would push the accumulated tail past the hard ceiling. The oversized messages then fall into the summarized region instead of being pinned.Why this approach: it changes nothing on healthy transcripts. The extra
msg_tokens > soft_ceilinggate means a long run of normal-sized turns under a tiny budget still honors the message-count floor exactly as today (the #9413 behavior is regression-tested). Only the genuine few-but-huge pathology engages the override, and all existing invariants hold: tool_call/result groups are never split (_align_boundary_backwardstill runs), the last user/assistant messages are still guaranteed in the tail (_ensure_last_*_message_in_tailrun afterwards and only grow), and the tail never shrinks below 3 messages.This also addresses the "Auto-Forget when compression targets fail" ask in #21916, which was closed with
sweeper:implemented-on-main— the token-budget tail protection on main is real, but it yields to the count floor, so the few-but-huge case documented there (37,299 → 34,080 tok, still far above target, session reset) still reproduces. This PR closes that gap.Related Issue
Fixes #21916
Type of Change
Changes Made
agent/context_compressor.py—_find_tail_cut_by_tokens: addedhard_ceiling(3×token_budget), a triple-gated hard-ceiling break in the backward walk (next message individually> soft_ceiling+ accumulated> hard_ceiling+ tail already ≥ 3 messages with the last user message inside), and a_tail_floorclamp so the fallbackmin()cannot silently re-expand the tail back to the count floor after the break fires. +39 lines, no behavior change outside the pathological case.tests/agent/test_compression_tail_token_cap.py— new: 5 tests covering the pathological shrink (below count floor, never below 3), tool-group integrity across the new cut, last-user-message retention, healthy-transcript no-op (count floor still honored under a tiny budget), and the pre-fix behavioral delta (the giant messages that the unfixed floor pins are excluded once the fix is active).How to Test
python -m pytest tests/agent/test_compression_tail_token_cap.py -v— 5 passed.python -m pytest tests/agent -k "tail or compress" -q— 98 passed, 1 skipped (pre-existing skip), 0 failures.agent/context_compressor.pyhunk and re-run step 1 — the pathological-tail tests fail because the count floor pins all giant tail messages and the cut never moves past them.Checklist
Code
fix(scope):,feat(scope):, etc.)scripts/run_tests.sh(the canonical CI-matching runner). Honest caveat: 20 environment-dependent failures on my macOS dev box (man-page binary tests, tempdir-path assumptions) — reproduced byte-for-byte identical at the merge-base without this change (the diff is 2 files, disjoint from every failing suite), so they are pre-existing local-environment issues, not regressions. Everything related to this change is green: the new test file 5/5,tests/agent -k "tail or compress"98 passed / 1 pre-existing skip. Deferring to CI as the clean-environment arbiter.Documentation & Housekeeping
docs/, docstrings) — N/A (behavior documented in code comments at the change site)cli-config.yaml.exampleif I added/changed config keys — N/A (no new config keys;hard_ceilingderives from the existingtoken_budget)CONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/AScreenshots / Logs
Production failure signature this fixes (long-running tool-heavy session, transcript middle already summarized, tail pinned by the count floor):