fix(agent): demote oversized tool results in protected compression tail (#61932)

[giggling-ginger]

Summary

Fixes the #61932 dead-end class: after multiple in-place compactions, a tool-heavy session can be short enough that almost every remaining message sits inside protect_last_n, yet those messages are huge completed read_file / tool outputs. The middle compress window then makes little or no token progress, preflight stalls, and the turn fails with:

Context length exceeded (… tokens). Cannot compress further.

This is a core compressor failure, not Desktop/Windows-specific. Algorithmic reproduction on Linux (no Desktop, no provider) matches the reported shape (~170k tokens stuck after multipass compress).

Changes

• Cap the prune path’s message-count floor at _MAX_TAIL_MESSAGE_FLOOR (same bound as tail-cut) so default protect_last_n=20 cannot freeze a bulky recent tool run against pruning.
• Add a pressure demotion pass: when the protected region still exceeds the soft tail budget (tail_token_budget * 1.5), demote large completed tool/file bodies inside that region while keeping a short recent floor (and the active user ask) intact. Last-resort demotion covers a single tool body larger than the soft budget alone.

Algorithmic repro (what this covers)

Post-compaction transcript: head + handoff summary + N unique large read_file results + large user prompt, with protect_last_n=20. Before: multipass compress drops a couple of rows while tokens stay ~170k (over context). After: first prune/compress pass reclaims the bulk of the protected-tail tool payload and recovers under the window.

Test plan

• scripts/run_tests.sh tests/agent/test_protected_tail_pressure_61932.py (new [Regression]: auto-compression still reaches Cannot compress further when protected recent tail is large (Desktop/openai-codex) #61932 regression)
• scripts/run_tests.sh tests/agent/test_context_compressor.py tests/agent/test_compressor_tool_call_budget.py tests/run_agent/test_infinite_compaction_loop.py tests/agent/test_compression_progress.py (177 passed)
• Optional: long Desktop tool-heavy session still worth a smoke check, but not required for this core-path fix

Closes #61932

[teknium1]

Thanks for isolating the remaining mismatch between token-budget tail cutting and the prune pre-pass. Current main still passes the uncapped protect_last_n into pruning at agent/context_compressor.py:2853-2855, while tail cutting already bounds it at agent/context_compressor.py:2698; this change addresses that live gap.

Suggested changes

• Add a focused test for the documented absolute-last-resort path at agent/context_compressor.py:1422-1432: retain one newest tool result larger than the soft ceiling after earlier pressure demotions, and assert that it is summarized without losing its tool_call_id. The current aggregate assertions in tests/agent/test_protected_tail_pressure_61932.py:133-141 and :173-181 establish recovery but do not prove that specific fallback.

Automated hermes-sweeper review.

[teknium1]

Suggestion: add a focused case for the absolute-last-resort branch. These aggregate savings assertions can pass after earlier tool bodies are demoted even if the newest oversized tool result is never summarized; assert that result is demoted and retains its tool_call_id.

[giggling-ginger]

Addressed in e1f8af0. Added a focused regression test that first demotes an earlier protected tool result, then forces the absolute-last-resort branch to summarize the newest tool result because it alone exceeds the soft ceiling. The test also asserts that its tool_call_id is preserved and the active user message remains unchanged. Verified with the focused test file (5 passed) and the related compressor suites (174 passed).

[teknium1]

Merged via #69830 (commit 18d83b4). Both your commits were cherry-picked with authorship preserved — your pressure-demotion + last-resort design won the four-way comparison on correct scoping. A regression test pinning the exact #61932 dead-end shape was added. Fixes #61932.