Skip to content

fix(agent): prevent infinite compression loop when aux model context is too small (#53008) - #57102

Closed
bone-bone-bone wants to merge 1 commit into
NousResearch:mainfrom
bone-bone-bone:fix/compression-threshold-safety-53008
Closed

fix(agent): prevent infinite compression loop when aux model context is too small (#53008)#57102
bone-bone-bone wants to merge 1 commit into
NousResearch:mainfrom
bone-bone-bone:fix/compression-threshold-safety-53008

Conversation

@bone-bone-bone

Copy link
Copy Markdown

Problem

When the auxiliary compression model has a context window smaller than the
main model's compression threshold, check_compression_model_feasibility()
unconditionally lowers threshold_tokens to aux_context — the aux model's
raw context size. This causes an infinite compression loop when:

  1. The session has already grown beyond the aux model's context (e.g.
    205K tokens vs 131K aux context)
  2. The lowered threshold forces preflight compression every turn
  3. The aux model cannot process 205K of input → compression is ~0.5%
    effective (205K → 204K)
  4. _compression_made_progress() sees "2 fewer messages" as progress,
    letting the 3-pass loop continue
  5. 204K > 131K threshold → next turn triggers again → repeat forever

User-facing symptoms:

⚠️ Session compressed 12 times — accuracy may degrade.
⚠️ Session compressed 13 times...
⚠️ Session compressed 16 times...

See #53008 for full analysis and reproduction details.

Fix (2 files, +32/−7)

1. agent/conversation_compression.py — safety margin

Replace new_threshold = aux_context with int(aux_context * 0.8).
The summarisation prompt template consumes part of the aux model's
context window, so using the raw context size can overflow it.

2. agent/turn_context.py — anti-loop guard

After the 3-pass compression loop, check if token reduction was
material. If reduction < 10% and tokens still exceed the threshold,
auto-raise the threshold to break the cycle:

threshold = max(session_tokens + 1, old_threshold * 1.5)

This is a last-resort guard: if compression is so ineffective that
every turn re-triggers it without making progress, the threshold
self-corrects.

Verification

A standalone reproduction script simulates the exact threshold-adjustment,
progress-check, and preflight-loop logic from the codebase:

Buggy: threshold 131K, session 205K → 50 turns, compression never stops 🔴
Fixed: threshold 104K (80% margin), session 205K → 0 triggers 🟢

Checklist

  • Edge case: aux_context * 0.8 stays above MINIMUM_CONTEXT_LENGTH (64K)
    for any aux model ≥ 80K context. Lower models already fail the hard
    floor check earlier.
  • The 1.5× threshold raise is bounded by the main model's context
    length (implicit via should_compresscontext_length comparison).
  • Existing behavior unchanged when aux model is large enough:
    aux_context >= threshold → no code path entered.

…is too small (NousResearch#53008)

When the auxiliary compression model's context window is smaller than
the main model's compression threshold, the threshold is unconditionally
set to aux_context. If the session has already grown beyond that size,
compression becomes nearly worthless yet triggers every turn, creating
an infinite loop.

Two changes:

1. conversation_compression.py: apply 80% safety margin on aux_context
   (the summarisation prompt template consumes part of the context)

2. turn_context.py: after the 3-pass preflight compression loop, check
   actual token reduction. If <10% and tokens still exceed threshold,
   auto-raise threshold to break the cycle.

See NousResearch#53008 for full analysis.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Jul 2, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Competing with #53235 for the same issue (#53008). This PR is a 2-file subset (conversation_compression.py 80% margin + turn_context.py anti-loop threshold guard); #53235 is broader (adds a proactive main-model fallback in context_compressor.py plus tests) and appears canonical. Same goal, different mechanism — flagging for a maintainer to pick.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for investigating the compression-loop failure mode.

This is an automated hermes-sweeper review. Current main already provides the requested no-infinite-loop guarantee:

  • agent/context_compressor.py:1348 records an ineffective attempt from the provider-reported prompt count when a completed compaction remains at or above its threshold.
  • agent/context_compressor.py:1442 blocks further automatic compaction after two ineffective attempts (and also guards repeated fallback compactions), rather than retrying every turn.
  • tests/agent/test_compaction_anti_thrash.py:130 covers the rough-preflight/real-usage sequence that could otherwise reopen the loop.
  • The load-bearing implementation is 7f9485707d0183af0c9f04e1153e1ec9bd98aa68 (fix(compaction): judge the anti-thrash verdict on real usage, not in should_compress), verified as an ancestor of current main.

The member discussion identified #53235 as the broader competing approach; its current-main anti-thrash guarantee now supersedes this PR's threshold-escape guard.

@teknium1 teknium1 closed this Jul 15, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants