Skip to content

fix(agent): prevent infinite compression loop when aux model context is too small (#53008) - #885

Open
hashbender wants to merge 1 commit into
mainfrom
mirror/pr-57102
Open

fix(agent): prevent infinite compression loop when aux model context is too small (#53008)#885
hashbender wants to merge 1 commit into
mainfrom
mirror/pr-57102

Conversation

@hashbender

Copy link
Copy Markdown
Owner

Problem

When the auxiliary compression model has a context window smaller than the
main model's compression threshold, check_compression_model_feasibility()
unconditionally lowers threshold_tokens to aux_context — the aux model's
raw context size. This causes an infinite compression loop when:

  1. The session has already grown beyond the aux model's context (e.g.
    205K tokens vs 131K aux context)
  2. The lowered threshold forces preflight compression every turn
  3. The aux model cannot process 205K of input → compression is ~0.5%
    effective (205K → 204K)
  4. _compression_made_progress() sees "2 fewer messages" as progress,
    letting the 3-pass loop continue
  5. 204K > 131K threshold → next turn triggers again → repeat forever

User-facing symptoms:

⚠️ Session compressed 12 times — accuracy may degrade.
⚠️ Session compressed 13 times...
⚠️ Session compressed 16 times...

See NousResearch#53008 for full analysis and reproduction details.

Fix (2 files, +32/−7)

1. agent/conversation_compression.py — safety margin

Replace new_threshold = aux_context with int(aux_context * 0.8).
The summarisation prompt template consumes part of the aux model's
context window, so using the raw context size can overflow it.

2. agent/turn_context.py — anti-loop guard

After the 3-pass compression loop, check if token reduction was
material. If reduction < 10% and tokens still exceed the threshold,
auto-raise the threshold to break the cycle:

threshold = max(session_tokens + 1, old_threshold * 1.5)

This is a last-resort guard: if compression is so ineffective that
every turn re-triggers it without making progress, the threshold
self-corrects.

Verification

A standalone reproduction script simulates the exact threshold-adjustment,
progress-check, and preflight-loop logic from the codebase:

Buggy: threshold 131K, session 205K → 50 turns, compression never stops 🔴
Fixed: threshold 104K (80% margin), session 205K → 0 triggers 🟢

Checklist

  • Edge case: aux_context * 0.8 stays above MINIMUM_CONTEXT_LENGTH (64K)
    for any aux model ≥ 80K context. Lower models already fail the hard
    floor check earlier.
  • The 1.5× threshold raise is bounded by the main model's context
    length (implicit via should_compresscontext_length comparison).
  • Existing behavior unchanged when aux model is large enough:
    aux_context >= threshold → no code path entered.

Mirror-of: NousResearch#57102
NousResearch#57102

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant