Skip to content

fix(compression): don't let config context_length auto-lower threshold - #58623

Open
AlexFucuson9 wants to merge 1 commit into
NousResearch:mainfrom
AlexFucuson9:alex/fix-compression-config-auto-lower
Open

fix(compression): don't let config context_length auto-lower threshold#58623
AlexFucuson9 wants to merge 1 commit into
NousResearch:mainfrom
AlexFucuson9:alex/fix-compression-config-auto-lower

Conversation

@AlexFucuson9

Copy link
Copy Markdown
Contributor

Summary

auxiliary.compression.context_length is intended to bound the compression model's input size. But the auto-lower logic used it as the model's natural context length, incorrectly lowering the session threshold.

Problem

When a user sets auxiliary.compression.context_length: 128000 to limit the summarizer's workload on a model with 800K natural context, the auto-lower reduces the threshold from 400K to 128K. This causes compression to fire on every turn with almost no token reduction, burning ~1.3M input tokens on ineffective summarization calls.

Root Cause

In conversation_compression.py, the auto-lower logic does:

if aux_context < threshold:  # aux_context = 128000 (from config)
    new_threshold = aux_context  # sets threshold to 128000

The config_context_length override is returned as aux_context by get_model_context_length(), so the auto-lower treats the user's input-size limit as the model's actual context window.

Fix

Resolve the model's NATURAL context length (without config override) separately, and use only the natural length for auto-lower decisions. The config override still bounds the compression model's input size.

Fixes #58407

auxiliary.compression.context_length is intended to bound the compression
model's input size. But the auto-lower logic used it as the model's
natural context length, incorrectly lowering the session threshold when
the config value is smaller than the threshold.

When a user sets context_length=128000 to limit the summarizer's workload
on a model with 800K natural context, the auto-lower would reduce the
threshold from 400K to 128K — causing compression to fire on every turn
with almost no token reduction.

Fix: resolve the model's natural context length (without config override)
separately, and use only the natural length for auto-lower decisions.
The config override still bounds the compression model's input size.

Fixes NousResearch#58407
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jul 5, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.
Duplicate of #58418 — both fix #58407 by excluding an explicit auxiliary.compression.context_length override from the auto-lower decision in check_compression_model_feasibility() (same file, same site). #58418 is earlier/canonical (#58435 already closed as its dup). Flagging for a maintainer to pick one.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating the every-turn compression loop; current main does reproduce the mechanism at agent/conversation_compression.py:260-303.

Problems

  • auxiliary.compression.context_length is currently documented in code as a feasibility hint for custom endpoints that cannot report /models (agent/agent_init.py:1628-1644). Ignoring it for auto-lowering can leave a threshold above that endpoint's declared capability.
  • The changed feasibility logic does not bound the actual summarizer request. The compressor serializes selected turns into a single prompt at agent/context_compressor.py:1971-1972 and sends it at :2114-2144.
  • The PR has no regression tests; the existing feasibility suite covers forwarding this override at tests/run_agent/test_compression_feasibility.py:176-197.

Suggested changes

Automated hermes-sweeper review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/compression Context compression and continuation sessions comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

auxiliary.compression.context_length config has unintended side effect: triggers auto-lower and causes compression to fire on every turn

3 participants