Skip to content

context_compressor: preserve absolute threshold override across model switches - #17883

Closed
BeliefanX wants to merge 1 commit into
NousResearch:mainfrom
BeliefanX:beliefanx/context-compression-threshold-tokens
Closed

context_compressor: preserve absolute threshold override across model switches#17883
BeliefanX wants to merge 1 commit into
NousResearch:mainfrom
BeliefanX:beliefanx/context-compression-threshold-tokens

Conversation

@BeliefanX

@BeliefanX BeliefanX commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add compression.threshold_tokens as an explicit absolute compression threshold override.
  • Preserve that user-configured absolute threshold across model switches, fallback restore, and context-length changes.
  • Keep runtime auxiliary compression feasibility lowering separate from the stored user override.

Implementation

  • ContextCompressor stores the user/config absolute value in threshold_tokens_override.
  • Active threshold is clamped to the current model context without losing the stored override, so switching back to a larger model can restore the intended threshold.
  • Auxiliary compression-model feasibility lowering now uses a live-session runtime cap instead of mutating the stored override.
  • The runtime cap survives primary-runtime restore/model updates, but does not become user configuration.

Compatibility / review scope

  • Existing ratio-based threshold behavior remains compatible when no absolute override is configured.
  • This PR stays independent of the gateway/delegation PRs and only covers context-compressor threshold semantics.
  • Current GitHub checks are green (success/skipped only).

Testing

  • /Users/fanxuxin/.hermes/hermes-agent/venv/bin/python -m pytest -o 'addopts=' tests/agent/test_context_compressor.py tests/run_agent/test_switch_model_context.py tests/run_agent/test_compression_feasibility.py -q — 116 passed
  • py_compile for touched runtime/config/test files — passed
  • Independent re-review verified explicit override preservation, runtime cap restore, and auxiliary feasibility semantics; no P0/P1/P2 issues.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels Apr 30, 2026
@BeliefanX
BeliefanX force-pushed the beliefanx/context-compression-threshold-tokens branch 4 times, most recently from 92203a0 to f7ae224 Compare May 6, 2026 08:27
@BeliefanX

Copy link
Copy Markdown
Contributor Author

Refreshed this PR onto current main and resolved the merge conflicts.

Current status after refresh:

  • branch is now mergeable against main
  • attribution / supply-chain / nix / e2e checks are passing
  • the remaining red test check appears to be the current upstream full-suite baseline, not this PR's focused change; failures are broad and match the same unrelated areas seen on other refreshed branches (Bedrock 1M, cron script empty-output note, Telegram topic hint debounce, browser Chromium guard, delegate heartbeat, sandbox cwd assertions, etc.)

I ran the relevant focused tests locally for this PR before force-pushing the refreshed branch.

@BeliefanX
BeliefanX force-pushed the beliefanx/context-compression-threshold-tokens branch from 9670c61 to daef20c Compare May 29, 2026 08:35
@BeliefanX
BeliefanX force-pushed the beliefanx/context-compression-threshold-tokens branch from daef20c to 43be3cc Compare June 6, 2026 14:36
@BeliefanX
BeliefanX force-pushed the beliefanx/context-compression-threshold-tokens branch from 43be3cc to d5c34c2 Compare June 14, 2026 08:01
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating the requested absolute-threshold behavior and for covering model-switch and fallback restoration.

Problems

  • Current main now derives thresholds from the effective input window after reserving model.max_tokens (agent/context_compressor.py:1008-1029). The PR's raw-context_length clamp would bypass that safety contract when salvaged.
  • compression.threshold_tokens also needs gateway cache invalidation: gateway/run.py:15623-15634 enumerates baked compressor settings, and the new key is absent.
  • The added user-facing setting needs docs. Current documentation states that threshold_tokens is always derived from threshold × context_length (website/docs/developer-guide/context-compression-and-caching.md:167-175).

Suggested changes

  • Integrate the override into the current _compute_threshold_tokens()/small-context safety model rather than restoring the older recomputation path.
  • Add the gateway cache key and a regression test, then update the compression configuration documentation.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
@BeliefanX

Copy link
Copy Markdown
Contributor Author

Closing this version after re-review against current main. The permanent absolute-threshold knob is not a confirmed product requirement, and this branch's raw-context path no longer fits the effective-input/output-reservation safety model. If a concrete need appears, it should return as a smaller current-main change with the required cache invalidation and documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants