feat: per-model compression threshold overrides by bennybuoy · Pull Request #60781 · NousResearch/hermes-agent

bennybuoy · 2026-07-08T08:21:32Z

Summary

Users who swap between models with very different context windows (e.g. a 256K model and a 1M model) need different compaction trigger points. A single global compression.threshold can't be optimal for both — 50% of 256K (128K) wastes a 1M window, while 25% of 256K (64K) compresses too aggressively on smaller models.

This PR adds compression.model_thresholds to config.yaml, letting users set per-model override fractions that are resolved by longest substring match:

compression:
  threshold: 0.50           # global default
  model_thresholds:
    glm-5.2-1M: 0.25        # 1M window → compact at ~250K
    glm-5.2: 0.70           # 256K window → compact at ~179K

Why this needs to exist

Model switching between large-context models is now common — users routinely switch between a 256K context model and a 1M context model in the same session (e.g. via /model). The existing codex_gpt55_autoraise hardcoded special case proves the need: different models need different compaction points. But that approach doesn't scale — we can't add a hardcoded override for every model family. A user-configurable per-model map generalizes the pattern and gives users control over their own model combinations.

Changes

resolve_model_threshold() helper in agent/context_compressor.py — module-level function importable by plugin context engines (e.g. LCM) for the same resolution logic
ContextCompressor.__init__ — accepts model_thresholds dict, resolves the effective threshold at construction time
ContextCompressor.update_model() — re-resolves the per-model threshold on every model switch
ContextEngine base class — stores model_thresholds and applies them in update_model() for plugin engines that don't override it
agent_init.py — reads model_thresholds from config and passes to both the built-in compressor and plugin engines
DEFAULT_CONFIG, cli-config.yaml.example, and docs updated
16 new tests covering the helper function, compressor init/switch, and base class behavior

Backward compatibility

Empty model_thresholds: {} (default) = identical behavior to before
The existing codex_gpt55_autoraise and Arcee Trinity hardcoded overrides still apply on top of this config
No changes to the wire format, tool schemas, or system prompt — no prompt caching impact

Test results

16 passed in 2.63s  (new tests)
31 passed in 13.11s (existing compression tests)
145 passed in 13.45s (existing config tests)

Users who swap between models with very different context windows (e.g. 256K and 1M tokens) need different compaction trigger points. A single global threshold can't be optimal for both — 50% of 256K (128K) wastes a 1M window, while 25% of 256K (64K) compresses too aggressively on smaller models. This adds compression.model_thresholds to config.yaml: compression: threshold: 0.50 # global default model_thresholds: glm-5.2-1M: 0.25 # 1M window -> compact at ~250K glm-5.2: 0.70 # 256K window -> compact at ~179K Keys are matched as substrings against the model name (longest match wins). The matched value replaces the global threshold for that model, working in both directions (raise and lower). Changes: - resolve_model_threshold() helper in context_compressor.py (importable by plugin engines like LCM) - ContextCompressor.__init__ accepts model_thresholds, resolves at construction and on every update_model() call - ContextEngine base class stores model_thresholds and applies them in update_model() for plugin engines that don't override it - agent_init.py reads model_thresholds from config and passes to both built-in compressor and plugin engines - DEFAULT_CONFIG, cli-config.yaml.example, and docs updated - 16 new tests covering helper, compressor init/switch, and base class

alt-glitch · 2026-07-08T08:27:41Z

This was generated by AI during triage.

Related: this implements the per-model compression-threshold feature tracked by #18733 (canonical open spec/anchor) and #24695, and competes with open impl PRs #19110, #24704, #38139. This PR resolves via a config compression.model_thresholds map (longest-substring match) with a resolve_model_threshold() helper importable by plugin context engines. Cluster has several open competing approaches — flagging for a maintainer to pick a canonical implementation. Not marking any as duplicate (different mechanisms).

Copilot

Pull request overview

Adds user-configurable per-model compression trigger thresholds so Hermes can compact earlier/later depending on the active model’s context window, including support for dynamic re-resolution on /model switches and documentation/test coverage to match.

Changes:

Introduces compression.model_thresholds config for longest-substring-match threshold overrides per model.
Adds resolve_model_threshold() and wires per-model override resolution into the built-in ContextCompressor init and model-switch path.
Propagates model_thresholds from config into both built-in and plugin context engines; updates docs and adds tests.

Reviewed changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
website/docs/user-guide/configuration.md	Documents `compression.model_thresholds` and matching semantics.
tests/run_agent/test_per_model_compression_threshold.py	Adds tests for threshold resolution, compressor behavior, and base engine update behavior.
hermes_cli/config.py	Adds `compression.model_thresholds` to `DEFAULT_CONFIG`.
cli-config.yaml.example	Adds example config entries for per-model threshold overrides.
agent/context_engine.py	Stores/uses `model_thresholds` in base engine `update_model()` and recalculates threshold tokens.
agent/context_compressor.py	Adds `resolve_model_threshold()` and applies per-model overrides in compressor init and `update_model()`.
agent/agent_init.py	Reads `compression.model_thresholds` from config and passes/propagates it to compressors/engines.
.gitignore	Ignores a top-level `build/` directory.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+    # Per-model threshold overrides (longest substring match wins).
+    # Engines that support per-model thresholds should read this dict
+    # in their update_model() override. The base class stores it but
+    # does not use it (threshold_percent is left untouched).
+    model_thresholds: dict = {}


        self.context_length = context_length
+        # Apply per-model threshold override if configured. Engines that
+        # override update_model() should call resolve_model_threshold() from
+        # agent.context_compressor for the same logic.
+        if self.model_thresholds and model:
+            from agent.context_compressor import resolve_model_threshold
+            self.threshold_percent = resolve_model_threshold(
+                model, self.model_thresholds, self.threshold_percent,
+            )
        self.threshold_tokens = int(context_length * self.threshold_percent)


…lback - model_thresholds: dict = {} → dict | None = None (mutable class attr shared across instances) - update_model() now falls back to _base_threshold_percent instead of the current threshold_percent, so switching from an overridden model to one with no match correctly resets to the global threshold - Added _base_threshold_percent field to ContextEngine base class

teknium1

Thanks for the focused per-model configuration proposal. The requested capability remains absent on current main, but this branch needs adaptation to two current invariants.

Problems

agent/agent_init.py:1842 wires the map only into compressor construction. Gateway cache reuse derives invalidation keys in gateway/run.py:15680-15716; this PR does not add compression.model_thresholds, so a live map edit would keep the old cached compressor.
Current agent/context_compressor.py:898-909 re-applies a 75% floor below 512K, added by 76381e2a8 to stop compaction loops. The resolver introduced here replaces the same threshold path, so its precedence relative to that safety floor needs an explicit decision and regression coverage.

Suggested changes

Include the map in gateway cache invalidation and test a map-only config edit.
Integrate the resolver with the current small-context floor rather than replacing that pipeline; validate finite, safe threshold values.

Automated hermes-sweeper review.

teknium1 · 2026-07-10T15:03:50Z

@@ -1825,6 +1842,7 @@ def _moa_reference_relay(event: str, **kwargs: Any) -> None:
            api_mode=agent.api_mode,


This is initialization-only, but the gateway reuses cached agents based on _CACHE_BUSTING_CONFIG_KEYS in gateway/run.py. Please include compression.model_thresholds there and add a map-only cache-invalidation test; otherwise live config edits keep the old threshold map.

teknium1 · 2026-07-10T15:03:50Z

@@ -883,6 +919,10 @@ def update_model(
        self.provider = provider


Current main now reapplies a 75% threshold floor below 512K on every model update (agent/context_compressor.py:898-909, commit 76381e2a8) to prevent compaction loops. Rebase this resolver into that path and specify/test whether a per-model override can intentionally bypass the floor.

bennybuoy · 2026-07-12T06:32:54Z

Closing in favor of a clean rebase. New PR will follow — same feature but rebased on current main, addresses both review points (gateway cache invalidation + small-context floor integration), and drops unrelated deletions.

Addresses teknium1 review feedback on PR #60781: 1. Gateway cache invalidation: added ('compression', 'model_thresholds') to _CACHE_BUSTING_CONFIG_KEYS so a live config edit to the map invalidates the cached compressor (previously kept stale thresholds). 2. Integrated resolver with small-context floor: per-model overrides are resolved FIRST, then the existing 75% floor for <512K models is applied on top. The floor is no longer replaced — it stacks. An override below 75% on a small-context model still gets floored to 75% (raise-only); an override above 75% wins. 3. Clean rebase on upstream main — no unrelated deletions or anti-thrashing changes. Only the per-model threshold feature is added. Changes: - resolve_model_threshold() module-level helper (longest substring match) - ContextCompressor.__init__ accepts model_thresholds dict - _base_threshold_percent stores the per-model resolved value - _config_threshold_percent stores the raw config value (fallback base) - update_model() re-resolves on /model switch, falls back to config value - ContextEngine base class update_model() applies overrides for plugin engines - agent_init.py reads compression.model_thresholds from config, passes to ctor - gateway/run.py cache busting key added - cli-config.yaml.example documents the feature - 17 tests covering resolve helper, compressor init (large/small context, override above/below floor), update_model (re-resolve, fallback), base class Co-authored-by: Copilot <copilot@github.com>

Addresses teknium1 review feedback on PR NousResearch#60781: 1. Gateway cache invalidation: added ('compression', 'model_thresholds') to _CACHE_BUSTING_CONFIG_KEYS so a live config edit to the map invalidates the cached compressor (previously kept stale thresholds). 2. Integrated resolver with small-context floor: per-model overrides are resolved FIRST, then the existing 75% floor for <512K models is applied on top. The floor is no longer replaced — it stacks. An override below 75% on a small-context model still gets floored to 75% (raise-only); an override above 75% wins. 3. Clean rebase on upstream main — no unrelated deletions or anti-thrashing changes. Only the per-model threshold feature is added. Changes: - resolve_model_threshold() module-level helper (longest substring match) - ContextCompressor.__init__ accepts model_thresholds dict - _base_threshold_percent stores the per-model resolved value - _config_threshold_percent stores the raw config value (fallback base) - update_model() re-resolves on /model switch, falls back to config value - ContextEngine base class update_model() applies overrides for plugin engines - agent_init.py reads compression.model_thresholds from config, passes to ctor - gateway/run.py cache busting key added - cli-config.yaml.example documents the feature - 17 tests covering resolve helper, compressor init (large/small context, override above/below floor), update_model (re-resolve, fallback), base class Co-authored-by: Copilot <copilot@github.com>

Copilot AI review requested due to automatic review settings July 8, 2026 08:21

Copilot started reviewing on behalf of bennybuoy July 8, 2026 08:21 View session

alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/config Config system, migrations, profiles P3 Low — cosmetic, nice to have labels Jul 8, 2026

Copilot AI reviewed Jul 8, 2026

View reviewed changes

teknium1 reviewed Jul 10, 2026

View reviewed changes

bennybuoy closed this Jul 12, 2026

bennybuoy mentioned this pull request Jul 12, 2026

feat: per-model compression threshold overrides #63020

Closed

teknium1 added the area/compression Context compression and continuation sessions label Jul 19, 2026

GottZ mentioned this pull request Aug 2, 2026

feat(agent): per-model / per-provider compression threshold overrides #19110

Open

9 tasks

Navigation Menu

Uh oh!

feat: per-model compression threshold overrides - #60781

feat: per-model compression threshold overrides#60781
bennybuoy wants to merge 2 commits into
NousResearch:mainfrom
bennybuoy:feat/per-model-compression-threshold

bennybuoy commented Jul 8, 2026

Uh oh!

alt-glitch commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 10, 2026

Uh oh!

teknium1 Jul 10, 2026

Uh oh!

bennybuoy commented Jul 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		@@ -1825,6 +1842,7 @@ def _moa_reference_relay(event: str, **kwargs: Any) -> None:
		api_mode=agent.api_mode,

		@@ -883,6 +919,10 @@ def update_model(
		self.provider = provider

Uh oh!

Conversation

bennybuoy commented Jul 8, 2026

Summary

Why this needs to exist

Changes

Backward compatibility

Test results

Uh oh!

alt-glitch commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 10, 2026

Choose a reason for hiding this comment

Uh oh!

bennybuoy commented Jul 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants