feat: per-model compression threshold overrides by bennybuoy · Pull Request #63020 · NousResearch/hermes-agent

bennybuoy · 2026-07-12T06:33:06Z

Per-model compression threshold overrides (v2, rebased)

Replaces #60781 (closed). Cleanly rebased on current main — no unrelated deletions or anti-thrashing changes. Addresses both points from @teknium1's review:

Review feedback addressed

1. Gateway cache invalidation — added ('compression', 'model_thresholds') to _CACHE_BUSTING_CONFIG_KEYS in gateway/run.py. A live config edit to the map now invalidates the cached compressor instead of silently keeping stale thresholds.

2. Integrated resolver with small-context floor — per-model overrides are resolved FIRST, then the existing 75% floor for <512K models is applied ON TOP. The floor is no longer replaced; it stacks:

Override below 75% on a small-context model → floored to 75% (raise-only)
Override above 75% on a small-context model → override wins
Override on a large-context model (≥512K) → override applies directly (no floor)
No override → global threshold + floor as before (fully backward compatible)

What it does

compression.model_thresholds in config.yaml maps model name substrings to threshold fractions. Longest match wins (so glm-5.2-1M beats glm-5.2). Re-resolved on /model switch.

compression:
  threshold: 0.50
  model_thresholds:
    "glm-5.2": 0.40
    "claude-sonnet": 0.35
    "gpt-5": 0.30

Files changed (6 files, +323/-11)

agent/context_compressor.py — resolve_model_threshold() helper, model_thresholds param, integration with _effective_threshold_percent()
agent/context_engine.py — base class update_model() applies overrides for plugin engines
agent/agent_init.py — reads compression.model_thresholds from config, passes to compressor
gateway/run.py — cache busting key added
cli-config.yaml.example — documents the feature
tests/run_agent/test_per_model_compression_threshold.py — 17 tests, all passing

Test results

17 passed in 2.82s

Tests cover: resolve helper (no overrides, empty model, exact/substring/longest match, no match, lower override), compressor init (large context with/without override, small context with override above/below floor, empty/None thresholds), update_model (re-resolve on switch, fall back to global), and ContextEngine base class.

Addresses teknium1 review feedback on PR NousResearch#60781: 1. Gateway cache invalidation: added ('compression', 'model_thresholds') to _CACHE_BUSTING_CONFIG_KEYS so a live config edit to the map invalidates the cached compressor (previously kept stale thresholds). 2. Integrated resolver with small-context floor: per-model overrides are resolved FIRST, then the existing 75% floor for <512K models is applied on top. The floor is no longer replaced — it stacks. An override below 75% on a small-context model still gets floored to 75% (raise-only); an override above 75% wins. 3. Clean rebase on upstream main — no unrelated deletions or anti-thrashing changes. Only the per-model threshold feature is added. Changes: - resolve_model_threshold() module-level helper (longest substring match) - ContextCompressor.__init__ accepts model_thresholds dict - _base_threshold_percent stores the per-model resolved value - _config_threshold_percent stores the raw config value (fallback base) - update_model() re-resolves on /model switch, falls back to config value - ContextEngine base class update_model() applies overrides for plugin engines - agent_init.py reads compression.model_thresholds from config, passes to ctor - gateway/run.py cache busting key added - cli-config.yaml.example documents the feature - 17 tests covering resolve helper, compressor init (large/small context, override above/below floor), update_model (re-resolve, fallback), base class Co-authored-by: Copilot <copilot@github.com>

Copilot

Pull request overview

Adds support for configuring per-model compression thresholds so Hermes can trigger context compaction at different usage points depending on the active model (resolved by longest substring match), while preserving existing compression behavior when no overrides are configured.

Changes:

Introduces compression.model_thresholds resolution logic and integrates it into compressor model-switch handling.
Extends the ContextEngine base update_model() to apply per-model threshold overrides for plugin engines.
Updates gateway cache-busting, example config documentation, and adds a dedicated test suite for override behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
`agent/context_compressor.py`	Adds `resolve_model_threshold()` and wires per-model overrides into compressor init + `update_model()`.
`agent/context_engine.py`	Applies per-model overrides in base `ContextEngine.update_model()` for engines that don’t override it.
`agent/agent_init.py`	Reads `compression.model_thresholds` from config and passes/propagates it to compressors/engines.
`gateway/run.py`	Busts gateway agent cache when `compression.model_thresholds` changes.
`cli-config.yaml.example`	Documents the new `compression.model_thresholds` configuration.
`tests/run_agent/test_per_model_compression_threshold.py`	Adds tests covering resolution rules, floor interactions, and model-switch re-resolution.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        _new_base = resolve_model_threshold(
+            model, self.model_thresholds, _config_pct,
+        )


+    best_key = ""
+    for key in model_thresholds:
+        if key in model and len(key) > len(best_key):
+            best_key = key
+    if best_key:
+        return float(model_thresholds[best_key])
+    return default


            api_mode=agent.api_mode,
        )
+        # Propagate per-model threshold overrides to plugin engines.
+        if compression_model_thresholds:
+            agent.context_compressor.model_thresholds = compression_model_thresholds


+  # Per-model threshold overrides: keys are substring-matched against the model
+  # name (longest match wins). Useful when some models need different compaction
+  # points — e.g. a 1M-context model can compress later (0.30) while a 128K
+  # model needs to compress earlier (0.60). The small-context floor (75% for
+  # <512K models) still applies on top of per-model overrides.


+    _raw_model_thresholds = _compression_cfg.get("model_thresholds", {})
+    if isinstance(_raw_model_thresholds, dict):
+        compression_model_thresholds = {
+            str(k): float(v) for k, v in _raw_model_thresholds.items()
+            if isinstance(v, (int, float)) and not isinstance(v, bool)
+        }
+    else:
+        compression_model_thresholds = {}


teknium1 · 2026-07-12T06:38:02Z

Thanks for preserving the small-context floor and adding gateway cache invalidation. The configurable built-in compressor premise is still present on current main.

Problems

The selected plugin engine is initialized through update_model() before this PR assigns model_thresholds (agent/agent_init.py:1802-1809). Since the new base ContextEngine.update_model() resolves the override during that call, the initial plugin model retains its global threshold. tests/run_agent/test_plugin_context_engine_init.py:40-67 covers this initialization path, but the new tests do not exercise it.
This also crosses the documented plugin boundary: website/docs/developer-guide/context-engine-plugin.md:168 says compression.* is specific to ContextCompressor; plugin engines own their config.
Please add the new public key to DEFAULT_CONFIG (hermes_cli/config.py:1412-1445) and the primary compression reference (website/docs/developer-guide/context-compression-and-caching.md:77-109), not only cli-config.yaml.example.

Suggested changes

Scope the map to ContextCompressor, or establish an explicit context-engine contract and configure it before initial update_model().
Add an AIAgent + selected-plugin initialization regression test.

Automated hermes-sweeper review.

@bennybuoy

Follow-up to the salvaged contributor commit, closing the three gaps flagged in the sweeper review: 1. Init ordering: assign compression.model_thresholds to a selected plugin context engine BEFORE the initial update_model() call in agent_init.py, so the initial model's override applies from init (previously it only took effect after the first /model switch). Base-class ContextEngine.update_model() now snapshots the pre-override percent once so repeated switches fall back to the engine's configured threshold, not a previous model's override. 2. DEFAULT_CONFIG: add compression.model_thresholds (empty map) to hermes_cli/config.py — additive key, no _config_version bump. 3. Docs: document the key in website/docs/developer-guide/context-compression-and-caching.md (yaml example, parameter table, dedicated section) and update the plugin-boundary note in context-engine-plugin.md to state the explicit context-engine contract for model_thresholds. Adds tests/run_agent/test_per_model_threshold_init_ordering.py: plugin-engine AIAgent init regression (override applies at init, empty map unchanged), DEFAULT_CONFIG key presence, floor interaction on the model-switch path (override below the small-context floor is raised to the floor; above the floor wins), and base-class config snapshot across repeated switches. Also maps @bennybuoy in contributors/emails/.

teknium1 · 2026-07-22T16:18:53Z

Merged via #69339 (commit f944e84). Your v2 commit was cherry-picked with authorship preserved; the three remaining review gaps (init ordering, DEFAULT_CONFIG key, docs) were closed in a follow-up commit. Thanks!

@bennybuoy

…h#63020) Follow-up to the salvaged contributor commit, closing the three gaps flagged in the sweeper review: 1. Init ordering: assign compression.model_thresholds to a selected plugin context engine BEFORE the initial update_model() call in agent_init.py, so the initial model's override applies from init (previously it only took effect after the first /model switch). Base-class ContextEngine.update_model() now snapshots the pre-override percent once so repeated switches fall back to the engine's configured threshold, not a previous model's override. 2. DEFAULT_CONFIG: add compression.model_thresholds (empty map) to hermes_cli/config.py — additive key, no _config_version bump. 3. Docs: document the key in website/docs/developer-guide/context-compression-and-caching.md (yaml example, parameter table, dedicated section) and update the plugin-boundary note in context-engine-plugin.md to state the explicit context-engine contract for model_thresholds. Adds tests/run_agent/test_per_model_threshold_init_ordering.py: plugin-engine AIAgent init regression (override applies at init, empty map unchanged), DEFAULT_CONFIG key presence, floor interaction on the model-switch path (override below the small-context floor is raised to the floor; above the floor wins), and base-class config snapshot across repeated switches. Also maps @bennybuoy in contributors/emails/.

@bennybuoy

…h#63020) Follow-up to the salvaged contributor commit, closing the three gaps flagged in the sweeper review: 1. Init ordering: assign compression.model_thresholds to a selected plugin context engine BEFORE the initial update_model() call in agent_init.py, so the initial model's override applies from init (previously it only took effect after the first /model switch). Base-class ContextEngine.update_model() now snapshots the pre-override percent once so repeated switches fall back to the engine's configured threshold, not a previous model's override. 2. DEFAULT_CONFIG: add compression.model_thresholds (empty map) to hermes_cli/config.py — additive key, no _config_version bump. 3. Docs: document the key in website/docs/developer-guide/context-compression-and-caching.md (yaml example, parameter table, dedicated section) and update the plugin-boundary note in context-engine-plugin.md to state the explicit context-engine contract for model_thresholds. Adds tests/run_agent/test_per_model_threshold_init_ordering.py: plugin-engine AIAgent init regression (override applies at init, empty map unchanged), DEFAULT_CONFIG key presence, floor interaction on the model-switch path (override below the small-context floor is raised to the floor; above the floor wins), and base-class config snapshot across repeated switches. Also maps @bennybuoy in contributors/emails/.

@bennybuoy

…h#63020) Follow-up to the salvaged contributor commit, closing the three gaps flagged in the sweeper review: 1. Init ordering: assign compression.model_thresholds to a selected plugin context engine BEFORE the initial update_model() call in agent_init.py, so the initial model's override applies from init (previously it only took effect after the first /model switch). Base-class ContextEngine.update_model() now snapshots the pre-override percent once so repeated switches fall back to the engine's configured threshold, not a previous model's override. 2. DEFAULT_CONFIG: add compression.model_thresholds (empty map) to hermes_cli/config.py — additive key, no _config_version bump. 3. Docs: document the key in website/docs/developer-guide/context-compression-and-caching.md (yaml example, parameter table, dedicated section) and update the plugin-boundary note in context-engine-plugin.md to state the explicit context-engine contract for model_thresholds. Adds tests/run_agent/test_per_model_threshold_init_ordering.py: plugin-engine AIAgent init regression (override applies at init, empty map unchanged), DEFAULT_CONFIG key presence, floor interaction on the model-switch path (override below the small-context floor is raised to the floor; above the floor wins), and base-class config snapshot across repeated switches. Also maps @bennybuoy in contributors/emails/.

Copilot AI review requested due to automatic review settings July 12, 2026 06:33

Copilot started reviewing on behalf of bennybuoy July 12, 2026 06:33 View session

Copilot AI reviewed Jul 12, 2026

View reviewed changes

alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/config Config system, migrations, profiles P3 Low — cosmetic, nice to have labels Jul 12, 2026

alt-glitch mentioned this pull request Jul 18, 2026

feat(agent): per-model / per-provider compression threshold overrides #19110

Closed

9 tasks

teknium1 added the area/compression Context compression and continuation sessions label Jul 19, 2026

teknium1 mentioned this pull request Jul 22, 2026

feat(compression): per-model threshold overrides in config #69339

Merged

This was referenced Jul 22, 2026

Per-model compression threshold overrides #37630

Closed

feat: Support fixed token threshold and per-model compression settings in config.yaml #24695

Closed

teknium1 closed this Jul 22, 2026

Navigation Menu

feat: per-model compression threshold overrides - #63020

feat: per-model compression threshold overrides#63020
bennybuoy wants to merge 1 commit into
NousResearch:mainfrom
bennybuoy:feat/per-model-compression-v2

bennybuoy commented Jul 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

teknium1 commented Jul 12, 2026

Uh oh!

teknium1 commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

bennybuoy commented Jul 12, 2026

Per-model compression threshold overrides (v2, rebased)

Review feedback addressed

What it does

Files changed (6 files, +323/-11)

Test results

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

teknium1 commented Jul 12, 2026

Problems

Suggested changes

Uh oh!

teknium1 commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants