Skip to content

feat(compression): add absolute token threshold via compression.threshold_tokens - #24279

Closed
DanielMaly wants to merge 1 commit into
NousResearch:mainfrom
DanielMaly:feat/threshold-tokens-v2
Closed

feat(compression): add absolute token threshold via compression.threshold_tokens#24279
DanielMaly wants to merge 1 commit into
NousResearch:mainfrom
DanielMaly:feat/threshold-tokens-v2

Conversation

@DanielMaly

@DanielMaly DanielMaly commented May 12, 2026

Copy link
Copy Markdown

Reopened from #22762 (auto-closed during fork sync).

Adds a new compression.threshold_tokens config option that triggers compression when the conversation exceeds an absolute token count, regardless of context window utilization.

This is useful for:

  • Long-running sessions where you want to proactively compress before hitting context limits
  • Enforcing consistent compression behavior across different model context sizes
  • Reducing latency and cost by compressing earlier in very long conversations

The threshold is checked alongside the existing utilization-based compression trigger — whichever fires first wins.

@alt-glitch alt-glitch added type/feature New feature or request comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists labels May 12, 2026
@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 13, 2026
@teknium1

teknium1 commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the absolute-threshold proposal. Current main still derives compression solely from compression.threshold (agent/agent_init.py:1502-1505; agent/context_compressor.py:990-1029), so the feature remains needed.

Problems

  • The implementation is stale: compressor construction moved from run_agent.py to agent/agent_init.py:1499-1828 in 9f408989c.
  • The cap would not survive a model switch or fallback. The PR updates the live percentage, but ContextCompressor.update_model() restores _configured_threshold_percent before recomputing (agent/context_compressor.py:904-917); live update paths include agent/agent_runtime_helpers.py:2017-2024 and agent/chat_completion_helpers.py:1531-1538.
  • No tests cover cap-vs-ratio selection or those update paths, and the user-facing compression references (website/docs/user-guide/configuration.md:737-752) need the new setting.

Suggested changes

  • Make the absolute cap a first-class compressor configuration value and apply it whenever thresholds are recomputed, with tests for model/context changes and output-token reservations.
  • Salvage this as a focused compression change; the background-review/curator/Honcho changes are unrelated to the stated feature, and main already contains background-review memory isolation (973f27e95).

Automated hermes-sweeper review.

@DanielMaly
DanielMaly force-pushed the feat/threshold-tokens-v2 branch from a4ae967 to a8b033e Compare July 13, 2026 16:05
@DanielMaly

DanielMaly commented Jul 13, 2026

Copy link
Copy Markdown
Author

Rework addressing sweeper feedback

This commit replaces the previous implementation with a clean rework that addresses all points from the automated review:

Changes from the review feedback

  1. Stale implementation fixed: The cap is now a first-class compressor config value (threshold_tokens_cap parameter on ContextCompressor.__init__), not a post-construction patch on the live instance in run_agent.py (which was stale — compressor construction moved to agent/agent_init.py in 9f408989c).

  2. Survives model switch/fallback: The cap is stored as an instance attribute and re-applied in update_model(), so it survives model switches and fallback activations. The old approach was undone by update_model() restoring _configured_threshold_percent before recomputing.

  3. Tests added: 9 tests in TestThresholdTokensCap covering:

    • Cap lower than ratio → cap wins
    • Cap higher than ratio → ratio wins
    • No cap → ratio only
    • Cap survives model switch (the core sweeper feedback)
    • Cap survives model switch to smaller window
    • Cap clamped to context length
    • Cap with max_tokens output-token reservation
    • Cap survives model switch with max_tokens change
    • Invalid cap values (zero, negative, non-numeric) treated as None

  4. Docs updated: Added threshold_tokens to the configuration reference in website/docs/user-guide/configuration.md.

  5. Unrelated changes removed: The background-review/curator/Honcho commits have been dropped (main already contains background-review memory isolation in 973f27e95). This is now a focused compression-only change.

Files changed

  • agent/context_compressor.py — new threshold_tokens_cap parameter, _coerce_threshold_tokens_cap(), _apply_threshold_tokens_cap(), applied in __init__ and update_model()
  • agent/agent_init.py — read config value, pass to constructor, display message
  • hermes_cli/config.py — DEFAULT_CONFIG default + show_config() display
  • gateway/run.py — cache-busting config key for gateway hot-reload
  • website/docs/user-guide/configuration.md — user-facing docs
  • tests/agent/test_context_compressor.py — 9 new tests

@DanielMaly
feat(compression): add absolute token threshold via compression.thres…
9358033 
…hold_tokens

Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.

This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.

Rework from PR NousResearch#24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
  (threshold_tokens_cap parameter on ContextCompressor.__init__),
  not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
  switches and fallback activations (the old approach was undone by
  update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
  a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
  context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
  already contains background-review memory isolation in 973f27e).

Config example:
  compression:
    threshold: 0.50
    threshold_tokens: 200000   # never compress later than 200K tokens
@DanielMaly
DanielMaly force-pushed the feat/threshold-tokens-v2 branch from a8b033e to 9358033 Compare July 13, 2026 16:21
@teknium1 teknium1 added the area/compression Context compression and continuation sessions label Jul 19, 2026
@teknium1 teknium1 mentioned this pull request Jul 22, 2026
Merged
teknium1 pushed a commit that referenced this pull request Jul 22, 2026
@DanielMaly @teknium1
feat(compression): add absolute token threshold via compression.thres…
4d49664 
…hold_tokens

Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.

This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.

Rework from PR #24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
  (threshold_tokens_cap parameter on ContextCompressor.__init__),
  not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
  switches and fallback activations (the old approach was undone by
  update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
  a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
  context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
  already contains background-review memory isolation in 973f27e).

Config example:
  compression:
    threshold: 0.50
    threshold_tokens: 200000   # never compress later than 200K tokens
teknium1 added a commit that referenced this pull request Jul 22, 2026
@teknium1
test(compression): behavioral + config wiring coverage for threshold_…
1c8a613 
…tokens

Follow-up for salvaged #24279:
- cli-config.yaml.example: document compression.threshold_tokens
  (commented-out, default null = disabled)
- contributors/emails: map maly.dan@gmail.com -> DanielMaly
- tests: should_compress() fires at the absolute cap below the pct
  threshold (first-fires-wins); DEFAULT_CONFIG ships None and 0/None
  are behavior-neutral incl. across update_model(); the small-context
  pct floor is unaffected by the cap and re-derives correctly on
  model switch
teknium1 pushed a commit that referenced this pull request Jul 22, 2026
@DanielMaly @teknium1
feat(compression): add absolute token threshold via compression.thres…
e5078e3 
…hold_tokens

Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.

This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.

Rework from PR #24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
  (threshold_tokens_cap parameter on ContextCompressor.__init__),
  not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
  switches and fallback activations (the old approach was undone by
  update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
  a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
  context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
  already contains background-review memory isolation in 973f27e).

Config example:
  compression:
    threshold: 0.50
    threshold_tokens: 200000   # never compress later than 200K tokens
teknium1 added a commit that referenced this pull request Jul 22, 2026
@teknium1
test(compression): behavioral + config wiring coverage for threshold_…
020bd1b 
…tokens

Follow-up for salvaged #24279:
- cli-config.yaml.example: document compression.threshold_tokens
  (commented-out, default null = disabled)
- contributors/emails: map maly.dan@gmail.com -> DanielMaly
- tests: should_compress() fires at the absolute cap below the pct
  threshold (first-fires-wins); DEFAULT_CONFIG ships None and 0/None
  are behavior-neutral incl. across update_model(); the small-context
  pct floor is unaffected by the cap and re-derives correctly on
  model switch
@teknium1 teknium1 mentioned this pull request Jul 22, 2026
Closed
@teknium1

teknium1 commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Merged via #69335 (commit 020bd1b). Your reworked commit was cherry-picked with authorship preserved — the update_model() survival you added post-review was exactly right. Thanks!

@teknium1 teknium1 closed this Jul 22, 2026
@DanielMaly
DanielMaly deleted the feat/threshold-tokens-v2 branch July 27, 2026 08:25
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
@DanielMaly @teknium1
feat(compression): add absolute token threshold via compression.thres…
9448b13 
…hold_tokens

Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.

This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.

Rework from PR NousResearch#24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
  (threshold_tokens_cap parameter on ContextCompressor.__init__),
  not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
  switches and fallback activations (the old approach was undone by
  update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
  a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
  context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
  already contains background-review memory isolation in ba371c6).

Config example:
  compression:
    threshold: 0.50
    threshold_tokens: 200000   # never compress later than 200K tokens
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
@teknium1
test(compression): behavioral + config wiring coverage for threshold_…
28cc46c 
…tokens

Follow-up for salvaged NousResearch#24279:
- cli-config.yaml.example: document compression.threshold_tokens
  (commented-out, default null = disabled)
- contributors/emails: map maly.dan@gmail.com -> DanielMaly
- tests: should_compress() fires at the absolute cap below the pct
  threshold (first-fires-wins); DEFAULT_CONFIG ships None and 0/None
  are behavior-neutral incl. across update_model(); the small-context
  pct floor is unaffected by the cap and re-derives correctly on
  model switch
33hodl pushed a commit to 33hodl/hermes-agent that referenced this pull request Aug 12, 2026
@DanielMaly @teknium1
feat(compression): add absolute token threshold via compression.thres…
db6b04c 
…hold_tokens

Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.

This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.

Rework from PR NousResearch#24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
  (threshold_tokens_cap parameter on ContextCompressor.__init__),
  not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
  switches and fallback activations (the old approach was undone by
  update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
  a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
  context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
  already contains background-review memory isolation in 973f27e).

Config example:
  compression:
    threshold: 0.50
    threshold_tokens: 200000   # never compress later than 200K tokens
33hodl pushed a commit to 33hodl/hermes-agent that referenced this pull request Aug 12, 2026
@teknium1
test(compression): behavioral + config wiring coverage for threshold_…
d6b6053 
…tokens

Follow-up for salvaged NousResearch#24279:
- cli-config.yaml.example: document compression.threshold_tokens
  (commented-out, default null = disabled)
- contributors/emails: map maly.dan@gmail.com -> DanielMaly
- tests: should_compress() fires at the absolute cap below the pct
  threshold (first-fires-wins); DEFAULT_CONFIG ships None and 0/None
  are behavior-neutral incl. across update_model(); the small-context
  pct floor is unaffected by the cap and re-derives correctly on
  model switch
prmartinow pushed a commit to prmartinow/hermes-agent that referenced this pull request Aug 26, 2026
@DanielMaly @teknium1
feat(compression): add absolute token threshold via compression.thres…
fa21f14 
…hold_tokens

Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.

This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.

Rework from PR NousResearch#24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
  (threshold_tokens_cap parameter on ContextCompressor.__init__),
  not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
  switches and fallback activations (the old approach was undone by
  update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
  a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
  context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
  already contains background-review memory isolation in 0422ec6).

Config example:
  compression:
    threshold: 0.50
    threshold_tokens: 200000   # never compress later than 200K tokens
prmartinow pushed a commit to prmartinow/hermes-agent that referenced this pull request Aug 26, 2026
@teknium1
test(compression): behavioral + config wiring coverage for threshold_…
c48223e 
…tokens

Follow-up for salvaged NousResearch#24279:
- cli-config.yaml.example: document compression.threshold_tokens
  (commented-out, default null = disabled)
- contributors/emails: map maly.dan@gmail.com -> DanielMaly
- tests: should_compress() fires at the absolute cap below the pct
  threshold (first-fires-wins); DEFAULT_CONFIG ships None and 0/None
  are behavior-neutral incl. across update_model(); the small-context
  pct floor is unaffected by the cap and re-derives correctly on
  model switch
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
@melon-xf
feat(compression): add absolute token threshold via compression.thres…
88c816d 
…hold_tokens

Add compression.threshold_tokens config option that sets an absolute
token cap for auto-compaction. When configured alongside the existing
ratio-based threshold, the effective trigger point is the lower of the
two, so compression never fires later than the user's preferred token
count regardless of which model is active.

This solves the problem where switching between models with different
context windows (e.g. 1M → 400K) shifts the absolute trigger point,
causing premature or delayed compression.

Rework from PR NousResearch#24279 addressing sweeper feedback:
- The cap is now a first-class compressor configuration value
  (threshold_tokens_cap parameter on ContextCompressor.__init__),
  not a post-construction patch on the live instance.
- Applied in both __init__ and update_model() so it survives model
  switches and fallback activations (the old approach was undone by
  update_model() restoring _configured_threshold_percent).
- Clamped to the model's context length so a cap above the window is
  a no-op (ratio-based threshold wins).
- Works with max_tokens output-token reservations.
- Added 9 tests covering cap-vs-ratio selection, model switch survival,
  context-length clamping, max_tokens interaction, and invalid values.
- Updated user-facing configuration docs.
- Removed unrelated background-review/curator/Honcho changes (main
  already contains background-review memory isolation in 973f27e).

Config example:
  compression:
    threshold: 0.50
    threshold_tokens: 200000   # never compress later than 200K tokens
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
@melon-xf
test(compression): behavioral + config wiring coverage for threshold_…
80d7e5c 
…tokens

Follow-up for salvaged NousResearch#24279:
- cli-config.yaml.example: document compression.threshold_tokens
  (commented-out, default null = disabled)
- contributors/emails: map maly.dan@gmail.com -> DanielMaly
- tests: should_compress() fires at the absolute cap below the pct
  threshold (first-fires-wins); DEFAULT_CONFIG ships None and 0/None
  are behavior-neutral incl. across update_model(); the small-context
  pct floor is unaffected by the cap and re-derives correctly on
  model switch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area/compression Context compression and continuation sessions comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@DanielMaly @teknium1 @alt-glitch