Skip to content

fix(compaction): skip compression when it cannot reduce the token count - #56415

Closed
golldyck wants to merge 1 commit into
NousResearch:mainfrom
golldyck:fix/compression-net-savings-guard
Closed

fix(compaction): skip compression when it cannot reduce the token count#56415
golldyck wants to merge 1 commit into
NousResearch:mainfrom
golldyck:fix/compression-net-savings-guard

Conversation

@golldyck

@golldyck golldyck commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Skips trajectory compaction when it can't actually reduce the token count. compress_trajectory (and its async twin) replaced a compressible middle region with a summary without checking that the summary is smaller than what it replaces. When the region was already small, the "summary" came out larger than the original. So the trajectory grew while still being marked was_compressed=True (observed: 406 tokens went up to 465, tokens_saved = -59).

The fix adds a net-savings guard: when the compressible region's token count is <= the summary target, return the trajectory unchanged instead of expanding it. This applies to both the sync and async paths (sibling call path included).

Related Issue

No separate issue filed; reproduces on current main.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • trajectory_compressor.py: guard both compress_trajectory and compress_trajectory_async so they skip compression when region_tokens <= summary_target_tokens, returning the trajectory unchanged.
  • tests/test_trajectory_compressor.py: sync and async tests asserting a tiny compressible region is left unchanged (no token growth, was_compressed not set).

How to Test

  1. On main: compressing a trajectory with a small compressible middle grows the token count yet reports was_compressed=True.
  2. Apply this PR.
  3. Run scripts/run_tests.sh tests/test_trajectory_compressor.py -q. The added net-savings tests pass (34 passed). Note: 3 tests (*_omits_temperature, kimi/moonshot) fail identically on unmodified main. They're unrelated to this change (temperature-omission for kimi models), not a regression from this PR.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(compaction): ...)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix (no unrelated commits)
  • I've run scripts/run_tests.sh tests/test_trajectory_compressor.py; the net-savings tests pass, and the only failures are the 3 pre-existing *_omits_temperature cases that also fail on main
  • I've added tests for my changes (sync + async)
  • I've tested on my platform: macOS 15

Documentation & Housekeeping

  • Documentation: N/A
  • cli-config.yaml.example: N/A
  • CONTRIBUTING.md / AGENTS.md: N/A
  • Cross-platform impact: N/A (pure token-count arithmetic)
  • Tool descriptions/schemas: N/A

compress_trajectory (and _async) replaced the compressible middle region with
a [CONTEXT SUMMARY] turn without checking that the region is actually larger
than the summary. When a large protected system prompt dominates the budget,
the compressible middle can be tiny; replacing e.g. a 2-token middle with a
~60-token summary GROWS the trajectory (tokens_saved negative), marks it
was_compressed, and still spends a summarization call — the opposite of the
intent, on exactly the hard over-budget cases.

Add a net-savings guard mirroring the code's own comment (net_savings =
region_tokens - summary_target_tokens): if the safely-compressible region is
no larger than summary_target_tokens, return the trajectory unchanged. Applied
to both the sync and async paths. Add sync+async regression tests.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state P2 Medium — degraded but workaround exists labels Jul 1, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: LGTM

Adds a guard to skip compression when the compressible region is smaller than the summary that would replace it. Prevents token count from growing instead of shrinking.

Looks Good

  • Clean guard in both sync and async paths
  • Well-tested with both sync and async test cases
  • Correctly returns trajectory unchanged when compression would not help

Reviewed by Hermes Agent

@golldyck

golldyck commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

Rechecked against current main: the change still rebases cleanly (no conflicts) and the compaction guard tests pass. For transparency, the only red tests in tests/test_trajectory_compressor.py are the three *_omits_temperature (kimi/moonshot) cases, which fail on unmodified main as well — they're unrelated to this net-savings guard. Happy to rebase if you'd like it refreshed before merge.

@teknium1 teknium1 added sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users area/compression Context compression and continuation sessions area/usage-cost Token accounting, usage reporting, billing, cost tracking labels Jul 15, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Merged via salvage PR #70243 as a clean cherry-pick, authorship preserved — thanks @golldyck! Both sync and async net-savings guards plus your regression tests are on main (merge commit 69339aa). The counterfactual run against pre-fix main confirmed the tests target the real 406→465-token growth bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/compression Context compression and continuation sessions area/usage-cost Token accounting, usage reporting, billing, cost tracking comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants