Improve context compression memory retention and release gates - #36222
Closed
KeyArgo wants to merge 4 commits into
Closed
Improve context compression memory retention and release gates#36222KeyArgo wants to merge 4 commits into
KeyArgo wants to merge 4 commits into
Conversation
P1-5: Calibrated token accountant — learns real/rough token ratio per
(provider, model) from update_from_response samples. Median of last 20
ratios applied by should_compress() to avoid premature compression when
tool schemas inflate rough estimates. Cold-start no-op.
P1-3: Absolute reserve trigger cap — _effective_trigger_tokens() caps
the percentage threshold at (context_length - 7120) so small windows
never plan to leave less than max_output + summarizer + margin free.
_last_compress_over_reserve flag surfaces terminal over-full condition.
P1-4: Softened SUMMARY_PREFIX — removes the "latest message WINS,
discard stale items" over-correction that broke anaphora resolution
("continue", "do the next one", "apply that to the other file").
Replaces with reference-resolution framing: use summary to understand
what the latest message refers to, don't proactively resume unless
asked. Keeps anti-hijack supersede logic, MEMORY.md authority, and end
marker. Previous hardened prefix frozen in _HISTORICAL_SUMMARY_PREFIXES
so persisted summaries re-normalize on next compaction.
P0-1 tests: 3 boundary stress cases for tool_use/result pairing at
compaction edges (machinery was already complete in the live code).
25 new tests; 119 total passing (94 pre-existing + 25 new).
P0-2 (summary-of-summary architectural rewrite) deferred to own sprint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…f-summary)
Replace the lossy iterative re-summarization path (_previous_summary fed
back as LLM input on every compaction) with an append-only segment list.
Key invariant: each turn is summarized EXACTLY ONCE. Segments are never
re-fed through the LLM; they are concatenated deterministically at render
time (_render_summary_from_segments). The LLM only ever sees NEW turns,
with PRIOR CONTEXT shown read-only to handle forward-references and avoid
duplication — not to be rewritten.
Changes:
- _summary_segments: append-only list of {start, end, text} dicts
- _turns_seen: global monotone counter for segment range-anchoring
- _render_summary_from_segments(): labels older segments, keeps newest
verbatim; pure concatenation, no LLM call
- _compact_old_segments(): merges oldest two segments when list exceeds
_SEGMENT_MERGE_THRESHOLD (5) — pure text concat, still no LLM call
- _context_for_new_turns(): builds the PRIOR CONTEXT block for delta calls
- _generate_summary() delta path: "PRIOR CONTEXT / NEW TURNS ONLY" prompt
instead of "PREVIOUS SUMMARY / UPDATE IT" prompt
- Backward compat: legacy _previous_summary from a persisted handoff
message is bootstrapped into _summary_segments on first re-compaction
- _previous_summary kept in sync = rendered view of all segments, so
fallback code paths and __new__-constructed test instances still work
- All getattr guards for __new__ instances (test_compress_focus.py pattern)
10 new tests in TestSegmentSummarization; 2 existing continuity tests
updated to match new delta-path prompt labels (intent preserved).
129 total passing (94 pre-existing + 35 huddle-fix tests).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
alt-glitch
added
type/feature
teknium1
reviewed
Jul 13, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the retention work. The goal remains relevant: current main still uses iterative previous-summary rewriting at agent/context_compressor.py:1956-1970.
Problems
agent/context_compressor.py:1432concatenates whole old segments, whileagent/context_compressor.py:1407-1418renders every segment. Limiting the list to five does not bound text size, so repeated compactions can make both the handoff and the next summarizer prompt grow without limit.agent/compression_memory.py:303-311inserts new atoms but never retires earlier same-session active tasks.retrieve_for_prompt()returns all active rows atagent/compression_memory.py:451-500;mark_superseded()exists at line 439 but is not used. A later “done” or cancellation segment can therefore re-inject an obsolete active task.- The branch base predates current compaction fixes such as
76381e2a8,7f9485707, and2c6e5877; the implementation needs focused reconciliation with those current lifecycle paths.
Suggested changes
- Bound rendered segment text across unlimited compactions and add a repeated-compaction regression.
- Reconcile/supersede same-session task atoms transactionally, with done, cancellation, and replacement tests.
Automated hermes-sweeper review.
| return | ||
| a = self._summary_segments[0] | ||
| b = self._summary_segments[1] | ||
| merged_text = (a.get("text") or "").strip() + "\n\n" + (b.get("text") or "").strip() |
Contributor
There was a problem hiding this comment.
This preserves every byte of both old segments. Because _render_summary_from_segments() renders all segment bodies, the five-entry limit does not bound the handoff or PRIOR CONTEXT; each later compaction can increase prompt size indefinitely. Please impose a total rendered-token/character budget and add a repeated-compaction regression.
| segment_id=segment_id, | ||
| start_turn=start_turn, | ||
| ) | ||
| for atom in atoms: |
Contributor
There was a problem hiding this comment.
Recording a later segment only upserts atoms from that segment; it never retires earlier same-session active task atoms. retrieve_for_prompt() will continue returning them, so a completed, cancelled, or replaced task can be reinjected as active state. Reconcile old task atoms transactionally and cover active→done/cancelled/replaced transitions.
teknium1
added
sweeper:risk-session-state
Contributor
|
Thanks @KeyArgo. The durable, typed compression-memory ledger with source-ref hashing and opt-in config is a genuinely novel retention design — nothing merged covers it (the closest, the #67938 memory-provider context handoff, is a different provider-owned path). The retention eval harness is independently reusable too. But this vehicle can't be salvaged:
Closing with credit. We'd genuinely welcome the ledger + eval harness back as a focused, rebased PR: atom supersession actually wired, a hard rendered-context budget, and session-boundary clearing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
scripts/run_tests.sh -j 16venv/bin/python -m py_compile agent/auxiliary_client.py agent/lsp/workspace.py cli.py gateway/platforms/matrix.py gateway/platforms/telegram.py hermes_cli/service_manager.py tools/file_tools.py tools/terminal_tool.py tools/vision_tools.pygit diff --cached --checkbefore commitNotes
.understand-anything/,agent/context_compressor.py.bak-20260527-growing-summary-budget