fix(agent): decay protected summaries after restart

[harjothkhara]

What does this PR do?

Decays protect_first_n after a restart when a persisted context-compression handoff summary is already inside the would-be protected head.

Before this change, _effective_protect_first_n() only decayed from in-memory compression_count / _previous_summary. After a gateway restart those are reset, so compress() could preserve the old handoff summary verbatim before it had a chance to rehydrate _previous_summary, leaving that old summary as a fossilized protected-head message.

The fix keeps the same in-memory decay behavior and detects persisted handoff summaries in the resumed-head region. If one is present, the compressor treats the session as already compacted. On the first post-restart compaction, self-heal scans the transcript for actual Hermes handoff summaries, folds any pre-summary live turns into the next summary prompt, and removes stacked summary fossils instead of preserving them verbatim. If summary generation falls back deterministically, the rehydrated previous summary is carried forward in a bounded redacted snapshot. Summary handoffs outside the bounded decay probe can be folded without decaying first-compaction head protection. When a forced user-leading summary must merge into the active user tail, the active request remains below the summary end marker and the merged summary remains rehydratable on the next compaction.

This intentionally implements the issue's Option 2 + Option 3 path instead of persisting a new compaction-state field. The no-system role pinning change is an adjacent pre-existing bug fix for already-decayed transcripts where the gateway path strips the system message before calling compress().

Related Issue

Fixes #57814

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Security fix
• Documentation update
• Tests (adding or improving test coverage)
• Refactor (no behavior change)
• New skill (bundled or hub)

Changes Made

• agent/context_compressor.py: pass messages into _effective_protect_first_n() so restart/resume state can be inferred from a persisted handoff summary in the protected head.
• agent/context_compressor.py: force the inserted/merged summary to keep a user-leading visible transcript when resumed no-system histories decay to an empty protected head.
• agent/context_compressor.py: align summary metadata/content detection and fold non-summary pre-handoff head turns into the next summary prompt.
• agent/context_compressor.py: limit restart decay inference to a named resumed-head probe window so unrelated summary-looking live-tail messages do not decay first-compaction protection.
• agent/context_compressor.py: extend first-pass self-heal folding through the transcript so short, zero-protect_first_n, and beyond-probe restart fossils are not preserved as protected tail.
• agent/context_compressor.py: preserve a bounded redacted previous-summary snapshot in deterministic fallback summaries.
• agent/context_compressor.py: keep active user tail content after the summary boundary when forced user-leading summaries merge into tail.
• agent/context_compressor.py: strip summary end markers even when live tail content follows, so merged summaries rehydrate without swallowing that tail.
• tests/agent/test_context_compressor_summary_continuity.py: add regression tests for protected-head and default protect_first_n=3 restart/resume fossilization cases.
• tests/agent/test_context_compressor_summary_continuity.py: add stacked-summary, metadata-only, protected-tail, fallback-preservation, zero-protect, beyond-probe, active-tail ordering, late-handoff tail preservation, merged-summary rehydration, live-tail head-protection, and restart-probe boundary regression coverage.
• tests/agent/test_context_compressor.py: add a no-system resumed-handoff role-ordering regression.

Duplicate-work checks

• gh issue view 57814 --repo NousResearch/hermes-agent --comments: issue is open with no comments at the time of implementation.
• gh pr list --repo NousResearch/hermes-agent --state all --search "57814": no matching PRs.
• gh pr list --repo NousResearch/hermes-agent --state all --search "protect_first_n decay state in-memory gateway restart compaction summaries": no matching PRs.
• Broader search for "compression protect_first_n summary gateway restart" found fix(compressor): shrink protect_first_n on recompaction (#17344) #17349 closed and feat(compression): add configurable warn_after_compressions threshold (#53876) #53958 unrelated to restart decay.

How to Test

1. Red proof on current main before the fix:
   scripts/run_tests.sh tests/agent/test_context_compressor_summary_continuity.py -- -k resume_handoff -q
   • The new regression test failed because the old summary remained in the compressed result.
2. Green focused proof after the fix:
   scripts/run_tests.sh tests/agent/test_context_compressor_summary_continuity.py tests/agent/test_context_compressor.py tests/agent/test_resume_stale_active_task.py -- -k "protect_first_n or context_summary_content or previous_summary or resume_handoff or default_protected_head or stacked_handoffs or metadata_summary_decay or without_system_still_starts or inherited_handoff or historical_prefixed or tail_summary_marker or restart_probe_boundary or protected_tail or fallback_preserves or zero_protect_first_n or beyond_restart_probe or double_compaction_user_tail or merge_into_tail or late_handoff or forced_leading" -q -p no:cacheprovider
   • 29 tests passed.
3. Full focused continuity file:
   scripts/run_tests.sh tests/agent/test_context_compressor_summary_continuity.py -- -q -p no:cacheprovider
   • 16 tests passed.
4. Static / compatibility checks:
   • .venv/bin/ruff check agent/context_compressor.py tests/agent/test_context_compressor_summary_continuity.py tests/agent/test_context_compressor.py tests/agent/test_resume_stale_active_task.py
   • .venv/bin/python scripts/check-windows-footguns.py agent/context_compressor.py tests/agent/test_context_compressor_summary_continuity.py tests/agent/test_context_compressor.py tests/agent/test_resume_stale_active_task.py
   • git diff --check

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Not applicable.

2026-07-13 shepherding refresh

• Rebased the contributor-owned branch from 349 commits behind onto then-current origin/main, preserving contributor authorship/history. Main advanced by one commit afterward; GitHub still reports the PR cleanly mergeable.
• Repaired a protected-tail boundary bug found by the private maintainer review: live tail turns before a fossil handoff now remain verbatim instead of entering the lossy summarizer input, fossil summaries cannot satisfy the surviving-user guard, and fallback telemetry counts only messages actually summarized.
• Added deterministic regressions for protected-tail preservation, user-role fossils, an all-fossil tail, and abort-then-retry wide self-healing. The restart probe's containment invariant, retry behavior, reserved marker semantics, and emitted-tail telemetry are now documented in code.
• Focused proof on final head 785783af4:
  • scripts/run_tests.sh tests/agent/test_context_compressor_summary_continuity.py tests/agent/test_context_compressor.py tests/agent/test_resume_stale_active_task.py -- -q -p no:cacheprovider — 177 passed.
  • .venv/bin/ruff check agent/context_compressor.py tests/agent/test_context_compressor_summary_continuity.py — passed.
  • .venv/bin/python scripts/check-windows-footguns.py agent/context_compressor.py tests/agent/test_context_compressor_summary_continuity.py — passed.
  • git diff --check — passed.
• Private cross-review: grogu confirmed the central bounded-probe/self-heal design and its load-bearing containment invariant; all concrete edge-case hardening requested in the completed pass was added. Mario was unavailable because the local Codex runner rejected its configured max reasoning value.
• GitHub Actions run 29300630299 is fully green: attribution/history, Ruff+ty, Windows, e2e, all eight test slices, supply-chain scans, both Docker architectures, and the aggregate required-check gate passed.
• Duplicate-work status is unchanged: issue [Bug]: protect_first_n decay state is in-memory — a gateway restart re-protects old compaction summaries, head grows unboundedly #57814 remains open and the timeline still points only to this implementation.

[teknium1]

Thanks for the thorough restart-decay coverage. The core premise is verified on current main: agent/context_compressor.py:2573-2588 derives decay only from in-memory state, while the persisted-handoff rehydration happens after head-boundary computation at agent/context_compressor.py:3066-3108.

Problems

• agent/context_compressor.py:3141 does not retain the full self-heal scan after an abort. The wide scan populates _previous_summary at :3153-3157, but an abort returns the original transcript at :3229-3265. The retry then fails the not self._previous_summary condition, scans only through compress_end, and clears that state at :3169-3175 if the fossil was in the protected tail. The existing abort/retry test uses a handoff inside the normal scan window, so it does not cover this path.

Suggested changes

• Scan the full resumed transcript whenever compression_count < 1, or restore rehydrated state on abort; add the beyond-compress_end abort/retry regression.

Automated hermes-sweeper review.

[teknium1]

This condition makes the abort/retry guarantee false for a fossil beyond compress_end: the first wide scan sets _previous_summary at lines 3153-3157, an abort returns the transcript unchanged, and retry skips the wide scan because _previous_summary is no longer empty. Scan on compression_count < 1 alone, or restore the prior state on abort, and add a protected-tail fossil abort/retry regression.

[harjothkhara]

Good catch — confirmed and fixed. The abort/retry path did strand the rehydrated state.

Root cause: the first-compaction self-heal scan populates _previous_summary from a beyond-window fossil, but the auth/network/abort_on_summary_failure abort returned the transcript unchanged without rolling that back. The retry (still compression_count == 0, now with a truthy _previous_summary) took the narrow rescan through compress_end, missed the tail fossil, and then discarded the rehydrated summary as cross-session leakage — copying the fossil forward as a stacked summary, exactly defeating this PR's restart-decay fix.

Fix (d556501): snapshot _previous_summary before the scan and restore it on the abort branch, so an aborted compaction is a true no-op and the retry re-runs the full first-compaction scan. Went with the "restore rehydrated state on abort" option since it also closes the discard path, not just the scan window.

Added test_restart_fossil_survives_summary_abort_then_retry — beyond-window fossil, first attempt aborts, asserts _previous_summary is rolled back and the retry still folds the fossil to a single summary. Red before the fix, green after; full compressor suite green. Also rebased onto current main.

[teknium1]

Merged via #69342 (commit 76e17bc). Your 12-commit series was squash-reapplied with your authorship preserved (the branch was 1210 commits behind — per-commit cherry-pick wasn't feasible). Follow-ups adapted test pins to task-snapshot grounding and composed your decay scan with the merged-handoff unwrap contract. Thanks!