fix: dedupe persisted compaction handoffs

[WXBR]

Summary

Fixes a recompression path where a persisted [CONTEXT COMPACTION] handoff can survive in the protected head/tail while the compressor also emits a fresh handoff. Repeated compression then carries duplicate handoff blocks forward instead of keeping only the folded summary.

The fix removes stale handoff-only messages during compression assembly. When a handoff was merged into a real tail message, it removes only the stale summary wrapper and preserves the original tail content.

Current merged-handoff format

Current main serializes merged handoffs as:

1. _MERGED_PRIOR_CONTEXT_HEADER
2. original prior-tail content
3. _MERGED_SUMMARY_DELIMITER
4. compaction summary
5. _SUMMARY_END_MARKER

The cleanup now parses that ordering and preserves everything before _MERGED_SUMMARY_DELIMITER except the wrapper header. It supports both string content and multimodal content blocks. Persisted legacy handoffs using the older summary + end marker + real tail ordering remain supported as well.

Why nearby compaction fixes do not cover this

This is related to the same broad compaction-handoff symptom family as #6212, #42830, #42895, and #43184, but it is a different root cause:

• fix(agent): keep compaction summaries as user role (#42768) #42830 changes role/marker behavior for emitted handoff summaries.
• fix(agent): historicalize persisted compaction handoffs #42895 historicalizes persisted handoff headings so stale tasks are not treated as current work.
• fix(tools): filter compaction summaries from session_search bookends and cap content length #43184 filters compaction summaries from session-search bookends.
• This PR handles recompression assembly itself: a stale handoff in a protected region must not be serialized into the newly compressed transcript again.

Changes

• Adds _strip_context_summary_handoff_message() to drop stale handoff-only messages.
• Preserves current delimiter-based merged tail content while removing the embedded old summary.
• Preserves legacy string and multimodal merged-tail formats already persisted in older sessions.
• Applies cleanup to protected head and tail messages before role/alternation decisions.
• Computes surviving-user and merge decisions from the cleaned message lists.
• Adds end-to-end recompression coverage asserting:
  • the real prior tail survives exactly once;
  • the stale summary is absent;
  • exactly one fresh handoff remains.
• Adds current and legacy multimodal preservation tests.

Validation

RED/GREEN against current origin/main (bb5fc723b):

• RED: the current delimiter-format recompression regression failed because CURRENT-MERGED-OLD-SUMMARY remained in the compressed output.
• GREEN: after the fix, the prior tail survives once, the old summary is absent, and only the fresh handoff remains.

Commands run:

PYTHONPATH="$PWD" ~/.hermes/hermes-agent/venv/bin/python -m pytest \
  tests/agent/test_context_compressor.py \
  tests/agent/test_context_compressor_summary_continuity.py \
  tests/agent/test_context_compressor_temporal_anchoring.py \
  tests/agent/test_context_compressor_session_end_clears_state.py \
  tests/agent/test_context_compressor_cross_session_guard.py \
  -q -o 'addopts='
# 172 passed

~/.hermes/hermes-agent/venv/bin/python -m ruff check \
  agent/context_compressor.py \
  tests/agent/test_context_compressor_summary_continuity.py
# All checks passed

git diff --check

[tonydwb]

Code Review Summary

Verdict: Approved

What was changed

Fix deduplication of persisted compaction handoffs. Previously, re-compression cycles would keep old handoff-only messages and accumulate duplicate [CONTEXT COMPACTION] markers. The new _strip_context_summary_handoff_message removes or unwraps these stale handoff messages during compression.

Observations

The _strip_context_summary_handoff_message classmethod correctly identifies summary messages, finds the _SUMMARY_END_MARKER, and either unwraps to reveal the content beneath or drops the message entirely if it was handoff-only. The fix is well-scoped to the context compressor.

Testing

Existing compression tests should cover this edge case. The fix adds a clear new code path for stripping, which is correctly integrated into the compress loop.

Security

No concerns.

---

Reviewed by Hermes Agent

[teknium1]

Thanks for isolating the protected-handoff duplication path. The current-main premise is still present: compress() rehydrates a handoff at agent/context_compressor.py:2957-2966, then copies the protected head at agent/context_compressor.py:3063-3073 before adding a new summary.

Problems

• agent/context_compressor.py:1754 assumes a merged handoff has tail text after _SUMMARY_END_MARKER. Current main deliberately writes merged content as prior tail text, _MERGED_SUMMARY_DELIMITER, summary, then the end marker (agent/context_compressor.py:3176-3189, a1a8a967e). This helper would therefore return None and discard the preserved tail message.
• The added merged-prefix test covers only that older ordering, not the current delimiter-based format recognized by _strip_summary_prefix() at agent/context_compressor.py:2233-2249.

Suggested changes

• Parse the current merged-tail delimiters and preserve the tail segment while removing the old embedded summary/metadata.
• Add a recompression test against that current serialization, asserting preserved tail text and exactly one fresh handoff.

Automated hermes-sweeper review.

[teknium1]

Current main serializes merged tail summaries as prior tail content + _MERGED_SUMMARY_DELIMITER + summary + this end marker (agent/context_compressor.py:3176-3189, a1a8a967e). This therefore has no remainder and returns None, dropping the preserved tail message. Parse the current delimiters and retain the tail segment instead.

[WXBR]

Addressed the current-main review in 055174454 and rebased the PR onto current main.

You were right about the serialization mismatch: the previous helper assumed real tail text followed _SUMMARY_END_MARKER, but current merged handoffs put the real prior tail before _MERGED_SUMMARY_DELIMITER and the end marker last.

The revised cleanup now:

• preserves the prior-tail segment before _MERGED_SUMMARY_DELIMITER;
• removes only the wrapper header and embedded stale summary;
• preserves current string and multimodal block content;
• retains compatibility with older summary + end marker + real tail handoffs;
• bases role/alternation decisions on the cleaned head and tail lists.

The new current-format regression failed on clean bb5fc723b with the old summary still present, then passed after the fix. The five adjacent compressor suites pass: 172 passed, with Ruff and git diff --check clean.

[teknium1]

Merged via #69297 (commit 5a3ee3c). Your commit was cherry-picked with authorship preserved — your marker predicates matched current emission exactly (verified live). A follow-up composed your unwrap logic with the newer head-copy skip and multi-fossil scan. Thanks!