Skip to content

fix(agent): strip _db_persisted on rotation compression assembly (salvage #57531, #57491) - #57574

Merged
kshitijk4poor merged 2 commits into
NousResearch:mainfrom
kshitijk4poor:salvage-57491
Jul 3, 2026
Merged

fix(agent): strip _db_persisted on rotation compression assembly (salvage #57531, #57491)#57574
kshitijk4poor merged 2 commits into
NousResearch:mainfrom
kshitijk4poor:salvage-57491

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Summary

Salvage of #57531 (@nankingjing) — cherry-picked to preserve authorship, rebased clean onto current main, and independently verified end-to-end. Fixes #57491.

Rotation compression (compression.in_place: false) on long-lived cached gateway sessions silently drops the compacted transcript from state.db. On restart the child session reloads a near-empty transcript → severe amnesia. Regression introduced by #50372 (e4c6d1b22), which switched flush dedup from id(msg) to an intrinsic _db_persisted marker.

Root cause: ContextCompressor.compress() assembles the post-rotation list with shallow messages[i].copy() for the protected head (context_compressor.py:2837) and tail (:2910). On a cached agent every live message already carries _db_persisted (stamped by incremental flushes), and .copy() propagates it. The child-session flush (run_agent.py:1820, if msg.get(_DB_PERSISTED_MARKER): continue) then skips every copied row — only brand-new post-compression appends reach the child DB.

Fix: strip _db_persisted when copying head/tail into the compressed transcript, via a small _fresh_compaction_message_copy() helper applied at both assembly sites. In-place compaction is untouched.

Independent verification (not just green units)

Reproduced against the real _flush_messages_to_session_db + real SessionDB on a temp HERMES_HOME, both variants:

Variant Unfixed main With fix
Standalone summary child gets 2/9 rows 9/9
_merge_summary_into_tail (summary folded into a marked tail dict) child gets 1/8 rows, 0 [CONTEXT COMPACTION] rows 8/8, summary present ✓

The merge-into-tail row exactly matches the WeChat production evidence in the issue ("Child #51: 17 messages, 0 rows containing [CONTEXT COMPACTION]") — the worst case where the summary itself is lost. The fix covers it because the tail copy is now stripped before the summary is merged into it.

Chokepoint check: the two post-assembly passes (_sanitize_tool_pairs, _strip_historical_media) operate on the already-stripped compressed list and never re-copy from the original messages, so sites 2837/2910 are the complete chokepoint — no marker can re-enter.

Mutation-tested: neutering the strip (revert helper to plain .copy()) makes test_rotation_child_session_flushes_full_compressed_transcript_with_markers fail — the regression test genuinely guards the fix.

Note on the local constant: the helper redefines _DB_PERSISTED_MARKER locally rather than importing it from run_agent — importing back would be circular (run_agent imports context_compressor at import time). Consolidating both literals into a shared constants module is a larger refactor out of scope for this bug.

Tests

  • tests/agent/test_context_compressor.py — unit: assembled compressed list carries no _db_persisted
  • tests/run_agent/test_compression_persistence.py — e2e: cached agent with all source messages pre-marked → rotation → child receives the full compressed row count
$ pytest tests/agent/test_context_compressor.py tests/run_agent/test_compression_persistence.py \
         tests/agent/test_context_compressor_session_end_clears_state.py \
         tests/agent/test_context_compressor_summary_continuity.py \
         tests/agent/test_context_compressor_cross_session_guard.py -q
164 passed

Supersedes #57531 and #57508 (both independent fixes for the same root cause; credit to @nankingjing and @rayjun).

nankingjing and others added 2 commits July 3, 2026 12:32
…transcript (NousResearch#57491)

Shallow messages[i].copy() during context compression propagated the
_db_persisted marker from cached gateway incremental flushes into the
post-rotation compressed list. _flush_messages_to_session_db then skipped
every row when writing to the new child session, so gateway restarts
lost the compacted transcript (severe amnesia).

Strip the marker in _fresh_compaction_message_copy() and add regression
tests for rotation flush + compressor assembly.

Fixes NousResearch#57491
…ep (NousResearch#57491)

Follow-up to the per-site strips from the review gate. The two copy-site
strips are correct but positional — a copy site added after the assembly
loops would re-leak _db_persisted into the child-session flush. Add a single
terminal sweep (_strip_persistence_markers) run once on the fully-assembled
compressed list so the invariant 'no compacted message leaves compress()
carrying a persistence marker' is structural, not dependent on copy-site order.

- agent/context_compressor.py: _strip_persistence_markers() called before
  compress() returns; helper docstring notes the sweep is the authoritative guard
- tests/agent/test_context_compressor.py: structural regression — neuter the
  per-site helper to a leaking copy, assert the terminal sweep still strips
- tests/run_agent/test_compression_persistence.py: pin the fixture assumption
  behind the exact-equality row-count assertion
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P1 High — major feature broken, no workaround sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state duplicate This issue or pull request already exists labels Jul 3, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #57508 (canonical, earliest of the same-mechanism cluster). This PR is a salvage of #57531 and fixes the same issue #57491 via the same mechanism (strip the _db_persisted marker from messages[i].copy() in context_compressor.py compressed-transcript assembly). Cluster fixing #57491: #57508 (earliest, 03:57), #57531 (salvage source, 04:49), #57574 (this, salvage). Maintainer to pick canonical; related family salvage: #48584.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P1 High — major feature broken, no workaround sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Rotation compression: _db_persisted marker copied via messages[i].copy() skips bulk flush to child session (cached gateway regression since #50372)

3 participants