fix(compression): preserve snapshot role alternation

[fangliquanflq]

Summary

• fold TODO snapshots into synthetic user tails without creating adjacent user messages or changing synthetic provenance
• separate durable TODO payloads from ephemeral retry, verification, and dropped-tool scaffolding at cleanup and compaction boundaries
• preserve strict user/assistant and tool-call/result sequencing across in-place persistence, rotated sessions, repeated verification, and last-resort salvage
• retain real-user provenance and structured-content behavior while removing stale synthetic instructions

Testing

• scripts/run_tests.sh tests/agent/test_compression_rotation_state.py -k TodoSnapshotScaffoldingTails (16 passed)
• scripts/run_tests.sh tests/agent/test_verification_continuation_budget.py tests/agent/test_salvage_grown_transcript.py tests/agent/test_message_sequence_repair.py (75 passed)

Related Issue

Closes #116880

[kvnloo]

[P1] Do not fold the durable TODO snapshot into ephemeral scaffolding while retaining the scaffolding flag.

The new else branch handles any trailing synthetic user row, including _empty_recovery_synthetic, _verification_stop_synthetic, _pre_verify_synthetic, and _dropped_toolcall_nudge, by appending the TODO text and preserving that flag.

Those flags have stronger downstream semantics than “not a real human turn”:

• agent/session_persistence.py::_EPHEMERAL_SCAFFOLDING_FLAGS makes the entire row non-durable. _todo_snapshot_synthetic is deliberately NOT in that tuple.
• agent/turn_final_response.py pops a trailing row carrying any of those ephemeral flags before the final response is appended.

So, for the added test's exact shape:

{"role":"user",
 "content":"Continue after recovering...",
 "_empty_recovery_synthetic": true}

this PR folds the live TODO snapshot into that row, leaves _empty_recovery_synthetic=true, and the next persistence/finalization boundary can skip/pop the snapshot together with the recovery nudge. The role-alternation assertion goes green, but the post-compression TODO-retention contract can be lost.

The old comment's “scaffolding tails must not absorb it” was load-bearing for more than provenance: the two payloads have different lifecycles.

Please keep those lifecycles separate. The regression should drive a real ephemeral-tail lifecycle, not only call _fold_todo_snapshot twice:

1. start with an _empty_recovery_synthetic (and ideally verification-nudge) user tail;
2. fold a pending TODO snapshot;
3. run the production persistence/final-response cleanup boundary;
4. assert the ephemeral nudge disappears but the TODO snapshot remains represented in the durable/live post-compression context, with strict role alternation preserved.

I would not clear the ephemeral flag just to keep the row: that would turn internal retry/verification text into durable user evidence, which #69292/#65919 explicitly prevent. The fix needs a sequence shape that preserves both strict alternation and the distinct durable-vs-ephemeral ownership of the two pieces.

[kvnloo]

The production change is broader than the new regression, so there is one useful test-shape upgrade before this becomes a strong salvage atom.

The new else runs for any trailing role=user row that _is_real_user_message() rejects.

That predicate is not just _empty_recovery_synthetic. Current head rejects synthetic users through:

• five explicit flags in _SYNTHETIC_USER_FLAGS;
• the _SYNTHETIC_USER_PREFIXES family;
• ContextCompressor._is_synthetic_compression_user_turn(...).

The new regression proves only the _empty_recovery_synthetic branch.

The smallest stronger invariant would parameterize representative synthetic categories, not add more implementation:

1. a flagged scaffold such as the existing empty-recovery row;
2. a prefix-only synthetic row;
3. a compressor-classified synthetic user row if that is a distinct shape.

For each case, assert:

• the original synthetic provenance remains authoritative;
• the TODO header appears exactly once after repeated folds;
• the resulting row is still not a real user according to _is_real_user_message;
• no adjacent user,user pair is emitted.

That pins the actual contract this hunk changes: every synthetic user tail may absorb the snapshot without becoming human intent or breaking role alternation, rather than only one current producer of synthetic rows.

[kvnloo]

Follow-up after the new commits: the broader synthetic-tail coverage exposed a real dependency that changes the salvage shape of this PR.

6b91b0e71219228106810c9df82516bc86cbf85c is not safe as a standalone donor atom anymore.

Its core fix folds the TODO snapshot into an ephemeral synthetic user row to preserve strict role alternation. But those retry/verification rows are intentionally removed later by persistence/final-response cleanup. Without the later work, the fix can therefore preserve alternation while silently dropping the durable TODO state.

The two follow-up commits are correctness dependencies, not optional cleanup:

• a2f3b504e1d9decb60f400b89b1d8fc0ae970764 preserves snapshot provenance across persistence;
• 41ca4a4ca9c979daae551d5b1271f0924c94bd7c separates the durable TODO payload from ephemeral retry scaffolding at cleanup/commit boundaries.

So the integration unit is now the three-commit chain (or a different simpler implementation that proves both invariants):

1. no adjacent user,user snapshot row;
2. ephemeral scaffolding can still disappear without taking the TODO snapshot with it.

The new parameterized synthetic-category regression plus test_ephemeral_tail_cleanup_preserves_durable_snapshot is the right evidence pair for that combined atom.

It would also help to update the PR summary before merge/carrierization, because the current body still reads like a two-file role-alternation fix while the branch now changes the durable compaction/persistence contract across five files.

[fangliquanflq]

Addressed all three points.

• Ephemeral lifecycle: the TODO payload is now projected to durable TODO-only state at persistence/rotation boundaries, while final-response, session-persistence, and verification cleanup remove retry/verification scaffolding without dropping the snapshot. Cleanup also rewinds paired tool-call/result rows when required and preserves strict role alternation.
• Synthetic-tail coverage: the regression now covers explicit flags, prefix-only scaffolding, and compressor-classified synthetic users. It verifies preserved synthetic provenance, one TODO header after repeated folds, non-human classification, and no adjacent user rows. Lifecycle tests additionally cover empty-response and dropped-tool nudges after real tool chains, plus repeated verification candidates.
• PR scope: the PR summary and test section now describe the durable compaction/persistence contract rather than only the original two-file alternation fix.

Validation:

• scripts/run_tests.sh tests/agent/test_compression_rotation_state.py -k TodoSnapshotScaffoldingTails — 16 passed
• scripts/run_tests.sh tests/agent/test_verification_continuation_budget.py tests/agent/test_salvage_grown_transcript.py tests/agent/test_message_sequence_repair.py — 75 passed