fix(compression): stop an aborted rotation from growing the parent it could not publish

[jackulau]

What does this PR do?

The compression rotation path flushes its un-persisted transcript to the parent (#47202) and then publishes:

try:
    agent._flush_messages_to_session_db(messages, conversation_history=persisted_history)
except Exception:
    pass  # best-effort — don't block compression on a flush error
...
agent._session_db.publish_compression_child(...)   # can still refuse

The abort handler rolls back the in-memory transcript and keeps agent.session_id on the parent. Its own comment says "keep the parent live and discard the stale compacted snapshot" — but the rows the flush just committed are not part of what it discards. Every failed rotation leaves the parent transcript longer than it found it, whatever the failure was.

Survivable once. Pathological when the failure is sticky: publish_compression_child refuses any parent whose row carries ended_at, nothing in this path clears it, so each auto-compaction appends another copy of the current turn to the transcript it was supposed to shrink. And the growth is then self-feeding — it satisfies this same file's

if _preflush_ok and isinstance(durable_parent, list) and len(durable_parent) > len(messages):
    messages = durable_parent

so the next attempt adopts the inflated snapshot as if it were genuine concurrent activity, and the in-memory transcript doubles too.

Three consecutive auto-compactions on such a parent, from the regression test's own log:

attempt 1  messages=20  -> aborted (failure_class=session_split_failed, "Compression parent already ended")
attempt 2  messages=20  -> aborted
attempt 3  compression: session=... grew before lease (20 -> 40 msgs); adopting durable snapshot
           messages=40->40  -> aborted

This checks that precondition before writing. A live parent reaches the flush exactly as before.

Why this is a safe place to check. It is a plain read of the row publish_compression_child is about to read anyway, and it raises the publish's own message, so split_status=aborted, failure_class=session_split_failed, the "Compression rotation aborted and rolled back to the parent session" warning and the in-memory rollback are byte-identical to today's post-publish abort. old_session_id moves above the flush for that reason: the except handler keys its rollback off that name, so hoisting it means a failure raised from here rolls the transcript back instead of leaving the failed attempt's compacted snapshot in place.

Deliberately not extended to the compression lease. A lease is re-acquirable and its loss is transient, so pre-checking it would abort rotations that would otherwise have committed. The ended-parent check is the opposite shape — it is sticky by construction, which is exactly what makes it worth paying for up front.

Related Issue

Refs #88197

Reported by @mayqhw: a live session went from ~52% to 166% of context in ~15 minutes and hit 400 invalid_request_error, with 303 unique messages stored as 2,611 rows (88% duplicate) after 7 aborted compression attempts. Their ended_at came from tui_gateway/server.py::_shutdown_sessions() stamping end_reason="tui_shutdown" on a still-active session when the TUI server auto-reloaded.

Scope — this is one of the two halves, and not the one in the title of the issue. It does not fix what marks a live session as ended. An affected session still aborts every attempt; it just stops making itself larger while it does. That half needs a maintainer decision (should _shutdown_sessions() stamp at all, should the publish guard read end_reason the way _check_transcript_write_guards does, or should the attach path reopen the way all eight reopen_session call sites do?) and the analysis is on the issue rather than in this PR.

Relationship to #85141, which is open against the same two files: it introduces _LINEAGE_CLOSED_END_REASONS and makes the write guard use it, deliberately leaving hygiene ends writable. tui_shutdown is a hygiene end by that taxonomy and #85141 does not touch the publish guard, so the two do not overlap — but #85141 reports this same amplification independently (25k → 126k messages on a /new parent), which is the second production sighting of an aborted rotation inflating the row it aborted on. That is the part this PR generalises: it holds for any abort cause, including the ones #85141 is fixing.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

• agent/conversation_compression.py — in the rotation branch: hoist old_session_id = agent.session_id above the Context compression silently loses unflushed messages (end_session without flush) #47202 flush, and read the parent row before flushing. If it already carries ended_at, raise the publish's own RuntimeError so nothing durable has been written. Fails open on an unreadable row — a cheap guard must not become a new way to lose compression.
• tests/agent/test_compression_rotation_state.py — TestAbortedRotationDoesNotGrowParent, 3 tests, driving the real _compress_context against a real SessionDB.

No behavior change for in-place compaction, for a rotation whose parent is live, or for any other publish failure (lease loss, busy, ambiguous lineage) — those still abort after the flush exactly as before.

How to Test

python -m pytest tests/agent/test_compression_rotation_state.py -q -k AbortedRotation

3 passed.

Mutation check, one rule at a time:

Mutation	Result
Never raise (if False and _parent_already_ended:) — i.e. today's behavior	1 failed — attempt 3 appended to the parent it could not publish, and the log shows the adoption firing at 20 -> 40 msgs
Always raise (if True:) — skip the flush unconditionally	2 failed — the live parent loses its #47202 tail, and the fail-open case stops rotating
Fail closed on an unreadable row (except Exception: _parent_already_ended = True)	1 failed — a transient read error becomes a lost rotation

Each mutation kills exactly the test that owns the rule, and no others.

Neighbouring suites, all on this branch:

pytest tests/agent/test_compression_rotation_state.py \
       tests/agent/test_compression_concurrent_fork.py \
       tests/agent/test_compression_orphan_recovery.py \
       tests/agent/test_compression_adoption_preserves_live_tail.py \
       tests/agent/test_compression_attempt_telemetry.py \
       tests/agent/test_preflight_compression_gate.py -q
  -> 73 passed

pytest tests/agent -q -k "compress or compact or rotation"
  -> 1 failed, 529 passed, 1 skipped

The one failure is test_compression_review_76354.py::TestF6ExecutorSaturation::test_cancelled_fence_skips_summary_work_before_start, and it is pre-existing and unrelated — verified by stashing both of my files and re-running it on pristine main (b20229312), where it fails identically with failure_class=commit_fence_cancelled.

ruff check on both files: clean.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate — issue timeline (no cross-references), PR search for 88197 and publish_compression_child, and a topic sweep of all 21,600 open PR titles for compaction/abort/duplicate/rotation terms. The nearest hits are fix(state): stop compressing /new parents after session_reset #85141 (different guard, different end-reason class — relationship spelled out above), fix(agent): adopt live compression continuation when flushing to a closed session #86409 and fix(agent): adopt live compression child on flush #79763 (both about adopting a live child when a flush hits a closed session, not about the rotation's own pre-publish write), and fix(compression): stop archive_and_compact replay duplicates and mark iteration-limit summaries terminal #87561 (archive_and_compact replay duplicates — the in-place path, which this PR does not touch).
• My PR contains only changes related to this fix (no unrelated commits)
• I've run pytest tests/ -q and all tests pass — ran the compression/rotation subset rather than the whole suite; results and the one pre-existing failure are in How to Test
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: Windows 11

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — the guard carries its own reasoning inline, including why the lease is excluded
• I've updated cli-config.yaml.example if I added/changed config keys — N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — the change is pure Python against SessionDB with no path, process or filesystem behavior. The reported incident is macOS and I verified the DB-level chain on Windows 11, so the fix is platform-neutral in both directions
• I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

From the issue, agent.log on the affected session:

compression attempt: failure_class=session_split_failed
  "Compression parent already ended: 20260817_100814_a25286"
  x7 between 13:28 and 13:52
303 unique messages -> 2,611 rows (88% duplicate)
522K tokens -> 1,783,431 -> request 1,663,655 -> 400 (max 1,048,576)

After this change those seven attempts still abort — the parent is still marked ended — but the row count stays at 303 and the session stays usable instead of bricking at the context limit.

[kshitijk4poor]

Merged via #88411. Your commit was cherry-picked onto the latest main with authorship preserved (rebase-merge).

Great find — the mutation table and the fail-open/fail-closed analysis were thorough. The fix is correct and well-tested.