feat(session-search): treat compression continuations as one conversation (lineage-aware recall)

[Skywind5487]

What does this PR do?

The bug class. Search results currently conflate generic parent ancestry with compression identity. A long conversation that context-compression rotated into a chain of physical session rows (root -[compression]-> mid -[compression]-> tip) is either merged with unrelated sessions — because any parent_session_id edge is treated as conversation identity — or split into fragments that recall misses: a match living only in a compressed continuation is dropped when the query is absent from the root transcript (upstream #6507). The same _discover seam already fails for /new-reset recall (#85756, fixed by #86652) and own-early-context recall (#90939, open #90974); this PR closes the compression-continuation half and makes the fresh-reset predecessor discoverable at the winner level.

The invariant. Only positive compression-continuation edges define one searchable conversation. The exact edge condition: the parent exists, the parent ended with end_reason='compression', the child is not a tool session, and the child's _branched_from/_delegate_from markers do not explicitly point at that parent (foreign markers pointing elsewhere do not disqualify). Generic parent ancestry (branches, delegates/subagents, tool/reset children) is not conversation identity. Resolution is fail-closed for missing parents and proven cycles, and query-bounded (exact B=2000 successful uncached-row-fetch budget, early-K stopping) with no semantic depth cap — a 10k-deep chain resolves correctly, not by capping depth.

Concrete behavioral contracts (from the implementation)

• Budget accounting is precise. B=2000 counts exactly one successful uncached lineage-node row fetch. Memo hits, absent-row fetches, and traversal-local cycle checks consume nothing; the archive-metadata batch queries (owner end_reason + per-message compacted) are not lineage-node fetches and do not count toward the budget.
• Budget exhaustion is conservative and surfaced, never silent. When a candidate's resolution would exceed the budget, the entire ranked scan stops — the bound candidate may own a higher-ranked root than every later candidate, so skipping it would violate ranking. The result is surfaced as truncated: true plus a warning ("results are a safe ranked prefix and may be incomplete") and stats.lineage_bound_hit; an exhausted partial path is operational uncertainty, not semantic evidence, so it is never memoized as unresolved and never fabricated into a root.
• Live-context visibility check. A hit whose content has left live context stays discoverable even when it belongs to the current lineage: owner ended by compression, owner has a fresh-reset end reason, or the matched message row is in-place-compacted (compacted=1).
• Fresh-reset predecessors stay discoverable. The winner phase threads out_of_context_end_reasons (reset set + new_session) so a /new-style predecessor of the current session is not hidden (session_search discovery hides ALL hits from the current session's own /new-reset lineage (gateway recall goes blind) #85756) — complementing fix(session-search): recall /new-reset sessions in the current lineage (#85756) #86652 at the winner level.
• Ranking and anchors are preserved. Winners follow the rank of each owner's first displayable anchor; a live current-lineage hit is skipped so a later compacted hit of the same owner can surface, but that owner never jumps ahead of a higher-ranked displayable winner from another session. The matched anchor keeps its raw owning session_id — it is never rewritten to the root.
• Current-session unresolved handling is conservative. An unresolved current session excludes only its own id, never broadening exclusion to an unproven ancestor.
• _scroll is deliberately unchanged in identity semantics. Scroll's guard is about live context, not search lineage: a live delegation/branch parent is still in the active context even though it is a distinct compression root, so scroll keeps generic parentage (fix: prevent false positives in recursive delete detection #68). This PR changes only discovery/winner semantics.

Snapshot / concurrent-writer note (explicit)

The winner phase runs inside a single read context + BEGIN read transaction (archive metadata, lineage resolution, and current-lineage ancestors; rolled back on exit). Candidate discovery (search_messages) happens on the existing connection model before that transaction, so the discover→resolve sequence is not one atomic snapshot; concurrent writers are not blocked. Missing parents and proven cycles are fail-closed at read time.

Implementation choices supporting that contract

Query-local resolver memo + path compression (a memo hit path-compresses the visited prefix to the known root); winner dedupe, current-session exclusion, and exact-title exclusion all using the same compression-root meaning (the exact-title slot is pre-resolved with the same resolver); preservation of the highest-ranked anchor per conversation; deferred expensive hydration; _chunked (size 400) batch reads for SQLite var-limit safety; SESSION_WINNERS_LINEAGE telemetry logging candidate/winner/work/memo/bound-hit counts so truncation is observable.

Related Issues

• Bug: session_search drops child/continuation-session hits by summarizing only the lineage root #6507 [open] — session_search drops child/continuation-session hits by summarizing only the lineage root. This PR's compression-root composition is the compression-continuation half of this bug class: it resolves and composes the full lineage, keeps the matched anchor (never rewrites to root), and summarizes from the lineage transcript, not just the resolved root.
• session_search discovery hides ALL hits from the current session's own /new-reset lineage (gateway recall goes blind) #85756 [closed, fixed by fix(session-search): recall /new-reset sessions in the current lineage (#85756) #86652 merged] — /new-reset lineage recall. The /new case (upstream fix(session-search): recall /new-reset sessions in the current lineage (#85756) #86652) and the compression case (this PR) are two halves of the same "same-lineage content is already in context" guard in _discover; this PR also makes the fresh-reset predecessor discoverable at the winner level via out_of_context_end_reasons.
• session_search can't recall the current session's own early context (long live session, term appears verbatim, 0 results) #90939 [open, fix(session_search): recall the current session own early context (#90939) #90974 open] — current session's own early context. Same _discover seam; complementary.

Overlap disposition (classified by contract, not file overlap)

Prior art — closed/unmerged; this PR is the first to land the contract

• fix(search): surface compaction-archived and compression-parent sessions in discovery #63144 — surface compaction-archived and compression-parent sessions in discovery (discovery half partially salvaged by fix(search): surface compacted sessions in discovery #69544, merged).
• fix: improve session_search lineage recall #6536 — improve session_search lineage recall.
• fix(session-search): read content from matched session #23561 / fix(session_search): load transcript from matched child session, not resolved root #20794 — read content from matched / child session (earliest Bug: session_search drops child/continuation-session hits by summarizing only the lineage root #6507 attempts).

Open upstream PRs — COEXIST (different surface, no equivalent invariant implemented)

• feat(desktop): add compression lineage navigation #81561 [open] — Desktop compression-lineage navigation UI, lazy-loads exact read-only segments. Non-search surface; notably shares the strict compression-continuation traversal edge definition (mutual validation).
• fix(state): opt-in logical lineage for JSON/JSONL session export #53992 [open] — export-path lineage stitching, reuses the same compression-only edge definition as get_compression_tip. Different surface (export), same edge semantics.
• Scope Telegram session_search to the current conversation #83829 [open] — scopes Telegram session_search to the current conversation; explicitly preserves compression-lineage recall. Different contract (scope).
• feat(cli): sessions list/search options, columns, pagination #75496 [open] — COEXIST (watch item). CLI /sessions list + sessions search UX with row-per-compression-chain dedupe. Its chain identity (compression-chain root, COMPRESSION_CHAIN_MAX_HOPS) matches this PR's positive-compression-continuation roots — not a conflicting identity — but its surface (listing/pagination/lane-scoping) is different and it does not implement this PR's bounded/fail-closed lineage reconstruction for the agent tool (no B=2000, no memo, no lineage summarization). If feat(cli): sessions list/search options, columns, pagination #75496 merges before this PR: stop mechanical rebasing, re-audit ownership of the lineage seams, and explicitly test whether its chain-identity semantics remain equivalent to compression-continuation roots. Its continued openness is not a veto on this PR.

Merged substrate (reused, not replaced)

#69544, #12960, #38393, #39062, #57685, #86652, #90357.

Test evidence

• tests/test_session_search_lineage.py — 37 tests: B=1999/2000/2001 boundary, 10k-deep chain, K=60 early-K, cycle/missing-parent fail-closed, current-session exclusion, anchor preservation, concurrent-writer lineage coherence.
• tests/tools/test_session_search.py — 51 tests.
• tests/test_hermes_state.py -k search — 19 state-search tests.
• Broad wrapper scripts/run_tests.sh: all three targeted files green. A full-suite run on a local Windows box is blocked by pre-existing environmental failures unrelated to this PR (POSIX-only modules termios/os.chown/AF_UNIX, missing anthropic SDK, broken aiohttp wheel, missing platform/API credentials); the run reached ~12k tests with zero failures in any file this PR touches. Upstream CI (Linux) is the authoritative broad gate.

Notes for maintainers

• Both fix(state): opt-in logical lineage for JSON/JSONL session export #53992 and feat(desktop): add compression lineage navigation #81561 independently use the same strict compression-continuation edge definition — strong cross-validation of this PR's identity semantics.
• Rebase note for fix(session_search): recall the current session own early context (#90939) #90974: both touch _discover in tools/session_search_tool.py; coordinate the compaction-escape-hatch handling (this PR replaces _is_compacted_message with the winner-phase _is_archived check, fix(session_search): recall the current session own early context (#90939) #90974 relies on the helper).
• Production code + tests only; no fork-internal docs.

Checklist

• Read the Contributing Guide
• Conventional Commits
• Searched for existing PRs — no duplicate (see Overlap disposition)
• PR contains only changes related to this feature (4 files, +1551/-98)
• Targeted suite via scripts/run_tests.sh — 107 passed (37+51+19)
• Broad wrapper run — see Test evidence (Windows env limits noted)
• Tests added (37 lineage tests + bound/cycle fixtures)
• Platform tested: Windows 11 (pending confirmation)

[Skywind5487]

Status update — ready for review.

• Rebased onto the latest upstream/main (40643cbaf9); the diff is the 4 intended files (+1551/-98).
• Local targeted suite via scripts/run_tests.sh (CI-parity wrapper): 107 passed — tests/test_session_search_lineage.py (37), tests/tools/test_session_search.py (51), tests/test_hermes_state.py -k search (19).
• Linux CI: full Python test suite (12 slices) + ruff/ty + OS-specific + attribution + OSV all green.
• Behavior contracts and overlap disposition (incl. feat(cli): sessions list/search options, columns, pagination #75496 watch note and fix(session_search): recall the current session own early context (#90939) #90974 seam coordination) are in the PR description above.

[Enough1122]

AI code review — automated review for reference, author can ignore or act on any point.

This is the right model and it is argued with unusual rigor: keeping compression-lineage identity (positive edges only — parent exists, parent ended by compression, not a tool child, branch/delegate markers not pointing at that parent) strictly separate from generic parentage, and then applying generic parentage deliberately where live-context semantics actually live (_scroll's guard and rebind, with in-place comments explaining why #68 does not reach there) is exactly the split that fixes both the over-merge and under-recall halves of the bug class. The boundary discipline is exemplary: memo-lookup → local-cycle-proof → budget-check ordering documented as correctness-significant, budget exhaustion never written to the memo ("operational uncertainty, never semantic evidence"), truncation surfaced as an explicit truncated + warning instead of a silently complete top-K, and tests pin the distinction between cycle-at-bound and bound-before-cycle as different outcomes — plus boundary budgets 1999/2000/2001, a 10k chain, oracle agreement, and coherent reads under a concurrent writer. Wrapping the winner phase in tool_error on crash (rather than falling back to raw results) is also correct: the current-context exclusion lives inside the winner phase, so an unfiltered fallback would leak live content.

Points:

1. The title slot was not migrated and now degrades open, not closed. _resolve_lineage changed to return Optional[str] (None on unresolved/budget-exhausted), but _title_match_result (~line 702) and _discover's pre-winner computation (~line 767) are untouched consumers. When resolution fails there: the same-lineage live-title guard if current_lineage_root and lineage_root == current_lineage_root is skipped entirely (falsy None), and excluded_lineage_roots=(title_lineage,) if title_lineage else () becomes empty — so a live in-context title match stops being hidden and a title-owned conversation can additionally surface in winner rows. That is precisely the dangling-parent/cycle/budget case this PR hardens everywhere else, failing open. Suggestion: thread the resolver's tri-state (root / proven-unresolved / budget-exhausted) into the title path like the winner phase does, and on unresolved fall back to conservative raw-id comparison (session_id == current_session_id) instead of dropping the guard.
2. Observability for dropped owners: candidates whose owner resolves to _UNRESOLVED are skipped from winners (if root is None: continue) per the stated fail-closed design, but stats has no counter for it. Adding e.g. lineage_unresolved_dropped to the SESSION_WINNERS_LINEAGE log line makes "FTS found the needle, why did the session disappear" answerable without a debugger.
3. Coverage note on the removed helper: deleting TestIsCompactedMessage alongside _is_compacted_message is fine because the behavior moved into the batched _is_archived, which discover-level tests exercise (test_lineage_winners_compacted_history_anchor_fallback, fresh-reset predecessor) — just flagging that the unit-level active=0/compacted=0 rewind-vs-compaction distinction now has no direct test anywhere; one assertion pinning that rewind rows (active=0, compacted=0) stay hidden would preserve that nuance cheaply.