[ljhade24]

What does this PR do?

Independent generic follow-up to #124988, based on recorded main snapshot at 5f045f842a60184748dda30acb9fecbd961cc18b. It adds caller-authored explicit-user deletion provenance and transactionally prepared minimal routing identities, exposed through public SessionDB receipt APIs for a plugin-owned durable backend-to-gateway handoff.

Draft / design decision requested: is a small generic transactional receipt store the preferred extension, and what retention/consumer-coordination policy should accompany it? The safe core piece is implemented and exercised; no synchronous plugin callback is run inside a live SQLite deletion transaction. If maintainers require a callback beyond the prepared-identity/read surface, its constrained writer/rollback semantics need agreement first. This draft is not a claim that the complete external cleanup/migration acceptance criteria are approved.

Related Issue

• Follow-up to fix(plugins): fire on_session_delete on every session-deletion path #124988 / [Feature]: Add an on_session_delete plugin hook (fire on every session-deletion path) #124511, not a competing observer implementation.
• Scope agreement: fix(plugins): fire on_session_delete on every session-deletion path #124988 (comment)
• Design question: fix(plugins): fire on_session_delete on every session-deletion path #124988 (comment)
• No code, commits or plugin-policy tables from a local paired-cleanup implementation are carried here. No overlap with the separate Desktop deletion-visibility PR.

Type of Change

• New feature (additive generic extension)
• Documentation and behavior-contract tests

Changes Made

• Actual selected-ID REST and session.delete RPC handlers set deletion_origin; no new wire parameter lets a client author its own provenance.
• Default/unknown origins, retention, empty sweeps, compensation and profile moves do not prepare external-cleanup evidence.
• After guard/fence checks, before row mutation, core snapshots exact removed IDs and minimal source/chat/thread/lineage identifiers. Receipt and deletion commit atomically; no-op and rollback leave no receipt. No messages, titles, summaries, model config, user names or credentials.
• Public bounded receipt replay, operation lookup, and explicit administrative reclamation. Store/profile scope derives from the actual database, not process launch state. Plugins own durable cursors, pending ledgers, opt-in and resource ownership checks, retries and external effects.
• The process-local post-commit observer in fix(plugins): fire on_session_delete on every session-deletion path #124988 is neither duplicated nor changed. Its ignored returns/fail-open dispatch must remain non-veto. No exactly-once external action guarantee.

Concrete consumer: an opt-in standalone plugin pairs an external messaging thread with stored conversation history and its own proven-owned files. It needs explicit user intent and routing identifiers even when Desktop commits in one process and the gateway consumes in another. That external policy is intentionally not added to core.

How to Test

Prepared an isolated PM Python 3.14 dev/test environment (python -m pm.build_env --source . --out .venv --group dev --group test), with a disposable development home and no copied credentials. Tested on macOS arm64.

scripts/run_tests.sh \
  tests/hermes_state/test_session_deletion_receipts.py \
  tests/hermes_state/test_delete_compression_chain.py \
  tests/hermes_state/test_delete_session_write_guards.py \
  tests/hermes_state/test_session_delete_profile_isolation.py \
  tests/hermes_state/test_session_lifecycle_status.py \
  tests/hermes_cli/test_session_deletion_provenance.py \
  tests/hermes_cli/test_web_routers_destructive_profile_scope.py \
  tests/hermes_cli/test_web_server_sessiondb_eventloop.py \
  tests/tui_gateway/test_session_deletion_handoff.py \
  -q --file-timeout 90 --file-retries 0 -j 2
# 97 passed, 0 failed

scripts/run_tests.sh tests/tui_gateway/test_tui_gateway_server.py \
  -q -k session_delete --file-timeout 90 --file-retries 0 -j 2
# 8 passed, 0 failed

python scripts/check --base 4808cabac34407bfc426dcef48515338be9c4160
# all 11 blocking checks passed

The cross-process test executes the real RPC handler against an actual temporary SessionDB, exits the backend abruptly after commit, then starts independent consumer processes which persist/restart a plugin-owned SQLite pending ledger using only public receipt reads. No external service is contacted.

Red-to-green observations: missing state keyword/API, missing REST provenance (zero receipts), missing RPC provenance in the cross-process handoff (zero receipts), and missing reclamation API. Contract tests additionally cover rollback, no-op/fences, active turn and compression guards, recursive delegate/compression IDs, profile isolation, real prune/move/empty paths, schema reconciliation, replay bounds and monotonic cursor reclamation.

Limitations / not green claims

• Broad tests/hermes_state/ run reported failures and exceeded the execution timeout; not a full-suite pass. Two representative failing files were re-run against untouched base: test_guest_durability_barriers.py (1 failure) and test_cross_vm_fs_wal_refusal.py (3 failures), reproducing the same failures. Other broad-run failures are not all baseline-classified.
• No Linux/Windows native execution or full website build.
• fix(plugins): fire on_session_delete on every session-deletion path #124988 was closed without merging. Closure reason/replacement has been requested; combined observer integration has not been tested or claimed.
• No automatic receipt expiry/consumer registry. Reclamation is administrative and must coordinate ALL consumers; the draft explicitly asks for a retention decision. Store restore/move/copy requires plugin scope/cursor reconciliation; this is not a distributed queue recovery protocol.
• CLI hermes sessions delete, interactive session browser deletion, and /exit --delete remain uninstrumented; intent must not be inferred from row absence.
• No arbitrary pre-delete callback, runtime plugin deployment, external deletion, production DB changes, restart, merge, or migration.

Checklist

• Read contribution/area rules and searched overlapping PRs
• Conventional commit; focused follow-up only
• Added contract docs/tests and considered profile/privacy/process boundaries
• Relevant focused tests and blocking lint passed
• Full suite passes (see limitations)
• Maintainer agreement on receipt storage/retention and readiness

Review follow-up — b8e04c9

• Receipt replay now uses _read_all / _read_retrying_ioerr. Regression coverage injects execute/fetchall transient IOERR and verifies replay on a real read-only connection; this is the primary read-only connection, not writable-reader-pool coverage.
• Missing receipt schema remains an explicit sqlite3.OperationalError: no such table: session_deletion_receipts for list/get; contract tests/documentation prevent interpreting schema-not-ready as empty replay.
• Registered the new docs page in website/sidebars.ts; named the three unsupported CLI deletion surfaces. Delegate-traversal optimization is deferred as a nonblocking performance nit. No transaction callbacks added.
• Exact nine-file focused scope: 97 passed; RPC deletion neighbors: 8 passed. Separate untouched base's six shared files: 68 passed. Official check: 11 blocking checks passed. Parent separately reran the receipt file: 26 passed. Source hashes matched before/after runs and the committed files match the tested artifact. Independent static patch review found no security/logic blockers.
• Isolated Python 3.11 execution of the exact unchanged typing import/Prelude alias reproduces Too few arguments for typing.Generator; actual 2, expected 3 on both this PR's source and recorded base. This is not a full Python 3.11 module import or pytest run; no unrelated compatibility fix is included.
• Initial test harness failures from inherited HOME are retained separately; reruns with disposable HOME passed without product-source changes. Full state-suite failures described above are not claimed resolved. No full website build, native integration or external cleanup deployment verified.
• Additional nonblocking review suggestions (writable pooled-reader retry coverage and persistent IOERR exhaustion) are not claimed covered. Draft and retention/consumer-coordination design questions remain. Maintainer workflow approval and review are still needed.

Conflict refresh — b72c6d5

Earlier test sections are historical pre-refresh evidence. This commit merges frozen upstream e06df9c2eae6901f4c88f464e0ae6e753c0428b8; published tree exactly matches the tested/reviewed candidate.

Retained upstream onboarding delete_chat wrapping/order and caller-authored user_rpc provenance, active-write protection and transactional receipts. No arbitrary transaction callback added. Two subprocess calls explicitly use check=False for current lint, preserving return-code assertions. Final 14-file regression: 140 passed; official scripts/check: 12 checks OK, health 0/0. Real onboarding RPC probe verified receipt commit before notification and no extra receipt on re-delete. Full state suite/website/native/plugin integration and retention coordination remain unverified; Draft retained.

Independent static review found no security/logic blockers; this does not imply the reviewer executed suites. PR-delta whitespace is clean; unrelated upstream whitespace findings were retained. Latest main d97007c32d6340f7ba79ac1dfb5773cc00846448 was separately checked with merge-tree (exit 0); its newer delta is release-tool scripts/tests. This is a conflict check, not runtime validation of a synthetic combined tree. Exact-head GitHub readback reported mergeable=true, mergeable_state=blocked; content conflicts are cleared, but review/design/workflow gates remain. No upstream PR merge, production code/database change, app edit or restart performed.

Recurrent conflict refresh — 27d9b69

Merged frozen upstream 7bda2a3368024f7ae1c70cef16583e7c2c891674. The REST conflict keeps upstream canonical authenticated/owner-fenced revision/generation/idle mutation instead of restoring the old direct writer. A minimal private Python _deletion_origin bridge preserves handler-authored explicit REST provenance through authority/runtime helpers; client payloads do not control provenance. State deletion captures final eligible local-reset/compression/delegate scope before retirement/deletion in the same transaction. Exact retry returns before preparation; mutation-receipt failure rolls preparation/deletion back together. No external cleanup executor, new plugin or transaction callback added.

Actual canonical regression: 27 files, 243 passed, 0 failed, 1 skipped (Linux-only lineage case skipped on Darwin). Blocking checks: 12 OK, health 0 blocking/0 advisory with explicit frozen upstream base. Exact published tree f07ddcd52750cedfccd0de9fda4a995c528762ff matches the tested candidate; tracked-source hashes were unchanged across final execution. Real HTTP regressions exercise live-owner/standalone receipts, stale/exact retry, profile isolation and rollback; state coverage includes reset lineage and queued-work refusal.

Independent static review of the full upstream-relative patch and surrounding authority/transaction helpers found no blocking security/logic defects. Nonblocking coverage suggestions remain: HTTP/WS-level provenance-spoof transport tests and a combined reset/compressed-delegate/surviving-branch receipt-vs-deleted-ID test. These are not claimed covered; reviewer did not execute suites. No full-suite/native integration or maintainer approval claim. Draft and retention/design/remote-CI gates remain.

[Finn763]

Thanks for the careful work here. The contract discipline is visible throughout: fail-closed provenance, no callbacks inside the write transaction, an additive surface, and docs that do not overclaim. I reviewed head 4808cab against the diff and ran the three new test files plus the delete-chain regression suite.

Test note, in the spirit of not softening: on a Python 3.11 environment, 23 of your 25 new tests pass. The two exceptions -- test_receipt_preserves_only_routing_identifiers and the cross-process handoff test -- fail because agent/turn_scripted_prelude.py:24 uses the PEP 696 two-argument typing.Generator[Y, S] form, which only parses on Python 3.13 or newer (the file is untouched by this PR; clean main reproduces it identically). With the semantically identical three-argument annotation, both pass, and the focused set is 34/34 green. Worth flagging separately to maintainers that requires-python (>=3.11) is out of sync with that file. Nothing in this PR causes it.

Findings, all small:

• list_session_deletion_receipts (hermes_state_deletion.py:64) reads via _read_ctx() directly, bypassing the _read_retrying_ioerr helper the other read accessors use; read-only pooled readers can hit the transient WAL disk-I/O window ([Desktop] Intermittent "Couldn't open <bot>'s chat": sqlite3.OperationalError: disk I/O error in read-only session DB FTS probe #100436 / [Bug] Repeated SQLITE_IOERR in SessionDB.get_session read path on WSL2 (ext4-on-vhdx), multi-process, starting at 0.21.0 — WAL stable but DELETE fallback collapses read throughput ~30000x #100871). Routing it through the _read_all-style retry would match house style.
• On a store whose writer has not reconciled the new table yet, the consumer's first read raises a bare sqlite3.OperationalError: no such table: session_deletion_receipts. Docs line 71 mention this abstractly; naming the exception (or normalizing it) would let plugin authors handle it deterministically.
• The new docs page is not registered in website/sidebars.ts, while sibling pages (application-declarations, catalog-submission) are.
• Scope: CLI user deletes (hermes sessions delete, the interactive browser, /exit --delete) still produce no receipts. The docs admit "not yet instrumented"; listing the surfaces or filing a follow-up issue would help consumers.
• Nit: the bulk path walks delegate children up to three times per transaction (guard pool, preparation, cascade); pass the collected set through if it grows.

On your open question: keep the receipt store; do not add a constrained transactional writer callback. A callback inside the deletion transaction either re-enters SQLite under the write lock or performs external work that a rollback can orphan -- which is precisely the coupling the observer was kept away from. The clean split is receipt = durable, fail-closed truth (this PR) and post-commit observer = best-effort wake-up (#124988), exactly as your docs frame it. If anything, keep constraining the receipt surface rather than growing a veto path.

Happy to iterate, and glad to help pressure-test the consumer cursor/reclaim protocol once maintainers weigh in.

[ljhade24]

@Finn763, thanks for the concrete review and for exercising the Python 3.11 path. I am addressing the receipt-reader retry path with regression coverage, the unreconciled-schema error contract, sidebar registration, and an explicit list of uninstrumented CLI deletion surfaces in an isolated checkout. I will retain the receipt-store design without transaction callbacks and keep this draft pending verification and maintainer decisions; the delegate-traversal optimization will stay separate unless needed for correctness.

I also noticed #124988 was closed without merging. Could you clarify the reason and whether a replacement or follow-up observer PR is planned? I will not assume the observer is available in main or fold your implementation into this PR. This draft remains a receipt/replay surface, with any wake-up integration separately verified.

[ljhade24]

@Finn763, the review follow-up is pushed as b8e04c9:

1. list_session_deletion_receipts now uses _read_all and the existing _read_retrying_ioerr path. Execute and fetchall transient-IOERR regressions went RED before the fix and GREEN afterwards. These tests exercise a real primary read-only connection, not the writable-reader pool; I am not claiming pool/exhaustion coverage.
2. Kept absent-schema failures explicit rather than silently returning an empty list. List/get tests and docs pin sqlite3.OperationalError: no such table: session_deletion_receipts; consumers must wait for writer schema reconciliation and must not infer that no deletion happened.
3. Registered the docs page in website/sidebars.ts.
4. Explicitly listed the unsupported CLI surfaces: hermes sessions delete, interactive session browser deletion, and /exit --delete.
5. Deferred delegate traversal optimization as a nonblocking performance follow-up; no callbacks or write/guard semantics changed.

Verification of the exact committed source on macOS/Python 3.14: nine focused files 97 passed, RPC deletion neighbors 8 passed, official 11 checks passed. Six shared regression files on the fixed separate recorded base 68 passed. I separately reran the receipt file (26 passed), verified source/patch hashes, and obtained independent static patch review with no security/logic blockers. Initial inherited-HOME harness failures were retained and rerun with disposable HOME; they are not product fixes or full-suite successes. Full state-suite/website/native integration remain unverified.

Separately reproduced the Python 3.11 Generator issue by executing the exact unchanged typing import/Prelude alias on both head-source and recorded base: identical TypeError: Too few arguments for typing.Generator; actual 2, expected 3. This is isolated annotation reproduction, not a full Python 3.11 module import/pytest pass; no unrelated compatibility fix is bundled.

The receipt-store/no-transaction-callback design remains. Keeping this Draft pending maintainer decisions on retention/all-consumer coordination and workflow approval; a re-review of this narrow follow-up would be welcome. My earlier question about #124988's closure/replacement remains open; no observer integration is assumed available.

[Finn763]

Thanks for the turnaround -- the _read_all switch in list_session_deletion_receipts (b8e04c9) is exactly the retry-path gap I flagged, and keeping absent-schema failures explicit reads right. I have not re-run the 3.11 suite against this head, so take this as a design-level ack rather than a full re-review; landing stays a maintainer call either way.

[ljhade24]

Conflict refresh pushed as b72c6d5, merging frozen upstream e06df9c2eae6901f4c88f464e0ae6e753c0428b8.

Retained upstream onboarding delete_chat wrapping/order and caller-authored user_rpc provenance, active-write protection and transactional receipts. No arbitrary transaction callback added. Two subprocess calls explicitly use check=False for current lint, preserving return-code assertions. Final 14-file regression: 140 passed; official scripts/check: 12 checks OK, health 0/0. Real onboarding RPC probe verified receipt commit before notification and no extra receipt on re-delete. Full state suite/website/native/plugin integration and retention coordination remain unverified; Draft retained.

Independent static review found no security/logic blockers, and the committed tree equals the tested/reviewed candidate. Separate merge-tree check against latest main d97007c32d6340f7ba79ac1dfb5773cc00846448 returned exit 0; exact-head GitHub readback reports mergeable=true (blocked, not dirty). Content conflicts are cleared, not maintainer approval or green remote CI. Existing Draft/readiness states are unchanged. No merge, production change or restart.

[ljhade24]

Recurrent conflicts resolved and pushed as 27d9b69, merging upstream 7bda2a3368024f7ae1c70cef16583e7c2c891674.

Kept upstream canonical REST mutation/ownership/revision/generation/idle/retirement/onboarding behavior rather than restoring the legacy direct delete writer. A minimal private handler-authored provenance bridge retains REST receipts; final eligible reset/compression/delegate identities and deletion/mutation receipts commit or roll back together. No new plugin, external cleanup executor or callback.

Final canonical regression 243 passed, 1 Linux-only skip on Darwin across 27 files; all 12 blocking checks OK. Published tree equals tested/reviewed tree. Independent static patch/helper review found no security/logic blockers. Nonblocking transport-spoof and combined lineage integration coverage suggestions remain explicit in the body.

Keeping Draft and existing maintainer design/retention/CI gates. No upstream merge, production changes, or full-suite/native E2E claim.