perf(desktop): bound observer event retention

[devin-ai-integration]

Summary

• Bound the SQLite-backed observer archive cache to 3,000 events per normalized (agent, channel) key and 12 least-recently-read channel keys while preserving causal order, identity deduplication, and the existing single-event append contract.
• Batch each decrypted archive page by channel so ingestion builds one identity set and performs one sort per page instead of sorting after every event.
• Prune inactive agents' live events, folded transcripts, and snapshots after a five-minute grace period on the existing active-turn cadence; active turns and mounted single-/multi-agent observer projections retain their state.

Related issue

Fixes #293

No duplicate issue or PR found.

Testing

TDD evidence:

• RED: the new retention suite failed the five intended contracts; the 1,000-event page recorded sort_calls=1000.
• GREEN: 7 retention tests pass, including archive cap/order/dedup, LRU eviction + notification + shared empty result, single-vs-batch equivalence, five-minute idle retention, active-turn retention, mounted projection retention, and transcript replay. The same page records sort_calls=1.
• Related observer/active-turn/transcript regressions: 174 passed.

Local gates, run on implementation commit d0a018e9d:

• pnpm --filter buzz check — passed (existing Biome diagnostics remain warnings/info).
• pnpm --filter buzz typecheck — passed.
• pnpm --filter buzz test — passed: 5,535 passed, 1 skipped.
• just test-unit — four out-of-scope failures reproduced unchanged on untouched PR base 9bffb8cab5152aa110acfd69cee96c8bd7b74a0a: tests::command_inventory_is_stable, tests::subcommand_counts_are_stable, tests::subcommand_names_are_stable, and migration::tests::embedded_migrator_contains_consolidated_initial_schema. This PR does not change Rust or CI configuration.

Full-window desktop smoke evidence:

DONE - what is implemented, tests added, gates run + results, evidence files/links

• Implementation is complete in the 11 desktop observer-store, retention, subscription-consumer, and test files shown by git diff --merge-base origin/main HEAD.
• Added RED-first coverage for bounded archive windows, ordering, deduplication, archive-key LRU eviction and notifications, shared empty results, single-sort batch ingestion, single-vs-batch equivalence, idle grace, active-turn retention, mounted projection retention, and transcript replay.
• Quantitative result: page_size=1000 sort_calls=1000 before and page_size=1000 sort_calls=1 after.
• Branch is pushed at 7934af2a9aff81a293f091c787ca15001deff414, including the user-pushed merge of current main; working tree is clean.
• The required NuncioCrew Gate passed on the pre-merge implementation head. Per the stop order, gates were not rerun after the user-pushed main merge.
• Raw evidence:
  • RED: /home/ubuntu/issue-293-evidence/red-observer-retention.log
  • GREEN: /home/ubuntu/issue-293-evidence/green-observer-retention.log
  • regressions: /home/ubuntu/issue-293-evidence/green-observer-regressions.log
  • before/after: /home/ubuntu/issue-293-evidence/archive-ingest-before-after.txt
  • check: /home/ubuntu/issue-293-evidence/gate-pnpm-buzz-check.log
  • typecheck: /home/ubuntu/issue-293-evidence/gate-pnpm-buzz-typecheck.log
  • desktop tests: /home/ubuntu/issue-293-evidence/gate-pnpm-buzz-test.log
  • Rust unit run and base reproduction: /home/ubuntu/issue-293-evidence/gate-just-test-unit.log, /home/ubuntu/issue-293-evidence/gate-just-test-unit-main-verification.log
  • full-window screenshot: /home/ubuntu/issue-293-evidence/issue-293-full-window.png
  • full-window runtime recording: /home/ubuntu/screencasts/pr307-observer-retention/pr307-observer-retention-edited.mp4
  • animated runtime evidence: /home/ubuntu/evidence/observer-retention/pr307-observer-retention.webp
  • additional full-window frames: /home/ubuntu/evidence/observer-retention/03-live-second.png, /home/ubuntu/evidence/observer-retention/11-bravo-isolated.png, /home/ubuntu/evidence/observer-retention/12-alpha-remount.png
  • runtime test plan: /home/ubuntu/evidence/observer-retention-test-plan.md
• Hosted runtime evidence and the advisory CI triage are also in the existing PR runtime-verification comment.

IN PROGRESS - what is half-done, which files, what state it is in

• Nothing is half-implemented; no uncommitted or unpushed files remain.
• No CI fixes are in progress. The user directed this session not to wait on or retrigger CI.

TODO / NEXT - remaining steps for whoever picks this up, known risks, conflicts expected

• Parent session owns any later CI serialization, branch update, conflict resolution, and merge. Do not merge from this session.
• If verification is repeated after the main merge, rerun the documented desktop gates; do not alter .github/.
• Advisory desktop E2E jobs reported failures outside the observer-retention paths. Three profile failures, four smoke-shard-2 failures, eight agent-integration failures, and two relay-backed notification failures were reproduced on the PR base. The relay-backed notification failures are order-dependent shared-relay profile-state pollution. Smoke shards 1 and 3 were only partially reproduced on the branch before the stop order, so no base-equivalence claim is made for that last batch.
• Current main merged cleanly into the feature branch; no known code conflicts remain.

Link to Devin session: https://app.devin.ai/sessions/fafa62bf95ed46489ace1c78029a6836
Requested by: @oscarlehuu

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

Runtime verification — desktop E2E mock bridge (d0a018e)

Drove the real NuncioCrew activity/transcript surfaces headed through the Playwright E2E mock bridge. Golden and adversarial paths pass; no visible regression.

Golden: 3,200-event archive page ingests; live updates keep flowing

A single archived observer page of 3,200 frames (200 over the cap) rendered the newest frame arch-3200 in 4.2s. Two frames injected afterward appeared in the already-mounted Alpha panel, verifying the per-agent projection subscription remains live after hydration.

Adversarial: projection isolation and unmount/remount retention

Bravo displayed only its own observer frame while Alpha received another frame off-panel.

Reopening Alpha within the grace period preserved both earlier frames, and a new post-remount frame rendered.

Regression coverage and limits

Committed workbench.spec.ts and observer-archive-policy.spec.ts: 7 passed headed.

The exact 3,000-event cap, 12-key LRU eviction, and five-minute expiry are not directly observable through the UI; the focused unit suite covers those values. This runtime run is a no-visible-regression check and was not repeated against the base commit.

Advisory CI failure triage

The three failures in Desktop E2E Integration (2/2) reproduce serially with byte-identical assertions on both this branch and a clean worktree at PR base 9bffb8cab: the two declared-owner profile tests and the notification home-badge test. They do not consume observer events; no production or test changes were made for them.

Written by Devin

[devin-ai-integration]

Summary

• Bound the SQLite-backed observer archive cache to 3,000 events per normalized (agent, channel) key and 12 least-recently-read channel keys while preserving causal order, identity deduplication, and the existing single-event append contract.
• Batch each decrypted archive page by channel so ingestion builds one identity set and performs one sort per page instead of sorting after every event.
• Prune inactive agents' live events, folded transcripts, and snapshots after a five-minute grace period on the existing active-turn cadence; active turns and mounted single-/multi-agent observer projections retain their state.

Related issue

Fixes #293

No duplicate issue or PR found.

Testing

TDD evidence:

• RED: the new retention suite failed the five intended contracts; the 1,000-event page recorded sort_calls=1000.
• GREEN: 7 retention tests pass, including archive cap/order/dedup, LRU eviction + notification + shared empty result, single-vs-batch equivalence, five-minute idle retention, active-turn retention, mounted projection retention, and transcript replay. The same page records sort_calls=1.
• Related observer/active-turn/transcript regressions: 174 passed.

Local gates, run on implementation commit d0a018e9d:

• pnpm --filter buzz check — passed (existing Biome diagnostics remain warnings/info).
• pnpm --filter buzz typecheck — passed.
• pnpm --filter buzz test — passed: 5,535 passed, 1 skipped.
• just test-unit — four out-of-scope failures reproduced unchanged on untouched PR base 9bffb8cab5152aa110acfd69cee96c8bd7b74a0a: tests::command_inventory_is_stable, tests::subcommand_counts_are_stable, tests::subcommand_names_are_stable, and migration::tests::embedded_migrator_contains_consolidated_initial_schema. This PR does not change Rust or CI configuration.

Full-window desktop smoke evidence:

DONE - what is implemented, tests added, gates run + results, evidence files/links

• Implementation is complete in the 11 desktop observer-store, retention, subscription-consumer, and test files shown by git diff --merge-base origin/main HEAD.
• Added RED-first coverage for bounded archive windows, ordering, deduplication, archive-key LRU eviction and notifications, shared empty results, single-sort batch ingestion, single-vs-batch equivalence, idle grace, active-turn retention, mounted projection retention, and transcript replay.
• Quantitative result: page_size=1000 sort_calls=1000 before and page_size=1000 sort_calls=1 after.
• Branch is pushed at 7934af2a9aff81a293f091c787ca15001deff414, including the user-pushed merge of current main; working tree is clean.
• The required NuncioCrew Gate passed on the pre-merge implementation head. Per the stop order, gates were not rerun after the user-pushed main merge.
• Raw evidence:
  • RED: /home/ubuntu/issue-293-evidence/red-observer-retention.log
  • GREEN: /home/ubuntu/issue-293-evidence/green-observer-retention.log
  • regressions: /home/ubuntu/issue-293-evidence/green-observer-regressions.log
  • before/after: /home/ubuntu/issue-293-evidence/archive-ingest-before-after.txt
  • check: /home/ubuntu/issue-293-evidence/gate-pnpm-buzz-check.log
  • typecheck: /home/ubuntu/issue-293-evidence/gate-pnpm-buzz-typecheck.log
  • desktop tests: /home/ubuntu/issue-293-evidence/gate-pnpm-buzz-test.log
  • Rust unit run and base reproduction: /home/ubuntu/issue-293-evidence/gate-just-test-unit.log, /home/ubuntu/issue-293-evidence/gate-just-test-unit-main-verification.log
  • full-window screenshot: /home/ubuntu/issue-293-evidence/issue-293-full-window.png
  • full-window runtime recording: /home/ubuntu/screencasts/pr307-observer-retention/pr307-observer-retention-edited.mp4
  • animated runtime evidence: /home/ubuntu/evidence/observer-retention/pr307-observer-retention.webp
  • additional full-window frames: /home/ubuntu/evidence/observer-retention/03-live-second.png, /home/ubuntu/evidence/observer-retention/11-bravo-isolated.png, /home/ubuntu/evidence/observer-retention/12-alpha-remount.png
  • runtime test plan: /home/ubuntu/evidence/observer-retention-test-plan.md
• Hosted runtime evidence and the advisory CI triage are also in the existing PR runtime-verification comment.

IN PROGRESS - what is half-done, which files, what state it is in

• Nothing is half-implemented; no uncommitted or unpushed files remain.
• No CI fixes are in progress. The user directed this session not to wait on or retrigger CI.

TODO / NEXT - remaining steps for whoever picks this up, known risks, conflicts expected

• Parent session owns any later CI serialization, branch update, conflict resolution, and merge. Do not merge from this session.
• If verification is repeated after the main merge, rerun the documented desktop gates; do not alter .github/.
• Advisory desktop E2E jobs reported failures outside the observer-retention paths. Three profile failures, four smoke-shard-2 failures, eight agent-integration failures, and two relay-backed notification failures were reproduced on the PR base. The relay-backed notification failures are order-dependent shared-relay profile-state pollution. Smoke shards 1 and 3 were only partially reproduced on the branch before the stop order, so no base-equivalence claim is made for that last batch.
• Current main merged cleanly into the feature branch; no known code conflicts remain.