diff --git a/.beads/interactions.jsonl b/.beads/interactions.jsonl index 2726ae70a3..57193fabf3 100644 --- a/.beads/interactions.jsonl +++ b/.beads/interactions.jsonl @@ -13,3 +13,24 @@ {"id":"int-a2361c4964a752d958cfc5fbb69bd077","kind":"field_change","created_at":"2026-07-29T21:16:21.533526502Z","actor":"jeremylongshore","issue_id":"buzz-ocv.5","extra":{"field":"status","new_value":"in_progress","old_value":"open"}} {"id":"int-4ee4835e436c4b7b6a00d0501b8b071c","kind":"field_change","created_at":"2026-07-30T04:47:01.972398591Z","actor":"jeremylongshore","issue_id":"buzz-ocv.6","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"DONE 2026-07-30. Mobile pairing sidecar deployed to prod + a REAL device paired (owner-confirmed handshake). Root cause: pinned relay-v0.2.0 image lacks buzz-pair-relay; fix pins the sidecar SEPARATELY via BUZZ_PAIR_RELAY_IMAGE (:main @9de8aff1) + absolute entrypoint, deployed sidecar-first (relay untouched), Caddy /pair route added. Verified wss://buzz-prod.intentsolutions.io/pair -> 101 Switching Protocols (Caddy->sidecar); sidecar logs show the live paired connection. intent-os PR #291 (overlay + test image-guard + runbook; 4/4 CI green). pairing-overlay-test now guards the image bug (5/5)."}} {"id":"int-ecdd102b39c09f97150839220f0b0d34","kind":"field_change","created_at":"2026-07-30T04:47:05.19034745Z","actor":"jeremylongshore","issue_id":"buzz-ocv.5","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"DONE 2026-07-30. Tauri desktop CORS origins (tauri://localhost + http://tauri.localhost) composed into version-controlled compose (intent-os PR #283), applied live to both hosts (PR #284), cors-config-test 2/2 in ci:drills. Final owner-gated gate met: Jeremy joined from the PACKAGED Linux desktop client to buzz-prod.intentsolutions.io (connected, admin, channels visible) — the packaged-client CORS verification the acceptance required."}} +{"id":"int-9d63f6e48fc6157929c47a230eede39a","kind":"field_change","created_at":"2026-07-30T05:45:53.661302817Z","actor":"jeremylongshore","issue_id":"buzz-ocv.2","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"DONE 2026-07-30 (ultracode Track-A workflow). Compensating edge controls shipped + applied. CODE: ops/buzz/scripts/apply-edge-controls.sh (plan-only default, --apply to install) + hermetic test, merged intent-os PR #292 (4/4 estate-CI green, adversarially verified sound). APPLIED LIVE on the dedicated prod host intent-ops-buzz: fail2ban 'caddy-buzz' jail (bans public 4xx-flooders from /var/log/caddy/buzz-access.log) + nftables 'inet buzz_edge' connlimit (128 concurrent NEW/IP on tcp 80+443 only) + buzz-edge-nft.service enabled (reboot-safe). VERIFIED no self-lock: SSH intact, relay _readiness 200 off-box, tailnet/port-22 never touched (WS/established never dropped). SCOPING: applied to the Buzz-DEDICATED prod host only; deliberately NOT applied host-wide on the shared VPS staging (its connlimit would rate-limit co-tenant estate stacks Plane/ERP/CRM) — request_body caps + security headers already exist in Caddy on both. Closes the last non-owner-gated piece of go-live edge hardening."}} +{"id":"int-d1b3bcb9725ff332970aacd020a3a368","kind":"field_change","created_at":"2026-07-30T05:54:52.463260258Z","actor":"jeremylongshore","issue_id":"buzz-bsy","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"DONE 2026-07-30 (ultracode Track-A workflow). Built ops/buzz/scripts/restore.sh — the real destructive three-store restore (Postgres pg_restore --clean --if-exists → media mc mirror --overwrite → git volume), gated behind --in-place with verify-before-destroy (sha256 every artifact + PGDMP header + manifest.env==--env cross-check; exit 2 writes nothing). Chose --in-place-required over auto-restore so decision-log/038's human-gate holds with ZERO updater.sh changes: the updater's flagless call exits 64 → rc5 page unchanged (updater-drill 6/6 still green). Hermetic restore-test.sh 6/6 with mutation-proven teeth; restore-drill-test 4/4 no regression; wired into ci:drills. Merged intent-os PR #293 (4/4 estate-CI green). Remaining real-tool proof = a host-side scratch-stack drill before first prod use (noted in runbook)."}} +{"id":"int-be2173d658fc095081a9083e49d64528","kind":"field_change","created_at":"2026-07-30T07:17:44.135065314Z","actor":"jeremylongshore","issue_id":"buzz-nry.2","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"DONE 2026-07-30. Production deployed from proven staging artifacts with FRESH secrets on the dedicated host intent-ops-buzz, serving buzz-prod.intentsolutions.io (169.58.95.32). The 'cut DNS over' (apex → buzz.) requirement is DROPPED by owner decision 2026-07-30: buzz-prod.intentsolutions.io is the PERMANENT production URL (no apex cutover). Prod DNS is therefore already on its permanent, correct name — nothing further to cut over."}} +{"id":"int-f42f74288896136765cfddee4c2de26e","kind":"field_change","created_at":"2026-07-30T07:17:46.752322494Z","actor":"jeremylongshore","issue_id":"buzz-nry.3","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"DONE 2026-07-30. Full go-live gate suite PASSED against prod (buzz-prod.intentsolutions.io): smoke 200/NIP-11/functional PASS; unauth-matrix 6/6 REJECT; functional-probe (member NIP-42 auth+publish+readback, un-invited refused) with cleanup CONFIRMED (baseline count 3). Was dependency-blocked by buzz-nry.2 (apex cutover) — that dependency is resolved by the owner decision to keep buzz-prod permanent, so this now closes with its already-captured green evidence."}} +{"id":"int-67f8b7a8627b3a18e0b7b5d935a2d3c1","kind":"field_change","created_at":"2026-07-30T07:18:01.27822535Z","actor":"jeremylongshore","issue_id":"buzz-nry","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"EPIC DONE 2026-07-30. Dedicated Buzz production VPS (intent-ops-buzz, 169.58.95.32) stood up to estate conventions (.1), deployed with fresh secrets serving buzz-prod.intentsolutions.io (.2), and the full go-live gate suite passed against it (.3). Owner decision 2026-07-30: buzz-prod.intentsolutions.io is the PERMANENT production URL — the 'cut the prod domain over' (apex) part of this epic is intentionally NOT done and is retired, not deferred. All three children closed."}} +{"id":"int-7879443495ecbf89b4dc0db155eeab9d","kind":"field_change","created_at":"2026-07-31T03:30:31.615907433Z","actor":"jeremylongshore","issue_id":"buzz-yfe.1","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"source-code reference written to ops/buzz/reference/BUZZ-SYSTEM-REFERENCE.md (45KB, file:line cited, from 8-agent sourcing workflow)"}} +{"id":"int-5068eed2df8e022fc89b5a792782fd39","kind":"field_change","created_at":"2026-07-31T03:30:33.412199224Z","actor":"jeremylongshore","issue_id":"buzz-yfe.2","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"estate-deployment reference written to ops/buzz/reference/BUZZ-ESTATE-DEPLOYMENT-REFERENCE.md (source-cited)"}} +{"id":"int-a7c6e307556516f672455c69f544d886","kind":"field_change","created_at":"2026-07-31T03:30:34.984414889Z","actor":"jeremylongshore","issue_id":"buzz-w92.1","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"AI Wire feed bot minted (8189ba10), relay member, joined ai-wire+0-general, first cards posted accepted:true 2026-07-31"}} +{"id":"int-ddb9fd8ab1b2e550f95a215b3bfc16e0","kind":"field_change","created_at":"2026-07-31T03:36:16.562161598Z","actor":"jeremylongshore","issue_id":"buzz-yfe.3","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"buzz-ops skill built (.claude/skills/buzz-ops/ with 3 bundled references); PASS /validate-skillmd standard tier"}} +{"id":"int-f5404dea1362f4e691fb6edda3e17b7d","kind":"field_change","created_at":"2026-07-31T03:36:18.134646786Z","actor":"jeremylongshore","issue_id":"buzz-yfe.4","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"buzz-ops agent built + PASS /validate-agent (14-field standalone spec)"}} +{"id":"int-85fdaa9e88769d07584be2fb1e04cf42","kind":"field_change","created_at":"2026-07-31T03:36:19.601340573Z","actor":"jeremylongshore","issue_id":"buzz-yfe.5","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"buzz-feed-curator agent built + PASS /validate-agent"}} +{"id":"int-8850efaef9cdedb9badb537fc62935ba","kind":"field_change","created_at":"2026-07-31T03:36:21.08100149Z","actor":"jeremylongshore","issue_id":"buzz-yfe.6","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"buzz-relay-admin agent built + PASS /validate-agent"}} +{"id":"int-4260303d729ddb42abd839cb31f38f27","kind":"field_change","created_at":"2026-07-31T03:36:22.349391594Z","actor":"jeremylongshore","issue_id":"buzz-yfe.7","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"all 3 agents PASS /validate-agent; skill PASS /validate-skillmd; reference-vs-code consistency 106 citations 100% resolve to real fork source (no fabrications)"}} +{"id":"int-f0b734dccd26f403f2e9576181b68a3f","kind":"field_change","created_at":"2026-07-31T03:37:20.957039522Z","actor":"jeremylongshore","issue_id":"buzz-yfe","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"All 7 children closed: 2 references + skill + 3 agents + validation; all PASS the IS validators"}} +{"id":"int-630ab91f0cf7e0e25edae67f26073244","kind":"field_change","created_at":"2026-07-31T04:06:15.368820216Z","actor":"jeremylongshore","issue_id":"buzz-w92.2","extra":{"field":"status","new_value":"in_progress","old_value":"open"}} +{"id":"int-cca6517e1c6ee0187914938b7edd0144","kind":"field_change","created_at":"2026-07-31T04:18:38.265063552Z","actor":"jeremylongshore","issue_id":"buzz-w92.2","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"pipeline built + hardened (fetch→dedupe(newest-25)→Groq/NVIDIA summary→card), rate-limit fallback chain, ops/buzz/wire/buzz-wire.py"}} +{"id":"int-78e02ffa7530bcd16898d0cbbffeec34","kind":"field_change","created_at":"2026-07-31T04:18:40.136787487Z","actor":"jeremylongshore","issue_id":"buzz-w92.3","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"easy-tier feeds live — OpenAI, HuggingFace, arXiv cs.AI, Google DeepMind; 4 seed cards posted with summaries"}} +{"id":"int-ca89c97ced65361b3070b31df6bec542","kind":"field_change","created_at":"2026-07-31T04:18:41.731887559Z","actor":"jeremylongshore","issue_id":"buzz-w92.8","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"3h cron deployed (~/bin/buzz-wire-cron.sh) + notify-lib liveness + registered in automations.md"}} +{"id":"int-e4c1bfdb34ec5b768b2fe150c50071fc","kind":"field_change","created_at":"2026-07-31T04:30:55.252637052Z","actor":"jeremylongshore","issue_id":"buzz-w92.7","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Daily ai-wire digest live — --digest composes cross-lab roundup (70b→8b→NVIDIA), posts to 1-ai-wire, cron 06:30 CT"}} +{"id":"int-873d15b79debfd144c2a44cbd6113969","kind":"field_change","created_at":"2026-07-31T14:04:52.409666792Z","actor":"jeremylongshore","issue_id":"buzz-w92.4","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Anthropic bridged via HTML-scrape source type (news/research/engineering -> anthropic-blogs/research), mirroring the estate monitors; Mistral added. buzz-w92.6 partially done (tooling/mcp/newsletters/ships/status live)."}} +{"id":"int-b6dc6f4abc00f98eb2fa156977ef6406","kind":"field_change","created_at":"2026-07-31T15:35:29.315597343Z","actor":"jeremylongshore","issue_id":"buzz-w92.5","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Filled groq/meta/mistral/cohere/perplexity/xai + cleaner anthropic via Olshansk RSS mirrors (from perception repo) — no RSSHub needed. Remaining empty (deepseek/qwen/moonshot/minimax/benchmarks/funding/openrouter) have no mirror."}} diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index b786ef1a58..bc5ccd5f3c 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -1,20 +1,67 @@ -{"_type":"issue","id":"buzz-2i6","title":"Install nak on the Buzz hosts and run the real functional-probe rehearsal, then enable the prod updater timer","description":"The updater lane is deployed to staging (dormant) but the real functional-probe rehearsal is blocked: nak (the NIP-42 auth+publish+readback CLI the probe uses) is absent on the Buzz hosts. buzz-admin IS in the relay image. Steps: install nak on staging (+ prod), run functional-probe.sh against the live staging relay (throwaway member auth+publish+readback, un-invited refused, cleans up), then a controlled no-op-digest updater rehearsal on staging, THEN install-updater-lane.sh intent-ops-buzz --enable-timer for prod. Parent: buzz-ocv.4.","notes":"REHEARSAL PARTIAL on live staging 2026-07-29 (nak copied to /usr/local/bin/nak on the staging host; jq present). VERIFIED LIVE: closed relay up+healthy+CORS-applied; buzz-admin add-member/remove-member/list-members work — EXACT syntax is 'buzz-admin add-member --pubkey \u003chex-or-npub\u003e' (NOT positional; positional errors with usage). nak emits 64-char hex from 'nak key public'; relay accepts hex. add+remove round-trip proven; test member cleaned up (relay left clean: owner + 1 pre-existing member). REMAINING: the nak publish/readback NIP-42 flow hung with reactive '--auth' — the closed relay likely needs '--force-pre-auth' (authenticate BEFORE the EVENT/REQ), and readback (nak req -i \u003cid\u003e) on a closed relay also needs auth so the member NSEC must be threaded into functional-probe.sh's readback() (currently it only gets the event id). NEXT: (1) prove 'NOSTR_SECRET_KEY=\u003cnsec\u003e nak event -k1 -c X --auth --force-pre-auth \u003cws\u003e' returns an id against staging; (2) prove readback with the member key; (3) bake the verified commands + --pubkey into functional-probe.sh add_member/del_member/publish/readback; (4) then no-op-digest updater rehearsal on staging; (5) install-updater-lane.sh intent-ops-buzz --enable-timer.\nREHEARSAL DONE on staging (intent-os PR #288): nak installed on the staging host; functional-probe.sh fixed to the real NIP-42 flow (--sec, public wss through Caddy, --force-pre-auth readback, --pubkey, keygen newline, global cleanup trap) and PROVEN GREEN E2E against the live staging relay — member publish+readback, un-invited refused; smoke-suite PASS; unauth-matrix 6/6. Self-cleaning; staging membership left clean. REMAINING: install nak on the PROD host + run the same 3 gates against buzz-prod off-network; then enable the prod updater timer.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T21:47:43Z","created_by":"jeremylongshore","updated_at":"2026-07-30T00:37:47Z","labels":["go-live","probe","updater"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-20l.6","title":"Add the Claude impl-agent identity as a second buzz-acp process that authors PRs from goose-filed issues only","description":"Second buzz-acp process: Claude (claude-agent-acp), PermissionMode acceptEdits, PR-scoped PAT, a /work cwd that clones REPOS/\u003crepo\u003e on demand; reads ONLY goose-authored issues (never the raw channel firehose — invariant 2). Handoff is a human @mention of the impl agent on goose's issue (invariant 1). Acceptance: human @mention on issue #N =\u003e impl agent opens a PR against that repo; CI gates the merge; the agent cannot merge. DEPENDS on the ACP-headless spike.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:11Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:11Z","labels":["agentic-ops","estate","github-bridge","impl-agent"],"dependencies":[{"issue_id":"buzz-20l.6","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:10Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.6","depends_on_id":"buzz-20l.1","type":"blocks","created_at":"2026-07-31T13:58:36Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.6","depends_on_id":"buzz-20l.5","type":"blocks","created_at":"2026-07-31T13:58:37Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":2,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-20l.5","title":"Configure the goose-triage profile — owner-gated, issue-filing only, reading the repo-channel firehose","description":"goose (MiniMax) runs owner-only, PermissionMode plan/dontAsk, no-merge PAT, subscribed to the repo-home channels; reads the raw bridge feed and files ONE GitHub issue per finding via the finding-id upsert. It NEVER authors code and NEVER hands off via chat @mention. Acceptance: a posted CI-failure card results in exactly one filed issue; goose has no write path to code.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:09Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:09Z","labels":["agentic-ops","estate","github-bridge","triage-agent"],"dependencies":[{"issue_id":"buzz-20l.5","depends_on_id":"buzz-20l.3","type":"blocks","created_at":"2026-07-31T13:58:34Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.5","depends_on_id":"buzz-20l.4","type":"blocks","created_at":"2026-07-31T13:58:34Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.5","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:09Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":2,"dependent_count":3,"comment_count":0} +{"_type":"issue","id":"buzz-20l.4","title":"Build the finding-id and level-triggered reconciler so every failure maps to exactly one auto-closing issue","description":"finding-id = hash(repo, workflow, normalized-failure) for idempotent issue upsert (create-only-if-absent); a periodic level-triggered reconciler computes currently-failing finding-ids and closes every finding-issue no longer in that set (do NOT rely on edge-triggered 'green run-\u003eclose' — Buzz drops replies #2459/#3587, agents go deaf in threads #2270). BUILD BEFORE the first repo channel goes live. Acceptance: three retries of one flaky failure =\u003e one issue; recovery =\u003e the issue auto-closes within one reconcile cycle.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:08Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:08Z","labels":["agentic-ops","estate","github-bridge","reliability"],"dependencies":[{"issue_id":"buzz-20l.4","depends_on_id":"buzz-20l.3","type":"blocks","created_at":"2026-07-31T13:58:33Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.4","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:08Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0} +{"_type":"issue","id":"buzz-20l.3","title":"Build buzz-gh-bridge: a webhook receiver plus a GitHub App on both orgs that posts repo events as cards into repo-home channels","description":"FastAPI receiver on intent-ops-buzz (~100 lines): verify X-Hub-Signature-256 HMAC, dedup on X-GitHub-Delivery, map owner/repo-\u003echannel_id, render event-\u003ecard, post via the EXISTING local docker-exec path in ops/buzz/wire/buzz-wire.py (no ssh hop). One GitHub App installed on jeremylongshore + intent-solutions-io subscribed to pull_request, pull_request_review, pull_request_review_comment, issues, issue_comment, workflow_run, push, release, deployment_status. Webhooks (real-time), not polling; CI status via the workflow_run event. Caddy route -\u003eloopback. Acceptance: a real PR/CI event posts a card into its repo channel; HMAC-invalid and duplicate deliveries are rejected.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:07Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:07Z","labels":["agentic-ops","estate","github-bridge"],"dependencies":[{"issue_id":"buzz-20l.3","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:06Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":3,"comment_count":0} +{"_type":"issue","id":"buzz-20l.2","title":"Record the agentic-Buzz architecture decisions in decision-log (division of labor, source-of-truth, the two invariants)","description":"Append a decision-log/040 (or new) entry: goose=issues / Claude|Codex=PRs; GitHub is source of truth and Buzz is a projection (NO NIP-34 issue/PR mirror); invariant 1 = the GitHub issue IS the handoff (no agent-to-agent chat @mention); invariant 2 = untrusted channel/webhook text never reaches the shell+creds process; 3-process topology (not per-repo). Acceptance: entry merged, cited by the build beads.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:05Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:05Z","labels":["agentic-ops","decision","estate","github-bridge"],"dependencies":[{"issue_id":"buzz-20l.2","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:05Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-20l.1","title":"Confirm Claude Code and Codex ACP harnesses run headless with API keys on the Buzz VPS (the build gate for the impl lane)","description":"Report source-4 flags upstream leans toward subscription/ChatGPT-login auth; we need headless API-key auth. Verify claude-agent-acp and codex-acp start, authenticate via API key, and complete a trivial task headless in an isolated container on intent-ops-buzz. BLOCKS the impl-agent bead. Acceptance: a headless ACP session with each harness completes a no-op task using an API key only, evidenced.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:04Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:04Z","labels":["agentic-ops","estate","github-bridge","spike"],"dependencies":[{"issue_id":"buzz-20l.1","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:03Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":2,"comment_count":0} +{"_type":"issue","id":"buzz-20l","title":"Build the agentic Buzz estate-ops surface — repo-home channels, a GitHub event bridge, and a goose-triage + Claude-impl division of labor","description":"Evolve Buzz from a chat+feed surface into the estate's agentic operations surface, replacing the dead Greptile/Gemini/CodeRabbit 'what was found' review stream on our own stack. Research: ops/buzz/RESEARCH-agentic-buzz-utilization-2026-07-31.md (14-agent ultracode workflow, 2026-07-31).\n\nOwner decisions locked: goose files ISSUES only (owner-gated, plan/dontAsk, no-merge PAT); Claude/Codex authors PRs (native buzz-acp harnesses, config.rs:696 — nothing to build at harness level); every ACTIVE repo (both orgs, minus archived/dead) gets a repo-home channel + lazy-create on next push; GitHub is the source of truth, Buzz is a projection (NO NIP-34 mirror). Two load-bearing invariants: (1) the GitHub issue IS the triage-\u003eimpl handoff, never an agent-to-agent chat @mention; (2) untrusted channel/webhook text never reaches the process holding a shell + git creds.\n\nDependency-ordered children below. Build gate: confirm Claude/Codex ACP run headless with API keys on our VPS before the impl lane.","status":"open","priority":1,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:57:05Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:57:05Z","labels":["agentic-ops","estate","github-bridge"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-1zr.1","title":"Stand up the staging drill harness for channel/identity/feed/notify actions","description":"A script suite that exercises the real actions against buzz-testing.intentsolutions.io with throwaway keys/channels, asserting the load-bearing rules: only-owner-can-delete returns the right error; a non-member post gets 403 relay_membership_required; add-member --role owner is rejected; provider-first naming; verify via channels list. Torn down after each run. AC: harness runs green on staging, prod untouched.","status":"open","priority":1,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:39:32Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:39:32Z","dependencies":[{"issue_id":"buzz-1zr.1","depends_on_id":"buzz-1zr","type":"parent-child","created_at":"2026-07-30T21:39:32Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-1zr","title":"Establish and run the Buzz testing procedure — staging drills, hermetic tests, agent evals","description":"A repeatable way to test the actions we build against Buzz, following the estate pattern (hermetic test in ci:drills + live drill) + the IS testing SOP (/audit-tests, /implement-tests, audit-harness). HARD SAFETY RULE: every destructive/live drill runs on the STAGING relay buzz-testing.intentsolutions.io (drills-only, never real members/channels) — NEVER prod (per ops/buzz/README.md). Covers the new surfaces: channel/identity ops, feed posting, sys-notification routing, and operator-agent behavior.","status":"open","priority":1,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:39:31Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:39:31Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-w92.2","title":"Build the feed ingestion pipeline (fetch -\u003e dedupe -\u003e LLM -\u003e card)","description":"The core pipeline: RSS/RSSHub fetch, dedupe against seen-state, LLM summarize+score (MiniMax/Groq keys from SOPS), format a mobile-first card (imeta hero + markdown + source badge + link), post to ai-wire + the matching *-wire. Reuse perception's ingestion shape (GCP service dead; code is the reference). AC: one real lab item -\u003e a rich card in ai-wire + the lab channel.","status":"closed","priority":1,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:08Z","created_by":"jeremylongshore","updated_at":"2026-07-31T04:18:38Z","started_at":"2026-07-31T04:06:15Z","closed_at":"2026-07-31T04:18:38Z","close_reason":"pipeline built + hardened (fetch→dedupe(newest-25)→Groq/NVIDIA summary→card), rate-limit fallback chain, ops/buzz/wire/buzz-wire.py","dependencies":[{"issue_id":"buzz-w92.2","depends_on_id":"buzz-w92","type":"parent-child","created_at":"2026-07-30T21:29:07Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-w92.2","depends_on_id":"buzz-yfe.5","type":"blocks","created_at":"2026-07-30T21:29:13Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0} +{"_type":"issue","id":"buzz-w92.1","title":"Posting primitive — feed bot key + buzz messages send","description":"DONE 2026-07-31: minted the AI Wire bot npub (8189ba10, /srv/buzz-agent/wire-bot.env), added to relay membership, set profile, joined ai-wire + 0-general, posted the first cards (accepted:true). This is the shared post path for wire + sys notifications.","status":"closed","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:07Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:30:35Z","closed_at":"2026-07-31T03:30:35Z","close_reason":"AI Wire feed bot minted (8189ba10), relay member, joined ai-wire+0-general, first cards posted accepted:true 2026-07-31","dependencies":[{"issue_id":"buzz-w92.1","depends_on_id":"buzz-w92","type":"parent-child","created_at":"2026-07-30T21:29:06Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":2,"comment_count":0} +{"_type":"issue","id":"buzz-w92","title":"AI-Wire feed engine — fetch, summarize, post curated cards","description":"The engine that fills ai-wire + the *-wire channels: per source, poll the feed (or RSSHub for scrape-only labs) -\u003e dedupe -\u003e LLM summarize/score -\u003e post a rich mobile card via the AI Wire bot. Anti-firehose: ai-wire is one curated digest; packs are opt-in. decision-log/040 D140 (fork-and-wire). Reuse the perception repo ingestion pattern + the estate ~/bin Anthropic monitor fleet. Posting primitive already proven (AI Wire bot live).","status":"open","priority":1,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:06Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:29:06Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-yfe.3","title":"Build the buzz-ops skill bundling the reference files","description":"A /skill-creator skill that bundles both reference docs so any session/agent that invokes it knows Buzz's source + estate layout. AC: SKILL.md passes /validate-skillmd; references bundled; encodes the load-bearing rules (only-owner-deletes, membership-required-to-post, provider-first naming, secret handling).","status":"closed","priority":1,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:14Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:36:16Z","closed_at":"2026-07-31T03:36:16Z","close_reason":"buzz-ops skill built (.claude/skills/buzz-ops/ with 3 bundled references); PASS /validate-skillmd standard tier","dependencies":[{"issue_id":"buzz-yfe.3","depends_on_id":"buzz-yfe.1","type":"blocks","created_at":"2026-07-30T21:28:59Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-yfe.3","depends_on_id":"buzz-yfe","type":"parent-child","created_at":"2026-07-30T21:27:13Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-yfe.3","depends_on_id":"buzz-yfe.2","type":"blocks","created_at":"2026-07-30T21:28:59Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":2,"dependent_count":3,"comment_count":0} +{"_type":"issue","id":"buzz-yfe.2","title":"Produce the Buzz estate-deployment reference","description":"How Buzz is wired in the estate: two VPSs, containers, secrets/keys, ops/buzz lane, deploy/backup/monitoring, sys-* notification path. AC: ops/buzz/reference/BUZZ-ESTATE-DEPLOYMENT-REFERENCE.md written, source-cited.","status":"closed","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:13Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:30:33Z","closed_at":"2026-07-31T03:30:33Z","close_reason":"estate-deployment reference written to ops/buzz/reference/BUZZ-ESTATE-DEPLOYMENT-REFERENCE.md (source-cited)","dependencies":[{"issue_id":"buzz-yfe.2","depends_on_id":"buzz-yfe","type":"parent-child","created_at":"2026-07-30T21:27:12Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} +{"_type":"issue","id":"buzz-yfe.1","title":"Produce the Buzz source-code reference (file:line)","description":"Map the block/buzz source: relay, CLI, admin, event kinds, auth/NIP-OA, ACP agent, personas/templates, media. IN FLIGHT via an 8-agent ultracode workflow -\u003e BUZZ-SYSTEM-REFERENCE.md. AC: ops/buzz/reference/BUZZ-SYSTEM-REFERENCE.md written, file:line cited.","status":"closed","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:12Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:30:31Z","closed_at":"2026-07-31T03:30:31Z","close_reason":"source-code reference written to ops/buzz/reference/BUZZ-SYSTEM-REFERENCE.md (45KB, file:line cited, from 8-agent sourcing workflow)","dependencies":[{"issue_id":"buzz-yfe.1","depends_on_id":"buzz-yfe","type":"parent-child","created_at":"2026-07-30T21:27:12Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":2,"comment_count":0} +{"_type":"issue","id":"buzz-yfe","title":"Encode durable Buzz operational expertise — reference, skill, operator agents","description":"Stop operating Buzz by trial-and-error. Produce source-grounded reference docs, a buzz-ops skill that bundles them, and validated operator agents that actually know how Buzz is wired. Built via /skill-creator and /agent-creator (IS 8-field spec), grounded in the references. Home: ops/buzz/reference/ (docs), intent-os .claude/ (skill+agents).","status":"closed","priority":1,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:11Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:37:21Z","closed_at":"2026-07-31T03:37:21Z","close_reason":"All 7 children closed: 2 references + skill + 3 agents + validation; all PASS the IS validators","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-ehv","title":"Community channel layout and identity roster — build clean, keep clean","description":"The buzz-prod community's channel architecture and member/agent identity hygiene. Ground truth + procedures: ops/buzz/RUNBOOK-channels.md (source-cited). SHIPPED this session (2026-07-31): the clean provider-first open layout (0-general pinned, ask, ai-wire, anthropic-* group, \u003cprovider\u003e-wire firehoses, topic *-wire, private sys-*); persona agents (Bumble/Fizz/Honey/CCA-F) purged; canonical names set (Buzz Admin, goose minimax3, AI Wire, humans); RUNBOOK-channels.md written. Open children below.","status":"open","priority":1,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:09Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:27:09Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-2i6","title":"Install nak on the Buzz hosts and run the real functional-probe rehearsal, then enable the prod updater timer","description":"The updater lane is deployed to staging (dormant) but the real functional-probe rehearsal is blocked: nak (the NIP-42 auth+publish+readback CLI the probe uses) is absent on the Buzz hosts. buzz-admin IS in the relay image. Steps: install nak on staging (+ prod), run functional-probe.sh against the live staging relay (throwaway member auth+publish+readback, un-invited refused, cleans up), then a controlled no-op-digest updater rehearsal on staging, THEN install-updater-lane.sh intent-ops-buzz --enable-timer for prod. Parent: buzz-ocv.4.","notes":"REHEARSAL PARTIAL on live staging 2026-07-29 (nak copied to /usr/local/bin/nak on the staging host; jq present). VERIFIED LIVE: closed relay up+healthy+CORS-applied; buzz-admin add-member/remove-member/list-members work — EXACT syntax is 'buzz-admin add-member --pubkey \u003chex-or-npub\u003e' (NOT positional; positional errors with usage). nak emits 64-char hex from 'nak key public'; relay accepts hex. add+remove round-trip proven; test member cleaned up (relay left clean: owner + 1 pre-existing member). REMAINING: the nak publish/readback NIP-42 flow hung with reactive '--auth' — the closed relay likely needs '--force-pre-auth' (authenticate BEFORE the EVENT/REQ), and readback (nak req -i \u003cid\u003e) on a closed relay also needs auth so the member NSEC must be threaded into functional-probe.sh's readback() (currently it only gets the event id). NEXT: (1) prove 'NOSTR_SECRET_KEY=\u003cnsec\u003e nak event -k1 -c X --auth --force-pre-auth \u003cws\u003e' returns an id against staging; (2) prove readback with the member key; (3) bake the verified commands + --pubkey into functional-probe.sh add_member/del_member/publish/readback; (4) then no-op-digest updater rehearsal on staging; (5) install-updater-lane.sh intent-ops-buzz --enable-timer.\nREHEARSAL DONE on staging (intent-os PR #288): nak installed on the staging host; functional-probe.sh fixed to the real NIP-42 flow (--sec, public wss through Caddy, --force-pre-auth readback, --pubkey, keygen newline, global cleanup trap) and PROVEN GREEN E2E against the live staging relay — member publish+readback, un-invited refused; smoke-suite PASS; unauth-matrix 6/6. Self-cleaning; staging membership left clean. REMAINING: install nak on the PROD host + run the same 3 gates against buzz-prod off-network; then enable the prod updater timer.\nAdvanced 2026-07-30 (ultracode Track-A workflow), timer NOT armed by design: (1) nak installed on prod host intent-ops-buzz (/usr/local/bin/nak, keygen verified); (2) wrapped-updater lane installed DORMANT via install-updater-lane.sh (no --enable-timer) — 6/6 lane scripts + both systemd units present, daemon-reload+syntax OK; (3) smoke-suite.sh + unauth-matrix.sh pushed (the installer omits them); (4) functional-probe rehearsal PASSED green against live prod (see buzz-nry.3 gate-suite run — member auth/publish/readback, un-invited refused, self-cleaned). VERIFIED dormant: timer disabled+inactive, updater.service never ran, serving digest unchanged (a0f672). REMAINING = arm the timer (systemctl enable --now) — HELD pending owner call: arming = unattended weekly pre-1.0 prod deploys with advisory-only scan, before apex cutover. Recommend arming AFTER apex cutover; lane is one command from armed.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T21:47:43Z","created_by":"jeremylongshore","updated_at":"2026-07-30T05:40:19Z","labels":["go-live","probe","updater"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-aet","title":"Author the authoritative Buzz naming and boundaries record and reconcile the blueprint ledger to prod-up","description":"Track B of the Buzz adoption plan (owner amendment 2026-07-29, do FIRST). Author the canonical naming+boundaries record in the fork 000-docs from the AUTHORITATIVE NAMING \u0026 REPOSITORY MODEL table: 6 assets (upstream block/buzz, code fork intent-solutions-io/buzz, contributor lab intent-solutions-io/intent-ops-buzz repo, production host intent-ops-buzz VPS, live ops lane intent-os ops/buzz, deferred plugin intent-solutions-io/intent-ops-buzz-plugin), the disambiguation rule (never bare 'intent-ops-buzz'), and the four-plane flow with two separate gates (contribution vs deployment). Reconcile the blueprint 001 completion ledger to reality (prod host built/deployed/verified pre-cutover, backups+restore proven), name the plugin repo deferred, and add the contributor-lab plane. Unblocks the parked contributor-lab architecture DRAFT (Track C). The amendment forbids durable architecture docs until this lands. Done-means: naming doc exists + internally consistent; ledger prod-up; no bare-intent-ops-buzz ambiguity; pnpm check green on the intent-os reconcile edit.","status":"closed","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T19:19:37Z","created_by":"jeremylongshore","updated_at":"2026-07-29T19:36:54Z","started_at":"2026-07-29T19:19:43Z","closed_at":"2026-07-29T19:36:54Z","close_reason":"Track B shipped: authored fork 000-docs/006-DR-STND-authoritative-naming-and-boundaries.md (canonical six-asset table + host-vs-repo disambiguation rule + four-plane flow + two-gate model); reconciled the 001 blueprint ledger to prod-up (E2/E2d/E3 states, ELab row, E9 plugin named intent-ops-buzz-plugin + DEFERRED, new Phase 6 pointer); added the intent-os ops/buzz/README 'Naming \u0026 planes' section. Merged: fork PR #11 + intent-os PR #281 (estate-CI 3 green contexts: gates 128s, drills 212s, gitleaks). pnpm check EXIT=0; no bare-intent-ops-buzz ambiguity remains.","labels":["architecture","ledger","naming"],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"buzz-nry.3","title":"Run the full go-live gate suite against production on the dedicated box","description":"The full E3 go-live gate suite to RUN against PROD before any invite (all BLOCKING, none assumed passed): unauth probe matrix off-network; functional membership probe (NIP-42 publish+readback, un-invited refused); backup restore drill (restore staging from prod's artifacts); updater planted-fault drill on staging; key runbooks rehearsed; resource caps verified. Status is tracked per-gate in the notes and in intent-os ops/buzz — the off-site backup leg and several gates remain OPEN; this description is scope, not a completion claim.","notes":"Go-live gates PARTIAL against prod: (1) functional membership probe GREEN — owner NIP-42 auth -\u003e publish -\u003e readback OK; un-invited key REFUSED (closed relay enforcing, member_count:1 on boot). (2) unauth HTTP probe matrix PASS — media PUT 405, git 404/405, hooks 400, admin 404; public readiness/NIP-11/web-client 200. (3) resource caps + loopback-only publish + isolated bridge net verified. REMAINING/BLOCKING before any invite: backup-restore drill, updater planted-fault drill, key runbooks — all depend on backup+updater wiring (buzz-ocv.4, not built).\nRESTORE DRILL PROVEN (buzz-nry.3 gate item): restored the first prod archive into an ISOLATED dev-box scratch project (prod/staging untouched, prod 4/4 healthy throughout). 40 tables restored; the exact prod-published event 9717ccaa physically present; relay_members=1 + member_count:1 on boot; media restored; relay identity stable (same key-\u003e35cd57ab); relay boots healthy; un-invited refused; restart persistence holds. RTO ~3min, RPO = backup point (18:36:41Z). Caveat: headless nak auth-on-REQ readback didn't complete in-harness (passed on prod). Evidence: ops/buzz/RUNBOOK-backup-restore.md. STILL BLOCKING before invite: updater planted-fault drill, key runbooks, off-site copy.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:31:30Z","created_by":"jeremylongshore","updated_at":"2026-07-29T18:45:39Z","labels":["hosting"],"dependencies":[{"issue_id":"buzz-nry.3","depends_on_id":"buzz-nry.2","type":"blocks","created_at":"2026-07-29T10:31:31Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-nry.3","depends_on_id":"buzz-nry","type":"parent-child","created_at":"2026-07-29T10:31:29Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"buzz-nry.2","title":"Deploy production from the proven staging artifacts with fresh secrets and cut DNS over","description":"Deploy prod from the proven staging artifacts (same digest-pinned compose) with FRESH secrets minted on the new box (new relay identity, new bootstrap owner key until the owner's desktop key swaps in, new db/redis/minio; staging keys never promote). DNS cuts over to the prod host at go-live; the shared stack renames to the staging domain. Deploy specifics + addressing in intent-os ops/buzz. Depends on the bootstrap child.","notes":"Prod deploy DONE (apex cutover pending): staging-proven digest-pinned compose deployed with FRESH prod secrets; 4 containers healthy; own Caddy ingress TLS + security headers. REMAINING (gated on the owner's client-generated desktop key): DNS cutover to the prod host, rename staging to the staging domain, swap BUZZ_DOMAIN/RELAY_URL/CORS to the apex, swap RELAY_OWNER_PUBKEY to the owner's desktop pubkey. Addressing + specifics in intent-os ops/buzz.","status":"open","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:31:22Z","created_by":"jeremylongshore","updated_at":"2026-07-29T18:17:43Z","labels":["hosting"],"dependencies":[{"issue_id":"buzz-nry.2","depends_on_id":"buzz-nry","type":"parent-child","created_at":"2026-07-29T10:31:22Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-nry.2","depends_on_id":"buzz-nry.1","type":"blocks","created_at":"2026-07-29T10:31:31Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0} +{"_type":"issue","id":"buzz-nry.3","title":"Run the full go-live gate suite against production on the dedicated box","description":"The full E3 go-live gate suite to RUN against PROD before any invite (all BLOCKING, none assumed passed): unauth probe matrix off-network; functional membership probe (NIP-42 publish+readback, un-invited refused); backup restore drill (restore staging from prod's artifacts); updater planted-fault drill on staging; key runbooks rehearsed; resource caps verified. Status is tracked per-gate in the notes and in intent-os ops/buzz — the off-site backup leg and several gates remain OPEN; this description is scope, not a completion claim.","notes":"Go-live gates PARTIAL against prod: (1) functional membership probe GREEN — owner NIP-42 auth -\u003e publish -\u003e readback OK; un-invited key REFUSED (closed relay enforcing, member_count:1 on boot). (2) unauth HTTP probe matrix PASS — media PUT 405, git 404/405, hooks 400, admin 404; public readiness/NIP-11/web-client 200. (3) resource caps + loopback-only publish + isolated bridge net verified. REMAINING/BLOCKING before any invite: backup-restore drill, updater planted-fault drill, key runbooks — all depend on backup+updater wiring (buzz-ocv.4, not built).\nRESTORE DRILL PROVEN (buzz-nry.3 gate item): restored the first prod archive into an ISOLATED dev-box scratch project (prod/staging untouched, prod 4/4 healthy throughout). 40 tables restored; the exact prod-published event 9717ccaa physically present; relay_members=1 + member_count:1 on boot; media restored; relay identity stable (same key-\u003e35cd57ab); relay boots healthy; un-invited refused; restart persistence holds. RTO ~3min, RPO = backup point (18:36:41Z). Caveat: headless nak auth-on-REQ readback didn't complete in-harness (passed on prod). Evidence: ops/buzz/RUNBOOK-backup-restore.md. STILL BLOCKING before invite: updater planted-fault drill, key runbooks, off-site copy.\nGATE SUITE PASSED against prod 2026-07-30 (ultracode Track-A workflow, non-destructive): smoke 200/NIP-11/functional PASS; unauth-matrix 6/6 REJECT; functional-probe (member NIP-42 auth+publish+readback, un-invited refused) with cleanup CONFIRMED (back to baseline count 3). The suite deliverable is DONE; this bead stays OPEN only because its dependency buzz-nry.2 (apex DNS cutover) is owner-gated and not yet done — close together when apex cuts over.","status":"closed","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:31:30Z","created_by":"jeremylongshore","updated_at":"2026-07-30T07:17:47Z","closed_at":"2026-07-30T07:17:47Z","close_reason":"DONE 2026-07-30. Full go-live gate suite PASSED against prod (buzz-prod.intentsolutions.io): smoke 200/NIP-11/functional PASS; unauth-matrix 6/6 REJECT; functional-probe (member NIP-42 auth+publish+readback, un-invited refused) with cleanup CONFIRMED (baseline count 3). Was dependency-blocked by buzz-nry.2 (apex cutover) — that dependency is resolved by the owner decision to keep buzz-prod permanent, so this now closes with its already-captured green evidence.","labels":["hosting"],"dependencies":[{"issue_id":"buzz-nry.3","depends_on_id":"buzz-nry.2","type":"blocks","created_at":"2026-07-29T10:31:31Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-nry.3","depends_on_id":"buzz-nry","type":"parent-child","created_at":"2026-07-29T10:31:29Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-nry.2","title":"Deploy production from the proven staging artifacts with fresh secrets and cut DNS over","description":"Deploy prod from the proven staging artifacts (same digest-pinned compose) with FRESH secrets minted on the new box (new relay identity, new bootstrap owner key until the owner's desktop key swaps in, new db/redis/minio; staging keys never promote). DNS cuts over to the prod host at go-live; the shared stack renames to the staging domain. Deploy specifics + addressing in intent-os ops/buzz. Depends on the bootstrap child.","notes":"Prod deploy DONE (apex cutover pending): staging-proven digest-pinned compose deployed with FRESH prod secrets; 4 containers healthy; own Caddy ingress TLS + security headers. REMAINING (gated on the owner's client-generated desktop key): DNS cutover to the prod host, rename staging to the staging domain, swap BUZZ_DOMAIN/RELAY_URL/CORS to the apex, swap RELAY_OWNER_PUBKEY to the owner's desktop pubkey. Addressing + specifics in intent-os ops/buzz.","status":"closed","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:31:22Z","created_by":"jeremylongshore","updated_at":"2026-07-30T07:17:44Z","closed_at":"2026-07-30T07:17:44Z","close_reason":"DONE 2026-07-30. Production deployed from proven staging artifacts with FRESH secrets on the dedicated host intent-ops-buzz, serving buzz-prod.intentsolutions.io (169.58.95.32). The 'cut DNS over' (apex → buzz.) requirement is DROPPED by owner decision 2026-07-30: buzz-prod.intentsolutions.io is the PERMANENT production URL (no apex cutover). Prod DNS is therefore already on its permanent, correct name — nothing further to cut over.","labels":["hosting"],"dependencies":[{"issue_id":"buzz-nry.2","depends_on_id":"buzz-nry","type":"parent-child","created_at":"2026-07-29T10:31:22Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-nry.2","depends_on_id":"buzz-nry.1","type":"blocks","created_at":"2026-07-29T10:31:31Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0} {"_type":"issue","id":"buzz-nry.1","title":"Bootstrap the new production host to estate conventions","description":"Bootstrap the dedicated prod host to estate ops/host conventions: Ubuntu 24.04, tailnet-only SSH, ufw, fail2ban, unattended-upgrades, docker, its own Caddy (separate ingress = separate failure domain), age host key + sops, borg client, Netdata (tailnet-bound). Host addressing, key paths, and access detail live ONLY in intent-os ops/buzz. Gated on the owner installing the dev-box SSH key.","status":"closed","priority":1,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:31:15Z","created_by":"jeremylongshore","updated_at":"2026-07-29T18:06:58Z","started_at":"2026-07-29T17:57:57Z","closed_at":"2026-07-29T18:06:58Z","close_reason":"intent-ops-buzz bootstrapped to estate baseline + verified: Ubuntu 24.04.4 LTS, hostname intent-ops-buzz, joined tailnet [tailnet addr — intent-os ops/buzz] (untagged/user-owned, matching estate); ufw default-deny (tailnet-trusted iface + public 80/443 only, public 22 CLOSED/times-out); sshd key-only + PermitRootLogin no + PasswordAuthentication no; a non-root sudo admin account (NOPASSWD, docker group); fail2ban + unattended-upgrades active; 2G swap swappiness=10; docker 29.6 (log-rotation + live-restore) + compose v5.3; caddy 2.11 (own ingress); age host key at [host key path — intent-os ops/host] (recipient [host age recipient — intent-os ops/host/secrets]); borg client; sops 3.9.4; Netdata bound loopback+tailnet only (127.0.0.1:19999 + [tailnet addr — intent-os ops/buzz]:19999, not public). 7/7 services active.","labels":["hosting"],"dependencies":[{"issue_id":"buzz-nry.1","depends_on_id":"buzz-nry","type":"parent-child","created_at":"2026-07-29T10:31:15Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0} -{"_type":"issue","id":"buzz-nry","title":"Stand up the dedicated Buzz production VPS and cut the prod domain over to it","description":"Track D: Buzz production runs on its OWN dedicated VPS (owner topology decision 2026-07-29, fork 000-docs/005) so a fast-moving pre-1.0 stack does not share a failure domain with revenue workloads. The shared-VPS stack becomes permanent STAGING (epic buzz-ocv). Prod = the proven staging artifacts (same digest-pinned compose) with FRESH secrets; staging keys never promote. All go-live gates run against prod before any invite. Concrete host detail (addressing, keys, DNS, deploy specifics) lives ONLY in the private intent-os ops/buzz lane.","notes":"GitHub: intent-solutions-io/buzz#9 — https://github.com/intent-solutions-io/buzz/issues/9. MIRROR RULE: bd-sync handles fan-out — `bd-sync note buzz-nry` and `bd-sync close buzz-nry` mirror to GH and Plane automatically.\n\nPlane: BUZZ-3 — (Plane, internal)\nProd pairing milestone 2026-07-30: mobile device pairing now LIVE on the production host (buzz-prod.intentsolutions.io) — sidecar + Caddy /pair deployed, owner confirmed a real device paired (intent-os PR #291). Prod relay + sidecar both healthy. Remaining under this prod epic: .2 apex DNS cutover off buzz-prod-\u003e buzz., .3 full go-live gate suite against prod.","status":"in_progress","priority":1,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:31:08Z","created_by":"jeremylongshore","updated_at":"2026-07-30T04:48:53Z","started_at":"2026-07-29T17:57:57Z","labels":["hosting"],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"buzz-bsy","title":"Build the Buzz three-store restore tool (restore.sh) and its restore drill","description":"The wrapped updater's ordered store-restore on a bad-migration revert calls ops/buzz/scripts/restore.sh (--recovery-point \u003cid\u003e --env \u003cenv\u003e) in intent-os. Until it exists+drilled, the updater fail-closes to exit 5 (manual page) rather than faking a DB rollback. Build restore.sh (pg_dump -Fc restore + MinIO media + git volume from a bound recovery point, dangling-ref walk) + a restore drill, so bad-migration recovery is fully unattended. Parent: wrapped-updater lane (buzz-ocv.4).","status":"open","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T20:13:56Z","created_by":"jeremylongshore","updated_at":"2026-07-29T20:13:56Z","labels":["backup","restore","updater"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-nry","title":"Stand up the dedicated Buzz production VPS and cut the prod domain over to it","description":"Track D: Buzz production runs on its OWN dedicated VPS (owner topology decision 2026-07-29, fork 000-docs/005) so a fast-moving pre-1.0 stack does not share a failure domain with revenue workloads. The shared-VPS stack becomes permanent STAGING (epic buzz-ocv). Prod = the proven staging artifacts (same digest-pinned compose) with FRESH secrets; staging keys never promote. All go-live gates run against prod before any invite. Concrete host detail (addressing, keys, DNS, deploy specifics) lives ONLY in the private intent-os ops/buzz lane.","notes":"GitHub: intent-solutions-io/buzz#9 — https://github.com/intent-solutions-io/buzz/issues/9. MIRROR RULE: bd-sync handles fan-out — `bd-sync note buzz-nry` and `bd-sync close buzz-nry` mirror to GH and Plane automatically.\n\nPlane: BUZZ-3 — (Plane, internal)\nProd pairing milestone 2026-07-30: mobile device pairing now LIVE on the production host (buzz-prod.intentsolutions.io) — sidecar + Caddy /pair deployed, owner confirmed a real device paired (intent-os PR #291). Prod relay + sidecar both healthy. Remaining under this prod epic: .2 apex DNS cutover off buzz-prod-\u003e buzz., .3 full go-live gate suite against prod.","status":"closed","priority":1,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:31:08Z","created_by":"jeremylongshore","updated_at":"2026-07-30T07:18:01Z","started_at":"2026-07-29T17:57:57Z","closed_at":"2026-07-30T07:18:01Z","close_reason":"EPIC DONE 2026-07-30. Dedicated Buzz production VPS (intent-ops-buzz, 169.58.95.32) stood up to estate conventions (.1), deployed with fresh secrets serving buzz-prod.intentsolutions.io (.2), and the full go-live gate suite passed against it (.3). Owner decision 2026-07-30: buzz-prod.intentsolutions.io is the PERMANENT production URL — the 'cut the prod domain over' (apex) part of this epic is intentionally NOT done and is retired, not deferred. All three children closed.","labels":["hosting"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-s8q","title":"Evaluate + owner-gated execute the Buzz relay upgrade off v0.2.0 via the wrapped updater (staging-rehearsed, pinned-sha — we'd be running AHEAD of Block's stable :latest)","description":"Finding 2026-07-31: Block's GHCR :latest relay tag == our current prod digest (a0f672, 'v0.2.0'); there is NO version-pinned relay release — v0.5.3-era relay code exists only in rolling :main / sha-\u003ccommit\u003e builds. So a relay upgrade = running UNRELEASED bleeding-edge code + a ~3-version schema migration on the live 49-person event store (exactly the D139 risk). Value is low relay-side (community-limit-\u003e5, NIP-11 max_limit, reconnect backoff); the agent reply-guard #3763 + Claude/Codex harnesses are AGENT-binary changes (rebuild our buzz-agent image, tracked under buzz-20l), NOT relay. Desktop clients already auto-updated to v0.5.3 and work against our relay (Nostr protocol-stable). Recommendation: stay on stable :latest until a real relay release exists OR buzz-20l needs a specific relay feature; if executed, pin a sha-\u003ccommit\u003e (not rolling :main), rehearse the migration on buzz-testing via ops/buzz/scripts/updater.sh, then prod with bound-snapshot + functional-probe + auto-revert. Owner-gated: this is an eyes-open 'run ahead of upstream stable' decision.","status":"open","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-08-01T02:40:44Z","created_by":"jeremylongshore","updated_at":"2026-08-01T02:40:44Z","labels":["agentic-ops","relay","updater"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-20l.8","title":"Cherry-pick the upstream Codex auto-reviewer (block/buzz #3715) onto the fork as a config injection","description":"buzz-acp-local approvals_reviewer=auto_review (#3715, OPEN upstream) — the closest drop-in for the dead Greptile PR-review. Cherry-pick as a fork config injection; does NOT touch the owner-gated remote-agents spec #3748. Acceptance: a PR triggers an automated Codex review comment on a test repo.","status":"open","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:13Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:13Z","labels":["agentic-ops","codex-review","estate","github-bridge"],"dependencies":[{"issue_id":"buzz-20l.8","depends_on_id":"buzz-20l.1","type":"blocks","created_at":"2026-07-31T13:58:38Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.8","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:12Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-20l.7","title":"Stand up repo-home channels for all active repos with lazy-create on next push","description":"One \u003crepo-name\u003e home channel per ACTIVE repo across both orgs (owner call: ALL active, ~74, minus archived and truly-dead; the ~15 dead repos get NO channel). Any other repo auto-creates its channel on its next real GitHub event (repo-channels.json is a cache derived from traffic, not a hand-authored census). Additive to the ~24 *-wire channels. Acceptance: active repos have home channels; a push to a channel-less repo auto-creates one; archived repos never get one.","status":"open","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:12Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:12Z","labels":["agentic-ops","channels","estate","github-bridge"],"dependencies":[{"issue_id":"buzz-20l.7","depends_on_id":"buzz-20l.3","type":"blocks","created_at":"2026-07-31T13:58:35Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.7","depends_on_id":"buzz-20l.4","type":"blocks","created_at":"2026-07-31T13:58:36Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.7","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:11Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":2,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-b6z.3","title":"Alert runtime relay errors/panics into sys-incidents and health into sys-health","description":"Today only deploy/backup failures page; a runtime relay panic/error goes unnoticed. Tail the relay logs (or a health endpoint) and route error/panic -\u003e sys-incidents, degraded-health -\u003e sys-health via the notifier. AC: a planted relay error lands in sys-incidents.","status":"open","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:40:57Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:40:57Z","dependencies":[{"issue_id":"buzz-b6z.3","depends_on_id":"buzz-b6z","type":"parent-child","created_at":"2026-07-30T21:40:57Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-b6z.3","depends_on_id":"buzz-0ts.1","type":"blocks","created_at":"2026-07-30T21:40:58Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-b6z","title":"Wire Buzz observability — trace export, log aggregation, error alerting, audit-chain","description":"Buzz has the bones (tracing_subscriber structured logs, OTel-export-ready relay code, the buzz-audit per-community SHA-256 hash-chain, Netdata host metrics, docker log rotation) but the wiring is incomplete. Close the gaps so a Buzz runtime problem is visible and pages. Feeds the sys-* channels (epic buzz-0ts).","status":"open","priority":2,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:40:55Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:40:55Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-1zr.3","title":"Golden behavioral evals for the 3 operator agents","description":"Golden-transcript evals (the opskit-golden pattern) for buzz-ops / buzz-feed-curator / buzz-relay-admin: assert each DOES the right sanctioned action and REFUSES illegal ones (DB surgery, deleting a non-owned channel, posting a secret, prod destructive drill). AC: eval suite green; a regression is caught.","status":"open","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:39:34Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:39:34Z","dependencies":[{"issue_id":"buzz-1zr.3","depends_on_id":"buzz-1zr","type":"parent-child","created_at":"2026-07-30T21:39:34Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-1zr.3","depends_on_id":"buzz-yfe.6","type":"blocks","created_at":"2026-07-30T21:39:36Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-1zr.3","depends_on_id":"buzz-yfe.5","type":"blocks","created_at":"2026-07-30T21:39:36Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-1zr.3","depends_on_id":"buzz-yfe.4","type":"blocks","created_at":"2026-07-30T21:39:35Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":3,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-1zr.2","title":"Hermetic tests for the new actions, wired into pnpm ci:drills","description":"Dry-run/mocked hermetic tests (no live relay) for channel-ops command construction, feed-card rendering (imeta), and notify topic-\u003echannel routing — same shape as the existing ops/buzz/tests/*.sh. AC: tests pass in ci:drills; a planted fault is caught.","status":"open","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:39:33Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:39:33Z","dependencies":[{"issue_id":"buzz-1zr.2","depends_on_id":"buzz-1zr","type":"parent-child","created_at":"2026-07-30T21:39:33Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-0ts.2","title":"Add af_buzz_transport to the af_dispatch chain (honest 2xx)","description":"A transport like vps-slack-transport.sh that posts to Buzz with honest 2xx classification, so Buzz alerts inherit dedup/rate-limit/spool/floor. AC: a governed alert routes through af_dispatch to a sys-* channel with a real receipt.","status":"open","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:19Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:29:19Z","dependencies":[{"issue_id":"buzz-0ts.2","depends_on_id":"buzz-0ts.1","type":"blocks","created_at":"2026-07-30T21:29:21Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-0ts.2","depends_on_id":"buzz-0ts","type":"parent-child","created_at":"2026-07-30T21:29:18Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0} +{"_type":"issue","id":"buzz-0ts.1","title":"Build buzz-notify.sh (topic -\u003e channel-uuid) + a notifier bot key","description":"A notifier bot (own key, separate from the AI Wire + goose keys) + buzz-notify.sh wrapping buzz messages send (kind:9), mapping a topic to a sys-* channel uuid. AC: a test alert posts to sys-incidents.","status":"open","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:18Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:29:18Z","dependencies":[{"issue_id":"buzz-0ts.1","depends_on_id":"buzz-0ts","type":"parent-child","created_at":"2026-07-30T21:29:18Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-0ts.1","depends_on_id":"buzz-w92.1","type":"blocks","created_at":"2026-07-30T21:29:21Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0} +{"_type":"issue","id":"buzz-0ts","title":"Route estate system notifications into the private sys-* channels","description":"Track B internal consolidation: the estate's ops alerts post into the private sys-* channels (incidents/health/backups/deploys/automation), retiring Slack/notify/Moshi. Keep the off-estate survivability floor (healthchecks.io + af_email_floor) and Ezekiel's blog emails. Inherits dedup/rate-limit/spool/floor from the governed af_dispatch chain.","status":"open","priority":2,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:17Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:29:17Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-w92.7","title":"Curation and the daily ai-wire digest (the anti-firehose moat)","description":"The curator promotes the day's best cross-lab items into one ai-wire digest + one announcements headline; members @mention to go deeper. AC: a real daily digest posts to ai-wire.","status":"closed","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:12Z","created_by":"jeremylongshore","updated_at":"2026-07-31T04:30:55Z","closed_at":"2026-07-31T04:30:55Z","close_reason":"Daily ai-wire digest live — --digest composes cross-lab roundup (70b→8b→NVIDIA), posts to 1-ai-wire, cron 06:30 CT","dependencies":[{"issue_id":"buzz-w92.7","depends_on_id":"buzz-w92.3","type":"blocks","created_at":"2026-07-30T21:29:16Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-w92.7","depends_on_id":"buzz-w92","type":"parent-child","created_at":"2026-07-30T21:29:11Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-w92.8","title":"Schedule the pipeline via cron and register it in automations.md","description":"Cron the ingestion + digest on the dev box (estate automation home), armed with notify-lib liveness markers, registered in mission-control/automations.md. AC: cron live + registry row + liveness sweep row.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:12Z","created_by":"jeremylongshore","updated_at":"2026-07-31T04:18:42Z","closed_at":"2026-07-31T04:18:42Z","close_reason":"3h cron deployed (~/bin/buzz-wire-cron.sh) + notify-lib liveness + registered in automations.md","dependencies":[{"issue_id":"buzz-w92.8","depends_on_id":"buzz-w92.3","type":"blocks","created_at":"2026-07-30T21:29:16Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-w92.8","depends_on_id":"buzz-w92","type":"parent-child","created_at":"2026-07-30T21:29:12Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-w92.4","title":"Wire the existing Anthropic monitor fleet into the anthropic-* channels","description":"The dev-box ~/bin monitors (claude-code-changelog, claude-blog, anthropic-news/research/engineering, anthropic-status) already summarize NEW items to Slack; add a Buzz post path -\u003e anthropic-changelog / anthropic-blogs / anthropic-research / anthropic-outages. AC: a real Anthropic release lands in anthropic-changelog.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:09Z","created_by":"jeremylongshore","updated_at":"2026-07-31T14:04:52Z","closed_at":"2026-07-31T14:04:52Z","close_reason":"Anthropic bridged via HTML-scrape source type (news/research/engineering -\u003e anthropic-blogs/research), mirroring the estate monitors; Mistral added. buzz-w92.6 partially done (tooling/mcp/newsletters/ships/status live).","dependencies":[{"issue_id":"buzz-w92.4","depends_on_id":"buzz-w92.1","type":"blocks","created_at":"2026-07-30T21:29:14Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-w92.4","depends_on_id":"buzz-w92","type":"parent-child","created_at":"2026-07-30T21:29:09Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-w92.3","title":"Ship the easy-tier provider feeds (clean RSS/status)","description":"Anthropic, OpenAI, Google/DeepMind, Groq, DeepSeek, MiniMax -\u003e their *-wire channels. AC: each posts real items on a schedule.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:08Z","created_by":"jeremylongshore","updated_at":"2026-07-31T04:18:40Z","closed_at":"2026-07-31T04:18:40Z","close_reason":"easy-tier feeds live — OpenAI, HuggingFace, arXiv cs.AI, Google DeepMind; 4 seed cards posted with summaries","dependencies":[{"issue_id":"buzz-w92.3","depends_on_id":"buzz-w92.2","type":"blocks","created_at":"2026-07-30T21:29:13Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-w92.3","depends_on_id":"buzz-w92","type":"parent-child","created_at":"2026-07-30T21:29:08Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":4,"comment_count":0} +{"_type":"issue","id":"buzz-yfe.6","title":"Build the buzz-relay-admin agent (membership, keys, backups, health)","description":"A /agent-creator agent for relay-admin ops (buzz-admin add/remove-member, keygen, backup/restore, health). AC: passes /validate-agent; encodes host access, buzz-admin-in-relay-container, backup/restore runbooks.","status":"closed","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:17Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:36:21Z","closed_at":"2026-07-31T03:36:21Z","close_reason":"buzz-relay-admin agent built + PASS /validate-agent","dependencies":[{"issue_id":"buzz-yfe.6","depends_on_id":"buzz-yfe.3","type":"blocks","created_at":"2026-07-30T21:29:03Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-yfe.6","depends_on_id":"buzz-yfe","type":"parent-child","created_at":"2026-07-30T21:27:16Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0} +{"_type":"issue","id":"buzz-yfe.7","title":"Validate the skill and all operator agents against the IS spec","description":"Run /validate-skillmd + /validate-agent on the skill and 3 agents; fix findings. AC: all green.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:17Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:36:22Z","closed_at":"2026-07-31T03:36:22Z","close_reason":"all 3 agents PASS /validate-agent; skill PASS /validate-skillmd; reference-vs-code consistency 106 citations 100% resolve to real fork source (no fabrications)","dependencies":[{"issue_id":"buzz-yfe.7","depends_on_id":"buzz-yfe.6","type":"blocks","created_at":"2026-07-30T21:29:05Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-yfe.7","depends_on_id":"buzz-yfe.4","type":"blocks","created_at":"2026-07-30T21:29:03Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-yfe.7","depends_on_id":"buzz-yfe","type":"parent-child","created_at":"2026-07-30T21:27:17Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-yfe.7","depends_on_id":"buzz-yfe.5","type":"blocks","created_at":"2026-07-30T21:29:04Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":3,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-yfe.5","title":"Build the buzz-feed-curator agent (AI-Wire posting)","description":"A /agent-creator agent that fetches feeds, summarizes, and posts rich cards to *-wire channels as the AI Wire bot. AC: passes /validate-agent; encodes the posting primitive, provider-first routing, membership requirement.","status":"closed","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:16Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:36:19Z","closed_at":"2026-07-31T03:36:19Z","close_reason":"buzz-feed-curator agent built + PASS /validate-agent","dependencies":[{"issue_id":"buzz-yfe.5","depends_on_id":"buzz-yfe.3","type":"blocks","created_at":"2026-07-30T21:29:02Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-yfe.5","depends_on_id":"buzz-yfe","type":"parent-child","created_at":"2026-07-30T21:27:15Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0} +{"_type":"issue","id":"buzz-yfe.4","title":"Build the buzz-ops operator agent (channel + identity management)","description":"A /agent-creator agent that manages channels + identities on the live relay per the runbook. AC: passes /validate-agent (8-field spec); encodes owner-key ownership, sanctioned CLI, secret-via-docker-exec-e, verify-via-channels-list.","status":"closed","priority":2,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:15Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:36:18Z","closed_at":"2026-07-31T03:36:18Z","close_reason":"buzz-ops agent built + PASS /validate-agent (14-field standalone spec)","dependencies":[{"issue_id":"buzz-yfe.4","depends_on_id":"buzz-yfe.3","type":"blocks","created_at":"2026-07-30T21:29:00Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-yfe.4","depends_on_id":"buzz-yfe","type":"parent-child","created_at":"2026-07-30T21:27:14Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0} +{"_type":"issue","id":"buzz-ehv.2","title":"Define and document the member join/onboarding model","description":"For the shared community: members JOIN the canonical open channels (they never template-create their own — templates make NEW channels and cause the duplicate-Welcome mess; see RUNBOOK-channels.md 1.6). AC: document which channels auto-join at onboarding vs opt-in, how a member joins an open *-wire pack, and the seed-content-before-invite step. Feeds the all-in onboarding epic (E5).","status":"open","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:11Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:27:11Z","dependencies":[{"issue_id":"buzz-ehv.2","depends_on_id":"buzz-ehv","type":"parent-child","created_at":"2026-07-30T21:27:10Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-bsy","title":"Build the Buzz three-store restore tool (restore.sh) and its restore drill","description":"The wrapped updater's ordered store-restore on a bad-migration revert calls ops/buzz/scripts/restore.sh (--recovery-point \u003cid\u003e --env \u003cenv\u003e) in intent-os. Until it exists+drilled, the updater fail-closes to exit 5 (manual page) rather than faking a DB rollback. Build restore.sh (pg_dump -Fc restore + MinIO media + git volume from a bound recovery point, dangling-ref walk) + a restore drill, so bad-migration recovery is fully unattended. Parent: wrapped-updater lane (buzz-ocv.4).","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T20:13:56Z","created_by":"jeremylongshore","updated_at":"2026-07-30T05:54:52Z","closed_at":"2026-07-30T05:54:52Z","close_reason":"DONE 2026-07-30 (ultracode Track-A workflow). Built ops/buzz/scripts/restore.sh — the real destructive three-store restore (Postgres pg_restore --clean --if-exists → media mc mirror --overwrite → git volume), gated behind --in-place with verify-before-destroy (sha256 every artifact + PGDMP header + manifest.env==--env cross-check; exit 2 writes nothing). Chose --in-place-required over auto-restore so decision-log/038's human-gate holds with ZERO updater.sh changes: the updater's flagless call exits 64 → rc5 page unchanged (updater-drill 6/6 still green). Hermetic restore-test.sh 6/6 with mutation-proven teeth; restore-drill-test 4/4 no regression; wired into ci:drills. Merged intent-os PR #293 (4/4 estate-CI green). Remaining real-tool proof = a host-side scratch-stack drill before first prod use (noted in runbook).","labels":["backup","restore","updater"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-ocv.6","title":"Add the mobile pairing sidecar overlay and validate a real desktop-to-mobile pairing","description":"Track E2 (external infra review; upstream #2734 / PR #2736): the first-party compose bundle lacks the buzz-pair-relay sidecar, so Android members cannot pair — and the all-in onboarding wave includes Android. Review the pinned image for the buzz-pair-relay entrypoint, add an estate overlay service (same image, entrypoint buzz-pair-relay, internal :5000, healthcheck) + a Caddy /pair* route. Claim it works only after a REAL desktop-to-mobile pairing succeeds; if wave 1 doesn't require it, document the limitation honestly instead.","notes":"PAIRING BLOCKED BY IMAGE (found 2026-07-30 during a live prod attempt): the pinned relay image ghcr.io/block/buzz@sha256:a0f672... (relay-v0.2.0) contains ONLY buzz-admin + buzz-relay in /usr/local/bin — the buzz-pair-relay binary is NOT in it (added in a NEWER buzz build; the Dockerfile builds it but our pinned digest predates/excludes it). So the pairing overlay's entrypoint 'buzz-pair-relay' -\u003e 'executable file not found'. NEAR-MISS: 'docker compose up -d' with the overlay recreated the relay (overlay patches relay env for BUZZ_PAIRING_RELAY_URL) AND the sidecar failed to start, which took the relay DOWN briefly; reverted (removed COMPOSE_FILE + overlay, base up) -\u003e relay healthy, prod restored. LESSONS: (1) pairing REQUIRES promoting a newer buzz image that ships buzz-pair-relay — do it via the drilled wrapped-updater (snapshot-\u003eprobe-\u003erevert), never a raw compose up on the live relay; (2) the overlay must PROBE the image for /usr/local/bin/buzz-pair-relay before starting, and bring up ONLY the sidecar service first (never recreate the relay on sidecar failure); (3) hermetic pairing-overlay-test passed because it mocked the binary — add a real-image binary-presence check. The relay DID advertise BUZZ_PAIRING_RELAY_URL correctly (app generated a valid nostrpair:// deep-link) — only the sidecar endpoint was missing.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:30:45Z","created_by":"jeremylongshore","updated_at":"2026-07-30T04:47:02Z","started_at":"2026-07-29T22:43:42Z","closed_at":"2026-07-30T04:47:02Z","close_reason":"DONE 2026-07-30. Mobile pairing sidecar deployed to prod + a REAL device paired (owner-confirmed handshake). Root cause: pinned relay-v0.2.0 image lacks buzz-pair-relay; fix pins the sidecar SEPARATELY via BUZZ_PAIR_RELAY_IMAGE (:main @9de8aff1) + absolute entrypoint, deployed sidecar-first (relay untouched), Caddy /pair route added. Verified wss://buzz-prod.intentsolutions.io/pair -\u003e 101 Switching Protocols (Caddy-\u003esidecar); sidecar logs show the live paired connection. intent-os PR #291 (overlay + test image-guard + runbook; 4/4 CI green). pairing-overlay-test now guards the image bug (5/5).","labels":["clients","hosting"],"dependencies":[{"issue_id":"buzz-ocv.6","depends_on_id":"buzz-ocv","type":"parent-child","created_at":"2026-07-29T10:30:45Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-ocv.5","title":"Set the Tauri desktop origins in the relay CORS config and verify with the packaged desktop client","description":"Track E1 (external infra review, verified vs upstream #3490): current env allows only https://the prod domain as an origin; packaged Tauri desktop clients present tauri://localhost (macOS/Linux) or http://tauri.localhost (Windows), so desktop join is BLOCKED. Verify the pinned image's CORS env handling, add the two Tauri origins (no permissive-CORS shortcuts), restart, confirm relay healthy + correct Access-Control-Allow-Origin echo. Final verification is a join from the PACKAGED desktop client on the owner's workstation, not a dev build.","notes":"CORS CONFIG FIXED (intent-os PR #283, merged via estate-CI 3 green). BUZZ_CORS_ORIGINS composed in version-controlled compose from required BUZZ_DOMAIN + tauri://localhost + http://tauri.localhost (environment: overrides env_file; non-secret Tauri origins out of sops). Closes the desktop-join gap AND a latent permissive-CORS gap (relay goes permissive only when the var is EMPTY; required BUZZ_DOMAIN makes empty impossible). Proof: ops/buzz/tests/cors-config-test.sh 2/2, wired into ci:drills. REMAINING (owner-gated): apply on staging (docker compose up -d relay) + verify a real join with the PACKAGED desktop client on Jeremy's workstation (dev build won't exercise Tauri origins).\nCORS APPLIED LIVE to BOTH hosts 2026-07-29 (intent-os PR #284 records; compose applied with backup + config-gate + relay recreate). Staging BUZZ_CORS_ORIGINS=https://the prod domain,tauri://localhost,http://tauri.localhost ; prod=…buzz-prod…,tauri…; relay healthy + _readiness 200 on each. Desktop-join unblocked on the live boxes. ONLY remaining: packaged-desktop-client verify on Jeremy's workstation (owner-gated).","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T16:30:36Z","created_by":"jeremylongshore","updated_at":"2026-07-30T04:47:05Z","started_at":"2026-07-29T21:16:21Z","closed_at":"2026-07-30T04:47:05Z","close_reason":"DONE 2026-07-30. Tauri desktop CORS origins (tauri://localhost + http://tauri.localhost) composed into version-controlled compose (intent-os PR #283), applied live to both hosts (PR #284), cors-config-test 2/2 in ci:drills. Final owner-gated gate met: Jeremy joined from the PACKAGED Linux desktop client to buzz-prod.intentsolutions.io (connected, admin, channels visible) — the packaged-client CORS verification the acceptance required.","labels":["clients","hosting"],"dependencies":[{"issue_id":"buzz-ocv.5","depends_on_id":"buzz-ocv","type":"parent-child","created_at":"2026-07-29T10:30:35Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-ocv.4","title":"Wire backups and the wrapped updater, register everything in the estate","description":"nightly pg_dump artifact into the borg set + three-store recovery point; weekly updater snapshot-promote-probe-revert; automations.md + catalog rows; E3 drill gates filed","notes":"Track E additions (external infra review 2026-07-29): the wrapped updater's promotion checklist MUST include — release-notes review; a staging boot; a probe-hang check (upstream #2723/PR #2724 — git conformance probe can wedge startup; our pinned digest boots healthy with BUZZ_GIT_CONFORMANCE_PROBE=true, but a future image must prove it on staging before promotion); a CORS/Tauri-origin regression check; a pairing regression check. Also confirmed: upstream's own 'backup' command is a checklist, not a backup — our pg_dump + three-store recovery-point design stands; off-site copy rides the estate B2/borg chain.\nBackup automation BUILT + RAN (evidence): scripts/backup.sh on prod produced encrypted borg archive intent-ops-buzz-prod-2026-07-29T183641Z (pg_dump 167KB schema-head-6 + media(6) + git + identity + manifest w/ sha256s); flock, retention 14/8/6, freshness marker, Slack notify (interim — Buzz replaces Slack post-migration), nonzero-on-fail. systemd buzz-backup.timer enabled (daily 04:20 UTC). Runbook ops/buzz/RUNBOOK-backup-restore.md. STILL OPEN: off-site B2 push (owner-gated estate-wide, Epic 1.1 B2 not provisioned) + borg repokey export off-box + Netdata backup-age alert + the wrapped updater.\nGIT CONFORMANCE PROBE risk (Priority 4) — evidence-based decision: pinned image = v0.2.0 / source rev 0d9be2f (built 2026-07-10). Upstream fix PR #2724 is still OPEN (NOT merged) -\u003e the bounded-timeout fix is NOT in our image. BUT fault test on a scratch relay (MinIO unreachable + BUZZ_GIT_CONFORMANCE_PROBE=true): the probe ran ('git object-store conformance probe A3 gate') and the relay EXITED with 's3 backend error' after ~63s (bounded by reqwest timeout) — it FAILS CLOSED, does not hang indefinitely. DECISION: KEEP the probe ON (real consistency gate; compose gates minio-healthy-before-relay; fails-closed not hangs), and MANDATE a deploy-wrapper HARD TIMEOUT (Priority 3) that bounds the relay-healthy wait + auto-reverts regardless of app behaviour — this covers the untested 'accepted-but-nonresponsive' case. Removal-of-risk condition: promote a digest containing merged #2724. Untested residual: MinIO-nonresponsive half-open (harder to reproduce safely).\nWrapped-updater half SHIPPED (intent-os PR #282, merged via estate-CI 3 green: gates 127s / drills 220s / gitleaks). Built ops/buzz/scripts/{updater,functional-probe,_repin-and-start}.sh over the drilled estate watchtower-gate/deploy-wrapper; hermetic planted-fault drill ops/buzz/tests/updater-drill.sh 6/6 (clean promote, bad-release auto-revert+store-restore+re-probe, hard-timeout hang guard rc124, scan-hold fail-closed, snapshot-abort, absent-restore manual page); systemd buzz-updater.{service,timer} (weekly Sun 05:30 UTC); runbook RUNBOOK-wrapped-updater.md; registered backup+updater timers in mission-control/automations.md; drill wired into ci:drills. STILL OPEN on .4: install the timer on the intent-ops-buzz production host + first real run; restore.sh three-store tool (follow-up bead filed). Real functional-probe-against-live-staging is buzz-ocv.3.\nUpdater lane DEPLOYED to staging (dormant) via install-updater-lane.sh (intent-os PR #284): 3 buzz scripts + 2 estate deps (watchtower-gate/deploy-wrapper -\u003e /srv/buzz/lib/) + systemd units; daemon-reload OK, timer disabled/inactive. Fixed container-name (buzz-relay-1) + governed-notify + systemd env wiring from live state. Prod updater install + timer-enable gated on the real staging functional-probe rehearsal (blocked on installing nak — new bead).\nGate TOOLING built + merged (intent-os PR #287, CI + hermetic tests, owner-authorized merge — reviewers unavailable): monitoring (outside-in-buzz control + backup-age watchdog 3/3), key-incident runbooks (7/7), smoke-suite + unauth-matrix (4/4), all in ci:drills. REMAINING = live runs: functional probe + unauth matrix against live prod off-network; install backup-age timer + outside-in probe; rehearse key runbooks. Parallel-authored by 3 subagents.","status":"in_progress","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T15:47:20Z","created_by":"jeremylongshore","updated_at":"2026-07-29T23:57:52Z","started_at":"2026-07-29T18:34:08Z","labels":["hosting"],"dependencies":[{"issue_id":"buzz-ocv.4","depends_on_id":"buzz-ocv","type":"parent-child","created_at":"2026-07-29T09:47:20Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-ocv.3","title":"Prove the deployment: smokes, functional membership probe, and the off-network unauth probe matrix","description":"/health + /_readiness + NIP-11; nak NIP-42 member publish/readback + un-invited refusal; media PUT / git packs / hooks endpoints all 401-403","notes":"Gate TOOLING built + merged (intent-os PR #287, CI + hermetic tests, owner-authorized merge — reviewers unavailable): monitoring (outside-in-buzz control + backup-age watchdog 3/3), key-incident runbooks (7/7), smoke-suite + unauth-matrix (4/4), all in ci:drills. REMAINING = live runs: functional probe + unauth matrix against live prod off-network; install backup-age timer + outside-in probe; rehearse key runbooks. Parallel-authored by 3 subagents.\nFunctional membership probe + smoke suite + unauth matrix PROVEN GREEN on live staging (PR #288). Prod runs remain (buzz-2i6).","status":"open","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T15:47:19Z","created_by":"jeremylongshore","updated_at":"2026-07-30T00:37:48Z","labels":["hosting"],"dependencies":[{"issue_id":"buzz-ocv.3","depends_on_id":"buzz-ocv","type":"parent-child","created_at":"2026-07-29T09:47:18Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"buzz-ocv.2","title":"Wire the estate Caddy ingress with the compensating edge controls","description":"site block + WS passthrough, body-size caps, timeouts, security headers; caddy validate + reload","status":"open","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T15:47:18Z","created_by":"jeremylongshore","updated_at":"2026-07-29T15:47:18Z","labels":["hosting"],"dependencies":[{"issue_id":"buzz-ocv.2","depends_on_id":"buzz-ocv","type":"parent-child","created_at":"2026-07-29T09:47:17Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-ocv.2","title":"Wire the estate Caddy ingress with the compensating edge controls","description":"site block + WS passthrough, body-size caps, timeouts, security headers; caddy validate + reload","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T15:47:18Z","created_by":"jeremylongshore","updated_at":"2026-07-30T05:45:54Z","closed_at":"2026-07-30T05:45:54Z","close_reason":"DONE 2026-07-30 (ultracode Track-A workflow). Compensating edge controls shipped + applied. CODE: ops/buzz/scripts/apply-edge-controls.sh (plan-only default, --apply to install) + hermetic test, merged intent-os PR #292 (4/4 estate-CI green, adversarially verified sound). APPLIED LIVE on the dedicated prod host intent-ops-buzz: fail2ban 'caddy-buzz' jail (bans public 4xx-flooders from /var/log/caddy/buzz-access.log) + nftables 'inet buzz_edge' connlimit (128 concurrent NEW/IP on tcp 80+443 only) + buzz-edge-nft.service enabled (reboot-safe). VERIFIED no self-lock: SSH intact, relay _readiness 200 off-box, tailnet/port-22 never touched (WS/established never dropped). SCOPING: applied to the Buzz-DEDICATED prod host only; deliberately NOT applied host-wide on the shared VPS staging (its connlimit would rate-limit co-tenant estate stacks Plane/ERP/CRM) — request_body caps + security headers already exist in Caddy on both. Closes the last non-owner-gated piece of go-live edge hardening.","labels":["hosting"],"dependencies":[{"issue_id":"buzz-ocv.2","depends_on_id":"buzz-ocv","type":"parent-child","created_at":"2026-07-29T09:47:17Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-ocv.1","title":"Provision DNS, secrets, and the /srv/buzz compose stack, booted closed from first start","description":"dns + sops env (relay identity key, bootstrap op owner key, HMAC, db/redis/minio creds) + estate-owned compose with digest-pinned image, loopback publish, resource caps","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T15:47:17Z","created_by":"jeremylongshore","updated_at":"2026-07-29T15:52:10Z","closed_at":"2026-07-29T15:52:10Z","close_reason":"the prod domain A record live (Porkbun lane); secrets generated (relay identity + bootstrap op owner + HMAC + db/redis/minio) deployed as /srv/buzz/.env mode 600 with sops master in intent-os ops/buzz/secrets/buzz.prod.env.sops (dev+VPS age recipients); estate-owned compose (digest-pinned image, loopback publish 3004/3084, mem/cpus/pids caps, own bridge net) deployed; all 4 containers healthy; _readiness 200; NIP-11 serving; closed-relay env from first boot","labels":["hosting"],"dependencies":[{"issue_id":"buzz-ocv.1","depends_on_id":"buzz-ocv","type":"parent-child","created_at":"2026-07-29T09:47:16Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"buzz-ocv","title":"Stand up the closed Buzz relay stack on the estate VPS behind the wrapped update lane","description":"RE-SCOPED 2026-07-29 (owner topology decision, fork 000-docs/005): this stack on the SHARED estate VPS is STAGING, permanently — same compose, same gates; restore drills and updater planted-fault drills run here, never on prod. Production moves to a DEDICATED VPS (Track D epic). Original scope: /srv/buzz compose stack (relay pinned by digest, Postgres 17, Redis 7, MinIO interim) closed from first boot, sops-held secrets, loopback publish behind estate Caddy, resource caps, DNS, smokes + unauth probe matrix, pg_dump backup wiring, wrapped updater. At cutover this env renames to the staging domain. E3 go-live gates run against PROD on the dedicated box before any invite.","notes":"GitHub: intent-solutions-io/buzz#8 — https://github.com/intent-solutions-io/buzz/issues/8. MIRROR RULE: bd-sync handles fan-out — `bd-sync note buzz-ocv` and `bd-sync close buzz-ocv` mirror to GH and Plane automatically.\n\nPlane: BUZZ-2 — (Plane, internal)\nTOPOLOGY RE-SCOPE (owner decision 2026-07-29, recorded as fork 000-docs/005): this shared-VPS stack is re-designated permanent STAGING — same compose, same gates; restore drills and updater planted-fault drills run here, never on prod. Production moves to a dedicated VPS (epic buzz-nry / GH #9 / Plane BUZZ-3). At cutover this env renames to the staging domain; staging keys never promote to prod. New Track-E children under this epic: .5 Tauri/CORS desktop-join fix, .6 mobile pairing sidecar overlay; updater promotion-checklist additions noted on .4.\nSession 2026-07-30: two children COMPLETE — .6 mobile pairing (real device paired; sidecar via BUZZ_PAIR_RELAY_IMAGE + Caddy /pair; intent-os PR #291) and .5 Tauri CORS (packaged Linux desktop client joined prod). Remaining under this staging epic: .2 (Caddy edge controls), .3 (smokes+unauth matrix), .4 (backups+updater register).","status":"in_progress","priority":2,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T15:47:04Z","created_by":"jeremylongshore","updated_at":"2026-07-30T04:47:23Z","started_at":"2026-07-29T15:47:21Z","labels":["hosting"],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-ocv","title":"Stand up the closed Buzz relay stack on the estate VPS behind the wrapped update lane","description":"RE-SCOPED 2026-07-29 (owner topology decision, fork 000-docs/005): this stack on the SHARED estate VPS is STAGING, permanently — same compose, same gates; restore drills and updater planted-fault drills run here, never on prod. Production moves to a DEDICATED VPS (Track D epic). Original scope: /srv/buzz compose stack (relay pinned by digest, Postgres 17, Redis 7, MinIO interim) closed from first boot, sops-held secrets, loopback publish behind estate Caddy, resource caps, DNS, smokes + unauth probe matrix, pg_dump backup wiring, wrapped updater. At cutover this env renames to the staging domain. E3 go-live gates run against PROD on the dedicated box before any invite.","notes":"GitHub: intent-solutions-io/buzz#8 — https://github.com/intent-solutions-io/buzz/issues/8. MIRROR RULE: bd-sync handles fan-out — `bd-sync note buzz-ocv` and `bd-sync close buzz-ocv` mirror to GH and Plane automatically.\n\nPlane: BUZZ-2 — (Plane, internal)\nTOPOLOGY RE-SCOPE (owner decision 2026-07-29, recorded as fork 000-docs/005): this shared-VPS stack is re-designated permanent STAGING — same compose, same gates; restore drills and updater planted-fault drills run here, never on prod. Production moves to a dedicated VPS (epic buzz-nry / GH #9 / Plane BUZZ-3). At cutover this env renames to the staging domain; staging keys never promote to prod. New Track-E children under this epic: .5 Tauri/CORS desktop-join fix, .6 mobile pairing sidecar overlay; updater promotion-checklist additions noted on .4.\nSession 2026-07-30: two children COMPLETE — .6 mobile pairing (real device paired; sidecar via BUZZ_PAIR_RELAY_IMAGE + Caddy /pair; intent-os PR #291) and .5 Tauri CORS (packaged Linux desktop client joined prod). Remaining under this staging epic: .2 (Caddy edge controls), .3 (smokes+unauth matrix), .4 (backups+updater register).\nTrack-A ultracode session 2026-07-30: child .2 (edge controls) CLOSED — fail2ban + nftables connlimit applied live on the prod host, verified no self-lock (intent-os PR #292). Remaining children: .3 (staging probe matrix), .4 (backups+updater register).","status":"in_progress","priority":2,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T15:47:04Z","created_by":"jeremylongshore","updated_at":"2026-07-30T05:46:37Z","started_at":"2026-07-29T15:47:21Z","labels":["hosting"],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-4ei.4","title":"Mirror the epic three-way and seed repo and estate memory","description":"GitHub cluster issue on intent-solutions-io/buzz + Plane BUZZ project via bd-sync link; auto-memory for the buzz working dir; estate memory buzz-adoption-state; private ops/buzz/README.md operator authority.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T04:24:21Z","created_by":"jeremylongshore","updated_at":"2026-07-29T04:36:11Z","closed_at":"2026-07-29T04:36:11Z","close_reason":"Three-way mirror live: buzz-4ei linked via bd-sync to intent-solutions-io/buzz#1 + Plane BUZZ-1 (comments fanned out); Plane project BUZZ created; estate anchor spine-8fl linked to the same pair; memories seeded (buzz working-dir auto-memory + intent-os buzz-adoption-state + ops/buzz/README.md operator authority)","labels":["bead-tooling"],"dependencies":[{"issue_id":"buzz-4ei.4","depends_on_id":"buzz-4ei","type":"parent-child","created_at":"2026-07-28T22:24:21Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-4ei.3","title":"Implement the missing test layers with the in-repo audit harness, Layer 1 git hooks required","description":"/implement-tests for gaps found by the baseline audit; in-repo enforcement only (cargo/vendored harness), additive files only, staged for review.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T04:24:20Z","created_by":"jeremylongshore","updated_at":"2026-07-29T06:21:36Z","closed_at":"2026-07-29T06:21:36Z","close_reason":"Merged in PR #6 (fa14f3f16): scripts/fork-gates/{additive-only,must-survive} + vendored audit-harness + 10-file hash-pin manifest + tracked lefthook-local.yml (escape-scan pre-commit; gates+verify pre-push) + decision record 000-docs/004. All four gates green on the merged tree; zero upstream-path edits","labels":["test-hygiene"],"dependencies":[{"issue_id":"buzz-4ei.3","depends_on_id":"buzz-4ei","type":"parent-child","created_at":"2026-07-28T22:24:20Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-4ei.2","title":"Run the testing-SOP baseline audit and produce TEST_AUDIT.md against the 7-layer taxonomy","description":"Diagnostic only: classify the fork, map upstream's suite (just test-unit / docker integration / e2e via buzz-test-client) to the 7 layers, write TEST_AUDIT.md. Staged, not auto-committed.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T04:24:19Z","created_by":"jeremylongshore","updated_at":"2026-07-29T04:26:01Z","closed_at":"2026-07-29T04:26:01Z","close_reason":"TEST_AUDIT.md written (diagnostic only): upstream suite strong across all 7 layers (~5700 Rust test fns, lefthook L1, clippy/biome/cargo-deny L2, Playwright E2E); fork-lane gaps = additive-only invariant gate + in-repo harness, handed to buzz-4ei.3","labels":["test-hygiene"],"dependencies":[{"issue_id":"buzz-4ei.2","depends_on_id":"buzz-4ei","type":"parent-child","created_at":"2026-07-28T22:24:19Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-4ei.1","title":"Author the master blueprint, decision record, deploy posture, and FORK.md as additive-only artifacts","description":"000-docs/{000-INDEX,001-PP-PLAN,002-DR-DECR,003-OD-DEPL} + FORK.md; zero upstream-path edits.","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T04:24:18Z","created_by":"jeremylongshore","updated_at":"2026-07-29T04:26:00Z","closed_at":"2026-07-29T04:26:00Z","close_reason":"Authored additive-only: 000-docs/{000-INDEX,001-PP-PLAN blueprint,002-DR-DECR decision record,003-OD-DEPL deploy posture} + FORK.md; git diff upstream/main --stat shows only fork-added paths","labels":["fork-infra"],"dependencies":[{"issue_id":"buzz-4ei.1","depends_on_id":"buzz-4ei","type":"parent-child","created_at":"2026-07-28T22:24:18Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-4ei","title":"Stand up the Intent Solutions fork infrastructure for the Buzz adoption","description":"Phase 1 / E1 of 000-docs/001-PP-PLAN-buzz-adoption-master-blueprint.md: fork + clone + beads/Dolt activation, master blueprint + decision record + deploy posture + FORK.md, testing-SOP baseline (TEST_AUDIT.md), three-way mirror (GitHub cluster issue + Plane BUZZ) and memory seeding. Additive-only: zero upstream-path edits.","notes":"GitHub: intent-solutions-io/buzz#1 — https://github.com/intent-solutions-io/buzz/issues/1. MIRROR RULE: bd-sync handles fan-out — `bd-sync note buzz-4ei` and `bd-sync close buzz-4ei` mirror to GH and Plane automatically.\n\nPlane: BUZZ-1 — (Plane, internal)\nE1 milestone: blueprint set + FORK.md + TEST_AUDIT.md authored (buzz-4ei.1, .2 closed); beads write-path root-caused (bd contributor-role fork routing) and fixed via --role maintainer; PR opening next. Note: epic was re-minted from buzz-ekg to buzz-4ei during the store rebuild.\nE1 shipped: PR intent-solutions-io/buzz#2 squash-merged to fork main (6cf4df96f) after CodeRabbit review (4 findings fixed in e49ed40a6, 2 declined with reasons). Children .1/.2/.4 closed with evidence; .3 (fork-lane test layers) remains open — epic stays in_progress until it lands. Deferred: epic-boundary Dolt tag until dolt-mcp-vcs is available (system dolt CLI is version-skewed vs bd's embedded engine).","status":"closed","priority":2,"issue_type":"epic","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T04:24:17Z","created_by":"jeremylongshore","updated_at":"2026-07-29T06:21:39Z","started_at":"2026-07-29T04:25:59Z","closed_at":"2026-07-29T06:21:39Z","close_reason":"E1 complete: all 4 children closed with evidence (PRs #2/#3/#5/#6 merged; blueprint ledger E1=COMPLETE; GH cluster #1 auto-closed by PR #6). Fork contract now machine-enforced. Phase 2 hosting opens per blueprint after the MX cutover.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"buzz-bkt","title":"acceptance probe","status":"closed","priority":2,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-29T04:23:48Z","created_by":"jeremylongshore","updated_at":"2026-07-29T04:23:49Z","started_at":"2026-07-29T04:23:49Z","closed_at":"2026-07-29T04:23:49Z","close_reason":"probe","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-20l.10","title":"Schema the goose-filed issue (finding, repro, executable acceptance, stop-conditions, blast-radius) and lint it in pnpm check (deferred)","description":"The filed issue is the program the impl agent executes — refuse to hand it an unschematized one. Adopt upstream #3928 'agent work contracts' schema when it lands. Deferred until goose triage precision is proven.","status":"open","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:15Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:15Z","labels":["agentic-ops","estate","github-bridge","schema"],"dependencies":[{"issue_id":"buzz-20l.10","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:14Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.10","depends_on_id":"buzz-20l.5","type":"blocks","created_at":"2026-07-31T13:58:39Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-20l.9","title":"Replay the dead reviewers' historical output as a golden eval set and MiniMax model-drift detector (deferred)","description":"Greptile/Gemini/CodeRabbit historical output on merged PRs = a free labeled dataset. Run goose/MiniMax over the same diffs to measure findings-recall + false-positive rate; use as a model-drift detector when MiniMax-M3 changes under us. Deferred until the triage lane is live.","status":"open","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T19:58:14Z","created_by":"jeremylongshore","updated_at":"2026-07-31T19:58:14Z","labels":["agentic-ops","estate","eval","github-bridge"],"dependencies":[{"issue_id":"buzz-20l.9","depends_on_id":"buzz-20l","type":"parent-child","created_at":"2026-07-31T13:58:13Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-20l.9","depends_on_id":"buzz-20l.5","type":"blocks","created_at":"2026-07-31T13:58:39Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-b6z.4","title":"Verify and surface the buzz-audit hash-chain (tamper-evident audit log)","description":"The buzz-audit crate keeps a per-community SHA-256 hash-chain. Add a periodic chain-verify (like the estate ico audit verify) + surface a break to sys-incidents. AC: chain-verify runs green; a planted break alerts.","status":"open","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:40:58Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:40:58Z","dependencies":[{"issue_id":"buzz-b6z.4","depends_on_id":"buzz-b6z","type":"parent-child","created_at":"2026-07-30T21:40:58Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-b6z.1","title":"Export relay OTel traces to SigNoz staging (set OTEL_EXPORTER_OTLP_ENDPOINT)","description":"The relay already attaches an OpenTelemetry exporter when OTEL_EXPORTER_OTLP_ENDPOINT is set (buzz-relay/src/main.rs). Point it at the estate SigNoz C8 gateway (+ OTEL_SERVICE_NAME=buzz-relay). AC: traces visible in SigNoz staging; prod untouched until verified on staging.","status":"open","priority":3,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:40:56Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:40:56Z","dependencies":[{"issue_id":"buzz-b6z.1","depends_on_id":"buzz-b6z","type":"parent-child","created_at":"2026-07-30T21:40:55Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-b6z.2","title":"Ship Buzz relay/agent logs off-box before they age out","description":"Docker json-file keeps only ~30MB (10m x3), so a crash's logs vanish. Ship to the estate log store (Loki/whatever the estate standardizes) or at minimum rotate into the borg backup. AC: relay logs survive a container recycle + are queryable.","status":"open","priority":3,"issue_type":"feature","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:40:56Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:40:56Z","dependencies":[{"issue_id":"buzz-b6z.2","depends_on_id":"buzz-b6z","type":"parent-child","created_at":"2026-07-30T21:40:56Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-1zr.4","title":"Wire the reference-vs-code citation-consistency check as a repeatable gate","description":"Promote the ad-hoc citation checker (106 citations -\u003e real fork file:line) into a committed script so BUZZ-SYSTEM-REFERENCE.md can't silently drift from block/buzz. AC: script committed + runnable; flags a broken citation.","status":"open","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:39:35Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:39:35Z","dependencies":[{"issue_id":"buzz-1zr.4","depends_on_id":"buzz-yfe.1","type":"blocks","created_at":"2026-07-30T21:39:37Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-1zr.4","depends_on_id":"buzz-1zr","type":"parent-child","created_at":"2026-07-30T21:39:35Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-0ts.3","title":"Wire the sys-* channels to their estate sources","description":"incidents\u003c-prod incidents; health\u003c-liveness sweeps + labs health; backups\u003c-borg/off-site/restore-drill; deploys\u003c-deploy-wrapper events; automation\u003c-cron-failures + liveness digest. AC: each sys-* channel receives its real events.","status":"open","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:20Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:29:20Z","dependencies":[{"issue_id":"buzz-0ts.3","depends_on_id":"buzz-0ts.2","type":"blocks","created_at":"2026-07-30T21:29:22Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-0ts.3","depends_on_id":"buzz-0ts","type":"parent-child","created_at":"2026-07-30T21:29:19Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0} +{"_type":"issue","id":"buzz-0ts.4","title":"Soak, then retire Slack/notify/Moshi (keep the off-estate floor + Ezekiel email)","description":"After a soak proving Buzz alerts are reliable, retire notify.sh/notify-lib Slack paths + Moshi. KEEP: off-estate healthchecks.io + af_email_floor (survivability floor, must stay off-estate) + Ezekiel's blog posting emails. AC: Slack retired, floor intact.","status":"open","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:20Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:29:20Z","dependencies":[{"issue_id":"buzz-0ts.4","depends_on_id":"buzz-0ts","type":"parent-child","created_at":"2026-07-30T21:29:20Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-0ts.4","depends_on_id":"buzz-0ts.3","type":"blocks","created_at":"2026-07-30T21:29:23Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-w92.6","title":"Add the topic feeds beyond the labs","description":"huggingface, openrouter, benchmarks, tooling, arxiv, mcp, funding, security, newsletters, ships (dogfood our own) -\u003e their *-wire channels. AC: each posts real items.","status":"open","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:11Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:29:11Z","dependencies":[{"issue_id":"buzz-w92.6","depends_on_id":"buzz-w92","type":"parent-child","created_at":"2026-07-30T21:29:10Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-w92.6","depends_on_id":"buzz-w92.3","type":"blocks","created_at":"2026-07-30T21:29:15Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-w92.5","title":"Add the medium and hard provider feeds (RSSHub for scrape-only labs)","description":"Medium: Mistral, Together, Perplexity, Cohere. Hard (self-host RSSHub on the VPS): xAI, Meta, Moonshot/Kimi, Qwen -\u003e their *-wire. AC: RSSHub deployed; each feed posts.","status":"closed","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:29:10Z","created_by":"jeremylongshore","updated_at":"2026-07-31T15:35:29Z","closed_at":"2026-07-31T15:35:29Z","close_reason":"Filled groq/meta/mistral/cohere/perplexity/xai + cleaner anthropic via Olshansk RSS mirrors (from perception repo) — no RSSHub needed. Remaining empty (deepseek/qwen/moonshot/minimax/benchmarks/funding/openrouter) have no mirror.","dependencies":[{"issue_id":"buzz-w92.5","depends_on_id":"buzz-w92","type":"parent-child","created_at":"2026-07-30T21:29:10Z","created_by":"jeremylongshore","metadata":"{}"},{"issue_id":"buzz-w92.5","depends_on_id":"buzz-w92.3","type":"blocks","created_at":"2026-07-30T21:29:15Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"buzz-ehv.1","title":"Decide and apply the Certified-Claude-Code-Architects channel","description":"The private CCA channel is owned by Jeremy's personal key (0ace65ad), so ops cannot delete/rename it without his key. Decide: keep as-is, rename (the coined 'Claude Code Certified Architect' has trademark risk — real credential is 'Claude Certified Architect (CCA)'), or retire. Owner-gated: needs Jeremy's key or his app action. AC: decision recorded + applied.","status":"open","priority":3,"issue_type":"task","owner":"jeremylongshore@users.noreply.github.com","created_at":"2026-07-31T03:27:10Z","created_by":"jeremylongshore","updated_at":"2026-07-31T03:27:10Z","dependencies":[{"issue_id":"buzz-ehv.1","depends_on_id":"buzz-ehv","type":"parent-child","created_at":"2026-07-30T21:27:09Z","created_by":"jeremylongshore","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0} diff --git a/.env.example b/.env.example index 3dc54856e7..b9bfcada0e 100644 --- a/.env.example +++ b/.env.example @@ -82,6 +82,19 @@ RELAY_URL=ws://localhost:3000 # BUZZ_GIT_PACK_CACHE_MAX_BYTES=5368709120 # BUZZ_GIT_PACK_CACHE_MAX_CONCURRENT_POPULATIONS=2 +# ----------------------------------------------------------------------------- +# S3-Compatible Object Storage (media + Git/CAS) +# ----------------------------------------------------------------------------- +# The local MinIO container is reachable from host processes at localhost:9000. +# Path style keeps the bucket in the URL path and is required by this local DNS +# setup. Use `virtual` only when the provider requires bucket-as-subdomain URLs. +BUZZ_S3_ENDPOINT=http://localhost:9000 +BUZZ_S3_ACCESS_KEY=buzz_dev +BUZZ_S3_SECRET_KEY=buzz_dev_secret +BUZZ_S3_BUCKET=buzz-media +BUZZ_S3_REGION=us-east-1 +BUZZ_S3_ADDRESSING_STYLE=path + # ----------------------------------------------------------------------------- # Media Upload Admission # ----------------------------------------------------------------------------- diff --git a/.github/workflows/auto-tag-on-release-pr-merge.yml b/.github/workflows/auto-tag-on-release-pr-merge.yml index db34fddc2c..3a090b3ebd 100644 --- a/.github/workflows/auto-tag-on-release-pr-merge.yml +++ b/.github/workflows/auto-tag-on-release-pr-merge.yml @@ -4,7 +4,7 @@ name: Auto-tag on Release PR Merge # prefix; the main chart lane also auto-detects a Chart.yaml version bump so # a chart feature PR can publish its own new version when merged: # -# version-bump/ → tag v → release.yml (desktop app) +# version-bump/ → tag desktop-v → release.yml (desktop app) # relay-release/ → tag relay-v → docker.yml (relay image) # chart-release/ → tag chart-v → helm-chart.yml (main helm chart) # push-chart-release/ → tag push-chart-v → push-gateway-helm-chart.yml @@ -35,12 +35,17 @@ permissions: jobs: auto-tag: + permissions: + contents: read + pull-requests: read + checks: read + statuses: read if: > github.event.pull_request.merged == true && github.event.pull_request.head.repo.full_name == github.repository runs-on: ubuntu-latest steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ github.event.pull_request.merge_commit_sha }} fetch-depth: 0 @@ -57,7 +62,7 @@ jobs: case "$BRANCH" in version-bump/*) VERSION="${BRANCH#version-bump/}" - TAG_PREFIX="v" ;; + TAG_PREFIX="desktop-v" ;; relay-release/*) VERSION="${BRANCH#relay-release/}" TAG_PREFIX="relay-v" ;; @@ -85,9 +90,33 @@ jobs: { echo "enabled=true" echo "tag=${TAG_PREFIX}${VERSION}" + if [[ "$TAG_PREFIX" == desktop-v ]]; then + echo "target_sha=${{ github.event.pull_request.merge_commit_sha }}" + echo "desktop=true" + else + echo "target_sha=$GITHUB_SHA" + echo "desktop=false" + fi } >> "$GITHUB_OUTPUT" echo "Tagging ${TAG_PREFIX}${VERSION}" + + - name: Verify immutable reviewed desktop candidate + if: steps.release.outputs.desktop == 'true' + env: + GH_TOKEN: ${{ github.token }} + VERSION: ${{ steps.release.outputs.tag }} + PR_NUMBER: ${{ github.event.pull_request.number }} + PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }} + PR_HEAD_REF: ${{ github.event.pull_request.head.ref }} + PR_BASE_REF: ${{ github.event.pull_request.base.ref }} + PR_HEAD_REPO: ${{ github.event.pull_request.head.repo.full_name }} + MERGE_SHA: ${{ github.event.pull_request.merge_commit_sha }} + run: | + VERSION="${VERSION#desktop-v}" + export VERSION + scripts/verify-desktop-release-merge.sh + - name: Create release tagger token if: steps.release.outputs.enabled == 'true' id: release-tagger @@ -102,21 +131,22 @@ jobs: env: GH_TOKEN: ${{ steps.release-tagger.outputs.token }} TAG: ${{ steps.release.outputs.tag }} + TARGET_SHA: ${{ steps.release.outputs.target_sha }} run: | set -euo pipefail # Check gh's exit status, not its output. A missing ref returns a 404 # JSON body on stdout, which must not be mistaken for an existing tag. if gh api "repos/$GITHUB_REPOSITORY/git/ref/tags/$TAG" --silent 2>/dev/null; then EXISTING_SHA="$(gh api "repos/$GITHUB_REPOSITORY/commits/$TAG" --jq .sha)" - if [ "$EXISTING_SHA" = "$GITHUB_SHA" ]; then - echo "Tag $TAG already exists at $GITHUB_SHA — skipping tag creation" + if [ "$EXISTING_SHA" = "$TARGET_SHA" ]; then + echo "Tag $TAG already exists at $TARGET_SHA — skipping tag creation" exit 0 else - echo "::error::Tag $TAG already exists at $EXISTING_SHA (expected $GITHUB_SHA)" + echo "::error::Tag $TAG already exists at $EXISTING_SHA (expected $TARGET_SHA)" exit 1 fi fi gh api --method POST "repos/$GITHUB_REPOSITORY/git/refs" \ -f ref="refs/tags/$TAG" \ - -f sha="$GITHUB_SHA" \ + -f sha="$TARGET_SHA" \ --silent diff --git a/.github/workflows/benchmark-harbor.yml b/.github/workflows/benchmark-harbor.yml index 31efe933c5..6024f00575 100644 --- a/.github/workflows/benchmark-harbor.yml +++ b/.github/workflows/benchmark-harbor.yml @@ -20,7 +20,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 15 steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 with: python-version: "3.12" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d4826d985f..60507182d5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -28,7 +28,7 @@ jobs: web: ${{ steps.filter.outputs.web }} mobile: ${{ steps.filter.outputs.mobile }} steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 2 - uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 # v4.0.2 @@ -76,6 +76,8 @@ jobs: - '.github/workflows/ci.yml' - name: Release workflow source contract run: scripts/test-release-ref-contract.sh + - name: Desktop release candidate contract + run: scripts/test-desktop-release-candidate.sh - name: Mobile release contract run: | scripts/test-mobile-release-contract.sh @@ -94,7 +96,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 with: @@ -115,7 +117,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: rui314/setup-mold@9c9c13bf4c3f1adef0cc596abc155580bcb04444 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 @@ -137,7 +139,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 2 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -233,7 +235,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - name: Get pnpm store directory id: pnpm-cache @@ -316,7 +318,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 # Reuse the relay binaries and backend test archive when none of their # inputs changed (desktop-only PRs hit this every time). The key covers @@ -389,7 +391,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - name: Start integration services run: | @@ -578,7 +580,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - name: Install cargo-nextest uses: taiki-e/install-action@0fd46367812ee04360509b4169d9f659d6892bb2 # v2.79.15 @@ -690,6 +692,18 @@ jobs: --run-ignored ignored-only env: DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz + - name: Workspace profile (kind:9033) gate tests + # Call-site integration for the 9033 authorization gate: open relay + # rosterless/steward transitions and the closed-relay admin/owner rule, + # against real Postgres. #[ignore]d in the default suite, selected + # explicitly here — see handlers::relay_admin::tests. + run: | + cargo nextest run \ + --archive-file target/ci/backend-integration-tests.tar.zst \ + -E 'package(buzz-relay) and test(/handlers::relay_admin::tests/)' \ + --run-ignored ignored-only + env: + DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz - name: NIP-ER reminder e2e # Feature e2e for NIP-ER (Event Reminders, kind:30300): write-path # validation, author-only read filtering, and scheduler delivery against @@ -702,6 +716,17 @@ jobs: --run-ignored ignored-only env: RELAY_URL: ws://localhost:3000 + - name: NIP-MP coordinate deletion guard + # Verifies the never-delete-newer invariant of soft_delete_by_coordinate: + # a stale tombstone (created_at earlier than the live head) spares that + # head, and an equal-timestamp tombstone deletes it. + run: | + cargo nextest run \ + --archive-file target/ci/backend-integration-tests.tar.zst \ + -E 'package(buzz-db) and test(coordinate_delete_spares_head_newer_than_the_deletion)' \ + --run-ignored ignored-only + env: + DATABASE_URL: postgres://buzz:${{ env.BUZZ_TEST_POSTGRES_PASSWORD }}@localhost:5432/buzz - name: Upload relay log if: failure() uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 @@ -719,7 +744,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 with: @@ -737,7 +762,7 @@ jobs: ./scripts/start-relay-for-tests.sh --no-build - name: Relay E2E tests run: | - cargo test -p buzz-test-client --test e2e_persona --test e2e_nostr_interop -- --ignored --nocapture + cargo test -p buzz-test-client --test e2e_persona --test e2e_team_catalog --test e2e_nostr_interop --test e2e_project -- --ignored --nocapture cargo test -p buzz-test-client --test e2e_relay invite -- --ignored --nocapture cargo test -p buzz-test-client --test e2e_relay nip43_membership_snapshots_are_rejected -- --ignored --nocapture env: @@ -760,7 +785,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 2 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -795,7 +820,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 2 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -856,7 +881,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - name: Dependency policy run: cargo-deny check @@ -868,7 +893,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - name: Check for dead API token references in client code run: | # Fail if dead API token patterns reappear in desktop, mobile, docs, or config. @@ -897,7 +922,7 @@ jobs: - x86_64-unknown-linux-musl - aarch64-unknown-linux-musl steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 with: @@ -934,7 +959,7 @@ jobs: env: TARGET: x86_64-pc-windows-msvc steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 # MSVC needs windows.h (aws-lc-sys et al.), so this runs on a real Windows # runner — hermit, used by the Linux jobs, does not provide MSVC. The # toolchain (1.95.0 + clippy via profile = default) comes from the @@ -1015,7 +1040,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 with: @@ -1029,6 +1054,7 @@ jobs: mkdir -p desktop/src-tauri/binaries touch "desktop/src-tauri/binaries/buzz-acp-$TARGET" touch "desktop/src-tauri/binaries/buzz-agent-$TARGET" + touch "desktop/src-tauri/binaries/buzz-backend-kubernetes-$TARGET" touch "desktop/src-tauri/binaries/buzz-dev-mcp-$TARGET" touch "desktop/src-tauri/binaries/git-credential-nostr-$TARGET" touch "desktop/src-tauri/binaries/buzz-$TARGET" diff --git a/.github/workflows/desktop-release-candidate.yml b/.github/workflows/desktop-release-candidate.yml new file mode 100644 index 0000000000..eddebea685 --- /dev/null +++ b/.github/workflows/desktop-release-candidate.yml @@ -0,0 +1,26 @@ +name: Desktop Release Candidate + +on: + pull_request: + branches: [main] + +permissions: + contents: read + +jobs: + validate: + name: Desktop Release Candidate + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 + with: + ref: ${{ github.event.pull_request.head.sha }} + fetch-depth: 0 + persist-credentials: false + - name: Validate immutable desktop candidate + if: startsWith(github.event.pull_request.head.ref, 'version-bump/') + env: + VERSION: ${{ github.event.pull_request.head.ref }} + run: | + VERSION="${VERSION#version-bump/}" + scripts/desktop_release.py validate --candidate HEAD --version "$VERSION" --repo "$GITHUB_REPOSITORY" diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index 52f21b28bc..564cd74e9d 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -101,7 +101,7 @@ jobs: steps: - name: Checkout - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 0 persist-credentials: false @@ -359,7 +359,7 @@ jobs: arch: arm64 steps: - name: Checkout - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 0 persist-credentials: false diff --git a/.github/workflows/helm-chart.yml b/.github/workflows/helm-chart.yml index e3d443d9f3..7118d16708 100644 --- a/.github/workflows/helm-chart.yml +++ b/.github/workflows/helm-chart.yml @@ -59,7 +59,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 with: # On chart-tag rescue dispatch, lint/render the tagged commit that the # publish job will package, not whatever `main` is when the dispatch @@ -119,7 +119,7 @@ jobs: permissions: contents: read steps: - - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 with: fetch-depth: 0 @@ -166,7 +166,7 @@ jobs: packages: write # push the chart to GHCR steps: - name: Checkout - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1 with: # On the rescue dispatch, build the tagged commit (github.ref is # `main` there); on a tag push, the default ref is already the tag. diff --git a/.github/workflows/linux-canary.yml b/.github/workflows/linux-canary.yml index 18d476e400..1664878770 100644 --- a/.github/workflows/linux-canary.yml +++ b/.github/workflows/linux-canary.yml @@ -21,7 +21,7 @@ jobs: name: Build Linux canary if: github.repository == 'block/buzz' runs-on: ubuntu-latest - container: ubuntu:22.04@sha256:0e0a0fc6d18feda9db1590da249ac93e8d5abfea8f4c3c0c849ce512b5ef8982 + container: ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 timeout-minutes: 60 permissions: contents: read @@ -166,7 +166,7 @@ jobs: - name: Build sidecars run: | - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh - name: Build Linux Tauri app diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c613924e57..02011ad386 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -1,14 +1,13 @@ name: Release +concurrency: + group: desktop-release-${{ github.ref }} + cancel-in-progress: false + on: push: tags: - - 'v[0-9]*' - workflow_dispatch: - inputs: - version: - description: "Semver version matching the v-prefixed dispatch tag" - required: true + - 'desktop-v[0-9]*' jobs: # Shared setup: verify the immutable release tag, determine the version, and @@ -19,23 +18,14 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 10 permissions: - contents: write + contents: read outputs: version: ${{ steps.version.outputs.version }} source_sha: ${{ steps.source.outputs.source_sha }} steps: - name: Determine version id: version - env: - EVENT_NAME: ${{ github.event_name }} - INPUT_VERSION: ${{ inputs.version }} - run: | - if [[ "$EVENT_NAME" == "push" ]]; then - VERSION="${GITHUB_REF_NAME#v}" - else - VERSION="$INPUT_VERSION" - fi - echo "version=$VERSION" >> "$GITHUB_OUTPUT" + run: echo "version=${GITHUB_REF_NAME#desktop-v}" >> "$GITHUB_OUTPUT" - name: Validate version env: @@ -46,7 +36,7 @@ jobs: exit 1 fi - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: fetch-depth: 0 persist-credentials: false @@ -56,42 +46,9 @@ jobs: env: VERSION: ${{ steps.version.outputs.version }} run: | - scripts/verify-release-ref.sh v "$VERSION" + scripts/verify-release-ref.sh desktop-v "$VERSION" echo "source_sha=$(git rev-parse 'HEAD^{commit}')" >> "$GITHUB_OUTPUT" - - name: Create versioned GitHub release - env: - VERSION: ${{ steps.version.outputs.version }} - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - RELEASE_SHA=$(git rev-parse HEAD) - NOTES="" - if [[ -f CHANGELOG.md ]]; then - NOTES=$(awk "/^## v${VERSION}\$/{found=1; next} found && /^## v/{exit} found && !/^\$/" CHANGELOG.md) - fi - if [[ -z "$NOTES" ]]; then - NOTES="Buzz Desktop v${VERSION}" - fi - PRERELEASE_FLAGS=() - if [[ "$VERSION" =~ -(test|alpha|beta|rc)([.-]|$) ]]; then - PRERELEASE_FLAGS=(--prerelease --latest=false) - fi - gh release create "v${VERSION}" \ - --target "$RELEASE_SHA" \ - --title "Buzz Desktop v${VERSION}" \ - --notes "$NOTES" \ - "${PRERELEASE_FLAGS[@]}" - - - name: Create rolling auto-update release - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - gh release create buzz-desktop-latest \ - --prerelease \ - --title "Buzz Desktop Auto-Update" \ - --notes "Rolling release for the Tauri auto-updater. Do not download manually — use the versioned release instead." \ - 2>/dev/null || true - release: name: Release if: github.repository == 'block/buzz' @@ -99,7 +56,7 @@ jobs: needs: setup timeout-minutes: 60 permissions: - contents: write + contents: read id-token: write # required by block/apple-codesign-action for OIDC outputs: archive_name: ${{ steps.artifacts.outputs.archive_name }} @@ -107,14 +64,14 @@ jobs: env: VERSION: ${{ needs.setup.outputs.version }} steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 persist-credentials: false - name: Verify tag-bound release source - run: scripts/verify-release-ref.sh v "$VERSION" + run: scripts/verify-release-ref.sh desktop-v "$VERSION" - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -134,7 +91,7 @@ jobs: - name: Build sidecars run: | - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh # Mesh rev derived from Cargo.lock (no lockstep edit on dep bump); cache key tracks it. @@ -272,13 +229,19 @@ jobs: fi echo "dmg=$DMG" >> "$GITHUB_OUTPUT" - # Find the updater .tar.gz and .sig + # Find the updater .tar.gz and .sig. Give each architecture a unique + # release basename before artifacts are merged by the final writer. ARCHIVE=$(find "$BUNDLE_DIR/macos" -name '*.tar.gz' ! -name '*.sig' -type f | head -1) SIG="${ARCHIVE}.sig" if [[ -z "$ARCHIVE" || ! -f "$SIG" ]]; then echo "::error::Updater archive or signature not found in $BUNDLE_DIR/macos" exit 1 fi + RENAMED="$(dirname "$ARCHIVE")/Buzz_${VERSION}_aarch64.app.tar.gz" + mv "$ARCHIVE" "$RENAMED" + mv "$SIG" "${RENAMED}.sig" + ARCHIVE="$RENAMED" + SIG="${RENAMED}.sig" echo "archive=$ARCHIVE" >> "$GITHUB_OUTPUT" echo "archive_name=$(basename "$ARCHIVE")" >> "$GITHUB_OUTPUT" echo "sig=$SIG" >> "$GITHUB_OUTPUT" @@ -289,23 +252,15 @@ jobs: env: SIG_PATH: ${{ steps.artifacts.outputs.sig }} - - name: Upload arm64 DMG to versioned GitHub release - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - DMG_PATH: ${{ steps.artifacts.outputs.dmg }} - run: gh release upload "v${VERSION}" "$DMG_PATH" --clobber - - - name: Upload updater archive to rolling release - if: github.ref == format('refs/tags/v{0}', needs.setup.outputs.version) - run: | - gh release upload buzz-desktop-latest \ - "$ARCHIVE_PATH" \ - "$SIG_PATH" \ - --clobber - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - ARCHIVE_PATH: ${{ steps.artifacts.outputs.archive }} - SIG_PATH: ${{ steps.artifacts.outputs.sig }} + - name: Stage Apple Silicon release artifacts + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: desktop-release-macos-arm64 + if-no-files-found: error + path: | + ${{ steps.artifacts.outputs.dmg }} + ${{ steps.artifacts.outputs.archive }} + ${{ steps.artifacts.outputs.sig }} release-macos-x64: name: Release macOS (Intel) @@ -314,7 +269,7 @@ jobs: needs: setup timeout-minutes: 60 permissions: - contents: write + contents: read id-token: write # required by block/apple-codesign-action for OIDC outputs: archive_name: ${{ steps.artifacts.outputs.archive_name }} @@ -323,14 +278,14 @@ jobs: VERSION: ${{ needs.setup.outputs.version }} TARGET: x86_64-apple-darwin steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 persist-credentials: false - name: Verify tag-bound release source - run: scripts/verify-release-ref.sh v "$VERSION" + run: scripts/verify-release-ref.sh desktop-v "$VERSION" - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -353,7 +308,7 @@ jobs: - name: Build sidecars run: | - cargo build --release --target "$TARGET" -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release --target "$TARGET" -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh "$TARGET" - name: Build unsigned Tauri app @@ -443,6 +398,11 @@ jobs: echo "::error::Updater archive or signature not found in $BUNDLE_DIR/macos" exit 1 fi + RENAMED="$(dirname "$ARCHIVE")/Buzz_${VERSION}_x64.app.tar.gz" + mv "$ARCHIVE" "$RENAMED" + mv "$SIG" "${RENAMED}.sig" + ARCHIVE="$RENAMED" + SIG="${RENAMED}.sig" echo "archive=$ARCHIVE" >> "$GITHUB_OUTPUT" echo "archive_name=$(basename "$ARCHIVE")" >> "$GITHUB_OUTPUT" echo "sig=$SIG" >> "$GITHUB_OUTPUT" @@ -453,34 +413,26 @@ jobs: env: SIG_PATH: ${{ steps.artifacts.outputs.sig }} - - name: Upload Intel DMG to versioned GitHub release - run: gh release upload "v${VERSION}" "$DMG_PATH" --clobber - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - DMG_PATH: ${{ steps.unsigned.outputs.dmg }} - - - name: Upload updater archive to rolling release - if: github.ref == format('refs/tags/v{0}', needs.setup.outputs.version) - run: | - gh release upload buzz-desktop-latest \ - "$ARCHIVE_PATH" \ - "$SIG_PATH" \ - --clobber - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - ARCHIVE_PATH: ${{ steps.artifacts.outputs.archive }} - SIG_PATH: ${{ steps.artifacts.outputs.sig }} + - name: Stage Intel macOS release artifacts + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: desktop-release-macos-x64 + if-no-files-found: error + path: | + ${{ steps.unsigned.outputs.dmg }} + ${{ steps.artifacts.outputs.archive }} + ${{ steps.artifacts.outputs.sig }} release-linux: name: Release Linux if: github.repository == 'block/buzz' runs-on: ubuntu-latest # Digest-pinned like the SHA-pinned actions below; Renovate keeps it fresh. - container: ubuntu:22.04@sha256:0e0a0fc6d18feda9db1590da249ac93e8d5abfea8f4c3c0c849ce512b5ef8982 + container: ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 needs: setup timeout-minutes: 60 permissions: - contents: write + contents: read env: # AppImage tools (linuxdeploy, appimagetool) are themselves AppImages. # Containers lack FUSE, so we must use the extract-and-run fallback. @@ -498,7 +450,7 @@ jobs: env: DEBIAN_FRONTEND: noninteractive run: | - # Must run first: bare ubuntu:22.04 ships without curl, wget, git, or + # Must run first: bare ubuntu:24.04 ships without curl, wget, git, or # ca-certificates. activate-hermit bootstraps via curl+HTTPS (needs # both), and actions/checkout falls back to a REST tarball without git. # Running as root — no sudo needed. @@ -543,7 +495,7 @@ jobs: apt-get update apt-get install -y --no-install-recommends gh - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 @@ -555,7 +507,7 @@ jobs: - name: Verify tag-bound release source env: VERSION: ${{ needs.setup.outputs.version }} - run: scripts/verify-release-ref.sh v "$VERSION" + run: scripts/verify-release-ref.sh desktop-v "$VERSION" - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -611,7 +563,7 @@ jobs: - name: Build sidecars run: | - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh - name: Generate release config @@ -689,29 +641,16 @@ jobs: SIG_PATH: ${{ steps.linux-artifacts.outputs.sig }} # NOTE: .deb is NOT auto-updatable (Tauri updater constraint — only AppImage supports it on Linux) - - name: Upload Linux artifacts to versioned GitHub release - env: - VERSION: ${{ needs.setup.outputs.version }} - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - DEB_PATH: ${{ steps.linux-artifacts.outputs.deb }} - APPIMAGE_PATH: ${{ steps.linux-artifacts.outputs.appimage }} - run: | - gh release upload "v$VERSION" \ - "$DEB_PATH" \ - "$APPIMAGE_PATH" \ - --clobber - - - name: Upload updater archive to rolling release - if: github.ref == format('refs/tags/v{0}', needs.setup.outputs.version) - run: | - gh release upload buzz-desktop-latest \ - "$ARCHIVE_PATH" \ - "$SIG_PATH" \ - --clobber - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - ARCHIVE_PATH: ${{ steps.linux-artifacts.outputs.archive }} - SIG_PATH: ${{ steps.linux-artifacts.outputs.sig }} + - name: Stage Linux release artifacts + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: desktop-release-linux-x64 + if-no-files-found: error + path: | + ${{ steps.linux-artifacts.outputs.deb }} + ${{ steps.linux-artifacts.outputs.appimage }} + ${{ steps.linux-artifacts.outputs.archive }} + ${{ steps.linux-artifacts.outputs.sig }} release-windows: name: Release Windows @@ -719,7 +658,7 @@ jobs: needs: setup timeout-minutes: 60 permissions: - contents: write + contents: read outputs: archive_name: ${{ steps.artifacts.outputs.archive_name }} sig: ${{ steps.read-sig.outputs.sig }} @@ -727,7 +666,7 @@ jobs: VERSION: ${{ needs.setup.outputs.version }} TARGET: x86_64-pc-windows-msvc steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 @@ -735,7 +674,7 @@ jobs: - name: Verify tag-bound release source shell: bash - run: scripts/verify-release-ref.sh v "$VERSION" + run: scripts/verify-release-ref.sh desktop-v "$VERSION" - uses: dtolnay/rust-toolchain@e081816240890017053eacbb1bdf337761dc5582 # 1.95.0 with: @@ -745,7 +684,7 @@ jobs: with: node-version: 24.14.1 # Disable dependency caching: a writable cache in this release workflow - # (contents: write, feeds a signed installer) is a poisoning vector. pnpm + # (contents: read, feeds a signed installer) is a poisoning vector. pnpm # install runs uncached below. package-manager-cache: false @@ -827,25 +766,14 @@ jobs: env: SIG_PATH: ${{ steps.artifacts.outputs.sig }} - - name: Upload Windows installer to versioned GitHub release - shell: bash - run: gh release upload "v${VERSION}" "$EXE_PATH" --clobber - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - EXE_PATH: ${{ steps.artifacts.outputs.exe }} - - - name: Upload updater archive to rolling release - if: github.ref == format('refs/tags/v{0}', needs.setup.outputs.version) - shell: bash - run: | - gh release upload buzz-desktop-latest \ - "$ARCHIVE_PATH" \ - "$SIG_PATH" \ - --clobber - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - ARCHIVE_PATH: ${{ steps.artifacts.outputs.archive }} - SIG_PATH: ${{ steps.artifacts.outputs.sig }} + - name: Stage Windows release artifacts + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 + with: + name: desktop-release-windows-x64 + if-no-files-found: error + path: | + ${{ steps.artifacts.outputs.exe }} + ${{ steps.artifacts.outputs.sig }} assemble-manifest: name: Assemble multi-platform latest.json @@ -853,7 +781,11 @@ jobs: if: | always() && needs.setup.result == 'success' && - github.ref == format('refs/tags/v{0}', needs.setup.outputs.version) + needs.release.result == 'success' && + needs.release-macos-x64.result == 'success' && + needs.release-linux.result == 'success' && + needs.release-windows.result == 'success' && + github.ref == format('refs/tags/desktop-v{0}', needs.setup.outputs.version) runs-on: ubuntu-latest needs: [setup, release, release-macos-x64, release-linux, release-windows] timeout-minutes: 10 @@ -863,14 +795,33 @@ jobs: VERSION: ${{ needs.setup.outputs.version }} GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 with: ref: ${{ needs.setup.outputs.source_sha }} fetch-depth: 0 persist-credentials: false - name: Verify tag-bound release source - run: scripts/verify-release-ref.sh v "$VERSION" + run: scripts/verify-release-ref.sh desktop-v "$VERSION" + + - name: Download staged release artifacts + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + pattern: desktop-release-* + path: staged-by-platform + + - name: Flatten staged artifacts without basename collisions + run: | + set -euo pipefail + mkdir staged + while IFS= read -r -d '' file; do + name="$(basename "$file")" + [[ ! -e "staged/$name" ]] || { + echo "::error::release artifact basename collision: $name" + exit 1 + } + cp "$file" "staged/$name" + done < <(find staged-by-platform -type f -print0) - name: Write signature files env: @@ -899,7 +850,7 @@ jobs: write_sig "$RESULT_LINUX" linux-x86_64 "$SIG_LINUX" write_sig "$RESULT_WIN" windows-x86_64 "$SIG_WIN" - - name: Verify archive URLs are accessible + - name: Verify draft release has every updater archive env: RESULT_ARM64: ${{ needs.release.result }} RESULT_X64: ${{ needs.release-macos-x64.result }} @@ -911,39 +862,19 @@ jobs: ARCHIVE_WIN: ${{ needs.release-windows.outputs.archive_name }} run: | set -euo pipefail - BASE="https://github.com/block/buzz/releases/download/buzz-desktop-latest" - ARCHIVES=() - - add_archive() { - local result="$1" platform="$2" archive="$3" - if [[ "$result" == "success" ]]; then - [[ -n "$archive" ]] || { echo "::error::Missing archive name for successful platform: $platform"; exit 1; } - ARCHIVES+=("$archive") - fi - } - - add_archive "$RESULT_ARM64" darwin-aarch64 "$ARCHIVE_ARM64" - add_archive "$RESULT_X64" darwin-x86_64 "$ARCHIVE_X64" - add_archive "$RESULT_LINUX" linux-x86_64 "$ARCHIVE_LINUX" - add_archive "$RESULT_WIN" windows-x86_64 "$ARCHIVE_WIN" - - for name in "${ARCHIVES[@]}"; do - echo "Checking $BASE/$name ..." - success=false - for attempt in 1 2 3; do - if curl -fsI "$BASE/$name" > /dev/null 2>&1; then - success=true - break - fi - echo "Attempt $attempt failed for $name, retrying in 10s..." - sleep 10 - done - if [ "$success" != "true" ]; then - echo "::error::Archive not accessible after 3 attempts: $BASE/$name" - exit 1 + assets=$(find staged -type f -exec basename {} \;) + for spec in \ + "$RESULT_ARM64:$ARCHIVE_ARM64" \ + "$RESULT_X64:$ARCHIVE_X64" \ + "$RESULT_LINUX:$ARCHIVE_LINUX" \ + "$RESULT_WIN:$ARCHIVE_WIN"; do + result="${spec%%:*}" + archive="${spec#*:}" + if [[ "$result" == success ]]; then + [[ -n "$archive" ]] || { echo "::error::successful platform has no archive"; exit 1; } + grep -Fxq "$archive" <<<"$assets" || { echo "::error::draft release missing $archive"; exit 1; } fi done - echo "All archive URLs verified." - name: Generate unified latest.json env: @@ -957,7 +888,7 @@ jobs: ARCHIVE_WIN: ${{ needs.release-windows.outputs.archive_name }} run: | set -euo pipefail - BASE="https://github.com/block/buzz/releases/download/buzz-desktop-latest" + BASE="https://github.com/block/buzz/releases/download/desktop-v${VERSION}" TRIPLES=() add_triple() { @@ -977,6 +908,45 @@ jobs: bash desktop/scripts/generate-oss-latest-json.sh "$VERSION" "${TRIPLES[@]}" > latest.json cat latest.json - - name: Upload latest.json to rolling release + - name: Create or verify versioned draft run: | - gh release upload buzz-desktop-latest latest.json --clobber + set -euo pipefail + NOTES_FILE="${RUNNER_TEMP}/release-notes.md" + awk "/^## v${VERSION}\$/{found=1; next} found && /^## v/{exit} found" CHANGELOG.md > "$NOTES_FILE" + [[ -s "$NOTES_FILE" ]] || { echo "::error::missing non-empty changelog block for v${VERSION}"; exit 1; } + PRERELEASE_FLAGS=() + if [[ "$VERSION" == *-* ]]; then + PRERELEASE_FLAGS=(--prerelease --latest=false) + fi + if gh release view "desktop-v${VERSION}" >/dev/null 2>&1; then + EXISTING_SHA=$(gh release view "desktop-v${VERSION}" --json targetCommitish --jq .targetCommitish) + IS_DRAFT=$(gh release view "desktop-v${VERSION}" --json isDraft --jq .isDraft) + [[ "$EXISTING_SHA" == "${{ needs.setup.outputs.source_sha }}" ]] || { + echo "::error::existing release targets $EXISTING_SHA, not the immutable source"; exit 1; + } + if [[ "$IS_DRAFT" != true ]]; then + echo "already_published=true" >> "$GITHUB_ENV" + fi + else + gh release create "desktop-v${VERSION}" \ + --draft \ + --target "${{ needs.setup.outputs.source_sha }}" \ + --title "Buzz Desktop v${VERSION}" \ + --notes-file "$NOTES_FILE" \ + "${PRERELEASE_FLAGS[@]}" + fi + + - name: Upload complete artifact set to versioned draft + if: env.already_published != 'true' + run: | + mapfile -t files < <(find staged -type f -print) + [[ "${#files[@]}" -gt 0 ]] || { echo "::error::no staged release artifacts"; exit 1; } + gh release upload "desktop-v${VERSION}" "${files[@]}" --clobber + + - name: Publish complete versioned release + if: env.already_published != 'true' + run: gh release edit "desktop-v${VERSION}" --draft=false + + - name: Upload latest.json to rolling release last + if: ${{ !contains(needs.setup.outputs.version, '-') }} + run: gh release upload buzz-desktop-latest latest.json --clobber diff --git a/.github/workflows/signed-macos-canary.yml b/.github/workflows/signed-macos-canary.yml index fb0656028a..0a3a513eef 100644 --- a/.github/workflows/signed-macos-canary.yml +++ b/.github/workflows/signed-macos-canary.yml @@ -93,7 +93,7 @@ jobs: - name: Build sidecars run: | - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p git-credential-nostr -p buzz-cli ./scripts/bundle-sidecars.sh # Mesh rev derived from Cargo.lock (no lockstep edit on dep bump); cache key tracks it. diff --git a/.github/workflows/sprig-image.yml b/.github/workflows/sprig-image.yml new file mode 100644 index 0000000000..5d5e12ae0c --- /dev/null +++ b/.github/workflows/sprig-image.yml @@ -0,0 +1,233 @@ +name: Sprig image + +# Builds and publishes the public agent container image as +# ghcr.io/block/buzz-sprig — the digest-pinned box the Kubernetes backend +# deploys agents into (see Dockerfile.sprig and docs/remote-agents.md). +# +# Strategy mirrors docker.yml (the relay image): each architecture builds on +# its native runner, pushes to GHCR by digest, then a merge job stitches the +# per-arch digests into one multi-arch manifest and attests provenance. +# No QEMU emulation. +# +# Triggers: +# - push to main (paths-filtered) → :main + :sha-<7> +# - tag sprig-v* → semver family (shared with sprig.yml's +# binary release — one tag versions both) +# - pull_request (paths-filtered) → build only, no push +# - workflow_dispatch → manual publish at the current ref +# +# NOTE: the first push creates the GHCR package PRIVATE by default. An org +# admin must flip ghcr.io/block/buzz-sprig to public once (Package settings → +# Change visibility). Subsequent pushes keep the visibility. + +on: + push: + branches: [main] + tags: ["sprig-v[0-9]*"] + paths: + - "Dockerfile.sprig" + - "scripts/sprig-entrypoint.sh" + - ".github/workflows/sprig-image.yml" + - "Cargo.toml" + - "Cargo.lock" + - "rust-toolchain.toml" + - "crates/**" + pull_request: + paths: + - "Dockerfile.sprig" + - "scripts/sprig-entrypoint.sh" + - ".github/workflows/sprig-image.yml" + workflow_dispatch: {} + +concurrency: + group: sprig-image-${{ github.ref }} + cancel-in-progress: ${{ github.ref_type == 'branch' && github.event_name == 'pull_request' }} + +permissions: {} + +env: + # Single source of truth for the image name; override with the + # GHCR_SPRIG_IMAGE repo variable (same pattern as docker.yml). + IMAGE_NAME: ${{ vars.GHCR_SPRIG_IMAGE != '' && vars.GHCR_SPRIG_IMAGE || 'ghcr.io/block/buzz-sprig' }} + +jobs: + build: + name: Build (${{ matrix.platform }}) + runs-on: ${{ matrix.runner }} + timeout-minutes: 60 + permissions: + contents: read + packages: write + id-token: write + attestations: write + strategy: + fail-fast: false + matrix: + include: + - platform: linux/amd64 + runner: ubuntu-24.04 + arch: amd64 + - platform: linux/arm64 + runner: ubuntu-24.04-arm + arch: arm64 + + steps: + - name: Checkout + uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 + with: + persist-credentials: false + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 + with: + # Same OOM cap as docker.yml — Rust compiles blow the 7GB runner + # at buildkit's default parallelism of 4. + buildkitd-config-inline: | + [worker.oci] + max-parallelism = 2 + + - name: Log in to GHCR + if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository + uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0 + with: + registry: ghcr.io + username: ${{ github.repository_owner }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract metadata + id: meta + uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0 + with: + images: ${{ env.IMAGE_NAME }} + # match=^sprig-v(.*)$ strips the tag prefix for the semver parser, + # exactly as docker.yml does for relay-v. :latest comes from + # flavor.latest=auto — stable semver only, never main pushes. + tags: | + type=ref,event=branch + type=sha,prefix=sha-,format=short + type=semver,pattern={{version}},match=^sprig-v(.*)$ + type=semver,pattern={{major}}.{{minor}},match=^sprig-v(.*)$ + labels: | + org.opencontainers.image.title=Buzz Sprig + org.opencontainers.image.description=Agent runtime image for Buzz remote agents (buzz-acp multicall + git + curl) + org.opencontainers.image.licenses=Apache-2.0 + + - name: Build and push by digest + id: build + uses: docker/build-push-action@f9f3042f7e2789586610d6e8b85c8f03e5195baf # v7.2.0 + with: + context: . + file: ./Dockerfile.sprig + platforms: ${{ matrix.platform }} + labels: ${{ steps.meta.outputs.labels }} + outputs: type=image,name=${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=${{ github.event_name != 'pull_request' }} + cache-from: | + type=registry,ref=${{ env.IMAGE_NAME }}-buildcache:${{ matrix.arch }} + cache-to: | + ${{ (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository) && format('type=registry,ref={0}-buildcache:{1},mode=max,compression=zstd', env.IMAGE_NAME, matrix.arch) || '' }} + + - name: Export digest + if: github.event_name != 'pull_request' + env: + DIGEST: ${{ steps.build.outputs.digest }} + run: | + mkdir -p /tmp/digests + touch "/tmp/digests/${DIGEST#sha256:}" + + - name: Upload digest + if: github.event_name != 'pull_request' + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: sprig-digest-${{ matrix.arch }} + path: /tmp/digests/* + if-no-files-found: error + retention-days: 1 + + merge: + name: Merge multi-arch manifest + if: github.event_name != 'pull_request' + runs-on: ubuntu-24.04 + needs: build + timeout-minutes: 15 + permissions: + contents: read + packages: write + id-token: write + attestations: write + + steps: + - name: Download per-arch digests + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + path: /tmp/digests + pattern: sprig-digest-* + merge-multiple: true + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@d7f5e7f509e45cec5c76c4d5afdd7de93d0b3df5 # v4.1.0 + + - name: Log in to GHCR + uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0 + with: + registry: ghcr.io + username: ${{ github.repository_owner }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract metadata + id: meta + uses: docker/metadata-action@80c7e94dd9b9319bd5eb7a0e0fe9291e23a2a2e9 # v6.1.0 + with: + images: ${{ env.IMAGE_NAME }} + # Must mirror the build job's tag matrix exactly (see docker.yml). + flavor: | + latest=auto + tags: | + type=ref,event=branch + type=sha,prefix=sha-,format=short + type=semver,pattern={{version}},match=^sprig-v(.*)$ + type=semver,pattern={{major}}.{{minor}},match=^sprig-v(.*)$ + + - name: Create and push manifest list + id: manifest + working-directory: /tmp/digests + env: + IMAGE_NAME: ${{ env.IMAGE_NAME }} + META_TAGS: ${{ steps.meta.outputs.tags }} + run: | + set -euo pipefail + tags=() + while IFS= read -r tag; do + [ -n "$tag" ] && tags+=("-t" "$tag") + done <<< "$META_TAGS" + + digests=() + for digest in *; do + digests+=("${IMAGE_NAME}@sha256:${digest}") + done + + docker buildx imagetools create "${tags[@]}" "${digests[@]}" + + first_tag=$(echo "$META_TAGS" | head -n1) + merged_digest=$(docker buildx imagetools inspect "$first_tag" \ + --format '{{json .Manifest}}' | jq -r '.digest') + echo "digest=${merged_digest}" >> "$GITHUB_OUTPUT" + + - name: Attest provenance for the merged image + # Verify with: gh attestation verify oci://ghcr.io/block/buzz-sprig: --owner block + uses: actions/attest-build-provenance@0f67c3f4856b2e3261c31976d6725780e5e4c373 # v4.1.1 + with: + subject-name: ${{ env.IMAGE_NAME }} + subject-digest: ${{ steps.manifest.outputs.digest }} + push-to-registry: true + + - name: Summary + env: + IMAGE_NAME: ${{ env.IMAGE_NAME }} + DIGEST: ${{ steps.manifest.outputs.digest }} + run: | + { + echo "### Sprig image published" + echo '```' + echo "${IMAGE_NAME}@${DIGEST}" + echo '```' + } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/sprig.yml b/.github/workflows/sprig.yml index b2dab3583f..5e50808b3b 100644 --- a/.github/workflows/sprig.yml +++ b/.github/workflows/sprig.yml @@ -42,7 +42,7 @@ jobs: - x86_64-unknown-linux-musl - aarch64-unknown-linux-musl steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - uses: cashapp/activate-hermit@cea9af7913204a965fd488637a8d1811bba2e616 # v1 @@ -116,7 +116,7 @@ jobs: permissions: contents: write steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - name: Download all Sprig artifacts uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 @@ -154,7 +154,7 @@ jobs: permissions: contents: write steps: - - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6 + - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3 - name: Download all Sprig artifacts uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 diff --git a/.gitignore b/.gitignore index 65ddcaf1c4..f26e74136c 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,10 @@ /dist/ /admin-web/dist/ +# Python cache +__pycache__/ +*.pyc + # lefthook-generated hook scripts (machine-specific) .hooks/ diff --git a/.release/desktop-candidate.json b/.release/desktop-candidate.json new file mode 100644 index 0000000000..1bf2efb66b --- /dev/null +++ b/.release/desktop-candidate.json @@ -0,0 +1,8 @@ +{ + "schema": 1, + "version": "0.5.3", + "base_sha": "54c8ef30a9bb9c59a4415a8a7ee84c7c5454b48a", + "previous_tag": "v0.5.2", + "tag": "desktop-v0.5.3", + "commit_count": 58 +} diff --git a/000-docs/001-PP-PLAN-buzz-adoption-master-blueprint.md b/000-docs/001-PP-PLAN-buzz-adoption-master-blueprint.md index 4c5533c3cd..8f632f707f 100644 --- a/000-docs/001-PP-PLAN-buzz-adoption-master-blueprint.md +++ b/000-docs/001-PP-PLAN-buzz-adoption-master-blueprint.md @@ -80,11 +80,15 @@ three-way (bead ↔ GitHub issue ↔ Plane project `BUZZ`). | E3 — Hardening + go-live gates (run against PROD) | 2 | in flight — membership probe + unauth HTTP matrix + resource caps + backup/restore PROVEN against prod; remaining BLOCKING: off-site backup leg, updater planted-fault drill (on staging), monitoring alerts exercised, full smoke suite, CORS client verify, pairing verify-or-documented, key runbooks rehearsed | — | | E4 — Headless administration | 2.5 | not started | — | | E5 — Team onboarding (all-in) | 3 | not started | — | -| E6 — Agent bridge (`@claude`, isolated) | 4 | not started | — | +| E6 — Agent bridge (isolated coding agent `goose minimax3`) | 4 | **LIVE on prod** (2026-07-30, decision-log/039, PR #301): `buzz-acp`+goose/MiniMax-M3 running as a member on the prod host, owner-gated + PR-gated, egress-isolated. Open: wire the prod PR loop (repo + scoped token + branch protection — owner inputs) | GH #? · PR #301 | | E7 — Governed-brain agent (`@bob`, BYOH) | 4 | staged (follow-up) | — | | ELab — Contributor laboratory (`intent-solutions-io/intent-ops-buzz` **repository**, Track C) | 6 | not started (repo not yet created — correct; depends on the naming record `006`) | — | | E8 — Upstream contribution lane (Track D) | 5 | qualified candidates filed | — | | E9 — Operator plugin (`intent-solutions-io/intent-ops-buzz-plugin`) | 7 | **DEFERRED** — scope gauged from the real install cycle (owner call 2026-07-29); repo not yet created | — | +| E10 — Community channel layout & identity roster (`buzz-ehv`) | 3 | **build DONE (2026-07-31)** — clean provider-first open layout live (`0-general` pinned, `ask`, `ai-wire`, `anthropic-*` group, `-wire` + topic `*-wire`, private `sys-*`); persona agents purged; canonical names set; `ops/buzz/RUNBOOK-channels.md` (source-cited). Open: CCA-channel decision (owner-gated), member join/onboarding model | fork beads `buzz-ehv` | +| E11 — Buzz operational expertise: reference + skill + agents (`buzz-yfe`) | 3 | **COMPLETE (2026-07-31)** — 2 source-cited references (`ops/buzz/reference/`, 100% citation-consistent with the fork); `buzz-ops` skill (PASS `/validate-skillmd`); 3 operator agents `buzz-ops`/`buzz-feed-curator`/`buzz-relay-admin` (PASS `/validate-agent`) | fork beads `buzz-yfe` | +| E12 — AI-Wire feed engine (`buzz-w92`) | 4 | **posting primitive DONE**; ingestion pipeline next — `AI Wire` bot live, first cards posted to `ai-wire`+`0-general`. Children: pipeline (RSSHub+feed→LLM→card), easy/medium/hard feeds, Anthropic monitor-fleet wire, curation+digest, cron | fork beads `buzz-w92` | +| E13 — Estate system notifications into `sys-*` (`buzz-0ts`) | 4 | **not started** (Track B) — `buzz-notify.sh`/`af_buzz_transport` → private `sys-*` channels, retiring Slack/notify/Moshi (keep off-estate floor + Ezekiel email) | fork beads `buzz-0ts` | Asset names are governed by `006-DR-STND-authoritative-naming-and-boundaries.md` (canonical). Never write the bare phrase `intent-ops-buzz` — it is either the diff --git a/AGENTS.md b/AGENTS.md index 7ff0eb4d47..4f03b312bc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -145,6 +145,10 @@ first, then implement handling in the relay. **Channel scoping**: Channels use `h` tags (NIP-29 group tag), not `e` tags. Filters and queries must scope to `h` tags when operating within a channel. +This applies to events *inside* a channel. Addressable events that describe a +channel carry its id in their `d` tag instead: kind:39000 (metadata), +kind:39001, kind:39002 (membership). `get_channels` resolves a user's channels +from the `d` tag of their kind:39002 events, not from `h`. **Agent-facing operations go in `buzz-cli`**: New agent-facing features belong in `buzz-cli` — add a subcommand there first, then wire the REST/WebSocket call in `client.rs`. `buzz-dev-mcp` (shell + file tools for `buzz-agent`) is separate. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 90cbbac0cf..892082d96c 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -139,7 +139,7 @@ The `kind` integer is the only dispatch switch. The relay routes, stores, and fa | 46001–46012 | KIND_WORKFLOW_* | Workflow execution events | | 20001 | KIND_PRESENCE_UPDATE | Ephemeral presence heartbeat | -`buzz-core` defines all 81 kinds as `pub const KIND_*: u32` and exports `ALL_KINDS: &[u32]`. Kinds are `u32` (NIP-01 specifies unsigned integer; `u32` covers the full range). Buzz uses both standard Nostr kinds (e.g., kind 7 for reactions) and custom ranges (40000+). +`buzz-core` defines each event kind as a `pub const u32` and exports the full registry as `ALL_KINDS: &[u32]` (127 kinds at the time of writing); `crates/buzz-core/src/kind.rs` is the source of truth for the current list. Kinds are `u32` (NIP-01 specifies unsigned integer; `u32` covers the full range). Buzz uses both standard Nostr kinds (e.g., kind 7 for reactions) and custom ranges (40000+). Note: `KIND_AUTH` (22242) is `pub const KIND_AUTH: u32` in `buzz-core/src/kind.rs` and imported by `buzz-relay/src/handlers/event.rs`. `KIND_CANVAS` (40100) is likewise `pub const KIND_CANVAS: u32` in `buzz-core/src/kind.rs`. @@ -447,7 +447,7 @@ The subscriber uses a **dedicated** `redis::aio::PubSub` connection — not from **Reconnection:** exponential backoff 1s → 30s (`backoff_secs * 2`). Backoff resets to 1s only after a clean stream end, not on each reconnect attempt. -**Presence:** `SET buzz:presence:{pubkey_hex} {status} EX 90` — 90-second TTL (3× the 30-second heartbeat interval). Single missed heartbeat does not cause presence flap. +**Presence:** `SET buzz:presence:{pubkey_hex} {status} EX 180` — 180-second TTL (3× the 60-second heartbeat interval). Single missed heartbeat does not cause presence flap. **Typing indicators:** ``` @@ -797,7 +797,7 @@ Docker Compose provides the full local development stack. All services include h | Pattern | Type | TTL | Purpose | |---------|------|-----|---------| | `buzz:channel:{uuid}` | Pub/Sub channel | — | Event fan-out (single-community form; shared multi-community Redis must use `buzz:{community}:channel:{uuid}` or equivalent) | -| `buzz:presence:{pubkey_hex}` | String | 90s | Online/away status (single-community form; shared multi-community Redis must scope by community) | +| `buzz:presence:{pubkey_hex}` | String | 180s | Online/away status (single-community form; shared multi-community Redis must scope by community) | | `buzz:typing:{channel_uuid}` | Sorted Set | 60s | Active typers (5s window; shared multi-community Redis must scope by community) | ### Full-Text Search (Postgres FTS) diff --git a/CHANGELOG.md b/CHANGELOG.md index cfd3b16d0a..71a4bbd449 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,106 @@ # Changelog +## v0.5.3 + +### Desktop and shared changes + +- Revert "chore(release): release Buzz Desktop version 0.5.3" ([#3960](https://github.com/block/buzz/pull/3960)) ([`bb34bc4d98fe4dabe847046103ac5e2859917ac5`](https://github.com/block/buzz/commit/bb34bc4d98fe4dabe847046103ac5e2859917ac5)) +- chore(release): release Buzz Desktop version 0.5.3 ([`d12b3d6a79d56a95fc99ce4fadd2d2235d5a3131`](https://github.com/block/buzz/commit/d12b3d6a79d56a95fc99ce4fadd2d2235d5a3131)) +- feat(desktop): import local Pocket voices ([#3259](https://github.com/block/buzz/pull/3259)) ([`c104eecfb38620de2c35c7e20a716f8658b5a6b1`](https://github.com/block/buzz/commit/c104eecfb38620de2c35c7e20a716f8658b5a6b1)) +- fix(desktop): open profiles from avatars ([#3751](https://github.com/block/buzz/pull/3751)) ([`39ce3dfc3cf2d12f0d6c64b4cd4293df86567663`](https://github.com/block/buzz/commit/39ce3dfc3cf2d12f0d6c64b4cd4293df86567663)) +- refactor(voice): extract reusable Pocket primitives + Pocket voice settings (relands #2467 + #3208) ([#3910](https://github.com/block/buzz/pull/3910)) ([`61ba9dfaa00852925058d1a024322fa53663a5bc`](https://github.com/block/buzz/commit/61ba9dfaa00852925058d1a024322fa53663a5bc)) +- feat(desktop): auto-enable huddle transcription for agents ([#3180](https://github.com/block/buzz/pull/3180)) ([`4632c55041c5d423d572a6f6411bb7b279c26f67`](https://github.com/block/buzz/commit/4632c55041c5d423d572a6f6411bb7b279c26f67)) +- feat(agent): optional reply guard reminds a silent turn to publish ([#3763](https://github.com/block/buzz/pull/3763)) ([`081f805d5ea25841ab885c7b67a568618a34aa59`](https://github.com/block/buzz/commit/081f805d5ea25841ab885c7b67a568618a34aa59)) +- feat(desktop): upgrade Pocket TTS model ([#3266](https://github.com/block/buzz/pull/3266)) ([`d48b0e0eec4d2958f90a3cafa9d974450abe8501`](https://github.com/block/buzz/commit/d48b0e0eec4d2958f90a3cafa9d974450abe8501)) +- feat(desktop): delete a message by clearing its edit to empty ([#3813](https://github.com/block/buzz/pull/3813)) ([`d88313f369acfa17973029787ee4c0bbea07fa51`](https://github.com/block/buzz/commit/d88313f369acfa17973029787ee4c0bbea07fa51)) +- feat(relay): raise hosted community limit to five ([#3829](https://github.com/block/buzz/pull/3829)) ([`10d5a26414dc90dc89fd27de74b21e105d4fa622`](https://github.com/block/buzz/commit/10d5a26414dc90dc89fd27de74b21e105d4fa622)) +- feat(desktop): locally stored NIP-49 encrypted key backup ([#2937](https://github.com/block/buzz/pull/2937)) ([`468647a51f858b29d27eaf9fd07bf90294f99d39`](https://github.com/block/buzz/commit/468647a51f858b29d27eaf9fd07bf90294f99d39)) +- fix(catalog): update Amp tagline ([#3806](https://github.com/block/buzz/pull/3806)) ([`f3e5e812677f6f14bffe16a7aa02642d56faca4b`](https://github.com/block/buzz/commit/f3e5e812677f6f14bffe16a7aa02642d56faca4b)) +- fix(desktop): channel topic and membership metadata cleanup ([#3642](https://github.com/block/buzz/pull/3642)) ([`9e8fcfda099652926b921bca7fcc9bfecab0e140`](https://github.com/block/buzz/commit/9e8fcfda099652926b921bca7fcc9bfecab0e140)) +- fix(desktop): align data deletion labels ([#2230](https://github.com/block/buzz/pull/2230)) ([`ede26863345a518ec46edd6d7692e0281883491b`](https://github.com/block/buzz/commit/ede26863345a518ec46edd6d7692e0281883491b)) +- fix(desktop): allow linux-only media items as dead code off-linux ([#3811](https://github.com/block/buzz/pull/3811)) ([`36571f4adcfdcf3714a17bd968c58c78bcbdd9ef`](https://github.com/block/buzz/commit/36571f4adcfdcf3714a17bd968c58c78bcbdd9ef)) +- fix(desktop): report authenticated relay recovery ([#3812](https://github.com/block/buzz/pull/3812)) ([`74cd5712191bffd84ae688d59bb8b451c6eec1b0`](https://github.com/block/buzz/commit/74cd5712191bffd84ae688d59bb8b451c6eec1b0)) +- fix(desktop): don't gate hover affordances on the hover media query ([#3657](https://github.com/block/buzz/pull/3657)) ([`29dfe4821ed577489a1879fd2a9bfe2a621a52b3`](https://github.com/block/buzz/commit/29dfe4821ed577489a1879fd2a9bfe2a621a52b3)) +- feat(relay): gate kind 30178 team-catalog reads behind the shared tag ([#3358](https://github.com/block/buzz/pull/3358)) ([`114d40d9d37f05eff83ee90347ed93fb3da512c5`](https://github.com/block/buzz/commit/114d40d9d37f05eff83ee90347ed93fb3da512c5)) +- test(desktop): click visible thread collapse guide ([#3800](https://github.com/block/buzz/pull/3800)) ([`b9e4ed616f39b812bc964e79c7a40223c4e93832`](https://github.com/block/buzz/commit/b9e4ed616f39b812bc964e79c7a40223c4e93832)) +- feat(desktop): raise the install ceiling and make installs observable ([#3368](https://github.com/block/buzz/pull/3368)) ([`d40a33290e75791aa7ecf3ce7a252b66c2e35966`](https://github.com/block/buzz/commit/d40a33290e75791aa7ecf3ce7a252b66c2e35966)) +- Add Devin as a preset ACP harness ([#3225](https://github.com/block/buzz/pull/3225)) ([`1b3ff96a5764303998fa629ff852e81f1a88d7ad`](https://github.com/block/buzz/commit/1b3ff96a5764303998fa629ff852e81f1a88d7ad)) +- feat(desktop): improve agent activity header ui ([#3321](https://github.com/block/buzz/pull/3321)) ([`4d47aa83455a9fd024121a596154cd311dca1d76`](https://github.com/block/buzz/commit/4d47aa83455a9fd024121a596154cd311dca1d76)) +- perf(presence): reduce heartbeat frequency ([#3783](https://github.com/block/buzz/pull/3783)) ([`bf139e8d0bdba10df9a5adbf16843140e0a78a59`](https://github.com/block/buzz/commit/bf139e8d0bdba10df9a5adbf16843140e0a78a59)) +- Tighten continuation message rows ([#3724](https://github.com/block/buzz/pull/3724)) ([`6e419b9f1c873549a7b40996970e0da7352adafb`](https://github.com/block/buzz/commit/6e419b9f1c873549a7b40996970e0da7352adafb)) +- Fix video reviews in thread replies ([#3719](https://github.com/block/buzz/pull/3719)) ([`f48f3f055fdd6030d3832f615f8c0d8e5a81261a`](https://github.com/block/buzz/commit/f48f3f055fdd6030d3832f615f8c0d8e5a81261a)) +- Make relay reconnect backoff authoritative ([#3774](https://github.com/block/buzz/pull/3774)) ([`cca8839034eb571a7ce943c3ace7f85a82330898`](https://github.com/block/buzz/commit/cca8839034eb571a7ce943c3ace7f85a82330898)) +- feat(desktop): add password-protected backups in settings ([#3701](https://github.com/block/buzz/pull/3701)) ([`bd0bff24bfd2cffa2b3b3a995f7628af5e460a5c`](https://github.com/block/buzz/commit/bd0bff24bfd2cffa2b3b3a995f7628af5e460a5c)) +- fix(desktop): reuse profiles when joining communities ([#2155](https://github.com/block/buzz/pull/2155)) ([`f44b5a2477f3979ae66e49153b11be36538cf859`](https://github.com/block/buzz/commit/f44b5a2477f3979ae66e49153b11be36538cf859)) +- fix(catalog): update Amp description ([#3758](https://github.com/block/buzz/pull/3758)) ([`61b96c9828d1dd54106b570d87a54edbc92bb9c4`](https://github.com/block/buzz/commit/61b96c9828d1dd54106b570d87a54edbc92bb9c4)) +- feat(catalog): resolve publisher display name in catalog detail pane ([#3640](https://github.com/block/buzz/pull/3640)) ([`02be413b823c356587e6e9f4d07f6cb06bb41c3c`](https://github.com/block/buzz/commit/02be413b823c356587e6e9f4d07f6cb06bb41c3c)) +- feat(mesh): upgrade embedded mesh to v0.74 and harden shared compute (split 1/2 of #3467) ([#3741](https://github.com/block/buzz/pull/3741)) ([`4933672eb4589e7208b312829ebddcd10dfa9dd3`](https://github.com/block/buzz/commit/4933672eb4589e7208b312829ebddcd10dfa9dd3)) +- Refine agent sharing dialog ([#3699](https://github.com/block/buzz/pull/3699)) ([`9a386a0defbf2b355ee17646c7c11817a535b85f`](https://github.com/block/buzz/commit/9a386a0defbf2b355ee17646c7c11817a535b85f)) +- desktop: enable getUserMedia in the Linux WebKitGTK webview ([#3607](https://github.com/block/buzz/pull/3607)) ([`c9aa55505c544c608ff71648bbfd21b235637f19`](https://github.com/block/buzz/commit/c9aa55505c544c608ff71648bbfd21b235637f19)) +- fix: align responsive agent views ([#3688](https://github.com/block/buzz/pull/3688)) ([`73589408db6fd96b87ac570935d414ecc4120f53`](https://github.com/block/buzz/commit/73589408db6fd96b87ac570935d414ecc4120f53)) +- Add macOS agent menu-bar menu ([#3565](https://github.com/block/buzz/pull/3565)) ([`d0a24bcb5210326da4c0b1e749ee3935621b329c`](https://github.com/block/buzz/commit/d0a24bcb5210326da4c0b1e749ee3935621b329c)) +- Fix pending message feedback ([#3543](https://github.com/block/buzz/pull/3543)) ([`4672ee55c4e4a7916c31bfeae5df2fb4384bed10`](https://github.com/block/buzz/commit/4672ee55c4e4a7916c31bfeae5df2fb4384bed10)) +- fix(desktop): remove remaining Projects panel fills ([#3742](https://github.com/block/buzz/pull/3742)) ([`c55e421a0629c74b9ffd96ee3ccde36f006196ed`](https://github.com/block/buzz/commit/c55e421a0629c74b9ffd96ee3ccde36f006196ed)) +- desktop: restore direct community member adds ([#3634](https://github.com/block/buzz/pull/3634)) ([`310df2ec33fbb075edf226ba18bf9a96d90ba81b`](https://github.com/block/buzz/commit/310df2ec33fbb075edf226ba18bf9a96d90ba81b)) +- fix(desktop): explain open agent access ([#2561](https://github.com/block/buzz/pull/2561)) ([`7fb008f9347b933b9a1da20a7afb070912b430e8`](https://github.com/block/buzz/commit/7fb008f9347b933b9a1da20a7afb070912b430e8)) +- fix(desktop): remove Projects overview card fills ([#3416](https://github.com/block/buzz/pull/3416)) ([`3b8567a05d4c40e667d061666feb7aa7bc38212d`](https://github.com/block/buzz/commit/3b8567a05d4c40e667d061666feb7aa7bc38212d)) +- fix(git): channel binding tooling + author remediation for unbound repos ([#3626](https://github.com/block/buzz/pull/3626)) ([`788b3c002bd2509455444f57f8a03a054b4b496a`](https://github.com/block/buzz/commit/788b3c002bd2509455444f57f8a03a054b4b496a)) +- feat: configure S3 URL addressing style ([#3400](https://github.com/block/buzz/pull/3400)) ([`7012d86d52fd188b27c7beedeaa132d9c1f61fa8`](https://github.com/block/buzz/commit/7012d86d52fd188b27c7beedeaa132d9c1f61fa8)) +- feat: add first-class OpenRouter provider support ([#1975](https://github.com/block/buzz/pull/1975)) ([`ab55fee81896d2b03edf5d2ca5012b715be2b93d`](https://github.com/block/buzz/commit/ab55fee81896d2b03edf5d2ca5012b715be2b93d)) +- feat(agent,acp): wire provider total_tokens through NIP-AM publish chain ([#3593](https://github.com/block/buzz/pull/3593)) ([`f95fdc1a102e17c6718a44323d9a2feaed702db7`](https://github.com/block/buzz/commit/f95fdc1a102e17c6718a44323d9a2feaed702db7)) + +### Other repository changes + +- fix(release): require exact-head approval for desktop tags ([#3973](https://github.com/block/buzz/pull/3973)) ([`54c8ef30a9bb9c59a4415a8a7ee84c7c5454b48a`](https://github.com/block/buzz/commit/54c8ef30a9bb9c59a4415a8a7ee84c7c5454b48a)) +- fix(release): make desktop tagging squash-safe ([#3965](https://github.com/block/buzz/pull/3965)) ([`db7e84d4f815127236b9cb080c5d374f48eaac09`](https://github.com/block/buzz/commit/db7e84d4f815127236b9cb080c5d374f48eaac09)) +- docs(nips): add single-coordinate manual-unread override layer and verification model to NIP-RS ([#2864](https://github.com/block/buzz/pull/2864)) ([`209536ade6c5ebf7fa82671d7ca0b74f599a40cc`](https://github.com/block/buzz/commit/209536ade6c5ebf7fa82671d7ca0b74f599a40cc)) +- fix(release): make immutable desktop release operable ([#3943](https://github.com/block/buzz/pull/3943)) ([`052174a148f9f6bcbb2b5a1d20ce0317645e49f8`](https://github.com/block/buzz/commit/052174a148f9f6bcbb2b5a1d20ce0317645e49f8)) +- docs: add VISION_REMOTE_AGENTS.md ([#3924](https://github.com/block/buzz/pull/3924)) ([`689617af7ad420c3266d5d2eb437757371327089`](https://github.com/block/buzz/commit/689617af7ad420c3266d5d2eb437757371327089)) +- fix(relay): align NIP-11 max_limit with REQ ceiling ([#3635](https://github.com/block/buzz/pull/3635)) ([`23f0c26b1ceba8e07bf3c160a1e08c7bda82ccd9`](https://github.com/block/buzz/commit/23f0c26b1ceba8e07bf3c160a1e08c7bda82ccd9)) +- fix(db): isolate usage metrics advisory-lock test on scratch DB ([#3670](https://github.com/block/buzz/pull/3670)) ([`dba97eecd9d8659c9c816cd6666fa6d687b6bca1`](https://github.com/block/buzz/commit/dba97eecd9d8659c9c816cd6666fa6d687b6bca1)) +- feat(release): make desktop releases immutable ([#3568](https://github.com/block/buzz/pull/3568)) ([`1dfd89ea67b4ebce0c4d10390f280ed4e7ddde8a`](https://github.com/block/buzz/commit/1dfd89ea67b4ebce0c4d10390f280ed4e7ddde8a)) +- Render mobile agent mention chips ([#3702](https://github.com/block/buzz/pull/3702)) ([`06582ee6f09e5f7454e4d8895d80a45c3cdb5e8a`](https://github.com/block/buzz/commit/06582ee6f09e5f7454e4d8895d80a45c3cdb5e8a)) +- fix(acp): preserve truncated thread context ([#3340](https://github.com/block/buzz/pull/3340)) ([`53771c8f5439f9c5c26876f0229bfcfe5da9b170`](https://github.com/block/buzz/commit/53771c8f5439f9c5c26876f0229bfcfe5da9b170)) +- docs(nips): specify kind:30621 multi-repo projects (NIP-MP) ([#3163](https://github.com/block/buzz/pull/3163)) ([`33bf7caa6ea474ccde2932c1ed05a90d7345c6e0`](https://github.com/block/buzz/commit/33bf7caa6ea474ccde2932c1ed05a90d7345c6e0)) +- feat(mobile): desktop-parity emoji and thread experience ([#3485](https://github.com/block/buzz/pull/3485)) ([`85edc0572a8540dedfa6562d40f0f875af0b5f61`](https://github.com/block/buzz/commit/85edc0572a8540dedfa6562d40f0f875af0b5f61)) +- fix(cli): resolve agents from owner records ([#3178](https://github.com/block/buzz/pull/3178)) ([`262f2392e3b7e09c78d582fb384672034d8551d5`](https://github.com/block/buzz/commit/262f2392e3b7e09c78d582fb384672034d8551d5)) +- feat(replica): portable heartbeat-token fence with snapshot-local reader routing ([#3268](https://github.com/block/buzz/pull/3268)) ([`63496cc1d4c6f1b7c613801bdcc694169dcf391a`](https://github.com/block/buzz/commit/63496cc1d4c6f1b7c613801bdcc694169dcf391a)) + +[Compare v0.5.2...desktop-v0.5.3](https://github.com/block/buzz/compare/v0.5.2...desktop-v0.5.3) + +## v0.5.2 + +- feat(cli): mirror Desktop mention delivery ([#3330](https://github.com/block/buzz/pull/3330)) ([`7adc46268`](https://github.com/block/buzz/commit/7adc46268d5e93f0b1d4dc8e700af22815dcac1b)) +- fix(desktop): deduplicate relay outage notification ([#3579](https://github.com/block/buzz/pull/3579)) ([`66e705492`](https://github.com/block/buzz/commit/66e7054928cc29395f828467c3e8c81b7408dd29)) +- fix(desktop): reconcile thread arrivals at bottom ([#3585](https://github.com/block/buzz/pull/3585)) ([`b42a8d447`](https://github.com/block/buzz/commit/b42a8d447e3a2b85b2313dc4fdd123731fd8bba3)) +- Improve emoji autocomplete matching ([#3571](https://github.com/block/buzz/pull/3571)) ([`259de6afb`](https://github.com/block/buzz/commit/259de6afbe0cc0d106e57ebdb2323064990e4122)) +- Fix shared agent avatar import profiles ([#3578](https://github.com/block/buzz/pull/3578)) ([`324bd6b46`](https://github.com/block/buzz/commit/324bd6b464de5751e12abbd155376046ce3d2afc)) +- Fix inline raster avatars in agent catalog ([#3581](https://github.com/block/buzz/pull/3581)) ([`7e9b77f72`](https://github.com/block/buzz/commit/7e9b77f72d82e019a99f074f1c9829be30c57ae1)) +- feat(agent): make Gemini and MLflow-route models usable through databricks_v2 ([#3569](https://github.com/block/buzz/pull/3569)) ([`4a1ebf25c`](https://github.com/block/buzz/commit/4a1ebf25c782fc6a68f0a69e6f866f793a259a1f)) + + +## v0.5.1 + +- perf(desktop): move observer-feed archive and decrypt commands off main thread ([#3415](https://github.com/block/buzz/pull/3415)) ([`294c8c821`](https://github.com/block/buzz/commit/294c8c821de51442a8c384c0bdb66b1a10224ca0)) +- fix(desktop): preserve shared agent fidelity ([#3553](https://github.com/block/buzz/pull/3553)) ([`f7a3988ba`](https://github.com/block/buzz/commit/f7a3988ba13b590d9a55a7e8413fc3fb5ffbef18)) +- feat(agent): route Claude/GPT model families to their native gateway wire ([#3538](https://github.com/block/buzz/pull/3538)) ([`6438dedf8`](https://github.com/block/buzz/commit/6438dedf83a9dbe1853e484326911bf6c7f1618c)) +- Refine community invite limits ([#3529](https://github.com/block/buzz/pull/3529)) ([`24d90d128`](https://github.com/block/buzz/commit/24d90d1280a9325c6cbcf8eea30ac54db5afd2cb)) +- feat(agent): fix Anthropic prompt caching with Databricks (+ MCP proxy/TLS passthrough) ([#3463](https://github.com/block/buzz/pull/3463)) ([`c405ad1d4`](https://github.com/block/buzz/commit/c405ad1d4b1da061c11b3d26761252d41dcc62d3)) +- feat: add explicit entry for claude-opus-5 in model config ([#2831](https://github.com/block/buzz/pull/2831)) ([`90e058ebf`](https://github.com/block/buzz/commit/90e058ebf68137e048a409aec6616519379ff726)) +- fix(desktop): clear stale thread new-message pill ([#3411](https://github.com/block/buzz/pull/3411)) ([`55a3ed7b9`](https://github.com/block/buzz/commit/55a3ed7b9217cee5b23e0a5441947dc929b2a38c)) +- fix(ci): ratchet file sizes against the base tree ([#3352](https://github.com/block/buzz/pull/3352)) ([`9227bdf58`](https://github.com/block/buzz/commit/9227bdf58ad6664ae3c1078888f2181ec19c4da4)) +- feat(desktop): apply WebKit rendering workarounds at startup on Linux ([#3271](https://github.com/block/buzz/pull/3271)) ([`3ece4461d`](https://github.com/block/buzz/commit/3ece4461df8a7b9663a8e68327483b8377d4086d)) +- fix(desktop): stabilize flaky DM expansion E2E ordering assertions ([#2004](https://github.com/block/buzz/pull/2004)) ([`913d564ce`](https://github.com/block/buzz/commit/913d564ce0f35924291bf3eeab6508517a6d8d1f)) +- fix(desktop): paint community rail full height ([#3382](https://github.com/block/buzz/pull/3382)) ([`1d3b810ad`](https://github.com/block/buzz/commit/1d3b810ad70d6325718ed91e723f32c4a376d5e1)) +- feat(desktop): add custom harness inline from agent dialogs ([#3252](https://github.com/block/buzz/pull/3252)) ([`b0503d80c`](https://github.com/block/buzz/commit/b0503d80c298b1ece3b0a43b41d316829a3379e7)) +- feat(desktop): refine agent catalog sharing ([#2439](https://github.com/block/buzz/pull/2439)) ([`a35771fc4`](https://github.com/block/buzz/commit/a35771fc441cdc3c6f517f419037206783b502d2)) +- fix(desktop): keep drafts out of the Inbox All view ([#3217](https://github.com/block/buzz/pull/3217)) ([`3afa129ee`](https://github.com/block/buzz/commit/3afa129ee785cc74d921d0ba969254a8255c4cc0)) +- fix(desktop): restore the inbox icon in the sidebar ([#3341](https://github.com/block/buzz/pull/3341)) ([`00ede2e7a`](https://github.com/block/buzz/commit/00ede2e7aa7eb95571b7db3ebbd163adbf6cf74e)) +- fix(desktop): gate codex-acp on a minimum supported version ([#3254](https://github.com/block/buzz/pull/3254)) ([`4e3998f36`](https://github.com/block/buzz/commit/4e3998f36e36d68b9a93dcbd85f0864450bb8f5f)) +- feat(cli): add users set-status command for NIP-38 profile status ([#3253](https://github.com/block/buzz/pull/3253)) ([`60158fce3`](https://github.com/block/buzz/commit/60158fce3e670f11bb35d42627857ccaea50ff06)) +- fix(composer): scope multiline block formatting ([#3246](https://github.com/block/buzz/pull/3246)) ([`5457c947a`](https://github.com/block/buzz/commit/5457c947a74f5ba4b979f9c6411aa7626a858387)) + + ## v0.5.0 - feat(invites): add use-limited invite links ([#3141](https://github.com/block/buzz/pull/3141)) ([`d500c2d5c`](https://github.com/block/buzz/commit/d500c2d5cf5d9aabe0ca4ebebfcafdbe5f5b7fd3)) diff --git a/Cargo.lock b/Cargo.lock index 3b60dc4579..937ead564a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -43,6 +43,20 @@ dependencies = [ "subtle", ] +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if 1.0.4", + "getrandom 0.3.4", + "once_cell", + "serde", + "version_check", + "zerocopy", +] + [[package]] name = "aho-corasick" version = "1.1.4" @@ -131,7 +145,7 @@ checksum = "5d0a66767aaf7d483c556386fb68ca2fba9347684d8bb17a4bd8b755851870f7" dependencies = [ "arrayvec", "aws-lc-rs", - "base64", + "base64 0.22.1", "byteorder", "minicbor", "rustls-pki-types", @@ -396,13 +410,23 @@ version = "1.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" +[[package]] +name = "atomic-write-file" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "84790c55b5704b0d35130bf16a4ce22a8e70eb0ea773522557524d9a4852663d" +dependencies = [ + "nix 0.30.1", + "rand 0.9.4", +] + [[package]] name = "attohttpc" version = "0.30.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "16e2cdb6d5ed835199484bb92bb8b3edd526effe995c61732580439c1a67e2e9" dependencies = [ - "base64", + "base64 0.22.1", "http", "log", "rustls", @@ -473,7 +497,7 @@ checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90" dependencies = [ "axum-core", "axum-macros", - "base64", + "base64 0.22.1", "bytes", "form_urlencoded", "futures-util", @@ -549,6 +573,12 @@ version = "1.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fd307490d624467aa6f74b0eabb77633d1f758a7b25f12bceb0b22e08d9726f6" +[[package]] +name = "base64" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8" + [[package]] name = "base64" version = "0.22.1" @@ -567,6 +597,12 @@ version = "0.11.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32637268377fc7b10a8c6d51de3e7fba1ce5dd371a96e342b34e6078db558e7f" +[[package]] +name = "beef" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3a8241f3ebb85c056b509d4327ad0358fbbba6ffb340bf388f26350aeda225b1" + [[package]] name = "bip39" version = "2.2.2" @@ -765,7 +801,7 @@ name = "buzz-acp" version = "0.1.0" dependencies = [ "anyhow", - "base64", + "base64 0.22.1", "buzz-core", "buzz-persona", "buzz-sdk", @@ -825,7 +861,7 @@ dependencies = [ "arc-swap", "async-trait", "axum", - "base64", + "base64 0.22.1", "getrandom 0.4.3", "hex", "nix 0.31.3", @@ -879,12 +915,32 @@ dependencies = [ "uuid", ] +[[package]] +name = "buzz-backend-kubernetes" +version = "0.1.0" +dependencies = [ + "chrono", + "hex", + "http", + "http-body-util", + "k8s-openapi", + "kube", + "nostr", + "rand 0.10.1", + "rustls", + "serde", + "serde_json", + "sha2 0.11.0", + "tokio", + "tower", +] + [[package]] name = "buzz-cli" version = "0.1.0" dependencies = [ "axum", - "base64", + "base64 0.22.1", "buzz-core", "buzz-persona", "buzz-sdk", @@ -925,7 +981,7 @@ dependencies = [ name = "buzz-core" version = "0.1.0" dependencies = [ - "base64", + "base64 0.22.1", "chrono", "hex", "hmac 0.13.0", @@ -949,6 +1005,8 @@ dependencies = [ "buzz-core", "chrono", "hex", + "metrics", + "metrics-util", "nostr", "rand 0.10.1", "serde", @@ -965,7 +1023,7 @@ dependencies = [ name = "buzz-dev-mcp" version = "0.1.0" dependencies = [ - "base64", + "base64 0.22.1", "buzz-cli", "buzz-core", "git-credential-nostr", @@ -1092,7 +1150,7 @@ dependencies = [ "appattest", "async-trait", "axum", - "base64", + "base64 0.22.1", "byteorder", "chrono", "getrandom 0.4.3", @@ -1127,7 +1185,7 @@ dependencies = [ "async-compression", "async-trait", "axum", - "base64", + "base64 0.22.1", "buzz-audit", "buzz-auth", "buzz-conformance", @@ -1237,8 +1295,9 @@ name = "buzz-test-client" version = "0.1.0" dependencies = [ "anyhow", - "base64", + "base64 0.22.1", "buzz-core", + "buzz-media", "buzz-sdk", "buzz-ws-client", "chrono", @@ -1262,6 +1321,25 @@ dependencies = [ "uuid", ] +[[package]] +name = "buzz-voice" +version = "0.1.0" +dependencies = [ + "atomic-write-file", + "hex", + "ort", + "ort-sys", + "rand 0.10.1", + "sentencepiece-model", + "serde", + "serde_json", + "sha2 0.11.0", + "sherpa-onnx", + "symphonia", + "tempfile", + "tokenizers", +] + [[package]] name = "buzz-workflow" version = "0.1.0" @@ -1329,6 +1407,26 @@ version = "1.11.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33" +[[package]] +name = "bzip2" +version = "0.4.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bdb116a6ef3f6c3698828873ad02c3014b3c85cadb88496095628e3ef1e347f8" +dependencies = [ + "bzip2-sys", + "libc", +] + +[[package]] +name = "bzip2-sys" +version = "0.1.13+1.0.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "225bff33b2141874fe80d71e07d6eec4f85c5c216453dd96388240f96e1acc14" +dependencies = [ + "cc", + "pkg-config", +] + [[package]] name = "castaway" version = "0.2.4" @@ -1577,6 +1675,7 @@ dependencies = [ "itoa", "rustversion", "ryu", + "serde", "static_assertions", ] @@ -2137,6 +2236,15 @@ dependencies = [ "syn 2.0.117", ] +[[package]] +name = "dary_heap" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b1e3a325bc115f096c8b77bbf027a7c2592230e70be2d985be950d3d5e60ebe" +dependencies = [ + "serde", +] + [[package]] name = "dashmap" version = "6.2.1" @@ -2174,7 +2282,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ccc2776f0c61eca1ca32528f85548abd1a4be8fb53d1b21c013e4f18da1e7090" dependencies = [ "data-encoding", - "syn 1.0.109", + "syn 2.0.117", ] [[package]] @@ -2626,6 +2734,12 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "esaxx-rs" +version = "0.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d817e038c30374a4bcb22f94d0a8a0e216958d4c3dcde369b1439fec4bdda6e6" + [[package]] name = "etcetera" version = "0.11.0" @@ -2683,6 +2797,12 @@ dependencies = [ "smallvec", ] +[[package]] +name = "extended" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "af9673d8203fcb076b19dfd17e38b3d4ae9f44959416ea532ce72415a6020365" + [[package]] name = "fancy-regex" version = "0.11.0" @@ -2693,6 +2813,17 @@ dependencies = [ "regex", ] +[[package]] +name = "fancy-regex" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e24cb5a94bcae1e5408b0effca5cd7172ea3c5755049c5f3af4cd283a165298" +dependencies = [ + "bit-set 0.8.0", + "regex-automata", + "regex-syntax", +] + [[package]] name = "fast-srgb8" version = "1.0.0" @@ -3084,7 +3215,7 @@ dependencies = [ name = "git-credential-nostr" version = "0.1.0" dependencies = [ - "base64", + "base64 0.22.1", "nostr", "serde_json", "zeroize", @@ -3094,7 +3225,7 @@ dependencies = [ name = "git-sign-nostr" version = "0.1.0" dependencies = [ - "base64", + "base64 0.22.1", "chrono", "hex", "libc", @@ -3269,7 +3400,7 @@ version = "1.0.0-rc.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5f89305dc8fe34e165eaf0eb12b6e294e12381d9df9a431bcc52a5809bab4319" dependencies = [ - "base64", + "base64 0.22.1", "bon", "bytes", "futures", @@ -3532,6 +3663,7 @@ dependencies = [ "http", "hyper", "hyper-util", + "log", "rustls", "rustls-native-certs", "tokio", @@ -3575,7 +3707,7 @@ version = "0.1.20" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0" dependencies = [ - "base64", + "base64 0.22.1", "bytes", "futures-channel", "futures-util", @@ -4192,6 +4324,31 @@ dependencies = [ "ucd-trie", ] +[[package]] +name = "jsonpath-rust" +version = "0.7.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c00ae348f9f8fd2d09f82a98ca381c60df9e0820d8d79fce43e649b4dc3128b" +dependencies = [ + "pest", + "pest_derive", + "regex", + "serde_json", + "thiserror 2.0.18", +] + +[[package]] +name = "k8s-openapi" +version = "0.26.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06d9e5e61dd037cdc51da0d7e2b2be10f497478ea7e120d85dad632adb99882b" +dependencies = [ + "base64 0.22.1", + "chrono", + "serde", + "serde_json", +] + [[package]] name = "kasuari" version = "0.4.12" @@ -4237,6 +4394,70 @@ version = "0.4.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e037a2e1d8d5fdbd49b16a4ea09d5d6401c1f29eca5ff29d03d3824dba16256a" +[[package]] +name = "kube" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "48e7bb0b6a46502cc20e4575b6ff401af45cfea150b34ba272a3410b78aa014e" +dependencies = [ + "k8s-openapi", + "kube-client", + "kube-core", +] + +[[package]] +name = "kube-client" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4987d57a184d2b5294fdad3d7fc7f278899469d21a4da39a8f6ca16426567a36" +dependencies = [ + "base64 0.22.1", + "bytes", + "chrono", + "either", + "futures", + "home", + "http", + "http-body", + "http-body-util", + "hyper", + "hyper-rustls", + "hyper-timeout", + "hyper-util", + "jsonpath-rust", + "k8s-openapi", + "kube-core", + "pem", + "rustls", + "secrecy", + "serde", + "serde_json", + "serde_yaml", + "thiserror 2.0.18", + "tokio", + "tokio-util", + "tower", + "tower-http", + "tracing", +] + +[[package]] +name = "kube-core" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "914bbb770e7bb721a06e3538c0edd2babed46447d128f7c21caa68747060ee73" +dependencies = [ + "chrono", + "derive_more", + "form_urlencoded", + "http", + "k8s-openapi", + "serde", + "serde-value", + "serde_json", + "thiserror 2.0.18", +] + [[package]] name = "lab" version = "0.11.0" @@ -4359,6 +4580,39 @@ version = "0.4.29" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "5e5032e24019045c762d3c0f28f5b6b8bbf38563a65908389bf7978758920897" +[[package]] +name = "logos" +version = "0.14.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7251356ef8cb7aec833ddf598c6cb24d17b689d20b993f9d11a3d764e34e6458" +dependencies = [ + "logos-derive", +] + +[[package]] +name = "logos-codegen" +version = "0.14.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59f80069600c0d66734f5ff52cc42f2dabd6b29d205f333d61fd7832e9e9963f" +dependencies = [ + "beef", + "fnv", + "lazy_static", + "proc-macro2", + "quote", + "regex-syntax", + "syn 2.0.117", +] + +[[package]] +name = "logos-derive" +version = "0.14.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24fb722b06a9dc12adb0963ed585f19fc61dc5413e6a9be9422ef92c091e731d" +dependencies = [ + "logos-codegen", +] + [[package]] name = "loom" version = "0.7.2" @@ -4418,6 +4672,22 @@ dependencies = [ "winapi", ] +[[package]] +name = "macro_rules_attribute" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b3ae8f6d608c795738406608304d30a2dfbdc8e58e44f7ba43236da5208ded3c" +dependencies = [ + "macro_rules_attribute-proc_macro", + "pastey", +] + +[[package]] +name = "macro_rules_attribute-proc_macro" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc04a4c58212d57930a24bf47d3fa87485264a3a054e9c10e042eb373573ad3c" + [[package]] name = "matchers" version = "0.2.0" @@ -4433,6 +4703,16 @@ version = "0.8.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3" +[[package]] +name = "matrixmultiply" +version = "0.3.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f607c237553f086e7043417a51df26b2eb899d3caff94e6a67592ff992fedc7" +dependencies = [ + "autocfg", + "rawpointer", +] + [[package]] name = "maybe-async" version = "0.2.11" @@ -4498,8 +4778,8 @@ dependencies = [ [[package]] name = "mesh-llm-api-client" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "hex", "mesh-llm-client", @@ -4508,8 +4788,8 @@ dependencies = [ [[package]] name = "mesh-llm-api-server" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "mesh-llm-api-client", @@ -4519,17 +4799,17 @@ dependencies = [ [[package]] name = "mesh-llm-build-info" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" [[package]] name = "mesh-llm-client" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "async-trait", - "base64", + "base64 0.22.1", "bytes", "crypto_box", "ed25519-dalek", @@ -4542,7 +4822,7 @@ dependencies = [ "mesh-llm-types", "model-artifact", "nostr-sdk", - "prost", + "prost 0.14.3", "rand 0.10.1", "rustls", "serde", @@ -4556,8 +4836,8 @@ dependencies = [ [[package]] name = "mesh-llm-config" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "dirs", @@ -4572,8 +4852,8 @@ dependencies = [ [[package]] name = "mesh-llm-embedded-runtime" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "mesh-llm-host-runtime", @@ -4582,8 +4862,8 @@ dependencies = [ [[package]] name = "mesh-llm-events" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "clap", @@ -4594,8 +4874,8 @@ dependencies = [ [[package]] name = "mesh-llm-gpu-bench" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "cc", @@ -4607,8 +4887,8 @@ dependencies = [ [[package]] name = "mesh-llm-guardrails" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "serde", "serde_json", @@ -4616,22 +4896,22 @@ dependencies = [ [[package]] name = "mesh-llm-hardware-profile" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "mesh-llm-native-runtime", ] [[package]] name = "mesh-llm-host-runtime" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "argon2", "async-trait", "axum", - "base64", + "base64 0.22.1", "bytes", "chacha20poly1305", "chrono", @@ -4647,7 +4927,6 @@ dependencies = [ "http", "http-body-util", "httparse", - "if-addrs", "iroh", "json5", "keyring", @@ -4681,7 +4960,7 @@ dependencies = [ "opentelemetry 0.31.0", "opentelemetry-otlp 0.31.1", "opentelemetry_sdk 0.31.0", - "prost", + "prost 0.14.3", "rand 0.10.1", "regex-lite", "reqwest 0.12.28", @@ -4695,6 +4974,7 @@ dependencies = [ "serde_yaml", "sha2 0.10.9", "skippy-coordinator", + "skippy-ffi", "skippy-protocol", "skippy-runtime", "skippy-server", @@ -4717,11 +4997,11 @@ dependencies = [ [[package]] name = "mesh-llm-identity" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "argon2", - "base64", + "base64 0.22.1", "chacha20poly1305", "chrono", "crypto_box", @@ -4739,8 +5019,8 @@ dependencies = [ [[package]] name = "mesh-llm-native-runtime" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "serde", @@ -4750,8 +5030,8 @@ dependencies = [ [[package]] name = "mesh-llm-node" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "mesh-llm-types", @@ -4764,13 +5044,13 @@ dependencies = [ [[package]] name = "mesh-llm-plugin" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "async-trait", - "prost", - "prost-build", + "prost 0.14.3", + "prost-build 0.14.3", "protoc-bin-vendored", "rmcp", "schemars", @@ -4781,8 +5061,8 @@ dependencies = [ [[package]] name = "mesh-llm-plugin-manager" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "dirs", @@ -4792,6 +5072,7 @@ dependencies = [ "reqwest 0.12.28", "serde", "serde_json", + "sha2 0.10.9", "tar", "tempfile", "zip", @@ -4799,29 +5080,29 @@ dependencies = [ [[package]] name = "mesh-llm-protocol" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "hex", "iroh", - "prost", + "prost 0.14.3", "serde_json", "sha2 0.10.9", ] [[package]] name = "mesh-llm-routing" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "iroh", ] [[package]] name = "mesh-llm-runtime-install" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "dirs", @@ -4843,8 +5124,8 @@ dependencies = [ [[package]] name = "mesh-llm-sdk" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "mesh-llm-api-client", @@ -4858,8 +5139,8 @@ dependencies = [ [[package]] name = "mesh-llm-skills" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "dirs", @@ -4869,8 +5150,8 @@ dependencies = [ [[package]] name = "mesh-llm-system" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "chrono", @@ -4892,8 +5173,8 @@ dependencies = [ [[package]] name = "mesh-llm-types" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "hex", "serde", @@ -4903,13 +5184,13 @@ dependencies = [ [[package]] name = "mesh-llm-ui" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" [[package]] name = "mesh-mixture-of-agents" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "async-trait", "mesh-llm-guardrails", @@ -4936,7 +5217,7 @@ version = "0.18.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1db0d8f1fc9e62caebd0319e11eaec5822b0186c171568f0480b46a0137f9108" dependencies = [ - "base64", + "base64 0.22.1", "evmap", "http-body-util", "hyper", @@ -4974,6 +5255,28 @@ dependencies = [ "sketches-ddsketch", ] +[[package]] +name = "miette" +version = "7.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f98efec8807c63c752b5bd61f862c165c115b0a35685bdcfd9238c7aeb592b7" +dependencies = [ + "cfg-if 1.0.4", + "miette-derive", + "unicode-width 0.1.14", +] + +[[package]] +name = "miette-derive" +version = "7.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db5b29714e950dbb20d5e6f74f9dcec4edbcc1067bb7f8ed198c097b8c1a818b" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + [[package]] name = "mime" version = "0.3.17" @@ -5035,8 +5338,8 @@ dependencies = [ [[package]] name = "model-artifact" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "async-trait", @@ -5046,8 +5349,8 @@ dependencies = [ [[package]] name = "model-hf" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "async-trait", @@ -5064,8 +5367,8 @@ dependencies = [ [[package]] name = "model-package" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "bytes", @@ -5084,16 +5387,16 @@ dependencies = [ [[package]] name = "model-ref" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "serde", ] [[package]] name = "model-resolver" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "model-artifact", @@ -5119,6 +5422,28 @@ dependencies = [ "uuid", ] +[[package]] +name = "monostate" +version = "0.1.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3341a273f6c9d5bef1908f17b7267bbab0e95c9bf69a0d4dcf8e9e1b2c76ef67" +dependencies = [ + "monostate-impl", + "serde", + "serde_core", +] + +[[package]] +name = "monostate-impl" +version = "0.1.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e4db6d5580af57bf992f59068d4ea26fd518574ff48d7639b255a36f9de6e7e9" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.117", +] + [[package]] name = "more-asserts" version = "0.3.1" @@ -5225,6 +5550,21 @@ dependencies = [ "tempfile", ] +[[package]] +name = "ndarray" +version = "0.17.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "520080814a7a6b4a6e9070823bb24b4531daac8c4627e08ba5de8c5ef2f2752d" +dependencies = [ + "matrixmultiply", + "num-complex", + "num-integer", + "num-traits", + "portable-atomic", + "portable-atomic-util", + "rawpointer", +] + [[package]] name = "ndk-context" version = "0.1.1" @@ -5373,9 +5713,9 @@ dependencies = [ [[package]] name = "nix" -version = "0.31.3" +version = "0.30.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d" +checksum = "74523f3a35e05aba87a1d978330aef40f67b0304ac79c1c00b294c9830543db6" dependencies = [ "bitflags 2.13.0", "cfg-if 1.0.4", @@ -5384,8 +5724,20 @@ dependencies = [ ] [[package]] -name = "nom" -version = "7.1.3" +name = "nix" +version = "0.31.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d" +dependencies = [ + "bitflags 2.13.0", + "cfg-if 1.0.4", + "cfg_aliases", + "libc", +] + +[[package]] +name = "nom" +version = "7.1.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a" dependencies = [ @@ -5457,11 +5809,11 @@ dependencies = [ [[package]] name = "nostr" -version = "0.44.6" +version = "0.44.7" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e826dd648489de2c5b293920e20b92932ef820302007c1987c758d4d06eeb2cf" +checksum = "c7d3d987ea7078dc36947cde532637c472a229426702e4331dd7667325378bd9" dependencies = [ - "base64", + "base64 0.22.1", "bech32", "bip39", "bitcoin_hashes", @@ -5501,9 +5853,9 @@ dependencies = [ [[package]] name = "nostr-relay-pool" -version = "0.44.1" +version = "0.44.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "91b2c039df4f96c4bf7dae52a74fd5516ad6dda83a11c0c69dea91b5255a4f37" +checksum = "c85c54d6ca9aae4ae2bf19a7663ba9db5f45f783f1d24aff55f006386b8b99a1" dependencies = [ "async-utility", "async-wsocket", @@ -5812,8 +6164,8 @@ checksum = "c08d65885ee38876c4f86fa503fb49d7b507c2b62552df7c70b2fce627e06381" [[package]] name = "openai-frontend" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "async-trait", "axum", @@ -5932,7 +6284,7 @@ dependencies = [ "opentelemetry-http", "opentelemetry-proto 0.31.0", "opentelemetry_sdk 0.31.0", - "prost", + "prost 0.14.3", "reqwest 0.12.28", "thiserror 2.0.18", ] @@ -5947,7 +6299,7 @@ dependencies = [ "opentelemetry 0.32.0", "opentelemetry-proto 0.32.0", "opentelemetry_sdk 0.32.1", - "prost", + "prost 0.14.3", "thiserror 2.0.18", "tokio", "tonic", @@ -5960,11 +6312,11 @@ version = "0.31.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a7175df06de5eaee9909d4805a3d07e28bb752c34cab57fa9cff549da596b30f" dependencies = [ - "base64", + "base64 0.22.1", "const-hex", "opentelemetry 0.31.0", "opentelemetry_sdk 0.31.0", - "prost", + "prost 0.14.3", "serde", "serde_json", "tonic", @@ -5979,7 +6331,7 @@ checksum = "56d658ba1faf63f7b9c492cfbe6e0ec365440a16132d3270c1065f7b33f1b638" dependencies = [ "opentelemetry 0.32.0", "opentelemetry_sdk 0.32.1", - "prost", + "prost 0.14.3", "tonic", "tonic-prost", ] @@ -6023,6 +6375,15 @@ version = "0.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "04744f49eae99ab78e0d5c0b603ab218f515ea8cfe5a456d7629ad883a3b6e7d" +[[package]] +name = "ordered-float" +version = "2.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "68f19d67e5a2795c94e73e0bb1cc1a7edeb2e28efd39e2e1c9b7a40c1108b11c" +dependencies = [ + "num-traits", +] + [[package]] name = "ordered-float" version = "4.6.0" @@ -6061,6 +6422,24 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "ort" +version = "2.0.0-rc.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7de3af33d24a745ffb8fab904b13478438d1cd52868e6f17735ef6e1f8bf133" +dependencies = [ + "ndarray", + "ort-sys", + "smallvec", + "tracing", +] + +[[package]] +name = "ort-sys" +version = "2.0.0-rc.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7b497d21a8b6fbb4b5a544f8fadb77e801a09ae0add9e411d31c6f89e3c1e90" + [[package]] name = "os_str_bytes" version = "6.6.1" @@ -6185,6 +6564,16 @@ dependencies = [ "hmac 0.12.1", ] +[[package]] +name = "pem" +version = "3.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d30c53c26bc5b31a98cd02d20f25a7c8567146caf63ed593a9d87b2775291be" +dependencies = [ + "base64 0.22.1", + "serde_core", +] + [[package]] name = "pem-rfc7468" version = "1.0.0" @@ -6242,6 +6631,16 @@ dependencies = [ "pest", ] +[[package]] +name = "petgraph" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3672b37090dbd86368a4145bc067582552b29c27377cad4e0a306c97f9bd7772" +dependencies = [ + "fixedbitset 0.5.7", + "indexmap", +] + [[package]] name = "petgraph" version = "0.8.3" @@ -6374,7 +6773,7 @@ version = "1.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "092791278e026273c1b65bbdcfbba3a300f2994c896bd01ab01da613c29c46f1" dependencies = [ - "base64", + "base64 0.22.1", "indexmap", "quick-xml 0.39.4", "serde", @@ -6440,13 +6839,22 @@ dependencies = [ "serde", ] +[[package]] +name = "portable-atomic-util" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a106d1259c23fac8e543272398ae0e3c0b8d33c88ed73d0cc71b0f1d902618" +dependencies = [ + "portable-atomic", +] + [[package]] name = "portmapper" version = "0.19.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "eb3713e4977408279158444a18c1a01ac9bf2e7eaf1fbfd1a19ac9cd18d90721" dependencies = [ - "base64", + "base64 0.22.1", "bytes", "derive_more", "hyper-util", @@ -6616,6 +7024,16 @@ dependencies = [ "unarray", ] +[[package]] +name = "prost" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2796faa41db3ec313a31f7624d9286acf277b52de526150b7e69f3debf891ee5" +dependencies = [ + "bytes", + "prost-derive 0.13.5", +] + [[package]] name = "prost" version = "0.14.3" @@ -6623,7 +7041,27 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d2ea70524a2f82d518bce41317d0fae74151505651af45faf1ffbd6fd33f0568" dependencies = [ "bytes", - "prost-derive", + "prost-derive 0.14.3", +] + +[[package]] +name = "prost-build" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf" +dependencies = [ + "heck", + "itertools", + "log", + "multimap", + "once_cell", + "petgraph 0.7.1", + "prettyplease", + "prost 0.13.5", + "prost-types 0.13.5", + "regex", + "syn 2.0.117", + "tempfile", ] [[package]] @@ -6636,15 +7074,28 @@ dependencies = [ "itertools", "log", "multimap", - "petgraph", + "petgraph 0.8.3", "prettyplease", - "prost", - "prost-types", + "prost 0.14.3", + "prost-types 0.14.3", "regex", "syn 2.0.117", "tempfile", ] +[[package]] +name = "prost-derive" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a56d757972c98b346a9b766e3f02746cde6dd1cd1d1d563472929fdd74bec4d" +dependencies = [ + "anyhow", + "itertools", + "proc-macro2", + "quote", + "syn 2.0.117", +] + [[package]] name = "prost-derive" version = "0.14.3" @@ -6658,13 +7109,35 @@ dependencies = [ "syn 2.0.117", ] +[[package]] +name = "prost-reflect" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b5edd582b62f5cde844716e66d92565d7faf7ab1445c8cebce6e00fba83ddb2" +dependencies = [ + "logos", + "miette", + "once_cell", + "prost 0.13.5", + "prost-types 0.13.5", +] + +[[package]] +name = "prost-types" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52c2c1bf36ddb1a1c396b3601a3cec27c2462e45f07c386894ec3ccf5332bd16" +dependencies = [ + "prost 0.13.5", +] + [[package]] name = "prost-types" version = "0.14.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8991c4cbdb8bc5b11f0b074ffe286c30e523de90fee5ba8132f1399f23cb3dd7" dependencies = [ - "prost", + "prost 0.14.3", ] [[package]] @@ -6731,6 +7204,33 @@ version = "3.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "95067976aca6421a523e491fce939a3e65249bac4b977adee0ee9771568e8aa3" +[[package]] +name = "protox" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6f352af331bf637b8ecc720f7c87bf903d2571fa2e14a66e9b2558846864b54a" +dependencies = [ + "bytes", + "miette", + "prost 0.13.5", + "prost-reflect", + "prost-types 0.13.5", + "protox-parse", + "thiserror 1.0.69", +] + +[[package]] +name = "protox-parse" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a3a462d115462c080ae000c29a47f0b3985737e5d3a995fcdbcaa5c782068dde" +dependencies = [ + "logos", + "miette", + "prost-types 0.13.5", + "thiserror 1.0.69", +] + [[package]] name = "pulldown-cmark" version = "0.13.4" @@ -7036,7 +7536,7 @@ dependencies = [ "thiserror 2.0.18", "unicode-segmentation", "unicode-truncate", - "unicode-width", + "unicode-width 0.2.2", ] [[package]] @@ -7099,7 +7599,7 @@ dependencies = [ "strum", "time", "unicode-segmentation", - "unicode-width", + "unicode-width 0.2.2", ] [[package]] @@ -7111,6 +7611,43 @@ dependencies = [ "bitflags 2.13.0", ] +[[package]] +name = "rawpointer" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3" + +[[package]] +name = "rayon" +version = "1.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d" +dependencies = [ + "either", + "rayon-core", +] + +[[package]] +name = "rayon-cond" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2964d0cf57a3e7a06e8183d14a8b527195c706b7983549cd5462d5aa3747438f" +dependencies = [ + "either", + "itertools", + "rayon", +] + +[[package]] +name = "rayon-core" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91" +dependencies = [ + "crossbeam-deque", + "crossbeam-utils", +] + [[package]] name = "redb" version = "3.1.3" @@ -7232,7 +7769,7 @@ version = "0.12.28" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "eddd3ca559203180a307f12d114c268abf583f59b03cb906fd0b3ff8646c1147" dependencies = [ - "base64", + "base64 0.22.1", "bytes", "encoding_rs", "futures-channel", @@ -7280,7 +7817,7 @@ version = "0.13.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "219c5811de6525e5416c7d5d53bb656d3afdbc6c5af816e0802bcfa42dbdc1c3" dependencies = [ - "base64", + "base64 0.22.1", "bytes", "encoding_rs", "futures-core", @@ -7365,12 +7902,12 @@ dependencies = [ [[package]] name = "rmcp" -version = "1.7.0" +version = "1.8.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0810a9f717d9828f475fe1f629f4c305c8464b7f496c3a854b58d29e65f4058e" +checksum = "1d1f571c72940a19d9532fe52dbea8bc9912bf1d766c2970bb824056b86f3f59" dependencies = [ "async-trait", - "base64", + "base64 0.22.1", "bytes", "chrono", "futures", @@ -7398,9 +7935,9 @@ dependencies = [ [[package]] name = "rmcp-macros" -version = "1.7.0" +version = "1.8.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6aefac48c364756e97f04c0401ba3231e8607882c7c1d92da0437dc16307904d" +checksum = "1aad0035b69380782d78ea95b508327e6deaa2235909053e596eea8f27b5e1d5" dependencies = [ "darling 0.23.0", "proc-macro2", @@ -7438,7 +7975,7 @@ dependencies = [ "async-trait", "aws-creds", "aws-region", - "base64", + "base64 0.22.1", "bytes", "cfg-if 1.0.4", "futures-util", @@ -7762,6 +8299,15 @@ dependencies = [ "cc", ] +[[package]] +name = "secrecy" +version = "0.10.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e891af845473308773346dc847b2c23ee78fe442e0472ac50e22a18a93d3ae5a" +dependencies = [ + "zeroize", +] + [[package]] name = "secret-service" version = "4.0.0" @@ -7839,6 +8385,18 @@ version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cd0b0ec5f1c1ca621c432a25813d8d60c88abe6d3e08a3eb9cf37d97a0fe3d73" +[[package]] +name = "sentencepiece-model" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40b87bf750a8322c3236d7aa63c1f4a6862187d00d2d8b038e1dfe263bfe43ec" +dependencies = [ + "miette", + "prost 0.13.5", + "prost-build 0.13.5", + "protox", +] + [[package]] name = "serde" version = "1.0.228" @@ -7849,6 +8407,16 @@ dependencies = [ "serde_derive", ] +[[package]] +name = "serde-value" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f3a1a3341211875ef120e117ea7fd5228530ae7e7036a779fdc9117be6b3282c" +dependencies = [ + "ordered-float 2.10.1", + "serde", +] + [[package]] name = "serde_bytes" version = "0.11.19" @@ -8049,6 +8617,28 @@ dependencies = [ "os_str_bytes", ] +[[package]] +name = "sherpa-onnx" +version = "1.13.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b142d3f255cb4e4b7808ea25869db6f5714e0a3550da355234483b4db552055" +dependencies = [ + "serde", + "serde_json", + "sherpa-onnx-sys", +] + +[[package]] +name = "sherpa-onnx-sys" +version = "1.13.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ffc951af03dc0653c0622158ca8a585a6f2bc43b7b06048cf0e5b5020005c227" +dependencies = [ + "bzip2", + "tar", + "ureq", +] + [[package]] name = "shlex" version = "1.3.0" @@ -8150,8 +8740,8 @@ checksum = "0c6f73aeb92d671e0cc4dca167e59b2deb6387c375391bc99ee743f326994a2b" [[package]] name = "skippy-cache" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "blake3", @@ -8160,40 +8750,40 @@ dependencies = [ [[package]] name = "skippy-coordinator" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "thiserror 2.0.18", ] [[package]] name = "skippy-ffi" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "libloading", ] [[package]] name = "skippy-metrics" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" [[package]] name = "skippy-protocol" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ - "prost", - "prost-build", + "prost 0.14.3", + "prost-build 0.14.3", "protoc-bin-vendored", "serde", ] [[package]] name = "skippy-runtime" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "libc", @@ -8206,13 +8796,14 @@ dependencies = [ [[package]] name = "skippy-server" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ + "ahash", "anyhow", "async-trait", "axum", - "base64", + "base64 0.22.1", "blake3", "clap", "futures-util", @@ -8234,8 +8825,8 @@ dependencies = [ [[package]] name = "skippy-topology" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.73.1#43103c5c40292be688ac0261129bcbab0e7b9132" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "serde", "serde_json", @@ -8319,6 +8910,18 @@ dependencies = [ "der", ] +[[package]] +name = "spm_precompiled" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5851699c4033c63636f7ea4cf7b7c1f1bf06d0cc03cfb42e711de5a5c46cf326" +dependencies = [ + "base64 0.13.1", + "nom", + "serde", + "unicode-segmentation", +] + [[package]] name = "sprig" version = "0.1.0" @@ -8347,7 +8950,7 @@ version = "0.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "05b44e85bf579a8eeb4ceaa77a3a523baf2bf0e9bac7e40f405d537b5d2d5ccb" dependencies = [ - "base64", + "base64 0.22.1", "bytes", "cfg-if 1.0.4", "chrono", @@ -8453,7 +9056,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "87a2bdd6e83f6b3ea525ca9fee568030508b58355a43d0b2c1674d5f79dcd65e" dependencies = [ "atoi", - "base64", + "base64 0.22.1", "bitflags 2.13.0", "byteorder", "chrono", @@ -8594,6 +9197,164 @@ version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a7973cce6668464ea31f176d85b13c7ab3bba2cb3b77a2ed26abd7801688010a" +[[package]] +name = "symphonia" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5773a4c030a19d9bfaa090f49746ff35c75dfddfa700df7a5939d5e076a57039" +dependencies = [ + "lazy_static", + "symphonia-bundle-flac", + "symphonia-bundle-mp3", + "symphonia-codec-aac", + "symphonia-codec-alac", + "symphonia-codec-pcm", + "symphonia-codec-vorbis", + "symphonia-core", + "symphonia-format-isomp4", + "symphonia-format-ogg", + "symphonia-format-riff", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-bundle-flac" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c91565e180aea25d9b80a910c546802526ffd0072d0b8974e3ebe59b686c9976" +dependencies = [ + "log", + "symphonia-core", + "symphonia-metadata", + "symphonia-utils-xiph", +] + +[[package]] +name = "symphonia-bundle-mp3" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4872dd6bb56bf5eac799e3e957aa1981086c3e613b27e0ac23b176054f7c57ed" +dependencies = [ + "lazy_static", + "log", + "symphonia-core", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-codec-aac" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4c263845aa86881416849c1729a54c7f55164f8b96111dba59de46849e73a790" +dependencies = [ + "lazy_static", + "log", + "symphonia-core", +] + +[[package]] +name = "symphonia-codec-alac" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8413fa754942ac16a73634c9dfd1500ed5c61430956b33728567f667fdd393ab" +dependencies = [ + "log", + "symphonia-core", +] + +[[package]] +name = "symphonia-codec-pcm" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e89d716c01541ad3ebe7c91ce4c8d38a7cf266a3f7b2f090b108fb0cb031d95" +dependencies = [ + "log", + "symphonia-core", +] + +[[package]] +name = "symphonia-codec-vorbis" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f025837c309cd69ffef572750b4a2257b59552c5399a5e49707cc5b1b85d1c73" +dependencies = [ + "log", + "symphonia-core", + "symphonia-utils-xiph", +] + +[[package]] +name = "symphonia-core" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ea00cc4f79b7f6bb7ff87eddc065a1066f3a43fe1875979056672c9ef948c2af" +dependencies = [ + "arrayvec", + "bitflags 1.3.2", + "bytemuck", + "lazy_static", + "log", +] + +[[package]] +name = "symphonia-format-isomp4" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "243739585d11f81daf8dac8d9f3d18cc7898f6c09a259675fc364b382c30e0a5" +dependencies = [ + "encoding_rs", + "log", + "symphonia-core", + "symphonia-metadata", + "symphonia-utils-xiph", +] + +[[package]] +name = "symphonia-format-ogg" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2b4955c67c1ed3aa8ae8428d04ca8397fbef6a19b2b051e73b5da8b1435639cb" +dependencies = [ + "log", + "symphonia-core", + "symphonia-metadata", + "symphonia-utils-xiph", +] + +[[package]] +name = "symphonia-format-riff" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2d7c3df0e7d94efb68401d81906eae73c02b40d5ec1a141962c592d0f11a96f" +dependencies = [ + "extended", + "log", + "symphonia-core", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-metadata" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "36306ff42b9ffe6e5afc99d49e121e0bd62fe79b9db7b9681d48e29fa19e6b16" +dependencies = [ + "encoding_rs", + "lazy_static", + "log", + "symphonia-core", +] + +[[package]] +name = "symphonia-utils-xiph" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ee27c85ab799a338446b68eec77abf42e1a6f1bb490656e121c6e27bfbab9f16" +dependencies = [ + "symphonia-core", + "symphonia-metadata", +] + [[package]] name = "syn" version = "1.0.109" @@ -8691,7 +9452,7 @@ version = "1.4.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fce91f2f0ec87dff7e6bcbbeb267439aa1188703003c6055193c821487400432" dependencies = [ - "unicode-width", + "unicode-width 0.2.2", ] [[package]] @@ -8765,9 +9526,9 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4676b37242ccbd1aabf56edb093a4827dc49086c0ffd764a5705899e0f35f8f7" dependencies = [ "anyhow", - "base64", + "base64 0.22.1", "bitflags 2.13.0", - "fancy-regex", + "fancy-regex 0.11.0", "filedescriptor", "finl_unicode", "fixedbitset 0.4.2", @@ -8917,6 +9678,39 @@ version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" +[[package]] +name = "tokenizers" +version = "0.22.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b238e22d44a15349529690fb07bd645cf58149a1b1e44d6cb5bd1641ff1a6223" +dependencies = [ + "ahash", + "aho-corasick", + "compact_str 0.9.1", + "dary_heap", + "derive_builder", + "esaxx-rs", + "fancy-regex 0.14.0", + "getrandom 0.3.4", + "itertools", + "log", + "macro_rules_attribute", + "monostate", + "paste", + "rand 0.9.4", + "rayon", + "rayon-cond", + "regex", + "regex-syntax", + "serde", + "serde_json", + "spm_precompiled", + "thiserror 2.0.18", + "unicode-normalization-alignments", + "unicode-segmentation", + "unicode_categories", +] + [[package]] name = "tokio" version = "1.52.3" @@ -9052,7 +9846,7 @@ version = "0.13.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "dad543404f98bfc969aeb71994105c592acfc6c43323fddcd016bb208d1c65cb" dependencies = [ - "base64", + "base64 0.22.1", "bytes", "futures-core", "futures-sink", @@ -9153,7 +9947,7 @@ checksum = "ac2a5518c70fa84342385732db33fb3f44bc4cc748936eb5833d2df34d6445ef" dependencies = [ "async-trait", "axum", - "base64", + "base64 0.22.1", "bytes", "h2", "http", @@ -9182,7 +9976,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "50849f68853be452acf590cde0b146665b8d507b3b8af17261df47e02c209ea0" dependencies = [ "bytes", - "prost", + "prost 0.14.3", "tonic", ] @@ -9192,8 +9986,8 @@ version = "0.14.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "73ab1b02061f83d519bba3caa167f88f261ef05720ab8ebc954ade70de3348e8" dependencies = [ - "prost", - "prost-types", + "prost 0.14.3", + "prost-types 0.14.3", "tonic", ] @@ -9223,6 +10017,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4cfcf7e2740e6fc6d4d688b4ef00650406bb94adf4731e43c096c3a19fe40840" dependencies = [ "async-compression", + "base64 0.22.1", "bitflags 2.13.0", "bytes", "futures-core", @@ -9482,6 +10277,15 @@ dependencies = [ "tinyvec", ] +[[package]] +name = "unicode-normalization-alignments" +version = "0.1.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43f613e4fa046e69818dd287fdc4bc78175ff20331479dab6e1b0f98d57062de" +dependencies = [ + "smallvec", +] + [[package]] name = "unicode-properties" version = "0.1.4" @@ -9502,9 +10306,15 @@ checksum = "16b380a1238663e5f8a691f9039c73e1cdae598a30e9855f541d29b08b53e9a5" dependencies = [ "itertools", "unicode-segmentation", - "unicode-width", + "unicode-width 0.2.2", ] +[[package]] +name = "unicode-width" +version = "0.1.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af" + [[package]] name = "unicode-width" version = "0.2.2" @@ -9517,6 +10327,12 @@ version = "0.2.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" +[[package]] +name = "unicode_categories" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39ec24b3121d976906ece63c9daad25b85969647682eee313cb5779fdd69e14e" + [[package]] name = "universal-hash" version = "0.5.1" @@ -9539,6 +10355,22 @@ version = "0.9.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1" +[[package]] +name = "ureq" +version = "2.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "02d1a66277ed75f640d608235660df48c8e3c19f3b4edb6a263315626cc3c01d" +dependencies = [ + "base64 0.22.1", + "flate2", + "log", + "once_cell", + "rustls", + "rustls-pki-types", + "url", + "webpki-roots 0.26.11", +] + [[package]] name = "url" version = "2.5.8" @@ -10530,7 +11362,7 @@ checksum = "3e1e496dcbe6a09017acdfaf48e1a646735e7ff5b2a49e2c7e081cca77a59bc8" dependencies = [ "anyhow", "async-trait", - "base64", + "base64 0.22.1", "bytes", "clap", "crc32fast", @@ -10567,7 +11399,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cb838aa8eb67d730af301584cf003caad407487606058292a6750711b603fbee" dependencies = [ "async-trait", - "base64", + "base64 0.22.1", "blake3", "bytemuck", "bytes", diff --git a/Cargo.toml b/Cargo.toml index 3ac7ee4cce..cc1dd0f9df 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -26,6 +26,8 @@ members = [ "crates/buzz-pair-relay", "crates/buzz-relay-mesh", "crates/buzz-dev-mcp", + "crates/buzz-voice", + "crates/buzz-backend-kubernetes", "examples/countdown-bot", ] exclude = ["desktop/src-tauri"] @@ -57,6 +59,13 @@ sqlx = { version = "0.9", features = [ redis = { version = "1.0", features = ["tokio-comp", "connection-manager", "tokio-rustls-comp"] } deadpool-redis = { version = "0.23", features = ["rt_tokio_1"] } +# Kubernetes (buzz-backend-kubernetes provider). No `ring` feature here: the +# process-level CryptoProvider is installed explicitly at startup, matching +# buzz-cli/buzz-acp/buzz-admin/buzz-relay/buzz-dev-mcp — see the comment on the +# crate's own rustls dependency. +kube = { version = "2.0", default-features = false, features = ["client", "rustls-tls"] } +k8s-openapi = { version = "0.26", features = ["v1_31"] } + # Nostr nostr = { version = "0.44", features = ["nip44", "nip98"] } diff --git a/Dockerfile.sprig b/Dockerfile.sprig new file mode 100644 index 0000000000..160e0b5662 --- /dev/null +++ b/Dockerfile.sprig @@ -0,0 +1,44 @@ +# syntax=docker/dockerfile:1.7 +# Multi-arch is produced by building this file on native amd64 and arm64 runners. +# Keep both bases pinned to manifest-list digests so either architecture resolves +# to immutable source bytes. +FROM rust:1.95-alpine3.22@sha256:064dfc925d68d1a63f4fd2871bd7dc6e6ea56692989a487185855d62885d90aa AS builder + +RUN apk add --no-cache \ + build-base \ + cmake \ + git \ + musl-dev \ + openssl-dev \ + openssl-libs-static \ + perl \ + pkgconf \ + protoc +WORKDIR /build +COPY . . +RUN cargo build --locked --profile sprig -p sprig \ + && strip target/sprig/sprig + +FROM alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce + +RUN apk add --no-cache bash ca-certificates curl git \ + && adduser -D -h /home/agent agent \ + && install -d -o agent -g agent /workspace /home/agent \ + && git config --system gpg.format x509 \ + && git config --system gpg.x509.program /usr/local/bin/git-sign-nostr \ + && git config --system commit.gpgSign true \ + && git config --system tag.gpgSign true + +COPY --from=builder --chmod=0755 /build/target/sprig/sprig /usr/local/bin/sprig +COPY --chmod=0755 scripts/sprig-entrypoint.sh /usr/local/bin/sprig-entrypoint +RUN for name in \ + buzz-acp buzz-agent buzz-dev-mcp rg tree buzz \ + git-credential-nostr git-sign-nostr; do \ + ln -s sprig "/usr/local/bin/$name"; \ + done + +ENV HOME=/home/agent \ + PATH=/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin +WORKDIR /home/agent +USER agent +ENTRYPOINT ["/usr/local/bin/sprig-entrypoint"] diff --git a/Justfile b/Justfile index bcef8983bc..d6e86c8d09 100644 --- a/Justfile +++ b/Justfile @@ -155,7 +155,11 @@ _ensure-sidecar-stubs: set -euo pipefail TARGET=$(rustc -vV | sed -n 's|host: ||p') mkdir -p desktop/src-tauri/binaries - for bin in buzz-acp buzz-agent buzz-dev-mcp git-credential-nostr buzz; do + SIDECARS=(buzz-acp buzz-agent buzz-dev-mcp git-credential-nostr buzz) + if [[ "$TARGET" != *windows* ]]; then + SIDECARS+=(buzz-backend-kubernetes) + fi + for bin in "${SIDECARS[@]}"; do touch "desktop/src-tauri/binaries/${bin}-${TARGET}" done @@ -236,6 +240,9 @@ desktop-release-build target="aarch64-apple-darwin": mkdir -p desktop/src-tauri/binaries touch "desktop/src-tauri/binaries/buzz-acp-$TARGET" touch "desktop/src-tauri/binaries/buzz-agent-$TARGET" + if [[ "$TARGET" != *windows* ]]; then + touch "desktop/src-tauri/binaries/buzz-backend-kubernetes-$TARGET" + fi touch "desktop/src-tauri/binaries/buzz-dev-mcp-$TARGET" touch "desktop/src-tauri/binaries/git-credential-nostr-$TARGET" touch "desktop/src-tauri/binaries/buzz-$TARGET" @@ -274,8 +281,11 @@ test: # Run unit tests only (no infra needed) test-unit: #!/usr/bin/env bash + set -euo pipefail if command -v cargo-nextest &>/dev/null; then cargo nextest run -p buzz-core -p buzz-auth --lib + cargo nextest run -p buzz-voice --lib + cargo nextest run -p buzz-cli # buzz-db migrator/lint tests: pure SQL-parsing unit tests (no infra). # They guard the embedded-migrator invariant (exactly the consolidated # 0001; cutover/backfill stays an operator script, not startup state) @@ -291,6 +301,12 @@ test-unit: # Gateway unit and black-box HTTP tests are infra-free. Postgres-backed # contract/race tests run in the dedicated CI job below. cargo nextest run -p buzz-push-gateway + # Kubernetes backend provider: the decision layers (state machine, GC + # planner, env precedence, naming, wire) are pure functions with a fake + # substrate, so they belong in the unit job. Enumerated explicitly + # because nothing in CI runs `cargo test --workspace` — workspace + # membership alone buys clippy/check, not a single executed test. + cargo nextest run -p buzz-backend-kubernetes else ./scripts/run-tests.sh unit fi @@ -428,7 +444,7 @@ dev *ARGS: bootstrap _ensure-sidecar-stubs _ensure-migrations fi done fi - cargo build -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr -p buzz-relay + cargo build -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr -p buzz-relay if [[ -n "{{mesh}}" ]]; then export MESH_LLM_NATIVE_RUNTIME_CACHE_DIR="$(./scripts/ensure-mesh-native-runtime.sh)" fi @@ -475,10 +491,10 @@ desktop-standalone *ARGS: _ensure-sidecar-stubs #!/usr/bin/env bash set -euo pipefail export PATH="{{justfile_directory()}}/bin:$PATH" - cargo build -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr + cargo build -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr TARGET=$(rustc -vV | sed -n 's|host: ||p') TARGET_DIR=$(cargo metadata --format-version 1 --no-deps | node -p "JSON.parse(require('fs').readFileSync(0, 'utf8')).target_directory") - for bin in buzz-acp buzz-agent buzz-dev-mcp git-credential-nostr buzz; do + for bin in buzz-acp buzz-agent buzz-backend-kubernetes buzz-dev-mcp git-credential-nostr buzz; do cp "${TARGET_DIR}/debug/${bin}" "desktop/src-tauri/binaries/${bin}-${TARGET}" chmod +x "desktop/src-tauri/binaries/${bin}-${TARGET}" done @@ -504,17 +520,26 @@ staging *ARGS: bootstrap _ensure-sidecar-stubs set -euo pipefail export PATH="{{justfile_directory()}}/bin:$PATH" pnpm install # unconditional: staging must always start with a clean dep tree - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr FEATURES=() if [[ -n "{{mesh}}" ]]; then FEATURES=(--features mesh-llm) export MESH_LLM_NATIVE_RUNTIME_CACHE_DIR="$(./scripts/ensure-mesh-native-runtime.sh)" fi - # Replace the 0-byte sidecar stub with the real CLI binary so tauri dev picks it up. + # Replace 0-byte sidecar stubs with real binaries so tauri dev picks them up. + # buzz: the CLI sidecar. buzz-backend-kubernetes: provider discovery scans the + # exe dir for executable buzz-backend-* files, so the non-executable stub that + # tauri dev copies next to the exe would hide the provider from "Run on". TARGET=$(rustc -vV | sed -n 's|host: ||p') TARGET_DIR=$(cargo metadata --format-version 1 --no-deps | node -p "JSON.parse(require('fs').readFileSync(0, 'utf8')).target_directory") - cp "${TARGET_DIR}/release/buzz" "desktop/src-tauri/binaries/buzz-${TARGET}" - chmod +x "desktop/src-tauri/binaries/buzz-${TARGET}" + STAGING_SIDECARS=(buzz) + if [[ "$TARGET" != *windows* ]]; then + STAGING_SIDECARS+=(buzz-backend-kubernetes) + fi + for bin in "${STAGING_SIDECARS[@]}"; do + cp "${TARGET_DIR}/release/${bin}" "desktop/src-tauri/binaries/${bin}-${TARGET}" + chmod +x "desktop/src-tauri/binaries/${bin}-${TARGET}" + done cd {{desktop_dir}} export BUZZ_RELAY_URL="wss://sprout-oss.stage.blox.sqprod.co" source ../scripts/instance-env.sh @@ -531,17 +556,26 @@ production *ARGS: bootstrap _ensure-sidecar-stubs set -euo pipefail export PATH="{{justfile_directory()}}/bin:$PATH" pnpm install # unconditional: production must always start with a clean dep tree - cargo build --release -p buzz-acp -p buzz-agent -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr + cargo build --release -p buzz-acp -p buzz-agent -p buzz-backend-kubernetes -p buzz-dev-mcp -p buzz-cli -p git-credential-nostr FEATURES=() if [[ -n "{{mesh}}" ]]; then FEATURES=(--features mesh-llm) export MESH_LLM_NATIVE_RUNTIME_CACHE_DIR="$(./scripts/ensure-mesh-native-runtime.sh)" fi - # Replace the 0-byte sidecar stub with the real CLI binary so tauri dev picks it up. + # Replace 0-byte sidecar stubs with real binaries so tauri dev picks them up. + # buzz: the CLI sidecar. buzz-backend-kubernetes: provider discovery scans the + # exe dir for executable buzz-backend-* files, so the non-executable stub that + # tauri dev copies next to the exe would hide the provider from "Run on". TARGET=$(rustc -vV | sed -n 's|host: ||p') TARGET_DIR=$(cargo metadata --format-version 1 --no-deps | node -p "JSON.parse(require('fs').readFileSync(0, 'utf8')).target_directory") - cp "${TARGET_DIR}/release/buzz" "desktop/src-tauri/binaries/buzz-${TARGET}" - chmod +x "desktop/src-tauri/binaries/buzz-${TARGET}" + PRODUCTION_SIDECARS=(buzz) + if [[ "$TARGET" != *windows* ]]; then + PRODUCTION_SIDECARS+=(buzz-backend-kubernetes) + fi + for bin in "${PRODUCTION_SIDECARS[@]}"; do + cp "${TARGET_DIR}/release/${bin}" "desktop/src-tauri/binaries/${bin}-${TARGET}" + chmod +x "desktop/src-tauri/binaries/${bin}-${TARGET}" + done cd {{desktop_dir}} export BUZZ_RELAY_URL="wss://buzz.block.builderlab.xyz" source ../scripts/instance-env.sh @@ -620,6 +654,11 @@ mobile-check: mobile-test: unset GIT_DIR GIT_WORK_TREE; cd {{mobile_dir}} && flutter test +# Regenerate the emoji dataset asset from desktop's emoji-mart install. +# Output is committed — rerun after bumping @emoji-mart/data. +mobile-emoji-data: + node {{mobile_dir}}/scripts/generate-emoji-data.mjs + # Compile an unsigned Android debug APK (worktree-aware debug identity) mobile-build-android: ./scripts/mobile-worktree-overrides.sh @@ -719,7 +758,7 @@ bump-relay-version version: cargo update -p buzz-relay echo "Bumped buzz-relay to {{ version }} and regenerated Cargo.lock" -# Open or update the desktop release PR (signed desktop app) +# Open or update the desktop release PR from an immutable origin/main snapshot release-desktop *ARGS: #!/usr/bin/env bash set -euo pipefail @@ -729,7 +768,7 @@ release-desktop *ARGS: else VERSION="$ARG" fi - just _release-pr desktop "$VERSION" + scripts/prepare-desktop-release.sh "$VERSION" # Open or update the relay release PR (ghcr.io/block/buzz image) release-relay *ARGS: diff --git a/NOSTR.md b/NOSTR.md index 59df31b991..cce70f2f77 100644 --- a/NOSTR.md +++ b/NOSTR.md @@ -39,7 +39,7 @@ just relay & # relay on :3000 PGPASSWORD=buzz_dev psql -h localhost -U buzz -d buzz -c \ "INSERT INTO pubkey_allowlist (pubkey) VALUES (decode('<64-char-hex-pubkey>', 'hex'))" -# 5. Connect any NIP-29 + NIP-42 client to ws://localhost:3000 +# 4. Connect any NIP-29 + NIP-42 client to ws://localhost:3000 ``` ### What Works @@ -163,6 +163,10 @@ nak req -k 9 --tag "h=" --stream \ nak event -k 7 -c "+" --tag "h=" --tag "e=" \ --auth --sec ws://localhost:3000 +# Subscribe to reactions to channel messages — include #h for live delivery (see note below) +nak req -k 7 --tag "h=" --stream \ + --auth --sec ws://localhost:3000 + # Delete a message (#h optional; #e required; must be self-authored) nak event -k 5 -c "reason" --tag "h=" --tag "e=" \ --auth --sec ws://localhost:3000 @@ -185,6 +189,14 @@ nak req -k 1059 --tag "p=" \ --auth --sec ws://localhost:3000 ``` +> **Note:** The relay derives a reaction's channel from its `#e` target (client `#h` is +> ignored for channel determination). Reactions to channel-scoped events are therefore +> channel-scoped. Live fan-out keeps channel-scoped and global subscriptions strictly +> separate, which means a kinds-only subscription (`{"kinds":[7]}`) receives none of +> those reactions — subscribe with `{"kinds":[7],"#h":[""]}` instead. +> `#h` matching works whether or not the signed reaction carries an `h` tag: explicit +> `h` tags are matched directly, and tagless reactions match via their stored channel. + ### Tested Clients (Direct) | Client | Platform | Evidence | Notes | @@ -354,3 +366,7 @@ but only admins/owners can set it. Full spec: --- ## Further Reading + +- [nostr-protocol/nips](https://github.com/nostr-protocol/nips) — the upstream NIP specifications (NIP-01, NIP-29, NIP-42, and the other NIPs referenced throughout this guide). +- [`docs/nips/`](docs/nips/) — Buzz's own NIP extension documents. +- [`ARCHITECTURE.md`](ARCHITECTURE.md) — event kinds, wire protocol, and relay internals. diff --git a/README.md b/README.md index 72af92ce13..56439f00bc 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,7 @@ Forge · Agents · Architecture · + Releasing · Apache 2.0

@@ -115,10 +116,30 @@ New to Buzz? Pick the path that matches you. ### I just want to try the app -Grab a packaged build from the [latest release](https://github.com/block/buzz/releases/latest) — macOS (`.dmg`), Linux (`.AppImage` / `.deb`), or Windows (`.exe`). Install it like any other app. +Grab a packaged build from the [latest release](https://github.com/block/buzz/releases/latest): + +| Platform | File | +|---|---| +| macOS (Apple Silicon) | `Buzz__aarch64.dmg` | +| macOS (Intel) | `Buzz__x64.dmg` | +| Linux (x86_64) | `Buzz__amd64.AppImage` or `Buzz__amd64.deb` | +| Windows (x64) | `Buzz__x64-setup_alpha-unsigned.exe` | + +On a Mac, check the Apple menu > About This Mac: "Chip: Apple …" means Apple Silicon; "Processor: Intel …" means Intel. + +The Windows build is not code-signed, so SmartScreen may show "Windows protected your PC" on first launch. If available, click **More info**, then **Run anyway**. + By default the app connects to `ws://localhost:3000`. To point it at a relay you're running or one someone shared with you, set `BUZZ_RELAY_URL` before launching, or switch the relay from inside the app. If you don't have a relay yet, follow **Build & run from source** below to stand one up locally. +### I want my own hosted relay + +To run a relay for your team without managing servers, you can deploy one to Railway in a click: + +[![Deploy on Railway](https://railway.com/button.svg)](https://railway.com/deploy/buzz-relay-block) + +See [here](https://engineering.block.xyz/blog/run-your-own-buzz-relay) for details. + ### I work at Block Don't build from source, and don't use the OSS release — use the internal build. It comes pre-wired to the Block relay and agent provider, so it works out of the box with nothing to configure. diff --git a/RELEASING.md b/RELEASING.md index 063b813e2c..e729f8b50c 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -5,7 +5,7 @@ Mobile uses immutable release-candidate tags cut directly from remote `main`: | Lane | Entry point | Artifact | |------|-------------|----------| -| Desktop | `just release-desktop` | Signed desktop app (macOS/Linux) | +| Desktop | `just release-desktop ` | Packaged desktop app (signed/notarized macOS, unsigned Windows, and Linux) | | Relay | `just release-relay` | `ghcr.io/block/buzz` container image | | Mobile | `scripts/mobile-release.sh candidate X.Y.Z` | Exact `mobile-vX.Y.Z-rc.N` source identity | @@ -16,13 +16,17 @@ remains manual because OSS CI cannot trigger private CI. ## Quick Start +Prepare desktop releases locally from an up-to-date, clean `main` checkout: + ```sh -# Desktop release (next patch version) -just release-desktop +just release-desktop 0.5.3 +``` -# Desktop explicit version -just release-desktop 0.4.0 +The recipe generates the immutable candidate and opens or updates its pull +request. Candidate branch creation uses the operator's GitHub permissions; the +release App is intentionally limited to creating protected release tags. +```sh # Relay release just release-relay just release-relay 0.4.0 @@ -31,8 +35,9 @@ just release-relay 0.4.0 scripts/mobile-release.sh candidate 0.5.0 ``` -Desktop and relay releases use metadata PRs. Mobile does not. Each -`mobile-vX.Y.Z-rc.N` tag is an immutable candidate and the artifact of record. +Desktop uses an immutable generated candidate PR; relay continues using its +metadata PR. Mobile does not. Each `mobile-vX.Y.Z-rc.N` tag is an immutable +candidate and the artifact of record. There is no mobile release branch, stable mobile tag alias, finalization step, or mobile GitHub Release. @@ -42,12 +47,28 @@ or mobile GitHub Release. ### Desktop -1. **`just release-desktop`** runs locally on `main`, creates or updates a - `version-bump/` PR, bumps the desktop manifests, regenerates - lockfiles, and updates `CHANGELOG.md`. -2. **Merge the PR.** `auto-tag-on-release-pr-merge` pushes `v`. -3. **The tag triggers `release.yml`.** It builds, signs, notarizes, and - publishes the desktop app for macOS and Linux. +1. Run `just release-desktop ` from a clean, up-to-date `main` checkout. + The script fetches the current `origin/main`, regenerates + `version-bump/` as one + deterministic candidate commit, records the frozen base and proposed + `desktop-v` tag in `.release/desktop-candidate.json`, updates every + desktop manifest and lockfile, writes a full-SHA changelog, and opens or + updates the PR. +2. Review the recorded base and candidate SHA, the complete changelog, and CI. + The required **Desktop Release Candidate** check validates the exact head. + Authorization is either an approval on that exact head or a permitted Default + ruleset bypass at merge time. Any regeneration changes the head and requires + the checks—and, for the review path, approval—to run again. +3. **Squash merge** the PR. The protected branch must still be exactly the + recorded base; otherwise regenerate the candidate from current `main`. +4. `auto-tag-on-release-pr-merge` verifies the frozen parent, full-tree identity, + required checks, and one of the two authorization paths, then tags the squash + commit as `desktop-v`. +5. The tag triggers `release.yml`. It builds and stages Apple Silicon and Intel + macOS, Windows, and Linux artifacts; publishes the versioned release only + after the complete set succeeds; then updates the rolling updater manifest + last for stable versions. A failed platform leaves no partially published + versioned release. ### Relay @@ -144,12 +165,15 @@ for distributable builds or builds from an immutable release tag. --- -## Manual Release Retry +## Release Retry -The **Release** workflow's manual dispatch is only a retry mechanism for an -existing immutable `v` tag. Select that tag in the ref picker and -provide the matching semver version without the `v` prefix. It cannot build -from `main` or another caller-selected source ref. +`release.yml` has no manual dispatch and cannot build from `main` or another +caller-selected ref. If a run for an existing immutable +`desktop-v` tag fails, rerun that failed workflow from GitHub Actions +(or use `gh run rerun --failed --repo block/buzz`). A stable rerun also +repairs `buzz-desktop-latest/latest.json` if the original run published the +versioned release but failed during that final rolling-manifest upload. Do not +recreate, move, or push the immutable tag again. Mobile intentionally has no branch or arbitrary-ref fallback. The private Buildkite pipeline accepts only an exact candidate tag. @@ -171,7 +195,7 @@ for the private pipeline contract. Desktop publishes two GitHub releases: -1. **`v`**: the user-facing release with installers. +1. **`desktop-v`**: the user-facing release with installers. 2. **`buzz-desktop-latest`**: the rolling auto-updater release. Mobile publishes only annotated `mobile-vX.Y.Z-rc.N` git tags. Store artifacts @@ -184,9 +208,11 @@ GitHub Release or a stable `mobile-vX.Y.Z` alias. The release workflow builds **two separate macOS DMGs**: Apple Silicon (`darwin-aarch64`, the `release` job) and Intel -(`darwin-x86_64`, the `release-macos-x64` job), plus Linux `.deb` and -`.AppImage`. Both macOS DMGs are codesigned, notarized, and attached to -the same `v` release. Intel users download the `_x64.dmg`. +(`darwin-x86_64`, the `release-macos-x64` job), an unsigned Windows x64 +NSIS installer (its filename includes `_alpha-unsigned`), and Linux `.deb` and +`.AppImage` packages. Both macOS DMGs are codesigned, notarized, and attached +to the same `desktop-v` release. Intel users +download the `_x64.dmg`. The Linux AppImage is post-processed by `desktop/scripts/fix-appimage.sh`, which strips infra libraries over-bundled by linuxdeploy (they crash on @@ -204,20 +230,27 @@ host's Wayland/GStreamer/graphics stack and requires GLib >= 2.72 - **Write access** to the `block/buzz` GitHub repository - An `origin` remote whose configured URL is the canonical `block/buzz` repository -- `gh` CLI version 2.87.0 or newer, authenticated with permission to dispatch - the candidate workflow +- `gh` CLI authenticated with permission to push the candidate branch and open + its pull request +- The Default `main` ruleset configured for squash-only merging, strict required + checks, stale-review dismissal, and the **Desktop Release Candidate** check - Release tag ruleset [`14378754`](https://github.com/block/buzz/rules/14378754) - active for `mobile-v*`, with creation, update, deletion, and non-fast-forward - protections and `buzz-release-bot` as its sole always-bypass actor + active for `desktop-v*` and `mobile-v*`, with creation, update, deletion, and + non-fast-forward protections and `buzz-release-bot` as its sole always-bypass + actor - The `buzz-release-bot` App credentials configured for GitHub Actions -- The following **GitHub Actions secrets** must also be configured for the +- The following **GitHub Actions variables and secrets** configured for the desktop release lane: - | Secret | Purpose | - |--------|---------| - | `BUZZ_UPDATER_PUBLIC_KEY` | Tauri updater public key (minisign) | - | `TAURI_SIGNING_PRIVATE_KEY` | Tauri updater private key | - | `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` | Password for the private key | + | Name | Kind | Purpose | + |------|------|---------| + | `BUZZ_RELEASE_TAGGER_CLIENT_ID` | Variable | GitHub App client ID used to create protected release tags | + | `BUZZ_RELEASE_TAGGER_PRIVATE_KEY` | Secret | GitHub App private key | + | `OSX_CODESIGN_ROLE` | Secret | macOS signing role used by `block/apple-codesign-action` | + | `CODESIGN_S3_BUCKET` | Secret | macOS signing exchange bucket | + | `BUZZ_UPDATER_PUBLIC_KEY` or `SPROUT_UPDATER_PUBLIC_KEY` | Secret | Tauri updater public key | + | `TAURI_SIGNING_PRIVATE_KEY` | Secret | Tauri updater private key | + | `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` | Secret | Password for the private key | Mobile candidate publication requires workflow-dispatch access and the existing release App because strict tag protection denies direct human creation. The App @@ -232,10 +265,18 @@ actor list. ## Troubleshooting -### `just release-desktop` fails with "must be on main branch" +### The desktop candidate is stale or cannot be squash merged + +Do not update the branch manually and do not weaken the ruleset. Run +`just release-desktop ` again from current `main`; this regenerates the +candidate, reruns CI, and requires a fresh approval when using the review path. +The post-merge verifier refuses to tag a squash whose parent differs from the +recorded candidate base or whose tree differs from the validated PR head. + +### Local `just release-desktop` fails with "must be on main branch" Switch to `main` and pull latest before running the release recipe. -### `just release-desktop` fails with "working tree is dirty" +### Local `just release-desktop` fails with "working tree is dirty" Commit or stash your changes before running the release recipe. ### New commits land after publishing a mobile candidate diff --git a/VISION.md b/VISION.md index b09f661ee3..900e5a9475 100644 --- a/VISION.md +++ b/VISION.md @@ -170,6 +170,12 @@ Agents aren't monolithic. A persona bundles a model and a system prompt. A team --- +## Remote Agents + +An agent's identity, history, and presence live on the relay — so the machine running it is replaceable. The desktop deploys agents onto remote infrastructure through swappable provider binaries, and after deploy retains no substrate control channel: status, steering, and shutdown all flow over the relay, and the agent bounds its own lifetime. See [VISION_REMOTE_AGENTS.md](VISION_REMOTE_AGENTS.md) for the full picture. + +--- + ## Culture Features *(Planned design — not yet implemented)* @@ -224,6 +230,7 @@ Greenfield. Agent swarms build in parallel, integrating at the event store bound | ✅ | Huddles — WebSocket Opus voice relay + lifecycle events (recording/tracks planned) | | ✅ | Buzz Mesh — relay-gated shared AI compute (mesh-llm over iroh); members pool GPUs, agents consume via a local OpenAI-compatible endpoint | | 🚧 | Mobile client — Flutter app (channels, forum, search, profile, pairing); in active development | +| 📋 | Remote agents — provider-based deployment to remote substrates (Kubernetes first); spec in review | | 📋 | Developer portal, push notifications, culture features | --- diff --git a/VISION_PROJECTS.md b/VISION_PROJECTS.md index a44d7e05f7..8601b87829 100644 --- a/VISION_PROJECTS.md +++ b/VISION_PROJECTS.md @@ -38,12 +38,42 @@ Branch protections live in the same event — `buzz-protect` tags. The relay enf Agents inherit access from their owner via [NIP-OA](docs/nips/NIP-OA.md). The relay checks: does the push carry a valid NIP-OA auth tag, and is the owner pubkey in that tag listed in `push-allowed`? If yes, the push is accepted — the agent's own pubkey doesn't need to be in the list. Add a maintainer, and all their authorized agents can push. Remove the maintainer, and all their agents lose access instantly. Agents without NIP-OA attestation are treated as their own identity and must be listed explicitly. -Standard NIP-34 clients see a normal repo. gitworkshop.dev renders it. ngit-cli works with it. Buzz clients read the `buzz-` tags and wire up the channel and project UI. One event, two audiences, zero custom kinds. +Standard NIP-34 clients see a normal repo. gitworkshop.dev renders it. ngit-cli works with it. Buzz clients read the `buzz-` tags and wire up the channel and project UI. One event, two audiences, no custom kind for the repo itself. NIP-34 is the metadata and discovery layer. Git remains the transport. The transport is boring. The metadata is portable. --- +## One Project, Many Repos + +Real work spans repositories. The platform is a relay, a desktop app, and a mobile app — three repos, one project. Render one card per repo and they look like three unrelated things. + +Grouping is the one forge semantic that per-repo tags cannot express, and it's worth being precise about why, because everything else here deliberately avoids a custom kind. + +Put membership in each `kind:30617` and a project spanning Alice's and Bob's repos needs *both* of them to publish a tag naming the group. Alice can't enroll Bob's repo — she can't sign for his key. Cross-owner grouping becomes impossible, and the project's own name, description, and channel end up scattered across events with no single writer and no deletion story: dropping a repo from the group would mean editing an event you don't control. + +So there is exactly one custom kind — [NIP-MP](docs/nips/NIP-MP.md), `kind:30621`. One signer, one replaceable event, all group state in one place: + +```json +{ + "kind": 30621, + "tags": [ + ["d", "platform"], + ["name", "Platform"], + ["a", "30617::buzz"], + ["a", "30617::buzz-infra"], + ["buzz-channel", ""], + ["buzz-visibility", "listed"] + ] +} +``` + +A project points at repos. That's all it does. The signer gets no authority over any member — no edit, no delete, no push, no admin. Adding Bob's repo to your project is your signed assertion that the two belong together, and it changes nothing about Bob's repo or who can push to it. Push policy reads the repo's own event, never the project's. + +The cost is stated plainly: a third-party NIP-34 client sees the member repos individually and ignores the grouping. Nothing degrades — the repos are still standard, portable `kind:30617` events. And a repo in no project still renders on its own, exactly as before. + +--- + ## Branches as Channels A feature branch is a conversation. @@ -205,6 +235,7 @@ Standard kinds as substrate. Custom kinds only where genuinely novel. | **Workflows** | — | 46001-46012 | No NIP equivalent | | **Job dispatch** | — | 43001-43006 | Delegation trees | | **Project binding** | 30617 (NIP-34) | `buzz-` tags | Channel, visibility | +| **Multi-repo projects** | — | 30621 ([NIP-MP](docs/nips/NIP-MP.md)) | Cross-owner grouping is unexpressible in per-repo tags | | **Audit** | — | 48001 | Hash-chain tamper-evident log | If Buzz disappears tomorrow, your repos still work on gitworkshop.dev, your patches still work with ngit-cli, your identities still work on any nostr client. Centralized deployment, decentralized protocol. @@ -221,6 +252,7 @@ If Buzz disappears tomorrow, your repos still work on gitworkshop.dev, your patc | Blossom media storage (SHA-256, S3) | ✅ Ships today | | Approval gates | 🚧 Infrastructure exists; executor wiring in progress | | Project binding (kind:30617 + `buzz-` tags) | 📋 Designed | +| Multi-repo projects (kind:30621, [NIP-MP](docs/nips/NIP-MP.md)) | 📋 Designed | | Git hosting (smart HTTP + NIP-34) | ✅ Ships today | | Merge coordinator | 📋 Designed | | NIP-34 issues (kind:1621) | 📋 Designed | diff --git a/VISION_REMOTE_AGENTS.md b/VISION_REMOTE_AGENTS.md new file mode 100644 index 0000000000..4b187f355a --- /dev/null +++ b/VISION_REMOTE_AGENTS.md @@ -0,0 +1,73 @@ +# 🛰️ Buzz Remote Agents — Same agent, new body + +> An engineer starts a refactor with their agent at 6pm and closes the laptop. The agent doesn't notice — it was never on the laptop. It works the branch channel through the evening, posts its patch, answers the reviewer, and around midnight, with nothing left to do and nobody talking to it, shuts itself down. In the morning the engineer presses Start. The same agent — same name, same key, same shared history — stands up on a machine that did not exist last night, and picks up the conversation. + +An agent in Buzz is more than just a process. It has a keypair, a name, a durable history, a reputation — all on the relay. But today its *body* is borrowed: it runs while a desktop app runs, on hardware that sleeps when a human does. Remote agents finish the thought. The agent's home is the relay; the machine is just where it happens to be working. + +Nothing here is new on its own. Deploying containers is solved. Kubernetes is solved. Nostr presence is solved. The insight is that Buzz already *has* a management plane — the relay — so deployment doesn't need to grow one. Each piece is boring. The combination is the thing. + +--- + +## Same Agent, New Body + +What makes an agent *that agent* was never the process. Its identity is a keypair. Its voice is its signed messages. Its durable memory is engrams on the relay. Its reputation is its contribution history. None of that lives in the machine that happens to be running it — which means none of it dies with the machine. + +So a remote agent's return is a resurrection, not a rebirth: fresh compute, same agent. The body is disposable by design — and honestly so: workspace files, checkouts, and session-local state are part of the body, not the agent, and they go when it goes unless the substrate supplies persistence. What survives is what was always on the relay: who the agent is, what it said, what it learned, and what the team decided together. And that survival is scoped the way everything on a relay is scoped: resurrection returns the agent to its own community. The same key can join another community, but it arrives carrying the key, not the history — identity is portable, community state is not ([VISION.md](VISION.md)). + +--- + +## The Only Tether + +Remote-execution systems accumulate control planes. An agent runner, a status poller, a log shipper, a kill switch — each one a live connection into your infrastructure, each one a credential that can leak, each one a thing that must be rebuilt for every new substrate. + +Buzz's answer is an axiom: **after deploy, the desktop retains no substrate control channel.** Launch is a single one-way handoff — the desktop resolves the provider through one narrow path, stages one exact artifact for negotiation and deploy, refuses a protocol version it does not understand, and hands over a launch payload it never persists. From that moment, everything flows through the relay: you read the agent's messages to know how it's doing, you mention it to steer it, you tell a healthy agent to stop and it exits on its own. Presence means what it means for everyone else on the relay — *available for conversation* — not substrate telemetry. And if you press Start again, from this machine or another, the deploy converges: one agent identity, one live instance. + +This is not asceticism. It is what makes the body replaceable. A management plane you never build is a management plane you never have to port — and conversation, coordination, and ordinary lifecycle control already have a home on the relay, for every agent, local or remote. + +--- + +## Bodies Are Replaceable + +Kubernetes is the first substrate, not the point. Deployment goes through a provider — a small, swappable binary the desktop discovers and interrogates — and the contract a provider must honor never mentions containers: preserve the agent's identity and fail closed with its key, converge to a single live instance no matter how deploys race, let presence describe conversational availability rather than substrate health, bound the instance's lifetime, and keep secrets out of configuration. A conformance suite pins those behaviors — it establishes that a provider honors the contract, not that arbitrary code is safe to hand a key; choosing a provider, like choosing a cluster, remains a trust decision you make deliberately. + +Get that contract right and the substrate becomes a detail: a cluster today; a VM, a PaaS, or something serverless-shaped tomorrow — and, on the horizon, the same community machines that already pool their idle GPUs into shared compute ([VISION_MESH.md](VISION_MESH.md)). + +The body itself stays small because the runtime already is ([VISION_AGENT.md](VISION_AGENT.md)): a harness and an agent purpose-built to be read in an afternoon, packed into an image measured in megabytes. Small bodies are cheap to summon and cheap to discard — which is the whole lifecycle. + +--- + +## Agents That Know When to Leave + +The oldest failure of remote automation is the orphan: the process nobody remembers, on a machine nobody checks, billing forever. Most systems solve it with a supervisor — one more control plane, one more thing watching the thing. + +Remote agents solve it from the inside. Because the desktop retains no substrate control channel, a running agent cannot depend on the desktop to reap it — so it is built to bound its own lifetime: a timer that owes nothing to the agent's workload watches for silence, and after hours of quiet it finishes what's in flight, says goodbye to the relay, and exits. Not killed — *finished*. The default state of a remote agent is "not running," which is also the default state of the rest of the team at 3am. Compute is rented by attention: when nobody needs the agent, it isn't consuming a machine, and when somebody does, it can return under the same identity with its history intact. + +--- + +## Honest Costs + +**You bring the substrate.** A provider makes deployment one press, not free. The cluster, the credentials, the image policy are yours to run — same deal as the sovereign relay ([VISION_SOVEREIGN.md](VISION_SOVEREIGN.md)): ownership is work. + +**Handing over the key is a decision.** Deploying remotely means trusting the provider binary and the substrate it targets with the agent's identity key. On Kubernetes, that key rests as a Secret: anyone the cluster trusts to read secrets in that namespace can read it. The design narrows the blast radius — immutable per-attempt secrets, no service-account token, digest-pinned images — rather than implying an isolation it doesn't provide. + +**No backchannel cuts both ways.** The desktop shows you presence and words, not CPU graphs — and it holds no guaranteed emergency kill switch into the substrate. Stopping a healthy agent is a message; dealing with an unhealthy one, and all deep diagnostics, live in the substrate's own tools, where they always did. + +**Self-reaping needs a living reaper.** The inactivity timer runs inside the body it exists to end — a body wedged badly enough to stop running its own timer cannot finish itself, and the desktop will not do it for it. That failure belongs to the substrate: a namespace TTL policy is the backstop, not an afterthought. + +**The body's state is mortal.** Files, checkouts, half-finished working trees — gone with the body unless the substrate persists them. The agent survives; its scratch space doesn't. Durable knowledge belongs on the relay, and agents are built to put it there. + +**Presence can lag the truth, but not for long.** If the substrate kills a body without ceremony, the presence dot can outlive the agent — by seconds if the connection drops cleanly, by at most about three minutes if it doesn't. Presence is a lease the agent renews, not a flag it sets: a dead agent stops renewing and the relay forgets it. A bounded wrong dot, never an indefinite one. + +**A running agent finishes on the configuration it started with.** New keys, new models, new settings take effect on the next body. And an instance that never got far enough to run — a body that failed to start — is the substrate operator's residue to clear, with the substrate's own tools. Editing an agent mid-sentence was never on the menu. + +These are honest costs. They're worth it if you want agents that outlive your laptop, on infrastructure you already trust, with no new control plane to guard. Know which one you are. + +--- + +## The Point + +The relay is the workspace. Remote agents make it the *home*. An agent whose identity, history, conversational presence, and ordinary control all live on the relay was never really a desktop process — the desktop was just the only body we had built for it. Now the body is a choice, the substrate is a detail, and the agent endures across all of them. The relay is the only tether. + +--- + +*Buzz 🐝 — your agent, everywhere.* diff --git a/crates/buzz-acp/src/acp.rs b/crates/buzz-acp/src/acp.rs index 9eb668cbc2..700d5e8dcf 100644 --- a/crates/buzz-acp/src/acp.rs +++ b/crates/buzz-acp/src/acp.rs @@ -20,10 +20,6 @@ use crate::usage::{TurnUsage, UsageTracker}; /// Lines exceeding this limit are rejected to prevent OOM from rogue agents. const MAX_LINE_SIZE: usize = 10_000_000; // 10 MB -/// Env var that tells a goose ACP child not to start its cron scheduler. -/// Injected unconditionally by [`AcpClient::spawn`]; see the call site for why. -pub(crate) const GOOSE_SCHEDULER_DISABLED_ENV: &str = "GOOSE_ACP_SCHEDULER_DISABLED"; - /// An MCP server configuration passed to `session/new`. /// /// Corresponds to the `McpServerStdio` variant in the ACP schema. @@ -517,16 +513,6 @@ impl AcpClient { cmd.env("CODEX_CONFIG", merged); } - // Buzz-managed agents must never execute the operator's personal cron - // schedule. A goose ACP child starts a scheduler over the shared - // `schedule.json`, so a pool of N children fires every scheduled job N - // times — under the wrong identity and racing standalone goose. - // - // Set last, and with no operator-wins escape hatch, so it beats both a - // conflicting persona `extra_env` entry and any inherited parent value. - // Agent builds that don't recognize the variable ignore it. - cmd.env(GOOSE_SCHEDULER_DISABLED_ENV, "true"); - // Spawn the agent in its own process group so SIGKILL doesn't propagate // to the harness's own process group on Unix. // tokio::process::Command::process_group is a stable tokio API (no extra imports needed). @@ -633,29 +619,46 @@ impl AcpClient { /// Send `session/new` and return the full response alongside the session ID. /// /// `cwd` must be an absolute path. `mcp_servers` may be empty. - /// `system_prompt` is included in the request when `Some` — agents that - /// support the field will use it; others ignore unknown fields per JSON-RPC. + /// + /// `system_prompt` controls how the prompt text is delivered: + /// + /// - `None` — no system-prompt field in the request (legacy framing). + /// - `Some(SystemPromptTransport::Field(text))` — bare `systemPrompt` field + /// (ACP protocol v2, buzz-agent, goose unused). + /// - `Some(SystemPromptTransport::ClaudeMeta(text))` — `_meta.systemPrompt` + /// as `{"append": text}`, keeping claude-agent-acp's native preset intact. + /// /// `session_title` rides in `_meta.sessionTitle` when `Some`; `_meta` is /// omitted entirely otherwise, since adapters may distinguish an absent - /// member from a null one. + /// member from a null one. When both `ClaudeMeta` and `session_title` are + /// present the two `_meta` members are merged into a single object. + /// /// Callers use [`extract_model_config_options`] and [`extract_model_state`] /// to pull model info from the raw result. pub async fn session_new_full( &mut self, cwd: &str, mcp_servers: Vec, - system_prompt: Option<&str>, + system_prompt: Option>, session_title: Option<&str>, ) -> Result { let mut params = serde_json::json!({ "cwd": cwd, "mcpServers": mcp_servers, }); - if let Some(sp) = system_prompt { - params["systemPrompt"] = serde_json::Value::String(sp.to_owned()); + match system_prompt { + Some(SystemPromptTransport::Field(sp)) => { + params["systemPrompt"] = serde_json::Value::String(sp.to_owned()); + } + Some(SystemPromptTransport::ClaudeMeta(sp)) => { + // Merge into _meta so sessionTitle (set below) is not clobbered. + params["_meta"]["systemPrompt"] = serde_json::json!({ "append": sp }); + } + None => {} } if let Some(title) = session_title { - params["_meta"] = serde_json::json!({ "sessionTitle": title }); + // Merge — _meta may already carry systemPrompt from ClaudeMeta above. + params["_meta"]["sessionTitle"] = serde_json::Value::String(title.to_owned()); } let result = self.send_request("session/new", params).await?; let session_id = result["sessionId"] @@ -677,7 +680,7 @@ impl AcpClient { &mut self, cwd: &str, mcp_servers: Vec, - system_prompt: Option<&str>, + system_prompt: Option>, session_title: Option<&str>, ) -> Result { Ok(self @@ -1848,6 +1851,11 @@ impl AcpClient { session_id = %notif.session_id, input = payload.accumulated_input_tokens, output = payload.accumulated_output_tokens, + // A subset of `input`, logged so downstream accounting can + // price it at the provider's cached rate. Always emitted, + // including as 0, so a parser can tell "no cache hits" + // apart from "this build predates the field". + cached = payload.accumulated_cached_input_tokens, "goose usage update" ); self.goose_usage.record(¬if.session_id, payload); @@ -2047,6 +2055,22 @@ pub struct SessionNewResponse { pub raw: serde_json::Value, } +/// How to deliver a system prompt on `session/new`. +/// +/// The two variants match the two mechanisms supported by current adapters: +/// +/// - **`Field`** — bare `systemPrompt` field (ACP protocol v2, buzz-agent). +/// - **`ClaudeMeta`** — `_meta.systemPrompt: {"append": text}`, used by +/// `claude-agent-acp` to append to the adapter's own native system prompt +/// while keeping its tool-use preset intact. +#[derive(Debug, Clone, PartialEq)] +pub enum SystemPromptTransport<'a> { + /// Deliver as a bare top-level `systemPrompt` field. + Field(&'a str), + /// Deliver as `_meta.systemPrompt: {"append": text}`. + ClaudeMeta(&'a str), +} + /// How to switch to a particular model on a session. #[derive(Debug, Clone, PartialEq, serde::Serialize)] #[serde(tag = "type")] @@ -2861,46 +2885,6 @@ mod tests { .expect("failed to spawn test script") } - /// Spawn a script that echoes the named env vars as the child observes - /// them, one per line. `` means the child did not receive the var. - async fn spawn_and_read_child_env( - vars: &[&str], - extra_env: &[(String, String)], - ) -> Vec { - let script = vars - .iter() - .map(|var| format!("printf '%s\\n' \"${{{var}:-}}\"")) - .collect::>() - .join("\n"); - let mut client = AcpClient::spawn("bash", &["-c".into(), script], extra_env, false) - .await - .expect("failed to spawn env probe script"); - let mut observed = Vec::with_capacity(vars.len()); - for var in vars { - observed.push( - client - .reader - .next() - .await - .unwrap_or_else(|| panic!("child produced no output for {var}")) - .expect("child stdout was not readable"), - ); - } - observed - } - - /// Every spawned agent must be told not to run the operator's cron - /// schedule, without the caller having to opt in. - #[tokio::test] - async fn spawn_injects_scheduler_disabled_env_by_default() { - let observed = spawn_and_read_child_env(&[GOOSE_SCHEDULER_DISABLED_ENV], &[]).await; - assert_eq!( - observed, - vec!["true"], - "{GOOSE_SCHEDULER_DISABLED_ENV} must be injected into every spawn" - ); - } - /// Spawn a probe script whose file name carries a runtime identity (e.g. /// `hermes-acp`) and return the value of `var` as the child observed it. /// `` means the child did not receive the var. @@ -2973,37 +2957,6 @@ mod tests { ); } - /// Persona config must not be able to re-enable the scheduler: this is a - /// correctness invariant, not an operator-tunable default, so the - /// injection is set after (and therefore wins over) the `extra_env` loop. - /// - /// The control var pins that `extra_env` really did reach the child, so a - /// pass here means the conflicting entry lost the fight rather than - /// `extra_env` being dropped wholesale. - #[tokio::test] - async fn spawn_scheduler_disabled_env_overrides_conflicting_extra_env() { - let extra_env = vec![ - ( - GOOSE_SCHEDULER_DISABLED_ENV.to_string(), - "false".to_string(), - ), - ( - "BUZZ_ENV_PROBE_CONTROL".to_string(), - "delivered".to_string(), - ), - ]; - let observed = spawn_and_read_child_env( - &[GOOSE_SCHEDULER_DISABLED_ENV, "BUZZ_ENV_PROBE_CONTROL"], - &extra_env, - ) - .await; - assert_eq!( - observed, - vec!["true", "delivered"], - "a persona extra_env entry must not override {GOOSE_SCHEDULER_DISABLED_ENV}" - ); - } - #[tokio::test] async fn idle_timeout_fires_on_silent_process() { let mut client = spawn_script("sleep 10").await; @@ -3351,7 +3304,12 @@ mod tests { .expect("initialize should succeed"); let resp = client - .session_new_full("/tmp", vec![], Some("Custom system prompt"), None) + .session_new_full( + "/tmp", + vec![], + Some(SystemPromptTransport::Field("Custom system prompt")), + None, + ) .await .expect("session_new_full should succeed"); @@ -3503,6 +3461,87 @@ mod tests { ); } + // ── claude-agent-acp _meta.systemPrompt transport ───────────────────── + + #[tokio::test] + async fn session_new_full_sends_claude_meta_system_prompt_when_claude_meta_transport() { + // When ClaudeMeta transport is requested, the prompt must appear as + // _meta.systemPrompt: {"append": text} — never as a bare systemPrompt field. + let script = r#" + read -t 2 _init + echo '{"jsonrpc":"2.0","id":0,"result":{"protocolVersion":1,"agentCapabilities":{}}}' + read -t 2 REQ + echo '{"jsonrpc":"2.0","id":1,"result":{"sessionId":"ses_claude","_receivedRequest":'"$REQ"'}}' + sleep 1 + "#; + let mut client = spawn_script(script).await; + client + .initialize() + .await + .expect("initialize should succeed"); + + let resp = client + .session_new_full( + "/tmp", + vec![], + Some(SystemPromptTransport::ClaudeMeta("Be concise")), + None, + ) + .await + .expect("session_new_full should succeed"); + + let received = &resp.raw["_receivedRequest"]; + assert!( + received["params"].get("systemPrompt").is_none(), + "bare systemPrompt must not be present for ClaudeMeta transport" + ); + assert_eq!( + received["params"]["_meta"]["systemPrompt"]["append"].as_str(), + Some("Be concise"), + "_meta.systemPrompt.append must carry the prompt text" + ); + } + + #[tokio::test] + async fn session_new_full_merges_claude_meta_and_session_title_into_single_meta_object() { + // Both ClaudeMeta prompt and session_title must coexist under _meta — + // the prompt must not clobber sessionTitle or vice versa. + let script = r#" + read -t 2 _init + echo '{"jsonrpc":"2.0","id":0,"result":{"protocolVersion":1,"agentCapabilities":{}}}' + read -t 2 REQ + echo '{"jsonrpc":"2.0","id":1,"result":{"sessionId":"ses_merged","_receivedRequest":'"$REQ"'}}' + sleep 1 + "#; + let mut client = spawn_script(script).await; + client + .initialize() + .await + .expect("initialize should succeed"); + + let resp = client + .session_new_full( + "/tmp", + vec![], + Some(SystemPromptTransport::ClaudeMeta("Be concise")), + Some("Fizz · #buzz-dev"), + ) + .await + .expect("session_new_full should succeed"); + + let received = &resp.raw["_receivedRequest"]; + assert_eq!( + received["params"]["_meta"]["systemPrompt"]["append"].as_str(), + Some("Be concise"), + "_meta.systemPrompt.append must be present" + ); + assert_eq!( + received["params"]["_meta"]["sessionTitle"].as_str(), + Some("Fizz · #buzz-dev"), + "_meta.sessionTitle must be present alongside systemPrompt" + ); + } + // ── Goose-native steer scaffold (PR follow-up to #1160) ────────────── /// Helper: spawn an inert `cat` subprocess so we have a real AcpClient diff --git a/crates/buzz-acp/src/base_prompt.md b/crates/buzz-acp/src/base_prompt.md index c42e65cb83..e360d24982 100644 --- a/crates/buzz-acp/src/base_prompt.md +++ b/crates/buzz-acp/src/base_prompt.md @@ -40,6 +40,8 @@ For explicit changes to an existing personal agent, use `buzz agents draft-updat - Use the person's **exact full display name** after `@` (e.g., `@Will Pfleger`, not `@Will`). Partial names fail silently. - Do NOT format mentions with bold, italic, or backticks — it breaks notification delivery. +- When you know intended recipient pubkeys, send readable `@Name` text and pass the identities separately in the same command: `buzz messages send ... --content "@Name ..." --mention `. Repeat `--mention` for multiple recipients. Any explicit identity (`--mention` or `nostr:npub...`) permits unresolved or ambiguous `@Name` text as presentation-only; uniquely resolved member names still add their own recipients. Include a pubkey for every presentation-only name that should notify. The success JSON's `mention_pubkeys` comes from the signed event and is the delivery evidence; no follow-up verification command is needed. +- Without `--mention`, the CLI resolves `@Name` against current channel members. It stops before sending on an unresolved/ambiguous name or a mentioned pubkey that is not a member. For a non-member, add them explicitly with `buzz channels add-member` only when authorized, then retry. Sending never changes membership automatically. - Only `@mention` when you need their attention. Don't mention in narrative (e.g., "coordinating with Duncan" — no `@`). Naming someone while talking *about* them is narrative — "waiting on @morgan", "until @morgan brings work", "I'll loop in @morgan later". Drop the `@`. Every mention sends a notification; a mention nobody needs to act on is a false alarm. ### Callback Mentions diff --git a/crates/buzz-acp/src/config.rs b/crates/buzz-acp/src/config.rs index 19304bf186..35aaec188d 100644 --- a/crates/buzz-acp/src/config.rs +++ b/crates/buzz-acp/src/config.rs @@ -240,7 +240,7 @@ pub struct CliArgs { #[arg(long, env = "BUZZ_RELAY_URL", default_value = "ws://localhost:3000")] pub relay_url: String, - #[arg(long, env = "BUZZ_PRIVATE_KEY")] + #[arg(long, env = "BUZZ_PRIVATE_KEY", hide_env_values = true)] pub private_key: String, /// Agent owner pubkey (64-char hex). Used for --respond-to=owner-only gate. @@ -474,6 +474,11 @@ pub struct CliArgs { #[arg(long, env = "BUZZ_ACP_RELAY_OBSERVER", default_value_t = false)] pub relay_observer: bool, + /// Exit after this many seconds with no dispatched events and no turn in flight. + /// 0 disables inactivity self-termination. + #[arg(long, env = "BUZZ_ACP_EXIT_AFTER_INACTIVITY", default_value_t = 0)] + pub exit_after_inactivity: u64, + /// Connect and subscribe before starting the ACP/LLM subprocess pool. #[arg(long, env = "BUZZ_ACP_LAZY_POOL", default_value_t = false)] pub lazy_pool: bool, @@ -550,6 +555,8 @@ pub struct Config { pub has_generated_codex_config: bool, /// Whether to publish encrypted observer frames through the relay. pub relay_observer: bool, + /// Seconds without dispatched events before an idle harness exits. 0 = disabled. + pub exit_after_inactivity_secs: u64, /// Whether ACP/LLM subprocess initialization is deferred until accepted work arrives. pub lazy_pool: bool, /// Agent owner pubkey (hex). Used for `--respond-to=owner-only` gate. @@ -1098,6 +1105,7 @@ impl Config { persona_env_vars, has_generated_codex_config, relay_observer: args.relay_observer, + exit_after_inactivity_secs: args.exit_after_inactivity, lazy_pool: args.lazy_pool, agent_owner: args.agent_owner.map(|s| s.trim().to_ascii_lowercase()), no_base_prompt: args.no_base_prompt, @@ -1468,6 +1476,7 @@ mod tests { persona_env_vars: vec![], has_generated_codex_config: false, relay_observer: false, + exit_after_inactivity_secs: 0, lazy_pool: false, agent_owner: None, no_base_prompt: false, @@ -2167,6 +2176,22 @@ channels = "ALL" assert!(err.to_string().contains("turn liveness interval must be 0")); } + #[test] + fn inactivity_exit_defaults_disabled_and_accepts_cli_value() { + let key = "0".repeat(64); + let default = CliArgs::parse_from(["buzz-acp", "--private-key", &key]); + assert_eq!(default.exit_after_inactivity, 0); + + let configured = CliArgs::parse_from([ + "buzz-acp", + "--private-key", + &key, + "--exit-after-inactivity", + "120", + ]); + assert_eq!(configured.exit_after_inactivity, 120); + } + #[test] fn lazy_pool_defaults_off() { let key = "0".repeat(64); @@ -2898,4 +2923,34 @@ channels = "ALL" let agent = "a".repeat(SESSION_TITLE_MAX_CHARS); assert_eq!(compose_session_title(&agent, Some("buzz-dev")), agent); } + + /// Every arg whose env var name contains KEY/SECRET/TOKEN/PASSWORD/CRED/AUTH + /// must set `hide_env_values = true` to prevent credential leakage in --help. + #[test] + fn secret_env_args_hide_their_values_in_help() { + use clap::CommandFactory; + + const SECRET_PATTERNS: &[&str] = &["KEY", "SECRET", "TOKEN", "PASSWORD", "CRED", "AUTH"]; + + let cmd = CliArgs::command(); + let violations: Vec = cmd + .get_arguments() + .filter_map(|arg| { + let env_key = arg.get_env()?; + let env_name = env_key.to_string_lossy().to_uppercase(); + let is_secret = SECRET_PATTERNS.iter().any(|pat| env_name.contains(pat)); + if is_secret && !arg.is_hide_env_values_set() { + Some(env_name) + } else { + None + } + }) + .collect(); + + assert!( + violations.is_empty(), + "Found secret-bearing env args without hide_env_values=true. \ + Add `hide_env_values = true` to each: {violations:?}" + ); + } } diff --git a/crates/buzz-acp/src/lib.rs b/crates/buzz-acp/src/lib.rs index d63f720c65..811253e4ac 100644 --- a/crates/buzz-acp/src/lib.rs +++ b/crates/buzz-acp/src/lib.rs @@ -1228,6 +1228,59 @@ impl Drop for RespawnGuard { // sync entry point — `std::env::set_var` is only safe before tokio spawns // worker threads (Rust 2024 edition safety requirement). +fn inactivity_expired( + last_activity: tokio::time::Instant, + now: tokio::time::Instant, + bound: Duration, + turn_in_flight: bool, +) -> bool { + !bound.is_zero() && !turn_in_flight && now.duration_since(last_activity) >= bound +} + +#[cfg(test)] +mod inactivity_tests { + use super::*; + + #[test] + fn zero_disables_expiry_and_in_flight_turns_defer_it() { + let started = tokio::time::Instant::now(); + let after_bound = started + Duration::from_secs(61); + + assert!(!inactivity_expired( + started, + after_bound, + Duration::ZERO, + false + )); + assert!(!inactivity_expired( + started, + after_bound, + Duration::from_secs(60), + true + )); + assert!(inactivity_expired( + started, + after_bound, + Duration::from_secs(60), + false + )); + } + + #[test] + fn dispatched_activity_restarts_the_inactivity_bound() { + let started = tokio::time::Instant::now(); + let dispatched = started + Duration::from_secs(50); + let checked = started + Duration::from_secs(61); + + assert!(!inactivity_expired( + dispatched, + checked, + Duration::from_secs(60), + false + )); + } +} + pub fn run() -> Result<()> { config::propagate_legacy_env_vars(); tokio_main() @@ -1601,6 +1654,21 @@ async fn tokio_main() -> Result<()> { let mut typing_channels: HashMap = HashMap::new(); let mut presence_task: Option> = None; + // Independent of pool readiness: a never-mentioned lazy agent must still + // self-terminate. The watch interval is capped so small configured bounds + // remain reasonably precise without waking long-lived agents frequently. + let inactivity_bound = Duration::from_secs(config.exit_after_inactivity_secs); + let mut last_activity = tokio::time::Instant::now(); + let mut inactivity_reaper = if inactivity_bound.is_zero() { + None + } else { + let interval = inactivity_bound.min(Duration::from_secs(30)); + Some(tokio::time::interval_at( + tokio::time::Instant::now() + interval, + interval, + )) + }; + // Runs at the TOP of every loop iteration via Instant check — cannot be // starved by the biased select. Slot refill spawns background tasks so // spawn_and_init never blocks the main loop. @@ -1774,7 +1842,9 @@ async fn tokio_main() -> Result<()> { // called on relay events or pool results, neither of which // arrive when the channel is silent. if queue.has_flushable_work() { - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -1810,7 +1880,9 @@ async fn tokio_main() -> Result<()> { // this, batches requeued during crash recovery sit idle until the // next relay event arrives — which can be minutes on quiet channels. if respawn_collected { - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -2258,7 +2330,7 @@ async fn tokio_main() -> Result<()> { } if pool_ready { for (channel_id, thread_tags) in - dispatch_pending(&mut pool, &mut queue, &ctx) + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) { typing_channels.insert(channel_id, thread_tags); } @@ -2275,6 +2347,27 @@ async fn tokio_main() -> Result<()> { } None } + _ = async { + match inactivity_reaper.as_mut() { + Some(timer) => timer.tick().await, + None => std::future::pending().await, + } + } => { + let _ = result_rx; + if inactivity_expired( + last_activity, + tokio::time::Instant::now(), + inactivity_bound, + queue.has_in_flight() || heartbeat_in_flight, + ) { + tracing::info!( + inactivity_seconds = config.exit_after_inactivity_secs, + "inactivity bound reached — exiting gracefully" + ); + let _ = shutdown_tx.send(()); + } + None + } _ = async { match heartbeat.as_mut() { Some(hb) => hb.tick().await, @@ -2287,7 +2380,7 @@ async fn tokio_main() -> Result<()> { } else if queue.has_flushable_work() { tracing::debug!("heartbeat_skipped_events"); for (channel_id, thread_tags) in - dispatch_pending(&mut pool, &mut queue, &ctx) + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) { typing_channels.insert(channel_id, thread_tags); } @@ -2385,7 +2478,9 @@ async fn tokio_main() -> Result<()> { { break; } - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -2408,7 +2503,9 @@ async fn tokio_main() -> Result<()> { tracing::error!("all agents dead — exiting"); break; } - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -2550,7 +2647,9 @@ async fn tokio_main() -> Result<()> { // tear down the in-flight task; on its completion the // queue drains. We still try here in case the in-flight // task has already returned. - for (channel_id, thread_tags) in dispatch_pending(&mut pool, &mut queue, &ctx) { + for (channel_id, thread_tags) in + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) + { typing_channels.insert(channel_id, thread_tags); } } @@ -2577,7 +2676,7 @@ async fn tokio_main() -> Result<()> { None, ); for (channel_id, thread_tags) in - dispatch_pending(&mut pool, &mut queue, &ctx) + dispatch_pending(&mut pool, &mut queue, &ctx, &mut last_activity) { typing_channels.insert(channel_id, thread_tags); } @@ -2911,6 +3010,7 @@ fn dispatch_pending( pool: &mut AgentPool, queue: &mut EventQueue, ctx: &Arc, + last_activity: &mut tokio::time::Instant, ) -> Vec<(Uuid, ThreadTags)> { let mut dispatched_channels = Vec::new(); loop { @@ -2990,6 +3090,7 @@ fn dispatch_pending( }, ); dispatched_channels.push((channel_id, typing_scope)); + *last_activity = tokio::time::Instant::now(); } tracing::debug!( dispatched = dispatched_channels.len(), @@ -3625,6 +3726,22 @@ mod agent_draft_prompt_tests { assert!(prompt.contains("single-quoted shell strings preserve `\\n` literally")); assert!(prompt.contains("buzz messages send ... --content -")); } + + #[test] + fn shared_base_prompt_teaches_single_command_mentions_and_preflight() { + let prompt = include_str!("base_prompt.md"); + assert!(prompt.contains("--mention ")); + assert!(prompt.contains("every presentation-only name that should notify")); + assert!( + prompt.contains("permits unresolved or ambiguous `@Name` text as presentation-only") + ); + assert!(prompt.contains("success JSON's `mention_pubkeys`")); + assert!(prompt.contains("no follow-up verification command is needed")); + assert!(prompt.contains("stops before sending")); + assert!(prompt + .contains("add them explicitly with `buzz channels add-member` only when authorized")); + assert!(prompt.contains("never changes membership automatically")); + } } fn default_heartbeat_prompt() -> String { @@ -5015,6 +5132,7 @@ mod build_mcp_servers_tests { persona_env_vars: vec![], has_generated_codex_config: false, relay_observer: false, + exit_after_inactivity_secs: 0, lazy_pool: false, agent_owner: None, no_base_prompt: false, @@ -5236,6 +5354,7 @@ mod error_outcome_emission_tests { persona_env_vars: vec![], has_generated_codex_config: false, relay_observer: false, + exit_after_inactivity_secs: 0, lazy_pool: false, agent_owner: None, no_base_prompt: false, diff --git a/crates/buzz-acp/src/pool.rs b/crates/buzz-acp/src/pool.rs index b1fd68d044..64edf68ee2 100644 --- a/crates/buzz-acp/src/pool.rs +++ b/crates/buzz-acp/src/pool.rs @@ -19,6 +19,7 @@ //! //! `AcpClient` is NOT Clone — ownership moves out on claim and back on return. +use std::cmp::Reverse; use std::collections::{HashMap, HashSet}; use std::sync::{Arc, Mutex}; use std::time::Duration; @@ -31,6 +32,7 @@ use uuid::Uuid; use crate::acp::{ extract_model_config_options, extract_model_state, model_in_catalog, resolve_model_switch_method, AcpClient, AcpError, McpServer, ModelSwitchMethod, StopReason, + SystemPromptTransport, }; use crate::config::{compose_session_title, DedupMode, PermissionMode}; use crate::observer; @@ -170,6 +172,13 @@ pub struct OwnedAgent { pub protocol_version: u32, } +/// Package name reported by `claude-agent-acp` in its `initialize` response. +/// Any adapter reporting this name supports `_meta.systemPrompt: {append: ...}` +/// on `session/new` — the feature landed in v0.6.0 (Oct 2025), before the +/// `@zed-industries/claude-code-acp` → `@agentclientprotocol/claude-agent-acp` +/// rename, so the new name is a reliable capability gate. +const CLAUDE_AGENT_ACP_NAME: &str = "@agentclientprotocol/claude-agent-acp"; + fn has_system_prompt_support( protocol_version: u32, agent_name: &str, @@ -177,20 +186,25 @@ fn has_system_prompt_support( ) -> bool { if agent_name == "goose" { goose_system_prompt_supported == Some(true) + } else if agent_name == CLAUDE_AGENT_ACP_NAME { + true } else { protocol_version >= 2 } } -fn session_new_system_prompt( +fn session_new_system_prompt<'a>( is_goose: bool, protocol_version: u32, - prompt: Option<&str>, -) -> Option<&str> { - if is_goose || protocol_version < 2 { + agent_name: &str, + prompt: Option<&'a str>, +) -> Option> { + if is_goose || (protocol_version < 2 && agent_name != CLAUDE_AGENT_ACP_NAME) { None + } else if agent_name == CLAUDE_AGENT_ACP_NAME { + prompt.map(SystemPromptTransport::ClaudeMeta) } else { - prompt + prompt.map(SystemPromptTransport::Field) } } @@ -800,6 +814,9 @@ pub enum IdleSwitchResult { /// 2 × CONTEXT_FETCH_TIMEOUT + CONTEXT_FETCH_RETRY_DELAY ≈ 6.5 s. const CONTEXT_FETCH_TIMEOUT: Duration = Duration::from_millis(3_000); +/// Short, single-attempt timeout for best-effort exact truncated-thread counts. +const CONTEXT_COUNT_TIMEOUT: Duration = Duration::from_millis(500); + /// Delay between the first failed context fetch and the single retry. const CONTEXT_FETCH_RETRY_DELAY: Duration = Duration::from_millis(500); @@ -903,6 +920,7 @@ async fn create_session_and_apply_model( session_new_system_prompt( is_goose, agent.protocol_version, + &agent.agent_name, combined_system_prompt.as_deref(), ), session_title.as_deref(), @@ -1897,6 +1915,18 @@ pub async fn run_prompt_task( None => prompt_sections.iter().map(String::as_str).collect(), }; + // Turn start, labelled exactly as `log_stop_reason` labels the end, so a + // log reads as start/stop pairs. Purely observational: an unpaired start is + // the only durable evidence that a turn was entered and never returned, and + // without it a stalled agent and an agent nobody woke leave identical logs — + // zero completions either way, so anything reading them afterwards has to + // guess which happened. + tracing::info!( + target: "pool::prompt", + "turn starting for {}", + prompt_label(&source) + ); + // When control_rx is Some (channel tasks), wrap the prompt in select! so // the main loop can cancel, interrupt, or rotate it. Heartbeats // (control_rx=None) take the simple await path — they are not controllable. @@ -2588,7 +2618,14 @@ async fn fetch_conversation_context( let last_event = batch.events.last()?; let tags = crate::queue::parse_thread_tags(&last_event.event); if let Some(root_id) = tags.root_event_id { - return fetch_thread_context(batch.channel_id, &root_id, limit, &ctx.rest_client).await; + return fetch_thread_context( + batch.channel_id, + &root_id, + limit, + ctx.agent_keys.public_key(), + &ctx.rest_client, + ) + .await; } // DM non-reply: fetch recent conversation history. @@ -2750,12 +2787,48 @@ async fn fetch_prompt_profile_lookup( } /// Fetch thread context via Nostr query: root event by ID + replies by `#e` tag. +/// +/// The reply query intentionally requests one more reply than the configured +/// display window. That sentinel event lets the prompt say `N of M, truncated` +/// when the relay has more thread history, instead of reporting the capped page +/// as the total. When the window is full, a best-effort `/count` attempts to +/// improve that lower-bound total; because it is a separate racy request, the +/// result is clamped to the sentinel-proven minimum. The query also asks for the +/// agent's newest reply separately so the next prompt can include the agent's +/// own prior turn even in busy threads where the recent-message window would +/// otherwise push it out. async fn fetch_thread_context( channel_id: Uuid, root_event_id: &str, limit: u32, + agent_pubkey: nostr::PublicKey, rest: &RestClient, ) -> Option { + fetch_thread_context_with( + channel_id, + root_event_id, + limit, + agent_pubkey, + |filters| async move { rest.query(&filters).await }, + |filters| async move { rest.count(&filters).await }, + ) + .await +} + +async fn fetch_thread_context_with( + channel_id: Uuid, + root_event_id: &str, + limit: u32, + agent_pubkey: nostr::PublicKey, + query: Query, + count: Count, +) -> Option +where + Query: Fn(Vec) -> QueryFut, + QueryFut: std::future::Future>, + Count: Fn(Vec) -> CountFut, + CountFut: std::future::Future>, +{ use nostr::{Alphabet, SingleLetterTag}; // Defense-in-depth: validate hex event ID. @@ -2774,7 +2847,8 @@ async fn fetch_thread_context( let h_tag = SingleLetterTag::lowercase(Alphabet::H); let ch_str = channel_id.to_string(); - // Two filters: (1) root event by ID, (2) replies with #e=root + #h=channel. + // Three filters: (1) root event by ID, (2) recent replies with #e=root + + // #h=channel plus a sentinel, and (3) the agent's newest reply for pinning. let root_filter = nostr::Filter::new().id(nostr::EventId::from_hex(root_event_id).ok()?); let replies_filter = nostr::Filter::new() .kinds([ @@ -2783,16 +2857,23 @@ async fn fetch_thread_context( ]) .custom_tags(e_tag, [root_event_id]) .custom_tags(h_tag, [ch_str.as_str()]) - .limit(limit as usize); + .limit(limit.saturating_add(1) as usize); + let agent_reply_filter = replies_filter.clone().author(agent_pubkey).limit(1); - fetch_with_retry(|| async { + let context = fetch_with_retry(|| async { match timeout( CONTEXT_FETCH_TIMEOUT, - rest.query(&[root_filter.clone(), replies_filter.clone()]), + query(vec![ + root_filter.clone(), + replies_filter.clone(), + agent_reply_filter.clone(), + ]), ) .await { - Ok(Ok(json)) => parse_nostr_thread_response(json, root_event_id), + Ok(Ok(json)) => { + parse_nostr_thread_response_with_meta(json, root_event_id, limit, &agent_pubkey) + } Ok(Err(e)) => { tracing::warn!( channel_id = %channel_id, @@ -2811,7 +2892,75 @@ async fn fetch_thread_context( } } }) - .await + .await; + + let mut parsed = context?; + + if matches!( + parsed.context, + ConversationContext::Thread { + truncated: true, + .. + } + ) { + let replies_count_filter = replies_filter.clone().limit(0); + if let Some(total) = fetch_thread_total( + channel_id, + &replies_count_filter, + parsed.root_present, + &count, + ) + .await + { + if let ConversationContext::Thread { + total: context_total, + .. + } = &mut parsed.context + { + let sentinel_minimum = *context_total; + // `/count` is a separate best-effort request after the message + // query. If replies are deleted between the two, the exact count + // can fall below the already-proven sentinel minimum; never + // render impossible labels like `13 of 12 messages, truncated`. + *context_total = total.max(sentinel_minimum); + } + } + } + + Some(parsed.context) +} + +/// Best-effort exact thread size for truncated context labels. +async fn fetch_thread_total( + channel_id: Uuid, + replies_filter: &nostr::Filter, + root_present: bool, + count: &Count, +) -> Option +where + Count: Fn(Vec) -> CountFut, + CountFut: std::future::Future>, +{ + let replies_count = + match timeout(CONTEXT_COUNT_TIMEOUT, count(vec![replies_filter.clone()])).await { + Ok(Ok(json)) => json.get("count").and_then(|v| v.as_u64())?, + Ok(Err(e)) => { + tracing::debug!( + channel_id = %channel_id, + "thread context count failed; using sentinel minimum: {e}" + ); + return None; + } + Err(_) => { + tracing::debug!( + channel_id = %channel_id, + "thread context count timed out; using sentinel minimum" + ); + return None; + } + }; + + Some(replies_count as usize + usize::from(root_present)) } /// Fetch DM context via Nostr query: recent messages in channel by `#h` tag. @@ -2964,48 +3113,110 @@ fn json_to_context_message(obj: &serde_json::Value) -> Option { /// Parse a Nostr query response (array of events) into thread context. /// -/// Separates the root event (matching `root_event_id`) from replies, sorts -/// chronologically by `created_at`. +/// Separates the root event (matching `root_event_id`) from replies, keeps the +/// newest `limit` replies returned by the sentinel query, then sorts the +/// displayed window chronologically for the prompt. If the agent's newest reply +/// is outside that window, keep it instead of the oldest displayed reply so the +/// next prompt always includes the agent's most recent prior turn. +#[cfg(test)] fn parse_nostr_thread_response( json: serde_json::Value, root_event_id: &str, + limit: u32, + agent_pubkey: &nostr::PublicKey, ) -> Option { + parse_nostr_thread_response_with_meta(json, root_event_id, limit, agent_pubkey) + .map(|parsed| parsed.context) +} + +struct ParsedThreadContext { + context: ConversationContext, + root_present: bool, +} + +fn parse_nostr_thread_response_with_meta( + json: serde_json::Value, + root_event_id: &str, + limit: u32, + agent_pubkey: &nostr::PublicKey, +) -> Option { let events = json.as_array()?; + let agent_pubkey_hex = agent_pubkey.to_hex(); let mut root_msg = None; let mut reply_msgs = Vec::new(); + let mut seen_reply_ids = HashSet::new(); for ev in events { let ev_id = ev.get("id").and_then(|v| v.as_str()).unwrap_or(""); if let Some(msg) = json_to_context_message(ev) { if ev_id == root_event_id { root_msg = Some(msg); - } else { + } else if seen_reply_ids.insert(ev_id.to_string()) { + let is_agent = msg.pubkey.eq_ignore_ascii_case(&agent_pubkey_hex); reply_msgs.push(( + ev_id.to_string(), ev.get("created_at").and_then(|v| v.as_u64()).unwrap_or(0), + is_agent, msg, )); } } } - // Sort replies chronologically. - reply_msgs.sort_by_key(|(ts, _)| *ts); + let root_present = root_msg.is_some(); + let fetched_total = reply_msgs.len() + usize::from(root_present); + let newest_agent_reply = reply_msgs + .iter() + .filter(|(_, _, is_agent, _)| *is_agent) + .max_by_key(|(_, ts, _, _)| *ts) + .cloned(); + + let truncated = reply_msgs.len() > limit as usize; + if truncated { + // The relay returns limited REQ results newest-first. Sort explicitly so + // the sentinel we drop is the oldest reply in the fetched window, not an + // arbitrary last element if the HTTP bridge ever changes iteration order. + reply_msgs.sort_by_key(|(_, ts, _, _)| Reverse(*ts)); + reply_msgs.truncate(limit as usize); + } + + if let Some(agent_reply) = newest_agent_reply { + let agent_reply_already_displayed = + reply_msgs.iter().any(|(id, _, _, _)| *id == agent_reply.0); + if !agent_reply_already_displayed { + reply_msgs.sort_by_key(|(_, ts, _, _)| *ts); + if let Some(oldest) = reply_msgs.first_mut() { + *oldest = agent_reply; + } + } + } + + // Sort displayed replies chronologically. + reply_msgs.sort_by_key(|(_, ts, _, _)| *ts); let mut messages = Vec::new(); if let Some(root) = root_msg { messages.push(root); } - messages.extend(reply_msgs.into_iter().map(|(_, msg)| msg)); + messages.extend(reply_msgs.into_iter().map(|(_, _, _, msg)| msg)); - let total = messages.len(); if messages.is_empty() { return None; } - Some(ConversationContext::Thread { - messages, - total, - truncated: false, // query returns all within limit + let total = if truncated { + fetched_total // all distinct fetched replies plus the root are proven visible history + } else { + messages.len() + }; + + Some(ParsedThreadContext { + context: ConversationContext::Thread { + messages, + total, + truncated, + }, + root_present, }) } @@ -3131,12 +3342,19 @@ fn classify_control_cancel_failure( } } -/// Log a stop reason at the appropriate tracing level. -fn log_stop_reason(source: &PromptSource, stop_reason: &StopReason) { - let label = match source { +/// How a turn's source is named in the `pool::prompt` log lines. +/// +/// Shared by the turn-start and turn-stop lines so a log can be read as pairs. +fn prompt_label(source: &PromptSource) -> String { + match source { PromptSource::Channel(cid) => format!("channel {cid}"), PromptSource::Heartbeat => "heartbeat".to_string(), - }; + } +} + +/// Log a stop reason at the appropriate tracing level. +fn log_stop_reason(source: &PromptSource, stop_reason: &StopReason) { + let label = prompt_label(source); match stop_reason { StopReason::EndTurn => { tracing::info!(target: "pool::prompt", "turn complete for {label}: end_turn"); @@ -3390,35 +3608,41 @@ fn acp_stop_to_core(r: &StopReason) -> buzz_core::agent_turn_metric::StopReason } } -/// Best-effort: build and publish a `kind:44200` NIP-AM agent turn metric event. +/// Build the `(turn, cumulative)` `TokenCounts` pair for a NIP-AM kind-44200 +/// payload from a completed `TurnUsage`. /// -/// Does nothing when `usage` is `None` (goose emitted no usage notification -/// for this turn) or when `owner_pubkey` is unconfigured (no NIP-AO identity). -/// Errors are logged at WARN and never surface to the caller — metric -/// publishing must never fail a turn. -async fn publish_agent_turn_metric( - ctx: &PromptContext, - usage: Option, - channel_id: Option, - session_id: &str, - turn_id: &str, - stop_reason: Option, +/// Extracted as a pure function so the mapping logic can be tested independently +/// of relay/crypto infrastructure. `publish_agent_turn_metric` is the only +/// production caller. +/// +/// - `turn` is `None` when `delta_reliable` is false; otherwise it carries the +/// per-turn i/o/total/cost deltas for this turn. +/// - `cumulative` always carries the session-aggregate i/o/cost totals. +/// `total_tokens` is `Some` only when the session accumulated a genuine +/// provider-reported total on every turn — never derived from i/o sums +/// (NIP-AM MUST NOT). +pub(crate) fn build_turn_metric_counts( + usage: &crate::usage::TurnUsage, +) -> ( + Option, + Option, ) { - use buzz_core::agent_turn_metric::{AgentTurnMetricPayload, TokenCounts}; - use nostr::{EventBuilder, Kind, Tag}; - - let (usage, owner_pk) = match (usage, ctx.agent_owner_pubkey.as_ref()) { - (Some(u), Some(pk)) => (u, pk), - _ => return, - }; + use buzz_core::agent_turn_metric::TokenCounts; let turn_counts = if usage.delta_reliable { Some(TokenCounts { input_tokens: usage.turn_input_tokens, output_tokens: usage.turn_output_tokens, - total_tokens: None, + // Field-local: present only when both the previous and current + // cumulative totals were available and monotonic. Never derived + // from input+output. + total_tokens: usage.turn_total_tokens, cost_usd: usage.turn_cost_usd, - cache_read_tokens: None, + // Field-local: present when the cumulative counter was monotonic + // across this turn. Zero means no cache hits this turn (not absent). + cache_read_tokens: usage.turn_cache_read_tokens, + // buzz-agent does not emit a cache-write count on the wire today; + // leave None rather than deriving it from other fields. cache_write_tokens: None, }) } else { @@ -3431,11 +3655,46 @@ async fn publish_agent_turn_metric( let cumulative_counts = Some(TokenCounts { input_tokens: Some(usage.cumulative_input_tokens), output_tokens: Some(usage.cumulative_output_tokens), - total_tokens: None, + // Present when every turn in the session reported a genuine provider + // total. None when the session has never emitted one or any turn lacked + // one. Never derived from input+output (NIP-AM MUST NOT). + total_tokens: usage.cumulative_total_tokens, cost_usd: usage.cumulative_cost_usd, - cache_read_tokens: None, + // Session-cumulative cache-read tokens; None when the harness never + // reported this field (e.g. goose or older buzz-agent sessions). + // Passes through directly — do not wrap in Some() as the field already + // carries provenance (None vs Some(0) are distinct meanings). + cache_read_tokens: usage.cumulative_cache_read_tokens, + // buzz-agent does not emit a cache-write count on the wire today; + // leave None rather than deriving it from other fields. cache_write_tokens: None, }); + (turn_counts, cumulative_counts) +} + +/// Best-effort: build and publish a `kind:44200` NIP-AM agent turn metric event. +/// +/// Does nothing when `usage` is `None` (goose emitted no usage notification +/// for this turn) or when `owner_pubkey` is unconfigured (no NIP-AO identity). +/// Errors are logged at WARN and never surface to the caller — metric +/// publishing must never fail a turn. +async fn publish_agent_turn_metric( + ctx: &PromptContext, + usage: Option, + channel_id: Option, + session_id: &str, + turn_id: &str, + stop_reason: Option, +) { + use buzz_core::agent_turn_metric::AgentTurnMetricPayload; + use nostr::{EventBuilder, Kind, Tag}; + + let (usage, owner_pk) = match (usage, ctx.agent_owner_pubkey.as_ref()) { + (Some(u), Some(pk)) => (u, pk), + _ => return, + }; + + let (turn_counts, cumulative_counts) = build_turn_metric_counts(&usage); let timestamp = chrono::Utc::now().to_rfc3339_opts(chrono::SecondsFormat::Millis, true); let payload = AgentTurnMetricPayload { harness: ctx.harness_name.clone(), @@ -3758,18 +4017,48 @@ mod tests { assert!(has_system_prompt_support(2, "goose", Some(true))); assert!(has_system_prompt_support(1, "goose", Some(true))); assert!(has_system_prompt_support(2, "buzz-agent", None)); + // Goose never receives system prompt via session/new (uses post-hoc method). assert_eq!( - session_new_system_prompt(true, 2, Some("instructions")), + session_new_system_prompt(true, 2, "goose", Some("instructions")), None ); + // Protocol-v2 non-goose gets Field transport. assert_eq!( - session_new_system_prompt(false, 2, Some("instructions")), - Some("instructions") + session_new_system_prompt(false, 2, "buzz-agent", Some("instructions")), + Some(SystemPromptTransport::Field("instructions")) ); + // Protocol-v1 non-goose, non-claude gets None (legacy user-message framing). assert_eq!( - session_new_system_prompt(false, 1, Some("instructions")), + session_new_system_prompt(false, 1, "codex", Some("instructions")), None ); + // claude-agent-acp gets ClaudeMeta transport regardless of protocol version. + assert_eq!( + session_new_system_prompt(false, 1, CLAUDE_AGENT_ACP_NAME, Some("instructions")), + Some(SystemPromptTransport::ClaudeMeta("instructions")) + ); + assert_eq!( + session_new_system_prompt(true, 1, CLAUDE_AGENT_ACP_NAME, Some("instructions")), + None, + "goose path must never produce a transport even when agent_name matches" + ); + } + + #[test] + fn claude_agent_acp_has_system_prompt_support_regardless_of_protocol_version() { + // claude-agent-acp declares protocolVersion:1 but supports _meta.systemPrompt; + // has_system_prompt_support must return true so user-message framing is suppressed. + assert!(has_system_prompt_support(1, CLAUDE_AGENT_ACP_NAME, None)); + assert!(has_system_prompt_support(2, CLAUDE_AGENT_ACP_NAME, None)); + } + + #[test] + fn old_zed_adapter_name_falls_through_to_protocol_version_gate() { + // The renamed @zed-industries package predates the _meta.systemPrompt support, + // so it must not be treated as capable and stays on legacy user-message framing. + let old_name = "@zed-industries/claude-code-acp"; + assert!(!has_system_prompt_support(1, old_name, None)); + assert!(has_system_prompt_support(2, old_name, None)); } #[test] @@ -4154,6 +4443,572 @@ mod tests { assert!(parse_dm_response(json, 12).is_none()); } + #[test] + fn test_parse_nostr_thread_response_marks_query_window_truncated() { + let agent = Keys::generate(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let agent_hex = agent.public_key().to_hex(); + let json = json!([ + { + "id": root_id, + "pubkey": "rootpub", + "content": "root", + "created_at": 1000 + }, + { + "id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "pubkey": agent_hex, + "content": "newest agent reply", + "created_at": 4000 + }, + { + "id": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "pubkey": "humanpub", + "content": "middle reply", + "created_at": 3000 + }, + { + "id": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "pubkey": "oldpub", + "content": "sentinel omitted reply", + "created_at": 2000 + } + ]); + + let ctx = parse_nostr_thread_response(json, root_id, 2, &agent.public_key()) + .expect("should parse"); + match ctx { + ConversationContext::Thread { + messages, + total, + truncated, + } => { + assert_eq!(messages.len(), 3); // root + 2 displayed replies + assert_eq!(total, 4); // root + displayed replies + sentinel + assert!(truncated); + assert_eq!(messages[0].content, "root"); + assert_eq!(messages[1].content, "middle reply"); + assert_eq!(messages[2].content, "newest agent reply"); + assert!(messages + .iter() + .all(|msg| msg.content != "sentinel omitted reply")); + } + _ => panic!("expected Thread context"), + } + } + + #[test] + fn test_parse_nostr_thread_response_not_truncated_below_limit() { + let agent = Keys::generate(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let json = json!([ + { + "id": root_id, + "pubkey": "rootpub", + "content": "root", + "created_at": 1000 + }, + { + "id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "pubkey": "replypub", + "content": "reply", + "created_at": 2000 + } + ]); + + let ctx = parse_nostr_thread_response(json, root_id, 2, &agent.public_key()) + .expect("should parse"); + match ctx { + ConversationContext::Thread { + messages, + total, + truncated, + } => { + assert_eq!(messages.len(), 2); + assert_eq!(total, 2); + assert!(!truncated); + } + _ => panic!("expected Thread context"), + } + } + + #[test] + fn test_parse_nostr_thread_response_keeps_agent_reply_outside_recent_window() { + let agent = Keys::generate(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let agent_hex = agent.public_key().to_hex(); + let json = json!([ + { + "id": root_id, + "pubkey": "rootpub", + "content": "root", + "created_at": 1000 + }, + { + "id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "pubkey": "humanpub", + "content": "newer human reply", + "created_at": 5000 + }, + { + "id": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "pubkey": "humanpub", + "content": "middle human reply", + "created_at": 4000 + }, + { + "id": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "pubkey": "humanpub", + "content": "oldest displayed reply without agent pin", + "created_at": 3000 + }, + { + "id": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + "pubkey": agent_hex, + "content": "agent reply outside recent window", + "created_at": 2000 + } + ]); + + let ctx = parse_nostr_thread_response(json, root_id, 2, &agent.public_key()) + .expect("should parse"); + match ctx { + ConversationContext::Thread { messages, .. } => { + assert_eq!(messages.len(), 3); // root + 2 displayed replies + assert_eq!(messages[0].content, "root"); + assert!(messages + .iter() + .any(|msg| msg.content == "agent reply outside recent window")); + assert!(messages + .iter() + .any(|msg| msg.content == "newer human reply")); + assert!(messages + .iter() + .all(|msg| msg.content != "middle human reply")); + assert!(messages + .iter() + .all(|msg| msg.content != "oldest displayed reply without agent pin")); + } + _ => panic!("expected Thread context"), + } + } + + #[tokio::test] + async fn test_fetch_thread_context_uses_exact_count_when_above_sentinel_minimum() { + let agent = Keys::generate(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let channel_id = Uuid::new_v4(); + let agent_pubkey = agent.public_key(); + let json = json!([ + thread_event(root_id, "rootpub", "root", 1000), + thread_event( + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "humanpub", + "newest reply", + 4000 + ), + thread_event( + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "humanpub", + "middle reply", + 3000 + ), + thread_event( + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "humanpub", + "sentinel reply", + 2000 + ) + ]); + + let ctx = fetch_thread_context_with( + channel_id, + root_id, + 2, + agent_pubkey, + move |filters| { + assert_thread_query_filters(&filters, channel_id, root_id, agent_pubkey, 3); + std::future::ready(Ok(json.clone())) + }, + move |filters| { + assert_thread_count_filter(&filters, channel_id, root_id); + std::future::ready(Ok(json!({ "count": 6 }))) + }, + ) + .await + .expect("thread context"); + + match ctx { + ConversationContext::Thread { + messages, + total, + truncated, + } => { + assert!(truncated); + assert_eq!(messages.len(), 3); + assert_eq!(total, 7); // 6 replies + root + } + _ => panic!("expected Thread context"), + } + } + + #[tokio::test] + async fn test_fetch_thread_context_does_not_add_missing_root_to_exact_count() { + let agent = Keys::generate(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let channel_id = Uuid::new_v4(); + let json = json!([ + thread_event( + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "humanpub", + "newest reply", + 4000 + ), + thread_event( + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "humanpub", + "middle reply", + 3000 + ), + thread_event( + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "humanpub", + "sentinel reply", + 2000 + ) + ]); + + let ctx = fetch_thread_context_with( + channel_id, + root_id, + 2, + agent.public_key(), + move |_filters| std::future::ready(Ok(json.clone())), + |_filters| std::future::ready(Ok(json!({ "count": 6 }))), + ) + .await + .expect("thread context"); + + match ctx { + ConversationContext::Thread { + messages, + total, + truncated, + } => { + assert!(truncated); + assert_eq!(messages.len(), 2); + assert_eq!(total, 6); + } + _ => panic!("expected Thread context"), + } + } + + #[tokio::test] + async fn test_fetch_thread_context_clamps_count_below_sentinel_minimum() { + let agent = Keys::generate(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let channel_id = Uuid::new_v4(); + let json = json!([ + thread_event(root_id, "rootpub", "root", 1000), + thread_event( + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "humanpub", + "newest reply", + 4000 + ), + thread_event( + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "humanpub", + "middle reply", + 3000 + ), + thread_event( + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "humanpub", + "sentinel reply", + 2000 + ) + ]); + + let ctx = fetch_thread_context_with( + channel_id, + root_id, + 2, + agent.public_key(), + move |_filters| std::future::ready(Ok(json.clone())), + |_filters| std::future::ready(Ok(json!({ "count": 1 }))), + ) + .await + .expect("thread context"); + + match ctx { + ConversationContext::Thread { + messages, + total, + truncated, + } => { + assert!(truncated); + assert_eq!(messages.len(), 3); + assert_eq!(total, 4); // root + displayed replies + sentinel minimum + } + _ => panic!("expected Thread context"), + } + } + + #[tokio::test] + async fn test_fetch_thread_context_preserves_sentinel_minimum_when_count_fails() { + let agent = Keys::generate(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let channel_id = Uuid::new_v4(); + let json = json!([ + thread_event(root_id, "rootpub", "root", 1000), + thread_event( + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "humanpub", + "newest reply", + 4000 + ), + thread_event( + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "humanpub", + "middle reply", + 3000 + ), + thread_event( + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "humanpub", + "sentinel reply", + 2000 + ) + ]); + + let ctx = fetch_thread_context_with( + channel_id, + root_id, + 2, + agent.public_key(), + move |_filters| std::future::ready(Ok(json.clone())), + |_filters| std::future::ready(Err(crate::relay::RelayError::Http("boom".into()))), + ) + .await + .expect("thread context"); + + match ctx { + ConversationContext::Thread { + messages, + total, + truncated, + } => { + assert!(truncated); + assert_eq!(messages.len(), 3); + assert_eq!(total, 4); // count failure leaves parser's sentinel minimum intact + } + _ => panic!("expected Thread context"), + } + } + + #[tokio::test] + async fn test_fetch_thread_context_deduplicates_and_pins_agent_reply() { + let agent = Keys::generate(); + let agent_hex = agent.public_key().to_hex(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let channel_id = Uuid::new_v4(); + let json = json!([ + thread_event(root_id, "rootpub", "root", 1000), + thread_event( + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "humanpub", + "newer human reply", + 5000 + ), + thread_event( + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "humanpub", + "middle human reply", + 4000 + ), + thread_event( + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + &agent_hex, + "agent reply outside recent window", + 2000 + ), + // Same event as the separately fetched author-filtered result; the + // parser should deduplicate it before pinning. + thread_event( + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + &agent_hex, + "agent reply outside recent window", + 2000 + ) + ]); + + let ctx = fetch_thread_context_with( + channel_id, + root_id, + 2, + agent.public_key(), + move |_filters| std::future::ready(Ok(json.clone())), + |_filters| std::future::ready(Ok(json!({ "count": 3 }))), + ) + .await + .expect("thread context"); + + match ctx { + ConversationContext::Thread { + messages, + total, + truncated, + } => { + assert!(truncated); + assert_eq!(total, 4); + assert_eq!(messages.len(), 3); + assert_eq!( + messages + .iter() + .filter(|msg| msg.content == "agent reply outside recent window") + .count(), + 1, + "separate agent-reply query must not duplicate the same event" + ); + assert!(messages + .iter() + .any(|msg| msg.content == "newer human reply")); + assert!(messages + .iter() + .all(|msg| msg.content != "middle human reply")); + } + _ => panic!("expected Thread context"), + } + } + + #[tokio::test] + async fn test_fetch_thread_context_uses_distinct_fetched_replies_as_minimum() { + let agent = Keys::generate(); + let agent_hex = agent.public_key().to_hex(); + let root_id = "1111111111111111111111111111111111111111111111111111111111111111"; + let channel_id = Uuid::new_v4(); + let json = json!([ + thread_event(root_id, "rootpub", "root", 1000), + thread_event( + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "humanpub", + "newest human reply", + 5000 + ), + thread_event( + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", + "humanpub", + "middle human reply", + 4000 + ), + thread_event( + "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc", + "humanpub", + "sentinel human reply", + 3000 + ), + thread_event( + "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", + &agent_hex, + "older distinct agent reply", + 2000 + ) + ]); + + let ctx = fetch_thread_context_with( + channel_id, + root_id, + 2, + agent.public_key(), + move |_filters| std::future::ready(Ok(json.clone())), + |_filters| std::future::ready(Err(crate::relay::RelayError::Http("boom".into()))), + ) + .await + .expect("thread context"); + + match ctx { + ConversationContext::Thread { + messages, + total, + truncated, + } => { + assert!(truncated); + assert_eq!(messages.len(), 3); + assert_eq!( + total, 5, + "root plus all four distinct fetched replies prove the lower bound" + ); + assert!(messages + .iter() + .any(|msg| msg.content == "older distinct agent reply")); + assert!(messages + .iter() + .any(|msg| msg.content == "newest human reply")); + assert!(messages + .iter() + .all(|msg| msg.content != "middle human reply")); + assert!(messages + .iter() + .all(|msg| msg.content != "sentinel human reply")); + } + _ => panic!("expected Thread context"), + } + } + + fn assert_thread_query_filters( + filters: &[nostr::Filter], + channel_id: Uuid, + root_id: &str, + agent_pubkey: nostr::PublicKey, + reply_limit: u64, + ) { + assert_eq!( + filters.len(), + 3, + "root, recent replies, and agent reply filters" + ); + + let root = serde_json::to_value(&filters[0]).expect("serialize root filter"); + assert_eq!(root.get("ids"), Some(&json!([root_id]))); + assert!(root.get("limit").is_none()); + + let replies = serde_json::to_value(&filters[1]).expect("serialize replies filter"); + assert_eq!(replies.get("kinds"), Some(&json!([9, 40002]))); + assert_eq!(replies.get("#e"), Some(&json!([root_id]))); + assert_eq!(replies.get("#h"), Some(&json!([channel_id.to_string()]))); + assert_eq!(replies.get("limit"), Some(&json!(reply_limit))); + assert!(replies.get("authors").is_none()); + + let agent = serde_json::to_value(&filters[2]).expect("serialize agent filter"); + assert_eq!(agent.get("kinds"), Some(&json!([9, 40002]))); + assert_eq!(agent.get("#e"), Some(&json!([root_id]))); + assert_eq!(agent.get("#h"), Some(&json!([channel_id.to_string()]))); + assert_eq!(agent.get("authors"), Some(&json!([agent_pubkey.to_hex()]))); + assert_eq!(agent.get("limit"), Some(&json!(1))); + } + + fn assert_thread_count_filter(filters: &[nostr::Filter], channel_id: Uuid, root_id: &str) { + assert_eq!(filters.len(), 1, "count should query only matching replies"); + + let count = serde_json::to_value(&filters[0]).expect("serialize count filter"); + assert_eq!(count.get("kinds"), Some(&json!([9, 40002]))); + assert_eq!(count.get("#e"), Some(&json!([root_id]))); + assert_eq!(count.get("#h"), Some(&json!([channel_id.to_string()]))); + assert_eq!(count.get("limit"), Some(&json!(0))); + assert!(count.get("ids").is_none()); + assert!(count.get("authors").is_none()); + } + + fn thread_event(id: &str, pubkey: &str, content: &str, created_at: u64) -> serde_json::Value { + json!({ + "id": id, + "pubkey": pubkey, + "content": content, + "created_at": created_at + }) + } + #[test] fn test_json_to_context_message_integer_timestamp() { let obj = json!({ @@ -5219,10 +6074,14 @@ mod tests { delta_reliable: true, turn_input_tokens: Some(100), turn_output_tokens: Some(50), + turn_total_tokens: None, turn_cost_usd: None, + turn_cache_read_tokens: None, cumulative_input_tokens: 100, cumulative_output_tokens: 50, + cumulative_total_tokens: None, cumulative_cost_usd: None, + cumulative_cache_read_tokens: None, model: None, }; // owner_pubkey = None → early return, no panic. @@ -5251,10 +6110,14 @@ mod tests { delta_reliable: true, turn_input_tokens: Some(200), turn_output_tokens: Some(80), + turn_total_tokens: None, turn_cost_usd: Some(0.001), + turn_cache_read_tokens: None, cumulative_input_tokens: 200, cumulative_output_tokens: 80, + cumulative_total_tokens: None, cumulative_cost_usd: Some(0.001), + cumulative_cache_read_tokens: None, model: None, }; // Will try to publish and fail (no real relay) but must not panic. @@ -5284,10 +6147,14 @@ mod tests { delta_reliable: true, turn_input_tokens: Some(50), turn_output_tokens: Some(20), + turn_total_tokens: None, turn_cost_usd: None, + turn_cache_read_tokens: None, cumulative_input_tokens: 150, cumulative_output_tokens: 70, + cumulative_total_tokens: None, cumulative_cost_usd: None, + cumulative_cache_read_tokens: None, model: None, }; // Must not panic; HTTP submit will fail (no real relay) — that's fine. @@ -5317,10 +6184,14 @@ mod tests { delta_reliable: false, // first turn from buzz-agent turn_input_tokens: None, turn_output_tokens: None, + turn_total_tokens: None, turn_cost_usd: None, + turn_cache_read_tokens: None, cumulative_input_tokens: 400, cumulative_output_tokens: 100, + cumulative_total_tokens: None, cumulative_cost_usd: None, + cumulative_cache_read_tokens: None, model: None, }; // Will try to publish (encrypt succeeds) and fail HTTP (no relay) — must not panic. @@ -5335,6 +6206,204 @@ mod tests { .await; } + /// `build_turn_metric_counts` maps exact turn and cumulative totals from + /// `TurnUsage` to the corresponding `TokenCounts.total_tokens` fields. + /// Reverting the production fields at the call site to `None` would break + /// this test; the test constrains the real code path. + #[test] + fn test_build_turn_metric_counts_exact_totals_map_through() { + let usage = crate::usage::TurnUsage { + session_id: "sess-total".to_string(), + turn_seq: 2, + delta_reliable: true, + turn_input_tokens: Some(100), + turn_output_tokens: Some(30), + turn_total_tokens: Some(130), // genuine per-turn total + turn_cost_usd: None, + turn_cache_read_tokens: None, + cumulative_input_tokens: 500, + cumulative_output_tokens: 120, + cumulative_total_tokens: Some(620), // genuine cumulative total + cumulative_cost_usd: None, + cumulative_cache_read_tokens: None, + model: None, + }; + + let (turn, cumulative) = crate::pool::build_turn_metric_counts(&usage); + + // Serialise to JSON — this is what ultimately goes on the wire. + let turn_json = serde_json::to_value(turn.as_ref().expect("turn counts present")).unwrap(); + let cum_json = + serde_json::to_value(cumulative.as_ref().expect("cumulative counts present")).unwrap(); + + // Per-turn total must be the genuine provider-reported value. + assert_eq!( + turn_json["totalTokens"], + serde_json::json!(130), + "per-turn total must map to TokenCounts.totalTokens in wire JSON" + ); + assert_eq!(turn_json["inputTokens"], serde_json::json!(100)); + assert_eq!(turn_json["outputTokens"], serde_json::json!(30)); + + // Cumulative total must be the genuine session total. + assert_eq!( + cum_json["totalTokens"], + serde_json::json!(620), + "cumulative total must map to TokenCounts.totalTokens in wire JSON" + ); + assert_eq!(cum_json["inputTokens"], serde_json::json!(500)); + assert_eq!(cum_json["outputTokens"], serde_json::json!(120)); + } + + /// When totals are absent, `build_turn_metric_counts` must produce null + /// `total_tokens` — never a derived input+output sum (NIP-AM MUST NOT). + /// Reverting the production fields to hardcoded `None` would leave this test + /// passing but input/output would disagree, making the null-path detectable. + #[test] + fn test_build_turn_metric_counts_null_totals_never_derived() { + let usage = crate::usage::TurnUsage { + session_id: "sess-nototal".to_string(), + turn_seq: 1, + delta_reliable: true, + turn_input_tokens: Some(200), + turn_output_tokens: Some(60), + turn_total_tokens: None, // provider did not supply a total + turn_cost_usd: None, + turn_cache_read_tokens: None, + cumulative_input_tokens: 200, + cumulative_output_tokens: 60, + cumulative_total_tokens: None, // session has no total + cumulative_cost_usd: None, + cumulative_cache_read_tokens: None, + model: None, + }; + + let (turn, cumulative) = crate::pool::build_turn_metric_counts(&usage); + + let turn_json = serde_json::to_value(turn.as_ref().expect("turn counts present")).unwrap(); + let cum_json = + serde_json::to_value(cumulative.as_ref().expect("cumulative counts present")).unwrap(); + + // total_tokens must be null in the wire JSON. + assert!( + turn_json["totalTokens"].is_null(), + "absent turn total must serialize as null — not derived from in+out" + ); + assert!( + cum_json["totalTokens"].is_null(), + "absent cumulative total must serialize as null — not derived from in+out" + ); + + // Input/output must still carry their real values. + assert_eq!( + turn_json["inputTokens"], + serde_json::json!(200), + "inputTokens must be present even when total is absent" + ); + assert_eq!( + turn_json["outputTokens"], + serde_json::json!(60), + "outputTokens must be present even when total is absent" + ); + + // The null total must not equal the input+output sum — it must be genuinely null. + let derived_sum = serde_json::json!(200u64 + 60u64); + assert_ne!( + turn_json["totalTokens"], derived_sum, + "total_tokens must never equal input+output when provider omitted it" + ); + } + + /// A payload with nonzero `accumulatedCachedInputTokens` on the second turn + /// must produce a kind:44200 payload where `cumulative.cacheReadTokens` is + /// nonzero and `turn.cacheReadTokens` reflects the per-turn delta. + /// This is the acceptance-criterion test: it proves the threading is live, + /// not hardcoded to None. + #[test] + fn test_build_turn_metric_counts_cache_read_tokens_thread_through() { + // Wire-parse a buzz-agent payload with cache, run it through the tracker, + // and verify the published TokenCounts carry the cache field. + let raw1 = serde_json::json!({ + "sessionId": "cache-sess", + "update": { + "sessionUpdate": "usage_update", + "accumulatedInputTokens": 15_091, + "accumulatedOutputTokens": 156, + "accumulatedCachedInputTokens": 5_033, + } + }); + let raw2 = serde_json::json!({ + "sessionId": "cache-sess", + "update": { + "sessionUpdate": "usage_update", + "accumulatedInputTokens": 28_500, + "accumulatedOutputTokens": 310, + "accumulatedCachedInputTokens": 11_000, + } + }); + + let mut tracker = crate::usage::UsageTracker::default(); + + // Turn 1 — establish baseline (delta unreliable, but cumulative still present). + tracker.begin_turn("cache-sess"); + if let crate::usage::GooseSessionUpdateVariant::UsageUpdate(p) = + serde_json::from_value::(raw1) + .unwrap() + .update + { + tracker.record("cache-sess", &p); + } + let t1 = tracker.take().expect("turn 1"); + + // Turn 1: cumulative must carry the cache count; turn delta is None (no baseline). + let (turn1, cum1) = crate::pool::build_turn_metric_counts(&t1); + // delta_reliable = false on first turn → no turn counts. + assert!(turn1.is_none(), "first turn: no reliable turn counts"); + let cum1 = cum1.expect("cumulative always present"); + assert_eq!( + cum1.cache_read_tokens, + Some(5_033), + "cumulative.cacheReadTokens must be 5033 after turn 1" + ); + + // Turn 2 — delta reliable. + tracker.begin_turn("cache-sess"); + if let crate::usage::GooseSessionUpdateVariant::UsageUpdate(p) = + serde_json::from_value::(raw2) + .unwrap() + .update + { + tracker.record("cache-sess", &p); + } + let t2 = tracker.take().expect("turn 2"); + + let (turn2, cum2) = crate::pool::build_turn_metric_counts(&t2); + + let turn2 = turn2.expect("reliable turn counts on turn 2"); + // Per-turn cache delta: 11_000 - 5_033 = 5_967. + assert_eq!( + turn2.cache_read_tokens, + Some(5_967), + "turn.cacheReadTokens must be the per-turn delta" + ); + // cache_write_tokens is always None — buzz-agent doesn't emit it. + assert!( + turn2.cache_write_tokens.is_none(), + "cache_write_tokens must be None — not emitted by buzz-agent" + ); + + let cum2 = cum2.expect("cumulative always present"); + assert_eq!( + cum2.cache_read_tokens, + Some(11_000), + "cumulative.cacheReadTokens must be 11_000 after turn 2" + ); + assert!( + cum2.cache_write_tokens.is_none(), + "cache_write_tokens must be None on cumulative too" + ); + } + fn make_prompt_context_no_owner() -> PromptContext { let agent_keys = nostr::Keys::generate(); make_prompt_context_impl(&agent_keys, None) diff --git a/crates/buzz-acp/src/queue.rs b/crates/buzz-acp/src/queue.rs index 029bf86dbf..5c960de202 100644 --- a/crates/buzz-acp/src/queue.rs +++ b/crates/buzz-acp/src/queue.rs @@ -646,6 +646,11 @@ impl EventQueue { self.in_flight_channels.contains(&channel_id) } + /// Whether any channel currently has a turn in flight. + pub fn has_in_flight(&self) -> bool { + !self.in_flight_channels.is_empty() + } + // ── Goose-native steer withhold (side table) ────────────────────────── // // While a goose-native `_goose/unstable/session/steer` write is in flight diff --git a/crates/buzz-acp/src/relay.rs b/crates/buzz-acp/src/relay.rs index c8312cc61e..aea5cee077 100644 --- a/crates/buzz-acp/src/relay.rs +++ b/crates/buzz-acp/src/relay.rs @@ -405,6 +405,19 @@ impl RestClient { .map_err(|e| RelayError::Http(e.to_string())) } + /// Count events via the HTTP bridge: `POST /count` with NIP-98 auth. + /// + /// Accepts a slice of `nostr::Filter` (serialized as JSON array). + /// Returns the bridge response as a `serde_json::Value` (usually `{ "count": n }`). + pub async fn count(&self, filters: &[nostr::Filter]) -> Result { + let body_bytes = serde_json::to_vec(filters) + .map_err(|e| RelayError::Http(format!("filter serialize error: {e}")))?; + let resp = self.bridge_post("/count", &body_bytes).await?; + resp.json() + .await + .map_err(|e| RelayError::Http(e.to_string())) + } + /// Submit a signed event via the HTTP bridge: `POST /events` with NIP-98 auth. /// /// The event must already be signed. Returns the relay response JSON. diff --git a/crates/buzz-acp/src/usage.rs b/crates/buzz-acp/src/usage.rs index a4f7abd3b3..56b772d12c 100644 --- a/crates/buzz-acp/src/usage.rs +++ b/crates/buzz-acp/src/usage.rs @@ -85,7 +85,25 @@ pub(crate) struct UsageUpdatePayload { pub context_limit: u64, pub accumulated_input_tokens: u64, pub accumulated_output_tokens: u64, + /// The cache-served subset of `accumulated_input_tokens`. + /// + /// `None` when the harness did not include the field (e.g. goose, which + /// never emits it). `Some(0)` when the harness explicitly reported zero + /// cache hits. The distinction matters: `None` means "we don't know", + /// while `Some(0)` means "provider confirmed no cache was used". + /// + /// Do NOT use `#[serde(default)]` here — that would collapse the absent + /// case into `Some(0)` and destroy provenance in the append-only archive. + pub accumulated_cached_input_tokens: Option, pub accumulated_cost: Option, + /// Session-cumulative genuine provider total tokens. Optional — only + /// emitted by buzz-agent when every turn in the session so far supplied a + /// provider-reported total. Absent for goose (field ignore-if-absent for + /// backward compat), for Anthropic-backed turns, and for sessions where any + /// turn lacked a provider total. NIP-AM forbids deriving this by summing + /// categories, so the UI must approximate when this field is absent. + #[serde(default)] + pub accumulated_total_tokens: Option, /// Effective model id for this turn. Optional — goose payloads that /// predate this field deserialize cleanly as `None`. #[serde(default)] @@ -107,6 +125,16 @@ struct SessionState { last_output: u64, /// Cumulative cost at the end of the LAST PUBLISHED turn. last_cost: Option, + /// Cumulative total tokens at the end of the LAST PUBLISHED turn. + /// `None` when the session has never emitted a provider total (Unseen) or + /// when any prior turn lacked one (poisoned). + last_total: Option, + /// Cumulative cache-read input tokens at the end of the LAST PUBLISHED turn. + /// `None` when the harness has never reported this field (e.g. goose). + /// `Some(n)` when at least one payload included the field. Field-local: + /// a decrease in this counter taints only the cache-read delta, not + /// `delta_reliable` or the input/output deltas. + last_cached_input: Option, } /// Per-turn usage record exposed to `TurnCompletionGuard` for NIP-AM publishing. @@ -125,15 +153,34 @@ pub struct TurnUsage { pub turn_input_tokens: Option, /// Per-turn output token delta; `None` when unreliable. pub turn_output_tokens: Option, + /// Per-turn total token delta; `None` when the cumulative total is + /// unavailable (no baseline, non-monotonic, or either snapshot was absent). + /// Field-local: a missing total never flips `delta_reliable` or invalidates + /// `turn_input_tokens`/`turn_output_tokens`. + pub turn_total_tokens: Option, /// Per-turn cost delta (`current − previous`); `None` when unreliable or /// either snapshot is missing. pub turn_cost_usd: Option, + /// Per-turn cache-read token delta (`current − previous`); `None` when no + /// baseline exists, either snapshot is `None` (harness did not report it), + /// or the cumulative counter decreased (field-local taint). Field-local: + /// a decrease here never flips `delta_reliable` or invalidates the + /// input/output deltas. + pub turn_cache_read_tokens: Option, /// Session-cumulative input tokens as reported by goose at end of turn. pub cumulative_input_tokens: u64, /// Session-cumulative output tokens as reported by goose at end of turn. pub cumulative_output_tokens: u64, + /// Session-cumulative genuine provider total tokens as reported by buzz-agent; + /// `None` when the session has never emitted one or any turn lacked one. + pub cumulative_total_tokens: Option, /// Session-cumulative estimated cost in USD; `None` if goose did not report it. pub cumulative_cost_usd: Option, + /// Session-cumulative cache-read input tokens as reported by buzz-agent. + /// `None` when the harness has never reported this field (e.g. goose or + /// any harness that omits `accumulatedCachedInputTokens`). + /// `Some(0)` when the harness reported zero cache hits. + pub cumulative_cache_read_tokens: Option, /// Effective model id for this turn (maps to NIP-AM `model`). `None` if the /// harness did not include the model in its usage notification. pub model: Option, @@ -212,6 +259,8 @@ impl UsageTracker { let current_input = payload.accumulated_input_tokens; let current_output = payload.accumulated_output_tokens; let current_cost = payload.accumulated_cost; + let current_total = payload.accumulated_total_tokens; + let current_cached_input = payload.accumulated_cached_input_tokens; // Determine whether this session is currently in-flight so we know // whether to set `pending`. We compute the delta regardless so that @@ -256,6 +305,32 @@ impl UsageTracker { } }; + // Total-token delta: field-local — never affects `delta_reliable` or + // the input/output deltas. Null when: no baseline exists, either + // snapshot is absent, or cumulative total decreased. + let turn_total = match self.sessions.get(session_id) { + Some(prev) => match (current_total, prev.last_total) { + (Some(cur), Some(p)) if cur >= p => Some(cur - p), + _ => None, // no baseline, absent on either side, or decrease + }, + None => None, // no baseline yet + }; + + // Cache-read token delta: field-local — never affects `delta_reliable` + // or the input/output deltas. Null when: no baseline exists, either + // snapshot is None (harness did not report the field), or the cumulative + // counter decreased (harness restart, overflow). + // Some(0) is a valid result when both snapshots are Some(0) — it means + // the harness confirmed zero cache hits this turn, not that data is absent. + let turn_cache_read = match self.sessions.get(session_id) { + Some(prev) => match (current_cached_input, prev.last_cached_input) { + (Some(cur), Some(p)) if cur >= p => Some(cur - p), + (Some(_), Some(_)) => None, // decrease → field-local taint + _ => None, // either snapshot absent → no delta + }, + None => None, // no baseline yet + }; + if is_in_flight { // In-flight-match: update pending with the latest cumulative values. // Baseline is NOT advanced here — it advances only on take(). @@ -265,10 +340,14 @@ impl UsageTracker { delta_reliable, turn_input_tokens: turn_input, turn_output_tokens: turn_output, + turn_total_tokens: turn_total, turn_cost_usd: turn_cost, + turn_cache_read_tokens: turn_cache_read, cumulative_input_tokens: current_input, cumulative_output_tokens: current_output, + cumulative_total_tokens: current_total, cumulative_cost_usd: current_cost, + cumulative_cache_read_tokens: current_cached_input, model: payload.model.clone(), }); } else if self.in_flight_session.is_none() { @@ -286,6 +365,8 @@ impl UsageTracker { last_input: current_input, last_output: current_output, last_cost: current_cost, + last_total: current_total, + last_cached_input: current_cached_input, }, ); } @@ -313,6 +394,8 @@ impl UsageTracker { last_input: record.cumulative_input_tokens, last_output: record.cumulative_output_tokens, last_cost: record.cumulative_cost_usd, + last_total: record.cumulative_total_tokens, + last_cached_input: record.cumulative_cache_read_tokens, }, ); Some(record) @@ -323,13 +406,68 @@ impl UsageTracker { mod tests { use super::*; + /// The camelCase key buzz-agent actually puts on the wire must land on the + /// field. A rename mismatch here would deserialize to None, and every trial + /// would be treated as "not reported" — the exact silent failure this field + /// was added to remove. + #[test] + fn cached_input_tokens_deserialize_from_the_wire_key() { + let p: UsageUpdatePayload = serde_json::from_value(serde_json::json!({ + "used": 15_247, + "contextLimit": 0, + "accumulatedInputTokens": 15_091, + "accumulatedOutputTokens": 156, + "accumulatedCachedInputTokens": 5_033, + })) + .expect("payload must deserialize"); + assert_eq!(p.accumulated_cached_input_tokens, Some(5_033)); + assert!(p.accumulated_cached_input_tokens.unwrap() <= p.accumulated_input_tokens); + } + + /// goose does not send the field; its payloads must deserialize with None — + /// not zero — so that "not reported" is preserved distinct from "reported zero". + #[test] + fn a_payload_without_the_cache_field_deserializes_as_none() { + let p: UsageUpdatePayload = serde_json::from_value(serde_json::json!({ + "used": 500, + "contextLimit": 200_000, + "accumulatedInputTokens": 400, + "accumulatedOutputTokens": 100, + })) + .expect("payload must deserialize without the cache field"); + assert!( + p.accumulated_cached_input_tokens.is_none(), + "absent field must be None, not Some(0)" + ); + } + + /// A harness that explicitly reports zero cache hits must produce Some(0), + /// not None — so downstream analytics can distinguish "confirmed zero" from + /// "not reported". + #[test] + fn a_payload_with_explicit_zero_cache_field_deserializes_as_some_zero() { + let p: UsageUpdatePayload = serde_json::from_value(serde_json::json!({ + "accumulatedInputTokens": 400, + "accumulatedOutputTokens": 100, + "accumulatedCachedInputTokens": 0, + })) + .expect("payload must deserialize with zero cache field"); + assert_eq!( + p.accumulated_cached_input_tokens, + Some(0), + "explicit zero must be Some(0), not None" + ); + } + fn payload(input: u64, output: u64, cost: Option) -> UsageUpdatePayload { UsageUpdatePayload { used: input + output, context_limit: 200_000, accumulated_input_tokens: input, accumulated_output_tokens: output, + accumulated_cached_input_tokens: None, accumulated_cost: cost, + accumulated_total_tokens: None, model: None, } } @@ -340,7 +478,9 @@ mod tests { context_limit: 0, accumulated_input_tokens: input, accumulated_output_tokens: output, + accumulated_cached_input_tokens: None, accumulated_cost: cost, + accumulated_total_tokens: None, model: None, } } @@ -836,7 +976,9 @@ mod tests { context_limit: 200_000, accumulated_input_tokens: input, accumulated_output_tokens: output, + accumulated_cached_input_tokens: None, accumulated_cost: cost, + accumulated_total_tokens: None, model: model.map(str::to_string), } } @@ -889,4 +1031,484 @@ mod tests { "TurnUsage.model must be None when payload omits the field" ); } + + // ── accumulatedTotalTokens: field-local delta, session poisoning ─────── + + fn payload_with_total(input: u64, output: u64, total: Option) -> UsageUpdatePayload { + UsageUpdatePayload { + used: input + output, + context_limit: 200_000, + accumulated_input_tokens: input, + accumulated_output_tokens: output, + accumulated_cached_input_tokens: None, + accumulated_cost: None, + accumulated_total_tokens: total, + model: None, + } + } + + #[test] + fn first_update_without_baseline_turn_total_is_none() { + // No baseline exists → turn total null, but delta_reliable/input/output + // follow the normal first-turn rule (delta_reliable = false). + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-t1"); + tracker.record("sess-t1", &payload_with_total(100, 20, Some(120))); + let usage = tracker.take().expect("pending"); + + assert!(!usage.delta_reliable, "first turn: delta unreliable"); + assert!( + usage.turn_total_tokens.is_none(), + "no baseline → turn total must be None" + ); + assert_eq!( + usage.cumulative_total_tokens, + Some(120), + "cumulative total passes through even on first turn" + ); + } + + #[test] + fn second_turn_with_totals_produces_turn_delta() { + let mut tracker = UsageTracker::default(); + // Turn 1 — establish baseline. + tracker.begin_turn("sess-t2"); + tracker.record("sess-t2", &payload_with_total(100, 20, Some(120))); + let _ = tracker.take(); + + // Turn 2 — delta is computable. + tracker.begin_turn("sess-t2"); + tracker.record("sess-t2", &payload_with_total(200, 50, Some(250))); + let usage = tracker.take().expect("pending"); + + assert!(usage.delta_reliable); + assert_eq!(usage.turn_total_tokens, Some(130)); // 250 - 120 + assert_eq!(usage.cumulative_total_tokens, Some(250)); + } + + #[test] + fn cumulative_total_decrease_leaves_turn_total_null_without_affecting_reliability() { + // Cumulative total decreases (e.g. counter reset) → turn total null, + // but delta_reliable and input/output are NOT affected (field-local). + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-t3"); + tracker.record("sess-t3", &payload_with_total(500, 100, Some(600))); + let _ = tracker.take(); + + tracker.begin_turn("sess-t3"); + // Cumulative total decreased: 600 → 50. + tracker.record("sess-t3", &payload_with_total(600, 150, Some(50))); + let usage = tracker.take().expect("pending"); + + assert!( + usage.delta_reliable, + "input/output decrease would flip reliability; total decrease must not" + ); + assert_eq!(usage.turn_input_tokens, Some(100)); + assert_eq!(usage.turn_output_tokens, Some(50)); + assert!( + usage.turn_total_tokens.is_none(), + "cumulative total decrease → turn total null (field-local)" + ); + assert_eq!( + usage.cumulative_total_tokens, + Some(50), + "cumulative total from payload still passes through" + ); + } + + #[test] + fn cumulative_total_absent_on_current_turn_leaves_turn_total_null() { + // Goose-shaped payload: no accumulatedTotalTokens field at all. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-t4"); + tracker.record("sess-t4", &payload_with_total(100, 20, Some(120))); + let _ = tracker.take(); + + // Second turn: goose omits the total field entirely. + tracker.begin_turn("sess-t4"); + tracker.record("sess-t4", &payload_with_total(200, 50, None)); + let usage = tracker.take().expect("pending"); + + assert!(usage.delta_reliable, "input/output delta unaffected"); + assert_eq!(usage.turn_input_tokens, Some(100)); + assert_eq!(usage.turn_output_tokens, Some(30)); + assert!( + usage.turn_total_tokens.is_none(), + "absent field → null turn total" + ); + assert!( + usage.cumulative_total_tokens.is_none(), + "absent cumulative total passes through as None" + ); + } + + #[test] + fn goose_shaped_payload_without_accumulated_total_deserializes_correctly() { + // goose payloads lack accumulatedTotalTokens; the field must default + // to None without a deserialization error (ignore-if-absent contract). + let json = r#"{ + "sessionUpdate": "usage_update", + "accumulatedInputTokens": 1000, + "accumulatedOutputTokens": 200, + "accumulatedCost": 0.01 + }"#; + let variant: GooseSessionUpdateVariant = + serde_json::from_str(json).expect("must deserialize without accumulatedTotalTokens"); + let payload = match variant { + GooseSessionUpdateVariant::UsageUpdate(p) => p, + _ => panic!("expected UsageUpdate"), + }; + assert!( + payload.accumulated_total_tokens.is_none(), + "absent accumulatedTotalTokens must default to None" + ); + + // And it must flow through the tracker correctly. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-goose-nototal"); + tracker.record("sess-goose-nototal", &payload); + let usage = tracker.take().expect("pending"); + assert!( + usage.cumulative_total_tokens.is_none(), + "goose-shaped payload must produce None cumulative_total_tokens" + ); + } + + #[test] + fn cumulative_total_absent_on_baseline_leaves_turn_total_null_on_second_turn() { + // Baseline was set without a total (e.g. first goose turn); second + // turn reports a total. No baseline to diff against → turn total None. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-t5"); + tracker.record("sess-t5", &payload_with_total(100, 20, None)); // no total + let _ = tracker.take(); + + tracker.begin_turn("sess-t5"); + tracker.record("sess-t5", &payload_with_total(200, 50, Some(250))); + let usage = tracker.take().expect("pending"); + + assert!(usage.delta_reliable, "input/output delta unaffected"); + assert!( + usage.turn_total_tokens.is_none(), + "absent baseline total → turn total null even when current has a total" + ); + assert_eq!(usage.cumulative_total_tokens, Some(250)); + } + + // ── cache-read token threading ────────────────────────────────────────── + + fn payload_with_cache( + input: u64, + output: u64, + cached_input: Option, + ) -> UsageUpdatePayload { + UsageUpdatePayload { + used: input + output, + context_limit: 200_000, + accumulated_input_tokens: input, + accumulated_output_tokens: output, + accumulated_cached_input_tokens: cached_input, + accumulated_cost: None, + accumulated_total_tokens: None, + model: None, + } + } + + #[test] + fn cache_read_first_turn_produces_none_turn_delta_and_passes_cumulative_through() { + // First turn has no baseline → turn cache delta must be None, but + // cumulative_cache_read_tokens must carry the reported value through. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-c1"); + tracker.record("sess-c1", &payload_with_cache(1000, 200, Some(500))); + let usage = tracker.take().expect("pending"); + + assert!( + usage.turn_cache_read_tokens.is_none(), + "first turn: no baseline → cache delta must be None" + ); + assert_eq!( + usage.cumulative_cache_read_tokens, + Some(500), + "cumulative cache read passes through on first turn" + ); + assert!(!usage.delta_reliable, "first turn is unreliable"); + } + + #[test] + fn cache_read_second_turn_delta_computed_correctly() { + // Second turn: cumulative cached 500 → 1200, delta = 700. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-c2"); + tracker.record("sess-c2", &payload_with_cache(1000, 200, Some(500))); + let _ = tracker.take(); + + tracker.begin_turn("sess-c2"); + tracker.record("sess-c2", &payload_with_cache(2000, 350, Some(1200))); + let usage = tracker.take().expect("pending"); + + assert!(usage.delta_reliable); + assert_eq!( + usage.turn_cache_read_tokens, + Some(700), + "cache delta = 1200 - 500 = 700" + ); + assert_eq!( + usage.cumulative_cache_read_tokens, + Some(1200), + "cumulative cache passes through" + ); + } + + #[test] + fn cache_read_decrease_nulls_turn_cache_but_leaves_delta_reliable() { + // Cache counter decrease → cache delta None (field-local taint), but + // delta_reliable and input/output deltas are NOT affected. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-c3"); + tracker.record("sess-c3", &payload_with_cache(1000, 200, Some(800))); + let _ = tracker.take(); + + tracker.begin_turn("sess-c3"); + // Cache counter decreased: 800 → 50. + tracker.record("sess-c3", &payload_with_cache(1500, 300, Some(50))); + let usage = tracker.take().expect("pending"); + + assert!( + usage.delta_reliable, + "cache decrease must NOT flip delta_reliable — field-local" + ); + assert_eq!( + usage.turn_input_tokens, + Some(500), + "input/output delta unaffected by cache decrease" + ); + assert_eq!(usage.turn_output_tokens, Some(100)); + assert!( + usage.turn_cache_read_tokens.is_none(), + "cache counter decrease → turn_cache_read_tokens None (field-local taint)" + ); + assert_eq!( + usage.cumulative_cache_read_tokens, + Some(50), + "cumulative still passes through from payload even on decrease" + ); + } + + #[test] + fn cache_read_explicit_zero_payload_after_explicit_zero_baseline_produces_some_zero_delta() { + // When both baseline and current are Some(0), turn_cache_read_tokens must + // be Some(0) — confirmed zero, not absent. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-c4"); + tracker.record("sess-c4", &payload_with_cache(1000, 200, Some(0))); + let _ = tracker.take(); + + tracker.begin_turn("sess-c4"); + tracker.record("sess-c4", &payload_with_cache(1500, 300, Some(0))); + let usage = tracker.take().expect("pending"); + + assert!(usage.delta_reliable); + assert_eq!( + usage.turn_cache_read_tokens, + Some(0), + "explicit zero on both sides → Some(0), not None" + ); + assert_eq!(usage.cumulative_cache_read_tokens, Some(0)); + } + + #[test] + fn cache_read_threads_through_setup_notification_baseline() { + // A setup notification (before begin_turn) with a nonzero cache count + // must update the committed baseline so the first real turn gets a + // correct delta from that starting point. + let mut tracker = UsageTracker::default(); + + // Setup notification: cumulative cache = 300. + tracker.record("sess-c5", &payload_with_cache(1000, 200, Some(300))); + + tracker.begin_turn("sess-c5"); + tracker.record("sess-c5", &payload_with_cache(1500, 350, Some(700))); + let usage = tracker.take().expect("pending"); + + assert!(usage.delta_reliable, "baseline from setup: reliable"); + assert_eq!( + usage.turn_cache_read_tokens, + Some(400), + "cache delta from setup baseline: 700 - 300 = 400" + ); + assert_eq!(usage.cumulative_cache_read_tokens, Some(700)); + } + + #[test] + fn cache_read_omitted_field_produces_none_cumulative_and_no_turn_delta() { + // A harness that omits accumulatedCachedInputTokens (e.g. goose) must + // produce None cumulative_cache_read_tokens — not Some(0) — and the + // turn delta must also be None even on the second turn. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-c6"); + // payload() uses None for accumulated_cached_input_tokens. + tracker.record("sess-c6", &payload(1000, 200, None)); + let t1 = tracker.take().expect("turn 1"); + + assert!( + t1.cumulative_cache_read_tokens.is_none(), + "goose-shaped payload: cumulative must be None, not Some(0)" + ); + assert!( + t1.turn_cache_read_tokens.is_none(), + "first turn always has no turn delta" + ); + + tracker.begin_turn("sess-c6"); + tracker.record("sess-c6", &payload(1500, 300, None)); + let t2 = tracker.take().expect("turn 2"); + + assert!( + t2.cumulative_cache_read_tokens.is_none(), + "continued goose session: cumulative must remain None" + ); + assert!( + t2.turn_cache_read_tokens.is_none(), + "absent field on both sides → no turn delta invented" + ); + assert!( + t2.delta_reliable, + "input/output reliability unaffected by absent cache field" + ); + } + + #[test] + fn cache_read_baseline_absent_then_present_produces_no_delta() { + // If the first turn omits the cache field (baseline stored as None) and + // the second turn reports a value, no delta can be computed — we have no + // baseline to subtract from. The cumulative value should still pass through. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-c7"); + tracker.record("sess-c7", &payload(1000, 200, None)); // no cache field + let _ = tracker.take(); + + tracker.begin_turn("sess-c7"); + tracker.record("sess-c7", &payload_with_cache(1500, 300, Some(400))); + let usage = tracker.take().expect("turn 2"); + + assert!( + usage.turn_cache_read_tokens.is_none(), + "absent baseline → no turn delta even when current has a value" + ); + assert_eq!( + usage.cumulative_cache_read_tokens, + Some(400), + "cumulative from current payload passes through" + ); + assert!(usage.delta_reliable, "input/output reliability unaffected"); + } + + #[test] + fn cache_read_baseline_present_then_absent_produces_no_delta() { + // If the first turn reports the cache field but the second omits it + // (harness switched), no delta should be produced and cumulative is None. + let mut tracker = UsageTracker::default(); + tracker.begin_turn("sess-c8"); + tracker.record("sess-c8", &payload_with_cache(1000, 200, Some(300))); + let _ = tracker.take(); + + tracker.begin_turn("sess-c8"); + tracker.record("sess-c8", &payload(1500, 300, None)); // no cache field + let usage = tracker.take().expect("turn 2"); + + assert!( + usage.turn_cache_read_tokens.is_none(), + "absent current → no turn delta" + ); + assert!( + usage.cumulative_cache_read_tokens.is_none(), + "absent field: cumulative must be None" + ); + assert!(usage.delta_reliable, "input/output reliability unaffected"); + } + + #[test] + fn pool_omitted_cache_field_publishes_no_cache_read_tokens_in_kind44200() { + // End-to-end: a buzz-agent or goose payload that omits the cache field + // must NOT publish cacheReadTokens in the kind:44200 event — neither + // in turn nor cumulative counts. + // + // This is the core acceptance test for Thufir's finding: the old code + // would publish cacheReadTokens: 0 for every harness regardless of + // whether the field was reported. + use crate::pool::build_turn_metric_counts; + + let usage = TurnUsage { + session_id: "sess-pool-none".into(), + turn_seq: 2, + delta_reliable: true, + turn_input_tokens: Some(400), + turn_output_tokens: Some(100), + turn_total_tokens: None, + turn_cost_usd: None, + turn_cache_read_tokens: None, + cumulative_input_tokens: 700, + cumulative_output_tokens: 200, + cumulative_total_tokens: None, + cumulative_cost_usd: None, + cumulative_cache_read_tokens: None, // harness did not report the field + model: None, + }; + + let (turn_counts, cumulative_counts) = build_turn_metric_counts(&usage); + + let turn = turn_counts.expect("turn counts must be present (delta reliable)"); + assert!( + turn.cache_read_tokens.is_none(), + "omitted cache field: turn cacheReadTokens must be absent from kind:44200" + ); + + let cumulative = cumulative_counts.expect("cumulative counts always present"); + assert!( + cumulative.cache_read_tokens.is_none(), + "omitted cache field: cumulative cacheReadTokens must be absent from kind:44200" + ); + } + + #[test] + fn pool_reported_cache_field_publishes_nonzero_cache_read_tokens_in_kind44200() { + // End-to-end: a buzz-agent payload with a nonzero cache count must + // publish cacheReadTokens in both turn and cumulative counts. + use crate::pool::build_turn_metric_counts; + + let usage = TurnUsage { + session_id: "sess-pool-some".into(), + turn_seq: 2, + delta_reliable: true, + turn_input_tokens: Some(400), + turn_output_tokens: Some(100), + turn_total_tokens: None, + turn_cost_usd: None, + turn_cache_read_tokens: Some(300), + cumulative_input_tokens: 700, + cumulative_output_tokens: 200, + cumulative_total_tokens: None, + cumulative_cost_usd: None, + cumulative_cache_read_tokens: Some(600), + model: None, + }; + + let (turn_counts, cumulative_counts) = build_turn_metric_counts(&usage); + + let turn = turn_counts.expect("turn counts present"); + assert_eq!( + turn.cache_read_tokens, + Some(300), + "nonzero turn cache: must appear in kind:44200 turn counts" + ); + + let cumulative = cumulative_counts.expect("cumulative counts present"); + assert_eq!( + cumulative.cache_read_tokens, + Some(600), + "nonzero cumulative cache: must appear in kind:44200 cumulative counts" + ); + } } diff --git a/crates/buzz-agent/README.md b/crates/buzz-agent/README.md index a2504db451..5d942777d5 100644 --- a/crates/buzz-agent/README.md +++ b/crates/buzz-agent/README.md @@ -21,9 +21,9 @@ HTTPS │ ▼ - Anthropic Messages API - or any OpenAI-compat - (vLLM, llama.cpp, OpenRouter, + Anthropic Messages API, + OpenRouter, or any OpenAI-compat + (vLLM, llama.cpp, Databricks, Block Gateway, Ollama, …) ``` @@ -50,6 +50,12 @@ OPENAI_COMPAT_MODEL=gpt-5 \ OPENAI_COMPAT_BASE_URL=https://api.openai.com/v1 \ ./target/release/buzz-agent +# Or OpenRouter +BUZZ_AGENT_PROVIDER=openrouter \ +OPENROUTER_API_KEY=sk-or-v1-... \ +OPENROUTER_MODEL=anthropic/claude-sonnet-4.5 \ + ./target/release/buzz-agent + # Or Databricks model serving via OAuth 2.0 PKCE BUZZ_AGENT_PROVIDER=databricks \ DATABRICKS_HOST=https://dbc-...cloud.databricks.com \ @@ -129,15 +135,18 @@ Everything is environment variables. No flags, no config files. (We are a subpro | Variable | Default | Notes | |---|---|---| -| `BUZZ_AGENT_PROVIDER` | — | Required. `anthropic`, `openai`, `databricks`, or `databricks_v2`. No implicit fallback — the agent errors at startup when this is unset. | +| `BUZZ_AGENT_PROVIDER` | — | Required. `anthropic`, `openai`, `openrouter`, `databricks`, or `databricks_v2`. No implicit fallback — the agent errors at startup when this is unset. | | `ANTHROPIC_API_KEY` | — | Required when provider=anthropic. | | `ANTHROPIC_MODEL` | — | Required when provider=anthropic. | | `ANTHROPIC_BASE_URL` | `https://api.anthropic.com` | | | `ANTHROPIC_API_VERSION` | `2023-06-01` | | | `OPENAI_COMPAT_API_KEY` | — | Required when provider=openai. | | `OPENAI_COMPAT_MODEL` | — | Required when provider=openai. | -| `OPENAI_COMPAT_BASE_URL` | `https://api.openai.com/v1` | Point at vLLM, llama.cpp, OpenRouter, Ollama, etc. | +| `OPENAI_COMPAT_BASE_URL` | `https://api.openai.com/v1` | Point at vLLM, llama.cpp, Ollama, etc. | | `OPENAI_COMPAT_API` | `auto` | `auto` \| `chat` \| `responses`. `auto` picks Responses for `*.openai.com`, Chat Completions everywhere else. | +| `OPENROUTER_API_KEY` | — | Required when provider=openrouter. | +| `OPENROUTER_MODEL` | — | Required when provider=openrouter. Use OpenRouter's `vendor/model` id, e.g. `anthropic/claude-sonnet-4.5`. | +| `OPENROUTER_BASE_URL` | `https://openrouter.ai/api/v1` | | | `DATABRICKS_HOST` | — | Required when provider=databricks or provider=databricks_v2. | | `DATABRICKS_MODEL` | — | Required when provider=databricks or provider=databricks_v2. | | `DATABRICKS_TOKEN` | — | Optional static bearer escape hatch. If unset, Databricks uses browser OAuth + refresh cache. | @@ -154,6 +163,67 @@ Everything is environment variables. No flags, no config files. (We are a subpro | `BUZZ_AGENT_MAX_LINE_BYTES` | `4194304` | 4 MiB. Hard cap on inbound JSON-RPC frames. | | `BUZZ_AGENT_MAX_HISTORY_BYTES` | `1048576` | 1 MiB. Old turns are evicted past this. | | `BUZZ_AGENT_MAX_TOOL_RESULT_TEXT_BYTES` | `51200` | 50 KiB. Per-result cap on tool-output text; oversize is middle-elided (head + tail kept) with an inline marker. Images are exempt. | +| `BUZZ_AGENT_REQUIRE_REPLY` | `0` (`1` on mesh) | `1` enables the [reply guard](#reply-guard) — remind the model to publish when a turn is about to end with nothing posted to Buzz. Desktop defaults it to `1` for Buzz shared-compute agents. | + + +## Reply Guard + +Off by default, except on Buzz shared-compute (mesh) agents, where Buzz Desktop +sets `BUZZ_AGENT_REQUIRE_REPLY=1` automatically. With it enabled, a turn that is +about to end without any recognized attempt to post to Buzz gets a reminder that +its assistant text is invisible to humans, and is rerolled. + +This exists because a Buzz agent's reasoning and tool output are not shown to +anyone. A turn that does real work and never posts is a silent failure — the +requester waits on a result that was produced and thrown away. + +Mesh agents get it by default because they run on small local models, which are +the ones most likely to do the work and then end the turn without publishing it. +Setting `BUZZ_AGENT_REQUIRE_REPLY=0` on the agent, persona, or global env opts a +mesh agent back out; the default never overrides an explicit value. + +**Advisory, never a trap.** At most two reminders, then the turn ends whether or +not anything was published. The guard catches accidental omission; it does not +compel speech. The reminder text explicitly licenses silence, because the +built-in system prompt says publishing is optional and silence is often the +correct outcome. + +**Recognition contract.** A turn counts as having replied when it issues a call +that: + +- resolves to a registered, non-hook tool (a hallucinated tool name is rejected + at preflight and never runs, so it must not disarm the guard), +- whose qualified name ends in `__shell` — i.e. the bare tool name is exactly + `shell`, which is `buzz-dev-mcp`'s shell tool and any other server's, and +- whose `command` argument contains `messages send` or `reactions add`. + +`messages send` also covers `messages send-diff`. Reactions count because the +built-in prompt directs agents to react rather than post a bare +acknowledgement, so nagging an agent that reacted would punish documented +behavior. + +Detection is checked **after** the per-turn tool-call cap +(`MAX_TOOL_CALLS_PER_TURN`) is applied: a publish-shaped call that was discarded +never ran. + +**It recognizes an attempt, not a successful publish.** Only the command text is +inspected, never the exit status. A send that fails still satisfies the guard — +which is fine, since a failed send already returns a non-zero exit and error +JSON to the model, louder feedback than a reminder. + +**Known limits**, both deliberate. A command assembled at runtime (`$CMD`) or +buried in a wrapper script is missed, so that turn is reminded despite having +posted. Text that merely quotes a send (`echo "buzz messages send"`) matches, so +that turn is not reminded. Missing a real post is the expensive direction, and +substring matching is the forgiving one there. Neither edge is pinned by a test; +the matcher is free to improve. + +**Budget.** Reminders ride the existing `_Stop` gate and share +`BUZZ_AGENT_STOP_MAX_REJECTIONS` — the outer cap on every end-turn objection. +At the default 3 both reminders fit; at 1 only one does; at 0 the guard is off +along with the hooks. A round carrying both a `_Stop` hook objection and a +reminder costs one rejection and delivers both texts. This is not a new +lifecycle hook — see [MCP_DRIVEN_HOOKS.md](../../docs/MCP_DRIVEN_HOOKS.md). ## Providers @@ -167,17 +237,24 @@ Everything is environment variables. No flags, no config files. (We are a subpro | vLLM | `openai` | `POST {base}/chat/completions` | any tool-calling model | | llama.cpp | `openai` | `POST {base}/chat/completions` | any tool-calling GGUF | | Ollama | `openai` | `POST {base}/chat/completions` | llama3.1, qwen2.5-coder | -| OpenRouter | `openai` | `POST {base}/chat/completions` | anything they route | | Block Gateway | `openai` | `POST {base}/chat/completions` | gpt-5, claude | +| OpenRouter | `openrouter` | `POST {base}/chat/completions` | anything they route (extended-thinking replay, provider-agnostic tool calling) | | Databricks | `databricks` | `POST {host}/serving-endpoints/{model}/invocations` | goose-claude-4-6-sonnet | | Databricks AI Gateway v2 | `databricks_v2` | `POST {host}/ai-gateway/{provider}/v1/...` | databricks-gpt-5-5, databricks-claude-opus-4-7 | -If `BUZZ_AGENT_PROVIDER=anthropic` is selected without `ANTHROPIC_API_KEY`, or `BUZZ_AGENT_PROVIDER=openai` is selected without `OPENAI_COMPAT_API_KEY`, the agent returns an error — there is no implicit fallback to another provider. +If `BUZZ_AGENT_PROVIDER=anthropic` is selected without `ANTHROPIC_API_KEY`, `BUZZ_AGENT_PROVIDER=openai` is selected without `OPENAI_COMPAT_API_KEY`, or `BUZZ_AGENT_PROVIDER=openrouter` is selected without `OPENROUTER_API_KEY`, the agent returns an error — there is no implicit fallback to another provider. `provider=openai` speaks two HTTP dialects: the [Responses API](https://platform.openai.com/docs/api-reference/responses) (`/v1/responses`, required for GPT-5 / o-series tool-calling on OpenAI's own service) and the [Chat Completions API](https://platform.openai.com/docs/api-reference/chat) (`/chat/completions`, the broadly-supported OpenAI-compatible wire format). By default (`OPENAI_COMPAT_API=auto`) the agent picks **Responses** when `OPENAI_COMPAT_BASE_URL` points at an `*.openai.com` host and **Chat Completions** everywhere else. Pin the choice explicitly with `OPENAI_COMPAT_API=chat` or `OPENAI_COMPAT_API=responses` for providers that diverge from the default (e.g. a Responses-compatible self-hosted gateway). +`provider=openrouter` is first-class, not routed through `provider=openai`: it speaks OpenAI's Chat Completions wire format but with OpenRouter-specific extensions layered on top — + +- `reasoning.effort` is set on the request when reasoning effort is configured. The request deliberately carries no `provider.require_parameters` filter: that filter routes only to endpoints advertising every parameter in the body, and 83 of 274 tools-capable OpenRouter models do not advertise `reasoning`, so it turns an effort setting into a hard 404 on a valid model id. A model that cannot reason answers without reasoning instead. +- The response's `reasoning_details` array (opaque extended-thinking payload) is captured and replayed byte-for-byte on the next turn's assistant message, so multi-turn tool use keeps the model's chain-of-thought. +- `anthropic/*` models get Anthropic-style `cache_control` breakpoints injected on the system message and the last two user messages. +- Retryable statuses (429 and typed `provider_overloaded` 503) honor the documented `Retry-After` header (clamped to a small ceiling — see `RETRY_AFTER_CAP_SECS` in `llm.rs` — since the sleep happens outside `BUZZ_AGENT_LLM_TIMEOUT_SECS`); 502 and untyped 503 retry with jittered backoff instead. `401` is treated as an expired/invalid key and refreshed once, while `402` (no credits) and `403` (guardrail/moderation/permission) fail immediately without retry. + `Provider` is a Rust `enum` with one `match` in `Llm::complete`. There is no trait, no `Box`, no async-trait. Adding a provider is a `match` arm and one `body`/`parse` pair in `llm.rs`. ## MCP Servers diff --git a/crates/buzz-agent/src/agent.rs b/crates/buzz-agent/src/agent.rs index 730e87b2e8..8e14fee195 100644 --- a/crates/buzz-agent/src/agent.rs +++ b/crates/buzz-agent/src/agent.rs @@ -14,13 +14,87 @@ use crate::mcp::ResultBudget; use crate::types::{ AgentError, ContentBlock, HistoryItem, ProviderStop, StopReason, ToolCall, ToolResult, - ToolResultContent, + ToolResultContent, TurnTotalState, }; use crate::wire::{self, WireSender}; const ERROR_REFLECTION_SUFFIX: &str = "\n\n[Reflect] Before retrying, identify the cause and change your approach."; +/// Maximum reply reminders emitted per prompt when `require_reply` is on. +/// +/// After this many, the turn is allowed to end whether or not anything was +/// published: the guard exists to catch accidental omission, not to compel +/// speech. The shared `stop_max_rejections` budget can cut this lower — see +/// [`Config::require_reply`](crate::config::Config::require_reply). +const MAX_REPLY_NAGS: u32 = 2; + +/// Server label on the synthetic reply-guard objection. +/// +/// Not a real MCP server. It rides the same tool-result path as `_Stop` hook +/// output, so the model sees `{hook, server, text}` attribution naming the +/// in-process guard rather than an MCP server that could be impersonated. +const REPLY_GUARD_SERVER: &str = "buzz-agent"; + +/// Reminder text emitted when a turn is about to end with nothing published. +/// +/// Explicitly licenses silence. The base prompt tells agents that publishing is +/// optional and "silence is usually correct"; a reminder that argued otherwise +/// would fight that instruction and make agents chattier. +const REPLY_GUARD_NAG: &str = "You are about to end this turn without calling `buzz messages send`. \ +Your assistant text and reasoning are never shown to anyone — if you did work, found an answer, \ +or hit a blocker that someone is waiting on, it exists only if you publish it. \ +If you already posted, or if silence is genuinely correct for this turn, ignore this and end your turn."; + +/// Whether `call` is a recognized attempt to publish a reply to Buzz. +/// +/// Recognizes an *attempt*, not a successful publish: the command text is +/// inspected, never the exit status. That is deliberate — a send that fails +/// already returns a non-zero exit and error JSON to the model, which is louder +/// feedback than the reminder this gates. +/// +/// `has` + `!is_hook` are the same checks the dispatcher uses to accept a call +/// (see `execute_calls`), so a hallucinated `fake__shell` — rejected at preflight +/// and never executed — cannot disarm the guard. They must stay *before* +/// [`is_reply_shaped`]: together with them, and only with them, the `__shell` +/// suffix is exactly equivalent to "the bare tool name is `shell`". +fn is_buzz_reply_call(call: &ToolCall, mcp: &McpRegistry) -> bool { + mcp.has(&call.name) && !mcp.is_hook(&call.name) && is_reply_shaped(&call.name, &call.arguments) +} + +/// Whether a tool name and arguments have the shape of a Buzz publish command. +/// +/// Split from [`is_buzz_reply_call`] only so the matcher is testable without a +/// live [`McpRegistry`]; callers must apply the registry checks first. +/// +/// On the name: `ends_with("__shell")` is exact rather than approximate *given* +/// those checks. Registration rejects `__` in both server names and bare tool +/// names, and qualified names are `{server}__{bare}`, so a trailing `__shell` can +/// only straddle the separator if the bare name starts with `_` — which `is_hook` +/// already excludes. Dropping the separator would not be exact: `powershell` and +/// `noshell` both end in `shell`. +/// +/// On the command: a deliberately coarse substring test, scoped to the structured +/// `command` field so unrelated metadata — a `description` that quotes a send — +/// cannot suppress the guard, and a non-string `command` is rejected rather than +/// coerced. Known limits, both accepted: a command assembled at runtime (`$CMD`) +/// or hidden in a wrapper script is missed, and text that merely quotes a send +/// (`echo "buzz messages send"`) matches. Missing a real post is the expensive +/// direction, and substring matching is the more forgiving one there. +fn is_reply_shaped(name: &str, arguments: &serde_json::Value) -> bool { + name.ends_with("__shell") + && arguments + .get("command") + .and_then(|v| v.as_str()) + .is_some_and(|cmd| { + // `messages send` also covers `messages send-diff`. `reactions + // add` counts because the base prompt directs agents to react + // rather than post a bare acknowledgement, so nagging an agent + // that reacted would punish documented-correct behavior. + cmd.contains("messages send") || cmd.contains("reactions add") + }) +} + pub struct RunCtx<'a> { pub cfg: &'a Config, /// Effective model for this session. Usually equals `cfg.model`; overridden @@ -60,6 +134,22 @@ pub struct RunCtx<'a> { /// Accumulated output tokens across all LLM rounds in this turn, for /// NIP-AM metric publishing. Reset to `None` at turn start in `run()`. pub turn_output_tokens: &'a mut Option, + /// The cache-served subset of `turn_input_tokens`, accumulated across all + /// LLM rounds in this turn. Reset to `None` at turn start in `run()`. + /// Consumers price this slice at the provider's cached rate; without it + /// every round of a growing conversation is billed at full price. + pub turn_cached_input_tokens: &'a mut Option, + /// Tri-state total-token accumulator for this turn. + /// + /// - `Unseen`: no usage-bearing response observed yet this turn (initial state). + /// - `Exact(n)`: every usage-bearing response so far reported a genuine + /// provider total; `n` is their sum. + /// - `Unknown`: at least one usage-bearing response lacked a provider total; + /// this turn can never produce a reliable total. + /// + /// Reset to `Unseen` at turn start in `run()`. Callers must not derive a + /// total by summing input+output — that is the UI display approximation only. + pub turn_total_state: &'a mut TurnTotalState, } impl RunCtx<'_> { @@ -78,12 +168,22 @@ impl RunCtx<'_> { // Reset per-turn token accumulators for this prompt. *self.turn_input_tokens = None; *self.turn_output_tokens = None; + *self.turn_cached_input_tokens = None; + *self.turn_total_state = TurnTotalState::Unseen; let mut round = 0u32; // Per-prompt `_Stop` objection count. Bounded per prompt (not per // session) so a stubborn exchange can't permanently disable the stop // guard for a long-lived session; `max_rounds` still caps the loop. let mut stop_rejections = 0u32; + // Reply-guard state for this prompt. `prompt()` *is* the turn, so + // locals here are per-turn by construction — same shape as + // `stop_rejections` above. + // + // Named for what it proves: a *recognized attempt* to publish, not a + // successful publish. See `is_buzz_reply_call`. + let mut buzz_reply_call_seen = false; + let mut reply_nags = 0u32; loop { if self.cfg.max_rounds > 0 && round >= self.cfg.max_rounds { return Ok(StopReason::MaxTurnRequests); @@ -175,6 +275,31 @@ impl RunCtx<'_> { *self.turn_output_tokens = Some(self.turn_output_tokens.unwrap_or(0).saturating_add(out)); } + // Accumulate the cache-served subset of this turn's input. Tracked + // separately from `turn_input_tokens` rather than subtracted from + // it: the input total must stay inclusive for the handoff gate, + // which cares how much context was sent, not what it cost. + if let Some(cached) = response.cached_input_tokens { + *self.turn_cached_input_tokens = Some( + self.turn_cached_input_tokens + .unwrap_or(0) + .saturating_add(cached), + ); + } + // Fold the provider-reported total into the turn tri-state, but only + // when this response was usage-bearing (had input or output tokens). + // A response with no usage at all is not evidence of a missing total + // and must not poison the accumulator. + // + // Shape assumption: documented OpenAI-compatible responses that carry + // `total_tokens` always co-report at least one of `prompt_tokens` / + // `completion_tokens`. A response that supplies only `total_tokens` + // with neither category is therefore not a supported shape and would + // be silently ignored here. If that shape is ever encountered, extend + // this gate rather than representing absent categories as zero. + if response.input_tokens.is_some() || response.output_tokens.is_some() { + *self.turn_total_state = self.turn_total_state.fold(response.total_tokens); + } if !response.reasoning.is_empty() { wire::send( @@ -213,6 +338,7 @@ impl RunCtx<'_> { self.history.push(HistoryItem::Assistant { text: response.text, tool_calls: Vec::new(), + reasoning_details: response.reasoning_details.clone(), }); let stop = map_stop(response.stop); // Only gate genuine end_turn — don't override max_tokens/refusal. @@ -220,7 +346,7 @@ impl RunCtx<'_> { if stop_rejections >= self.cfg.stop_max_rejections { return Ok(stop); } - let objections = self + let mut objections = self .mcp .call_hooks( "_Stop", @@ -229,6 +355,17 @@ impl RunCtx<'_> { &self.cfg.hook_servers, ) .await; + // Reply guard shares this gate and this budget, so a round + // carrying both a hook objection and a reply reminder costs + // one rejection and delivers both texts. + if self.cfg.require_reply + && !buzz_reply_call_seen + && reply_nags < MAX_REPLY_NAGS + { + reply_nags += 1; + objections + .push((REPLY_GUARD_SERVER.to_string(), REPLY_GUARD_NAG.to_string())); + } if !objections.is_empty() { stop_rejections = stop_rejections.saturating_add(1); push_hook_outputs_as_tool_results(self.history, "_Stop", &objections); @@ -246,9 +383,15 @@ impl RunCtx<'_> { ); calls.truncate(MAX_TOOL_CALLS_PER_TURN); } + // Deliberately after truncation: a publish-shaped call that was + // discarded never runs, so it must not suppress the reminder. + if self.cfg.require_reply && !buzz_reply_call_seen { + buzz_reply_call_seen = calls.iter().any(|c| is_buzz_reply_call(c, self.mcp)); + } self.history.push(HistoryItem::Assistant { text: response.text, tool_calls: calls.clone(), + reasoning_details: response.reasoning_details, }); if let Some(stop) = self.execute_calls(&calls).await { @@ -679,7 +822,11 @@ pub(crate) fn push_hook_outputs_as_tool_results( provider_id: provider_id.clone(), name: tool_name, arguments: serde_json::json!({}), + // Synthesised locally, so there is no provider wire form to + // preserve. + provider_extra: Default::default(), }], + reasoning_details: None, }); history.push(HistoryItem::ToolResult(ToolResult { provider_id, @@ -744,3 +891,158 @@ fn map_stop(p: ProviderStop) -> StopReason { ProviderStop::Refusal => StopReason::Refusal, } } + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + /// The shapes the guard must recognize as a publish attempt. Callers apply + /// the registry checks first; these cover the name suffix and command text. + #[test] + fn reply_shape_matches_documented_send_forms() { + for cmd in [ + "buzz messages send --channel X --content Y", + "buzz --relay wss://r messages send --channel X --content Y", + "/abs/path/buzz messages send", + "printf 'hi' | buzz messages send --content -", + "buzz messages send-diff --diff -", + "buzz reactions add --event E --emoji +", + // Assembled through another shell: rev 3's tokenizer missed this. + r#"sh -c "buzz messages send --channel X""#, + ] { + assert!( + is_reply_shaped("dev__shell", &json!({ "command": cmd })), + "expected {cmd:?} to count as a publish attempt" + ); + } + } + + /// Commands that do real work but do not reply in the originating + /// conversation must still be nagged. + #[test] + fn reply_shape_rejects_non_reply_commands() { + for cmd in [ + "buzz messages get --channel X", + "buzz channels list", + "buzz reactions remove --event E", + "buzz pr open --title T", + "buzz social publish --content hi", + "buzz notes set --name n", + "cargo test -p buzz-agent", + ] { + assert!( + !is_reply_shaped("dev__shell", &json!({ "command": cmd })), + "expected {cmd:?} not to count as a publish attempt" + ); + } + } + + /// The `__` separator is load-bearing: `ends_with("shell")` alone would + /// accept any registered tool whose name merely ends in those letters, and + /// `has()` proves registration, not the bare name. + #[test] + fn reply_shape_requires_the_qname_separator() { + let args = json!({ "command": "buzz messages send --channel X" }); + for name in [ + "dev__powershell", + "dev__noshell", + "shell", + "dev__send_message", + ] { + assert!( + !is_reply_shaped(name, &args), + "{name} must not satisfy the shell-tool check" + ); + } + assert!(is_reply_shaped("dev__shell", &args)); + assert!(is_reply_shaped("buzz-dev-mcp__shell", &args)); + } + + /// Only the field that carries the executable command counts. Searching + /// serialized arguments instead would let arbitrary metadata disarm the + /// guard, turning a description into an attempted send. + #[test] + fn reply_shape_reads_only_the_command_field() { + assert!(!is_reply_shaped( + "dev__shell", + &json!({ "description": "buzz messages send --channel X" }) + )); + assert!(!is_reply_shaped( + "dev__shell", + &json!({ "workdir": "buzz messages send" }) + )); + // Malformed `command` is rejected, not coerced — and must not panic. + assert!(!is_reply_shaped("dev__shell", &json!({ "command": 42 }))); + assert!(!is_reply_shaped("dev__shell", &json!({ "command": null }))); + assert!(!is_reply_shaped("dev__shell", &json!({}))); + assert!(!is_reply_shaped("dev__shell", &json!("not an object"))); + } + + /// A9 regression: `reasoning_details` contributes real bytes to + /// `estimated_bytes` (see `types.rs::HistoryItem::size_with`), so a + /// history item carrying a large opaque reasoning array must actually + /// drive `truncate_history` eviction — not be silently invisible to the + /// sizing gate that decides what survives a turn. + #[test] + fn truncate_history_evicts_oldest_turn_with_reasoning_details() { + let big_reasoning = json!([{ "type": "reasoning.text", "text": "x".repeat(400) }]); + let mut history = vec![ + HistoryItem::User("first question".into()), + HistoryItem::Assistant { + text: "first answer".into(), + tool_calls: vec![], + reasoning_details: Some(big_reasoning), + }, + HistoryItem::User("second question".into()), + HistoryItem::Assistant { + text: "second answer".into(), + tool_calls: vec![], + reasoning_details: None, + }, + ]; + + let total_before: usize = history.iter().map(HistoryItem::estimated_bytes).sum(); + // Budget below the total but above the second (smaller) turn alone, + // so only the oldest user+assistant pair — the one carrying + // reasoning_details — must be dropped. + let max_bytes = total_before - 100; + assert!( + max_bytes > 0, + "test fixture must leave room to evict only one turn" + ); + + truncate_history(&mut history, max_bytes); + + assert_eq!( + history.len(), + 2, + "the oldest user+assistant turn (with reasoning_details) must be evicted" + ); + assert!(matches!(&history[0], HistoryItem::User(s) if s == "second question")); + assert!( + matches!(&history[1], HistoryItem::Assistant { text, .. } if text == "second answer") + ); + let total_after: usize = history.iter().map(HistoryItem::estimated_bytes).sum(); + assert!(total_after <= max_bytes); + } + + #[test] + fn truncate_history_noop_when_under_budget() { + let mut history = vec![ + HistoryItem::User("hi".into()), + HistoryItem::Assistant { + text: "hello".into(), + tool_calls: vec![], + reasoning_details: None, + }, + ]; + let original_len = history.len(); + truncate_history(&mut history, 1_000_000); + assert_eq!( + history.len(), + original_len, + "under budget must not evict anything" + ); + } +} diff --git a/crates/buzz-agent/src/config.rs b/crates/buzz-agent/src/config.rs index 864fa7d237..afbda5379d 100644 --- a/crates/buzz-agent/src/config.rs +++ b/crates/buzz-agent/src/config.rs @@ -671,6 +671,8 @@ pub enum Provider { /// Databricks AI Gateway v2. Routes by model family through the gateway's /// OpenAI Responses, Anthropic Messages, or MLflow Chat Completions paths. DatabricksV2, + /// OpenRouter multi-provider gateway. Routes to `{base_url}/chat/completions` with bearer auth. Wire format is OpenAI-chat-compatible. + OpenRouter, } /// Which OpenAI-family HTTP API to call. Set via `OPENAI_COMPAT_API` @@ -718,6 +720,16 @@ pub struct Config { /// Maximum `_Stop` rejections per prompt. Default 3. Set to 0 to /// disable `_Stop` hooks entirely (agent always honors end_turn). pub stop_max_rejections: u32, + /// Remind the model to publish when a turn is about to end without any + /// recognized attempt to post to Buzz. Default off; opt in per agent with + /// `BUZZ_AGENT_REQUIRE_REPLY=1`. + /// + /// Advisory only: at most `MAX_REPLY_NAGS` reminders (see `agent.rs`), + /// then the turn ends regardless. Bounded by the same + /// `stop_max_rejections` budget as `_Stop` hooks, which is the outer cap on + /// all end-turn objections — at the default 3 both reminders fit; at 1 only + /// one does; at 0 the guard is off with the hooks. + pub require_reply: bool, /// Hook server allowlist. See [`HookServers`] for variant semantics. /// Default (env unset/empty) is `None` — hooks are off unless the /// operator explicitly opts in. @@ -738,6 +750,14 @@ pub struct Config { /// Thinking/reasoning effort level. `None` = use provider default (no /// thinking config sent). Set via `BUZZ_AGENT_THINKING_EFFORT`. pub thinking_effort: Option, + /// Emit Anthropic `cache_control` breakpoints on the stable prefix + /// (tools + system prompt) and the rolling conversation tail. Default on; + /// disable with `BUZZ_AGENT_PROMPT_CACHING=0`. Consulted on every route that + /// speaks the Anthropic caching dialect: first-party Anthropic, the + /// DatabricksV2 Claude route, and OpenRouter's `anthropic/*` models. The + /// Databricks gateway does not auto-cache, so without this the surfaced + /// `cache_read_input_tokens` is structurally always 0. + pub prompt_caching: bool, } impl Config { @@ -748,6 +768,7 @@ impl Config { env("BUZZ_AGENT_PROVIDER").as_deref(), env("ANTHROPIC_API_KEY").as_deref(), env("OPENAI_COMPAT_API_KEY").as_deref(), + env("OPENROUTER_API_KEY").as_deref(), )?; // Universal model override — takes priority over provider-specific model @@ -790,6 +811,16 @@ impl Config { databricks_host.ok_or_else(|| "config: DATABRICKS_HOST required".to_string())?, OpenAiApi::Chat, // only read by OpenAI/legacy Databricks dispatch ), + Provider::OpenRouter => ( + req("OPENROUTER_API_KEY")?, + resolve_model( + buzz_agent_model.as_deref(), + env("OPENROUTER_MODEL").as_deref(), + ) + .ok_or_else(|| "config: OPENROUTER_MODEL required".to_string())?, + env_or("OPENROUTER_BASE_URL", "https://openrouter.ai/api/v1"), + OpenAiApi::Chat, // OpenRouter uses Chat Completions only + ), }; let system_prompt = match (env("BUZZ_AGENT_SYSTEM_PROMPT"), env("BUZZ_AGENT_SYSTEM_PROMPT_FILE")) { (Some(_), Some(_)) => return Err( @@ -830,9 +861,11 @@ impl Config { max_parallel_tools: parse_env("BUZZ_AGENT_MAX_PARALLEL_TOOLS", 8usize)?, hook_timeout: Duration::from_millis(parse_env("BUZZ_AGENT_HOOK_TIMEOUT_MS", 2500u64)?), stop_max_rejections: parse_env("BUZZ_AGENT_STOP_MAX_REJECTIONS", 3u32)?, + require_reply: parse_env("BUZZ_AGENT_REQUIRE_REPLY", 0u8)? != 0, hook_servers: parse_hook_servers_env("MCP_HOOK_SERVERS"), hints_enabled: parse_env("BUZZ_AGENT_NO_HINTS", 0u8)? == 0, thinking_effort: parse_thinking_effort(env("BUZZ_AGENT_THINKING_EFFORT").as_deref())?, + prompt_caching: parse_env("BUZZ_AGENT_PROMPT_CACHING", 1u8)? != 0, }; cfg.validate()?; Ok(cfg) @@ -871,9 +904,11 @@ impl Config { max_parallel_tools: 1, hook_timeout: Duration::from_secs(1), stop_max_rejections: 0, + require_reply: false, hook_servers: HookServers::None, hints_enabled: false, thinking_effort: None, + prompt_caching: false, } } @@ -993,6 +1028,7 @@ fn resolve_provider( requested: Option<&str>, anthropic_key: Option<&str>, openai_key: Option<&str>, + openrouter_key: Option<&str>, ) -> Result { match requested.map(str::trim).filter(|s| !s.is_empty()) { Some(raw) => { @@ -1008,6 +1044,8 @@ fn resolve_provider( ), "databricks" => Ok(Provider::Databricks), "databricks_v2" | "databricks-v2" => Ok(Provider::DatabricksV2), + "openrouter" if present_nonempty(openrouter_key) => Ok(Provider::OpenRouter), + "openrouter" => Err("config: OPENROUTER_API_KEY required".into()), _ => Err(format!( "config: BUZZ_AGENT_PROVIDER={raw} not supported" )), @@ -1225,11 +1263,11 @@ mod tests { #[test] fn resolve_provider_keeps_requested_provider_when_token_present() { assert_eq!( - resolve_provider(Some("anthropic"), Some("sk-ant"), None,).unwrap(), + resolve_provider(Some("anthropic"), Some("sk-ant"), None, None).unwrap(), Provider::Anthropic ); assert_eq!( - resolve_provider(Some("openai"), None, Some("sk-openai"),).unwrap(), + resolve_provider(Some("openai"), None, Some("sk-openai"), None).unwrap(), Provider::OpenAi ); } @@ -1237,17 +1275,17 @@ mod tests { #[test] fn resolve_provider_errors_when_requested_provider_key_missing() { // No fallback — missing key returns an error regardless of Databricks availability. - let err = resolve_provider(Some("anthropic"), None, None).unwrap_err(); + let err = resolve_provider(Some("anthropic"), None, None, None).unwrap_err(); assert!(err.contains("ANTHROPIC_API_KEY required"), "{err}"); - let err = resolve_provider(Some("openai-compat"), None, Some(" ")).unwrap_err(); + let err = resolve_provider(Some("openai-compat"), None, Some(" "), None).unwrap_err(); assert!(err.contains("OPENAI_COMPAT_API_KEY required"), "{err}"); } #[test] fn resolve_provider_errors_when_provider_env_absent() { // No implicit inference — absent BUZZ_AGENT_PROVIDER is an error. - let err = resolve_provider(None, None, None).unwrap_err(); + let err = resolve_provider(None, None, None, None).unwrap_err(); assert!(err.contains("BUZZ_AGENT_PROVIDER is required"), "{err}"); } @@ -1257,19 +1295,19 @@ mod tests { // When BUZZ_AGENT_PROVIDER=databricks, resolve_provider succeeds regardless // of DATABRICKS_HOST/MODEL (those are validated later in from_env()). assert_eq!( - resolve_provider(Some("databricks"), None, None).unwrap(), + resolve_provider(Some("databricks"), None, None, None).unwrap(), Provider::Databricks ); // Missing key for other providers still errors — no Databricks fallback. - let err = resolve_provider(Some("openai"), None, None).unwrap_err(); + let err = resolve_provider(Some("openai"), None, None, None).unwrap_err(); assert!(err.contains("OPENAI_COMPAT_API_KEY required"), "{err}"); - let err = resolve_provider(None, None, None).unwrap_err(); + let err = resolve_provider(None, None, None, None).unwrap_err(); assert!(err.contains("BUZZ_AGENT_PROVIDER is required"), "{err}"); } #[test] fn resolve_provider_unsupported_error_preserves_user_casing() { - let err = resolve_provider(Some("OpenAIish"), None, None).unwrap_err(); + let err = resolve_provider(Some("OpenAIish"), None, None, None).unwrap_err(); assert!(err.contains("BUZZ_AGENT_PROVIDER=OpenAIish")); } @@ -2657,6 +2695,9 @@ mod tests { if p == "databricks" { return openai_result(&m); } + if p == "openrouter" { + return (ALL_7.to_vec(), Some("medium")); + } // openai-compat, unknown, empty → all-7, default medium. (ALL_7.to_vec(), Some("medium")) } @@ -2708,4 +2749,18 @@ mod tests { ); } } + + #[test] + fn resolve_provider_openrouter_with_key() { + assert_eq!( + resolve_provider(Some("openrouter"), None, None, Some("sk-or-123")).unwrap(), + Provider::OpenRouter + ); + } + + #[test] + fn resolve_provider_openrouter_missing_key() { + let err = resolve_provider(Some("openrouter"), None, None, None).unwrap_err(); + assert!(err.contains("OPENROUTER_API_KEY")); + } } diff --git a/crates/buzz-agent/src/handoff.rs b/crates/buzz-agent/src/handoff.rs index 9c27c6606d..3b0feefecf 100644 --- a/crates/buzz-agent/src/handoff.rs +++ b/crates/buzz-agent/src/handoff.rs @@ -259,7 +259,11 @@ fn push_history_snippet(out: &mut String, item: &HistoryItem) { out.push_str(s); out.push('\n'); } - HistoryItem::Assistant { text, tool_calls } => { + HistoryItem::Assistant { + text, + tool_calls, + reasoning_details: _, + } => { out.push_str("[assistant] "); if !text.is_empty() { out.push_str(text); diff --git a/crates/buzz-agent/src/lib.rs b/crates/buzz-agent/src/lib.rs index e141b9860f..9a45bf4c98 100644 --- a/crates/buzz-agent/src/lib.rs +++ b/crates/buzz-agent/src/lib.rs @@ -100,6 +100,19 @@ struct Session { accumulated_input_tokens: u64, /// Session-cumulative output tokens across all turns. accumulated_output_tokens: u64, + /// Session-cumulative cache-served input tokens across all turns — a subset + /// of `accumulated_input_tokens`, not an addition to it. Emitted alongside + /// it so a consumer can price the cached slice at the provider's discounted + /// rate instead of assuming every input token cost full price. + accumulated_cached_input_tokens: u64, + /// Session-cumulative total-token state across all turns. + /// + /// Mirrors the per-turn `TurnTotalState` tri-state: starts `Unseen`, + /// becomes `Exact(n)` as turns with genuine provider totals complete, + /// transitions permanently to `Unknown` when any turn lacks a total or + /// when the cumulative would otherwise decrease. Only emitted in the + /// `usage_update` notification when `Exact`. + accumulated_total_state: crate::types::TurnTotalState, } fn die(msg: String) -> ! { @@ -426,6 +439,8 @@ async fn session_new(app: &Arc, id: Value, params: Value, wire_tx: &WireSen effective_model: None, accumulated_input_tokens: 0, accumulated_output_tokens: 0, + accumulated_cached_input_tokens: 0, + accumulated_total_state: crate::types::TurnTotalState::Unseen, }, ); drop(sessions); @@ -672,6 +687,8 @@ async fn run_prompt(app: Arc, id: Value, params: Value, wire_tx: WireSender .unwrap_or(&app.cfg.model); let mut turn_input_tokens: Option = None; let mut turn_output_tokens: Option = None; + let mut turn_cached_input_tokens: Option = None; + let mut turn_total_state = crate::types::TurnTotalState::Unseen; let mut ctx = RunCtx { cfg: &app.cfg, effective_model: effective_model_str, @@ -690,6 +707,8 @@ async fn run_prompt(app: Arc, id: Value, params: Value, wire_tx: WireSender last_request_history_bytes: &mut last_request_history_bytes, turn_input_tokens: &mut turn_input_tokens, turn_output_tokens: &mut turn_output_tokens, + turn_cached_input_tokens: &mut turn_cached_input_tokens, + turn_total_state: &mut turn_total_state, }; let result = ctx.run(p.prompt).await; if let Some(s) = app.sessions.lock().await.get_mut(&sid) { @@ -722,31 +741,54 @@ async fn run_prompt(app: Arc, id: Value, params: Value, wire_tx: WireSender s.accumulated_output_tokens = s .accumulated_output_tokens .saturating_add(turn_output_tokens.unwrap_or(0)); - Some((s.accumulated_input_tokens, s.accumulated_output_tokens)) + s.accumulated_cached_input_tokens = s + .accumulated_cached_input_tokens + .saturating_add(turn_cached_input_tokens.unwrap_or(0)); + // Fold the per-turn total state into the session cumulative. + // Unknown poisons the session permanently; Exact adds to running sum; + // Unseen (turn emitted no usage) leaves the cumulative unchanged. + // Uses TurnTotalState::merge_session, which applies the same + // checked-add / overflow-poisons contract as the per-response fold. + s.accumulated_total_state = + s.accumulated_total_state.merge_session(turn_total_state); + Some(( + s.accumulated_input_tokens, + s.accumulated_output_tokens, + s.accumulated_cached_input_tokens, + s.accumulated_total_state, + )) } else { // Session is gone — the accumulated baseline no longer exists, so // there is nothing correct to emit. Skip the usage notification. None } }; - if let Some((accumulated_in, accumulated_out)) = accumulated { - wire::send( - &wire_tx, - goose_session_update( - &sid, - json!({ - "sessionUpdate": "usage_update", - // used: total tokens as a context-usage proxy; - // contextLimit: 0 (buzz-agent has no context limit tracking). - "used": accumulated_in.saturating_add(accumulated_out), - "contextLimit": 0u64, - "accumulatedInputTokens": accumulated_in, - "accumulatedOutputTokens": accumulated_out, - "model": effective_model_str, - }), - ), - ) - .await; + if let Some((accumulated_in, accumulated_out, accumulated_cached, accumulated_total)) = + accumulated + { + // Build the usage_update payload. `accumulatedTotalTokens` is only + // included when the cumulative is exactly known — never when Unseen + // (no total ever observed) or Unknown (at least one turn lacked a + // total). A goose consumer that doesn't recognise the field ignores it. + let mut update = serde_json::json!({ + "sessionUpdate": "usage_update", + // used: total tokens as a context-usage proxy; + // contextLimit: 0 (buzz-agent has no context limit tracking). + "used": accumulated_in.saturating_add(accumulated_out), + "contextLimit": 0u64, + "accumulatedInputTokens": accumulated_in, + "accumulatedOutputTokens": accumulated_out, + // A subset of accumulatedInputTokens, not an addition to + // it. Extends goose's usage_update shape; a consumer that + // does not know the field ignores it and prices exactly as + // it did before. + "accumulatedCachedInputTokens": accumulated_cached, + "model": effective_model_str, + }); + if let crate::types::TurnTotalState::Exact(total) = accumulated_total { + update["accumulatedTotalTokens"] = serde_json::json!(total); + } + wire::send(&wire_tx, goose_session_update(&sid, update)).await; } } match result { diff --git a/crates/buzz-agent/src/llm.rs b/crates/buzz-agent/src/llm.rs index 23cef24e72..73c7e1faf2 100644 --- a/crates/buzz-agent/src/llm.rs +++ b/crates/buzz-agent/src/llm.rs @@ -3,7 +3,7 @@ use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::Arc; use reqwest::Client; -use serde_json::{json, Value}; +use serde_json::{json, Map, Value}; use tokio::sync::Mutex; use tokio::time::Instant; @@ -146,6 +146,18 @@ impl Llm { .await?; parse_anthropic(v) } + Provider::OpenRouter => { + let mut body = + openai_body(cfg, system_prompt, history, tools, effective_model, None); + apply_openrouter_mutations( + &mut body, + cfg.thinking_effort, + effective_model, + cfg.prompt_caching, + ); + let v = self.post_openrouter(cfg, &body).await?; + parse_openai_with_reasoning_details(v) + } Provider::OpenAi | Provider::Databricks => { self.openai_request( cfg, @@ -248,6 +260,16 @@ impl Llm { }); Ok(parse_anthropic(self.post_anthropic(cfg, &body).await?)?.text) } + Provider::OpenRouter => { + let body = openrouter_summary_body( + effective_model, + system_prompt, + user_prompt, + max_output_tokens, + ); + let v = self.post_openrouter(cfg, &body).await?; + Ok(parse_openai(v)?.text) + } Provider::OpenAi | Provider::Databricks => { let r = self .openai_request( @@ -652,6 +674,32 @@ impl Llm { } } + async fn post_openrouter(&self, cfg: &Config, body: &Value) -> Result { + let url = format!("{}/chat/completions", cfg.base_url.trim_end_matches('/')); + let mut bearer = self.auth.bearer().await?; + let mut refreshed = false; + loop { + match openrouter_post(&self.http, &url, body, &bearer).await { + Err(AgentError::LlmAuth(_)) if !refreshed => { + refreshed = true; + let new_bearer = self.auth.refresh_now(&bearer).await?; + // A static key refreshes to itself — a byte-identical retry + // would be a guaranteed duplicate request against a key the + // server just rejected. Fail terminal immediately; the retry + // is only meaningful when the source can actually mint a + // distinct token (e.g., a PKCE OAuth source). + if new_bearer == bearer { + return Err(AgentError::LlmAuth( + "401: static key rejected — update key in agent settings".into(), + )); + } + bearer = new_bearer; + } + result => return result, + } + } + } + /// If `err` names `/v1/responses` / "use the Responses API", latch a /// sticky upgrade so subsequent OpenAI calls hit Responses. Logged once. fn try_upgrade(&self, err: &AgentError) -> bool { @@ -695,7 +743,11 @@ fn anthropic_body( messages.push(json!({ "role": "user", "content": [{ "type": "text", "text": text }] })); } - HistoryItem::Assistant { text, tool_calls } => { + HistoryItem::Assistant { + text, + tool_calls, + reasoning_details: _, + } => { flush(&mut messages, &mut pending); let mut content: Vec = Vec::new(); if !text.is_empty() { @@ -720,6 +772,12 @@ fn anthropic_body( } } flush(&mut messages, &mut pending); + // Rolling cache breakpoint: mark the tail of the (append-only) conversation + // so the next turn re-reads this whole prefix from cache instead of paying + // full input price for it. See `stamp_rolling_cache_breakpoint`. + if cfg.prompt_caching { + stamp_rolling_cache_breakpoint(&mut messages); + } let tools_json: Vec = tools .iter() .map(|t| { @@ -727,8 +785,19 @@ fn anthropic_body( "name": t.name, "description": t.description, "input_schema": t.input_schema }) }) .collect(); + // Static prefix breakpoint: caching the `system` block caches the whole + // prefix up to and including it — and the prefix order is + // `tools -> system -> messages`, so this single marker caches tools + + // system together. Requires the structured (array) form of `system`; skip + // it for an empty prompt since Anthropic rejects empty text blocks. + let system_value = if cfg.prompt_caching && !system_prompt.is_empty() { + json!([{ "type": "text", "text": system_prompt, + "cache_control": { "type": "ephemeral" } }]) + } else { + json!(system_prompt) + }; let mut body = json!({ "model": effective_model, "max_tokens": cfg.max_output_tokens, - "system": system_prompt, "messages": messages }); + "system": system_value, "messages": messages }); if let Some(e) = effort { let (thinking, output_config) = crate::config::anthropic_thinking_config(effective_model, e, cfg.max_output_tokens); @@ -745,6 +814,41 @@ fn anthropic_body( body } +/// Attach ephemeral `cache_control` markers to the tail of the conversation so +/// the next turn re-reads the whole prior prefix from cache (~0.1x input price) +/// rather than re-billing it as fresh input. Anthropic caches the prefix up to +/// and including each marked block. +/// +/// We mark the last content block of the last *two* messages, not just the +/// final one. Each Anthropic breakpoint walks back at most 20 content blocks to +/// find a prior cache entry, and one agentic turn can append ~17 blocks at the +/// default `max_parallel_tools` (1 assistant text + N `tool_use` + +/// N `tool_result`). With only a tail marker, consecutive breakpoints sit a +/// full turn apart, which slips past the 20-block window as soon as parallelism +/// rises or a turn carries extra blocks — and the miss is silent. Marking the +/// last two messages halves the gap (to ~N+1 blocks), keeping a live cache +/// entry comfortably within reach. Uses 2 of the 4 allowed breakpoints; the +/// static `system` marker is the third. +/// +/// A no-op for messages whose content is empty or whose tail block is not a +/// JSON object. +fn stamp_rolling_cache_breakpoint(messages: &mut [Value]) { + let n = messages.len(); + // The two most-recently-appended messages (the current turn's tool results + // and the assistant turn before them). `checked_sub` + `flatten` skips the + // second index when there is only one message. + for idx in [n.checked_sub(1), n.checked_sub(2)].into_iter().flatten() { + if let Some(block) = messages[idx] + .get_mut("content") + .and_then(Value::as_array_mut) + .and_then(|c| c.last_mut()) + .and_then(Value::as_object_mut) + { + block.insert("cache_control".into(), json!({ "type": "ephemeral" })); + } + } +} + fn anthropic_tool_result_content(content: &[ToolResultContent]) -> Vec { content .iter() @@ -787,20 +891,40 @@ fn openai_body( flush_images(&mut messages, &mut pending_images); messages.push(json!({ "role": "user", "content": text })); } - HistoryItem::Assistant { text, tool_calls } => { + HistoryItem::Assistant { + text, + tool_calls, + reasoning_details, + } => { flush_images(&mut messages, &mut pending_images); let mut msg = serde_json::Map::new(); msg.insert("role".into(), json!("assistant")); msg.insert("content".into(), json!(text.as_str())); + if let Some(details) = reasoning_details { + msg.insert("reasoning_details".into(), details.clone()); + } if !tool_calls.is_empty() { let calls: Vec = tool_calls .iter() .map(|c| { - json!({ - "id": c.provider_id, "type": "function", - "function": { "name": c.name, - "arguments": serde_json::to_string(&c.arguments) - .unwrap_or_else(|_| "{}".into()) } }) + let mut call = serde_json::Map::new(); + call.insert("id".into(), json!(c.provider_id)); + call.insert("type".into(), json!("function")); + // Provider-owned fields go back beside `function`, + // which is where the provider put them. Position is + // load-bearing, not cosmetic: Gemini rejects a + // `thoughtSignature` nested inside `function{}` with + // the same 400 it gives for one that is missing. + for (k, v) in &c.provider_extra { + call.insert(k.clone(), v.clone()); + } + call.insert( + "function".into(), + json!({ "name": c.name, + "arguments": serde_json::to_string(&c.arguments) + .unwrap_or_else(|_| "{}".into()) }), + ); + Value::Object(call) }) .collect(); msg.insert("tool_calls".into(), Value::Array(calls)); @@ -886,7 +1010,11 @@ fn responses_body( "role": "user", "content": [{ "type": "input_text", "text": text }], })), - HistoryItem::Assistant { text, tool_calls } => { + HistoryItem::Assistant { + text, + tool_calls, + reasoning_details: _, + } => { if !text.is_empty() { input.push(json!({ "role": "assistant", @@ -967,13 +1095,53 @@ fn is_responses_required_error(body: &str) -> bool { || b.contains("use the responses api") } +/// OpenAI-family code names that appear as their own segment in a Databricks v2 +/// endpoint name (the GPT-5 launch aliases). The `gpt` family itself is matched +/// separately by segment prefix so `gpt`, `gpt5`, and the `gpt` of a split +/// `gpt-5` all qualify. +const DATABRICKS_V2_OPENAI_CODE_NAMES: &[&str] = &["sol", "luna", "terra"]; + +/// Anthropic (Claude) family and release code names that appear as their own +/// segment in a Databricks v2 endpoint name — the `claude` prefix, the family +/// names (`opus`, `sonnet`, `haiku`), and the release code names (`mythos`, +/// `fable`). Getting a Claude model onto the Anthropic Messages route is what +/// lets it carry a `cache_control` breakpoint; an endpoint that matches none of +/// these falls through to the MLflow (OpenAI-wire) path, where Anthropic prompt +/// caching is structurally impossible and the discount is silently lost. +const DATABRICKS_V2_CLAUDE_NAMES: &[&str] = + &["claude", "opus", "sonnet", "haiku", "mythos", "fable"]; + +/// Split a Databricks v2 endpoint name into its lowercase alphanumeric segments, +/// breaking on any non-alphanumeric delimiter (`-`, `_`, `.`, `/`, …). E.g. +/// `Databricks-Claude-Opus-5` -> `["databricks", "claude", "opus", "5"]`. +fn model_name_segments(model: &str) -> Vec { + model + .split(|c: char| !c.is_ascii_alphanumeric()) + .filter(|s| !s.is_empty()) + .map(str::to_ascii_lowercase) + .collect() +} + fn databricks_v2_route_for_model(model: &str) -> DatabricksV2Route { - // Databricks v2 catalog names currently identify OpenAI-shaped GPT-5 - // models and Anthropic-shaped Claude models by these substrings. - let lower = model.to_ascii_lowercase(); - if lower.contains("gpt-5") || lower.contains("gpt5") { + // The v2 catalog exposes no family field, so the wire format is inferred + // from the endpoint name. Discovery deliberately keeps arbitrary custom + // aliases, so we match whole name *segments* rather than raw substrings: a + // substring test would misroute unrelated names — `consolidated-llama` + // (`sol`), `terraform-coder` (`terra`), `corpus-reranker`/`octopus-model` + // (`opus`) — onto a wire whose request shape their backend can't parse, + // turning a caching optimization into a hard request/parse failure. Segment + // matching still accepts real prefixed names like `goose-opus-5`. + let segments = model_name_segments(model); + let has_named_segment = + |names: &[&str]| segments.iter().any(|seg| names.contains(&seg.as_str())); + // `gpt` family: any segment beginning with `gpt` — covers `gpt`, `gpt5`, and + // the `gpt` segment of a split `gpt-5`, without matching mid-word. + let is_gpt_family = segments.iter().any(|seg| seg.starts_with("gpt")); + // OpenAI is checked before Claude so a name carrying both markers resolves + // to the OpenAI wire (preserving the prior `gpt-5`-first precedence). + if is_gpt_family || has_named_segment(DATABRICKS_V2_OPENAI_CODE_NAMES) { DatabricksV2Route::OpenAiResponses - } else if lower.contains("claude") { + } else if has_named_segment(DATABRICKS_V2_CLAUDE_NAMES) { DatabricksV2Route::AnthropicMessages } else { DatabricksV2Route::MlflowChatCompletions @@ -1028,10 +1196,15 @@ fn parse_responses(v: Value) -> Result { let args: Value = serde_json::from_str(raw).map_err(|e| { AgentError::Llm(format!("function_call.arguments not valid JSON: {e}")) })?; + // No passthrough on this route: `responses_body` replays a + // function call as `{call_id, name, arguments}` and the Responses + // API asks for nothing else, so an empty map keeps the request + // byte-identical to before. tool_calls.push(make_tool_call( str_field(item, "call_id"), str_field(item, "name"), args, + Default::default(), )?); } Some("reasoning") => { @@ -1079,13 +1252,25 @@ fn parse_responses(v: Value) -> Result { }; let input_tokens = sum_usage(&v, &["input_tokens"]); let output_tokens = sum_usage(&v, &["output_tokens"]); + // The Responses API nests the cache split under `input_tokens_details`. + let cached_input_tokens = usage_first( + &v, + &["cache_read_input_tokens"], + &[("input_tokens_details", "cached_tokens")], + ); + // Responses API reports a genuine provider total. Read it directly — + // never derived, so it stays None when the provider omits it. + let total_tokens = sum_usage(&v, &["total_tokens"]); Ok(LlmResponse { text, tool_calls, stop, input_tokens, + cached_input_tokens, output_tokens, + total_tokens, reasoning, + reasoning_details: None, }) } @@ -1131,19 +1316,70 @@ fn anthropic_input_tokens(v: &Value) -> Option { ) } -/// Input-token total for OpenAI Chat Completions and Databricks responses. -/// OpenAI's `prompt_tokens` is already inclusive. Databricks uses the same -/// `prompt_tokens` wire field but ALSO reports Anthropic-style cache fields -/// alongside it, so we sum them; the cache fields are simply absent (and -/// contribute 0) for vanilla OpenAI. +/// Input-token total for OpenAI Chat Completions and Databricks MLflow-route +/// responses. `prompt_tokens` is already the inclusive input total on both, so +/// it is read alone and never summed with the cache fields. +/// +/// Vanilla OpenAI nests the cache split under `prompt_tokens_details` and +/// `prompt_tokens` includes it. The Databricks MLflow route reports the split +/// with the flat Anthropic spelling (`cache_read_input_tokens`) *alongside* an +/// already-inclusive `prompt_tokens` — so summing double-counts. Verified on +/// `databricks-glm-5-2` (2026-07-28): `prompt_tokens 13320`, +/// `cache_read_input_tokens 13312`, `completion_tokens 30`, `total_tokens +/// 13350`; since `prompt_tokens + completion_tokens == total_tokens`, the 13312 +/// cached tokens are contained in the 13320, not additional to it. Summing gave +/// 26632 — nearly double — inflating both the context-budget gate and cost. +/// +/// This differs from Anthropic's native route (see [`anthropic_input_tokens`]), +/// where `input_tokens` genuinely EXCLUDES the cache fields and must be summed. +/// The two never collide here: the router sends `claude*` models to the +/// Anthropic route, so `parse_openai` only ever sees inclusive `prompt_tokens`. fn openai_chat_input_tokens(v: &Value) -> Option { - sum_usage( + sum_usage(v, &["prompt_tokens"]) +} + +/// First present value among `usage.` and `usage..` pairs. +/// +/// Cache counts are the one usage figure providers do not agree on the shape of. +/// Anthropic puts `cache_read_input_tokens` flat on `usage`; OpenAI nests the +/// same quantity one level down, under `prompt_tokens_details` on +/// `/chat/completions` and `input_tokens_details` on `/responses`. [`sum_usage`] +/// only reads flat keys, which is why the OpenAI split was invisible for so +/// long: `prompt_tokens` is already inclusive, so the *total* was right and +/// nothing looked broken while the discount silently went unclaimed. +/// +/// Returns the first candidate that resolves, not a sum — these are alternative +/// spellings of one number, so adding them would double-count on Databricks, +/// which reports both shapes. +fn usage_first(v: &Value, flat: &[&str], nested: &[(&str, &str)]) -> Option { + let usage = v.get("usage")?; + for f in flat { + if let Some(n) = usage.get(*f).and_then(Value::as_u64) { + return Some(n); + } + } + for (outer, leaf) in nested { + if let Some(n) = usage + .get(*outer) + .and_then(|o| o.get(*leaf)) + .and_then(Value::as_u64) + { + return Some(n); + } + } + None +} + +/// Cache-read tokens for an OpenAI Chat Completions response. +/// +/// `prompt_tokens_details.cached_tokens` is where vanilla OpenAI reports it. +/// The flat Anthropic spelling is checked first for Databricks, which routes +/// Anthropic models through an OpenAI-shaped envelope. +fn openai_chat_cached_tokens(v: &Value) -> Option { + usage_first( v, - &[ - "prompt_tokens", - "cache_read_input_tokens", - "cache_creation_input_tokens", - ], + &["cache_read_input_tokens"], + &[("prompt_tokens_details", "cached_tokens")], ) } @@ -1151,6 +1387,76 @@ fn str_field(v: &Value, key: &str) -> String { v.get(key).and_then(Value::as_str).unwrap_or("").to_owned() } +/// Append `part` to `buf` on its own line, ignoring empties. +fn push_part(buf: &mut String, part: &str) { + if part.is_empty() { + return; + } + if !buf.is_empty() { + buf.push('\n'); + } + buf.push_str(part); +} + +/// Split an OpenAI-shaped `message.content` into `(text, reasoning)`. +/// +/// Standard OpenAI sends a string. Several models on the Databricks MLflow route +/// — Gemini, Qwen35, gpt-oss — send an array of typed blocks instead, and +/// `as_str()` yields nothing for an array, so their entire answer was being +/// discarded: no error, no warning, just a turn that looked like the model had +/// said nothing. `parse_anthropic` already walks a block array; this gives +/// `parse_openai` the same tolerance. +fn openai_content_parts(content: Option<&Value>) -> (String, String) { + let mut text = String::new(); + let mut reasoning = String::new(); + match content { + Some(Value::String(s)) => text.push_str(s), + Some(Value::Array(blocks)) => { + for b in blocks { + match b.get("type").and_then(Value::as_str) { + Some("text") => push_part(&mut text, &str_field(b, "text")), + Some("reasoning") => match b.get("summary").and_then(Value::as_array) { + // Gemini nests the prose one level down under `summary`. + Some(summary) => { + for s in summary { + push_part(&mut reasoning, &str_field(s, "text")); + } + } + None => push_part(&mut reasoning, &str_field(b, "text")), + }, + // An untyped block carrying text is still the model talking; + // treating it as text loses nothing and keeps one more + // provider out of the silent-empty-answer failure mode. + _ => push_part(&mut text, &str_field(b, "text")), + } + } + } + _ => {} + } + (text, reasoning) +} + +/// Make `provider_id` unique across one assistant turn's tool calls. +/// +/// Gemini returns the function name as the id, so two parallel calls to the same +/// function arrive sharing one id — and that id is what pairs a `role:"tool"` +/// result back to its call, leaving two results indistinguishable. Rewriting is +/// safe because both halves of that pairing are re-emitted from this same value; +/// the provider never sees its original id again. +fn dedupe_provider_ids(calls: &mut [ToolCall]) { + let mut seen: BTreeSet = BTreeSet::new(); + for c in calls.iter_mut() { + if seen.contains(&c.provider_id) { + let mut n = 2; + while seen.contains(&format!("{}-{n}", c.provider_id)) { + n += 1; + } + c.provider_id = format!("{}-{n}", c.provider_id); + } + seen.insert(c.provider_id.clone()); + } +} + fn parse_anthropic(v: Value) -> Result { let stop = map_stop(v.get("stop_reason").and_then(Value::as_str)); let mut tool_calls = Vec::new(); @@ -1173,10 +1479,12 @@ fn parse_anthropic(v: Value) -> Result { reasoning.push_str(t); } } + // Anthropic's replay shape is fully modelled, so nothing to keep. Some("tool_use") => tool_calls.push(make_tool_call( str_field(b, "id"), str_field(b, "name"), b.get("input").cloned().unwrap_or(Value::Null), + Default::default(), )?), _ => {} } @@ -1184,17 +1492,58 @@ fn parse_anthropic(v: Value) -> Result { } let input_tokens = anthropic_input_tokens(&v); let output_tokens = sum_usage(&v, &["output_tokens"]); + // Anthropic reports the cache split flat on `usage`. Note this is already + // part of `input_tokens` above, which sums it in deliberately. + let cached_input_tokens = usage_first(&v, &["cache_read_input_tokens"], &[]); Ok(LlmResponse { text, tool_calls, stop, input_tokens, + cached_input_tokens, output_tokens, + // Anthropic reports only category counts; NIP-AM forbids deriving a + // total from them. Always None for this provider. + total_tokens: None, reasoning, + reasoning_details: None, }) } fn parse_openai(v: Value) -> Result { + // A5: error-inside-200 check — choice-level `finish_reason == "error"` + if let Some(choice) = v + .get("choices") + .and_then(Value::as_array) + .and_then(|a| a.first()) + { + if choice.get("finish_reason").and_then(Value::as_str) == Some("error") { + let err = choice.get("error").cloned().unwrap_or(Value::Null); + // OpenRouter's `error.code` is numeric; other OpenAI-compat hosts + // may send a string. Accept either rather than discarding the + // typed code as "unknown". + let code = err + .get("code") + .and_then(|c| { + c.as_str() + .map(str::to_string) + .or_else(|| c.as_i64().map(|n| n.to_string())) + }) + .unwrap_or_else(|| "unknown".into()); + let message = err + .get("message") + .and_then(Value::as_str) + .unwrap_or("provider error in 200 response"); + let error_type = err + .get("metadata") + .and_then(|m| m.get("error_type")) + .and_then(Value::as_str); + return Err(AgentError::Llm(match error_type { + Some(et) => format!("provider error ({code}, {et}): {message}"), + None => format!("provider error ({code}): {message}"), + })); + } + } let choice = v .get("choices") .and_then(Value::as_array) @@ -1204,17 +1553,23 @@ fn parse_openai(v: Value) -> Result { let msg = choice .get("message") .ok_or_else(|| AgentError::Llm("missing message".into()))?; - let text = str_field(msg, "content"); + let (text, block_reasoning) = openai_content_parts(msg.get("content")); // DeepSeek and vLLM-style OpenAI-compat hosts expose reasoning tokens on the // message object. Prefer `reasoning_content` (DeepSeek's field name); fall - // back to `reasoning` (some other providers). Both are absent for standard - // OpenAI responses, which leaves this empty without any special-casing. + // back to `reasoning` (some other providers), and last to reasoning blocks + // found inside `content`. All three are absent for standard OpenAI + // responses, which leaves this empty without any special-casing. let reasoning = { let rc = str_field(msg, "reasoning_content"); - if rc.is_empty() { + let rc = if rc.is_empty() { str_field(msg, "reasoning") } else { rc + }; + if rc.is_empty() { + block_reasoning + } else { + rc } }; let mut tool_calls = Vec::new(); @@ -1226,26 +1581,64 @@ fn parse_openai(v: Value) -> Result { let raw = f.get("arguments").and_then(Value::as_str).unwrap_or("{}"); let args: Value = serde_json::from_str(raw) .map_err(|e| AgentError::Llm(format!("tool_call.arguments not valid JSON: {e}")))?; + // Everything on the wire object we do not model, kept for replay. + let extra = tc + .as_object() + .map(|o| { + o.iter() + .filter(|(k, _)| !matches!(k.as_str(), "id" | "type" | "function")) + .map(|(k, v)| (k.clone(), v.clone())) + .collect() + }) + .unwrap_or_default(); tool_calls.push(make_tool_call( str_field(tc, "id"), str_field(f, "name"), args, + extra, )?); } } + dedupe_provider_ids(&mut tool_calls); let input_tokens = openai_chat_input_tokens(&v); let output_tokens = sum_usage(&v, &["completion_tokens"]); + let cached_input_tokens = openai_chat_cached_tokens(&v); + // OpenAI Chat Completions reports a genuine provider total. Read it + // directly — never derived, so it stays None when the provider omits it. + let total_tokens = sum_usage(&v, &["total_tokens"]); Ok(LlmResponse { text, tool_calls, stop, input_tokens, + cached_input_tokens, output_tokens, + total_tokens, reasoning, + reasoning_details: None, }) } -fn make_tool_call(id: String, name: String, args: Value) -> Result { +fn parse_openai_with_reasoning_details(v: Value) -> Result { + let reasoning_details = v + .get("choices") + .and_then(Value::as_array) + .and_then(|a| a.first()) + .and_then(|c| c.get("message")) + .and_then(|m| m.get("reasoning_details")) + .filter(|rd| rd.is_array()) + .cloned(); + let mut response = parse_openai(v)?; + response.reasoning_details = reasoning_details; + Ok(response) +} + +fn make_tool_call( + id: String, + name: String, + args: Value, + provider_extra: Map, +) -> Result { if id.is_empty() || name.is_empty() { return Err(AgentError::Llm("tool_call missing id or name".into())); } @@ -1262,6 +1655,7 @@ fn make_tool_call(id: String, name: String, args: Value) -> Result Result, AgentError> { match cfg.provider { - Provider::Anthropic | Provider::OpenAi => { + Provider::Anthropic | Provider::OpenAi | Provider::OpenRouter => { Ok(Arc::new(StaticTokenSource::new(cfg.api_key.clone()))) } Provider::Databricks | Provider::DatabricksV2 => { @@ -1554,6 +1948,29 @@ pub(crate) fn build_token_source(cfg: &Config) -> Result, A } } +/// Build the request body for `Llm::summarize` on `Provider::OpenRouter`. +/// Extracted so tests can assert on the actual wire shape instead of a +/// hand-rolled literal — summaries never carry `reasoning` (see +/// `apply_openrouter_mutations`, which the summary path never calls). +/// It spells the token limit `max_tokens` directly for the same reason: the +/// mutation that renames it is never applied here. +fn openrouter_summary_body( + effective_model: &str, + system_prompt: &str, + user_prompt: &str, + max_output_tokens: u32, +) -> Value { + json!({ + "model": effective_model, + "stream": false, + "max_tokens": max_output_tokens, + "messages": [ + { "role": "system", "content": system_prompt }, + { "role": "user", "content": user_prompt }, + ], + }) +} + /// Return a clone of `body` with any top-level `"model"` field removed. /// Used for Databricks model-serving, which encodes the model in the URL /// path and rejects the field in the body. @@ -1568,12 +1985,353 @@ fn strip_model(body: &Value) -> Value { } } +#[derive(Debug)] +enum OpenRouterErrorClass { + Retryable(Option), + Unknown, +} + +/// Ceiling applied to the server-supplied `Retry-After` header before we +/// sleep on it. OpenRouter can advertise waits up to an hour, but +/// `openrouter_post`'s per-attempt sleep happens *outside* +/// `Client::timeout` (`cfg.llm_timeout`, default 240s) — an unclamped hint +/// could keep a single turn alive for up to two full-duration sleeps across +/// `MAX_RETRIES`. Clamping (never rejecting) keeps us honoring the server's +/// backoff signal while bounding worst-case turn latency to a value smaller +/// than the connect/response timeout. +const RETRY_AFTER_CAP_SECS: u64 = 60; + +fn parse_retry_after_header(headers: &reqwest::header::HeaderMap) -> Option { + let val = headers.get(reqwest::header::RETRY_AFTER)?.to_str().ok()?; + let secs: u64 = val.trim().parse().ok()?; + (secs > 0).then(|| std::time::Duration::from_secs(secs.min(RETRY_AFTER_CAP_SECS))) +} + +fn classify_openrouter_error( + status: u16, + body: &str, + header_retry_after: Option, +) -> OpenRouterErrorClass { + let parsed: Option = serde_json::from_str(body).ok(); + let error_type = parsed + .as_ref() + .and_then(|v| v.get("error")) + .and_then(|e| e.get("metadata")) + .and_then(|m| m.get("error_type")) + .and_then(Value::as_str); + // OpenRouter's documented retry hint is the HTTP `Retry-After` header + // (see https://openrouter.ai/docs/api_reference/errors-and-debugging); + // no current doc specifies a body-level retry field, so we don't parse one. + let retry_after = header_retry_after; + + match (status, error_type) { + (429, _) => OpenRouterErrorClass::Retryable(retry_after), + (502, _) => OpenRouterErrorClass::Retryable(None), + (503, Some("provider_overloaded")) => OpenRouterErrorClass::Retryable(retry_after), + _ => OpenRouterErrorClass::Unknown, + } +} + +/// The one place the parameter-routing failure is worded. OpenRouter reports it +/// two ways — a 404 `No endpoints found ...` (what a `require_parameters`-style +/// or unsupported-parameter body actually returns) and an untyped 503 — and both +/// mean the same thing to the user: the model id is fine, the request shape is +/// not serveable by any endpoint behind it. +fn openrouter_parameter_routing_error(error_body: &str) -> AgentError { + AgentError::Llm(format!( + "no OpenRouter endpoint supports the requested parameters — \ + check model, effort, and tool requirements: {error_body}" + )) +} + +async fn openrouter_post( + http: &Client, + url: &str, + body: &Value, + bearer: &str, +) -> Result { + let body_bytes = + serde_json::to_vec(body).map_err(|e| AgentError::Llm(format!("serialize: {e}")))?; + let call_start = std::time::Instant::now(); + for attempt in 0..MAX_RETRIES { + let resp = match http + .post(url) + .header("content-type", "application/json") + .header("HTTP-Referer", "https://github.com/block/buzz") + .header("X-OpenRouter-Title", "Buzz") + .bearer_auth(bearer) + .body(body_bytes.clone()) + .send() + .await + { + Ok(r) => r, + Err(e) => { + if attempt + 1 < MAX_RETRIES && is_retryable_transport_error(&e) { + tracing::warn!( + attempt = attempt + 1, + max_attempts = MAX_RETRIES, + error = %e, + "llm: openrouter transport error, retrying" + ); + backoff_with_jitter(attempt).await; + continue; + } + return Err(terminal_llm_error( + call_start.elapsed(), + attempt + 1, + &format!("transport: {e}"), + )); + } + }; + let status = resp.status(); + // Unlike the generic `post` path, OpenRouter's static-key auth makes + // 401 and 403 distinguishable: 401 is an invalid/expired key (worth + // one refresh-and-retry), while OpenRouter documents 403 as a + // guardrail/moderation/permission rejection the same key will always + // reproduce. Refreshing a static key returns the identical key, so + // classifying 403 as `LlmAuth` would just waste a duplicate request + // and surface Desktop's unrelated "access denied" copy. + if status == 401 { + return Err(AgentError::LlmAuth(read_error_body(resp).await)); + } + if status == 403 { + return Err(AgentError::Llm(format!( + "{status}: {}", + read_error_body(resp).await + ))); + } + if status == 402 { + return Err(AgentError::Llm( + "OpenRouter credits exhausted — check https://openrouter.ai/credits".into(), + )); + } + if status == 404 { + // OpenRouter overloads 404: a genuinely unknown/unavailable model id + // and a valid model whose parameter set no endpoint can serve both + // land here. Discriminate on the full parameter-routing phrase rather + // than a prefix — "No endpoints found for "-style bodies are + // about the model, and reporting a parameter problem as + // `LlmModelNotFound` (or vice versa) sends the user to the wrong fix. + let error_body = read_error_body(resp).await; + if error_body.contains("No endpoints found that can handle the requested parameters") { + return Err(openrouter_parameter_routing_error(&error_body)); + } + return Err(AgentError::LlmModelNotFound(format!( + "{status}: {error_body}" + ))); + } + // A6: status+error_type retry matrix + // 499 (Client Closed Request) is included: OpenRouter may emit it when a + // turn times out mid-stream. Will added 499 to the shared `post()` path in + // #2175 for the same reason; OpenRouter must match to avoid silently opting + // out of that stall-surfacing recovery. + if status.is_server_error() || status == 429 || status.as_u16() == 499 { + let header_retry_after = parse_retry_after_header(resp.headers()); + let error_body = read_error_body(resp).await; + let should_retry = if attempt + 1 < MAX_RETRIES { + match classify_openrouter_error(status.as_u16(), &error_body, header_retry_after) { + OpenRouterErrorClass::Retryable(delay) => { + if let Some(d) = delay { + tokio::time::sleep(d).await; + } else { + backoff_with_jitter(attempt).await; + } + true + } + OpenRouterErrorClass::Unknown => { + backoff_with_jitter(attempt).await; + true + } + } + } else { + false + }; + if should_retry { + continue; + } + // Terminal: classify for the user + return if status == 429 { + Err(terminal_llm_error( + call_start.elapsed(), + attempt + 1, + &format!("rate limited: {error_body}"), + )) + } else { + let parsed: Option = serde_json::from_str(&error_body).ok(); + let has_error_type = parsed + .as_ref() + .and_then(|v| v.get("error")) + .and_then(|e| e.get("metadata")) + .and_then(|m| m.get("error_type")) + .and_then(Value::as_str) + .is_some(); + Err(if !has_error_type && status.as_u16() == 503 { + openrouter_parameter_routing_error(&error_body) + } else { + terminal_llm_error( + call_start.elapsed(), + attempt + 1, + &format!("exhausted retries: {status}: {error_body}"), + ) + }) + }; + } + if !status.is_success() { + return Err(AgentError::Llm(format!( + "{status}: {}", + read_error_body(resp).await + ))); + } + if let Some(len) = resp.content_length() { + if len as usize > MAX_LLM_RESPONSE_BYTES { + return Err(AgentError::Llm(format!( + "response too large: {len} > {MAX_LLM_RESPONSE_BYTES}" + ))); + } + } + let mut buf: Vec = Vec::new(); + let mut stream = resp; + loop { + match stream.chunk().await { + Ok(Some(chunk)) => { + if buf.len() + chunk.len() > MAX_LLM_RESPONSE_BYTES { + return Err(AgentError::Llm(format!( + "response exceeded {MAX_LLM_RESPONSE_BYTES} bytes" + ))); + } + buf.extend_from_slice(&chunk); + } + Ok(None) => break, + Err(e) => { + return Err(terminal_llm_error( + call_start.elapsed(), + attempt + 1, + &format!("body read: {e}"), + )) + } + } + } + return serde_json::from_slice(&buf).map_err(|e| AgentError::Llm(format!("json: {e}"))); + } + Err(terminal_llm_error( + call_start.elapsed(), + MAX_RETRIES, + "exhausted retries", + )) +} + +fn apply_openrouter_mutations( + body: &mut Value, + effort: Option, + effective_model: &str, + prompt_caching: bool, +) { + if let Some(obj) = body.as_object_mut() { + // OpenRouter's Chat Completions API spells the output cap `max_tokens`; + // `max_completion_tokens` is OpenAI-native and only 53 of 274 + // tools-capable OpenRouter models advertise it. Sending the OpenAI + // spelling risks the cap being dropped on the floor by the endpoint we + // route to, so translate it here rather than at the shared `openai_body` + // (which OpenAI and Databricks also use, where the OpenAI spelling is + // correct). + if let Some(max_tokens) = obj.remove("max_completion_tokens") { + obj.insert("max_tokens".into(), max_tokens); + } + + // A2/A3: Add OpenRouter reasoning object when effort is configured. + // Deliberately NOT paired with `provider.require_parameters`: that filter + // routes only to endpoints advertising every parameter in the body, and + // 83 of 274 tools-capable models do not advertise `reasoning`, so it turns + // an opt-in effort setting into a hard 404 ("No endpoints found that can + // handle the requested parameters") on a model id that is perfectly + // valid. OpenRouter already best-effort routes on `tools`/`max_tokens` + // without the filter, so we accept a request served without reasoning + // over a request that cannot be served at all. + if let Some(e) = effort { + obj.insert( + "reasoning".into(), + json!({ "effort": e.openai_effort_str() }), + ); + } + + // A7: Anthropic cache_control injection for anthropic/* models. Gated on + // `prompt_caching` (`BUZZ_AGENT_PROMPT_CACHING`) for the same reason as + // the native Anthropic route: these are Anthropic-dialect breakpoints on + // an Anthropic model, so the documented kill switch must reach them too. + if prompt_caching && effective_model.starts_with("anthropic/") { + apply_anthropic_cache_control(obj); + } + } +} + +fn apply_anthropic_cache_control(body: &mut serde_json::Map) { + if let Some(messages) = body.get_mut("messages").and_then(Value::as_array_mut) { + // Cache the system message + if let Some(system_msg) = messages + .iter_mut() + .find(|m| m.get("role").and_then(Value::as_str) == Some("system")) + { + if let Some(content) = system_msg.get("content").and_then(Value::as_str) { + let content_str = content.to_string(); + if let Some(obj) = system_msg.as_object_mut() { + obj.insert( + "content".into(), + json!([{ + "type": "text", + "text": content_str, + "cache_control": { "type": "ephemeral" } + }]), + ); + } + } + } + + // Cache last 2 user messages (skip image-only ones — A7 mixed-content regression) + let mut user_count = 0; + for msg in messages.iter_mut().rev() { + if msg.get("role").and_then(Value::as_str) != Some("user") { + continue; + } + // Only cache string content (plain text user messages), not array content + // (image batches from tool results). This prevents corrupting image-only + // user messages by converting them to text cache breakpoints. + if let Some(content) = msg.get("content").and_then(Value::as_str) { + let content_str = content.to_string(); + if let Some(obj) = msg.as_object_mut() { + obj.insert( + "content".into(), + json!([{ + "type": "text", + "text": content_str, + "cache_control": { "type": "ephemeral" } + }]), + ); + } + user_count += 1; + } + if user_count >= 2 { + break; + } + } + } + // Cache the last tool definition + if let Some(tools) = body.get_mut("tools").and_then(Value::as_array_mut) { + if let Some(last_tool) = tools.last_mut() { + if let Some(function) = last_tool.get_mut("function").and_then(Value::as_object_mut) { + function.insert("cache_control".into(), json!({ "type": "ephemeral" })); + } + } + } +} + #[cfg(test)] mod tests { use super::*; use crate::config::{Config, HookServers, OpenAiApi, Provider}; use crate::types::{HistoryItem, ToolCall, ToolResult, ToolResultContent}; + use std::collections::VecDeque; use std::time::Duration; + use tokio::sync::Mutex; use tracing_subscriber::layer::SubscriberExt; fn cfg(provider: Provider) -> Config { @@ -1597,6 +2355,7 @@ mod tests { max_parallel_tools: 1, hook_timeout: Duration::from_secs(1), stop_max_rejections: 0, + require_reply: false, hook_servers: HookServers::None, api_key: "key".into(), model: "model".into(), @@ -1606,6 +2365,7 @@ mod tests { prefer_mesh_for_auto: false, hints_enabled: true, thinking_effort: None, + prompt_caching: true, } } @@ -2208,7 +2968,9 @@ mod tests { provider_id: "toolu_1".into(), name: "dev__view_image".into(), arguments: serde_json::json!({"source":"x.png"}), + provider_extra: Default::default(), }], + reasoning_details: None, }, HistoryItem::ToolResult(ToolResult { provider_id: "toolu_1".into(), @@ -2257,7 +3019,9 @@ mod tests { provider_id: "call_abc".into(), name: "dev__shell".into(), arguments: serde_json::json!({"command": "ls"}), + provider_extra: Default::default(), }], + reasoning_details: None, }, HistoryItem::ToolResult(ToolResult { provider_id: "call_abc".into(), @@ -2355,7 +3119,9 @@ mod tests { provider_id: "call_x".into(), name: "t".into(), arguments: serde_json::json!({}), + provider_extra: Default::default(), }], + reasoning_details: None, }, ]; let body = responses_body(&cfg_responses(), "system", &history, &[], "model", None); @@ -2462,26 +3228,111 @@ mod tests { #[test] fn databricks_v2_routes_by_model_family() { + use DatabricksV2Route::{AnthropicMessages, MlflowChatCompletions, OpenAiResponses}; for (model, route, path) in [ + // OpenAI-shaped: the gpt family plus the GPT-5 code names. ( "databricks-gpt-5-5", - DatabricksV2Route::OpenAiResponses, + OpenAiResponses, "/ai-gateway/openai/v1/responses", ), + ("gpt-4o", OpenAiResponses, "/ai-gateway/openai/v1/responses"), + // The intentional dashless `gpt5` spelling still routes to OpenAI. + ("gpt5", OpenAiResponses, "/ai-gateway/openai/v1/responses"), + ( + "databricks-gpt-5-6-luna", + OpenAiResponses, + "/ai-gateway/openai/v1/responses", + ), + ( + "databricks-gpt-5-6-sol", + OpenAiResponses, + "/ai-gateway/openai/v1/responses", + ), + ( + "databricks-terra", + OpenAiResponses, + "/ai-gateway/openai/v1/responses", + ), + // Anthropic-shaped: the claude prefix, the family names, and the + // release code names — each must reach the cache-capable route even + // when the endpoint name omits the literal "claude". ( "databricks-claude-opus-4-7", - DatabricksV2Route::AnthropicMessages, + AnthropicMessages, "/ai-gateway/anthropic/v1/messages", ), ( - "custom-tool-model", - DatabricksV2Route::MlflowChatCompletions, - "/ai-gateway/mlflow/v1/chat/completions", + "goose-opus-5", + AnthropicMessages, + "/ai-gateway/anthropic/v1/messages", ), - ] { - let got = databricks_v2_route_for_model(model); - assert_eq!(got, route, "model={model}"); - assert_eq!(databricks_v2_path(got), path, "model={model}"); + ( + "databricks-sonnet-5", + AnthropicMessages, + "/ai-gateway/anthropic/v1/messages", + ), + ( + "databricks-haiku-4-5", + AnthropicMessages, + "/ai-gateway/anthropic/v1/messages", + ), + ( + "databricks-mythos-5", + AnthropicMessages, + "/ai-gateway/anthropic/v1/messages", + ), + ( + "databricks-fable-5", + AnthropicMessages, + "/ai-gateway/anthropic/v1/messages", + ), + // Case-insensitive. + ( + "Databricks-Claude-Opus-5", + AnthropicMessages, + "/ai-gateway/anthropic/v1/messages", + ), + // Unrecognised names still fall through to the MLflow chat route. + ( + "custom-tool-model", + MlflowChatCompletions, + "/ai-gateway/mlflow/v1/chat/completions", + ), + ( + "databricks-gemini-3-pro", + MlflowChatCompletions, + "/ai-gateway/mlflow/v1/chat/completions", + ), + // Collision guard: short code names must match only as whole + // segments, never as substrings of an unrelated custom alias. + // Each of these embeds a marker (`sol`, `terra`, `opus`) mid-word + // and must stay on the MLflow fallback, not adopt a wire its + // backend can't parse. + ( + "consolidated-llama", + MlflowChatCompletions, + "/ai-gateway/mlflow/v1/chat/completions", + ), + ( + "terraform-coder", + MlflowChatCompletions, + "/ai-gateway/mlflow/v1/chat/completions", + ), + ( + "corpus-reranker", + MlflowChatCompletions, + "/ai-gateway/mlflow/v1/chat/completions", + ), + ( + "octopus-model", + MlflowChatCompletions, + "/ai-gateway/mlflow/v1/chat/completions", + ), + ] { + let got = databricks_v2_route_for_model(model); + assert_eq!(got, route, "model={model}"); + assert_eq!(databricks_v2_path(got), path, "model={model}"); } } @@ -2548,13 +3399,16 @@ mod tests { provider_id: "toolu_a".into(), name: "dev__view_image".into(), arguments: serde_json::json!({"source": "a.png"}), + provider_extra: Default::default(), }, ToolCall { provider_id: "toolu_b".into(), name: "dev__view_image".into(), arguments: serde_json::json!({"source": "b.png"}), + provider_extra: Default::default(), }, ], + reasoning_details: None, }, HistoryItem::ToolResult(ToolResult { provider_id: "toolu_a".into(), @@ -2604,6 +3458,101 @@ mod tests { assert_eq!(imgs[1]["image_url"]["url"], "data:image/png;base64,bbb"); } + // ---- prompt caching (cache_control) body-shape tests ---- + + #[test] + fn anthropic_body_stamps_cache_control_when_enabled() { + let body = anthropic_body( + &cfg(Provider::DatabricksV2), + "sys", + &[ + HistoryItem::User("hello".into()), + HistoryItem::Assistant { + text: "hi".into(), + tool_calls: vec![], + reasoning_details: None, + }, + HistoryItem::User("more".into()), + ], + &[], + "databricks-claude-opus-5", + None, + ); + // Static prefix: system promoted to a structured block carrying the marker. + assert_eq!(body["system"][0]["type"], "text"); + assert_eq!(body["system"][0]["text"], "sys"); + assert_eq!(body["system"][0]["cache_control"]["type"], "ephemeral"); + // Leapfrog: the last block of the last TWO messages is marked; earlier + // ones are not. Three distinct user/assistant/user turns → messages[1] + // and messages[2] marked, messages[0] clean. + let msgs = body["messages"].as_array().unwrap(); + assert_eq!(msgs.len(), 3); + let tail_block = |m: &Value| m["content"].as_array().unwrap().last().unwrap().clone(); + assert_eq!(tail_block(&msgs[2])["cache_control"]["type"], "ephemeral"); + assert_eq!(tail_block(&msgs[1])["cache_control"]["type"], "ephemeral"); + assert!( + tail_block(&msgs[0]).get("cache_control").is_none(), + "only the last two messages carry a breakpoint" + ); + } + + #[test] + fn anthropic_body_single_message_stamps_only_one_breakpoint() { + // With a single message there is no second turn to leapfrog to; the + // checked_sub(2) index is skipped rather than panicking. + let body = anthropic_body( + &cfg(Provider::DatabricksV2), + "sys", + &[HistoryItem::User("hello".into())], + &[], + "databricks-claude-opus-5", + None, + ); + let msgs = body["messages"].as_array().unwrap(); + assert_eq!(msgs.len(), 1); + assert_eq!( + msgs[0]["content"].as_array().unwrap().last().unwrap()["cache_control"]["type"], + "ephemeral" + ); + } + + #[test] + fn anthropic_body_no_cache_control_when_disabled() { + let mut c = cfg(Provider::DatabricksV2); + c.prompt_caching = false; + let body = anthropic_body( + &c, + "sys", + &[HistoryItem::User("hello".into())], + &[], + "databricks-claude-opus-5", + None, + ); + // system stays a bare string; no marker anywhere. + assert_eq!(body["system"], "sys"); + let last_block = &body["messages"][0]["content"][0]; + assert!(last_block.get("cache_control").is_none()); + } + + #[test] + fn anthropic_body_empty_system_stays_string_even_when_caching() { + // An empty system prompt must not become an empty text block — + // Anthropic rejects those. Caching is still applied to the tail. + let body = anthropic_body( + &cfg(Provider::DatabricksV2), + "", + &[HistoryItem::User("hello".into())], + &[], + "databricks-claude-opus-5", + None, + ); + assert_eq!(body["system"], ""); + assert_eq!( + body["messages"][0]["content"][0]["cache_control"]["type"], + "ephemeral" + ); + } + // ---- ThinkingEffort body-shape tests ---- #[test] @@ -3483,6 +4432,93 @@ mod tests { assert_eq!(parse_anthropic(v).unwrap().input_tokens, None); } + /// A Gemini reply as the Databricks MLflow route actually returns it: + /// block-array `content`, and a `thoughtSignature` beside `function`. + fn gemini_choice() -> Value { + json!({"choices": [{"finish_reason": "tool_calls", "message": { + "role": "assistant", + "content": [ + {"type": "reasoning", "summary": [{"type": "summary_text", "text": "weighing it"}]}, + {"type": "text", "text": "391"} + ], + "tool_calls": [{ + "id": "get_weather", "type": "function", "thoughtSignature": "SIG-A", + "function": {"name": "get_weather", "arguments": "{\"city\":\"Paris\"}"} + }] + }}]}) + } + + #[test] + fn parse_openai_reads_text_out_of_a_block_array() { + // Before this, `as_str()` on the array yielded "" and the model's answer + // was discarded with no error at all. + let r = parse_openai(gemini_choice()).unwrap(); + assert_eq!(r.text, "391"); + assert_eq!(r.reasoning, "weighing it"); + } + + #[test] + fn parse_openai_still_reads_a_plain_string_content() { + let v = json!({"choices": [{"finish_reason": "stop", "message": { + "role": "assistant", "content": "plain"}}]}); + let r = parse_openai(v).unwrap(); + assert_eq!(r.text, "plain"); + assert_eq!(r.reasoning, ""); + } + + #[test] + fn parse_openai_keeps_unmodelled_tool_call_fields() { + let r = parse_openai(gemini_choice()).unwrap(); + let extra = &r.tool_calls[0].provider_extra; + assert_eq!(extra.get("thoughtSignature"), Some(&json!("SIG-A"))); + // `id`/`type`/`function` are modelled, so they must not be duplicated + // into the passthrough — they would be re-emitted twice. + assert!(!extra.contains_key("id")); + assert!(!extra.contains_key("type")); + assert!(!extra.contains_key("function")); + } + + #[test] + fn openai_body_replays_the_signature_beside_function_not_inside_it() { + // Position is what the gateway checks: nested inside `function{}` it is + // rejected with the same 400 as a missing signature. + let r = parse_openai(gemini_choice()).unwrap(); + let history = vec![HistoryItem::Assistant { + text: r.text.clone(), + tool_calls: r.tool_calls.clone(), + reasoning_details: None, + }]; + let body = openai_body( + &cfg(Provider::DatabricksV2), + "sys", + &history, + &[], + "databricks-gemini-3-6-flash", + None, + ); + let call = &body["messages"][1]["tool_calls"][0]; + assert_eq!(call["thoughtSignature"], json!("SIG-A")); + assert!(call["function"].get("thoughtSignature").is_none()); + assert_eq!(call["function"]["name"], json!("get_weather")); + } + + #[test] + fn parse_openai_makes_duplicate_tool_call_ids_unique() { + // Gemini returns the function name as the id, so parallel calls to one + // function collide and their results become indistinguishable. + let v = json!({"choices": [{"finish_reason": "tool_calls", "message": { + "role": "assistant", "content": "", + "tool_calls": [ + {"id": "get_weather", "type": "function", + "function": {"name": "get_weather", "arguments": "{\"city\":\"Paris\"}"}}, + {"id": "get_weather", "type": "function", + "function": {"name": "get_weather", "arguments": "{\"city\":\"Rome\"}"}} + ]}}]}); + let r = parse_openai(v).unwrap(); + assert_eq!(r.tool_calls[0].provider_id, "get_weather"); + assert_eq!(r.tool_calls[1].provider_id, "get_weather-2"); + } + #[test] fn parse_openai_uses_prompt_tokens() { let v = serde_json::json!({ @@ -3493,20 +4529,34 @@ mod tests { } #[test] - fn parse_openai_databricks_sums_cache_fields() { - // Databricks uses the OpenAI chat wire format (prompt_tokens) but also - // reports Anthropic-style cache fields; the inclusive total sums them. + fn parse_openai_databricks_prompt_tokens_already_inclusive() { + // Databricks' MLflow route uses the OpenAI chat wire format + // (prompt_tokens) but ALSO reports the flat Anthropic-style + // cache_read_input_tokens. prompt_tokens is already inclusive of that + // slice, so the total is prompt_tokens alone — summing double-counts. + // Values are the live databricks-glm-5-2 response (2026-07-28), where + // prompt_tokens + completion_tokens == total_tokens proves inclusivity. let v = serde_json::json!({ "choices": [{"finish_reason": "stop", "message": {"content": "hi"}}], "usage": { - "prompt_tokens": 200, - "completion_tokens": 4, - "total_tokens": 204, - "cache_read_input_tokens": 800, - "cache_creation_input_tokens": 0 + "prompt_tokens": 13320, + "completion_tokens": 30, + "total_tokens": 13350, + "cache_read_input_tokens": 13312, + "prompt_tokens_details": {"cached_tokens": 13312} } }); - assert_eq!(parse_openai(v).unwrap().input_tokens, Some(1000)); + let r = parse_openai(v).unwrap(); + assert_eq!( + r.input_tokens, + Some(13320), + "prompt_tokens is the inclusive total" + ); + assert_eq!(r.cached_input_tokens, Some(13312)); + assert!( + r.cached_input_tokens.unwrap() <= r.input_tokens.unwrap(), + "the cached slice is a subset of the input total" + ); } #[test] @@ -3517,6 +4567,194 @@ mod tests { assert_eq!(parse_openai(v).unwrap().input_tokens, None); } + // ── total_tokens parsing ─────────────────────────────────────────────── + + #[test] + fn parse_openai_chat_total_tokens_present_is_read() { + // Chat Completions: `usage.total_tokens` is a genuine provider total. + let v = serde_json::json!({ + "choices": [{"finish_reason": "stop", "message": {"content": "ok"}}], + "usage": {"prompt_tokens": 100, "completion_tokens": 25, "total_tokens": 125} + }); + let r = parse_openai(v).unwrap(); + assert_eq!(r.total_tokens, Some(125)); + assert_eq!(r.input_tokens, Some(100)); + assert_eq!(r.output_tokens, Some(25)); + } + + #[test] + fn parse_openai_chat_total_tokens_absent_is_none() { + // Chat Completions without `total_tokens` → None, not a derived sum. + let v = serde_json::json!({ + "choices": [{"finish_reason": "stop", "message": {"content": "ok"}}], + "usage": {"prompt_tokens": 100, "completion_tokens": 25} + }); + let r = parse_openai(v).unwrap(); + assert_eq!(r.total_tokens, None); + } + + #[test] + fn parse_responses_total_tokens_present_is_read() { + // Responses API: `usage.total_tokens` is a genuine provider total. + let v = serde_json::json!({ + "output": [{"type": "message", "content": [{"type": "output_text", "text": "hi"}]}], + "status": "completed", + "usage": {"input_tokens": 80, "output_tokens": 20, "total_tokens": 100} + }); + let r = parse_responses(v).unwrap(); + assert_eq!(r.total_tokens, Some(100)); + assert_eq!(r.input_tokens, Some(80)); + assert_eq!(r.output_tokens, Some(20)); + } + + #[test] + fn parse_responses_total_tokens_absent_is_none() { + // Responses API without `total_tokens` → None. + let v = serde_json::json!({ + "output": [{"type": "message", "content": [{"type": "output_text", "text": "hi"}]}], + "status": "completed", + "usage": {"input_tokens": 80, "output_tokens": 20} + }); + let r = parse_responses(v).unwrap(); + assert_eq!(r.total_tokens, None); + } + + #[test] + fn parse_anthropic_total_tokens_always_none() { + // Anthropic reports only category counts; NIP-AM forbids deriving a total. + // total_tokens must always be None regardless of what the response contains — + // including if a future Anthropic API version unexpectedly adds total_tokens. + let v = serde_json::json!({ + "content": [{"type": "text", "text": "hi"}], + "stop_reason": "end_turn", + "usage": { + "input_tokens": 100, + "cache_read_input_tokens": 50, + "cache_creation_input_tokens": 0, + "output_tokens": 30, + // Unexpected field: parse_anthropic must ignore this and return None. + "total_tokens": 180 + } + }); + let r = parse_anthropic(v).unwrap(); + assert!( + r.total_tokens.is_none(), + "Anthropic must never supply a total_tokens value" + ); + // Verify other fields still parse correctly. + assert_eq!(r.input_tokens, Some(150)); // inclusive sum with cache + assert_eq!(r.output_tokens, Some(30)); + } + + #[test] + fn parse_openai_reads_nested_cached_tokens() { + // The shape vanilla OpenAI actually returns, captured from a live + // /chat/completions probe on gpt-5.6-luna: `prompt_tokens` is already + // inclusive and the cache split is nested one level down. Reading only + // flat keys left the discount unclaimed while the total looked correct, + // which is why this went unnoticed. + let v = serde_json::json!({ + "choices": [{"finish_reason": "stop", "message": {"content": "OK"}}], + "usage": { + "prompt_tokens": 5229, + "completion_tokens": 4, + "total_tokens": 5233, + "prompt_tokens_details": {"audio_tokens": 0, "cached_tokens": 5226, + "cache_write_tokens": 0} + } + }); + let r = parse_openai(v).unwrap(); + assert_eq!(r.input_tokens, Some(5229), "total must stay inclusive"); + assert_eq!(r.cached_input_tokens, Some(5226)); + } + + #[test] + fn parse_openai_cache_write_round_reports_zero_cached() { + // First request of a cold prefix: the provider writes the cache and + // serves nothing from it. `Some(0)` not `None` — the split was reported, + // it was simply zero, and a consumer must be able to tell the two apart. + let v = serde_json::json!({ + "choices": [{"finish_reason": "stop", "message": {"content": "OK"}}], + "usage": { + "prompt_tokens": 5229, + "completion_tokens": 4, + "prompt_tokens_details": {"cached_tokens": 0, "cache_write_tokens": 5226} + } + }); + assert_eq!(parse_openai(v).unwrap().cached_input_tokens, Some(0)); + } + + #[test] + fn parse_openai_no_cache_detail_is_none() { + let v = serde_json::json!({ + "choices": [{"finish_reason": "stop", "message": {"content": "hi"}}], + "usage": {"prompt_tokens": 123, "completion_tokens": 4} + }); + assert_eq!(parse_openai(v).unwrap().cached_input_tokens, None); + } + + #[test] + fn parse_openai_prefers_flat_anthropic_spelling_over_nested() { + // Databricks reports both shapes for the same quantity. Take one, never + // the sum, or the cached slice double-counts. cache_read (800) is a + // subset of the inclusive prompt_tokens (1000), as it must be. + let v = serde_json::json!({ + "choices": [{"finish_reason": "stop", "message": {"content": "hi"}}], + "usage": { + "prompt_tokens": 1000, + "completion_tokens": 4, + "cache_read_input_tokens": 800, + "prompt_tokens_details": {"cached_tokens": 800} + } + }); + let r = parse_openai(v).unwrap(); + assert_eq!(r.input_tokens, Some(1000)); + assert_eq!(r.cached_input_tokens, Some(800)); + assert!(r.cached_input_tokens.unwrap() <= r.input_tokens.unwrap()); + } + + #[test] + fn parse_anthropic_reports_cache_read_as_cached() { + // Anthropic's `input_tokens` EXCLUDES cached, so the inclusive total is + // a sum -- but the cached slice must still be a subset of that total. + let v = serde_json::json!({ + "stop_reason": "end_turn", + "content": [{"type": "text", "text": "hi"}], + "usage": { + "input_tokens": 100, + "output_tokens": 7, + "cache_read_input_tokens": 900, + "cache_creation_input_tokens": 50 + } + }); + let r = parse_anthropic(v).unwrap(); + assert_eq!(r.input_tokens, Some(1050)); + assert_eq!(r.cached_input_tokens, Some(900)); + assert!(r.cached_input_tokens.unwrap() <= r.input_tokens.unwrap()); + } + + #[test] + fn parse_responses_reads_nested_cached_tokens() { + // The Responses API nests the same figure under a different key than + // /chat/completions does. + let v = serde_json::json!({ + "status": "completed", + "output": [{ + "type": "message", + "role": "assistant", + "content": [{"type": "output_text", "text": "hi"}] + }], + "usage": { + "input_tokens": 4000, + "output_tokens": 9, + "input_tokens_details": {"cached_tokens": 3584} + } + }); + let r = parse_responses(v).unwrap(); + assert_eq!(r.input_tokens, Some(4000)); + assert_eq!(r.cached_input_tokens, Some(3584)); + } + #[test] fn parse_responses_uses_input_tokens() { let v = serde_json::json!({ @@ -3843,4 +5081,1476 @@ mod tests { }); assert_eq!(parse_responses(v).unwrap().output_tokens, None); } + + // ---- A3: OpenRouter body-shape tests ---- + + fn tools_vec() -> Vec { + vec![ToolDef { + name: "dev__shell".into(), + description: "run a shell command".into(), + input_schema: serde_json::json!({ + "type": "object", + "properties": {"command": {"type": "string"}}, + }), + }] + } + + #[test] + fn openrouter_body_tools_with_effort() { + let mut c = cfg(Provider::OpenRouter); + c.thinking_effort = Some(ThinkingEffort::High); + let mut body = openai_body( + &c, + "system", + &[HistoryItem::User("hi".into())], + &tools_vec(), + "anthropic/claude-opus-4-7", + None, + ); + apply_openrouter_mutations( + &mut body, + c.thinking_effort, + "anthropic/claude-opus-4-7", + true, + ); + assert_eq!(body["reasoning"]["effort"], "high"); + // `openai_body` is always called with `effort=None` on the OpenRouter + // path (line 151): OpenRouter uses its own `reasoning` object, not the + // OpenAI-style `reasoning_effort` field. Verify the field is structurally + // absent — not merely removed by a no-op cleanup. + assert!( + body.get("reasoning_effort").is_none(), + "reasoning_effort must never appear on the OpenRouter body path: \ + openai_body is called with effort=None, so the field is never emitted" + ); + assert!( + body.get("provider").is_none(), + "provider.require_parameters hard-404s models that do not advertise \ + every parameter we send" + ); + assert!(!body["tools"].as_array().unwrap().is_empty()); + assert_eq!( + body["max_tokens"], 1024, + "token limit must carry openai_body's value under OpenRouter's spelling" + ); + assert!( + body.get("max_completion_tokens").is_none(), + "max_completion_tokens is the OpenAI-native spelling; OpenRouter reads max_tokens" + ); + } + + #[test] + fn openrouter_body_tools_no_effort() { + let c = cfg(Provider::OpenRouter); + let mut body = openai_body( + &c, + "system", + &[HistoryItem::User("hi".into())], + &tools_vec(), + "anthropic/claude-opus-4-7", + None, + ); + apply_openrouter_mutations(&mut body, None, "anthropic/claude-opus-4-7", true); + assert!( + body.get("reasoning").is_none(), + "reasoning must be absent when effort is None" + ); + assert!( + body.get("reasoning_effort").is_none(), + "reasoning_effort must never appear on the OpenRouter body path" + ); + assert!( + body.get("provider").is_none(), + "tools alone must not add a provider routing filter" + ); + } + + #[test] + fn openrouter_body_empty_tools_with_effort() { + let mut c = cfg(Provider::OpenRouter); + c.thinking_effort = Some(ThinkingEffort::Medium); + let mut body = openai_body( + &c, + "system", + &[HistoryItem::User("hi".into())], + &[], + "anthropic/claude-opus-4-7", + None, + ); + apply_openrouter_mutations( + &mut body, + c.thinking_effort, + "anthropic/claude-opus-4-7", + true, + ); + assert_eq!(body["reasoning"]["effort"], "medium"); + assert!( + body.get("provider").is_none(), + "83 of 274 tools-capable models do not advertise `reasoning`; filtering on \ + it would 404 them instead of answering without reasoning" + ); + } + + #[test] + fn openrouter_body_empty_tools_no_effort() { + let c = cfg(Provider::OpenRouter); + let mut body = openai_body( + &c, + "system", + &[HistoryItem::User("hi".into())], + &[], + "anthropic/claude-opus-4-7", + None, + ); + apply_openrouter_mutations(&mut body, None, "anthropic/claude-opus-4-7", true); + assert!(body.get("reasoning").is_none()); + assert!( + body.get("provider").is_none(), + "no body shape adds a provider routing filter" + ); + } + + /// `BUZZ_AGENT_PROMPT_CACHING=0` must reach the OpenRouter `anthropic/*` + /// route too, not just the native Anthropic Messages routes. The switch and + /// this route landed in separate changes, so nothing but this test stops the + /// gate from being dropped and the kill switch silently becoming a no-op on + /// a route that emits Anthropic-dialect breakpoints. + #[test] + fn openrouter_body_caching_disabled_emits_no_cache_control() { + let c = cfg(Provider::OpenRouter); + let mut body = openai_body( + &c, + "system", + &[HistoryItem::User("hi".into())], + &tools_vec(), + "anthropic/claude-opus-4-7", + None, + ); + apply_openrouter_mutations(&mut body, None, "anthropic/claude-opus-4-7", false); + assert!( + !body.to_string().contains("cache_control"), + "BUZZ_AGENT_PROMPT_CACHING=0 must suppress every breakpoint: {body}" + ); + // The switch is scoped to caching — the routing-contract mutations that + // make the request serveable at all must still be applied. + assert_eq!( + body.get("max_tokens").and_then(Value::as_u64), + Some(u64::from(c.max_output_tokens)), + "the max_tokens rename is not part of the caching gate" + ); + } + + /// The paired positive case: with caching on, the same body does carry + /// breakpoints. Without this twin, a mutation that hard-disabled caching + /// outright would still leave the test above green. + #[test] + fn openrouter_body_caching_enabled_emits_cache_control() { + let c = cfg(Provider::OpenRouter); + let mut body = openai_body( + &c, + "system", + &[HistoryItem::User("hi".into())], + &tools_vec(), + "anthropic/claude-opus-4-7", + None, + ); + apply_openrouter_mutations(&mut body, None, "anthropic/claude-opus-4-7", true); + assert!( + body.to_string().contains("cache_control"), + "caching on must still emit breakpoints: {body}" + ); + } + + /// The rename moves an existing value; it never invents a token limit for a + /// body that did not carry one. + #[test] + fn openrouter_body_without_token_limit_gains_none() { + let mut body = json!({ "model": "vendor/model", "messages": [] }); + apply_openrouter_mutations(&mut body, None, "vendor/model", true); + assert!(body.get("max_tokens").is_none()); + assert!(body.get("max_completion_tokens").is_none()); + } + + #[test] + fn openrouter_summary_carries_neither_reasoning_nor_provider() { + let body = openrouter_summary_body( + "anthropic/claude-opus-4-7", + "summarize", + "text to summarize", + 1024, + ); + assert_eq!(body["model"], "anthropic/claude-opus-4-7"); + assert_eq!(body["messages"][0]["role"], "system"); + assert_eq!(body["messages"][1]["content"], "text to summarize"); + assert_eq!(body["max_tokens"], 1024); + assert!( + body.get("max_completion_tokens").is_none(), + "summary body must use OpenRouter's token-limit spelling" + ); + assert!( + body.get("reasoning").is_none(), + "summary body must not carry reasoning" + ); + assert!( + body.get("provider").is_none(), + "summary body must not carry provider" + ); + } + + /// The token-limit rename belongs to `apply_openrouter_mutations`, not to + /// `openai_body` — which OpenAI and Databricks also use, and where + /// `max_completion_tokens` is the correct spelling. + #[test] + fn openai_body_keeps_max_completion_tokens_when_unmutated() { + let body = openai_body( + &cfg(Provider::OpenAi), + "system", + &[HistoryItem::User("hi".into())], + &[], + "model", + None, + ); + assert_eq!(body["max_completion_tokens"], 1024); + assert!(body.get("max_tokens").is_none()); + } + + // ---- A5: error-inside-200 ---- + + #[test] + fn parse_openai_error_inside_200_returns_error() { + let v = serde_json::json!({ + "choices": [{ + "finish_reason": "error", + "error": { + "code": 503, + "message": "No endpoints found that support tool use" + } + }] + }); + let err = parse_openai(v).unwrap_err(); + match &err { + AgentError::Llm(s) => { + assert!(s.contains("provider error (503)"), "got: {s}"); + assert!(s.contains("No endpoints found"), "got: {s}"); + } + _ => panic!("expected AgentError::Llm, got: {err:?}"), + } + } + + #[test] + fn parse_openai_error_inside_200_accepts_string_code() { + let v = serde_json::json!({ + "choices": [{ + "finish_reason": "error", + "error": { + "code": "insufficient_quota", + "message": "quota exceeded" + } + }] + }); + let err = parse_openai(v).unwrap_err(); + match &err { + AgentError::Llm(s) => assert!( + s.contains("provider error (insufficient_quota)"), + "got: {s}" + ), + _ => panic!("expected AgentError::Llm, got: {err:?}"), + } + } + + #[test] + fn parse_openai_error_inside_200_surfaces_error_type() { + let v = serde_json::json!({ + "choices": [{ + "finish_reason": "error", + "error": { + "code": 429, + "message": "Rate limit exceeded", + "metadata": { "error_type": "rate_limit_exceeded" } + } + }] + }); + let err = parse_openai(v).unwrap_err(); + match &err { + AgentError::Llm(s) => { + assert!(s.contains("429"), "got: {s}"); + assert!(s.contains("rate_limit_exceeded"), "got: {s}"); + } + _ => panic!("expected AgentError::Llm, got: {err:?}"), + } + } + + #[test] + fn parse_openai_normal_stop_not_affected_by_error_check() { + let v = serde_json::json!({ + "choices": [{"finish_reason": "stop", "message": {"content": "hello"}}], + "usage": {"prompt_tokens": 10, "completion_tokens": 5} + }); + let r = parse_openai(v).unwrap(); + assert_eq!(r.text, "hello"); + assert_eq!(r.stop, ProviderStop::EndTurn); + } + + #[test] + fn parse_openai_tool_calls_not_affected_by_error_check() { + let v = serde_json::json!({ + "choices": [{ + "finish_reason": "tool_calls", + "message": { + "content": "", + "tool_calls": [{ + "id": "call_1", + "type": "function", + "function": {"name": "test", "arguments": "{}"} + }] + } + }] + }); + let r = parse_openai(v).unwrap(); + assert_eq!(r.stop, ProviderStop::ToolUse); + assert_eq!(r.tool_calls.len(), 1); + } + + // ---- A6: OpenRouter retry classification ---- + + #[test] + fn classify_429_rate_limit_body_retry_after_is_ignored() { + // M1: no documented body-level retry field, so a `retry_after` key + // inside `error.metadata` is inert — only the HTTP header (passed + // separately) can produce a delay. + let body = + r#"{"error":{"metadata":{"error_type":"rate_limit_exceeded","retry_after":2.5}}}"#; + match classify_openrouter_error(429, body, None) { + OpenRouterErrorClass::Retryable(None) => {} + other => panic!("expected Retryable(None), got: {other:?}"), + } + } + + #[test] + fn classify_429_rate_limit_without_retry_after() { + let body = r#"{"error":{"metadata":{"error_type":"rate_limit_exceeded"}}}"#; + match classify_openrouter_error(429, body, None) { + OpenRouterErrorClass::Retryable(None) => {} + other => panic!("expected Retryable(None), got: {other:?}"), + } + } + + #[test] + fn classify_429_prefers_http_header_over_body() { + // Body-level retry hints are no longer parsed (M1: undocumented field); + // the HTTP header is the only source, and it's honored when present. + let body = r#"{"error":{"metadata":{"error_type":"rate_limit_exceeded"}}}"#; + let header = Some(Duration::from_secs(3)); + match classify_openrouter_error(429, body, header) { + OpenRouterErrorClass::Retryable(Some(d)) => { + assert_eq!(d, Duration::from_secs(3), "HTTP header must be honored"); + } + other => panic!("expected Retryable with header delay, got: {other:?}"), + } + } + + #[test] + fn classify_502_provider_unavailable() { + let body = r#"{"error":{"metadata":{"error_type":"provider_unavailable"}}}"#; + match classify_openrouter_error(502, body, None) { + OpenRouterErrorClass::Retryable(None) => {} + other => panic!("expected Retryable(None) for 502, got: {other:?}"), + } + } + + #[test] + fn classify_503_provider_overloaded_with_retry_after() { + // No documented body-level retry field (M1); the header is the only + // source `classify_openrouter_error` consults. + let body = r#"{"error":{"metadata":{"error_type":"provider_overloaded"}}}"#; + let header = Some(Duration::from_secs(5)); + match classify_openrouter_error(503, body, header) { + OpenRouterErrorClass::Retryable(Some(d)) => { + assert_eq!(d, Duration::from_secs(5)); + } + other => panic!("expected Retryable with delay, got: {other:?}"), + } + } + + #[test] + fn classify_503_untyped_is_unknown() { + let body = r#"{"error":{"message":"No endpoints found"}}"#; + match classify_openrouter_error(503, body, None) { + OpenRouterErrorClass::Unknown => {} + other => panic!("expected Unknown for untyped 503, got: {other:?}"), + } + } + + #[test] + fn classify_500_untyped_is_unknown() { + match classify_openrouter_error(500, r#"{"error":{"message":"internal"}}"#, None) { + OpenRouterErrorClass::Unknown => {} + other => panic!("expected Unknown for 500, got: {other:?}"), + } + } + + #[test] + fn parse_retry_after_header_valid() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert(reqwest::header::RETRY_AFTER, "5".parse().unwrap()); + assert_eq!( + parse_retry_after_header(&headers), + Some(Duration::from_secs(5)) + ); + } + + #[test] + fn parse_retry_after_header_zero_rejected() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert(reqwest::header::RETRY_AFTER, "0".parse().unwrap()); + assert_eq!(parse_retry_after_header(&headers), None); + } + + #[test] + fn parse_retry_after_header_over_cap_clamped() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert(reqwest::header::RETRY_AFTER, "3601".parse().unwrap()); + assert_eq!( + parse_retry_after_header(&headers), + Some(Duration::from_secs(RETRY_AFTER_CAP_SECS)), + "over-cap hints clamp to the ceiling rather than being dropped" + ); + } + + #[test] + fn parse_retry_after_header_at_cap_unclamped() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert( + reqwest::header::RETRY_AFTER, + RETRY_AFTER_CAP_SECS.to_string().parse().unwrap(), + ); + assert_eq!( + parse_retry_after_header(&headers), + Some(Duration::from_secs(RETRY_AFTER_CAP_SECS)) + ); + } + + #[test] + fn parse_retry_after_header_missing() { + let headers = reqwest::header::HeaderMap::new(); + assert_eq!(parse_retry_after_header(&headers), None); + } + + #[test] + fn parse_retry_after_header_non_numeric_ignored() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert( + reqwest::header::RETRY_AFTER, + "Wed, 21 Oct 2026 07:28:00 GMT".parse().unwrap(), + ); + assert_eq!(parse_retry_after_header(&headers), None); + } + + // ---- A7: Anthropic cache_control with mixed content ---- + + #[test] + fn anthropic_cache_control_mixed_text_tool_image_history() { + let history = vec![ + HistoryItem::User("first question".into()), + HistoryItem::Assistant { + text: String::new(), + tool_calls: vec![ToolCall { + provider_id: "toolu_1".into(), + name: "dev__view_image".into(), + arguments: serde_json::json!({"source": "x.png"}), + provider_extra: Default::default(), + }], + reasoning_details: None, + }, + HistoryItem::ToolResult(ToolResult { + provider_id: "toolu_1".into(), + content: vec![ + ToolResultContent::Text("10×10 image".into()), + ToolResultContent::Image { + data: "aW1n".into(), + mime_type: "image/png".into(), + }, + ], + is_error: false, + }), + HistoryItem::User("second question about the image".into()), + HistoryItem::User("third question".into()), + ]; + let mut body = openai_body( + &cfg(Provider::OpenRouter), + "system", + &history, + &tools_vec(), + "anthropic/claude-opus-4-7", + None, + ); + apply_openrouter_mutations(&mut body, None, "anthropic/claude-opus-4-7", true); + let messages = body["messages"].as_array().unwrap(); + + // System message should have cache_control + let system = &messages[0]; + assert_eq!(system["content"][0]["cache_control"]["type"], "ephemeral"); + + // Image-batch user messages (containing image_url blocks) must NOT have cache_control + let image_user_msgs: Vec<_> = messages + .iter() + .filter(|m| { + m.get("role").and_then(Value::as_str) == Some("user") + && m.get("content") + .and_then(Value::as_array) + .map(|a| { + a.iter() + .any(|b| b.get("type").and_then(Value::as_str) == Some("image_url")) + }) + .unwrap_or(false) + }) + .collect(); + assert!( + !image_user_msgs.is_empty(), + "should have image user messages" + ); + for img_msg in &image_user_msgs { + let content = img_msg["content"].as_array().unwrap(); + for block in content { + assert!( + block.get("cache_control").is_none(), + "image-only user message must not receive cache_control" + ); + } + } + + // Exactly 2 text user messages should have cache_control (skipping image-only ones) + let cached_text_count = messages + .iter() + .filter(|m| { + m.get("role").and_then(Value::as_str) == Some("user") + && m.get("content") + .and_then(Value::as_array) + .map(|a| { + a.iter().any(|b| { + b.get("type").and_then(Value::as_str) == Some("text") + && b.get("cache_control").is_some() + }) + }) + .unwrap_or(false) + }) + .count(); + assert_eq!( + cached_text_count, 2, + "exactly 2 text user messages should have cache_control" + ); + + // Last tool def should have cache_control + let tools = body["tools"].as_array().unwrap(); + let last_tool = tools.last().unwrap(); + assert_eq!(last_tool["function"]["cache_control"]["type"], "ephemeral"); + } + + #[test] + fn anthropic_cache_control_image_only_user_does_not_consume_slot() { + // An image-only user message between two text user messages must not + // consume a cache breakpoint slot — both text messages should get cached. + let history = vec![ + HistoryItem::User("text message one".into()), + HistoryItem::Assistant { + text: String::new(), + tool_calls: vec![ToolCall { + provider_id: "toolu_1".into(), + name: "dev__view_image".into(), + arguments: serde_json::json!({"source": "x.png"}), + provider_extra: Default::default(), + }], + reasoning_details: None, + }, + HistoryItem::ToolResult(ToolResult { + provider_id: "toolu_1".into(), + content: vec![ToolResultContent::Image { + data: "aW1n".into(), + mime_type: "image/png".into(), + }], + is_error: false, + }), + HistoryItem::User("text message two".into()), + HistoryItem::User("text message three".into()), + ]; + let mut body = openai_body( + &cfg(Provider::OpenRouter), + "system", + &history, + &[], + "anthropic/claude-opus-4-7", + None, + ); + apply_openrouter_mutations(&mut body, None, "anthropic/claude-opus-4-7", true); + let messages = body["messages"].as_array().unwrap(); + + // Count text user messages that got cache_control + let cached_text_count = messages + .iter() + .filter(|m| { + m.get("role").and_then(Value::as_str) == Some("user") + && m.get("content") + .and_then(Value::as_array) + .map(|a| a.iter().any(|b| b.get("cache_control").is_some())) + .unwrap_or(false) + }) + .count(); + assert_eq!( + cached_text_count, 2, + "image-only user messages must not consume a cache breakpoint slot" + ); + } + + // ---- A9: reasoning_details round-trip ---- + + #[test] + fn parse_openai_with_reasoning_details_captures_array() { + let details = serde_json::json!([ + {"type": "thinking", "content": "Let me consider..."}, + {"type": "thinking", "content": "The answer is 42."} + ]); + let v = serde_json::json!({ + "choices": [{ + "finish_reason": "tool_calls", + "message": { + "content": "", + "reasoning_details": details, + "tool_calls": [{ + "id": "call_1", + "type": "function", + "function": {"name": "test", "arguments": "{}"} + }] + } + }], + "usage": {"prompt_tokens": 10, "completion_tokens": 5} + }); + let r = parse_openai_with_reasoning_details(v).unwrap(); + assert_eq!(r.reasoning_details, Some(details)); + } + + #[test] + fn parse_openai_with_reasoning_details_none_when_absent() { + let v = serde_json::json!({ + "choices": [{ + "finish_reason": "stop", + "message": {"content": "hello"} + }] + }); + let r = parse_openai_with_reasoning_details(v).unwrap(); + assert!( + r.reasoning_details.is_none(), + "reasoning_details must be None when not in response" + ); + } + + /// M2: a malformed `reasoning_details` shape (null or a bare object, + /// rather than the documented array) must be omitted at the parse + /// boundary, not stored and later replayed into the next request. + #[test] + fn parse_openai_with_reasoning_details_omits_non_array_shapes() { + let null_shape = serde_json::json!({ + "choices": [{ + "finish_reason": "stop", + "message": {"content": "hello", "reasoning_details": null} + }] + }); + let r = parse_openai_with_reasoning_details(null_shape).unwrap(); + assert!( + r.reasoning_details.is_none(), + "null reasoning_details must be omitted, not stored as Some(Null)" + ); + + let object_shape = serde_json::json!({ + "choices": [{ + "finish_reason": "stop", + "message": { + "content": "hello", + "reasoning_details": {"type": "thinking", "content": "not an array"} + } + }] + }); + let r = parse_openai_with_reasoning_details(object_shape).unwrap(); + assert!( + r.reasoning_details.is_none(), + "a bare-object reasoning_details must be omitted, not stored as Some(object)" + ); + } + + #[test] + fn parse_openai_plain_never_captures_reasoning_details() { + let v = serde_json::json!({ + "choices": [{ + "finish_reason": "stop", + "message": { + "content": "hello", + "reasoning_details": [{"type": "thinking", "content": "hmm"}] + } + }] + }); + let r = parse_openai(v).unwrap(); + assert!( + r.reasoning_details.is_none(), + "plain parse_openai must never capture reasoning_details (OpenAI/Databricks regression)" + ); + } + + #[test] + fn reasoning_details_two_request_round_trip() { + let details = serde_json::json!([ + {"type": "thinking", "content": "Step 1: analyze the request."}, + {"type": "thinking", "content": "Step 2: call the tool."} + ]); + // Request 1: model returns a tool call with reasoning_details + let response1 = serde_json::json!({ + "choices": [{ + "finish_reason": "tool_calls", + "message": { + "content": "", + "reasoning_details": details, + "tool_calls": [{ + "id": "call_abc", + "type": "function", + "function": {"name": "dev__shell", "arguments": "{\"command\":\"ls\"}"} + }] + } + }], + "usage": {"prompt_tokens": 50, "completion_tokens": 20} + }); + let r1 = parse_openai_with_reasoning_details(response1).unwrap(); + assert_eq!(r1.reasoning_details, Some(details.clone())); + + // Build history as the agent would: assistant turn with reasoning_details, + // followed by a tool result. + let history = vec![ + HistoryItem::User("run ls".into()), + HistoryItem::Assistant { + text: String::new(), + tool_calls: r1.tool_calls, + reasoning_details: r1.reasoning_details, + }, + HistoryItem::ToolResult(ToolResult { + provider_id: "call_abc".into(), + content: vec![ToolResultContent::Text("file.txt".into())], + is_error: false, + }), + ]; + + // Request 2: build the body for the continuation + let body = openai_body( + &cfg(Provider::OpenRouter), + "system", + &history, + &[], + "anthropic/claude-opus-4-7", + None, + ); + let messages = body["messages"].as_array().unwrap(); + + // The assistant message must carry the identical reasoning_details array + let assistant_msg = messages + .iter() + .find(|m| m.get("role").and_then(Value::as_str) == Some("assistant")) + .expect("assistant message must exist"); + assert_eq!( + assistant_msg["reasoning_details"], details, + "reasoning_details must be replayed byte-for-byte on the assistant message" + ); + + // The assistant message must appear BEFORE the tool result + let assistant_idx = messages + .iter() + .position(|m| m.get("role").and_then(Value::as_str) == Some("assistant")) + .unwrap(); + let tool_idx = messages + .iter() + .position(|m| m.get("role").and_then(Value::as_str) == Some("tool")) + .unwrap(); + assert!( + assistant_idx < tool_idx, + "assistant with reasoning_details must precede tool result" + ); + } + + #[test] + fn reasoning_details_none_emits_no_field_in_body() { + let history = vec![ + HistoryItem::User("hello".into()), + HistoryItem::Assistant { + text: "hi back".into(), + tool_calls: Vec::new(), + reasoning_details: None, + }, + ]; + let body = openai_body( + &cfg(Provider::OpenRouter), + "system", + &history, + &[], + "anthropic/claude-opus-4-7", + None, + ); + let messages = body["messages"].as_array().unwrap(); + let assistant_msg = messages + .iter() + .find(|m| m.get("role").and_then(Value::as_str) == Some("assistant")) + .expect("assistant message must exist"); + assert!( + assistant_msg.get("reasoning_details").is_none(), + "assistant with None reasoning_details must not emit the field" + ); + } + + #[test] + fn reasoning_details_charged_to_estimated_bytes() { + let details = serde_json::json!([ + {"type": "thinking", "content": "A long chain of reasoning tokens here."} + ]); + let with = HistoryItem::Assistant { + text: "text".into(), + tool_calls: Vec::new(), + reasoning_details: Some(details.clone()), + }; + let without = HistoryItem::Assistant { + text: "text".into(), + tool_calls: Vec::new(), + reasoning_details: None, + }; + assert!( + with.estimated_bytes() > without.estimated_bytes(), + "reasoning_details must contribute to estimated_bytes" + ); + assert!( + with.context_pressure_bytes() > without.context_pressure_bytes(), + "reasoning_details must contribute to context_pressure_bytes" + ); + let details_size = serde_json::to_vec(&details).unwrap().len(); + assert_eq!( + with.estimated_bytes() - without.estimated_bytes(), + details_size, + "reasoning_details contribution must equal its serialized size" + ); + } + + #[test] + fn reasoning_details_not_replayed_in_anthropic_body() { + let history = vec![ + HistoryItem::User("hi".into()), + HistoryItem::Assistant { + text: "ok".into(), + tool_calls: Vec::new(), + reasoning_details: Some( + serde_json::json!([{"type": "thinking", "content": "hmm"}]), + ), + }, + ]; + let body = anthropic_body( + &cfg(Provider::Anthropic), + "system", + &history, + &[], + "claude-opus-4-7", + None, + ); + let messages = body["messages"].as_array().unwrap(); + let assistant = messages + .iter() + .find(|m| m.get("role").and_then(Value::as_str) == Some("assistant")) + .unwrap(); + assert!( + assistant.get("reasoning_details").is_none(), + "anthropic_body must not replay reasoning_details" + ); + } + + #[test] + fn reasoning_details_not_replayed_in_responses_body() { + let history = vec![ + HistoryItem::User("hi".into()), + HistoryItem::Assistant { + text: "ok".into(), + tool_calls: Vec::new(), + reasoning_details: Some( + serde_json::json!([{"type": "thinking", "content": "hmm"}]), + ), + }, + ]; + let body = responses_body(&cfg_responses(), "system", &history, &[], "model", None); + let body_str = serde_json::to_string(&body).unwrap(); + assert!( + !body_str.contains("reasoning_details"), + "responses_body must not replay reasoning_details" + ); + } + + // ---- T4: openrouter_post transport-level regressions ---- + // + // These stub an HTTP server directly and drive `openrouter_post` (not the + // classifier in isolation), proving the retry/attempt-accounting and + // header behavior the classifier-only tests above cannot see. + + /// One canned response: status, body, and any extra headers (e.g. + /// `Retry-After`) to send back for a single request. + struct CannedResponse { + status: u16, + body: String, + extra_headers: Vec<(String, String)>, + } + + impl CannedResponse { + fn new(status: u16, body: &str) -> Self { + Self { + status, + body: body.into(), + extra_headers: Vec::new(), + } + } + + fn with_header(mut self, name: &str, value: &str) -> Self { + self.extra_headers.push((name.into(), value.into())); + self + } + } + + fn status_line(status: u16) -> &'static str { + match status { + 200 => "200 OK", + 401 => "401 Unauthorized", + 402 => "402 Payment Required", + 403 => "403 Forbidden", + 404 => "404 Not Found", + 429 => "429 Too Many Requests", + 499 => "499 Client Closed Request", + 500 => "500 Internal Server Error", + 502 => "502 Bad Gateway", + 503 => "503 Service Unavailable", + _ => panic!("unsupported status {status} in test stub"), + } + } + + /// Spawns a stub HTTP server that pops one `CannedResponse` per request + /// (repeating the last one once the queue is exhausted, so an + /// over-budget attempt count is visible rather than hanging), and + /// captures each request's raw header block for header-attribution + /// assertions. Returns (url, captured_header_blocks, attempt_counter). + async fn spawn_openrouter_stub( + responses: Vec, + ) -> ( + String, + Arc>>, + Arc, + ) { + use std::sync::atomic::{AtomicU32, Ordering}; + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + use tokio::net::TcpListener; + + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("http://{}", listener.local_addr().unwrap()); + let queue = Arc::new(Mutex::new(VecDeque::from(responses))); + let captured: Arc>> = Arc::new(Mutex::new(Vec::new())); + let attempts = Arc::new(AtomicU32::new(0)); + let captured_clone = captured.clone(); + let attempts_clone = attempts.clone(); + tokio::spawn(async move { + loop { + let (mut sock, _) = match listener.accept().await { + Ok(p) => p, + Err(_) => return, + }; + let queue = queue.clone(); + let captured = captured_clone.clone(); + let attempts = attempts_clone.clone(); + tokio::spawn(async move { + let mut buf = Vec::new(); + let mut tmp = [0u8; 4096]; + while !buf.windows(4).any(|w| w == b"\r\n\r\n") { + match sock.read(&mut tmp).await { + Ok(0) | Err(_) => return, + Ok(n) => buf.extend_from_slice(&tmp[..n]), + } + if buf.len() > 1_000_000 { + return; + } + } + let header_end = buf.windows(4).position(|w| w == b"\r\n\r\n").unwrap() + 4; + let header_str = String::from_utf8_lossy(&buf[..header_end]).into_owned(); + // Drain any remaining body per Content-Length so `Connection: + // close` doesn't race the client's write. + let content_length: usize = header_str + .lines() + .find_map(|line| { + line.to_ascii_lowercase() + .strip_prefix("content-length:") + .and_then(|v| v.trim().parse().ok()) + }) + .unwrap_or(0); + let mut body_len = buf.len() - header_end; + while body_len < content_length { + match sock.read(&mut tmp).await { + Ok(0) | Err(_) => break, + Ok(n) => body_len += n, + } + } + captured.lock().await.push(header_str); + attempts.fetch_add(1, Ordering::SeqCst); + + let mut q = queue.lock().await; + let canned = if q.len() > 1 { + q.pop_front().unwrap() + } else { + // Repeat the final canned response so a test bug that + // over-retries produces a visible extra attempt + // instead of a hung connection. + let last = q.front().unwrap(); + CannedResponse { + status: last.status, + body: last.body.clone(), + extra_headers: last.extra_headers.clone(), + } + }; + drop(q); + + let mut resp = format!( + "HTTP/1.1 {}\r\nContent-Type: application/json\r\nContent-Length: {}\r\n", + status_line(canned.status), + canned.body.len() + ); + for (name, value) in &canned.extra_headers { + resp.push_str(&format!("{name}: {value}\r\n")); + } + resp.push_str("Connection: close\r\n\r\n"); + resp.push_str(&canned.body); + let _ = sock.write_all(resp.as_bytes()).await; + let _ = sock.shutdown().await; + }); + } + }); + (url, captured, attempts) + } + + /// A 403 (guardrail/moderation/permission rejection, per OpenRouter docs) + /// must NOT be classified as `LlmAuth`: refreshing a static key returns + /// the identical key, so retrying would just waste a duplicate request. + /// Exactly one attempt, plain `AgentError::Llm` with the body preserved. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_403_single_attempt_not_auth_error() { + let (url, _captured, attempts) = spawn_openrouter_stub(vec![CannedResponse::new( + 403, + r#"{"error":{"message":"model flagged by moderation"}}"#, + )]) + .await; + let http = Client::builder() + .timeout(Duration::from_secs(5)) + .build() + .unwrap(); + let err = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .unwrap_err(); + assert!( + matches!(&err, AgentError::Llm(s) if s.contains("403") && s.contains("model flagged by moderation")), + "403 must surface as AgentError::Llm with status+body, not LlmAuth: got {err:?}" + ); + assert_eq!( + attempts.load(std::sync::atomic::Ordering::SeqCst), + 1, + "403 must not be retried (a refreshed static key is identical)" + ); + } + + /// A 402 short-circuits on the first attempt: no retry, one request, + /// the actionable credits-exhausted message. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_402_single_attempt_short_circuit() { + let (url, _captured, attempts) = spawn_openrouter_stub(vec![CannedResponse::new( + 402, + r#"{"error":{"message":"payment required"}}"#, + )]) + .await; + let http = Client::builder() + .timeout(Duration::from_secs(5)) + .build() + .unwrap(); + let err = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .unwrap_err(); + assert!( + matches!(&err, AgentError::Llm(s) if s.contains("credits exhausted")), + "got {err:?}" + ); + assert_eq!( + attempts.load(std::sync::atomic::Ordering::SeqCst), + 1, + "402 must not be retried" + ); + } + + /// A 404 whose body is OpenRouter's parameter-routing rejection is NOT a + /// missing model: the id is valid and no endpoint behind it can serve the + /// request shape. It must surface the actionable routing message rather than + /// `LlmModelNotFound`, which sends the user hunting a model-name typo. + /// Body text is the one OpenRouter actually returned in the live probe run. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_404_no_endpoints_found_is_parameter_routing_error() { + let (url, _captured, attempts) = spawn_openrouter_stub(vec![CannedResponse::new( + 404, + r#"{"error":{"message":"No endpoints found that can handle the requested parameters. To learn more about provider routing, visit: https://openrouter.ai/docs/guides/routing/provider-selection","code":404}}"#, + )]) + .await; + let http = Client::builder() + .timeout(Duration::from_secs(5)) + .build() + .unwrap(); + let err = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .unwrap_err(); + assert!( + matches!(&err, AgentError::Llm(s) if s.contains("no OpenRouter endpoint supports")), + "parameter-routing 404 must not be reported as a missing model: got {err:?}" + ); + assert_eq!( + attempts.load(std::sync::atomic::Ordering::SeqCst), + 1, + "404 must not be retried" + ); + } + + /// Every other 404 still maps to `LlmModelNotFound`, including one that + /// shares the `No endpoints found` prefix but is about the model rather than + /// the parameters — the discriminator is narrow enough that a genuinely + /// unavailable model keeps its own error kind (Desktop renders + /// model-not-found differently from a generic LLM failure). + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_404_unknown_model_stays_model_not_found() { + let (url, _captured, _attempts) = spawn_openrouter_stub(vec![CannedResponse::new( + 404, + r#"{"error":{"message":"No endpoints found for vendor/nonexistent-model.","code":404}}"#, + )]) + .await; + let http = Client::builder() + .timeout(Duration::from_secs(5)) + .build() + .unwrap(); + let err = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .unwrap_err(); + assert!( + matches!(&err, AgentError::LlmModelNotFound(s) if s.contains("404") && s.contains("vendor/nonexistent-model")), + "a model-level 404 must stay LlmModelNotFound: got {err:?}" + ); + } + + /// A 429 with `Retry-After: 1` sleeps for that duration before the retry + /// succeeds — proving the header value is actually honored, not just + /// classified. + #[tokio::test(flavor = "current_thread")] + async fn openrouter_post_429_honors_retry_after_header() { + let (url, _captured, attempts) = spawn_openrouter_stub(vec![ + CannedResponse::new(429, r#"{"error":{"message":"rate limited"}}"#) + .with_header("Retry-After", "1"), + CannedResponse::new(200, r#"{"choices":[{"message":{"content":"ok"}}]}"#), + ]) + .await; + let http = Client::builder() + .timeout(Duration::from_secs(30)) + .build() + .unwrap(); + let before = std::time::Instant::now(); + let out = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .expect("second attempt succeeds"); + assert_eq!(out["choices"][0]["message"]["content"], "ok"); + assert!( + before.elapsed() >= Duration::from_secs(1), + "must sleep at least the Retry-After hint" + ); + assert_eq!(attempts.load(std::sync::atomic::Ordering::SeqCst), 2); + } + + /// A `Retry-After` far beyond `RETRY_AFTER_CAP_SECS` must not stall the + /// retry loop for anywhere near its advertised duration — proving the + /// cap is enforced end-to-end in `openrouter_post`'s actual sleep, not + /// merely in the isolated `parse_retry_after_header` unit tests above. + /// Runs on a paused clock so a real 999999s wait would hang the test + /// instead of silently passing. + #[tokio::test(start_paused = true)] + async fn openrouter_post_429_retry_sleep_capped_despite_huge_retry_after() { + let (url, _captured, attempts) = spawn_openrouter_stub(vec![ + CannedResponse::new(429, r#"{"error":{"message":"rate limited"}}"#) + .with_header("Retry-After", "999999"), + CannedResponse::new(200, r#"{"choices":[{"message":{"content":"ok"}}]}"#), + ]) + .await; + let http = Client::builder().build().unwrap(); + let before = tokio::time::Instant::now(); + let out = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .expect("second attempt succeeds"); + assert_eq!(out["choices"][0]["message"]["content"], "ok"); + assert!( + before.elapsed() <= Duration::from_secs(RETRY_AFTER_CAP_SECS + 5), + "retry sleep must be clamped to RETRY_AFTER_CAP_SECS ({RETRY_AFTER_CAP_SECS}s), \ + not the header's 999999s: elapsed {:?}", + before.elapsed() + ); + assert_eq!(attempts.load(std::sync::atomic::Ordering::SeqCst), 2); + } + + /// An untyped 503 (no `error.metadata.error_type`) exhausts all + /// `MAX_RETRIES` attempts, then returns the actionable routing message — + /// proving attempt accounting terminates rather than retrying forever. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_untyped_503_exhausts_retries_into_actionable_message() { + let canned = CannedResponse::new(503, r#"{"error":{"message":"no capacity"}}"#); + let (url, _captured, attempts) = spawn_openrouter_stub(vec![canned]).await; + let http = Client::builder() + .timeout(Duration::from_secs(30)) + .build() + .unwrap(); + let err = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .unwrap_err(); + assert!( + matches!(&err, AgentError::Llm(s) if s.contains("no OpenRouter endpoint supports")), + "got {err:?}" + ); + assert_eq!( + attempts.load(std::sync::atomic::Ordering::SeqCst), + MAX_RETRIES, + "must exhaust exactly MAX_RETRIES attempts, no more" + ); + } + + /// Attribution headers (`HTTP-Referer`, `X-OpenRouter-Title`) are on the + /// actual wire request, not merely asserted against a body fixture. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_sends_attribution_headers() { + let (url, captured, _attempts) = spawn_openrouter_stub(vec![CannedResponse::new( + 200, + r#"{"choices":[{"message":{"content":"ok"}}]}"#, + )]) + .await; + let http = Client::builder() + .timeout(Duration::from_secs(5)) + .build() + .unwrap(); + openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .expect("200 succeeds"); + let headers = captured.lock().await; + let header_str = headers + .first() + .expect("one request captured") + .to_lowercase(); + assert!( + header_str.contains("http-referer: https://github.com/block/buzz"), + "got: {header_str}" + ); + assert!( + header_str.contains("x-openrouter-title: buzz"), + "got: {header_str}" + ); + } + + /// A 499 response is retried and the call succeeds on the second attempt, + /// mirroring the shared `post()` path (#2175). + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_retries_499_then_succeeds() { + let (url, _captured, attempts) = spawn_openrouter_stub(vec![ + CannedResponse::new(499, ""), + CannedResponse::new(200, r#"{"choices":[{"message":{"content":"ok"}}]}"#), + ]) + .await; + let http = Client::builder() + .timeout(Duration::from_secs(30)) + .build() + .unwrap(); + let out = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .expect("retry after 499 should succeed"); + assert_eq!(out["choices"][0]["message"]["content"], "ok"); + assert_eq!( + attempts.load(std::sync::atomic::Ordering::SeqCst), + 2, + "exactly one 499 retry" + ); + } + + /// A 502 without `provider_unavailable` still retries (the classifier's + /// unconditional-retry branch), then succeeds on attempt 2 — proving the + /// generic 502 path isn't accidentally routed to `Unknown`/terminal. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_502_retries_then_succeeds() { + let (url, _captured, attempts) = spawn_openrouter_stub(vec![ + CannedResponse::new(502, r#"{"error":{"message":"bad gateway"}}"#), + CannedResponse::new(200, r#"{"choices":[{"message":{"content":"ok"}}]}"#), + ]) + .await; + let http = Client::builder() + .timeout(Duration::from_secs(30)) + .build() + .unwrap(); + let out = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .expect("retry succeeds"); + assert_eq!(out["choices"][0]["message"]["content"], "ok"); + assert_eq!(attempts.load(std::sync::atomic::Ordering::SeqCst), 2); + } + + /// A 200 response whose body is truncated mid-stream (connection closed + /// before the full Content-Length is delivered) must surface the error + /// through `terminal_llm_error`, not a bare `AgentError::Llm("read: …")` — + /// the caller needs cumulative duration and attempt-count context to diagnose + /// an upstream that silently drops connections. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn openrouter_post_truncated_200_body_wraps_in_terminal_error() { + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + use tokio::net::TcpListener; + + // Spawn a stub that sends a 200 with Content-Length > actual body, + // then closes the connection — reqwest sees a truncated stream. + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let url = format!("http://{}", listener.local_addr().unwrap()); + tokio::spawn(async move { + if let Ok((mut sock, _)) = listener.accept().await { + let mut buf = [0u8; 4096]; + // Read until end-of-headers, ignore body + loop { + match sock.read(&mut buf).await { + Ok(0) | Err(_) => break, + Ok(n) => { + if buf[..n].windows(4).any(|w| w == b"\r\n\r\n") { + break; + } + } + } + } + // Claim 1 MB of body, send only 10 bytes, then close. + let _ = sock + .write_all( + b"HTTP/1.1 200 OK\r\nContent-Type: application/json\r\n\ + Content-Length: 1048576\r\nConnection: close\r\n\r\n\ + {truncated", + ) + .await; + let _ = sock.shutdown().await; + } + }); + + let http = Client::builder() + .timeout(Duration::from_secs(5)) + .build() + .unwrap(); + let err = openrouter_post(&http, &format!("{url}/x"), &json!({}), "key") + .await + .unwrap_err(); + assert!( + matches!(&err, AgentError::Llm(s) if s.contains("body read")), + "truncated body must surface as AgentError::Llm with 'body read': got {err:?}" + ); + assert!( + matches!(&err, AgentError::Llm(s) if s.contains("cumulative")), + "body-read error must include terminal_llm_error's cumulative context: got {err:?}" + ); + } + + /// A `TokenSource` whose `refresh_now` always returns the identical token — + /// models a static API key whose bytes never change on refresh. + struct StaticAuth { + token: String, + } + + #[async_trait::async_trait] + impl TokenSource for StaticAuth { + async fn bearer(&self) -> Result { + Ok(self.token.clone()) + } + async fn refresh_now(&self, _rejected: &str) -> Result { + Ok(self.token.clone()) // static: same bytes every time + } + } + + /// A `TokenSource` that returns a stale token from `bearer()` and a + /// distinct fresh token from `refresh_now()`, modelling a PKCE OAuth source. + struct MintingAuth { + stale: String, + fresh: String, + refreshes: std::sync::atomic::AtomicU32, + } + + #[async_trait::async_trait] + impl TokenSource for MintingAuth { + async fn bearer(&self) -> Result { + Ok(self.stale.clone()) + } + async fn refresh_now(&self, _rejected: &str) -> Result { + self.refreshes + .fetch_add(1, std::sync::atomic::Ordering::SeqCst); + Ok(self.fresh.clone()) + } + } + + /// A static key 401: `refresh_now` returns the same bytes — the second + /// wire request would be byte-identical, so the retry must be skipped. + /// Exactly one wire request reaches the server. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn post_openrouter_static_key_401_single_attempt_no_retry() { + use std::sync::atomic::Ordering; + + let (url, _captured, attempts) = spawn_openrouter_stub(vec![CannedResponse::new( + 401, + r#"{"error":{"message":"invalid api key"}}"#, + )]) + .await; + let auth = Arc::new(StaticAuth { + token: "static-key".into(), + }); + let llm = llm_with(auth); + let mut c = cfg(Provider::OpenRouter); + c.base_url = url; + + let err = llm.post_openrouter(&c, &json!({})).await.unwrap_err(); + assert!( + matches!(&err, AgentError::LlmAuth(s) if s.contains("static key rejected")), + "static 401 must surface as LlmAuth with 'static key rejected': got {err:?}" + ); + assert_eq!( + attempts.load(Ordering::SeqCst), + 1, + "exactly one wire request — no duplicate retry for a static key" + ); + } + + /// A minting-source 401: `refresh_now` produces a distinct fresh token, so + /// the retry is legitimate. The stub accepts the fresh token's second + /// request with 200 and exactly two wire requests reach the server. + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn post_openrouter_minting_source_401_retries_with_fresh_token() { + use std::sync::atomic::Ordering; + + // Stub: always 401 for bearer "stale", 200 for anything else. + // We repurpose `spawn_auth_stub` here: it rejects `Bearer stale`, + // accepts `Bearer fresh`. + let always_401 = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false)); + let base = spawn_auth_stub(always_401, 401).await; + + let auth = Arc::new(MintingAuth { + stale: "stale".into(), + fresh: "fresh".into(), + refreshes: std::sync::atomic::AtomicU32::new(0), + }); + let llm = llm_with(auth.clone()); + let mut c = cfg(Provider::OpenRouter); + c.base_url = base; + + let result = llm.post_openrouter(&c, &json!({})).await; + // `spawn_auth_stub` returns `{"ok":true}` on success. + assert!( + result.is_ok(), + "minting-source retry should succeed: {result:?}" + ); + assert_eq!( + auth.refreshes.load(Ordering::SeqCst), + 1, + "exactly one refresh" + ); + } } diff --git a/crates/buzz-agent/src/mcp.rs b/crates/buzz-agent/src/mcp.rs index 744b10dc7a..9ae125a0b7 100644 --- a/crates/buzz-agent/src/mcp.rs +++ b/crates/buzz-agent/src/mcp.rs @@ -52,6 +52,31 @@ const PASSTHROUGH_ENV: &[&str] = &[ "GIT_ASKPASS", "GIT_SSH_COMMAND", "GIT_CONFIG_GLOBAL", + // Proxy — on a host whose only route out is a CONNECT proxy, dropping + // these does not degrade the tools, it blinds them: apt, curl, pip and git + // all connect directly instead, and the egress firewall resets the socket. + // The agent then reports "Connection reset by peer" and concludes the + // environment has no network, which is indistinguishable in the transcript + // from a task that is genuinely offline. + // + // Both cases are needed. curl and git read the lowercase spellings, most + // Go and Python tooling reads the uppercase ones, and libcurl deliberately + // ignores uppercase HTTP_PROXY (CGI ambiguity), so keeping only one form + // silently breaks half the toolchain. + "HTTP_PROXY", + "HTTPS_PROXY", + "NO_PROXY", + "ALL_PROXY", + "http_proxy", + "https_proxy", + "no_proxy", + "all_proxy", + // TLS trust — a proxy that terminates TLS presents its own CA, and an + // image whose trust store does not carry it fails every https fetch with a + // verification error. Same class of failure as the proxy vars: the parent + // was configured correctly and the child could not see it. + "SSL_CERT_FILE", + "SSL_CERT_DIR", // Buzz identity — dev-mcp writes NOSTR_PRIVATE_KEY to a keyfile then // removes it from its own env (children never see it). BUZZ_PRIVATE_KEY // and BUZZ_RELAY_URL are kept for the buzz CLI. BUZZ_AUTH_TAG is a @@ -1015,6 +1040,41 @@ mod content_tests { fn passthrough_includes_buzz_owner_attestation() { assert!(PASSTHROUGH_ENV.contains(&"BUZZ_AUTH_TAG")); } + + #[test] + fn passthrough_carries_proxy_configuration_to_tools() { + // On a proxy-only host this is the difference between an agent that can + // install a package and one that reports the network is down. Both + // spellings: libcurl ignores uppercase HTTP_PROXY, and Go/Python + // tooling largely ignores the lowercase set. + for var in [ + "HTTP_PROXY", + "HTTPS_PROXY", + "NO_PROXY", + "ALL_PROXY", + "http_proxy", + "https_proxy", + "no_proxy", + "all_proxy", + ] { + assert!( + PASSTHROUGH_ENV.contains(&var), + "{var} must survive env_clear() or every MCP tool loses the proxy" + ); + } + } + + #[test] + fn passthrough_carries_tls_trust_to_tools() { + // A TLS-terminating proxy presents its own CA; without these the child + // rejects every https fetch even though the proxy itself is reachable. + for var in ["SSL_CERT_FILE", "SSL_CERT_DIR"] { + assert!( + PASSTHROUGH_ENV.contains(&var), + "{var} must survive env_clear() or https fails inside tools" + ); + } + } use rmcp::model::Content; #[cfg(windows)] diff --git a/crates/buzz-agent/src/types.rs b/crates/buzz-agent/src/types.rs index d29e975e03..343a75bf72 100644 --- a/crates/buzz-agent/src/types.rs +++ b/crates/buzz-agent/src/types.rs @@ -1,5 +1,5 @@ use serde::Deserialize; -use serde_json::Value; +use serde_json::{Map, Value}; /// Byte-equivalent charged to the handoff/context-pressure gate for a single /// image tool result. The gate maps bytes to tokens at 1 byte/token (see @@ -62,6 +62,7 @@ pub enum HistoryItem { Assistant { text: String, tool_calls: Vec, + reasoning_details: Option, }, ToolResult(ToolResult), } @@ -83,7 +84,11 @@ impl HistoryItem { fn size_with(&self, content_size: fn(&ToolResultContent) -> usize) -> usize { match self { Self::User(s) => s.len(), - Self::Assistant { text, tool_calls } => { + Self::Assistant { + text, + tool_calls, + reasoning_details, + } => { text.len() + tool_calls .iter() @@ -93,8 +98,20 @@ impl HistoryItem { + serde_json::to_vec(&c.arguments) .map(|b| b.len()) .unwrap_or(0) + // `provider_extra` (e.g. a Gemini + // `thoughtSignature`) is re-serialized into + // every replayed call, so it counts toward the + // request body and the context-pressure gate. + + serde_json::to_vec(&c.provider_extra) + .map(|b| b.len()) + .unwrap_or(0) }) .sum::() + + reasoning_details + .as_ref() + .and_then(|v| serde_json::to_vec(v).ok()) + .map(|b| b.len()) + .unwrap_or(0) } Self::ToolResult(r) => { r.provider_id.len() + r.content.iter().map(content_size).sum::() @@ -108,6 +125,17 @@ pub struct ToolCall { pub provider_id: String, pub name: String, pub arguments: Value, + /// Fields the provider put on the tool call that we do not model, kept so + /// the assistant turn can be replayed the way it arrived. + /// + /// Gemini on the Databricks MLflow route returns a `thoughtSignature` per + /// call and *requires* it echoed back: replaying without it fails the whole + /// request with `Function call is missing a thought_signature in functionCall + /// parts`. For an agent loop that lands on the very first tool call, so the + /// model is unusable without this. Carrying whatever we did not model, + /// rather than naming that one field, means the next provider with an opaque + /// per-call token needs no change here. + pub provider_extra: Map, } #[derive(Debug, Clone)] @@ -139,10 +167,28 @@ pub struct LlmResponse { /// tokens, so reading it alone would undercount). Used to gate handoff on /// the real token budget rather than a byte estimate. pub input_tokens: Option, + /// The portion of `input_tokens` the provider served from its prompt cache, + /// or `None` when the response reported no cache split. Providers bill this + /// slice at a large discount (roughly 10x for both OpenAI and Anthropic), + /// so a consumer that prices all of `input_tokens` at the full rate + /// *overstates* cost — by a lot on an append-only agent loop, where most of + /// each request is a prefix the provider already has. + /// + /// This is a subset of `input_tokens`, never an addition to it: every + /// provider we speak to reports an inclusive input total, so adding this + /// would double-count. + pub cached_input_tokens: Option, /// Output tokens the provider reported for this request, or `None` if the /// response carried no usage. Used to accumulate per-turn output counts /// for NIP-AM metric publishing. pub output_tokens: Option, + /// Provider-reported total tokens for this request, or `None` when the + /// provider does not report a genuine total. Present for OpenAI-shaped + /// responses (`usage.total_tokens`). Always `None` for Anthropic, which + /// reports only category counts; NIP-AM forbids summing categories into a + /// total. Callers must not derive this by summing `input_tokens + + /// output_tokens` — that is what the UI display approximation is for. + pub total_tokens: Option, /// Reasoning/thinking content emitted by the model before its answer, if /// any. Non-empty when the provider returns extended-thinking tokens: /// @@ -152,6 +198,10 @@ pub struct LlmResponse { /// /// Empty string when the provider returned no reasoning content. pub reasoning: String, + /// Raw `reasoning_details` array from an OpenRouter response, if present. + /// Replayed on subsequent turns so the model can continue its chain-of-thought. + /// `None` for all non-OpenRouter providers. + pub reasoning_details: Option, } #[derive(Debug, Clone, Copy, PartialEq)] @@ -170,6 +220,94 @@ pub struct ToolDef { pub input_schema: Value, } +/// Tri-state accumulator for provider-reported total tokens within one ACP turn. +/// +/// Tracks whether every usage-bearing LLM response in the turn supplied a genuine +/// provider total. Used to accumulate a reliable per-turn total and contribute to +/// the session-cumulative total. +/// +/// - `Unseen`: no usage-bearing response observed yet (initial state for each turn). +/// - `Exact(n)`: every response so far reported a total; `n` is their sum. +/// - `Unknown`: at least one response lacked a total — permanently poisoned for +/// this turn. The session-cumulative also transitions to Unknown when any turn +/// lands Unknown, and stays there until a new session resets it. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum TurnTotalState { + #[default] + Unseen, + Exact(u64), + Unknown, +} + +impl TurnTotalState { + /// Add two exact token counts with overflow protection. + /// + /// Returns `Exact(acc + n)` on success or `Unknown` on overflow. + /// This is the single implementation of the checked-add / overflow-poisons + /// contract; both `fold()` and `merge_session()` call this helper so a + /// change to overflow semantics needs to be made in exactly one place. + fn checked_exact_sum(acc: u64, n: u64) -> TurnTotalState { + match acc.checked_add(n) { + Some(sum) => TurnTotalState::Exact(sum), + None => TurnTotalState::Unknown, + } + } + + /// Fold one provider-reported total into the current state. + /// + /// `total`: `Some(n)` when the provider included a genuine total on this + /// response; `None` when it was absent (e.g. Anthropic, or an OpenAI + /// response that omits usage). Absence of a total on any usage-bearing + /// response poisons the whole turn. + /// + /// Overflow is handled by `checked_exact_sum`: a saturated value would + /// not be a genuine provider-reported total, so overflow → `Unknown`. + pub fn fold(self, total: Option) -> TurnTotalState { + match (self, total) { + // Already poisoned — stays Unknown regardless. + (TurnTotalState::Unknown, _) => TurnTotalState::Unknown, + // No total from this response — poison the accumulator. + (_, None) => TurnTotalState::Unknown, + // First response with a total. + (TurnTotalState::Unseen, Some(n)) => TurnTotalState::Exact(n), + // Subsequent response — delegate to the shared checked-sum helper. + (TurnTotalState::Exact(acc), Some(n)) => Self::checked_exact_sum(acc, n), + } + } + + /// Merge a completed turn's total state into the session-cumulative state. + /// + /// This is the turn→session boundary accumulation: + /// - An `Unseen` turn (no usage-bearing responses) leaves the cumulative unchanged. + /// - Any `Unknown` side poisons the session permanently. + /// - Two `Exact` values are summed via `checked_exact_sum`; overflow → `Unknown`. + /// + /// The checked-add logic lives in `checked_exact_sum`; both this function and + /// `fold()` call that helper so overflow semantics are defined once. + pub fn merge_session(self, turn: TurnTotalState) -> TurnTotalState { + match (self, turn) { + // Either side poisoned → session is poisoned. + (TurnTotalState::Unknown, _) | (_, TurnTotalState::Unknown) => TurnTotalState::Unknown, + // Turn had no usage-bearing responses → no change to cumulative. + (acc, TurnTotalState::Unseen) => acc, + // First exact turn — adopt its value. + (TurnTotalState::Unseen, TurnTotalState::Exact(n)) => TurnTotalState::Exact(n), + // Add to running exact sum — delegate to the shared checked-sum helper. + (TurnTotalState::Exact(acc), TurnTotalState::Exact(n)) => { + Self::checked_exact_sum(acc, n) + } + } + } + + /// Consume the exact value if present; `None` for `Unseen` or `Unknown`. + pub fn exact_value(self) -> Option { + match self { + TurnTotalState::Exact(n) => Some(n), + _ => None, + } + } +} + #[derive(Debug, Clone, Copy, PartialEq)] pub enum StopReason { EndTurn, @@ -342,6 +480,41 @@ mod tests { assert!(item.estimated_bytes() >= 3_118_884); } + #[test] + fn assistant_size_counts_provider_extra() { + // A Gemini `thoughtSignature` rides the wire on every replayed call, so + // both size measures must see it — otherwise `truncate_history` and the + // handoff gate under-count and let the real request exceed the budget. + let mut extra = Map::new(); + extra.insert("thoughtSignature".into(), Value::String("S".repeat(500))); + let with_extra = HistoryItem::Assistant { + text: String::new(), + tool_calls: vec![ToolCall { + provider_id: "id".into(), + name: "t".into(), + arguments: Value::Null, + provider_extra: extra, + }], + reasoning_details: None, + }; + let without_extra = HistoryItem::Assistant { + text: String::new(), + tool_calls: vec![ToolCall { + provider_id: "id".into(), + name: "t".into(), + arguments: Value::Null, + provider_extra: Map::new(), + }], + reasoning_details: None, + }; + assert!(with_extra.estimated_bytes() > without_extra.estimated_bytes() + 500); + assert_eq!( + with_extra.estimated_bytes(), + with_extra.context_pressure_bytes(), + "provider_extra is text, so both measures must agree" + ); + } + #[test] fn text_content_size_is_identical_for_both_measures() { // Only images diverge; text must size the same under both paths. @@ -351,3 +524,138 @@ mod tests { assert_eq!(item.estimated_bytes(), item.context_pressure_bytes()); } } + +#[cfg(test)] +mod turn_total_state_tests { + use super::TurnTotalState; + + // ── TurnTotalState::fold ─────────────────────────────────────────────── + + #[test] + fn fold_first_response_with_total_becomes_exact() { + let state = TurnTotalState::Unseen; + assert_eq!(state.fold(Some(100)), TurnTotalState::Exact(100)); + } + + #[test] + fn fold_first_response_without_total_becomes_unknown() { + // Missing total on any usage-bearing response poisons the turn. + let state = TurnTotalState::Unseen; + assert_eq!(state.fold(None), TurnTotalState::Unknown); + } + + #[test] + fn multiple_provider_rounds_all_with_totals_sum_correctly() { + // Multiple rounds all reporting a genuine total → Exact with their sum. + let state = TurnTotalState::Unseen; + let state = state.fold(Some(100)); + let state = state.fold(Some(50)); + let state = state.fold(Some(75)); + assert_eq!(state, TurnTotalState::Exact(225)); + } + + #[test] + fn mixed_present_and_missing_totals_within_one_turn_poisons_accumulator() { + // First round has a total, second does not → Unknown (permanently poisoned). + let state = TurnTotalState::Unseen; + let state = state.fold(Some(100)); // Exact(100) + let state = state.fold(None); // Missing → Unknown + assert_eq!(state, TurnTotalState::Unknown); + // Further rounds with totals don't un-poison. + let state = state.fold(Some(50)); + assert_eq!(state, TurnTotalState::Unknown); + } + + #[test] + fn unknown_stays_unknown_regardless_of_subsequent_totals() { + // Once poisoned, no subsequent total can recover the state. + let state = TurnTotalState::Unknown; + assert_eq!(state.fold(Some(999)), TurnTotalState::Unknown); + assert_eq!(state.fold(None), TurnTotalState::Unknown); + } + + #[test] + fn exact_value_returns_some_only_for_exact_variant() { + assert_eq!(TurnTotalState::Unseen.exact_value(), None); + assert_eq!(TurnTotalState::Unknown.exact_value(), None); + assert_eq!(TurnTotalState::Exact(42).exact_value(), Some(42)); + } + + #[test] + fn default_is_unseen() { + let state: TurnTotalState = Default::default(); + assert_eq!(state, TurnTotalState::Unseen); + } + + // ── overflow: fold ───────────────────────────────────────────────────── + + #[test] + fn fold_overflow_poisons_turn_not_saturates() { + // u64::MAX + 1 would saturate; checked_add must poison instead. + let state = TurnTotalState::Exact(u64::MAX); + assert_eq!( + state.fold(Some(1)), + TurnTotalState::Unknown, + "overflow in fold() must produce Unknown, not Exact(u64::MAX)" + ); + } + + // ── TurnTotalState::merge_session ────────────────────────────────────── + + #[test] + fn merge_session_unseen_turn_leaves_cumulative_unchanged() { + // An Unseen turn (no usage-bearing responses) must not alter the cumulative. + assert_eq!( + TurnTotalState::Exact(100).merge_session(TurnTotalState::Unseen), + TurnTotalState::Exact(100), + ); + assert_eq!( + TurnTotalState::Unseen.merge_session(TurnTotalState::Unseen), + TurnTotalState::Unseen, + ); + } + + #[test] + fn merge_session_exact_turn_adds_to_exact_cumulative() { + assert_eq!( + TurnTotalState::Exact(100).merge_session(TurnTotalState::Exact(50)), + TurnTotalState::Exact(150), + ); + } + + #[test] + fn merge_session_first_exact_turn_from_unseen_adopts_value() { + assert_eq!( + TurnTotalState::Unseen.merge_session(TurnTotalState::Exact(200)), + TurnTotalState::Exact(200), + ); + } + + #[test] + fn merge_session_unknown_turn_poisons_cumulative_permanently() { + assert_eq!( + TurnTotalState::Exact(100).merge_session(TurnTotalState::Unknown), + TurnTotalState::Unknown, + ); + // Poisoned session stays poisoned even with Unseen turn. + assert_eq!( + TurnTotalState::Unknown.merge_session(TurnTotalState::Unseen), + TurnTotalState::Unknown, + ); + // Poisoned session stays poisoned even with another Exact turn. + assert_eq!( + TurnTotalState::Unknown.merge_session(TurnTotalState::Exact(999)), + TurnTotalState::Unknown, + ); + } + + #[test] + fn merge_session_overflow_poisons_not_saturates() { + // Overflow at the session boundary must also produce Unknown. + assert_eq!( + TurnTotalState::Exact(u64::MAX).merge_session(TurnTotalState::Exact(1)), + TurnTotalState::Unknown, + "overflow in merge_session() must produce Unknown, not Exact(u64::MAX)" + ); + } +} diff --git a/crates/buzz-agent/tests/bin/fake_mcp.rs b/crates/buzz-agent/tests/bin/fake_mcp.rs index 0bbd1d3478..5b660da48c 100644 --- a/crates/buzz-agent/tests/bin/fake_mcp.rs +++ b/crates/buzz-agent/tests/bin/fake_mcp.rs @@ -33,6 +33,11 @@ //! — expose a `_PostCompact` hook tool //! FAKE_MCP_POSTCOMPACT_TEXT=text //! — `_PostCompact` returns this (default: "") +//! FAKE_MCP_SHELL_TOOL=1 — expose a tool whose bare name is `shell` +//! (registered as `__shell`), taking a +//! `command` string. Lets a test drive the +//! reply guard's recognition of a real, +//! registered shell tool. use std::io::{BufRead, Write}; @@ -76,6 +81,7 @@ fn make_tools( desc: &str, include_stop_hook: bool, include_post_compact_hook: bool, + include_shell_tool: bool, ) -> Vec { let mut tools: Vec = (0..count) .map(|i| { @@ -100,6 +106,17 @@ fn make_tools( "inputSchema": { "type": "object", "properties": {} }, })); } + if include_shell_tool { + tools.push(json!({ + "name": "shell", + "description": "run a shell command", + "inputSchema": { + "type": "object", + "properties": { "command": { "type": "string" } }, + "required": ["command"], + }, + })); + } tools } @@ -136,6 +153,7 @@ fn main() { let stop_count_limit: usize = env_usize("FAKE_MCP_STOP_COUNT", usize::MAX); let mut stop_calls_seen: usize = 0; let post_compact_hook = env_flag("FAKE_MCP_POSTCOMPACT_HOOK"); + let shell_tool = env_flag("FAKE_MCP_SHELL_TOOL"); let post_compact_text = std::env::var("FAKE_MCP_POSTCOMPACT_TEXT").unwrap_or_default(); // Use a channel-based stdin reader so notifications (which carry no id) @@ -206,7 +224,13 @@ fn main() { write_response( id, json!({ - "tools": make_tools(tool_count, &desc, stop_hook, post_compact_hook) + "tools": make_tools( + tool_count, + &desc, + stop_hook, + post_compact_hook, + shell_tool, + ) }), ); } diff --git a/crates/buzz-agent/tests/fake_llm.rs b/crates/buzz-agent/tests/fake_llm.rs index f2a86ac4b3..f782a9d476 100644 --- a/crates/buzz-agent/tests/fake_llm.rs +++ b/crates/buzz-agent/tests/fake_llm.rs @@ -771,6 +771,26 @@ fn openai_text_with_usage(content: &str, input_tokens: u64, output_tokens: u64) }) } +/// An OpenAI chat completion response WITH i/o usage but WITHOUT `total_tokens`. +/// Simulates a provider that omits the genuine total from its usage block. +/// buzz-agent must treat this turn's total as Unknown and poison the cumulative. +fn openai_text_with_usage_no_total(content: &str, input_tokens: u64, output_tokens: u64) -> Value { + json!({ + "id": "cc-nt", "object": "chat.completion", "model": "fake-model", + "choices": [{ + "index": 0, + "message": { "role": "assistant", "content": content }, + "finish_reason": "stop", + }], + "usage": { + "prompt_tokens": input_tokens, + "completion_tokens": output_tokens, + // total_tokens deliberately absent — simulates Anthropic or any + // provider that does not report a genuine total. + }, + }) +} + /// Returns true when `v` is a `_goose/unstable/session/update` usage_update /// notification. fn is_usage_update(v: &Value) -> bool { @@ -1123,3 +1143,164 @@ async fn steer_rejected_on_empty_prompt() { assert!(saw_reject, "empty steer prompt was not rejected"); h.shutdown().await; } + +// ─── Session-boundary total accumulation ──────────────────────────────────── + +/// Once a usage-bearing turn lacks a provider total, the session cumulative +/// becomes Unknown and `accumulatedTotalTokens` must be absent from subsequent +/// `usage_update` notifications — even if later turns supply a total. +/// +/// Sequence: turn 1 has total, turn 2 lacks total → session poisoned, turn 3 +/// has total → still poisoned. Only turn 1 must carry `accumulatedTotalTokens`. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn session_total_poisoned_by_missing_total_and_stays_poisoned() { + let url = spawn_fake_llm(vec![ + openai_text_with_usage("t1", 10, 5), // total present → Exact(15) + openai_text_with_usage_no_total("t2", 20, 8), // total absent → Unknown + openai_text_with_usage("t3", 15, 6), // total present → still Unknown + ]) + .await; + let mut h = Harness::spawn(&url).await; + let sid = init_session(&mut h).await; + + // ── Turn 1: total present ─────────────────────────────────────────────── + let p1 = h + .send( + "session/prompt", + json!({"sessionId": sid, "prompt": [{"type":"text","text":"t1"}]}), + ) + .await; + let (frames1, _) = recv_until_with_drain(&mut h, |v| v["id"] == p1).await; + let usage1 = frames1 + .iter() + .find(|v| is_usage_update(v)) + .expect("usage_update for turn 1"); + assert_eq!( + usage1["params"]["update"]["accumulatedTotalTokens"], + json!(15u64), + "turn 1 has genuine total; accumulatedTotalTokens must be 15" + ); + + // ── Turn 2: total absent — session is now poisoned ────────────────────── + let p2 = h + .send( + "session/prompt", + json!({"sessionId": sid, "prompt": [{"type":"text","text":"t2"}]}), + ) + .await; + let (frames2, _) = recv_until_with_drain(&mut h, |v| v["id"] == p2).await; + let usage2 = frames2 + .iter() + .find(|v| is_usage_update(v)) + .expect("usage_update for turn 2"); + assert!( + usage2["params"]["update"]["accumulatedTotalTokens"].is_null() + || usage2["params"]["update"] + .get("accumulatedTotalTokens") + .is_none(), + "turn 2 lacked total; accumulatedTotalTokens must be absent/null; got: {usage2:#?}" + ); + + // ── Turn 3: total present, but session is still poisoned ───────────────── + let p3 = h + .send( + "session/prompt", + json!({"sessionId": sid, "prompt": [{"type":"text","text":"t3"}]}), + ) + .await; + let (frames3, _) = recv_until_with_drain(&mut h, |v| v["id"] == p3).await; + let usage3 = frames3 + .iter() + .find(|v| is_usage_update(v)) + .expect("usage_update for turn 3"); + assert!( + usage3["params"]["update"]["accumulatedTotalTokens"].is_null() + || usage3["params"]["update"].get("accumulatedTotalTokens").is_none(), + "session is poisoned; accumulatedTotalTokens must remain absent even after a total-bearing turn; got: {usage3:#?}" + ); + + // i/o counters are unaffected by total poisoning. + assert_eq!( + usage3["params"]["update"]["accumulatedInputTokens"], + json!(45u64), + "poisoned total must not discard input accumulation" + ); + assert_eq!( + usage3["params"]["update"]["accumulatedOutputTokens"], + json!(19u64), + "poisoned total must not discard output accumulation" + ); + + h.shutdown().await; +} + +/// A new session starts fresh and can accumulate an exact total independently +/// of any previous session. This verifies `accumulated_total_state` is reset +/// to `Unseen` on `session/new`, not inherited from a prior session. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn new_session_resets_total_accumulation() { + // Session A: two turns both with totals → Exact should accumulate. + // Session B (new session/new call): starts fresh. + let url = spawn_fake_llm(vec![ + // Session A, turn 1 + openai_text_with_usage("s1t1", 10, 5), + // Session A, turn 2 + openai_text_with_usage("s1t2", 20, 8), + // Session B, turn 1 + openai_text_with_usage("s2t1", 30, 10), + ]) + .await; + let mut h = Harness::spawn(&url).await; + let sid_a = init_session(&mut h).await; + + // Session A, turn 1 + let p1 = h + .send( + "session/prompt", + json!({"sessionId": sid_a, "prompt": [{"type":"text","text":"s1t1"}]}), + ) + .await; + let (frames1, _) = recv_until_with_drain(&mut h, |v| v["id"] == p1).await; + let u1 = frames1.iter().find(|v| is_usage_update(v)).expect("usage1"); + assert_eq!( + u1["params"]["update"]["accumulatedTotalTokens"], + json!(15u64), + "session A turn 1 accumulated total" + ); + + // Session A, turn 2 — cumulative total is 15+28=43 + let p2 = h + .send( + "session/prompt", + json!({"sessionId": sid_a, "prompt": [{"type":"text","text":"s1t2"}]}), + ) + .await; + let (frames2, _) = recv_until_with_drain(&mut h, |v| v["id"] == p2).await; + let u2 = frames2.iter().find(|v| is_usage_update(v)).expect("usage2"); + assert_eq!( + u2["params"]["update"]["accumulatedTotalTokens"], + json!(43u64), + "session A turn 2 cumulative total must be 15+28=43" + ); + + // Start a new session — must reset accumulated_total_state to Unseen. + let sid_b = init_session(&mut h).await; + assert_ne!(sid_a, sid_b, "sessions must have distinct IDs"); + + // Session B, turn 1 — total 30+10=40. Must NOT start from 43. + let p3 = h + .send( + "session/prompt", + json!({"sessionId": sid_b, "prompt": [{"type":"text","text":"s2t1"}]}), + ) + .await; + let (frames3, _) = recv_until_with_drain(&mut h, |v| v["id"] == p3).await; + let u3 = frames3.iter().find(|v| is_usage_update(v)).expect("usage3"); + assert_eq!( + u3["params"]["update"]["accumulatedTotalTokens"], + json!(40u64), + "new session must start fresh — accumulated total must be 40, not 83" + ); + + h.shutdown().await; +} diff --git a/crates/buzz-agent/tests/regressions.rs b/crates/buzz-agent/tests/regressions.rs index 2e0b579c84..abb4f7b311 100644 --- a/crates/buzz-agent/tests/regressions.rs +++ b/crates/buzz-agent/tests/regressions.rs @@ -1819,3 +1819,465 @@ async fn cancel_sends_notifications_cancelled_to_any_mcp_server() { let _ = std::fs::remove_file(&call_received_marker); h.shutdown().await; } + +// --------------------------------------------------------------------------- +// Reply guard (`BUZZ_AGENT_REQUIRE_REPLY`) +// +// The guard reminds the model to publish when a turn is about to end without +// any recognized attempt to post to Buzz. It rides the existing `_Stop` gate +// and shares its rejection budget, so most of these tests count LLM calls: +// each reminder costs exactly one extra round. +// --------------------------------------------------------------------------- + +/// Number of reply-guard reminders present in one captured LLM request. +/// +/// A reminder is a tool-role message whose JSON body is attributed to the +/// in-process guard (`server: "buzz-agent"`) at the `_Stop` hook point — the +/// same lower-trust shape as real hook output. +fn reply_nag_count(request: &Value) -> usize { + request["messages"] + .as_array() + .map(|msgs| { + msgs.iter() + .filter(|m| { + m["role"] == "tool" + && serde_json::from_str::(m["content"].as_str().unwrap_or("")) + .map(|p| p["hook"] == "_Stop" && p["server"] == "buzz-agent") + .unwrap_or(false) + }) + .count() + }) + .unwrap_or(0) +} + +/// A publish-shaped call to a real registered shell tool. +fn openai_shell_send(id: &str) -> Value { + openai_tool_call( + id, + "fake__shell", + json!({ "command": "buzz messages send --channel c --content hi" }), + ) +} + +/// Run one prompt to completion, answering any permission requests, and +/// return the final response. +async fn prompt_to_completion(h: &mut Harness, sid: &str) -> Value { + let p = h + .send( + "session/prompt", + json!({"sessionId": sid, "prompt": [{"type":"text","text":"go"}]}), + ) + .await; + loop { + let v = h.recv().await; + if v.get("method") == Some(&json!("session/request_permission")) { + let id = v["id"].clone(); + h.write(json!({ + "jsonrpc": "2.0", + "id": id, + "result": { "outcome": { "outcome": "selected", "optionId": "allow" } }, + })) + .await; + continue; + } + if v["id"] == json!(p) { + return v; + } + } +} + +/// Default off: a silent turn ends on the first end_turn with no extra round. +/// This is the invariant that keeps the feature free for everyone who hasn't +/// opted in. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_off_by_default() { + let llm = spawn_capturing_llm(vec![openai_text("done"), openai_text("unexpected")]).await; + let mut h = Harness::spawn(&llm.url).await; + let sid = init_session(&mut h, json!([])).await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 1, + "guard must be inert when unset, got {} LLM calls", + captured.len() + ); + h.shutdown().await; +} + +/// `BUZZ_AGENT_REQUIRE_REPLY=0` is off too — the toggle is numeric, so a +/// literal `0` must not read as "set, therefore on". +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_explicit_zero_is_off() { + let llm = spawn_capturing_llm(vec![openai_text("done"), openai_text("unexpected")]).await; + let mut h = Harness::spawn_with_env(&llm.url, &[("BUZZ_AGENT_REQUIRE_REPLY", "0")]).await; + let sid = init_session(&mut h, json!([])).await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 1, + "REQUIRE_REPLY=0 must behave as off, got {} LLM calls", + captured.len() + ); + h.shutdown().await; +} + +/// Opted in and silent: exactly two reminders, then the turn is allowed to +/// end. The guard is advisory — it must never trap a turn. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_nags_twice_then_lets_the_turn_end() { + // Budget defaults to 3, so the cap that stops the loop here is + // MAX_REPLY_NAGS = 2, not the rejection budget. + let llm = spawn_capturing_llm(vec![ + openai_text("silent-1"), + openai_text("silent-2"), + openai_text("silent-3"), + openai_text("must-not-be-requested"), + ]) + .await; + let mut h = Harness::spawn_with_env(&llm.url, &[("BUZZ_AGENT_REQUIRE_REPLY", "1")]).await; + let sid = init_session(&mut h, json!([])).await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 3, + "expected 2 reminders then end_turn (3 LLM calls), got {}", + captured.len() + ); + assert_eq!( + reply_nag_count(&captured[0]), + 0, + "reminder before any end_turn" + ); + assert_eq!(reply_nag_count(&captured[1]), 1); + assert_eq!(reply_nag_count(&captured[2]), 2); + + // The reminder must name the command it wants and license silence, so it + // cannot fight the base prompt's "silence is usually correct". + let msgs = captured[2]["messages"].as_array().unwrap(); + let nag = msgs + .iter() + .filter_map(|m| serde_json::from_str::(m["content"].as_str().unwrap_or("")).ok()) + .find(|p| p["server"] == "buzz-agent") + .expect("reminder body"); + let text = nag["text"].as_str().unwrap_or(""); + assert!( + text.contains("buzz messages send"), + "reminder should name the command: {text}" + ); + assert!( + text.contains("silence is genuinely correct"), + "reminder must license silence: {text}" + ); + h.shutdown().await; +} + +/// A real publish attempt through a registered shell tool satisfies the guard: +/// no reminder, no extra round. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_satisfied_by_registered_shell_send() { + let llm = spawn_capturing_llm(vec![ + openai_shell_send("tc1"), + openai_text("posted"), + openai_text("must-not-be-requested"), + ]) + .await; + let mut h = Harness::spawn_with_env(&llm.url, &[("BUZZ_AGENT_REQUIRE_REPLY", "1")]).await; + let sid = init_session_with_fake_mcp( + &mut h, + &[("FAKE_MCP_TOOL_COUNT", "1"), ("FAKE_MCP_SHELL_TOOL", "1")], + ) + .await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 2, + "a recognized send must not be nagged, got {} LLM calls", + captured.len() + ); + assert_eq!(reply_nag_count(&captured[1]), 0); + h.shutdown().await; +} + +/// A publish-shaped call to a shell tool that is *not registered* never runs — +/// preflight rejects it — so it must not disarm the guard. This is what the +/// `has`/`is_hook` checks in the predicate buy. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_ignores_unregistered_shell_tool() { + // FAKE_MCP_SHELL_TOOL is absent, so `fake__shell` is a hallucination. + let llm = spawn_capturing_llm(vec![ + openai_shell_send("tc1"), + openai_text("silent-1"), + openai_text("silent-2"), + ]) + .await; + let mut h = Harness::spawn_with_env( + &llm.url, + &[ + ("BUZZ_AGENT_REQUIRE_REPLY", "1"), + ("BUZZ_AGENT_STOP_MAX_REJECTIONS", "1"), + ], + ) + .await; + let sid = init_session_with_fake_mcp(&mut h, &[("FAKE_MCP_TOOL_COUNT", "1")]).await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 3, + "expected the hallucinated call to still be nagged, got {} LLM calls", + captured.len() + ); + let msgs = captured[1]["messages"].as_array().unwrap(); + assert!( + msgs.iter() + .any(|m| m["role"] == "tool" + && m["content"].as_str().unwrap_or("").contains("unknown tool")), + "expected preflight to reject the call: {msgs:?}" + ); + assert_eq!(reply_nag_count(&captured[2]), 1); + h.shutdown().await; +} + +/// A publish-shaped call discarded by the per-turn tool-call cap never runs, +/// so it must not suppress the reminder either. Pins the check's placement +/// after `calls.truncate(MAX_TOOL_CALLS_PER_TURN)`. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_ignores_calls_lost_to_the_turn_cap() { + // 64 filler calls (the cap) followed by the publish attempt, which is + // therefore truncated away. The shell tool *is* registered here, so only + // the placement — not tool identity — can explain the reminder. + let mut calls: Vec = (0..64) + .map(|i| { + json!({ + "id": format!("c{i}"), + "type": "function", + "function": { "name": "fake__tool_0", "arguments": "{}" }, + }) + }) + .collect(); + calls.push(json!({ + "id": "c-send", + "type": "function", + "function": { + "name": "fake__shell", + "arguments": json!({ "command": "buzz messages send --channel c --content hi" }) + .to_string(), + }, + })); + let truncated_send = json!({ + "id": "cc-trunc", "object": "chat.completion", "model": "fake-model", + "choices": [{ + "index": 0, + "message": { "role": "assistant", "content": null, "tool_calls": calls }, + "finish_reason": "tool_calls", + }], + }); + let llm = spawn_capturing_llm(vec![ + truncated_send, + openai_text("silent-1"), + openai_text("silent-2"), + ]) + .await; + let mut h = Harness::spawn_with_env( + &llm.url, + &[ + ("BUZZ_AGENT_REQUIRE_REPLY", "1"), + ("BUZZ_AGENT_STOP_MAX_REJECTIONS", "1"), + ], + ) + .await; + let sid = init_session_with_fake_mcp( + &mut h, + &[("FAKE_MCP_TOOL_COUNT", "1"), ("FAKE_MCP_SHELL_TOOL", "1")], + ) + .await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 3, + "a truncated send must still be nagged, got {} LLM calls", + captured.len() + ); + assert_eq!(reply_nag_count(&captured[2]), 1); + h.shutdown().await; +} + +/// The shared `_Stop` rejection budget is the outer cap: at 1 the guard gets +/// one reminder instead of two. Documented degradation, not a bug. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_bounded_by_stop_rejection_budget() { + let llm = spawn_capturing_llm(vec![ + openai_text("silent-1"), + openai_text("silent-2"), + openai_text("must-not-be-requested"), + ]) + .await; + let mut h = Harness::spawn_with_env( + &llm.url, + &[ + ("BUZZ_AGENT_REQUIRE_REPLY", "1"), + ("BUZZ_AGENT_STOP_MAX_REJECTIONS", "1"), + ], + ) + .await; + let sid = init_session(&mut h, json!([])).await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 2, + "budget 1 must allow exactly one reminder, got {} LLM calls", + captured.len() + ); + assert_eq!(reply_nag_count(&captured[1]), 1); + h.shutdown().await; +} + +/// Budget 0 disables every objection at the gate, including this one. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_off_when_stop_budget_is_zero() { + let llm = spawn_capturing_llm(vec![openai_text("done"), openai_text("unexpected")]).await; + let mut h = Harness::spawn_with_env( + &llm.url, + &[ + ("BUZZ_AGENT_REQUIRE_REPLY", "1"), + ("BUZZ_AGENT_STOP_MAX_REJECTIONS", "0"), + ], + ) + .await; + let sid = init_session(&mut h, json!([])).await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 1, + "budget 0 must disable the guard, got {} LLM calls", + captured.len() + ); + h.shutdown().await; +} + +/// The two axes are independent inside one shared budget: a round carrying +/// both a `_Stop` hook objection and a reminder costs one rejection and +/// delivers both texts, and once the reminders are spent the hook objection +/// continues alone. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn reply_guard_combines_with_stop_hook_objection() { + // The hook objects on its first 3 calls, then clears. Reminders stop + // after 2, so round 3 must carry the hook text and no new reminder. + let llm = spawn_capturing_llm(vec![ + openai_text("silent-1"), + openai_text("silent-2"), + openai_text("silent-3"), + openai_text("silent-4"), + ]) + .await; + let mut h = Harness::spawn_with_env( + &llm.url, + &[ + ("BUZZ_AGENT_REQUIRE_REPLY", "1"), + ("MCP_HOOK_SERVERS", "fake"), + ("BUZZ_AGENT_STOP_MAX_REJECTIONS", "10"), + ], + ) + .await; + let sid = init_session_with_fake_mcp( + &mut h, + &[ + ("FAKE_MCP_TOOL_COUNT", "1"), + ("FAKE_MCP_STOP_HOOK", "1"), + ("FAKE_MCP_STOP_TEXT", "you have open todos"), + ("FAKE_MCP_STOP_COUNT", "3"), + ], + ) + .await; + + let r = prompt_to_completion(&mut h, &sid).await; + assert_eq!(r["result"]["stopReason"], "end_turn"); + + let captured = llm.captured.lock().await; + assert_eq!( + captured.len(), + 4, + "expected 3 objecting rounds then a clear end, got {}", + captured.len() + ); + + let hook_objections = |req: &Value| -> usize { + req["messages"] + .as_array() + .map(|msgs| { + msgs.iter() + .filter(|m| { + m["content"] + .as_str() + .unwrap_or("") + .contains("you have open todos") + }) + .count() + }) + .unwrap_or(0) + }; + + // Round 2 carries one of each — a single rejection bought both texts. + assert_eq!(reply_nag_count(&captured[1]), 1); + assert_eq!(hook_objections(&captured[1]), 1); + // Round 4: the hook objected three times, the guard only twice. + assert_eq!(reply_nag_count(&captured[3]), 2); + assert_eq!(hook_objections(&captured[3]), 3); + h.shutdown().await; +} + +/// An unparseable toggle is a startup error, not a silent default. `parse_env` +/// is generic over `FromStr`, so this also pins the numeric type: a `bool` +/// field would have rejected the documented `1`. +#[test] +fn reply_guard_rejects_unparseable_toggle() { + let out = std::process::Command::new(env!("CARGO_BIN_EXE_buzz-agent")) + .env("BUZZ_AGENT_PROVIDER", "openai") + .env("OPENAI_COMPAT_API_KEY", "test") + .env("OPENAI_COMPAT_MODEL", "fake-model") + .env("BUZZ_AGENT_REQUIRE_REPLY", "true") + .stdin(Stdio::null()) + .output() + .expect("run buzz-agent"); + assert!( + !out.status.success(), + "expected a config error exit, got {:?}", + out.status + ); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + stderr.contains("BUZZ_AGENT_REQUIRE_REPLY"), + "expected the offending key in the error, got: {stderr}" + ); +} diff --git a/crates/buzz-backend-kubernetes/Cargo.toml b/crates/buzz-backend-kubernetes/Cargo.toml new file mode 100644 index 0000000000..1cd17030db --- /dev/null +++ b/crates/buzz-backend-kubernetes/Cargo.toml @@ -0,0 +1,36 @@ +[package] +name = "buzz-backend-kubernetes" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +description = "Kubernetes backend provider for Buzz remote agents (docs/remote-agents.md)" + +[[bin]] +name = "buzz-backend-kubernetes" +path = "src/main.rs" + +[dependencies] +kube = { workspace = true } +k8s-openapi = { workspace = true } +nostr = { workspace = true } +tokio = { workspace = true } +serde = { workspace = true } +serde_json = { workspace = true } +sha2 = { workspace = true } +hex = { workspace = true } +rand = { workspace = true } +chrono = { workspace = true } +http = "1" +http-body-util = "0.1" + +# Explicit rustls dep with the ring provider — required to install the +# process-level CryptoProvider at startup. Without it this binary panics on its +# first TLS connection to the apiserver: the release build compiles every +# sidecar in one cargo invocation (.github/workflows/release.yml), which unifies +# both ring and aws-lc-rs features and leaves rustls unable to auto-select a +# provider. Same dependency and reason as crates/buzz-cli/Cargo.toml. +rustls = { version = "0.23", default-features = false, features = ["ring", "std"] } + +[dev-dependencies] +tower = { workspace = true } diff --git a/crates/buzz-backend-kubernetes/src/classify.rs b/crates/buzz-backend-kubernetes/src/classify.rs new file mode 100644 index 0000000000..e9cfaba402 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/classify.rs @@ -0,0 +1,377 @@ +//! The deploy state machine (spec §Deploy State Machine), as a pure function. +//! +//! `classify` maps a verified observation plus the desired create intent to +//! one [`Action`]. It performs no I/O, so every row of the spec's table is a +//! unit test with no cluster. `reconcile` executes actions and re-enters. +//! +//! Two invariants are structural rather than remembered: +//! +//! * [`Action::Delete`] carries the [`Fence`] from the exact observation that +//! authorized it. There is no way to build a delete without one, so a later +//! helper cannot re-read and silently substitute a fresher fence. +//! * The pull-failure classifier ([`PullFailure`]) reaches only +//! [`Action::Report`] and [`Action::Observe`]. It is absent from +//! `Action::Delete`'s type, so "reason strings are never deletion +//! authority" is enforced by the compiler. + +use crate::intent::Fingerprint; + +/// The compare-and-delete fence: UID + resourceVersion from the observation +/// that authorized the deletion. A failed precondition means the object +/// changed since the read — re-enter, never retry the delete. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Fence { + pub uid: String, + pub resource_version: String, +} + +/// Why a pod that never started looks permanently broken. Reporting only. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum PullFailure { + /// Registry auth: a 403/401 `ImagePullBackOff` retries forever without + /// ever succeeding, so "the pull retries" is false for this case. + Unauthorized, + /// The digest or repository does not exist at that registry. + ManifestUnknown, + /// The image has no variant for the node's architecture. + ArchMismatch, +} + +/// The container's startup state, already decoded from pod status. Decoding +/// happens at the edge so this module stays free of API types. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Startup { + /// `state.running` — the harness process is up. This, not pod phase, is + /// what "live" means. + Started, + /// Started once and reached a terminal phase (Succeeded/Failed). + Terminated, + /// Never started, and self-healing is plausible: unschedulable during + /// scale-from-zero, an image pull in progress, a transient + /// `CreateContainerConfigError` whose Secret exists. + NeverStartedRecoverable, + /// Never started, and the provider *verified* the cause — not a reason + /// string. Either the referenced Secret is confirmed absent by a + /// most-recent read, or the image reference is structurally invalid. + NeverStartedProvablyBroken, + /// Never started; the pull is failing in a way that will not self-heal. + /// Still recoverable in the *never delete* sense — this only changes what + /// we report and how long we wait. + NeverStartedPullFailing(PullFailure), +} + +/// A pod that passed identity and ownership verification: label-selected, +/// full-pubkey annotation equal to the derived pubkey, management marker +/// present. Constructing this type is the verification step's output, so an +/// unverified object cannot reach `classify` at all. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct VerifiedPod { + pub name: String, + pub fence: Fence, + /// Set once the apiserver accepts a delete. In Kubernetes there is no + /// `Terminating` phase — a pod being gracefully deleted stays in phase + /// `Running` for its whole grace period — so this must be checked + /// *before* startup state or the dying pod reads as the no-op row. + pub deletion_marked: bool, + pub startup: Startup, + /// The `buzz.block.xyz/create-intent` annotation as recorded at create. + /// `None` for a pod written before the annotation existed, which counts + /// as divergence. + pub recorded_intent: Option, +} + +/// What the reconciler should do next. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Action { + /// Create the pod, then wait for the harness container to start. + Create, + /// Compare-and-delete, poll for actual disappearance, then re-enter. + Delete { name: String, fence: Fence }, + /// Wait for a deletion already in flight, then re-enter. + AwaitDisappearance { name: String }, + /// Strict no-op: return this `agent_id`, mutate nothing. + NoOp { agent_id: String }, + /// Keep observing until started or the operation deadline expires; on + /// expiry report the latest condition. Never deletes, on this call or any + /// later one. + Observe { name: String }, + /// Surface an actionable condition immediately rather than burning the + /// deadline on a failure that will not self-heal. + Report { name: String, failure: PullFailure }, +} + +/// Apply the spec's ordered rules to one verified observation. +/// +/// `desired` is the freshly computed create intent; comparison is always +/// recorded-annotation vs freshly-computed, never a diff against the live pod +/// (admission defaulting would make every pod look divergent). +pub fn classify(observed: Option<&VerifiedPod>, desired: &Fingerprint) -> Action { + let Some(pod) = observed else { + // Row: no instance → create. First deploy, or after GC. + return Action::Create; + }; + + // Row: deletion-marked, ANY phase. Checked before startup state because + // there is no `Terminating` phase to match on — a gracefully deleting pod + // reports phase `Running` throughout its grace period, so testing startup + // first would mistake it for the live no-op row and return an id that + // evaporates. + if pod.deletion_marked { + return Action::AwaitDisappearance { + name: pod.name.clone(), + }; + } + + match &pod.startup { + // Row: live and started → strict no-op. Start must never kill a live + // agent mid-turn, whatever the fingerprint says. + Startup::Started => Action::NoOp { + agent_id: pod.name.clone(), + }, + + // Row: terminated → delete residue, then re-enter to create. This is + // the normal restart path — how a user revives a reaped agent. + Startup::Terminated => Action::Delete { + name: pod.name.clone(), + fence: pod.fence.clone(), + }, + + // Row: never started, provably non-recoverable → fenced replace. + // "Provably" means a verified absence or a structural defect, never a + // reason string. + Startup::NeverStartedProvablyBroken => Action::Delete { + name: pod.name.clone(), + fence: pod.fence.clone(), + }, + + // Inside the recoverable row: a pull that will not self-heal is + // reported immediately instead of consuming the 600s deadline. This + // changes reporting and wait behavior only — no delete authority. + Startup::NeverStartedPullFailing(failure) => Action::Report { + name: pod.name.clone(), + failure: *failure, + }, + + // Row: never started, recoverable — split on create-intent + // divergence. Divergence is evidence of a config change the user is + // waiting on, and it is the *only* thing that replaces a + // never-started pod. Pod age triggers nothing: any finite age + // threshold collides with Cluster Autoscaler's own pod-age delays, + // and delete-recreate resets exactly the age it keys on. + Startup::NeverStartedRecoverable => { + if pod.recorded_intent.as_ref() == Some(desired) { + Action::Observe { + name: pod.name.clone(), + } + } else { + Action::Delete { + name: pod.name.clone(), + fence: pod.fence.clone(), + } + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn fp(seed: &str) -> Fingerprint { + Fingerprint::for_test(seed) + } + + fn pod(startup: Startup, intent: Option) -> VerifiedPod { + VerifiedPod { + name: "buzz-agent-abc123def456".into(), + fence: Fence { + uid: "uid-1".into(), + resource_version: "rv-1".into(), + }, + deletion_marked: false, + startup, + recorded_intent: intent, + } + } + + #[test] + fn no_instance_creates() { + assert_eq!(classify(None, &fp("a")), Action::Create); + } + + #[test] + fn started_pod_is_strict_no_op() { + let p = pod(Startup::Started, Some(fp("a"))); + assert_eq!( + classify(Some(&p), &fp("a")), + Action::NoOp { + agent_id: p.name.clone() + } + ); + } + + /// The asymmetry the spec states plainly: an edit cannot reach a started + /// pod until it exits, but it *can* reach a never-started one — the + /// never-started pod is the one the user is editing because it did not + /// start. + #[test] + fn started_pod_no_ops_even_when_intent_diverges() { + let p = pod(Startup::Started, Some(fp("old"))); + assert_eq!( + classify(Some(&p), &fp("new")), + Action::NoOp { + agent_id: p.name.clone() + } + ); + } + + /// In Kubernetes a gracefully deleting pod stays in phase `Running`. If + /// the deletion mark were checked after startup state, this pod would + /// take the no-op row and `deploy` would return an id that evaporates. + #[test] + fn deletion_mark_beats_every_startup_state() { + for startup in [ + Startup::Started, + Startup::Terminated, + Startup::NeverStartedRecoverable, + Startup::NeverStartedProvablyBroken, + Startup::NeverStartedPullFailing(PullFailure::Unauthorized), + ] { + let mut p = pod(startup.clone(), Some(fp("a"))); + p.deletion_marked = true; + assert_eq!( + classify(Some(&p), &fp("a")), + Action::AwaitDisappearance { + name: p.name.clone() + }, + "deletion mark ignored for {startup:?}" + ); + } + } + + #[test] + fn terminated_pod_is_replaced() { + let p = pod(Startup::Terminated, Some(fp("a"))); + assert_eq!( + classify(Some(&p), &fp("a")), + Action::Delete { + name: p.name.clone(), + fence: p.fence.clone() + } + ); + } + + /// A never-started winner is repairable: pod exists, Secret confirmed + /// absent, container never started. A later deploy must delete-recreate + /// rather than no-op — the test that pins started-not-phase as the no-op + /// criterion. + #[test] + fn provably_broken_never_started_pod_is_replaced() { + let p = pod(Startup::NeverStartedProvablyBroken, Some(fp("a"))); + assert_eq!( + classify(Some(&p), &fp("a")), + Action::Delete { + name: p.name.clone(), + fence: p.fence.clone() + } + ); + } + + /// The anti-livelock rule: identical desired intent means *never* delete, + /// however long the pod has been pending. Age is not an input to this + /// function at all, which is the strongest way to say so. + #[test] + fn recoverable_with_matching_intent_only_observes() { + let p = pod(Startup::NeverStartedRecoverable, Some(fp("same"))); + assert_eq!( + classify(Some(&p), &fp("same")), + Action::Observe { + name: p.name.clone() + } + ); + } + + /// Repeated identical Starts can never delete anything — the same + /// classification, arbitrarily many times. + #[test] + fn repeated_identical_starts_never_delete() { + let p = pod(Startup::NeverStartedRecoverable, Some(fp("same"))); + for _ in 0..100 { + assert!(!matches!( + classify(Some(&p), &fp("same")), + Action::Delete { .. } + )); + } + } + + /// The wedge escape: the user corrected a resource request or image, so + /// the never-started pod is built from configuration they have since + /// changed. Without this row the edit could never materialize. + #[test] + fn recoverable_with_divergent_intent_is_replaced() { + let p = pod(Startup::NeverStartedRecoverable, Some(fp("old"))); + assert_eq!( + classify(Some(&p), &fp("new")), + Action::Delete { + name: p.name.clone(), + fence: p.fence.clone() + } + ); + } + + /// A pod predating the annotation has no recorded intent — that is + /// absence, which the spec groups with divergence. + #[test] + fn missing_recorded_intent_counts_as_divergence() { + let p = pod(Startup::NeverStartedRecoverable, None); + assert_eq!( + classify(Some(&p), &fp("any")), + Action::Delete { + name: p.name.clone(), + fence: p.fence.clone() + } + ); + } + + /// Permanent-looking pull failures report immediately instead of burning + /// 600s — and, critically, never delete. + #[test] + fn pull_failures_report_and_never_delete() { + for failure in [ + PullFailure::Unauthorized, + PullFailure::ManifestUnknown, + PullFailure::ArchMismatch, + ] { + let p = pod(Startup::NeverStartedPullFailing(failure), Some(fp("a"))); + // Divergent intent too — still no delete from this arm. + for desired in [fp("a"), fp("different")] { + assert_eq!( + classify(Some(&p), &desired), + Action::Report { + name: p.name.clone(), + failure + } + ); + } + } + } + + /// Every delete carries the fence from the observation that authorized + /// it. Exhaustive over the delete-producing states, so a future arm that + /// forgets is caught here rather than in a cluster. + #[test] + fn every_delete_carries_the_authorizing_fence() { + let states = [ + (Startup::Terminated, fp("a")), + (Startup::NeverStartedProvablyBroken, fp("a")), + (Startup::NeverStartedRecoverable, fp("divergent")), + ]; + for (startup, desired) in states { + let p = pod(startup, Some(fp("a"))); + match classify(Some(&p), &desired) { + Action::Delete { fence, .. } => assert_eq!(fence, p.fence), + other => panic!("expected Delete, got {other:?}"), + } + } + } +} diff --git a/crates/buzz-backend-kubernetes/src/client.rs b/crates/buzz-backend-kubernetes/src/client.rs new file mode 100644 index 0000000000..0c3bed65b7 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/client.rs @@ -0,0 +1,182 @@ +//! Cluster auth and client construction (spec §Cluster auth, +//! `docs/remote-agents.md:985-995`). +//! +//! Standard kubeconfig resolution (`$KUBECONFIG` → `~/.kube/config`). +//! `provider_config` carries `context` and `namespace` only — credentials +//! never transit config (I2, `:196-198`). + +use kube::config::{ExecConfig, KubeConfigOptions, Kubeconfig}; +use kube::{Client, Config}; +use std::path::{Path, PathBuf}; + +/// Directories prepended to `PATH` before the client is built. +/// +/// Kubeconfigs at Block near-universally authenticate through `exec` +/// credential plugins (`aws eks get-token`, `gke-gcloud-auth-plugin`) that +/// resolve via `PATH` — and this provider inherits a Finder-launched +/// desktop's minimal `PATH`, which contains none of the places those plugins +/// install to (`:989-994`). +const PATH_PREPEND: [&str; 2] = ["/opt/homebrew/bin", "/usr/local/bin"]; + +/// Compute the new `PATH` value: plugin directories first, inherited entries +/// after, in order. Pure so the ordering can be tested without mutating the +/// process's environment. +fn prepended_path(home: Option<&Path>, existing: &std::ffi::OsStr) -> Option { + let mut dirs: Vec = PATH_PREPEND.iter().map(PathBuf::from).collect(); + if let Some(home) = home { + dirs.push(home.join(".local/bin")); + } + // An empty inherited PATH splits into one empty entry, which POSIX + // resolves as the current directory — a place a credential plugin should + // never be looked up. Drop empties rather than propagate them. + dirs.extend(std::env::split_paths(existing).filter(|p| !p.as_os_str().is_empty())); + std::env::join_paths(dirs).ok() +} + +/// Prepend the plugin directories to this process's `PATH`. +/// +/// Modifies the provider's own environment, which is sound here: one process +/// per operation, called before any client or task exists, and the child +/// processes that read it are exactly the credential plugins this exists for. +fn prepend_plugin_path() { + let home = std::env::var_os("HOME"); + let existing = std::env::var_os("PATH").unwrap_or_default(); + if let Some(joined) = prepended_path(home.as_ref().map(Path::new), &existing) { + std::env::set_var("PATH", joined); + } +} + +/// Is `command` runnable — an executable on `PATH`, or an existing path? +fn resolves_on_path(command: &str) -> bool { + if command.contains(std::path::MAIN_SEPARATOR) { + return Path::new(command).is_file(); + } + std::env::var_os("PATH") + .map(|path| std::env::split_paths(&path).any(|dir| dir.join(command).is_file())) + .unwrap_or(false) +} + +/// The exec plugin the selected context authenticates with, if any. +/// +/// Read from the kubeconfig directly rather than from `Config`, which does not +/// expose it. A read failure yields `None`: this lookup exists only to improve +/// an error message, and must never be the thing that fails a deploy. +fn exec_plugin_for(context: Option<&str>) -> Option { + let kubeconfig = Kubeconfig::read().ok()?; + let context_name = context + .map(str::to_string) + .or_else(|| kubeconfig.current_context.clone())?; + let user_name = kubeconfig + .contexts + .iter() + .find(|c| c.name == context_name) + .and_then(|c| c.context.as_ref()) + .and_then(|c| c.user.clone())?; + kubeconfig + .auth_infos + .iter() + .find(|a| a.name == user_name) + .and_then(|a| a.auth_info.as_ref()) + .and_then(|a| a.exec.clone()) +} + +/// Turn a client-construction failure into an error a user can act on. +/// +/// When the context authenticates through an exec plugin that is not on +/// `PATH`, that is almost always the cause, and the actionable fact is the +/// plugin's name — not a kube-rs error chain (`:994-995`). +fn explain(context: Option<&str>, error: &kube::Error) -> String { + if let Some(command) = exec_plugin_for(context).and_then(|e| e.command) { + if !resolves_on_path(&command) { + return format!( + "kubeconfig context {} authenticates with the credential plugin \ + {command:?}, which is not on PATH. Install it or add its \ + directory to PATH, then try again.", + context.unwrap_or("(current)") + ); + } + } + format!( + "could not connect to the cluster using kubeconfig context {}: {error}", + context.unwrap_or("(current)") + ) +} + +/// Build a client for the selected context. +pub async fn connect(context: Option<&str>) -> Result { + prepend_plugin_path(); + + let options = KubeConfigOptions { + context: context.map(str::to_string), + ..Default::default() + }; + let config = Config::from_kubeconfig(&options).await.map_err(|e| { + // A named context that does not exist is a user typo, and the + // kube-rs message for it is already specific. + format!( + "could not load kubeconfig for context {}: {e}", + context.unwrap_or("(current)") + ) + })?; + + Client::try_from(config).map_err(|e| explain(context, &e)) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The three plugin directories must end up ahead of the inherited PATH, + /// or a Finder-launched desktop never finds `aws`/`gke-gcloud-auth-plugin`. + /// Tested on the pure computation: mutating the process PATH here would + /// race every other test in the binary. + #[test] + fn plugin_directories_are_prepended_in_order() { + let joined = prepended_path( + Some(Path::new("/tmp/fake-home")), + std::ffi::OsStr::new("/inherited/bin:/usr/bin"), + ) + .unwrap(); + let dirs: Vec = std::env::split_paths(&joined).collect(); + assert_eq!( + dirs, + [ + "/opt/homebrew/bin", + "/usr/local/bin", + "/tmp/fake-home/.local/bin", + "/inherited/bin", + "/usr/bin", + ] + .map(PathBuf::from) + ); + } + + /// No `HOME` is not a failure — the two absolute directories still apply. + #[test] + fn missing_home_still_prepends_the_absolute_directories() { + let joined = prepended_path(None, std::ffi::OsStr::new("/inherited/bin")).unwrap(); + let dirs: Vec = std::env::split_paths(&joined).collect(); + assert_eq!( + dirs, + ["/opt/homebrew/bin", "/usr/local/bin", "/inherited/bin"].map(PathBuf::from) + ); + } + + /// An empty inherited PATH must not produce an empty entry, which the + /// shell and `resolves_on_path` would both read as the cwd. + #[test] + fn empty_inherited_path_yields_no_empty_entry() { + let joined = prepended_path(None, std::ffi::OsStr::new("")).unwrap(); + let dirs: Vec = std::env::split_paths(&joined).collect(); + assert_eq!( + dirs, + ["/opt/homebrew/bin", "/usr/local/bin"].map(PathBuf::from) + ); + } + + #[test] + fn resolves_absolute_paths_directly() { + assert!(resolves_on_path("/bin/sh")); + assert!(!resolves_on_path("/nonexistent/plugin-binary")); + } +} diff --git a/crates/buzz-backend-kubernetes/src/cluster.rs b/crates/buzz-backend-kubernetes/src/cluster.rs new file mode 100644 index 0000000000..755f41f6c8 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/cluster.rs @@ -0,0 +1,427 @@ +//! The real [`Substrate`]: kube-rs against a live apiserver. +//! +//! Everything that *decides* lives in `classify`/`gc`; this module only +//! performs I/O and maps apiserver responses onto the trait's vocabulary. +//! Three mappings here are normative rather than incidental: +//! +//! * **409 is discriminated on `Status.reason`, never on the code.** A create +//! 409 is `AlreadyExists`; a delete 409 from a failed precondition is +//! `Conflict`. Branching on `code == 409` conflates a lost create race with +//! a stale fence and is the trap the spec names (`:780-794`). +//! * **Reads leave `resourceVersion` unset**, which is the quorum read. `"0"` +//! is the cache read, and a confirmed absence from a cache is proof of +//! nothing (`:761-769`). +//! * **Deletes never set `grace_period_seconds`**, so the object's own 60s +//! budget applies. Passing `0` is a force-kill that discards the shutdown +//! window the pod declares (`:1185-1189`). + +use crate::classify::Fence; +use crate::reconcile::{CreateOutcome, DeleteOutcome, Substrate}; +use chrono::{DateTime, Utc}; +use k8s_openapi::api::core::v1::{Namespace, Pod, Secret}; +use kube::api::{Api, DeleteParams, GetParams, ListParams, PostParams, Preconditions}; +use kube::core::ErrorResponse; +use kube::{Client, Resource}; +use std::time::{Duration, Instant}; + +/// `Status.reason` values we branch on. Spelled once so the two 409 arms are +/// visibly the same discriminator read two ways. +/// +/// These are wire strings `apimachinery` chooses, not names this crate picks: +/// `StatusReasonAlreadyExists`, `StatusReasonConflict`, `StatusReasonNotFound`, +/// and `StatusReasonForbidden` in `k8s.io/apimachinery/pkg/apis/meta/v1/types.go`. +/// kube-core types `ErrorResponse::reason` as a bare `String`, so there is no +/// upstream constant to bind to and the spelling is pinned by test instead. +const REASON_ALREADY_EXISTS: &str = "AlreadyExists"; +const REASON_CONFLICT: &str = "Conflict"; +const REASON_NOT_FOUND: &str = "NotFound"; +const REASON_FORBIDDEN: &str = "Forbidden"; + +/// The apiserver-backed substrate for one deploy operation. +pub struct Cluster { + client: Client, + namespace: String, + /// Start of *this operation*, for the deadline. Monotonic: the 600s budget + /// must not move when the wall clock does. + started: Instant, +} + +/// The typed API error underneath a `kube::Error`, if it is one. +fn api_error(error: &kube::Error) -> Option<&ErrorResponse> { + match error { + kube::Error::Api(response) => Some(response), + _ => None, + } +} + +/// Does this error carry the given `Status.reason`? +fn reason_is(error: &kube::Error, reason: &str) -> bool { + api_error(error).is_some_and(|e| e.reason == reason) +} + +impl Cluster { + pub fn new(client: Client, namespace: &str) -> Self { + Self { + client, + namespace: namespace.to_string(), + started: Instant::now(), + } + } + + fn pods(&self) -> Api { + Api::namespaced(self.client.clone(), &self.namespace) + } + + fn secrets(&self) -> Api { + Api::namespaced(self.client.clone(), &self.namespace) + } + + /// List an object kind through the raw client so the response's HTTP + /// `Date` header is reachable. + /// + /// `Api::list` returns only the decoded body, and the apiserver's clock is + /// the *only* clock the orphan-Secret age gate may use — a desktop's local + /// clock running fast computes every in-flight Secret as expired + /// (`:1321-1335`). So the list goes through `Client::send`, which hands + /// back the whole `http::Response`. + async fn list_with_date( + &self, + selector: &str, + ) -> Result<(Vec, Option>), String> + where + K: Resource + + Clone + + serde::de::DeserializeOwned + + std::fmt::Debug, + K::DynamicType: Default, + { + let dt = K::DynamicType::default(); + let url = K::url_path(&dt, Some(&self.namespace)); + // resourceVersion deliberately unset: quorum read. + let params = ListParams { + label_selector: Some(selector.to_string()), + ..Default::default() + }; + let request = kube::core::Request::new(url) + .list(¶ms) + .map_err(|e| format!("could not build a list request: {e}"))?; + let (parts, body) = request.into_parts(); + let response = self + .client + .send(http::Request::from_parts(parts, body.into())) + .await + .map_err(|e| format!("could not list {}: {e}", K::plural(&dt)))?; + + // Parsed before the body is consumed, and independently of it: a + // missing or malformed header is not a list failure, it just means the + // orphan sweep has no clock and skips. + let server_now = response + .headers() + .get(http::header::DATE) + .and_then(|v| v.to_str().ok()) + .and_then(|v| DateTime::parse_from_rfc2822(v).ok()) + .map(|v| v.with_timezone(&Utc)); + + let bytes = http_body_util::BodyExt::collect(response.into_body()) + .await + .map_err(|e| format!("could not read the {} list body: {e}", K::plural(&dt)))? + .to_bytes(); + let list: kube::core::ObjectList = serde_json::from_slice(&bytes) + .map_err(|e| format!("could not decode the {} list: {e}", K::plural(&dt)))?; + + Ok((list.items, server_now)) + } +} + +impl Substrate for Cluster { + async fn ensure_namespace(&self, namespace: &str) -> Result<(), String> { + let api: Api = Api::all(self.client.clone()); + if api + .get_opt(namespace) + .await + .map_err(|e| format!("could not check whether namespace {namespace} exists: {e}"))? + .is_some() + { + return Ok(()); + } + + let spec = Namespace { + metadata: kube::core::ObjectMeta { + name: Some(namespace.to_string()), + ..Default::default() + }, + ..Default::default() + }; + match api.create(&PostParams::default(), &spec).await { + Ok(_) => Ok(()), + // Someone else created it between our check and our create. That + // is the desired end state, not a failure. + Err(e) if reason_is(&e, REASON_ALREADY_EXISTS) => Ok(()), + // Namespace-create is frequently denied on shared clusters. Name + // the exact command an operator runs, and never silently fall back + // to `default` — deploying an agent into someone else's namespace + // is worse than refusing (`:1002-1005`). + Err(e) if reason_is(&e, REASON_FORBIDDEN) => Err(format!( + "not authorized to create namespace {namespace}. Ask a cluster \ + administrator to run `kubectl create namespace {namespace}`, \ + then try again." + )), + Err(e) => Err(format!("could not create namespace {namespace}: {e}")), + } + } + + async fn list_pods(&self, selector: &str) -> Result<(Vec, Option>), String> { + self.list_with_date::(selector).await + } + + async fn list_secrets(&self, selector: &str) -> Result, String> { + Ok(self.list_with_date::(selector).await?.0) + } + + async fn secret_exists(&self, name: &str) -> Result { + // `GetParams::default()` leaves resourceVersion unset — the quorum + // read this check requires to be proof of anything. + match self.secrets().get_with(name, &GetParams::default()).await { + Ok(_) => Ok(true), + Err(e) if reason_is(&e, REASON_NOT_FOUND) => Ok(false), + Err(e) => Err(format!("could not check whether secret {name} exists: {e}")), + } + } + + async fn create_secret(&self, secret: &Secret) -> Result<(), String> { + let name = secret.metadata.name.clone().unwrap_or_default(); + self.secrets() + .create(&PostParams::default(), secret) + .await + .map(|_| ()) + .map_err(|e| format!("could not create secret {name}: {e}")) + } + + async fn create_pod(&self, pod: &Pod) -> Result { + let name = pod.metadata.name.clone().unwrap_or_default(); + match self.pods().create(&PostParams::default(), pod).await { + Ok(_) => Ok(CreateOutcome::Created), + // The deterministic name is taken: a concurrent attempt won the + // election. Discriminated on the reason — a 409 whose reason is + // `Conflict` is a different condition and must not be read as a + // lost race. + Err(e) if reason_is(&e, REASON_ALREADY_EXISTS) => Ok(CreateOutcome::AlreadyExists), + Err(e) => Err(format!("could not create pod {name}: {e}")), + } + } + + async fn delete_pod(&self, name: &str, fence: &Fence) -> Result { + let params = DeleteParams { + preconditions: Some(Preconditions { + uid: Some(fence.uid.clone()), + resource_version: Some(fence.resource_version.clone()), + }), + // grace_period_seconds deliberately unset: the pod's own 60s + // budget applies. + ..Default::default() + }; + match self.pods().delete(name, ¶ms).await { + Ok(_) => Ok(DeleteOutcome::Accepted), + Err(e) if reason_is(&e, REASON_NOT_FOUND) => Ok(DeleteOutcome::NotFound), + // The object changed since the observation that authorized this + // delete. Same HTTP code as the create race above, different + // reason, different meaning. + Err(e) if reason_is(&e, REASON_CONFLICT) => Ok(DeleteOutcome::PreconditionFailed), + Err(e) => Err(format!("could not delete pod {name}: {e}")), + } + } + + async fn delete_secret(&self, name: &str) -> Result<(), String> { + match self.secrets().delete(name, &DeleteParams::default()).await { + Ok(_) => Ok(()), + // Already gone is the desired end state. + Err(e) if reason_is(&e, REASON_NOT_FOUND) => Ok(()), + Err(e) => Err(format!("could not delete secret {name}: {e}")), + } + } + + async fn get_pod(&self, name: &str) -> Result, String> { + match self.pods().get_with(name, &GetParams::default()).await { + Ok(pod) => Ok(Some(pod)), + Err(e) if reason_is(&e, REASON_NOT_FOUND) => Ok(None), + Err(e) => Err(format!("could not read pod {name}: {e}")), + } + } + + async fn sleep(&self, duration: Duration) { + tokio::time::sleep(duration).await; + } + + fn elapsed(&self) -> Duration { + self.started.elapsed() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use http::Response; + use kube::client::Body; + use std::sync::{Arc, Mutex}; + use tower::service_fn; + + fn list_response(date: Option<&str>) -> Response { + let mut response = Response::builder().status(200); + if let Some(date) = date { + response = response.header(http::header::DATE, date); + } + response + .body(Body::from( + serde_json::to_vec(&serde_json::json!({ + "apiVersion": "v1", + "kind": "PodList", + "metadata": {"resourceVersion": "17"}, + "items": [{ + "apiVersion": "v1", + "kind": "Pod", + "metadata": {"name": "sprig"}, + "spec": {"containers": [{"name": "agent", "image": "example.invalid/sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}]} + }] + })) + .unwrap(), + )) + .unwrap() + } + + async fn list_through_real_request_path( + date: Option<&'static str>, + ) -> (Vec, Option>, String) { + let observed_uri = Arc::new(Mutex::new(None)); + let service_uri = Arc::clone(&observed_uri); + let service = service_fn(move |request: http::Request| { + let service_uri = Arc::clone(&service_uri); + async move { + *service_uri.lock().unwrap() = Some(request.uri().to_string()); + Ok::<_, std::convert::Infallible>(list_response(date)) + } + }); + let cluster = Cluster::new(Client::new(service, "ignored"), "owned-ns"); + let result = cluster + .list_with_date::("app.kubernetes.io/managed-by=buzz-backend-kubernetes") + .await + .unwrap(); + let uri = observed_uri.lock().unwrap().take().unwrap(); + (result.0, result.1, uri) + } + + /// Exercise the shipped `Request` + `Client::send` seam. A fake + /// reconciler would not prove that kube-rs emits a quorum list request or + /// that the apiserver's clock survives body decoding. + #[tokio::test] + async fn list_with_date_uses_a_quorum_request_and_returns_the_server_clock() { + let (pods, server_now, uri) = + list_through_real_request_path(Some("Sun, 02 Aug 2026 04:00:00 GMT")).await; + + assert_eq!(pods.len(), 1, "fixture must contain one decoded pod"); + assert_eq!(pods[0].metadata.name.as_deref(), Some("sprig")); + assert!(uri.starts_with("/api/v1/namespaces/owned-ns/pods?")); + assert!( + uri.contains("labelSelector=app.kubernetes.io%2Fmanaged-by%3Dbuzz-backend-kubernetes") + ); + assert!( + !uri.contains("resourceVersion"), + "cache read leaked into {uri}" + ); + assert_eq!( + server_now.unwrap().to_rfc3339(), + "2026-08-02T04:00:00+00:00" + ); + } + + /// Header failure is deliberately not list failure: without a trustworthy + /// apiserver clock the orphan sweep skips, but normal reconciliation still + /// receives the decoded objects. + #[tokio::test] + async fn list_with_date_keeps_items_when_the_server_clock_is_unusable() { + for date in [Some("not a date"), None] { + let (pods, server_now, _) = list_through_real_request_path(date).await; + assert_eq!(pods.len(), 1, "fixture must contain one decoded pod"); + assert!(server_now.is_none(), "unexpected clock for {date:?}"); + } + } + + /// A typed apiserver error, as kube-rs surfaces it. + fn api(reason: &str, code: u16) -> kube::Error { + kube::Error::Api(ErrorResponse { + status: "Failure".into(), + message: String::new(), + reason: reason.into(), + code, + }) + } + + /// The one discriminator the whole file rests on, and the trap the spec + /// predicts: "an implementation that branches on the code alone will + /// eventually take the adoption path on a failed delete or vice versa" + /// (`:788-790`). + /// + /// Both of these are 409. Reading the *code* makes them identical; reading + /// `Status.reason` keeps a lost create race and a stale fence apart. The + /// mutation that must fail this test is `e.reason == …` → `e.code == 409`, + /// which no other test in the crate would catch — the fakes never produce + /// a real `kube::Error`. + #[test] + fn the_two_409s_are_never_conflated() { + let already_exists = api(REASON_ALREADY_EXISTS, 409); + let conflict = api(REASON_CONFLICT, 409); + + assert!(reason_is(&already_exists, REASON_ALREADY_EXISTS)); + assert!(reason_is(&conflict, REASON_CONFLICT)); + // The cross terms are the whole point. + assert!(!reason_is(&already_exists, REASON_CONFLICT)); + assert!(!reason_is(&conflict, REASON_ALREADY_EXISTS)); + } + + /// A transport-level failure is not an apiserver verdict. It must fall + /// through to the error arm rather than being read as any reason — a + /// connection reset silently classified as `NotFound` would report a pod + /// as confirmed-absent, which the classifier treats as proof. + #[test] + fn a_non_api_error_carries_no_reason() { + let transport = kube::Error::LinesCodecMaxLineLengthExceeded; + assert!(api_error(&transport).is_none()); + for reason in [ + REASON_ALREADY_EXISTS, + REASON_CONFLICT, + REASON_NOT_FOUND, + REASON_FORBIDDEN, + ] { + assert!(!reason_is(&transport, reason), "matched {reason}"); + } + } + + /// `reason` is `#[serde(default)]` in kube-core, so an apiserver that + /// omits it yields an empty string. That must match nothing rather than + /// matching an empty pattern by accident. + #[test] + fn an_absent_reason_matches_nothing() { + let bare = api("", 409); + assert!(!reason_is(&bare, REASON_ALREADY_EXISTS)); + assert!(!reason_is(&bare, REASON_CONFLICT)); + } + + /// The consts are the apiserver's spelling, asserted against literals + /// rather than against themselves. + /// + /// Every other test here references the consts symbolically on both sides + /// — fixture *and* assertion — which is true for any pair of distinct + /// values. That tests the discriminator is self-consistent, not that it is + /// correct: swapping the two 409 values inverts `AlreadyExists` and + /// `Conflict` at a real apiserver (`:788-790`'s failure, reached by + /// editing a string instead of a branch) with every other test still + /// green. These are `apimachinery`'s wire strings and kube-core exposes no + /// constant for them, so a literal is the only external anchor available. + /// Found by Quinn's mutation matrix; M2/M3/M4 survived without it. + #[test] + fn the_reason_consts_are_the_apiservers_spelling() { + assert_eq!(REASON_ALREADY_EXISTS, "AlreadyExists"); + assert_eq!(REASON_CONFLICT, "Conflict"); + assert_eq!(REASON_NOT_FOUND, "NotFound"); + assert_eq!(REASON_FORBIDDEN, "Forbidden"); + } +} diff --git a/crates/buzz-backend-kubernetes/src/config.rs b/crates/buzz-backend-kubernetes/src/config.rs new file mode 100644 index 0000000000..4d96735b7b --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/config.rs @@ -0,0 +1,470 @@ +//! `provider_config` parsing and the `info` config schema +//! (spec §`provider_config` v1 fields, `docs/remote-agents.md:1384-1389`). +//! +//! Nine fields, all optional except `image` (required at parse time; the +//! schema offers the published sprig image as a prefill default — §Image). +//! No credential field exists, by I2: cluster auth comes from ambient +//! kubeconfig resolution and nothing else (`:196-198`). + +use crate::image::{self, ImageRef}; + +/// Resource requests and limits (§Pod shape: 1cpu/2Gi → 2cpu/4Gi, all four +/// configurable — `cargo build` in an agent workspace makes 500m/1Gi +/// unrealistic). +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Resources { + pub cpu_request: String, + pub memory_request: String, + pub cpu_limit: String, + pub memory_limit: String, +} + +impl Default for Resources { + fn default() -> Self { + Self { + cpu_request: "1".into(), + memory_request: "2Gi".into(), + cpu_limit: "2".into(), + memory_limit: "4Gi".into(), + } + } +} + +/// Default inactivity budget: the I5 opt-in (§Auto-Stop). The config field and +/// `BUZZ_ACP_EXIT_AFTER_INACTIVITY` are one knob, not two. +pub const DEFAULT_INACTIVITY_SECONDS: u64 = 7200; + +/// Default `image` schema prefill: the published sprig image, in tag+digest +/// form so the tag stays human-traceable to its git SHA while the digest does +/// the pinning (§Image — tag-only refs are rejected; `image::parse` drops the +/// tag on normalization). This is a UI prefill, not a baked fallback: `image` +/// stays required, an empty value still fails closed, and the value always +/// arrives explicitly in `provider_config`, so create-intent fingerprints are +/// unaffected by provider upgrades. +pub const DEFAULT_IMAGE: &str = "ghcr.io/block/buzz-sprig:sha-6530b58@sha256:17facfc7608d8ddb33bc056c9aaba1098f4ef6abe5655702fbfd7584d1f74d76"; + +/// Fixed nonzero UID/GID for the agent container (§Pod shape hardening). +pub const RUN_AS_UID: i64 = 10001; +pub const RUN_AS_GID: i64 = 10001; + +/// Writable workspace root; also `HOME` and the harness's cwd +/// (§Working directory). +pub const WORKSPACE_PATH: &str = "/home/agent"; + +/// `terminationGracePeriodSeconds` — a declared budget, not a derived sum +/// (§Pod shape). Kubernetes' default 30s would SIGKILL the harness mid-drain. +pub const TERMINATION_GRACE_SECONDS: i64 = 60; + +/// The only restart policy v1 ships. `OnFailure` is double-gated on the +/// harness exit-code contract *and* a crash-loop classification row the state +/// machine does not have (`:1121-1139`); until both land the provider refuses +/// the combination rather than shipping against an undefended convention. +pub const RESTART_POLICY: &str = "Never"; + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ProviderConfig { + /// kubeconfig context; `None` uses the current context. + pub context: Option, + pub namespace: String, + pub image: ImageRef, + pub resources: Resources, + /// `None` when `inactivity_seconds` was 0 — refused in v1, see [`parse`]. + pub inactivity_seconds: Option, + pub service_account: Option, +} + +/// Read an optional non-empty string field. Rejects non-string scalars rather +/// than stringifying them, so a mistyped field is named at the boundary. +fn optional_string(cfg: &serde_json::Value, field: &str) -> Result, String> { + match cfg.get(field) { + None | Some(serde_json::Value::Null) => Ok(None), + Some(serde_json::Value::String(s)) if s.trim().is_empty() => Ok(None), + Some(serde_json::Value::String(s)) => Ok(Some(s.trim().to_string())), + Some(other) => Err(format!( + "provider_config.{field} must be a string, got {other}" + )), + } +} + +/// Read an optional unsigned integer. The desktop's form omits blank numeric +/// fields rather than sending `""`, but a hand-crafted payload may send a +/// numeric string — accept both, refuse anything else. +fn optional_u64(cfg: &serde_json::Value, field: &str) -> Result, String> { + match cfg.get(field) { + None | Some(serde_json::Value::Null) => Ok(None), + Some(serde_json::Value::Number(n)) => n.as_u64().map(Some).ok_or_else(|| { + format!("provider_config.{field} must be a non-negative integer, got {n}") + }), + Some(serde_json::Value::String(s)) if s.trim().is_empty() => Ok(None), + Some(serde_json::Value::String(s)) => s.trim().parse::().map(Some).map_err(|_| { + format!("provider_config.{field} must be a non-negative integer, got {s:?}") + }), + Some(other) => Err(format!( + "provider_config.{field} must be a non-negative integer, got {other}" + )), + } +} + +/// A Kubernetes namespace name: RFC 1123 label, ≤63 chars. Validated here so a +/// typo fails with a named field instead of an apiserver rejection partway +/// through a deploy. +fn valid_namespace(name: &str) -> bool { + !name.is_empty() + && name.len() <= 63 + && name.starts_with(|c: char| c.is_ascii_lowercase() || c.is_ascii_digit()) + && name.ends_with(|c: char| c.is_ascii_lowercase() || c.is_ascii_digit()) + && name + .chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-') +} + +pub fn parse(cfg: &serde_json::Value) -> Result { + if !cfg.is_object() && !cfg.is_null() { + return Err("provider_config must be a JSON object".to_string()); + } + + let namespace = optional_string(cfg, "namespace")?.ok_or_else(|| { + "provider_config.namespace is required: the info schema supplies a \ + generated default, so an empty value means the form was cleared" + .to_string() + })?; + if !valid_namespace(&namespace) { + return Err(format!( + "provider_config.namespace {namespace:?} is not a valid Kubernetes \ + namespace (lowercase alphanumerics and '-', ≤63 characters)" + )); + } + + let image = image::parse(optional_string(cfg, "image")?.unwrap_or_default().as_str())?; + + let defaults = Resources::default(); + let resources = Resources { + cpu_request: optional_string(cfg, "cpu_request")?.unwrap_or(defaults.cpu_request), + memory_request: optional_string(cfg, "memory_request")?.unwrap_or(defaults.memory_request), + cpu_limit: optional_string(cfg, "cpu_limit")?.unwrap_or(defaults.cpu_limit), + memory_limit: optional_string(cfg, "memory_limit")?.unwrap_or(defaults.memory_limit), + }; + + // `inactivity_seconds: 0` is a legal, blessed value in the spec (§Auto-Stop) + // meaning "no auto-stop" — but it selects `restartPolicy: OnFailure`, which + // §Pod shape forbids until the harness exit-code contract is pinned AND the + // state machine gains a crash-loop row. Refusing the *combination* is what + // the spec asks for; silently downgrading to `Never` would ship an + // indefinite agent that dies on its first crash. + let inactivity_seconds = match optional_u64(cfg, "inactivity_seconds")? { + None => Some(DEFAULT_INACTIVITY_SECONDS), + Some(0) => { + return Err( + "provider_config.inactivity_seconds: 0 (indefinite lifetime) is not \ + supported in this version: it requires restartPolicy OnFailure, \ + which is gated on the harness exit-code contract. Set a positive \ + number of seconds." + .to_string(), + ) + } + Some(n) => Some(n), + }; + + Ok(ProviderConfig { + context: optional_string(cfg, "context")?, + namespace, + image, + resources, + inactivity_seconds, + service_account: optional_string(cfg, "service_account")?, + }) +} + +/// A fresh `buzz-agents-` namespace default. +/// +/// Computed per `info` call, which is how "random default" is satisfied with +/// zero UI changes: the schema's `default` prefills the form (§K8s Namespace). +pub fn generated_namespace() -> String { + use rand::RngExt; + const ALPHABET: &[u8] = b"abcdefghijklmnopqrstuvwxyz0123456789"; + let mut rng = rand::rng(); + let suffix: String = (0..6) + .map(|_| ALPHABET[rng.random_range(0..ALPHABET.len())] as char) + .collect(); + format!("buzz-agents-{suffix}") +} + +/// The `config_schema` returned by `info`. Drives the UI form: +/// `properties[*].default` prefill, scalar coercion, `required` gating +/// (`:407-411`). +pub fn config_schema() -> serde_json::Value { + let defaults = Resources::default(); + serde_json::json!({ + "type": "object", + "properties": { + "context": { + "type": "string", + "title": "Kubeconfig context", + "description": "Context from your kubeconfig. Leave empty to use the current context." + }, + "namespace": { + "type": "string", + "title": "Namespace", + "description": "Created if it does not exist.", + "default": generated_namespace() + }, + "image": { + "type": "string", + "title": "Agent image", + "description": "Digest-pinned image containing the buzz-acp runtime ABI, e.g. ghcr.io/block/buzz-sprig@sha256:. Tags alone are not accepted: this pod holds the agent's private key.", + "default": DEFAULT_IMAGE + }, + "cpu_request": { + "type": "string", "title": "CPU request", "default": defaults.cpu_request + }, + "memory_request": { + "type": "string", "title": "Memory request", "default": defaults.memory_request + }, + "cpu_limit": { + "type": "string", "title": "CPU limit", "default": defaults.cpu_limit + }, + "memory_limit": { + "type": "string", "title": "Memory limit", "default": defaults.memory_limit + }, + "inactivity_seconds": { + "type": "number", + "title": "Stop after inactivity (seconds)", + "description": "The agent exits after this long with no work, and can be started again at any time.", + "default": DEFAULT_INACTIVITY_SECONDS + }, + "service_account": { + "type": "string", + "title": "Service account", + "description": "Scheduling/RBAC identity only. No API token is mounted." + } + }, + "required": ["namespace", "image"] + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn digest_ref() -> String { + format!("ghcr.io/block/buzz-sprig@sha256:{}", "a".repeat(64)) + } + + fn minimal() -> serde_json::Value { + serde_json::json!({"namespace": "buzz-agents-abc123", "image": digest_ref()}) + } + + #[test] + fn applies_spec_defaults() { + let c = parse(&minimal()).unwrap(); + assert_eq!(c.resources, Resources::default()); + assert_eq!(c.resources.cpu_request, "1"); + assert_eq!(c.resources.memory_request, "2Gi"); + assert_eq!(c.resources.cpu_limit, "2"); + assert_eq!(c.resources.memory_limit, "4Gi"); + assert_eq!(c.inactivity_seconds, Some(DEFAULT_INACTIVITY_SECONDS)); + assert_eq!(c.context, None); + assert_eq!(c.service_account, None); + } + + #[test] + fn all_four_resources_are_configurable() { + let mut cfg = minimal(); + cfg["cpu_request"] = "500m".into(); + cfg["memory_request"] = "1Gi".into(); + cfg["cpu_limit"] = "4".into(); + cfg["memory_limit"] = "8Gi".into(); + let c = parse(&cfg).unwrap(); + assert_eq!( + c.resources, + Resources { + cpu_request: "500m".into(), + memory_request: "1Gi".into(), + cpu_limit: "4".into(), + memory_limit: "8Gi".into(), + } + ); + } + + /// The desktop's form omits blank numeric fields; a hand-crafted payload + /// may send a numeric string. Both must mean the same thing. + #[test] + fn inactivity_accepts_number_string_and_omission() { + let mut cfg = minimal(); + cfg["inactivity_seconds"] = serde_json::json!(300); + assert_eq!(parse(&cfg).unwrap().inactivity_seconds, Some(300)); + + cfg["inactivity_seconds"] = serde_json::json!("300"); + assert_eq!(parse(&cfg).unwrap().inactivity_seconds, Some(300)); + + cfg["inactivity_seconds"] = serde_json::json!(""); + assert_eq!( + parse(&cfg).unwrap().inactivity_seconds, + Some(DEFAULT_INACTIVITY_SECONDS) + ); + } + + /// Indefinite lifetime selects `OnFailure`, which is gated. Refuse rather + /// than silently downgrade — a downgraded agent dies on its first crash + /// while the user believes they asked for indefinite. + #[test] + fn refuses_indefinite_lifetime() { + let mut cfg = minimal(); + cfg["inactivity_seconds"] = serde_json::json!(0); + let err = parse(&cfg).unwrap_err(); + assert!(err.contains("inactivity_seconds"), "got: {err}"); + assert!( + err.contains("OnFailure"), + "error should name the gate: {err}" + ); + } + + #[test] + fn rejects_negative_and_non_numeric_inactivity() { + for bad in [ + serde_json::json!(-1), + serde_json::json!(1.5), + serde_json::json!("soon"), + serde_json::json!(true), + ] { + let mut cfg = minimal(); + cfg["inactivity_seconds"] = bad.clone(); + assert!(parse(&cfg).is_err(), "accepted {bad}"); + } + } + + #[test] + fn image_is_required_and_must_be_digest_pinned() { + let mut cfg = minimal(); + cfg.as_object_mut().unwrap().remove("image"); + assert!(parse(&cfg).unwrap_err().contains("provider_config.image")); + + cfg["image"] = "ghcr.io/block/buzz-sprig:latest".into(); + assert!(parse(&cfg).unwrap_err().contains("digest-pinned")); + } + + #[test] + fn rejects_invalid_namespace_names() { + for bad in [ + "", + "Buzz-Agents", + "-leading", + "trailing-", + "has_underscore", + &"n".repeat(64), + ] { + let mut cfg = minimal(); + cfg["namespace"] = bad.into(); + assert!(parse(&cfg).is_err(), "accepted namespace {bad:?}"); + } + } + + /// I2 corollary: there is no config path for cluster credentials, so a + /// caller that tries to supply one gets no effect from it. Asserting the + /// parsed struct has no such field is the closest a test can get to + /// "the type makes it impossible". + #[test] + fn credential_fields_have_no_effect() { + let mut cfg = minimal(); + cfg["token"] = "hunter2".into(); + cfg["client_key"] = "hunter2".into(); + let c = parse(&cfg).unwrap(); + let rendered = format!("{c:?}"); + assert!( + !rendered.contains("hunter2"), + "config absorbed a credential: {rendered}" + ); + } + + #[test] + fn mistyped_string_fields_are_named() { + let mut cfg = minimal(); + cfg["namespace"] = serde_json::json!(42); + assert!(parse(&cfg) + .unwrap_err() + .contains("provider_config.namespace")); + } + + #[test] + fn generated_namespaces_are_fresh_and_valid() { + let a = generated_namespace(); + let b = generated_namespace(); + assert_ne!(a, b, "namespace default is not random"); + assert!(valid_namespace(&a), "{a} is not a valid namespace"); + assert!(a.starts_with("buzz-agents-")); + assert_eq!(a.len(), "buzz-agents-".len() + 6); + } + + /// The schema's own namespace default must be a value the parser accepts — + /// otherwise the UI prefills a form that fails on submit. + #[test] + fn schema_default_namespace_round_trips_through_parse() { + let schema = config_schema(); + let default = schema["properties"]["namespace"]["default"] + .as_str() + .unwrap(); + let cfg = serde_json::json!({"namespace": default, "image": digest_ref()}); + assert_eq!(parse(&cfg).unwrap().namespace, default); + } + + /// Same guarantee for the image prefill: the schema's default must be a + /// value `image::parse` accepts, or the UI prefills a form that fails on + /// submit. Its tag+digest form normalizes to the tagless canonical form. + #[test] + fn schema_default_image_round_trips_through_parse() { + let schema = config_schema(); + let default = schema["properties"]["image"]["default"].as_str().unwrap(); + assert_eq!(default, DEFAULT_IMAGE); + let cfg = serde_json::json!({"namespace": "buzz-agents-abc123", "image": default}); + let parsed = parse(&cfg).unwrap(); + assert_eq!( + parsed.image.as_str(), + "ghcr.io/block/buzz-sprig@sha256:17facfc7608d8ddb33bc056c9aaba1098f4ef6abe5655702fbfd7584d1f74d76" + ); + } + + /// Nine fields exactly (§`provider_config` v1 fields). The cap is 20; the + /// count is pinned so a field added without a spec change is caught here. + #[test] + fn schema_declares_exactly_the_nine_v1_fields() { + let schema = config_schema(); + let props = schema["properties"].as_object().unwrap(); + let mut keys: Vec<&str> = props.keys().map(String::as_str).collect(); + keys.sort(); + assert_eq!( + keys, + [ + "context", + "cpu_limit", + "cpu_request", + "image", + "inactivity_seconds", + "memory_limit", + "memory_request", + "namespace", + "service_account" + ] + ); + assert_eq!( + schema["required"], + serde_json::json!(["namespace", "image"]) + ); + } + + /// I2's key lint rejects any field whose word-split contains + /// secret|password|token|key|credential. A schema field tripping it would + /// make every deploy fail validation desktop-side (`:185-198`). + #[test] + fn no_schema_field_trips_the_i2_key_lint() { + const BANNED: [&str; 5] = ["secret", "password", "token", "key", "credential"]; + let schema = config_schema(); + for field in schema["properties"].as_object().unwrap().keys() { + for word in field.split(['_', '-']) { + assert!( + !BANNED.contains(&word), + "field {field:?} contains I2-banned word {word:?}" + ); + } + } + } +} diff --git a/crates/buzz-backend-kubernetes/src/env.rs b/crates/buzz-backend-kubernetes/src/env.rs new file mode 100644 index 0000000000..badff621e8 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/env.rs @@ -0,0 +1,732 @@ +//! Building the pod environment (spec §Launch data, §Entrypoint mapping table). +//! +//! The three tiers are resolved *here*, before serialization, because a +//! Kubernetes Secret's `data` is a flat map with no precedence of its own: if +//! two tiers supplied the same key, whichever entry landed in the map would +//! win silently. Resolving in-provider makes later-wins explicit and testable. + +use crate::wire::{AgentPayload, LaunchBlock}; +use std::collections::BTreeMap; + +/// Keys the authoritative tier owns. +/// +/// Load-bearing, not documentation: tier 3 *clears* every key on this list +/// before writing its own values, so a key the authoritative tier has no value +/// for is **removed** rather than left holding a lower-tier value. Plain +/// overwrite is not enough — most of these are written conditionally +/// (`BUZZ_ACP_AGENT_ARGS` only when `launch.args` is non-empty, +/// `BUZZ_ACP_RESPOND_TO` only when set), and without the clear, a lower tier +/// could supply the value for exactly the cases the authoritative tier stays +/// silent on. Clearing is also what the local spawn does: the desktop strips +/// reserved keys from user env before the authoritative layer is written +/// (`env_vars.rs:54-57`), so absent-means-absent in both paths. +const AUTHORITATIVE_KEYS: &[&str] = &[ + "BUZZ_RELAY_URL", + "BUZZ_PRIVATE_KEY", + "NOSTR_PRIVATE_KEY", + "BUZZ_AUTH_TAG", + "BUZZ_ACP_AGENT_OWNER", + "BUZZ_ACP_AGENT_COMMAND", + "BUZZ_ACP_AGENT_ARGS", + "BUZZ_ACP_RESPOND_TO", + "BUZZ_ACP_RESPOND_TO_ALLOWLIST", + "BUZZ_ACP_MCP_COMMAND", + "BUZZ_ACP_EXIT_AFTER_INACTIVITY", + START_NONCE_KEY, +]; + +/// The attempt's generation, as the harness sees it. Also the Secret's name +/// suffix — one generation, one identity — so the reconciler restamps this on +/// every create attempt rather than letting the caller's value persist across +/// a retry. +pub const START_NONCE_KEY: &str = "BUZZ_MANAGED_AGENT_START_NONCE"; + +/// Presence is the only remote liveness signal (I3), so a launch that +/// suppresses it is non-conforming (L1 item 2) — and unlike a reserved-key +/// collision, there is no "authoritative value" to overwrite it with. Refuse. +const FORBIDDEN_KEY: &str = "BUZZ_ACP_NO_PRESENCE"; + +/// Kubernetes' own cap on the summed value bytes of a Secret +/// (`MaxSecretSize`, `pkg/apis/core/types.go`). Enforced here so an oversized +/// env surfaces as a named provider error rather than an apiserver rejection +/// partway through a deploy. +const MAX_SECRET_BYTES: usize = 1024 * 1024; + +/// A POSIX-shaped env var name: `[A-Za-z_][A-Za-z0-9_]*`. +/// +/// Kubernetes validates Secret *keys* as `IsConfigMapKey` +/// (`[-._a-zA-Z0-9]+`), which is looser — `foo.bar` is a legal Secret key. +/// What the kubelet then does with such a key **changed between versions**: +/// through 1.29 it filtered invalid env names out of `envFrom` and emitted an +/// `InvalidEnvironmentVariableNames` warning event +/// (`pkg/kubelet/kubelet_pods.go:646,654` at v1.29.0); from 1.30 that filter +/// is gone (KEP-4369) and the key is injected verbatim. The same manifest +/// would silently drop a variable on one cluster and set it on another, so we +/// fail closed on the provider side and get one deterministic behavior. +fn is_posix_env_key(key: &str) -> bool { + let mut chars = key.chars(); + match chars.next() { + Some(c) if c == '_' || c.is_ascii_alphabetic() => {} + _ => return false, + } + chars.all(|c| c == '_' || c.is_ascii_alphanumeric()) +} + +/// An identity component (L1 item 1) is present only if it is nonempty after +/// trimming — and the **trimmed form is what gets stored**. The validator and +/// the writer must never disagree about the value: a guard that accepts +/// `" wss://relay "` and then writes it with the padding intact has only +/// moved the failure from a loud refusal to a connect error in the harness. +fn identity_component(value: &str) -> Option<&str> { + let trimmed = value.trim(); + (!trimmed.is_empty()).then_some(trimmed) +} + +/// The harness's `allowlist` gate mode, spelled as the desktop serializes +/// `RespondTo` (kebab-case) and as `buzz-acp`'s CLI parses it. +const RESPOND_TO_ALLOWLIST: &str = "allowlist"; + +/// Every gate mode `buzz-acp` accepts, spelled as its `clap::ValueEnum` parses +/// them (`config.rs:95-101`, kebab-case via `RespondTo`'s `Display`). +/// +/// Deliberately the **harness's** four and not the desktop's three: the desktop +/// rejects `nobody` on purpose (`managed_agents/types.rs:871-880`), but the +/// harness starts fine with it. This guard exists to cover non-desktop callers, +/// so inheriting a desktop-only narrowing would refuse a launch that works. +const RESPOND_TO_MODES: [&str; 4] = ["owner-only", RESPOND_TO_ALLOWLIST, "anyone", "nobody"]; + +/// Refuse a respond-to gate the harness will reject at config parse. +/// +/// The local spawn path re-validates this before spawning — "doing it here +/// means we never spawn a doomed process" (`runtime.rs:378`) — but the deploy +/// path projects the record's fields straight through. Without this, a gate +/// the harness refuses becomes a pod that exits 1 at startup; `restartPolicy: +/// Never` turns that into `Terminated` → `Delete` → recreate, and each cycle +/// leaves a Secret the in-call path never reaps (only a later deploy's orphan +/// sweep does, at `ORPHAN_SECRET_MIN_AGE_SECS`). The user-visible ending is +/// "startup not confirmed", indistinguishable from a slow cluster. +/// +/// Mirrors `buzz-acp`'s own rules exactly (`config.rs:95-101,996-1004,629-641`), +/// deliberately including their asymmetry: the allowlist is validated **only** +/// in allowlist mode, and merely warned about otherwise. Validating it in +/// every mode would refuse a deploy whose identical local spawn succeeds — +/// and a stale list is already harmless here, since +/// `BUZZ_ACP_RESPOND_TO_ALLOWLIST` is an authoritative key that tier 3 clears. +fn validate_respond_to_gate(respond_to: &str, allowlist: Option<&[String]>) -> Result<(), String> { + // Exact, untrimmed: `clap` does not trim, so `" allowlist "` is `rc=2` at + // the harness — a parse failure even earlier than the config errors below. + if !RESPOND_TO_MODES.contains(&respond_to) { + return Err(format!( + "deploy refused: respond_to {respond_to:?} is not a mode the \ + harness accepts (expected one of {}) — the pod would fail to \ + parse its arguments, be replaced, and leave a Secret behind on \ + every attempt", + RESPOND_TO_MODES.join(", ") + )); + } + if respond_to != RESPOND_TO_ALLOWLIST { + return Ok(()); + } + let entries = allowlist.unwrap_or_default(); + if entries.is_empty() { + return Err(format!( + "deploy refused: respond_to is {RESPOND_TO_ALLOWLIST:?} but the \ + allowlist is empty — the harness refuses this at startup, so the \ + pod would fail, be replaced, and leave a Secret behind on every \ + attempt" + )); + } + for entry in entries { + let trimmed = entry.trim(); + if trimmed.len() != 64 || !trimmed.chars().all(|c| c.is_ascii_hexdigit()) { + return Err(format!( + "deploy refused: invalid pubkey in respond_to_allowlist: \ + {entry:?} (must be exactly 64 hex characters)" + )); + } + } + Ok(()) +} + +/// Inputs the provider itself supplies to the authoritative tier. +pub struct AuthoritativeInputs<'a> { + /// The attempt's generation token — also the Secret's name suffix, so the + /// lifecycle correlator and the Secret generation are one identity. + pub generation: &'a str, + /// Resolved from `provider_config.inactivity_seconds`; `None` when the + /// indefinite opt-in was chosen (which this version refuses elsewhere). + pub inactivity_seconds: Option, +} + +/// Resolve the full pod environment. +/// +/// Order is the spec's, and the function body is deliberately three writes in +/// that order — tier 1, tier 2, tier 3 — so "later wins" is visible rather +/// than argued. +pub fn build_env( + agent: &AgentPayload, + auth: AuthoritativeInputs<'_>, +) -> Result, String> { + let default_launch = LaunchBlock::default(); + let launch = agent.launch.as_ref().unwrap_or(&default_launch); + + let mut env: BTreeMap = BTreeMap::new(); + + // Tier 1 — overridable behavior defaults. + env.extend(launch.policy_env.clone()); + + // Tier 2 — user/layered env. The descriptor already merged + // global < persona < agent, so `agent.env_vars` is NOT re-merged on top + // (§Launch data tier 2) — doing so would resurrect a layer the desktop + // already resolved. When the desktop predates the `launch` block we fall + // back to the legacy field, which is the only case it is the truth. + if agent.launch.is_some() { + env.extend(launch.env.clone()); + } else { + env.extend(agent.env_vars.clone()); + } + + // Validate what the lower tiers contributed, before the authoritative + // tier overwrites any of it. A reserved-key collision is NOT fatal: the + // spec's precedence is later-wins, so tier 3 simply overwrites it, which + // is exactly what a local spawn does. Only a key that has no + // authoritative counterpart to overwrite it — presence suppression — is + // a refusal. + for key in env.keys() { + if !is_posix_env_key(key) { + return Err(format!( + "env key {key:?} is not a POSIX environment variable name \ + ([A-Za-z_][A-Za-z0-9_]*); Kubernetes would treat it \ + inconsistently across cluster versions" + )); + } + if key.eq_ignore_ascii_case(FORBIDDEN_KEY) { + return Err(format!( + "{FORBIDDEN_KEY} must not be set on a remote agent: presence \ + is the only signal that a remote agent is alive" + )); + } + } + + // Tier 3 — authoritative. Every key it owns is cleared first, then the + // values it has are written, so it wins at a key whether or not it has a + // value there (see [`AUTHORITATIVE_KEYS`]). + for key in AUTHORITATIVE_KEYS { + env.remove(*key); + } + // Identity comes from top-level payload fields, never from `env_vars` + // (§Reserved-key rule). All three components must be nonempty: an agent + // that cannot reach a relay is the identityless launch L1 item 1 exists to + // prevent, and a blank field would otherwise sail through into the Secret + // and produce a pod that starts, fails to connect, and looks like a + // network problem. + let Some(relay_url) = identity_component(&agent.relay_url) else { + return Err("deploy refused: relay_url is empty — the agent would have \ + no relay to connect to" + .to_string()); + }; + env.insert("BUZZ_RELAY_URL".into(), relay_url.to_string()); + env.insert("BUZZ_PRIVATE_KEY".into(), agent.private_key_nsec.clone()); + // The git credential/signing helpers read NOSTR_PRIVATE_KEY. + env.insert("NOSTR_PRIVATE_KEY".into(), agent.private_key_nsec.clone()); + + // Owner: at least one of these must resolve, or the harness cannot match + // `!shutdown` and §Stop describes a mechanism that does not work. + let auth_tag = agent.auth_tag.as_deref().and_then(identity_component); + let owner = launch.owner_pubkey.as_deref().and_then(identity_component); + match (auth_tag, owner) { + (None, None) => { + return Err("deploy refused: neither auth_tag nor launch.owner_pubkey \ + resolved — without an owner the agent cannot honor \ + !shutdown" + .to_string()) + } + (tag, own) => { + if let Some(t) = tag { + env.insert("BUZZ_AUTH_TAG".into(), t.to_string()); + } + if let Some(o) = own { + env.insert("BUZZ_ACP_AGENT_OWNER".into(), o.to_string()); + } + } + } + + // The harness and MCP binaries are resolved against the *image's* PATH. + // A host path forwarded from the desktop is guaranteed absent in the + // container (§Launch data, host-resolved values). + if let Some(command) = launch.command.as_deref().filter(|c| !c.is_empty()) { + env.insert("BUZZ_ACP_AGENT_COMMAND".into(), command.to_string()); + } + if !launch.args.is_empty() { + // Comma-joined because that is what the harness's CLI parser decodes, + // and what the desktop's local spawn does. An argument containing a + // comma is unrepresentable in both paths; inventing an escaping + // scheme here would produce args the harness cannot decode. + env.insert("BUZZ_ACP_AGENT_ARGS".into(), launch.args.join(",")); + } + env.insert("BUZZ_ACP_MCP_COMMAND".into(), "buzz-dev-mcp".into()); + + if let Some(respond_to) = agent.respond_to.as_deref().filter(|s| !s.is_empty()) { + validate_respond_to_gate(respond_to, agent.respond_to_allowlist.as_deref())?; + env.insert("BUZZ_ACP_RESPOND_TO".into(), respond_to.to_string()); + } + if let Some(list) = agent + .respond_to_allowlist + .as_ref() + .filter(|l| !l.is_empty()) + { + env.insert("BUZZ_ACP_RESPOND_TO_ALLOWLIST".into(), list.join(",")); + } + + if let Some(secs) = auth.inactivity_seconds { + env.insert("BUZZ_ACP_EXIT_AFTER_INACTIVITY".into(), secs.to_string()); + } + // The generation token doubles as the lifecycle-frame correlator, so pod + // logs and observer frames share one identity (§K8s Secrets). + env.insert(START_NONCE_KEY.into(), auth.generation.to_string()); + + let total: usize = env.values().map(String::len).sum(); + if total > MAX_SECRET_BYTES { + return Err(format!( + "agent environment is {total} bytes; Kubernetes caps Secret data \ + at {MAX_SECRET_BYTES}" + )); + } + + Ok(env) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn payload_json(extra_agent: serde_json::Value) -> AgentPayload { + let mut agent = serde_json::json!({ + "name": "a", + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1example", + "auth_tag": "tag-1", + }); + let (serde_json::Value::Object(base), serde_json::Value::Object(extra)) = + (&mut agent, extra_agent) + else { + panic!("expected objects") + }; + base.extend(extra); + serde_json::from_value(agent).unwrap() + } + + fn build(agent: &AgentPayload) -> Result, String> { + build_env( + agent, + AuthoritativeInputs { + generation: "gen0001", + inactivity_seconds: Some(7200), + }, + ) + } + + #[test] + fn identity_comes_from_top_level_fields() { + let env = build(&payload_json(serde_json::json!({}))).unwrap(); + assert_eq!(env["BUZZ_RELAY_URL"], "wss://relay.example"); + assert_eq!(env["BUZZ_PRIVATE_KEY"], "nsec1example"); + assert_eq!(env["NOSTR_PRIVATE_KEY"], "nsec1example"); + assert_eq!(env["BUZZ_AUTH_TAG"], "tag-1"); + } + + /// Wren's amendment, and the spec's later-wins rule: a lower tier that + /// spoofs an authoritative key is *overwritten*, not refused. Refusing + /// would diverge from the local spawn, where the same env is written + /// before the authoritative layer and simply loses. + #[test] + fn lower_tiers_cannot_spoof_authoritative_values() { + let agent = payload_json(serde_json::json!({ + "launch": { + "command": "goose", + "policy_env": { + "BUZZ_PRIVATE_KEY": "nsec1attacker", + "BUZZ_MANAGED_AGENT_START_NONCE": "forged", + }, + "env": { + "BUZZ_RELAY_URL": "wss://attacker.example", + "NOSTR_PRIVATE_KEY": "nsec1attacker", + "BUZZ_AUTH_TAG": "forged-tag", + "BUZZ_ACP_AGENT_OWNER": "cafe", + "BUZZ_ACP_AGENT_COMMAND": "/bin/sh", + "BUZZ_ACP_MCP_COMMAND": "/bin/sh", + "BUZZ_ACP_EXIT_AFTER_INACTIVITY": "0", + }, + "owner_pubkey": "beef" + } + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_PRIVATE_KEY"], "nsec1example"); + assert_eq!(env["NOSTR_PRIVATE_KEY"], "nsec1example"); + assert_eq!(env["BUZZ_RELAY_URL"], "wss://relay.example"); + assert_eq!(env["BUZZ_AUTH_TAG"], "tag-1"); + assert_eq!(env["BUZZ_ACP_AGENT_OWNER"], "beef"); + assert_eq!(env["BUZZ_ACP_AGENT_COMMAND"], "goose"); + assert_eq!(env["BUZZ_ACP_MCP_COMMAND"], "buzz-dev-mcp"); + assert_eq!(env["BUZZ_ACP_EXIT_AFTER_INACTIVITY"], "7200"); + assert_eq!(env["BUZZ_MANAGED_AGENT_START_NONCE"], "gen0001"); + } + + /// Tier 1 is *overridable* — user env beats policy defaults, matching the + /// local spawn, where the user layer is written after them. Getting this + /// backwards would make remote agents ignore overrides local agents honor. + #[test] + fn user_env_overrides_policy_defaults() { + let agent = payload_json(serde_json::json!({ + "launch": { + "policy_env": {"GOOSE_MODE": "auto", "BUZZ_ACP_MODEL": "sonnet"}, + "env": {"GOOSE_MODE": "chat"}, + "owner_pubkey": "beef" + } + })); + let env = build(&agent).unwrap(); + assert_eq!(env["GOOSE_MODE"], "chat"); + assert_eq!(env["BUZZ_ACP_MODEL"], "sonnet"); + } + + /// `launch.env` already contains the merged user env, so re-merging the + /// legacy field would undo a layering the desktop already resolved. + #[test] + fn legacy_env_vars_are_not_remerged_when_launch_present() { + let agent = payload_json(serde_json::json!({ + "env_vars": {"STALE": "yes", "SHARED": "legacy"}, + "launch": {"env": {"SHARED": "resolved"}, "owner_pubkey": "beef"} + })); + let env = build(&agent).unwrap(); + assert_eq!(env["SHARED"], "resolved"); + assert!(!env.contains_key("STALE"), "legacy env_vars re-merged"); + } + + /// ...but a desktop predating the `launch` block has nothing else to + /// offer, so the legacy field is the truth in exactly that case. + #[test] + fn legacy_env_vars_used_when_launch_absent() { + let agent = payload_json(serde_json::json!({"env_vars": {"API": "v"}})); + let env = build(&agent).unwrap(); + assert_eq!(env["API"], "v"); + } + + #[test] + fn refuses_when_no_owner_resolves() { + let agent = payload_json(serde_json::json!({"auth_tag": null})); + let err = build(&agent).unwrap_err(); + assert!(err.contains("!shutdown"), "unhelpful error: {err}"); + } + + /// An empty string is not an owner. Without this the refusal is + /// bypassable by a blank field and the pod launches unable to be stopped. + #[test] + fn empty_owner_fields_count_as_absent() { + for blank in ["", " "] { + let agent = payload_json(serde_json::json!({ + "auth_tag": blank, + "launch": {"owner_pubkey": blank} + })); + assert!( + build(&agent).is_err(), + "whitespace resolved as an owner: {blank:?}" + ); + } + } + + /// The other half of every identity guard: what is *stored*. A validator + /// that trims and a writer that doesn't disagree about the value, and the + /// padding reaches the harness inside the Secret. Assert on the stored + /// string — asserting only that the deploy was accepted passes either way. + #[test] + fn identity_components_are_stored_trimmed() { + let agent = payload_json(serde_json::json!({ + "relay_url": " wss://relay.example ", + "auth_tag": " tag-1 ", + "launch": {"owner_pubkey": " beefcafe "} + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_RELAY_URL"], "wss://relay.example"); + assert_eq!(env["BUZZ_AUTH_TAG"], "tag-1"); + assert_eq!(env["BUZZ_ACP_AGENT_OWNER"], "beefcafe"); + } + + /// L1 item 1's third identity component. The nsec arm is enforced in + /// `naming.rs` and the owner arm above; without this one an agent + /// deploys with nothing to connect to — a pod that starts, fails at the + /// relay, and reads as a network fault rather than a refused launch. + #[test] + fn refuses_empty_relay_url() { + for blank in ["", " "] { + let agent = payload_json(serde_json::json!({"relay_url": blank})); + let err = build(&agent).unwrap_err(); + assert!(err.contains("relay_url"), "unhelpful error: {err}"); + } + } + + #[test] + fn owner_pubkey_alone_is_sufficient() { + let agent = payload_json(serde_json::json!({ + "auth_tag": null, + "launch": {"owner_pubkey": "beefcafe"} + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_ACP_AGENT_OWNER"], "beefcafe"); + assert!(!env.contains_key("BUZZ_AUTH_TAG")); + } + + #[test] + fn refuses_presence_suppression() { + let agent = payload_json(serde_json::json!({ + "launch": {"env": {"BUZZ_ACP_NO_PRESENCE": "1"}, "owner_pubkey": "beef"} + })); + let err = build(&agent).unwrap_err(); + assert!(err.contains("BUZZ_ACP_NO_PRESENCE"), "got: {err}"); + } + + /// `foo.bar` is a legal Secret key but not a legal env name: pre-1.30 + /// kubelets drop it, 1.30+ inject it. Refuse rather than behave + /// differently depending on the cluster. + #[test] + fn refuses_non_posix_env_keys() { + for bad in ["foo.bar", "foo-bar", "1LEADING", "", "has space"] { + let agent = payload_json(serde_json::json!({ + "launch": {"env": {bad: "v"}, "owner_pubkey": "beef"} + })); + assert!(build(&agent).is_err(), "accepted non-POSIX key {bad:?}"); + } + } + + #[test] + fn args_are_comma_joined_and_omitted_when_empty() { + let agent = payload_json(serde_json::json!({ + "launch": {"command": "goose", "args": ["run", "--no-session"], "owner_pubkey": "b"} + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_ACP_AGENT_ARGS"], "run,--no-session"); + + let agent = payload_json(serde_json::json!({ + "launch": {"command": "goose", "args": [], "owner_pubkey": "b"} + })); + assert!(!build(&agent).unwrap().contains_key("BUZZ_ACP_AGENT_ARGS")); + } + + /// The top-level `model`/`provider` fields are display inputs; their + /// environment consequence is per-runtime and arrives already resolved + /// inside `launch`. A provider-side mapping is wrong for three of the + /// four built-in runtimes. + #[test] + fn provider_never_maps_model_or_provider_itself() { + let agent = payload_json(serde_json::json!({ + "model": "claude-opus", "provider": "anthropic", + "launch": {"owner_pubkey": "beef"} + })); + let env = build(&agent).unwrap(); + for key in [ + "BUZZ_AGENT_PROVIDER", + "BUZZ_AGENT_MODEL", + "GOOSE_PROVIDER", + "GOOSE_MODEL", + ] { + assert!(!env.contains_key(key), "provider mapped {key} itself"); + } + } + + /// `turn_timeout_seconds` is deprecated and ignored upstream; the local + /// spawn does not emit it either. + #[test] + fn turn_timeout_is_not_mapped() { + let agent = payload_json(serde_json::json!({ + "turn_timeout_seconds": 30, "launch": {"owner_pubkey": "b"} + })); + let env = build(&agent).unwrap(); + assert!(!env.keys().any(|k| k.contains("TURN_TIMEOUT"))); + } + + #[test] + fn inactivity_omitted_when_unset() { + let agent = payload_json(serde_json::json!({"launch": {"owner_pubkey": "b"}})); + let env = build_env( + &agent, + AuthoritativeInputs { + generation: "g", + inactivity_seconds: None, + }, + ) + .unwrap(); + assert!(!env.contains_key("BUZZ_ACP_EXIT_AFTER_INACTIVITY")); + } + + /// Structural guard over the whole authoritative list at once: whatever + /// the lower tiers contain, no authoritative key holds a lower-tier value + /// — including the conditionally-written ones the authoritative tier has + /// nothing to say about, which must be **absent** rather than spoofed. + /// This test caught exactly that: `BUZZ_ACP_AGENT_ARGS` is only written + /// when `launch.args` is non-empty, so plain later-wins overwrite left the + /// spoofed value in place. + #[test] + fn no_authoritative_key_retains_a_lower_tier_value() { + let spoofed: serde_json::Map = AUTHORITATIVE_KEYS + .iter() + .map(|k| ((*k).to_string(), serde_json::json!("SPOOFED"))) + .collect(); + // Split across both lower tiers: policy_env and env are separate + // insertion points, and a fix that only cleared one would pass a + // single-tier test. + let agent = payload_json(serde_json::json!({ + "launch": { + "command": "goose", + "policy_env": spoofed.clone(), + "env": spoofed, + "owner_pubkey": "beef" + } + })); + let env = build(&agent).unwrap(); + for key in AUTHORITATIVE_KEYS { + assert_ne!( + env.get(*key).map(String::as_str), + Some("SPOOFED"), + "{key} kept its lower-tier value" + ); + } + // The keys the authoritative tier had no value for are gone, not + // merely different. + for absent in [ + "BUZZ_ACP_AGENT_ARGS", + "BUZZ_ACP_RESPOND_TO", + "BUZZ_ACP_RESPOND_TO_ALLOWLIST", + ] { + assert!(!env.contains_key(absent), "{absent} survived the clear"); + } + } + + /// A 64-hex pubkey, the only allowlist entry shape the harness accepts. + fn pubkey(fill: char) -> String { + std::iter::repeat_n(fill, 64).collect() + } + + /// The gate the harness refuses first (`config.rs:996-1004`). Refusing it + /// here is the difference between one error message and an unbounded + /// fail-replace loop that leaves a Secret per attempt. + #[test] + fn allowlist_mode_with_an_empty_list_is_refused() { + for empty in [serde_json::json!([]), serde_json::Value::Null] { + let agent = payload_json(serde_json::json!({ + "respond_to": "allowlist", + "respond_to_allowlist": empty, + })); + let err = build(&agent).unwrap_err(); + assert!( + err.contains("the allowlist is empty"), + "unexpected error: {err}" + ); + } + } + + /// `config.rs:629-641` — each entry must be exactly 64 hex characters. + /// The rejects are the distinct ways to miss that: too short, right length + /// but not hex, empty, and one character short of valid. + #[test] + fn an_allowlist_entry_that_is_not_64_hex_is_refused() { + for bad in ["abc1234", &"z".repeat(64), "", &pubkey('a')[..63]] { + let agent = payload_json(serde_json::json!({ + "respond_to": "allowlist", + "respond_to_allowlist": [pubkey('a'), bad], + })); + let err = build(&agent).unwrap_err(); + assert!( + err.contains("must be exactly 64 hex characters"), + "{bad:?} was accepted; error was: {err}" + ); + } + } + + /// The positive control: the guard refuses bad gates, not every gate. + /// Without this, a validator that refused unconditionally would pass both + /// tests above. + #[test] + fn a_valid_allowlist_gate_is_accepted_and_comma_joined() { + let agent = payload_json(serde_json::json!({ + "respond_to": "allowlist", + "respond_to_allowlist": [pubkey('a'), pubkey('b')], + })); + let env = build(&agent).unwrap(); + assert_eq!(env["BUZZ_ACP_RESPOND_TO"], "allowlist"); + assert_eq!( + env["BUZZ_ACP_RESPOND_TO_ALLOWLIST"], + format!("{},{}", pubkey('a'), pubkey('b')) + ); + } + + /// The harness validates the allowlist **only** in allowlist mode and + /// merely warns otherwise (`config.rs:1005-1010`). A stricter provider + /// would refuse a deploy whose identical local spawn succeeds, so this + /// pins the asymmetry rather than leaving it to look like an oversight. + #[test] + fn a_junk_allowlist_is_tolerated_outside_allowlist_mode() { + for mode in ["owner-only", "anyone"] { + let agent = payload_json(serde_json::json!({ + "respond_to": mode, + "respond_to_allowlist": ["not-a-pubkey"], + })); + let env = build(&agent) + .unwrap_or_else(|e| panic!("{mode} with a stale list must deploy: {e}")); + assert_eq!(env["BUZZ_ACP_RESPOND_TO"], mode); + } + } + + /// `respond_to` is an opaque `String` on the wire but a `clap::ValueEnum` + /// at the harness, so an unrecognized mode dies at `rc=2` — before config + /// parsing runs at all, earlier than either refusal above. Measured + /// against the built binary: `invalid value 'npub1abc' for '--respond-to'`. + /// This is the shape our own fixture carried until it was corrected. + #[test] + fn a_mode_the_harness_cannot_parse_is_refused() { + for bad in ["npub1abc", "OWNER-ONLY", "owner_only", "allowlistt", "x"] { + let agent = payload_json(serde_json::json!({ "respond_to": bad })); + let err = build(&agent).unwrap_err(); + assert!( + err.contains("is not a mode the harness accepts"), + "{bad:?} was accepted; error was: {err}" + ); + } + } + + /// `clap` does not trim its value-enum input, so a padded mode is `rc=2` + /// even though the same string trimmed is valid. Measured: `invalid value + /// ' allowlist ' for '--respond-to'`. Trimming here would accept a deploy + /// the harness refuses — the exact direction this guard exists to prevent. + #[test] + fn a_padded_mode_is_refused_because_clap_does_not_trim() { + for padded in [" allowlist ", "allowlist ", " owner-only", "\tnobody"] { + let agent = payload_json(serde_json::json!({ + "respond_to": padded, + "respond_to_allowlist": [pubkey('a')], + })); + let err = build(&agent).unwrap_err(); + assert!( + err.contains("is not a mode the harness accepts"), + "{padded:?} was accepted; error was: {err}" + ); + } + } + + /// Positive control for the mode check, and the reason it validates the + /// harness's four rather than the desktop's three: `nobody` is rejected by + /// `parse_wire` on purpose (`managed_agents/types.rs:871-880`) but starts + /// fine at the harness. A guard mirroring the desktop enum would refuse a + /// working launch from a non-desktop caller — the callers this guard is + /// for. Without this test, refusing `nobody` would pass everything above. + #[test] + fn every_mode_the_harness_accepts_is_deployable() { + for mode in ["owner-only", "allowlist", "anyone", "nobody"] { + let agent = payload_json(serde_json::json!({ + "respond_to": mode, + "respond_to_allowlist": [pubkey('a')], + })); + let env = build(&agent) + .unwrap_or_else(|e| panic!("{mode} is valid at the harness but was refused: {e}")); + assert_eq!(env["BUZZ_ACP_RESPOND_TO"], mode); + } + } +} diff --git a/crates/buzz-backend-kubernetes/src/gc.rs b/crates/buzz-backend-kubernetes/src/gc.rs new file mode 100644 index 0000000000..2e18825aa4 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/gc.rs @@ -0,0 +1,368 @@ +//! Preflight garbage collection (spec §K8s GC, `docs/remote-agents.md:1282-1335`). +//! +//! GC runs on every deploy, after identity derivation and before the state +//! transition. It deletes terminated pods (and their referenced Secrets) and +//! age-eligible orphan Secrets — every one of which must pass the full-pubkey +//! annotation check *and* carry the management marker. An unmarked object is +//! never GC'd regardless of its labels. +//! +//! The decision layer here is pure. The effectful caller supplies the observed +//! objects and the apiserver's clock; this module decides what may be deleted. + +use crate::naming::AgentIdentity; +use crate::observe::{referenced_secret, secret_is_ours}; +use chrono::{DateTime, Utc}; +use k8s_openapi::api::core::v1::{Pod, Secret}; + +/// The deploy operation deadline (spec §Deploy: `timeout: 600s`). +pub const OPERATION_DEADLINE_SECS: i64 = 600; + +/// An unreferenced Secret is GC-eligible only once it is older than **twice** +/// the deploy deadline. Rationale: Secret-create → pod-create is not atomic +/// against an independent GC pass, so without the gate a concurrent attempt's +/// preflight GC can delete a Secret whose pod has not been created yet and +/// strand that deploy. The age bound makes "unreferenced" mean "provably +/// abandoned" — any attempt that could still reference it has exceeded its own +/// deadline (`:1301-1319`). +pub const ORPHAN_SECRET_MIN_AGE_SECS: i64 = 2 * OPERATION_DEADLINE_SECS; + +/// What a GC pass decided to delete. Names only: the caller re-reads each +/// object's own fence at delete time. +#[derive(Debug, Default, PartialEq, Eq)] +pub struct GcPlan { + /// Terminated, verified, marker-bearing pods. + pub pods: Vec, + /// Age-eligible, verified, marker-bearing orphan Secrets. + pub secrets: Vec, +} + +/// Plan a GC pass. +/// +/// `now` is the apiserver's clock — the HTTP `Date` header from the very list +/// call that produced `secrets`. `None` means the header was absent or +/// unparseable, in which case **orphan-Secret GC is skipped entirely** rather +/// than falling back to local time: this provider runs on a user's desktop, +/// and a local clock fast by more than the margin does not race — it +/// deterministically computes every in-flight Secret as expired, on every +/// pass, reopening exactly the interleaving the gate exists to close +/// (`:1321-1335`). A deferred cleanup is free; a wrong deletion is not. +/// +/// Terminated-pod GC does not use the clock and is unaffected. +pub fn plan( + identity: &AgentIdentity, + pods: &[Pod], + secrets: &[Secret], + terminated: impl Fn(&Pod) -> bool, + now: Option>, +) -> GcPlan { + // Only pods that pass the full fence participate — in either direction. + // An unverified pod is neither deleted nor allowed to protect a Secret: + // it cannot be ours, so its `envFrom` cannot reference our generation. + let ours: Vec<&Pod> = pods + .iter() + .filter(|p| { + crate::observe::verify(p, identity, crate::classify::Startup::Started).is_some() + }) + .collect(); + + let doomed_pods: Vec<&&Pod> = ours.iter().filter(|p| terminated(p)).collect(); + + // A Secret referenced by ANY existing pod is protected — deliberately + // including not-yet-started pods, whose `envFrom` is exactly as + // load-bearing as a running pod's (`:1262-1264`). Pods being GC'd in this + // same pass are excluded, so their Secrets go with them. + let doomed_names: Vec<&str> = doomed_pods + .iter() + .filter_map(|p| p.metadata.name.as_deref()) + .collect(); + let protected: Vec = ours + .iter() + .filter(|p| !doomed_names.contains(&p.metadata.name.as_deref().unwrap_or_default())) + .filter_map(|p| referenced_secret(p)) + .collect(); + + let mut plan = GcPlan { + pods: doomed_names.iter().map(|n| n.to_string()).collect(), + secrets: doomed_pods + .iter() + .filter_map(|p| referenced_secret(p)) + .collect(), + }; + + // Orphan sweep: only with a server clock. + if let Some(now) = now { + for secret in secrets { + if !secret_is_ours(secret, identity) { + continue; + } + let Some(name) = secret.metadata.name.as_deref() else { + continue; + }; + if protected.contains(&name.to_string()) || plan.secrets.iter().any(|s| s == name) { + continue; + } + let Some(created) = secret.metadata.creation_timestamp.as_ref() else { + // No server-assigned timestamp means no age proof. Skip. + continue; + }; + if (now - created.0).num_seconds() >= ORPHAN_SECRET_MIN_AGE_SECS { + plan.secrets.push(name.to_string()); + } + } + } + + plan +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::naming::{ANNOTATION_PUBKEY_FULL, LABEL_MANAGED_BY}; + use k8s_openapi::api::core::v1::{Container, EnvFromSource, PodSpec, SecretEnvSource}; + use k8s_openapi::apimachinery::pkg::apis::meta::v1::{ObjectMeta, Time}; + use std::collections::BTreeMap; + + fn identity() -> AgentIdentity { + use nostr::nips::nip19::ToBech32; + let keys = nostr::Keys::generate(); + AgentIdentity::from_nsec(&keys.secret_key().to_bech32().unwrap()).unwrap() + } + + fn pod_named(id: &AgentIdentity, name: &str, secret: Option<&str>) -> Pod { + Pod { + metadata: ObjectMeta { + name: Some(name.into()), + uid: Some(format!("uid-{name}")), + resource_version: Some("1".into()), + labels: Some(id.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + id.pubkey_hex().to_string(), + )] + .into_iter() + .collect::>(), + ), + ..Default::default() + }, + spec: secret.map(|s| PodSpec { + containers: vec![Container { + name: "agent".into(), + env_from: Some(vec![EnvFromSource { + secret_ref: Some(SecretEnvSource { + name: s.into(), + optional: Some(false), + }), + ..Default::default() + }]), + ..Default::default() + }], + ..Default::default() + }), + ..Default::default() + } + } + + fn secret_named(id: &AgentIdentity, name: &str, age_secs: i64, now: DateTime) -> Secret { + Secret { + metadata: ObjectMeta { + name: Some(name.into()), + labels: Some(id.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + id.pubkey_hex().to_string(), + )] + .into_iter() + .collect::>(), + ), + creation_timestamp: Some(Time(now - chrono::Duration::seconds(age_secs))), + ..Default::default() + }, + ..Default::default() + } + } + + fn never(_: &Pod) -> bool { + false + } + fn always(_: &Pod) -> bool { + true + } + + #[test] + fn terminated_pods_and_their_secrets_are_collected_together() { + let id = identity(); + let now = Utc::now(); + let pod = pod_named(&id, "buzz-agent-dead", Some("buzz-agent-dead-gen1")); + let plan = plan(&id, &[pod], &[], always, Some(now)); + assert_eq!(plan.pods, ["buzz-agent-dead"]); + assert_eq!(plan.secrets, ["buzz-agent-dead-gen1"]); + } + + #[test] + fn live_pods_are_never_collected() { + let id = identity(); + let pod = pod_named(&id, "buzz-agent-live", Some("buzz-agent-live-gen1")); + let plan = plan(&id, &[pod], &[], never, Some(Utc::now())); + assert_eq!(plan, GcPlan::default()); + } + + /// The auto-repair fence applies to GC identically: an object that lacks + /// the marker, or carries a different pubkey, is never touched — however + /// well its labels match. + #[test] + fn unmarked_and_mismatched_objects_are_never_collected() { + let id = identity(); + let other = identity(); + let now = Utc::now(); + + let mut unmarked = pod_named(&id, "look-alike", Some("look-alike-gen1")); + let mut labels = id.labels(); + labels.remove(LABEL_MANAGED_BY); + unmarked.metadata.labels = Some(labels); + + let mut foreign = pod_named(&id, "someone-elses", Some("someone-elses-gen1")); + foreign.metadata.annotations = Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + other.pubkey_hex().to_string(), + )] + .into_iter() + .collect(), + ); + + let mut unmarked_secret = secret_named(&id, "orphan-unmarked", 100_000, now); + unmarked_secret.metadata.labels = Some(BTreeMap::new()); + let mut foreign_secret = secret_named(&id, "orphan-foreign", 100_000, now); + foreign_secret.metadata.annotations = Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + other.pubkey_hex().to_string(), + )] + .into_iter() + .collect(), + ); + + let plan = plan( + &id, + &[unmarked, foreign], + &[unmarked_secret, foreign_secret], + always, + Some(now), + ); + assert_eq!( + plan, + GcPlan::default(), + "GC touched an object it does not own" + ); + } + + /// The interleaving the age gate exists to close: attempt A creates its + /// Secret; concurrent attempt B's preflight GC runs before A creates its + /// pod. Without the gate B deletes A's Secret and strands A. + #[test] + fn young_unreferenced_secrets_are_protected() { + let id = identity(); + let now = Utc::now(); + let fresh = secret_named(&id, "buzz-agent-x-gen-inflight", 5, now); + assert_eq!( + plan(&id, &[], &[fresh], never, Some(now)), + GcPlan::default() + ); + } + + /// Past twice the deadline, any attempt that could still reference the + /// Secret has exceeded its own deadline — so it is provably abandoned. + #[test] + fn secrets_older_than_twice_the_deadline_are_collected() { + let id = identity(); + let now = Utc::now(); + let old = secret_named( + &id, + "buzz-agent-x-gen-abandoned", + ORPHAN_SECRET_MIN_AGE_SECS + 1, + now, + ); + let plan = plan(&id, &[], &[old], never, Some(now)); + assert_eq!(plan.secrets, ["buzz-agent-x-gen-abandoned"]); + } + + /// The boundary itself, both sides. `>= 1200s` is eligible. + #[test] + fn age_gate_boundary_is_exact() { + let id = identity(); + let now = Utc::now(); + let just_under = secret_named(&id, "under", ORPHAN_SECRET_MIN_AGE_SECS - 1, now); + let exactly = secret_named(&id, "exact", ORPHAN_SECRET_MIN_AGE_SECS, now); + assert!(plan(&id, &[], &[just_under], never, Some(now)) + .secrets + .is_empty()); + assert_eq!( + plan(&id, &[], &[exactly], never, Some(now)).secrets, + ["exact"] + ); + } + + /// The same-clock rule. No apiserver `Date` header → skip the orphan + /// sweep entirely. A local clock fast by more than the margin would + /// silently delete every in-flight Secret on every pass. + #[test] + fn without_a_server_clock_the_orphan_sweep_is_skipped() { + let id = identity(); + let now = Utc::now(); + let ancient = secret_named(&id, "buzz-agent-x-gen-ancient", 10_000_000, now); + let plan = plan(&id, &[], &[ancient], never, None); + assert!( + plan.secrets.is_empty(), + "orphan swept without a server clock — a fast local clock would delete live Secrets" + ); + } + + /// ...but terminated-pod GC does not consult the clock, so it still runs. + #[test] + fn terminated_pod_gc_runs_without_a_server_clock() { + let id = identity(); + let pod = pod_named(&id, "buzz-agent-dead", Some("buzz-agent-dead-gen1")); + let plan = plan(&id, &[pod], &[], always, None); + assert_eq!(plan.pods, ["buzz-agent-dead"]); + assert_eq!(plan.secrets, ["buzz-agent-dead-gen1"]); + } + + /// "Existing" includes not-yet-started pods: a Secret referenced by a pod + /// still pulling its image must not be swept, however old it is. + #[test] + fn secrets_referenced_by_a_pending_pod_are_protected() { + let id = identity(); + let now = Utc::now(); + let pending = pod_named(&id, "buzz-agent-pending", Some("buzz-agent-pending-gen1")); + let old = secret_named(&id, "buzz-agent-pending-gen1", 10_000_000, now); + let plan = plan(&id, &[pending], &[old], never, Some(now)); + assert!(plan.secrets.is_empty(), "swept a referenced Secret"); + } + + /// A Secret with no server-assigned creationTimestamp has no age proof, + /// so it is skipped rather than assumed old. + #[test] + fn secrets_without_a_creation_timestamp_are_skipped() { + let id = identity(); + let now = Utc::now(); + let mut no_timestamp = secret_named(&id, "buzz-agent-x-gen-unknown", 10_000_000, now); + no_timestamp.metadata.creation_timestamp = None; + assert!(plan(&id, &[], &[no_timestamp], never, Some(now)) + .secrets + .is_empty()); + } + + /// A Secret belonging to a pod being collected in this same pass goes with + /// it, and must not be listed twice. + #[test] + fn a_collected_pods_secret_is_listed_once() { + let id = identity(); + let now = Utc::now(); + let dead = pod_named(&id, "buzz-agent-dead", Some("buzz-agent-dead-gen1")); + let its_secret = secret_named(&id, "buzz-agent-dead-gen1", 10_000_000, now); + let plan = plan(&id, &[dead], &[its_secret], always, Some(now)); + assert_eq!(plan.secrets, ["buzz-agent-dead-gen1"]); + } +} diff --git a/crates/buzz-backend-kubernetes/src/image.rs b/crates/buzz-backend-kubernetes/src/image.rs new file mode 100644 index 0000000000..409989fda9 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/image.rs @@ -0,0 +1,184 @@ +//! Image reference validation (spec §Image). +//! +//! The object holding this reference runs with an nsec, so the reference must +//! be **immutable**. Registry tags are mutable pointers — Kubernetes itself +//! distinguishes them from digests for exactly this reason — so a tag-only +//! reference is rejected, not just `:latest`. +//! +//! There is no parse-time fallback: `image` is required, and its absence +//! fails closed with a named field. The published `ghcr.io/block/buzz-sprig` +//! digest is offered only as a schema `default` (a UI prefill the desktop +//! submits explicitly — see `config::DEFAULT_IMAGE`), so the create-intent +//! fingerprint never depends on compiled-in provider state. + +/// A validated, digest-qualified image reference. +/// +/// The inner string is always in canonical tagless form `name@sha256:`: +/// `name:tag@sha256:…` normalizes by dropping the tag, because the tag is +/// decorative once a digest pins the content, and leaving it in would make +/// two references to identical bytes produce different create-intent +/// fingerprints. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ImageRef(String); + +impl ImageRef { + pub fn as_str(&self) -> &str { + &self.0 + } +} + +impl std::fmt::Display for ImageRef { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +/// Parse and normalize a user-supplied image reference. +pub fn parse(raw: &str) -> Result { + let reference = raw.trim(); + if reference.is_empty() { + return Err("provider_config.image is required: no image is assumed \ + at parse time, so the digest-pinned image to run must be \ + given explicitly" + .to_string()); + } + + let mut parts = reference.split('@'); + let name_and_tag = parts.next().unwrap_or_default(); + let digest = match (parts.next(), parts.next()) { + (Some(d), None) => d, + (None, _) => { + return Err(format!( + "provider_config.image {reference:?} is not digest-pinned: a \ + tag is a mutable pointer, and this object runs with the \ + agent's private key. Use name@sha256:<64 hex chars>" + )) + } + (Some(_), Some(_)) => { + return Err(format!( + "provider_config.image {reference:?} contains more than one '@'" + )) + } + }; + + let hex = digest.strip_prefix("sha256:").ok_or_else(|| { + format!("provider_config.image digest {digest:?} must start with 'sha256:'") + })?; + // Lowercase only: OCI canonicalizes digest hex, and accepting uppercase + // would let two spellings of one digest produce two fingerprints. + if hex.len() != 64 + || !hex + .chars() + .all(|c| c.is_ascii_digit() || ('a'..='f').contains(&c)) + { + return Err(format!( + "provider_config.image digest {digest:?} must be exactly 64 \ + lowercase hex characters" + )); + } + + // Drop any tag: `name:tag@sha256:…` and `name@sha256:…` name the same + // bytes and must fingerprint identically. Only a *final* colon segment + // that isn't a port counts as a tag — `host:5000/name` has no tag. + let name = match name_and_tag.rfind(':') { + Some(colon) if !name_and_tag[colon + 1..].contains('/') => &name_and_tag[..colon], + _ => name_and_tag, + }; + if name.is_empty() { + return Err(format!( + "provider_config.image {reference:?} has no repository name" + )); + } + + Ok(ImageRef(format!("{name}@sha256:{hex}"))) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn accepts_digest_pinned_reference() { + let d = "a".repeat(64); + let r = parse(&format!("ghcr.io/block/buzz-sprig@sha256:{d}")).unwrap(); + assert_eq!(r.as_str(), format!("ghcr.io/block/buzz-sprig@sha256:{d}")); + } + + /// The normalization that keeps the fingerprint stable: two spellings of + /// the same bytes must produce one reference. + #[test] + fn strips_tag_from_tag_plus_digest_form() { + let d = "b".repeat(64); + let tagged = parse(&format!("ghcr.io/block/buzz-sprig:v1.2@sha256:{d}")).unwrap(); + let plain = parse(&format!("ghcr.io/block/buzz-sprig@sha256:{d}")).unwrap(); + assert_eq!(tagged, plain); + } + + /// A registry port is not a tag. `host:5000/name` must keep its port. + #[test] + fn registry_port_is_not_mistaken_for_a_tag() { + let d = "c".repeat(64); + let r = parse(&format!("localhost:5000/buzz-sprig@sha256:{d}")).unwrap(); + assert_eq!(r.as_str(), format!("localhost:5000/buzz-sprig@sha256:{d}")); + } + + #[test] + fn port_and_tag_together_drops_only_the_tag() { + let d = "d".repeat(64); + let r = parse(&format!("localhost:5000/buzz-sprig:dev@sha256:{d}")).unwrap(); + assert_eq!(r.as_str(), format!("localhost:5000/buzz-sprig@sha256:{d}")); + } + + /// Wren's amendment: *every* tag-only reference is rejected, not just + /// `:latest`. A `sha-` tag is traceable but still movable. + #[test] + fn rejects_every_tag_only_reference() { + for bad in [ + "ghcr.io/block/buzz-sprig:latest", + "ghcr.io/block/buzz-sprig:v1.2.3", + "ghcr.io/block/buzz-sprig:sha-abc1234", + "ghcr.io/block/buzz-sprig", + "localhost:5000/buzz-sprig", + ] { + let err = parse(bad).unwrap_err(); + assert!(err.contains("digest-pinned"), "for {bad:?} got: {err}"); + } + } + + /// Uppercase hex is a second spelling of one digest; accepting it would + /// let the same image fingerprint two ways. + #[test] + fn rejects_uppercase_digest_hex() { + let d = "A".repeat(64); + assert!(parse(&format!("img@sha256:{d}")).is_err()); + } + + #[test] + fn rejects_malformed_digests() { + let short = "a".repeat(63); + let long = "a".repeat(65); + let ok = "a".repeat(64); + for bad in [ + format!("img@sha256:{short}"), + format!("img@sha256:{long}"), + format!("img@sha512:{ok}"), + format!("img@{ok}"), + format!("img@sha256:{}", "g".repeat(64)), + format!("img@sha256:{ok}@sha256:{ok}"), + format!("@sha256:{ok}"), + ] { + assert!(parse(&bad).is_err(), "accepted {bad:?}"); + } + } + + /// Parsing has no fallback (the schema default is a UI prefill, not a + /// parse-time substitute), so an absent image is an error that names the + /// field rather than a silent fallback. + #[test] + fn empty_reference_names_the_field() { + for empty in ["", " "] { + let err = parse(empty).unwrap_err(); + assert!(err.contains("provider_config.image"), "got: {err}"); + } + } +} diff --git a/crates/buzz-backend-kubernetes/src/intent.rs b/crates/buzz-backend-kubernetes/src/intent.rs new file mode 100644 index 0000000000..73218f83bf --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/intent.rs @@ -0,0 +1,309 @@ +//! The create-intent fingerprint (spec §Deploy State Machine, create-intent +//! fingerprint, `docs/remote-agents.md:796-828`). +//! +//! The fingerprint is an unkeyed SHA-256 over a canonical serialization of the +//! provider's non-secret create-intent template. A plain hash is safe *only* +//! because of the scope rule: the input covers exactly the provider-controlled +//! fields that can affect scheduling or container creation, and **never Secret +//! data or attempt identity**. Hashing low-entropy secrets into a +//! world-readable annotation would be a dictionary oracle. +//! +//! That rule is enforced structurally rather than remembered. [`IntentTemplate`] +//! is a *pre-binding* type: it has no field that can hold Secret material or a +//! generation token, so there is no expression that hashes one. The +//! per-attempt Secret name never appears — the pod's `envFrom` is represented +//! by the fixed [`SECRET_PLACEHOLDER`], because otherwise every attempt would +//! diverge from every other by construction. +//! +//! Server- and admission-produced output (UID, `resourceVersion`, timestamps, +//! defaulted fields, the annotation itself) is excluded the same way: the +//! serializer is only ever handed this template, never a live `Pod`, so the +//! exclusion is checkable by inspection. + +use crate::image::ImageRef; +use serde::Serialize; +use sha2::{Digest, Sha256}; + +/// Stands in for the per-attempt Secret name in the `envFrom` position. +/// A real generation token here would make every attempt diverge. +const SECRET_PLACEHOLDER: &str = ""; + +/// The recorded/computed create intent: a hex SHA-256 digest. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Fingerprint(String); + +impl Fingerprint { + pub fn as_str(&self) -> &str { + &self.0 + } + + /// Read a fingerprint off a pod annotation. Any recorded string is + /// accepted verbatim: comparison is equality against a freshly computed + /// value, so a malformed annotation simply reads as divergence — which is + /// the correct outcome for a pod this provider version did not write. + pub fn from_annotation(value: &str) -> Self { + Self(value.to_string()) + } + + #[cfg(test)] + pub fn for_test(seed: &str) -> Self { + Self(format!("test-{seed}")) + } +} + +impl std::fmt::Display for Fingerprint { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +/// The non-secret, pre-binding description of the pod this deploy would +/// create. Every field is provider-controlled and scheduling-relevant; there +/// is deliberately no field for env values, Secret data, or the generation. +#[derive(Debug, Clone, PartialEq, Eq, Serialize)] +pub struct IntentTemplate { + /// Schema version of the template itself. Bumping it re-fingerprints every + /// pod, which is the intended way to roll out a pod-shape change. + pub template_version: u32, + pub namespace: String, + /// Normalized, digest-qualified image reference. + pub image: String, + pub cpu_request: String, + pub memory_request: String, + pub cpu_limit: String, + pub memory_limit: String, + pub service_account: Option, + pub restart_policy: &'static str, + pub termination_grace_period_seconds: i64, + /// Env *keys* only, sorted. Keys are pod-shape (a renamed key changes the + /// container's contract); values are Secret material and must not be here. + pub env_keys: Vec, + /// Fixed placeholder for the per-attempt Secret in `envFrom`. + pub env_from_secret: &'static str, + pub workspace_mount_path: String, + pub run_as_user: i64, + pub run_as_group: i64, +} + +/// Current template schema version. +pub const TEMPLATE_VERSION: u32 = 1; + +impl IntentTemplate { + /// Compute the fingerprint. `serde_json` on a struct with declared field + /// order plus pre-sorted `env_keys` is a canonical serialization: the same + /// template always produces the same bytes. + pub fn fingerprint(&self) -> Fingerprint { + let canonical = serde_json::to_vec(self).expect("intent template is plain data"); + Fingerprint(hex::encode(Sha256::digest(&canonical))) + } + + /// Build from resolved pod-shape inputs. `env_keys` is sorted here rather + /// than at the call site so key ordering can never leak into the digest. + /// + /// The fixed pod-shape constants are read from [`crate::config`] rather + /// than passed in: `pod::build_pod` stamps the pod from those same + /// constants, so the fingerprint cannot describe a pod shape different + /// from the one actually created. Threading them through as arguments + /// would make that agreement a thing to test instead of a thing that holds. + pub fn new( + namespace: &str, + image: &ImageRef, + resources: &crate::config::Resources, + service_account: Option<&str>, + env_keys: impl IntoIterator, + ) -> Self { + let mut env_keys: Vec = env_keys.into_iter().collect(); + env_keys.sort(); + Self { + template_version: TEMPLATE_VERSION, + namespace: namespace.to_string(), + image: image.as_str().to_string(), + cpu_request: resources.cpu_request.clone(), + memory_request: resources.memory_request.clone(), + cpu_limit: resources.cpu_limit.clone(), + memory_limit: resources.memory_limit.clone(), + service_account: service_account.map(str::to_string), + restart_policy: crate::config::RESTART_POLICY, + termination_grace_period_seconds: crate::config::TERMINATION_GRACE_SECONDS, + env_keys, + env_from_secret: SECRET_PLACEHOLDER, + workspace_mount_path: crate::config::WORKSPACE_PATH.to_string(), + run_as_user: crate::config::RUN_AS_UID, + run_as_group: crate::config::RUN_AS_GID, + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::config::Resources; + + fn image(byte: char) -> ImageRef { + crate::image::parse(&format!( + "ghcr.io/block/buzz-sprig@sha256:{}", + byte.to_string().repeat(64) + )) + .unwrap() + } + + fn template() -> IntentTemplate { + IntentTemplate::new( + "buzz-agents", + &image('a'), + &Resources::default(), + None, + ["BUZZ_RELAY_URL".to_string(), "GOOSE_MODE".to_string()], + ) + } + + #[test] + fn fingerprint_is_deterministic() { + assert_eq!(template().fingerprint(), template().fingerprint()); + } + + #[test] + fn fingerprint_is_hex_sha256() { + let fp = template().fingerprint(); + assert_eq!(fp.as_str().len(), 64); + assert!(fp.as_str().chars().all(|c| c.is_ascii_hexdigit())); + } + + /// Key *order* must not reach the digest, or two identical environments + /// built in different orders would look like a config change. + #[test] + fn env_key_order_does_not_affect_the_digest() { + let a = IntentTemplate::new( + "ns", + &image('a'), + &Resources::default(), + None, + ["A".to_string(), "B".to_string(), "C".to_string()], + ); + let b = IntentTemplate::new( + "ns", + &image('a'), + &Resources::default(), + None, + ["C".to_string(), "A".to_string(), "B".to_string()], + ); + assert_eq!(a.fingerprint(), b.fingerprint()); + } + + /// A mutation applied to a fresh template clone, named for its assertion + /// message. + type Mutation = (&'static str, Box); + + /// Every scheduling-relevant knob must move the digest — this is the + /// wedge escape (§Deploy State Machine never-started recoverable row). + /// Exhaustive by construction: each mutation is applied to a fresh clone. + #[test] + fn every_scheduling_field_changes_the_digest() { + let base = template(); + let baseline = base.fingerprint(); + + let mutations: Vec = vec![ + ( + "template_version", + Box::new(|t: &mut IntentTemplate| t.template_version += 1), + ), + ( + "namespace", + Box::new(|t: &mut IntentTemplate| t.namespace = "other".into()), + ), + ( + "image", + Box::new(|t: &mut IntentTemplate| t.image = image('b').as_str().into()), + ), + ( + "cpu_request", + Box::new(|t: &mut IntentTemplate| t.cpu_request = "4".into()), + ), + ( + "memory_request", + Box::new(|t: &mut IntentTemplate| t.memory_request = "8Gi".into()), + ), + ( + "cpu_limit", + Box::new(|t: &mut IntentTemplate| t.cpu_limit = "8".into()), + ), + ( + "memory_limit", + Box::new(|t: &mut IntentTemplate| t.memory_limit = "16Gi".into()), + ), + ( + "service_account", + Box::new(|t: &mut IntentTemplate| t.service_account = Some("sa".into())), + ), + ( + "restart_policy", + Box::new(|t: &mut IntentTemplate| t.restart_policy = "OnFailure"), + ), + ( + "grace_period", + Box::new(|t: &mut IntentTemplate| t.termination_grace_period_seconds = 30), + ), + ( + "env_keys", + Box::new(|t: &mut IntentTemplate| t.env_keys.push("NEW_KEY".into())), + ), + ( + "workspace_mount_path", + Box::new(|t: &mut IntentTemplate| t.workspace_mount_path = "/w".into()), + ), + ( + "run_as_user", + Box::new(|t: &mut IntentTemplate| t.run_as_user = 2000), + ), + ( + "run_as_group", + Box::new(|t: &mut IntentTemplate| t.run_as_group = 2000), + ), + ]; + + for (name, mutate) in mutations { + let mut t = base.clone(); + mutate(&mut t); + assert_ne!( + t.fingerprint(), + baseline, + "{name} did not affect the digest" + ); + } + } + + /// The scope rule, asserted on the bytes: no Secret value and no + /// generation token can appear in the serialization, because the type has + /// nowhere to put them. The placeholder is what `envFrom` contributes. + #[test] + fn serialization_contains_no_secret_material_or_attempt_identity() { + let json = serde_json::to_string(&template()).unwrap(); + for forbidden in ["nsec1", "SPOOFED", "wss://", "gen0001"] { + assert!( + !json.contains(forbidden), + "template leaked {forbidden}: {json}" + ); + } + assert!(json.contains(SECRET_PLACEHOLDER)); + } + + /// Two attempts for the same agent differ only in generation, which is + /// absent from the template — so their fingerprints must be equal, or the + /// divergence discriminator would fire on every single deploy. + #[test] + fn attempts_differing_only_by_generation_do_not_diverge() { + // There is no generation input to pass; that *is* the property. The + // test states it explicitly so a future field addition breaks here. + assert_eq!(template().fingerprint(), template().fingerprint()); + let json = serde_json::to_string(&template()).unwrap(); + assert_eq!(json.matches(SECRET_PLACEHOLDER).count(), 1); + } + + /// A recorded annotation this provider version did not write reads as + /// divergence rather than an error. + #[test] + fn unrecognized_annotation_reads_as_divergence() { + let recorded = Fingerprint::from_annotation("not-a-digest"); + assert_ne!(recorded, template().fingerprint()); + } +} diff --git a/crates/buzz-backend-kubernetes/src/main.rs b/crates/buzz-backend-kubernetes/src/main.rs new file mode 100644 index 0000000000..5d521a9d37 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/main.rs @@ -0,0 +1,199 @@ +//! Kubernetes backend provider for Buzz remote agents +//! (spec `docs/remote-agents.md`). +//! +//! One process per operation: read exactly one JSON request from stdin, write +//! exactly one JSON response to stdout, exit. The exit code carries exactly +//! one bit — 0 for a response that was produced, 1 for a failure to produce +//! one. Everything a caller needs to distinguish is *inside* the response's +//! `ok` field, because a provider that encoded outcomes in exit codes would +//! have a second, redundant error channel to keep in sync (§Provider Protocol). + +mod classify; +mod client; +mod cluster; +mod config; +mod env; +mod gc; +mod image; +mod intent; +mod naming; +mod observe; +mod pod; +mod reconcile; +mod wire; + +use std::io::Read; +use wire::{Request, Response}; + +/// The provider a shared-compute agent resolves to. Refused here as the +/// spec's backstop: a mesh agent runs on the relay's compute, so deploying it +/// as a pod would create a second, contending consumer of the same agent +/// identity (`:214-219`). +const RELAY_MESH_PROVIDER: &str = "relay-mesh"; + +fn main() { + // rustls needs a process-level provider before the first TLS connection. + // The release build compiles every sidecar in one cargo invocation, which + // unifies the `ring` and `aws-lc-rs` features and leaves rustls unable to + // auto-select — so this is an explicit install, not a default. + let _ = rustls::crypto::ring::default_provider().install_default(); + + let mut input = String::new(); + if let Err(e) = std::io::stdin().read_to_string(&mut input) { + // No request means no request_id and no response contract to honor. + // This is the one path that exits nonzero. + eprintln!("could not read the request from stdin: {e}"); + std::process::exit(1); + } + + let response = respond(&input); + println!( + "{}", + serde_json::to_string(&response).unwrap_or_else(|e| { + // The response types are plain data; this cannot fail in practice, + // and a hand-built object is still a conforming response. + format!(r#"{{"ok":false,"error":"could not serialize a response: {e}"}}"#) + }) + ); +} + +/// Produce the single response for one request. Separated from `main` so the +/// whole dispatch is testable without a process. +fn respond(input: &str) -> Response { + // Parsed as raw JSON first: the relay-mesh refusal below MUST see the wire + // value, and `AgentPayload` deliberately does not carry `provider`. + let raw: serde_json::Value = match serde_json::from_str(input) { + Ok(value) => value, + Err(e) => return Response::error(format!("request is not valid JSON: {e}")), + }; + + if let Some(refusal) = refuse_relay_mesh(&raw) { + return Response::error(refusal); + } + + let request: Request = match serde_json::from_value(raw) { + Ok(request) => request, + Err(e) => return Response::error(format!("could not understand the request: {e}")), + }; + + match request { + Request::Info => Response::info(), + Request::Deploy(deploy) => { + let runtime = match tokio::runtime::Builder::new_current_thread() + .enable_all() + .build() + { + Ok(runtime) => runtime, + Err(e) => return Response::error(format!("could not start the runtime: {e}")), + }; + match runtime.block_on(deploy_agent(&deploy)) { + Ok(agent_id) => Response::deployed(agent_id), + Err(e) => Response::error(e), + } + } + } +} + +/// Refuse a shared-compute agent, reading the **raw wire value**. +/// +/// Trimmed before comparing: the desktop's own layers disagree about padding +/// (`relay_mesh.rs:17` and `effective_config/mod.rs:46` trim; the deploy guard +/// at `agents_deploy.rs:116` did not), and `non_blank` preserves surrounding +/// whitespace on a non-blank value. A backstop that shares its bypass with the +/// layer it backs is not a backstop. +fn refuse_relay_mesh(raw: &serde_json::Value) -> Option { + let provider = raw.get("agent")?.get("provider")?.as_str()?; + (provider.trim() == RELAY_MESH_PROVIDER).then(|| { + "deploy refused: this agent is configured for shared compute \ + (relay-mesh), which runs on the relay rather than in a pod. \ + Switch the agent to a local runtime before deploying it to \ + Kubernetes." + .to_string() + }) +} + +/// Run one deploy to a terminal outcome. +async fn deploy_agent(request: &wire::DeployRequest) -> Result { + let cfg = config::parse(&request.provider_config)?; + // Identity before any cluster contact: a malformed nsec is a refusal, not + // a failed connection (§Deploy State Machine step 0). + let identity = naming::AgentIdentity::from_nsec(&request.agent.private_key_nsec)?; + + // One generation for this operation's first attempt; the reconciler mints + // its own per attempt and restamps the correlator to match. + let env = env::build_env( + &request.agent, + env::AuthoritativeInputs { + generation: &naming::new_generation(), + inactivity_seconds: cfg.inactivity_seconds, + }, + )?; + + let client = client::connect(cfg.context.as_deref()).await?; + let substrate = cluster::Cluster::new(client, &cfg.namespace); + reconcile::deploy(&substrate, &identity, &cfg, env).await +} + +#[cfg(test)] +mod tests { + use super::*; + + fn error_of(response: &Response) -> String { + let json = serde_json::to_value(response).unwrap(); + assert_eq!(json["ok"], false, "expected a refusal: {json}"); + json["error"].as_str().unwrap().to_string() + } + + /// The spec's backstop for the relay-mesh MUST. The desktop refuses first + /// (`agents_deploy.rs:116`); this is the layer that owes the obligation. + #[test] + fn refuses_a_relay_mesh_agent() { + let request = r#"{"op":"deploy","agent":{ + "relay_url":"wss://r","private_key_nsec":"nsec1x","provider":"relay-mesh"}, + "provider_config":{"namespace":"ns"}}"#; + assert!(error_of(&respond(request)).contains("relay-mesh")); + } + + /// Padding must not bypass the backstop. Reachable by construction: + /// `GlobalConfig.provider` is a bare `Option` with no trim on + /// write, and `non_blank` rejects whitespace-only while preserving + /// surrounding whitespace on everything else. + #[test] + fn refuses_a_padded_relay_mesh_agent() { + let request = r#"{"op":"deploy","agent":{ + "relay_url":"wss://r","private_key_nsec":"nsec1x","provider":" relay-mesh "}, + "provider_config":{"namespace":"ns"}}"#; + assert!(error_of(&respond(request)).contains("relay-mesh")); + } + + /// The refusal must not fire on a normal agent — a guard that refuses + /// everything passes its own test and ships a provider that deploys + /// nothing. + #[test] + fn does_not_refuse_a_normal_provider() { + let raw: serde_json::Value = + serde_json::from_str(r#"{"agent":{"provider":"openai"}}"#).unwrap(); + assert!(refuse_relay_mesh(&raw).is_none()); + // …nor when the field is absent entirely, which is the common case: + // `AgentPayload` does not carry `provider`. + let bare: serde_json::Value = serde_json::from_str(r#"{"agent":{}}"#).unwrap(); + assert!(refuse_relay_mesh(&bare).is_none()); + } + + /// Malformed input still produces exactly one conforming response. + #[test] + fn malformed_input_is_an_in_band_error() { + assert!(error_of(&respond("not json")).contains("valid JSON")); + assert!(error_of(&respond(r#"{"op":"undeploy"}"#)).contains("understand")); + } + + /// `info` answers without touching a cluster — it is what the desktop + /// calls to render the config form, before any kubeconfig exists. + #[test] + fn info_answers_with_the_protocol_version_and_schema() { + let json = serde_json::to_value(respond(r#"{"op":"info"}"#)).unwrap(); + assert_eq!(json["ok"], true); + assert_eq!(json["protocol_version"], wire::PROTOCOL_VERSION); + assert!(json["config_schema"]["properties"]["namespace"].is_object()); + } +} diff --git a/crates/buzz-backend-kubernetes/src/naming.rs b/crates/buzz-backend-kubernetes/src/naming.rs new file mode 100644 index 0000000000..4b9d7ea035 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/naming.rs @@ -0,0 +1,223 @@ +//! Identity derivation and the object-naming contract (spec §Pod shape). +//! +//! Every name, label, and annotation below is derived from the pubkey the +//! provider decoded itself from `private_key_nsec` — never from a +//! caller-supplied pubkey (§Deploy State Machine step 0). + +use nostr::nips::nip19::FromBech32; + +/// `app.kubernetes.io/managed-by` value: the management marker's identity half. +pub const MANAGED_BY: &str = "buzz-backend-kubernetes"; + +/// Label key carrying [`MANAGED_BY`]. +pub const LABEL_MANAGED_BY: &str = "app.kubernetes.io/managed-by"; + +/// Label key carrying [`BINDING_VERSION`] — the marker's schema half. +pub const LABEL_BINDING_VERSION: &str = "buzz.block.xyz/binding-version"; + +/// Schema version of the object layout this provider writes. Bumped when the +/// pod/Secret shape changes in a way a older provider would mis-handle. +pub const BINDING_VERSION: &str = "1"; + +/// Label key: truncated pubkey, the reconciliation and GC selector. +pub const LABEL_AGENT_PUBKEY: &str = "buzz.block.xyz/agent-pubkey"; + +/// Annotation key: full pubkey. Load-bearing — the truncated label is +/// collision-*resistant*, this is what makes it safe (§Deploy State Machine +/// step 1). +pub const ANNOTATION_PUBKEY_FULL: &str = "buzz.block.xyz/agent-pubkey-full"; + +/// Annotation key: the recorded create-intent fingerprint. +pub const ANNOTATION_CREATE_INTENT: &str = "buzz.block.xyz/create-intent"; + +/// Annotation key: the image reference this generation actually resolved to, +/// for post-hoc attribution (§Image). +pub const ANNOTATION_IMAGE: &str = "buzz.block.xyz/image"; + +/// An agent identity the provider derived itself, plus every name it implies. +/// +/// Constructing this type is the *only* way to obtain the names — so a +/// caller-supplied pubkey cannot reach a selector by any path. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct AgentIdentity { + pubkey_hex: String, +} + +impl AgentIdentity { + /// Derive from the payload's `private_key_nsec`. + /// + /// Accepts bech32 `nsec1…`; a malformed or undecodable key is an + /// immediate error, before any substrate read or mutation + /// (§Deploy State Machine step 0). + pub fn from_nsec(nsec: &str) -> Result { + let secret = nostr::SecretKey::from_bech32(nsec.trim()) + .map_err(|_| "private_key_nsec is not a decodable nsec1 key".to_string())?; + let keys = nostr::Keys::new(secret); + Ok(Self { + pubkey_hex: keys.public_key().to_hex(), + }) + } + + /// Full 64-hex public key — the annotation value and the comparison + /// operand for candidate authentication. + pub fn pubkey_hex(&self) -> &str { + &self.pubkey_hex + } + + /// Selector label value: first 32 hex chars (128 bits). A full hex pubkey + /// is 64 chars and label values cap at 63, which is why this is truncated + /// and why the annotation check is normative rather than decorative. + pub fn label_pubkey(&self) -> &str { + &self.pubkey_hex[..32] + } + + /// Deterministic pod name, also the returned `agent_id`. + pub fn pod_name(&self) -> String { + format!("buzz-agent-{}", &self.pubkey_hex[..12]) + } + + /// Per-attempt Secret name. `generation` is a fresh random token per + /// create attempt — never reused — which is what makes payload and Secret + /// atomic at the pod-spec boundary (§K8s Secrets). + pub fn secret_name(&self, generation: &str) -> String { + format!("buzz-agent-{}-{}", &self.pubkey_hex[..12], generation) + } + + /// Label selector matching this identity's objects *and* our management + /// marker. Selecting on the marker as well as the identity means an + /// unmarked look-alike never even enters the candidate list. + pub fn selector(&self) -> String { + format!( + "{LABEL_AGENT_PUBKEY}={},{LABEL_MANAGED_BY}={MANAGED_BY}", + self.label_pubkey() + ) + } + + /// The label set stamped on every object this provider creates. + pub fn labels(&self) -> std::collections::BTreeMap { + [ + ( + LABEL_AGENT_PUBKEY.to_string(), + self.label_pubkey().to_string(), + ), + (LABEL_MANAGED_BY.to_string(), MANAGED_BY.to_string()), + ( + LABEL_BINDING_VERSION.to_string(), + BINDING_VERSION.to_string(), + ), + ] + .into_iter() + .collect() + } +} + +/// A fresh generation token: 8 lowercase hex chars from the OS RNG. +/// +/// Appears in the Secret name and as `BUZZ_MANAGED_AGENT_START_NONCE`, so the +/// Secret generation and the harness's lifecycle-frame correlator are one +/// identity (§Launch data tier 3). +pub fn new_generation() -> String { + use rand::RngExt; + let n: u32 = rand::rng().random(); + format!("{n:08x}") +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A fixed test key. Deriving the pubkey (rather than hardcoding both + /// halves) is the point: the test exercises the same derivation the + /// reconciler depends on. + fn identity() -> AgentIdentity { + let keys = nostr::Keys::generate(); + let nsec = { + use nostr::nips::nip19::ToBech32; + keys.secret_key().to_bech32().unwrap() + }; + let id = AgentIdentity::from_nsec(&nsec).unwrap(); + assert_eq!(id.pubkey_hex(), keys.public_key().to_hex()); + id + } + + #[test] + fn rejects_malformed_nsec() { + for bad in ["", "nsec1", "not-a-key", "npub1abc"] { + assert!( + AgentIdentity::from_nsec(bad).is_err(), + "accepted malformed key {bad:?}" + ); + } + } + + #[test] + fn tolerates_surrounding_whitespace() { + let keys = nostr::Keys::generate(); + use nostr::nips::nip19::ToBech32; + let nsec = keys.secret_key().to_bech32().unwrap(); + let padded = format!(" {nsec}\n"); + assert_eq!( + AgentIdentity::from_nsec(&padded).unwrap().pubkey_hex(), + keys.public_key().to_hex() + ); + } + + /// Kubernetes label *values* cap at 63 chars; a full hex pubkey is 64, + /// one over. That one-char overflow is the whole reason the selector is + /// truncated, so it gets an explicit test. + #[test] + fn label_value_fits_kubernetes_limit() { + let id = identity(); + assert_eq!(id.pubkey_hex().len(), 64); + assert_eq!(id.label_pubkey().len(), 32); + assert!(id.label_pubkey().len() <= 63); + } + + #[test] + fn pod_name_is_deterministic_and_dns_safe() { + let id = identity(); + assert_eq!(id.pod_name(), id.pod_name()); + assert_eq!( + id.pod_name(), + format!("buzz-agent-{}", &id.pubkey_hex()[..12]) + ); + assert!(id.pod_name().len() <= 253); + assert!(id + .pod_name() + .chars() + .all(|c| c.is_ascii_lowercase() || c.is_ascii_digit() || c == '-')); + } + + /// Two attempts must never share a Secret name — that uniqueness is what + /// stops a losing contender from overwriting the winner's identity. + #[test] + fn secret_names_are_per_attempt() { + let id = identity(); + let a = id.secret_name(&new_generation()); + let b = id.secret_name(&new_generation()); + assert_ne!(a, b); + assert!(a.starts_with(&id.pod_name())); + assert!(a.len() <= 253); + } + + #[test] + fn selector_requires_the_management_marker() { + let id = identity(); + let sel = id.selector(); + assert!(sel.contains(&format!("{LABEL_AGENT_PUBKEY}={}", id.label_pubkey()))); + assert!(sel.contains(&format!("{LABEL_MANAGED_BY}={MANAGED_BY}"))); + } + + #[test] + fn every_created_object_carries_the_marker() { + let labels = identity().labels(); + assert_eq!( + labels.get(LABEL_MANAGED_BY).map(String::as_str), + Some(MANAGED_BY) + ); + assert_eq!( + labels.get(LABEL_BINDING_VERSION).map(String::as_str), + Some(BINDING_VERSION) + ); + } +} diff --git a/crates/buzz-backend-kubernetes/src/observe.rs b/crates/buzz-backend-kubernetes/src/observe.rs new file mode 100644 index 0000000000..1c6c883563 --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/observe.rs @@ -0,0 +1,592 @@ +//! Decoding API objects into verified observations (spec §Deploy State +//! Machine step 1). +//! +//! Pure: `Pod` in, [`VerifiedPod`] out. Keeping the decode here means the +//! conformance tests drive the *shipped* decoder with real API types rather +//! than a test-only stand-in, and it keeps `classify.rs` free of API types. +//! +//! Verification is the gate, not a filter: [`verify`] returns `None` for any +//! object whose full-pubkey annotation does not equal the derived pubkey or +//! that lacks the management marker, so an unverified object cannot reach +//! classification, deletion, or the returned `agent_id`. + +use crate::classify::{Fence, PullFailure, Startup, VerifiedPod}; +use crate::intent::Fingerprint; +use crate::naming::{ + AgentIdentity, ANNOTATION_CREATE_INTENT, ANNOTATION_PUBKEY_FULL, BINDING_VERSION, + LABEL_BINDING_VERSION, LABEL_MANAGED_BY, MANAGED_BY, +}; +use k8s_openapi::api::core::v1::{Pod, Secret}; + +/// Container name the provider creates; status is read from this container. +pub const CONTAINER_NAME: &str = "agent"; + +/// The startup state, or a state that cannot be settled without one more read. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum StartupObservation { + Resolved(Startup), + /// `CreateContainerConfigError` — recoverable *unless* the referenced + /// Secret is confirmed absent by a most-recent read. The kubelet's reason + /// string is a hint; the provider verifies before treating it as fatal + /// (§Deploy State Machine: "provably" means a verified absence, never a + /// reason string). + ConfigErrorPendingSecretCheck { + secret_name: String, + }, +} + +/// Does this object carry the management marker (§Pod shape)? +/// +/// Identity labels prove identity; the marker asserts protocol ownership. +/// Without it an object that merely matches our schema fails closed. +fn has_marker(labels: Option<&std::collections::BTreeMap>) -> bool { + let Some(labels) = labels else { return false }; + labels.get(LABEL_MANAGED_BY).map(String::as_str) == Some(MANAGED_BY) + && labels.get(LABEL_BINDING_VERSION).map(String::as_str) == Some(BINDING_VERSION) +} + +/// Does the full-pubkey annotation equal the derived pubkey? +/// +/// The 32-hex label is collision-*resistant*, not collision-free, which is +/// why this check is normative rather than decorative (`:1152-1166`). +fn annotation_matches( + annotations: Option<&std::collections::BTreeMap>, + identity: &AgentIdentity, +) -> bool { + annotations + .and_then(|a| a.get(ANNOTATION_PUBKEY_FULL)) + .map(|v| v == identity.pubkey_hex()) + .unwrap_or(false) +} + +/// Is this Secret ours and this identity's? The same fence GC applies before +/// deleting anything. +pub fn secret_is_ours(secret: &Secret, identity: &AgentIdentity) -> bool { + has_marker(secret.metadata.labels.as_ref()) + && annotation_matches(secret.metadata.annotations.as_ref(), identity) +} + +/// Decode a pod's startup state from its status. +/// +/// "Started" means `state.running` on our container — not pod phase. A pod can +/// sit in phase `Running` with a container that never started, and a pod being +/// gracefully deleted stays in phase `Running` for its whole grace period. +pub fn decode_startup(pod: &Pod) -> StartupObservation { + use StartupObservation::Resolved; + + let status = pod.status.as_ref(); + let phase = status.and_then(|s| s.phase.as_deref()); + + let container = status + .and_then(|s| s.container_statuses.as_ref()) + .and_then(|cs| cs.iter().find(|c| c.name == CONTAINER_NAME)); + + if let Some(state) = container.and_then(|c| c.state.as_ref()) { + if state.running.is_some() { + return Resolved(Startup::Started); + } + if state.terminated.is_some() { + return Resolved(Startup::Terminated); + } + if let Some(waiting) = state.waiting.as_ref() { + let reason = waiting.reason.as_deref().unwrap_or_default(); + let message = waiting.message.as_deref().unwrap_or_default(); + return match reason { + // Structurally invalid reference: no retry can fix it. + "InvalidImageName" => Resolved(Startup::NeverStartedProvablyBroken), + "ErrImagePull" | "ImagePullBackOff" => match classify_pull_failure(message) { + Some(failure) => Resolved(Startup::NeverStartedPullFailing(failure)), + None => Resolved(Startup::NeverStartedRecoverable), + }, + "CreateContainerConfigError" => match referenced_secret(pod) { + Some(secret_name) => { + StartupObservation::ConfigErrorPendingSecretCheck { secret_name } + } + None => Resolved(Startup::NeverStartedRecoverable), + }, + _ => Resolved(Startup::NeverStartedRecoverable), + }; + } + } + + // No container status yet (unscheduled, image pulling before the kubelet + // reports, quota-blocked). A terminal phase without container status still + // means the pod is done. + match phase { + Some("Succeeded") | Some("Failed") => Resolved(Startup::Terminated), + _ => Resolved(Startup::NeverStartedRecoverable), + } +} + +/// The Secret name this pod's `envFrom` references, if any. +pub fn referenced_secret(pod: &Pod) -> Option { + pod.spec + .as_ref()? + .containers + .iter() + .flat_map(|c| c.env_from.iter().flatten()) + .find_map(|source| source.secret_ref.as_ref().map(|r| r.name.clone())) +} + +/// Classify a pull failure from the kubelet's message. +/// +/// Reporting only — [`PullFailure`] is structurally excluded from +/// `Action::Delete`, so a wrong guess here can delay a report but can never +/// destroy anything. `None` means "no permanent cause recognized", which +/// leaves the pod on the ordinary observational path. +fn classify_pull_failure(message: &str) -> Option { + let m = message.to_ascii_lowercase(); + if m.contains("401") + || m.contains("unauthorized") + || m.contains("403") + || m.contains("denied") + || m.contains("authentication required") + { + return Some(PullFailure::Unauthorized); + } + if m.contains("manifest unknown") + || m.contains("not found") + || m.contains("manifest_unknown") + || m.contains("repository does not exist") + { + return Some(PullFailure::ManifestUnknown); + } + if m.contains("no match for platform") || m.contains("no matching manifest") { + return Some(PullFailure::ArchMismatch); + } + None +} + +/// The redacted, actionable condition text for a pull failure. +/// +/// Names the registry and the immutable reference — never credentials, and +/// never the kubelet's raw message, which can echo a registry token. +pub fn pull_failure_message(failure: PullFailure, image: &str) -> String { + let registry = image.split('/').next().unwrap_or(image); + match failure { + PullFailure::Unauthorized => format!( + "the cluster is not authorized to pull {image} from {registry}. \ + This pull retries indefinitely and will not succeed on its own: \ + grant the cluster's nodes access to that registry." + ), + PullFailure::ManifestUnknown => { + format!("{registry} has no image at {image}. Check the digest and repository.") + } + PullFailure::ArchMismatch => format!( + "{image} has no variant for the architecture of the nodes it was \ + scheduled on." + ), + } +} + +/// The latest actionable condition for a pod that has not started, redacted. +/// +/// Two sources, deliberately treated differently: +/// +/// * The container's waiting **reason** is included; its **message** is not. +/// Waiting messages are kubelet-composed and echo the thing that failed — +/// for a pull that is the registry request, which can carry credential +/// material. The reason token alone (`ImagePullBackOff`, +/// `CreateContainerConfigError`) is the diagnostic; the message adds +/// exposure, not information the user can act on. +/// * Pod-condition messages **are** included. They are scheduler- and +/// kubelet-composed from the pod's own spec and cluster capacity +/// ("0/3 nodes are available: Insufficient memory"), which is precisely the +/// actionable half and contains nothing derived from Secret data. +pub fn condition(pod: &Pod) -> Option { + let status = pod.status.as_ref()?; + + if let Some(state) = status + .container_statuses + .as_ref() + .and_then(|cs| cs.iter().find(|c| c.name == CONTAINER_NAME)) + .and_then(|c| c.state.as_ref()) + { + if let Some(waiting) = state.waiting.as_ref() { + if let Some(reason) = waiting.reason.as_deref() { + return Some(format!("the container is waiting, reason {reason}")); + } + } + // Exit code and reason only — the terminated `message` is + // process-composed output and falls under the same redaction rule as + // waiting messages. + if let Some(terminated) = state.terminated.as_ref() { + return Some(match terminated.reason.as_deref() { + Some(reason) => format!( + "the container exited with code {} ({reason})", + terminated.exit_code + ), + None => format!("the container exited with code {}", terminated.exit_code), + }); + } + } + + if let Some((type_, reason, message)) = status.conditions.as_ref().and_then(|cs| { + cs.iter().find(|c| c.status == "False").map(|c| { + ( + c.type_.clone(), + c.reason.clone().unwrap_or_default(), + c.message.clone().unwrap_or_default(), + ) + }) + }) { + let detail = [reason, message] + .into_iter() + .filter(|s| !s.is_empty()) + .collect::>() + .join(": "); + return Some(if detail.is_empty() { + format!("pod condition {type_} is false") + } else { + format!("pod condition {type_} is false: {detail}") + }); + } + + status.phase.as_deref().map(|p| format!("the pod is {p}")) +} + +/// Verify a label-selected pod and decode it, or reject it. +/// +/// `startup` is supplied by the caller because settling +/// `CreateContainerConfigError` needs a most-recent Secret read the pure layer +/// must not perform. +pub fn verify(pod: &Pod, identity: &AgentIdentity, startup: Startup) -> Option { + if !has_marker(pod.metadata.labels.as_ref()) { + return None; + } + if !annotation_matches(pod.metadata.annotations.as_ref(), identity) { + return None; + } + Some(VerifiedPod { + name: pod.metadata.name.clone()?, + fence: Fence { + uid: pod.metadata.uid.clone()?, + resource_version: pod.metadata.resource_version.clone()?, + }, + deletion_marked: pod.metadata.deletion_timestamp.is_some(), + startup, + recorded_intent: pod + .metadata + .annotations + .as_ref() + .and_then(|a| a.get(ANNOTATION_CREATE_INTENT)) + .map(|v| Fingerprint::from_annotation(v)), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use k8s_openapi::api::core::v1::{ + ContainerState, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, + ContainerStatus, PodStatus, + }; + use k8s_openapi::apimachinery::pkg::apis::meta::v1::{ObjectMeta, Time}; + use std::collections::BTreeMap; + + fn identity() -> AgentIdentity { + use nostr::nips::nip19::ToBech32; + let keys = nostr::Keys::generate(); + AgentIdentity::from_nsec(&keys.secret_key().to_bech32().unwrap()).unwrap() + } + + fn base_pod(id: &AgentIdentity) -> Pod { + Pod { + metadata: ObjectMeta { + name: Some(id.pod_name()), + uid: Some("uid-1".into()), + resource_version: Some("100".into()), + labels: Some(id.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + id.pubkey_hex().to_string(), + )] + .into_iter() + .collect::>(), + ), + ..Default::default() + }, + ..Default::default() + } + } + + fn with_container_state(mut pod: Pod, state: ContainerState) -> Pod { + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: CONTAINER_NAME.into(), + state: Some(state), + ..Default::default() + }]), + ..Default::default() + }); + pod + } + + fn waiting(reason: &str, message: &str) -> ContainerState { + ContainerState { + waiting: Some(ContainerStateWaiting { + reason: Some(reason.into()), + message: Some(message.into()), + }), + ..Default::default() + } + } + + #[test] + fn running_container_is_started() { + let id = identity(); + let pod = with_container_state( + base_pod(&id), + ContainerState { + running: Some(ContainerStateRunning::default()), + ..Default::default() + }, + ); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::Started) + ); + } + + /// Pod phase is not the criterion. A pod in phase `Running` whose + /// container never started must NOT read as started, or the reconciler + /// no-ops on a pod that will never serve. + #[test] + fn phase_running_with_waiting_container_is_not_started() { + let id = identity(); + let pod = with_container_state(base_pod(&id), waiting("ContainerCreating", "")); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::NeverStartedRecoverable) + ); + } + + #[test] + fn terminated_container_is_terminated() { + let id = identity(); + let pod = with_container_state( + base_pod(&id), + ContainerState { + terminated: Some(ContainerStateTerminated { + exit_code: 0, + ..Default::default() + }), + ..Default::default() + }, + ); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::Terminated) + ); + } + + /// A terminal phase with no container status (evicted before the kubelet + /// reported) is still terminated — otherwise the residue is never GC'd. + #[test] + fn terminal_phase_without_container_status_is_terminated() { + let id = identity(); + for phase in ["Succeeded", "Failed"] { + let mut pod = base_pod(&id); + pod.status = Some(PodStatus { + phase: Some(phase.into()), + ..Default::default() + }); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::Terminated), + "phase {phase}" + ); + } + } + + #[test] + fn invalid_image_name_is_provably_broken() { + let id = identity(); + let pod = with_container_state(base_pod(&id), waiting("InvalidImageName", "bad ref")); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::NeverStartedProvablyBroken) + ); + } + + /// Permanent pull failures are recognized from the message; anything + /// unrecognized stays on the ordinary observational path rather than + /// being guessed at. + #[test] + fn permanent_pull_failures_are_classified() { + let id = identity(); + let cases = [ + ("401 Unauthorized", PullFailure::Unauthorized), + ("pull access denied", PullFailure::Unauthorized), + ("manifest unknown", PullFailure::ManifestUnknown), + ( + "no match for platform in manifest", + PullFailure::ArchMismatch, + ), + ]; + for (message, expected) in cases { + let pod = with_container_state(base_pod(&id), waiting("ErrImagePull", message)); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::NeverStartedPullFailing(expected)), + "message {message:?}" + ); + } + + let pod = with_container_state( + base_pod(&id), + waiting("ImagePullBackOff", "dial tcp: i/o timeout"), + ); + assert_eq!( + decode_startup(&pod), + StartupObservation::Resolved(Startup::NeverStartedRecoverable), + "a transient network failure must not be reported as permanent" + ); + } + + /// The kubelet's reason string is a hint, not proof: a config error defers + /// to a most-recent Secret read before anything is called broken. + #[test] + fn config_error_defers_to_a_secret_read() { + let id = identity(); + let mut pod = + with_container_state(base_pod(&id), waiting("CreateContainerConfigError", "")); + pod.spec = Some(k8s_openapi::api::core::v1::PodSpec { + containers: vec![k8s_openapi::api::core::v1::Container { + name: CONTAINER_NAME.into(), + env_from: Some(vec![k8s_openapi::api::core::v1::EnvFromSource { + secret_ref: Some(k8s_openapi::api::core::v1::SecretEnvSource { + name: "buzz-agent-abc-gen1".into(), + optional: Some(false), + }), + ..Default::default() + }]), + ..Default::default() + }], + ..Default::default() + }); + assert_eq!( + decode_startup(&pod), + StartupObservation::ConfigErrorPendingSecretCheck { + secret_name: "buzz-agent-abc-gen1".into() + } + ); + } + + /// The auto-repair fence: an object that matches our schema but lacks the + /// marker, or carries someone else's pubkey, is never verified — so it can + /// never be no-op'd against, deleted, or returned as an `agent_id`. + #[test] + fn unmarked_or_mismatched_objects_fail_verification() { + let id = identity(); + let other = identity(); + + let mut unmarked = base_pod(&id); + unmarked.metadata.labels = Some(BTreeMap::new()); + assert!( + verify(&unmarked, &id, Startup::Started).is_none(), + "unmarked pod verified" + ); + + let mut wrong_version = base_pod(&id); + let mut labels = id.labels(); + labels.insert(LABEL_BINDING_VERSION.to_string(), "999".to_string()); + wrong_version.metadata.labels = Some(labels); + assert!(verify(&wrong_version, &id, Startup::Started).is_none()); + + let mut mismatched = base_pod(&id); + mismatched.metadata.annotations = Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + other.pubkey_hex().to_string(), + )] + .into_iter() + .collect(), + ); + assert!( + verify(&mismatched, &id, Startup::Started).is_none(), + "collision verified" + ); + + let mut missing = base_pod(&id); + missing.metadata.annotations = Some(BTreeMap::new()); + assert!(verify(&missing, &id, Startup::Started).is_none()); + + assert!( + verify(&base_pod(&id), &id, Startup::Started).is_some(), + "own pod rejected" + ); + } + + /// The fence must come from the observed object, and the deletion mark + /// must be read even though the phase says `Running`. + #[test] + fn verified_pod_carries_the_fence_and_deletion_mark() { + let id = identity(); + let mut pod = base_pod(&id); + pod.metadata.deletion_timestamp = Some(Time(chrono::Utc::now())); + let verified = verify(&pod, &id, Startup::Started).unwrap(); + assert_eq!(verified.fence.uid, "uid-1"); + assert_eq!(verified.fence.resource_version, "100"); + assert!(verified.deletion_marked); + } + + /// A pod with no recorded intent reads as `None`, which the classifier + /// groups with divergence. + #[test] + fn missing_intent_annotation_decodes_as_none() { + let id = identity(); + assert!(verify(&base_pod(&id), &id, Startup::Started) + .unwrap() + .recorded_intent + .is_none()); + } + + /// A pull-failure report must name the registry and the immutable + /// reference and nothing else — never the kubelet's raw message, which + /// can echo a registry token. + #[test] + fn pull_failure_messages_are_actionable_and_redacted() { + let image = format!("ghcr.io/block/buzz-sprig@sha256:{}", "a".repeat(64)); + for failure in [ + PullFailure::Unauthorized, + PullFailure::ManifestUnknown, + PullFailure::ArchMismatch, + ] { + let msg = pull_failure_message(failure, &image); + assert!(msg.contains("ghcr.io"), "{msg}"); + assert!(msg.contains(&image), "{msg}"); + for secret in ["Bearer", "password", "nsec1", "token"] { + assert!(!msg.contains(secret), "leaked {secret}: {msg}"); + } + } + } + + #[test] + fn secret_ownership_requires_marker_and_annotation() { + let id = identity(); + let other = identity(); + let ours = Secret { + metadata: ObjectMeta { + labels: Some(id.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + id.pubkey_hex().to_string(), + )] + .into(), + ), + ..Default::default() + }, + ..Default::default() + }; + assert!(secret_is_ours(&ours, &id)); + assert!(!secret_is_ours(&ours, &other)); + + let mut unmarked = ours.clone(); + unmarked.metadata.labels = Some(BTreeMap::new()); + assert!(!secret_is_ours(&unmarked, &id)); + } +} diff --git a/crates/buzz-backend-kubernetes/src/pod.rs b/crates/buzz-backend-kubernetes/src/pod.rs new file mode 100644 index 0000000000..725f98fc7d --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/pod.rs @@ -0,0 +1,446 @@ +//! Pod and Secret construction (spec §Pod shape, §K8s Secrets). +//! +//! The builder is pure: it turns resolved inputs into API objects and performs +//! no I/O, so every normative field is a unit assertion. + +use crate::config::{ + ProviderConfig, RESTART_POLICY, RUN_AS_GID, RUN_AS_UID, TERMINATION_GRACE_SECONDS, + WORKSPACE_PATH, +}; +use crate::intent::{Fingerprint, IntentTemplate}; +use crate::naming::{ + AgentIdentity, ANNOTATION_CREATE_INTENT, ANNOTATION_IMAGE, ANNOTATION_PUBKEY_FULL, +}; +use k8s_openapi::api::core::v1::{ + Capabilities, Container, EmptyDirVolumeSource, EnvFromSource, Pod, PodSecurityContext, PodSpec, + ResourceRequirements, SeccompProfile, Secret, SecretEnvSource, SecurityContext, Volume, + VolumeMount, +}; +use k8s_openapi::apimachinery::pkg::api::resource::Quantity; +use k8s_openapi::apimachinery::pkg::apis::meta::v1::ObjectMeta; +use std::collections::BTreeMap; + +/// Volume name for the agent's writable workspace. +const WORKSPACE_VOLUME: &str = "workspace"; + +/// The container name. Fixed: log and exec tooling addresses it by name. +const CONTAINER_NAME: &str = "agent"; + +/// Build the per-attempt Secret holding the resolved environment. +/// +/// `immutable: true` — the Secret is written once per attempt and never +/// updated, which is what lets the pod's `envFrom` reference be treated as an +/// atomic binding to this exact payload (§K8s Secrets). +pub fn build_secret( + identity: &AgentIdentity, + namespace: &str, + generation: &str, + env: BTreeMap, +) -> Secret { + Secret { + metadata: ObjectMeta { + name: Some(identity.secret_name(generation)), + namespace: Some(namespace.to_string()), + labels: Some(identity.labels()), + annotations: Some( + [( + ANNOTATION_PUBKEY_FULL.to_string(), + identity.pubkey_hex().to_string(), + )] + .into_iter() + .collect(), + ), + ..Default::default() + }, + string_data: Some(env), + immutable: Some(true), + ..Default::default() + } +} + +/// Build the pod for one create attempt. +/// +/// The `fingerprint` is computed from [`intent_template`] over a type that +/// cannot contain the generation or any Secret value, so it is stable across +/// attempts of the same configuration. +pub fn build_pod( + identity: &AgentIdentity, + cfg: &ProviderConfig, + generation: &str, + fingerprint: &Fingerprint, +) -> Pod { + let annotations: BTreeMap = [ + ( + ANNOTATION_PUBKEY_FULL.to_string(), + identity.pubkey_hex().to_string(), + ), + ( + ANNOTATION_CREATE_INTENT.to_string(), + fingerprint.as_str().to_string(), + ), + (ANNOTATION_IMAGE.to_string(), cfg.image.as_str().to_string()), + ] + .into_iter() + .collect(); + + let requests: BTreeMap = [ + ( + "cpu".to_string(), + Quantity(cfg.resources.cpu_request.clone()), + ), + ( + "memory".to_string(), + Quantity(cfg.resources.memory_request.clone()), + ), + ] + .into_iter() + .collect(); + let limits: BTreeMap = [ + ("cpu".to_string(), Quantity(cfg.resources.cpu_limit.clone())), + ( + "memory".to_string(), + Quantity(cfg.resources.memory_limit.clone()), + ), + ] + .into_iter() + .collect(); + + let container = Container { + name: CONTAINER_NAME.to_string(), + image: Some(cfg.image.as_str().to_string()), + // No `command`/`args`: the image's entrypoint execs the harness as + // PID 1 (§Entrypoint). Overriding it here would be how a provider + // accidentally puts a shell in front of the signal receiver. + env_from: Some(vec![EnvFromSource { + secret_ref: Some(SecretEnvSource { + name: identity.secret_name(generation), + optional: Some(false), + }), + ..Default::default() + }]), + resources: Some(ResourceRequirements { + requests: Some(requests), + limits: Some(limits), + ..Default::default() + }), + volume_mounts: Some(vec![VolumeMount { + name: WORKSPACE_VOLUME.to_string(), + mount_path: WORKSPACE_PATH.to_string(), + ..Default::default() + }]), + security_context: Some(SecurityContext { + allow_privilege_escalation: Some(false), + capabilities: Some(Capabilities { + drop: Some(vec!["ALL".to_string()]), + ..Default::default() + }), + // `readOnlyRootFilesystem` is deliberately unset: the sprig + // toolchain writes outside the workspace mount (§Pod shape). + ..Default::default() + }), + ..Default::default() + }; + + Pod { + metadata: ObjectMeta { + name: Some(identity.pod_name()), + namespace: Some(cfg.namespace.clone()), + labels: Some(identity.labels()), + annotations: Some(annotations), + ..Default::default() + }, + spec: Some(PodSpec { + containers: vec![container], + restart_policy: Some(RESTART_POLICY.to_string()), + termination_grace_period_seconds: Some(TERMINATION_GRACE_SECONDS), + // The agent runs prompted, untrusted code while holding an nsec; + // an ambient ServiceAccount token would be an API-stealable + // credential it never needs (§Pod shape hardening). Naming a + // service account selects a scheduling/RBAC identity and MUST NOT + // re-enable token mounting. + automount_service_account_token: Some(false), + service_account_name: cfg.service_account.clone(), + security_context: Some(PodSecurityContext { + run_as_non_root: Some(true), + run_as_user: Some(RUN_AS_UID), + run_as_group: Some(RUN_AS_GID), + fs_group: Some(RUN_AS_GID), + seccomp_profile: Some(SeccompProfile { + type_: "RuntimeDefault".to_string(), + ..Default::default() + }), + ..Default::default() + }), + volumes: Some(vec![Volume { + name: WORKSPACE_VOLUME.to_string(), + empty_dir: Some(EmptyDirVolumeSource::default()), + ..Default::default() + }]), + ..Default::default() + }), + ..Default::default() + } +} + +/// The create-intent template for this configuration (§Deploy State Machine). +pub fn intent_template( + cfg: &ProviderConfig, + env_keys: impl IntoIterator, +) -> IntentTemplate { + IntentTemplate::new( + &cfg.namespace, + &cfg.image, + &cfg.resources, + cfg.service_account.as_deref(), + env_keys, + ) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::config; + + fn identity() -> AgentIdentity { + use nostr::nips::nip19::ToBech32; + let keys = nostr::Keys::generate(); + AgentIdentity::from_nsec(&keys.secret_key().to_bech32().unwrap()).unwrap() + } + + fn provider_config() -> ProviderConfig { + config::parse(&serde_json::json!({ + "namespace": "buzz-agents-test", + "image": format!("ghcr.io/block/buzz-sprig@sha256:{}", "a".repeat(64)), + })) + .unwrap() + } + + fn pod() -> Pod { + let cfg = provider_config(); + build_pod( + &identity(), + &cfg, + "gen00001", + &intent_template(&cfg, ["BUZZ_RELAY_URL".to_string()]).fingerprint(), + ) + } + + fn spec(pod: &Pod) -> &PodSpec { + pod.spec.as_ref().unwrap() + } + + /// Every hardening default from §Pod shape, asserted individually so a + /// dropped one names itself. + #[test] + fn hardening_defaults_are_all_present() { + let pod = pod(); + let spec = spec(&pod); + assert_eq!(spec.automount_service_account_token, Some(false)); + + let sc = spec + .security_context + .as_ref() + .expect("pod security context"); + assert_eq!(sc.run_as_non_root, Some(true)); + assert_eq!(sc.run_as_user, Some(RUN_AS_UID)); + assert_ne!(sc.run_as_user, Some(0), "root UID"); + assert_eq!(sc.run_as_group, Some(RUN_AS_GID)); + assert_eq!( + sc.seccomp_profile.as_ref().map(|p| p.type_.as_str()), + Some("RuntimeDefault") + ); + + let csc = spec.containers[0] + .security_context + .as_ref() + .expect("container sc"); + assert_eq!(csc.allow_privilege_escalation, Some(false)); + assert_eq!( + csc.capabilities.as_ref().and_then(|c| c.drop.clone()), + Some(vec!["ALL".to_string()]) + ); + assert_ne!(csc.privileged, Some(true)); + } + + /// The forbidden host-namespace and hostPath escapes, asserted as absence. + #[test] + fn never_uses_host_namespaces_or_host_paths() { + let pod = pod(); + let spec = spec(&pod); + assert!(spec.host_pid.is_none() || spec.host_pid == Some(false)); + assert!(spec.host_network.is_none() || spec.host_network == Some(false)); + assert!(spec.host_ipc.is_none() || spec.host_ipc == Some(false)); + for volume in spec.volumes.as_ref().unwrap() { + assert!( + volume.host_path.is_none(), + "hostPath volume {}", + volume.name + ); + } + } + + /// `Never` only. `OnFailure` is gated on the harness exit-code contract + /// *and* a crash-loop classification row (`:1121-1139`); the config layer + /// refuses `inactivity_seconds: 0` so this arm is unreachable, and the + /// assertion keeps it that way. + #[test] + fn restart_policy_is_never() { + assert_eq!(spec(&pod()).restart_policy.as_deref(), Some("Never")); + } + + /// 60s, not Kubernetes' default 30s — which would SIGKILL the harness + /// mid-drain and leave presence stale-online (§Pod shape). + #[test] + fn declares_the_sixty_second_grace_budget() { + assert_eq!(spec(&pod()).termination_grace_period_seconds, Some(60)); + } + + /// The pod must not override the image's entrypoint: the image execs the + /// harness as PID 1, and a `command` here is how a shell ends up in front + /// of the signal receiver (§Entrypoint). + #[test] + fn does_not_override_the_image_entrypoint() { + let pod = pod(); + let container = &spec(&pod).containers[0]; + assert!(container.command.is_none(), "overrode the entrypoint"); + assert!(container.args.is_none()); + } + + #[test] + fn workspace_is_an_emptydir_mounted_at_home() { + let pod = pod(); + let spec = spec(&pod); + let volume = &spec.volumes.as_ref().unwrap()[0]; + assert!(volume.empty_dir.is_some()); + assert!(volume.persistent_volume_claim.is_none()); + let mount = &spec.containers[0].volume_mounts.as_ref().unwrap()[0]; + assert_eq!(mount.name, volume.name); + assert_eq!(mount.mount_path, WORKSPACE_PATH); + } + + #[test] + fn resources_carry_the_configured_requests_and_limits() { + let mut cfg = provider_config(); + cfg.resources.cpu_limit = "4".into(); + let pod = build_pod(&identity(), &cfg, "g", &Fingerprint::from_annotation("f")); + let r = spec(&pod).containers[0].resources.as_ref().unwrap(); + assert_eq!(r.requests.as_ref().unwrap()["cpu"], Quantity("1".into())); + assert_eq!( + r.requests.as_ref().unwrap()["memory"], + Quantity("2Gi".into()) + ); + assert_eq!(r.limits.as_ref().unwrap()["cpu"], Quantity("4".into())); + assert_eq!(r.limits.as_ref().unwrap()["memory"], Quantity("4Gi".into())); + } + + /// `envFrom` must point at this attempt's Secret and must NOT be optional: + /// an optional reference starts the container with no identity at all, + /// turning a missing-Secret bug into an agent that silently cannot + /// authenticate. + #[test] + fn env_from_references_this_attempts_secret_and_is_required() { + let id = identity(); + let cfg = provider_config(); + let pod = build_pod(&id, &cfg, "gen00042", &Fingerprint::from_annotation("f")); + let source = &spec(&pod).containers[0].env_from.as_ref().unwrap()[0]; + let secret_ref = source.secret_ref.as_ref().unwrap(); + assert_eq!(secret_ref.name, id.secret_name("gen00042")); + assert_eq!(secret_ref.optional, Some(false)); + assert!(source.config_map_ref.is_none()); + } + + /// Identity, ownership marker, and the recorded intent all travel on the + /// pod — the GC and reconciliation fences read exactly these. + #[test] + fn pod_carries_identity_marker_and_recorded_intent() { + let id = identity(); + let cfg = provider_config(); + let fp = intent_template(&cfg, ["A".to_string()]).fingerprint(); + let pod = build_pod(&id, &cfg, "g", &fp); + let meta = &pod.metadata; + assert_eq!(meta.name.as_deref(), Some(id.pod_name().as_str())); + assert_eq!(meta.namespace.as_deref(), Some("buzz-agents-test")); + assert_eq!(meta.labels.as_ref().unwrap(), &id.labels()); + let ann = meta.annotations.as_ref().unwrap(); + assert_eq!(ann[ANNOTATION_PUBKEY_FULL], id.pubkey_hex()); + assert_eq!(ann[ANNOTATION_CREATE_INTENT], fp.as_str()); + assert_eq!(ann[ANNOTATION_IMAGE], cfg.image.as_str()); + } + + /// The Secret is immutable and marker-bearing: immutability is what makes + /// the pod's `envFrom` an atomic binding, and the marker is what GC + /// requires before it will delete anything. + #[test] + fn secret_is_immutable_marked_and_holds_the_env() { + let id = identity(); + let env: BTreeMap = + [("BUZZ_RELAY_URL".to_string(), "wss://r".to_string())].into(); + let secret = build_secret(&id, "ns", "gen1", env.clone()); + assert_eq!(secret.immutable, Some(true)); + assert_eq!(secret.string_data.as_ref().unwrap(), &env); + assert_eq!( + secret.metadata.name.as_deref(), + Some(id.secret_name("gen1").as_str()) + ); + assert_eq!(secret.metadata.labels.as_ref().unwrap(), &id.labels()); + assert_eq!( + secret.metadata.annotations.as_ref().unwrap()[ANNOTATION_PUBKEY_FULL], + id.pubkey_hex() + ); + // `data` must stay unset — setting both is an apiserver rejection. + assert!(secret.data.is_none()); + } + + /// Naming a service account selects a scheduling identity; it must not + /// re-enable token mounting (§Pod shape hardening, `:1221-1225`). + #[test] + fn service_account_does_not_re_enable_token_mounting() { + let mut cfg = provider_config(); + cfg.service_account = Some("agent-sa".into()); + let pod = build_pod(&identity(), &cfg, "g", &Fingerprint::from_annotation("f")); + assert_eq!(spec(&pod).service_account_name.as_deref(), Some("agent-sa")); + assert_eq!(spec(&pod).automount_service_account_token, Some(false)); + } + + /// The fingerprint recorded on the pod is the one the classifier will + /// recompute — pinned end-to-end so a builder change that forgets to feed + /// the template a field cannot pass silently. + #[test] + fn recorded_fingerprint_matches_a_fresh_computation() { + let cfg = provider_config(); + let keys = ["BUZZ_RELAY_URL".to_string(), "GOOSE_MODE".to_string()]; + let fp = intent_template(&cfg, keys.clone()).fingerprint(); + let pod = build_pod(&identity(), &cfg, "gen-a", &fp); + let recorded = Fingerprint::from_annotation( + &pod.metadata.annotations.as_ref().unwrap()[ANNOTATION_CREATE_INTENT], + ); + assert_eq!(recorded, intent_template(&cfg, keys).fingerprint()); + } + + /// Two attempts differing only in generation must record the *same* + /// fingerprint, or the divergence discriminator fires on every deploy and + /// the never-started row deletes healthy pending pods. + #[test] + fn generation_does_not_change_the_recorded_fingerprint() { + let cfg = provider_config(); + let keys = ["BUZZ_RELAY_URL".to_string()]; + let a = intent_template(&cfg, keys.clone()).fingerprint(); + let b = intent_template(&cfg, keys).fingerprint(); + let id = identity(); + let pod_a = build_pod(&id, &cfg, "gen-1", &a); + let pod_b = build_pod(&id, &cfg, "gen-2", &b); + let read = + |p: &Pod| p.metadata.annotations.as_ref().unwrap()[ANNOTATION_CREATE_INTENT].clone(); + assert_eq!(read(&pod_a), read(&pod_b)); + // ...while the Secret they reference differs. + let secret_of = |p: &Pod| { + spec(p).containers[0].env_from.as_ref().unwrap()[0] + .secret_ref + .as_ref() + .unwrap() + .name + .clone() + }; + assert_ne!(secret_of(&pod_a), secret_of(&pod_b)); + } +} diff --git a/crates/buzz-backend-kubernetes/src/reconcile.rs b/crates/buzz-backend-kubernetes/src/reconcile.rs new file mode 100644 index 0000000000..df2f99789f --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/reconcile.rs @@ -0,0 +1,1585 @@ +//! The deploy loop: executes [`crate::classify`]'s actions against a substrate +//! and re-enters (spec §Deploy State Machine). +//! +//! The substrate is a trait so the conformance tests drive this exact +//! reconciler — the shipped code path, not a test-only reimplementation — with +//! a fake cluster and a fake clock. +//! +//! Two shapes are worth naming up front, because they are what keep the loop +//! terminating: +//! +//! * **Success means the harness container started** (`:696-699`). There is no +//! "deployed but not confirmed" success: `deploy` returns an `agent_id` or an +//! in-band error carrying the latest condition. The wire has no third form. +//! * **A create-conflict loser never repairs.** It verifies the winner, drops +//! its own Secret, and *observes* until the winner starts. Applying the +//! divergence row to the pod that just beat it is exactly the ping-pong the +//! spec forbids (`:845-850`), and the escape it names is the *next* deploy, +//! not this one. + +use crate::classify::{self, Action, Fence, Startup, VerifiedPod}; +use crate::config::ProviderConfig; +use crate::gc; +use crate::naming::AgentIdentity; +use crate::observe::{self, StartupObservation}; +use chrono::{DateTime, Utc}; +use k8s_openapi::api::core::v1::{Pod, Secret}; +use std::collections::BTreeMap; +use std::time::Duration; + +/// Outcome of a create against the deterministic pod name. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum CreateOutcome { + Created, + /// 409 with `Status.reason: AlreadyExists` — a concurrent attempt won. + /// Discriminated on the typed reason, never on the HTTP code alone: 409 is + /// also `Conflict`, which means a failed precondition (`:780-794`). + AlreadyExists, +} + +/// Outcome of a fenced delete. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum DeleteOutcome { + Accepted, + /// Already gone. Delete-not-found is success (`:842`). + NotFound, + /// 409 with `Status.reason: Conflict` — the object changed since the + /// observation that authorized this delete. Neither an error nor + /// permission to retry: re-enter and classify what exists now + /// (`:775-777`). + PreconditionFailed, +} + +/// The cluster operations the reconciler needs. Everything here is I/O; +/// everything that decides is pure and lives in `classify`/`gc`. +#[allow(async_fn_in_trait)] +pub trait Substrate { + /// Create the namespace if absent. On RBAC denial the error MUST name the + /// literal `kubectl create namespace ` command and MUST NOT fall + /// back to `default` (`:1002-1005`). + async fn ensure_namespace(&self, namespace: &str) -> Result<(), String>; + + /// Most-recent read of the pods matching this identity's selector, plus + /// the apiserver's clock from the same call's HTTP `Date` header. `None` + /// clock means the header was absent or unparseable, which makes the + /// orphan-Secret sweep skip (`:1321-1335`). + async fn list_pods(&self, selector: &str) -> Result<(Vec, Option>), String>; + + async fn list_secrets(&self, selector: &str) -> Result, String>; + + /// Most-recent existence check (`resourceVersion` explicitly unset, not + /// `"0"`): the classifier treats a confirmed absence as proof, so a + /// possibly-stale cache read would be proof of nothing (`:761-769`). + async fn secret_exists(&self, name: &str) -> Result; + + async fn create_secret(&self, secret: &Secret) -> Result<(), String>; + + async fn create_pod(&self, pod: &Pod) -> Result; + + /// Compare-and-delete against the fence from the authorizing observation. + /// Uses the object's own grace period — never `grace_period_seconds: 0`, + /// which is a force-kill that discards the declared 60s shutdown budget + /// (`:1185-1189`). + async fn delete_pod(&self, name: &str, fence: &Fence) -> Result; + + /// Best-effort: the Secret may already be gone, which is success. + async fn delete_secret(&self, name: &str) -> Result<(), String>; + + /// Read one pod by name, most-recent. `None` is a confirmed absence. + async fn get_pod(&self, name: &str) -> Result, String>; + + async fn sleep(&self, duration: Duration); + + /// Monotonic elapsed time since the operation began. Fake-clock driven in + /// tests; the deadline must not depend on wall-clock adjustments. + fn elapsed(&self) -> Duration; +} + +/// Interval between reconciler polls. Short enough that a fast start is +/// reported promptly, long enough not to hammer the apiserver for 600s. +const POLL_INTERVAL: Duration = Duration::from_secs(2); + +/// The deploy operation deadline (spec §Deploy: `timeout: 600s`). +const DEADLINE: Duration = Duration::from_secs(gc::OPERATION_DEADLINE_SECS as u64); + +/// Settle a pod's startup state, performing the one most-recent Secret read +/// that `CreateContainerConfigError` requires. +async fn settle(substrate: &impl Substrate, pod: &Pod) -> Result { + Ok(match observe::decode_startup(pod) { + StartupObservation::Resolved(startup) => startup, + StartupObservation::ConfigErrorPendingSecretCheck { secret_name } => { + // A confirmed absence is proof; anything else stays recoverable, + // because the kubelet's reason string alone is not evidence. + if substrate.secret_exists(&secret_name).await? { + Startup::NeverStartedRecoverable + } else { + Startup::NeverStartedProvablyBroken + } + } + }) +} + +/// Observe the single pod owned by this identity, verified. +/// +/// `Ok(None)` conflates "absent" with "present but not ours" on purpose *here* +/// — both mean the classifier has nothing it may act on. The create path +/// separates them, because only there is the difference actionable. +async fn observe_pod( + substrate: &impl Substrate, + identity: &AgentIdentity, +) -> Result, String> { + let Some(pod) = substrate.get_pod(&identity.pod_name()).await? else { + return Ok(None); + }; + let startup = settle(substrate, &pod).await?; + Ok(observe::verify(&pod, identity, startup)) +} + +/// The latest condition to report, read fresh at the moment of reporting. +/// +/// Reading it here rather than threading it through every loop iteration is +/// what "the *latest* redacted condition" (`:689`) asks for, and it costs a +/// read only on the paths that are already failing. +async fn latest_condition(substrate: &impl Substrate, identity: &AgentIdentity) -> String { + match substrate.get_pod(&identity.pod_name()).await { + Ok(Some(pod)) => observe::condition(&pod) + .unwrap_or_else(|| "no condition reported by the cluster".to_string()), + Ok(None) => "the pod no longer exists".to_string(), + Err(e) => format!("the pod's condition could not be read: {e}"), + } +} + +/// Preflight GC (§K8s GC). Failures are logged and swallowed: GC is hygiene, +/// and a deploy must not fail because a stale object could not be listed or +/// removed. A denial that actually blocks this deploy resurfaces at create, +/// where the message names the operation the user was denied. +async fn preflight_gc(substrate: &impl Substrate, identity: &AgentIdentity) { + if let Err(e) = try_preflight_gc(substrate, identity).await { + eprintln!("gc: preflight pass skipped: {e}"); + } +} + +async fn try_preflight_gc( + substrate: &impl Substrate, + identity: &AgentIdentity, +) -> Result<(), String> { + let selector = identity.selector(); + let (pods, server_now) = substrate.list_pods(&selector).await?; + let secrets = substrate.list_secrets(&selector).await?; + + let mut terminated: Vec = Vec::new(); + for pod in &pods { + if matches!(settle(substrate, pod).await?, Startup::Terminated) { + if let Some(name) = pod.metadata.name.clone() { + terminated.push(name); + } + } + } + + let plan = gc::plan( + identity, + &pods, + &secrets, + |pod| { + pod.metadata + .name + .as_deref() + .map(|n| terminated.iter().any(|t| t == n)) + .unwrap_or(false) + }, + server_now, + ); + + for name in &plan.pods { + // Re-read to fence the delete against the object we just observed; a + // pod that changed since the list is simply skipped this pass. + let Some(pod) = substrate.get_pod(name).await? else { + continue; + }; + let (Some(uid), Some(rv)) = ( + pod.metadata.uid.clone(), + pod.metadata.resource_version.clone(), + ) else { + continue; + }; + if !matches!(settle(substrate, &pod).await?, Startup::Terminated) { + continue; + } + let fence = Fence { + uid, + resource_version: rv, + }; + if let Err(e) = substrate.delete_pod(name, &fence).await { + eprintln!("gc: could not delete terminated pod {name}: {e}"); + } + } + for name in &plan.secrets { + if let Err(e) = substrate.delete_secret(name).await { + eprintln!("gc: could not delete secret {name}: {e}"); + } + } + Ok(()) +} + +/// Wait for a pod to actually disappear. +/// +/// Mandatory before recreating: `DELETE` returns success while the object +/// still exists, and the deterministic name stays taken for the whole grace +/// period (`:1177-1195`). +async fn await_disappearance(substrate: &impl Substrate, name: &str) -> Result<(), String> { + while substrate.elapsed() < DEADLINE { + if substrate.get_pod(name).await?.is_none() { + return Ok(()); + } + substrate.sleep(POLL_INTERVAL).await; + } + Err(format!( + "timed out after {}s waiting for {name} to finish terminating", + DEADLINE.as_secs() + )) +} + +/// Is `secret` referenced by any pod that currently exists under this +/// identity's selector? +/// +/// Protection deliberately spans *all* our pods, not just the winner: an +/// `envFrom` reference from a pod still pulling its image is exactly as +/// load-bearing as one from a running pod (`:1261-1264`). +async fn secret_is_referenced( + substrate: &impl Substrate, + identity: &AgentIdentity, + secret: &str, +) -> Result { + let (pods, _) = substrate.list_pods(&identity.selector()).await?; + Ok(pods + .iter() + .filter_map(observe::referenced_secret) + .any(|name| name == secret)) +} + +/// Drop this attempt's own Secret once nothing references it. +/// +/// Only ever called with a name this process generated, and gated on the +/// reference check: "never the winner's, never any Secret referenced by an +/// existing pod" (`:1259-1264`). Failure is logged, not fatal — a leaked +/// Secret is collected by the age-gated sweep. +async fn drop_own_secret(substrate: &impl Substrate, identity: &AgentIdentity, secret: &str) { + match secret_is_referenced(substrate, identity, secret).await { + Ok(false) => { + if let Err(e) = substrate.delete_secret(secret).await { + eprintln!("could not clean up own unreferenced secret {secret}: {e}"); + } + } + Ok(true) => {} + Err(e) => eprintln!("could not check whether {secret} is still referenced: {e}"), + } +} + +/// Lost the create race: adopt the elected winner. +/// +/// Observe-only by construction — this function has no delete edge for the +/// pod. A winner that is terminated or provably broken is reported, not +/// repaired; the spec's escape is "a *subsequent* deploy that walks in and +/// observes that never-started divergent winner replaces it normally" +/// (`:849-850`). +async fn adopt_winner( + substrate: &impl Substrate, + identity: &AgentIdentity, + own_secret: &str, +) -> Result { + let name = identity.pod_name(); + + // Verify before adopting. A pod under our deterministic name that fails + // the marker/annotation check is not ours to adopt, wait for, or touch — + // and it will never become ours, so this is terminal rather than a retry. + match substrate.get_pod(&name).await? { + None => {} + Some(pod) => { + let startup = settle(substrate, &pod).await?; + if observe::verify(&pod, identity, startup).is_none() { + drop_own_secret(substrate, identity, own_secret).await; + return Err(format!( + "a pod named {name} already exists in this namespace but is not \ + managed by this provider for this agent (it lacks the management \ + marker or carries a different agent identity). Remove it, or \ + deploy this agent to a different namespace." + )); + } + } + } + + drop_own_secret(substrate, identity, own_secret).await; + + // Then wait for the winner exactly as we would wait for our own pod: + // success still means the harness container started. + loop { + if substrate.elapsed() >= DEADLINE { + return Err(format!( + "startup not confirmed within {}s for {name} (another deploy of this \ + agent created it): {}", + DEADLINE.as_secs(), + latest_condition(substrate, identity).await + )); + } + match observe_pod(substrate, identity).await? { + Some(pod) if matches!(pod.startup, Startup::Started) => return Ok(pod.name), + Some(pod) if pod.deletion_marked => { + return Err(format!( + "{name} was created by another deploy of this agent and is already \ + being deleted; try again" + )) + } + Some(pod) + if matches!( + pod.startup, + Startup::Terminated | Startup::NeverStartedProvablyBroken + ) => + { + return Err(format!( + "{name} was created by another deploy of this agent and did not \ + start: {}", + latest_condition(substrate, identity).await + )) + } + // Gone again, or still coming up: keep observing under this + // operation's deadline. + _ => substrate.sleep(POLL_INTERVAL).await, + } + } +} + +/// Run the deploy state machine to a terminal outcome: the started pod's name, +/// or an in-band error carrying the latest condition. +pub async fn deploy( + substrate: &impl Substrate, + identity: &AgentIdentity, + cfg: &ProviderConfig, + env: BTreeMap, +) -> Result { + substrate.ensure_namespace(&cfg.namespace).await?; + preflight_gc(substrate, identity).await; + + let desired = crate::pod::intent_template(cfg, env.keys().cloned()).fingerprint(); + + // Has THIS call created a pod? Set once its create lands. The replacement + // rows below are for residue from a previous life; once this call has made + // its own attempt, a replace-classification means that attempt failed — + // and startup verification is part of create, so the failure is reported + // in-band rather than retried. Without this bound a deterministic startup + // failure (the harness starts, rejects its configuration, exits) is + // delete-recreated every poll for the whole deadline, minting an immutable + // Secret per cycle — measured live at 107 Secrets in one 600s call, every + // one younger than the orphan sweep's age gate. + let mut created_this_call = false; + + loop { + if substrate.elapsed() >= DEADLINE { + return Err(format!( + "startup not confirmed within {}s for {}: {}", + DEADLINE.as_secs(), + identity.pod_name(), + latest_condition(substrate, identity).await + )); + } + + let observed = observe_pod(substrate, identity).await?; + match classify::classify(observed.as_ref(), &desired) { + // The only success edge: the harness container is running. + Action::NoOp { agent_id } => return Ok(agent_id), + + // Self-healing states. Never delete, on this call or any later one + // — what replaces a never-started pod is a config change, never a + // deadline (`:717-729`). + Action::Observe { .. } => substrate.sleep(POLL_INTERVAL).await, + + // A pull that will not self-heal: report now rather than spend the + // remaining deadline on it. Still no delete authority. + Action::Report { name, failure } => { + return Err(format!( + "{name} did not start: {}", + observe::pull_failure_message(failure, cfg.image.as_str()) + )) + } + + Action::AwaitDisappearance { name } => await_disappearance(substrate, &name).await?, + + Action::Delete { name, fence } => { + // This call already made its own attempt, and that attempt is + // what the classification wants replaced: it terminated (the + // deterministic startup failure — the harness starts, rejects + // its configuration, exits) or was proven broken. Replacing it + // here retries the identical configuration against the same + // cluster: a hot delete/mint/create cycle every poll for the + // whole deadline, an immutable Secret per cycle — measured + // live at 107 Secrets in one 600s call, all younger than the + // orphan sweep's age gate. Report in-band instead. The residue + // is deliberate: the next Start's preflight GC collects the + // terminated pod and its referenced Secret together, so retry + // is gated on fresh owner intent and litter stays bounded at + // one pod + one Secret per press. + if created_this_call { + return Err(format!( + "{name} was created by this deploy and did not stay \ + running: {}. Not retrying in this call — an immediate \ + exit recurs until its cause is fixed. Check the \ + agent's configuration and press Start to try again.", + latest_condition(substrate, identity).await + )); + } + match substrate.delete_pod(&name, &fence).await? { + // Accepted or already gone: both need the disappearance + // poll before the name is free again. + DeleteOutcome::Accepted | DeleteOutcome::NotFound => { + await_disappearance(substrate, &name).await? + } + // The object changed since the observation that authorized + // this delete. Discard the action and re-classify — never + // retry with a fresher fence, which would delete something + // we never examined. Sleep before re-entering: the losing + // race is against another writer, and re-reading at full + // speed is a busy-retry with no better odds than a paced + // one. + DeleteOutcome::PreconditionFailed => substrate.sleep(POLL_INTERVAL).await, + } + } + + Action::Create => { + let generation = crate::naming::new_generation(); + let secret_name = identity.secret_name(&generation); + + // The generation is minted *per attempt*, and it is two things + // at once: the Secret's name suffix and the lifecycle + // correlator the harness reports. `build_env` stamped the + // caller's generation, so on any attempt after the first the + // two would name different generations — pod logs correlating + // to a Secret that is not the one mounted. Restamp so there is + // exactly one generation per attempt (§K8s Secrets). + let mut env = env.clone(); + env.insert(crate::env::START_NONCE_KEY.to_string(), generation.clone()); + + // Secret first: the pod's spec references this exact name, so + // payload and Secret are atomic at the pod-spec boundary. + let secret = crate::pod::build_secret(identity, &cfg.namespace, &generation, env); + substrate.create_secret(&secret).await?; + + let pod = crate::pod::build_pod(identity, cfg, &generation, &desired); + match substrate.create_pod(&pod).await? { + // Re-enter rather than wait inline: the next iteration + // observes what we just created and runs the same rows + // every other state runs through. One loop, one table. + // + // Sleep first. A just-created pod cannot already be + // started, so the immediate observation has no outcome but + // "still coming up" — and if it ever came back + // unverifiable, re-entering without advancing the clock + // would hot-spin creates against the apiserver for the + // whole deadline. + CreateOutcome::Created => { + created_this_call = true; + substrate.sleep(POLL_INTERVAL).await + } + CreateOutcome::AlreadyExists => { + return adopt_winner(substrate, identity, &secret_name).await + } + } + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::config::Resources; + use crate::naming::{ANNOTATION_CREATE_INTENT, ANNOTATION_PUBKEY_FULL, LABEL_MANAGED_BY}; + use k8s_openapi::api::core::v1::{ + ContainerState, ContainerStateRunning, ContainerStateTerminated, ContainerStateWaiting, + ContainerStatus, PodStatus, + }; + use k8s_openapi::apimachinery::pkg::apis::meta::v1::Time; + use std::cell::RefCell; + use std::future::Future; + use std::pin::pin; + use std::task::{Context, Poll, RawWaker, RawWakerVTable, Waker}; + + /// Mutates the pod map after a poll, so a test can script a pod that + /// starts (or vanishes) partway through an observation loop. + type PollHook = Box)>; + + /// A scripted cluster. Single-threaded on purpose: the reconciler is one + /// process per operation, and `RefCell` keeps the assertions readable. + /// + /// This drives the *shipped* `deploy` — the point of the `Substrate` seam. + /// Every fake here answers with real `k8s_openapi` objects, so the decode + /// and verification layers under test are the ones that run in a cluster. + #[derive(Default)] + struct Fake { + pods: RefCell>, + secrets: RefCell>, + /// Server clock for the GC age gate; `None` models a missing `Date`. + server_now: Option>, + /// Elapsed time, advanced only by `sleep` — a fake clock, so a 600s + /// deadline test runs instantly and cannot flake on a slow machine. + elapsed: RefCell, + /// Queued create outcomes; the default is `Created`. + create_outcomes: RefCell>, + /// Installed when a create loses the race. A winner must be *absent* + /// at the observation that decides to create and *present* by the time + /// the create lands — pre-seeding it instead makes the loop no-op + /// before it ever reaches the create edge. + winner: RefCell>, + /// Every Secret ever created, retained across deletion. + created_secrets: RefCell>, + /// Queued delete outcomes; the default is `Accepted`. + delete_outcomes: RefCell>, + /// Every mutating call, in order — the anti-mutation assertions read + /// this rather than guessing from final state. + calls: RefCell>, + /// Applied to the pod map after each poll, so a test can script a pod + /// that starts (or vanishes) partway through an observation loop. + on_poll: RefCell>, + /// `ensure_namespace` fails with this, if set. + namespace_error: Option, + } + + impl Fake { + fn with_pod(self, pod: Pod) -> Self { + self.pods + .borrow_mut() + .insert(pod.metadata.name.clone().unwrap(), pod); + self + } + fn log(&self, entry: impl Into) { + self.calls.borrow_mut().push(entry.into()); + } + fn mutations(&self) -> Vec { + self.calls.borrow().clone() + } + } + + impl Substrate for Fake { + async fn ensure_namespace(&self, namespace: &str) -> Result<(), String> { + match &self.namespace_error { + Some(e) => Err(e.clone()), + None => { + self.log(format!("ensure_namespace {namespace}")); + Ok(()) + } + } + } + + async fn list_pods( + &self, + _selector: &str, + ) -> Result<(Vec, Option>), String> { + Ok(( + self.pods.borrow().values().cloned().collect(), + self.server_now, + )) + } + + async fn list_secrets(&self, _selector: &str) -> Result, String> { + Ok(self.secrets.borrow().clone()) + } + + async fn secret_exists(&self, name: &str) -> Result { + Ok(self + .secrets + .borrow() + .iter() + .any(|s| s.metadata.name.as_deref() == Some(name))) + } + + async fn create_secret(&self, secret: &Secret) -> Result<(), String> { + let name = secret.metadata.name.clone().unwrap(); + self.log(format!("create_secret {name}")); + self.secrets.borrow_mut().push(secret.clone()); + // Kept even after the Secret is deleted: assertions about what an + // attempt *wrote* must not be silently vacuous once cleanup runs. + self.created_secrets.borrow_mut().push(secret.clone()); + Ok(()) + } + + async fn create_pod(&self, pod: &Pod) -> Result { + let name = pod.metadata.name.clone().unwrap(); + self.log(format!("create_pod {name}")); + let outcome = if self.create_outcomes.borrow().is_empty() { + CreateOutcome::Created + } else { + self.create_outcomes.borrow_mut().remove(0) + }; + if outcome == CreateOutcome::Created { + // The apiserver stamps these on admission; a builder never + // carries them. Without them the pod fails `verify`'s fence + // extraction and the loop can never see what it just created. + let mut pod = pod.clone(); + pod.metadata.uid = Some(format!("uid-created-{}", self.calls.borrow().len())); + pod.metadata.resource_version = Some("1".into()); + self.pods.borrow_mut().insert(name, pod); + } else if let Some(winner) = self.winner.borrow_mut().take() { + // The concurrent attempt's pod becomes visible exactly when our + // create is rejected — the ordering a real race produces. + self.pods.borrow_mut().insert(name, winner); + } + Ok(outcome) + } + + async fn delete_pod(&self, name: &str, fence: &Fence) -> Result { + self.log(format!( + "delete_pod {name} uid={} rv={}", + fence.uid, fence.resource_version + )); + let outcome = if self.delete_outcomes.borrow().is_empty() { + DeleteOutcome::Accepted + } else { + self.delete_outcomes.borrow_mut().remove(0) + }; + if outcome == DeleteOutcome::Accepted { + self.pods.borrow_mut().remove(name); + } + Ok(outcome) + } + + async fn delete_secret(&self, name: &str) -> Result<(), String> { + self.log(format!("delete_secret {name}")); + self.secrets + .borrow_mut() + .retain(|s| s.metadata.name.as_deref() != Some(name)); + Ok(()) + } + + async fn get_pod(&self, name: &str) -> Result, String> { + Ok(self.pods.borrow().get(name).cloned()) + } + + async fn sleep(&self, duration: Duration) { + *self.elapsed.borrow_mut() += duration; + let hooks = self.on_poll.borrow(); + let mut pods = self.pods.borrow_mut(); + for hook in hooks.iter() { + hook(&mut pods); + } + } + + fn elapsed(&self) -> Duration { + *self.elapsed.borrow() + } + } + + fn identity() -> AgentIdentity { + use nostr::nips::nip19::ToBech32; + let keys = nostr::Keys::generate(); + AgentIdentity::from_nsec(&keys.secret_key().to_bech32().unwrap()).unwrap() + } + + fn config() -> ProviderConfig { + ProviderConfig { + context: None, + namespace: "buzz-agents-test".into(), + image: crate::image::parse(&format!( + "ghcr.io/block/buzz-sprig@sha256:{}", + "a".repeat(64) + )) + .unwrap(), + resources: Resources::default(), + inactivity_seconds: Some(7200), + service_account: None, + } + } + + fn env() -> BTreeMap { + [("BUZZ_RELAY_URL".to_string(), "wss://r".to_string())] + .into_iter() + .collect() + } + + /// A pod exactly as this provider would have created it — same builder the + /// reconciler uses, so verification is exercised rather than bypassed. + fn our_pod(id: &AgentIdentity, cfg: &ProviderConfig, state: Option) -> Pod { + let fp = crate::pod::intent_template(cfg, env().keys().cloned()).fingerprint(); + let mut pod = crate::pod::build_pod(id, cfg, "gen-existing", &fp); + pod.metadata.uid = Some("uid-1".into()); + pod.metadata.resource_version = Some("100".into()); + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: state.map(|s| { + vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(s), + ..Default::default() + }] + }), + ..Default::default() + }); + pod + } + + fn running() -> ContainerState { + ContainerState { + running: Some(ContainerStateRunning::default()), + ..Default::default() + } + } + + fn terminated() -> ContainerState { + ContainerState { + terminated: Some(ContainerStateTerminated::default()), + ..Default::default() + } + } + + fn waiting(reason: &str) -> ContainerState { + ContainerState { + waiting: Some(ContainerStateWaiting { + reason: Some(reason.into()), + message: None, + }), + ..Default::default() + } + } + + fn run(fake: &Fake, id: &AgentIdentity, cfg: &ProviderConfig) -> Result { + block_on(deploy(fake, id, cfg, env())) + } + + /// Minimal executor: the fake never yields to a reactor (no timers, no + /// I/O — `sleep` just advances a counter), so polling to completion is + /// sufficient and avoids pulling a runtime into the unit job. + fn block_on(fut: impl Future) -> T { + fn noop_waker() -> Waker { + fn nop(_: *const ()) {} + fn clone(_: *const ()) -> RawWaker { + RawWaker::new(std::ptr::null(), &VTABLE) + } + static VTABLE: RawWakerVTable = RawWakerVTable::new(clone, nop, nop, nop); + unsafe { Waker::from_raw(RawWaker::new(std::ptr::null(), &VTABLE)) } + } + + let waker = noop_waker(); + let mut cx = Context::from_waker(&waker); + let mut fut = pin!(fut); + match fut.as_mut().poll(&mut cx) { + Poll::Ready(v) => v, + Poll::Pending => panic!("fake substrate future parked — it has no reactor"), + } + } + + // ---- the state machine's rows, end to end ------------------------------- + + /// First deploy: Secret before pod, and the returned id is the pod name. + #[test] + fn creates_secret_then_pod_and_returns_the_pod_name() { + let id = identity(); + let cfg = config(); + let fake = Fake::default(); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + if let Some(pod) = pods.get_mut(&name) { + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]), + ..Default::default() + }); + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + + let calls = fake.mutations(); + let secret_at = calls + .iter() + .position(|c| c.starts_with("create_secret")) + .unwrap(); + let pod_at = calls + .iter() + .position(|c| c.starts_with("create_pod")) + .unwrap(); + assert!( + secret_at < pod_at, + "pod created before its Secret: {calls:?}" + ); + } + + /// The strict no-op row: a started pod returns its id having mutated + /// nothing at all. Asserted on the *call log*, not on final state — a + /// delete-then-recreate would leave identical final state. + #[test] + fn started_pod_is_a_zero_mutation_no_op() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(running()))); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert_eq!( + fake.mutations(), + [format!("ensure_namespace {}", cfg.namespace)], + "the no-op row mutated something" + ); + } + + /// ...including when the desired intent has diverged. Edits reach a + /// started pod only via the next generation (`:861-865`). + #[test] + fn started_pod_no_ops_under_divergent_intent() { + let id = identity(); + let cfg = config(); + let mut pod = our_pod(&id, &cfg, Some(running())); + pod.metadata.annotations.as_mut().unwrap().insert( + ANNOTATION_CREATE_INTENT.to_string(), + "stale-fingerprint".into(), + ); + let fake = Fake::default().with_pod(pod); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "divergence deleted a started pod" + ); + } + + /// Terminated → fenced delete → disappearance → recreate. The normal + /// restart path, and the fence must carry the observed uid/rv. + #[test] + fn terminated_pod_is_replaced_with_a_fenced_delete() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(terminated()))); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + if let Some(pod) = pods.get_mut(&name) { + if pod + .status + .as_ref() + .and_then(|s| s.container_statuses.as_ref()) + .is_none() + { + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]), + ..Default::default() + }); + } + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + let calls = fake.mutations(); + assert!( + calls + .iter() + .any(|c| c == &format!("delete_pod {} uid=uid-1 rv=100", id.pod_name())), + "delete was not fenced to the observed uid+resourceVersion: {calls:?}" + ); + assert!(calls.iter().any(|c| c.starts_with("create_pod"))); + } + + /// A failed precondition is neither an error nor permission to retry the + /// delete: re-enter and classify what exists now (`:775-777`). Here the + /// object has become a *started* pod, so the correct outcome is the no-op + /// row — never a second delete with a fresher fence. + #[test] + fn precondition_failure_reclassifies_instead_of_retrying() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(terminated()))); + fake.delete_outcomes + .borrow_mut() + .push(DeleteOutcome::PreconditionFailed); + // The writer we lost the race to: between our observation and our + // delete, the pod under this name became a *started* one with a new + // resourceVersion. Installed on the poll that follows the failed + // precondition, so the sequence is observe → delete → lose → re-observe. + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + let started = { + let mut p = our_pod(&id, &cfg, Some(running())); + p.metadata.resource_version = Some("200".into()); + p + }; + Box::new(move |pods: &mut BTreeMap| { + pods.insert(name.clone(), started.clone()); + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + let deletes: Vec<_> = fake + .mutations() + .into_iter() + .filter(|c| c.starts_with("delete_pod")) + .collect(); + // Two call sites legitimately target a terminated pod: preflight GC's + // sweep (which is the one that loses the precondition here) and the + // state machine's terminated row. Both are fenced on their own + // observation. What must never happen is a *third* — a retry of the + // failed delete — so the invariant is the fence, not the count: every + // delete carries rv=100, the version we observed. A retry would carry + // the racing writer's rv=200. + assert_eq!(deletes.len(), 2, "unexpected delete traffic: {deletes:?}"); + assert!( + deletes.iter().all(|d| d.contains("rv=100")), + "retried with a fresher fence: {deletes:?}" + ); + } + + /// The anti-livelock rule, at the loop level: a recoverable pod whose + /// intent matches is observed until the deadline and **never** deleted — + /// the case that would otherwise reset the pod age Cluster Autoscaler + /// keys on (`:689`). + #[test] + fn recoverable_pod_with_matching_intent_is_never_deleted() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(waiting("Unschedulable")))); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("startup not confirmed"), "got: {err}"); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "deleted a recoverable pod: {:?}", + fake.mutations() + ); + assert!(fake.elapsed() >= DEADLINE, "gave up before the deadline"); + } + + /// The same pod, provisioned late: the observation loop must *succeed* + /// when the autoscaler eventually lands the node, not merely avoid + /// deleting. + #[test] + fn recoverable_pod_that_starts_late_succeeds() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(waiting("Unschedulable")))); + let polls = RefCell::new(0); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + *polls.borrow_mut() += 1; + if *polls.borrow() >= 5 { + if let Some(pod) = pods.get_mut(&name) { + pod.status.as_mut().unwrap().container_statuses = + Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]); + } + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!(fake.elapsed() < DEADLINE); + } + + /// A permanent pull failure reports immediately rather than burning the + /// deadline — and still never deletes. + #[test] + fn permanent_pull_failure_reports_without_deleting() { + let id = identity(); + let cfg = config(); + let mut pod = our_pod(&id, &cfg, None); + pod.status = Some(PodStatus { + phase: Some("Pending".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(ContainerState { + waiting: Some(ContainerStateWaiting { + reason: Some("ImagePullBackOff".into()), + message: Some("manifest unknown".into()), + }), + ..Default::default() + }), + ..Default::default() + }]), + ..Default::default() + }); + let fake = Fake::default().with_pod(pod); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("no image at"), "got: {err}"); + assert!( + fake.elapsed() < DEADLINE, + "burned the deadline on a permanent failure" + ); + assert!(!fake.mutations().iter().any(|c| c.starts_with("delete_pod"))); + } + + /// The kubelet's pull *message* can echo a registry request; only the + /// classified outcome and the image reference reach the user. + #[test] + fn pull_failure_message_is_not_echoed_verbatim() { + let id = identity(); + let cfg = config(); + let mut pod = our_pod(&id, &cfg, None); + pod.status = Some(PodStatus { + phase: Some("Pending".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(ContainerState { + waiting: Some(ContainerStateWaiting { + reason: Some("ErrImagePull".into()), + message: Some( + "unauthorized: authentication required, token=SUPERSECRET".into(), + ), + }), + ..Default::default() + }), + ..Default::default() + }]), + ..Default::default() + }); + let fake = Fake::default().with_pod(pod); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!( + !err.contains("SUPERSECRET"), + "kubelet message echoed: {err}" + ); + } + + /// A pod being gracefully deleted stays in phase `Running` for its whole + /// grace period. The reconciler must wait it out and recreate, not return + /// an id that evaporates. + #[test] + fn deletion_marked_pod_is_awaited_then_recreated() { + let id = identity(); + let cfg = config(); + let mut dying = our_pod(&id, &cfg, Some(running())); + dying.metadata.deletion_timestamp = Some(Time(Utc::now())); + let fake = Fake::default().with_pod(dying); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + match pods + .get(&name) + .map(|p| p.metadata.deletion_timestamp.is_some()) + { + // The grace period elapses: the object disappears. + Some(true) => { + pods.remove(&name); + } + // The replacement we create then starts. + Some(false) => { + let pod = pods.get_mut(&name).unwrap(); + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]), + ..Default::default() + }); + } + None => {} + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "deleted a pod that was already terminating" + ); + } + + /// `CreateContainerConfigError` is settled by a most-recent Secret read, + /// never by the reason string: Secret present → recoverable (observe). + #[test] + fn config_error_with_a_present_secret_is_recoverable() { + let id = identity(); + let cfg = config(); + let pod = our_pod(&id, &cfg, Some(waiting("CreateContainerConfigError"))); + let referenced = crate::observe::referenced_secret(&pod).unwrap(); + let fake = Fake::default().with_pod(pod); + fake.secrets.borrow_mut().push(crate::pod::build_secret( + &id, + &cfg.namespace, + "gen-existing", + env(), + )); + assert_eq!(referenced, id.secret_name("gen-existing")); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("startup not confirmed"), "got: {err}"); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "deleted a pod whose Secret exists" + ); + } + + /// ...and Secret confirmed absent → provably broken → fenced replace. + #[test] + fn config_error_with_a_confirmed_absent_secret_is_replaced() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod( + &id, + &cfg, + Some(waiting("CreateContainerConfigError")), + )); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + if let Some(pod) = pods.get_mut(&name) { + if pod + .status + .as_ref() + .and_then(|s| s.container_statuses.as_ref()) + .is_none() + { + pod.status = Some(PodStatus { + phase: Some("Running".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]), + ..Default::default() + }); + } + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!(fake.mutations().iter().any(|c| c.starts_with("delete_pod"))); + } + + /// The generation is the Secret's name suffix *and* the lifecycle + /// correlator inside it. They must name the same generation, or pod logs + /// point at a Secret that was never mounted. The caller stamps one + /// generation into the env once; each create attempt mints its own, so + /// only a restamp keeps the pair together across a retry. + #[test] + fn each_attempt_stamps_its_own_generation_into_its_own_secret() { + let id = identity(); + let cfg = config(); + // First attempt loses the create race and is cleaned up; the operation + // then adopts. Two creates, two generations. + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(our_pod(&id, &cfg, Some(running()))); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + run(&fake, &id, &cfg).unwrap(); + + let created = fake.created_secrets.borrow(); + // Guard the guard: this assertion is over a list that cleanup empties, + // so an empty list would make every check below vacuously true. + assert_eq!(created.len(), 1, "expected one create attempt"); + for secret in created.iter() { + let name = secret.metadata.name.as_deref().unwrap(); + let nonce = secret + .string_data + .as_ref() + .and_then(|d| d.get(crate::env::START_NONCE_KEY)) + .expect("no lifecycle correlator in the Secret"); + assert!( + name.ends_with(nonce.as_str()), + "secret {name} carries a correlator for a different generation ({nonce})" + ); + } + } + + // ---- bounded replacement ------------------------------------------------ + + /// Installs a hook that makes every pod under `name` crash-exit before the + /// next poll — a deterministic startup failure (the harness starts, rejects + /// its configuration, and exits) as observed live: the container reaches + /// `state.terminated` with a nonzero exit code. + fn crash_exits_immediately(fake: &Fake, name: String) { + fake.on_poll + .borrow_mut() + .push(Box::new(move |pods: &mut BTreeMap| { + if let Some(pod) = pods.get_mut(&name) { + if pod + .status + .as_ref() + .and_then(|s| s.container_statuses.as_ref()) + .is_none() + { + pod.status = Some(PodStatus { + phase: Some("Failed".into()), + container_statuses: Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(ContainerState { + terminated: Some(ContainerStateTerminated { + exit_code: 1, + reason: Some("Error".into()), + ..Default::default() + }), + ..Default::default() + }), + ..Default::default() + }]), + ..Default::default() + }); + } + } + })); + } + + /// A pod that this call created and that crash-exits deterministically is + /// reported in-band after exactly ONE attempt — never hot-replaced until + /// the deadline. The live failure this pins: one delete/create cycle every + /// ~4s minted 107 immutable Secrets in a single 600s deploy call, all + /// younger than the orphan sweep's age gate. + #[test] + fn a_deterministic_crash_exit_is_reported_not_hot_replaced() { + let id = identity(); + let cfg = config(); + let fake = Fake::default(); + crash_exits_immediately(&fake, id.pod_name()); + + let err = run(&fake, &id, &cfg).unwrap_err(); + + let creates = fake + .mutations() + .iter() + .filter(|c| c.starts_with("create_secret")) + .count(); + assert_eq!( + creates, 1, + "hot replacement loop: {creates} Secrets minted in one deploy call" + ); + assert!( + err.contains("exited with code 1"), + "error does not carry the exit: {err}" + ); + // The failed attempt is left in place as evidence — no cleanup on the + // error path. Its Secret stays referenced by the terminated pod, so + // the next deploy's preflight GC collects both together. + assert!( + !fake + .mutations() + .iter() + .any(|c| c.starts_with("delete_pod") || c.starts_with("delete_secret")), + "the failed attempt was cleaned up on the error path: {:?}", + fake.mutations() + ); + assert!( + fake.elapsed() < DEADLINE, + "burned the whole deadline on a deterministic failure" + ); + } + + /// The revive path is untouched: terminated residue from a *previous* + /// life is still replaced — the bound is on pods this call created, not + /// on the row. + #[test] + fn pre_existing_terminated_residue_is_still_replaced_once() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(terminated()))); + crash_exits_immediately(&fake, id.pod_name()); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("exited with code 1"), "got: {err}"); + + let calls = fake.mutations(); + let deletes = calls.iter().filter(|c| c.starts_with("delete_pod")).count(); + let creates = calls + .iter() + .filter(|c| c.starts_with("create_secret")) + .count(); + // Exactly one residue delete (the normal restart path) and one fresh + // attempt — then report, not another cycle. + assert_eq!(deletes, 1, "unexpected delete traffic: {calls:?}"); + assert_eq!(creates, 1, "unexpected create traffic: {calls:?}"); + } + + /// Retry is gated on fresh owner intent: the *next* deploy call clears + /// the crashed attempt (preflight GC collects the terminated pod and its + /// referenced Secret together) and makes exactly one new attempt — total + /// litter stays one pod + one Secret however many times Start is pressed. + #[test] + fn the_next_deploy_collects_the_crashed_attempt_before_its_own_attempt() { + let id = identity(); + let cfg = config(); + let fake = Fake::default(); + crash_exits_immediately(&fake, id.pod_name()); + + run(&fake, &id, &cfg).unwrap_err(); + // Second Start: fresh call, fresh clock. + *fake.elapsed.borrow_mut() = Duration::ZERO; + run(&fake, &id, &cfg).unwrap_err(); + + assert_eq!( + fake.secrets.borrow().len(), + 1, + "crashed attempts accumulated Secrets across calls" + ); + assert_eq!(fake.pods.borrow().len(), 1, "crashed pods accumulated"); + } + + // ---- the auto-repair fence ---------------------------------------------- + + /// An object under our deterministic name that lacks the management + /// marker is not ours. The reconciler must not adopt it, delete it, or + /// return its name — it fails closed to the operator (`:1156-1162`). + #[test] + fn an_unmarked_look_alike_is_never_touched_or_adopted() { + let id = identity(); + let cfg = config(); + let mut look_alike = our_pod(&id, &cfg, Some(running())); + look_alike + .metadata + .labels + .as_mut() + .unwrap() + .remove(LABEL_MANAGED_BY); + let fake = Fake::default(); + // Our create loses to the object already sitting on the name. + *fake.winner.borrow_mut() = Some(look_alike); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("not managed by this provider"), "got: {err}"); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "deleted an object we do not own" + ); + } + + /// Same fence, other direction: the marker is present but the annotation + /// carries a different agent's pubkey. The 32-hex label is + /// collision-resistant, not collision-free (`:1152-1155`). + #[test] + fn a_pubkey_mismatch_is_never_adopted() { + let id = identity(); + let other = identity(); + let cfg = config(); + let mut foreign = our_pod(&id, &cfg, Some(running())); + foreign.metadata.annotations.as_mut().unwrap().insert( + ANNOTATION_PUBKEY_FULL.to_string(), + other.pubkey_hex().to_string(), + ); + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(foreign); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("different agent identity"), "got: {err}"); + assert!(!fake.mutations().iter().any(|c| c.starts_with("delete_pod"))); + } + + // ---- create-conflict convergence --------------------------------------- + + /// The loser adopts the winner, returns the winner's id, and deletes + /// **only its own** Secret — never the winner's (`:1259-1264`). + #[test] + fn create_loser_adopts_the_winner_and_drops_only_its_own_secret() { + let id = identity(); + let cfg = config(); + let winner = our_pod(&id, &cfg, Some(running())); + let winners_secret = crate::observe::referenced_secret(&winner).unwrap(); + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(winner); + fake.secrets.borrow_mut().push(crate::pod::build_secret( + &id, + &cfg.namespace, + "gen-existing", + env(), + )); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + + let deleted: Vec<_> = fake + .mutations() + .into_iter() + .filter(|c| c.starts_with("delete_secret")) + .collect(); + assert_eq!( + deleted.len(), + 1, + "expected exactly our own Secret dropped: {deleted:?}" + ); + assert!( + !deleted[0].contains(&winners_secret), + "deleted the winner's Secret: {deleted:?}" + ); + assert!( + fake.secrets + .borrow() + .iter() + .any(|s| s.metadata.name.as_deref() == Some(winners_secret.as_str())), + "the winner's Secret is gone" + ); + } + + /// The loser must not apply the divergence row to the pod that just beat + /// it — that is the ping-pong the spec forbids (`:845-850`). A divergent + /// *started* winner is adopted as-is. + #[test] + fn create_loser_does_not_replace_a_divergent_winner() { + let id = identity(); + let cfg = config(); + let mut winner = our_pod(&id, &cfg, Some(running())); + winner.metadata.annotations.as_mut().unwrap().insert( + ANNOTATION_CREATE_INTENT.to_string(), + "a-different-intent".into(), + ); + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(winner); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!( + !fake.mutations().iter().any(|c| c.starts_with("delete_pod")), + "the create loser deleted the winner" + ); + } + + /// The loser waits for the winner to *start* — adopting a not-yet-started + /// winner as success would report an agent that is not running. + #[test] + fn create_loser_waits_for_the_winner_to_start() { + let id = identity(); + let cfg = config(); + let fake = Fake::default(); + *fake.winner.borrow_mut() = Some(our_pod(&id, &cfg, Some(waiting("ContainerCreating")))); + fake.create_outcomes + .borrow_mut() + .push(CreateOutcome::AlreadyExists); + let polls = RefCell::new(0); + fake.on_poll.borrow_mut().push({ + let name = id.pod_name(); + Box::new(move |pods: &mut BTreeMap| { + *polls.borrow_mut() += 1; + if *polls.borrow() >= 3 { + if let Some(pod) = pods.get_mut(&name) { + pod.status.as_mut().unwrap().container_statuses = + Some(vec![ContainerStatus { + name: crate::observe::CONTAINER_NAME.into(), + state: Some(running()), + ..Default::default() + }]); + } + } + }) + }); + + assert_eq!(run(&fake, &id, &cfg).unwrap(), id.pod_name()); + assert!(fake.elapsed() > Duration::ZERO, "did not wait at all"); + } + + // ---- deadline and namespace -------------------------------------------- + + /// Deadline expiry reports the *latest* condition, not a generic timeout + /// (`:699-701`), and triggers no cleanup (`:720-724`). + #[test] + fn deadline_expiry_reports_the_condition_and_cleans_up_nothing() { + let id = identity(); + let cfg = config(); + let fake = Fake::default().with_pod(our_pod(&id, &cfg, Some(waiting("ContainerCreating")))); + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("ContainerCreating"), "generic timeout: {err}"); + let calls = fake.mutations(); + assert!( + !calls.iter().any(|c| c.starts_with("delete_")), + "deadline expiry triggered cleanup: {calls:?}" + ); + } + + /// An RBAC denial on namespace create fails the deploy with the literal + /// command to run, before any Secret is written (`:1002-1005`). + #[test] + fn namespace_denial_fails_before_writing_any_secret() { + let id = identity(); + let cfg = config(); + let fake = Fake { + namespace_error: Some(format!( + "not authorized to create namespaces: run `kubectl create namespace {}`", + cfg.namespace + )), + ..Default::default() + }; + + let err = run(&fake, &id, &cfg).unwrap_err(); + assert!(err.contains("kubectl create namespace"), "got: {err}"); + assert!( + fake.mutations().is_empty(), + "wrote something after a namespace denial: {:?}", + fake.mutations() + ); + } + + /// GC failures are hygiene, not deploy failures: a list the user cannot + /// perform must not block a deploy they can. + #[test] + fn a_gc_failure_does_not_fail_the_deploy() { + struct GcDenied(Fake); + impl Substrate for GcDenied { + async fn ensure_namespace(&self, ns: &str) -> Result<(), String> { + self.0.ensure_namespace(ns).await + } + async fn list_pods( + &self, + _s: &str, + ) -> Result<(Vec, Option>), String> { + Err("forbidden: cannot list pods".into()) + } + async fn list_secrets(&self, _s: &str) -> Result, String> { + Err("forbidden: cannot list secrets".into()) + } + async fn secret_exists(&self, n: &str) -> Result { + self.0.secret_exists(n).await + } + async fn create_secret(&self, s: &Secret) -> Result<(), String> { + self.0.create_secret(s).await + } + async fn create_pod(&self, p: &Pod) -> Result { + self.0.create_pod(p).await + } + async fn delete_pod(&self, n: &str, f: &Fence) -> Result { + self.0.delete_pod(n, f).await + } + async fn delete_secret(&self, n: &str) -> Result<(), String> { + self.0.delete_secret(n).await + } + async fn get_pod(&self, n: &str) -> Result, String> { + self.0.get_pod(n).await + } + async fn sleep(&self, d: Duration) { + self.0.sleep(d).await + } + fn elapsed(&self) -> Duration { + self.0.elapsed() + } + } + + let id = identity(); + let cfg = config(); + let inner = Fake::default().with_pod(our_pod(&id, &cfg, Some(running()))); + let fake = GcDenied(inner); + + assert_eq!( + block_on(deploy(&fake, &id, &cfg, env())).unwrap(), + id.pod_name() + ); + } +} diff --git a/crates/buzz-backend-kubernetes/src/wire.rs b/crates/buzz-backend-kubernetes/src/wire.rs new file mode 100644 index 0000000000..89b78b364d --- /dev/null +++ b/crates/buzz-backend-kubernetes/src/wire.rs @@ -0,0 +1,250 @@ +//! The stdin/stdout JSON protocol (spec §Provider Protocol). +//! +//! One process per operation: one JSON object in, one JSON object out. +//! These types are this provider's view of the contract; the golden fixtures +//! in `tests/fixtures/provider-wire/` are the arbiter shared with the desktop. + +use serde::{Deserialize, Serialize}; +use std::collections::BTreeMap; + +/// The wire-contract version this provider speaks (spec §Info). +pub const PROTOCOL_VERSION: u32 = 1; + +/// Request envelope. `op` discriminates; unknown ops are an in-band error. +/// +/// `request_id` is deliberately absent from every variant. The desktop sends +/// it, but the exchange is one request and one response per process, so there +/// is nothing to correlate and nothing in the response schema to echo it into +/// (`:387`, `:416` — neither response shape carries it). Serde ignores it on +/// the way in, so a caller that sends it is accepted; typing it would only +/// create a field nothing reads. +#[derive(Debug, Deserialize)] +#[serde(tag = "op", rename_all = "lowercase")] +pub enum Request { + Info, + Deploy(Box), +} + +#[derive(Debug, Deserialize)] +pub struct DeployRequest { + pub agent: AgentPayload, + #[serde(default)] + pub provider_config: serde_json::Value, +} + +/// The agent payload (spec §Deploy). +/// +/// Only the fields this binding actually consumes are typed. `name` (display +/// name), `model`, `provider`, and `turn_timeout_seconds` are deliberately +/// absent: object names derive from the pubkey, not the display name +/// (`:1150-1152`), the model/provider pair arrives already resolved inside +/// `launch`, and the timeout is ignored upstream — typing any of them would +/// invite a provider-side remap the spec forbids. +#[derive(Debug, Deserialize)] +pub struct AgentPayload { + pub relay_url: String, + pub private_key_nsec: String, + #[serde(default)] + pub auth_tag: Option, + #[serde(default)] + pub respond_to: Option, + #[serde(default)] + pub respond_to_allowlist: Option>, + /// User env, already merged global < persona < agent by the desktop and + /// already stripped of reserved keys. Superseded by `launch.env` when + /// `launch` is present — a provider MUST NOT re-merge it on top + /// (§Launch data, precedence tier 2). + #[serde(default)] + pub env_vars: BTreeMap, + /// The desktop-resolved launch contract. Absent only from a desktop + /// predating Known Defect 3's fix. + #[serde(default)] + pub launch: Option, +} + +/// Desktop-resolved launch data (spec §Launch data). +#[derive(Debug, Default, Deserialize)] +pub struct LaunchBlock { + /// Command *name*, resolved against the image's PATH — never a host path. + #[serde(default)] + pub command: Option, + #[serde(default)] + pub args: Vec, + /// Layered env: baked → runtime metadata → definition → global → persona + /// → agent. Precedence tier 2. + #[serde(default)] + pub env: BTreeMap, + /// Overridable behavior defaults. Precedence tier 1 — user env beats + /// these, matching the local spawn. + #[serde(default)] + pub policy_env: BTreeMap, + /// Resolved workspace owner (hex). The respond-to gate's one + /// irreducible input; without it or `auth_tag` the harness cannot match + /// `!shutdown`. + #[serde(default)] + pub owner_pubkey: Option, +} + +/// Response envelope. Serialized flat — `{"ok": true, …}` — because the +/// desktop reads `ok`, `error`, and `agent_id` off the top level. +#[derive(Debug, Serialize)] +#[serde(untagged)] +pub enum Response { + Info(InfoResponse), + Deploy(DeployResponse), + Error(ErrorResponse), +} + +#[derive(Debug, Serialize)] +pub struct InfoResponse { + pub ok: bool, + pub name: &'static str, + pub version: &'static str, + pub protocol_version: u32, + pub description: &'static str, + pub config_schema: serde_json::Value, +} + +#[derive(Debug, Serialize)] +pub struct DeployResponse { + pub ok: bool, + pub agent_id: String, +} + +#[derive(Debug, Serialize)] +pub struct ErrorResponse { + pub ok: bool, + pub error: String, +} + +impl Response { + pub fn error(message: impl Into) -> Self { + Response::Error(ErrorResponse { + ok: false, + error: message.into(), + }) + } + + /// The provider's self-description (spec §Info). Pure — no cluster + /// contact — because the desktop calls it to render the config form + /// before a kubeconfig is known to exist. + pub fn info() -> Self { + Response::Info(InfoResponse { + ok: true, + name: "kubernetes", + version: env!("CARGO_PKG_VERSION"), + protocol_version: PROTOCOL_VERSION, + description: "Runs agents as pods in a Kubernetes cluster", + config_schema: crate::config::config_schema(), + }) + } + + pub fn deployed(agent_id: impl Into) -> Self { + Response::Deploy(DeployResponse { + ok: true, + agent_id: agent_id.into(), + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parses_info_request() { + let r: Request = serde_json::from_str(r#"{"op":"info","request_id":"abc"}"#).unwrap(); + assert!(matches!(r, Request::Info)); + } + + /// The desktop sends `request_id` on every call, but the exchange is 1:1 + /// per process — a provider that hard-required it would fail a + /// conforming-but-minimal caller for no safety gain. + #[test] + fn request_id_is_optional() { + let r: Request = serde_json::from_str(r#"{"op":"info"}"#).unwrap(); + assert!(matches!(r, Request::Info)); + } + + #[test] + fn rejects_unknown_op() { + assert!(serde_json::from_str::(r#"{"op":"undeploy"}"#).is_err()); + } + + /// Payload fields this binding does not consume must not break parsing: + /// the desktop sends `model`, `provider`, `system_prompt` and more, and a + /// provider that rejected them would break on every real deploy. + #[test] + fn ignores_unconsumed_payload_fields() { + let json = r#"{ + "op":"deploy","request_id":"r1", + "agent":{ + "name":"a","relay_url":"wss://r","private_key_nsec":"nsec1x", + "model":"gpt-5","provider":"openai","system_prompt":"hi", + "turn_timeout_seconds":30,"parallelism":10, + "agent_command":"goose","agent_args":[] + }, + "provider_config":{"namespace":"ns"} + }"#; + let r: Request = serde_json::from_str(json).unwrap(); + let Request::Deploy(d) = r else { + panic!("wrong op") + }; + assert_eq!(d.agent.relay_url, "wss://r"); + assert!(d.agent.launch.is_none()); + } + + #[test] + fn parses_launch_block() { + let json = r#"{ + "op":"deploy", + "agent":{ + "relay_url":"wss://r","private_key_nsec":"nsec1x", + "launch":{ + "command":"goose","args":["run","--x"], + "env":{"GOOSE_MODEL":"m"}, + "policy_env":{"GOOSE_MODE":"auto"}, + "owner_pubkey":"deadbeef" + } + } + }"#; + let Request::Deploy(d) = serde_json::from_str::(json).unwrap() else { + panic!("wrong op") + }; + let l = d.agent.launch.unwrap(); + assert_eq!(l.command.as_deref(), Some("goose")); + assert_eq!(l.args, ["run", "--x"]); + assert_eq!(l.env["GOOSE_MODEL"], "m"); + assert_eq!(l.policy_env["GOOSE_MODE"], "auto"); + assert_eq!(l.owner_pubkey.as_deref(), Some("deadbeef")); + } + + /// A null `owner_pubkey`/`auth_tag` must parse (the refusal is a policy + /// decision made later, with a specific message), not fail as a type error. + #[test] + fn null_owner_fields_parse() { + let json = r#"{"op":"deploy","agent":{ + "relay_url":"wss://r","private_key_nsec":"nsec1x","auth_tag":null, + "launch":{"owner_pubkey":null} + }}"#; + let Request::Deploy(d) = serde_json::from_str::(json).unwrap() else { + panic!("wrong op") + }; + assert!(d.agent.auth_tag.is_none()); + assert!(d.agent.launch.unwrap().owner_pubkey.is_none()); + } + + /// The desktop reads `ok`/`error`/`agent_id` off the top level, so the + /// enum must serialize flat with no variant tag. + #[test] + fn responses_serialize_flat() { + let v = serde_json::to_value(Response::deployed("buzz-agent-abc")).unwrap(); + assert_eq!(v["ok"], true); + assert_eq!(v["agent_id"], "buzz-agent-abc"); + assert!(v.get("Deploy").is_none()); + + let v = serde_json::to_value(Response::error("boom")).unwrap(); + assert_eq!(v["ok"], false); + assert_eq!(v["error"], "boom"); + } +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/README.md b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/README.md new file mode 100644 index 0000000000..56a972f894 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/README.md @@ -0,0 +1,39 @@ +# Provider wire fixtures + +The shared arbiter for the stdin/stdout contract between the desktop +(`agents_deploy.rs`) and this provider (spec §Provider Protocol). + +Each `*.request.json` is a request the desktop can emit; each matching +`*.response.json` is the exact response this provider produces for it. The +provider side is asserted by `tests/wire_fixtures.rs`; the desktop side should +assert that its emitted payloads parse as the corresponding request. + +Three rules keep these useful rather than decorative: + +* **Requests are recorded, not invented.** A fixture that no caller emits + tests a contract nobody has. "Recorded" means *executed and transcribed* — + `deploy-full-launch.request.json` is the output of the desktop's real + `build_launch_block` → `deploy_payload_json` path, not a shape derived by + reading those functions. Deriving it is how this fixture acquired four + impossible values at once: a `respond_to` that was a pubkey where the + desktop serializes a kebab-case `RespondTo` enum, allowlist and owner + values failing `validate_respond_to_allowlist`'s 64-hex rule + (`types.rs:897`), an invented `BUZZ_ACP_PARALLELISM` where the emitter + writes `BUZZ_ACP_AGENTS` (`runtime.rs:729`), and a `launch.env` key from + no layer of `resolve_effective_harness_descriptor`. +* **The provider cannot police this file, so the desktop must.** Every field + above is one this provider is deliberately indifferent to — `respond_to` is + an opaque `Option`, the allowlist an opaque `Vec`, + `policy_env` an arbitrary map — so `the_full_desktop_payload_is_accepted` + passes on invented data exactly as happily as on recorded data. The + enforcement is the desktop's whole-object equality test, which *builds* the + payload and compares it to this file. A completeness guard (the case-list + directory scan in `wire_fixtures.rs`) stops a case from going missing; it + cannot tell you a case is false. +* **Responses are byte-compared after key-sorted re-serialization**, so a + field rename or a type change fails here rather than in a desktop that + silently reads `undefined`. + +`deploy-*` fixtures cover only responses reachable without a cluster — +refusals and malformed input. A successful deploy needs an apiserver and is +covered by the conformance suite, not by a static fixture. diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json new file mode 100644 index 0000000000..28fe6ce90e --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json @@ -0,0 +1,52 @@ +{ + "op": "deploy", + "request_id": "req-6", + "agent": { + "agent_args": [], + "agent_command": "goose", + "auth_tag": "tag-1", + "env_vars": { + "USER_KEY": "user-value" + }, + "idle_timeout_seconds": null, + "launch": { + "args": [ + "acp" + ], + "command": "goose", + "env": { + "GOOSE_MODEL": "gpt-5", + "GOOSE_PROVIDER": "openai", + "USER_KEY": "user-value" + }, + "owner_pubkey": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "policy_env": { + "BUZZ_ACP_AGENTS": "10", + "BUZZ_ACP_LAZY_POOL": "true", + "BUZZ_ACP_MODEL": "gpt-5", + "BUZZ_ACP_RELAY_OBSERVER": "true", + "BUZZ_ACP_SESSION_TITLE": "worker", + "GOOSE_MODE": "auto" + } + }, + "max_turn_duration_seconds": null, + "model": "gpt-5", + "name": "worker", + "parallelism": 10, + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "provider": "openai", + "relay_url": "wss://relay.example", + "respond_to": "allowlist", + "respond_to_allowlist": [ + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + ], + "system_prompt": null, + "turn_timeout_seconds": 300 + }, + "provider_config": { + "namespace": "buzz-agents-test", + "image": "ghcr.io/block/buzz-sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "inactivity_seconds": 3600 + } +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.request.json new file mode 100644 index 0000000000..4f56082bbb --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.request.json @@ -0,0 +1,9 @@ +{ + "op": "deploy", + "request_id": "req-5", + "agent": { + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5" + }, + "provider_config": {"namespace": "buzz-agents-test", "image": "ghcr.io/block/buzz-sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.response.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.response.json new file mode 100644 index 0000000000..c6011b899a --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-no-owner.response.json @@ -0,0 +1 @@ +{"ok":false,"error":"deploy refused: neither auth_tag nor launch.owner_pubkey resolved — without an owner the agent cannot honor !shutdown"} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.request.json new file mode 100644 index 0000000000..2160909b8f --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.request.json @@ -0,0 +1,11 @@ +{ + "op": "deploy", + "request_id": "req-3", + "agent": { + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "auth_tag": "tag-1", + "provider": " relay-mesh " + }, + "provider_config": {"namespace": "buzz-agents-test", "image": "ghcr.io/block/buzz-sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.response.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.response.json new file mode 100644 index 0000000000..0f0c995fd8 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh-padded.response.json @@ -0,0 +1 @@ +{"ok":false,"error":"deploy refused: this agent is configured for shared compute (relay-mesh), which runs on the relay rather than in a pod. Switch the agent to a local runtime before deploying it to Kubernetes."} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.request.json new file mode 100644 index 0000000000..71ce9f067e --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.request.json @@ -0,0 +1,12 @@ +{ + "op": "deploy", + "request_id": "req-2", + "agent": { + "name": "mesh-agent", + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "auth_tag": "tag-1", + "provider": "relay-mesh" + }, + "provider_config": {"namespace": "buzz-agents-test", "image": "ghcr.io/block/buzz-sprig@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.response.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.response.json new file mode 100644 index 0000000000..0f0c995fd8 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-relay-mesh.response.json @@ -0,0 +1 @@ +{"ok":false,"error":"deploy refused: this agent is configured for shared compute (relay-mesh), which runs on the relay rather than in a pod. Switch the agent to a local runtime before deploying it to Kubernetes."} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.request.json new file mode 100644 index 0000000000..232d9dcf43 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.request.json @@ -0,0 +1,10 @@ +{ + "op": "deploy", + "request_id": "req-4", + "agent": { + "relay_url": "wss://relay.example", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "auth_tag": "tag-1" + }, + "provider_config": {"namespace": "buzz-agents-test", "image": "ghcr.io/block/buzz-sprig:latest"} +} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.response.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.response.json new file mode 100644 index 0000000000..e67fa6a47f --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-tag-image.response.json @@ -0,0 +1 @@ +{"ok":false,"error":"provider_config.image \"ghcr.io/block/buzz-sprig:latest\" is not digest-pinned: a tag is a mutable pointer, and this object runs with the agent's private key. Use name@sha256:<64 hex chars>"} diff --git a/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/info.request.json b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/info.request.json new file mode 100644 index 0000000000..db99c86c00 --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/info.request.json @@ -0,0 +1 @@ +{"op":"info","request_id":"req-1"} diff --git a/crates/buzz-backend-kubernetes/tests/wire_fixtures.rs b/crates/buzz-backend-kubernetes/tests/wire_fixtures.rs new file mode 100644 index 0000000000..b3049a98ef --- /dev/null +++ b/crates/buzz-backend-kubernetes/tests/wire_fixtures.rs @@ -0,0 +1,222 @@ +//! Golden wire fixtures (spec §Provider Protocol). +//! +//! These drive the **built binary** over a real pipe rather than calling an +//! in-process function: the contract the desktop depends on is +//! `stdin → one JSON object on stdout → exit code`, and an in-process test +//! would assert the shape of a value while skipping the three things that +//! actually break — the process writing nothing, writing two objects, or +//! signalling the outcome through the exit code. + +use std::io::Write; +use std::path::{Path, PathBuf}; +use std::process::{Command, Stdio}; + +fn fixtures() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")).join("tests/fixtures/provider-wire") +} + +/// Feed one request to the binary; return `(stdout, exit code)`. +fn run(request: &str) -> (String, i32) { + let mut child = Command::new(env!("CARGO_BIN_EXE_buzz-backend-kubernetes")) + // A kubeconfig that does not exist, so a fixture that accidentally + // reaches the cluster fails loudly here instead of depending on + // whatever cluster the developer is pointed at. + .env("KUBECONFIG", "/nonexistent/kubeconfig-for-fixture-tests") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .spawn() + .expect("could not run the provider binary"); + child + .stdin + .take() + .expect("no stdin") + .write_all(request.as_bytes()) + .expect("could not write the request"); + let out = child.wait_with_output().expect("provider did not exit"); + ( + String::from_utf8(out.stdout).expect("stdout was not UTF-8"), + out.status.code().unwrap_or(-1), + ) +} + +fn read(name: &str) -> String { + std::fs::read_to_string(fixtures().join(name)) + .unwrap_or_else(|e| panic!("could not read fixture {name}: {e}")) +} + +/// Every response fixture, byte-compared after key-sorted re-serialization so +/// a field rename fails here rather than in a desktop reading `undefined`. +#[test] +fn responses_match_their_fixtures() { + let cases = [ + "deploy-relay-mesh", + "deploy-relay-mesh-padded", + "deploy-tag-image", + "deploy-no-owner", + ]; + // The list must cover every response fixture on disk. A literal array is + // never empty, so `!is_empty()` would assert nothing; what can actually go + // wrong is a fixture added to the directory and never added here, which + // reads as a passing suite that exercises one case fewer than it appears to. + let mut on_disk: Vec = std::fs::read_dir(fixtures()) + .expect("could not read the fixture directory") + .filter_map(|entry| entry.ok()?.file_name().into_string().ok()) + .filter_map(|name| Some(name.strip_suffix(".response.json")?.to_string())) + .collect(); + on_disk.sort(); + let mut listed: Vec = cases.iter().map(|c| c.to_string()).collect(); + listed.sort(); + assert_eq!(on_disk, listed, "response fixtures and cases disagree"); + + for case in cases { + let (stdout, code) = run(&read(&format!("{case}.request.json"))); + assert_eq!(code, 0, "{case}: a produced response must exit 0"); + + // Exactly one object, terminated by exactly one newline. Two responses + // would leave the desktop's reader holding a second one forever. + assert_eq!( + stdout.matches('\n').count(), + 1, + "{case}: expected exactly one line, got {stdout:?}" + ); + + let actual: serde_json::Value = + serde_json::from_str(&stdout).unwrap_or_else(|e| panic!("{case}: {e}: {stdout:?}")); + let expected: serde_json::Value = + serde_json::from_str(&read(&format!("{case}.response.json"))).unwrap(); + assert_eq!( + actual, expected, + "{case}: response drifted from its fixture" + ); + } +} + +/// `info` is checked on the fields the desktop reads rather than byte-for-byte: +/// the namespace default is randomly generated per call (§K8s Namespace), so a +/// golden copy of it would be a test that fails every run. +#[test] +fn info_response_carries_the_contract_fields() { + let (stdout, code) = run(&read("info.request.json")); + assert_eq!(code, 0); + let info: serde_json::Value = serde_json::from_str(&stdout).unwrap(); + assert_eq!(info["ok"], true); + assert_eq!(info["protocol_version"], 1); + assert_eq!(info["name"], "kubernetes"); + let schema = &info["config_schema"]; + assert_eq!( + schema["required"], + serde_json::json!(["namespace", "image"]) + ); + let default = schema["properties"]["namespace"]["default"] + .as_str() + .expect("no generated namespace default"); + assert!( + default.starts_with("buzz-agents-"), + "unexpected namespace default: {default}" + ); + let image_default = schema["properties"]["image"]["default"] + .as_str() + .expect("no image default"); + assert!( + image_default.starts_with("ghcr.io/block/buzz-sprig:") + && image_default.contains("@sha256:"), + "unexpected image default: {image_default}" + ); +} + +/// The desktop's richest payload must parse. No response fixture: this one +/// reaches the cluster, so its outcome depends on a kubeconfig. What it +/// guards is that every field the desktop sends is *accepted* — a payload the +/// provider rejects at parse time is a deploy that never starts. +#[test] +fn the_full_desktop_payload_is_accepted() { + let (stdout, code) = run(&read("deploy-full-launch.request.json")); + assert_eq!(code, 0); + let response: serde_json::Value = serde_json::from_str(&stdout).unwrap(); + let error = response["error"].as_str().unwrap_or_default(); + // It fails — there is no cluster — but it must fail at the *connection*, + // having accepted every field above it. + assert!( + error.contains("kubeconfig"), + "the full payload was rejected before reaching the cluster: {error}" + ); +} + +/// Sami's pre-registered respond-to matrix, driven through the built binary. +/// +/// `build_env` runs at `main.rs:124`, `client::connect` at `:132`, so under a +/// kubeconfig that cannot exist the error string *is* the ordering assertion: +/// "kubeconfig" means the gate passed and we reached the cluster, anything +/// else means we refused before writing a Secret. A test asserting only +/// `ok: false` would pass on the connection error and prove nothing. +/// +/// The cases are applied to the real full-launch request so each one differs +/// from a known-good deploy in exactly the field under test. +#[test] +fn the_respond_to_gate_matches_the_harness_acceptance_surface() { + let key_a = "a".repeat(64); + let padded_upper = format!(" {} ", "A".repeat(64)); + // (name, respond_to, allowlist, must reach the cluster) + let cases: Vec<(&str, &str, Option>, bool)> = vec![ + ("allowlist + []", "allowlist", Some(vec![]), false), + ("allowlist + absent", "allowlist", None, false), + ( + "allowlist + junk", + "allowlist", + Some(vec!["beefcafe".into()]), + false, + ), + ("unparseable mode", "npub1abc", None, false), + ("padded mode", " allowlist ", None, false), + ( + "allowlist + two valid", + "allowlist", + Some(vec![key_a.clone(), "b".repeat(64)]), + true, + ), + ( + "owner-only + junk list", + "owner-only", + Some(vec!["beefcafe".into()]), + true, + ), + ( + "allowlist + padded upper", + "allowlist", + Some(vec![padded_upper]), + true, + ), + ("nobody", "nobody", None, true), + ("anyone", "anyone", None, true), + ]; + + let base: serde_json::Value = + serde_json::from_str(&read("deploy-full-launch.request.json")).unwrap(); + + for (name, mode, allowlist, reaches_cluster) in cases { + let mut request = base.clone(); + let agent = &mut request["agent"]; + agent["respond_to"] = serde_json::json!(mode); + agent["respond_to_allowlist"] = match &allowlist { + Some(list) => serde_json::json!(list), + None => serde_json::Value::Null, + }; + + let (stdout, code) = run(&request.to_string()); + assert_eq!(code, 0, "{name}: provider did not exit cleanly"); + let response: serde_json::Value = serde_json::from_str(&stdout).unwrap(); + let error = response["error"].as_str().unwrap_or_default(); + let reached = error.contains("kubeconfig"); + + assert_eq!( + reached, reaches_cluster, + "{name}: expected reaches_cluster={reaches_cluster}, got error: {error}" + ); + if !reaches_cluster { + assert!( + error.contains("deploy refused"), + "{name}: refused, but not by the gate: {error}" + ); + } + } +} diff --git a/crates/buzz-cli/README.md b/crates/buzz-cli/README.md index 40699459fc..a2dcdce6d2 100644 --- a/crates/buzz-cli/README.md +++ b/crates/buzz-cli/README.md @@ -56,6 +56,7 @@ buzz reactions get --event buzz users get # your own profile buzz users get --pubkey # single user buzz users get --pubkey --pubkey # batch (max 200) +buzz users get --name Honey --owner me # exact-name lookup in your managed agents buzz users set-presence --status online buzz users set-status --text "heads down on the CLI" --emoji "🚀" buzz users set-status --clear # remove your status diff --git a/crates/buzz-cli/src/commands/messages.rs b/crates/buzz-cli/src/commands/messages.rs index 290cc59fa8..40a9ae80b5 100644 --- a/crates/buzz-cli/src/commands/messages.rs +++ b/crates/buzz-cli/src/commands/messages.rs @@ -9,8 +9,7 @@ use crate::validate::{ validate_content_size, validate_hex64, validate_uuid, MAX_DIFF_BYTES, }; use buzz_sdk::mentions::{ - extract_at_mentions_with_known, extract_nostr_uris, merge_mentions, strip_code_regions, - MENTION_CAP, + extract_at_mentions_with_known, extract_nostr_uris, strip_code_regions, MENTION_CAP, }; /// Extract the thread root event ID from a Nostr tag array. @@ -119,47 +118,82 @@ async fn resolve_channel_id(client: &BuzzClient, event_id: &str) -> Result>, + has_explicit_mentions: bool, +) -> Result, CliError> { + let mut resolved = Vec::new(); + for name in names { + match name_to_pubkeys + .get(name) + .map(Vec::as_slice) + .unwrap_or_default() + { + [pubkey] => resolved.push(pubkey.clone()), + [] if has_explicit_mentions => {} + [] => { + return Err(CliError::Usage(format!( + "mention '@{name}' does not match a current channel member; retry with --mention " + ))) + } + _ if has_explicit_mentions => {} + candidates => { + return Err(CliError::Usage(format!( + "mention '@{name}' is ambiguous; candidates: {}. Retry with --mention ", + candidates.join(", ") + ))) + } + } + } + Ok(resolved) +} + +/// Resolve mention text against the channel membership snapshot. /// -/// Queries kind 39002 (channel members) then kind 0 (profiles), parses -/// display names once, and feeds them to [`extract_at_mentions_with_known`] -/// for multi-word matching. On any I/O or parse failure, returns an empty -/// vec — auto-tagging is best-effort and must never block a send. +/// Returns both the current member set and uniquely name-resolved pubkeys. +/// Lookup failures are fatal when mention processing is requested: publishing +/// visible mention text without its intended `p` tag is worse than not sending. async fn resolve_content_mentions( client: &BuzzClient, channel_id: &str, content: &str, -) -> Vec { - if !content.contains('@') { - return vec![]; + has_explicit_mentions: bool, +) -> Result<(Vec, Vec), CliError> { + let stripped = strip_code_regions(content); + if !stripped.contains('@') && !has_explicit_mentions { + return Ok((vec![], vec![])); } - // 1. Membership list (kind 39002 is parameterized-replaceable, addressed by `d` tag). let members_filter = serde_json::json!({ "kinds": [39002], "#d": [channel_id], "limit": 1, }); - let member_pubkeys = match fetch_member_pubkeys(client, &members_filter).await { - Some(pks) if !pks.is_empty() => pks, - _ => return vec![], - }; + let member_pubkeys = fetch_member_pubkeys(client, &members_filter) + .await + .ok_or_else(|| { + CliError::Other("could not load channel membership for mention preflight".into()) + })?; + + if !stripped.contains('@') { + return Ok((member_pubkeys, vec![])); + } - // 2. Profiles for those members (kind 0). let profiles_filter = serde_json::json!({ "kinds": [0], "authors": member_pubkeys, "limit": member_pubkeys.len(), }); - let profile_events = match fetch_events(client, &profiles_filter).await { - Some(v) => v, - None => return vec![], - }; + let profile_events = fetch_events(client, &profiles_filter) + .await + .ok_or_else(|| { + CliError::Other("could not load member profiles for mention resolution".into()) + })?; - // 3. Single parse: extract (pubkey, display_name) pairs from profile JSON. let mut name_to_pubkeys: std::collections::HashMap> = std::collections::HashMap::new(); - let mut display_names: Vec = Vec::new(); + let mut display_names = Vec::new(); for e in &profile_events { let Some(pubkey) = e.get("pubkey").and_then(|v| v.as_str()) else { continue; @@ -178,26 +212,82 @@ async fn resolve_content_mentions( else { continue; }; - let lower = name.to_ascii_lowercase(); name_to_pubkeys - .entry(lower) + .entry(name.to_ascii_lowercase()) .or_default() .push(pubkey.to_string()); display_names.push(name.to_string()); } - // 4. Two-pass extraction: known multi-word names first, single-word fallback. - let known_refs: Vec<&str> = display_names.iter().map(|s| s.as_str()).collect(); - let names = extract_at_mentions_with_known(content, &known_refs); + let known_refs: Vec<&str> = display_names.iter().map(String::as_str).collect(); + let names = extract_at_mentions_with_known(&stripped, &known_refs); + let resolved = resolve_names_to_pubkeys(&names, &name_to_pubkeys, has_explicit_mentions)?; + Ok((member_pubkeys, resolved)) +} + +fn normalize_explicit_mentions(values: &[String]) -> Result, CliError> { + let mut normalized = Vec::new(); + for value in values { + let pubkey = PublicKey::parse(value.trim()) + .map_err(|_| CliError::Usage(format!("invalid --mention pubkey: {value}")))?; + let hex = pubkey.to_hex(); + if !normalized.contains(&hex) { + normalized.push(hex); + } + } + if normalized.len() > MENTION_CAP { + return Err(CliError::Usage(format!( + "too many --mention values (max {MENTION_CAP})" + ))); + } + Ok(normalized) +} + +fn merge_message_mentions( + explicit: &[String], + uri_pubkeys: &[String], + auto_resolved: &[String], +) -> Result, CliError> { + let mut mentions = Vec::new(); + for pubkey in explicit + .iter() + .chain(uri_pubkeys.iter()) + .chain(auto_resolved.iter()) + { + if !mentions.contains(pubkey) { + mentions.push(pubkey.clone()); + } + } + if mentions.len() > MENTION_CAP { + return Err(CliError::Usage(format!( + "too many unique message mentions (max {MENTION_CAP})" + ))); + } + Ok(mentions) +} - // 5. Look up matched names → pubkeys via the map we already built. - names +fn missing_members(mentions: &[String], members: &[String]) -> Vec { + let members: std::collections::HashSet<&str> = members.iter().map(String::as_str).collect(); + mentions .iter() - .flat_map(|n| name_to_pubkeys.get(n).into_iter().flatten()) + .filter(|pk| !members.contains(pk.as_str())) .cloned() .collect() } +fn event_mention_pubkeys(event: &nostr::Event) -> Vec { + event + .tags + .iter() + .filter_map(|tag| { + let parts = tag.as_slice(); + (parts.first().map(String::as_str) == Some("p")) + .then(|| parts.get(1).cloned()) + .flatten() + }) + .collect() +} + /// Fetch raw events for `filter` via the relay's `/query` endpoint. /// Returns `None` on any I/O or parse failure. async fn fetch_events( @@ -478,6 +568,7 @@ pub struct SendMessageParams { pub reply_to: Option, pub broadcast: bool, pub files: Vec, + pub mentions: Vec, } pub async fn cmd_send_message( @@ -495,6 +586,30 @@ pub async fn cmd_send_message( } let channel_uuid = parse_uuid(&p.channel_id)?; + let explicit_mentions = normalize_explicit_mentions(&p.mentions)?; + let stripped = strip_code_regions(&p.content); + let uri_pubkeys = extract_nostr_uris(&stripped); + // Supplying any identity explicitly authorizes unresolved or ambiguous @Name text + // as presentation-only, matching Desktop's separate visible-label and p-tag model. + // Uniquely resolvable member names still add their own p-tags; callers must supply + // every intended identity whose visible label cannot be resolved uniquely. + let has_explicit_mentions = !explicit_mentions.is_empty() || !uri_pubkeys.is_empty(); + let (member_pubkeys, auto_resolved) = + resolve_content_mentions(client, &p.channel_id, &p.content, has_explicit_mentions).await?; + let mention_pubkeys = merge_message_mentions(&explicit_mentions, &uri_pubkeys, &auto_resolved)?; + + let missing = missing_members(&mention_pubkeys, &member_pubkeys); + if !missing.is_empty() { + return Err(CliError::Usage( + serde_json::json!({ + "message": "mentioned pubkeys are not channel members; add them explicitly before retrying", + "missing_member_pubkeys": missing, + "add_member_command": format!("buzz channels add-member --channel {} --pubkey --role ", p.channel_id), + }) + .to_string(), + )); + } + // Upload files and build imeta tags let mut media_tags: Vec> = Vec::new(); let mut media_content = String::new(); @@ -526,16 +641,7 @@ pub async fn cmd_send_message( None }; - // Resolve @name mentions in the author-written body only — not the media markdown we - // append above, which is derived from upload metadata and can't carry `@names`. - let mut auto_resolved = resolve_content_mentions(client, &p.channel_id, &p.content).await; - - // NIP-27: also extract nostr:npub1… inline references (skipping code regions) - let stripped = strip_code_regions(&p.content); - let uri_pubkeys = extract_nostr_uris(&stripped); - merge_mentions(&mut auto_resolved, &uri_pubkeys, MENTION_CAP); - - let mention_refs: Vec<&str> = auto_resolved.iter().map(|s| s.as_str()).collect(); + let mention_refs: Vec<&str> = mention_pubkeys.iter().map(String::as_str).collect(); let builder = match p.kind { Some(45001) => { @@ -572,9 +678,17 @@ pub async fn cmd_send_message( }; let event = client.sign_event(builder)?; - + let emitted_mentions = event_mention_pubkeys(&event); let resp = client.submit_event(event).await?; - println!("{}", normalize_write_response(&resp)); + let mut output: serde_json::Value = serde_json::from_str(&normalize_write_response(&resp)) + .unwrap_or_else(|_| serde_json::json!({ "response": resp })); + if let Some(object) = output.as_object_mut() { + object.insert( + "mention_pubkeys".into(), + serde_json::json!(emitted_mentions), + ); + } + println!("{output}"); Ok(()) } @@ -765,6 +879,7 @@ pub async fn dispatch( reply_to, broadcast, files, + mentions, } => { cmd_send_message( client, @@ -775,6 +890,7 @@ pub async fn dispatch( reply_to, broadcast, files, + mentions, }, ) .await @@ -876,7 +992,11 @@ pub async fn dispatch( #[cfg(test)] mod tests { - use super::{find_root_from_tags, match_profiles_by_name, parse_member_pubkeys}; + use super::{ + event_mention_pubkeys, find_root_from_tags, match_profiles_by_name, merge_message_mentions, + missing_members, normalize_explicit_mentions, parse_member_pubkeys, + resolve_names_to_pubkeys, + }; use buzz_sdk::mentions::{ extract_at_mentions_with_known, extract_at_names, match_names_to_profiles, MentionProfile, }; @@ -1103,6 +1223,94 @@ mod tests { assert_eq!(parse_member_pubkeys(&event), vec![PK_VALID_A, PK_VALID_A]); } + #[test] + fn explicit_mentions_accept_hex_and_npub_and_deduplicate() { + use nostr::ToBech32; + let npub = nostr::PublicKey::from_hex(PK_VALID_A) + .unwrap() + .to_bech32() + .unwrap(); + assert_eq!( + normalize_explicit_mentions(&[PK_VALID_A.into(), npub]).unwrap(), + vec![PK_VALID_A] + ); + assert!(normalize_explicit_mentions(&["not-a-key".into()]).is_err()); + } + + #[test] + fn explicit_mentions_authorize_presentation_text_without_name_resolution() { + let names = vec!["renamed user".into()]; + let profiles = std::collections::HashMap::new(); + assert_eq!( + resolve_names_to_pubkeys(&names, &profiles, true).unwrap(), + Vec::::new() + ); + assert!(resolve_names_to_pubkeys(&names, &profiles, false).is_err()); + } + + #[test] + fn explicit_mentions_authorize_ambiguous_presentation_text() { + let names = vec!["alice".into()]; + let profiles = std::collections::HashMap::from([( + "alice".into(), + vec![PK_VALID_A.into(), PK_VALID_B.into()], + )]); + assert_eq!( + resolve_names_to_pubkeys(&names, &profiles, true).unwrap(), + Vec::::new() + ); + let error = resolve_names_to_pubkeys(&names, &profiles, false).unwrap_err(); + assert!(error.to_string().contains(PK_VALID_A)); + assert!(error.to_string().contains(PK_VALID_B)); + } + + #[test] + fn explicit_mentions_make_all_at_names_presentation_only() { + let names = vec!["alice".into(), "bob".into()]; + let profiles = std::collections::HashMap::from([("alice".into(), vec![PK_VALID_A.into()])]); + assert_eq!( + resolve_names_to_pubkeys(&names, &profiles, true).unwrap(), + vec![PK_VALID_A] + ); + assert!(resolve_names_to_pubkeys(&names, &profiles, false).is_err()); + } + + #[test] + fn combined_mention_union_errors_instead_of_truncating() { + let explicit: Vec = (0..50).map(|i| format!("explicit-{i}")).collect(); + assert!(merge_message_mentions(&explicit, &[], &["resolved-bob".into()]).is_err()); + + let mut with_duplicate = explicit.clone(); + with_duplicate.push(explicit[0].clone()); + assert_eq!( + merge_message_mentions(&with_duplicate, &[explicit[1].clone()], &[]) + .unwrap() + .len(), + 50 + ); + } + + #[test] + fn membership_preflight_lists_only_missing_mentions() { + assert_eq!( + missing_members( + &[PK_VALID_A.into(), PK_VALID_B.into()], + &[PK_VALID_A.into()] + ), + vec![PK_VALID_B] + ); + } + + #[test] + fn mention_evidence_comes_from_signed_event_tags() { + use nostr::{EventBuilder, Keys, Tag}; + let event = EventBuilder::text_note("hello") + .tags(vec![Tag::parse(["p", PK_VALID_A]).unwrap()]) + .sign_with_keys(&Keys::generate()) + .unwrap(); + assert_eq!(event_mention_pubkeys(&event), vec![PK_VALID_A]); + } + // ---- match_profiles_by_name (author resolution for `messages search --author`) ---- fn profile_event( diff --git a/crates/buzz-cli/src/commands/mod.rs b/crates/buzz-cli/src/commands/mod.rs index 8691590636..1ccc37a702 100644 --- a/crates/buzz-cli/src/commands/mod.rs +++ b/crates/buzz-cli/src/commands/mod.rs @@ -12,9 +12,37 @@ pub mod notes; pub mod pack; pub mod patches; pub mod pr; +pub mod projects; pub mod reactions; pub mod repos; pub mod social; pub mod upload; pub mod users; pub mod workflows; + +use crate::{client::normalize_write_response, error::CliError}; + +/// Parse a relay write-response JSON blob, mapping a duplicate (dominated) +/// write to [`CliError::Conflict`] with the caller-supplied message. +/// +/// Used by every command that publishes an NIP-33 addressable event and +/// needs to tell accepted from duplicate/dominated. +pub fn parse_write_response(raw: &str, conflict_msg: &str) -> Result { + let response: serde_json::Value = serde_json::from_str(raw) + .map_err(|e| CliError::Other(format!("relay response is not JSON: {e} ({raw})")))?; + let accepted = response + .get("accepted") + .and_then(serde_json::Value::as_bool) + .unwrap_or(false); + let message = response + .get("message") + .and_then(serde_json::Value::as_str) + .unwrap_or(""); + if !accepted { + return Err(CliError::Other(format!("relay rejected event: {message}"))); + } + if message == "duplicate" || message.starts_with("duplicate:") { + return Err(CliError::Conflict(conflict_msg.to_string())); + } + Ok(normalize_write_response(raw)) +} diff --git a/crates/buzz-cli/src/commands/projects.rs b/crates/buzz-cli/src/commands/projects.rs new file mode 100644 index 0000000000..e6798dbfc4 --- /dev/null +++ b/crates/buzz-cli/src/commands/projects.rs @@ -0,0 +1,1198 @@ +//! `buzz projects` commands — NIP-MP kind:30621 write path. +//! +//! All mutations follow a read-modify-write pattern: +//! 1. Fetch the caller's own live head via `kinds:[30621] + authors:[self] + #d:[slug]`. +//! 2. Mutate the tag set (strip `auth`, apply change). +//! 3. Re-validate the full envelope through Layer A before submitting. +//! 4. Set `created_at = head.created_at + 1` (never wall-clock) to avoid +//! overwriting a concurrently advancing head. +//! +//! Limitations recorded in this phase: +//! - Relay hints are read-preserved but not authored (`--repo` carries +//! a coordinate only; existing hinted tags survive RMW unchanged). +//! - `delete` targets signer-self only (NIP-OA owner-delete path deferred). +//! - Deletion durability against later arrival (watermark follow-up) is +//! not in scope. + +use buzz_core::kind::KIND_PROJECT; +use buzz_sdk::{ + build_delete_addressable, build_project, build_project_with_tags, ProjectMemberCoord, + PROJECT_D_MAX_LEN, +}; +use nostr::{Event, EventBuilder, Tag, Timestamp}; + +use crate::client::BuzzClient; +use crate::commands::parse_write_response; +use crate::error::CliError; + +// ── Buzz repo-ID grammar (bare --repo shorthand) ───────────────────────────── + +/// Pattern for a Buzz-hosted repo identifier (bare `--repo` shorthand). +/// `[a-zA-Z0-9._-]{1,64}` — no colons, so guaranteed collision-free with +/// `30617::` full coordinates. +fn is_bare_repo_id(s: &str) -> bool { + !s.is_empty() + && s.len() <= 64 + && s.chars() + .all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '_' || c == '-') +} + +/// Expand a CLI `--repo` argument into a full `30617::` coordinate. +/// +/// Bare form (`[a-zA-Z0-9._-]{1,64}`): owner defaults to the caller's pubkey. +/// Full form (`30617::`): used verbatim. +fn expand_repo_coord(s: &str, caller_pubkey: &str) -> Result { + if is_bare_repo_id(s) { + // Bare form: expand to full coordinate with caller as owner. + let full = format!("30617:{caller_pubkey}:{s}"); + ProjectMemberCoord::parse_full(&full) + .map_err(|e| CliError::Usage(format!("invalid repo coordinate: {e}"))) + } else { + // Full form: must be parseable as a complete coordinate. + ProjectMemberCoord::parse_full(s) + .map_err(|e| CliError::Usage(format!("invalid repo coordinate: {e}"))) + } +} + +// ── Head-fetch helper ───────────────────────────────────────────────────────── + +fn parse_events(json: &str) -> Result, CliError> { + serde_json::from_str(json) + .map_err(|e| CliError::Other(format!("failed to parse relay response: {e}"))) +} + +/// Fetch the caller's own live kind:30621 head for `slug`. +async fn fetch_own_project(client: &BuzzClient, slug: &str) -> Result, CliError> { + fetch_project(client, slug, None).await +} + +/// Fetch a project head by slug and optional owner pubkey. +async fn fetch_project( + client: &BuzzClient, + slug: &str, + owner: Option<&str>, +) -> Result, CliError> { + let pubkey = match owner { + Some(pk) => { + crate::validate::validate_hex64(pk)?; + pk.to_string() + } + None => client.keys().public_key().to_hex(), + }; + let filter = serde_json::json!({ + "kinds": [KIND_PROJECT], + "authors": [pubkey], + "#d": [slug], + "limit": 1, + }); + let raw = client.query(&filter).await?; + let mut events = parse_events(&raw)?; + events.sort_by_key(|e| std::cmp::Reverse(e.created_at)); + Ok(events.into_iter().next()) +} + +// ── Tag helpers ─────────────────────────────────────────────────────────────── + +fn tag_name(tag: &Tag) -> Option<&str> { + tag.as_slice().first().map(String::as_str) +} + +fn tag_value(tag: &Tag) -> Option<&str> { + tag.as_slice().get(1).map(String::as_str) +} + +fn make_tag(parts: &[&str]) -> Result { + Tag::parse(parts.iter().copied()) + .map_err(|e| CliError::Other(format!("tag construction failed: {e}"))) +} + +// ── Submit helper ───────────────────────────────────────────────────────────── + +async fn submit_project(client: &BuzzClient, builder: EventBuilder) -> Result<(), CliError> { + let event = client.sign_event(builder)?; + let raw = client.submit_event(event).await?; + println!( + "{}", + parse_write_response(&raw, "project changed concurrently; retry")? + ); + Ok(()) +} + +// ── Build helpers ───────────────────────────────────────────────────────────── + +/// Advance the `created_at` counter off an observed head. +fn next_timestamp(head: &Event) -> Result { + head.created_at + .as_secs() + .checked_add(1) + .map(Timestamp::from) + .ok_or_else(|| CliError::Other("project timestamp cannot be advanced".into())) +} + +/// Strip `auth` from a tag list and pass the resulting envelope through +/// Layer A validation. Returns a validated `EventBuilder` at `next_ts`. +fn rebuild_project( + content: &str, + tags: Vec, + next_ts: Timestamp, +) -> Result { + // Strip auth tags. + let clean_tags: Vec = tags + .into_iter() + .filter(|t| tag_name(t) != Some("auth")) + .collect(); + + build_project_with_tags(content, clean_tags) + .map_err(|e| CliError::Other(format!("envelope validation failed: {e}"))) + .map(|b| b.custom_created_at(next_ts)) +} + +// ── Command implementations ─────────────────────────────────────────────────── + +/// `buzz projects create` +pub async fn cmd_create( + client: &BuzzClient, + slug: &str, + repos: &[String], + name: Option<&str>, + description: Option<&str>, + channel: Option<&str>, + visibility: Option<&str>, +) -> Result<(), CliError> { + // ── Local validation (all checks before any .await) ─────────────────── + validate_project_slug(slug)?; + + let caller_pubkey = client.keys().public_key().to_hex(); + + // Expand and validate repo coordinates. + let members: Vec = repos + .iter() + .map(|r| expand_repo_coord(r, &caller_pubkey)) + .collect::, _>>()?; + + // Dedupe: preserve first occurrence, reject duplicates with Usage. + let mut seen = std::collections::HashSet::new(); + for m in &members { + if !seen.insert(m.coord.clone()) { + return Err(CliError::Usage(format!( + "duplicate --repo coordinate in this invocation: {:?}", + m.coord + ))); + } + } + + // Validate optional metadata (early, before any network call). + if let Some(ch) = channel { + crate::validate::validate_uuid(ch)?; + } + if let Some(vis) = visibility { + validate_visibility(vis)?; + } + if let Some(n) = name { + if n.len() > 256 { + return Err(CliError::Usage(format!( + "project name must not exceed 256 bytes (got {})", + n.len() + ))); + } + } + + // ── Network: collision preflight ────────────────────────────────────── + if fetch_own_project(client, slug).await?.is_some() { + return Err(CliError::Conflict(format!( + "project {slug:?} already exists; use 'buzz projects update' to modify it" + ))); + } + + // ── Build via Layer B (enforces all writer policy) ──────────────────── + let builder = build_project(slug, name, description, &members, channel, visibility) + .map_err(|e| CliError::Usage(e.to_string()))?; + submit_project(client, builder).await +} + +/// `buzz projects get` +pub async fn cmd_get(client: &BuzzClient, slug: &str, owner: Option<&str>) -> Result<(), CliError> { + validate_project_slug(slug)?; + let resp = match fetch_project(client, slug, owner).await? { + Some(event) => serde_json::json!({ + "event_id": event.id.to_hex(), + "pubkey": event.pubkey.to_hex(), + "created_at": event.created_at.as_secs(), + "kind": event.kind.as_u16(), + "tags": event.tags.iter().map(|t| t.as_slice().to_vec()).collect::>(), + "content": event.content, + }), + None => { + let owner_desc = owner.unwrap_or("current identity"); + return Err(CliError::NotFound(format!( + "project {slug:?} not found for {owner_desc}" + ))); + } + }; + println!("{resp}"); + Ok(()) +} + +/// `buzz projects list` +pub async fn cmd_list( + client: &BuzzClient, + owner: Option<&str>, + limit: Option, +) -> Result<(), CliError> { + let pubkey = match owner { + Some(pk) => { + crate::validate::validate_hex64(pk)?; + pk.to_string() + } + None => client.keys().public_key().to_hex(), + }; + let mut filter = serde_json::json!({ + "kinds": [KIND_PROJECT], + "authors": [pubkey], + }); + if let Some(n) = limit { + filter["limit"] = serde_json::json!(n); + } + let resp = client.query(&filter).await?; + println!("{resp}"); + Ok(()) +} + +/// `buzz projects add-repo` +pub async fn cmd_add_repo( + client: &BuzzClient, + slug: &str, + repos: &[String], +) -> Result<(), CliError> { + validate_project_slug(slug)?; + let caller_pubkey = client.keys().public_key().to_hex(); + + // ── Local validation before any .await ──────────────────────────────── + let new_members: Vec = repos + .iter() + .map(|r| expand_repo_coord(r, &caller_pubkey)) + .collect::, _>>()?; + + // Dedupe within this invocation: first occurrence wins, duplicate → Usage. + let mut seen = std::collections::HashSet::new(); + for m in &new_members { + if !seen.insert(m.coord.clone()) { + return Err(CliError::Usage(format!( + "duplicate --repo coordinate in this invocation: {:?}", + m.coord + ))); + } + } + + // ── Network: fetch head ─────────────────────────────────────────────── + let head = fetch_own_project(client, slug) + .await? + .ok_or_else(|| CliError::NotFound(format!("project {slug:?} not found")))?; + let next_ts = next_timestamp(&head)?; + + // Build the new tag set: keep existing tags (including hinted members), + // append new members only if not already present (by coordinate). + let mut tags: Vec = head.tags.iter().cloned().collect(); + let existing_coords: std::collections::HashSet = head + .tags + .iter() + .filter(|t| tag_name(t) == Some("a")) + .filter_map(|t| tag_value(t).map(String::from)) + .collect(); + let mut added = 0usize; + for m in &new_members { + if !existing_coords.contains(m.coord.as_str()) { + let parts = m.to_tag_parts(); + let parts_ref: Vec<&str> = parts.iter().map(String::as_str).collect(); + tags.push( + Tag::parse(parts_ref.iter().copied()) + .map_err(|e| CliError::Other(format!("member tag construction failed: {e}")))?, + ); + added += 1; + } + } + + // All requested coordinates were already present — no change to publish. + if added == 0 { + return Err(CliError::Conflict(format!( + "all requested repositories are already members of project {slug:?}" + ))); + } + + let builder = rebuild_project(&head.content, tags, next_ts)?; + submit_project(client, builder).await +} + +/// `buzz projects remove-repo` +pub async fn cmd_remove_repo( + client: &BuzzClient, + slug: &str, + repos: &[String], +) -> Result<(), CliError> { + validate_project_slug(slug)?; + let caller_pubkey = client.keys().public_key().to_hex(); + + // ── Local validation before any .await ──────────────────────────────── + let to_remove: Vec = repos + .iter() + .map(|r| expand_repo_coord(r, &caller_pubkey)) + .collect::, _>>()?; + + // ── Network: fetch head ─────────────────────────────────────────────── + let head = fetch_own_project(client, slug) + .await? + .ok_or_else(|| CliError::NotFound(format!("project {slug:?} not found")))?; + let next_ts = next_timestamp(&head)?; + + // Verify all requested repos exist in the project. + let existing_coords: std::collections::HashSet = head + .tags + .iter() + .filter(|t| tag_name(t) == Some("a")) + .filter_map(|t| tag_value(t).map(String::from)) + .collect(); + for m in &to_remove { + if !existing_coords.contains(m.coord.as_str()) { + return Err(CliError::NotFound(format!( + "project {slug:?} does not contain member {:?}", + m.coord + ))); + } + } + + let remove_coords: std::collections::HashSet<&str> = + to_remove.iter().map(|m| m.coord.as_str()).collect(); + + // Keep all tags except auth and the removed members. + let tags: Vec = head + .tags + .iter() + .filter(|t| { + if tag_name(t) == Some("auth") { + return false; + } + if tag_name(t) == Some("a") { + if let Some(coord) = tag_value(t) { + return !remove_coords.contains(coord); + } + } + true + }) + .cloned() + .collect(); + + // Single rebuild validates the full envelope and strips any remaining auth. + let builder = rebuild_project(&head.content, tags, next_ts)?; + submit_project(client, builder).await +} + +/// `buzz projects update` +/// +/// Requires at least one setter or clearer; a no-op call is a usage error. +#[allow(clippy::too_many_arguments)] +pub async fn cmd_update( + client: &BuzzClient, + slug: &str, + name: Option<&str>, + clear_name: bool, + description: Option<&str>, + clear_description: bool, + channel: Option<&str>, + clear_channel: bool, + visibility: Option<&str>, + clear_visibility: bool, +) -> Result<(), CliError> { + // Guard: at least one mutation required. The clap `ArgGroup` with + // `required(true).multiple(true)` enforces this at parse time; this + // runtime check is a defense-in-depth safety net for callers that invoke + // `cmd_update` directly (e.g. tests and future programmatic callers). + let has_mutation = name.is_some() + || clear_name + || description.is_some() + || clear_description + || channel.is_some() + || clear_channel + || visibility.is_some() + || clear_visibility; + if !has_mutation { + return Err(CliError::Usage( + "buzz projects update requires at least one of: \ + --name, --clear-name, --description, --clear-description, \ + --channel, --clear-channel, --visibility, --clear-visibility" + .into(), + )); + } + + validate_project_slug(slug)?; + if let Some(ch) = channel { + crate::validate::validate_uuid(ch)?; + } + if let Some(vis) = visibility { + validate_visibility(vis)?; + } + + let head = fetch_own_project(client, slug) + .await? + .ok_or_else(|| CliError::NotFound(format!("project {slug:?} not found")))?; + let next_ts = next_timestamp(&head)?; + + // Build the new tag set. For each singleton metadata field: + // - setter present: replace value (strip old, append new) + // - clear flag set: drop the tag + // - neither: keep existing + // Non-singleton / non-metadata tags (d, a, unknown) are preserved as-is. + let singleton_fields = ["name", "description", "buzz-channel", "buzz-visibility"]; + let mut tags: Vec = head + .tags + .iter() + .filter(|t| { + if tag_name(t) == Some("auth") { + return false; + } + // Drop singletons we're replacing or clearing. + if let Some(field) = tag_name(t) { + if singleton_fields.contains(&field) { + let clear = match field { + "name" => clear_name || name.is_some(), + "description" => clear_description || description.is_some(), + "buzz-channel" => clear_channel || channel.is_some(), + "buzz-visibility" => clear_visibility || visibility.is_some(), + _ => false, + }; + return !clear; + } + } + true + }) + .cloned() + .collect(); + + // Append new singleton values. + if let Some(n) = name { + tags.push(make_tag(&["name", n])?); + } + if let Some(d) = description { + tags.push(make_tag(&["description", d])?); + } + if let Some(ch) = channel { + tags.push(make_tag(&["buzz-channel", ch])?); + } + if let Some(vis) = visibility { + tags.push(make_tag(&["buzz-visibility", vis])?); + } + + let builder = build_project_with_tags(&head.content, tags) + .map_err(|e| CliError::Other(format!("envelope validation failed: {e}")))? + .custom_created_at(next_ts); + submit_project(client, builder).await +} + +/// `buzz projects delete` +/// +/// Head-based and verified: +/// 1. Fetch own live head — `NotFound` if absent. +/// 2. Build tombstone at `head.created_at + 1`. +/// 3. Submit. +/// 4. Re-query the coordinate; if a newer head survived → `Conflict`. +pub async fn cmd_delete(client: &BuzzClient, slug: &str) -> Result<(), CliError> { + validate_project_slug(slug)?; + + let head = fetch_own_project(client, slug) + .await? + .ok_or_else(|| CliError::NotFound(format!("project {slug:?} not found")))?; + let next_ts = next_timestamp(&head)?; + + let pubkey_hex = client.keys().public_key().to_hex(); + let tombstone = build_delete_addressable(KIND_PROJECT, &pubkey_hex, slug) + .map_err(|e| CliError::Other(format!("failed to build delete event: {e}")))? + .custom_created_at(next_ts); + + let event = client.sign_event(tombstone)?; + let raw = client.submit_event(event).await?; + parse_write_response(&raw, "delete event was dominated; a newer head exists")?; + + // Post-submit verification: re-query to confirm the head is gone. + if let Some(survivor) = fetch_own_project(client, slug).await? { + // A newer head survived the tombstone. + return Err(CliError::Conflict(format!( + "project {slug:?} still exists (head at {}); a concurrent write raced the delete", + survivor.created_at.as_secs() + ))); + } + + println!("{}", serde_json::json!({ "deleted": slug, "status": "ok" })); + Ok(()) +} + +// ── Validation helpers ──────────────────────────────────────────────────────── + +/// Validate a project slug: non-empty, ≤1024 bytes, verbatim. +/// Does NOT impose the Buzz repo-ID grammar — project slugs are more permissive. +fn validate_project_slug(slug: &str) -> Result<(), CliError> { + if slug.is_empty() { + return Err(CliError::Usage("project slug must not be empty".into())); + } + if slug.len() > PROJECT_D_MAX_LEN { + return Err(CliError::Usage(format!( + "project slug must not exceed {PROJECT_D_MAX_LEN} bytes (got {})", + slug.len() + ))); + } + Ok(()) +} + +/// Validate a `buzz-visibility` value at the writer level. +fn validate_visibility(vis: &str) -> Result<(), CliError> { + if vis != "listed" && vis != "unlisted" { + return Err(CliError::Usage(format!( + "visibility must be 'listed' or 'unlisted' (got {vis:?})" + ))); + } + Ok(()) +} + +// ── Dispatch ────────────────────────────────────────────────────────────────── + +pub async fn dispatch(cmd: crate::ProjectsCmd, client: &BuzzClient) -> Result<(), CliError> { + use crate::ProjectsCmd; + match cmd { + ProjectsCmd::Create { + slug, + repo, + name, + description, + channel, + visibility, + } => { + cmd_create( + client, + &slug, + &repo, + name.as_deref(), + description.as_deref(), + channel.as_deref(), + visibility.map(|v| v.as_str()), + ) + .await + } + ProjectsCmd::Get { slug, owner } => cmd_get(client, &slug, owner.as_deref()).await, + ProjectsCmd::List { owner, limit } => cmd_list(client, owner.as_deref(), limit).await, + ProjectsCmd::AddRepo { slug, repo } => cmd_add_repo(client, &slug, &repo).await, + ProjectsCmd::RemoveRepo { slug, repo } => cmd_remove_repo(client, &slug, &repo).await, + ProjectsCmd::Update { + slug, + name, + clear_name, + description, + clear_description, + channel, + clear_channel, + visibility, + clear_visibility, + } => { + cmd_update( + client, + &slug, + name.as_deref(), + clear_name, + description.as_deref(), + clear_description, + channel.as_deref(), + clear_channel, + visibility.map(|v| v.as_str()), + clear_visibility, + ) + .await + } + ProjectsCmd::Delete { slug } => cmd_delete(client, &slug).await, + } +} + +// ── Tests ───────────────────────────────────────────────────────────────────── + +#[cfg(test)] +mod tests { + use buzz_sdk::{validate_project_envelope, PROJECT_MEMBER_CAP}; + use nostr::Tag; + + use super::*; + + // ── Coordinate expansion ────────────────────────────────────────────────── + + const OWNER_HEX: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + const OWNER_B_HEX: &str = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; + + #[test] + fn expand_repo_coord_bare_expands_with_caller_pubkey() { + let coord = expand_repo_coord("my-repo", OWNER_HEX).unwrap(); + assert_eq!(coord.coord, format!("30617:{OWNER_HEX}:my-repo")); + } + + #[test] + fn expand_repo_coord_full_passes_through() { + let full = format!("30617:{OWNER_HEX}:some-repo"); + let coord = expand_repo_coord(&full, OWNER_B_HEX).unwrap(); + // Owner from the full coord, not the caller. + assert_eq!(coord.coord, full); + } + + #[test] + fn expand_repo_coord_full_cross_owner() { + let full = format!("30617:{OWNER_B_HEX}:infra"); + let coord = expand_repo_coord(&full, OWNER_HEX).unwrap(); + assert_eq!(coord.coord, full); + } + + #[test] + fn expand_repo_coord_rejects_uppercase_owner() { + let upper = "30617:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA:buzz"; + assert!(expand_repo_coord(upper, OWNER_HEX).is_err()); + } + + #[test] + fn expand_repo_coord_rejects_coordinate_shaped_bare_value() { + // A value with a colon is never a bare id. + let not_bare = "30617:something"; + // parse_full will fail because it's not a valid full coordinate either. + assert!(expand_repo_coord(not_bare, OWNER_HEX).is_err()); + } + + // ── validate_project_slug ───────────────────────────────────────────────── + + #[test] + fn validate_project_slug_accepts_normal() { + assert!(validate_project_slug("my-project").is_ok()); + assert!(validate_project_slug("platform:v2").is_ok()); // colons allowed — more permissive than repo-id + } + + #[test] + fn validate_project_slug_rejects_empty() { + assert!(validate_project_slug("").is_err()); + } + + #[test] + fn validate_project_slug_rejects_over_1024() { + let long = "a".repeat(1025); + assert!(validate_project_slug(&long).is_err()); + } + + #[test] + fn validate_project_slug_accepts_1024() { + let at_limit = "a".repeat(1024); + assert!(validate_project_slug(&at_limit).is_ok()); + } + + // ── validate_visibility ─────────────────────────────────────────────────── + + #[test] + fn validate_visibility_accepts_listed_and_unlisted() { + assert!(validate_visibility("listed").is_ok()); + assert!(validate_visibility("unlisted").is_ok()); + } + + #[test] + fn validate_visibility_rejects_unknown_token() { + assert!(validate_visibility("chartreuse").is_err()); + assert!(validate_visibility("").is_err()); + } + + // ── is_bare_repo_id ─────────────────────────────────────────────────────── + + #[test] + fn bare_repo_id_accepts_valid() { + assert!(is_bare_repo_id("buzz")); + assert!(is_bare_repo_id("my-repo_1.0")); + } + + #[test] + fn bare_repo_id_rejects_colon() { + assert!(!is_bare_repo_id("30617:something")); + assert!(!is_bare_repo_id("has:colon")); + } + + #[test] + fn bare_repo_id_rejects_empty() { + assert!(!is_bare_repo_id("")); + } + + #[test] + fn bare_repo_id_rejects_over_64() { + let long = "a".repeat(65); + assert!(!is_bare_repo_id(&long)); + } + + // ── tag helpers ─────────────────────────────────────────────────────────── + + fn make_test_tag(parts: &[&str]) -> Tag { + Tag::parse(parts.iter().copied()).unwrap() + } + + // ── rebuild_project: hinted / unknown tag preservation ─────────────────── + + #[test] + fn rebuild_project_preserves_hinted_member_tags() { + // A member 'a' tag with a relay hint must survive RMW untouched. + let coord = format!("30617:{OWNER_HEX}:buzz"); + let hint = "wss://relay.example.com"; + let tags = vec![ + make_test_tag(&["d", "platform"]), + Tag::parse(["a", &coord, hint]).unwrap(), + ]; + let ts = Timestamp::from(1_700_000_001u64); + let b = rebuild_project("", tags, ts).unwrap(); + let ev = b.sign_with_keys(&nostr::Keys::generate()).expect("sign"); + let a_tag = ev + .tags + .iter() + .find(|t| tag_name(t) == Some("a")) + .expect("a tag present"); + assert_eq!( + a_tag.as_slice(), + &["a".to_string(), coord, hint.to_string()], + "relay hint must survive rebuild" + ); + } + + #[test] + fn rebuild_project_preserves_unknown_tags() { + let tags = vec![ + make_test_tag(&["d", "platform"]), + make_test_tag(&["future-metadata", "value"]), + ]; + let ts = Timestamp::from(1_700_000_001u64); + let b = rebuild_project("", tags, ts).unwrap(); + let ev = b.sign_with_keys(&nostr::Keys::generate()).expect("sign"); + assert!(ev + .tags + .iter() + .any(|t| tag_name(t) == Some("future-metadata"))); + } + + #[test] + fn rebuild_project_strips_auth_tag() { + let tags = vec![ + make_test_tag(&["d", "platform"]), + make_test_tag(&["auth", &"a".repeat(64), "kind=30617", &"b".repeat(128)]), + ]; + let ts = Timestamp::from(1_700_000_001u64); + let b = rebuild_project("", tags, ts).unwrap(); + let ev = b.sign_with_keys(&nostr::Keys::generate()).expect("sign"); + assert!( + !ev.tags.iter().any(|t| tag_name(t) == Some("auth")), + "auth tag must be stripped" + ); + } + + #[test] + fn rebuild_project_rejects_over_cap_foreign_head() { + // A foreign head with 65 members must fail Layer A on republish. + let mut tags = vec![make_test_tag(&["d", "wide"])]; + for i in 0..=64u32 { + let coord = format!("30617:{OWNER_HEX}:repo-{i:02}"); + tags.push(make_test_tag(&["a", &coord])); + } + assert_eq!( + tags.iter().filter(|t| tag_name(t) == Some("a")).count(), + 65, + "65 a-tags" + ); + let ts = Timestamp::from(1_700_000_001u64); + // rebuild_project strips auth, but 65 a-tags still exceeds cap. + assert!( + rebuild_project("", tags, ts).is_err(), + "over-cap foreign head must fail rebuild" + ); + } + + #[test] + fn rebuild_project_at_exact_cap_succeeds() { + let mut tags = vec![make_test_tag(&["d", "wide"])]; + for i in 0..PROJECT_MEMBER_CAP { + let coord = format!("30617:{OWNER_HEX}:repo-{i:02}"); + tags.push(make_test_tag(&["a", &coord])); + } + let ts = Timestamp::from(1_700_000_001u64); + assert!(rebuild_project("", tags, ts).is_ok()); + } + + // ── clear-flag semantics ────────────────────────────────────────────────── + + /// Build a minimal head Event for testing update semantics without the relay. + fn make_head_tags(extra: &[Tag]) -> Vec { + let mut tags = vec![make_test_tag(&["d", "platform"])]; + tags.extend_from_slice(extra); + tags + } + + #[allow(clippy::too_many_arguments)] + fn apply_update_tags( + head_tags: Vec, + name: Option<&str>, + clear_name: bool, + description: Option<&str>, + clear_description: bool, + channel: Option<&str>, + clear_channel: bool, + visibility: Option<&str>, + clear_visibility: bool, + ) -> Vec { + // Replicate the tag-mutation logic from cmd_update (sans relay I/O). + let singleton_fields = ["name", "description", "buzz-channel", "buzz-visibility"]; + let mut tags: Vec = head_tags + .iter() + .filter(|t| { + if tag_name(t) == Some("auth") { + return false; + } + if let Some(field) = tag_name(t) { + if singleton_fields.contains(&field) { + let clear = match field { + "name" => clear_name || name.is_some(), + "description" => clear_description || description.is_some(), + "buzz-channel" => clear_channel || channel.is_some(), + "buzz-visibility" => clear_visibility || visibility.is_some(), + _ => false, + }; + return !clear; + } + } + true + }) + .cloned() + .collect(); + if let Some(n) = name { + tags.push(make_test_tag(&["name", n])); + } + if let Some(d) = description { + tags.push(make_test_tag(&["description", d])); + } + if let Some(ch) = channel { + tags.push(make_test_tag(&["buzz-channel", ch])); + } + if let Some(vis) = visibility { + tags.push(make_test_tag(&["buzz-visibility", vis])); + } + tags + } + + #[test] + fn update_omission_preserves_existing_field() { + let head = make_head_tags(&[make_test_tag(&["name", "Old Name"])]); + let result = apply_update_tags(head, None, false, None, false, None, false, None, false); + assert!(result.iter().any(|t| tag_value(t) == Some("Old Name"))); + } + + #[test] + fn update_setter_replaces_existing_field() { + let head = make_head_tags(&[make_test_tag(&["name", "Old Name"])]); + let result = apply_update_tags( + head, + Some("New Name"), + false, + None, + false, + None, + false, + None, + false, + ); + assert!(result.iter().any(|t| tag_value(t) == Some("New Name"))); + assert!(!result.iter().any(|t| tag_value(t) == Some("Old Name"))); + } + + #[test] + fn update_clear_drops_existing_field() { + let head = make_head_tags(&[make_test_tag(&["name", "Old Name"])]); + let result = apply_update_tags(head, None, true, None, false, None, false, None, false); + assert!(!result.iter().any(|t| tag_name(t) == Some("name"))); + } + + #[test] + fn update_clear_visibility_drops_tag() { + let head = make_head_tags(&[make_test_tag(&["buzz-visibility", "unlisted"])]); + let result = apply_update_tags(head, None, false, None, false, None, false, None, true); + assert!(!result + .iter() + .any(|t| tag_name(t) == Some("buzz-visibility"))); + } + + #[test] + fn update_exactly_one_singleton_after_replace() { + // Start with a buzz-channel; replace with a new one; must have exactly one. + let uuid1 = "3580ca9b-47b4-4af9-b22a-1068778f26c6"; + let uuid2 = "00000000-0000-0000-0000-000000000000"; + let head = make_head_tags(&[make_test_tag(&["buzz-channel", uuid1])]); + let result = apply_update_tags( + head, + None, + false, + None, + false, + Some(uuid2), + false, + None, + false, + ); + let channels: Vec<_> = result + .iter() + .filter(|t| tag_name(t) == Some("buzz-channel")) + .collect(); + assert_eq!(channels.len(), 1); + assert_eq!(tag_value(channels[0]), Some(uuid2)); + } + + // ── duplicate-member rejection on republish ─────────────────────────────── + + #[test] + fn duplicate_member_in_foreign_head_fails_rebuild() { + let coord = format!("30617:{OWNER_HEX}:buzz"); + let tags = vec![ + make_test_tag(&["d", "platform"]), + make_test_tag(&["a", &coord]), + make_test_tag(&["a", &coord]), // duplicate + ]; + let ts = Timestamp::from(1_700_000_001u64); + assert!(rebuild_project("", tags, ts).is_err()); + } + + // ── validate_project_envelope integration ──────────────────────────────── + + #[test] + fn validate_project_envelope_accepts_hinted_member() { + let coord = format!("30617:{OWNER_HEX}:buzz"); + let tags = vec![ + make_test_tag(&["d", "platform"]), + Tag::parse(["a", &coord, "wss://relay.example.com"]).unwrap(), + ]; + assert!(validate_project_envelope(&tags, "").is_ok()); + } + + #[test] + fn validate_project_envelope_rejects_four_element_member() { + let coord = format!("30617:{OWNER_HEX}:buzz"); + let tags = vec![ + make_test_tag(&["d", "platform"]), + Tag::parse(["a", &coord, "wss://relay.example.com", "extra"]).unwrap(), + ]; + assert!(validate_project_envelope(&tags, "").is_err()); + } + + // ── next_timestamp ordering ─────────────────────────────────────────────── + + /// `next_timestamp` must return `head.created_at + 1` regardless of the wall + /// clock. NIP-MP Deletion rule: a tombstone older than the live head does + /// NOT remove it, so we must advance strictly off the observed head — never + /// use wall-clock time, which could be behind a head that was bumped + /// multiple times in the same second. + #[test] + fn next_timestamp_returns_head_plus_one_when_head_is_ahead_of_wall_clock() { + // Build a minimal signed event with a created_at far in the future. + let keys = nostr::Keys::generate(); + let far_future_ts = Timestamp::from(9_999_999_999u64); // year 2286 + let tags = vec![ + make_test_tag(&["d", "platform"]), + make_test_tag(&["a", &format!("30617:{OWNER_HEX}:buzz")]), + ]; + let builder = rebuild_project("", tags, far_future_ts).expect("valid head envelope"); + let head = builder.sign_with_keys(&keys).expect("sign"); + // Verify the event actually has our future timestamp. + assert_eq!(head.created_at, far_future_ts); + + // next_timestamp must return far_future + 1, not now(). + let next = next_timestamp(&head).expect("no overflow"); + assert_eq!( + next.as_secs(), + far_future_ts.as_secs() + 1, + "tombstone must be strictly after head, even when head is far in the future" + ); + } + + // ── empty update guard ──────────────────────────────────────────────────── + + /// `cmd_update` with no setters or clearers must return `CliError::Usage` + /// before making any network call. The guard is synchronous (before the + /// first `.await`) so we can drive it with a dummy client whose address + /// would reject any real connection attempt. + #[tokio::test] + async fn empty_update_returns_usage_error_before_any_network_call() { + let keys = nostr::Keys::generate(); + // Port 9 is the discard protocol — any real connect will be refused + // immediately, but the guard fires before the first await so this + // never reaches the network. + let client = crate::client::BuzzClient::new("http://127.0.0.1:9".into(), keys, None, None) + .expect("client construction"); + + let err = cmd_update( + &client, "my-slug", None, false, // name / clear_name + None, false, // description / clear_description + None, false, // channel / clear_channel + None, false, // visibility / clear_visibility + ) + .await + .expect_err("empty update must fail"); + + assert!( + matches!(err, CliError::Usage(_)), + "expected CliError::Usage, got {err:?}" + ); + } + + // ── no-network malformed-input tests ───────────────────────────────────── + // + // All three cases use port 9 (discard protocol): any real connection is + // refused immediately, but local validation fires before the first .await + // so the network is never touched. + + fn discard_client() -> crate::client::BuzzClient { + let keys = nostr::Keys::generate(); + crate::client::BuzzClient::new("http://127.0.0.1:9".into(), keys, None, None) + .expect("client construction") + } + + /// Invalid visibility token must return Usage before touching the relay. + #[tokio::test] + async fn create_invalid_visibility_returns_usage_before_any_network_call() { + let client = discard_client(); + let err = cmd_create( + &client, + "my-slug", + &["buzz".to_string()], + None, + None, + None, + Some("chartreuse"), + ) + .await + .expect_err("invalid visibility must fail"); + assert!( + matches!(err, CliError::Usage(_)), + "expected CliError::Usage for invalid visibility, got {err:?}" + ); + } + + /// A name longer than 256 bytes must return Usage before touching the relay. + #[tokio::test] + async fn create_overlong_name_returns_usage_before_any_network_call() { + let client = discard_client(); + let long_name = "a".repeat(257); + let err = cmd_create( + &client, + "my-slug", + &["buzz".to_string()], + Some(&long_name), + None, + None, + None, + ) + .await + .expect_err("overlong name must fail"); + assert!( + matches!(err, CliError::Usage(_)), + "expected CliError::Usage for overlong name, got {err:?}" + ); + } + + /// A malformed --repo coordinate must return Usage before touching the relay. + #[tokio::test] + async fn create_malformed_repo_returns_usage_before_any_network_call() { + let client = discard_client(); + let err = cmd_create( + &client, + "my-slug", + &["nope:bad".to_string()], + None, + None, + None, + None, + ) + .await + .expect_err("malformed repo must fail"); + assert!( + matches!(err, CliError::Usage(_)), + "expected CliError::Usage for malformed repo, got {err:?}" + ); + } + + /// A malformed --repo coordinate on add-repo must return Usage before touching the relay. + #[tokio::test] + async fn add_repo_malformed_coord_returns_usage_before_any_network_call() { + let client = discard_client(); + let err = cmd_add_repo(&client, "my-slug", &["nope:bad".to_string()]) + .await + .expect_err("malformed repo must fail"); + assert!( + matches!(err, CliError::Usage(_)), + "expected CliError::Usage for malformed repo on add-repo, got {err:?}" + ); + } + + /// A malformed --repo coordinate on remove-repo must return Usage before touching the relay. + #[tokio::test] + async fn remove_repo_malformed_coord_returns_usage_before_any_network_call() { + let client = discard_client(); + let err = cmd_remove_repo(&client, "my-slug", &["nope:bad".to_string()]) + .await + .expect_err("malformed repo must fail"); + assert!( + matches!(err, CliError::Usage(_)), + "expected CliError::Usage for malformed repo on remove-repo, got {err:?}" + ); + } + + // ── duplicate --repo within one invocation ──────────────────────────────── + + /// Supplying the same coordinate twice in one create call must return Usage + /// (names the duplicate) before any network call. + #[tokio::test] + async fn create_duplicate_repo_returns_usage_before_any_network_call() { + let client = discard_client(); + let coord = format!("30617:{OWNER_HEX}:buzz"); + let err = cmd_create( + &client, + "my-slug", + &[coord.clone(), coord.clone()], + None, + None, + None, + None, + ) + .await + .expect_err("duplicate repo must fail"); + assert!( + matches!(err, CliError::Usage(_)), + "expected CliError::Usage for duplicate repo, got {err:?}" + ); + // Error message must name the duplicate coordinate. + assert!( + format!("{err}").contains("buzz"), + "Usage message must name the duplicate coordinate, got {err:?}" + ); + } + + /// Supplying the same coordinate twice in one add-repo call must return Usage + /// (names the duplicate) before any network call. + #[tokio::test] + async fn add_repo_duplicate_coord_returns_usage_before_any_network_call() { + let client = discard_client(); + let coord = format!("30617:{OWNER_HEX}:buzz"); + let err = cmd_add_repo(&client, "my-slug", &[coord.clone(), coord.clone()]) + .await + .expect_err("duplicate repo must fail"); + assert!( + matches!(err, CliError::Usage(_)), + "expected CliError::Usage for duplicate repo on add-repo, got {err:?}" + ); + } + + // ── create collision guard ──────────────────────────────────────────────── + + // The create-collision Conflict path is pinned by the live transcript + // (step: duplicate create → Conflict, exit=5). No relay mock is available + // for a unit test; the no-network tests above cover all pre-await paths. + + // ── add-repo no-op guard ────────────────────────────────────────────────── + + // The add-repo no-op Conflict path is pinned by the live transcript + // (step 7: buzz already present → exit=5). No relay mock is available + // for a unit test; the async no-network tests above cover all pre-await paths. +} diff --git a/crates/buzz-cli/src/commands/repos.rs b/crates/buzz-cli/src/commands/repos.rs index 0f570df1aa..15e064d9c3 100644 --- a/crates/buzz-cli/src/commands/repos.rs +++ b/crates/buzz-cli/src/commands/repos.rs @@ -4,7 +4,8 @@ use buzz_core::{ }; use nostr::{Event, EventBuilder, Tag, Timestamp}; -use crate::client::{normalize_write_response, BuzzClient}; +use crate::client::BuzzClient; +use crate::commands::parse_write_response; use crate::error::CliError; use crate::validate::validate_repo_id; @@ -83,27 +84,36 @@ fn build_protection_tag( Tag::parse(values).map_err(tag_error) } -enum ProtectionChange { - Set(Box), - Remove(String), +enum RepoChange { + SetProtection(Box), + RemoveProtection(String), + /// Bind (or rebind) the repo to a channel: replaces every existing + /// `buzz-channel` tag with exactly one carrying the validated UUID. + BindChannel(String), } fn build_updated_repo_announcement( existing: &Event, - change: ProtectionChange, + change: RepoChange, ) -> Result { let repo_id = repo_id_from_event(existing)?; - let (pattern, replacement) = match change { - ProtectionChange::Set(tag) => { + // What to strip beyond `auth` (always stripped), and what to append. + let (removed_pattern, removed_channel, replacement) = match change { + RepoChange::SetProtection(tag) => { let pattern = protection_pattern(&tag) .ok_or_else(|| CliError::Other("replacement is not a protection tag".into()))? .to_string(); - (pattern, Some(*tag)) + (Some(pattern), false, Some(*tag)) } - ProtectionChange::Remove(pattern) => { + RepoChange::RemoveProtection(pattern) => { RefPattern::parse(&pattern) .map_err(|error| CliError::Usage(format!("invalid ref pattern: {error}")))?; - (pattern, None) + (Some(pattern), false, None) + } + RepoChange::BindChannel(channel) => { + crate::validate::validate_uuid(&channel)?; + let tag = Tag::parse(["buzz-channel", channel.as_str()]).map_err(tag_error)?; + (None, true, Some(tag)) } }; @@ -111,7 +121,13 @@ fn build_updated_repo_announcement( .tags .iter() .filter(|tag| { - !has_tag_name(tag, "auth") && protection_pattern(tag) != Some(pattern.as_str()) + if has_tag_name(tag, "auth") { + return false; + } + if removed_channel && has_tag_name(tag, "buzz-channel") { + return false; + } + removed_pattern.is_none() || protection_pattern(tag) != removed_pattern.as_deref() }) .cloned() .collect(); @@ -171,25 +187,10 @@ fn protection_rules_json(event: &Event) -> Result { } fn validate_write_response(raw: &str) -> Result { - let response: serde_json::Value = serde_json::from_str(raw) - .map_err(|error| CliError::Other(format!("relay response is not JSON: {error} ({raw})")))?; - let accepted = response - .get("accepted") - .and_then(serde_json::Value::as_bool) - .unwrap_or(false); - let message = response - .get("message") - .and_then(serde_json::Value::as_str) - .unwrap_or(""); - if !accepted { - return Err(CliError::Other(format!("relay rejected event: {message}"))); - } - if message == "duplicate" || message.starts_with("duplicate:") { - return Err(CliError::Conflict( - "repository changed concurrently; fetch the latest rules and retry".into(), - )); - } - Ok(normalize_write_response(raw)) + parse_write_response( + raw, + "repository changed concurrently; fetch the latest rules and retry", + ) } async fn submit_repo_update(client: &BuzzClient, builder: EventBuilder) -> Result<(), CliError> { @@ -199,21 +200,30 @@ async fn submit_repo_update(client: &BuzzClient, builder: EventBuilder) -> Resul Ok(()) } -pub async fn cmd_create_repo( - client: &BuzzClient, +/// Build the kind:30617 announcement for `repos create`, including the +/// `buzz-channel` binding when requested. +/// +/// Pure (no I/O) so the emitted tags are unit-testable. Exactly one +/// validated `buzz-channel` tag is appended — the tag is the git ACL +/// (issue #3527: without it the relay 404s every clone/fetch/push), so the +/// UUID is shape-validated here and its existence/membership is the relay's +/// authority at git-access time, same posture as `repos bind`. +#[allow(clippy::too_many_arguments)] +fn build_create_announcement( repo_id: &str, name: Option<&str>, description: Option<&str>, clone_urls: &[String], web_url: Option<&str>, relays: &[String], -) -> Result<(), CliError> { + channel: Option<&str>, +) -> Result { validate_repo_id(repo_id)?; let clone_refs: Vec<&str> = clone_urls.iter().map(|s| s.as_str()).collect(); let relay_refs: Vec<&str> = relays.iter().map(|s| s.as_str()).collect(); - let builder = buzz_sdk::build_repo_announcement( + let mut builder = buzz_sdk::build_repo_announcement( repo_id, name, description, @@ -223,6 +233,33 @@ pub async fn cmd_create_repo( ) .map_err(|e| CliError::Other(format!("build_repo_announcement failed: {e}")))?; + if let Some(channel) = channel { + crate::validate::validate_uuid(channel)?; + builder = builder.tag(Tag::parse(["buzz-channel", channel]).map_err(tag_error)?); + } + Ok(builder) +} + +#[allow(clippy::too_many_arguments)] +pub async fn cmd_create_repo( + client: &BuzzClient, + repo_id: &str, + name: Option<&str>, + description: Option<&str>, + clone_urls: &[String], + web_url: Option<&str>, + relays: &[String], + channel: Option<&str>, +) -> Result<(), CliError> { + let builder = build_create_announcement( + repo_id, + name, + description, + clone_urls, + web_url, + relays, + channel, + )?; let event = client.sign_event(builder)?; let resp = client.submit_event(event).await?; println!("{resp}"); @@ -320,7 +357,8 @@ async fn cmd_protect_set( require_patch, )?; let event = current_repo(client, repo_id).await?; - let builder = build_updated_repo_announcement(&event, ProtectionChange::Set(Box::new(tag)))?; + let builder = + build_updated_repo_announcement(&event, RepoChange::SetProtection(Box::new(tag)))?; submit_repo_update(client, builder).await } @@ -341,8 +379,27 @@ async fn cmd_protect_remove( "repository {repo_id:?} has no protection rule for {ref_pattern:?}" ))); } + let builder = build_updated_repo_announcement( + &event, + RepoChange::RemoveProtection(ref_pattern.to_string()), + )?; + submit_repo_update(client, builder).await +} + +/// Bind (or rebind) a repository to a channel — the fix path for issue +/// #3527's permanently-404 repos. Publishes a read-modify-write update of +/// the caller's own kind:30617 with exactly one `buzz-channel` tag; all +/// other metadata (protections, name, description, future tags) is +/// preserved by the same machinery `repos protect` uses. +/// +/// The UUID is validated for *shape* only — deliberately. Channel existence +/// and the caller's membership are the relay's authority at git-access +/// time; a CLI-side network pre-check would just be TOCTOU with extra +/// latency. +async fn cmd_bind_repo(client: &BuzzClient, repo_id: &str, channel: &str) -> Result<(), CliError> { + let event = current_repo(client, repo_id).await?; let builder = - build_updated_repo_announcement(&event, ProtectionChange::Remove(ref_pattern.to_string()))?; + build_updated_repo_announcement(&event, RepoChange::BindChannel(channel.to_string()))?; submit_repo_update(client, builder).await } @@ -356,6 +413,7 @@ pub async fn dispatch(cmd: crate::ReposCmd, client: &BuzzClient) -> Result<(), C clone_urls, web, relays, + channel, } => { cmd_create_repo( client, @@ -365,11 +423,13 @@ pub async fn dispatch(cmd: crate::ReposCmd, client: &BuzzClient) -> Result<(), C &clone_urls, web.as_deref(), &relays, + channel.as_deref(), ) .await } ReposCmd::Get { id, owner } => cmd_get_repo(client, &id, owner.as_deref()).await, ReposCmd::List { owner, limit } => cmd_list_repos(client, owner.as_deref(), limit).await, + ReposCmd::Bind { id, channel } => cmd_bind_repo(client, &id, &channel).await, ReposCmd::Protect(command) => match command { ReposProtectCmd::List { id } => cmd_protect_list(client, &id).await, ReposProtectCmd::Set { @@ -403,8 +463,8 @@ mod tests { use nostr::{EventBuilder, Keys, Kind, Tag, Timestamp}; use super::{ - build_protection_tag, build_updated_repo_announcement, protection_rules_json, - validate_write_response, ProtectionChange, + build_create_announcement, build_protection_tag, build_updated_repo_announcement, + protection_rules_json, validate_write_response, RepoChange, }; fn signed_repo(tags: Vec, content: &str, created_at: u64) -> nostr::Event { @@ -439,7 +499,7 @@ mod tests { let updated = build_updated_repo_announcement( &existing, - ProtectionChange::Set(Box::new(replacement)), + RepoChange::SetProtection(Box::new(replacement)), ) .expect("build update") .sign_with_keys(&Keys::generate()) @@ -501,7 +561,7 @@ mod tests { let updated = build_updated_repo_announcement( &existing, - ProtectionChange::Remove("refs/heads/main".into()), + RepoChange::RemoveProtection("refs/heads/main".into()), ) .expect("build removal") .sign_with_keys(&Keys::generate()) @@ -538,7 +598,7 @@ mod tests { let error = build_updated_repo_announcement( &existing, - ProtectionChange::Set(Box::new(replacement)), + RepoChange::SetProtection(Box::new(replacement)), ) .expect_err("malformed existing rule must fail closed"); @@ -564,7 +624,7 @@ mod tests { let error = build_updated_repo_announcement( &existing, - ProtectionChange::Set(Box::new(replacement)), + RepoChange::SetProtection(Box::new(replacement)), ) .expect_err("the 51st rule must be rejected"); @@ -615,6 +675,145 @@ mod tests { .is_some_and(|error| error.contains("needs pattern + at least one rule"))); } + #[test] + fn bind_channel_replaces_duplicates_and_preserves_everything_else() { + let channel = uuid::Uuid::new_v4().to_string(); + let existing = signed_repo( + vec![ + tag(&["d", "demo"]), + tag(&["name", "Demo"]), + // Two stale bindings — e.g. from a buggy or vanilla client. + tag(&["buzz-channel", "old-and-broken"]), + tag(&["buzz-channel", &uuid::Uuid::new_v4().to_string()]), + tag(&["auth", &"a".repeat(64), "kind=30617", &"b".repeat(128)]), + tag(&["buzz-protect", "refs/heads/main", "push:admin"]), + tag(&["future-metadata", "preserve-me"]), + ], + "repository content", + 100, + ); + + let updated = + build_updated_repo_announcement(&existing, RepoChange::BindChannel(channel.clone())) + .expect("build bind update") + .sign_with_keys(&Keys::generate()) + .expect("sign bind update"); + + assert_eq!(updated.content, "repository content"); + assert_eq!(updated.created_at.as_secs(), 101); + // Exactly one binding remains, and it is the requested one. + let bindings: Vec<_> = updated + .tags + .iter() + .filter(|tag| tag.as_slice().first().map(String::as_str) == Some("buzz-channel")) + .collect(); + assert_eq!(bindings.len(), 1); + assert_eq!(bindings[0].as_slice(), ["buzz-channel", channel.as_str()]); + // Auth stripped (relay re-stamps); everything else preserved. + assert!(!updated + .tags + .iter() + .any(|tag| tag.as_slice().first().map(String::as_str) == Some("auth"))); + assert!(updated + .tags + .iter() + .any(|tag| tag.as_slice() == ["buzz-protect", "refs/heads/main", "push:admin"])); + assert!(updated + .tags + .iter() + .any(|tag| tag.as_slice() == ["future-metadata", "preserve-me"])); + assert!(updated + .tags + .iter() + .any(|tag| tag.as_slice() == ["name", "Demo"])); + } + + #[test] + fn bind_channel_adds_binding_to_unbound_repo() { + let channel = uuid::Uuid::new_v4().to_string(); + let existing = signed_repo(vec![tag(&["d", "demo"])], "", 10); + + let updated = + build_updated_repo_announcement(&existing, RepoChange::BindChannel(channel.clone())) + .expect("build bind update") + .sign_with_keys(&Keys::generate()) + .expect("sign bind update"); + + assert!(updated + .tags + .iter() + .any(|tag| tag.as_slice() == ["buzz-channel", channel.as_str()])); + } + + #[test] + fn bind_channel_rejects_malformed_uuid() { + let existing = signed_repo(vec![tag(&["d", "demo"])], "", 10); + + let error = + build_updated_repo_announcement(&existing, RepoChange::BindChannel("nope".into())) + .expect_err("malformed channel id must not build an update"); + + assert!(matches!(error, crate::error::CliError::Usage(_))); + } + + /// Issue #3527: `repos create --channel` must emit exactly one + /// `buzz-channel` tag so the primary create command stops producing + /// repos the relay 404s forever. + #[test] + fn create_with_channel_emits_exactly_one_binding_tag() { + let channel = uuid::Uuid::new_v4().to_string(); + let event = build_create_announcement( + "demo", + Some("Demo"), + None, + &["https://relay.example/git/owner/demo".to_string()], + None, + &[], + Some(&channel), + ) + .expect("build create announcement") + .sign_with_keys(&Keys::generate()) + .expect("sign create announcement"); + + assert_eq!(event.kind, Kind::Custom(30617)); + let bindings: Vec<_> = event + .tags + .iter() + .filter(|tag| tag.as_slice().first().map(String::as_str) == Some("buzz-channel")) + .collect(); + assert_eq!(bindings.len(), 1, "exactly one buzz-channel tag"); + assert_eq!(bindings[0].as_slice(), ["buzz-channel", channel.as_str()]); + // The standard metadata still rides along. + assert!(event.tags.iter().any(|tag| tag.as_slice() == ["d", "demo"])); + assert!(event + .tags + .iter() + .any(|tag| tag.as_slice() == ["name", "Demo"])); + } + + #[test] + fn create_without_channel_emits_no_binding_tag() { + let event = build_create_announcement("demo", None, None, &[], None, &[], None) + .expect("build create announcement") + .sign_with_keys(&Keys::generate()) + .expect("sign create announcement"); + + assert!( + !event + .tags + .iter() + .any(|tag| tag.as_slice().first().map(String::as_str) == Some("buzz-channel")), + "no --channel means no binding tag (vanilla NIP-34 stays possible)" + ); + } + + #[test] + fn create_rejects_malformed_channel_uuid() { + let error = build_create_announcement("demo", None, None, &[], None, &[], Some("nope")) + .expect_err("malformed channel id must not build an announcement"); + assert!(matches!(error, crate::error::CliError::Usage(_))); + } + #[test] fn duplicate_write_response_is_a_conflict() { let error = validate_write_response( diff --git a/crates/buzz-cli/src/commands/users.rs b/crates/buzz-cli/src/commands/users.rs index f5a0bee879..7c15d285a0 100644 --- a/crates/buzz-cli/src/commands/users.rs +++ b/crates/buzz-cli/src/commands/users.rs @@ -1,4 +1,7 @@ -use crate::client::{normalize_write_response, BuzzClient}; +use buzz_core::kind::KIND_MANAGED_AGENT; +use nostr::PublicKey; + +use crate::client::{extract_d_tag, normalize_write_response, BuzzClient}; use crate::error::CliError; use crate::validate::validate_hex64; @@ -13,6 +16,7 @@ pub async fn cmd_get_users( client: &BuzzClient, pubkeys: &[String], name: Option<&str>, + owner: Option<&str>, format: &crate::OutputFormat, ) -> Result<(), CliError> { if let Some(query) = name { @@ -21,7 +25,11 @@ pub async fn cmd_get_users( "--name and --pubkey are mutually exclusive".into(), )); } - return search_by_name(client, query, format).await; + return search_by_name(client, query, owner, format).await; + } + + if owner.is_some() { + return Err(CliError::Usage("--owner requires --name".into())); } for pk in pubkeys { @@ -76,68 +84,269 @@ pub async fn cmd_get_users( Ok(()) } -/// Search for users by display name via NIP-50 full-text search on kind:0 profiles. -/// Returns [] if the relay does not implement NIP-50 search. -async fn search_by_name( +fn effective_owner(client: &BuzzClient) -> String { + client + .auth_tag_owner_hex() + .unwrap_or_else(|| client.keys().public_key().to_hex()) +} + +fn resolve_owner(client: &BuzzClient, owner: Option<&str>) -> Result, CliError> { + owner + .map(|owner| { + if owner == "me" { + Ok(effective_owner(client)) + } else { + PublicKey::parse(owner) + .map(|pubkey| pubkey.to_hex()) + .map_err(|e| { + CliError::Usage(format!("--owner must be `me`, a pubkey, or npub: {e}")) + }) + } + }) + .transpose() +} + +fn owned_agent_pubkeys_from_events(events: &[serde_json::Value], query: &str) -> Vec { + let mut pubkeys: Vec = events + .iter() + .filter_map(|event| { + let content: serde_json::Value = + serde_json::from_str(event.get("content")?.as_str()?).ok()?; + let name = content.get("name")?.as_str()?; + if !name.eq_ignore_ascii_case(query) { + return None; + } + let pubkey = extract_d_tag(event); + (!pubkey.is_empty()).then_some(pubkey) + }) + .collect(); + pubkeys.sort(); + pubkeys.dedup(); + pubkeys +} + +async fn owned_agent_pubkeys_by_name( client: &BuzzClient, + owner: &str, query: &str, - format: &crate::OutputFormat, -) -> Result<(), CliError> { - if query.trim().is_empty() { - return Err(CliError::Usage("--name cannot be empty".into())); - } - +) -> Result, CliError> { let filter = serde_json::json!({ - "kinds": [0], - "search": query, - "limit": 100 + "kinds": [KIND_MANAGED_AGENT], + "authors": [owner], }); - let raw = client.query(&filter).await?; - - // Parse and filter client-side for case-insensitive substring match - // on display_name or name fields (NIP-50 may return broader matches). - let events: serde_json::Value = serde_json::from_str(&raw) - .map_err(|e| CliError::Other(format!("failed to parse response: {e}")))?; + let events = client.query_all(filter).await?; + Ok(owned_agent_pubkeys_from_events(&events, query)) +} - let Some(arr) = events.as_array() else { - println!("[]"); - return Ok(()); - }; +fn profile_content(event: &serde_json::Value) -> serde_json::Map { + event + .get("content") + .and_then(|value| value.as_str()) + .and_then(|content| serde_json::from_str::(content).ok()) + .and_then(|content| content.as_object().cloned()) + .unwrap_or_default() +} +fn name_search_profiles(events: &[serde_json::Value], query: &str) -> Vec { let lower_query = query.to_ascii_lowercase(); - let profiles: Vec = arr + events .iter() .filter_map(|event| { - let content_str = event.get("content").and_then(|v| v.as_str())?; - let content: serde_json::Value = serde_json::from_str(content_str).ok()?; - let display_name = content + let mut profile = profile_content(event); + let display_name = profile .get("display_name") - .and_then(|v| v.as_str()) + .and_then(|value| value.as_str()) + .unwrap_or(""); + let name = profile + .get("name") + .and_then(|value| value.as_str()) .unwrap_or(""); - let name = content.get("name").and_then(|v| v.as_str()).unwrap_or(""); if !display_name.to_ascii_lowercase().contains(&lower_query) && !name.to_ascii_lowercase().contains(&lower_query) { return None; } - let mut profile = content; - if let Some(obj) = profile.as_object_mut() { - obj.insert( - "pubkey".to_string(), - serde_json::json!(event.get("pubkey").and_then(|v| v.as_str()).unwrap_or("")), - ); + profile.insert( + "pubkey".to_string(), + serde_json::json!(event + .get("pubkey") + .and_then(|value| value.as_str()) + .unwrap_or("")), + ); + Some(serde_json::Value::Object(profile)) + }) + .collect() +} + +fn auth_tag_values(event: &serde_json::Value) -> Vec<&serde_json::Value> { + event + .get("tags") + .and_then(|tags| tags.as_array()) + .into_iter() + .flatten() + .filter(|tag| { + tag.as_array() + .and_then(|values| values.first()) + .and_then(|value| value.as_str()) + == Some("auth") + }) + .collect() +} + +fn auth_conditions_apply(auth_tag: &serde_json::Value, event: &serde_json::Value) -> bool { + let Some(conditions) = auth_tag + .as_array() + .and_then(|values| values.get(2)) + .and_then(|value| value.as_str()) + else { + return false; + }; + let Some(kind) = event.get("kind").and_then(|value| value.as_u64()) else { + return false; + }; + let Some(created_at) = event.get("created_at").and_then(|value| value.as_u64()) else { + return false; + }; + + conditions.split('&').all(|clause| { + if let Some(value) = clause.strip_prefix("kind=") { + value.parse::() == Ok(kind) + } else if let Some(value) = clause.strip_prefix("created_at<") { + value.parse::().is_ok_and(|bound| created_at < bound) + } else if let Some(value) = clause.strip_prefix("created_at>") { + value.parse::().is_ok_and(|bound| created_at > bound) + } else { + clause.is_empty() + } + }) +} + +fn owner_verification(event: &serde_json::Value, expected_owner: &str) -> &'static str { + let Some(agent_pubkey) = event + .get("pubkey") + .and_then(|value| value.as_str()) + .and_then(|value| PublicKey::parse(value).ok()) + else { + return "invalid_agent_pubkey"; + }; + let auth_tags = auth_tag_values(event); + let [auth_tag] = auth_tags.as_slice() else { + return if auth_tags.is_empty() { + "missing_auth" + } else { + "multiple_auth_tags" + }; + }; + let Ok(auth_tag_json) = serde_json::to_string(auth_tag) else { + return "invalid_auth"; + }; + match buzz_sdk::nip_oa::verify_auth_tag(&auth_tag_json, &agent_pubkey) { + Ok(owner) if owner.to_hex() != expected_owner => "owner_mismatch", + Ok(_) if !auth_conditions_apply(auth_tag, event) => "condition_mismatch", + Ok(_) => "verified", + Err(_) => "invalid_auth", + } +} + +fn owner_scoped_profiles( + events: &[serde_json::Value], + pubkeys: &[String], + owner: &str, + effective_owner: &str, +) -> Vec { + pubkeys + .iter() + .map(|pubkey| { + let event = events.iter().find(|event| { + event.get("pubkey").and_then(|value| value.as_str()) == Some(pubkey.as_str()) + }); + let mut profile = event.map(profile_content).unwrap_or_default(); + let verification = if PublicKey::parse(pubkey).is_err() { + "invalid_agent_pubkey" + } else { + event + .map(|event| owner_verification(event, owner)) + .unwrap_or("missing_profile") + }; + profile.insert("pubkey".to_string(), serde_json::json!(pubkey)); + profile.insert("verification".to_string(), serde_json::json!(verification)); + profile.insert( + "owned_by_me".to_string(), + serde_json::json!(verification == "verified" && owner == effective_owner), + ); + if verification == "verified" { + profile.insert("owner_pubkey".to_string(), serde_json::json!(owner)); } - Some(profile) + serde_json::Value::Object(profile) }) - .collect(); + .collect() +} + +/// Search for users by display name. Owner-scoped searches resolve managed-agent records +/// and verify their profiles; unscoped searches use NIP-50 and return [] if unsupported. +async fn search_by_name( + client: &BuzzClient, + query: &str, + owner: Option<&str>, + format: &crate::OutputFormat, +) -> Result<(), CliError> { + if query.trim().is_empty() { + return Err(CliError::Usage("--name cannot be empty".into())); + } + + let owner = resolve_owner(client, owner)?; + let profiles = if let Some(owner) = owner { + let pubkeys = owned_agent_pubkeys_by_name(client, &owner, query).await?; + if pubkeys.is_empty() { + println!("[]"); + return Ok(()); + } + let valid_pubkeys: Vec<&String> = pubkeys + .iter() + .filter(|pubkey| PublicKey::parse(pubkey.as_str()).is_ok()) + .collect(); + let events = if valid_pubkeys.is_empty() { + Vec::new() + } else { + let filter = serde_json::json!({ + "kinds": [0], + "authors": valid_pubkeys, + "limit": valid_pubkeys.len(), + }); + let raw = client.query(&filter).await?; + serde_json::from_str(&raw) + .map_err(|e| CliError::Other(format!("failed to parse response: {e}")))? + }; + owner_scoped_profiles(&events, &pubkeys, &owner, &effective_owner(client)) + } else { + let filter = serde_json::json!({ + "kinds": [0], + "search": query, + "limit": 100 + }); + let raw = client.query(&filter).await?; + let events: Vec = serde_json::from_str(&raw) + .map_err(|e| CliError::Other(format!("failed to parse response: {e}")))?; + name_search_profiles(&events, query) + }; let output = match format { crate::OutputFormat::Compact => { let compact: Vec = profiles .iter() - .map(|p| serde_json::json!({ - "pubkey": p.get("pubkey").cloned().unwrap_or_default(), - "display_name": p.get("display_name").or_else(|| p.get("name")).cloned().unwrap_or_default(), - })) + .map(|p| { + let mut value = serde_json::json!({ + "pubkey": p.get("pubkey").cloned().unwrap_or_default(), + "display_name": p.get("display_name").or_else(|| p.get("name")).cloned().unwrap_or_default(), + }); + if let Some(obj) = value.as_object_mut() { + for field in ["owner_pubkey", "owned_by_me", "verification"] { + if let Some(field_value) = p.get(field) { + obj.insert(field.to_string(), field_value.clone()); + } + } + } + value + }) .collect(); serde_json::to_string(&compact).unwrap_or_default() } @@ -327,9 +536,11 @@ pub async fn dispatch( ) -> Result<(), CliError> { use crate::UsersCmd; match cmd { - UsersCmd::Get { pubkeys, name } => { - cmd_get_users(client, &pubkeys, name.as_deref(), format).await - } + UsersCmd::Get { + pubkeys, + name, + owner, + } => cmd_get_users(client, &pubkeys, name.as_deref(), owner.as_deref(), format).await, UsersCmd::SetProfile { name, avatar, @@ -362,9 +573,178 @@ pub async fn dispatch( #[cfg(test)] mod tests { - use super::presence_subject; + use super::{ + owned_agent_pubkeys_from_events, owner_scoped_profiles, owner_verification, + presence_subject, + }; + use nostr::Keys; use serde_json::json; + #[test] + fn owned_agent_lookup_matches_exact_name_case_insensitively() { + let events = vec![ + json!({"content": r#"{"name":"Honey"}"#, "tags": [["d", "b"]]}), + json!({"content": r#"{"name":"Honeybee"}"#, "tags": [["d", "c"]]}), + json!({"content": r#"{"name":"honey"}"#, "tags": [["d", "a"]]}), + ]; + assert_eq!( + owned_agent_pubkeys_from_events(&events, "Honey"), + vec!["a", "b"] + ); + } + + #[test] + fn owned_agent_lookup_ignores_malformed_events() { + let events = vec![ + json!({"content": "not json", "tags": [["d", "a"]]}), + json!({"content": r#"{"name":"Honey"}"#, "tags": [["p", "b"]]}), + ]; + assert!(owned_agent_pubkeys_from_events(&events, "Honey").is_empty()); + } + + fn profile_event(agent_keys: &Keys, auth_tags: Vec) -> serde_json::Value { + json!({ + "pubkey": agent_keys.public_key().to_hex(), + "kind": 0, + "created_at": 100, + "content": r#"{"display_name":"Renamed Honey"}"#, + "tags": auth_tags, + }) + } + + #[test] + fn owner_verification_requires_one_valid_auth_tag_for_requested_owner() { + let owner_keys = Keys::generate(); + let agent_keys = Keys::generate(); + let foreign_owner_keys = Keys::generate(); + let valid_tag: serde_json::Value = serde_json::from_str( + &buzz_sdk::nip_oa::compute_auth_tag(&owner_keys, &agent_keys.public_key(), "kind=0") + .unwrap(), + ) + .unwrap(); + let foreign_tag: serde_json::Value = serde_json::from_str( + &buzz_sdk::nip_oa::compute_auth_tag( + &foreign_owner_keys, + &agent_keys.public_key(), + "kind=9", + ) + .unwrap(), + ) + .unwrap(); + + assert_eq!( + owner_verification( + &profile_event(&agent_keys, vec![valid_tag.clone()]), + &owner_keys.public_key().to_hex(), + ), + "verified" + ); + assert_eq!( + owner_verification( + &profile_event(&agent_keys, vec![foreign_tag]), + &owner_keys.public_key().to_hex(), + ), + "owner_mismatch" + ); + assert_eq!( + owner_verification( + &profile_event(&agent_keys, vec![]), + &owner_keys.public_key().to_hex() + ), + "missing_auth" + ); + assert_eq!( + owner_verification( + &profile_event(&agent_keys, vec![valid_tag.clone(), valid_tag]), + &owner_keys.public_key().to_hex(), + ), + "multiple_auth_tags" + ); + assert_eq!( + owner_verification( + &profile_event( + &agent_keys, + vec![json!([ + "auth", + owner_keys.public_key().to_hex(), + "kind=9", + "0".repeat(128) + ])], + ), + &owner_keys.public_key().to_hex(), + ), + "invalid_auth" + ); + } + + #[test] + fn owner_verification_requires_conditions_to_apply_to_profile_event() { + let owner_keys = Keys::generate(); + let agent_keys = Keys::generate(); + let verification = |conditions: &str| { + let auth_tag: serde_json::Value = serde_json::from_str( + &buzz_sdk::nip_oa::compute_auth_tag( + &owner_keys, + &agent_keys.public_key(), + conditions, + ) + .unwrap(), + ) + .unwrap(); + owner_verification( + &profile_event(&agent_keys, vec![auth_tag]), + &owner_keys.public_key().to_hex(), + ) + }; + + assert_eq!(verification("kind=9"), "condition_mismatch"); + assert_eq!(verification("created_at<100"), "condition_mismatch"); + assert_eq!(verification("created_at>100"), "condition_mismatch"); + assert_eq!( + verification("kind=0&created_at>99&created_at<101"), + "verified" + ); + } + + #[test] + fn owner_scoped_profiles_keep_drifted_and_missing_profiles_without_claiming_ownership() { + let owner_keys = Keys::generate(); + let agent_keys = Keys::generate(); + let missing_keys = Keys::generate(); + let auth_tag: serde_json::Value = serde_json::from_str( + &buzz_sdk::nip_oa::compute_auth_tag(&owner_keys, &agent_keys.public_key(), "kind=0") + .unwrap(), + ) + .unwrap(); + let events = vec![profile_event(&agent_keys, vec![auth_tag])]; + let pubkeys = vec![ + agent_keys.public_key().to_hex(), + missing_keys.public_key().to_hex(), + "malformed".to_string(), + ]; + + let profiles = owner_scoped_profiles( + &events, + &pubkeys, + &owner_keys.public_key().to_hex(), + &owner_keys.public_key().to_hex(), + ); + + assert_eq!(profiles[0]["display_name"], "Renamed Honey"); + assert_eq!(profiles[0]["verification"], "verified"); + assert_eq!(profiles[0]["owned_by_me"], true); + assert_eq!( + profiles[0]["owner_pubkey"], + owner_keys.public_key().to_hex() + ); + assert_eq!(profiles[1]["verification"], "missing_profile"); + assert_eq!(profiles[1]["owned_by_me"], false); + assert!(profiles[1].get("owner_pubkey").is_none()); + assert_eq!(profiles[2]["verification"], "invalid_agent_pubkey"); + assert_eq!(profiles[2]["owned_by_me"], false); + assert!(profiles[2].get("owner_pubkey").is_none()); + } + #[test] fn presence_subject_uses_p_tag() { let event = json!({"pubkey": "relay", "tags": [["p", "user"]]}); diff --git a/crates/buzz-cli/src/lib.rs b/crates/buzz-cli/src/lib.rs index 0b46734584..f745e7b280 100644 --- a/crates/buzz-cli/src/lib.rs +++ b/crates/buzz-cli/src/lib.rs @@ -82,11 +82,11 @@ struct Cli { relay: String, /// Nostr private key (hex or nsec). This is the CLI's identity. - #[arg(long, env = "BUZZ_PRIVATE_KEY")] + #[arg(long, env = "BUZZ_PRIVATE_KEY", hide_env_values = true)] private_key: Option, /// NIP-OA auth tag JSON (owner attestation). Injected into every signed event. - #[arg(long, env = "BUZZ_AUTH_TAG")] + #[arg(long, env = "BUZZ_AUTH_TAG", hide_env_values = true)] auth_tag: Option, /// Output format: 'json' (default, full fields) or 'compact' (reduced fields). @@ -212,6 +212,9 @@ enum Cmd { /// Announce and discover git repositories (NIP-34) #[command(subcommand)] Repos(ReposCmd), + /// Create and manage multi-repo projects (NIP-MP) + #[command(subcommand)] + Projects(ProjectsCmd), /// Send, get, list, and set status on git patches (NIP-34) #[command(subcommand)] Patches(PatchesCmd), @@ -369,6 +372,9 @@ pub enum MessagesCmd { /// Attach file(s) — uploads and includes as imeta tags #[arg(long = "file")] files: Vec, + /// Pubkey to mention (hex or npub; repeatable). Supplying any explicit identity permits unresolved or ambiguous @Name text as presentation-only; uniquely resolved member names still notify. + #[arg(long = "mention")] + mentions: Vec, }, /// Send a code diff / patch to a channel SendDiff { @@ -808,6 +814,9 @@ pub enum UsersCmd { /// Search by display name (case-insensitive substring match) #[arg(long = "name")] name: Option, + /// Scope an exact-name agent lookup to its owner (`me`, hex, or npub) + #[arg(long = "owner", requires = "name")] + owner: Option, }, /// Update the current identity's profile #[command(name = "set-profile")] @@ -1126,6 +1135,11 @@ pub enum ReposCmd { /// Preferred Nostr relay(s) for repo discovery — can be specified multiple times #[arg(long = "nostr-relay")] relays: Vec, + /// Channel UUID to bind the repo to. The `buzz-channel` tag is the + /// git ACL: without it the relay 404s every clone/fetch/push until + /// the author runs `buzz repos bind` (issue #3527). + #[arg(long)] + channel: Option, }, /// Get a repository announcement Get { @@ -1145,6 +1159,20 @@ pub enum ReposCmd { #[arg(long)] limit: Option, }, + /// Bind (or rebind) one of your repositories to a channel. + /// + /// The `buzz-channel` tag on the announcement is the git ACL: the relay + /// authorizes clone/fetch/push by membership in the bound channel. A + /// repo announced without it (e.g. by a vanilla NIP-34 client) returns + /// 404 for everyone until its author binds it here. + Bind { + /// Repository identifier (d-tag). + #[arg(long)] + id: String, + /// Channel UUID to bind. Replaces any existing binding. + #[arg(long)] + channel: String, + }, /// Manage branch and tag protection rules on one of your repositories. #[command(subcommand)] Protect(ReposProtectCmd), @@ -1202,6 +1230,122 @@ pub enum RepoPushRole { Member, } +/// Visibility of a multi-repo project listing. +#[derive(Clone, Copy, Debug, clap::ValueEnum)] +pub enum ProjectVisibility { + /// Project appears in public listings (default). + Listed, + /// Project is hidden from public listings. + Unlisted, +} + +impl ProjectVisibility { + pub fn as_str(self) -> &'static str { + match self { + ProjectVisibility::Listed => "listed", + ProjectVisibility::Unlisted => "unlisted", + } + } +} + +#[derive(Subcommand)] +pub enum ProjectsCmd { + /// Create a new multi-repo project (NIP-MP kind:30621) + /// + /// Requires at least one --repo. Fails with Conflict if the project already exists. + Create { + /// Project identifier (slug), up to 1024 bytes + slug: String, + /// Member repository coordinate: bare Buzz repo id (e.g. `buzz`) or full + /// `30617::` for cross-owner or colon-bearing repo ids. + /// At least one --repo is required. + #[arg(long = "repo", required = true)] + repo: Vec, + /// Display name (≤256 bytes) + #[arg(long)] + name: Option, + /// Description (≤2048 bytes) + #[arg(long)] + description: Option, + /// Associated Buzz channel UUID + #[arg(long)] + channel: Option, + /// Visibility: `listed` (default) or `unlisted` + #[arg(long)] + visibility: Option, + }, + /// Get a project by slug + Get { + /// Project slug + slug: String, + /// Owner pubkey (64-char hex). Defaults to the current identity. + #[arg(long)] + owner: Option, + }, + /// List projects + List { + /// Owner pubkey (64-char hex). Defaults to the current identity. + #[arg(long)] + owner: Option, + /// Maximum number of results + #[arg(long)] + limit: Option, + }, + /// Add one or more member repositories to a project + #[command(name = "add-repo")] + AddRepo { + /// Project slug + slug: String, + /// Member repository coordinate (bare id or full `30617::`) + #[arg(long = "repo", required = true)] + repo: Vec, + }, + /// Remove one or more member repositories from a project + #[command(name = "remove-repo")] + RemoveRepo { + /// Project slug + slug: String, + /// Member repository coordinate to remove (bare id or full `30617::`) + #[arg(long = "repo", required = true)] + repo: Vec, + }, + /// Update project metadata (at least one setter or clearer required) + #[command(group = clap::ArgGroup::new("mutation").required(true).multiple(true))] + Update { + /// Project slug + slug: String, + /// Set the display name + #[arg(long, group = "mutation")] + name: Option, + /// Remove the display name + #[arg(long, group = "mutation", conflicts_with = "name")] + clear_name: bool, + /// Set the description + #[arg(long, group = "mutation")] + description: Option, + /// Remove the description + #[arg(long, group = "mutation", conflicts_with = "description")] + clear_description: bool, + /// Set the associated Buzz channel UUID + #[arg(long, group = "mutation")] + channel: Option, + /// Remove the associated channel + #[arg(long, group = "mutation", conflicts_with = "channel")] + clear_channel: bool, + /// Set visibility: `listed` or `unlisted` + #[arg(long, group = "mutation")] + visibility: Option, + /// Remove the visibility tag (absence defaults to `listed`) + #[arg(long, group = "mutation", conflicts_with = "visibility")] + clear_visibility: bool, + }, + /// Delete a project (head-based tombstone; verified after submit) + Delete { + /// Project slug + slug: String, + }, +} + #[derive(Subcommand)] pub enum PatchesCmd { /// Send a git patch (NIP-34 kind:1617) @@ -1743,6 +1887,41 @@ pub enum ModerationCmd { }, } +/// Normalize hand-authored `BUZZ_AUTH_TAG` input to strict JSON. +/// +/// `.env` files and shell exports sometimes carry the tag in the unquoted +/// shorthand `[auth,,,]` (quotes dropped by hand). +/// When the input is not valid JSON but is bracket-delimited, rewrite it as +/// a JSON array of the comma-separated fields (an empty field `,,` becomes +/// `""`, matching the canonical form `["auth","hex","","hex"]`). +/// +/// This is presentation-layer leniency at the configuration edge only: the +/// output is always fed through the SDK's strict `parse_auth_tag` / +/// `verify_auth_tag`, which enforce structure, hex, the conditions grammar, +/// and the BIP-340 signature. Inputs that are already valid JSON — or not +/// recognizable as the shorthand — are returned unchanged so the strict +/// parser reports the error on the original bytes. +fn normalize_auth_tag_input(input: &str) -> String { + let trimmed = input.trim(); + if serde_json::from_str::(trimmed).is_ok() { + return trimmed.to_owned(); + } + if trimmed.starts_with('[') && trimmed.ends_with(']') { + let fields: Vec<&str> = trimmed[1..trimmed.len() - 1] + .split(',') + .map(str::trim) + .collect(); + // Only a plausible 4-field auth tag is rewritten; anything else is + // passed through untouched for the strict parser to reject with an + // error that references the caller's original input. + if fields.len() == 4 && !fields.iter().any(|f| f.contains('"')) { + // serde_json cannot fail serializing a Vec<&str>. + return serde_json::to_string(&fields).expect("string array serializes"); + } + } + trimmed.to_owned() +} + async fn run(cli: Cli) -> Result<(), CliError> { let relay_url = client::normalize_relay_url(&cli.relay); @@ -1763,17 +1942,28 @@ async fn run(cli: Cli) -> Result<(), CliError> { .map_err(|e| CliError::Key(format!("invalid BUZZ_PRIVATE_KEY: {e}")))?; // NIP-OA: parse and verify the auth tag if provided. + // + // `BUZZ_AUTH_TAG` is hand-authored configuration, so the unquoted raw + // shorthand `[auth,hex,,hex]` is normalized to JSON here — at this input + // edge only. The SDK grammar and the `x-auth-tag` wire format stay strict + // JSON; all validation and signature verification happen on the strict + // path below, unchanged. let (auth_tag, auth_tag_json) = match cli.auth_tag { - Some(ref json) if !json.is_empty() => { - let tag = buzz_sdk::nip_oa::parse_auth_tag(json) + Some(ref input) if !input.is_empty() => { + let json = normalize_auth_tag_input(input); + let tag = buzz_sdk::nip_oa::parse_auth_tag(&json) .map_err(|e| CliError::Auth(format!("BUZZ_AUTH_TAG is malformed: {e}")))?; - buzz_sdk::nip_oa::verify_auth_tag(json, &keys.public_key()).map_err(|e| { + buzz_sdk::nip_oa::verify_auth_tag(&json, &keys.public_key()).map_err(|e| { CliError::Auth(format!( "BUZZ_AUTH_TAG verification failed for pubkey {}: {e}", keys.public_key().to_hex() )) })?; - (Some(tag), Some(json.clone())) + // Canonical wire form derives from the parsed-and-verified tag + // (same shape as buzz-acp's RestClient), never from raw input. + let canonical = serde_json::to_string(tag.as_slice()) + .map_err(|e| CliError::Auth(format!("BUZZ_AUTH_TAG serialization failed: {e}")))?; + (Some(tag), Some(canonical)) } _ => (None, None), }; @@ -1794,6 +1984,7 @@ async fn run(cli: Cli) -> Result<(), CliError> { Cmd::Social(sub) => commands::social::dispatch(sub, &client).await, Cmd::Notes(sub) => commands::notes::dispatch(sub, &client).await, Cmd::Repos(sub) => commands::repos::dispatch(sub, &client).await, + Cmd::Projects(sub) => commands::projects::dispatch(sub, &client).await, Cmd::Patches(sub) => commands::patches::dispatch(sub, &client).await, Cmd::Issues(sub) => commands::issues::dispatch(sub, &client).await, Cmd::Pr(sub) => commands::pr::dispatch(sub, &client).await, @@ -1810,6 +2001,51 @@ mod tests { use super::*; use clap::CommandFactory; + /// Raw shorthand `[auth,hex,,hex]` normalizes to strict JSON; the empty + /// conditions field becomes `""`. + #[test] + fn normalize_auth_tag_raw_shorthand() { + let owner = "a".repeat(64); + let sig = "b".repeat(128); + + let raw = format!("[auth,{owner},,{sig}]"); + let json = normalize_auth_tag_input(&raw); + let parsed: Vec = serde_json::from_str(&json).expect("output must be JSON"); + assert_eq!(parsed, vec!["auth", &owner, "", &sig]); + + // With conditions and surrounding whitespace (shell/.env artifacts). + let raw = format!(" [auth, {owner} , kind=9, {sig}] \n"); + let json = normalize_auth_tag_input(&raw); + let parsed: Vec = serde_json::from_str(&json).expect("output must be JSON"); + assert_eq!(parsed, vec!["auth", &owner, "kind=9", &sig]); + } + + /// Valid JSON input passes through byte-identical (modulo outer trim) — + /// the normalizer must never rewrite well-formed input. + #[test] + fn normalize_auth_tag_json_passthrough() { + let owner = "a".repeat(64); + let sig = "b".repeat(128); + let json_in = serde_json::json!(["auth", owner, "kind=9", sig]).to_string(); + assert_eq!(normalize_auth_tag_input(&json_in), json_in); + } + + /// Inputs that are neither JSON nor a plausible 4-field shorthand pass + /// through unchanged, so the strict parser rejects the original bytes. + #[test] + fn normalize_auth_tag_leaves_garbage_untouched() { + for garbage in [ + "not a tag", + "[auth,too,few]", + "[a,b,c,d,e]", + r#"[auth,"quoted",x,y]"#, // quote chars => not the shorthand + "[]", + "{\"auth\":1}", + ] { + assert_eq!(normalize_auth_tag_input(garbage), garbage.trim()); + } + } + /// Smoke test: CLI definition is valid and parseable. #[test] fn cli_definition_is_valid() { @@ -1858,6 +2094,7 @@ mod tests { "pack", "patches", "pr", + "projects", "reactions", "repos", "social", @@ -1988,7 +2225,7 @@ mod tests { ); assert_eq!( names(&cmd, "repos"), - vec!["create", "get", "list", "protect"] + vec!["bind", "create", "get", "list", "protect"] ); let repos = cmd .get_subcommands() @@ -2013,6 +2250,18 @@ mod tests { names(&cmd, "patches"), vec!["get", "list", "send", "status"] ); + assert_eq!( + names(&cmd, "projects"), + vec![ + "add-repo", + "create", + "delete", + "get", + "list", + "remove-repo", + "update" + ] + ); assert_eq!( names(&cmd, "issues"), vec!["create", "get", "list", "status"] @@ -2050,8 +2299,9 @@ mod tests { ("pack", 2), ("patches", 4), ("pr", 5), + ("projects", 7), ("reactions", 3), - ("repos", 4), + ("repos", 5), ("social", 7), ("upload", 1), ("users", 5), @@ -2075,4 +2325,153 @@ mod tests { ); } } + + /// Collect all args (recursing into subcommands) whose env var name looks + /// like a credential but does NOT have `hide_env_values` set. + fn collect_unhidden_secret_args(cmd: &clap::Command) -> Vec<(String, String)> { + const SECRET_PATTERNS: &[&str] = &["KEY", "SECRET", "TOKEN", "PASSWORD", "CRED", "AUTH"]; + + let mut violations: Vec<(String, String)> = Vec::new(); + + for arg in cmd.get_arguments() { + if let Some(env_key) = arg.get_env() { + let env_name = env_key.to_string_lossy().to_uppercase(); + let is_secret = SECRET_PATTERNS.iter().any(|pat| env_name.contains(pat)); + if is_secret && !arg.is_hide_env_values_set() { + violations.push((cmd.get_name().to_string(), env_name)); + } + } + } + + for sub in cmd.get_subcommands() { + violations.extend(collect_unhidden_secret_args(sub)); + } + + violations + } + + /// Every arg whose env var name contains KEY/SECRET/TOKEN/PASSWORD/CRED/AUTH + /// must set `hide_env_values = true` to prevent credential leakage in --help. + #[test] + fn secret_env_args_hide_their_values_in_help() { + let cmd = Cli::command(); + let violations = collect_unhidden_secret_args(&cmd); + assert!( + violations.is_empty(), + "Found secret-bearing env args without hide_env_values=true. \ + Add `hide_env_values = true` to each:\n{}", + violations + .iter() + .map(|(cmd, env)| format!(" command={cmd:?} env={env:?}")) + .collect::>() + .join("\n") + ); + } + + // ── projects update mutation group ──────────────────────────────────────── + + /// Multiple independent fields must be accepted in the same invocation. + #[test] + fn projects_update_multi_field_is_accepted() { + assert!( + Cli::try_parse_from([ + "buzz", + "projects", + "update", + "my-slug", + "--name", + "X", + "--description", + "Y", + ]) + .is_ok(), + "--name and --description together must be accepted" + ); + } + + /// A setter for one field and a clearer for a different field must be accepted. + #[test] + fn projects_update_setter_with_other_clearer_is_accepted() { + assert!( + Cli::try_parse_from([ + "buzz", + "projects", + "update", + "my-slug", + "--name", + "X", + "--clear-description", + ]) + .is_ok(), + "--name with --clear-description must be accepted" + ); + } + + /// A setter and its own clearer are mutually exclusive — clap must reject this. + #[test] + fn projects_update_setter_with_own_clearer_is_rejected() { + assert!( + Cli::try_parse_from([ + "buzz", + "projects", + "update", + "my-slug", + "--name", + "X", + "--clear-name", + ]) + .is_err(), + "--name and --clear-name together must be rejected by clap" + ); + } + + /// Providing no mutation options at all must be rejected by clap (required group). + #[test] + fn projects_update_no_mutation_is_rejected_by_clap() { + // Without credentials, a valid parse would reach authentication and fail + // with auth_error — but a clap-level rejection happens before any I/O. + // We verify it's a clap error (not just any error) by checking the error + // kind is not a runtime/auth failure — Cli::try_parse_from returns Err + // immediately for argument violations. + assert!( + Cli::try_parse_from(["buzz", "projects", "update", "my-slug"]).is_err(), + "update with no setters or clearers must be rejected at parse time" + ); + } + + /// An unrecognised visibility token must be rejected by clap before any I/O. + #[test] + fn projects_create_invalid_visibility_is_rejected_by_clap() { + assert!( + Cli::try_parse_from([ + "buzz", + "projects", + "create", + "my-slug", + "--repo", + "buzz", + "--visibility", + "chartreuse", + ]) + .is_err(), + "--visibility chartreuse must be rejected at parse time" + ); + } + + /// An unrecognised visibility token on update must be rejected by clap before any I/O. + #[test] + fn projects_update_invalid_visibility_is_rejected_by_clap() { + assert!( + Cli::try_parse_from([ + "buzz", + "projects", + "update", + "my-slug", + "--visibility", + "chartreuse", + ]) + .is_err(), + "--visibility chartreuse on update must be rejected at parse time" + ); + } } diff --git a/crates/buzz-core/src/git_perms.rs b/crates/buzz-core/src/git_perms.rs index 53acd704b9..391781163b 100644 --- a/crates/buzz-core/src/git_perms.rs +++ b/crates/buzz-core/src/git_perms.rs @@ -15,6 +15,30 @@ use crate::channel::MemberRole; use std::fmt; +/// Machine-readable token prefixing the push-policy denial for a kind:30617 +/// announcement with no `buzz-channel` binding. +/// +/// This is a **declared cross-component contract**, not a log string. Known +/// consumers switch on it: +/// - relay `api/git/policy.rs` — produces [`GIT_NO_CHANNEL_BINDING_BODY`] +/// - desktop `src-tauri/commands/project_git_workflow.rs` — merge-failure +/// classifier maps it to a structured `no_channel_binding` error code +/// - desktop `src/features/projects/lib/projectBranchErrors.ts` — dialog +/// copy matcher (TS re-types the literal; its test pins the value) +pub const GIT_NO_CHANNEL_BINDING_TOKEN: &str = "no_channel_binding"; + +/// Full push-policy denial body for an unbound repository. +/// +/// Format: `: `. The trailing prose deliberately +/// repeats the token's meaning because desktops already in the field match +/// the exact phrase `no channel binding` (spaces, not underscores — the +/// token alone would NOT satisfy that matcher). Do not "fix" the redundancy: +/// removing the phrase silently breaks every shipped desktop, and removing +/// the token breaks the structured consumers above. A relay-side test pins +/// both matchers. +pub const GIT_NO_CHANNEL_BINDING_BODY: &str = + "no_channel_binding: repository has no channel binding"; + /// Maximum number of `buzz-protect` tags per repo. pub const MAX_PROTECTION_RULES: usize = 50; /// Maximum character length of a ref pattern. diff --git a/crates/buzz-core/src/kind.rs b/crates/buzz-core/src/kind.rs index afec52305a..b1be7c5038 100644 --- a/crates/buzz-core/src/kind.rs +++ b/crates/buzz-core/src/kind.rs @@ -182,29 +182,43 @@ pub const P_GATED_KINDS: &[u32] = &[ /// or more than one `shared` tag) so no ambiguous heads can exist. pub const KIND_PERSONA: u32 = 30175; -/// Returns `true` if `kind` uses the author-only-unless-shared read model -/// (currently only `KIND_PERSONA` / 30175). +/// Kinds that use the author-only-unless-shared read model. /// /// Events of these kinds may only be delivered to foreign readers when the -/// event carries exactly `["shared", "true"]`. Used by all relay read -/// chokepoints: REQ historical delivery, live fan-out, COUNT fallback, -/// and the `ids`-lookup result gate. -pub fn is_persona_shared_kind(kind: u32) -> bool { - kind == KIND_PERSONA +/// event carries exactly `["shared", "true"]`. Every relay read chokepoint +/// consults this set: REQ historical delivery, live fan-out, COUNT fallback, +/// the `ids`-lookup result gate, both HTTP surfaces, and the pre-`LIMIT` SQL +/// visibility pushdown in `buzz-db`. +/// +/// Membership is a privacy decision, not a convenience: adding a kind here +/// makes its events invisible to foreign readers until their author opts in, +/// and the opt-in must be a `shared` TAG (not a content field) so that +/// toggling it leaves content bytes — and any content hash derived from them — +/// unchanged. +/// +/// `KIND_TEAM` (30176) is deliberately NOT a member. Its writers never emit +/// `shared`, so catalog opt-in semantics do not describe it; it needs +/// owner-private read semantics instead, which is a separate change. +pub const SHARED_GATED_KINDS: &[u32] = &[KIND_PERSONA, KIND_TEAM_CATALOG]; + +/// Returns `true` if `kind` uses the author-only-unless-shared read model +/// (see [`SHARED_GATED_KINDS`]). +pub fn is_shared_gated_kind(kind: u32) -> bool { + SHARED_GATED_KINDS.contains(&kind) } -/// Returns `true` if the event is a persona-shared-catalog kind AND the -/// requester is NOT the author AND the event does NOT carry `["shared", -/// "true"]`. All three conditions must hold to withhold the event. +/// Returns `true` if the event is a shared-gated kind AND the requester is NOT +/// the author AND the event does NOT carry `["shared", "true"]`. All three +/// conditions must hold to withhold the event. /// /// This is the per-event gate used by REQ historical delivery, live fan-out, /// and COUNT fallback paths. It is intentionally independent of -/// `is_author_only_event` — persona events with `["shared", "true"]` MUST +/// `is_author_only_event` — shared-gated events with `["shared", "true"]` MUST /// reach foreign readers; stripping them at the author-only layer would break /// the catalog query. -pub fn is_unshared_persona_event(event: &nostr::Event, requester_pubkey_bytes: &[u8]) -> bool { +pub fn is_unshared_gated_event(event: &nostr::Event, requester_pubkey_bytes: &[u8]) -> bool { let kind = event.kind.as_u16() as u32; - if !is_persona_shared_kind(kind) { + if !is_shared_gated_kind(kind) { return false; } // Author reads are always allowed. @@ -212,18 +226,23 @@ pub fn is_unshared_persona_event(event: &nostr::Event, requester_pubkey_bytes: & return false; } // Foreign reader: allowed only if the event is explicitly shared. - !persona_event_is_shared(event) + !event_is_shared(event) } /// Returns `true` if the event carries exactly one `["shared", "true"]` tag. /// +/// Kind-agnostic: this is purely the tag-shape predicate. The kind check lives +/// in [`is_shared_gated_kind`], so callers that need "is this event shared" +/// for a kind they already know (e.g. a client deciding whether its own +/// retained head is published) can use this directly. +/// /// Requires the tag to have exactly two elements so that a three-element shape /// like `["shared","true","extra"]` is NOT treated as shared. Ingest enforces /// the same exact shape, so a well-stored event either has no `shared` tag /// (author-only) or exactly one with precisely two elements and value `"true"` /// (community-readable). This helper fails closed on any non-exact shape /// independently of ingest guarantees. -pub fn persona_event_is_shared(event: &nostr::Event) -> bool { +pub fn event_is_shared(event: &nostr::Event) -> bool { let mut count = 0usize; for tag in event.tags.iter() { let parts = tag.as_slice(); @@ -258,6 +277,34 @@ pub const KIND_TEAM: u32 = 30176; /// since these events are world-readable on the relay. pub const KIND_MANAGED_AGENT: u32 = 30177; +/// NIP-AP: Team Catalog projection (parameterized replaceable, owner-authored). +/// +/// The shareable projection of a team, addressed by `(pubkey, kind, d_tag)` +/// where `d_tag` is the team's stable id. Content is a versioned JSON body +/// carrying sanitized team fields plus ordered, EMBEDDED member definition +/// projections. +/// +/// # Why this is not a `shared` tag on [`KIND_TEAM`] +/// +/// A team's members live in kind 30175 events that are author-only unless +/// individually shared, so a foreign reader of a shared team could never +/// hydrate its members. This kind therefore embeds the member projections +/// rather than referencing them: the share is atomic, it covers built-in +/// members that have no 30175 head at all, it is immune to local-id/d-tag +/// divergence, and an unshared 30175 stays private. Kind 30176's wire body is +/// untouched, so device sync keeps its contract. +/// +/// # Access control +/// +/// Member of [`SHARED_GATED_KINDS`]: author-only unless the event carries +/// exactly `["shared", "true"]`. Ingest additionally requires exactly one +/// non-empty, bounded `d` tag — generic NIP-33 storage maps a missing `d` to +/// the empty coordinate, which would collapse every team into one slot. +/// +/// Content carries only sanitized fields: no env vars, no `respond_to` +/// allowlist pubkeys, no source or local ids, no filesystem paths, no secrets. +pub const KIND_TEAM_CATALOG: u32 = 30178; + // NIP-56 reporting /// NIP-56: Report an event, pubkey, or blob to relay moderators (kind:1984). /// @@ -562,6 +609,15 @@ pub const KIND_GIT_STATUS_CLOSED: u32 = 1632; /// NIP-34: Status — Draft. pub const KIND_GIT_STATUS_DRAFT: u32 = 1633; +/// NIP-MP: Multi-repo project — a named grouping of `kind:30617` repository +/// announcements (parameterized replaceable, d=project slug). +/// +/// Members are `a` tags holding `30617::` coordinates, so one +/// project may span repositories owned by different pubkeys. The signer gains no +/// authority over any member: push policy reads the repository's own +/// announcement, never a project. See `docs/nips/NIP-MP.md`. +pub const KIND_PROJECT: u32 = 30621; + /// All registered kind constants — used for duplicate detection and iteration. pub const ALL_KINDS: &[u32] = &[ KIND_PROFILE, @@ -586,6 +642,7 @@ pub const ALL_KINDS: &[u32] = &[ KIND_PERSONA, KIND_TEAM, KIND_MANAGED_AGENT, + KIND_TEAM_CATALOG, KIND_REPORT, KIND_PRODUCT_FEEDBACK, KIND_NIP29_PUT_USER, @@ -691,6 +748,7 @@ pub const ALL_KINDS: &[u32] = &[ KIND_GIT_STATUS_MERGED, KIND_GIT_STATUS_CLOSED, KIND_GIT_STATUS_DRAFT, + KIND_PROJECT, ]; /// Returns `true` if `kind` is in the ephemeral range (20000–29999). @@ -784,9 +842,11 @@ const _: () = assert!(is_replaceable(KIND_AGENT_PROFILE)); // 10100 ∈ 10000– const _: () = assert!(is_parameterized_replaceable(KIND_PERSONA)); // 30175 ∈ 30000–39999 const _: () = assert!(is_parameterized_replaceable(KIND_TEAM)); // 30176 ∈ 30000–39999 const _: () = assert!(is_parameterized_replaceable(KIND_MANAGED_AGENT)); // 30177 ∈ 30000–39999 +const _: () = assert!(is_parameterized_replaceable(KIND_TEAM_CATALOG)); // 30178 ∈ 30000–39999 const _: () = assert!(is_parameterized_replaceable(KIND_WORKFLOW_DEF)); // 30620 ∈ 30000–39999 const _: () = assert!(is_parameterized_replaceable(KIND_EVENT_REMINDER)); // 30300 ∈ 30000–39999 const _: () = assert!(is_parameterized_replaceable(KIND_DM_VISIBILITY)); // 30622 ∈ 30000–39999 +const _: () = assert!(is_parameterized_replaceable(KIND_PROJECT)); // 30621 ∈ 30000–39999 const _: () = assert!(is_parameterized_replaceable(KIND_THREAD_SUMMARY)); // 39005 ∈ 30000–39999 const _: () = assert!(is_parameterized_replaceable(KIND_WINDOW_BOUNDS)); // 39006 ∈ 30000–39999 @@ -858,64 +918,68 @@ mod tests { } } - // ── persona_event_is_shared / is_unshared_persona_event ────────────── + // ── event_is_shared / is_unshared_gated_event ──────────────────────── - fn make_persona_event(tags: &[&[&str]]) -> nostr::Event { + fn make_event_of_kind(kind: u32, tags: &[&[&str]]) -> nostr::Event { use nostr::{EventBuilder, Keys, Kind, Tag}; let keys = Keys::generate(); let tag_vec: Vec = tags .iter() .map(|parts| Tag::parse(parts.iter().copied()).unwrap()) .collect(); - EventBuilder::new(Kind::Custom(KIND_PERSONA as u16), "") + EventBuilder::new(Kind::Custom(kind as u16), "") .tags(tag_vec) .sign_with_keys(&keys) .unwrap() } + fn make_persona_event(tags: &[&[&str]]) -> nostr::Event { + make_event_of_kind(KIND_PERSONA, tags) + } + #[test] - fn persona_event_is_shared_true_tag() { + fn event_is_shared_true_tag() { let ev = make_persona_event(&[&["d", "my-agent"], &["shared", "true"]]); - assert!(persona_event_is_shared(&ev)); + assert!(event_is_shared(&ev)); } #[test] - fn persona_event_is_shared_no_tag() { + fn event_is_shared_no_tag() { let ev = make_persona_event(&[&["d", "my-agent"]]); - assert!(!persona_event_is_shared(&ev)); + assert!(!event_is_shared(&ev)); } #[test] - fn persona_event_is_shared_wrong_value() { + fn event_is_shared_wrong_value() { let ev = make_persona_event(&[&["d", "my-agent"], &["shared", "false"]]); - assert!(!persona_event_is_shared(&ev)); + assert!(!event_is_shared(&ev)); } #[test] - fn persona_event_is_shared_duplicate_shared_tags() { + fn event_is_shared_duplicate_shared_tags() { // Two ["shared","true"] tags → ambiguous; not considered shared. let ev = make_persona_event(&[&["d", "my-agent"], &["shared", "true"], &["shared", "true"]]); - assert!(!persona_event_is_shared(&ev)); + assert!(!event_is_shared(&ev)); } #[test] - fn persona_event_is_shared_three_element_tag_not_shared() { + fn event_is_shared_three_element_tag_not_shared() { // ["shared","true","extra"] — three elements — must NOT be treated as shared. // The helper fails closed on any non-exact shape independently of ingest guarantees. let ev = make_persona_event(&[&["d", "my-agent"], &["shared", "true", "extra"]]); - assert!(!persona_event_is_shared(&ev)); + assert!(!event_is_shared(&ev)); } #[test] - fn persona_event_is_shared_one_element_tag_not_shared() { + fn event_is_shared_one_element_tag_not_shared() { // ["shared"] — only one element — not shared (fails the == 2 check). let ev = make_persona_event(&[&["d", "my-agent"], &["shared"]]); - assert!(!persona_event_is_shared(&ev)); + assert!(!event_is_shared(&ev)); } #[test] - fn is_unshared_persona_event_author_always_allowed() { + fn is_unshared_gated_event_author_always_allowed() { // Even without a shared tag the event author should not be blocked. use nostr::{EventBuilder, Keys, Kind, Tag}; let keys = Keys::generate(); @@ -924,32 +988,83 @@ mod tests { .sign_with_keys(&keys) .unwrap(); let author_bytes = keys.public_key().to_bytes(); - assert!(!is_unshared_persona_event(&ev, &author_bytes)); + assert!(!is_unshared_gated_event(&ev, &author_bytes)); } #[test] - fn is_unshared_persona_event_foreign_no_tag() { + fn is_unshared_gated_event_foreign_no_tag() { let ev = make_persona_event(&[&["d", "my-agent"]]); let foreign = [0u8; 32]; - assert!(is_unshared_persona_event(&ev, &foreign)); + assert!(is_unshared_gated_event(&ev, &foreign)); } #[test] - fn is_unshared_persona_event_foreign_shared_tag() { + fn is_unshared_gated_event_foreign_shared_tag() { let ev = make_persona_event(&[&["d", "my-agent"], &["shared", "true"]]); let foreign = [0u8; 32]; - assert!(!is_unshared_persona_event(&ev, &foreign)); + assert!(!is_unshared_gated_event(&ev, &foreign)); } #[test] - fn is_unshared_persona_event_non_persona_kind_passthrough() { + fn is_unshared_gated_event_ungated_kind_passthrough() { use nostr::{EventBuilder, Keys, Kind}; let keys = Keys::generate(); let ev = EventBuilder::new(Kind::Custom(KIND_TEAM as u16), "") .sign_with_keys(&keys) .unwrap(); let foreign = [0u8; 32]; - // Non-persona kinds are never blocked by this gate. - assert!(!is_unshared_persona_event(&ev, &foreign)); + // Kinds outside SHARED_GATED_KINDS are never blocked by this gate. + assert!(!is_unshared_gated_event(&ev, &foreign)); + } + + #[test] + fn is_unshared_gated_event_team_catalog_foreign_no_tag() { + // The gate must cover 30178 identically to 30175 — an unshared team + // catalog projection is author-only. + let ev = make_event_of_kind(KIND_TEAM_CATALOG, &[&["d", "team-1"]]); + let foreign = [0u8; 32]; + assert!(is_unshared_gated_event(&ev, &foreign)); + } + + #[test] + fn is_unshared_gated_event_team_catalog_foreign_shared_tag() { + let ev = make_event_of_kind(KIND_TEAM_CATALOG, &[&["d", "team-1"], &["shared", "true"]]); + let foreign = [0u8; 32]; + assert!(!is_unshared_gated_event(&ev, &foreign)); + } + + #[test] + fn is_unshared_gated_event_team_catalog_author_always_allowed() { + use nostr::{EventBuilder, Keys, Kind, Tag}; + let keys = Keys::generate(); + let ev = EventBuilder::new(Kind::Custom(KIND_TEAM_CATALOG as u16), "") + .tags(vec![Tag::parse(["d", "team-1"]).unwrap()]) + .sign_with_keys(&keys) + .unwrap(); + let author_bytes = keys.public_key().to_bytes(); + assert!(!is_unshared_gated_event(&ev, &author_bytes)); + } + + #[test] + fn is_unshared_gated_event_team_catalog_malformed_shared_tag_fails_closed() { + // A three-element `shared` tag can never be stored (ingest rejects it), + // but the read gate must independently treat it as NOT shared. + let ev = make_event_of_kind( + KIND_TEAM_CATALOG, + &[&["d", "team-1"], &["shared", "true", "extra"]], + ); + let foreign = [0u8; 32]; + assert!(is_unshared_gated_event(&ev, &foreign)); + } + + #[test] + fn shared_gated_kinds_membership() { + assert!(is_shared_gated_kind(KIND_PERSONA)); + assert!(is_shared_gated_kind(KIND_TEAM_CATALOG)); + // 30176 has owner-private semantics, not catalog opt-in semantics: its + // writers never emit `shared`, so gating it here would hide every team + // from its own delegated readers. + assert!(!is_shared_gated_kind(KIND_TEAM)); + assert!(!is_shared_gated_kind(KIND_MANAGED_AGENT)); } } diff --git a/crates/buzz-db/Cargo.toml b/crates/buzz-db/Cargo.toml index 01f1e172b6..6f76a11bc1 100644 --- a/crates/buzz-db/Cargo.toml +++ b/crates/buzz-db/Cargo.toml @@ -21,6 +21,8 @@ tracing = { workspace = true } thiserror = { workspace = true } nostr = { workspace = true } rand = { workspace = true } +metrics = { workspace = true } [dev-dependencies] tokio = { workspace = true } +metrics-util = { workspace = true } diff --git a/crates/buzz-db/src/event.rs b/crates/buzz-db/src/event.rs index 520fd1536f..a670a13402 100644 --- a/crates/buzz-db/src/event.rs +++ b/crates/buzz-db/src/event.rs @@ -11,12 +11,19 @@ use uuid::Uuid; use buzz_core::kind::{ event_kind_i32, is_ephemeral, is_parameterized_replaceable, KIND_AUTH, KIND_EVENT_REMINDER, - KIND_HUDDLE_STARTED, + KIND_HUDDLE_STARTED, SHARED_GATED_KINDS, }; use buzz_core::{CommunityId, StoredEvent}; use crate::error::{DbError, Result}; +/// Largest page [`query_events`] will return when [`EventQuery::max_limit`] is +/// unset — the effective ceiling on any client-requested `limit`. +/// +/// This is the value the relay advertises as NIP-11 `limitation.max_limit`, so +/// the advertised ceiling and the enforced one cannot drift. +pub const DEFAULT_MAX_PAGE_LIMIT: i64 = 1_000; + /// Optional filters for [`query_events`]. #[derive(Debug, Clone)] pub struct EventQuery { @@ -67,17 +74,19 @@ pub struct EventQuery { /// channel-less global events. Applied before SQL `LIMIT` so access-filtered /// historical pages have exact exhaustion semantics. pub channel_ids: Option>, - /// Override the default limit clamp (1000). Used by COUNT fallback path - /// which needs to fetch all matching events for post-filter counting. - /// When None, the default clamp of 1000 applies. + /// Override the default page clamp ([`DEFAULT_MAX_PAGE_LIMIT`]). Used by + /// the COUNT fallback path, which needs to fetch all matching events for + /// post-filter counting. When None, the default clamp applies. pub max_limit: Option, - /// Persona visibility reader: when set, append an SQL visibility clause - /// for kind 30175 before ORDER/LIMIT so private personas are excluded from - /// the candidate page rather than discarded after it. + /// Shared-gated visibility reader: when set, append an SQL visibility + /// clause for every kind in [`SHARED_GATED_KINDS`] before ORDER/LIMIT so + /// private events are excluded from the candidate page rather than + /// discarded after it. /// - /// The clause is: `AND (kind != 30175 OR pubkey = $reader OR tags @> ?)`, - /// where `?` is the JSONB literal `[["shared","true"]]`. The GIN index on - /// `tags` (migration 0004, jsonb_path_ops) makes the containment check fast. + /// The clause is: `AND (kind NOT IN (...) OR pubkey = $reader OR tags @> ?)`, + /// where the `IN` list is [`SHARED_GATED_KINDS`] and `?` is the JSONB + /// literal `[["shared","true"]]`. The GIN index on `tags` (migration 0004, + /// jsonb_path_ops) makes the containment check fast. /// /// NOTE: `tags @> '[["shared","true"]]'` uses JSONB containment, which /// matches any tag array that is a superset of `[["shared","true"]]` — it @@ -85,7 +94,7 @@ pub struct EventQuery { /// 2` exact-shape check ensures such malformed tags are never stored, so the /// SQL pushdown is sound. Keeping `event_visible_to_reader` as post-filter /// defense-in-depth catches any residual mismatch. - pub persona_reader: Option>, + pub shared_gated_reader: Option>, } impl EventQuery { @@ -114,7 +123,7 @@ impl EventQuery { e_tags: None, channel_ids: None, max_limit: None, - persona_reader: None, + shared_gated_reader: None, } } } @@ -316,6 +325,17 @@ pub async fn insert_event( /// Uses `QueryBuilder` for dynamic filter composition — avoids string concatenation /// while keeping all user values in bind parameters. pub async fn query_events(pool: &PgPool, q: &EventQuery) -> Result> { + let mut conn = pool.acquire().await?; + query_events_on(&mut conn, q).await +} + +/// [`query_events`] on a specific session — the replica-routing path runs +/// follow-up (aux) queries on the exact reader connection whose heartbeat +/// observation proved coverage for the page they annotate. +pub(crate) async fn query_events_on( + conn: &mut sqlx::PgConnection, + q: &EventQuery, +) -> Result> { // Composite cursor requires both halves. if q.before_id.is_some() && q.until.is_none() { return Err(DbError::InvalidData( @@ -344,7 +364,7 @@ pub async fn query_events(pool: &PgPool, q: &EventQuery) -> Result Result '[["shared","true"]]') + // Clause: AND (kind NOT IN (30175, 30178) OR pubkey = $reader + // OR tags @> '[["shared","true"]]') // // The JSONB containment check is served by idx_events_tags_gin (migration // 0004, jsonb_path_ops). `tags @> '[["shared","true"]]'` matches any array // that contains exactly the sub-array — a two-element `["shared","true"]` - // tag passes; a tag-absent event does not. Because ingest now requires - // exactly two elements for the shared tag (parts.len() == 2), no stored - // event can carry a three-element superset. - if let Some(ref reader_bytes) = q.persona_reader { - let kind_30175: i32 = 30175; + // tag passes; a tag-absent event does not. Because ingest requires exactly + // two elements for the shared tag (parts.len() == 2), no stored event can + // carry a three-element superset. + if let Some(ref reader_bytes) = q.shared_gated_reader { let shared_containment = serde_json::json!([["shared", "true"]]); - qb.push(format!(" AND ({col_prefix}kind != ")); - qb.push_bind(kind_30175); - qb.push(format!(" OR {col_prefix}pubkey = ")); + qb.push(format!(" AND ({col_prefix}kind NOT IN (")); + let mut sep = qb.separated(", "); + for kind in SHARED_GATED_KINDS { + sep.push_bind(*kind as i32); + } + qb.push(format!(") OR {col_prefix}pubkey = ")); qb.push_bind(reader_bytes.clone()); qb.push(format!(" OR {col_prefix}tags @> ")); qb.push_bind(shared_containment); @@ -538,7 +561,7 @@ pub async fn query_events(pool: &PgPool, q: &EventQuery) -> Result Result Result { + let mut conn = pool.acquire().await?; + count_events_on(&mut conn, q).await +} + +/// [`count_events`] on a specific session — the replica-routing path runs +/// the count on the exact reader connection whose heartbeat observation +/// proved its predicate. +pub(crate) async fn count_events_on(conn: &mut sqlx::PgConnection, q: &EventQuery) -> Result { // Empty list means "match nothing" — return 0 immediately. if q.kinds.as_deref().is_some_and(|k| k.is_empty()) { return Ok(0); @@ -730,7 +761,7 @@ pub async fn count_events(pool: &PgPool, q: &EventQuery) -> Result { } } - let row = qb.build().fetch_one(pool).await?; + let row = qb.build().fetch_one(&mut *conn).await?; let cnt: i64 = row.try_get("cnt")?; Ok(cnt) @@ -758,7 +789,8 @@ pub async fn soft_delete_event( } /// Soft-delete the live row for an addressable coordinate -/// `(kind, pubkey, d_tag)` — the NIP-33 replacement key. +/// `(kind, pubkey, d_tag)` — the NIP-33 replacement key — provided it is not +/// newer than the deletion request. /// /// Used by `handle_a_tag_deletion` to honour NIP-09 a-tag deletions for any /// parameterized-replaceable kind. The WHERE clause mirrors @@ -766,23 +798,45 @@ pub async fn soft_delete_event( /// `channel_id` is intentionally NOT in the key (NIP-33 replacement is global /// per the spec — `channel_id` is stored for query scoping, not identity). /// +/// `deletion_created_at_secs` is the deletion event's own `created_at`. NIP-09 +/// scopes an `a`-tag deletion to versions at or before that instant, so a +/// delayed or replayed tombstone signed between two versions must not erase the +/// newer replacement. `events.created_at` is immutable per row, so the predicate +/// guarantees a tombstone can never erase a version newer than itself — the UPDATE +/// re-evaluates its WHERE clause after any lock wait, so a replacement that races +/// the deletion and lands with a later `created_at` is always spared. +/// +/// This does NOT guarantee deletion completeness when a same-coordinate +/// replacement races the deletion: the deletion may evaluate its predicate before +/// the replacement arrives, miss the incoming head, and return `Ok(false)`. That +/// outcome is state-identical to the deletion having arrived first (old head +/// gone, new head present), which is a valid Nostr ordering — Nostr never fixes +/// the order of concurrent writes from different signers, and even same-signer +/// ordering is advisory. The return value feeds only a debug log, not a +/// correctness gate. +/// /// Returns `Ok(true)` if a row was deleted, `Ok(false)` if no live row matched -/// (already deleted, or never existed). +/// (already deleted, never existed, or strictly newer than the deletion). pub async fn soft_delete_by_coordinate( pool: &PgPool, community_id: CommunityId, kind: i32, pubkey: &[u8], d_tag: &str, + deletion_created_at_secs: i64, ) -> Result { + let deletion_created_at = DateTime::from_timestamp(deletion_created_at_secs, 0) + .ok_or(DbError::InvalidTimestamp(deletion_created_at_secs))?; let result = sqlx::query( "UPDATE events SET deleted_at = NOW() \ - WHERE community_id = $1 AND kind = $2 AND pubkey = $3 AND d_tag = $4 AND deleted_at IS NULL", + WHERE community_id = $1 AND kind = $2 AND pubkey = $3 AND d_tag = $4 AND deleted_at IS NULL \ + AND created_at <= $5", ) .bind(community_id.as_uuid()) .bind(kind) .bind(pubkey) .bind(d_tag) + .bind(deletion_created_at) .execute(pool) .await?; @@ -990,6 +1044,21 @@ pub async fn get_events_by_ids( pool: &PgPool, community_id: CommunityId, ids: &[&[u8]], +) -> Result> { + if ids.is_empty() { + return Ok(vec![]); + } + let mut conn = pool.acquire().await?; + get_events_by_ids_on(&mut conn, community_id, ids).await +} + +/// [`get_events_by_ids`] on a specific session — the replica-routing path +/// runs the query on the exact reader connection whose heartbeat +/// observation proved its predicate. +pub(crate) async fn get_events_by_ids_on( + conn: &mut sqlx::PgConnection, + community_id: CommunityId, + ids: &[&[u8]], ) -> Result> { if ids.is_empty() { return Ok(vec![]); @@ -1008,7 +1077,7 @@ pub async fn get_events_by_ids( } qb.push(")"); - let rows = qb.build().fetch_all(pool).await?; + let rows = qb.build().fetch_all(&mut *conn).await?; let mut out = Vec::with_capacity(rows.len()); for row in rows { diff --git a/crates/buzz-db/src/feed.rs b/crates/buzz-db/src/feed.rs index 511a2a6083..40e58d0d06 100644 --- a/crates/buzz-db/src/feed.rs +++ b/crates/buzz-db/src/feed.rs @@ -132,6 +132,29 @@ pub async fn query_mentions( accessible_channel_ids: &[Uuid], since: Option>, limit: i64, +) -> Result> { + let mut conn = pool.acquire().await?; + query_mentions_on( + &mut conn, + community, + pubkey_bytes, + accessible_channel_ids, + since, + limit, + ) + .await +} + +/// [`query_mentions`] on a specific session — the replica-routing path runs +/// the query on the exact reader connection whose heartbeat observation +/// proved its predicate. +pub(crate) async fn query_mentions_on( + conn: &mut sqlx::PgConnection, + community: CommunityId, + pubkey_bytes: &[u8], + accessible_channel_ids: &[Uuid], + since: Option>, + limit: i64, ) -> Result> { let mut qb = build_mentions_query( community, @@ -140,7 +163,7 @@ pub async fn query_mentions( since, limit, ); - let rows = qb.build().fetch_all(pool).await?; + let rows = qb.build().fetch_all(&mut *conn).await?; collect_stored_events(rows) } @@ -193,6 +216,27 @@ pub async fn query_needs_action( accessible_channel_ids: &[Uuid], since: Option>, limit: i64, +) -> Result> { + let mut conn = pool.acquire().await?; + query_needs_action_on( + &mut conn, + community, + pubkey_bytes, + accessible_channel_ids, + since, + limit, + ) + .await +} + +/// [`query_needs_action`] on a specific session — see [`query_mentions_on`]. +pub(crate) async fn query_needs_action_on( + conn: &mut sqlx::PgConnection, + community: CommunityId, + pubkey_bytes: &[u8], + accessible_channel_ids: &[Uuid], + since: Option>, + limit: i64, ) -> Result> { let mut qb = build_needs_action_query( community, @@ -201,7 +245,7 @@ pub async fn query_needs_action( since, limit, ); - let rows = qb.build().fetch_all(pool).await?; + let rows = qb.build().fetch_all(&mut *conn).await?; collect_stored_events(rows) } @@ -241,9 +285,21 @@ pub async fn query_activity( accessible_channel_ids: &[Uuid], since: Option>, limit: i64, +) -> Result> { + let mut conn = pool.acquire().await?; + query_activity_on(&mut conn, community, accessible_channel_ids, since, limit).await +} + +/// [`query_activity`] on a specific session — see [`query_mentions_on`]. +pub(crate) async fn query_activity_on( + conn: &mut sqlx::PgConnection, + community: CommunityId, + accessible_channel_ids: &[Uuid], + since: Option>, + limit: i64, ) -> Result> { let mut qb = build_activity_query(community, accessible_channel_ids, since, limit); - let rows = qb.build().fetch_all(pool).await?; + let rows = qb.build().fetch_all(&mut *conn).await?; collect_stored_events(rows) } diff --git a/crates/buzz-db/src/lib.rs b/crates/buzz-db/src/lib.rs index 2a3ba9a63e..9b26876747 100644 --- a/crates/buzz-db/src/lib.rs +++ b/crates/buzz-db/src/lib.rs @@ -55,7 +55,7 @@ pub mod user; pub mod workflow; pub use error::{DbError, Result}; -pub use event::{EventQuery, ReactionEventInsertOutcome}; +pub use event::{EventQuery, ReactionEventInsertOutcome, DEFAULT_MAX_PAGE_LIMIT}; use chrono::{DateTime, Utc}; use sqlx::postgres::{PgConnection, PgPoolOptions}; @@ -181,12 +181,289 @@ pub struct Db { /// route here (see [`Db::read`]); locks, transactions, and anything /// consistency-critical stays on `pool`. pub(crate) read_pool: Option, + /// Maximum connections configured for the read-replica pool (from + /// [`DbConfig::read_max_connections`], defaulting to the writer's + /// sizing). Kept separately from `max_connections` so + /// [`Db::read_pool_stats`] reports the reader's own ceiling — a + /// utilisation gauge derived from the writer's max would understate + /// reader saturation by exactly the ratio of the two pool sizes. + pub(crate) read_max_connections: u32, /// Freshness fence gating cursor-page routing to the replica. /// /// Starts closed; a background probe ([`replica_fence::run_probe`]) - /// advances it after each verified writer→replica LSN handshake. When - /// closed or stale, every cursor page routes to the writer. + /// commits heartbeat tokens and retains proof entries. Routing proves + /// coverage per request on the serving reader session; when the ring is + /// empty or stale, every routed read stays on the writer. pub(crate) fence: std::sync::Arc, + /// Bounded-staleness routing budget `B`: a read routed under + /// [`RoutePredicate::Bounded`] may be served from a proved replica + /// session only when the proved heartbeat entry is at most this old. + /// `None` disables the bounded arm entirely (the rollout default) — + /// bounded-stale read semantics are a product decision, not an + /// invariant, so the gate ships off. + pub(crate) replica_read_max_age: Option, + /// Whether the reader endpoint supports the Aurora PostgreSQL identity + /// function ([`replica_fence::AURORA_IDENTITY_FN`]) — probed + /// once per process on the first routed read (on a plain autocommit + /// checkout, outside any request transaction) and cached. Unset means + /// not yet probed (or the probe hit a transient error and will retry). + /// Shared across `Db` clones. + pub(crate) reader_aurora_identity: std::sync::Arc>, +} + +/// The session that served (or will serve) a routed read, so follow-up +/// queries in the same request (the channel-window aux closure) run on the +/// **same proved snapshot** — a different pooled reader session may sit at a +/// different replay position, and even the same connection advances its +/// snapshot between autocommit statements. +/// +/// `Replica` holds the request's `REPEATABLE READ, READ ONLY` transaction: +/// the heartbeat observation was its first statement, so the snapshot the +/// proof was taken against is exactly the snapshot every follow-up sees. +/// Dropping the session rolls the read-only transaction back and returns +/// the connection to the pool. +/// +/// `Writer` carries the writer pool: follow-ups there are authoritative by +/// construction and need no session pinning. +pub struct ReadSession { + inner: ReadSessionInner, +} + +enum ReadSessionInner { + /// The proved replica request transaction (snapshot-anchored), plus the + /// writer pool so a mid-request replica failure (e.g. a hot-standby + /// recovery conflict cancelling the held snapshot) degrades the session + /// to the writer instead of surfacing an error: degraded capacity, + /// never holes — and never a 500 the writer could have served. + Replica { + tx: sqlx::Transaction<'static, sqlx::Postgres>, + writer: PgPool, + }, + /// The writer pool (cheap clone; Arc-backed). + Writer(PgPool), +} + +impl ReadSession { + /// Query events on this session (see [`Db::query_events`]). + /// + /// If the proved replica transaction fails mid-request, the session + /// permanently degrades to the writer and the query is re-run there. + /// The writer is always at or ahead of any replica replay position, so + /// the degraded follow-up can only observe *more* than the proof-time + /// snapshot, never less — fresher aux rows, the same failure semantics + /// as a request that routed to the writer to begin with. + pub async fn query_events(&mut self, q: &EventQuery) -> Result> { + let degraded = match &mut self.inner { + ReadSessionInner::Replica { tx, writer } => { + match event::query_events_on(tx, q).await { + Ok(rows) => return Ok(rows), + Err(e) => { + tracing::warn!( + error = %e, + "replica session query failed mid-request; degrading to writer" + ); + // Deliberately not a `buzz_db_route_decision` event: + // the page's route was already recorded, and the + // offload metric must stay one-event-per-request. + metrics::counter!("buzz_db_read_session_degraded").increment(1); + writer.clone() + } + } + } + ReadSessionInner::Writer(pool) => return event::query_events(pool, q).await, + }; + // Replacing the inner drops the replica transaction (rolling it + // back and returning the reader connection to its pool). + self.inner = ReadSessionInner::Writer(degraded.clone()); + event::query_events(°raded, q).await + } + + /// Whether this session is a proved replica connection (observability). + pub fn is_replica(&self) -> bool { + matches!(self.inner, ReadSessionInner::Replica { .. }) + } +} + +/// Where one routed read is served (see [`Db::route_read`]). +enum RouteDecision { + /// A reader request transaction whose first-statement heartbeat + /// observation proved this fence entry — the page runs inside it. The + /// `&'static str` is the metric reason (`covered`/`fresh`); the caller + /// records the route only once the page is actually served from the + /// replica, so a post-verification writer re-run or a mid-query replica + /// failure emits exactly one `buzz_db_route_decision` event per request + /// (the offload percentage is read straight off `decision="replica"`). + Replica( + sqlx::Transaction<'static, sqlx::Postgres>, + replica_fence::TokenEntry, + &'static str, + ), + /// Fail closed: serve from the writer pool (already recorded). + Writer, +} + +/// The ONLY place [`route_proof::ChannelScoped`] can be constructed. A +/// crate-root tuple struct would be mintable via `ChannelScoped(())` from +/// every descendant module — tuple-struct field privacy is module-scoped — +/// so the token lives in its own module and E0423 enforces the invariant. +mod route_proof { + use uuid::Uuid; + + /// Proof that a query/page can only return rows with + /// `channel_id IS NOT NULL` — the domain of the commit-time floor guard + /// (migration 0021). `channel_ids` (retains channel-NULL rows) and + /// `global_only = false` are explicitly NOT proofs. + /// + /// Each constructor keys off *how* its path proves channel-bearing-ness: + /// a pinned query filter, a bare `Uuid` argument, or a `NOT NULL` column + /// reached through an inner join. Do not add a universal constructor + /// callers reshape their inputs to fit, and never fabricate a throwaway + /// `EventQuery` purely to mint a token — the proof must be the SQL's + /// shape, not "someone assembled a struct". + #[derive(Clone, Copy)] + pub(crate) struct ChannelScoped(()); + + impl ChannelScoped { + /// Constructor 1: the query pins a single channel + /// (`EventQuery.channel_id = Some(_)`, compiled to a + /// `channel_id = $n` predicate). This proof covers BOTH query + /// builders — the SELECT builder (`event::query_events_on`) and the + /// COUNT builder (`event::count_events`) pin identically; if the + /// two ever drift, this comment is a lie and the routed COUNT seam + /// is unsound. + /// Sound under conjunction: any additional clause (e.g. + /// `channel_ids`, which alone retains channel-NULL rows) is ANDed, + /// and `channel_id = ` never matches NULL — the pin strictly + /// narrows and cannot be widened back out to global rows. + pub(crate) fn from_pinned_channel(q: &crate::event::EventQuery) -> Option { + q.channel_id.map(|_| ChannelScoped(())) + } + + /// Constructor 2 (thread pages): the page is an inner JOIN from + /// `thread_metadata` to `events`, and `thread_metadata.channel_id` + /// is `UUID NOT NULL` — every writer that creates a row passes a + /// concrete channel (`ThreadMetadataParams.channel_id: Uuid`, + /// non-Option). Channel-bearing by construction of the join, not by + /// query predicate. + pub(crate) fn from_thread_metadata_join() -> Self { + ChannelScoped(()) + } + + /// Constructor 3 (channel windows): the channel arrives as a bare + /// `Uuid` argument and the SQL binds it unconditionally + /// (`e.channel_id = $2` in `get_channel_window_on`); every served + /// row is channel-bearing. No `EventQuery` exists on this path. + pub(crate) fn from_channel_id(_channel_id: Uuid) -> Self { + ChannelScoped(()) + } + } +} +use route_proof::ChannelScoped; + +/// The predicate one routed read must satisfy (see [`Db::route_read`]). +/// +/// Discipline: no `Default`, no `Deserialize`, stays non-`pub` — any of +/// those re-opens the [`ChannelScoped`] mint. +enum RoutePredicate { + /// Bounded staleness: the proved entry must be within the configured + /// read budget `B` (default off). Bounds TIME — the page misses at most + /// the freshest `B` of writes. Sound for ANY query shape, including + /// global (channel-NULL) rows: it relies only on heartbeat commit order, + /// not the floor guard. + Bounded, + /// Completeness: the proved wall must cover the page's upper bound. + /// Bounds CONTENT — every row at/below `upper` is present, meaningful + /// even when the cursor is hours old, where `B`-freshness says nothing. + /// Sound ONLY on the floor guard's domain (channel-bearing rows), hence + /// the proof token. `upper` is non-optional: the no-upper-bound + /// post-verifying case is [`RoutePredicate::CoveredPostVerified`]. + /// + /// Bounds INSERT-completeness only — "no missing rows", not "no extra + /// rows". Soft deletes are `UPDATE .. SET deleted_at` commits outside + /// the floor guard and never touch `created_at`, so a covered page can + /// briefly serve a row the writer already excludes; deletion visibility + /// is bounded by replication lag under `FENCE_STALENESS` (30s), not by + /// `upper` or `B`. Do not extend the covered arm to a surface that + /// cannot absorb extra rows (this is why the routed COUNT seam is + /// bounded-only). + Covered { + upper: DateTime, + /// Never read — the field exists so constructing this variant + /// requires minting the token through `route_proof`. + #[allow(dead_code)] + proof: ChannelScoped, + }, + /// Forward-walking thread pages: no upper bound is derivable from the + /// cursor; the caller post-verifies the served rows against the proved + /// wall (full page + tail at/below the wall, else re-run on the writer). + /// Only the thread path constructs this — a general routed caller does + /// no post-verification and must never self-certify. + CoveredPostVerified { + #[allow(dead_code)] + proof: ChannelScoped, + }, + /// Either arm admits, covered tried first (it has no budget dependence). + /// For general routed reads that are channel-pinned AND carry an + /// `until` upper bound. + BoundedOrCovered { + upper: DateTime, + /// Never read — see [`RoutePredicate::Covered::proof`]. + #[allow(dead_code)] + proof: ChannelScoped, + }, +} + +impl RoutePredicate { + /// A channel-window request: cursor pages are covered-only — for deep + /// keyset pages only coverage answers "have all rows below the cursor + /// replayed?" — and a head fetch is bounded. The channel id is the + /// bare-`Uuid` proof that the window SQL pins a channel. + fn from_channel_cursor(channel_id: Uuid, cursor: &Option<(DateTime, Vec)>) -> Self { + match cursor { + Some((ts, _)) => RoutePredicate::Covered { + upper: *ts, + proof: ChannelScoped::from_channel_id(channel_id), + }, + None => RoutePredicate::Bounded, + } + } + + /// General entry point for the routed query seams: derives the strongest + /// sound predicate from the query shape. Never produces a covered arm + /// without both a channel-scope proof AND a real upper bound. + /// + /// `routing_enabled` is whether `BUZZ_REPLICA_READ_MAX_AGE_MS` is set + /// (non-zero). When it is NOT, this returns `Bounded` — which the zero + /// budget then fails closed — so the new seams are genuinely dark at + /// the deploy default even for channel-pinned queries carrying `until`. + /// Without this gate, `BoundedOrCovered` would take the covered arm + /// (which has no budget dependence) and route on day one with no env + /// var set and no kill switch short of removing the replica URL + /// (Dawn's covered-at-zero-budget catch). The pre-existing cursor + /// paths (`Covered`/`CoveredPostVerified` from channel windows and + /// thread pages) intentionally still route at B=0 — status quo, + /// unchanged. + fn for_query(q: &event::EventQuery, routing_enabled: bool) -> Self { + if !routing_enabled { + return RoutePredicate::Bounded; + } + match (ChannelScoped::from_pinned_channel(q), q.until) { + (Some(proof), Some(upper)) => RoutePredicate::BoundedOrCovered { upper, proof }, + _ => RoutePredicate::Bounded, + } + } +} + +/// Map the configured read budget (`BUZZ_REPLICA_READ_MAX_AGE_MS`) to the +/// runtime gate: `0` disables bounded-staleness routing; anything above the +/// fence staleness gate is clamped to it (an entry older than the staleness +/// gate never routes anyway, so a larger budget would only misrepresent the +/// config). +fn read_budget_from_ms(ms: u64) -> Option { + match ms { + 0 => None, + ms => Some(Duration::from_millis(ms).min(replica_fence::FENCE_STALENESS)), + } } /// Snapshot of Postgres connection pool utilisation. @@ -232,6 +509,9 @@ pub struct DbConfig { pub read_database_url: Option, /// Maximum number of connections in the pool. pub max_connections: u32, + /// Maximum connections in the read-replica pool (env + /// `BUZZ_DB_READ_POOL_SIZE`). `None` inherits [`Self::max_connections`]. + pub read_max_connections: Option, /// Minimum number of idle connections to maintain. pub min_connections: u32, /// Seconds to wait when acquiring a connection before timing out. @@ -240,6 +520,13 @@ pub struct DbConfig { pub max_lifetime_secs: u64, /// Seconds a connection may sit idle before being closed. pub idle_timeout_secs: u64, + /// Replica read budget `B` in milliseconds (bounded arm, env + /// `BUZZ_REPLICA_READ_MAX_AGE_MS`). `0` disables bounded-staleness + /// routing — the rollout default. Values above + /// [`replica_fence::FENCE_STALENESS`] are clamped to it: an entry older + /// than the staleness gate never routes anyway, so a larger budget + /// would only misrepresent the config. + pub replica_read_max_age_ms: u64, } impl Default for DbConfig { @@ -251,10 +538,12 @@ impl Default for DbConfig { database_url: "postgres://buzz:buzz_dev@localhost:5432/buzz".to_string(), // sadscan:disable np.postgres.1 read_database_url: None, max_connections: 20, + read_max_connections: None, min_connections: 2, acquire_timeout_secs: 3, max_lifetime_secs: 1800, idle_timeout_secs: 600, + replica_read_max_age_ms: 0, } } } @@ -361,15 +650,22 @@ impl Db { /// proof hold for every insert path that goes through this pool. pub async fn new(config: &DbConfig) -> Result { let pool = Self::connect_pool(config, &config.database_url, true).await?; + let read_max_connections = config + .read_max_connections + .unwrap_or(config.max_connections); let read_pool = match &config.read_database_url { - Some(url) => Some(Self::connect_pool(config, url, false).await?), + Some(url) => Some(Self::connect_read_pool(config, url, read_max_connections)?), None => None, }; + let replica_read_max_age = read_budget_from_ms(config.replica_read_max_age_ms); Ok(Self { pool, max_connections: config.max_connections, read_pool, + read_max_connections, fence: std::sync::Arc::new(replica_fence::ReplicaFence::new()), + replica_read_max_age, + reader_aurora_identity: std::sync::Arc::new(std::sync::OnceLock::new()), }) } @@ -401,13 +697,90 @@ impl Db { Ok(options.connect(url).await?) } + /// Reader acquire timeout — deliberately far below the writer's + /// (seconds-denominated) timeout. Failing closed to the writer must be + /// fast: a saturated reader pool that made routed reads wait the full + /// writer-style timeout would add dead latency during exactly the load + /// spike the offload exists for. A miss here surfaces as + /// `writer/reader_acquire_timeout` (see [`Db::proved_reader`] for why + /// the reason names the mechanism rather than a diagnosis). + const READER_ACQUIRE_TIMEOUT: Duration = Duration::from_millis(150); + + /// Connect the read-replica pool **lazily** — no connection is + /// attempted at construction, so a reader that is down at boot cannot + /// crash the relay (it starts all-writer with the fence closed and + /// recovers when the replica returns). + /// + /// `min_connections` is pinned to 0 explicitly: sqlx's lazy pool still + /// spawns an eager background connect task to satisfy a nonzero + /// minimum, which would reintroduce boot-time reader dial attempts (and + /// their log noise) that "lazy" is meant to avoid. With 0, connections + /// are dialed only on first acquire; the ~10-minute reaper never tops + /// the pool back up, which is fine — routed reads re-fill it on demand. + /// + /// No floor guard: replica sessions are read-only, the trigger never + /// fires there (see [`Db::connect_pool`]). + fn connect_read_pool(config: &DbConfig, url: &str, max_connections: u32) -> Result { + Ok(PgPoolOptions::new() + .max_connections(max_connections) + .min_connections(0) + .acquire_timeout(Self::READER_ACQUIRE_TIMEOUT) + .max_lifetime(Duration::from_secs(config.max_lifetime_secs)) + .idle_timeout(Duration::from_secs(config.idle_timeout_secs)) + .connect_lazy(url)?) + } + + /// Spawn a one-shot reader reachability probe that only WARNs. + /// + /// With a lazy pool and `min_connections(0)`, nothing dials the replica + /// until the first routed read — so a misconfigured `READ_DATABASE_URL` + /// would otherwise be invisible until traffic arrives and quietly falls + /// back to the writer. This ping is the only boot-time reader-down + /// visibility; it must never gate startup or [`Db::spawn_fence_probe`]. + /// + /// On success it also primes the Aurora identity capability cache + /// ([`Db::reader_aurora_identity`]) on the connection it already holds, + /// so the first routed read doesn't spend a second acquire (up to + /// another [`Db::READER_ACQUIRE_TIMEOUT`]) inside + /// [`Db::reader_aurora_capability_on`]. Prime failure is fine: the routed + /// path re-probes on the connection it already holds, so a failed prime + /// costs a round trip rather than a second acquire budget. + pub fn spawn_read_pool_boot_ping(&self) { + let Some(read_pool) = self.read_pool.clone() else { + return; + }; + let aurora_identity = self.reader_aurora_identity.clone(); + tokio::spawn(async move { + match read_pool.acquire().await { + Ok(mut conn) => { + tracing::info!("read replica reachable at boot"); + match replica_fence::reader_supports_aurora_identity(&mut conn).await { + Ok(supported) => { + let _ = aurora_identity.set(supported); + } + Err(e) => tracing::debug!( + error = %e, + "aurora identity boot prime failed; first routed read will probe" + ), + } + } + Err(e) => tracing::warn!( + "read replica unreachable at boot; serving all-writer until it recovers: {e}" + ), + } + }); + } + /// Creates a `Db` from an existing `PgPool` (useful in tests). pub fn from_pool(pool: PgPool) -> Self { Self { max_connections: pool.options().get_max_connections(), + read_max_connections: pool.options().get_max_connections(), pool, read_pool: None, fence: std::sync::Arc::new(replica_fence::ReplicaFence::new()), + replica_read_max_age: None, + reader_aurora_identity: std::sync::Arc::new(std::sync::OnceLock::new()), } } @@ -421,12 +794,21 @@ impl Db { pub fn from_pools(pool: PgPool, read_pool: PgPool) -> Self { Self { max_connections: pool.options().get_max_connections(), + read_max_connections: read_pool.options().get_max_connections(), pool, read_pool: Some(read_pool), fence: std::sync::Arc::new(replica_fence::ReplicaFence::new()), + replica_read_max_age: None, + reader_aurora_identity: std::sync::Arc::new(std::sync::OnceLock::new()), } } + /// Test hook: set the head-fetch routing budget (Predicate A), which + /// [`Db::from_pools`] leaves disabled. + pub fn set_replica_read_max_age_for_tests(&mut self, budget: Option) { + self.replica_read_max_age = budget; + } + /// The freshness fence gating replica routing (see [`replica_fence`]). pub fn fence(&self) -> &std::sync::Arc { &self.fence @@ -438,7 +820,7 @@ impl Db { /// Ordering matters (Perci, PR #2084 review): this must run **after** /// the migration decision. On a relay with `BUZZ_AUTO_MIGRATE` off, the /// writer pool arms the GUC regardless, but if migration 0021 has not - /// been applied there is no trigger enforcing it — and an LSN probe + /// been applied there is no trigger enforcing it — and a heartbeat probe /// would open the fence over an unenforced floor. So the probe is gated /// on an unconditional two-part verification against the live schema: /// catalog shape ([`replica_fence::verify_floor_guard_catalog`]) and @@ -449,14 +831,13 @@ impl Db { /// stays closed: every cursor page routes to the writer. The relay keeps /// serving — degraded capacity, never holes. pub async fn spawn_fence_probe(&self) -> Result { - let Some(read_pool) = &self.read_pool else { + if self.read_pool.is_none() { return Ok(false); - }; + } replica_fence::verify_floor_guard_catalog(&self.pool).await?; replica_fence::verify_floor_guard_behavior(&self.pool).await?; tokio::spawn(replica_fence::run_probe( self.pool.clone(), - read_pool.clone(), std::sync::Arc::clone(&self.fence), )); Ok(true) @@ -465,11 +846,13 @@ impl Db { /// The pool for lag-tolerant reads: the read replica when configured, /// otherwise the writer pool. /// - /// Routing contract — a query may use this pool only when a stale (bounded - /// replication lag) result is acceptable to its caller. Keyset-cursor - /// pagination over immutable history qualifies; head-of-channel fetches, - /// auth/membership checks, locks, and anything inside a transaction do not. - pub fn read(&self) -> &PgPool { + /// Removed as a public escape hatch (Dawn, review of 1b0aa0dfa): the + /// raw replica pool carries **no fence proof**, which is exactly the + /// bug class the routed-read machinery exists to eliminate. All replica + /// reads must go through [`Db::route_read`]-backed entry points; this + /// remains only for the fence's own plumbing tests. + #[cfg(test)] + fn read(&self) -> &PgPool { self.read_pool.as_ref().unwrap_or(&self.pool) } @@ -478,6 +861,153 @@ impl Db { self.read_pool.is_some() } + /// Open a reader request transaction and complete the connection-local + /// half of the fence proof: `BEGIN ISOLATION LEVEL REPEATABLE READ, READ + /// ONLY`, then observe the heartbeat token/epoch as the transaction's + /// **first statement** — anchoring the snapshot every follow-up + /// statement (page, participants, aux closure) sees to exactly the + /// snapshot the proof was taken against — and resolve it against the + /// retained ring. Returns the open transaction together with the + /// strongest [`replica_fence::TokenEntry`] its observation supports, or + /// the fail-closed reason for route metrics. + /// + /// `REPEATABLE READ` is the strongest isolation a hot standby supports + /// (`SERIALIZABLE` is writer-only); `READ ONLY` documents intent and + /// rejects accidental writes. Everything but `Ok` fails closed — begin + /// failure, missing heartbeat row (migration not yet replayed there), + /// observation error, epoch mismatch, or a token below every retained + /// entry all route the request to the writer. + async fn proved_reader( + &self, + read_pool: &PgPool, + ) -> std::result::Result< + ( + sqlx::Transaction<'static, sqlx::Postgres>, + replica_fence::TokenEntry, + ), + &'static str, + > { + // One checkout per routed read. The Aurora capability probe and the + // read-only transaction share a single `acquire()` so the request path + // spends exactly one READER_ACQUIRE_TIMEOUT budget. Probing through + // `read_pool` separately would spend a second budget whenever the + // capability is uncached — i.e. after a failed boot ping, which is + // precisely the reader-unavailable case the bound must hold for. + let conn = match read_pool.acquire().await { + Ok(conn) => conn, + Err(sqlx::Error::PoolTimedOut) => { + tracing::warn!("reader pool acquire timed out; routing to writer"); + return Err("reader_acquire_timeout"); + } + Err(e) => { + tracing::warn!(error = %e, "reader connection acquire failed; routing to writer"); + return Err("reader_validation_error"); + } + }; + let mut conn = conn; + let aurora = self.reader_aurora_capability_on(&mut conn).await; + let mut tx = match sqlx::Transaction::begin( + conn, + Some(sqlx::SqlStr::from_static( + "BEGIN ISOLATION LEVEL REPEATABLE READ, READ ONLY", + )), + ) + .await + { + Ok(tx) => tx, + // The acquire miss gets its own reason code: the reader pool's + // short acquire timeout (READER_ACQUIRE_TIMEOUT) makes this the + // fast fail-closed path under load, and + // `buzz_db_route_decision{decision="writer",reason="reader_acquire_timeout"}` + // is the operator's alert signal for a struggling reader pool. + // + // The reason deliberately names the mechanism, not a diagnosis: + // `PoolTimedOut` proves only that no connection was handed out + // within the 150ms budget. That budget includes cold connect + // (TCP+TLS+auth), and sqlx's `size` counts in-flight dials, so + // this fires for slow connection establishment as well as for + // established-connection contention — and neither `size == 0` + // nor `size >= max` recovers the missing causal bit (in-flight + // dials hold a size slot, and a cold burst can push + // `active = size - idle` toward max with zero busy connections). + // Runbook: correlate with `buzz_db_read_pool_active` / `_max` + // and reader connection health/latency; high active suggests + // contention, but this metric alone does not distinguish + // contention from slow connects. Note the gauge is a coarse + // sample (BUZZ_POOL_METRICS_INTERVAL_SECS, default 10s) while + // the event it explains lasts ~150ms — a short burst may fall + // between samples entirely, so absence of elevated active is + // NOT evidence of a cold connect. + Err(sqlx::Error::PoolTimedOut) => { + tracing::warn!("reader pool acquire timed out; routing to writer"); + return Err("reader_acquire_timeout"); + } + Err(e) => { + tracing::warn!(error = %e, "reader transaction begin failed; routing to writer"); + return Err("reader_validation_error"); + } + }; + let obs = match replica_fence::observe_heartbeat(&mut tx, aurora).await { + Ok(Some(observation)) => observation, + Ok(None) => return Err("reader_validation_error"), + Err(e) => { + tracing::warn!(error = %e, "heartbeat observation failed; routing to writer"); + return Err("reader_validation_error"); + } + }; + match self.fence.resolve(obs.token, obs.epoch) { + replica_fence::ResolveOutcome::Proved(entry) => { + tracing::debug!( + token = obs.token, + proved_token = entry.token, + backend = %obs.backend, + "reader snapshot proved fence coverage" + ); + Ok((tx, entry)) + } + replica_fence::ResolveOutcome::EpochMismatch => Err("reader_validation_error"), + replica_fence::ResolveOutcome::TokenBehind => Err("reader_token_behind"), + } + } + + /// Whether the reader endpoint supports the Aurora PostgreSQL identity + /// function ([`replica_fence::AURORA_IDENTITY_FN`]), probed + /// once per process and cached (see [`Db::reader_aurora_identity`]). + /// The probe runs on a plain autocommit checkout — never inside the + /// request transaction, where an undefined-function error would abort + /// it. Probe failure (acquire or transient) degrades to the plain + /// identity tuple for THIS request without caching, so a later request + /// retries; identity is evidence, never a routing gate. + /// Aurora capability on a connection the caller already holds, so the + /// routed path never spends a second acquire budget. + async fn reader_aurora_capability_on( + &self, + conn: &mut sqlx::pool::PoolConnection, + ) -> bool { + if let Some(cached) = self.reader_aurora_identity.get() { + return *cached; + } + match replica_fence::reader_supports_aurora_identity(conn).await { + Ok(supported) => *self.reader_aurora_identity.get_or_init(|| supported), + Err(e) => { + tracing::debug!(error = %e, "aurora identity probe failed; will retry"); + false + } + } + } + + /// Record one route decision (Rev 2 observability): which path, where it + /// went, and why. + fn record_route(path: &'static str, decision: &'static str, reason: &'static str) { + metrics::counter!( + "buzz_db_route_decision", + "path" => path, + "decision" => decision, + "reason" => reason, + ) + .increment(1); + } + /// Run pending database migrations. pub async fn migrate(&self) -> Result<()> { migration::run_migrations(&self.pool).await @@ -502,11 +1032,18 @@ impl Db { } /// Pool utilisation stats for the read-replica pool, when configured. + /// + /// `max` is the **reader's** ceiling ([`Db::read_max_connections`]), not + /// the writer's: `buzz_db_read_pool_active / buzz_db_read_pool_max` is + /// the operator's utilisation signal for tuning `BUZZ_DB_READ_POOL_SIZE`, + /// and deriving it from the writer's max would misreport saturation by + /// exactly the ratio of the two pool sizes — in the direction that hides + /// the problem. pub fn read_pool_stats(&self) -> Option { self.read_pool.as_ref().map(|p| DbPoolStats { size: p.size(), idle: p.num_idle() as u32, - max: self.max_connections, + max: self.read_max_connections, }) } @@ -1094,15 +1631,126 @@ impl Db { } /// Queries events matching the given filter parameters. + /// + /// Always reads from the WRITER pool. If the result influences a write + /// or a permission decision, this is the method to call. Display-path + /// callers that tolerate bounded staleness should use + /// [`Db::query_events_routed`] instead — converting a caller is an + /// explicit, per-callsite decision, never a change to this method. pub async fn query_events(&self, q: &EventQuery) -> Result> { event::query_events(&self.pool, q).await } + /// [`Db::query_events`] with replica routing — the opt-in fast path for + /// display reads. + /// + /// Rule of thumb: **if the result influences a write or a permission, + /// it reads from the writer** — do not convert such a caller to this + /// method. Every new caller must be added to the caller-classification + /// table in `PLANS/REPLICA_FULL_READ_ROUTING_DESIGN.md`. + /// + /// Routing derives the strongest sound predicate from the query shape + /// ([`RoutePredicate::for_query`]): a channel-pinned query with an + /// `until` upper bound may be served covered (provably complete below + /// the fence wall); anything else is bounded-staleness only. The whole + /// seam is gated on `BUZZ_REPLICA_READ_MAX_AGE_MS` (default off): when + /// unset, even covered-eligible queries stay on the writer, so merging + /// this seam is a true no-op until the budget is configured. Every + /// failure fails closed to the writer. + pub async fn query_events_routed( + &self, + path: &'static str, + q: &EventQuery, + ) -> Result> { + let predicate = RoutePredicate::for_query(q, self.replica_read_max_age.is_some()); + match self.route_read(path, predicate).await { + RouteDecision::Replica(mut tx, _entry, reason) => { + match event::query_events_on(&mut tx, q).await { + Ok(events) => { + Self::record_route(path, "replica", reason); + Ok(events) + } + Err(e) => { + // Mid-query replica failure: fail closed to the + // writer rather than surfacing a routed error. + tracing::warn!(path, "replica read failed; re-running on writer: {e}"); + Self::record_route(path, "writer", "replica_error"); + event::query_events(&self.pool, q).await + } + } + } + RouteDecision::Writer => event::query_events(&self.pool, q).await, + } + } + + /// [`Db::query_events_routed`] restricted to the BOUNDED arm — for + /// reads whose result feeds a COUNT rather than a displayed page. + /// + /// The covered arm bounds insert-completeness only; stale deletions can + /// briefly inflate the result set (see [`RoutePredicate::Covered`]). A + /// display page absorbs that per-row; a number derived from the rows + /// does not. Same classification-table requirement as + /// [`Db::query_events_routed`]. + pub async fn query_events_routed_bounded( + &self, + path: &'static str, + q: &EventQuery, + ) -> Result> { + match self.route_read(path, RoutePredicate::Bounded).await { + RouteDecision::Replica(mut tx, _entry, reason) => { + match event::query_events_on(&mut tx, q).await { + Ok(events) => { + Self::record_route(path, "replica", reason); + Ok(events) + } + Err(e) => { + tracing::warn!(path, "replica read failed; re-running on writer: {e}"); + Self::record_route(path, "writer", "replica_error"); + event::query_events(&self.pool, q).await + } + } + } + RouteDecision::Writer => event::query_events(&self.pool, q).await, + } + } + /// Count events matching the given query (NIP-45 COUNT support). + /// + /// Always reads from the WRITER pool — see [`Db::query_events`] for the + /// writer-vs-routed rule. pub async fn count_events(&self, q: &EventQuery) -> Result { event::count_events(&self.pool, q).await } + /// [`Db::count_events`] with replica routing — same contract, rules, + /// and classification-table requirement as [`Db::query_events_routed`]. + /// + /// Counts route on the BOUNDED arm only, never covered: the covered + /// arm bounds insert-completeness but not deletion visibility (soft + /// deletes are UPDATEs outside the floor guard), and a count has no + /// downstream per-row re-filter to absorb extra rows — a silently + /// inflated number for up to `FENCE_STALENESS` is a different product + /// statement than a page briefly showing a deleted row. `Bounded` ties + /// the error to the accepted budget `B`. + pub async fn count_events_routed(&self, path: &'static str, q: &EventQuery) -> Result { + match self.route_read(path, RoutePredicate::Bounded).await { + RouteDecision::Replica(mut tx, _entry, reason) => { + match event::count_events_on(&mut tx, q).await { + Ok(count) => { + Self::record_route(path, "replica", reason); + Ok(count) + } + Err(e) => { + tracing::warn!(path, "replica count failed; re-running on writer: {e}"); + Self::record_route(path, "writer", "replica_error"); + event::count_events(&self.pool, q).await + } + } + } + RouteDecision::Writer => event::count_events(&self.pool, q).await, + } + } + /// Return whether a creator-signed huddle-start event links a parent /// channel to an ephemeral huddle channel. pub async fn huddle_started_link_exists( @@ -1165,16 +1813,27 @@ impl Db { event::soft_delete_event(&self.pool, community_id, event_id).await } - /// Soft-delete the live row for an addressable coordinate `(kind, pubkey, d_tag)`. - /// Used by NIP-09 a-tag deletion for parameterized-replaceable kinds. + /// Soft-delete the live row for an addressable coordinate `(kind, pubkey, d_tag)` + /// when it is not newer than the deletion request. + /// Used by NIP-09 a-tag deletion for parameterized-replaceable kinds; + /// `deletion_created_at_secs` is the deletion event's `created_at`. pub async fn soft_delete_by_coordinate( &self, community_id: CommunityId, kind: i32, pubkey: &[u8], d_tag: &str, + deletion_created_at_secs: i64, ) -> Result { - event::soft_delete_by_coordinate(&self.pool, community_id, kind, pubkey, d_tag).await + event::soft_delete_by_coordinate( + &self.pool, + community_id, + kind, + pubkey, + d_tag, + deletion_created_at_secs, + ) + .await } /// Atomically soft-delete an event and decrement thread reply counters. @@ -1222,6 +1881,37 @@ impl Db { event::get_events_by_ids(&self.pool, community_id, ids).await } + /// [`Db::get_events_by_ids`] with replica routing — same contract and + /// classification-table requirement as [`Db::query_events_routed`]. + /// + /// By-id fetches route on the BOUNDED arm only: an id list carries no + /// channel pin, so no fence floor can prove insert-completeness — the + /// covered arm is structurally unavailable. Used for FTS hit hydration, + /// where a missing row degrades to a skipped search hit downstream. + pub async fn get_events_by_ids_routed( + &self, + path: &'static str, + community_id: CommunityId, + ids: &[&[u8]], + ) -> Result> { + match self.route_read(path, RoutePredicate::Bounded).await { + RouteDecision::Replica(mut tx, _entry, reason) => { + match event::get_events_by_ids_on(&mut tx, community_id, ids).await { + Ok(events) => { + Self::record_route(path, "replica", reason); + Ok(events) + } + Err(e) => { + tracing::warn!(path, "replica read failed; re-running on writer: {e}"); + Self::record_route(path, "writer", "replica_error"); + event::get_events_by_ids(&self.pool, community_id, ids).await + } + } + } + RouteDecision::Writer => event::get_events_by_ids(&self.pool, community_id, ids).await, + } + } + /// Exclusively claim a batch of due matcher jobs from one community. pub async fn claim_due_push_match_batch( &self, @@ -1990,19 +2680,25 @@ impl Db { /// Fetch replies under a root event. /// - /// Routing: the head fetch (`cursor: None`) always reads the writer. - /// Cursor-bearing pages may read the replica pool when one is configured - /// AND the freshness fence is open ([`replica_fence`]). Thread pagination - /// walks forward from oldest to newest, so a replica page is served only - /// when it is provably complete: + /// Routing mirrors [`Db::get_channel_window_with_session`]: a head + /// fetch (`cursor: None`) is Predicate A (bounded staleness, gated by + /// the default-off head budget); cursor pages are Predicate B + /// (completeness). Thread pagination walks **forward** from oldest to + /// newest, so a cursor carries no upper bound — instead the served page + /// is post-verified against the wall the serving session proved: /// /// - an under-`limit` page is a candidate terminal page — the client /// treats it as EOF, so it is re-run on the writer to keep the EOF /// decision authoritative (a lagged replica could truncate the tail); - /// - a full page whose newest row exceeds the fence could straddle a row - /// the replica has not replayed (commit order is not `created_at` - /// order), so it is also re-run on the writer. Only a full page that - /// sits entirely at or below the fence is served from the replica. + /// - a full page whose newest row exceeds the proved fence wall could + /// straddle a row the session has not replayed (commit order is not + /// `created_at` order), so it is also re-run on the writer. Only a + /// full page that sits entirely at or below the proved wall is served + /// from the replica. + /// + /// A head fetch routed under Predicate A skips the re-run: bounded + /// staleness (missing at most the freshest budget-window of replies) is + /// exactly the semantic the head gate accepts. pub async fn get_thread_replies( &self, community_id: CommunityId, @@ -2011,25 +2707,59 @@ impl Db { limit: u32, cursor: Option<&[u8]>, ) -> Result> { - if cursor.is_some() && self.has_read_pool() && self.fence.verified_through().is_some() { - let replies = thread::get_thread_replies( - self.read(), + let (path, predicate): (&'static str, RoutePredicate) = match cursor { + Some(_) => ( + "thread_cursor", + RoutePredicate::CoveredPostVerified { + proof: ChannelScoped::from_thread_metadata_join(), + }, + ), + None => ("thread_head", RoutePredicate::Bounded), + }; + if let RouteDecision::Replica(mut tx, entry, reason) = + self.route_read(path, predicate).await + { + match thread::get_thread_replies_on( + &mut tx, community_id, root_event_id, depth_limit, limit, cursor, ) - .await?; - let full = replies.len() >= limit as usize; - let below_fence = replies - .last() - .is_some_and(|tail| self.fence.covers(tail.created_at)); - if full && below_fence { - return Ok(replies); + .await + { + Ok(replies) => { + if cursor.is_none() { + // Predicate A: bounded-stale head page, served as proved. + Self::record_route(path, "replica", reason); + return Ok(replies); + } + let full = replies.len() >= limit as usize; + let below_fence = replies + .last() + .is_some_and(|tail| tail.created_at <= entry.fence_wall); + if full && below_fence { + Self::record_route(path, "replica", reason); + return Ok(replies); + } + // Candidate terminal page, or page reaching above the + // proved wall — verify against the writer. Recorded as + // the request's ONLY route event: the replica leg was + // discarded, so counting it would overstate offload. + Self::record_route("thread_eof", "writer", "stale"); + } + Err(e) => { + // Mid-request replica failure (e.g. a hot-standby + // recovery conflict) fails closed to the writer. + tracing::warn!( + error = %e, + path, + "replica thread query failed; re-running on writer" + ); + Self::record_route(path, "writer", "replica_error"); + } } - // Candidate terminal page, or page reaching above the fence — - // verify against the writer. } thread::get_thread_replies( &self.pool, @@ -2053,15 +2783,8 @@ impl Db { /// One channel window: top-level rows + summaries + server `has_more`. /// - /// Routing: the head fetch (`cursor: None`) always reads the writer — it - /// must include just-committed events. A cursor-bearing page scrolls - /// *backward* into history bounded above by the cursor timestamp - /// (`created_at < ts`, or `= ts` with the id tiebreak), so it may read - /// the replica when one is configured AND the freshness fence covers the - /// cursor timestamp: every row the page could contain is then provably - /// replayed on the replica ([`replica_fence`]). Pages whose cursor - /// reaches above the fence — the freshest sliver of history — stay on - /// the writer. + /// Convenience wrapper over [`Db::get_channel_window_with_session`] for + /// callers with no follow-up queries; the serving session is released. pub async fn get_channel_window( &self, community_id: CommunityId, @@ -2070,11 +2793,199 @@ impl Db { cursor: Option<(DateTime, Vec)>, kind_filter: Option<&[u32]>, ) -> Result { - let pool = match &cursor { - Some((ts, _)) if self.has_read_pool() && self.fence.covers(*ts) => self.read(), - _ => &self.pool, + self.get_channel_window_with_session(community_id, channel_id, limit, cursor, kind_filter) + .await + .map(|(window, _session)| window) + } + + /// [`Db::get_channel_window`], additionally returning the session that + /// served the page so request-scoped follow-ups (the aux closure) run on + /// the same proved connection. + /// + /// Routing: + /// + /// - **Cursor page** (Predicate B — completeness): scrolls *backward* + /// into history bounded above by the cursor timestamp (`created_at < + /// ts`, or `= ts` with the id tiebreak), so it may be served by a + /// replica session when one is configured AND that session **proves** + /// coverage of the cursor timestamp: the heartbeat token/epoch is + /// observed on the exact connection that will serve the page and + /// resolved against the fence's retained ring ([`replica_fence`]). + /// - **Head fetch** (Predicate A — bounded staleness): served by a + /// proved replica session only when the head gate is configured + /// ([`DbConfig::replica_read_max_age_ms`], default off) and the + /// proved entry is within the budget. This trades a bounded staleness + /// window (budget plus probe cadence) on the GET leg for writer + /// offload. NOTE: enabling the budget also breaks read-your-own-writes + /// on the GET leg; the client-side WS `since`-overlap union intended + /// to cover fresh events has NOT shipped yet — do not enable + /// `BUZZ_REPLICA_HEAD_MAX_AGE_SECS` until it has, proven by a + /// post-then-immediately-refetch test. + /// + /// Every failure fails closed to the writer and is recorded in + /// `buzz_db_route_decision`. + pub async fn get_channel_window_with_session( + &self, + community_id: CommunityId, + channel_id: Uuid, + limit: u32, + cursor: Option<(DateTime, Vec)>, + kind_filter: Option<&[u32]>, + ) -> Result<(thread::ChannelWindow, ReadSession)> { + let path: &'static str = if cursor.is_some() { + "channel_cursor" + } else { + "channel_head" + }; + match self + .route_read( + path, + RoutePredicate::from_channel_cursor(channel_id, &cursor), + ) + .await + { + RouteDecision::Replica(mut tx, _entry, reason) => { + match thread::get_channel_window_on( + &mut tx, + community_id, + channel_id, + limit, + cursor.clone(), + kind_filter, + ) + .await + { + Ok(window) => { + Self::record_route(path, "replica", reason); + return Ok(( + window, + ReadSession { + inner: ReadSessionInner::Replica { + tx, + writer: self.pool.clone(), + }, + }, + )); + } + Err(e) => { + // A mid-request replica failure (e.g. a hot-standby + // recovery conflict cancelling the held snapshot) + // fails closed to the writer: a stale-but-served + // page, never an error the writer could have + // answered. Dropping `tx` rolls the reader + // transaction back. + tracing::warn!( + error = %e, + path, + "replica window query failed; re-running on writer" + ); + Self::record_route(path, "writer", "replica_error"); + } + } + } + RouteDecision::Writer => {} + } + let window = thread::get_channel_window( + &self.pool, + community_id, + channel_id, + limit, + cursor, + kind_filter, + ) + .await?; + Ok(( + window, + ReadSession { + inner: ReadSessionInner::Writer(self.pool.clone()), + }, + )) + } + + /// Shared route decision for one read: evaluate the predicate against a + /// proved reader session and record the decision. Fail closed to the + /// writer everywhere. + async fn route_read(&self, path: &'static str, predicate: RoutePredicate) -> RouteDecision { + let Some(read_pool) = &self.read_pool else { + Self::record_route(path, "writer", "disabled"); + return RouteDecision::Writer; + }; + // Cheap prechecks on the shared ring before spending a reader + // checkout; the connection-local observation still has to prove it. + let Some(newest) = self.fence.newest() else { + Self::record_route(path, "writer", "uninitialized"); + return RouteDecision::Writer; + }; + // Precheck helpers against the newest shared entry: if the newest + // cannot satisfy an arm, no proved (older-or-equal) entry can. + let bounded_precheck = + |budget: &Option| -> std::result::Result<(), &'static str> { + match budget { + Some(budget) if newest.committed_at.elapsed() <= *budget => Ok(()), + Some(_) => Err("stale"), + None => Err("disabled"), + } + }; + let covered_precheck = |upper: &DateTime| -> std::result::Result<(), &'static str> { + if *upper <= newest.fence_wall { + Ok(()) + } else { + Err("stale") + } }; - thread::get_channel_window(pool, community_id, channel_id, limit, cursor, kind_filter).await + let precheck = match &predicate { + RoutePredicate::Bounded => bounded_precheck(&self.replica_read_max_age), + RoutePredicate::Covered { upper, .. } => covered_precheck(upper), + // No upper bound: the caller post-verifies served rows. + RoutePredicate::CoveredPostVerified { .. } => Ok(()), + // Covered first (no budget dependence), else bounded. + RoutePredicate::BoundedOrCovered { upper, .. } => { + covered_precheck(upper).or_else(|_| bounded_precheck(&self.replica_read_max_age)) + } + }; + if let Err(reason) = precheck { + Self::record_route(path, "writer", reason); + return RouteDecision::Writer; + } + match self.proved_reader(read_pool).await { + Ok((tx, entry)) => { + // Re-evaluate against the entry the session actually proved + // (it may be older than the shared newest). + let bounded_holds = || { + self.replica_read_max_age + .is_some_and(|budget| entry.committed_at.elapsed() <= budget) + }; + let verdict: Option<&'static str> = match &predicate { + RoutePredicate::Bounded => bounded_holds().then_some("fresh"), + RoutePredicate::Covered { upper, .. } => { + (*upper <= entry.fence_wall).then_some("covered") + } + // No upper bound: the caller post-verifies the served + // rows against the proved wall. + RoutePredicate::CoveredPostVerified { .. } => Some("covered"), + RoutePredicate::BoundedOrCovered { upper, .. } => { + if *upper <= entry.fence_wall { + Some("covered") + } else { + bounded_holds().then_some("fresh") + } + } + }; + match verdict { + Some(reason) => RouteDecision::Replica(tx, entry, reason), + None => { + // The session proves an older entry than the + // predicate needs (replication lag) — fail closed. + Self::record_route(path, "writer", "stale"); + RouteDecision::Writer + } + } + } + Err(reason) => { + Self::record_route(path, "writer", reason); + RouteDecision::Writer + } + } } /// Look up a single thread_metadata row by event_id. @@ -2239,6 +3150,67 @@ impl Db { .await } + /// [`Db::query_feed_mentions`] with replica routing — same contract and + /// classification-table requirement as [`Db::query_events_routed`]. + /// + /// Feed queries route on the BOUNDED arm only: the `accessible_channel_ids` + /// parameter admits community-global rows alongside channel rows, so no + /// single channel's fence floor can prove completeness — the covered arm + /// is structurally unavailable, not merely unchosen. + pub async fn query_feed_mentions_routed( + &self, + path: &'static str, + community: CommunityId, + pubkey_bytes: &[u8], + accessible_channel_ids: &[Uuid], + since: Option>, + limit: i64, + ) -> Result> { + match self.route_read(path, RoutePredicate::Bounded).await { + RouteDecision::Replica(mut tx, _entry, reason) => { + match feed::query_mentions_on( + &mut tx, + community, + pubkey_bytes, + accessible_channel_ids, + since, + limit, + ) + .await + { + Ok(events) => { + Self::record_route(path, "replica", reason); + Ok(events) + } + Err(e) => { + tracing::warn!(path, "replica read failed; re-running on writer: {e}"); + Self::record_route(path, "writer", "replica_error"); + feed::query_mentions( + &self.pool, + community, + pubkey_bytes, + accessible_channel_ids, + since, + limit, + ) + .await + } + } + } + RouteDecision::Writer => { + feed::query_mentions( + &self.pool, + community, + pubkey_bytes, + accessible_channel_ids, + since, + limit, + ) + .await + } + } + } + /// Find events that require action from the given pubkey. pub async fn query_feed_needs_action( &self, @@ -2259,6 +3231,63 @@ impl Db { .await } + /// [`Db::query_feed_needs_action`] with replica routing — BOUNDED arm + /// only; see [`Db::query_feed_mentions_routed`] for why the covered arm + /// is structurally unavailable to feed queries. + pub async fn query_feed_needs_action_routed( + &self, + path: &'static str, + community: CommunityId, + pubkey_bytes: &[u8], + accessible_channel_ids: &[Uuid], + since: Option>, + limit: i64, + ) -> Result> { + match self.route_read(path, RoutePredicate::Bounded).await { + RouteDecision::Replica(mut tx, _entry, reason) => { + match feed::query_needs_action_on( + &mut tx, + community, + pubkey_bytes, + accessible_channel_ids, + since, + limit, + ) + .await + { + Ok(events) => { + Self::record_route(path, "replica", reason); + Ok(events) + } + Err(e) => { + tracing::warn!(path, "replica read failed; re-running on writer: {e}"); + Self::record_route(path, "writer", "replica_error"); + feed::query_needs_action( + &self.pool, + community, + pubkey_bytes, + accessible_channel_ids, + since, + limit, + ) + .await + } + } + } + RouteDecision::Writer => { + feed::query_needs_action( + &self.pool, + community, + pubkey_bytes, + accessible_channel_ids, + since, + limit, + ) + .await + } + } + } + /// Find recent activity across accessible channels. pub async fn query_feed_activity( &self, @@ -2270,6 +3299,53 @@ impl Db { feed::query_activity(&self.pool, community, accessible_channel_ids, since, limit).await } + /// [`Db::query_feed_activity`] with replica routing — BOUNDED arm only; + /// see [`Db::query_feed_mentions_routed`] for why the covered arm is + /// structurally unavailable to feed queries. + pub async fn query_feed_activity_routed( + &self, + path: &'static str, + community: CommunityId, + accessible_channel_ids: &[Uuid], + since: Option>, + limit: i64, + ) -> Result> { + match self.route_read(path, RoutePredicate::Bounded).await { + RouteDecision::Replica(mut tx, _entry, reason) => { + match feed::query_activity_on( + &mut tx, + community, + accessible_channel_ids, + since, + limit, + ) + .await + { + Ok(events) => { + Self::record_route(path, "replica", reason); + Ok(events) + } + Err(e) => { + tracing::warn!(path, "replica read failed; re-running on writer: {e}"); + Self::record_route(path, "writer", "replica_error"); + feed::query_activity( + &self.pool, + community, + accessible_channel_ids, + since, + limit, + ) + .await + } + } + } + RouteDecision::Writer => { + feed::query_activity(&self.pool, community, accessible_channel_ids, since, limit) + .await + } + } + } + /// Create a new API token record. #[allow(clippy::too_many_arguments)] pub async fn create_api_token( @@ -2923,8 +3999,33 @@ impl Db { } /// Returns `true` if `pubkey` (64-char hex) is a member of `community`. + /// + /// Replica-routed on the bounded arm — the one PERMISSION read routed by + /// explicit product decision (bounded-stale membership beats the 10s + /// cache it replaced). Admits and revokes may lag by at most the budget + /// `B`; everything else fails closed to the writer, exactly like + /// [`Db::query_events_routed_bounded`]. Not precedent for routing other + /// permission reads. pub async fn is_relay_member(&self, community: CommunityId, pubkey: &str) -> Result { - relay_members::is_relay_member(&self.pool, community, pubkey).await + let path = "relay_membership"; + match self.route_read(path, RoutePredicate::Bounded).await { + RouteDecision::Replica(mut tx, _entry, reason) => { + match relay_members::is_relay_member_on(&mut tx, community, pubkey).await { + Ok(is_member) => { + Self::record_route(path, "replica", reason); + Ok(is_member) + } + Err(e) => { + tracing::warn!(path, "replica read failed; re-running on writer: {e}"); + Self::record_route(path, "writer", "replica_error"); + relay_members::is_relay_member(&self.pool, community, pubkey).await + } + } + } + RouteDecision::Writer => { + relay_members::is_relay_member(&self.pool, community, pubkey).await + } + } } /// Returns the relay member record for `pubkey` in `community`, or `None` if not found. @@ -3020,6 +4121,12 @@ impl Db { relay_members::bootstrap_owner(&self.pool, community, owner_pubkey).await } + /// Returns `true` if any member of `community` holds the `admin` or + /// `owner` role. + pub async fn has_admin_or_owner(&self, community: CommunityId) -> Result { + relay_members::has_admin_or_owner(&self.pool, community).await + } + /// Atomically transfers ownership of `community` to `new_owner_pubkey`, /// demoting the previous owner(s) to `member`. Verifies /// `expected_owner_pubkey` matches the current owner inside the same @@ -4164,23 +5271,92 @@ mod tests { #[tokio::test] #[ignore = "requires Postgres"] - async fn duplicate_nip_rs_discriminator_tags_keep_legacy_retention() { + async fn coordinate_delete_spares_head_newer_than_the_deletion() { use nostr::{EventBuilder, Keys, Kind, Tag, Timestamp}; let db = setup_db().await; let community = CommunityId::from_uuid(make_community(&db.pool).await); let keys = Keys::generate(); + let kind = buzz_core::kind::KIND_PROJECT as i32; + let d_tag = "stale-tombstone-project"; + let pubkey = keys.public_key().to_bytes().to_vec(); let base = Timestamp::now().as_secs(); - for (case, tags) in [ - ( - "duplicate-d", - vec![ - Tag::parse(["d", &format!("read-state:{}", "c".repeat(32))]) - .expect("first d tag"), - Tag::parse(["d", &format!("read-state:{}", "d".repeat(32))]) - .expect("second d tag"), - Tag::parse(["t", "read-state"]).expect("t tag"), + let version = |content: &str, offset: u64| { + EventBuilder::new(Kind::Custom(buzz_core::kind::KIND_PROJECT as u16), content) + .tags(vec![Tag::parse(["d", d_tag]).expect("d tag")]) + .custom_created_at(Timestamp::from(base + offset)) + .sign_with_keys(&keys) + .expect("sign project version") + }; + + for (content, offset) in [("v1", 0), ("v2", 100)] { + assert!( + db.replace_parameterized_event(community, &version(content, offset), d_tag, None) + .await + .expect("store project version") + .1 + ); + } + + // Tombstone timestamped between V1 and V2: it authorizes deleting V1, + // never the newer head that replaced it. + let stale_deleted = db + .soft_delete_by_coordinate(community, kind, &pubkey, d_tag, (base + 50) as i64) + .await + .expect("stale coordinate delete"); + assert!( + !stale_deleted, + "a tombstone older than the live head must delete nothing" + ); + + let live_content: Option = sqlx::query_scalar( + "SELECT content FROM events \ + WHERE community_id=$1 AND kind=$2 AND pubkey=$3 AND d_tag=$4 AND deleted_at IS NULL", + ) + .bind(community.as_uuid()) + .bind(kind) + .bind(&pubkey) + .bind(d_tag) + .fetch_optional(&db.pool) + .await + .expect("read live head"); + assert_eq!( + live_content.as_deref(), + Some("v2"), + "the newer head must survive a stale tombstone" + ); + + // A tombstone at or after the head's own timestamp still deletes it. + let current_deleted = db + .soft_delete_by_coordinate(community, kind, &pubkey, d_tag, (base + 100) as i64) + .await + .expect("current coordinate delete"); + assert!( + current_deleted, + "a tombstone at the head's timestamp must delete it (NIP-09 is at-or-before)" + ); + } + + #[tokio::test] + #[ignore = "requires Postgres"] + async fn duplicate_nip_rs_discriminator_tags_keep_legacy_retention() { + use nostr::{EventBuilder, Keys, Kind, Tag, Timestamp}; + + let db = setup_db().await; + let community = CommunityId::from_uuid(make_community(&db.pool).await); + let keys = Keys::generate(); + let base = Timestamp::now().as_secs(); + + for (case, tags) in [ + ( + "duplicate-d", + vec![ + Tag::parse(["d", &format!("read-state:{}", "c".repeat(32))]) + .expect("first d tag"), + Tag::parse(["d", &format!("read-state:{}", "d".repeat(32))]) + .expect("second d tag"), + Tag::parse(["t", "read-state"]).expect("t tag"), ], ), ( @@ -4715,15 +5891,21 @@ mod tests { #[tokio::test] #[ignore = "requires Postgres"] async fn test_usage_metrics_lock_has_single_owner_and_releases_on_drop() { - let database_url = - std::env::var("TEST_DATABASE_URL").unwrap_or_else(|_| TEST_DB_URL.into()); - let pool = PgPoolOptions::new() - .max_connections(2) - .connect(&database_url) + // Use a private scratch database — not the shared TEST_DATABASE_URL. + // Postgres advisory locks are per-database; hardcoding the production + // USAGE_METRICS_LOCK_KEY (0x4255_5A5A_4D45_5452) on the shared test DB + // races any live buzz-relay on the same database (see #3619). + let admin_url = std::env::var("TEST_DATABASE_URL").unwrap_or_else(|_| TEST_DB_URL.into()); + let admin = PgPoolOptions::new() + .max_connections(1) + .connect(&admin_url) .await - .expect("connect to test DB"); + .expect("connect admin to create scratch db"); + let (pool, scratch_name) = create_scratch_db(&admin, "usage_metrics_lock").await; let first = Db::from_pool(pool.clone()); - let second = Db::from_pool(pool); + let second = Db::from_pool(pool.clone()); + // Same key as production (`buzz-relay` USAGE_METRICS_LOCK_KEY) — safe here + // because the scratch DB is empty of other holders. let key = 0x4255_5A5A_4D45_5452; let mut leader = first @@ -4750,6 +5932,11 @@ mod tests { .is_some(), "dropping the detached session releases its advisory lock" ); + + // Release any remaining session state before DROP DATABASE. + drop(first); + drop(second); + drop_scratch_db(&admin, pool, &scratch_name).await; } #[tokio::test] @@ -5436,6 +6623,163 @@ mod tests { assert!(db.read_pool_stats().is_none()); } + #[test] + fn read_budget_zero_disables_and_large_values_clamp_to_staleness() { + assert_eq!(read_budget_from_ms(0), None, "0 = bounded routing off"); + assert_eq!( + read_budget_from_ms(1000), + Some(std::time::Duration::from_millis(1000)) + ); + assert_eq!( + read_budget_from_ms(10_000_000), + Some(replica_fence::FENCE_STALENESS), + "budgets above the staleness gate clamp to it" + ); + } + + /// Truth table for [`RoutePredicate::for_query`]: the strongest sound + /// predicate per query shape, and — the deploy-day default row — that + /// `routing_enabled = false` (BUZZ_REPLICA_READ_MAX_AGE_MS unset) + /// forces `Bounded` even for covered-eligible shapes, so the zero + /// budget fails the new seams closed (Dawn's covered-at-zero-budget + /// catch, design doc rev 5). + #[test] + fn for_query_predicate_truth_table() { + let community = CommunityId::from_uuid(Uuid::new_v4()); + let channel = Uuid::new_v4(); + let until = chrono::Utc::now(); + + let pinned_with_until = { + let mut q = event::EventQuery::for_community(community); + q.channel_id = Some(channel); + q.until = Some(until); + q + }; + let pinned_no_until = { + let mut q = event::EventQuery::for_community(community); + q.channel_id = Some(channel); + q + }; + let unpinned_with_until = { + let mut q = event::EventQuery::for_community(community); + q.until = Some(until); + q + }; + let global_only = { + let mut q = event::EventQuery::for_community(community); + q.global_only = true; + q.until = Some(until); + q + }; + + // Deploy-day default: budget unset ⇒ Bounded regardless of shape. + // The zero budget then fails Bounded closed, so the new seams + // record writer/disabled — merging with no env var set is a no-op. + assert!( + matches!( + RoutePredicate::for_query(&pinned_with_until, false), + RoutePredicate::Bounded + ), + "budget unset must not reach the covered arm even when eligible" + ); + + // Budget set + channel pin + until ⇒ the strongest predicate. + assert!(matches!( + RoutePredicate::for_query(&pinned_with_until, true), + RoutePredicate::BoundedOrCovered { .. } + )); + + // Missing either covered precondition ⇒ Bounded. + assert!(matches!( + RoutePredicate::for_query(&pinned_no_until, true), + RoutePredicate::Bounded + )); + assert!(matches!( + RoutePredicate::for_query(&unpinned_with_until, true), + RoutePredicate::Bounded + )); + // global_only implies `channel_id = None`, so the channel-pin + // precondition fails and no covered arm is possible — `for_query` + // never inspects `global_only` itself; the row holds because + // constructor 1 (channel pin) returns None for an unpinned query. + assert!(matches!( + RoutePredicate::for_query(&global_only, true), + RoutePredicate::Bounded + )); + } + + /// The pre-existing cursor paths are NOT budget-gated: a channel-window + /// cursor page still derives `Covered` with no `routing_enabled` input + /// at all — at B=0 today it routes covered, and that status quo is + /// intentionally unchanged by the `for_query` gate (Max's matrix row: + /// old paths route at budget-unset; only the new seams go dark). + #[test] + fn channel_cursor_predicate_is_not_budget_gated() { + let channel = Uuid::new_v4(); + let cursor = Some((chrono::Utc::now(), vec![1u8; 32])); + assert!(matches!( + RoutePredicate::from_channel_cursor(channel, &cursor), + RoutePredicate::Covered { .. } + )); + // Head fetch (no cursor) is bounded — gated by the budget. + assert!(matches!( + RoutePredicate::from_channel_cursor(channel, &None), + RoutePredicate::Bounded + )); + } + + /// D5 wiring: `read_pool_stats().max` must be the READER pool's own + /// ceiling, not the writer's — `buzz_db_read_pool_active / _max` is the + /// operator's utilisation signal and inheriting the writer's max hides + /// reader saturation by exactly the sizing ratio. Pure wiring test: + /// `connect_lazy` never touches the network, but it does spawn the + /// pool reaper task, which needs a Tokio runtime — hence + /// `#[tokio::test]` despite the test body itself never awaiting. + #[tokio::test] + async fn read_pool_stats_reports_reader_ceiling_not_writer() { + let writer = sqlx::postgres::PgPoolOptions::new() + .max_connections(20) + .connect_lazy(TEST_DB_URL) + .expect("lazy writer pool"); + let reader = sqlx::postgres::PgPoolOptions::new() + .max_connections(40) + .connect_lazy(TEST_DB_URL) + .expect("lazy reader pool"); + let db = Db::from_pools(writer, reader); + assert_eq!(db.pool_stats().max, 20); + assert_eq!( + db.read_pool_stats().expect("read pool configured").max, + 40, + "reader gauge must report the reader's own ceiling" + ); + } + + /// D4 wiring: the reader pool is built lazily with `min_connections(0)` + /// and the short reader acquire timeout — construction must succeed + /// with no replica listening (reader-down at boot must not crash the + /// relay), and `read_max_connections` must honour + /// `DbConfig::read_max_connections` over the writer sizing. + /// `#[tokio::test]` because `connect_lazy` spawns the pool reaper task, + /// which needs a Tokio runtime even though nothing is dialed. + #[tokio::test] + async fn connect_read_pool_is_lazy_and_independently_sized() { + let config = DbConfig { + max_connections: 20, + read_max_connections: Some(7), + ..DbConfig::default() + }; + // Unroutable per RFC 5737 TEST-NET-1: proves nothing is dialed at + // construction time. + let pool = Db::connect_read_pool(&config, "postgres://user:pw@192.0.2.1:5432/none", 7) + .expect("lazy construction must not dial the replica"); + assert_eq!(pool.options().get_max_connections(), 7); + assert_eq!(pool.options().get_min_connections(), 0); + assert_eq!( + pool.options().get_acquire_timeout(), + Db::READER_ACQUIRE_TIMEOUT + ); + } + /// Channel window: head fetch (no cursor) reads the WRITER; cursor pages /// read the REPLICA. Divergent fixtures prove which pool served each. #[tokio::test] @@ -5519,18 +6863,20 @@ mod tests { drop_scratch_db(&admin, writer, &wname).await; } - /// Thread replies: head fetch reads the writer; a FULL cursor page is - /// served by the replica; an UNDER-limit cursor page (candidate terminal - /// page) is re-run on the writer so a lagged replica can never truncate - /// the tail into a false EOF. + /// Fail-closed on a mid-request replica failure (Dawn, review of + /// 1b0aa0dfa): a replica-routed page whose query errors *after* the + /// proof (the live shape is a hot-standby recovery conflict — 40001 / + /// 25P02 — cancelling the held snapshot under `max_standby_streaming_delay`) + /// must be re-run on the writer and served, never surfaced as an error + /// the writer could have answered. Degraded capacity, never holes. #[tokio::test] #[ignore = "requires Postgres"] - async fn thread_replies_cursor_pages_route_to_replica_with_writer_terminal_verification() { + async fn replica_window_failure_falls_back_to_writer() { let admin = PgPool::connect(&admin_url().await) .await .expect("connect admin"); - let (writer, wname) = create_scratch_db(&admin, "routing_tw").await; - let (replica, rname) = create_scratch_db(&admin, "routing_tr").await; + let (writer, wname) = create_scratch_db(&admin, "fb_w").await; + let (replica, rname) = create_scratch_db(&admin, "fb_r").await; let author = nostr::Keys::generate(); let community = Uuid::new_v4(); @@ -5539,107 +6885,78 @@ mod tests { seed_community_channel(&replica, community, channel, &author).await; let base = 1_700_000_000u64; - let root = signed_event_at(&author, "root", base); + let m1 = signed_event_at(&author, "m1", base); + let m2 = signed_event_at(&author, "m2", base + 10); + let m3 = signed_event_at(&author, "m3", base + 20); for pool in [&writer, &replica] { - insert_top_level(pool, community, channel, &root).await; - } - - // Writer holds replies r1..r5; the lagged replica only has r1..r3. - let replies: Vec = (1..=5) - .map(|i| signed_event_at(&author, &format!("r{i}"), base + 10 * i as u64)) - .collect(); - for reply in &replies { - insert_thread_reply(&writer, community, channel, &root, reply).await; - } - for reply in &replies[..3] { - insert_thread_reply(&replica, community, channel, &root, reply).await; + for ev in [&m1, &m2, &m3] { + insert_top_level(pool, community, channel, ev).await; + } } + let marker = signed_event_at(&author, "replica-only-marker", base + 5); + insert_top_level(&replica, community, channel, &marker).await; let db = Db::from_pools(writer.clone(), replica.clone()); - // Open the fence through "now" — fixture history is far in the past. db.fence().force_open_for_tests(chrono::Utc::now()); let cid = CommunityId::from_uuid(community); - // Page 1 (no cursor) → writer. - let page1 = db - .get_thread_replies(cid, root.id.as_bytes(), Some(10), 2, None) + let head = db + .get_channel_window(cid, channel, 1, None, None) .await - .expect("page 1"); - let contents: Vec<&str> = page1 - .iter() - .map(|r| r.stored_event.event.content.as_str()) - .collect(); - assert_eq!(contents, vec!["r1", "r2"], "head page from writer"); + .expect("head window"); + let cursor = head.next_cursor.expect("has_more implies next_cursor"); - // Page 2: replica serves a FULL page (r3 exists there) — but wait: - // replica has r1..r3, page after r2 with limit 2 returns only [r3] - // (under limit) → terminal-verification re-runs on the writer, which - // returns [r3, r4]. A lag-truncated EOF must never surface. - let cur2 = thread_cursor(page1.last().expect("page 1 non-empty")); - let page2 = db - .get_thread_replies(cid, root.id.as_bytes(), Some(10), 2, Some(&cur2)) + // Guard against a vacuous pass: the cursor page must actually be + // replica-eligible before we break the replica. + let healthy = db + .get_channel_window(cid, channel, 10, Some(cursor.clone()), None) .await - .expect("page 2"); - let contents: Vec<&str> = page2 - .iter() - .map(|r| r.stored_event.event.content.as_str()) - .collect(); - assert_eq!( - contents, - vec!["r3", "r4"], - "under-limit replica page must be re-verified on the writer" + .expect("healthy cursor window"); + assert!( + healthy + .rows + .iter() + .any(|r| r.stored_event.event.content == "replica-only-marker"), + "fixture must route the cursor page to the replica while healthy" ); - // Full-page replica serve: with limit 1, the page after r2 is [r3] — - // exactly `limit` rows, so the replica result stands. Prove it came - // from the replica with a replica-only divergent reply. - let ghost = signed_event_at(&author, "replica-only-ghost", base + 25); - insert_thread_reply(&replica, community, channel, &root, &ghost).await; - let page_replica = db - .get_thread_replies(cid, root.id.as_bytes(), Some(10), 1, Some(&cur2)) + // Break the replica AFTER the proof point: the heartbeat table stays + // intact (the observation succeeds), the page query then fails. + sqlx::query("DROP TABLE events CASCADE") + .execute(&replica) .await - .expect("full replica page"); - let contents: Vec<&str> = page_replica + .expect("drop replica events"); + + let page = db + .get_channel_window(cid, channel, 10, Some(cursor), None) + .await + .expect("replica failure must fall back to the writer, not error"); + let contents: Vec<&str> = page + .rows .iter() .map(|r| r.stored_event.event.content.as_str()) .collect(); assert_eq!( contents, - vec!["replica-only-ghost"], - "a full cursor page must be served by the replica" + vec!["m2", "m1"], + "fallback page must be the writer's answer (no replica marker)" ); - // Same query with no replica configured reads the writer and cannot - // see the ghost. - let db_writer_only = Db::from_pool(writer.clone()); - let page_writer = db_writer_only - .get_thread_replies(cid, root.id.as_bytes(), Some(10), 1, Some(&cur2)) - .await - .expect("writer-only page"); - let contents: Vec<&str> = page_writer - .iter() - .map(|r| r.stored_event.event.content.as_str()) - .collect(); - assert_eq!(contents, vec!["r3"], "unset replica falls back to writer"); - drop_scratch_db(&admin, replica, &rname).await; drop_scratch_db(&admin, writer, &wname).await; } - /// Channel DESC scrollback, out-of-order commit adversary: the replica is - /// missing a MIDDLE row (`m2`) because a transaction with an older - /// client-signed `created_at` committed late and has not replayed yet. - /// The replica's cursor page would be `[m1]` — silently skipping `m2` - /// forever, since the next cursor advances past it. The fence must route - /// any cursor above it to the writer. + /// [`replica_window_failure_falls_back_to_writer`] for the thread-replies + /// path: a replica-routed thread page whose query errors after the proof + /// re-runs on the writer instead of surfacing an error. #[tokio::test] #[ignore = "requires Postgres"] - async fn channel_cursor_above_fence_stays_on_writer_preventing_middle_hole() { + async fn replica_thread_failure_falls_back_to_writer() { let admin = PgPool::connect(&admin_url().await) .await .expect("connect admin"); - let (writer, wname) = create_scratch_db(&admin, "fence_cw").await; - let (replica, rname) = create_scratch_db(&admin, "fence_cr").await; + let (writer, wname) = create_scratch_db(&admin, "fbt_w").await; + let (replica, rname) = create_scratch_db(&admin, "fbt_r").await; let author = nostr::Keys::generate(); let community = Uuid::new_v4(); @@ -5648,79 +6965,1156 @@ mod tests { seed_community_channel(&replica, community, channel, &author).await; let base = 1_700_000_000u64; - let m1 = signed_event_at(&author, "m1", base); - let m2 = signed_event_at(&author, "m2-late-commit", base + 10); - let m3 = signed_event_at(&author, "m3", base + 20); - let m4 = signed_event_at(&author, "m4", base + 30); - for ev in [&m1, &m2, &m3, &m4] { - insert_top_level(&writer, community, channel, ev).await; + let root = signed_event_at(&author, "root", base); + for pool in [&writer, &replica] { + insert_top_level(pool, community, channel, &root).await; } - // Replica replayed everything EXCEPT the late-committed m2. - for ev in [&m1, &m3, &m4] { - insert_top_level(&replica, community, channel, ev).await; + let replies: Vec = (1..=3) + .map(|i| signed_event_at(&author, &format!("r{i}"), base + 10 * i as u64)) + .collect(); + for pool in [&writer, &replica] { + for reply in &replies { + insert_thread_reply(pool, community, channel, &root, reply).await; + } } + // Replica-only divergent reply between r2 and r3 marks replica serves. + let ghost = signed_event_at(&author, "replica-only-ghost", base + 25); + insert_thread_reply(&replica, community, channel, &root, &ghost).await; let db = Db::from_pools(writer.clone(), replica.clone()); + db.fence().force_open_for_tests(chrono::Utc::now()); let cid = CommunityId::from_uuid(community); - // Head page (writer): [m4, m3]; cursor lands on m3 (base+20). - let head = db - .get_channel_window(cid, channel, 2, None, None) + let page1 = db + .get_thread_replies(cid, root.id.as_bytes(), Some(10), 2, None) .await - .expect("head window"); - let cursor = head.next_cursor.expect("has_more implies next_cursor"); + .expect("head page"); + let cur = thread_cursor(page1.last().expect("page 1 non-empty")); - // Fence closed → cursor page must come from the writer: m2 present. - let contents = |w: &thread::ChannelWindow| -> Vec { - w.rows - .iter() - .map(|r| r.stored_event.event.content.clone()) - .collect() - }; - let page_closed = db - .get_channel_window(cid, channel, 10, Some(cursor.clone()), None) + // Healthy: the full page after r2 is the replica's [ghost]. + let healthy = db + .get_thread_replies(cid, root.id.as_bytes(), Some(10), 1, Some(&cur)) .await - .expect("cursor page, fence closed"); + .expect("healthy replica page"); assert_eq!( - contents(&page_closed), - vec!["m2-late-commit".to_string(), "m1".to_string()], - "fence closed: cursor pages route to the writer" + healthy[0].stored_event.event.content, "replica-only-ghost", + "fixture must route the cursor page to the replica while healthy" ); - // Fence open but BELOW the cursor timestamp (covers base+5 only): - // the cursor (base+20) is not covered → writer again. - db.fence().force_open_for_tests( - chrono::DateTime::from_timestamp(base as i64 + 5, 0).expect("ts"), - ); - let page_below = db - .get_channel_window(cid, channel, 10, Some(cursor.clone()), None) + sqlx::query("DROP TABLE events CASCADE") + .execute(&replica) .await - .expect("cursor page, fence below cursor"); - assert_eq!( - contents(&page_below), - vec!["m2-late-commit".to_string(), "m1".to_string()], - "cursor above the fence must stay on the writer" - ); + .expect("drop replica events"); - // Counterfactual pinning the hazard: were the fence (wrongly) open - // through now, the replica would serve the page WITHOUT m2 — the - // permanent-skip hole this fence exists to prevent. - db.fence().force_open_for_tests(chrono::Utc::now()); - let page_hazard = db - .get_channel_window(cid, channel, 10, Some(cursor), None) + let page = db + .get_thread_replies(cid, root.id.as_bytes(), Some(10), 1, Some(&cur)) .await - .expect("cursor page, fence wrongly open"); + .expect("replica failure must fall back to the writer, not error"); assert_eq!( - contents(&page_hazard), - vec!["m1".to_string()], - "fixture models the inversion: an over-open fence would skip m2" + page[0].stored_event.event.content, "r3", + "fallback page must be the writer's answer" ); drop_scratch_db(&admin, replica, &rname).await; drop_scratch_db(&admin, writer, &wname).await; } - /// Thread ASC pagination, out-of-order commit adversary: the replica + /// Mid-request degradation of the held session (Dawn, review of + /// 1b0aa0dfa): when the proved replica transaction dies between the page + /// and an aux follow-up (stand-in: `pg_terminate_backend` on the reader + /// connection, the same tx-fatal shape as a recovery-conflict cancel), + /// [`ReadSession::query_events`] must re-run the query on the writer and + /// permanently degrade the session instead of surfacing the error. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn read_session_degrades_to_writer_when_replica_connection_dies() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "deg_w").await; + let (replica, rname) = create_scratch_db(&admin, "deg_r").await; + + let author = nostr::Keys::generate(); + let community = Uuid::new_v4(); + let channel = Uuid::new_v4(); + seed_community_channel(&writer, community, channel, &author).await; + seed_community_channel(&replica, community, channel, &author).await; + + let base = 1_700_000_000u64; + let m1 = signed_event_at(&author, "m1", base); + let m2 = signed_event_at(&author, "m2", base + 10); + for pool in [&writer, &replica] { + for ev in [&m1, &m2] { + insert_top_level(pool, community, channel, ev).await; + } + } + // Writer-only row proves the degraded aux ran on the writer. + let fresh = signed_event_at(&author, "fresh-writer-only", base + 20); + insert_top_level(&writer, community, channel, &fresh).await; + + let db = Db::from_pools(writer.clone(), replica.clone()); + db.fence().force_open_for_tests(chrono::Utc::now()); + let cid = CommunityId::from_uuid(community); + + let head = db + .get_channel_window(cid, channel, 1, None, None) + .await + .expect("head window"); + let cursor = head.next_cursor.expect("has_more implies next_cursor"); + let (_window, mut session) = db + .get_channel_window_with_session(cid, channel, 10, Some(cursor), None) + .await + .expect("routed cursor window"); + assert!( + session.is_replica(), + "fixture must route this page to the replica" + ); + + // Kill the reader's backend out from under the held transaction. + sqlx::query( + "SELECT pg_terminate_backend(pid) FROM pg_stat_activity \ + WHERE datname = $1 AND pid <> pg_backend_pid()", + ) + .bind(&rname) + .execute(&admin) + .await + .expect("terminate replica backends"); + + let mut aux = EventQuery::for_community(cid); + aux.channel_id = Some(channel); + let rows = session + .query_events(&aux) + .await + .expect("session must degrade to the writer, not error"); + assert!( + rows.iter() + .any(|se| se.event.content == "fresh-writer-only"), + "degraded aux must be served by the writer" + ); + assert!( + !session.is_replica(), + "the session must be permanently degraded to the writer" + ); + + drop(session); + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// Snapshot continuity (Wren, review of 17ea2ff6a): the routed request + /// runs inside ONE `REPEATABLE READ, READ ONLY` transaction whose first + /// statement was the heartbeat observation — so a row committed on the + /// replica *after* the proof must be invisible to every follow-up + /// statement in the same request (page, participants, aux). This + /// distinguishes the transaction contract from mere connection reuse: + /// autocommit statements on the same backend advance their snapshot + /// per statement and WOULD see the mid-request row. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn routed_request_holds_one_snapshot_across_page_and_aux() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "snap_w").await; + let (replica, rname) = create_scratch_db(&admin, "snap_r").await; + + let author = nostr::Keys::generate(); + let community = Uuid::new_v4(); + let channel = Uuid::new_v4(); + seed_community_channel(&writer, community, channel, &author).await; + seed_community_channel(&replica, community, channel, &author).await; + + let base = 1_700_000_000u64; + let m1 = signed_event_at(&author, "m1", base); + let m2 = signed_event_at(&author, "m2", base + 10); + for pool in [&writer, &replica] { + for ev in [&m1, &m2] { + insert_top_level(pool, community, channel, ev).await; + } + } + + let db = Db::from_pools(writer.clone(), replica.clone()); + db.fence().force_open_for_tests(chrono::Utc::now()); + let cid = CommunityId::from_uuid(community); + + // Head page on the writer yields the cursor for a replica-routed page. + let head = db + .get_channel_window(cid, channel, 1, None, None) + .await + .expect("head window"); + let cursor = head.next_cursor.expect("has_more implies next_cursor"); + + // Route the cursor page to the replica and HOLD the session. + let (window, mut session) = db + .get_channel_window_with_session(cid, channel, 10, Some(cursor), None) + .await + .expect("routed cursor window"); + assert!( + session.is_replica(), + "fixture must route this page to the replica" + ); + assert_eq!(window.rows.len(), 1, "page after m2 is [m1]"); + + // Mid-request: a new event commits on the replica (stands in for + // replay advancing between the page and the aux closure). + let mid = signed_event_at(&author, "mid-request-commit", base + 5); + insert_top_level(&replica, community, channel, &mid).await; + + // A fresh autocommit statement on ANOTHER session sees it — the row + // is really there (control for the assertion below). + let mut control = EventQuery::for_community(cid); + control.channel_id = Some(channel); + let visible_elsewhere = event::query_events(&replica, &control) + .await + .expect("control query"); + assert!( + visible_elsewhere + .iter() + .any(|se| se.event.content == "mid-request-commit"), + "control: the mid-request row must be committed and visible to a new snapshot" + ); + + // The held request session must NOT see it: its snapshot was + // anchored by the heartbeat observation, before the commit. + let mut aux = EventQuery::for_community(cid); + aux.channel_id = Some(channel); + let in_request = session.query_events(&aux).await.expect("aux query"); + assert!( + !in_request + .iter() + .any(|se| se.event.content == "mid-request-commit"), + "request transaction must hold the proof-time snapshot; a \ + mid-request commit leaking in means the aux ran outside the \ + request transaction (autocommit connection reuse)" + ); + // Rows from the proof-time snapshot are still served. + assert!( + in_request.iter().any(|se| se.event.content == "m1"), + "proof-time rows must remain visible in the request snapshot" + ); + + drop(session); + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// Head gate (Predicate A): with the budget unset, a head fetch reads + /// the writer even over an open fence; with a budget set and a fresh + /// proved entry, the head page is served by the replica session + /// (bounded staleness accepted); with a budget the fence entry exceeds, + /// the head page falls back to the writer. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn head_fetch_routes_by_configured_budget() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "head_w").await; + let (replica, rname) = create_scratch_db(&admin, "head_r").await; + + let author = nostr::Keys::generate(); + let community = Uuid::new_v4(); + let channel = Uuid::new_v4(); + seed_community_channel(&writer, community, channel, &author).await; + seed_community_channel(&replica, community, channel, &author).await; + + let base = 1_700_000_000u64; + let shared = signed_event_at(&author, "shared", base); + for pool in [&writer, &replica] { + insert_top_level(pool, community, channel, &shared).await; + } + // Divergent heads prove which pool served the fetch. + let fresh = signed_event_at(&author, "fresh-writer-only", base + 30); + insert_top_level(&writer, community, channel, &fresh).await; + let marker = signed_event_at(&author, "replica-only-marker", base + 20); + insert_top_level(&replica, community, channel, &marker).await; + + let mut db = Db::from_pools(writer.clone(), replica.clone()); + db.fence().force_open_for_tests(chrono::Utc::now()); + let cid = CommunityId::from_uuid(community); + let head_contents = |w: &thread::ChannelWindow| -> Vec { + w.rows + .iter() + .map(|r| r.stored_event.event.content.clone()) + .collect() + }; + + // Budget unset (rollout default): head → writer, fence open or not. + let head = db + .get_channel_window(cid, channel, 2, None, None) + .await + .expect("head, gate off"); + assert_eq!( + head_contents(&head), + vec!["fresh-writer-only".to_string(), "shared".to_string()], + "head routing must default off" + ); + + // Budget set, entry fresh (just recorded): head → replica. + db.set_replica_read_max_age_for_tests(Some(std::time::Duration::from_secs(5))); + let head = db + .get_channel_window(cid, channel, 2, None, None) + .await + .expect("head, gate on"); + assert_eq!( + head_contents(&head), + vec!["replica-only-marker".to_string(), "shared".to_string()], + "a fresh proved entry within budget must serve the head from the replica" + ); + + // Entry older than the budget: head falls back to the writer. + db.fence().close(); + db.fence().force_open_for_tests_at( + chrono::Utc::now(), + std::time::Instant::now() - std::time::Duration::from_secs(10), + ); + let head = db + .get_channel_window(cid, channel, 2, None, None) + .await + .expect("head, entry too old"); + assert_eq!( + head_contents(&head), + vec!["fresh-writer-only".to_string(), "shared".to_string()], + "an over-budget entry must fail the head gate closed" + ); + + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// End-to-end deploy-default proof for the NEW routed seams: with the + /// budget unset, a covered-eligible query (channel-pinned + `until`) + /// through [`Db::query_events_routed`] is served by the WRITER — the + /// `for_query` gate keeps the covered arm dark (rev 5). With the budget + /// set and a fresh proved entry, the same query routes to the replica. + /// Divergent fixtures prove which pool served each read. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn query_events_routed_defaults_dark_and_routes_covered_when_enabled() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "qer_w").await; + let (replica, rname) = create_scratch_db(&admin, "qer_r").await; + + let author = nostr::Keys::generate(); + let community = Uuid::new_v4(); + let channel = Uuid::new_v4(); + seed_community_channel(&writer, community, channel, &author).await; + seed_community_channel(&replica, community, channel, &author).await; + + let base = 1_700_000_000u64; + let shared = signed_event_at(&author, "shared", base); + for pool in [&writer, &replica] { + insert_top_level(pool, community, channel, &shared).await; + } + let writer_only = signed_event_at(&author, "writer-only", base + 10); + insert_top_level(&writer, community, channel, &writer_only).await; + let replica_only = signed_event_at(&author, "replica-only", base + 20); + insert_top_level(&replica, community, channel, &replica_only).await; + + let mut db = Db::from_pools(writer.clone(), replica.clone()); + db.fence().force_open_for_tests(chrono::Utc::now()); + let cid = CommunityId::from_uuid(community); + + // Covered-eligible shape: channel-pinned with an `until` upper + // bound below the (now) fence wall. + let q = { + let mut q = EventQuery::for_community(cid); + q.channel_id = Some(channel); + q.until = chrono::DateTime::from_timestamp((base + 60) as i64, 0); + q + }; + let contents = |evs: &[StoredEvent]| -> std::collections::BTreeSet { + evs.iter().map(|e| e.event.content.clone()).collect() + }; + + // Deploy default: budget unset ⇒ writer, even though the shape is + // covered-eligible and the fence is open. + let rows = db + .query_events_routed("test_routed", &q) + .await + .expect("routed query, gate off"); + assert!( + contents(&rows).contains("writer-only"), + "budget unset must serve the writer" + ); + assert!( + !contents(&rows).contains("replica-only"), + "budget unset must not reach the replica via the covered arm" + ); + + // Budget set ⇒ the covered arm serves it from the replica. + db.set_replica_read_max_age_for_tests(Some(std::time::Duration::from_secs(5))); + let rows = db + .query_events_routed("test_routed", &q) + .await + .expect("routed query, gate on"); + assert!( + contents(&rows).contains("replica-only"), + "budget set + covered-eligible must route to the replica" + ); + assert!(!contents(&rows).contains("writer-only")); + + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// COUNT is bounded-only (rev 5 deletion-visibility rule): a + /// covered-eligible shape must NOT let a count take the covered arm. + /// With the budget unset the count reads the WRITER even with an open + /// fence; with the budget set and a fresh entry it reads the replica + /// under the bounded arm. Divergent row counts prove the serving pool. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn count_events_routed_is_bounded_only() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "cnt_w").await; + let (replica, rname) = create_scratch_db(&admin, "cnt_r").await; + + let author = nostr::Keys::generate(); + let community = Uuid::new_v4(); + let channel = Uuid::new_v4(); + seed_community_channel(&writer, community, channel, &author).await; + seed_community_channel(&replica, community, channel, &author).await; + + let base = 1_700_000_000u64; + // Writer: 2 rows. Replica: 1 row. + for (i, content) in ["a", "b"].iter().enumerate() { + let ev = signed_event_at(&author, content, base + i as u64); + insert_top_level(&writer, community, channel, &ev).await; + } + let ev = signed_event_at(&author, "c", base); + insert_top_level(&replica, community, channel, &ev).await; + + let mut db = Db::from_pools(writer.clone(), replica.clone()); + db.fence().force_open_for_tests(chrono::Utc::now()); + let cid = CommunityId::from_uuid(community); + + // Covered-eligible shape on purpose: pinned + until. A count must + // ignore that eligibility. + let q = { + let mut q = EventQuery::for_community(cid); + q.channel_id = Some(channel); + q.until = chrono::DateTime::from_timestamp((base + 60) as i64, 0); + q + }; + + // Budget unset ⇒ bounded arm disabled ⇒ writer. + let n = db + .count_events_routed("test_count", &q) + .await + .expect("count, gate off"); + assert_eq!(n, 2, "budget unset must count on the writer"); + + // Budget set + fresh entry ⇒ bounded arm ⇒ replica. + db.set_replica_read_max_age_for_tests(Some(std::time::Duration::from_secs(5))); + let n = db + .count_events_routed("test_count", &q) + .await + .expect("count, gate on"); + assert_eq!(n, 1, "budget set must count on the replica (bounded)"); + + // Entry older than the budget ⇒ bounded fails ⇒ writer. Covered + // would still hold here (upper <= wall) — proving count never + // consults it. + db.fence().close(); + db.fence().force_open_for_tests_at( + chrono::Utc::now(), + std::time::Instant::now() - std::time::Duration::from_secs(10), + ); + let n = db + .count_events_routed("test_count", &q) + .await + .expect("count, entry too old"); + assert_eq!( + n, 2, + "an over-budget entry must fail the count closed to the writer, \ + even when the covered arm would admit the shape" + ); + + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// Routed relay-membership check: budget unset ⇒ writer; budget set + + /// fresh proved entry ⇒ replica (bounded arm); over-budget entry ⇒ + /// writer. Divergent membership rows prove which pool answered. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn is_relay_member_is_bounded_routed_and_fails_closed() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "mem_w").await; + let (replica, rname) = create_scratch_db(&admin, "mem_r").await; + + let community = Uuid::new_v4(); + for pool in [&writer, &replica] { + sqlx::query("INSERT INTO communities (id, host) VALUES ($1, $2)") + .bind(community) + .bind(format!("member-routing-{}.example", community.simple())) + .execute(pool) + .await + .expect("insert community"); + } + let cid = CommunityId::from_uuid(community); + let writer_only = "aa".repeat(32); + let replica_only = "bb".repeat(32); + relay_members::add_relay_member(&writer, cid, &writer_only, "member", None) + .await + .expect("seed writer member"); + relay_members::add_relay_member(&replica, cid, &replica_only, "member", None) + .await + .expect("seed replica member"); + + let mut db = Db::from_pools(writer.clone(), replica.clone()); + db.fence().force_open_for_tests(chrono::Utc::now()); + + // Budget unset ⇒ bounded arm disabled ⇒ writer. + assert!( + db.is_relay_member(cid, &writer_only) + .await + .expect("gate off"), + "budget unset must answer from the writer" + ); + assert!(!db.is_relay_member(cid, &replica_only).await.unwrap()); + + // Budget set + fresh entry ⇒ replica. + db.set_replica_read_max_age_for_tests(Some(std::time::Duration::from_secs(5))); + assert!( + db.is_relay_member(cid, &replica_only) + .await + .expect("gate on"), + "budget set must answer from the replica" + ); + assert!(!db.is_relay_member(cid, &writer_only).await.unwrap()); + + // Entry older than the budget ⇒ fail closed to the writer. Close + // first so no prior fresh entry can be the one proved (matches the + // count test; today `force_open_for_tests_at` also clears the ring). + db.fence().close(); + db.fence().force_open_for_tests_at( + chrono::Utc::now(), + std::time::Instant::now() - std::time::Duration::from_secs(10), + ); + assert!( + db.is_relay_member(cid, &writer_only) + .await + .expect("entry too old"), + "an over-budget entry must fail closed to the writer" + ); + + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// Community separation across every routed seam, verified on + /// REPLICA-SERVED reads. + /// + /// The pre-existing feed/event scoping tests prove the shared SQL + /// builders confine rows to one community, but they exercise those + /// builders through the WRITER wrapper. `_on` variants are + /// executor-only refactors, so scoping *should* be identical — this + /// test refuses to take that on faith and re-proves it through the + /// routed executor, on a snapshot the replica actually served. + /// + /// Construction: two communities A and B exist in BOTH databases with + /// the same ids. The replica additionally holds a `replica-only` row in + /// each — divergent fixtures, so any row bearing that content proves + /// the replica (not the writer) served the read. Every assertion + /// requests A and demands B's rows never appear, including B's + /// `replica-only` row, which is the one a leaky predicate would surface. + /// The routed fallback must cost ONE reader acquire budget, even when the + /// Aurora capability cache is cold. + /// + /// Regression test for a stacked-budget bug found at `9fa3c9c0b`: the + /// capability probe used to `acquire()` from the pool itself and return + /// `false` *uncached* on `PoolTimedOut`, so the routed read then spent a + /// SECOND `READER_ACQUIRE_TIMEOUT` inside `begin`. Measured 302ms against + /// a ~150ms documented bound. Boot priming + /// ([`Db::spawn_read_pool_boot_ping`]) hid it only when the boot ping + /// SUCCEEDED — and a reader that is unavailable at boot is exactly the + /// case the bound is specified for, so the two failures are correlated. + /// + /// The fixture reproduces that state deliberately: a size-1 reader whose + /// sole connection is established and then HELD (so every further acquire + /// must time out), with `reader_aurora_identity` asserted cold. It routes + /// through `count_events_routed` rather than calling `proved_reader` + /// directly, because `buzz_db_route_decision` is emitted by `route_read` + /// — a direct call would prove the timing but never emit the label. + /// + /// Timing uses an upper bound of 2x the budget minus a margin: it must + /// fail for two stacked budgets (~300ms) while tolerating scheduler + /// jitter on one (~150ms). Asserting a lower bound too would pin the + /// budget's own value, which `reader_acquire_timeout_is_the_documented_budget` + /// already covers. + #[tokio::test(flavor = "current_thread")] + #[ignore = "requires Postgres"] + async fn routed_fallback_spends_one_acquire_budget_when_aurora_cache_is_cold() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (seed, wname) = create_scratch_db(&admin, "one_budget").await; + seed.close().await; + let base = admin_url().await; + let scratch_url = { + let idx = base.rfind('/').expect("db url has a path segment"); + format!("{}/{}", &base[..idx], wname) + }; + + // `Db::new` so the writer arms the floor guard and the reader is the + // real lazy `connect_read_pool` pool (min_connections=0, 150ms + // acquire timeout). Reader is sized 1 so holding one connection + // saturates it. + let mut db = Db::new(&DbConfig { + database_url: scratch_url.clone(), + read_database_url: Some(scratch_url), + max_connections: 4, + read_max_connections: Some(1), + ..DbConfig::default() + }) + .await + .expect("connect armed Db with size-1 lazy reader"); + db.fence().force_open_for_tests(chrono::Utc::now()); + db.set_replica_read_max_age_for_tests(Some(Duration::from_secs(5))); + + let read_pool = db.read_pool.clone().expect("reader pool configured"); + // Establish and hold the reader's only connection: saturated. + let held = read_pool + .acquire() + .await + .expect("establish the reader's sole connection"); + assert_eq!( + db.read_max_connections, 1, + "reader max must report 1 for this fixture to test saturation" + ); + assert_eq!( + read_pool.size(), + 1, + "the sole reader connection is established and held" + ); + // The bug is only observable with the capability cache cold; if a + // future change primes it here, this fixture would silently stop + // discriminating. + assert!( + db.reader_aurora_identity.get().is_none(), + "Aurora capability must be UNPRIMED (post-boot-ping-failure state)" + ); + + let recorder = metrics_util::debugging::DebuggingRecorder::new(); + let snapshotter = recorder.snapshotter(); + let query = EventQuery::for_community(CommunityId::from_uuid(Uuid::new_v4())); + + // The recorder is installed thread-locally, so it must stay installed + // across the `.await` — hence the guard form rather than + // `with_local_recorder`, whose closure cannot host an await. The + // `current_thread` flavor keeps the route decision on this thread; on + // a multi-thread runtime the emit could land on a worker where no + // local recorder is installed and the label assertions would vacuously + // see an empty snapshot. + let start = std::time::Instant::now(); + let count = { + let _guard = metrics::set_default_local_recorder(&recorder); + db.count_events_routed("one_budget_probe", &query).await + } + .expect("writer fallback still answers the read"); + let elapsed = start.elapsed(); + + assert_eq!(count, 0, "writer answered on an empty scratch database"); + assert!( + elapsed < Duration::from_millis(250), + "routed fallback must spend ONE {}ms acquire budget, not two; took {}ms", + Db::READER_ACQUIRE_TIMEOUT.as_millis(), + elapsed.as_millis() + ); + + let reasons: std::collections::HashMap<(String, String), u64> = snapshotter + .snapshot() + .into_vec() + .into_iter() + .filter(|(key, ..)| key.key().name() == "buzz_db_route_decision") + .map(|(key, _, _, value)| { + let metrics_util::debugging::DebugValue::Counter(n) = value else { + panic!("buzz_db_route_decision must be a counter"); + }; + let labels: Vec<_> = key.key().labels().collect(); + let get = |name: &str| { + labels + .iter() + .find(|l| l.key() == name) + .map(|l| l.value().to_owned()) + .unwrap_or_default() + }; + ((get("decision"), get("reason")), n) + }) + .collect(); + + assert_eq!( + reasons.get(&("writer".to_owned(), "reader_acquire_timeout".to_owned())), + Some(&1), + "saturated reader must fall back as writer/reader_acquire_timeout; got {reasons:?}" + ); + // `reader_validation_error` would mean we misclassified a timeout as a + // broken reader, and `pool_busy` is the retired name — neither may + // appear in ANY emitted label. + assert!( + !reasons + .keys() + .any(|(_, reason)| reason == "reader_validation_error" || reason == "pool_busy"), + "no reader_validation_error or retired pool_busy label may be emitted; got {reasons:?}" + ); + + drop(held); + drop_scratch_db(&admin, db.pool.clone(), &wname).await; + } + + #[tokio::test] + #[ignore = "requires Postgres"] + async fn routed_reads_are_confined_to_the_requested_community() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "sep_w").await; + let (replica, rname) = create_scratch_db(&admin, "sep_r").await; + + let author = nostr::Keys::generate(); + let (comm_a, chan_a) = (Uuid::new_v4(), Uuid::new_v4()); + let (comm_b, chan_b) = (Uuid::new_v4(), Uuid::new_v4()); + for pool in [&writer, &replica] { + seed_community_channel(pool, comm_a, chan_a, &author).await; + seed_community_channel(pool, comm_b, chan_b, &author).await; + } + + // A p-tag mention is what makes a row eligible for the mentions and + // needs-action feeds. Kind 9 satisfies mentions + activity; + // needs-action admits only approval/reminder kinds, so each + // community also gets a kind-46010 row. + let mentioned = nostr::Keys::generate(); + let mentioned_hex = mentioned.public_key().to_hex(); + let mentioned_bytes = mentioned.public_key().to_bytes(); + let tagged_kind = |kind: u16, content: &str, secs: u64| { + nostr::EventBuilder::new(nostr::Kind::Custom(kind), content) + .tags([nostr::Tag::parse(["p", mentioned_hex.as_str()]).expect("p tag")]) + .custom_created_at(nostr::Timestamp::from(secs)) + .sign_with_keys(&author) + .expect("sign event") + }; + let tagged = |content: &str, secs: u64| tagged_kind(9, content, secs); + + let base = 1_700_000_000u64; + // Shared rows (both DBs) + replica-only rows (divergence) per community. + let a_shared = tagged("a-shared", base); + let b_shared = tagged("b-shared", base + 1); + for pool in [&writer, &replica] { + insert_top_level(pool, comm_a, chan_a, &a_shared).await; + insert_mentions( + pool, + CommunityId::from_uuid(comm_a), + &a_shared, + Some(chan_a), + ) + .await + .expect("mentions a-shared"); + insert_top_level(pool, comm_b, chan_b, &b_shared).await; + insert_mentions( + pool, + CommunityId::from_uuid(comm_b), + &b_shared, + Some(chan_b), + ) + .await + .expect("mentions b-shared"); + } + let a_replica_only = tagged("a-replica-only", base + 10); + let b_replica_only = tagged("b-replica-only", base + 11); + insert_top_level(&replica, comm_a, chan_a, &a_replica_only).await; + insert_mentions( + &replica, + CommunityId::from_uuid(comm_a), + &a_replica_only, + Some(chan_a), + ) + .await + .expect("mentions a-replica-only"); + insert_top_level(&replica, comm_b, chan_b, &b_replica_only).await; + insert_mentions( + &replica, + CommunityId::from_uuid(comm_b), + &b_replica_only, + Some(chan_b), + ) + .await + .expect("mentions b-replica-only"); + + // Needs-action fixtures: approval kind, replica-only in BOTH + // communities, so the assertion below is replica-served on A and + // must still not see B's. + let a_approval = tagged_kind(46010, "a-approval-replica-only", base + 20); + let b_approval = tagged_kind(46010, "b-approval-replica-only", base + 21); + insert_top_level(&replica, comm_a, chan_a, &a_approval).await; + insert_mentions( + &replica, + CommunityId::from_uuid(comm_a), + &a_approval, + Some(chan_a), + ) + .await + .expect("mentions a-approval"); + insert_top_level(&replica, comm_b, chan_b, &b_approval).await; + insert_mentions( + &replica, + CommunityId::from_uuid(comm_b), + &b_approval, + Some(chan_b), + ) + .await + .expect("mentions b-approval"); + + let mut db = Db::from_pools(writer.clone(), replica.clone()); + db.fence().force_open_for_tests(chrono::Utc::now()); + db.set_replica_read_max_age_for_tests(Some(std::time::Duration::from_secs(5))); + let cid_a = CommunityId::from_uuid(comm_a); + + let contents = |evs: &[StoredEvent]| -> std::collections::BTreeSet { + evs.iter().map(|e| e.event.content.clone()).collect() + }; + // Every routed seam must (a) have been served by the replica — + // proven by a divergent row absent from the writer — and (b) contain + // no row belonging to community B. All B fixtures are named `b-*`, + // so the leak check is a single prefix scan. + let assert_a_only = |rows: &[StoredEvent], marker: &str, seam: &str| { + let got = contents(rows); + assert!( + got.contains(marker), + "{seam}: must be replica-served (divergent row `{marker}` absent from writer); got {got:?}" + ); + assert!( + !got.iter().any(|c| c.starts_with("b-")), + "{seam}: community B rows leaked into a community A read; got {got:?}" + ); + }; + + // 1. Generic query — covered arm (channel-pinned + `until`). + let mut q = EventQuery::for_community(cid_a); + q.channel_id = Some(chan_a); + q.until = chrono::DateTime::from_timestamp((base + 60) as i64, 0); + let rows = db + .query_events_routed("sep_query", &q) + .await + .expect("routed query"); + assert_a_only(&rows, "a-replica-only", "query_events_routed"); + + // 2. Generic query — bounded arm (no channel pin at all, so a + // missing community predicate could not be masked by the pin). + let unpinned = EventQuery::for_community(cid_a); + let rows = db + .query_events_routed_bounded("sep_query_bounded", &unpinned) + .await + .expect("routed bounded query"); + assert_a_only(&rows, "a-replica-only", "query_events_routed_bounded"); + + // 3. COUNT — bounded-only. Community A holds 3 rows on the replica + // (shared + replica-only + approval) but only 1 on the writer, + // and 3 more exist in community B. Exactly 3 proves the read was + // both replica-served and community-confined. + let count = db + .count_events_routed("sep_count", &unpinned) + .await + .expect("routed count"); + assert_eq!( + count, 3, + "count must see A's three replica rows only — not B's, not the writer's one" + ); + + // 4. By-ID hydration — ids carry no channel pin, and B's ids are + // requested alongside A's. Only A's may hydrate. + let ids: Vec<&[u8]> = vec![ + a_shared.id.as_bytes(), + a_replica_only.id.as_bytes(), + b_shared.id.as_bytes(), + b_replica_only.id.as_bytes(), + ]; + let rows = db + .get_events_by_ids_routed("sep_by_ids", cid_a, &ids) + .await + .expect("routed by-ids"); + assert_a_only(&rows, "a-replica-only", "get_events_by_ids_routed"); + + // 5-7. All three feed builders, each given BOTH channels as + // accessible — so only the community predicate can exclude B. + let both = [chan_a, chan_b]; + let rows = db + .query_feed_mentions_routed("sep_feed", cid_a, &mentioned_bytes, &both, None, 50) + .await + .expect("routed mentions"); + assert_a_only(&rows, "a-replica-only", "query_feed_mentions_routed"); + + let rows = db + .query_feed_needs_action_routed("sep_feed", cid_a, &mentioned_bytes, &both, None, 50) + .await + .expect("routed needs action"); + assert_a_only( + &rows, + "a-approval-replica-only", + "query_feed_needs_action_routed", + ); + + let rows = db + .query_feed_activity_routed("sep_feed", cid_a, &both, None, 50) + .await + .expect("routed activity"); + assert_a_only(&rows, "a-replica-only", "query_feed_activity_routed"); + + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// D4: a LAZY reader pool (connect_lazy, min_connections=0, never yet + /// used) must still let [`Db::spawn_fence_probe`] verify the writer's + /// floor guard and spawn — reader-down or reader-idle at boot must not + /// disable fence probing. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn lazy_reader_pool_still_spawns_fence_probe() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (seed, wname) = create_scratch_db(&admin, "lazy_w").await; + seed.close().await; + + let writer_url = { + let base = admin_url().await; + let idx = base.rfind('/').expect("db url has a path segment"); + format!("{}/{}", &base[..idx], wname) + }; + // `Db::new` (not `from_pools`) so the WRITER pool arms the + // `buzz.created_at_floor` GUC — `spawn_fence_probe` verifies the + // floor guard on a writer connection, and `create_scratch_db`'s + // plain `PgPool::connect` never arms it. The reader is still the + // lazy `connect_read_pool` pool this test is about. + let db = Db::new(&DbConfig { + database_url: writer_url.clone(), + read_database_url: Some(writer_url), + max_connections: 2, + ..DbConfig::default() + }) + .await + .expect("connect armed Db with lazy reader"); + + let spawned = db + .spawn_fence_probe() + .await + .expect("floor-guard verification must pass on the migrated writer"); + assert!(spawned, "a configured (lazy) reader must spawn the probe"); + + drop_scratch_db(&admin, db.pool.clone(), &wname).await; + } + + /// Thread replies: head fetch reads the writer; a FULL cursor page is + /// served by the replica; an UNDER-limit cursor page (candidate terminal + /// page) is re-run on the writer so a lagged replica can never truncate + /// the tail into a false EOF. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn thread_replies_cursor_pages_route_to_replica_with_writer_terminal_verification() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "routing_tw").await; + let (replica, rname) = create_scratch_db(&admin, "routing_tr").await; + + let author = nostr::Keys::generate(); + let community = Uuid::new_v4(); + let channel = Uuid::new_v4(); + seed_community_channel(&writer, community, channel, &author).await; + seed_community_channel(&replica, community, channel, &author).await; + + let base = 1_700_000_000u64; + let root = signed_event_at(&author, "root", base); + for pool in [&writer, &replica] { + insert_top_level(pool, community, channel, &root).await; + } + + // Writer holds replies r1..r5; the lagged replica only has r1..r3. + let replies: Vec = (1..=5) + .map(|i| signed_event_at(&author, &format!("r{i}"), base + 10 * i as u64)) + .collect(); + for reply in &replies { + insert_thread_reply(&writer, community, channel, &root, reply).await; + } + for reply in &replies[..3] { + insert_thread_reply(&replica, community, channel, &root, reply).await; + } + + let db = Db::from_pools(writer.clone(), replica.clone()); + // Open the fence through "now" — fixture history is far in the past. + db.fence().force_open_for_tests(chrono::Utc::now()); + let cid = CommunityId::from_uuid(community); + + // Page 1 (no cursor) → writer. + let page1 = db + .get_thread_replies(cid, root.id.as_bytes(), Some(10), 2, None) + .await + .expect("page 1"); + let contents: Vec<&str> = page1 + .iter() + .map(|r| r.stored_event.event.content.as_str()) + .collect(); + assert_eq!(contents, vec!["r1", "r2"], "head page from writer"); + + // Page 2: replica serves a FULL page (r3 exists there) — but wait: + // replica has r1..r3, page after r2 with limit 2 returns only [r3] + // (under limit) → terminal-verification re-runs on the writer, which + // returns [r3, r4]. A lag-truncated EOF must never surface. + let cur2 = thread_cursor(page1.last().expect("page 1 non-empty")); + let page2 = db + .get_thread_replies(cid, root.id.as_bytes(), Some(10), 2, Some(&cur2)) + .await + .expect("page 2"); + let contents: Vec<&str> = page2 + .iter() + .map(|r| r.stored_event.event.content.as_str()) + .collect(); + assert_eq!( + contents, + vec!["r3", "r4"], + "under-limit replica page must be re-verified on the writer" + ); + + // Full-page replica serve: with limit 1, the page after r2 is [r3] — + // exactly `limit` rows, so the replica result stands. Prove it came + // from the replica with a replica-only divergent reply. + let ghost = signed_event_at(&author, "replica-only-ghost", base + 25); + insert_thread_reply(&replica, community, channel, &root, &ghost).await; + let page_replica = db + .get_thread_replies(cid, root.id.as_bytes(), Some(10), 1, Some(&cur2)) + .await + .expect("full replica page"); + let contents: Vec<&str> = page_replica + .iter() + .map(|r| r.stored_event.event.content.as_str()) + .collect(); + assert_eq!( + contents, + vec!["replica-only-ghost"], + "a full cursor page must be served by the replica" + ); + + // Same query with no replica configured reads the writer and cannot + // see the ghost. + let db_writer_only = Db::from_pool(writer.clone()); + let page_writer = db_writer_only + .get_thread_replies(cid, root.id.as_bytes(), Some(10), 1, Some(&cur2)) + .await + .expect("writer-only page"); + let contents: Vec<&str> = page_writer + .iter() + .map(|r| r.stored_event.event.content.as_str()) + .collect(); + assert_eq!(contents, vec!["r3"], "unset replica falls back to writer"); + + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// Channel DESC scrollback, out-of-order commit adversary: the replica is + /// missing a MIDDLE row (`m2`) because a transaction with an older + /// client-signed `created_at` committed late and has not replayed yet. + /// The replica's cursor page would be `[m1]` — silently skipping `m2` + /// forever, since the next cursor advances past it. The fence must route + /// any cursor above it to the writer. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn channel_cursor_above_fence_stays_on_writer_preventing_middle_hole() { + let admin = PgPool::connect(&admin_url().await) + .await + .expect("connect admin"); + let (writer, wname) = create_scratch_db(&admin, "fence_cw").await; + let (replica, rname) = create_scratch_db(&admin, "fence_cr").await; + + let author = nostr::Keys::generate(); + let community = Uuid::new_v4(); + let channel = Uuid::new_v4(); + seed_community_channel(&writer, community, channel, &author).await; + seed_community_channel(&replica, community, channel, &author).await; + + let base = 1_700_000_000u64; + let m1 = signed_event_at(&author, "m1", base); + let m2 = signed_event_at(&author, "m2-late-commit", base + 10); + let m3 = signed_event_at(&author, "m3", base + 20); + let m4 = signed_event_at(&author, "m4", base + 30); + for ev in [&m1, &m2, &m3, &m4] { + insert_top_level(&writer, community, channel, ev).await; + } + // Replica replayed everything EXCEPT the late-committed m2. + for ev in [&m1, &m3, &m4] { + insert_top_level(&replica, community, channel, ev).await; + } + + let db = Db::from_pools(writer.clone(), replica.clone()); + let cid = CommunityId::from_uuid(community); + + // Head page (writer): [m4, m3]; cursor lands on m3 (base+20). + let head = db + .get_channel_window(cid, channel, 2, None, None) + .await + .expect("head window"); + let cursor = head.next_cursor.expect("has_more implies next_cursor"); + + // Fence closed → cursor page must come from the writer: m2 present. + let contents = |w: &thread::ChannelWindow| -> Vec { + w.rows + .iter() + .map(|r| r.stored_event.event.content.clone()) + .collect() + }; + let page_closed = db + .get_channel_window(cid, channel, 10, Some(cursor.clone()), None) + .await + .expect("cursor page, fence closed"); + assert_eq!( + contents(&page_closed), + vec!["m2-late-commit".to_string(), "m1".to_string()], + "fence closed: cursor pages route to the writer" + ); + + // Fence open but BELOW the cursor timestamp (covers base+5 only): + // the cursor (base+20) is not covered → writer again. + db.fence().force_open_for_tests( + chrono::DateTime::from_timestamp(base as i64 + 5, 0).expect("ts"), + ); + let page_below = db + .get_channel_window(cid, channel, 10, Some(cursor.clone()), None) + .await + .expect("cursor page, fence below cursor"); + assert_eq!( + contents(&page_below), + vec!["m2-late-commit".to_string(), "m1".to_string()], + "cursor above the fence must stay on the writer" + ); + + // Counterfactual pinning the hazard: were the fence (wrongly) open + // through now, the replica would serve the page WITHOUT m2 — the + // permanent-skip hole this fence exists to prevent. + db.fence().force_open_for_tests(chrono::Utc::now()); + let page_hazard = db + .get_channel_window(cid, channel, 10, Some(cursor), None) + .await + .expect("cursor page, fence wrongly open"); + assert_eq!( + contents(&page_hazard), + vec!["m1".to_string()], + "fixture models the inversion: an over-open fence would skip m2" + ); + + drop_scratch_db(&admin, replica, &rname).await; + drop_scratch_db(&admin, writer, &wname).await; + } + + /// Thread ASC pagination, out-of-order commit adversary: the replica /// holds a FULL page whose newest row (`r4`) has a later key than a /// not-yet-replayed row (`r3`). The old under-limit check alone would /// serve `[r4]` and the client cursor would advance past `r3` forever. @@ -6012,21 +8406,39 @@ mod tests { let base = admin_url().await; let idx = base.rfind('/').expect("db url has a path segment"); - let db = Db::new(&DbConfig { - database_url: format!("{}/{}", &base[..idx], wname), - read_database_url: Some(format!("{}/{}", &base[..idx], rname)), + let writer_url = format!("{}/{}", &base[..idx], wname); + let replica_url = format!("{}/{}", &base[..idx], rname); + + // Healthy schema: verification passes, probe starts. A SEPARATE Db + // instance, because its background probe legitimately opens its own + // fence (the heartbeat probe is writer-side only) — the refusal + // assertions below must run against a fence whose spawns were all + // refused. + let db_healthy = Db::new(&DbConfig { + database_url: writer_url.clone(), + read_database_url: Some(replica_url.clone()), max_connections: 2, ..DbConfig::default() }) .await .expect("connect armed Db with replica"); - - // Healthy schema: verification passes, probe starts. assert!( - db.spawn_fence_probe().await.expect("verification passes"), + db_healthy + .spawn_fence_probe() + .await + .expect("verification passes"), "probe must start on a verified schema" ); + let db = Db::new(&DbConfig { + database_url: writer_url, + read_database_url: Some(replica_url), + max_connections: 2, + ..DbConfig::default() + }) + .await + .expect("connect armed Db with replica"); + // Sabotage A: catalog-shaped no-op — same trigger, gutted function // body. Catalog check alone would pass; behavior check must refuse. sqlx::query( @@ -6066,6 +8478,10 @@ mod tests { "fence must remain closed when verification refuses the probe" ); + db_healthy.pool.close().await; + if let Some(rp) = &db_healthy.read_pool { + rp.close().await; + } db.pool.close().await; if let Some(rp) = &db.read_pool { rp.close().await; diff --git a/crates/buzz-db/src/migration.rs b/crates/buzz-db/src/migration.rs index 1d1b7e05d4..6985916bba 100644 --- a/crates/buzz-db/src/migration.rs +++ b/crates/buzz-db/src/migration.rs @@ -347,6 +347,7 @@ mod tests { "push_gateway_delivery_auth_replays", "push_gateway_delivery_request_replays", "product_feedback", + "replica_heartbeat", ] { if normalized[insert_pos..].contains(&format!("'{value}'")) { globals.insert(value.to_owned()); @@ -560,7 +561,7 @@ mod tests { let mut migrations: Vec<_> = MIGRATOR.iter().collect(); migrations.sort_by_key(|migration| migration.version); - assert_eq!(migrations.len(), 25); + assert_eq!(migrations.len(), 26); assert_eq!(migrations[0].version, 1); assert_eq!(&*migrations[0].description, "initial schema"); assert!(migrations[0] @@ -904,6 +905,20 @@ mod tests { desired_schema.contains("CREATE TABLE join_policy_acceptances"), "desired-state schema must include join-policy evidence used by invite claims", ); + + // Replica heartbeat (this branch, renumbered to 0026 after + // 0025_relay_invites landed on main): the fence's portable read-side + // observation. A single CHECK'd row makes the token update the + // serialization point (multi-pod commit ordering), and the epoch + // column is what detects token resets — both are load-bearing for + // the routing proof. + assert_eq!(migrations[25].version, 26); + let heartbeat = migrations[25].sql.as_str(); + assert!(heartbeat.contains("CREATE TABLE replica_heartbeat")); + assert!(heartbeat.contains("CHECK (id = 1)")); + assert!(heartbeat.contains("epoch")); + assert!(heartbeat.contains("INSERT INTO replica_heartbeat (id) VALUES (1)")); + assert!(heartbeat.contains("_operator_global_tables")); } #[test] @@ -1146,7 +1161,7 @@ mod tests { run_migrations(&pool) .await .expect("retry succeeds after operator repair"); - assert_eq!(applied_versions(&pool).await.last().copied(), Some(25)); + assert_eq!(applied_versions(&pool).await.last().copied(), Some(26)); } #[tokio::test] diff --git a/crates/buzz-db/src/relay_members.rs b/crates/buzz-db/src/relay_members.rs index 3805745f9b..402229cdec 100644 --- a/crates/buzz-db/src/relay_members.rs +++ b/crates/buzz-db/src/relay_members.rs @@ -29,14 +29,41 @@ pub struct RelayMember { /// Returns `true` if `pubkey` (64-char hex) is a member of `community`. pub async fn is_relay_member(pool: &PgPool, community: CommunityId, pubkey: &str) -> Result { + let mut conn = pool.acquire().await?; + is_relay_member_on(&mut conn, community, pubkey).await +} + +/// [`is_relay_member`] on a specific session — the replica-routing path runs +/// the lookup on the exact reader connection whose heartbeat observation +/// proved fence coverage. +pub(crate) async fn is_relay_member_on( + conn: &mut sqlx::PgConnection, + community: CommunityId, + pubkey: &str, +) -> Result { let row = sqlx::query("SELECT 1 FROM relay_members WHERE community_id = $1 AND pubkey = $2") .bind(community.as_uuid()) .bind(pubkey) - .fetch_optional(pool) + .fetch_optional(conn) .await?; Ok(row.is_some()) } +/// Returns `true` if any member of `community` holds the `admin` or `owner` +/// role. Open relays don't *enforce* the roster, but startup +/// (`bootstrap_owner`) and operator provisioning still populate it — this is +/// how the workspace-profile gate detects whether a steward exists. +pub async fn has_admin_or_owner(pool: &PgPool, community: CommunityId) -> Result { + let row = sqlx::query( + "SELECT 1 FROM relay_members \ + WHERE community_id = $1 AND role IN ('admin', 'owner') LIMIT 1", + ) + .bind(community.as_uuid()) + .fetch_optional(pool) + .await?; + Ok(row.is_some()) +} + /// Returns the relay member record for `pubkey` in `community`, or `None`. pub async fn get_relay_member( pool: &PgPool, @@ -376,7 +403,7 @@ pub enum TransferResult { /// Default maximum number of communities a single pubkey can own. Enforced at /// the relay layer — the authoritative layer — so that concurrent transfers or /// transfer-vs-create races cannot both pass a preflight count. -pub const MAX_COMMUNITIES_PER_OWNER: i64 = 3; +pub const MAX_COMMUNITIES_PER_OWNER: i64 = 5; /// Effective per-owner community limit for this deployment. /// diff --git a/crates/buzz-db/src/replica_fence.rs b/crates/buzz-db/src/replica_fence.rs index 03bc1c77f0..c840db393e 100644 --- a/crates/buzz-db/src/replica_fence.rs +++ b/crates/buzz-db/src/replica_fence.rs @@ -9,38 +9,59 @@ //! (`clock_timestamp()`, evaluated inside commit processing). Enforcement //! is armed per session via the `buzz.created_at_floor` GUC, which the //! relay's writer pool sets on every connection. -//! 2. **Ordered LSN handshake** (this module): on one pinned writer -//! connection, three separately-awaited statements sample +//! 2. **Ordered heartbeat handshake** (this module): on one pinned writer +//! connection, separately-awaited statements sample //! `S = clock_timestamp()`, then scan `pg_stat_activity` for the oldest -//! open transaction, then capture `L = pg_current_wal_lsn()` **last**. -//! Once the replica reports `pg_last_wal_replay_lsn() >= L`, every -//! transaction partitions into exactly three buckets: -//! (a) finished before the activity scan — its commit WAL precedes `L`, -//! so the replica has replayed it; +//! open transaction, then — **last** — commit heartbeat token `M` via a +//! single-row `UPDATE replica_heartbeat ... RETURNING token, epoch` +//! (migration 0026). Because the single-row UPDATE serializes all pods' +//! probes, tokens are globally commit-ordered. A reader **session** that +//! observes `token >= M` on its own connection has, by WAL/storage replay +//! order, also replayed every commit that preceded M's commit; every +//! transaction then partitions into exactly three buckets: +//! (a) finished before the activity scan — its commit precedes `M`'s +//! commit, so the replica session has replayed it; //! (b) open at the activity scan — represented by `xact_start`, so it is //! bounded by the `oldest_xact_start` term; //! (c) started after the activity scan — its deferred floor guard runs //! after `S`, so it cannot commit a row with //! `created_at < S - floor`. -//! There is no fourth bucket. The fence therefore advances to -//! `min(oldest_xact_start, S) - floor - clock_margin`, and every -//! channel-window row with `created_at <= fence` is on the replica. +//! There is no fourth bucket. Each committed token `M` therefore proves a +//! **fence wall** of `min(oldest_xact_start, S) - floor - clock_margin`: +//! every channel-window row with `created_at <= fence_wall(M)` is present +//! on any reader session observing `token >= M`. +//! +//! Unlike the previous WAL-LSN observation (`pg_last_wal_replay_lsn()`, which +//! Aurora reader endpoints hide), the token observation is portable and — +//! critically — **snapshot-local**: routing opens a `REPEATABLE READ, READ +//! ONLY` transaction on the reader session that will serve the page and +//! observes the heartbeat as its first statement, so the proof binds to the +//! exact snapshot every follow-up statement in the request (page, +//! participants, aux closure) reads from — never to a different pooled +//! session (readers behind one endpoint may sit at different replay +//! positions), and never to a later autocommit snapshot on the same wire. +//! An observed token lower than the newest retained `M` is ordinary +//! replication lag, not a fault; the resolver simply proves from an older +//! retained `M`. Regression detection is writer-side only: a non-monotonic +//! `RETURNING token` or an epoch change (restore/re-seed) clears the retained +//! ring, so no stale entry can masquerade as fresh coverage. //! //! Everything fails **closed**: probe errors, masked `pg_stat_activity` -//! visibility, NULL/absent replica LSN (Aurora observability differences), -//! non-advancing replay, or probe staleness all close the fence, which routes -//! all reads back to the writer — degraded capacity, never holes. +//! visibility, an unreadable heartbeat row on the reader session, an epoch +//! mismatch, or an observed token below every retained entry all route the +//! request back to the writer — degraded capacity, never holes. //! //! Operational bypasses (sessions without the GUC, `session_replication_role //! = replica` restores) are outside the proof by design and require holding //! the fence closed for their duration; see `migrations/0021`. -use std::sync::atomic::{AtomicI64, Ordering}; -use std::sync::Arc; -use std::time::Duration; +use std::collections::VecDeque; +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; use chrono::{DateTime, Utc}; -use sqlx::{PgPool, Row}; +use sqlx::{PgConnection, PgPool, Row}; +use uuid::Uuid; /// Seconds of `created_at` history the commit-time floor guard tolerates. /// @@ -58,79 +79,231 @@ pub const CREATED_AT_FLOOR_SECS: i64 = 960; /// between machines. pub const FENCE_CLOCK_MARGIN_SECS: i64 = 5; -/// How often the probe samples the writer and checks the replica. -pub const PROBE_INTERVAL: Duration = Duration::from_secs(5); +/// How often the probe samples the writer and commits a heartbeat token. +/// +/// 500ms keeps the cadence at least 2x under the smallest sensible bounded +/// budget (`BUZZ_REPLICA_READ_MAX_AGE_MS`, deploy plan 1000ms) so +/// eligibility doesn't flap between beats. Cost is one single-row UPDATE +/// tuple of WAL per beat per pod — ~20 beats/s fleet-wide, <0.1% of the +/// writer. +pub const PROBE_INTERVAL: Duration = Duration::from_millis(500); -/// A fence older than this is stale: the probe has stopped confirming -/// freshness and the fence closes until a new handshake completes. +/// A fence whose newest entry is older than this is stale: the probe has +/// stopped committing tokens and routing eligibility closes until a new +/// handshake completes. +/// +/// Note this is an availability hygiene gate, not a soundness requirement: +/// a retained entry's proof (`token >= M` on a session implies every row +/// `<= fence_wall(M)` is present there) never decays. Closing on staleness +/// just stops spending reader checkouts once the probe is evidently dead. pub const FENCE_STALENESS: Duration = Duration::from_secs(30); -/// Sentinel: fence closed (no verified replica coverage). -const CLOSED: i64 = i64::MIN; +/// How many `(token, fence_wall)` entries the fence retains. At one probe +/// per [`PROBE_INTERVAL`] (500ms) this is ~60 seconds of history — a reader +/// session lagging further than that behind the newest token fails closed +/// (routes to the writer) rather than proving from thin air. Aurora reader +/// lag is typically tens of milliseconds; a reader minutes behind is a +/// fault, not a routing candidate. +const RING_CAPACITY: usize = 120; -/// Shared fence state. `Db` holds an `Arc` of this; the probe task advances -/// it and cursor routing consults it. -#[derive(Debug)] +// The retained window must outlast the staleness gate: if the ring held +// less than FENCE_STALENESS of history, a non-stale newest entry could +// coexist with proved-but-evicted older entries, failing sessions closed +// for capacity rather than lag. Compile-checked so a future cadence or +// capacity tweak can't silently shrink the window below the gate. +const _: () = assert!( + RING_CAPACITY as u64 * PROBE_INTERVAL.as_millis() as u64 > FENCE_STALENESS.as_millis() as u64, + "fence ring must retain more history than the staleness gate" +); + +/// One retained heartbeat observation: proof material for reader sessions. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct TokenEntry { + /// The committed heartbeat token `M`. + pub token: i64, + /// Monotonic instant captured just before `M` was committed. `elapsed()` + /// bounds (from above) how old a session observing `token >= M` can be — + /// the freshness term of the head-routing predicate. + pub committed_at: Instant, + /// `min(oldest_xact_start, S) - floor - clock_margin` for `M`'s + /// handshake: every channel-window row with `created_at <= fence_wall` + /// is present on any session observing `token >= M`. + pub fence_wall: DateTime, +} + +#[derive(Debug, Default)] +struct FenceInner { + /// Epoch the retained ring belongs to. `None` until the first probe — + /// or after the test hook, whose injected entry deliberately bypasses + /// the epoch comparison in [`ReplicaFence::resolve`]. + epoch: Option, + /// Retained entries in strictly increasing token order. + ring: VecDeque, +} + +/// Outcome of recording one probe sample. +#[derive(Debug, PartialEq, Eq)] +pub enum RecordOutcome { + /// Entry retained; proofs may cite it. + Recorded, + /// The token went backwards within the same epoch — a restore that kept + /// the old epoch. The ring was cleared and the entry discarded; the + /// probe must rotate the epoch before recording again (a reader still on + /// the pre-rewind timeline could otherwise observe a *higher* token that + /// proves nothing about the new timeline). + TokenRegression, +} + +/// Outcome of resolving one reader-session observation against the ring. +/// Everything but [`ResolveOutcome::Proved`] fails closed (routes to the +/// writer); the variants exist so route metrics can name the reason. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ResolveOutcome { + /// The observation proves this retained entry. + Proved(TokenEntry), + /// The observed epoch is not the ring's epoch — the session is on a + /// different timeline (restore) or the ring was rotated under it. + EpochMismatch, + /// The observed token is below every retained entry: the reader lags + /// further than the ring's history (or the ring is empty). + TokenBehind, +} + +impl ResolveOutcome { + /// The proved entry, if any. + pub fn proved(self) -> Option { + match self { + ResolveOutcome::Proved(entry) => Some(entry), + _ => None, + } + } +} + +/// Shared fence state. `Db` holds an `Arc` of this; the probe task records +/// entries and per-request routing resolves proofs against it. +#[derive(Debug, Default)] pub struct ReplicaFence { - /// Unix micros of the newest verified-complete timestamp, or `CLOSED`. - fence_micros: AtomicI64, - /// Unix micros when the fence was last advanced (staleness check). - updated_micros: AtomicI64, + inner: Mutex, } impl ReplicaFence { - /// A new fence, initially closed. + /// A new fence, initially closed (empty ring). pub fn new() -> Self { - Self { - fence_micros: AtomicI64::new(CLOSED), - updated_micros: AtomicI64::new(CLOSED), - } + Self::default() } - /// Close the fence: all cursor reads route to the writer. + /// Close the fence: drop all retained proofs; reads route to the writer. pub fn close(&self) { - self.fence_micros.store(CLOSED, Ordering::Relaxed); + let mut inner = self.inner.lock().expect("fence lock poisoned"); + inner.ring.clear(); } - fn advance(&self, fence: DateTime) { - self.fence_micros - .store(fence.timestamp_micros(), Ordering::Relaxed); - self.updated_micros - .store(Utc::now().timestamp_micros(), Ordering::Relaxed); + /// Record one probe sample. Epoch changes (re-seed) clear the ring and + /// start a new one under the observed epoch — sound, because an entry + /// only proves commits on its own timeline and readers must match the + /// epoch to cite it. A same-epoch token regression is the unsafe case: + /// see [`RecordOutcome::TokenRegression`]. + pub fn record( + &self, + token: i64, + epoch: Uuid, + committed_at: Instant, + fence_wall: DateTime, + ) -> RecordOutcome { + let mut inner = self.inner.lock().expect("fence lock poisoned"); + if inner.epoch != Some(epoch) { + inner.ring.clear(); + inner.epoch = Some(epoch); + } else if inner.ring.back().is_some_and(|last| token <= last.token) { + inner.ring.clear(); + return RecordOutcome::TokenRegression; + } + if inner.ring.len() == RING_CAPACITY { + inner.ring.pop_front(); + } + inner.ring.push_back(TokenEntry { + token, + committed_at, + fence_wall, + }); + RecordOutcome::Recorded } - /// The current fence, or `None` when closed or stale. + /// Resolve the strongest proof a reader session's observation supports: + /// the greatest retained entry with `entry.token <= observed_token`, + /// provided the observed epoch matches the ring's. Non-`Proved` outcomes + /// fail closed; they are distinguished only for route metrics. + pub fn resolve(&self, observed_token: i64, observed_epoch: Uuid) -> ResolveOutcome { + let inner = self.inner.lock().expect("fence lock poisoned"); + match inner.epoch { + Some(e) if e != observed_epoch => return ResolveOutcome::EpochMismatch, + // `None` with a non-empty ring only happens via the test hook; + // the epoch comparison is deliberately skipped there. + _ => {} + } + inner + .ring + .iter() + .rev() + .find(|entry| entry.token <= observed_token) + .copied() + .map_or(ResolveOutcome::TokenBehind, ResolveOutcome::Proved) + } + + /// The newest retained entry, staleness-gated. Used as the cheap + /// pre-check before spending a reader checkout, and for observability. + pub fn newest(&self) -> Option { + let inner = self.inner.lock().expect("fence lock poisoned"); + inner + .ring + .back() + .filter(|entry| entry.committed_at.elapsed() <= FENCE_STALENESS) + .copied() + } + + /// Age of the newest retained entry, ungated (observability: how long + /// since the probe last committed a token). + pub fn heartbeat_age(&self) -> Option { + let inner = self.inner.lock().expect("fence lock poisoned"); + inner.ring.back().map(|entry| entry.committed_at.elapsed()) + } + + /// The newest fence wall, or `None` when closed or stale. /// - /// Rows with `created_at <= fence` are verified present on the replica. + /// Rows with `created_at <= fence` are verified present on a reader + /// session that proves the newest entry; whether a *given* session does + /// is decided per request via [`ReplicaFence::resolve`]. pub fn verified_through(&self) -> Option> { - let raw = self.fence_micros.load(Ordering::Relaxed); - if raw == CLOSED { - return None; - } - let updated = self.updated_micros.load(Ordering::Relaxed); - let age_micros = Utc::now().timestamp_micros().saturating_sub(updated); - if age_micros > FENCE_STALENESS.as_micros() as i64 { - return None; - } - DateTime::from_timestamp_micros(raw) + self.newest().map(|entry| entry.fence_wall) } - /// Whether the replica verifiably holds every channel-window row at or - /// before `ts`. + /// Whether some retained entry's wall covers `ts` — the cheap routing + /// pre-check (the connection-local observation still has to prove it). pub fn covers(&self, ts: DateTime) -> bool { self.verified_through().is_some_and(|fence| ts <= fence) } /// Test hook: force the fence open through `ts` without a probe. - /// Used by routing tests that stand up a divergent fake replica. + /// Injects an entry any observed token satisfies (`i64::MIN`) with no + /// epoch recorded, so the epoch comparison is bypassed — routing tests + /// stand up a divergent fake replica whose heartbeat epoch differs from + /// the writer's. pub fn force_open_for_tests(&self, ts: DateTime) { - self.advance(ts); + self.force_open_for_tests_at(ts, Instant::now()); } -} -impl Default for ReplicaFence { - fn default() -> Self { - Self::new() + /// [`ReplicaFence::force_open_for_tests`] with an explicit commit + /// instant, for pinning age-gated behavior (head-budget and staleness + /// tests inject entries "committed" in the past). + pub fn force_open_for_tests_at(&self, ts: DateTime, committed_at: Instant) { + let mut inner = self.inner.lock().expect("fence lock poisoned"); + inner.epoch = None; + inner.ring.clear(); + inner.ring.push_back(TokenEntry { + token: i64::MIN, + committed_at, + fence_wall: ts, + }); } } @@ -332,15 +505,20 @@ struct WriterSample { /// Oldest open transaction among other client backends at scan time, /// or `None` when no transaction was open. oldest_xact_start: Option>, - /// `L`: writer `pg_current_wal_lsn()` captured last, as text. - wal_lsn: String, + /// `M`: the heartbeat token committed **last**, after the scan. + token: i64, + /// Heartbeat epoch returned with `M`. + epoch: Uuid, + /// Monotonic instant captured immediately before committing `M` — an + /// upper bound on how old a session observing `token >= M` can be. + committed_at: Instant, } /// Errors that close the fence. All variants are logged and treated /// identically: fail closed. #[derive(Debug, thiserror::Error)] pub enum ProbeError { - /// A probe query against writer or replica failed. + /// A probe query against the writer failed. #[error("writer probe query failed: {0}")] Writer(#[from] sqlx::Error), /// `pg_stat_activity` hid state for another backend that could hold an @@ -353,14 +531,15 @@ pub enum ProbeError { /// Number of other client backends with masked/unknown state. masked: i64, }, - /// The replica returned NULL for the replay-LSN comparison. - #[error("replica did not report a comparable replay LSN")] - ReplicaLsnUnavailable, + /// The single heartbeat row (migration 0026) is missing on the writer. + #[error("replica_heartbeat row missing on the writer — migration 0026 not applied?")] + HeartbeatRowMissing, } -/// Take one ordered writer sample: S, then activity scan, then L **last**. +/// Take one ordered writer sample: S, then activity scan, then commit the +/// heartbeat token **last**. /// -/// The three statements are separately awaited on a single pinned connection; +/// The statements are separately awaited on a single pinned connection; /// a single SELECT would not guarantee evaluation order across the /// subexpressions, reopening the race this ordering exists to close. async fn sample_writer(writer: &PgPool) -> Result { @@ -393,8 +572,8 @@ async fn sample_writer(writer: &PgPool) -> Result { // // Prepared transactions (2PC) are a bucket of their own: while // prepared they have left `pg_stat_activity` but can still commit - // after `L`. Their deferred floor guard already ran at PREPARE, so - // `pg_prepared_xacts.prepared` bounds their rows exactly like + // after the token. Their deferred floor guard already ran at PREPARE, + // so `pg_prepared_xacts.prepared` bounds their rows exactly like // `xact_start`; fold it into the same minimum. let row = sqlx::query( r#" @@ -423,80 +602,171 @@ async fn sample_writer(writer: &PgPool) -> Result { } let oldest_xact_start: Option> = row.get("oldest_xact_start"); - // 3. L last. - let wal_lsn: String = sqlx::query_scalar("SELECT pg_current_wal_lsn()::text") - .fetch_one(&mut *conn) - .await?; + // 3. Token commit LAST, on the same pinned connection. The single-row + // UPDATE serializes concurrent pods' probes, so RETURNING token is + // globally commit-ordered. `committed_at` is captured before the + // round trip so `elapsed()` over-estimates the observation's age — + // the conservative direction for the head-freshness bound. + let committed_at = Instant::now(); + let row = sqlx::query( + "UPDATE replica_heartbeat SET token = token + 1 WHERE id = 1 RETURNING token, epoch", + ) + .fetch_optional(&mut *conn) + .await? + .ok_or(ProbeError::HeartbeatRowMissing)?; Ok(WriterSample { sampled_at, oldest_xact_start, - wal_lsn, + token: row.get("token"), + epoch: row.get("epoch"), + committed_at, }) } -/// Whether the replica has replayed at least through `wal_lsn`. -/// -/// The comparison happens on the replica in pg_lsn domain. The -/// `pg_is_in_recovery()` gate is load-bearing: after crash recovery or -/// promotion a *primary* returns a static non-NULL `pg_last_wal_replay_lsn()` -/// rather than NULL, so NULL-checking alone would not reliably detect a -/// misrouted "replica" URL. Not-in-recovery, NULL replay LSN, or Aurora -/// hiding either is an error → fence closes. -async fn replica_covers(replica: &PgPool, wal_lsn: &str) -> Result { - let covered: Option = sqlx::query_scalar( - r#" - SELECT CASE - WHEN pg_is_in_recovery() THEN pg_last_wal_replay_lsn() >= $1::pg_lsn - ELSE NULL - END - "#, - ) - .bind(wal_lsn) - .fetch_one(replica) - .await?; - covered.ok_or(ProbeError::ReplicaLsnUnavailable) +/// The fence wall proved by one handshake: +/// `min(oldest_xact_start, S) - floor - clock_margin`. +fn fence_wall(sample_s: DateTime, oldest_xact_start: Option>) -> DateTime { + let lower = match oldest_xact_start { + Some(oldest) => oldest.min(sample_s), + None => sample_s, + }; + lower + - chrono::Duration::seconds(CREATED_AT_FLOOR_SECS) + - chrono::Duration::seconds(FENCE_CLOCK_MARGIN_SECS) } -/// Run one full handshake and, on success, advance the fence. +/// Run one full handshake and record the resulting `(token, fence_wall)`. /// -/// Returns the new fence value for observability. `Ok(None)` means the -/// replica has not yet replayed past the sample; the fence is left as-is -/// (staleness will close it if this persists). -pub async fn probe_once( - writer: &PgPool, - replica: &PgPool, - fence: &ReplicaFence, -) -> Result>, ProbeError> { +/// On a same-epoch token regression (the writer was restored from a backup +/// that kept its epoch), the retained ring has already been cleared by +/// [`ReplicaFence::record`]; this additionally **rotates the epoch** on the +/// writer and records the rotated token, so a reader still serving the +/// pre-rewind timeline (whose old, higher token would otherwise satisfy +/// `token >= M`) fails the epoch check instead of proving stale coverage. +pub async fn probe_once(writer: &PgPool, fence: &ReplicaFence) -> Result { let sample = sample_writer(writer).await?; - if !replica_covers(replica, &sample.wal_lsn).await? { - return Ok(None); + let wall = fence_wall(sample.sampled_at, sample.oldest_xact_start); + match fence.record(sample.token, sample.epoch, sample.committed_at, wall) { + RecordOutcome::Recorded => Ok(TokenEntry { + token: sample.token, + committed_at: sample.committed_at, + fence_wall: wall, + }), + RecordOutcome::TokenRegression => { + tracing::warn!( + token = sample.token, + "replica heartbeat token regressed within its epoch (restore?); rotating epoch" + ); + // The rotation commit happens after this sample's activity scan, + // so the same three-bucket argument (and the same wall) holds + // for the rotated token. + let committed_at = Instant::now(); + let row = sqlx::query( + "UPDATE replica_heartbeat SET epoch = gen_random_uuid(), token = token + 1 \ + WHERE id = 1 RETURNING token, epoch", + ) + .fetch_optional(writer) + .await? + .ok_or(ProbeError::HeartbeatRowMissing)?; + let token: i64 = row.get("token"); + let epoch: Uuid = row.get("epoch"); + // A fresh epoch always clears and records; regression is + // impossible against an empty ring. + fence.record(token, epoch, committed_at, wall); + Ok(TokenEntry { + token, + committed_at, + fence_wall: wall, + }) + } } - let lower = match sample.oldest_xact_start { - Some(oldest) => oldest.min(sample.sampled_at), - None => sample.sampled_at, +} + +/// The Aurora **PostgreSQL** instance-identity function. Named once so the +/// capability probe and the observation query can never disagree — and +/// pinned by a unit test, because the MySQL-family spelling +/// (`aurora_server_id`) is a near-miss that would make the capability probe +/// cache a permanent `false` on real Aurora (42883) and silently strip the +/// instance id from canary evidence. +pub const AURORA_IDENTITY_FN: &str = "aurora_db_instance_identifier"; + +/// Whether this reader endpoint supports [`AURORA_IDENTITY_FN`] — probed +/// ONCE per process on a plain autocommit checkout, never inside a request +/// transaction (an undefined-function error would abort the transaction +/// and fail the proof). `Ok(false)` is the definitive "not Aurora" answer +/// (undefined_function, SQLSTATE 42883); transient errors surface as `Err` +/// so the caller can retry the probe on a later request instead of caching +/// a wrong answer. +pub async fn reader_supports_aurora_identity(conn: &mut PgConnection) -> Result { + match sqlx::query(sqlx::AssertSqlSafe(format!( + "SELECT {AURORA_IDENTITY_FN}()" + ))) + .fetch_one(&mut *conn) + .await + { + Ok(_) => Ok(true), + Err(sqlx::Error::Database(e)) if e.code().as_deref() == Some("42883") => Ok(false), + Err(e) => Err(e), + } +} + +/// Observe the heartbeat on a specific reader session — the +/// connection-local half of the proof. Returns the observed token/epoch +/// plus the backend identity of the session for route-decision evidence: +/// `addr:port pid=N` (`local` on unix sockets), prefixed with the Aurora +/// instance id when `aurora` is set (only pass `true` after +/// [`reader_supports_aurora_identity`] confirmed it — the function +/// reference fails at parse time on plain Postgres). `None` when the row +/// is missing there (migration not yet replayed): fail closed. +pub async fn observe_heartbeat( + conn: &mut PgConnection, + aurora: bool, +) -> Result, sqlx::Error> { + const ADDR_PID: &str = "COALESCE(host(inet_server_addr()) || ':' || \ + inet_server_port()::text, 'local') || ' pid=' || pg_backend_pid()::text"; + let sql = if aurora { + format!( + "SELECT token, epoch, {AURORA_IDENTITY_FN}() || ' @ ' || {ADDR_PID} AS backend \ + FROM replica_heartbeat WHERE id = 1" + ) + } else { + format!( + "SELECT token, epoch, {ADDR_PID} AS backend \ + FROM replica_heartbeat WHERE id = 1" + ) }; - let new_fence = lower - - chrono::Duration::seconds(CREATED_AT_FLOOR_SECS) - - chrono::Duration::seconds(FENCE_CLOCK_MARGIN_SECS); - fence.advance(new_fence); - Ok(Some(new_fence)) + let row = sqlx::query(sqlx::AssertSqlSafe(sql)) + .fetch_optional(&mut *conn) + .await?; + Ok(row.map(|r| HeartbeatObservation { + token: r.get("token"), + epoch: r.get("epoch"), + backend: r.get("backend"), + })) } -/// Background probe loop: sample every `PROBE_INTERVAL`, close the fence on -/// any error. Runs for the life of the process. -pub async fn run_probe(writer: PgPool, replica: PgPool, fence: Arc) { +/// One reader-session heartbeat observation (see [`observe_heartbeat`]). +#[derive(Debug, Clone)] +pub struct HeartbeatObservation { + /// The token the session has replayed through. + pub token: i64, + /// The epoch the session observes — must match the ring's. + pub epoch: Uuid, + /// Backend identity of the observed session, so live evidence records + /// which reader served both proof and page. + pub backend: String, +} + +/// Background probe loop: commit a heartbeat token every `PROBE_INTERVAL`; +/// close the fence on any error. Runs for the life of the process. +pub async fn run_probe(writer: PgPool, fence: Arc) { let mut interval = tokio::time::interval(PROBE_INTERVAL); interval.set_missed_tick_behavior(tokio::time::MissedTickBehavior::Delay); loop { interval.tick().await; - match probe_once(&writer, &replica, &fence).await { - Ok(Some(_)) => {} - Ok(None) => { - // Replica behind the sample: leave the fence; staleness - // closes it if the replica stays behind. - tracing::debug!("replica fence: replay behind writer sample"); - } + match probe_once(&writer, &fence).await { + Ok(_) => {} Err(e) => { fence.close(); tracing::warn!(error = %e, "replica fence probe failed; fence closed"); @@ -515,14 +785,50 @@ mod tests { std::env::var("TEST_DATABASE_URL").unwrap_or_else(|_| TEST_DB_URL.into()) } + /// A private scratch database with migrations applied: the probe tests + /// mutate the singleton heartbeat row (rewind/rotate), which must never + /// race the shared dev database or each other. + async fn scratch_db() -> (PgPool, PgPool, String) { + let admin = PgPool::connect(&test_db_url()) + .await + .expect("connect admin"); + let name = format!("fence_probe_{}", uuid::Uuid::new_v4().simple()); + sqlx::query(sqlx::AssertSqlSafe(format!("CREATE DATABASE {name}"))) + .execute(&admin) + .await + .expect("create scratch db"); + let base = test_db_url(); + let idx = base.rfind('/').expect("db url has a path segment"); + let pool = PgPool::connect(&format!("{}/{}", &base[..idx], name)) + .await + .expect("connect scratch db"); + crate::migration::run_migrations(&pool) + .await + .expect("migrate scratch db"); + (admin, pool, name) + } + + async fn drop_scratch_db(admin: &PgPool, pool: PgPool, name: &str) { + pool.close().await; + let _ = sqlx::query(sqlx::AssertSqlSafe(format!( + "DROP DATABASE IF EXISTS {name} WITH (FORCE)" + ))) + .execute(admin) + .await; + } + #[test] - fn fence_starts_closed_and_opens_on_advance() { + fn fence_starts_closed_and_opens_on_record() { let fence = ReplicaFence::new(); assert!(fence.verified_through().is_none(), "must start closed"); assert!(!fence.covers(Utc::now() - chrono::Duration::days(365))); let ts = Utc::now(); - fence.advance(ts); + let epoch = Uuid::new_v4(); + assert_eq!( + fence.record(1, epoch, Instant::now(), ts), + RecordOutcome::Recorded + ); assert_eq!(fence.verified_through(), Some(ts)); assert!(fence.covers(ts - chrono::Duration::seconds(1))); assert!(fence.covers(ts), "boundary is inclusive"); @@ -537,19 +843,116 @@ mod tests { fn stale_fence_reads_as_closed() { let fence = ReplicaFence::new(); let ts = Utc::now(); - fence - .fence_micros - .store(ts.timestamp_micros(), Ordering::Relaxed); - // Last update older than the staleness budget. - let stale = (Utc::now() - - chrono::Duration::from_std(FENCE_STALENESS).expect("duration") - - chrono::Duration::seconds(1)) - .timestamp_micros(); - fence.updated_micros.store(stale, Ordering::Relaxed); + // Newest entry committed longer ago than the staleness budget. + let stale_instant = Instant::now() - (FENCE_STALENESS + Duration::from_secs(1)); + fence.record(1, Uuid::new_v4(), stale_instant, ts); assert!( fence.verified_through().is_none(), "a fence the probe stopped confirming must read as closed" ); + // heartbeat_age is deliberately ungated (observability). + assert!(fence.heartbeat_age().expect("entry retained") > FENCE_STALENESS); + } + + /// Resolve picks the greatest retained entry <= the observed token — + /// a lagged reader proves from an older wall, never from thin air. + #[test] + fn resolve_picks_greatest_retained_token_at_or_below_observation() { + let fence = ReplicaFence::new(); + let epoch = Uuid::new_v4(); + let base = Utc::now(); + for (token, secs) in [(10i64, 0i64), (20, 10), (30, 20)] { + fence.record( + token, + epoch, + Instant::now(), + base + chrono::Duration::seconds(secs), + ); + } + + // Exact hit. + assert_eq!(fence.resolve(20, epoch).proved().expect("proof").token, 20); + // Between entries: prove from the older one. + assert_eq!(fence.resolve(25, epoch).proved().expect("proof").token, 20); + // Ahead of everything retained: newest. + assert_eq!( + fence.resolve(1000, epoch).proved().expect("proof").token, + 30 + ); + // Behind everything retained: no proof. + assert_eq!( + fence.resolve(9, epoch), + ResolveOutcome::TokenBehind, + "token below ring fails closed" + ); + // Wrong epoch: no proof, regardless of token. + assert_eq!( + fence.resolve(1000, Uuid::new_v4()), + ResolveOutcome::EpochMismatch, + "epoch mismatch fails closed" + ); + } + + /// An epoch change clears the ring and starts a new one; a same-epoch + /// token regression clears the ring and reports the fault. + #[test] + fn record_epoch_change_resets_and_same_epoch_regression_fails() { + let fence = ReplicaFence::new(); + let epoch_a = Uuid::new_v4(); + let ts = Utc::now(); + fence.record(10, epoch_a, Instant::now(), ts); + fence.record(11, epoch_a, Instant::now(), ts); + + // New epoch, lower token: fine — new timeline, old proofs dropped. + let epoch_b = Uuid::new_v4(); + assert_eq!( + fence.record(3, epoch_b, Instant::now(), ts), + RecordOutcome::Recorded + ); + assert_eq!( + fence.resolve(11, epoch_a), + ResolveOutcome::EpochMismatch, + "entries from the old epoch must be gone" + ); + assert_eq!(fence.resolve(3, epoch_b).proved().expect("proof").token, 3); + + // Same epoch, non-increasing token: regression → cleared + reported. + assert_eq!( + fence.record(3, epoch_b, Instant::now(), ts), + RecordOutcome::TokenRegression + ); + assert!( + fence.verified_through().is_none(), + "ring cleared on regression" + ); + assert_eq!( + fence.resolve(i64::MAX, epoch_b), + ResolveOutcome::TokenBehind + ); + } + + /// The ring is bounded: old entries fall off and stop proving coverage. + #[test] + fn ring_capacity_evicts_oldest_entries() { + let fence = ReplicaFence::new(); + let epoch = Uuid::new_v4(); + let ts = Utc::now(); + for token in 0..(RING_CAPACITY as i64 + 10) { + fence.record(token, epoch, Instant::now(), ts); + } + assert_eq!( + fence.resolve(5, epoch), + ResolveOutcome::TokenBehind, + "evicted tokens must no longer prove coverage" + ); + assert_eq!( + fence + .resolve(i64::MAX, epoch) + .proved() + .expect("proof") + .token, + RING_CAPACITY as i64 + 9 + ); } /// The activity scan must (a) represent another session's open @@ -582,9 +985,14 @@ mod tests { oldest <= during.sampled_at, "xact_start precedes the sample that observed it" ); - // S is captured before the activity scan, L after: the sample's - // ordering invariant. + // S is captured before the activity scan, the token commit after: + // the sample's ordering invariant. assert!(during.sampled_at >= before.sampled_at); + assert!( + during.token > before.token, + "each sample must commit a strictly newer token" + ); + assert_eq!(during.epoch, before.epoch, "epoch is stable across samples"); tx.rollback().await.expect("rollback"); } @@ -641,21 +1049,140 @@ mod tests { .expect("drop role"); } - /// A primary (non-replica) database returns NULL from - /// `pg_last_wal_replay_lsn()`; the probe must fail closed, never - /// synthesize freshness. This is also the Aurora-observability guard: - /// if the reader endpoint hides replay LSNs, routing stays writer-only. + /// The Aurora PostgreSQL identity function name is exact — the + /// MySQL-family near-miss (`aurora_server_id`) would make the + /// capability probe cache a permanent false on real Aurora and + /// silently strip the instance id from canary evidence (Wren, delta + /// review of a472327). AWS reference: aurora_db_instance_identifier() + /// (Aurora PostgreSQL user guide; also awslabs/pg-collector). + #[test] + fn aurora_identity_function_name_is_the_postgres_one() { + assert_eq!(AURORA_IDENTITY_FN, "aurora_db_instance_identifier"); + } + + /// The Aurora identity capability probe must answer a definitive + /// `false` on plain Postgres (undefined_function), not error — and the + /// error path must not poison the connection for later statements. #[tokio::test] #[ignore = "requires Postgres"] - async fn probe_fails_closed_when_replica_lsn_unavailable() { + async fn aurora_identity_probe_reports_false_on_plain_postgres() { let pool = PgPool::connect(&test_db_url()).await.expect("connect"); + let mut conn = pool.acquire().await.expect("conn"); + assert!( + !reader_supports_aurora_identity(&mut conn) + .await + .expect("probe must not error on plain postgres"), + "plain postgres must report no aurora identity support" + ); + // The failed function lookup must not have wedged the session. + let one: i32 = sqlx::query_scalar("SELECT 1") + .fetch_one(&mut *conn) + .await + .expect("connection usable after probe"); + assert_eq!(one, 1); + } + + /// End-to-end probe against a real database: each probe commits a + /// strictly newer token, records a retained entry, and a session on the + /// same database observes a token/epoch that resolves that entry. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn probe_commits_tokens_and_sessions_prove_coverage() { + let (admin, pool, name) = scratch_db().await; + let fence = ReplicaFence::new(); + + let first = probe_once(&pool, &fence).await.expect("first probe"); + let second = probe_once(&pool, &fence).await.expect("second probe"); + assert!(second.token > first.token, "tokens strictly increase"); + assert!( + second.fence_wall >= first.fence_wall + || second.fence_wall + > first.fence_wall - chrono::Duration::seconds(FENCE_CLOCK_MARGIN_SECS), + "walls advance with the clock (modulo an open transaction)" + ); + + // A "reader" session on the same database observes at least the + // second token and proves the newest retained entry. + let mut conn = pool.acquire().await.expect("reader conn"); + let obs = observe_heartbeat(&mut conn, false) + .await + .expect("observe") + .expect("heartbeat row present"); + assert!(obs.token >= second.token); + assert!( + obs.backend.contains(" pid="), + "backend identity must carry the backend pid, got {:?}", + obs.backend + ); + // TCP fixtures also carry addr:port; unix-socket fixtures read 'local'. + assert!( + obs.backend.starts_with("local pid=") || obs.backend.contains(':'), + "backend identity must carry addr:port or 'local', got {:?}", + obs.backend + ); + let proof = fence + .resolve(obs.token, obs.epoch) + .proved() + .expect("proof resolves"); + assert_eq!(proof.token, second.token, "newest retained entry cited"); + + // An epoch nobody committed proves nothing. + assert_eq!( + fence.resolve(obs.token, Uuid::new_v4()), + ResolveOutcome::EpochMismatch + ); + + drop(conn); + drop_scratch_db(&admin, pool, &name).await; + } + + /// A same-epoch token rewind on the writer (restore adversary) must not + /// leave proofs standing: the probe rotates the epoch, so a reader still + /// on the pre-rewind timeline — observing a *higher* token under the old + /// epoch — fails the epoch check instead of proving stale coverage. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn probe_rotates_epoch_on_same_epoch_token_regression() { + let (admin, pool, name) = scratch_db().await; let fence = ReplicaFence::new(); - fence.advance(Utc::now()); // pretend a previous handshake succeeded - // Using the primary as its own "replica": replay LSN is NULL. - let err = probe_once(&pool, &pool, &fence) + let before = probe_once(&pool, &fence).await.expect("probe"); + let mut conn = pool.acquire().await.expect("conn"); + let old_epoch = observe_heartbeat(&mut conn, false) + .await + .expect("observe") + .expect("row") + .epoch; + + // Rewind the token in place, keeping the epoch: the restore shape. + sqlx::query("UPDATE replica_heartbeat SET token = 0 WHERE id = 1") + .execute(&pool) + .await + .expect("rewind token"); + + let after = probe_once(&pool, &fence).await.expect("recovery probe"); + // The pre-rewind observation must no longer prove anything. + assert_eq!( + fence.resolve(before.token, old_epoch), + ResolveOutcome::EpochMismatch, + "old-epoch observations must fail closed after rotation" + ); + // A fresh observation on the new timeline proves the rotated entry. + let obs = observe_heartbeat(&mut conn, false) .await - .expect_err("NULL replay LSN must be an error"); - assert!(matches!(err, ProbeError::ReplicaLsnUnavailable)); + .expect("observe") + .expect("row"); + assert_ne!(obs.epoch, old_epoch, "epoch rotated"); + assert_eq!( + fence + .resolve(obs.token, obs.epoch) + .proved() + .expect("proof") + .token, + after.token + ); + + drop(conn); + drop_scratch_db(&admin, pool, &name).await; } } diff --git a/crates/buzz-db/src/thread.rs b/crates/buzz-db/src/thread.rs index 3f92212dd5..007677e258 100644 --- a/crates/buzz-db/src/thread.rs +++ b/crates/buzz-db/src/thread.rs @@ -349,6 +349,30 @@ pub async fn get_thread_replies( depth_limit: Option, limit: u32, cursor: Option<&[u8]>, +) -> Result> { + let mut conn = pool.acquire().await?; + get_thread_replies_on( + &mut conn, + community_id, + root_event_id, + depth_limit, + limit, + cursor, + ) + .await +} + +/// [`get_thread_replies`] on a specific session — the replica-routing path +/// runs the page on the exact reader connection whose heartbeat observation +/// proved coverage (the proof is connection-local; a different pooled +/// session may sit at a different replay position). +pub(crate) async fn get_thread_replies_on( + conn: &mut sqlx::PgConnection, + community_id: CommunityId, + root_event_id: &[u8], + depth_limit: Option, + limit: u32, + cursor: Option<&[u8]>, ) -> Result> { // Decode cursor bytes -> keyset (timestamp, optional event_id) for the // WHERE condition. Layout: 8-byte BE i64 seconds, then the raw event_id. @@ -445,7 +469,7 @@ pub async fn get_thread_replies( } q = q.bind(limit as i32); - let rows = q.fetch_all(pool).await?; + let rows = q.fetch_all(&mut *conn).await?; let mut replies = Vec::with_capacity(rows.len()); for row in rows { @@ -569,6 +593,31 @@ pub async fn get_channel_window( limit: u32, cursor: Option<(DateTime, Vec)>, kind_filter: Option<&[u32]>, +) -> Result { + let mut conn = pool.acquire().await?; + get_channel_window_on( + &mut conn, + community_id, + channel_id, + limit, + cursor, + kind_filter, + ) + .await +} + +/// [`get_channel_window`] on a specific session — the replica-routing path +/// runs the page (and its participants batch) on the exact reader connection +/// whose heartbeat observation proved coverage (the proof is +/// connection-local; a different pooled session may sit at a different +/// replay position). +pub(crate) async fn get_channel_window_on( + conn: &mut sqlx::PgConnection, + community_id: CommunityId, + channel_id: Uuid, + limit: u32, + cursor: Option<(DateTime, Vec)>, + kind_filter: Option<&[u32]>, ) -> Result { let mut param_idx = 3u32; // $1 is community_id, $2 is channel_id let mut sql = String::from( @@ -637,7 +686,7 @@ pub async fn get_channel_window( // The +1 probe row is the server-internal has_more evidence. q = q.bind(limit as i64 + 1); - let mut db_rows = q.fetch_all(pool).await?; + let mut db_rows = q.fetch_all(&mut *conn).await?; let has_more = db_rows.len() > limit as usize; db_rows.truncate(limit as usize); @@ -722,7 +771,7 @@ pub async fn get_channel_window( ) .bind(community_id.as_uuid()) .bind(&roots) - .fetch_all(pool) + .fetch_all(&mut *conn) .await?; let mut by_root: std::collections::HashMap, Vec>> = diff --git a/crates/buzz-media/src/config.rs b/crates/buzz-media/src/config.rs index 047c08475e..3c70e4afe1 100644 --- a/crates/buzz-media/src/config.rs +++ b/crates/buzz-media/src/config.rs @@ -1,5 +1,38 @@ //! Media storage configuration. +use std::str::FromStr; + +/// S3 URL addressing style shared by media and Git/CAS storage. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, serde::Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum S3AddressingStyle { + /// Put the bucket in the request path (`https://endpoint/bucket/key`). + /// + /// This preserves compatibility with the bundled MinIO deployments, whose + /// internal DNS only resolves the endpoint hostname. + #[default] + Path, + /// Put the bucket in the hostname (`https://bucket.endpoint/key`). + /// + /// This is the standard S3 form and is required by providers such as new + /// Railway Storage Buckets. + Virtual, +} + +impl FromStr for S3AddressingStyle { + type Err = String; + + fn from_str(value: &str) -> Result { + match value { + "path" => Ok(Self::Path), + "virtual" => Ok(Self::Virtual), + _ => Err(format!( + "BUZZ_S3_ADDRESSING_STYLE must be 'path' or 'virtual', got {value:?}" + )), + } + } +} + fn default_max_video_bytes() -> u64 { 524_288_000 // 500 MB } @@ -31,6 +64,9 @@ pub struct MediaConfig { /// the value is not meaningfully checked. #[serde(default = "default_s3_region")] pub s3_region: String, + /// S3 URL addressing style. Defaults to path style for MinIO compatibility. + #[serde(default)] + pub s3_addressing_style: S3AddressingStyle, /// Maximum upload size for images (bytes). Default: 50 MB. pub max_image_bytes: u64, /// Maximum upload size for animated GIFs (bytes). Default: 10 MB. @@ -123,7 +159,8 @@ impl MediaConfig { #[cfg(test)] mod tests { - use super::MediaConfig; + use super::{MediaConfig, S3AddressingStyle}; + use std::str::FromStr; fn valid_config() -> MediaConfig { MediaConfig { @@ -132,6 +169,7 @@ mod tests { s3_secret_key: "s".to_string(), s3_bucket: "buzz-media".to_string(), s3_region: "us-east-1".to_string(), + s3_addressing_style: S3AddressingStyle::Path, max_image_bytes: 1, max_gif_bytes: 1, max_video_bytes: 1, @@ -143,6 +181,35 @@ mod tests { } } + #[test] + fn addressing_style_parses_supported_values() { + assert_eq!( + S3AddressingStyle::from_str("path"), + Ok(S3AddressingStyle::Path) + ); + assert_eq!( + S3AddressingStyle::from_str("virtual"), + Ok(S3AddressingStyle::Virtual) + ); + } + + #[test] + fn addressing_style_defaults_to_path() { + assert_eq!(S3AddressingStyle::default(), S3AddressingStyle::Path); + } + + #[test] + fn addressing_style_rejects_unknown_or_ambiguous_values() { + for invalid in ["", "auto", "PATH", "virtual-hosted"] { + let error = + S3AddressingStyle::from_str(invalid).expect_err("must reject invalid style"); + assert!( + error.contains("BUZZ_S3_ADDRESSING_STYLE must be 'path' or 'virtual'"), + "unexpected error for {invalid:?}: {error}" + ); + } + } + #[test] fn upload_record_knobs_default_off_and_validate() { assert!(valid_config().validate().is_ok()); diff --git a/crates/buzz-media/src/lib.rs b/crates/buzz-media/src/lib.rs index ac05ea6d51..67896d4ef2 100644 --- a/crates/buzz-media/src/lib.rs +++ b/crates/buzz-media/src/lib.rs @@ -17,7 +17,7 @@ pub use bucket_index::{ classify_key, fold_bucket_listing, BucketAggregate, BucketSnapshot, CommunityStorage, KeyClass, Page, SweepError, }; -pub use config::MediaConfig; +pub use config::{MediaConfig, S3AddressingStyle}; pub use error::MediaError; pub use storage::{BlobHeadMeta, BlobMeta, ByteStream, MediaStorage}; pub use types::BlobDescriptor; diff --git a/crates/buzz-media/src/storage.rs b/crates/buzz-media/src/storage.rs index 0e9809af2f..cbf980201f 100644 --- a/crates/buzz-media/src/storage.rs +++ b/crates/buzz-media/src/storage.rs @@ -5,7 +5,7 @@ use std::pin::Pin; use buzz_core::tenant::{CommunityId, TenantContext}; -use crate::config::MediaConfig; +use crate::config::{MediaConfig, S3AddressingStyle}; use crate::error::MediaError; use bytes::Bytes; use s3::creds::Credentials; @@ -61,8 +61,11 @@ impl MediaStorage { } .map_err(|e| MediaError::StorageError(e.to_string()))?; let bucket = Bucket::new(&config.s3_bucket, region, creds) - .map_err(|e| MediaError::StorageError(e.to_string()))? - .with_path_style(); + .map_err(|e| MediaError::StorageError(e.to_string()))?; + let bucket = match config.s3_addressing_style { + S3AddressingStyle::Path => bucket.with_path_style(), + S3AddressingStyle::Virtual => bucket, + }; Ok(Self { bucket }) } @@ -285,6 +288,7 @@ mod tests { s3_secret_key: secret.to_string(), s3_bucket: "buzz-media".to_string(), s3_region: "us-west-2".to_string(), + s3_addressing_style: S3AddressingStyle::Path, max_image_bytes: 50 * 1024 * 1024, max_gif_bytes: 10 * 1024 * 1024, max_video_bytes: 524_288_000, @@ -309,6 +313,23 @@ mod tests { } } + #[test] + fn client_constructor_applies_both_addressing_styles() { + let path = MediaStorage::new(&storage_config("buzz_dev", "buzz_dev_secret")) + .expect("path-style client"); + assert!(path.bucket.is_path_style()); + assert_eq!(path.bucket.url(), "http://localhost:9000/buzz-media"); + + let mut virtual_config = storage_config("buzz_dev", "buzz_dev_secret"); + virtual_config.s3_addressing_style = S3AddressingStyle::Virtual; + let virtual_hosted = MediaStorage::new(&virtual_config).expect("virtual-hosted client"); + assert!(virtual_hosted.bucket.is_subdomain_style()); + assert_eq!( + virtual_hosted.bucket.url(), + "http://buzz-media.localhost:9000" + ); + } + #[test] fn partial_static_keys_are_rejected() { let err = match MediaStorage::new(&storage_config("buzz_dev", "")) { diff --git a/crates/buzz-media/src/upload.rs b/crates/buzz-media/src/upload.rs index 478ac114ef..524b033280 100644 --- a/crates/buzz-media/src/upload.rs +++ b/crates/buzz-media/src/upload.rs @@ -570,6 +570,7 @@ mod tests { s3_secret_key: String::new(), s3_bucket: String::new(), s3_region: "us-east-1".to_string(), + s3_addressing_style: crate::config::S3AddressingStyle::Path, max_image_bytes: 50 * 1024 * 1024, max_gif_bytes: 10 * 1024 * 1024, max_video_bytes: 524_288_000, diff --git a/crates/buzz-media/src/validation.rs b/crates/buzz-media/src/validation.rs index ee940dfb24..f1387fc9d6 100644 --- a/crates/buzz-media/src/validation.rs +++ b/crates/buzz-media/src/validation.rs @@ -949,6 +949,7 @@ mod tests { s3_secret_key: String::new(), s3_bucket: String::new(), s3_region: "us-east-1".to_string(), + s3_addressing_style: crate::config::S3AddressingStyle::Path, max_image_bytes: 50 * 1024 * 1024, max_gif_bytes: 10 * 1024 * 1024, max_video_bytes: 524_288_000, diff --git a/crates/buzz-media/tests/static_creds_minio.rs b/crates/buzz-media/tests/static_creds_minio.rs index d7591238c2..4c8c10702c 100644 --- a/crates/buzz-media/tests/static_creds_minio.rs +++ b/crates/buzz-media/tests/static_creds_minio.rs @@ -1,5 +1,5 @@ -//! Live round-trip test for the **static-credentials** S3 path against a local -//! MinIO, guarded by `#[ignore]`. +//! Live round-trip test for the **static-credentials** S3 path against an +//! S3-compatible service. It is guarded by `#[ignore]`. //! //! This is the path local/dev and any static-key deployment uses //! (`s3_access_key`/`s3_secret_key` both non-empty -> `Credentials::new`). It @@ -15,7 +15,8 @@ //! ``` //! //! Overridable via `BUZZ_S3_ENDPOINT` / `BUZZ_S3_ACCESS_KEY` / -//! `BUZZ_S3_SECRET_KEY` / `BUZZ_S3_BUCKET`. +//! `BUZZ_S3_SECRET_KEY` / `BUZZ_S3_BUCKET` / `BUZZ_S3_REGION` / +//! `BUZZ_S3_ADDRESSING_STYLE`. The default remains `path` for MinIO. use buzz_media::config::MediaConfig; use buzz_media::storage::MediaStorage; @@ -29,7 +30,11 @@ fn minio_config() -> MediaConfig { s3_secret_key: std::env::var("BUZZ_S3_SECRET_KEY") .unwrap_or_else(|_| "buzz_dev_secret".to_string()), s3_bucket: std::env::var("BUZZ_S3_BUCKET").unwrap_or_else(|_| "buzz-media".to_string()), - s3_region: "us-east-1".to_string(), + s3_region: std::env::var("BUZZ_S3_REGION").unwrap_or_else(|_| "us-east-1".to_string()), + s3_addressing_style: std::env::var("BUZZ_S3_ADDRESSING_STYLE") + .unwrap_or_else(|_| "path".to_string()) + .parse() + .expect("BUZZ_S3_ADDRESSING_STYLE must be path or virtual"), max_image_bytes: 50 * 1024 * 1024, max_gif_bytes: 10 * 1024 * 1024, max_video_bytes: 524_288_000, diff --git a/crates/buzz-pubsub/src/lib.rs b/crates/buzz-pubsub/src/lib.rs index eae8c5ef9e..4f1690beef 100644 --- a/crates/buzz-pubsub/src/lib.rs +++ b/crates/buzz-pubsub/src/lib.rs @@ -328,7 +328,7 @@ impl PubSubManager { publisher::publish_event(&self.pool, ctx, topic, event).await } - /// Set presence with 60s TTL. Call on connect and every 30s heartbeat. + /// Set presence with 180s TTL. Call on connect and every 60s heartbeat. pub async fn set_presence( &self, ctx: &TenantContext, diff --git a/crates/buzz-pubsub/src/presence.rs b/crates/buzz-pubsub/src/presence.rs index 178ba7550a..e0c9dfd6c9 100644 --- a/crates/buzz-pubsub/src/presence.rs +++ b/crates/buzz-pubsub/src/presence.rs @@ -1,7 +1,7 @@ //! Presence tracking — online/away status with TTL. //! -//! Stored as `SET buzz:{community}:presence:{pubkey_hex} "online" EX 90`. -//! TTL is 3x the 30s heartbeat interval so a single missed heartbeat doesn't +//! Stored as `SET buzz:{community}:presence:{pubkey_hex} "online" EX 180`. +//! TTL is 3x the 60s heartbeat interval so a single missed heartbeat doesn't //! cause presence flap. Clean disconnect deletes immediately. use buzz_core::TenantContext; @@ -12,8 +12,8 @@ use std::collections::HashMap; use crate::error::PubSubError; use crate::topic::BUZZ_PREFIX; -/// 3x the 30s heartbeat — single missed heartbeat won't cause presence flap. -pub const PRESENCE_TTL_SECS: u64 = 90; +/// 3x the 60s heartbeat — single missed heartbeat won't cause presence flap. +pub const PRESENCE_TTL_SECS: u64 = 180; /// Returns the Redis key for the presence entry of `pubkey` under `ctx`. pub fn presence_key(ctx: &TenantContext, pubkey: &PublicKey) -> String { @@ -109,6 +109,12 @@ mod tests { TenantContext::resolved(CommunityId::from_uuid(Uuid::from_u128(id)), host) } + #[test] + fn presence_ttl_is_three_one_minute_heartbeat_windows() { + assert_eq!(PRESENCE_TTL_SECS, 180); + assert_eq!(PRESENCE_TTL_SECS, 3 * 60); + } + #[test] fn test_presence_key_format() { let pubkey = make_pubkey(); diff --git a/crates/buzz-relay/Cargo.toml b/crates/buzz-relay/Cargo.toml index 01f78a2d49..41bdc3b9e9 100644 --- a/crates/buzz-relay/Cargo.toml +++ b/crates/buzz-relay/Cargo.toml @@ -84,8 +84,8 @@ async-compression = { version = "0.4.42", features = ["tokio", "gzip"] } dev = ["buzz-auth/dev"] [dev-dependencies] -mesh-llm-sdk = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.73.1", package = "mesh-llm-sdk", default-features = false, features = ["client", "serving"] } -mesh-llm-host-runtime = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.73.1", package = "mesh-llm-host-runtime", default-features = false, features = ["dynamic-native-runtime"] } +mesh-llm-sdk = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-sdk", default-features = false, features = ["client", "serving"] } +mesh-llm-host-runtime = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-host-runtime", default-features = false, features = ["dynamic-native-runtime"] } buzz-core = { workspace = true, features = ["test-utils"] } buzz-auth = { workspace = true, features = ["dev"] } reqwest = { workspace = true } diff --git a/crates/buzz-relay/examples/mesh_agent_e2e.rs b/crates/buzz-relay/examples/mesh_agent_e2e.rs index b6f723f35a..345ca4c746 100644 --- a/crates/buzz-relay/examples/mesh_agent_e2e.rs +++ b/crates/buzz-relay/examples/mesh_agent_e2e.rs @@ -278,7 +278,9 @@ async fn agent_chat_in_isolated_home( .env("OPENAI_COMPAT_API_KEY", "buzz-mesh-local") .env("OPENAI_COMPAT_API", "chat") .env("BUZZ_AGENT_MAX_OUTPUT_TOKENS", "4096") - .env("BUZZ_AGENT_THINKING_EFFORT", "none") + // No BUZZ_AGENT_THINKING_EFFORT: apply_relay_mesh_env() deliberately + // leaves it unset so each model's chat template picks its own default. + // Pinning a value here would test a config the product does not ship. .stdin(Stdio::piped()) .stdout(Stdio::piped()) .stderr(Stdio::null()); diff --git a/crates/buzz-relay/src/api/bridge.rs b/crates/buzz-relay/src/api/bridge.rs index 2e00d6bd2f..a118ff453f 100644 --- a/crates/buzz-relay/src/api/bridge.rs +++ b/crates/buzz-relay/src/api/bridge.rs @@ -462,9 +462,9 @@ async fn handle_channel_window_filter( .as_ref() .map(|ks| ks.iter().map(|k| k.as_u16() as u32).collect()); - let window = state + let (window, mut session) = state .db - .get_channel_window( + .get_channel_window_with_session( tenant.community(), ch_id, limit, @@ -485,7 +485,12 @@ async fn handle_channel_window_filter( // 2. Aux closure: reactions/deletions/edits targeting retained rows, plus // deletions targeting those aux events (the transitive second hop). - // One round trip for the client instead of an #e fan-out. + // One round trip for the client instead of an #e fan-out. Runs in the + // SAME request transaction that served the window: when the page came + // from a proved replica session, the heartbeat observation anchored a + // REPEATABLE READ snapshot, so the aux hops see exactly the state the + // proof covered — another pooled session (or even another autocommit + // statement) could sit at a different replay position. if extension_flag(raw, "include_aux") && !row_ids_hex.is_empty() { let mut seen_aux: std::collections::HashSet = std::collections::HashSet::new(); @@ -495,8 +500,7 @@ async fn handle_channel_window_filter( aux_query.kinds = Some(hop_kinds.iter().map(|k| *k as i32).collect()); aux_query.e_tags = Some(std::mem::take(&mut hop_ids)); aux_query.limit = Some(1000); - let aux_events = state - .db + let aux_events = session .query_events(&aux_query) .await .map_err(|e| internal_error(&format!("window aux error: {e}")))?; @@ -1083,7 +1087,8 @@ async fn query_events_authed( let type_events = match canonical { "mentions" => state .db - .query_feed_mentions( + .query_feed_mentions_routed( + "bridge_feed", tenant.community(), &pubkey_bytes, &accessible_channels, @@ -1094,7 +1099,8 @@ async fn query_events_authed( .map_err(|e| internal_error(&format!("feed mentions error: {e}")))?, "needs_action" => state .db - .query_feed_needs_action( + .query_feed_needs_action_routed( + "bridge_feed", tenant.community(), &pubkey_bytes, &accessible_channels, @@ -1105,7 +1111,13 @@ async fn query_events_authed( .map_err(|e| internal_error(&format!("feed needs_action error: {e}")))?, "activity" => state .db - .query_feed_activity(tenant.community(), &accessible_channels, since, remaining) + .query_feed_activity_routed( + "bridge_feed", + tenant.community(), + &accessible_channels, + since, + remaining, + ) .await .map_err(|e| internal_error(&format!("feed activity error: {e}")))?, _ => continue, @@ -1224,10 +1236,10 @@ async fn query_events_authed( extract_channel_from_filter(filter), &accessible_channels, ); - // Persona visibility pushdown: must mirror WS REQ so that a page of newer - // private personas does not starve older shared ones off the candidate page. - if crate::handlers::req::filter_can_match_persona_shared_kinds(filter) { - query.persona_reader = Some(pubkey_bytes.clone()); + // Shared-gated visibility pushdown: must mirror WS REQ so that a page of + // newer private events does not starve older shared ones off the page. + if crate::handlers::req::filter_can_match_shared_gated_kinds(filter) { + query.shared_gated_reader = Some(pubkey_bytes.clone()); } match extract_before_id(raw) { @@ -1271,7 +1283,7 @@ async fn query_events_authed( let db = state.db.clone(); let mut catchall_results = stream::iter(catchall_queries.into_iter().map(|(idx, query)| { let db = db.clone(); - async move { (idx, db.query_events(&query).await) } + async move { (idx, db.query_events_routed("bridge_query", &query).await) } })) .buffered(crate::handlers::req::FILTER_QUERY_CONCURRENCY); @@ -1441,11 +1453,11 @@ async fn count_events_authed( filter, &authed_pubkey_hex, ); - // Force per-event fallback for filters that can match kind:30175 — - // the fast SQL count_events() path has no per-event gate and would - // over-count foreign unshared persona events (existence leak). - let needs_persona_filtering = - crate::handlers::req::filter_can_match_persona_shared_kinds(filter); + // Force per-event fallback for filters that can match a shared-gated + // kind — the fast SQL count_events() path has no per-event gate and + // would over-count foreign unshared events (existence leak). + let needs_shared_gate_filtering = + crate::handlers::req::filter_can_match_shared_gated_kinds(filter); // If filter targets a specific channel, verify access. if let Some(ch_id) = extract_channel_from_filter(filter) { @@ -1460,10 +1472,10 @@ async fn count_events_authed( tenant.community(), ) .await; - // Persona visibility pushdown: same as REQ and /query paths, so the - // fallback's query_events call doesn't over-fetch private persona rows. - if needs_persona_filtering { - query.persona_reader = Some(pubkey_bytes.clone()); + // Shared-gated visibility pushdown: same as REQ and /query paths, so + // the fallback's query_events call doesn't over-fetch private rows. + if needs_shared_gate_filtering { + query.shared_gated_reader = Some(pubkey_bytes.clone()); } let author_is_self = filter.authors.as_ref().is_some_and(|authors| { !authors.is_empty() @@ -1474,9 +1486,9 @@ async fn count_events_authed( if crate::handlers::req::filter_fully_pushable(filter) && (!needs_author_only_filtering || author_is_self) && !needs_result_gated_filtering - && !needs_persona_filtering + && !needs_shared_gate_filtering { - match state.db.count_events(&query).await { + match state.db.count_events_routed("bridge_count", &query).await { Ok(n) => total += n as u64, Err(e) => { return Err(internal_error(&format!("count error: {e}"))); @@ -1486,7 +1498,11 @@ async fn count_events_authed( // Fallback: query + post-filter for non-pushable constraints. let mut q = query; crate::handlers::req::apply_count_fallback_limit(&mut q); - match state.db.query_events(&q).await { + match state + .db + .query_events_routed_bounded("bridge_count_fallback", &q) + .await + { Ok(stored_events) => { if crate::handlers::req::count_fallback_exceeded(stored_events.len()) { metrics::counter!("buzz_count_fallback_rejections_total").increment(1); @@ -1525,10 +1541,10 @@ async fn count_events_authed( ) .await; query.channel_ids = Some(accessible_channels.to_vec()); - // Persona visibility pushdown: pre-filter before ORDER/LIMIT on the - // fallback query_events path. - if needs_persona_filtering { - query.persona_reader = Some(pubkey_bytes.clone()); + // Shared-gated visibility pushdown: pre-filter before ORDER/LIMIT on + // the fallback query_events path. + if needs_shared_gate_filtering { + query.shared_gated_reader = Some(pubkey_bytes.clone()); } let author_is_self = filter.authors.as_ref().is_some_and(|authors| { @@ -1540,10 +1556,10 @@ async fn count_events_authed( if crate::handlers::req::filter_fully_pushable(filter) && (!needs_author_only_filtering || author_is_self) && !needs_result_gated_filtering - && !needs_persona_filtering + && !needs_shared_gate_filtering { query.limit = None; - match state.db.count_events(&query).await { + match state.db.count_events_routed("bridge_count", &query).await { Ok(n) => total += n as u64, Err(e) => { return Err(internal_error(&format!("count error: {e}"))); @@ -1552,7 +1568,11 @@ async fn count_events_authed( } else { // Fallback: query a bounded candidate set + post-filter. crate::handlers::req::apply_count_fallback_limit(&mut query); - match state.db.query_events(&query).await { + match state + .db + .query_events_routed_bounded("bridge_count_fallback", &query) + .await + { Ok(stored_events) => { if crate::handlers::req::count_fallback_exceeded(stored_events.len()) { metrics::counter!("buzz_count_fallback_rejections_total").increment(1); @@ -1725,7 +1745,7 @@ async fn handle_bridge_search( let id_refs: Vec<&[u8]> = hit_ids.iter().map(|b| b.as_slice()).collect(); let stored_events = state .db - .get_events_by_ids(tenant.community(), &id_refs) + .get_events_by_ids_routed("bridge_search_hydrate", tenant.community(), &id_refs) .await .map_err(|e| internal_error(&format!("search fetch error: {e}")))?; @@ -3022,6 +3042,27 @@ mod tests { assert_eq!(extract_page_offset(&raw, None), None); } + /// Offsets are sized from the *clamped* limit the DB will honor, not from + /// what the client asked for. `filter_to_query_params` clamps an absent or + /// over-ceiling `limit` to `DEFAULT_MAX_PAGE_LIMIT` (guarded in + /// `handlers::req::tests::req_filter_limit_clamps_to_advertised_nip11_max_limit`) + /// and that clamped value is what arrives here — so page N starts exactly + /// N-1 full pages in. Sizing from an unclamped limit would step past rows + /// the previous page never returned. + #[test] + fn extract_page_offset_sizes_pages_from_clamped_limit() { + let clamped = buzz_db::DEFAULT_MAX_PAGE_LIMIT; + + assert_eq!( + extract_page_offset(&serde_json::json!({ "page": 2 }), Some(clamped)), + Some(clamped) + ); + assert_eq!( + extract_page_offset(&serde_json::json!({ "page": 3 }), Some(clamped)), + Some(clamped * 2) + ); + } + #[test] fn extract_depth_limit_valid() { let raw = serde_json::json!({ "depth_limit": 3 }); diff --git a/crates/buzz-relay/src/api/git/binding.rs b/crates/buzz-relay/src/api/git/binding.rs new file mode 100644 index 0000000000..7ee0eccb23 --- /dev/null +++ b/crates/buzz-relay/src/api/git/binding.rs @@ -0,0 +1,128 @@ +//! Repo → channel binding resolution, shared by the read gate and push policy. +//! +//! The `buzz-channel` tag on a kind:30617 announcement IS the git ACL: the +//! read gate (SEC-005, `transport::authorize_git_read`) and the push policy +//! endpoint (`policy::hook_callback`) both authorize against membership in +//! the bound channel. Before this module they each parsed the tag with their +//! own code that agreed only by coincidence; the resolver makes the +//! agreement structural. +//! +//! # First-tag, fail-closed semantics +//! +//! Only the *first* `buzz-channel` tag is considered, and it must carry a +//! valid UUID. A malformed first binding resolves to [`RepoBinding::Broken`] +//! even if a later duplicate tag is valid — an ambiguous announcement must +//! fail closed, not silently resolve to whichever duplicate happens to +//! parse. If this ever became "find the first *parseable* tag", an author +//! who can append a second `buzz-channel` tag would pick the channel. +//! +//! # What this deliberately does NOT do +//! +//! No DB access. A well-formed UUID that names a nonexistent or deleted +//! channel still resolves to [`RepoBinding::Bound`]; each gate's own +//! membership lookup then denies (`get_member_role` joins +//! `channels … deleted_at IS NULL`, so a dead channel is indistinguishable +//! from a non-member — the info-leak-safe posture). Likewise each gate keeps +//! its own archived-channel policy: push denies on archived channels, read +//! does not, and this resolver must not unify that asymmetry as a side +//! effect. + +use uuid::Uuid; + +/// How a kind:30617 announcement binds (or fails to bind) a channel. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RepoBinding { + /// No `buzz-channel` tag at all. The announcement author (the only + /// identity that can rebind — 30617 is keyed by `(author, d)`) may be + /// offered remediation; everyone else gets the generic denial. + NotBound, + /// First `buzz-channel` tag carries a valid UUID. + Bound(Uuid), + /// First `buzz-channel` tag exists but its value is not a UUID. + /// Fail closed with the generic denial — never remediation, which + /// would leak that the repo exists. + Broken, +} + +/// Resolve the channel binding of a kind:30617 announcement from its tags. +pub fn resolve_repo_binding(event: &nostr::Event) -> RepoBinding { + let Some(first) = event + .tags + .iter() + .find(|t| t.as_slice().first().map(String::as_str) == Some("buzz-channel")) + else { + return RepoBinding::NotBound; + }; + match first.as_slice().get(1).map(|v| Uuid::parse_str(v)) { + Some(Ok(id)) => RepoBinding::Bound(id), + _ => RepoBinding::Broken, + } +} + +#[cfg(test)] +mod tests { + use nostr::{EventBuilder, Keys, Kind, Tag}; + + use super::{resolve_repo_binding, RepoBinding}; + + fn announcement(tags: Vec) -> nostr::Event { + EventBuilder::new(Kind::Custom(30617), "") + .tags(tags) + .sign_with_keys(&Keys::generate()) + .expect("sign 30617") + } + + #[test] + fn extracts_valid_uuid() { + let ch = uuid::Uuid::new_v4(); + let event = announcement(vec![ + Tag::parse(["d", "repo"]).unwrap(), + Tag::parse(["buzz-channel", &ch.to_string()]).unwrap(), + ]); + assert_eq!(resolve_repo_binding(&event), RepoBinding::Bound(ch)); + } + + #[test] + fn absent_tag_is_not_bound() { + let event = announcement(vec![Tag::parse(["d", "repo"]).unwrap()]); + assert_eq!(resolve_repo_binding(&event), RepoBinding::NotBound); + } + + #[test] + fn malformed_and_empty_values_are_broken_not_absent() { + let malformed = announcement(vec![ + Tag::parse(["d", "repo"]).unwrap(), + Tag::parse(["buzz-channel", "not-a-uuid"]).unwrap(), + ]); + assert_eq!(resolve_repo_binding(&malformed), RepoBinding::Broken); + + let empty = announcement(vec![ + Tag::parse(["d", "repo"]).unwrap(), + Tag::parse(["buzz-channel"]).unwrap(), + ]); + assert_eq!(resolve_repo_binding(&empty), RepoBinding::Broken); + } + + #[test] + fn fails_closed_on_ambiguous_duplicate_bindings() { + let ch = uuid::Uuid::new_v4(); + let other = uuid::Uuid::new_v4(); + + // Malformed first + valid second: the ambiguity denies; the valid + // duplicate must NOT win, or the duplicate picks the channel. + let malformed_first = announcement(vec![ + Tag::parse(["d", "repo"]).unwrap(), + Tag::parse(["buzz-channel", "not-a-uuid"]).unwrap(), + Tag::parse(["buzz-channel", &ch.to_string()]).unwrap(), + ]); + assert_eq!(resolve_repo_binding(&malformed_first), RepoBinding::Broken); + + // Valid first + different second: first wins deterministically. + let valid_first = announcement(vec![ + Tag::parse(["d", "repo"]).unwrap(), + Tag::parse(["buzz-channel", &ch.to_string()]).unwrap(), + Tag::parse(["buzz-channel", &other.to_string()]).unwrap(), + ]); + assert_eq!(resolve_repo_binding(&valid_first), RepoBinding::Bound(ch)); + } +} diff --git a/crates/buzz-relay/src/api/git/cas_publish.rs b/crates/buzz-relay/src/api/git/cas_publish.rs index 635dcf2a67..50bb36d818 100644 --- a/crates/buzz-relay/src/api/git/cas_publish.rs +++ b/crates/buzz-relay/src/api/git/cas_publish.rs @@ -1370,6 +1370,15 @@ mod tests { ); } + #[test] + fn published_head_moves_to_surviving_branch_after_current_branch_deletion() { + let refs = BTreeMap::from([("refs/heads/master".to_string(), "1".repeat(40))]); + assert_eq!( + resolve_published_head(&refs, "refs/heads/main".to_string(), "refs/heads/main"), + "refs/heads/master" + ); + } + #[test] fn digest_from_key_strips_prefix() { let k = format!("manifests/{}", "a".repeat(64)); @@ -1583,22 +1592,15 @@ mod tests { } fn live_store() -> GitStore { - let endpoint = std::env::var("BUZZ_GIT_S3_ENDPOINT") - .or_else(|_| std::env::var("BUZZ_S3_ENDPOINT")) - .unwrap_or_else(|_| "http://localhost:9000".into()); - let access_key = std::env::var("BUZZ_GIT_S3_ACCESS_KEY") - .or_else(|_| std::env::var("BUZZ_S3_ACCESS_KEY")) - .unwrap_or_else(|_| "buzz_dev".into()); - let secret_key = std::env::var("BUZZ_GIT_S3_SECRET_KEY") - .or_else(|_| std::env::var("BUZZ_S3_SECRET_KEY")) - .unwrap_or_else(|_| "buzz_dev_secret".into()); - let bucket = std::env::var("BUZZ_GIT_S3_BUCKET") - .or_else(|_| std::env::var("BUZZ_S3_BUCKET")) - .unwrap_or_else(|_| "buzz-media".into()); - let region = std::env::var("BUZZ_GIT_S3_REGION") - .or_else(|_| std::env::var("BUZZ_S3_REGION")) - .unwrap_or_else(|_| "us-east-1".into()); - GitStore::new(&endpoint, &access_key, &secret_key, &bucket, ®ion).expect("connect minio") + GitStore::new( + "http://localhost:9000", + "buzz_dev", + "buzz_dev_secret", + "buzz-media", + "us-east-1", + buzz_media::config::S3AddressingStyle::Path, + ) + .expect("connect local MinIO") } fn tenant() -> TenantContext { diff --git a/crates/buzz-relay/src/api/git/hydrate.rs b/crates/buzz-relay/src/api/git/hydrate.rs index 064d01923e..3ce809d18f 100644 --- a/crates/buzz-relay/src/api/git/hydrate.rs +++ b/crates/buzz-relay/src/api/git/hydrate.rs @@ -543,8 +543,15 @@ mod tests { #[tokio::test] async fn materialized_repo_is_created_under_configured_scratch_dir() { let scratch = TempDir::new().unwrap(); - let store = GitStore::new("http://localhost:9000", "x", "x", "x", "us-east-1") - .expect("construct store"); + let store = GitStore::new( + "http://localhost:9000", + "x", + "x", + "x", + "us-east-1", + buzz_media::config::S3AddressingStyle::Path, + ) + .expect("construct store"); let manifest = Manifest { version: 1, head: "refs/heads/main".into(), @@ -587,8 +594,9 @@ mod tests { "buzz_dev_secret", "buzz-git", "us-east-1", + buzz_media::config::S3AddressingStyle::Path, ) - .expect("connect minio") + .expect("connect local MinIO") } /// Build a tiny on-disk repo, return (pack bytes, head_oid). diff --git a/crates/buzz-relay/src/api/git/mod.rs b/crates/buzz-relay/src/api/git/mod.rs index ab0510fbeb..dd69d7dc36 100644 --- a/crates/buzz-relay/src/api/git/mod.rs +++ b/crates/buzz-relay/src/api/git/mod.rs @@ -22,6 +22,7 @@ use tower_http::limit::RequestBodyLimitLayer; use crate::state::AppState; +pub mod binding; pub mod cas_publish; pub mod hook; pub mod hydrate; diff --git a/crates/buzz-relay/src/api/git/policy.rs b/crates/buzz-relay/src/api/git/policy.rs index fd6c4fb688..32d63f4600 100644 --- a/crates/buzz-relay/src/api/git/policy.rs +++ b/crates/buzz-relay/src/api/git/policy.rs @@ -42,7 +42,10 @@ use tracing::{error, warn}; use uuid::Uuid; use buzz_core::channel::MemberRole; -use buzz_core::git_perms::{evaluate_push, parse_protection_tags, Denial, RefUpdate, UpdateKind}; +use buzz_core::git_perms::{ + evaluate_push, parse_protection_tags, Denial, RefUpdate, UpdateKind, + GIT_NO_CHANNEL_BINDING_BODY, +}; use buzz_db::EventQuery; use crate::state::AppState; @@ -297,12 +300,29 @@ pub async fn hook_policy_check( } }; - // 6. Resolve channel and check archived state (applies to ALL pushers including owner). - let channel_id = tags - .iter() - .find(|t| t.first().map(|s| s.as_str()) == Some("buzz-channel")) - .and_then(|t| t.get(1)) - .and_then(|id| Uuid::parse_str(id).ok()); + // 6. Resolve channel binding via the shared resolver (same first-tag, + // fail-closed semantics as the read gate) and check archived state + // (applies to ALL pushers including owner). + // + // `Broken` denies HERE, before owner resolution: a malformed or + // ambiguous first binding fails closed for *everyone*, exactly like the + // read gate. Letting it fall through as "unbound" would hand the owner + // short-circuit below a push path through a binding the read gate + // refuses to honor — the tri-state exists precisely so Broken and + // NotBound cannot collapse. Only genuinely-NotBound repos proceed, and + // only they may earn the remediation-token denial. + let channel_id = match crate::api::git::binding::resolve_repo_binding(&repo_event.event) { + crate::api::git::binding::RepoBinding::Bound(id) => Some(id), + crate::api::git::binding::RepoBinding::NotBound => None, + crate::api::git::binding::RepoBinding::Broken => { + warn!(repo = %req.repo_id, "hook callback: broken buzz-channel binding"); + // Deliberately NOT the no_channel_binding token body: the + // remediation contract is NotBound-only. A broken binding is + // ambiguity, and ambiguity gets a generic denial (matching the + // read gate's posture for the same announcement). + return (StatusCode::FORBIDDEN, "invalid channel binding").into_response(); + } + }; if let Some(ch_id) = channel_id { match state.db.get_channel(community, ch_id).await { @@ -350,7 +370,10 @@ pub async fn hook_policy_check( match channel_id { None => { warn!(repo = %req.repo_id, "hook callback: no buzz-channel binding"); - return (StatusCode::FORBIDDEN, "no channel binding").into_response(); + // Declared cross-component contract — see the const docs in + // buzz-core::git_perms for who consumes the token and why + // the body also repeats the legacy phrase. + return (StatusCode::FORBIDDEN, GIT_NO_CHANNEL_BINDING_BODY).into_response(); } Some(ch_id) => { match state @@ -482,6 +505,30 @@ mod tests { assert!(!verify_hmac(b"wrong-secret", &req)); } + /// Deploy-skew guard for the unbound-repo deny body. The token + /// (`no_channel_binding`, underscores) and the legacy phrase + /// (`no channel binding`, spaces) do NOT contain each other, so the body + /// must carry both: the token for structured consumers (Desktop's merge + /// classifier and dialog matcher), the phrase for desktops already in + /// the field that prose-match it. Relay ships continuously and Desktop + /// on release cadence — dropping the phrase strands every old desktop + /// on a new relay. Asserted against the shared consts, not re-typed + /// literals, so the const and this test cannot drift apart separately. + #[test] + fn no_channel_binding_body_satisfies_old_and_new_matchers() { + assert!( + GIT_NO_CHANNEL_BINDING_BODY.starts_with(&format!( + "{}: ", + buzz_core::git_perms::GIT_NO_CHANNEL_BINDING_TOKEN + )), + "new structured consumers match the token prefix" + ); + assert!( + GIT_NO_CHANNEL_BINDING_BODY.contains("no channel binding"), + "shipped desktops prose-match this exact phrase (spaces, not underscores)" + ); + } + #[test] fn hmac_tampered_repo_id_rejected() { let secret = b"test-secret"; @@ -772,4 +819,169 @@ printf '%s' "$HMAC_INPUT" | openssl dgst -sha256 -hmac "{secret}" -hex 2>/dev/nu "Single-ref HMAC mismatch!\n Rust: {rust_sig}\n Bash: {bash_sig}" ); } + + // ── hook_policy_check binding gate (requires Postgres) ────────────── + + const TEST_DB_URL: &str = "postgres://buzz:buzz_dev@localhost:5432/buzz"; // sadscan:disable np.postgres.1 + + async fn policy_test_state() -> Arc { + let mut config = crate::config::Config::from_env().expect("default config loads"); + config.require_relay_membership = false; + config.redis_url = "redis://127.0.0.1:1".to_string(); + config.database_url = std::env::var("BUZZ_TEST_DATABASE_URL") + .or_else(|_| std::env::var("DATABASE_URL")) + .unwrap_or_else(|_| TEST_DB_URL.to_string()); + let pool = sqlx::PgPool::connect(&config.database_url) + .await + .expect("connect test DB"); + let db = buzz_db::Db::from_pool(pool.clone()); + let redis_pool = deadpool_redis::Config::from_url(&config.redis_url) + .create_pool(Some(deadpool_redis::Runtime::Tokio1)) + .expect("redis pool"); + let pubsub = Arc::new( + buzz_pubsub::PubSubManager::new(&config.redis_url, redis_pool.clone()) + .await + .expect("pubsub manager"), + ); + let audit = buzz_audit::AuditService::new(pool.clone()); + let auth = buzz_auth::AuthService::new(config.auth.clone()); + let search = buzz_search::SearchService::new(pool.clone()); + let workflow_engine = Arc::new(buzz_workflow::WorkflowEngine::new( + db.clone(), + buzz_workflow::WorkflowConfig::default(), + )); + let media_storage = buzz_media::MediaStorage::new(&config.media).expect("media storage"); + let (state, _audit_shutdown) = AppState::new( + config, + db, + redis_pool, + audit, + pubsub, + auth, + search, + workflow_engine, + nostr::Keys::generate(), + media_storage, + ); + Arc::new(state) + } + + /// Announce `repo_id` with the given tags, then push to it as its own + /// announcement author and return the response. + async fn owner_push_response( + state: &Arc, + community: buzz_core::CommunityId, + keys: &nostr::Keys, + repo_id: &str, + binding_tags: Vec, + ) -> axum::response::Response { + use nostr::{EventBuilder, Kind, Tag}; + + let mut tags = vec![Tag::parse(["d", repo_id]).unwrap()]; + tags.extend(binding_tags); + let event = EventBuilder::new(Kind::Custom(30617), "") + .tags(tags) + .sign_with_keys(keys) + .expect("sign 30617"); + state + .db + .insert_event(community, &event, None) + .await + .expect("insert 30617"); + + let owner_hex = keys.public_key().to_hex(); + let mut req = HookCallbackRequest { + repo_id: repo_id.to_string(), + repo_owner: owner_hex.clone(), + community_id: community.as_uuid().to_string(), + pusher_pubkey: owner_hex, + ref_updates: vec![HookRefUpdate { + old_oid: "0".repeat(40), + new_oid: "2".repeat(40), + ref_name: "refs/heads/main".to_string(), + is_ancestor: false, + }], + timestamp: SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_secs(), + signature: String::new(), + }; + let secret = state.config.git_hook_hmac_secret.clone(); + sign_request(&mut req, secret.as_bytes()); + hook_policy_check(State(Arc::clone(state)), Json(req)).await + } + + async fn body_string(response: axum::response::Response) -> (StatusCode, String) { + let status = response.status(); + let bytes = axum::body::to_bytes(response.into_body(), usize::MAX) + .await + .expect("read body"); + (status, String::from_utf8(bytes.to_vec()).expect("utf-8")) + } + + /// The tri-state trap the resolver exists to prevent: a broken (malformed + /// or ambiguous-first) binding must fail closed for EVERYONE on push — + /// including the announcement author — *before* the owner short-circuit + /// grants `MemberRole::Owner`. Collapsing `Broken` into "unbound" hands + /// the owner a push path through a binding the read gate refuses to + /// honor. The remediation token stays reserved for genuinely NotBound. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn push_gate_denies_owner_through_broken_binding() { + use nostr::{Keys, Tag}; + + let state = policy_test_state().await; + let host = format!("policy-{}.example", uuid::Uuid::new_v4().simple()); + let community = state + .db + .ensure_configured_community(&host) + .await + .expect("community") + .id; + let keys = Keys::generate(); + + // Malformed first + valid-looking second: the ambiguity must deny, + // and the parseable duplicate must not rescue the push. + let response = owner_push_response( + &state, + community, + &keys, + &format!("repo-{}", uuid::Uuid::new_v4().simple()), + vec![ + Tag::parse(["buzz-channel", "not-a-uuid"]).unwrap(), + Tag::parse(["buzz-channel", &uuid::Uuid::new_v4().to_string()]).unwrap(), + ], + ) + .await; + let (status, body) = body_string(response).await; + assert_eq!(status, StatusCode::FORBIDDEN); + assert_eq!( + body, "invalid channel binding", + "owner pushing through a broken binding must be denied generically" + ); + assert!( + !body.contains(buzz_core::git_perms::GIT_NO_CHANNEL_BINDING_TOKEN), + "remediation token is NotBound-only; Broken must never earn it" + ); + + // Control: the same owner pushing a genuinely NEVER-BOUND repo is + // allowed (owner authority over an unbound announcement is the + // long-standing push semantics). This pins the denial above to + // Broken specifically, not to some broader regression. + let response = owner_push_response( + &state, + community, + &keys, + &format!("repo-{}", uuid::Uuid::new_v4().simple()), + vec![], + ) + .await; + let (status, body) = body_string(response).await; + assert_eq!( + status, + StatusCode::OK, + "owner push to a never-bound repo must remain allowed (got body: {body})" + ); + } } diff --git a/crates/buzz-relay/src/api/git/store.rs b/crates/buzz-relay/src/api/git/store.rs index 43d210e648..bdfca8dcf2 100644 --- a/crates/buzz-relay/src/api/git/store.rs +++ b/crates/buzz-relay/src/api/git/store.rs @@ -174,7 +174,9 @@ pub struct GitStore { impl GitStore { /// Build a client against an S3-compatible endpoint (e.g. MinIO). /// - /// Uses path-style addressing for MinIO compatibility; AWS S3 accepts both. + /// `addressing_style` is shared with media storage so both paths sign and + /// route requests consistently. Path style supports the bundled MinIO DNS; + /// virtual-hosted style supports standard S3 and providers such as Railway. /// /// Credential selection mirrors [`buzz_media::MediaStorage::new`]: /// - both `access_key` and `secret_key` non-empty → static credentials @@ -190,6 +192,7 @@ impl GitStore { secret_key: &str, bucket_name: &str, region: &str, + addressing_style: buzz_media::config::S3AddressingStyle, ) -> Result { let region = Region::Custom { region: region.into(), @@ -209,9 +212,11 @@ impl GitStore { } } .map_err(|e| StoreError::Backend(S3Error::Credentials(e)))?; - let bucket = Bucket::new(bucket_name, region, creds) - .map_err(StoreError::Backend)? - .with_path_style(); + let bucket = Bucket::new(bucket_name, region, creds).map_err(StoreError::Backend)?; + let bucket = match addressing_style { + buzz_media::config::S3AddressingStyle::Path => bucket.with_path_style(), + buzz_media::config::S3AddressingStyle::Virtual => bucket, + }; Ok(Self { bucket: Arc::from(bucket), }) @@ -950,6 +955,7 @@ mod tests { "buzz_dev_secret", "buzz-git", "us-west-2", + buzz_media::config::S3AddressingStyle::Path, ) .expect("static creds should build a git store"); match store.bucket.region { @@ -958,6 +964,34 @@ mod tests { } } + #[test] + fn constructor_applies_both_addressing_styles() { + for (style, expected_url, path_style) in [ + ( + buzz_media::config::S3AddressingStyle::Path, + "https://storage.example/buzz-git", + true, + ), + ( + buzz_media::config::S3AddressingStyle::Virtual, + "https://buzz-git.storage.example", + false, + ), + ] { + let store = GitStore::new( + "https://storage.example", + "buzz_dev", + "buzz_dev_secret", + "buzz-git", + "us-east-1", + style, + ) + .expect("construct git store"); + assert_eq!(store.bucket.url(), expected_url); + assert_eq!(store.bucket.is_path_style(), path_style); + } + } + #[test] fn partial_static_keys_are_rejected() { for (access, secret) in [("buzz_dev", ""), ("", "buzz_dev_secret")] { @@ -967,6 +1001,7 @@ mod tests { secret, "buzz-git", "us-east-1", + buzz_media::config::S3AddressingStyle::Path, ) { Ok(_) => { panic!("partial static creds must not silently use the credential chain") @@ -998,14 +1033,29 @@ mod probe { } fn store() -> GitStore { + // This is the dedicated backend conformance path, so all connection and + // signing inputs are overridable for a real provider such as Railway. + // The hydrate/CAS live tests use explicit local MinIO fixtures instead. + let endpoint = + std::env::var("BUZZ_S3_ENDPOINT").unwrap_or_else(|_| "http://localhost:9000".into()); + let access_key = std::env::var("BUZZ_S3_ACCESS_KEY").unwrap_or_else(|_| "buzz_dev".into()); + let secret_key = + std::env::var("BUZZ_S3_SECRET_KEY").unwrap_or_else(|_| "buzz_dev_secret".into()); + let bucket = std::env::var("BUZZ_S3_BUCKET").unwrap_or_else(|_| "buzz-git".into()); + let region = std::env::var("BUZZ_S3_REGION").unwrap_or_else(|_| "us-east-1".into()); + let addressing_style = std::env::var("BUZZ_S3_ADDRESSING_STYLE") + .unwrap_or_else(|_| "path".into()) + .parse() + .expect("BUZZ_S3_ADDRESSING_STYLE must be path or virtual"); GitStore::new( - "http://localhost:9000", - "buzz_dev", - "buzz_dev_secret", - "buzz-git", - "us-east-1", + &endpoint, + &access_key, + &secret_key, + &bucket, + ®ion, + addressing_style, ) - .expect("connect minio") + .expect("connect S3-compatible storage") } fn sha256_hex(b: &[u8]) -> String { diff --git a/crates/buzz-relay/src/api/git/transport.rs b/crates/buzz-relay/src/api/git/transport.rs index df5bdd4c3e..d3118d8a76 100644 --- a/crates/buzz-relay/src/api/git/transport.rs +++ b/crates/buzz-relay/src/api/git/transport.rs @@ -28,6 +28,7 @@ use tokio::process::Command; use tower_http::limit::RequestBodyLimitLayer; use tracing::{error, info, warn}; +use super::binding::{resolve_repo_binding, RepoBinding}; use super::cas_publish::{cas_publish, CasError, ParentState, PublishLimits}; use super::hook::install_hook; use super::hydrate::{ @@ -377,7 +378,15 @@ fn hydrate_error_to_response(owner: &str, repo: &str, err: HydrateError) -> Resp /// error all deny. There is deliberately **no repo-owner bypass**: an owner /// removed from the bound channel loses read access, which is the exact /// exploit shape this gate closes. Every denial is the same generic 404 as a -/// nonexistent repo so membership cannot be probed through the git endpoints. +/// nonexistent repo so membership cannot be probed through the git endpoints +/// — with exactly one carve-out: a **never-bound** repo read by its own +/// **announcement author** returns a 404 whose body tells the author how to +/// bind it (issue #3527: a vanilla NIP-34 client can announce without a +/// `buzz-channel` tag, and the repo then 404s forever with no explanation +/// for anyone). The author already knows the repo exists — they announced it +/// — so the remediation body leaks nothing, and only the author can rebind +/// (kind:30617 is keyed by `(author, d)`). A *broken* binding stays generic +/// even for the author: ambiguity fails closed. async fn authorize_git_read( db: &buzz_db::Db, community: buzz_core::CommunityId, @@ -415,9 +424,32 @@ async fn authorize_git_read( } }; - let Some(channel_id) = repo_bound_channel_id(&repo_event.event) else { - warn!(repo = %repo_name, "git read gate: missing/malformed buzz-channel binding (deny)"); - return Err(denied()); + let channel_id = match resolve_repo_binding(&repo_event.event) { + RepoBinding::Bound(id) => id, + RepoBinding::NotBound => { + // Remediation carve-out: author of a never-bound announcement. + // Status stays 404 — byte-identical to every other denial at the + // status level — so denial *class* is still unprobeable; only + // the body differs, and only for the one identity that already + // knows the repo exists. The body is a single verb-first line: + // Desktop error paths that keep one line keep the instruction. + if repo_event.event.pubkey == *caller { + warn!(repo = %repo_name, "git read gate: unbound repo read by its author (deny with remediation)"); + return Err(( + StatusCode::NOT_FOUND, + format!( + "run: buzz repos bind --id {repo_name} --channel — repository {repo_name:?} has no channel binding, so the relay cannot authorize access" + ), + ) + .into_response()); + } + warn!(repo = %repo_name, "git read gate: missing buzz-channel binding (deny)"); + return Err(denied()); + } + RepoBinding::Broken => { + warn!(repo = %repo_name, "git read gate: malformed buzz-channel binding (deny)"); + return Err(denied()); + } }; match db @@ -433,24 +465,6 @@ async fn authorize_git_read( } } -/// Extract the `buzz-channel` UUID from a kind:30617 announcement. -/// -/// First-tag semantics, matching the push policy endpoint: only the *first* -/// `buzz-channel` tag is considered, and it must carry a valid UUID. A -/// malformed first binding denies even if a later duplicate tag is valid — -/// an ambiguous announcement must fail closed, not silently resolve to -/// whichever duplicate happens to parse. -fn repo_bound_channel_id(event: &nostr::Event) -> Option { - let first = event - .tags - .iter() - .find(|t| t.as_slice().first().map(String::as_str) == Some("buzz-channel"))?; - first - .as_slice() - .get(1) - .and_then(|v| uuid::Uuid::parse_str(v).ok()) -} - /// Pure decision for [`authorize_git_read`]: a read requires a current /// active membership row whose role the relay recognizes. /// @@ -1050,7 +1064,7 @@ pub async fn receive_pack( state.config.bind_addr.port() ); let hooks_dir = repo.path().join("hooks").display().to_string(); - let hook_env = vec![ + let mut hook_env = vec![ ("BUZZ_HOOK_URL", hook_url), ( "BUZZ_HOOK_SECRET", @@ -1063,13 +1077,8 @@ pub async fn receive_pack( auth.tenant.community().as_uuid().to_string(), ), ("BUZZ_PUSHER_PUBKEY", pusher_hex.clone()), - // Override any repo-local core.hooksPath setting; defense in - // depth even though the hydrated workspace has no inherited - // config. - ("GIT_CONFIG_COUNT", "1".to_string()), - ("GIT_CONFIG_KEY_0", "core.hooksPath".to_string()), - ("GIT_CONFIG_VALUE_0", hooks_dir), ]; + hook_env.extend(receive_pack_git_config(hooks_dir)); // Run receive-pack against the tempdir. Returns the *owned* subprocess // output (PackOutput) — crucially NOT a Response, so the post-push @@ -1097,6 +1106,23 @@ pub async fn receive_pack( Ok(finalize_push(&state, ctx).await) } +/// Per-process git configuration for the hydrated receive-pack workspace. +fn receive_pack_git_config(hooks_dir: String) -> Vec<(&'static str, String)> { + vec![ + // Override any repo-local core.hooksPath setting; defense in depth + // even though the hydrated workspace has no inherited config. + ("GIT_CONFIG_COUNT", "2".to_string()), + ("GIT_CONFIG_KEY_0", "core.hooksPath".to_string()), + ("GIT_CONFIG_VALUE_0", hooks_dir), + // A bare repository rejects deletion of its symbolic HEAD branch by + // default. Hydrated repositories are ephemeral, and cas_publish + // selects a surviving branch for the next manifest HEAD, so allow + // receive-pack to apply the deletion before that selection runs. + ("GIT_CONFIG_KEY_1", "receive.denyDeleteCurrent".to_string()), + ("GIT_CONFIG_VALUE_1", "ignore".to_string()), + ] +} + /// Buffered output of a `git --stateless-rpc` subprocess. /// /// The handler holds this as an owned value between subprocess completion @@ -1907,11 +1933,148 @@ mod track_c_tests { use buzz_core::CommunityId; use nostr::{EventBuilder, Keys, Kind, Tag}; use std::collections::BTreeMap; + use std::io::Write; + use std::process::Output; fn oid_sha1() -> String { "cb09a769da1c01f458fa6959d4e8eded38fac8d3".to_string() } + fn run_test_git(cwd: &Path, args: &[&str], extra_env: &[(&str, String)]) -> Output { + let mut cmd = std::process::Command::new("git"); + cmd.current_dir(cwd) + .args(args) + .env_clear() + .env("PATH", std::env::var("PATH").unwrap_or_default()) + .env("GIT_CONFIG_NOSYSTEM", "1") + .env("GIT_CONFIG_GLOBAL", "/dev/null") + .env("HOME", "/dev/null"); + for (key, value) in extra_env { + cmd.env(key, value); + } + cmd.output().expect("run git") + } + + fn run_test_receive_pack(repo: &Path, request: &[u8], extra_env: &[(&str, String)]) -> Output { + let mut cmd = std::process::Command::new("git"); + cmd.arg("receive-pack") + .arg("--stateless-rpc") + .arg(repo) + .stdin(std::process::Stdio::piped()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .env_clear() + .env("PATH", std::env::var("PATH").unwrap_or_default()) + .env("GIT_CONFIG_NOSYSTEM", "1") + .env("GIT_CONFIG_GLOBAL", "/dev/null") + .env("HOME", "/dev/null"); + for (key, value) in extra_env { + cmd.env(key, value); + } + + let mut child = cmd.spawn().expect("spawn receive-pack"); + child + .stdin + .take() + .expect("receive-pack stdin") + .write_all(request) + .expect("write receive-pack request"); + child.wait_with_output().expect("wait for receive-pack") + } + + fn assert_git_success(output: Output, operation: &str) { + assert!( + output.status.success(), + "{operation} failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + } + + #[test] + fn receive_pack_config_allows_deleting_current_branch() { + let root = tempfile::TempDir::new().expect("tempdir"); + let remote = root.path().join("remote.git"); + let source = root.path().join("source"); + let remote_arg = remote.to_str().expect("utf-8 remote path"); + let source_arg = source.to_str().expect("utf-8 source path"); + + assert_git_success( + run_test_git( + root.path(), + &["init", "--bare", "--initial-branch=main", remote_arg], + &[], + ), + "initialize bare remote", + ); + assert_git_success( + run_test_git( + root.path(), + &["init", "--initial-branch=main", source_arg], + &[], + ), + "initialize source repository", + ); + assert_git_success( + run_test_git(source.as_path(), &["config", "user.name", "Buzz Test"], &[]), + "configure user name", + ); + assert_git_success( + run_test_git( + source.as_path(), + &["config", "user.email", "buzz-test@example.com"], + &[], + ), + "configure user email", + ); + std::fs::write(source.join("README.md"), "test\n").expect("write fixture"); + assert_git_success( + run_test_git(source.as_path(), &["add", "README.md"], &[]), + "stage fixture", + ); + assert_git_success( + run_test_git(source.as_path(), &["commit", "-m", "fixture"], &[]), + "commit fixture", + ); + assert_git_success( + run_test_git( + source.as_path(), + &["push", remote_arg, "main:main", "main:master"], + &[], + ), + "seed main and master", + ); + + let oid_output = run_test_git(remote.as_path(), &["rev-parse", "refs/heads/main"], &[]); + assert!(oid_output.status.success()); + let old_oid = String::from_utf8(oid_output.stdout) + .expect("utf-8 oid") + .trim() + .to_string(); + let command = format!( + "{old_oid} {} refs/heads/main\0report-status\n", + "0".repeat(40) + ); + let mut request = format!("{:04x}", command.len() + 4).into_bytes(); + request.extend_from_slice(command.as_bytes()); + request.extend_from_slice(b"0000"); + + let git_config = receive_pack_git_config(remote.join("hooks").display().to_string()); + let output = run_test_receive_pack(remote.as_path(), &request, &git_config); + assert!( + output.status.success(), + "receive-pack failed: {}", + String::from_utf8_lossy(&output.stderr) + ); + assert!( + !receive_pack_report_rejected(&output.stdout), + "receive-pack rejected the deletion: {}", + String::from_utf8_lossy(&output.stdout) + ); + + assert!(!remote.join("refs/heads/main").exists()); + assert!(remote.join("refs/heads/master").exists()); + } + /// A gzip-encoded request body is transparently inflated before it /// reaches the git subprocess. Git's smart-HTTP client gzips the /// upload-pack/receive-pack request body past a size threshold (fires @@ -2454,75 +2617,30 @@ mod sec005_read_gate_tests { .expect("sign 30617") } - #[test] - fn repo_bound_channel_id_extracts_valid_uuid() { - let keys = Keys::generate(); - let ch = uuid::Uuid::new_v4(); - let event = announcement( - &keys, - vec![ - Tag::parse(["d", "r"]).unwrap(), - Tag::parse(["buzz-channel", &ch.to_string()]).unwrap(), - ], - ); - assert_eq!(repo_bound_channel_id(&event), Some(ch)); - } - - #[test] - fn repo_bound_channel_id_rejects_absent_and_malformed_bindings() { - let keys = Keys::generate(); - let absent = announcement(&keys, vec![Tag::parse(["d", "r"]).unwrap()]); - assert_eq!(repo_bound_channel_id(&absent), None); - - let malformed = announcement( - &keys, - vec![ - Tag::parse(["d", "r"]).unwrap(), - Tag::parse(["buzz-channel", "not-a-uuid"]).unwrap(), - ], - ); - assert_eq!(repo_bound_channel_id(&malformed), None); - - let empty = announcement( - &keys, - vec![ - Tag::parse(["d", "r"]).unwrap(), - Tag::parse(["buzz-channel"]).unwrap(), - ], - ); - assert_eq!(repo_bound_channel_id(&empty), None); + // Binding *parse* semantics (first-tag fails-closed, duplicate-tag + // ambiguity, malformed vs. absent) are unit-tested where the resolver + // lives: `super::super::binding`. The tests below prove the *gate* wires + // each resolver outcome to the right response — allow, generic denial + // body, or the author remediation body — which the resolver tests + // cannot see. + + /// Collapse an `authorize_git_read` denial to `(status, body)` so tests + /// can assert on the exact bytes a git client would see. A blind + /// `.is_err()` cannot distinguish the generic 404 from the remediation + /// 404 — and that distinction IS the security property. + async fn denial_parts(result: Result<(), Response>) -> (StatusCode, String) { + let response = result.expect_err("expected a denial"); + let status = response.status(); + let bytes = axum::body::to_bytes(response.into_body(), usize::MAX) + .await + .expect("read denial body"); + ( + status, + String::from_utf8(bytes.to_vec()).expect("utf-8 body"), + ) } - #[test] - fn repo_bound_channel_id_fails_closed_on_ambiguous_duplicate_bindings() { - // First-tag semantics: a malformed first binding must deny even when - // a later duplicate tag is valid. An ambiguous announcement must not - // silently resolve to whichever duplicate happens to parse. - let keys = Keys::generate(); - let ch = uuid::Uuid::new_v4(); - let malformed_first = announcement( - &keys, - vec![ - Tag::parse(["d", "r"]).unwrap(), - Tag::parse(["buzz-channel", "not-a-uuid"]).unwrap(), - Tag::parse(["buzz-channel", &ch.to_string()]).unwrap(), - ], - ); - assert_eq!(repo_bound_channel_id(&malformed_first), None); - - // And the mirror image: a valid first binding wins, matching the - // push policy endpoint's first-tag resolution. - let other = uuid::Uuid::new_v4(); - let valid_first = announcement( - &keys, - vec![ - Tag::parse(["d", "r"]).unwrap(), - Tag::parse(["buzz-channel", &ch.to_string()]).unwrap(), - Tag::parse(["buzz-channel", &other.to_string()]).unwrap(), - ], - ); - assert_eq!(repo_bound_channel_id(&valid_first), Some(ch)); - } + const GENERIC_DENIAL: &str = "repository not found"; // ── authorize_git_read matrix (requires Postgres) ──────────────────── @@ -2544,6 +2662,11 @@ mod sec005_read_gate_tests { Missing, /// `buzz-channel` tag whose value is not a UUID. Malformed, + /// `buzz-channel` tag carrying a well-formed UUID that names no + /// channel. The resolver reports `Bound`; the membership lookup + /// (whose SQL joins `channels … deleted_at IS NULL`) then returns + /// no role — the deliberate phase-1 posture for dead bindings. + UnknownChannel, } struct RepoFixture { @@ -2609,6 +2732,9 @@ mod sec005_read_gate_tests { Binding::Malformed => { tags.push(Tag::parse(["buzz-channel", "not-a-uuid"]).unwrap()); } + Binding::UnknownChannel => { + tags.push(Tag::parse(["buzz-channel", &uuid::Uuid::new_v4().to_string()]).unwrap()); + } } let event = announcement(&owner_keys, tags); db.insert_event(community, &event, None) @@ -2676,32 +2802,63 @@ mod sec005_read_gate_tests { #[tokio::test] #[ignore = "requires Postgres"] async fn read_gate_denies_missing_or_malformed_binding_and_absent_repo() { - // Missing buzz-channel tag → deny even for a channel member. + // Missing buzz-channel tag → deny even for a channel member, with + // the generic body: the remediation carve-out is author-only. let f = setup_repo(Binding::Missing).await; let member = f.member_keys.public_key(); - assert!( - authorize_git_read(&f.db, f.community, &member, &f.owner_hex, &f.repo) - .await - .is_err(), - "announcement without buzz-channel binding must deny" + let (status, body) = denial_parts( + authorize_git_read(&f.db, f.community, &member, &f.owner_hex, &f.repo).await, + ) + .await; + assert_eq!(status, StatusCode::NOT_FOUND); + assert_eq!( + body, GENERIC_DENIAL, + "unbound repo read by a NON-author must get the generic body — \ + remediation for anyone but the announcement author leaks repo existence" ); - // Malformed buzz-channel tag → deny. + // Malformed buzz-channel tag → deny with the generic body EVEN FOR + // THE AUTHOR. This is the assertion that pins the carve-out to + // NotBound: if it ever fires on Broken, this fails on bytes, not + // on Ok/Err (which cannot see the difference). let g = setup_repo(Binding::Malformed).await; - let member_g = g.member_keys.public_key(); - assert!( - authorize_git_read(&g.db, g.community, &member_g, &g.owner_hex, &g.repo) - .await - .is_err(), - "announcement with malformed buzz-channel binding must deny" + let g_owner = g.owner_keys.public_key(); + let (status, body) = denial_parts( + authorize_git_read(&g.db, g.community, &g_owner, &g.owner_hex, &g.repo).await, + ) + .await; + assert_eq!(status, StatusCode::NOT_FOUND); + assert_eq!( + body, GENERIC_DENIAL, + "broken binding must stay generic even for the author (ambiguity fails closed)" + ); + + // Well-formed UUID naming a nonexistent channel → resolver says + // Bound, membership lookup finds nothing → generic denial for + // everyone, author included. The dead-channel case must be + // indistinguishable from non-membership (phase-1 posture; ingest + // validation closes the front door in phase 2). + let u = setup_repo(Binding::UnknownChannel).await; + let u_owner = u.owner_keys.public_key(); + let (status, body) = denial_parts( + authorize_git_read(&u.db, u.community, &u_owner, &u.owner_hex, &u.repo).await, + ) + .await; + assert_eq!(status, StatusCode::NOT_FOUND); + assert_eq!( + body, GENERIC_DENIAL, + "binding to a nonexistent channel must deny generically, even for the author" ); // Nonexistent announcement → deny. - assert!( - authorize_git_read(&f.db, f.community, &member, &f.owner_hex, "no-such-repo") - .await - .is_err(), - "nonexistent repo must deny" + let (status, body) = denial_parts( + authorize_git_read(&f.db, f.community, &member, &f.owner_hex, "no-such-repo").await, + ) + .await; + assert_eq!(status, StatusCode::NOT_FOUND); + assert_eq!( + body, GENERIC_DENIAL, + "nonexistent repo must deny generically" ); // Owner-mismatch: URL owner differs from announcement author → deny. @@ -2722,6 +2879,53 @@ mod sec005_read_gate_tests { ); } + #[tokio::test] + #[ignore = "requires Postgres"] + async fn read_gate_gives_author_of_unbound_repo_remediation_body() { + // Issue #3527: the author of a never-bound announcement is the one + // identity that can fix it (30617 is keyed by (author, d)) and the + // one identity remediation cannot leak anything to. Status must stay + // 404 — identical to every other denial — with the bind command in + // the body. + let f = setup_repo(Binding::Missing).await; + let author = f.owner_keys.public_key(); + + let response = authorize_git_read(&f.db, f.community, &author, &f.owner_hex, &f.repo) + .await + .expect_err("unbound repo must still deny its author"); + assert_eq!(response.status(), StatusCode::NOT_FOUND); + // Guard against a future "tidy" into Json(...) or a custom + // IntoResponse: git prints `remote:` lines only for text/plain + // bodies — any other content-type makes the remediation silently + // invisible in the user's terminal with no failing assertion. + assert_eq!( + response + .headers() + .get(axum::http::header::CONTENT_TYPE) + .and_then(|v| v.to_str().ok()), + Some("text/plain; charset=utf-8"), + "remediation body must stay text/plain or git clients will swallow it" + ); + let bytes = axum::body::to_bytes(response.into_body(), usize::MAX) + .await + .expect("read remediation body"); + let body = String::from_utf8(bytes.to_vec()).expect("utf-8 body"); + assert!( + body.starts_with(&format!("run: buzz repos bind --id {}", f.repo)), + "remediation must lead with the actionable command (got {body:?})" + ); + assert_ne!(body, GENERIC_DENIAL); + + // Same repo, same state, different caller: a member of some channel + // who is not the author still gets the generic body. + let member = f.member_keys.public_key(); + let (_, body) = denial_parts( + authorize_git_read(&f.db, f.community, &member, &f.owner_hex, &f.repo).await, + ) + .await; + assert_eq!(body, GENERIC_DENIAL, "remediation is author-only"); + } + #[tokio::test] #[ignore = "requires Postgres"] async fn read_gate_follows_current_announcement_not_stale_registry() { @@ -2740,10 +2944,18 @@ mod sec005_read_gate_tests { ); let owner_pk = f.owner_keys.public_key().to_bytes().to_vec(); + // Tombstone timestamped after the announcement, per NIP-09's + // at-or-before scoping in `soft_delete_by_coordinate`. let deleted = - f.db.soft_delete_by_coordinate(f.community, 30617, &owner_pk, &f.repo) - .await - .expect("soft delete 30617"); + f.db.soft_delete_by_coordinate( + f.community, + 30617, + &owner_pk, + &f.repo, + chrono::Utc::now().timestamp() + 60, + ) + .await + .expect("soft delete 30617"); assert!(deleted, "precondition: a live announcement row was deleted"); assert!( diff --git a/crates/buzz-relay/src/config.rs b/crates/buzz-relay/src/config.rs index a1691349d6..85a0ca2efe 100644 --- a/crates/buzz-relay/src/config.rs +++ b/crates/buzz-relay/src/config.rs @@ -56,6 +56,10 @@ pub struct Config { /// Optional read-replica connection URL (e.g. an Aurora `cluster-ro-` /// endpoint). Unset means all reads stay on the writer. pub read_database_url: Option, + /// Replica read budget `B` in milliseconds (`BUZZ_REPLICA_READ_MAX_AGE_MS`). + /// `0` (the default) disables bounded-staleness replica routing; see + /// [`buzz_db::DbConfig::replica_read_max_age_ms`]. + pub replica_read_max_age_ms: u64, /// Redis connection URL used by the pub/sub manager. pub redis_url: String, /// Maximum connections in the shared Redis pool. Defaults to 16. @@ -72,6 +76,11 @@ pub struct Config { /// the per-pod pool and requests fail on acquire timeout while the /// database sits idle. pub db_pool_size: u32, + /// Maximum connections in the Postgres read-replica pool + /// (`BUZZ_DB_READ_POOL_SIZE`). Defaults to `db_pool_size`. Sized + /// independently so reader capacity can be tuned against the replica's + /// headroom without touching the writer pool. + pub db_read_pool_size: Option, /// Public WebSocket URL of this relay, advertised in NIP-11. pub relay_url: String, /// Public WebSocket URL of the dedicated device-pairing relay, when configured. @@ -423,6 +432,27 @@ impl Config { .map(|v| v.trim().to_string()) .filter(|v| !v.is_empty()); + // The old seconds-denominated name is a hard startup error, not an + // alias: silently honouring it would mean 1000x the intended budget. + if std::env::var("BUZZ_REPLICA_HEAD_MAX_AGE_SECS").is_ok() { + return Err(ConfigError::InvalidValue( + "BUZZ_REPLICA_HEAD_MAX_AGE_SECS was renamed to BUZZ_REPLICA_READ_MAX_AGE_MS \ + (note: milliseconds, not seconds); refusing to start" + .to_string(), + )); + } + + // Replica read budget: 0 = off (the rollout default), so this is a + // non-negative parse, unlike `positive_u64_from_env`. + let replica_read_max_age_ms = match std::env::var("BUZZ_REPLICA_READ_MAX_AGE_MS") { + Ok(raw) => raw.trim().parse::().map_err(|_| { + ConfigError::InvalidValue( + "BUZZ_REPLICA_READ_MAX_AGE_MS must be a non-negative integer".to_string(), + ) + })?, + Err(_) => 0, + }; + let redis_url = std::env::var("REDIS_URL").unwrap_or_else(|_| "redis://localhost:6379".to_string()); @@ -438,6 +468,11 @@ impl Config { .filter(|&v| v > 0) .unwrap_or(50); + let db_read_pool_size = std::env::var("BUZZ_DB_READ_POOL_SIZE") + .ok() + .and_then(|v| v.parse::().ok()) + .filter(|&v| v > 0); + let relay_url = std::env::var("RELAY_URL").unwrap_or_else(|_| "ws://localhost:3000".to_string()); @@ -630,6 +665,16 @@ impl Config { .and_then(|v| v.parse().ok()) .unwrap_or(9102); + let s3_addressing_style = match std::env::var("BUZZ_S3_ADDRESSING_STYLE") { + Ok(value) => value.parse().map_err(ConfigError::InvalidValue)?, + Err(std::env::VarError::NotPresent) => buzz_media::config::S3AddressingStyle::default(), + Err(std::env::VarError::NotUnicode(_)) => { + return Err(ConfigError::InvalidValue( + "BUZZ_S3_ADDRESSING_STYLE must be valid Unicode and one of 'path' or 'virtual'" + .to_string(), + )); + } + }; let media = buzz_media::MediaConfig { s3_endpoint: std::env::var("BUZZ_S3_ENDPOINT") .unwrap_or_else(|_| "http://localhost:9000".to_string()), @@ -641,6 +686,7 @@ impl Config { s3_region: std::env::var("BUZZ_S3_REGION") .or_else(|_| std::env::var("AWS_REGION")) .unwrap_or_else(|_| "us-east-1".to_string()), + s3_addressing_style, max_image_bytes: std::env::var("BUZZ_MAX_IMAGE_BYTES") .ok() .and_then(|v| v.parse().ok()) @@ -887,9 +933,11 @@ impl Config { bind_addr, database_url, read_database_url, + replica_read_max_age_ms, redis_url, redis_pool_size, db_pool_size, + db_read_pool_size, relay_url, pairing_relay_url, max_connections, @@ -990,6 +1038,11 @@ mod tests { !config.require_media_get_auth, "require_media_get_auth should default to false for staged client rollout" ); + assert_eq!( + config.media.s3_addressing_style, + buzz_media::config::S3AddressingStyle::Path, + "S3 addressing must default to path style for bundled MinIO compatibility" + ); assert!( config.join_policy.is_none(), "join_policy should default to None so policy prompts and acceptance receipts are opt-in" @@ -1000,6 +1053,61 @@ mod tests { ); } + #[test] + fn s3_addressing_style_env_accepts_virtual_and_rejects_invalid_values() { + let _guard = ENV_MUTEX.lock().unwrap(); + let previous = std::env::var_os("BUZZ_S3_ADDRESSING_STYLE"); + + std::env::set_var("BUZZ_S3_ADDRESSING_STYLE", "virtual"); + let configured = Config::from_env() + .expect("virtual style config") + .media + .s3_addressing_style; + + std::env::set_var("BUZZ_S3_ADDRESSING_STYLE", "auto"); + let invalid = Config::from_env(); + + if let Some(value) = previous { + std::env::set_var("BUZZ_S3_ADDRESSING_STYLE", value); + } else { + std::env::remove_var("BUZZ_S3_ADDRESSING_STYLE"); + } + + assert_eq!(configured, buzz_media::config::S3AddressingStyle::Virtual); + assert!(matches!( + invalid, + Err(ConfigError::InvalidValue(ref message)) + if message.contains("BUZZ_S3_ADDRESSING_STYLE must be 'path' or 'virtual'") + )); + } + + #[cfg(unix)] + #[test] + fn s3_addressing_style_env_rejects_non_unicode_values() { + use std::os::unix::ffi::OsStringExt; + + let _guard = ENV_MUTEX.lock().unwrap(); + let previous = std::env::var_os("BUZZ_S3_ADDRESSING_STYLE"); + std::env::set_var( + "BUZZ_S3_ADDRESSING_STYLE", + std::ffi::OsString::from_vec(vec![0xff]), + ); + + let invalid = Config::from_env(); + + if let Some(value) = previous { + std::env::set_var("BUZZ_S3_ADDRESSING_STYLE", value); + } else { + std::env::remove_var("BUZZ_S3_ADDRESSING_STYLE"); + } + + assert!(matches!( + invalid, + Err(ConfigError::InvalidValue(ref message)) + if message.contains("must be valid Unicode") + )); + } + #[test] fn redis_pool_size_env_override_and_invalid_fallback() { let _guard = ENV_MUTEX.lock().unwrap(); @@ -1050,6 +1158,35 @@ mod tests { assert_eq!(junk, 50, "unparsable value must fall back to the default"); } + #[test] + fn db_read_pool_size_env_override_and_invalid_fallback() { + let _guard = ENV_MUTEX.lock().unwrap(); + let previous = std::env::var_os("BUZZ_DB_READ_POOL_SIZE"); + + std::env::remove_var("BUZZ_DB_READ_POOL_SIZE"); + let unset = Config::from_env().expect("config").db_read_pool_size; + + std::env::set_var("BUZZ_DB_READ_POOL_SIZE", "40"); + let overridden = Config::from_env().expect("config").db_read_pool_size; + + std::env::set_var("BUZZ_DB_READ_POOL_SIZE", "0"); + let zero = Config::from_env().expect("config").db_read_pool_size; + + std::env::set_var("BUZZ_DB_READ_POOL_SIZE", "not-a-number"); + let junk = Config::from_env().expect("config").db_read_pool_size; + + if let Some(value) = previous { + std::env::set_var("BUZZ_DB_READ_POOL_SIZE", value); + } else { + std::env::remove_var("BUZZ_DB_READ_POOL_SIZE"); + } + + assert_eq!(unset, None, "unset must inherit the writer pool sizing"); + assert_eq!(overridden, Some(40)); + assert_eq!(zero, None, "zero must fall back to inheriting"); + assert_eq!(junk, None, "unparsable value must fall back to inheriting"); + } + #[test] fn read_database_url_unset_or_blank_is_none() { let _guard = ENV_MUTEX.lock().unwrap(); @@ -1078,6 +1215,58 @@ mod tests { ); } + #[test] + fn replica_read_max_age_defaults_off_and_rejects_junk() { + let _guard = ENV_MUTEX.lock().unwrap(); + let previous = std::env::var_os("BUZZ_REPLICA_READ_MAX_AGE_MS"); + let previous_old = std::env::var_os("BUZZ_REPLICA_HEAD_MAX_AGE_SECS"); + std::env::remove_var("BUZZ_REPLICA_HEAD_MAX_AGE_SECS"); + + std::env::remove_var("BUZZ_REPLICA_READ_MAX_AGE_MS"); + let unset = Config::from_env().expect("config").replica_read_max_age_ms; + + std::env::set_var("BUZZ_REPLICA_READ_MAX_AGE_MS", "1000"); + let set = Config::from_env().expect("config").replica_read_max_age_ms; + + std::env::set_var("BUZZ_REPLICA_READ_MAX_AGE_MS", "0"); + let zero = Config::from_env().expect("config").replica_read_max_age_ms; + + std::env::set_var("BUZZ_REPLICA_READ_MAX_AGE_MS", "soon"); + let junk = Config::from_env(); + + // The retired seconds-denominated name must be a hard startup + // error even alongside a valid new-name value: silently ignoring + // it (or honouring it) would mean 1000x the intended budget. + std::env::set_var("BUZZ_REPLICA_READ_MAX_AGE_MS", "1000"); + std::env::set_var("BUZZ_REPLICA_HEAD_MAX_AGE_SECS", "5"); + let old_name = Config::from_env(); + + std::env::remove_var("BUZZ_REPLICA_HEAD_MAX_AGE_SECS"); + if let Some(value) = previous { + std::env::set_var("BUZZ_REPLICA_READ_MAX_AGE_MS", value); + } else { + std::env::remove_var("BUZZ_REPLICA_READ_MAX_AGE_MS"); + } + if let Some(value) = previous_old { + std::env::set_var("BUZZ_REPLICA_HEAD_MAX_AGE_SECS", value); + } + + assert_eq!(unset, 0, "replica read routing must default off"); + assert_eq!(set, 1000); + assert_eq!(zero, 0, "explicit 0 is off"); + assert!( + junk.is_err(), + "an unparsable budget must fail loudly, not silently disable" + ); + match old_name { + Err(ConfigError::InvalidValue(message)) => assert!( + message.contains("BUZZ_REPLICA_READ_MAX_AGE_MS"), + "the error must name the replacement env var, got: {message}" + ), + other => panic!("old env name must hard-fail startup, got {other:?}"), + } + } + #[test] fn audit_logging_defaults_on_and_accepts_explicit_off() { let _guard = ENV_MUTEX.lock().unwrap(); diff --git a/crates/buzz-relay/src/handlers/count.rs b/crates/buzz-relay/src/handlers/count.rs index 614e54d7a0..3eeab5e807 100644 --- a/crates/buzz-relay/src/handlers/count.rs +++ b/crates/buzz-relay/src/handlers/count.rs @@ -7,8 +7,8 @@ use tracing::warn; use crate::connection::{AuthState, ConnectionState}; use crate::handlers::req::{ - event_visible_to_reader, filter_can_match_persona_shared_kinds, - filter_can_match_result_gated_kinds, result_gated_count_safe_for_pushdown, + event_visible_to_reader, filter_can_match_result_gated_kinds, + filter_can_match_shared_gated_kinds, result_gated_count_safe_for_pushdown, }; use crate::protocol::RelayMessage; use crate::state::AppState; @@ -103,11 +103,11 @@ pub async fn handle_count( // fast-path count_events() cannot be used because it doesn't do // per-event author filtering. let needs_author_only_filtering = super::req::filter_can_match_author_only_kinds(filter); - // Determine if this filter can match kind 30175 (persona) — if so, the - // fast-path must be bypassed because it has no per-event shared-tag check. - // A fast count over 30175 would include foreign unshared persona events, - // leaking the existence of private agent activity. - let needs_persona_filtering = filter_can_match_persona_shared_kinds(filter); + // Determine if this filter can match a shared-gated kind (30175, 30178) + // — if so, the fast path must be bypassed because it has no per-event + // shared-tag check. A fast count over those kinds would include foreign + // unshared events, leaking the existence of private agent activity. + let needs_shared_gate_filtering = filter_can_match_shared_gated_kinds(filter); // Determine if this filter can match result-gated kinds (44200, 30622) // that require a per-event owner check. When the fast SQL path would // count matching rows without calling reader_authorized_for_event, a @@ -157,10 +157,10 @@ pub async fn handle_count( conn.tenant.community(), ) .await; - // Persona visibility pushdown: pre-filter the fallback query_events - // candidate page before ORDER/LIMIT. - if needs_persona_filtering { - query.persona_reader = Some(pubkey_bytes.clone()); + // Shared-gated visibility pushdown: pre-filter the fallback + // query_events candidate page before ORDER/LIMIT. + if needs_shared_gate_filtering { + query.shared_gated_reader = Some(pubkey_bytes.clone()); } let author_is_self = filter.authors.as_ref().is_some_and(|authors| { !authors.is_empty() @@ -171,9 +171,9 @@ pub async fn handle_count( if super::req::filter_fully_pushable(filter) && (!needs_author_only_filtering || author_is_self) && !needs_result_gated_filtering - && !needs_persona_filtering + && !needs_shared_gate_filtering { - match state.db.count_events(&query).await { + match state.db.count_events_routed("count_req", &query).await { Ok(n) => total += n as u64, Err(e) => { conn.send(RelayMessage::closed(&sub_id, &format!("error: {e}"))); @@ -184,7 +184,11 @@ pub async fn handle_count( // Fallback: query + post-filter for non-pushable constraints. let mut q = query; super::req::apply_count_fallback_limit(&mut q); - match state.db.query_events(&q).await { + match state + .db + .query_events_routed_bounded("count_req_fallback", &q) + .await + { Ok(stored_events) => { if super::req::count_fallback_exceeded(stored_events.len()) { metrics::counter!("buzz_count_fallback_rejections_total").increment(1); @@ -226,9 +230,9 @@ pub async fn handle_count( ) .await; query.channel_ids = Some(accessible_channels.to_vec()); - // Persona visibility pushdown for the fallback query_events path. - if needs_persona_filtering { - query.persona_reader = Some(pubkey_bytes.clone()); + // Shared-gated visibility pushdown for the fallback query_events path. + if needs_shared_gate_filtering { + query.shared_gated_reader = Some(pubkey_bytes.clone()); } let author_is_self = filter.authors.as_ref().is_some_and(|authors| { @@ -240,10 +244,10 @@ pub async fn handle_count( if super::req::filter_fully_pushable(filter) && (!needs_author_only_filtering || author_is_self) && !needs_result_gated_filtering - && !needs_persona_filtering + && !needs_shared_gate_filtering { query.limit = None; // COUNT doesn't need a row limit - match state.db.count_events(&query).await { + match state.db.count_events_routed("count_req", &query).await { Ok(n) => total += n as u64, Err(e) => { conn.send(RelayMessage::closed(&sub_id, &format!("error: {e}"))); @@ -253,7 +257,11 @@ pub async fn handle_count( } else { // Fallback: query a bounded candidate set + post-filter. super::req::apply_count_fallback_limit(&mut query); - match state.db.query_events(&query).await { + match state + .db + .query_events_routed_bounded("count_req_fallback", &query) + .await + { Ok(stored_events) => { if super::req::count_fallback_exceeded(stored_events.len()) { metrics::counter!("buzz_count_fallback_rejections_total").increment(1); diff --git a/crates/buzz-relay/src/handlers/event.rs b/crates/buzz-relay/src/handlers/event.rs index 88dd5f5180..a9cdffcdec 100644 --- a/crates/buzz-relay/src/handlers/event.rs +++ b/crates/buzz-relay/src/handlers/event.rs @@ -7,7 +7,7 @@ use tracing::{debug, error, info, warn}; use buzz_core::event::StoredEvent; use buzz_core::kind::{ - event_kind_u32, is_ephemeral, is_unshared_persona_event, AUTHOR_ONLY_KINDS, + event_kind_u32, is_ephemeral, is_unshared_gated_event, AUTHOR_ONLY_KINDS, KIND_AGENT_OBSERVER_FRAME, KIND_GIFT_WRAP, KIND_PRESENCE_UPDATE, }; use buzz_core::observer::{ @@ -151,10 +151,10 @@ pub async fn filter_fanout_by_access( matches }; - // Persona shared-read gate (fan-out): kind 30175 events fan out to all - // connections only when carrying ["shared","true"]. Unshared personas - // are delivered only to the author's own connections, matching REQ semantics. - let matches = if buzz_core::kind::is_persona_shared_kind(event_kind_u32(&stored_event.event)) { + // Shared-read gate (fan-out): SHARED_GATED_KINDS events fan out to all + // connections only when carrying ["shared","true"]. Unshared ones are + // delivered only to the author's own connections, matching REQ semantics. + let matches = if buzz_core::kind::is_shared_gated_kind(event_kind_u32(&stored_event.event)) { let author = stored_event.event.pubkey.to_bytes(); matches .into_iter() @@ -167,7 +167,7 @@ pub async fn filter_fanout_by_access( return true; } // Foreign connection: allowed only if the event is shared. - !is_unshared_persona_event(&stored_event.event, &pk) + !is_unshared_gated_event(&stored_event.event, &pk) }) .collect() } else { diff --git a/crates/buzz-relay/src/handlers/ingest.rs b/crates/buzz-relay/src/handlers/ingest.rs index a30b0e714d..fcd0d70728 100644 --- a/crates/buzz-relay/src/handlers/ingest.rs +++ b/crates/buzz-relay/src/handlers/ingest.rs @@ -28,12 +28,12 @@ use buzz_core::kind::{ KIND_NIP29_EDIT_METADATA, KIND_NIP29_JOIN_REQUEST, KIND_NIP29_LEAVE_REQUEST, KIND_NIP29_PUT_USER, KIND_NIP29_REMOVE_USER, KIND_NIP43_LEAVE_REQUEST, KIND_NIP65_RELAY_LIST_METADATA, KIND_PERSONA, KIND_PIN_LIST, KIND_PRESENCE_UPDATE, - KIND_PRODUCT_FEEDBACK, KIND_PROFILE, KIND_REACTION, KIND_READ_STATE, KIND_REPORT, + KIND_PRODUCT_FEEDBACK, KIND_PROFILE, KIND_PROJECT, KIND_REACTION, KIND_READ_STATE, KIND_REPORT, KIND_STREAM_MESSAGE, KIND_STREAM_MESSAGE_BOOKMARKED, KIND_STREAM_MESSAGE_DIFF, KIND_STREAM_MESSAGE_EDIT, KIND_STREAM_MESSAGE_PINNED, KIND_STREAM_MESSAGE_SCHEDULED, - KIND_STREAM_MESSAGE_V2, KIND_STREAM_REMINDER, KIND_TEAM, KIND_TEXT_NOTE, KIND_USER_STATUS, - KIND_WORKFLOW_DEF, KIND_WORKFLOW_TRIGGER, RELAY_ADMIN_ADD_MEMBER, RELAY_ADMIN_CHANGE_ROLE, - RELAY_ADMIN_REMOVE_MEMBER, RELAY_ADMIN_SET_WORKSPACE_PROFILE, + KIND_STREAM_MESSAGE_V2, KIND_STREAM_REMINDER, KIND_TEAM, KIND_TEAM_CATALOG, KIND_TEXT_NOTE, + KIND_USER_STATUS, KIND_WORKFLOW_DEF, KIND_WORKFLOW_TRIGGER, RELAY_ADMIN_ADD_MEMBER, + RELAY_ADMIN_CHANGE_ROLE, RELAY_ADMIN_REMOVE_MEMBER, RELAY_ADMIN_SET_WORKSPACE_PROFILE, }; use buzz_core::tenant::TenantContext; use buzz_core::verification::verify_event; @@ -214,7 +214,7 @@ fn required_scope_for_kind(kind: u32, event: &Event) -> Result Ok(Scope::MessagesWrite), KIND_CONTACT_LIST | KIND_READ_STATE | KIND_USER_STATUS | KIND_AGENT_ENGRAM | KIND_EVENT_REMINDER | KIND_PERSONA | KIND_TEAM | KIND_MANAGED_AGENT - | super::push_lease::KIND_PUSH_LEASE => { + | KIND_TEAM_CATALOG | super::push_lease::KIND_PUSH_LEASE => { Ok(Scope::UsersWrite) } // NIP-AM: agent turn metrics are agent-authored global events (encrypted to owner). @@ -301,6 +301,9 @@ fn required_scope_for_kind(kind: u32, event: &Event) -> Result Ok(Scope::ChannelsWrite), // NIP-34: Git repository events KIND_GIT_REPO_ANNOUNCEMENT | KIND_GIT_REPO_STATE => Ok(Scope::ReposWrite), + // NIP-MP: a project is repository metadata — grouping repositories needs + // the same scope as announcing them. + KIND_PROJECT => Ok(Scope::ReposWrite), KIND_GIT_PATCH | KIND_GIT_PULL_REQUEST | KIND_GIT_PR_UPDATE @@ -419,10 +422,12 @@ pub(crate) fn is_global_only_kind(kind: u32) -> bool { | KIND_AGENT_PROFILE // NIP-AP: persona definitions (30175): owner-authored, keyed by (pubkey, kind, d_tag). | KIND_PERSONA - // NIP-AP: team (30176) + managed-agent (30177) definitions: owner-authored, - // keyed by (pubkey, kind, d_tag). A stray `h` tag must not channel-scope them. + // NIP-AP: team (30176) + managed-agent (30177) definitions and the + // team-catalog projection (30178): owner-authored, keyed by + // (pubkey, kind, d_tag). A stray `h` tag must not channel-scope them. | KIND_TEAM | KIND_MANAGED_AGENT + | KIND_TEAM_CATALOG // NIP-34: git events use `a` tags (repo reference), not `h` tags (channel scope). // Parameterized replaceable kinds are keyed by (pubkey, kind, d_tag). | KIND_GIT_REPO_ANNOUNCEMENT @@ -435,6 +440,10 @@ pub(crate) fn is_global_only_kind(kind: u32) -> bool { | KIND_GIT_STATUS_MERGED | KIND_GIT_STATUS_CLOSED | KIND_GIT_STATUS_DRAFT + // NIP-MP: projects are addressed by (pubkey, kind, d_tag). The + // `buzz-channel` tag is a metadata reference, not a routing directive, + // so a project's state is never channel-scoped. + | KIND_PROJECT // Community moderation commands (9040–9044): community-global // direct commands, same model as the NIP-43 9030-series. A stray // `h` tag must never channel-scope them (pinned contract — @@ -1029,37 +1038,27 @@ fn validate_engram_envelope(event: &Event) -> Result<(), String> { Ok(()) } -/// Validate the envelope of a kind:30175 persona event. -/// -/// Enforces: -/// * exactly one `d` tag with a non-empty value matching the slug grammar -/// `^[a-z0-9][a-z0-9_-]{0,63}$`. -/// * at most one `shared` tag; if present, its value must be exactly `"true"`. +/// Enforce the `shared`-tag shape shared by every kind in +/// [`buzz_core::kind::SHARED_GATED_KINDS`]: at most one `shared` tag, and if +/// present it must be exactly `["shared", "true"]`. /// -/// Without the `d`-tag check, an empty d-tag collapses every persona into the -/// `(pubkey, 30175, "")` slot — last-write-wins data loss. +/// This ensures no ambiguous heads: either an event has no `shared` tag +/// (author-only) or exactly `["shared", "true"]` (community-readable). Any +/// other value (`"false"`, `"1"`, extra elements, duplicate tags) is rejected +/// at ingest so read-path helpers — including the SQL-level `tags @> +/// '[["shared","true"]]'` containment clause, which would otherwise match a +/// three-element superset — can treat stored events as unambiguously one or the +/// other. /// -/// The `shared` tag rule ensures no ambiguous heads: either an event has no -/// `shared` tag (author-only) or exactly `["shared", "true"]` (community- -/// readable). Any other value (`"false"`, `"1"`, extra tags) is rejected at -/// ingest so read-path helpers can treat stored events as unambiguously one or -/// the other. -fn validate_persona_envelope(event: &Event) -> Result<(), String> { - let mut d_tags: Vec<&str> = Vec::new(); +/// `label` names the kind in error messages (e.g. `"persona event"`). +fn validate_shared_tag(event: &Event, label: &str) -> Result<(), String> { let mut shared_count = 0usize; for tag in event.tags.iter() { let parts = tag.as_slice(); - if parts.len() >= 2 && parts[0].as_str() == "d" { - d_tags.push(&parts[1]); - } if !parts.is_empty() && parts[0].as_str() == "shared" { - // Exact shape required: ["shared", "true"] — exactly two elements, - // second element exactly "true". Extra elements are rejected so that - // a three-element tag like ["shared","true","extra"] cannot be stored - // and later misread as shared by the SQL-level visibility clause. if parts.len() != 2 || parts[1].as_str() != "true" { return Err(format!( - "persona event `shared` tag must be exactly [\"shared\",\"true\"] (got {:?})", + "{label} `shared` tag must be exactly [\"shared\",\"true\"] (got {:?})", parts.iter().map(|s| s.as_str()).collect::>() )); } @@ -1068,39 +1067,380 @@ fn validate_persona_envelope(event: &Event) -> Result<(), String> { } if shared_count > 1 { return Err(format!( - "persona event must have at most one `shared` tag (got {shared_count})" + "{label} must have at most one `shared` tag (got {shared_count})" )); } + Ok(()) +} + +/// Return the event's single `d` tag value, requiring exactly one tag whose +/// value is non-empty, at most 64 characters, and free of Unicode control +/// characters and whitespace. +/// +/// Without this check an empty `d` tag collapses every event of the kind into +/// the `(pubkey, kind, "")` slot — last-write-wins data loss. The character +/// bound keeps the value usable as a NIP-33 coordinate (`::`) +/// and as a log field: an embedded newline or tab would break line-oriented +/// consumers of both. +/// +/// Tags are counted by their first element alone, so a valueless `["d"]` +/// counts. Skipping it would let `["d"]` plus `["d", "team-1"]` pass the +/// exactly-one rule, and a NIP-33 consumer that reads `["d"]` as an +/// empty-valued first `d` tag would then address the event at `""` where this +/// relay addresses it at `"team-1"`. +/// +/// `label` names the kind in error messages (e.g. `"persona event"`). +fn single_bounded_d_tag<'a>(event: &'a Event, label: &str) -> Result<&'a str, String> { + let d_tags: Vec> = event + .tags + .iter() + .filter_map(|tag| { + let parts = tag.as_slice(); + (parts.first().map(|name| name.as_str()) == Some("d")) + .then(|| parts.get(1).map(|value| value.as_str())) + }) + .collect(); if d_tags.len() != 1 { return Err(format!( - "persona event must have exactly one `d` tag (got {})", + "{label} must have exactly one `d` tag (got {})", d_tags.len() )); } - let d = d_tags[0]; + let d = d_tags[0].unwrap_or_default(); if d.is_empty() { - return Err("persona event `d` tag must not be empty".to_string()); + return Err(format!("{label} `d` tag must not be empty")); } - // Slug grammar: ^[a-z0-9][a-z0-9_-]{0,63}$ - if d.len() > 64 { + let char_count = d.chars().count(); + if char_count > 64 { return Err(format!( - "persona event `d` tag too long ({} chars, max 64)", - d.len() + "{label} `d` tag too long ({char_count} chars, max 64)" )); } + if d.chars().any(|c| c.is_control() || c.is_whitespace()) { + return Err(format!( + "{label} `d` tag must not contain control characters or whitespace" + )); + } + Ok(d) +} + +/// Validate the envelope of a kind:30175 persona event. +/// +/// Enforces the shared-gated `shared`-tag shape ([`validate_shared_tag`]) plus +/// exactly one `d` tag matching the persona slug grammar +/// `^[a-z0-9][a-z0-9_-]{0,63}$`. +fn validate_persona_envelope(event: &Event) -> Result<(), String> { + const LABEL: &str = "persona event"; + validate_shared_tag(event, LABEL)?; + let d = single_bounded_d_tag(event, LABEL)?; + // Slug grammar: ^[a-z0-9][a-z0-9_-]{0,63}$ let bytes = d.as_bytes(); if !bytes[0].is_ascii_lowercase() && !bytes[0].is_ascii_digit() { - return Err( - "persona event `d` tag must start with a lowercase letter or digit".to_string(), - ); + return Err(format!( + "{LABEL} `d` tag must start with a lowercase letter or digit" + )); } if !bytes[1..] .iter() .all(|&b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'_' || b == b'-') { - return Err( - "persona event `d` tag must match [a-z0-9_-] after the first character".to_string(), - ); + return Err(format!( + "{LABEL} `d` tag must match [a-z0-9_-] after the first character" + )); + } + Ok(()) +} + +/// Validate the envelope of a kind:30178 team-catalog event. +/// +/// Enforces the shared-gated `shared`-tag shape ([`validate_shared_tag`]) plus +/// exactly one non-empty, bounded `d` tag. +/// +/// Deliberately NOT the persona slug grammar: a team's `d` tag is its stable +/// local id, which is either a UUID or a built-in identifier such as +/// `builtin-team:welcome` — the colon is not slug-legal, and rewriting ids to +/// fit would break NIP-33 addressing against the team's own kind:30176 head. +fn validate_team_catalog_envelope(event: &Event) -> Result<(), String> { + const LABEL: &str = "team-catalog event"; + validate_shared_tag(event, LABEL)?; + single_bounded_d_tag(event, LABEL)?; + Ok(()) +} + +/// Maximum number of member `a` tags on a kind:30621 project. +/// +/// Counted over raw tags, not distinct coordinates: a duplicate-heavy event +/// naming one coordinate thousands of times would otherwise be bounded only by +/// the relay frame limit (`config.rs`), so the cap must be checked before any +/// set proportional to the tag list is built. +const PROJECT_MEMBER_CAP: usize = 64; + +/// Maximum byte length of a project `name` tag value. +const PROJECT_NAME_MAX_LEN: usize = 256; + +/// Maximum byte length of a project `description` tag value. +const PROJECT_DESCRIPTION_MAX_LEN: usize = 2048; + +/// Maximum byte length of `buzz-channel` and `buzz-visibility` tag values. +/// +/// Both are opaque strings at the relay layer; the bound exists only so an +/// unbounded value cannot ride into storage on a tag ingest does not interpret. +const PROJECT_METADATA_TAG_MAX_LEN: usize = 256; + +/// Metadata tags a project may carry at most once each. +/// +/// Duplicates would make the effective value reader-dependent — one client +/// taking the first, another the last. +const PROJECT_SINGLETON_METADATA_TAGS: [&str; 4] = + ["name", "description", "buzz-channel", "buzz-visibility"]; + +/// The kind segment every project member coordinate must carry: a project groups +/// repository *announcements*, so a coordinate naming any other kind (notably +/// kind:30618 repository state) is malformed. +const PROJECT_MEMBER_KIND_SEGMENT: &str = "30617"; +const _: () = assert!(KIND_GIT_REPO_ANNOUNCEMENT == 30617); + +/// A validation failure from [`validate_project_envelope`] or +/// [`parse_project_member_coordinate`]. +/// +/// Carries the stable NIP-MP rule identifier alongside the human-readable +/// rejection message. The rule ID allows the fixture oracle and any future +/// cross-implementation conformance test to assert *which* rule fired, not just +/// that rejection occurred — an implementation cannot pass a reject fixture by +/// refusing for an unrelated reason. +/// +/// The eight IDs match the `reject_rules` strings in `NIP-MP.fixtures.json` +/// exactly: `d-cardinality`, `d-empty`, `member-cap`, `member-tag-arity`, +/// `member-coordinate-malformed`, `member-duplicate`, `metadata-cardinality`, +/// `metadata-length`. +#[derive(Debug)] +struct ProjectRejection { + /// Stable rule identifier matching the fixture file's `reject_rules` set. + rule: &'static str, + /// Human-readable explanation forwarded to the client's NOTICE/OK message. + message: String, +} + +impl std::fmt::Display for ProjectRejection { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "[{}] {}", self.rule, self.message) + } +} + +impl ProjectRejection { + fn new(rule: &'static str, message: impl Into) -> Self { + Self { + rule, + message: message.into(), + } + } +} + +/// Validate the envelope of a kind:30621 NIP-MP project event. +/// +/// Enforces the structural contract in `docs/nips/NIP-MP.md` — exactly one +/// non-empty `d` tag, at most [`PROJECT_MEMBER_CAP`] member `a` tags each +/// holding a canonical `30617::` +/// coordinate with no duplicates, and bounded metadata. +/// +/// Deliberately absent: any membership authorization. The signer may reference +/// any repository coordinate, including another owner's, because membership +/// grants nothing — push policy reads the repository's own kind:30617 +/// (`api/git/policy.rs`) and never a project. Owner-only replacement comes free +/// from NIP-33 addressing. +/// +/// Duplicates are rejected rather than deduped: a relay cannot rewrite tags +/// inside a signed event without invalidating its id and signature, so the +/// choice is reject or force every consumer to apply a first-wins rule. +fn validate_project_envelope(event: &Event) -> Result<(), ProjectRejection> { + let mut d_tags: Vec<&str> = Vec::new(); + let mut members: Vec<&str> = Vec::new(); + let mut name: Option<&str> = None; + let mut description: Option<&str> = None; + let mut buzz_channel: Option<&str> = None; + let mut buzz_visibility: Option<&str> = None; + let mut singleton_counts = [0usize; PROJECT_SINGLETON_METADATA_TAGS.len()]; + + for tag in event.tags.iter() { + let parts = tag.as_slice(); + let Some(tag_name) = parts.first().map(|s| s.as_str()) else { + continue; + }; + let value = parts.get(1).map(|s| s.as_str()).unwrap_or(""); + match tag_name { + "d" => d_tags.push(value), + "a" => members.push(value), + _ => { + if let Some(i) = PROJECT_SINGLETON_METADATA_TAGS + .iter() + .position(|k| *k == tag_name) + { + singleton_counts[i] += 1; + match tag_name { + "name" => name = Some(value), + "description" => description = Some(value), + "buzz-channel" => buzz_channel = Some(value), + "buzz-visibility" => buzz_visibility = Some(value), + _ => {} + } + } + } + } + } + + // `d-cardinality` / `d-empty`: under NIP-33 a missing `d` is treated as + // empty, which collapses every such project into the `(pubkey, 30621, "")` + // slot where unrelated projects silently overwrite each other. Several `d` + // tags make the address reader-dependent. Length is bounded by the generic + // `D_TAG_MAX_LEN` check the ingest pipeline already applies. + if d_tags.len() != 1 { + return Err(ProjectRejection::new( + "d-cardinality", + format!( + "project event must have exactly one `d` tag (got {})", + d_tags.len() + ), + )); + } + if d_tags[0].is_empty() { + return Err(ProjectRejection::new( + "d-empty", + "project event `d` tag must not be empty", + )); + } + + // `member-cap` before `member-coordinate-malformed` and `member-duplicate`: + // refuse on count before doing per-tag work. + if members.len() > PROJECT_MEMBER_CAP { + return Err(ProjectRejection::new( + "member-cap", + format!( + "project event must have at most {PROJECT_MEMBER_CAP} member `a` tags (got {})", + members.len() + ), + )); + } + // `member-tag-arity`: every member `a` tag has exactly 2 or 3 elements per + // NIP-01's `a` tag grammar. A one-element tag names no coordinate; a fourth + // element has no defined meaning, and accepting it would let a writer park + // unbounded unvalidated data in a position no consumer reads. + for tag in event.tags.iter() { + let parts = tag.as_slice(); + if parts.first().map(|s| s.as_str()) == Some("a") && !(2..=3).contains(&parts.len()) { + return Err(ProjectRejection::new( + "member-tag-arity", + format!( + "project event member `a` tag must have exactly 2 or 3 elements (got {})", + parts.len() + ), + )); + } + } + let mut seen = std::collections::HashSet::with_capacity(members.len()); + for member in &members { + parse_project_member_coordinate(member)?; + if !seen.insert(*member) { + return Err(ProjectRejection::new( + "member-duplicate", + format!("project event has duplicate member coordinate {member:?}"), + )); + } + } + + for (i, count) in singleton_counts.iter().enumerate() { + if *count > 1 { + return Err(ProjectRejection::new( + "metadata-cardinality", + format!( + "project event must have at most one `{}` tag (got {count})", + PROJECT_SINGLETON_METADATA_TAGS[i] + ), + )); + } + } + if let Some(name) = name { + if name.len() > PROJECT_NAME_MAX_LEN { + return Err(ProjectRejection::new( + "metadata-length", + format!( + "project event `name` tag too long ({} bytes, max {PROJECT_NAME_MAX_LEN})", + name.len() + ), + )); + } + } + if let Some(description) = description { + if description.len() > PROJECT_DESCRIPTION_MAX_LEN { + return Err(ProjectRejection::new( + "metadata-length", + format!( + "project event `description` tag too long ({} bytes, max {PROJECT_DESCRIPTION_MAX_LEN})", + description.len() + ), + )); + } + } + if let Some(buzz_channel) = buzz_channel { + if buzz_channel.len() > PROJECT_METADATA_TAG_MAX_LEN { + return Err(ProjectRejection::new( + "metadata-length", + format!( + "project event `buzz-channel` tag too long ({} bytes, max {PROJECT_METADATA_TAG_MAX_LEN})", + buzz_channel.len() + ), + )); + } + } + if let Some(buzz_visibility) = buzz_visibility { + if buzz_visibility.len() > PROJECT_METADATA_TAG_MAX_LEN { + return Err(ProjectRejection::new( + "metadata-length", + format!( + "project event `buzz-visibility` tag too long ({} bytes, max {PROJECT_METADATA_TAG_MAX_LEN})", + buzz_visibility.len() + ), + )); + } + } + Ok(()) +} + +/// Check that `coordinate` is a canonical repository-announcement address. +/// +/// Splits on the first two colons only, matching how NIP-09 deletion handling +/// parses coordinates (`side_effects.rs`), so a repository whose `d` tag +/// contains a colon stays addressable and a project can never disagree with a +/// deletion about where the `d` value begins. +fn parse_project_member_coordinate(coordinate: &str) -> Result<(), ProjectRejection> { + let malformed = || { + ProjectRejection::new( + "member-coordinate-malformed", + format!( + "project event member `a` tag must be \ + `{PROJECT_MEMBER_KIND_SEGMENT}::` (got {coordinate:?})" + ), + ) + }; + let mut segments = coordinate.splitn(3, ':'); + let (Some(kind), Some(owner), Some(repo_d)) = + (segments.next(), segments.next(), segments.next()) + else { + return Err(malformed()); + }; + if kind != PROJECT_MEMBER_KIND_SEGMENT { + return Err(malformed()); + } + // Lowercase-only: `#a` filter matching is byte-exact, so an uppercase-owner + // head would be invisible to the lowercase-coordinate queries readers issue. + if owner.len() != 64 + || !owner + .bytes() + .all(|b| b.is_ascii_hexdigit() && !b.is_ascii_uppercase()) + { + return Err(malformed()); + } + if repo_d.is_empty() { + return Err(malformed()); } Ok(()) } @@ -2070,6 +2410,16 @@ async fn ingest_event_inner( .map_err(|e| IngestError::Rejected(format!("invalid: {e}")))?; } + if kind_u32 == KIND_TEAM_CATALOG { + validate_team_catalog_envelope(&event) + .map_err(|e| IngestError::Rejected(format!("invalid: {e}")))?; + } + + if kind_u32 == KIND_PROJECT { + validate_project_envelope(&event) + .map_err(|e| IngestError::Rejected(format!("invalid: {e}")))?; + } + // Track pre-created channel UUID for compensation on insert failure. let mut pre_created_channel: Option = None; @@ -2481,7 +2831,12 @@ async fn ingest_event_inner( crate::handlers::side_effects::handle_side_effects(tenant, kind_u32, &event, state) .await { - warn!(event_id = %event_id_hex, kind = kind_u32, "Side effect failed: {e}"); + // error!, not warn!: the event was accepted but its side effects + // (channel creation, git repo seeding, …) did not run — the relay + // is now in a state the client believes it isn't. Production runs + // RUST_LOG=error, so warn! made these failures invisible during + // the #3527 triage. + error!(event_id = %event_id_hex, kind = kind_u32, "Side effect failed: {e}"); } } @@ -3592,6 +3947,24 @@ mod tests { assert!(err.contains("`d` tag"), "got: {err}"); } + #[test] + fn persona_envelope_rejects_valueless_d_tag() { + // A lone ["d"] carries no value; it must fail as a missing value, not + // be skipped as though the event had no `d` tag at all. + let ev = make_persona(&[&["d"]]); + let err = validate_persona_envelope(&ev).unwrap_err(); + assert!(err.contains("must not be empty"), "got: {err}"); + } + + #[test] + fn persona_envelope_rejects_valueless_plus_valued_d_tags() { + // Counting only tags with a value would see one `d` here and accept the + // event, breaking the exactly-one rule. + let ev = make_persona(&[&["d"], &["d", "slug-a"]]); + let err = validate_persona_envelope(&ev).unwrap_err(); + assert!(err.contains("exactly one `d` tag"), "got: {err}"); + } + #[test] fn persona_envelope_rejects_too_long() { let slug = "a".repeat(65); @@ -3718,6 +4091,552 @@ mod tests { ); } + // ─── team-catalog (30178) envelope tests ───────────────────────────────── + + fn make_team_catalog(tags: &[&[&str]]) -> Event { + make_event_with_tags( + KIND_TEAM_CATALOG, + r#"{"v":1,"name":"Team","members":[]}"#, + tags, + ) + } + + #[test] + fn team_catalog_envelope_accepts_uuid_d_tag() { + let ev = make_team_catalog(&[&["d", "0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0"]]); + assert!(validate_team_catalog_envelope(&ev).is_ok()); + } + + #[test] + fn team_catalog_envelope_accepts_builtin_colon_d_tag() { + // Built-in team ids carry a colon (`builtin-team:welcome`), which the + // persona slug grammar forbids. The catalog `d` tag must accept them so + // a built-in team can be shared under its real local id. + let ev = make_team_catalog(&[&["d", "builtin-team:welcome"]]); + assert!(validate_team_catalog_envelope(&ev).is_ok()); + } + + #[test] + fn team_catalog_envelope_accepts_shared_true() { + let ev = make_team_catalog(&[&["d", "team-1"], &["shared", "true"]]); + assert!(validate_team_catalog_envelope(&ev).is_ok()); + } + + #[test] + fn team_catalog_envelope_rejects_missing_d_tag() { + let ev = make_team_catalog(&[]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("exactly one `d` tag"), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_rejects_empty_d_tag() { + // An empty d-tag collapses every team into the (pubkey, 30178, "") slot. + let ev = make_team_catalog(&[&["d", ""]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("must not be empty"), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_rejects_duplicate_d_tags() { + let ev = make_team_catalog(&[&["d", "team-1"], &["d", "team-2"]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("exactly one `d` tag"), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_rejects_valueless_d_tag() { + // A lone ["d"] carries no value; it must fail as a missing value, not + // be skipped as though the event had no `d` tag at all. + let ev = make_team_catalog(&[&["d"]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("must not be empty"), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_rejects_valueless_plus_valued_d_tags() { + // Counting only tags with a value would see one `d` here and accept the + // event. A NIP-33 consumer that reads ["d"] as an empty-valued first + // `d` tag would then address this event at "" where we address it at + // "team-1". + let ev = make_team_catalog(&[&["d"], &["d", "team-1"]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("exactly one `d` tag"), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_bounds_d_tag_by_chars_not_bytes() { + // 64 multi-byte characters is 192 bytes; the documented bound is + // characters, so this must be accepted. + let d = "é".repeat(64); + assert!(d.len() > 64, "fixture must exceed the bound in bytes"); + let ev = make_team_catalog(&[&["d", &d]]); + assert!(validate_team_catalog_envelope(&ev).is_ok()); + } + + #[test] + fn team_catalog_envelope_rejects_too_long_d_tag() { + let d = "a".repeat(65); + let ev = make_team_catalog(&[&["d", &d]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("too long"), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_accepts_max_length_d_tag() { + let d = "a".repeat(64); + let ev = make_team_catalog(&[&["d", &d]]); + assert!(validate_team_catalog_envelope(&ev).is_ok()); + } + + #[test] + fn team_catalog_envelope_rejects_whitespace_d_tag() { + // A newline in the d-tag would break the NIP-33 coordinate and any + // line-oriented log consumer. + let ev = make_team_catalog(&[&["d", "team\n1"]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("control characters"), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_rejects_shared_false() { + let ev = make_team_catalog(&[&["d", "team-1"], &["shared", "false"]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("\"true\""), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_rejects_shared_three_elements() { + // Same exact-shape rule as personas: a three-element tag would match the + // SQL containment clause `tags @> '[["shared","true"]]'` as a superset. + let ev = make_team_catalog(&[&["d", "team-1"], &["shared", "true", "extra"]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("[\"shared\",\"true\"]"), "got: {err}"); + } + + #[test] + fn team_catalog_envelope_rejects_duplicate_shared_tags() { + let ev = make_team_catalog(&[&["d", "team-1"], &["shared", "true"], &["shared", "true"]]); + let err = validate_team_catalog_envelope(&ev).unwrap_err(); + assert!(err.contains("at most one"), "got: {err}"); + } + + #[test] + fn team_catalog_is_in_scope_allowlist() { + let dummy = make_dummy_event(); + assert_eq!( + required_scope_for_kind(KIND_TEAM_CATALOG, &dummy).unwrap(), + Scope::UsersWrite, + ); + } + + #[test] + fn team_catalog_is_global_only() { + assert!(is_global_only_kind(KIND_TEAM_CATALOG)); + assert!(!requires_h_channel_scope(KIND_TEAM_CATALOG)); + } + + // ─── project (NIP-MP kind:30621) envelope tests ────────────────────────── + + const OWNER_A: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + const OWNER_B: &str = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; + + fn make_project(tags: &[&[&str]]) -> Event { + make_event_with_tags(KIND_PROJECT, "", tags) + } + + fn member_coord(owner: &str, repo_d: &str) -> String { + format!("30617:{owner}:{repo_d}") + } + + #[test] + fn project_envelope_accepts_minimal() { + let ev = make_project(&[&["d", "platform"]]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + #[test] + fn project_envelope_accepts_full_cross_owner_membership() { + // The motivating case: one project spanning two owners' repositories. + let a = member_coord(OWNER_A, "buzz"); + let b = member_coord(OWNER_B, "buzz-infra"); + let ev = make_project(&[ + &["d", "platform"], + &["name", "Platform"], + &["description", "Relay, desktop, and mobile."], + &["a", &a], + &["a", &b], + &["buzz-channel", "3580ca9b-47b4-4af9-b22a-1068778f26c6"], + &["buzz-visibility", "listed"], + ]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + #[test] + fn project_envelope_accepts_zero_members() { + // Legal at the protocol layer: the natural state after removing a final + // member. The create UI requires >= 1; the relay must not. + let ev = make_project(&[&["d", "empty"], &["name", "Empty"]]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + #[test] + fn project_envelope_accepts_same_repo_d_under_two_owners() { + // The NIP-34 fork case. Identity is the whole coordinate, so these are + // two distinct members, not a duplicate. + let a = member_coord(OWNER_A, "buzz"); + let b = member_coord(OWNER_B, "buzz"); + let ev = make_project(&[&["d", "forks"], &["a", &a], &["a", &b]]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + #[test] + fn project_envelope_accepts_member_repo_d_containing_colon() { + // Coordinates split on the first two colons only, matching NIP-09 + // deletion parsing, so a colon-bearing repository `d` stays addressable. + let coord = member_coord(OWNER_A, "group:repo"); + let ev = make_project(&[&["d", "external"], &["a", &coord]]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + #[test] + fn project_envelope_accepts_member_cap_boundary() { + let coords: Vec = (0..PROJECT_MEMBER_CAP) + .map(|i| member_coord(OWNER_A, &format!("repo-{i}"))) + .collect(); + let mut tags: Vec> = vec![vec!["d", "wide"]]; + tags.extend(coords.iter().map(|c| vec!["a", c.as_str()])); + let tag_refs: Vec<&[&str]> = tags.iter().map(|t| t.as_slice()).collect(); + let ev = make_project(&tag_refs); + assert!( + validate_project_envelope(&ev).is_ok(), + "exactly {PROJECT_MEMBER_CAP} members must be accepted" + ); + } + + #[test] + fn project_envelope_ignores_unknown_tags() { + // Forward compatibility: a newer writer's extra metadata must not + // invalidate the event for this relay. + let ev = make_project(&[&["d", "platform"], &["future-field", "whatever"]]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + #[test] + fn project_envelope_rejects_missing_d_tag() { + let ev = make_project(&[&["name", "No Identity"]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("exactly one `d` tag"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_multiple_d_tags() { + let ev = make_project(&[&["d", "one"], &["d", "two"]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("exactly one `d` tag"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_empty_d_tag() { + // An empty `d` collapses every such project into the (pubkey, 30621, "") + // slot, where unrelated projects silently overwrite each other. + let ev = make_project(&[&["d", ""]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!(err.to_string().contains("must not be empty"), "got: {err}"); + } + + #[test] + fn project_envelope_rejects_valueless_d_tag() { + // `["d"]` with no value is treated as empty, not as absent. + let ev = make_project(&[&["d"]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!(err.to_string().contains("must not be empty"), "got: {err}"); + } + + #[test] + fn project_envelope_rejects_duplicate_member_coordinate() { + let coord = member_coord(OWNER_A, "buzz"); + let ev = make_project(&[&["d", "platform"], &["a", &coord], &["a", &coord]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("duplicate member coordinate"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_member_cap_exceeded() { + let coords: Vec = (0..=PROJECT_MEMBER_CAP) + .map(|i| member_coord(OWNER_A, &format!("repo-{i}"))) + .collect(); + let mut tags: Vec> = vec![vec!["d", "wide"]]; + tags.extend(coords.iter().map(|c| vec!["a", c.as_str()])); + let tag_refs: Vec<&[&str]> = tags.iter().map(|t| t.as_slice()).collect(); + let ev = make_project(&tag_refs); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!(err.to_string().contains("at most 64 member"), "got: {err}"); + } + + #[test] + fn project_envelope_rejects_duplicate_heavy_list_on_cap_not_duplicate() { + // The cap counts raw `a` tags, so a duplicate-heavy list is refused on + // count — parse volume is never bounded only by the frame limit. + let coord = member_coord(OWNER_A, "buzz"); + let mut tags: Vec> = vec![vec!["d", "wide"]]; + for _ in 0..=PROJECT_MEMBER_CAP { + tags.push(vec!["a", coord.as_str()]); + } + let tag_refs: Vec<&[&str]> = tags.iter().map(|t| t.as_slice()).collect(); + let ev = make_project(&tag_refs); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("at most 64 member"), + "cap must be evaluated before the duplicate set is built, got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_member_wrong_kind_prefix() { + // kind:30618 is repository *state*; a project groups announcements. + let coord = format!("30618:{OWNER_A}:buzz"); + let ev = make_project(&[&["d", "platform"], &["a", &coord]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("member `a` tag must be"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_member_owner_not_hex() { + let coord = member_coord(&"z".repeat(64), "buzz"); + let ev = make_project(&[&["d", "platform"], &["a", &coord]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("member `a` tag must be"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_member_owner_uppercase_hex() { + // `#a` filter matching is byte-exact: an uppercase-owner head would be + // invisible to the lowercase-coordinate queries every reader issues. + let coord = member_coord(&"A".repeat(64), "buzz"); + let ev = make_project(&[&["d", "platform"], &["a", &coord]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("member `a` tag must be"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_member_owner_wrong_length() { + let coord = member_coord(&"a".repeat(63), "buzz"); + let ev = make_project(&[&["d", "platform"], &["a", &coord]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("member `a` tag must be"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_member_empty_repo_d() { + let coord = member_coord(OWNER_A, ""); + let ev = make_project(&[&["d", "platform"], &["a", &coord]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("member `a` tag must be"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_member_missing_segment() { + let coord = format!("30617:{OWNER_A}"); + let ev = make_project(&[&["d", "platform"], &["a", &coord]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("member `a` tag must be"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_valueless_member_tag() { + // A one-element `a` tag names no coordinate — caught by the arity check + // (rule 4) before the coordinate parse (rule 5) even runs. + let ev = make_project(&[&["d", "platform"], &["a"]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("exactly 2 or 3 elements"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_rejects_duplicate_metadata_tags() { + // Every singleton metadata tag is bounded: a duplicate would make the + // effective value reader-dependent. + for tag_name in PROJECT_SINGLETON_METADATA_TAGS { + let ev = make_project(&[&["d", "platform"], &[tag_name, "x"], &[tag_name, "y"]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string() + .contains(&format!("at most one `{tag_name}` tag")), + "duplicate `{tag_name}` must be rejected, got: {err}" + ); + } + } + + #[test] + fn project_envelope_rejects_name_too_long() { + let name = "x".repeat(PROJECT_NAME_MAX_LEN + 1); + let ev = make_project(&[&["d", "platform"], &["name", &name]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("`name` tag too long"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_accepts_name_at_max_length() { + let name = "x".repeat(PROJECT_NAME_MAX_LEN); + let ev = make_project(&[&["d", "platform"], &["name", &name]]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + #[test] + fn project_envelope_rejects_description_too_long() { + let description = "x".repeat(PROJECT_DESCRIPTION_MAX_LEN + 1); + let ev = make_project(&[&["d", "platform"], &["description", &description]]); + let err = validate_project_envelope(&ev).unwrap_err(); + assert!( + err.to_string().contains("`description` tag too long"), + "got: {err}" + ); + } + + #[test] + fn project_envelope_accepts_description_at_max_length() { + let description = "x".repeat(PROJECT_DESCRIPTION_MAX_LEN); + let ev = make_project(&[&["d", "platform"], &["description", &description]]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + /// Membership is an assertion, not a permission grant: the relay must accept + /// a project naming a repository the signer does not own. Cross-owner + /// grouping is the entire point of the kind, and it is safe precisely because + /// membership confers nothing. + #[test] + fn project_envelope_accepts_member_owned_by_another_pubkey() { + let stranger = member_coord(OWNER_B, "not-mine"); + let ev = make_project(&[&["d", "collection"], &["a", &stranger]]); + assert!(validate_project_envelope(&ev).is_ok()); + } + + #[test] + fn project_is_in_scope_allowlist() { + let dummy = make_dummy_event(); + assert_eq!( + required_scope_for_kind(KIND_PROJECT, &dummy).unwrap(), + Scope::ReposWrite, + "a project is repository metadata — same scope as announcing a repo" + ); + } + + #[test] + fn project_is_global_only() { + // `buzz-channel` is a metadata reference, not a routing directive. + assert!(is_global_only_kind(KIND_PROJECT)); + assert!(!requires_h_channel_scope(KIND_PROJECT)); + } + + #[test] + fn project_is_parameterized_replaceable() { + // Owner-only editing comes free from NIP-33 addressing: replacement is + // keyed by (pubkey, kind, d), so one signer can never overwrite another's + // project. No relay-side permission check exists or is needed. + assert!(is_parameterized_replaceable(KIND_PROJECT)); + } + + /// Drive every case in the shared NIP-MP fixture file against + /// `validate_project_envelope`. All 11 accept cases must pass; all 20 + /// reject cases must return an error whose rule is in the case's allowed + /// `reject_rules` set — an implementation cannot pass by rejecting for an + /// unrelated reason. This is the machine-readable oracle the spec promises. + #[test] + fn project_envelope_validates_all_shared_fixtures() { + #[derive(serde::Deserialize)] + struct FixtureFile { + cases: Vec, + } + #[derive(serde::Deserialize)] + struct Case { + name: String, + expect: String, + #[serde(default)] + reject_rules: Vec, + template: Template, + } + #[derive(serde::Deserialize)] + struct Template { + content: String, + tags: Vec>, + } + + let raw = include_str!("../../../../docs/nips/NIP-MP.fixtures.json"); + let file: FixtureFile = serde_json::from_str(raw).expect("fixture file must parse"); + + for case in &file.cases { + let tag_strs: Vec> = case + .template + .tags + .iter() + .map(|t| t.iter().map(|s| s.as_str()).collect()) + .collect(); + let tag_refs: Vec<&[&str]> = tag_strs.iter().map(|t| t.as_slice()).collect(); + let ev = make_event_with_tags(KIND_PROJECT, &case.template.content, &tag_refs); + let result = validate_project_envelope(&ev); + match case.expect.as_str() { + "accept" => assert!( + result.is_ok(), + "fixture {:?} expected accept, got err: {:?}", + case.name, + result.unwrap_err() + ), + "reject" => { + let rejection = match result { + Err(r) => r, + Ok(()) => { + panic!("fixture {:?} expected reject, but was accepted", case.name) + } + }; + assert!( + case.reject_rules.iter().any(|r| r == rejection.rule), + "fixture {:?} fired rule {:?}, which is not in allowed set {:?}", + case.name, + rejection.rule, + case.reject_rules, + ); + } + other => panic!( + "unknown expect value {:?} in fixture {:?}", + other, case.name + ), + } + } + } + // ─── agent_turn_metric envelope tests ──────────────────────────────────── /// Build an event for kind:44200 with the given tags and content. diff --git a/crates/buzz-relay/src/handlers/relay_admin.rs b/crates/buzz-relay/src/handlers/relay_admin.rs index 3f58a9c2aa..3782f2c516 100644 --- a/crates/buzz-relay/src/handlers/relay_admin.rs +++ b/crates/buzz-relay/src/handlers/relay_admin.rs @@ -10,7 +10,7 @@ //! | 9030 | Add member | admin or owner | //! | 9031 | Remove member | admin or owner | //! | 9032 | Change role | owner only | -//! | 9033 | Set workspace profile (icon) | admin or owner | +//! | 9033 | Set workspace profile (icon) | admin or owner; on an open relay whose community has no admin/owner row at all, any authenticated sender (see [`may_set_workspace_profile`]) | use std::sync::Arc; @@ -94,6 +94,35 @@ fn validate_workspace_icon(icon: &str) -> Result<(), String> { Ok(()) } +/// Whether `sender_role` may set the workspace profile (kind:9033). +/// +/// Closed relays (`membership_enforced == true`) require an `admin`/`owner` +/// row in `relay_members` — the enforced roster is the authority. Open relays +/// don't *enforce* the roster, but the data can still exist: startup +/// bootstraps `RELAY_OWNER_PUBKEY` as `owner` regardless of the flag +/// (`main.rs`), as does operator provisioning. So the rule is steward-wins: +/// +/// - a steward (any admin/owner row) exists → admin/owner only, exactly like +/// a closed relay. An open relay with a configured owner keeps its icon +/// owner-controlled instead of last-write-wins for every authenticated key. +/// - genuinely rosterless (e.g. a community created by +/// `ensure_configured_community`, which writes no owner row) → any +/// NIP-42-authenticated sender may set the icon, mirroring how open relays +/// gate every other write. Without this the icon is permanently unsettable: +/// the desktop deliberately shows the icon editor on open relays (see +/// `canEditIcon` in `EditCommunityDialog.tsx`, #2640) and defers to this +/// relay-side check, which used to always say no. +fn may_set_workspace_profile( + sender_role: &str, + membership_enforced: bool, + community_has_steward: bool, +) -> bool { + if !membership_enforced && !community_has_steward { + return true; + } + sender_role == "admin" || sender_role == "owner" +} + /// A relay-admin command failure, carrying the *category* of the failure so /// the ingest seam can map it to the right NIP-01 prefix and HTTP status. /// @@ -230,9 +259,33 @@ async fn execute_relay_admin_command( // kind:9033 — Set workspace profile (icon). Handled before p-tag // extraction: it targets the relay itself, not a member pubkey. if kind == RELAY_ADMIN_SET_WORKSPACE_PROFILE { - if sender_role != "admin" && sender_role != "owner" { + // Steward detection only matters on open relays (closed relays gate on + // the sender's own role either way), so skip the extra query there. + let community_has_steward = if state.config.require_relay_membership { + true + } else { + state + .db + .has_admin_or_owner(tenant.community()) + .await + .map_err(|e| format!("database error: {e}"))? + }; + if !may_set_workspace_profile( + sender_role, + state.config.require_relay_membership, + community_has_steward, + ) { return Err("actor not authorized: must be admin or owner".to_string()); } + if sender_role != "admin" && sender_role != "owner" { + // Rosterless-open-relay admit: 9033 writes no audit row and + // publishes no announcement event (unlike 9030/9031), so this warn + // is the only durable attribution of who changed the icon. + warn!( + sender = %sender_hex, + "workspace profile change admitted without a roster role (open relay, no steward)" + ); + } // Empty or missing icon tag clears the workspace icon. let icon = extract_tag_value(event, "icon").unwrap_or_default(); @@ -562,6 +615,46 @@ mod tests { assert!(validate_workspace_icon("").is_ok()); } + /// Closed relay (membership enforced): only an admin/owner row in + /// `relay_members` may set the workspace profile — a plain member, or a + /// pubkey with no row at all (empty role), must be refused. The steward + /// flag is irrelevant when membership is enforced (call sites pass `true`, + /// but the rule must not depend on it). + #[test] + fn closed_relay_requires_admin_or_owner_for_workspace_profile() { + for steward in [true, false] { + assert!(may_set_workspace_profile("owner", true, steward)); + assert!(may_set_workspace_profile("admin", true, steward)); + assert!(!may_set_workspace_profile("member", true, steward)); + assert!(!may_set_workspace_profile("", true, steward)); + } + } + + /// Open relay with a steward: startup bootstraps `RELAY_OWNER_PUBKEY` as + /// `owner` regardless of `require_relay_membership`, so an open relay's + /// community can hold admin/owner rows. When one exists, the icon stays + /// steward-only — the fix must not widen an owner-controlled icon to + /// every authenticated key. + #[test] + fn open_relay_with_steward_keeps_workspace_profile_steward_only() { + assert!(may_set_workspace_profile("owner", false, true)); + assert!(may_set_workspace_profile("admin", false, true)); + assert!(!may_set_workspace_profile("member", false, true)); + assert!(!may_set_workspace_profile("", false, true)); + } + + /// Open relay, genuinely rosterless (no admin/owner row anywhere): any + /// authenticated sender may set the icon — including the roleless (empty + /// role) case, which is *every* sender there. This is the bug being + /// fixed: the desktop shows the icon editor on open relays (#2640) but + /// the relay refused every 9033. + #[test] + fn rosterless_open_relay_admits_any_authenticated_sender_for_workspace_profile() { + assert!(may_set_workspace_profile("", false, false)); + assert!(may_set_workspace_profile("member", false, false)); + assert!(may_set_workspace_profile("owner", false, false)); + } + #[test] fn workspace_icon_https_ok() { assert!(validate_workspace_icon("https://example.com/icon.png").is_ok()); @@ -591,4 +684,216 @@ mod tests { let long_data = format!("data:image/png;base64,{}", "A".repeat(98_304)); assert!(validate_workspace_icon(&long_data).is_err()); } + + // ─── Call-site integration: the 9033 gate wired to real config + DB ──── + // + // The unit tests above pin `may_set_workspace_profile`'s truth table, but + // not its wiring: mutation-testing showed that inverting + // `state.config.require_relay_membership` at the call site — an exact + // inversion of the security contract — survives the default suite. These + // tests drive `handle_relay_admin_event` with a real `AppState` against + // Postgres, on both relay modes, so the wiring itself is pinned. Selected + // explicitly in CI's Backend Integration job; requires local Postgres + // (and hard-fails rather than skipping when it is unreachable). + + const TEST_DB_URL: &str = "postgres://buzz:buzz_dev@localhost:5432/buzz"; // sadscan:disable np.postgres.1 + + /// Build a real `AppState` + tenant for a fresh community on `host`, with + /// `require_relay_membership` set as given. Mirrors + /// `api::invites::tests::invite_test_state`. + async fn workspace_profile_test_state( + host: &str, + require_relay_membership: bool, + ) -> (Arc, TenantContext) { + let mut config = crate::config::Config::from_env().expect("config from env"); + let database_url = std::env::var("BUZZ_TEST_DATABASE_URL") + .or_else(|_| std::env::var("DATABASE_URL")) + .unwrap_or_else(|_| TEST_DB_URL.to_string()); + config.database_url = database_url.clone(); + config.redis_url = "redis://127.0.0.1:1".to_string(); + config.relay_url = format!("wss://{host}"); + config.require_relay_membership = require_relay_membership; + + let pool = sqlx::PgPool::connect(&database_url) + .await + .expect("requires reachable Postgres"); + let db = buzz_db::Db::from_pool(pool.clone()); + let record = db + .ensure_configured_community(host) + .await + .expect("ensure community"); + let tenant = TenantContext::resolved(record.id, host); + + let redis_pool = deadpool_redis::Config::from_url(&config.redis_url) + .create_pool(Some(deadpool_redis::Runtime::Tokio1)) + .expect("redis pool config"); + let pubsub = Arc::new( + buzz_pubsub::PubSubManager::new(&config.redis_url, redis_pool.clone()) + .await + .expect("pubsub manager"), + ); + let audit = buzz_audit::AuditService::new(pool.clone()); + let auth = buzz_auth::AuthService::new(config.auth.clone()); + let search = buzz_search::SearchService::new(pool.clone()); + let workflow_engine = Arc::new(buzz_workflow::WorkflowEngine::new( + db.clone(), + buzz_workflow::WorkflowConfig::default(), + )); + let media_storage = buzz_media::MediaStorage::new(&config.media).expect("media storage"); + let (state, _audit_shutdown) = AppState::new( + config, + db, + redis_pool, + audit, + pubsub, + auth, + search, + workflow_engine, + Keys::generate(), + media_storage, + ); + (Arc::new(state), tenant) + } + + /// Sign a fresh kind:9033 with `icon` and run it through the real + /// admission + command path. + async fn submit_9033( + state: &Arc, + tenant: &TenantContext, + keys: &Keys, + icon: &str, + ) -> Result<(), RelayAdminError> { + let event = EventBuilder::new(Kind::Custom(9033), "") + .tags(vec![Tag::parse(["icon", icon]).expect("icon tag")]) + .sign_with_keys(keys) + .expect("sign 9033"); + handle_relay_admin_event(tenant, state, &event).await + } + + async fn stored_icon(state: &Arc, tenant: &TenantContext) -> Option { + state + .db + .get_community_icon(tenant.community()) + .await + .expect("read icon") + } + + /// Open relay (`require_relay_membership = false`): a rosterless + /// community admits any authenticated sender, but the moment a steward + /// (admin/owner row) exists the gate reverts to steward-only. + /// + /// Discriminating: fails if the call site inverts or drops + /// `require_relay_membership`, or stops consulting `has_admin_or_owner`. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn open_relay_9033_admits_roleless_only_until_a_steward_exists() { + let host = format!("icon-gate-open-{}.example", uuid::Uuid::new_v4().simple()); + let (state, tenant) = workspace_profile_test_state(&host, false).await; + let roleless = Keys::generate(); + let owner = Keys::generate(); + + // Rosterless: the roleless sender may set the icon. + submit_9033(&state, &tenant, &roleless, "https://example.com/open.png") + .await + .expect("rosterless open relay must admit an authenticated sender"); + assert_eq!( + stored_icon(&state, &tenant).await.as_deref(), + Some("https://example.com/open.png"), + "icon must actually be stored" + ); + + // Seed a steward — the same roleless sender must now be refused, and + // the previously stored icon must survive the refused attempt. + state + .db + .add_relay_member( + tenant.community(), + &owner.public_key().to_hex(), + "owner", + None, + ) + .await + .expect("seed owner"); + let refused = submit_9033(&state, &tenant, &roleless, "https://evil.example/pwn.png").await; + assert_eq!( + refused, + Err(RelayAdminError::Rejected( + "actor not authorized: must be admin or owner".to_string() + )), + "an open relay with a steward must refuse a roleless sender" + ); + assert_eq!( + stored_icon(&state, &tenant).await.as_deref(), + Some("https://example.com/open.png"), + "refused attempt must not mutate the icon" + ); + + // The steward still can. + submit_9033(&state, &tenant, &owner, "https://example.com/owner.png") + .await + .expect("the steward must retain icon control"); + assert_eq!( + stored_icon(&state, &tenant).await.as_deref(), + Some("https://example.com/owner.png") + ); + } + + /// Closed relay (`require_relay_membership = true`): admin/owner only — + /// a plain member and a roleless key are refused even though the + /// community also *looks* rosterless-then-stewarded to the open-relay + /// branch. Together with the open-relay test this kills the inverted-flag + /// mutant: no assignment of the flag satisfies both. + #[tokio::test] + #[ignore = "requires Postgres"] + async fn closed_relay_9033_still_requires_admin_or_owner() { + let host = format!("icon-gate-closed-{}.example", uuid::Uuid::new_v4().simple()); + let (state, tenant) = workspace_profile_test_state(&host, true).await; + let roleless = Keys::generate(); + let member = Keys::generate(); + let admin = Keys::generate(); + state + .db + .add_relay_member( + tenant.community(), + &member.public_key().to_hex(), + "member", + None, + ) + .await + .expect("seed member"); + state + .db + .add_relay_member( + tenant.community(), + &admin.public_key().to_hex(), + "admin", + None, + ) + .await + .expect("seed admin"); + + for (keys, label) in [(&roleless, "roleless"), (&member, "member")] { + let refused = submit_9033(&state, &tenant, keys, "https://evil.example/pwn.png").await; + assert_eq!( + refused, + Err(RelayAdminError::Rejected( + "actor not authorized: must be admin or owner".to_string() + )), + "closed relay must refuse a {label} sender" + ); + } + assert_eq!( + stored_icon(&state, &tenant).await, + None, + "refused attempts must not set an icon" + ); + + submit_9033(&state, &tenant, &admin, "https://example.com/closed.png") + .await + .expect("closed-relay admin must set the icon"); + assert_eq!( + stored_icon(&state, &tenant).await.as_deref(), + Some("https://example.com/closed.png") + ); + } } diff --git a/crates/buzz-relay/src/handlers/req.rs b/crates/buzz-relay/src/handlers/req.rs index d3ddd3e5d3..2aed12cd7f 100644 --- a/crates/buzz-relay/src/handlers/req.rs +++ b/crates/buzz-relay/src/handlers/req.rs @@ -7,8 +7,8 @@ use tracing::{debug, warn}; use buzz_core::filter::filters_match; use buzz_core::kind::{ - is_unshared_persona_event, AUTHOR_ONLY_KINDS, KIND_AGENT_ENGRAM, KIND_AGENT_TURN_METRIC, - KIND_DM_VISIBILITY, KIND_PERSONA, P_GATED_KINDS, RESULT_GATED_KINDS, + is_unshared_gated_event, AUTHOR_ONLY_KINDS, KIND_AGENT_ENGRAM, KIND_AGENT_TURN_METRIC, + KIND_DM_VISIBILITY, P_GATED_KINDS, RESULT_GATED_KINDS, SHARED_GATED_KINDS, }; use buzz_core::tenant::TenantContext; use buzz_db::EventQuery; @@ -22,7 +22,6 @@ use crate::connection::{AuthState, ConnectionState}; use crate::protocol::RelayMessage; use crate::state::AppState; -const MAX_HISTORICAL_LIMIT: i64 = 2_000; const MAX_SUBSCRIPTIONS: usize = 1024; /// Maximum `query_events` calls in flight per multi-filter REQ / bridge query. @@ -290,11 +289,11 @@ pub async fn handle_req( let mut params = filter_to_query_params(filter, per_filter_channel, conn.tenant.community()); apply_access_scope_to_query(&mut params, per_filter_channel, &accessible_channels); - // Persona visibility pushdown: set reader bytes so query_events appends - // the SQL visibility clause before ORDER/LIMIT, preventing newer private - // personas from starving older shared ones off the page. - if filter_can_match_persona_shared_kinds(filter) { - params.persona_reader = Some(pubkey_bytes.clone()); + // Shared-gated visibility pushdown: set reader bytes so query_events + // appends the SQL visibility clause before ORDER/LIMIT, preventing + // newer private events from starving older shared ones off the page. + if filter_can_match_shared_gated_kinds(filter) { + params.shared_gated_reader = Some(pubkey_bytes.clone()); } (idx, per_filter_channel, params) }) @@ -310,7 +309,7 @@ pub async fn handle_req( |(idx, per_filter_channel, params)| { let db = db.clone(); async move { - let filter_events = db.query_events(¶ms).await; + let filter_events = db.query_events_routed("req_historical", ¶ms).await; (idx, per_filter_channel, filter_events) } }, @@ -416,10 +415,24 @@ pub async fn handle_req( ); } -/// Handle a NIP-50 search REQ: query Postgres FTS, fetch full events, deliver results, EOSE. -/// Search subscriptions are one-shot — no persistent subscription is registered. +/// FTS candidate hits fetched per page. Pages are always full regardless of +/// the requested limit — post-filtering discards an unpredictable share of +/// hits, so the scan fetches candidates in full pages rather than sizing +/// pages to the request. +const SEARCH_PAGE_SIZE: u32 = 100; + /// Maximum FTS pages to fetch per filter (prevents unbounded loops). -const MAX_SEARCH_PAGES: u32 = 10; +/// +/// Derived from the advertised page ceiling rather than fixed: the scan +/// budget is a resource policy — at most one advertised page ceiling's worth +/// of candidates per filter — and deriving it keeps the budget tracking the +/// ceiling if the ceiling ever moves. This bounds candidates *scanned*, not +/// events *emitted*: post-filtering (NIP-01 match, channel access, reader +/// visibility, dedup) can discard any number of candidates, so a result +/// smaller than the requested limit remains possible and is not a NIP-11 +/// violation — `max_limit` promises a clamp on the request, not a count in +/// the response. +const MAX_SEARCH_PAGES: u32 = (buzz_db::DEFAULT_MAX_PAGE_LIMIT as u32).div_ceil(SEARCH_PAGE_SIZE); /// Resolve request-local channel access, repairing a stale cache-negative. /// @@ -501,6 +514,8 @@ pub(crate) fn build_search_channel_scope_filter( }) } +/// Handle a NIP-50 search REQ: query Postgres FTS, fetch full events, deliver results, EOSE. +/// Search subscriptions are one-shot — no persistent subscription is registered. #[allow(clippy::too_many_arguments)] async fn handle_search_req( sub_id: &str, @@ -535,8 +550,8 @@ async fn handle_search_req( let limit = filter .limit - .map(|l| (l as u32).min(MAX_HISTORICAL_LIMIT as u32)) - .unwrap_or(MAX_HISTORICAL_LIMIT as u32); + .map(|l| (l as u32).min(buzz_db::DEFAULT_MAX_PAGE_LIMIT as u32)) + .unwrap_or(buzz_db::DEFAULT_MAX_PAGE_LIMIT as u32); if limit == 0 { continue; // NIP-01: limit 0 means "no results from this filter" @@ -583,13 +598,11 @@ async fn handle_search_req( let since = filter.since.map(|s| s.as_secs() as i64); let until = filter.until.map(|u| u.as_secs() as i64); - // Paginate: keep fetching pages until we've emitted `limit` results - // or exhausted the search result set. This ensures post-filtering - // doesn't silently reduce the result count below the requested limit. + // Paginate: keep fetching pages until we've emitted `limit` results or + // exhausted the search result set. Post-filtering discards an unpredictable + // share of each page, so continuing past short yields gives the scan a + // chance — not a guarantee — of filling the requested limit. let mut emitted: u32 = 0; - // Always fetch full pages (100) regardless of limit — post-filtering - // may discard many hits, so we need headroom to fill the requested limit. - let per_page: u32 = 100; for page in 1..=MAX_SEARCH_PAGES { if emitted >= limit { @@ -605,7 +618,7 @@ async fn handle_search_req( since, until, page, - per_page, + per_page: SEARCH_PAGE_SIZE, mode: buzz_search::SearchMode::FullText, }; @@ -617,9 +630,9 @@ async fn handle_search_req( } }; - // A short page is the last page: FTS returns up to `per_page` hits, - // so fewer than that means the result set is exhausted. - let exhausted = search_result.hits.len() < per_page as usize; + // A short page is the last page: FTS returns up to a full page of + // hits, so fewer than that means the result set is exhausted. + let exhausted = search_result.hits.len() < SEARCH_PAGE_SIZE as usize; let page_empty = search_result.hits.is_empty(); let hit_ids: Vec<[u8; 32]> = @@ -629,7 +642,7 @@ async fn handle_search_req( let id_refs: Vec<&[u8]> = hit_ids.iter().map(|b| b.as_slice()).collect(); let events = match state .db - .get_events_by_ids(tenant.community(), &id_refs) + .get_events_by_ids_routed("req_search_hydrate", tenant.community(), &id_refs) .await { Ok(evs) => evs, @@ -878,8 +891,8 @@ fn filter_to_query_params( .and_then(|u| chrono::DateTime::from_timestamp(u.as_secs() as i64, 0)); let limit = filter .limit - .map(|l| (l as i64).min(MAX_HISTORICAL_LIMIT)) - .unwrap_or(MAX_HISTORICAL_LIMIT); + .map(|l| (l as i64).min(buzz_db::DEFAULT_MAX_PAGE_LIMIT)) + .unwrap_or(buzz_db::DEFAULT_MAX_PAGE_LIMIT); // Push author filter into SQL. Single-author uses the indexed `pubkey` column; // multi-author uses the `authors` IN-list pushdown added in the pure-nostr PR. @@ -1137,19 +1150,20 @@ pub(crate) fn filter_can_match_author_only_kinds(filter: &Filter) -> bool { }) } -/// Returns `true` if the filter CAN match kind 30175 (persona) — meaning it -/// either has no `kinds` constraint (wildcard) or explicitly includes 30175. +/// Returns `true` if the filter CAN match any kind in [`SHARED_GATED_KINDS`] — +/// meaning it either has no `kinds` constraint (wildcard) or explicitly includes +/// one of them. /// /// Used by the COUNT handler to force the per-event fallback path, which calls -/// `is_unshared_persona_event` on each row. The fast SQL `count_events()` path +/// `is_unshared_gated_event` on each row. The fast SQL `count_events()` path /// has no per-event access check, so it would over-count foreign unshared -/// persona events — leaking the existence of persona activity even without -/// returning content. -pub(crate) fn filter_can_match_persona_shared_kinds(filter: &Filter) -> bool { - filter - .kinds - .as_ref() - .is_none_or(|ks| ks.iter().any(|k| k.as_u16() as u32 == KIND_PERSONA)) +/// events — leaking the existence of private persona/team-catalog activity even +/// without returning content. +pub(crate) fn filter_can_match_shared_gated_kinds(filter: &Filter) -> bool { + filter.kinds.as_ref().is_none_or(|ks| { + ks.iter() + .any(|k| SHARED_GATED_KINDS.contains(&(k.as_u16() as u32))) + }) } /// Returns `true` if the filter CAN match result-gated kinds — meaning it @@ -1208,8 +1222,9 @@ pub(crate) fn is_author_only_event(event: &nostr::Event, requester_pubkey_bytes: /// /// 1. **Author-only kinds** (`AUTHOR_ONLY_KINDS`, e.g. kind 30300/30350): only /// the author may read their own events. -/// 2. **Persona shared-gate** (kind 30175 without `["shared","true"]`): the -/// event is only visible to the author unless explicitly opted into sharing. +/// 2. **Shared-gate** (`SHARED_GATED_KINDS`, e.g. kind 30175/30178 without +/// `["shared","true"]`): the event is only visible to the author unless +/// explicitly opted into sharing. /// 3. **Result-gated kinds** (kind 44200/30622 etc.): `reader_authorized_for_event` /// carries the per-event ownership check. /// @@ -1223,7 +1238,7 @@ pub(crate) fn event_visible_to_reader(event: &nostr::Event, requester_pubkey_byt if is_author_only_event(event, requester_pubkey_bytes) { return false; } - if is_unshared_persona_event(event, requester_pubkey_bytes) { + if is_unshared_gated_event(event, requester_pubkey_bytes) { return false; } let requester_pubkey_hex = hex::encode(requester_pubkey_bytes); @@ -1416,6 +1431,83 @@ mod tests { ) } + /// NIP-11 `limitation.max_limit` as this relay actually advertises it. + fn advertised_max_limit() -> i64 { + crate::nip11::RelayInfo::build( + None, + None, + false, + crate::config::DEFAULT_MAX_FRAME_BYTES, + None, + ) + .limitation + .expect("limitation") + .max_limit + .expect("max_limit") as i64 + } + + #[test] + fn req_filter_limit_clamps_to_advertised_nip11_max_limit() { + let advertised = advertised_max_limit(); + + let community = buzz_core::tenant::CommunityId::from_uuid(uuid::Uuid::new_v4()); + + // A filter asking for more than the relay advertises is clamped down to + // exactly the advertised ceiling — the NIP-11 document is the promise, + // this is the enforcement. + let greedy = filter_to_query_params( + &Filter::new().limit(advertised as usize * 10), + None, + community, + ); + assert_eq!(greedy.limit, Some(advertised)); + + // A filter with no `limit` gets the same ceiling, not something larger. + let unbounded = filter_to_query_params(&Filter::new(), None, community); + assert_eq!(unbounded.limit, Some(advertised)); + + // Neither sets `max_limit`, so `query_events` applies its own default + // clamp. That default must equal the advertised value too, or the + // clamp above would be undone one layer down. + assert_eq!(greedy.max_limit, None); + assert_eq!(unbounded.max_limit, None); + assert_eq!(buzz_db::DEFAULT_MAX_PAGE_LIMIT, advertised); + + // Under-ceiling requests are honored verbatim. + let modest = filter_to_query_params(&Filter::new().limit(10), None, community); + assert_eq!(modest.limit, Some(10)); + } + + /// The NIP-50 search path clamps its emission target to the advertised + /// ceiling like every other REQ, but the number of candidates it will scan + /// is bounded a second time by the page budget. This pins the resource + /// policy: the budget covers exactly one advertised page ceiling's worth of + /// candidates — no less (a ceiling raise must not silently shrink the scan + /// relative to what clients may request) and no hand-tuned spare (the budget + /// must stay derived, not drift back into a magic number). It deliberately + /// does NOT claim search fills the emitted limit — post-filtering can + /// discard any number of candidates. + #[test] + fn search_scan_capacity_covers_advertised_nip11_max_limit() { + let advertised = advertised_max_limit(); + let capacity = i64::from(MAX_SEARCH_PAGES) * i64::from(SEARCH_PAGE_SIZE); + + assert!( + capacity >= advertised, + "NIP-50 scans at most {capacity} candidates ({MAX_SEARCH_PAGES} pages of \ + {SEARCH_PAGE_SIZE}) but NIP-11 advertises {advertised} — the scan budget \ + no longer covers the advertised ceiling" + ); + + // The budget is derived, not hand-tuned: one page under the derived + // count must be insufficient, or the ceiling could rise without the + // page count following it. + assert!( + capacity - i64::from(SEARCH_PAGE_SIZE) < advertised, + "scan budget has a spare page of slack — derive it from the ceiling" + ); + } + #[test] fn count_fallback_fetches_one_extra_candidate() { let mut query = diff --git a/crates/buzz-relay/src/handlers/side_effects.rs b/crates/buzz-relay/src/handlers/side_effects.rs index 65d04ef0ba..660a55fef3 100644 --- a/crates/buzz-relay/src/handlers/side_effects.rs +++ b/crates/buzz-relay/src/handlers/side_effects.rs @@ -2167,9 +2167,18 @@ async fn handle_a_tag_deletion( }; // Safe cast: NIP-33 kinds are 30000–39999, well within i32. let kind_i32 = k as i32; + // NIP-09 scopes an a-tag deletion to versions at or before the + // deletion's own created_at, so a stale/replayed tombstone can never + // erase a newer replacement head. let deleted = state .db - .soft_delete_by_coordinate(tenant.community(), kind_i32, &pubkey_bytes, d_tag) + .soft_delete_by_coordinate( + tenant.community(), + kind_i32, + &pubkey_bytes, + d_tag, + event.created_at.as_secs() as i64, + ) .await .map_err(|e| { anyhow::anyhow!( diff --git a/crates/buzz-relay/src/main.rs b/crates/buzz-relay/src/main.rs index 3ed820d3c5..799cf9cf60 100644 --- a/crates/buzz-relay/src/main.rs +++ b/crates/buzz-relay/src/main.rs @@ -102,6 +102,7 @@ async fn main() -> anyhow::Result<()> { // spans under the correct service identity. let resource = telemetry::service_resource(); let tracer_init = telemetry::try_init_tracer(resource.clone()); + let otel_enabled = matches!(&tracer_init, telemetry::TracerInit::Enabled(_)); let otel_layer = match &tracer_init { telemetry::TracerInit::Enabled(p) => { use opentelemetry::trace::TracerProvider as _; @@ -109,12 +110,18 @@ async fn main() -> anyhow::Result<()> { } _ => None, }; + let trace_context_lookup = telemetry::TraceContextLookup::default(); + let trace_context_lookup_layer = otel_enabled.then(|| { + trace_context_lookup + .clone() + .with_filter(tracing_subscriber::filter::LevelFilter::OFF) + }); tracing_subscriber::registry() .with( fmt::layer() .json() - .flatten_event(true) + .event_format(trace_context_lookup.json_formatter(otel_enabled)) .with_filter(log_env_filter(std::env::var("RUST_LOG").ok().as_deref())), ) .with(otel_layer.map(|layer| { @@ -122,6 +129,7 @@ async fn main() -> anyhow::Result<()> { std::env::var("BUZZ_OTEL_FILTER").ok().as_deref(), )) })) + .with(trace_context_lookup_layer) .init(); // Log any exporter-build failure now that the subscriber is installed. @@ -158,7 +166,9 @@ async fn main() -> anyhow::Result<()> { let db_config = DbConfig { database_url: config.database_url.clone(), read_database_url: config.read_database_url.clone(), + replica_read_max_age_ms: config.replica_read_max_age_ms, max_connections: config.db_pool_size, + read_max_connections: config.db_read_pool_size, ..DbConfig::default() }; let db = Db::new(&db_config).await.map_err(|e| { @@ -166,7 +176,11 @@ async fn main() -> anyhow::Result<()> { anyhow::anyhow!("DB connection failed: {e}") })?; if db.has_read_pool() { - info!("Postgres connected (writer + read replica)"); + info!("Postgres connected (writer + lazy read replica pool)"); + // Reader-down at boot must not crash or block the relay; this warn-only + // ping is the sole boot-time visibility that the replica is unreachable + // (the lazy pool with min_connections=0 dials nothing until first use). + db.spawn_read_pool_boot_ping(); } else { info!("Postgres connected"); } @@ -991,6 +1005,12 @@ async fn main() -> anyhow::Result<()> { metrics::gauge!("buzz_db_replica_fence_open").set(0.0); } } + // Probe liveness, ungated by staleness: how long since + // the probe last committed a heartbeat token. + if let Some(age) = pool_state.db.fence().heartbeat_age() { + metrics::gauge!("buzz_db_replica_heartbeat_age_seconds") + .set(age.as_secs_f64()); + } } let rs = pool_state.redis_pool.status(); diff --git a/crates/buzz-relay/src/nip11.rs b/crates/buzz-relay/src/nip11.rs index a8e397dd21..2575ddd7ba 100644 --- a/crates/buzz-relay/src/nip11.rs +++ b/crates/buzz-relay/src/nip11.rs @@ -89,6 +89,11 @@ pub struct RelayLimitation { /// Canonical `RelayLimitation` advertised by this relay. /// +/// `max_limit` is [`buzz_db::DEFAULT_MAX_PAGE_LIMIT`], the same constant the +/// REQ path clamps filter limits to, so the advertised ceiling and the +/// enforced one cannot drift (see +/// `handlers::req::tests::req_filter_limit_clamps_to_advertised_nip11_max_limit`). +/// /// `auth_required` is always `true`: the REQ, EVENT, and COUNT handlers /// unconditionally reject connections that are not in /// `AuthState::Authenticated`. This is independent of the REST API token @@ -103,7 +108,7 @@ fn relay_limitation(max_message_length: usize) -> RelayLimitation { max_message_length: Some(max_message_length as u64), max_subscriptions: Some(1024), max_filters: Some(10), - max_limit: Some(10_000), + max_limit: Some(buzz_db::DEFAULT_MAX_PAGE_LIMIT as u32), max_subid_length: Some(256), min_pow_difficulty: None, auth_required: true, diff --git a/crates/buzz-relay/src/state.rs b/crates/buzz-relay/src/state.rs index 758c001b96..58a869a995 100644 --- a/crates/buzz-relay/src/state.rs +++ b/crates/buzz-relay/src/state.rs @@ -697,6 +697,7 @@ impl AppState { &config.media.s3_secret_key, &config.media.s3_bucket, &config.media.s3_region, + config.media.s3_addressing_style, ) .expect("media storage was already constructed with this S3 config"); let git_pack_cache = Arc::new( diff --git a/crates/buzz-relay/src/telemetry.rs b/crates/buzz-relay/src/telemetry.rs index 11c6d03512..91bd92f0f3 100644 --- a/crates/buzz-relay/src/telemetry.rs +++ b/crates/buzz-relay/src/telemetry.rs @@ -23,9 +23,157 @@ //! - `OTEL_TRACES_SAMPLER` (default: `parentbased_always_on`) //! - `OTEL_TRACES_SAMPLER_ARG` +use std::{ + fmt, + sync::{Arc, OnceLock}, +}; + +use opentelemetry::trace::{SpanId, TraceContextExt as _, TraceId}; use opentelemetry_otlp::ExporterBuildError; use opentelemetry_sdk::{resource::EnvResourceDetector, trace::SdkTracerProvider, Resource}; -use tracing_subscriber::EnvFilter; +use tracing::{Event, Subscriber}; +use tracing_subscriber::{ + fmt::{ + format::{Format, FormatEvent, FormatFields, Json, Writer}, + FmtContext, + }, + registry::LookupSpan, + EnvFilter, Layer, +}; + +/// Captures the subscriber dispatch used to resolve tracing span IDs to their +/// OpenTelemetry contexts. +#[derive(Clone, Default)] +pub struct TraceContextLookup { + dispatch: Arc>, +} + +impl TraceContextLookup { + /// Build a JSON formatter backed by this subscriber dispatch lookup. + pub fn json_formatter(&self, enabled: bool) -> TraceContextJson { + TraceContextJson { + inner: tracing_subscriber::fmt::format().json().flatten_event(true), + enabled, + context_lookup: self.clone(), + } + } + + fn nearest_otel_context(&self, span_id: &tracing::span::Id) -> Option { + let dispatch = self.dispatch.get()?.upgrade()?; + let registry = dispatch.downcast_ref::()?; + + let context = registry.span(span_id)?.scope().find_map(|span| { + let context = tracing_opentelemetry::get_otel_context(&span.id(), &dispatch)?; + context.span().span_context().is_valid().then_some(context) + }); + context + } +} + +impl Layer for TraceContextLookup { + fn on_register_dispatch(&self, subscriber: &tracing::Dispatch) { + let _ = self.dispatch.set(subscriber.downgrade()); + } +} + +/// JSON event formatter that adds the active OpenTelemetry trace context. +/// +/// Datadog recognizes the OpenTelemetry-standard `trace_id` and `span_id` +/// fields when they are lowercase hexadecimal strings. Events outside a valid +/// OpenTelemetry span retain the standard `tracing-subscriber` JSON format. +pub struct TraceContextJson { + inner: Format, + enabled: bool, + context_lookup: TraceContextLookup, +} + +struct CorrelationWriter<'writer> { + inner: Writer<'writer>, + trace_id: TraceId, + span_id: SpanId, + injected: bool, +} + +impl fmt::Write for CorrelationWriter<'_> { + fn write_str(&mut self, value: &str) -> fmt::Result { + if self.injected { + return self.inner.write_str(value); + } + + let Some(object_start) = value.find('{') else { + return self.inner.write_str(value); + }; + self.inner.write_str(&value[..=object_start])?; + write!( + self.inner, + "\"trace_id\":\"{}\",\"span_id\":\"{}\",", + self.trace_id, self.span_id + )?; + self.injected = true; + self.inner.write_str(&value[object_start + 1..]) + } +} + +impl FormatEvent for TraceContextJson +where + S: Subscriber + for<'lookup> LookupSpan<'lookup>, + N: for<'writer> FormatFields<'writer> + 'static, +{ + fn format_event( + &self, + ctx: &FmtContext<'_, S, N>, + mut writer: Writer<'_>, + event: &Event<'_>, + ) -> fmt::Result { + if !self.enabled { + return self.inner.format_event(ctx, writer, event); + } + + let otel_context = match event.parent() { + Some(span_id) => self.context_lookup.nearest_otel_context(span_id), + None if event.is_contextual() => Some(opentelemetry::Context::current()), + None => None, + }; + let Some(otel_context) = otel_context else { + return self.inner.format_event(ctx, writer, event); + }; + let otel_span = otel_context.span(); + let span_context = otel_span.span_context(); + + if !span_context.is_valid() { + return self.inner.format_event(ctx, writer, event); + } + + let trace_id = span_context.trace_id(); + let span_id = span_context.span_id(); + + // Events may define fields with the correlation names themselves. In + // that uncommon case, overwrite them rather than emitting duplicate + // JSON keys. Preserve the allocation-free streaming path for ordinary + // events. + let fields = event.metadata().fields(); + if fields.field("trace_id").is_some() || fields.field("span_id").is_some() { + let mut json = String::new(); + self.inner + .format_event(ctx, Writer::new(&mut json), event)?; + let mut object: serde_json::Map = + serde_json::from_str(json.trim_end()).map_err(|_| fmt::Error)?; + object.insert("trace_id".into(), trace_id.to_string().into()); + object.insert("span_id".into(), span_id.to_string().into()); + writer.write_str(&serde_json::to_string(&object).map_err(|_| fmt::Error)?)?; + return writeln!(writer); + } + + let mut writer = CorrelationWriter { + inner: writer, + trace_id, + span_id, + injected: false, + }; + self.inner + .format_event(ctx, Writer::new(&mut writer), event) + } +} /// Build the filter for spans exported through OpenTelemetry. /// @@ -122,8 +270,13 @@ fn classify_exporter_result( #[cfg(test)] mod tests { use super::*; - use opentelemetry::KeyValue; - use std::sync::Mutex; + use opentelemetry::{trace::TracerProvider as _, KeyValue}; + use opentelemetry_sdk::trace::InMemorySpanExporter; + use std::{ + io, + sync::{Arc, Mutex}, + }; + use tracing_subscriber::prelude::*; // Env vars are process-global — serialize tests that mutate them to prevent // cross-test races when the suite runs with multiple threads. @@ -137,6 +290,209 @@ mod tests { .map(|(_, v)| v.to_string()) } + #[derive(Clone)] + struct CapturingWriter(Arc>>); + + impl io::Write for CapturingWriter { + fn write(&mut self, bytes: &[u8]) -> io::Result { + self.0.lock().unwrap().extend_from_slice(bytes); + Ok(bytes.len()) + } + + fn flush(&mut self) -> io::Result<()> { + Ok(()) + } + } + + #[test] + fn trace_context_json_correlates_nested_span_logs() { + let output = Arc::new(Mutex::new(Vec::new())); + let output_writer = Arc::clone(&output); + let exporter = InMemorySpanExporter::default(); + let provider = SdkTracerProvider::builder() + .with_simple_exporter(exporter.clone()) + .build(); + let tracer = provider.tracer("trace-context-json-test"); + let context_lookup = TraceContextLookup::default(); + + let subscriber = tracing_subscriber::registry() + .with( + tracing_subscriber::fmt::layer() + .json() + .event_format(context_lookup.json_formatter(true)) + .with_writer(move || CapturingWriter(Arc::clone(&output_writer))) + .with_filter(tracing_subscriber::filter::filter_fn(|metadata| { + metadata.target() != "stdout_filtered" + })), + ) + .with( + tracing_opentelemetry::layer() + .with_tracer(tracer) + .with_filter(tracing_subscriber::filter::filter_fn(|metadata| { + !matches!(metadata.target(), "filtered" | "otel_event_filtered") + })), + ) + .with( + context_lookup + .clone() + .with_filter(tracing_subscriber::filter::LevelFilter::OFF), + ); + + tracing::subscriber::with_default(subscriber, || { + let explicit = tracing::info_span!("explicit"); + let root = tracing::info_span!("root"); + root.in_scope(|| { + tracing::info!(answer = 42, "root event"); + tracing::info!( + trace_id = "event-provided-trace", + span_id = "event-provided-span", + "colliding-fields event" + ); + tracing::info!(parent: &explicit, "explicit-parent event"); + tracing::info!(parent: None, "explicit-root event"); + let child = tracing::info_span!("child"); + child.in_scope(|| tracing::info!("child event")); + + let filtered_child = tracing::info_span!(target: "filtered", "filtered-child"); + filtered_child.in_scope(|| tracing::info!("filtered-child event")); + tracing::info!( + parent: &filtered_child, + "explicit-filtered-child event" + ); + + let stdout_filtered_child = + tracing::info_span!(target: "stdout_filtered", "stdout-filtered-child"); + stdout_filtered_child.in_scope(|| tracing::info!("stdout-filtered-child event")); + + tracing::info!(target: "otel_event_filtered", "otel-filtered event"); + }); + let filtered = tracing::info_span!(target: "filtered", "filtered"); + filtered.in_scope(|| tracing::info!("filtered-span event")); + tracing::info!("unscoped event"); + }); + + provider.force_flush().unwrap(); + let spans = exporter.get_finished_spans().unwrap(); + let root = spans.iter().find(|span| span.name == "root").unwrap(); + let explicit = spans.iter().find(|span| span.name == "explicit").unwrap(); + let child = spans.iter().find(|span| span.name == "child").unwrap(); + let stdout_filtered_child = spans + .iter() + .find(|span| span.name == "stdout-filtered-child") + .unwrap(); + + let bytes = output.lock().unwrap().clone(); + let output = String::from_utf8(bytes).unwrap(); + let lines: Vec<&str> = output.lines().collect(); + let logs: Vec = lines + .iter() + .map(|line| serde_json::from_str(line).unwrap()) + .collect(); + assert_eq!(logs.len(), 11); + + assert_eq!(logs[0]["message"], "root event"); + assert_eq!(logs[0]["answer"], 42); + assert_eq!( + logs[0]["trace_id"], + root.span_context.trace_id().to_string() + ); + assert_eq!(logs[0]["span_id"], root.span_context.span_id().to_string()); + assert_eq!(logs[0]["trace_id"].as_str().unwrap().len(), 32); + assert_eq!(logs[0]["span_id"].as_str().unwrap().len(), 16); + + assert_eq!(logs[1]["message"], "colliding-fields event"); + assert_eq!( + logs[1]["trace_id"], + root.span_context.trace_id().to_string() + ); + assert_eq!(logs[1]["span_id"], root.span_context.span_id().to_string()); + assert_eq!(lines[1].matches("\"trace_id\":").count(), 1); + assert_eq!(lines[1].matches("\"span_id\":").count(), 1); + + assert_eq!(logs[2]["message"], "explicit-parent event"); + assert_eq!( + logs[2]["trace_id"], + explicit.span_context.trace_id().to_string() + ); + assert_eq!( + logs[2]["span_id"], + explicit.span_context.span_id().to_string() + ); + + assert_eq!(logs[3]["message"], "explicit-root event"); + assert!(logs[3].get("trace_id").is_none()); + assert!(logs[3].get("span_id").is_none()); + + assert_eq!(logs[4]["message"], "child event"); + assert_eq!( + logs[4]["trace_id"], + child.span_context.trace_id().to_string() + ); + assert_eq!(logs[4]["span_id"], child.span_context.span_id().to_string()); + assert_eq!(logs[0]["trace_id"], logs[4]["trace_id"]); + + assert_eq!(logs[5]["message"], "filtered-child event"); + assert_eq!( + logs[5]["trace_id"], + root.span_context.trace_id().to_string() + ); + assert_eq!(logs[5]["span_id"], root.span_context.span_id().to_string()); + + assert_eq!(logs[6]["message"], "explicit-filtered-child event"); + assert_eq!( + logs[6]["trace_id"], + root.span_context.trace_id().to_string() + ); + assert_eq!(logs[6]["span_id"], root.span_context.span_id().to_string()); + + assert_eq!(logs[7]["message"], "stdout-filtered-child event"); + assert_eq!( + logs[7]["trace_id"], + stdout_filtered_child.span_context.trace_id().to_string() + ); + assert_eq!( + logs[7]["span_id"], + stdout_filtered_child.span_context.span_id().to_string() + ); + + assert_eq!(logs[8]["message"], "otel-filtered event"); + assert_eq!( + logs[8]["trace_id"], + root.span_context.trace_id().to_string() + ); + assert_eq!(logs[8]["span_id"], root.span_context.span_id().to_string()); + + assert_eq!(logs[9]["message"], "filtered-span event"); + assert!(logs[9].get("trace_id").is_none()); + assert!(logs[9].get("span_id").is_none()); + + assert_eq!(logs[10]["message"], "unscoped event"); + assert!(logs[10].get("trace_id").is_none()); + assert!(logs[10].get("span_id").is_none()); + } + + #[test] + fn trace_context_lookup_does_not_enable_callsites() { + let context_lookup = TraceContextLookup::default(); + let subscriber = tracing_subscriber::registry().with( + context_lookup + .clone() + .with_filter(tracing_subscriber::filter::LevelFilter::OFF), + ); + + tracing::subscriber::with_default(subscriber, || { + assert!(context_lookup + .dispatch + .get() + .and_then(tracing::dispatcher::WeakDispatch::upgrade) + .is_some()); + assert!(!tracing::enabled!( + target: "trace_context_lookup_filter_test", + tracing::Level::ERROR + )); + }); + } + #[test] fn test_service_resource_default_when_env_unset() { let _guard = ENV_LOCK.lock().unwrap(); diff --git a/crates/buzz-sdk/src/builders.rs b/crates/buzz-sdk/src/builders.rs index 8cc9c8650a..9a139f0377 100644 --- a/crates/buzz-sdk/src/builders.rs +++ b/crates/buzz-sdk/src/builders.rs @@ -11,8 +11,8 @@ use buzz_core::{ KIND_GIT_STATUS_CLOSED, KIND_GIT_STATUS_DRAFT, KIND_GIT_STATUS_MERGED, KIND_GIT_STATUS_OPEN, KIND_IA_ARCHIVE_REQUEST, KIND_IA_UNARCHIVE_REQUEST, KIND_MODERATION_BAN, KIND_MODERATION_RESOLVE_REPORT, KIND_MODERATION_TIMEOUT, - KIND_MODERATION_UNBAN, KIND_MODERATION_UNTIMEOUT, KIND_PRESENCE_UPDATE, KIND_USER_STATUS, - KIND_WORKFLOW_DEF, KIND_WORKFLOW_TRIGGER, + KIND_MODERATION_UNBAN, KIND_MODERATION_UNTIMEOUT, KIND_PRESENCE_UPDATE, KIND_PROJECT, + KIND_USER_STATUS, KIND_WORKFLOW_DEF, KIND_WORKFLOW_TRIGGER, }, observer::{ content_looks_like_nip44, OBSERVER_AGENT_TAG, OBSERVER_FRAME_CONTROL, OBSERVER_FRAME_TAG, @@ -1499,12 +1499,7 @@ pub fn build_workflow_delete( author_pubkey: &str, workflow_id: Uuid, ) -> Result { - let pk = check_pubkey_hex(author_pubkey, "author_pubkey")?; - let tags = vec![tag(&[ - "a", - &format!("{}:{pk}:{workflow_id}", KIND_WORKFLOW_DEF), - ])?]; - Ok(EventBuilder::new(Kind::Custom(KIND_DELETION as u16), "").tags(tags)) + build_delete_addressable(KIND_WORKFLOW_DEF, author_pubkey, &workflow_id.to_string()) } /// Build a workflow trigger event (kind 46020). @@ -1838,6 +1833,364 @@ pub fn build_unarchive_identity_request( ) } +// ─── NIP-MP: Multi-repo projects (kind:30621) ──────────────────────────────── +// +// Public surface: +// • `validate_project_envelope` — Layer A protocol validator (8 ingest rules) +// • `build_project_with_tags` — Layer A raw builder (content + tags, no canonicalization) +// • `ProjectMemberCoord` — parsed member coordinate + optional relay hint +// • `build_project` — Layer B writer-policy builder +// • `build_delete_addressable` — generic NIP-09 kind:5 coordinate delete +// +// Byte-length bounds from NIP-MP §Relay Processing: +/// Maximum byte length of a project `d` tag value. +pub const PROJECT_D_MAX_LEN: usize = 1024; +/// Maximum byte length of a project `name` tag value. +pub const PROJECT_NAME_MAX: usize = 256; +/// Maximum byte length of a project `description` tag value. +pub const PROJECT_DESCRIPTION_MAX: usize = 2048; +/// Maximum byte length of a project `buzz-channel` tag value. +pub const PROJECT_CHANNEL_MAX: usize = 256; +/// Maximum byte length of a project `buzz-visibility` tag value. +pub const PROJECT_VISIBILITY_MAX: usize = 256; +/// Maximum number of `a` member tags per project event (checked before dedup). +pub const PROJECT_MEMBER_CAP: usize = 64; + +/// A validated NIP-MP member `a`-tag coordinate with an optional relay hint. +/// +/// Equality and `Hash` are by `coord` only (per spec: duplicate detection ignores hint). +#[derive(Clone, Debug)] +pub struct ProjectMemberCoord { + /// The full `30617::` coordinate string. + pub coord: String, + /// Optional opaque relay hint (third `a`-tag element, never validated by content). + pub hint: Option, +} + +impl PartialEq for ProjectMemberCoord { + fn eq(&self, other: &Self) -> bool { + self.coord == other.coord + } +} + +impl Eq for ProjectMemberCoord {} + +impl std::hash::Hash for ProjectMemberCoord { + fn hash(&self, state: &mut H) { + self.coord.hash(state); + } +} + +impl ProjectMemberCoord { + /// Parse a full `30617::` coordinate string. + /// + /// Accepts an optional relay hint as the third colon-separated element + /// after the split, but the split is always first-two-colons: kind, owner, + /// everything-else-as-repo-d. + /// + /// Rules enforced: + /// - Exactly three segments after splitting on the first two colons + /// - First segment must be the literal string `"30617"` + /// - Second segment must be exactly 64 lowercase hex characters + /// - Third segment (repo-d) must be non-empty + /// - Uppercase owners are rejected (never normalized) + pub fn parse_full(coord: &str) -> Result { + // Split on first two colons only: kind:owner:rest + let mut parts = coord.splitn(3, ':'); + let kind_part = parts.next().unwrap_or(""); + let owner_part = parts.next().unwrap_or(""); + let rest = parts.next().unwrap_or(""); + + if kind_part != "30617" { + return Err(SdkError::InvalidInput(format!( + "member coordinate must start with '30617:' (got kind {kind_part:?})" + ))); + } + if owner_part.len() != 64 || !owner_part.chars().all(|c| c.is_ascii_hexdigit()) { + return Err(SdkError::InvalidInput(format!( + "member owner must be a 64-character hex pubkey (got {owner_part:?})" + ))); + } + // Reject uppercase (spec: lowercase hex required) + if owner_part.chars().any(|c| c.is_ascii_uppercase()) { + return Err(SdkError::InvalidInput( + "member owner hex must be lowercase".into(), + )); + } + if rest.is_empty() { + return Err(SdkError::InvalidInput( + "member coordinate repo-d must not be empty".into(), + )); + } + Ok(ProjectMemberCoord { + coord: format!("30617:{owner_part}:{rest}"), + hint: None, + }) + } + + /// Returns the `a`-tag element slice: `[coord]` or `[coord, hint]`. + pub fn to_tag_parts(&self) -> Vec { + let mut parts = vec!["a".to_string(), self.coord.clone()]; + if let Some(h) = &self.hint { + parts.push(h.clone()); + } + parts + } +} + +/// **Layer A**: Validate a complete kind:30621 envelope against the 8 NIP-MP +/// ingest rules. This is the single source of protocol truth used by both +/// `build_project_with_tags` (raw path) and `build_project` (policy path). +/// +/// Rules enforced (matches relay `buzz-db` ingest logic): +/// 1. `d` cardinality: exactly one `d` tag. +/// 2. `d` value: non-empty, ≤1024 bytes. +/// 3. Member cap: raw count of every `a` tag ≤ 64 (checked **before** per-tag +/// parsing, matching relay rule order). +/// 4. Member tag arity: every `a` tag has 2 or 3 elements (no more, no fewer). +/// 5. Member coordinate grammar: first-two-colons split; kind literal `"30617"`; +/// owner lowercase 64-hex; repo-d non-empty verbatim. +/// 6. Member deduplication: coordinate equality only (hint ignored); any +/// coordinate that appears more than once is a duplicate. +/// 7. Singleton metadata: each of `name`, `description`, `buzz-channel`, +/// `buzz-visibility` appears at most once. +/// 8. Metadata byte lengths: `name` ≤256, `description` ≤2048, +/// `buzz-channel` ≤256, `buzz-visibility` ≤256. +pub fn validate_project_envelope(tags: &[Tag], _content: &str) -> Result<(), SdkError> { + // --- Rule 1 & 2: d tag --- + let d_tags: Vec<&Tag> = tags.iter().filter(|t| tag_name(t) == Some("d")).collect(); + match d_tags.len() { + 0 => { + return Err(SdkError::InvalidInput( + "project must have exactly one 'd' tag (rule: d-cardinality)".into(), + )) + } + 1 => {} + _ => { + return Err(SdkError::InvalidInput( + "project must have exactly one 'd' tag (rule: d-cardinality)".into(), + )) + } + } + let d_val = tag_value(d_tags[0]).unwrap_or(""); + if d_val.is_empty() { + return Err(SdkError::InvalidInput( + "project 'd' tag must not be empty (rule: d-empty)".into(), + )); + } + if d_val.len() > PROJECT_D_MAX_LEN { + return Err(SdkError::InvalidInput(format!( + "project 'd' tag exceeds {PROJECT_D_MAX_LEN} bytes (rule: d-empty)" + ))); + } + + let a_tags: Vec<&Tag> = tags.iter().filter(|t| tag_name(t) == Some("a")).collect(); + + // --- Rule 3: member cap (checked before per-tag parsing, matching relay rule order) --- + if a_tags.len() > PROJECT_MEMBER_CAP { + return Err(SdkError::InvalidInput(format!( + "project exceeds member cap of {PROJECT_MEMBER_CAP} (got {}) (rule: member-cap)", + a_tags.len() + ))); + } + + // --- Rule 4: member arity --- + for a in &a_tags { + let len = a.as_slice().len() - 1; // exclude the "a" name element + if !(1..=2).contains(&len) { + return Err(SdkError::InvalidInput(format!( + "member 'a' tag must have 1 or 2 value elements (got {len}) (rule: member-tag-arity)" + ))); + } + } + + // --- Rules 5 & 6: coordinate grammar + deduplication --- + let mut seen_coords: std::collections::HashSet = std::collections::HashSet::new(); + for a in &a_tags { + let coord_val = tag_value(a).unwrap_or(""); + ProjectMemberCoord::parse_full(coord_val).map_err(|e| { + SdkError::InvalidInput(format!("{e} (rule: member-coordinate-malformed)")) + })?; + if !seen_coords.insert(coord_val.to_string()) { + return Err(SdkError::InvalidInput(format!( + "duplicate member coordinate {coord_val:?} (rule: member-duplicate)" + ))); + } + } + + // --- Rules 7 & 8: singleton metadata + byte bounds --- + let singleton_fields = [ + ( + "name", + PROJECT_NAME_MAX, + "metadata-cardinality", + "metadata-length", + ), + ( + "description", + PROJECT_DESCRIPTION_MAX, + "metadata-cardinality", + "metadata-length", + ), + ( + "buzz-channel", + PROJECT_CHANNEL_MAX, + "metadata-cardinality", + "metadata-length", + ), + ( + "buzz-visibility", + PROJECT_VISIBILITY_MAX, + "metadata-cardinality", + "metadata-length", + ), + ]; + for (field, max_bytes, card_rule, len_rule) in singleton_fields { + let matches: Vec<&Tag> = tags.iter().filter(|t| tag_name(t) == Some(field)).collect(); + if matches.len() > 1 { + return Err(SdkError::InvalidInput(format!( + "project must have at most one '{field}' tag (rule: {card_rule})" + ))); + } + if let Some(t) = matches.first() { + let val = tag_value(t).unwrap_or(""); + if val.len() > max_bytes { + return Err(SdkError::InvalidInput(format!( + "'{field}' tag exceeds {max_bytes} bytes (rule: {len_rule})" + ))); + } + } + } + + Ok(()) +} + +/// Helper: tag name (first element). +fn tag_name(tag: &Tag) -> Option<&str> { + tag.as_slice().first().map(String::as_str) +} + +/// Helper: tag value (second element). +fn tag_value(tag: &Tag) -> Option<&str> { + tag.as_slice().get(1).map(String::as_str) +} + +/// **Layer A raw builder**: Build a kind:30621 project event from a raw +/// `content` string and a raw `tags` slice, without any canonicalization. +/// +/// Validates the entire envelope through `validate_project_envelope` before +/// accepting it. The caller is responsible for supplying the correct `d` tag. +/// This is the path exercised by fixture conformance tests and by read-modify- +/// write mutations in the CLI. +pub fn build_project_with_tags(content: &str, tags: Vec) -> Result { + validate_project_envelope(&tags, content)?; + Ok(EventBuilder::new(Kind::Custom(KIND_PROJECT as u16), content).tags(tags)) +} + +/// **Layer B writer-policy builder**: Build a kind:30621 project event with +/// enforced writer policy: +/// - The `d` tag is constructed from `slug`; `check_project_slug` rejects +/// an empty or over-length slug. +/// - `channel` must be a valid UUID string. +/// - `visibility` must be `"listed"` or `"unlisted"`. +/// - Content is always empty. +/// - Member coordinates are parsed through `ProjectMemberCoord::parse_full`. +/// +/// The resulting envelope is validated through Layer A before the builder is +/// returned. +pub fn build_project( + slug: &str, + name: Option<&str>, + description: Option<&str>, + members: &[ProjectMemberCoord], + channel: Option<&str>, + visibility: Option<&str>, +) -> Result { + // Slug validation + if slug.is_empty() { + return Err(SdkError::InvalidInput( + "project slug must not be empty".into(), + )); + } + if slug.len() > PROJECT_D_MAX_LEN { + return Err(SdkError::InvalidInput(format!( + "project slug must not exceed {PROJECT_D_MAX_LEN} bytes (got {})", + slug.len() + ))); + } + + // Channel UUID validation + if let Some(ch) = channel { + uuid::Uuid::parse_str(ch).map_err(|_| { + SdkError::InvalidInput(format!("buzz-channel must be a valid UUID (got {ch:?})")) + })?; + } + + // Visibility enum validation + if let Some(vis) = visibility { + if vis != "listed" && vis != "unlisted" { + return Err(SdkError::InvalidInput(format!( + "buzz-visibility must be 'listed' or 'unlisted' (got {vis:?})" + ))); + } + } + + let mut tags: Vec = Vec::new(); + tags.push(tag(&["d", slug])?); + + if let Some(n) = name { + tags.push(tag(&["name", n])?); + } + if let Some(d) = description { + tags.push(tag(&["description", d])?); + } + for m in members { + let tag_parts = m.to_tag_parts(); + let parts: Vec<&str> = tag_parts.iter().map(|s| s.as_str()).collect(); + // Safety: to_tag_parts always produces ["a", coord, ...hint] + tags.push( + Tag::parse(parts.iter().copied()).map_err(|e| SdkError::InvalidTag(e.to_string()))?, + ); + } + if let Some(ch) = channel { + tags.push(tag(&["buzz-channel", ch])?); + } + if let Some(vis) = visibility { + tags.push(tag(&["buzz-visibility", vis])?); + } + + build_project_with_tags("", tags) +} + +/// **Generic NIP-09 coordinate delete**: Build a kind:5 deletion event with +/// a single `a`-tag addressing `::`. +/// +/// Validates: +/// - `kind` is an addressable kind (10000–19999 or 30000–39999). +/// - `pubkey` is a 64-character lowercase hex string. +/// - `d` is non-empty. +/// +/// `build_workflow_delete` delegates to this function. +pub fn build_delete_addressable( + kind: u32, + pubkey: &str, + d: &str, +) -> Result { + let is_addressable = (10000..20000).contains(&kind) || (30000..40000).contains(&kind); + if !is_addressable { + return Err(SdkError::InvalidInput(format!( + "kind {kind} is not an addressable kind (must be 10000–19999 or 30000–39999)" + ))); + } + let pk = check_pubkey_hex(pubkey, "pubkey")?; + if d.is_empty() { + return Err(SdkError::InvalidInput("d must not be empty".into())); + } + let coord = format!("{kind}:{pk}:{d}"); + let tags = vec![tag(&["a", &coord])?]; + Ok(EventBuilder::new(Kind::Custom(KIND_DELETION as u16), "").tags(tags)) +} + #[cfg(test)] mod tests { use super::*; @@ -3884,4 +4237,280 @@ mod tests { .iter() .any(|t| t.as_slice().first().map(String::as_str) == Some("replaced-by"))); } + + // ── NIP-MP cap-before-arity ordering ───────────────────────────────────── + + /// When an envelope exceeds the member cap AND contains a malformed `a` tag, + /// the validator must fire `member-cap` (rule 3) — not `member-tag-arity` + /// (rule 4). This matches the relay's ingest ordering and means a client + /// sending an oversized list never receives a per-tag parse error. + #[test] + fn validate_project_envelope_cap_wins_over_arity_when_both_fail() { + let owner = "a".repeat(64); + // Build 65 well-formed `a` tags — enough to trigger the cap. + let mut tags = vec![Tag::parse(["d", "platform"]).unwrap()]; + for i in 0..65usize { + let coord = format!("30617:{owner}:repo-{i}"); + tags.push(Tag::parse(["a", &coord]).unwrap()); + } + // Also add one malformed tag (four elements) that would fire + // member-tag-arity if evaluated before the cap check. + let coord_extra = format!("30617:{owner}:repo-extra"); + tags.push( + Tag::parse([ + "a", + &coord_extra, + "wss://relay.example.com", + "extra-element", + ]) + .unwrap(), + ); + + let err = validate_project_envelope(&tags, "").unwrap_err(); + let msg = err.to_string(); + assert!( + msg.contains("member-cap"), + "expected member-cap to win, got: {msg}" + ); + assert!( + !msg.contains("member-tag-arity"), + "arity rule must not fire before cap rule, got: {msg}" + ); + } + + // ── Layer B writer-policy builder ─────────────────────────────────────── + + const OWNER64: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + const VALID_UUID: &str = "3580ca9b-47b4-4af9-b22a-1068778f26c6"; + + fn member_coord(repo: &str) -> ProjectMemberCoord { + ProjectMemberCoord::parse_full(&format!("30617:{OWNER64}:{repo}")).unwrap() + } + + #[test] + fn build_project_emitted_envelope_has_correct_shape() { + // slug, name, description, channel, visibility, and one member. + let m = member_coord("buzz"); + let ev = sign( + build_project( + "my-proj", + Some("My Project"), + Some("A description"), + &[m], + Some(VALID_UUID), + Some("listed"), + ) + .expect("Layer B must accept valid inputs"), + ); + + // Kind must be 30621. + assert_eq!(ev.kind.as_u16(), KIND_PROJECT as u16); + // Content must be empty (Layer B policy). + assert!(ev.content.is_empty(), "content must be empty"); + + let all_tags: Vec> = ev.tags.iter().map(|t| t.as_slice().to_vec()).collect(); + + // d tag must be present exactly once. + let d_tags: Vec<_> = all_tags.iter().filter(|t| t[0] == "d").collect(); + assert_eq!(d_tags.len(), 1); + assert_eq!(d_tags[0][1], "my-proj"); + + // name, description, buzz-channel, buzz-visibility present. + let name_tags: Vec<_> = all_tags.iter().filter(|t| t[0] == "name").collect(); + assert_eq!(name_tags.len(), 1); + assert_eq!(name_tags[0][1], "My Project"); + + let desc_tags: Vec<_> = all_tags.iter().filter(|t| t[0] == "description").collect(); + assert_eq!(desc_tags.len(), 1); + assert_eq!(desc_tags[0][1], "A description"); + + let ch_tags: Vec<_> = all_tags.iter().filter(|t| t[0] == "buzz-channel").collect(); + assert_eq!(ch_tags.len(), 1); + assert_eq!(ch_tags[0][1], VALID_UUID); + + let vis_tags: Vec<_> = all_tags + .iter() + .filter(|t| t[0] == "buzz-visibility") + .collect(); + assert_eq!(vis_tags.len(), 1); + assert_eq!(vis_tags[0][1], "listed"); + + // member a tag. + let a_tags: Vec<_> = all_tags.iter().filter(|t| t[0] == "a").collect(); + assert_eq!(a_tags.len(), 1); + assert_eq!(a_tags[0][1], format!("30617:{OWNER64}:buzz")); + } + + #[test] + fn build_project_optional_fields_absent_when_not_supplied() { + let m = member_coord("core"); + let ev = sign( + build_project("my-proj", None, None, &[m], None, None) + .expect("minimal build must succeed"), + ); + let names: Vec<_> = ev + .tags + .iter() + .filter(|t| t.as_slice().first().map(|s| s.as_str()) == Some("name")) + .collect(); + assert!(names.is_empty(), "name tag must not be emitted when absent"); + } + + #[test] + fn build_project_rejects_empty_slug() { + let m = member_coord("r"); + let err = build_project("", None, None, &[m], None, None).unwrap_err(); + assert!( + matches!(err, SdkError::InvalidInput(_)), + "empty slug must be InvalidInput, got: {err:?}" + ); + assert!(err.to_string().contains("empty")); + } + + #[test] + fn build_project_rejects_overlong_slug() { + let long_slug = "a".repeat(PROJECT_D_MAX_LEN + 1); + let m = member_coord("r"); + let err = build_project(&long_slug, None, None, &[m], None, None).unwrap_err(); + assert!(matches!(err, SdkError::InvalidInput(_))); + } + + #[test] + fn build_project_rejects_invalid_channel_uuid() { + let m = member_coord("r"); + let err = build_project("slug", None, None, &[m], Some("not-a-uuid"), None).unwrap_err(); + assert!(matches!(err, SdkError::InvalidInput(_))); + assert!(err.to_string().contains("UUID") || err.to_string().contains("uuid")); + } + + #[test] + fn build_project_rejects_invalid_visibility_token() { + let m = member_coord("r"); + let err = build_project("slug", None, None, &[m], None, Some("chartreuse")).unwrap_err(); + assert!(matches!(err, SdkError::InvalidInput(_))); + assert!(err.to_string().contains("listed") || err.to_string().contains("unlisted")); + } + + #[test] + fn build_project_rejects_over_cap_members() { + let members: Vec<_> = (0..=PROJECT_MEMBER_CAP) + .map(|i| member_coord(&format!("repo-{i}"))) + .collect(); + let err = build_project("slug", None, None, &members, None, None).unwrap_err(); + assert!(matches!(err, SdkError::InvalidInput(_))); + assert!( + err.to_string().contains("member-cap"), + "over-cap must report member-cap, got: {err}" + ); + } + + #[test] + fn build_project_rejects_duplicate_members() { + let m = member_coord("same"); + let err = build_project("slug", None, None, &[m.clone(), m], None, None).unwrap_err(); + assert!(matches!(err, SdkError::InvalidInput(_))); + assert!( + err.to_string().contains("dedup") || err.to_string().contains("duplicate"), + "duplicate member must report dedup, got: {err}" + ); + } + + #[test] + fn build_project_content_is_always_empty() { + // build_project forces content="" regardless; Layer A also enforces + // that the envelope is valid. Any non-empty content would be dropped. + // This test pins the Layer B content-forced-empty policy. + let m = member_coord("r"); + let ev = sign(build_project("slug", None, None, &[m], None, None).unwrap()); + assert!( + ev.content.is_empty(), + "Layer B must always emit empty content" + ); + } + + // ── NIP-MP conformance fixtures ────────────────────────────────────────── + // `build_project_with_tags` directly. Accept cases must build; reject + // cases must fail with an error message containing the expected rule name. + // A count assertion guards against silent omissions. + // + // `include_str!` path is relative to this source file. + fn nip_mp_fixture_tags(json_tags: &serde_json::Value) -> Vec { + json_tags + .as_array() + .unwrap() + .iter() + .map(|t| { + let parts: Vec = t + .as_array() + .unwrap() + .iter() + .map(|v| v.as_str().unwrap().to_string()) + .collect(); + let parts_ref: Vec<&str> = parts.iter().map(String::as_str).collect(); + Tag::parse(parts_ref.iter().copied()) + .unwrap_or_else(|e| panic!("fixture tag parse error: {e}\n raw: {t}")) + }) + .collect() + } + + #[test] + fn nip_mp_fixtures_all_31_cases_exercised() { + const FIXTURE_JSON: &str = include_str!("../../../docs/nips/NIP-MP.fixtures.json"); + + let data: serde_json::Value = + serde_json::from_str(FIXTURE_JSON).expect("fixture JSON must parse"); + let cases = data["cases"].as_array().expect("cases must be array"); + + // Count gate: the spec says "required to test against this one file" + // with the exact count as-shipped. + assert_eq!( + cases.len(), + 31, + "expected 31 fixture cases, got {} — was NIP-MP.fixtures.json edited?", + cases.len() + ); + + let mut accept_count = 0usize; + let mut reject_count = 0usize; + + for case in cases { + let name = case["name"].as_str().unwrap(); + let expect = case["expect"].as_str().unwrap(); + let template = &case["template"]; + let content = template["content"].as_str().unwrap_or(""); + let tags = nip_mp_fixture_tags(&template["tags"]); + + match expect { + "accept" => { + build_project_with_tags(content, tags).unwrap_or_else(|e| { + panic!("fixture '{name}' (accept) must build successfully, got: {e}") + }); + accept_count += 1; + } + "reject" => { + let reject_rules = case["reject_rules"] + .as_array() + .expect("reject case must have reject_rules") + .iter() + .map(|r| r.as_str().unwrap().to_string()) + .collect::>(); + + let err = build_project_with_tags(content, tags).unwrap_err(); + let err_msg = err.to_string(); + + // The error must mention at least one of the expected rules. + let rule_matched = reject_rules.iter().any(|r| err_msg.contains(r.as_str())); + assert!( + rule_matched, + "fixture '{name}' rejected with wrong rule.\n expected one of: {reject_rules:?}\n got error: {err_msg}" + ); + reject_count += 1; + } + other => panic!("fixture '{name}' has unknown expect value: {other:?}"), + } + } + + assert_eq!(accept_count, 11, "expected 11 accept cases"); + assert_eq!(reject_count, 20, "expected 20 reject cases"); + } } diff --git a/crates/buzz-test-client/Cargo.toml b/crates/buzz-test-client/Cargo.toml index 40a08f3d19..e495c16300 100644 --- a/crates/buzz-test-client/Cargo.toml +++ b/crates/buzz-test-client/Cargo.toml @@ -36,6 +36,7 @@ sha2 = { workspace = true } sqlx = { workspace = true } chrono = { workspace = true } s3 = { version = "0.37", package = "rust-s3", default-features = false, features = ["tokio-rustls-tls", "fail-on-err", "tags"] } +buzz-media = { workspace = true } buzz-sdk = { workspace = true } [[bin]] diff --git a/crates/buzz-test-client/tests/e2e_git.rs b/crates/buzz-test-client/tests/e2e_git.rs index 63281fd18f..3c82e31764 100644 --- a/crates/buzz-test-client/tests/e2e_git.rs +++ b/crates/buzz-test-client/tests/e2e_git.rs @@ -21,6 +21,7 @@ use std::path::{Path, PathBuf}; use std::process::Command; use std::time::Duration; +use buzz_media::S3AddressingStyle; use nostr::{EventBuilder, Keys, Kind, Tag}; use s3::creds::Credentials; use s3::{Bucket, Region}; @@ -61,6 +62,27 @@ async fn post_event(event: &nostr::Event) { ); } +/// Create a channel (kind:9007) owned by `keys` and return its UUID. +/// +/// The git read gate (SEC-005) authorizes against membership in the channel +/// named by the announcement's `buzz-channel` tag, so every repo these tests +/// announce must be bound to a channel its owner belongs to — creating the +/// channel makes the creator its owner-member. +async fn create_test_channel(keys: &Keys) -> String { + let channel_uuid = uuid::Uuid::new_v4().to_string(); + let event = EventBuilder::new(Kind::Custom(9007), "") + .tags(vec![ + Tag::parse(["h", &channel_uuid]).unwrap(), + Tag::parse(["name", &format!("git-e2e-{channel_uuid}")]).unwrap(), + Tag::parse(["channel_type", "stream"]).unwrap(), + Tag::parse(["visibility", "open"]).unwrap(), + ]) + .sign_with_keys(keys) + .unwrap(); + post_event(&event).await; + channel_uuid +} + /// Run `git` with the Buzz credential helper and isolated config. fn git_status(args: &[&str], cwd: &Path, owner_nsec: &str) -> std::process::Output { let helper = credential_helper(); @@ -115,29 +137,55 @@ struct PointerSnapshot { } impl GitS3Probe { - fn from_env() -> Self { - let endpoint = std::env::var("BUZZ_GIT_S3_ENDPOINT") - .or_else(|_| std::env::var("BUZZ_S3_ENDPOINT")) - .unwrap_or_else(|_| "http://localhost:9000".to_string()); - let access_key = std::env::var("BUZZ_GIT_S3_ACCESS_KEY") - .or_else(|_| std::env::var("BUZZ_S3_ACCESS_KEY")) - .unwrap_or_else(|_| "buzz_dev".to_string()); - let secret_key = std::env::var("BUZZ_GIT_S3_SECRET_KEY") - .or_else(|_| std::env::var("BUZZ_S3_SECRET_KEY")) - .unwrap_or_else(|_| "buzz_dev_secret".to_string()); - let bucket = std::env::var("BUZZ_GIT_S3_BUCKET") - .or_else(|_| std::env::var("BUZZ_S3_BUCKET")) - .unwrap_or_else(|_| "buzz-media".to_string()); - + fn bucket( + endpoint: String, + access_key: &str, + secret_key: &str, + bucket_name: &str, + region_name: String, + addressing_style: S3AddressingStyle, + ) -> Box { let region = Region::Custom { - region: "us-east-1".into(), + region: region_name, endpoint, }; - let creds = Credentials::new(Some(&access_key), Some(&secret_key), None, None, None) + let creds = Credentials::new(Some(access_key), Some(secret_key), None, None, None) .expect("S3 credentials"); - let bucket = Bucket::new(&bucket, region, creds) - .expect("S3 bucket") - .with_path_style(); + let bucket = Bucket::new(bucket_name, region, creds).expect("S3 bucket"); + match addressing_style { + S3AddressingStyle::Path => bucket.with_path_style(), + S3AddressingStyle::Virtual => bucket, + } + } + + fn from_env() -> Self { + // These E2E assertions inspect the relay's backing bucket directly, so + // they must receive the same provider connection and URL style as the + // relay. Unit/live MinIO probes in buzz-relay keep explicit local + // fixtures and do not need provider overrides. + let endpoint = std::env::var("BUZZ_S3_ENDPOINT") + .unwrap_or_else(|_| "http://localhost:9000".to_string()); + let access_key = + std::env::var("BUZZ_S3_ACCESS_KEY").unwrap_or_else(|_| "buzz_dev".to_string()); + let secret_key = + std::env::var("BUZZ_S3_SECRET_KEY").unwrap_or_else(|_| "buzz_dev_secret".to_string()); + let bucket_name = + std::env::var("BUZZ_S3_BUCKET").unwrap_or_else(|_| "buzz-media".to_string()); + let region_name = + std::env::var("BUZZ_S3_REGION").unwrap_or_else(|_| "us-east-1".to_string()); + let addressing_style = std::env::var("BUZZ_S3_ADDRESSING_STYLE") + .unwrap_or_else(|_| "path".to_string()) + .parse::() + .expect("BUZZ_S3_ADDRESSING_STYLE must be 'path' or 'virtual'"); + + let bucket = Self::bucket( + endpoint, + &access_key, + &secret_key, + &bucket_name, + region_name, + addressing_style, + ); Self { bucket } } @@ -192,6 +240,31 @@ impl GitS3Probe { } } +#[test] +fn git_s3_probe_builds_both_addressing_styles() { + let path = GitS3Probe::bucket( + "https://storage.example".to_string(), + "access", + "secret", + "buzz-media", + "us-east-1".to_string(), + S3AddressingStyle::Path, + ); + assert!(path.is_path_style()); + assert_eq!(path.url(), "https://storage.example/buzz-media"); + + let virtual_hosted = GitS3Probe::bucket( + "https://storage.example".to_string(), + "access", + "secret", + "buzz-media", + "auto".to_string(), + S3AddressingStyle::Virtual, + ); + assert!(virtual_hosted.is_subdomain_style()); + assert_eq!(virtual_hosted.url(), "https://buzz-media.storage.example"); +} + #[tokio::test] #[ignore = "requires live relay + MinIO + git"] async fn git_clone_push_fetch_force_roundtrip() { @@ -204,10 +277,15 @@ async fn git_clone_push_fetch_force_roundtrip() { let s3 = GitS3Probe::from_env(); // Announce the repo (kind:30617) so the relay creates the bare repo + hook. + // The `buzz-channel` binding is the repo's ACL: without it the read gate + // 404s even for the owner (issue #3527), so bind to a channel the owner + // just created (and therefore belongs to). + let channel = create_test_channel(&owner).await; let announce = EventBuilder::new(Kind::from(30617), "") .tags(vec![ Tag::parse(["d", &repo]).unwrap(), Tag::parse(["name", "e2e git repo"]).unwrap(), + Tag::parse(["buzz-channel", &channel]).unwrap(), ]) .sign_with_keys(&owner) .unwrap(); @@ -341,10 +419,12 @@ async fn git_concurrent_push_one_wins_and_repo_recovers() { let repo = format!("e2e-git-concurrent-{}", std::process::id()); let s3 = GitS3Probe::from_env(); + let channel = create_test_channel(&owner).await; let announce = EventBuilder::new(Kind::from(30617), "") .tags(vec![ Tag::parse(["d", &repo]).unwrap(), Tag::parse(["name", "e2e concurrent git repo"]).unwrap(), + Tag::parse(["buzz-channel", &channel]).unwrap(), ]) .sign_with_keys(&owner) .unwrap(); diff --git a/crates/buzz-test-client/tests/e2e_persona.rs b/crates/buzz-test-client/tests/e2e_persona.rs index b3b1f7f6b2..4f37e22e16 100644 --- a/crates/buzz-test-client/tests/e2e_persona.rs +++ b/crates/buzz-test-client/tests/e2e_persona.rs @@ -1324,7 +1324,7 @@ async fn test_persona_http_query_cross_author_gate() { /// /// A foreign authenticated caller counting `{kinds:[30175],authors:[victim]}` /// must count only shared heads — not unshared ones — on both the fast SQL -/// path (prevented by `needs_persona_filtering`) and the fallback path. +/// path (prevented by `needs_shared_gate_filtering`) and the fallback path. #[tokio::test] #[ignore] async fn test_persona_http_count_cross_author_gate() { @@ -1403,7 +1403,7 @@ async fn test_persona_http_count_cross_author_gate() { /// event is returned. /// /// Verifies at `312014d5e`: this test fails there because `query_events` did -/// not have the `persona_reader` SQL clause and the private rows starved the +/// not have the `shared_gated_reader` SQL clause and the private rows starved the /// shared one off the page. #[tokio::test] #[ignore] diff --git a/crates/buzz-test-client/tests/e2e_project.rs b/crates/buzz-test-client/tests/e2e_project.rs new file mode 100644 index 0000000000..c0a05e4674 --- /dev/null +++ b/crates/buzz-test-client/tests/e2e_project.rs @@ -0,0 +1,491 @@ +//! End-to-end tests for kind:30621 multi-repo project events (NIP-MP). +//! +//! The ingest unit tests in `buzz-relay` pin the envelope contract in isolation. +//! These tests cover the three behaviors that only exist once an event reaches +//! storage, plus proof that the envelope validator is actually wired into the +//! live write path: +//! - a valid cross-owner project round-trips through its NIP-33 coordinate; +//! - replacement is keyed by `(pubkey, 30621, d)` — newer wins for one author, +//! and two authors sharing a `d` hold two independent projects (this is what +//! makes owner-only editing free rather than a relay permission check); +//! - a NIP-09 `a`-tag tombstone removes the project coordinate and leaves every +//! referenced kind:30617 announcement untouched, because membership is an +//! assertion about repositories and never authority over them; +//! - malformed envelopes are refused by the relay, not merely by the validator. +//! +//! See `docs/nips/NIP-MP.md` for the normative contract. +//! +//! # Running +//! +//! Start the relay, then run: +//! +//! ```text +//! RELAY_URL=ws://localhost:3000 cargo test -p buzz-test-client --test e2e_project -- --ignored +//! ``` + +use std::time::Duration; + +use buzz_test_client::BuzzTestClient; +use nostr::{Alphabet, EventBuilder, Filter, Keys, Kind, SingleLetterTag, Tag, Timestamp}; + +const PROJECT_KIND: u16 = 30621; +const REPO_ANNOUNCEMENT_KIND: u16 = 30617; + +fn relay_url() -> String { + std::env::var("RELAY_URL").unwrap_or_else(|_| "ws://localhost:3000".to_string()) +} + +fn sub_id(name: &str) -> String { + format!("e2e-project-{name}-{}", uuid::Uuid::new_v4()) +} + +/// A short unique suffix so concurrent runs never collide on a `d` tag. +fn unique(prefix: &str) -> String { + format!("{prefix}-{}", &uuid::Uuid::new_v4().to_string()[..8]) +} + +fn member_coord(owner: &Keys, repo_d: &str) -> String { + format!( + "{REPO_ANNOUNCEMENT_KIND}:{}:{repo_d}", + owner.public_key().to_hex() + ) +} + +/// Build a project event. `members` are canonical `30617::` +/// coordinates; `created_at` defaults to now when `None`. +fn project_event( + keys: &Keys, + d_tag: &str, + name: &str, + members: &[String], + created_at: Option, +) -> nostr::Event { + let mut tags = vec![ + Tag::parse(["d", d_tag]).unwrap(), + Tag::parse(["name", name]).unwrap(), + ]; + tags.extend( + members + .iter() + .map(|m| Tag::parse(["a", m.as_str()]).unwrap()), + ); + let builder = EventBuilder::new(Kind::Custom(PROJECT_KIND), "").tags(tags); + match created_at { + Some(ts) => builder.custom_created_at(Timestamp::from(ts)), + None => builder, + } + .sign_with_keys(keys) + .unwrap() +} + +/// Announce a repository so a project has a real coordinate to reference. +fn repo_announcement(keys: &Keys, repo_d: &str) -> nostr::Event { + EventBuilder::new(Kind::Custom(REPO_ANNOUNCEMENT_KIND), "") + .tags(vec![ + Tag::parse(["d", repo_d]).unwrap(), + Tag::parse(["name", repo_d]).unwrap(), + ]) + .sign_with_keys(keys) + .unwrap() +} + +/// A NIP-09 `a`-tag-only deletion at a NIP-33 coordinate. No `e` tag, so the +/// relay takes the coordinate-delete path rather than the event-id path. +/// `created_at` defaults to now when `None`. +fn coordinate_delete(keys: &Keys, kind: u16, d_tag: &str, created_at: Option) -> nostr::Event { + let coord = format!("{kind}:{}:{d_tag}", keys.public_key().to_hex()); + let builder = + EventBuilder::new(Kind::Custom(5), "") + .tags(vec![Tag::parse(["a", coord.as_str()]).unwrap()]); + match created_at { + Some(ts) => builder.custom_created_at(Timestamp::from(ts)), + None => builder, + } + .sign_with_keys(keys) + .unwrap() +} + +fn addressable_filter(kind: u16, author: &Keys, d_tag: &str) -> Filter { + Filter::new() + .kind(Kind::Custom(kind)) + .author(author.public_key()) + .custom_tags(SingleLetterTag::lowercase(Alphabet::D), [d_tag]) +} + +/// Subscribe with `filter` and drain to EOSE. +async fn query(client: &mut BuzzTestClient, name: &str, filter: Filter) -> Vec { + let sid = sub_id(name); + client + .subscribe(&sid, vec![filter]) + .await + .expect("subscribe"); + client + .collect_until_eose(&sid, Duration::from_secs(5)) + .await + .expect("collect events") +} + +#[tokio::test] +#[ignore] +async fn test_project_publish_and_query_returns_cross_owner_members() { + let url = relay_url(); + let owner = Keys::generate(); + let other = Keys::generate(); + let d_tag = unique("project"); + + let members = vec![ + member_coord(&owner, "buzz"), + member_coord(&other, "buzz-infra"), + ]; + + let mut client = BuzzTestClient::connect(&url, &owner) + .await + .expect("connect"); + + let event = project_event(&owner, &d_tag, "Platform", &members, None); + let ok = client.send_event(event).await.expect("send project"); + assert!(ok.accepted, "relay rejected project event: {}", ok.message); + + let events = query( + &mut client, + "query", + addressable_filter(PROJECT_KIND, &owner, &d_tag), + ) + .await; + + assert_eq!(events.len(), 1, "expected exactly one project event"); + let stored: Vec<&str> = events[0] + .tags + .iter() + .filter_map(|t| { + let parts = t.as_slice(); + (parts.first().map(|s| s.as_str()) == Some("a")).then(|| parts[1].as_str()) + }) + .collect(); + assert_eq!( + stored, members, + "both members must survive the round trip, including the one owned by another pubkey" + ); + + client.disconnect().await.expect("disconnect"); +} + +#[tokio::test] +#[ignore] +async fn test_project_replacement_keeps_only_newest_for_same_author_and_d() { + let url = relay_url(); + let owner = Keys::generate(); + let d_tag = unique("project-replace"); + let now = Timestamp::now().as_secs(); + + let mut client = BuzzTestClient::connect(&url, &owner) + .await + .expect("connect"); + + let first = project_event(&owner, &d_tag, "Old", &[], Some(now - 100)); + let ok = client.send_event(first).await.expect("send old"); + assert!(ok.accepted, "relay rejected old project: {}", ok.message); + + let members = vec![member_coord(&owner, "buzz")]; + let second = project_event(&owner, &d_tag, "New", &members, Some(now)); + let ok = client.send_event(second).await.expect("send new"); + assert!(ok.accepted, "relay rejected new project: {}", ok.message); + + let events = query( + &mut client, + "replace", + addressable_filter(PROJECT_KIND, &owner, &d_tag), + ) + .await; + + assert_eq!( + events.len(), + 1, + "NIP-33: only the newest head should remain" + ); + let name = events[0] + .tags + .iter() + .find_map(|t| { + let parts = t.as_slice(); + (parts.first().map(|s| s.as_str()) == Some("name")).then(|| parts[1].as_str()) + }) + .expect("name tag"); + assert_eq!(name, "New", "the newer head must win"); + + client.disconnect().await.expect("disconnect"); +} + +/// Owner-only editing is a property of the addressable model, not a relay +/// permission check: two authors publishing the same `d` occupy two coordinates, +/// so neither can overwrite the other. This is the test that would fail if the +/// kind were ever classified as plain-replaceable or keyed on `d` alone. +#[tokio::test] +#[ignore] +async fn test_project_same_d_under_two_authors_are_independent() { + let url = relay_url(); + let alice = Keys::generate(); + let bob = Keys::generate(); + let d_tag = unique("project-shared-d"); + + let mut alice_client = BuzzTestClient::connect(&url, &alice) + .await + .expect("connect"); + let ok = alice_client + .send_event(project_event(&alice, &d_tag, "Alice", &[], None)) + .await + .expect("send alice"); + assert!( + ok.accepted, + "relay rejected alice's project: {}", + ok.message + ); + + let mut bob_client = BuzzTestClient::connect(&url, &bob).await.expect("connect"); + let ok = bob_client + .send_event(project_event(&bob, &d_tag, "Bob", &[], None)) + .await + .expect("send bob"); + assert!(ok.accepted, "relay rejected bob's project: {}", ok.message); + + for (label, keys, expected_name) in [("alice", &alice, "Alice"), ("bob", &bob, "Bob")] { + let events = query( + &mut alice_client, + label, + addressable_filter(PROJECT_KIND, keys, &d_tag), + ) + .await; + assert_eq!( + events.len(), + 1, + "{label} should still hold their own project at the shared `d`" + ); + let name = events[0] + .tags + .iter() + .find_map(|t| { + let parts = t.as_slice(); + (parts.first().map(|s| s.as_str()) == Some("name")).then(|| parts[1].as_str()) + }) + .expect("name tag"); + assert_eq!(name, expected_name, "{label}'s project was overwritten"); + } + + alice_client.disconnect().await.expect("disconnect"); + bob_client.disconnect().await.expect("disconnect"); +} + +/// Deleting a project must delete only the grouping. A project is metadata about +/// repositories; if a tombstone at the project coordinate cascaded to the +/// referenced kind:30617s, adding a repo to someone's project would become a way +/// to destroy it. +#[tokio::test] +#[ignore] +async fn test_project_tombstone_deletes_coordinate_and_spares_members() { + let url = relay_url(); + let owner = Keys::generate(); + let repo_d = unique("repo"); + let project_d = unique("project-tombstone"); + + let mut client = BuzzTestClient::connect(&url, &owner) + .await + .expect("connect"); + + let ok = client + .send_event(repo_announcement(&owner, &repo_d)) + .await + .expect("send announcement"); + assert!(ok.accepted, "relay rejected announcement: {}", ok.message); + + let members = vec![member_coord(&owner, &repo_d)]; + let ok = client + .send_event(project_event(&owner, &project_d, "Doomed", &members, None)) + .await + .expect("send project"); + assert!(ok.accepted, "relay rejected project: {}", ok.message); + + let before = query( + &mut client, + "tombstone-pre", + addressable_filter(PROJECT_KIND, &owner, &project_d), + ) + .await; + assert_eq!(before.len(), 1, "project should be live before deletion"); + + let ok = client + .send_event(coordinate_delete(&owner, PROJECT_KIND, &project_d, None)) + .await + .expect("send tombstone"); + assert!(ok.accepted, "relay rejected tombstone: {}", ok.message); + + let after = query( + &mut client, + "tombstone-post", + addressable_filter(PROJECT_KIND, &owner, &project_d), + ) + .await; + assert!( + after.is_empty(), + "tombstone should remove the project coordinate, got {} event(s)", + after.len() + ); + + let repo = query( + &mut client, + "member-after", + addressable_filter(REPO_ANNOUNCEMENT_KIND, &owner, &repo_d), + ) + .await; + assert_eq!( + repo.len(), + 1, + "deleting a project must not touch the repositories it referenced" + ); + + client.disconnect().await.expect("disconnect"); +} + +/// NIP-09 scopes an `a`-tag deletion to versions at or before the deletion's own +/// `created_at`. A tombstone signed between V1 and V2 — delayed in transit or +/// replayed by a third party — must therefore retire V1 only and leave the newer +/// V2 head live. Before the timestamp predicate landed in +/// `soft_delete_by_coordinate`, the coordinate delete was timestamp-blind and +/// this sequence silently destroyed V2. +#[tokio::test] +#[ignore] +async fn test_stale_tombstone_between_versions_leaves_newer_project_live() { + let url = relay_url(); + let owner = Keys::generate(); + let project_d = unique("project-stale-tombstone"); + let now = Timestamp::now().as_secs(); + + let mut client = BuzzTestClient::connect(&url, &owner) + .await + .expect("connect"); + + let ok = client + .send_event(project_event( + &owner, + &project_d, + "V1", + &[], + Some(now - 100), + )) + .await + .expect("send v1"); + assert!(ok.accepted, "relay rejected V1: {}", ok.message); + + let ok = client + .send_event(project_event(&owner, &project_d, "V2", &[], Some(now))) + .await + .expect("send v2"); + assert!(ok.accepted, "relay rejected V2: {}", ok.message); + + // Timestamped strictly between V1 and V2: valid for V1, stale for V2. + let ok = client + .send_event(coordinate_delete( + &owner, + PROJECT_KIND, + &project_d, + Some(now - 50), + )) + .await + .expect("send stale tombstone"); + assert!( + ok.accepted, + "a well-formed tombstone is still an acceptable event: {}", + ok.message + ); + + let after = query( + &mut client, + "stale-tombstone", + addressable_filter(PROJECT_KIND, &owner, &project_d), + ) + .await; + + assert_eq!( + after.len(), + 1, + "a tombstone older than the live head must not delete it, got {} event(s)", + after.len() + ); + let name = after[0] + .tags + .iter() + .find_map(|t| { + let parts = t.as_slice(); + (parts.first().map(|s| s.as_str()) == Some("name")).then(|| parts[1].as_str()) + }) + .expect("surviving head must carry its name tag"); + assert_eq!(name, "V2", "the surviving head must be the newer version"); + + client.disconnect().await.expect("disconnect"); +} + +/// Proves the envelope validator is reachable from the live write path — a unit +/// test of `validate_project_envelope` cannot show that ingest calls it. +#[tokio::test] +#[ignore] +async fn test_project_malformed_envelope_rejected_by_relay() { + let url = relay_url(); + let owner = Keys::generate(); + let mut client = BuzzTestClient::connect(&url, &owner) + .await + .expect("connect"); + + let duplicate = member_coord(&owner, "buzz"); + // Each case pairs a malformed event with the substring its rejection must + // carry, so a refusal for an unrelated reason cannot satisfy the assertion. + let cases: Vec<(&str, nostr::Event, &str)> = vec![ + ( + "duplicate member coordinate", + project_event( + &owner, + &unique("project-dup"), + "Dup", + &[duplicate.clone(), duplicate], + None, + ), + "duplicate member coordinate", + ), + ( + "member coordinate naming the wrong kind", + project_event( + &owner, + &unique("project-badkind"), + "Bad kind", + &[format!("30618:{}:buzz", owner.public_key().to_hex())], + None, + ), + "member `a` tag must be", + ), + ( + "member coordinate with an uppercase-hex owner", + project_event( + &owner, + &unique("project-upper"), + "Uppercase", + &[format!("{REPO_ANNOUNCEMENT_KIND}:{}:buzz", "A".repeat(64))], + None, + ), + "member `a` tag must be", + ), + ]; + + for (label, event, expected) in cases { + let ok = client.send_event(event).await.expect("send"); + assert!( + !ok.accepted, + "relay must reject a project with a {label}, got OK: {}", + ok.message + ); + assert!( + ok.message.contains(expected), + "rejection for {label} must name the rule that fired, got: {}", + ok.message + ); + } + + client.disconnect().await.expect("disconnect"); +} diff --git a/crates/buzz-test-client/tests/e2e_team_catalog.rs b/crates/buzz-test-client/tests/e2e_team_catalog.rs new file mode 100644 index 0000000000..ce313d1fe9 --- /dev/null +++ b/crates/buzz-test-client/tests/e2e_team_catalog.rs @@ -0,0 +1,484 @@ +//! End-to-end tests for kind:30178 team-catalog events (NIP-AP). +//! +//! Kind 30178 is the shareable projection of a team. It joins kind:30175 in +//! `SHARED_GATED_KINDS`, so these tests assert the wire behaviour of that gate +//! at every read chokepoint (REQ, `ids` lookup, COUNT, live fan-out) plus the +//! ingest envelope rules that make the gate sound: +//! - Exactly one non-empty, bounded `d` tag — the team's stable local id, which +//! may contain a colon (`builtin-team:welcome`) unlike a persona slug. +//! - `shared`, if present, is exactly `["shared", "true"]`. +//! +//! # Running +//! +//! Start the relay, then run: +//! +//! ```text +//! RELAY_URL=ws://localhost:3000 cargo test --test e2e_team_catalog -- --ignored +//! ``` + +use std::time::Duration; + +use buzz_test_client::{BuzzTestClient, RelayMessage}; +use nostr::{Alphabet, EventBuilder, Filter, Keys, Kind, SingleLetterTag, Tag, Timestamp}; + +const TEAM_CATALOG_KIND: u16 = 30178; + +fn relay_url() -> String { + std::env::var("RELAY_URL").unwrap_or_else(|_| "ws://localhost:3000".to_string()) +} + +fn sub_id(name: &str) -> String { + format!("e2e-team-catalog-{name}-{}", uuid::Uuid::new_v4()) +} + +fn catalog_content(name: &str) -> String { + serde_json::json!({ "v": 1, "name": name, "members": [] }).to_string() +} + +/// Build a kind:30178 event, optionally carrying the `["shared","true"]` opt-in. +fn catalog_event(keys: &Keys, d_tag: &str, shared: bool) -> nostr::Event { + catalog_event_at(keys, d_tag, shared, Timestamp::now().as_secs()) +} + +/// Same as [`catalog_event`] with an explicit `created_at`, so NIP-33 head +/// ordering is deterministic instead of resolved by event-id tie-break. +fn catalog_event_at(keys: &Keys, d_tag: &str, shared: bool, created_at: u64) -> nostr::Event { + let mut tags = vec![Tag::parse(["d", d_tag]).unwrap()]; + if shared { + tags.push(Tag::parse(["shared", "true"]).unwrap()); + } + EventBuilder::new( + Kind::Custom(TEAM_CATALOG_KIND), + catalog_content("Test Team"), + ) + .tags(tags) + .custom_created_at(Timestamp::from(created_at)) + .sign_with_keys(keys) + .unwrap() +} + +fn author_filter(author: &Keys) -> Filter { + Filter::new() + .kind(Kind::Custom(TEAM_CATALOG_KIND)) + .author(author.public_key()) +} + +fn coordinate_filter(author: &Keys, d_tag: &str) -> Filter { + author_filter(author).custom_tags(SingleLetterTag::lowercase(Alphabet::D), [d_tag]) +} + +fn d_tag_of(event: &nostr::Event) -> Option<&str> { + event.tags.iter().find_map(|t| { + let parts = t.as_slice(); + if parts.first().map(|p| p.as_str()) != Some("d") { + return None; + } + Some(parts.get(1)?.as_str()) + }) +} + +/// The author's own unshared projection round-trips at its NIP-33 coordinate. +/// +/// The `d` tag is a UUID, matching the desktop team id — proof the envelope does +/// NOT apply the persona slug grammar. +#[tokio::test] +#[ignore] +async fn test_team_catalog_publish_and_query_own_unshared() { + let url = relay_url(); + let keys = Keys::generate(); + let d_tag = uuid::Uuid::new_v4().to_string(); + + let mut client = BuzzTestClient::connect(&url, &keys).await.expect("connect"); + let event = catalog_event(&keys, &d_tag, false); + let event_id = event.id; + let ok = client.send_event(event).await.expect("send catalog"); + assert!(ok.accepted, "relay rejected catalog event: {}", ok.message); + + let sid = sub_id("own-unshared"); + client + .subscribe(&sid, vec![coordinate_filter(&keys, &d_tag)]) + .await + .expect("subscribe"); + let events = client + .collect_until_eose(&sid, Duration::from_secs(5)) + .await + .expect("collect"); + + assert_eq!(events.len(), 1, "author must see own unshared projection"); + assert_eq!(events[0].id, event_id); + + client.disconnect().await.expect("disconnect"); +} + +/// A built-in team id (`builtin-team:welcome`) is accepted as the `d` tag. +/// +/// The colon is illegal in a persona slug; rewriting the id to fit would break +/// NIP-33 addressing against the team's own kind:30176 head. +#[tokio::test] +#[ignore] +async fn test_team_catalog_accepts_builtin_colon_d_tag() { + let url = relay_url(); + let keys = Keys::generate(); + let d_tag = format!("builtin-team:{}", &uuid::Uuid::new_v4().to_string()[..8]); + + let mut client = BuzzTestClient::connect(&url, &keys).await.expect("connect"); + let ok = client + .send_event(catalog_event(&keys, &d_tag, true)) + .await + .expect("send catalog"); + assert!( + ok.accepted, + "relay rejected colon-bearing team id: {}", + ok.message + ); + + client.disconnect().await.expect("disconnect"); +} + +/// Ingest refuses an empty `d` tag: generic NIP-33 storage maps it to the empty +/// coordinate, collapsing every team into one `(pubkey, 30178, "")` slot. +#[tokio::test] +#[ignore] +async fn test_team_catalog_rejects_empty_d_tag() { + let url = relay_url(); + let keys = Keys::generate(); + + let mut client = BuzzTestClient::connect(&url, &keys).await.expect("connect"); + let ok = client + .send_event(catalog_event(&keys, "", false)) + .await + .expect("send catalog"); + assert!(!ok.accepted, "empty d-tag must be rejected"); + assert!( + ok.message.contains("invalid:"), + "expected an `invalid:` refusal, got: {}", + ok.message + ); + + client.disconnect().await.expect("disconnect"); +} + +/// Ingest refuses a valueless `["d"]` tag alongside a valued one. Counting only +/// tags that carry a value would see exactly one `d` here and accept the event; +/// a NIP-33 consumer that reads `["d"]` as an empty-valued first `d` tag would +/// then address the event at `""` where this relay addresses it at the team id. +#[tokio::test] +#[ignore] +async fn test_team_catalog_rejects_valueless_plus_valued_d_tags() { + let url = relay_url(); + let keys = Keys::generate(); + let d_tag = uuid::Uuid::new_v4().to_string(); + + let event = EventBuilder::new( + Kind::Custom(TEAM_CATALOG_KIND), + catalog_content("Two d tags"), + ) + .tags(vec![ + Tag::parse(["d"]).unwrap(), + Tag::parse(["d", d_tag.as_str()]).unwrap(), + ]) + .sign_with_keys(&keys) + .unwrap(); + + let mut client = BuzzTestClient::connect(&url, &keys).await.expect("connect"); + let ok = client.send_event(event).await.expect("send catalog"); + assert!( + !ok.accepted, + "a valueless `d` tag must count toward the exactly-one rule" + ); + assert!( + ok.message.contains("invalid:"), + "expected an `invalid:` refusal, got: {}", + ok.message + ); + + client.disconnect().await.expect("disconnect"); +} + +/// Ingest refuses a malformed `shared` tag. A three-element tag would satisfy +/// the SQL containment clause `tags @> '[["shared","true"]]'` as a superset +/// while the in-process gate reads it as unshared — the two layers must agree, +/// so such an event can never be stored. +#[tokio::test] +#[ignore] +async fn test_team_catalog_rejects_three_element_shared_tag() { + let url = relay_url(); + let keys = Keys::generate(); + let d_tag = uuid::Uuid::new_v4().to_string(); + + let event = EventBuilder::new( + Kind::Custom(TEAM_CATALOG_KIND), + catalog_content("Malformed"), + ) + .tags(vec![ + Tag::parse(["d", d_tag.as_str()]).unwrap(), + Tag::parse(["shared", "true", "extra"]).unwrap(), + ]) + .sign_with_keys(&keys) + .unwrap(); + + let mut client = BuzzTestClient::connect(&url, &keys).await.expect("connect"); + let ok = client.send_event(event).await.expect("send catalog"); + assert!(!ok.accepted, "three-element shared tag must be rejected"); + assert!( + ok.message.contains("invalid:"), + "expected an `invalid:` refusal, got: {}", + ok.message + ); + + client.disconnect().await.expect("disconnect"); +} + +/// REQ historical delivery: a foreign reader receives only shared projections, +/// while the author receives both of their own. +#[tokio::test] +#[ignore] +async fn test_team_catalog_foreign_sees_only_shared() { + let url = relay_url(); + let author_keys = Keys::generate(); + let foreign_keys = Keys::generate(); + + let d_unshared = format!("priv-{}", uuid::Uuid::new_v4()); + let d_shared = format!("pub-{}", uuid::Uuid::new_v4()); + + let mut author = BuzzTestClient::connect(&url, &author_keys) + .await + .expect("connect author"); + let shared_event = catalog_event(&author_keys, &d_shared, true); + let shared_id = shared_event.id; + let ok = author + .send_event(catalog_event(&author_keys, &d_unshared, false)) + .await + .expect("send unshared"); + assert!(ok.accepted, "unshared ingest rejected: {}", ok.message); + let ok = author.send_event(shared_event).await.expect("send shared"); + assert!(ok.accepted, "shared ingest rejected: {}", ok.message); + + let mut foreign = BuzzTestClient::connect(&url, &foreign_keys) + .await + .expect("connect foreign"); + let sid = sub_id("fg-all"); + foreign + .subscribe(&sid, vec![author_filter(&author_keys)]) + .await + .expect("subscribe"); + let events = foreign + .collect_until_eose(&sid, Duration::from_secs(5)) + .await + .expect("collect"); + + assert!( + !events + .iter() + .any(|e| d_tag_of(e) == Some(d_unshared.as_str())), + "foreign reader must NOT see the unshared projection" + ); + assert!( + events.iter().any(|e| e.id == shared_id), + "foreign reader must see the shared projection" + ); + + let sid_author = sub_id("auth-all"); + author + .subscribe(&sid_author, vec![author_filter(&author_keys)]) + .await + .expect("subscribe author"); + let author_events = author + .collect_until_eose(&sid_author, Duration::from_secs(5)) + .await + .expect("collect author"); + assert!( + author_events.len() >= 2, + "author must see both own projections, got {}", + author_events.len() + ); + + author.disconnect().await.expect("disconnect author"); + foreign.disconnect().await.expect("disconnect foreign"); +} + +/// Knowing an event id does NOT grant access: `{ids:[unshared]}` returns nothing +/// to a foreign reader. +#[tokio::test] +#[ignore] +async fn test_team_catalog_ids_lookup_unshared_returns_nothing_to_foreign() { + let url = relay_url(); + let author_keys = Keys::generate(); + let foreign_keys = Keys::generate(); + + let event = catalog_event(&author_keys, &uuid::Uuid::new_v4().to_string(), false); + let event_id = event.id; + + let mut author = BuzzTestClient::connect(&url, &author_keys) + .await + .expect("connect author"); + let ok = author.send_event(event).await.expect("send"); + assert!(ok.accepted, "ingest rejected: {}", ok.message); + author.disconnect().await.expect("disconnect author"); + + let mut foreign = BuzzTestClient::connect(&url, &foreign_keys) + .await + .expect("connect foreign"); + let sid = sub_id("ids-unshared"); + foreign + .subscribe(&sid, vec![Filter::new().id(event_id)]) + .await + .expect("subscribe"); + let events = foreign + .collect_until_eose(&sid, Duration::from_secs(5)) + .await + .expect("collect"); + + assert!( + events.is_empty(), + "ids-lookup of an unshared projection must return nothing, got {:?}", + events.iter().map(|e| e.id).collect::>() + ); + + foreign.disconnect().await.expect("disconnect foreign"); +} + +/// COUNT must take the per-event fallback for kind:30178 so the aggregate does +/// not leak the existence of unshared projections. +#[tokio::test] +#[ignore] +async fn test_team_catalog_count_excludes_foreign_unshared() { + let url = relay_url(); + let author_keys = Keys::generate(); + let foreign_keys = Keys::generate(); + + let mut author = BuzzTestClient::connect(&url, &author_keys) + .await + .expect("connect author"); + let ok = author + .send_event(catalog_event( + &author_keys, + &uuid::Uuid::new_v4().to_string(), + false, + )) + .await + .expect("send unshared"); + assert!(ok.accepted, "unshared rejected: {}", ok.message); + let ok = author + .send_event(catalog_event( + &author_keys, + &uuid::Uuid::new_v4().to_string(), + true, + )) + .await + .expect("send shared"); + assert!(ok.accepted, "shared rejected: {}", ok.message); + author.disconnect().await.expect("disconnect author"); + + let mut foreign = BuzzTestClient::connect(&url, &foreign_keys) + .await + .expect("connect foreign"); + let sid = sub_id("count"); + let count_msg = serde_json::json!(["COUNT", sid, author_filter(&author_keys)]); + foreign.send_raw(&count_msg).await.expect("send COUNT"); + + let count = match foreign.recv_event(Duration::from_secs(5)).await { + Ok(RelayMessage::Count { count, .. }) => count, + Ok(RelayMessage::Closed { message, .. }) => panic!("COUNT closed unexpectedly: {message}"), + Ok(other) => panic!("unexpected relay message for COUNT: {other:?}"), + Err(e) => panic!("unexpected error for COUNT: {e}"), + }; + assert_eq!( + count, 1, + "foreign COUNT must see only the shared projection, got {count}" + ); + + foreign.disconnect().await.expect("disconnect foreign"); +} + +/// Live fan-out honours the gate, and unsharing (a NIP-33 replacement that drops +/// the `shared` tag) retracts the projection from foreign readers. +#[tokio::test] +#[ignore] +async fn test_team_catalog_live_fanout_and_unshare_retracts() { + let url = relay_url(); + let author_keys = Keys::generate(); + let foreign_keys = Keys::generate(); + + let d_tag = uuid::Uuid::new_v4().to_string(); + let now = Timestamp::now().as_secs(); + let (t0, t1, t2) = (now.saturating_sub(2), now.saturating_sub(1), now); + + // Subscribe BEFORE publishing, scoped to this author so parallel tests + // publishing their own 30178s cannot trip the leak assertion. + let mut foreign = BuzzTestClient::connect(&url, &foreign_keys) + .await + .expect("connect foreign"); + let sid = sub_id("fanout"); + foreign + .subscribe(&sid, vec![author_filter(&author_keys)]) + .await + .expect("subscribe"); + let _ = foreign + .collect_until_eose(&sid, Duration::from_secs(5)) + .await + .expect("drain eose"); + + let mut author = BuzzTestClient::connect(&url, &author_keys) + .await + .expect("connect author"); + + // Unshared publish must NOT reach the foreign connection. + let ok = author + .send_event(catalog_event_at(&author_keys, &d_tag, false, t0)) + .await + .expect("send unshared"); + assert!(ok.accepted, "unshared rejected: {}", ok.message); + match foreign.recv_event(Duration::from_millis(750)).await { + Err(buzz_test_client::TestClientError::Timeout) => {} + Ok(RelayMessage::Event { event, .. }) if event.kind == Kind::Custom(TEAM_CATALOG_KIND) => { + panic!("unshared projection leaked to foreign live subscription"); + } + Ok(_) => {} + Err(e) => panic!("unexpected error awaiting fan-out: {e}"), + } + + // Shared replacement MUST reach it. + let shared_event = catalog_event_at(&author_keys, &d_tag, true, t1); + let shared_id = shared_event.id; + let ok = author.send_event(shared_event).await.expect("send shared"); + assert!(ok.accepted, "shared rejected: {}", ok.message); + let delivered = loop { + match foreign.recv_event(Duration::from_secs(5)).await { + Ok(RelayMessage::Event { event, .. }) if event.id == shared_id => break true, + Ok(_) => continue, + Err(buzz_test_client::TestClientError::Timeout) => break false, + Err(e) => panic!("unexpected error awaiting shared fan-out: {e}"), + } + }; + assert!( + delivered, + "shared projection must fan out to foreign readers" + ); + + // Unshare: replace at the same coordinate without the tag. Subsequent + // foreign REQs must return nothing. + let ok = author + .send_event(catalog_event_at(&author_keys, &d_tag, false, t2)) + .await + .expect("send unshare"); + assert!(ok.accepted, "unshare rejected: {}", ok.message); + + let sid_post = sub_id("post-unshare"); + foreign + .subscribe(&sid_post, vec![coordinate_filter(&author_keys, &d_tag)]) + .await + .expect("subscribe post"); + let after = foreign + .collect_until_eose(&sid_post, Duration::from_secs(5)) + .await + .expect("collect post"); + assert!( + after.is_empty(), + "unsharing must retract the projection from foreign readers, got {} event(s)", + after.len() + ); + + author.disconnect().await.expect("disconnect author"); + foreign.disconnect().await.expect("disconnect foreign"); +} diff --git a/crates/buzz-voice/Cargo.toml b/crates/buzz-voice/Cargo.toml new file mode 100644 index 0000000000..beff5b4a54 --- /dev/null +++ b/crates/buzz-voice/Cargo.toml @@ -0,0 +1,25 @@ +[package] +name = "buzz-voice" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +description = "Reusable local voice primitives for Buzz" + +[dependencies] +atomic-write-file = "0.3" +hex = { workspace = true } +ort = { version = "=2.0.0-rc.12", default-features = false, features = ["api-24", "ndarray", "std"] } +ort-sys = { version = "=2.0.0-rc.12", features = ["disable-linking"] } +rand = "0.10" +sentencepiece-model = "0.1" +serde = { version = "1", features = ["derive"] } +serde_json = "1" +sha2 = { workspace = true } +sherpa-onnx = "1.12" +symphonia = { version = "0.5", default-features = false, features = ["aac", "aiff", "alac", "flac", "isomp4", "mp3", "ogg", "pcm", "vorbis", "wav"] } +tokenizers = { version = "0.22", default-features = false, features = ["fancy-regex"] } + +[dev-dependencies] +tempfile = "3" diff --git a/crates/buzz-voice/src/imported.rs b/crates/buzz-voice/src/imported.rs new file mode 100644 index 0000000000..6f0ea71cad --- /dev/null +++ b/crates/buzz-voice/src/imported.rs @@ -0,0 +1,730 @@ +//! Device-local Pocket reference voice validation, canonicalization, and storage. + +use std::{ + fs, + io::Write, + path::{Path, PathBuf}, +}; + +use atomic_write_file::AtomicWriteFile; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use symphonia::core::{ + audio::SampleBuffer, codecs::DecoderOptions, errors::Error as SymphoniaError, + formats::FormatOptions, io::MediaSourceStream, meta::MetadataOptions, probe::Hint, +}; + +const MAX_SOURCE_BYTES: u64 = 25 * 1024 * 1024; +const MIN_SAMPLE_RATE: u32 = 8_000; +const MAX_SAMPLE_RATE: u32 = 96_000; +const MIN_DURATION_SECONDS: f64 = 2.0; +const MAX_DURATION_SECONDS: f64 = 30.0; +pub const CANONICAL_SAMPLE_RATE: u32 = 32_000; +const REGISTRY_VERSION: u32 = 1; +const REGISTRY_FILE: &str = "registry.json"; + +#[derive(Clone, Debug, Deserialize, Serialize, PartialEq, Eq)] +#[serde(rename_all = "camelCase")] +pub struct ImportedVoice { + pub key: String, + pub display_name: String, + pub content_hash: String, + pub file_name: String, +} + +#[derive(Default, Deserialize, Serialize)] +#[serde(rename_all = "camelCase")] +struct ImportedVoiceRegistry { + version: u32, + voices: Vec, +} + +#[derive(Clone, Debug)] +pub struct PocketVoiceLibrary { + root: PathBuf, +} + +impl PocketVoiceLibrary { + pub fn new(root: impl Into) -> Self { + Self { root: root.into() } + } + + pub fn root(&self) -> &Path { + &self.root + } + + fn registry_path(&self) -> PathBuf { + self.root.join(REGISTRY_FILE) + } + + pub fn load(&self) -> Result, String> { + let path = self.registry_path(); + if !path.exists() { + return Ok(Vec::new()); + } + let bytes = + fs::read(&path).map_err(|error| format!("could not read imported voices: {error}"))?; + let registry: ImportedVoiceRegistry = serde_json::from_slice(&bytes) + .map_err(|error| format!("imported voice registry is invalid: {error}"))?; + if registry.version > REGISTRY_VERSION { + return Err(format!( + "imported voice registry version {} is newer than this Buzz build supports", + registry.version + )); + } + Ok(registry + .voices + .into_iter() + .filter(valid_identity) + .filter(|voice| self.resolve_file(voice).is_ok()) + .collect()) + } + + fn save(&self, voices: &[ImportedVoice]) -> Result<(), String> { + ensure_storage_dir(&self.root)?; + let payload = serde_json::to_vec_pretty(&ImportedVoiceRegistry { + version: REGISTRY_VERSION, + voices: voices.to_vec(), + }) + .map_err(|error| format!("could not encode imported voice registry: {error}"))?; + atomic_write_restricted(&self.registry_path(), &payload) + .map_err(|error| format!("could not save imported voice registry: {error}")) + } + + pub fn resolve_file(&self, voice: &ImportedVoice) -> Result { + if !valid_identity(voice) { + return Err("Imported voice registry contains an invalid file identity".to_string()); + } + let path = self.root.join(&voice.file_name); + if !is_regular_file_without_symlink(&path) { + return Err(format!("Imported voice {} is missing", voice.display_name)); + } + let bytes = + fs::read(&path).map_err(|error| format!("could not verify imported voice: {error}"))?; + if hex::encode(Sha256::digest(bytes)) != voice.content_hash { + return Err(format!( + "Imported voice {} does not match its content identity", + voice.display_name + )); + } + Ok(path) + } + + pub fn find(&self, key: &str) -> Result, String> { + Ok(self.load()?.into_iter().find(|voice| voice.key == key)) + } + + pub fn import_path(&self, source: &Path) -> Result { + let metadata = fs::metadata(source) + .map_err(|error| format!("could not inspect selected audio: {error}"))?; + if metadata.len() > MAX_SOURCE_BYTES { + return Err("Voice audio must be 25 MB or smaller".to_string()); + } + let extension = source + .extension() + .and_then(|extension| extension.to_str()) + .map(str::to_ascii_lowercase) + .ok_or_else(|| "Voice audio must have a supported file extension".to_string())?; + let samples = if extension == "wav" { + let source_bytes = fs::read(source) + .map_err(|error| format!("could not read selected audio: {error}"))?; + decode_wav(&source_bytes)? + } else { + decode_media(source, &extension)? + }; + let canonical_samples = resample_linear(&samples.samples, samples.sample_rate); + let canonical = encode_pcm16_wav(&canonical_samples, CANONICAL_SAMPLE_RATE); + let hash = hex::encode(Sha256::digest(&canonical)); + let key = format!("pocket:imported:{hash}"); + let file_name = format!("{hash}.wav"); + let display_name = source + .file_stem() + .and_then(|name| name.to_str()) + .map(str::trim) + .filter(|name| !name.is_empty()) + .unwrap_or("Imported voice") + .chars() + .take(80) + .collect::(); + + ensure_storage_dir(&self.root)?; + let file_path = self.root.join(&file_name); + let file_created = !file_path.exists(); + if file_created { + atomic_write_restricted(&file_path, &canonical) + .map_err(|error| format!("could not save imported voice audio: {error}"))?; + } else { + if !is_regular_file_without_symlink(&file_path) { + return Err("Imported voice storage contains an unsafe file entry".to_string()); + } + let existing = fs::read(&file_path) + .map_err(|error| format!("could not verify imported voice audio: {error}"))?; + if hex::encode(Sha256::digest(&existing)) != hash { + return Err("Imported voice storage contains mismatched audio data".to_string()); + } + } + + let mut imported = ImportedVoice { + key, + display_name, + content_hash: hash, + file_name, + }; + let mut voices = self.load()?; + if let Some(existing) = voices + .iter() + .find(|voice| voice.content_hash == imported.content_hash) + { + imported = existing.clone(); + } else { + voices.push(imported.clone()); + } + if let Err(error) = self.save(&voices) { + if file_created { + let _ = fs::remove_file(&file_path); + } + return Err(error); + } + Ok(imported) + } + + pub fn delete(&self, key: &str) -> Result<(), String> { + let mut voices = self.load()?; + let index = voices + .iter() + .position(|voice| voice.key == key) + .ok_or_else(|| format!("Unknown imported voice: {key}"))?; + let previous_voices = voices.clone(); + let removed = voices.remove(index); + self.save(&voices)?; + let path = self.root.join(removed.file_name); + match fs::remove_file(path) { + Ok(()) => Ok(()), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(error) => { + self.save(&previous_voices).map_err(|rollback_error| { + format!( + "Imported voice audio could not be deleted ({error}), and its registry \ + entry could not be restored ({rollback_error})" + ) + })?; + Err(format!( + "Imported voice audio could not be deleted: {error}" + )) + } + } + } +} + +#[derive(Clone, Copy, Debug, PartialEq)] +pub struct PcmStats { + pub sample_count: usize, + pub sample_rate: u32, + pub duration_seconds: f64, + pub peak: f32, + pub rms: f32, + pub non_silent_samples: usize, +} + +impl PcmStats { + pub fn analyze(samples: &[f32], sample_rate: u32) -> Self { + let peak = samples + .iter() + .filter(|sample| sample.is_finite()) + .fold(0.0_f32, |peak, sample| peak.max(sample.abs())); + let square_sum = samples + .iter() + .filter(|sample| sample.is_finite()) + .map(|sample| sample * sample) + .sum::(); + let rms = if samples.is_empty() { + 0.0 + } else { + (square_sum / samples.len() as f32).sqrt() + }; + Self { + sample_count: samples.len(), + sample_rate, + duration_seconds: if sample_rate == 0 { + 0.0 + } else { + samples.len() as f64 / f64::from(sample_rate) + }, + peak, + rms, + non_silent_samples: samples + .iter() + .filter(|sample| sample.is_finite() && sample.abs() >= 0.001) + .count(), + } + } + + pub fn is_non_silent(self) -> bool { + self.peak >= 0.001 && self.rms >= 0.0001 && self.non_silent_samples > 0 + } +} + +pub fn write_pcm16_wav(path: &Path, samples: &[f32], sample_rate: u32) -> Result<(), String> { + let bytes = encode_pcm16_wav(samples, sample_rate); + fs::write(path, bytes).map_err(|error| format!("could not write PCM evidence: {error}")) +} + +fn ensure_storage_dir(path: &Path) -> Result<(), String> { + fs::create_dir_all(path) + .map_err(|error| format!("could not create local voice storage: {error}"))?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + fs::set_permissions(path, fs::Permissions::from_mode(0o700)) + .map_err(|error| format!("could not restrict local voice storage: {error}"))?; + } + Ok(()) +} + +fn atomic_write_restricted(path: &Path, payload: &[u8]) -> Result<(), String> { + let resolved = fs::canonicalize(path).unwrap_or_else(|_| path.to_path_buf()); + let mut file = AtomicWriteFile::open(&resolved) + .map_err(|error| format!("open {} for atomic write: {error}", resolved.display()))?; + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + file.set_permissions(fs::Permissions::from_mode(0o600)) + .map_err(|error| format!("set {} permissions: {error}", resolved.display()))?; + } + file.write_all(payload) + .map_err(|error| format!("write {}: {error}", resolved.display()))?; + file.commit() + .map_err(|error| format!("commit {}: {error}", resolved.display())) +} + +fn valid_hash(hash: &str) -> bool { + hash.len() == 64 && hash.bytes().all(|byte| byte.is_ascii_hexdigit()) +} + +fn valid_identity(voice: &ImportedVoice) -> bool { + valid_hash(&voice.content_hash) + && voice.key == format!("pocket:imported:{}", voice.content_hash) + && voice.file_name == format!("{}.wav", voice.content_hash) +} + +fn is_regular_file_without_symlink(path: &Path) -> bool { + fs::symlink_metadata(path) + .is_ok_and(|metadata| metadata.file_type().is_file() && !metadata.file_type().is_symlink()) +} + +#[derive(Debug)] +struct DecodedAudio { + sample_rate: u32, + samples: Vec, +} + +fn decode_wav(bytes: &[u8]) -> Result { + if bytes.len() < 12 || &bytes[..4] != b"RIFF" || &bytes[8..12] != b"WAVE" { + return Err("Selected file is not a valid RIFF/WAVE file".to_string()); + } + let mut offset = 12usize; + let mut format = None; + let mut data = None; + while offset.checked_add(8).is_some_and(|end| end <= bytes.len()) { + let id = &bytes[offset..offset + 4]; + let size = + u32::from_le_bytes(bytes[offset + 4..offset + 8].try_into().unwrap_or([0; 4])) as usize; + let start = offset + 8; + let end = start.checked_add(size).ok_or("WAV chunk size overflow")?; + if end > bytes.len() { + return Err("Selected WAV contains a truncated chunk".to_string()); + } + if id == b"fmt " { + format = Some(&bytes[start..end]); + } else if id == b"data" { + data = Some(&bytes[start..end]); + } + offset = end + (size & 1); + } + let format = format.ok_or("Selected WAV has no format chunk")?; + let data = data.ok_or("Selected WAV has no audio data")?; + if format.len() < 16 { + return Err("Selected WAV has an invalid format chunk".to_string()); + } + let encoding = u16::from_le_bytes(format[0..2].try_into().unwrap_or([0; 2])); + let encoding = if encoding == 0xfffe && format.len() >= 40 { + u16::from_le_bytes(format[24..26].try_into().unwrap_or([0; 2])) + } else { + encoding + }; + let channels = u16::from_le_bytes(format[2..4].try_into().unwrap_or([0; 2])); + let sample_rate = u32::from_le_bytes(format[4..8].try_into().unwrap_or([0; 4])); + let block_align = u16::from_le_bytes(format[12..14].try_into().unwrap_or([0; 2])) as usize; + let bits = u16::from_le_bytes(format[14..16].try_into().unwrap_or([0; 2])); + if channels == 0 || channels > 8 { + return Err("Voice WAV must contain between 1 and 8 channels".to_string()); + } + if !(MIN_SAMPLE_RATE..=MAX_SAMPLE_RATE).contains(&sample_rate) { + return Err("Voice WAV sample rate must be between 8 and 96 kHz".to_string()); + } + let bytes_per_sample = usize::from(bits.div_ceil(8)); + if block_align != bytes_per_sample * usize::from(channels) + || block_align == 0 + || data.len() % block_align != 0 + { + return Err("Voice WAV has invalid sample alignment".to_string()); + } + if !matches!((encoding, bits), (1, 8 | 16 | 24 | 32) | (3, 32)) { + return Err("Voice WAV must contain PCM or 32-bit float audio".to_string()); + } + let frames = data.len() / block_align; + let duration = frames as f64 / f64::from(sample_rate); + if !(MIN_DURATION_SECONDS..=MAX_DURATION_SECONDS).contains(&duration) { + return Err("Voice WAV must be between 2 and 30 seconds long".to_string()); + } + + let mut samples = Vec::with_capacity(frames); + for frame in data.chunks_exact(block_align) { + let mut mono = 0.0_f32; + for chunk in frame.chunks_exact(bytes_per_sample) { + let sample = match (encoding, bits) { + (1, 8) => (f32::from(chunk[0]) - 128.0) / 128.0, + (1, 16) => f32::from(i16::from_le_bytes([chunk[0], chunk[1]])) / 32768.0, + (1, 24) => { + let raw = i32::from_le_bytes([ + chunk[0], + chunk[1], + chunk[2], + if chunk[2] & 0x80 == 0 { 0 } else { 0xff }, + ]); + raw as f32 / 8_388_608.0 + } + (1, 32) => { + i32::from_le_bytes(chunk.try_into().map_err(|_| "invalid PCM sample")?) as f32 + / 2_147_483_648.0 + } + (3, 32) => f32::from_le_bytes( + chunk + .try_into() + .map_err(|_| "invalid floating-point sample")?, + ), + _ => unreachable!(), + }; + if !sample.is_finite() { + return Err("Voice WAV contains non-finite samples".to_string()); + } + mono += sample; + } + samples.push((mono / f32::from(channels)).clamp(-1.0, 1.0)); + } + let stats = PcmStats::analyze(&samples, sample_rate); + if !stats.is_non_silent() { + return Err("Voice WAV is silent or too quiet to clone".to_string()); + } + Ok(DecodedAudio { + sample_rate, + samples, + }) +} + +fn decode_media(source: &Path, extension: &str) -> Result { + let supported = ["m4a", "mp3", "flac", "ogg", "oga", "aif", "aiff"]; + if !supported.contains(&extension) { + return Err(format!( + "Unsupported voice audio format .{extension}. Choose WAV, M4A, MP3, FLAC, OGG, or AIFF" + )); + } + + let file = fs::File::open(source) + .map_err(|error| format!("could not read selected audio: {error}"))?; + let media = MediaSourceStream::new(Box::new(file), Default::default()); + let mut hint = Hint::new(); + hint.with_extension(extension); + let probed = symphonia::default::get_probe() + .format( + &hint, + media, + &FormatOptions::default(), + &MetadataOptions::default(), + ) + .map_err(|error| format!("could not recognize selected audio: {error}"))?; + let mut format = probed.format; + let track = format + .default_track() + .ok_or_else(|| "Selected audio has no decodable track".to_string())?; + let track_id = track.id; + let mut decoder = symphonia::default::get_codecs() + .make(&track.codec_params, &DecoderOptions::default()) + .map_err(|error| format!("could not initialize audio decoder: {error}"))?; + let mut sample_rate = None; + let mut samples = Vec::new(); + + loop { + let packet = match format.next_packet() { + Ok(packet) => packet, + Err(SymphoniaError::ResetRequired) => { + return Err("Selected audio changes format mid-stream".to_string()); + } + Err(SymphoniaError::IoError(error)) + if error.kind() == std::io::ErrorKind::UnexpectedEof => + { + break; + } + Err(error) => return Err(format!("could not read selected audio: {error}")), + }; + if packet.track_id() != track_id { + continue; + } + let decoded = match decoder.decode(&packet) { + Ok(decoded) => decoded, + Err(SymphoniaError::DecodeError(_)) => continue, + Err(error) => return Err(format!("could not decode selected audio: {error}")), + }; + let spec = *decoded.spec(); + if !(MIN_SAMPLE_RATE..=MAX_SAMPLE_RATE).contains(&spec.rate) { + return Err("Voice audio sample rate must be between 8 and 96 kHz".to_string()); + } + if sample_rate.is_some_and(|rate| rate != spec.rate) { + return Err("Selected audio changes sample rate mid-stream".to_string()); + } + sample_rate = Some(spec.rate); + let channels = spec.channels.count(); + if channels == 0 || channels > 8 { + return Err("Voice audio must contain between 1 and 8 channels".to_string()); + } + let mut buffer = SampleBuffer::::new(decoded.capacity() as u64, spec); + buffer.copy_interleaved_ref(decoded); + for frame in buffer.samples().chunks_exact(channels) { + let mono = frame.iter().copied().sum::() / channels as f32; + if !mono.is_finite() { + return Err("Voice audio contains non-finite samples".to_string()); + } + samples.push(mono.clamp(-1.0, 1.0)); + } + if samples.len() as f64 > MAX_DURATION_SECONDS * f64::from(spec.rate) { + return Err("Voice audio must be between 2 and 30 seconds long".to_string()); + } + } + + let sample_rate = + sample_rate.ok_or_else(|| "Selected audio contains no samples".to_string())?; + validate_decoded_audio(&samples, sample_rate)?; + Ok(DecodedAudio { + sample_rate, + samples, + }) +} + +fn validate_decoded_audio(samples: &[f32], sample_rate: u32) -> Result<(), String> { + let stats = PcmStats::analyze(samples, sample_rate); + if !(MIN_DURATION_SECONDS..=MAX_DURATION_SECONDS).contains(&stats.duration_seconds) { + return Err("Voice audio must be between 2 and 30 seconds long".to_string()); + } + if !stats.is_non_silent() { + return Err("Voice audio is silent or too quiet to clone".to_string()); + } + Ok(()) +} + +fn resample_linear(samples: &[f32], source_rate: u32) -> Vec { + if source_rate == CANONICAL_SAMPLE_RATE { + return samples.to_vec(); + } + let output_len = ((samples.len() as u64 * u64::from(CANONICAL_SAMPLE_RATE) + + u64::from(source_rate) / 2) + / u64::from(source_rate)) as usize; + (0..output_len) + .map(|index| { + let source = index as f64 * f64::from(source_rate) / f64::from(CANONICAL_SAMPLE_RATE); + let left = source.floor() as usize; + let fraction = (source - left as f64) as f32; + let a = samples[left.min(samples.len() - 1)]; + let b = samples[(left + 1).min(samples.len() - 1)]; + a + (b - a) * fraction + }) + .collect() +} + +fn encode_pcm16_wav(samples: &[f32], sample_rate: u32) -> Vec { + let data_len = (samples.len() * 2) as u32; + let mut bytes = Vec::with_capacity(44 + data_len as usize); + bytes.extend_from_slice(b"RIFF"); + bytes.extend_from_slice(&(36 + data_len).to_le_bytes()); + bytes.extend_from_slice(b"WAVEfmt "); + bytes.extend_from_slice(&16_u32.to_le_bytes()); + bytes.extend_from_slice(&1_u16.to_le_bytes()); + bytes.extend_from_slice(&1_u16.to_le_bytes()); + bytes.extend_from_slice(&sample_rate.to_le_bytes()); + bytes.extend_from_slice(&(sample_rate * 2).to_le_bytes()); + bytes.extend_from_slice(&2_u16.to_le_bytes()); + bytes.extend_from_slice(&16_u16.to_le_bytes()); + bytes.extend_from_slice(b"data"); + bytes.extend_from_slice(&data_len.to_le_bytes()); + for sample in samples { + let value = (sample.clamp(-1.0, 1.0) * f32::from(i16::MAX)).round() as i16; + bytes.extend_from_slice(&value.to_le_bytes()); + } + bytes +} + +#[cfg(test)] +mod tests { + use super::*; + + fn fixture(sample_rate: u32, seconds: usize, amplitude: f32) -> Vec { + let samples = (0..sample_rate as usize * seconds) + .map(|index| { + amplitude + * (std::f32::consts::TAU * 220.0 * index as f32 / sample_rate as f32).sin() + }) + .collect::>(); + encode_pcm16_wav(&samples, sample_rate) + } + + fn stereo_fixture(sample_rate: u32, seconds: usize, amplitude: f32) -> Vec { + let mono = fixture(sample_rate, seconds, amplitude); + let mono_data = &mono[44..]; + let mut stereo_data = Vec::with_capacity(mono_data.len() * 2); + for sample in mono_data.chunks_exact(2) { + stereo_data.extend_from_slice(sample); + stereo_data.extend_from_slice(sample); + } + let mut stereo = mono[..44].to_vec(); + stereo[4..8].copy_from_slice(&(36 + stereo_data.len() as u32).to_le_bytes()); + stereo[22..24].copy_from_slice(&2_u16.to_le_bytes()); + stereo[28..32].copy_from_slice(&(sample_rate * 4).to_le_bytes()); + stereo[32..34].copy_from_slice(&4_u16.to_le_bytes()); + stereo[40..44].copy_from_slice(&(stereo_data.len() as u32).to_le_bytes()); + stereo.extend_from_slice(&stereo_data); + stereo + } + + #[test] + fn imports_persists_reloads_and_deletes_canonical_voice() { + let temp = tempfile::tempdir().expect("temp voice workspace"); + let source = temp.path().join("My voice.wav"); + fs::write(&source, fixture(44_100, 2, 0.5)).expect("write source"); + let library = PocketVoiceLibrary::new(temp.path().join("library")); + + let imported = library.import_path(&source).expect("import voice"); + assert!(imported.key.starts_with("pocket:imported:")); + assert_eq!(imported.display_name, "My voice"); + + let relaunched = PocketVoiceLibrary::new(library.root()); + assert_eq!( + relaunched.load().expect("reload registry"), + vec![imported.clone()] + ); + let stored = relaunched + .resolve_file(&imported) + .expect("resolve stored voice"); + let decoded = decode_wav(&fs::read(&stored).expect("read stored voice")) + .expect("decode canonical voice"); + assert_eq!(decoded.sample_rate, CANONICAL_SAMPLE_RATE); + assert_eq!(decoded.samples.len(), CANONICAL_SAMPLE_RATE as usize * 2); + + assert_eq!( + relaunched.import_path(&source).expect("idempotent import"), + imported + ); + assert_eq!(relaunched.load().expect("deduplicated registry").len(), 1); + + relaunched.delete(&imported.key).expect("delete voice"); + assert!(relaunched.load().expect("empty registry").is_empty()); + assert!(!stored.exists()); + } + + #[test] + fn common_stereo_audio_is_downmixed_to_canonical_mono() { + let temp = tempfile::tempdir().expect("temp voice workspace"); + let source = temp.path().join("stereo.wav"); + fs::write(&source, stereo_fixture(44_100, 2, 0.5)).expect("write stereo"); + let library = PocketVoiceLibrary::new(temp.path().join("library")); + + let imported = library.import_path(&source).expect("import stereo"); + let stored = library + .resolve_file(&imported) + .expect("resolve stored voice"); + let decoded = decode_wav(&fs::read(stored).expect("read stored voice")) + .expect("decode canonical voice"); + assert_eq!(decoded.sample_rate, CANONICAL_SAMPLE_RATE); + assert_eq!(decoded.samples.len(), CANONICAL_SAMPLE_RATE as usize * 2); + } + + #[test] + #[ignore = "requires BUZZ_VOICE_IMPORT_TEST_DIR with common-format fixtures"] + fn imports_common_audio_format_fixtures() { + let fixtures = + PathBuf::from(std::env::var("BUZZ_VOICE_IMPORT_TEST_DIR").expect("fixture directory")); + let temp = tempfile::tempdir().expect("temp voice workspace"); + let library = PocketVoiceLibrary::new(temp.path().join("library")); + + for file_name in [ + "voice.wav", + "voice.m4a", + "voice.mp3", + "voice.flac", + "voice.ogg", + "voice.aiff", + ] { + let imported = library + .import_path(&fixtures.join(file_name)) + .unwrap_or_else(|error| panic!("import {file_name}: {error}")); + let stored = library + .resolve_file(&imported) + .unwrap_or_else(|error| panic!("resolve {file_name}: {error}")); + let decoded = decode_wav(&fs::read(stored).expect("read canonical voice")) + .expect("decode canonical voice"); + assert_eq!(decoded.sample_rate, CANONICAL_SAMPLE_RATE); + assert!(decoded.samples.len() >= CANONICAL_SAMPLE_RATE as usize * 2); + } + } + + #[test] + fn invalid_unsupported_and_silent_files_do_not_mutate_registry() { + let temp = tempfile::tempdir().expect("temp voice workspace"); + let library = PocketVoiceLibrary::new(temp.path().join("library")); + + let garbage = temp.path().join("garbage.wav"); + fs::write(&garbage, b"not a wave").expect("write garbage"); + assert!(library + .import_path(&garbage) + .expect_err("garbage rejected") + .contains("RIFF/WAVE")); + + let silent = temp.path().join("silent.wav"); + fs::write(&silent, fixture(32_000, 2, 0.0)).expect("write silence"); + assert!(library + .import_path(&silent) + .expect_err("silence rejected") + .contains("silent")); + + let unsupported_container = temp.path().join("voice.txt"); + fs::write(&unsupported_container, b"not audio").expect("write unsupported container"); + assert!(library + .import_path(&unsupported_container) + .expect_err("container rejected") + .contains("Unsupported voice audio format")); + + let mut unsupported = fixture(32_000, 2, 0.5); + unsupported[20..22].copy_from_slice(&6_u16.to_le_bytes()); + let unsupported_path = temp.path().join("unsupported.wav"); + fs::write(&unsupported_path, unsupported).expect("write unsupported"); + assert!(library + .import_path(&unsupported_path) + .expect_err("unsupported rejected") + .contains("PCM or 32-bit float")); + + assert!(library.load().expect("unchanged registry").is_empty()); + } + + #[test] + fn pcm_analysis_distinguishes_signal_from_silence() { + let signal = (0..24_000) + .map(|index| (std::f32::consts::TAU * 440.0 * index as f32 / 24_000.0).sin() * 0.5) + .collect::>(); + let signal_stats = PcmStats::analyze(&signal, 24_000); + assert!(signal_stats.is_non_silent()); + assert_eq!(signal_stats.duration_seconds, 1.0); + assert!(signal_stats.peak > 0.49); + assert!(signal_stats.rms > 0.3); + + let silence = vec![0.0; 24_000]; + assert!(!PcmStats::analyze(&silence, 24_000).is_non_silent()); + } +} diff --git a/crates/buzz-voice/src/lib.rs b/crates/buzz-voice/src/lib.rs new file mode 100644 index 0000000000..e4b4ebfed3 --- /dev/null +++ b/crates/buzz-voice/src/lib.rs @@ -0,0 +1,23 @@ +//! Reusable local voice primitives for Buzz. + +pub mod imported; +pub mod pocket; + +pub use pocket::{ + april_model_info, load_text_to_speech, load_voice_style, PocketModelInfo, PocketTts, + VoiceStyle, DEFAULT_VOICE, SAMPLE_RATE, VOICE_FILE_EXT, +}; + +/// One immutable artifact required by the April Pocket bundle. +/// +/// `filename` is the bundle-relative file name, `sha256` pins its contents, +/// `size_bytes` supports download progress and validation, and `quantized` +/// identifies the INT8 components. +pub type PocketModelArtifact = pocket::PocketModelArtifact; + +/// Language bundle selected from the pinned export. +pub const APRIL_BUNDLE_ID: &str = pocket::APRIL_BUNDLE_ID; +/// Pinned upstream export repository. +pub const APRIL_MODEL_ID: &str = pocket::APRIL_MODEL_ID; +/// Pinned revision containing the April bundle. +pub const APRIL_MODEL_REVISION: &str = pocket::APRIL_MODEL_REVISION; diff --git a/crates/buzz-voice/src/pocket.rs b/crates/buzz-voice/src/pocket.rs new file mode 100644 index 0000000000..0c6174a8dc --- /dev/null +++ b/crates/buzz-voice/src/pocket.rs @@ -0,0 +1,167 @@ +//! April 2026 Pocket TTS engine for Buzz Desktop. +//! +//! The `english_2026-04` bundle uses SentencePiece tokenization, a learned +//! voice BOS embedding, recurrent FlowLM state, and stateful Mimi decoding. +//! Buzz selects the upstream three-graph INT8 variant while retaining the +//! full-precision Mimi encoder and text conditioner specified by that variant. +//! +//! ## Attribution +//! +//! - Pocket TTS and Mimi: Kyutai, CC-BY-4.0. +//! - ONNX export: KevinAHM/pocket-tts-onnx, CC-BY-4.0. +//! - Reference voice: Kyutai's Mary preset (VCTK p333), CC-BY-4.0. +//! +//! `huddle::models` writes the complete attribution beside the cached bytes. + +use std::path::{Path, PathBuf}; +use std::sync::Mutex; + +use sherpa_onnx::Wave; + +#[path = "pocket_april.rs"] +mod pocket_april; +#[path = "pocket_models.rs"] +mod pocket_models; + +use pocket_april::{prepare_april_prompt, AprilPocketTts}; +pub use pocket_models::{ + april_model_info, PocketModelArtifact, PocketModelInfo, APRIL_BUNDLE_ID, APRIL_MODEL_ID, + APRIL_MODEL_REVISION, +}; + +/// Pocket TTS emits 24 kHz mono PCM. +pub const SAMPLE_RATE: u32 = 24_000; + +/// Bundled reference voice name without its extension. +pub const DEFAULT_VOICE: &str = "reference_sample"; + +/// Pocket voice files are reference WAVs. +pub const VOICE_FILE_EXT: &str = "wav"; + +const TTS_NUM_THREADS: usize = 1; + +/// Loaded reference voice samples and their original sample rate. +#[derive(Debug, Clone)] +pub struct VoiceStyle { + samples: Vec, + sample_rate: i32, +} + +/// Load a Pocket reference voice WAV from disk. +pub fn load_voice_style(path: &Path) -> Result { + let path_str = path + .to_str() + .ok_or_else(|| format!("voice path is not valid UTF-8: {}", path.display()))?; + let wave = Wave::read(path_str) + .ok_or_else(|| format!("could not read voice WAV at {}", path.display()))?; + let samples = wave.samples().to_vec(); + if samples.is_empty() { + return Err(format!("voice WAV is empty: {}", path.display())); + } + Ok(VoiceStyle { + samples, + sample_rate: wave.sample_rate(), + }) +} + +/// Resident April INT8 Pocket TTS engine. +pub struct PocketTts { + inner: Mutex, +} + +/// Load Buzz Desktop's pinned April INT8 model. +pub fn load_text_to_speech(model_dir: &str) -> Result { + let dir = PathBuf::from(model_dir); + for artifact in april_model_info().artifacts { + let path = dir.join(artifact.filename); + if !path.is_file() { + return Err(format!( + "incomplete Pocket TTS {} INT8 bundle: missing {}", + APRIL_BUNDLE_ID, + path.display() + )); + } + } + Ok(PocketTts { + inner: Mutex::new(AprilPocketTts::load(&dir, TTS_NUM_THREADS)?), + }) +} + +impl PocketTts { + /// Split text into synthesis units that satisfy the bundle's exact + /// 50-token input limit. + pub fn split_text_into_chunks(&self, text: &str) -> Result, String> { + let Some(prepared) = prepare_april_prompt(text) else { + return Ok(Vec::new()); + }; + self.inner + .lock() + .map_err(|_| "Pocket TTS engine lock poisoned".to_string())? + .split_prompt(&prepared) + } + + /// Synthesize text with the supplied reference voice. + /// + /// Pocket detects language from text and this model uses one synthesis + /// step, so `_lang` and `_steps` intentionally do not affect output. + pub fn synth_chunk( + &self, + text: &str, + _lang: &str, + style: &VoiceStyle, + _steps: usize, + ) -> Result, String> { + let Some(prepared) = prepare_april_prompt(text) else { + return Ok(Vec::new()); + }; + let mut engine = self + .inner + .lock() + .map_err(|_| "Pocket TTS engine lock poisoned".to_string())?; + let chunks = engine.split_prompt(&prepared)?; + let mut samples = Vec::new(); + for chunk in chunks { + let prepared = prepare_april_prompt(&chunk) + .ok_or_else(|| "Pocket TTS prompt chunk became empty".to_string())?; + samples.extend(engine.synth_chunk(&prepared, style)?); + } + Ok(samples) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn desktop_model_is_april_int8_only() { + let info = april_model_info(); + assert_eq!(info.max_token_per_chunk, 50); + assert_eq!(info.sample_rate, SAMPLE_RATE); + assert!(info + .artifacts + .iter() + .any(|artifact| artifact.filename == "flow_lm_main_int8.onnx")); + assert!(!info + .artifacts + .iter() + .any(|artifact| artifact.filename == "flow_lm_main.onnx")); + } + + #[test] + #[ignore = "requires BUZZ_POCKET_TEST_MODEL_DIR"] + fn production_api_emits_non_silent_april_int8_pcm() { + let dir = std::env::var("BUZZ_POCKET_TEST_MODEL_DIR") + .expect("set BUZZ_POCKET_TEST_MODEL_DIR to an April INT8 model directory"); + let engine = load_text_to_speech(&dir).expect("load April INT8 engine"); + let style = load_voice_style(&Path::new(&dir).join("reference_sample.wav")) + .expect("load reference voice"); + let samples = engine + .synth_chunk("Bright birds begin beside the bay.", "en", &style, 1) + .expect("synthesize through the production API"); + + assert!(!samples.is_empty()); + assert!(samples.iter().all(|sample| sample.is_finite())); + assert!(samples.iter().any(|sample| sample.abs() > 1.0e-6)); + } +} diff --git a/crates/buzz-voice/src/pocket_april.rs b/crates/buzz-voice/src/pocket_april.rs new file mode 100644 index 0000000000..43826df5c9 --- /dev/null +++ b/crates/buzz-voice/src/pocket_april.rs @@ -0,0 +1,940 @@ +//! Native ONNX loader for Pocket TTS `english_2026-04`. +//! +//! The bundle uses SentencePiece, prepends a learned BOS voice embedding, and +//! describes recurrent state tensors in `bundle.json`. This module supplies +//! that frontend and state loop while reusing the ONNX Runtime linked by the +//! Desktop speech stack. + +use std::borrow::Cow; +use std::f32::consts::TAU; +use std::fs; +use std::path::{Path, PathBuf}; + +use ort::session::{Session, SessionInputValue}; +use ort::value::{DynValue, Tensor}; +use rand::{Rng, RngExt}; +use sentencepiece_model::SentencePieceModel; +use serde::Deserialize; +use sherpa_onnx::LinearResampler; +use tokenizers::models::unigram::Unigram; +use tokenizers::pre_tokenizers::metaspace::{Metaspace, PrependScheme}; +use tokenizers::Tokenizer; + +use super::VoiceStyle; + +const FILE_BUNDLE: &str = "bundle.json"; +const FILE_MIMI_ENCODER: &str = "mimi_encoder.onnx"; +const FILE_TEXT_CONDITIONER: &str = "text_conditioner.onnx"; +const FILE_FLOW_MAIN_INT8: &str = "flow_lm_main_int8.onnx"; +const FILE_FLOW_INT8: &str = "flow_lm_flow_int8.onnx"; +const FILE_MIMI_DECODER_INT8: &str = "mimi_decoder_int8.onnx"; + +const MODEL_LANGUAGE: &str = "english_2026-04"; +const DEFAULT_TEMPERATURE: f32 = 0.7; +const EOS_LOGIT_THRESHOLD: f32 = -4.0; +const DECODER_CHUNK_FRAMES: usize = 12; +const TOKENS_PER_SECOND_ESTIMATE: f32 = 3.0; +const GENERATION_SECONDS_PADDING: f32 = 2.0; + +#[derive(Debug, Deserialize)] +struct Bundle { + schema_version: u32, + language: String, + sample_rate: usize, + frame_rate: f32, + samples_per_frame: usize, + latent_dim: usize, + conditioning_dim: usize, + insert_bos_before_voice: bool, + pad_with_spaces_for_short_inputs: bool, + remove_semicolons: bool, + model_recommended_frames_after_eos: Option, + max_token_per_chunk: usize, + tokenizer_file: String, + bos_before_voice_file: String, + flow_lm_state_manifest: Vec, + mimi_state_manifest: Vec, +} + +#[derive(Debug, Clone, Deserialize)] +struct StateSpec { + input_name: String, + output_name: String, + dtype: StateDtype, + shape: Vec, + fill: StateFill, +} + +#[derive(Debug, Clone, Copy, Deserialize)] +#[serde(rename_all = "lowercase")] +enum StateDtype { + #[serde(rename = "float32")] + Float32, + #[serde(rename = "int64")] + Int64, + Bool, +} + +#[derive(Debug, Clone, Copy, Deserialize)] +#[serde(rename_all = "lowercase")] +enum StateFill { + Empty, + Nan, + Ones, + Zeros, +} + +struct StateValue { + spec: StateSpec, + value: DynValue, +} + +struct CachedVoice { + samples_ptr: usize, + samples_len: usize, + sample_rate: i32, + embeddings: Vec, +} + +pub(crate) struct AprilPocketTts { + bundle: Bundle, + tokenizer: Tokenizer, + bos_embedding: Vec, + mimi_encoder: Session, + text_conditioner: Session, + flow_main: Session, + flow: Session, + mimi_decoder: Session, + cached_voice: Option, +} + +#[derive(Debug, Clone, PartialEq)] +pub(crate) struct AprilPreparedPrompt { + pub(crate) text: String, + pub(crate) frames_after_eos: usize, +} + +pub(crate) fn prepare_april_prompt(input: &str) -> Option { + let trimmed = input.trim(); + if trimmed.is_empty() { + return None; + } + + let mut cleaned = String::with_capacity(trimmed.len()); + let mut last_was_space = false; + for ch in trimmed.chars() { + if ch.is_whitespace() { + if !last_was_space { + cleaned.push(' '); + } + last_was_space = true; + } else { + cleaned.push(ch); + last_was_space = false; + } + } + + let first = cleaned.chars().next().expect("cleaned non-empty above"); + if first.is_lowercase() { + let upper: String = first.to_uppercase().collect(); + let mut iter = cleaned.chars(); + iter.next(); + cleaned = upper + iter.as_str(); + } + + let last = cleaned + .chars() + .next_back() + .expect("cleaned non-empty above"); + if last.is_alphanumeric() { + cleaned.push('.'); + } + + let word_count = cleaned.split_whitespace().count(); + Some(AprilPreparedPrompt { + text: cleaned, + // Mirror the bundle's upstream heuristic: three generated frames plus + // two trailing frames for short prompts, one plus two otherwise. + frames_after_eos: if word_count <= 4 { 5 } else { 3 }, + }) +} + +impl AprilPocketTts { + pub(crate) fn load(dir: &Path, num_threads: usize) -> Result { + if num_threads == 0 { + return Err("Pocket TTS num_threads must be at least 1".to_string()); + } + let bundle_path = dir.join(FILE_BUNDLE); + let bundle: Bundle = serde_json::from_slice( + &fs::read(&bundle_path) + .map_err(|err| format!("read {}: {err}", bundle_path.display()))?, + ) + .map_err(|err| format!("parse {}: {err}", bundle_path.display()))?; + + if bundle.schema_version != 2 { + return Err(format!( + "unsupported Pocket TTS bundle schema {} in {}", + bundle.schema_version, + bundle_path.display() + )); + } + if bundle.language != MODEL_LANGUAGE { + return Err(format!( + "expected Pocket TTS language {MODEL_LANGUAGE}, got {}", + bundle.language + )); + } + if bundle.sample_rate != 24_000 + || bundle.frame_rate != 12.5 + || bundle.samples_per_frame != 1_920 + || bundle.latent_dim != 32 + || bundle.conditioning_dim != 1024 + { + return Err(format!( + "unexpected Pocket TTS dimensions: sample_rate={}, frame_rate={}, samples_per_frame={}, latent_dim={}, conditioning_dim={}", + bundle.sample_rate, + bundle.frame_rate, + bundle.samples_per_frame, + bundle.latent_dim, + bundle.conditioning_dim + )); + } + if !bundle.insert_bos_before_voice { + return Err("April Pocket TTS bundle must insert BOS before voice".to_string()); + } + if bundle.pad_with_spaces_for_short_inputs + || bundle.remove_semicolons + || bundle.model_recommended_frames_after_eos.is_some() + || bundle.max_token_per_chunk != 50 + { + return Err("unsupported April Pocket TTS prompt-policy metadata".to_string()); + } + + let tokenizer_path = dir.join(&bundle.tokenizer_file); + let tokenizer = load_tokenizer(&tokenizer_path)?; + let bos_path = dir.join(&bundle.bos_before_voice_file); + let bos_embedding = read_npy_f32(&bos_path)?; + if bos_embedding.len() != bundle.conditioning_dim { + return Err(format!( + "{} has {} values; expected {}", + bos_path.display(), + bos_embedding.len(), + bundle.conditioning_dim + )); + } + + let flow_main = FILE_FLOW_MAIN_INT8; + let flow = FILE_FLOW_INT8; + let mimi_decoder = FILE_MIMI_DECODER_INT8; + + Ok(Self { + // The INT8 layout quantizes only the three generation graphs; + // voice encoding and text conditioning remain full precision. + mimi_encoder: load_session(dir.join(FILE_MIMI_ENCODER), num_threads)?, + text_conditioner: load_session(dir.join(FILE_TEXT_CONDITIONER), num_threads)?, + flow_main: load_session(dir.join(flow_main), num_threads)?, + flow: load_session(dir.join(flow), num_threads)?, + mimi_decoder: load_session(dir.join(mimi_decoder), num_threads)?, + bundle, + tokenizer, + bos_embedding, + cached_voice: None, + }) + } + + pub(crate) fn split_prompt( + &self, + prepared: &AprilPreparedPrompt, + ) -> Result, String> { + if self.token_count(&prepared.text)? <= self.bundle.max_token_per_chunk { + return Ok(vec![prepared.text.clone()]); + } + + let mut chunks = Vec::new(); + let mut current = String::new(); + for word in prepared.text.split_whitespace() { + let candidate = if current.is_empty() { + word.to_string() + } else { + format!("{current} {word}") + }; + if self.prepared_token_count(&candidate)? <= self.bundle.max_token_per_chunk { + current = candidate; + continue; + } + if !current.is_empty() { + chunks.push(std::mem::take(&mut current)); + } + + if self.prepared_token_count(word)? <= self.bundle.max_token_per_chunk { + current = word.to_string(); + continue; + } + + let mut fragment = String::new(); + for ch in word.chars() { + let candidate = format!("{fragment}{ch}"); + if !fragment.is_empty() + && self.prepared_token_count(&candidate)? > self.bundle.max_token_per_chunk + { + chunks.push(std::mem::take(&mut fragment)); + } + fragment.push(ch); + } + current = fragment; + } + if !current.is_empty() { + chunks.push(current); + } + + chunks + .into_iter() + .map(|text| { + let chunk = prepare_april_prompt(&text) + .ok_or_else(|| "Pocket TTS prompt chunk became empty".to_string())?; + let token_count = self.token_count(&chunk.text)?; + if token_count > self.bundle.max_token_per_chunk { + return Err(format!( + "Pocket TTS prompt chunk has {token_count} tokens; maximum is {}", + self.bundle.max_token_per_chunk + )); + } + Ok(chunk.text) + }) + .collect() + } + + pub(crate) fn synth_chunk( + &mut self, + prepared: &AprilPreparedPrompt, + style: &VoiceStyle, + ) -> Result, String> { + let voice_embeddings = self.voice_embeddings(style)?; + let mut flow_state = self.condition_voice(&voice_embeddings)?; + let token_ids = self + .tokenizer + .encode(prepared.text.as_str(), false) + .map_err(|err| format!("tokenize Pocket TTS prompt: {err}"))? + .get_ids() + .iter() + .copied() + .map(i64::from) + .collect::>(); + if token_ids.is_empty() { + return Ok(Vec::new()); + } + if token_ids.len() > self.bundle.max_token_per_chunk { + return Err(format!( + "Pocket TTS prompt has {} tokens; split_text_into_chunks maximum is {}", + token_ids.len(), + self.bundle.max_token_per_chunk + )); + } + + let token_count = token_ids.len(); + let text_embeddings = self.text_embeddings(token_ids)?; + self.run_flow_main_prefix(&text_embeddings, &mut flow_state)?; + let max_frames = estimate_max_frames(token_count, self.bundle.frame_rate); + let latents = + self.generate_latents(max_frames, prepared.frames_after_eos, &mut flow_state)?; + self.decode_latents(&latents) + } + + fn prepared_token_count(&self, text: &str) -> Result { + let prepared = prepare_april_prompt(text) + .ok_or_else(|| "Pocket TTS prompt chunk became empty".to_string())?; + self.token_count(&prepared.text) + } + + fn token_count(&self, text: &str) -> Result { + Ok(self + .tokenizer + .encode(text, false) + .map_err(|err| format!("tokenize Pocket TTS prompt: {err}"))? + .get_ids() + .len()) + } + + fn voice_embeddings(&mut self, style: &VoiceStyle) -> Result, String> { + let key = ( + style.samples.as_ptr() as usize, + style.samples.len(), + style.sample_rate, + ); + if let Some(cached) = &self.cached_voice { + if (cached.samples_ptr, cached.samples_len, cached.sample_rate) == key { + return Ok(cached.embeddings.clone()); + } + } + + let samples = if style.sample_rate == self.bundle.sample_rate as i32 { + style.samples.clone() + } else { + LinearResampler::create(style.sample_rate, self.bundle.sample_rate as i32) + .ok_or_else(|| { + format!( + "create Pocket TTS resampler {}Hz -> {}Hz", + style.sample_rate, self.bundle.sample_rate + ) + })? + .resample(&style.samples, true) + }; + let audio = Tensor::from_array(( + vec![1_i64, 1, samples.len() as i64], + samples.into_boxed_slice(), + )) + .map_err(ort_error("create voice audio tensor"))?; + let outputs = self + .mimi_encoder + .run(ort::inputs!["audio" => audio]) + .map_err(ort_error("run Mimi encoder"))?; + let (_, encoded) = outputs[0] + .try_extract_tensor::() + .map_err(ort_error("extract Mimi encoder output"))?; + if !encoded.len().is_multiple_of(self.bundle.conditioning_dim) { + return Err(format!( + "Mimi encoder returned {} values, not divisible by {}", + encoded.len(), + self.bundle.conditioning_dim + )); + } + let mut embeddings = + Vec::with_capacity(self.bos_embedding.len().saturating_add(encoded.len())); + embeddings.extend_from_slice(&self.bos_embedding); + embeddings.extend_from_slice(encoded); + self.cached_voice = Some(CachedVoice { + samples_ptr: key.0, + samples_len: key.1, + sample_rate: key.2, + embeddings: embeddings.clone(), + }); + Ok(embeddings) + } + + fn condition_voice(&mut self, embeddings: &[f32]) -> Result, String> { + let frames = embeddings.len() / self.bundle.conditioning_dim; + let sequence = Tensor::::new( + &ort::memory::Allocator::default(), + [1_i64, 0, self.bundle.latent_dim as i64], + ) + .map_err(ort_error("create empty voice sequence"))?; + let text_embeddings = Tensor::from_array(( + vec![1_i64, frames as i64, self.bundle.conditioning_dim as i64], + embeddings.to_vec().into_boxed_slice(), + )) + .map_err(ort_error("create voice embedding tensor"))?; + let mut state = initialize_state(&self.bundle.flow_lm_state_manifest)?; + let mut inputs = vec![ + (Cow::Borrowed("sequence"), SessionInputValue::from(sequence)), + ( + Cow::Borrowed("text_embeddings"), + SessionInputValue::from(text_embeddings), + ), + ]; + append_state_inputs(&mut inputs, &state); + let mut outputs = self + .flow_main + .run(inputs) + .map_err(ort_error("condition Pocket TTS voice"))?; + replace_state_from_outputs(&mut state, &mut outputs)?; + Ok(state) + } + + fn text_embeddings(&mut self, token_ids: Vec) -> Result, String> { + let tokens = Tensor::from_array(( + vec![1_i64, token_ids.len() as i64], + token_ids.into_boxed_slice(), + )) + .map_err(ort_error("create token tensor"))?; + let outputs = self + .text_conditioner + .run(ort::inputs!["token_ids" => tokens]) + .map_err(ort_error("run text conditioner"))?; + let (_, embeddings) = outputs[0] + .try_extract_tensor::() + .map_err(ort_error("extract text embeddings"))?; + Ok(embeddings.to_vec()) + } + + fn run_flow_main_prefix( + &mut self, + text_embeddings: &[f32], + state: &mut [StateValue], + ) -> Result<(), String> { + if !text_embeddings + .len() + .is_multiple_of(self.bundle.conditioning_dim) + { + return Err(format!( + "text conditioner returned {} values, not divisible by {}", + text_embeddings.len(), + self.bundle.conditioning_dim + )); + } + let frames = text_embeddings.len() / self.bundle.conditioning_dim; + let sequence = Tensor::::new( + &ort::memory::Allocator::default(), + [1_i64, 0, self.bundle.latent_dim as i64], + ) + .map_err(ort_error("create empty text sequence"))?; + let text_embeddings = Tensor::from_array(( + vec![1_i64, frames as i64, self.bundle.conditioning_dim as i64], + text_embeddings.to_vec().into_boxed_slice(), + )) + .map_err(ort_error("create text embedding tensor"))?; + let mut inputs = vec![ + (Cow::Borrowed("sequence"), SessionInputValue::from(sequence)), + ( + Cow::Borrowed("text_embeddings"), + SessionInputValue::from(text_embeddings), + ), + ]; + append_state_inputs(&mut inputs, state); + let mut outputs = self + .flow_main + .run(inputs) + .map_err(ort_error("prime Pocket TTS text state"))?; + replace_state_from_outputs(state, &mut outputs) + } + + fn generate_latents( + &mut self, + max_frames: usize, + frames_after_eos: usize, + state: &mut [StateValue], + ) -> Result, String> { + let mut current = vec![f32::NAN; self.bundle.latent_dim]; + let mut latents = Vec::with_capacity(max_frames * self.bundle.latent_dim); + let mut eos_step = None; + let mut rng = rand::rng(); + + for step in 0..max_frames { + let sequence = Tensor::from_array(( + vec![1_i64, 1, self.bundle.latent_dim as i64], + current.clone().into_boxed_slice(), + )) + .map_err(ort_error("create latent input"))?; + let text_embeddings = Tensor::::new( + &ort::memory::Allocator::default(), + [1_i64, 0, self.bundle.conditioning_dim as i64], + ) + .map_err(ort_error("create empty text input"))?; + let mut inputs = vec![ + (Cow::Borrowed("sequence"), SessionInputValue::from(sequence)), + ( + Cow::Borrowed("text_embeddings"), + SessionInputValue::from(text_embeddings), + ), + ]; + append_state_inputs(&mut inputs, state); + let mut outputs = self + .flow_main + .run(inputs) + .map_err(ort_error("run Pocket TTS Flow LM"))?; + let conditioning = outputs[0] + .try_extract_tensor::() + .map_err(ort_error("extract Flow LM conditioning"))? + .1 + .to_vec(); + let eos_logit = outputs[1] + .try_extract_tensor::() + .map_err(ort_error("extract Flow LM EOS logit"))? + .1 + .first() + .copied() + .ok_or_else(|| "Flow LM returned empty EOS logit".to_string())?; + replace_state_from_outputs(state, &mut outputs)?; + + if eos_logit > EOS_LOGIT_THRESHOLD && eos_step.is_none() { + eos_step = Some(step); + } + if eos_step.is_some_and(|eos| step >= eos + frames_after_eos) { + break; + } + + let mut noise = + normal_noise(&mut rng, self.bundle.latent_dim, DEFAULT_TEMPERATURE.sqrt()); + let conditioning = Tensor::from_array(( + vec![1_i64, self.bundle.conditioning_dim as i64], + conditioning.into_boxed_slice(), + )) + .map_err(ort_error("create flow conditioning"))?; + let s = Tensor::from_array((vec![1_i64, 1], vec![0.0_f32].into_boxed_slice())) + .map_err(ort_error("create flow start tensor"))?; + let t = Tensor::from_array((vec![1_i64, 1], vec![1.0_f32].into_boxed_slice())) + .map_err(ort_error("create flow end tensor"))?; + let x = Tensor::from_array(( + vec![1_i64, self.bundle.latent_dim as i64], + noise.clone().into_boxed_slice(), + )) + .map_err(ort_error("create flow noise tensor"))?; + let outputs = self + .flow + .run(ort::inputs![ + "c" => conditioning, + "s" => s, + "t" => t, + "x" => x, + ]) + .map_err(ort_error("run Pocket TTS flow"))?; + let flow = outputs[0] + .try_extract_tensor::() + .map_err(ort_error("extract Pocket TTS flow"))? + .1; + if flow.len() != noise.len() { + return Err(format!( + "flow returned {} values; expected {}", + flow.len(), + noise.len() + )); + } + for (sample, delta) in noise.iter_mut().zip(flow) { + *sample += *delta; + } + current.clone_from(&noise); + latents.extend_from_slice(&noise); + } + Ok(latents) + } + + fn decode_latents(&mut self, latents: &[f32]) -> Result, String> { + if latents.is_empty() { + return Ok(Vec::new()); + } + if !latents.len().is_multiple_of(self.bundle.latent_dim) { + return Err(format!( + "latent buffer has {} values, not divisible by {}", + latents.len(), + self.bundle.latent_dim + )); + } + let frame_count = latents.len() / self.bundle.latent_dim; + let mut state = initialize_state(&self.bundle.mimi_state_manifest)?; + let mut audio = Vec::new(); + + for start in (0..frame_count).step_by(DECODER_CHUNK_FRAMES) { + let end = (start + DECODER_CHUNK_FRAMES).min(frame_count); + let values = + latents[start * self.bundle.latent_dim..end * self.bundle.latent_dim].to_vec(); + let latent = Tensor::from_array(( + vec![1_i64, (end - start) as i64, self.bundle.latent_dim as i64], + values.into_boxed_slice(), + )) + .map_err(ort_error("create Mimi latent tensor"))?; + let mut inputs = vec![(Cow::Borrowed("latent"), SessionInputValue::from(latent))]; + append_state_inputs(&mut inputs, &state); + let mut outputs = self + .mimi_decoder + .run(inputs) + .map_err(ort_error("run Mimi decoder"))?; + let samples = outputs[0] + .try_extract_tensor::() + .map_err(ort_error("extract Mimi audio"))? + .1; + audio.extend_from_slice(samples); + replace_state_from_outputs(&mut state, &mut outputs)?; + } + Ok(audio) + } +} + +fn load_session(path: PathBuf, num_threads: usize) -> Result { + if !path.is_file() { + return Err(format!("missing Pocket TTS file: {}", path.display())); + } + Session::builder() + .map_err(ort_error("create ONNX session builder"))? + .with_intra_threads(num_threads) + .map_err(|err| format!("configure ONNX intra-op threads: {err}"))? + .with_inter_threads(1) + .map_err(|err| format!("configure ONNX inter-op threads: {err}"))? + .commit_from_file(&path) + .map_err(|err| format!("load {}: {err}", path.display())) +} + +fn load_tokenizer(path: &Path) -> Result { + let sentencepiece = SentencePieceModel::from_file(path) + .map_err(|err| format!("load {}: {err}", path.display()))?; + let trainer = sentencepiece + .trainer() + .ok_or_else(|| format!("{} has no SentencePiece trainer metadata", path.display()))?; + let normalizer = sentencepiece.normalizer().ok_or_else(|| { + format!( + "{} has no SentencePiece normalizer metadata", + path.display() + ) + })?; + if normalizer.name() != "identity" { + return Err(format!( + "{} uses unsupported SentencePiece normalizer {:?}", + path.display(), + normalizer.name() + )); + } + + let vocab = sentencepiece + .pieces() + .iter() + .map(|piece| (piece.piece().to_owned(), f64::from(piece.score()))) + .collect(); + let mut tokenizer = Tokenizer::new( + Unigram::from( + vocab, + Some(trainer.unk_id() as usize), + trainer.byte_fallback(), + ) + .map_err(|err| format!("construct tokenizer from {}: {err}", path.display()))?, + ); + // SentencePiece's identity normalizer still escapes spaces as U+2581 and + // prepends one marker to the input before unigram segmentation. + tokenizer.with_pre_tokenizer(Some(Metaspace::new('▁', PrependScheme::Always, false))); + Ok(tokenizer) +} + +fn initialize_state(specs: &[StateSpec]) -> Result, String> { + specs + .iter() + .cloned() + .map(|spec| { + let len = shape_len(&spec.shape)?; + let value = match spec.dtype { + StateDtype::Float32 => { + let fill = match spec.fill { + StateFill::Nan => f32::NAN, + StateFill::Empty | StateFill::Zeros => 0.0, + StateFill::Ones => 1.0, + }; + if len == 0 { + Tensor::::new(&ort::memory::Allocator::default(), spec.shape.clone()) + .map_err(ort_error("create empty float state tensor"))? + .into_dyn() + } else { + Tensor::from_array((spec.shape.clone(), vec![fill; len].into_boxed_slice())) + .map_err(ort_error("create float state tensor"))? + .into_dyn() + } + } + StateDtype::Int64 => { + let fill = i64::from(matches!(spec.fill, StateFill::Ones)); + if len == 0 { + Tensor::::new(&ort::memory::Allocator::default(), spec.shape.clone()) + .map_err(ort_error("create empty integer state tensor"))? + .into_dyn() + } else { + Tensor::from_array((spec.shape.clone(), vec![fill; len].into_boxed_slice())) + .map_err(ort_error("create integer state tensor"))? + .into_dyn() + } + } + StateDtype::Bool => { + let fill = matches!(spec.fill, StateFill::Ones); + if len == 0 { + Tensor::::new(&ort::memory::Allocator::default(), spec.shape.clone()) + .map_err(ort_error("create empty bool state tensor"))? + .into_dyn() + } else { + Tensor::from_array((spec.shape.clone(), vec![fill; len].into_boxed_slice())) + .map_err(ort_error("create bool state tensor"))? + .into_dyn() + } + } + }; + Ok(StateValue { spec, value }) + }) + .collect() +} + +fn append_state_inputs<'a>( + inputs: &mut Vec<(Cow<'a, str>, SessionInputValue<'a>)>, + state: &'a [StateValue], +) { + for value in state { + inputs.push(( + Cow::Borrowed(value.spec.input_name.as_str()), + SessionInputValue::from(&value.value), + )); + } +} + +fn replace_state_from_outputs( + state: &mut [StateValue], + outputs: &mut ort::session::SessionOutputs<'_>, +) -> Result<(), String> { + for value in state { + value.value = outputs + .remove(&value.spec.output_name) + .ok_or_else(|| format!("missing state output {}", value.spec.output_name))?; + } + Ok(()) +} + +fn shape_len(shape: &[i64]) -> Result { + shape.iter().try_fold(1_usize, |len, &dim| { + let dim = usize::try_from(dim).map_err(|_| format!("negative state dimension {dim}"))?; + len.checked_mul(dim) + .ok_or_else(|| format!("state shape overflows usize: {shape:?}")) + }) +} + +fn estimate_max_frames(token_count: usize, frame_rate: f32) -> usize { + ((token_count as f32 / TOKENS_PER_SECOND_ESTIMATE + GENERATION_SECONDS_PADDING) * frame_rate) + .ceil() as usize +} + +fn normal_noise(rng: &mut impl Rng, len: usize, std_dev: f32) -> Vec { + let mut out = Vec::with_capacity(len); + while out.len() < len { + let u1 = rng.random::().max(f32::MIN_POSITIVE); + let u2 = rng.random::(); + let radius = (-2.0_f32 * u1.ln()).sqrt() * std_dev; + out.push(radius * (TAU * u2).cos()); + if out.len() < len { + out.push(radius * (TAU * u2).sin()); + } + } + out +} + +fn read_npy_f32(path: &Path) -> Result, String> { + let bytes = fs::read(path).map_err(|err| format!("read {}: {err}", path.display()))?; + if bytes.len() < 10 || &bytes[..6] != b"\x93NUMPY" { + return Err(format!("{} is not a NumPy array", path.display())); + } + let major = bytes[6]; + let header_len_bytes = match major { + 1 => 2, + 2 | 3 => 4, + _ => { + return Err(format!( + "unsupported NumPy version {major} in {}", + path.display() + )) + } + }; + let header_start = 8 + header_len_bytes; + if bytes.len() < header_start { + return Err(format!("truncated NumPy header in {}", path.display())); + } + let header_len = if header_len_bytes == 2 { + u16::from_le_bytes([bytes[8], bytes[9]]) as usize + } else { + u32::from_le_bytes([bytes[8], bytes[9], bytes[10], bytes[11]]) as usize + }; + let data_start = header_start + .checked_add(header_len) + .ok_or_else(|| format!("NumPy header overflow in {}", path.display()))?; + if data_start > bytes.len() { + return Err(format!("truncated NumPy data in {}", path.display())); + } + let header = std::str::from_utf8(&bytes[header_start..data_start]) + .map_err(|err| format!("invalid NumPy header in {}: {err}", path.display()))?; + if !(header.contains("'descr': ' impl FnOnce(ort::Error) -> String { + move |err| format!("{context}: {err}") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn shape_len_supports_empty_state_dimensions() { + assert_eq!(shape_len(&[1, 128, 0]).expect("shape"), 0); + assert_eq!(shape_len(&[2, 1, 8, 1000, 64]).expect("shape"), 1_024_000); + } + + #[test] + fn normal_noise_has_requested_length() { + let mut rng = rand::rng(); + assert_eq!(normal_noise(&mut rng, 1, 1.0).len(), 1); + assert_eq!(normal_noise(&mut rng, 32, 1.0).len(), 32); + } + + #[test] + fn generation_frame_estimate_scales_with_token_count() { + assert_eq!(estimate_max_frames(3, 12.5), 38); + assert_eq!(estimate_max_frames(300, 12.5), 1_275); + } + + #[test] + #[ignore = "requires BUZZ_POCKET_TEST_MODEL_DIR"] + fn tokenizer_matches_sentencepiece_reference_including_unknown_words() { + let dir = std::env::var("BUZZ_POCKET_TEST_MODEL_DIR") + .expect("set BUZZ_POCKET_TEST_MODEL_DIR to the verified April bundle"); + let tokenizer = + load_tokenizer(&Path::new(&dir).join("tokenizer.model")).expect("load April tokenizer"); + let cases: &[(&str, &[u32])] = &[ + ("Yep.", &[2462, 263]), + ("Hello there.", &[2994, 310, 263]), + ( + "quizzaciously xyzzy.", + &[ + 260, 1157, 1818, 362, 1814, 323, 260, 568, 327, 1818, 327, 263, + ], + ), + ("I'm listening.", &[268, 264, 283, 260, 604, 273, 263]), + ]; + for (text, expected) in cases { + let encoding = tokenizer.encode(*text, false).expect("tokenize"); + assert_eq!(encoding.get_ids(), *expected, "{text}"); + } + } + + #[test] + #[ignore = "requires BUZZ_POCKET_TEST_MODEL_DIR"] + fn loader_splits_oversized_prompts_at_bundle_token_limit() { + let dir = std::env::var("BUZZ_POCKET_TEST_MODEL_DIR") + .expect("set BUZZ_POCKET_TEST_MODEL_DIR to the verified April bundle"); + let engine = AprilPocketTts::load(Path::new(&dir), 1).expect("load April bundle"); + let text = "This deliberately long sentence repeats ordinary English words so the exact SentencePiece token limit is exercised without relying on punctuation, and it keeps adding more material until the prompt must be divided into multiple independently safe generation chunks before the recurrent state cache can be exhausted."; + let prepared = prepare_april_prompt(text).expect("prepare prompt"); + let chunks = engine.split_prompt(&prepared).expect("split prompt"); + + assert!(chunks.len() > 1); + assert!(chunks.iter().all(|chunk| { + engine.token_count(chunk).expect("tokenize chunk") <= engine.bundle.max_token_per_chunk + })); + } + + #[test] + #[ignore = "requires BUZZ_POCKET_TEST_MODEL_DIR"] + fn gary_provost_long_sentence_respects_bundle_token_limit() { + let dir = std::env::var("BUZZ_POCKET_TEST_MODEL_DIR") + .expect("set BUZZ_POCKET_TEST_MODEL_DIR to the verified April bundle"); + let engine = AprilPocketTts::load(Path::new(&dir), 1).expect("load April bundle"); + let text = "And sometimes, when I am certain the reader is rested, I will engage him with a sentence of considerable length, a sentence that burns with energy and builds with all the impetus of a crescendo, the roll of the drums, the crash of the cymbals–sounds that say listen to this, it is important."; + let prepared = prepare_april_prompt(text).expect("prepare prompt"); + let chunks = engine.split_prompt(&prepared).expect("split long sentence"); + let token_counts: Vec<_> = chunks + .iter() + .map(|chunk| engine.token_count(chunk).expect("count tokens")) + .collect(); + + assert_eq!( + chunks, + [ + "And sometimes, when I am certain the reader is rested, I will engage him with a sentence of considerable length, a sentence that burns with energy and builds with all the.", + "Impetus of a crescendo, the roll of the drums, the crash of the cymbals–sounds that say listen to this, it is important.", + ] + ); + assert_eq!(token_counts, [48, 44]); + } +} diff --git a/crates/buzz-voice/src/pocket_models.rs b/crates/buzz-voice/src/pocket_models.rs new file mode 100644 index 0000000000..ba3f92849c --- /dev/null +++ b/crates/buzz-voice/src/pocket_models.rs @@ -0,0 +1,137 @@ +//! Immutable capabilities for Buzz Desktop's April Pocket TTS bundle. + +/// Pinned upstream export repository. +pub const APRIL_MODEL_ID: &str = "KevinAHM/pocket-tts-onnx"; + +/// Pinned revision containing the `english_2026-04` bundle. +pub const APRIL_MODEL_REVISION: &str = "58a6d00cf13d239b6748cb0769f35c580a8f606c"; + +/// Language bundle selected from the pinned export. +pub const APRIL_BUNDLE_ID: &str = "english_2026-04"; + +/// Maximum input size declared by the April bundle. +pub const APRIL_MAX_TOKEN_PER_CHUNK: usize = 50; + +/// One immutable artifact required by the April INT8 runtime. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct PocketModelArtifact { + pub filename: &'static str, + pub sha256: &'static str, + pub size_bytes: u64, + pub quantized: bool, +} + +/// Capabilities of Buzz Desktop's sole Pocket model. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct PocketModelInfo { + /// Language bundle selected from the pinned export. + pub bundle_id: &'static str, + /// Upstream model repository. + pub source_model_id: &'static str, + /// Pinned upstream model revision. + pub revision: &'static str, + /// PCM output sample rate. + pub sample_rate: u32, + /// Maximum input size declared by the bundle. + pub max_token_per_chunk: usize, + /// Immutable files required by the runtime. + pub artifacts: &'static [PocketModelArtifact], + /// Components quantized in the selected bundle. + pub quantized_components: &'static [&'static str], +} + +const INT8_ARTIFACTS: [PocketModelArtifact; 8] = [ + PocketModelArtifact { + filename: "bundle.json", + sha256: "bab643150f437f37df080a710520ff39ed9ebd9a339f8ebdc739f7eddfc28b3f", + size_bytes: 24_381, + quantized: false, + }, + PocketModelArtifact { + filename: "bos_before_voice.npy", + sha256: "f46edf4f7007b7ba4ea58831f49d003e59e167b4641c44bb3addfe9231a780b1", + size_bytes: 4_224, + quantized: false, + }, + PocketModelArtifact { + filename: "tokenizer.model", + sha256: "d461765ae179566678c93091c5fa6f2984c31bbe990bf1aa62d92c64d91bc3f6", + size_bytes: 59_339, + quantized: false, + }, + PocketModelArtifact { + filename: "flow_lm_main_int8.onnx", + sha256: "f9bd8106b79a0192c1c43399ab938fb24900a95c1c599870d75a884e99000116", + size_bytes: 76_341_079, + quantized: true, + }, + PocketModelArtifact { + filename: "flow_lm_flow_int8.onnx", + sha256: "3dd781ee5abee9e195320bf0106bebd6372a852b3b36352524ee78b40554635d", + size_bytes: 9_962_530, + quantized: true, + }, + PocketModelArtifact { + filename: "mimi_decoder_int8.onnx", + sha256: "3630450a3297a101792a6ac66619ebc70ab916b265e6220c2afaef8b1673f925", + size_bytes: 22_684_077, + quantized: true, + }, + PocketModelArtifact { + filename: "mimi_encoder.onnx", + sha256: "853e2ca623b8782d94c3745ec6133bfdff7ce33d9b11128bd29ea03f28d76e3d", + size_bytes: 39_768_446, + quantized: false, + }, + PocketModelArtifact { + filename: "text_conditioner.onnx", + sha256: "4ecee995fb69f85c7a7493d11f7b5ee15d9950facc7ab3f5c9c49ef1e03847bb", + size_bytes: 16_388_344, + quantized: false, + }, +]; + +const INT8_COMPONENTS: [&str; 3] = ["flow_lm_main", "flow_lm_flow", "mimi_decoder"]; + +/// Return immutable metadata for Buzz Desktop's April INT8 model. +pub const fn april_model_info() -> PocketModelInfo { + PocketModelInfo { + bundle_id: APRIL_BUNDLE_ID, + source_model_id: APRIL_MODEL_ID, + revision: APRIL_MODEL_REVISION, + sample_rate: 24_000, + max_token_per_chunk: APRIL_MAX_TOKEN_PER_CHUNK, + artifacts: &INT8_ARTIFACTS, + quantized_components: &INT8_COMPONENTS, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn metadata_matches_pinned_int8_layout() { + let info = april_model_info(); + assert_eq!(info.artifacts.len(), 8); + assert_eq!( + info.quantized_components, + ["flow_lm_main", "flow_lm_flow", "mimi_decoder"] + ); + assert_eq!( + info.artifacts + .iter() + .map(|artifact| artifact.size_bytes) + .sum::(), + 165_232_420 + ); + assert!(info + .artifacts + .iter() + .any(|artifact| { artifact.filename == "mimi_encoder.onnx" && !artifact.quantized })); + assert!(!info + .artifacts + .iter() + .any(|artifact| artifact.filename == "mimi_encoder_int8.onnx")); + } +} diff --git a/crates/buzz-voice/tests/pocket_import_audio.rs b/crates/buzz-voice/tests/pocket_import_audio.rs new file mode 100644 index 0000000000..8578c368d4 --- /dev/null +++ b/crates/buzz-voice/tests/pocket_import_audio.rs @@ -0,0 +1,133 @@ +use std::{ + fs, + path::{Path, PathBuf}, +}; + +use buzz_voice::{ + imported::{write_pcm16_wav, PcmStats, PocketVoiceLibrary}, + pocket::{load_text_to_speech, load_voice_style, DEFAULT_VOICE, SAMPLE_RATE, VOICE_FILE_EXT}, +}; + +const PREVIEW_TEXT: &str = "This is an objective Pocket voice preview."; + +fn required_path(name: &str) -> PathBuf { + std::env::var_os(name) + .map(PathBuf::from) + .unwrap_or_else(|| panic!("{name} must point to the required local test path")) +} + +fn checked_in_voice() -> PathBuf { + Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../desktop/src-tauri/resources/pocket-voices/eve.wav") +} + +fn evidence_dir() -> PathBuf { + std::env::var_os("BUZZ_VOICE_EVIDENCE_DIR") + .map(PathBuf::from) + .unwrap_or_else(|| { + Path::new(env!("CARGO_MANIFEST_DIR")).join("../../target/buzz-voice-evidence") + }) +} + +fn synthesize(model_dir: &Path, voice_path: &Path, text: &str) -> (Vec, PcmStats) { + let engine = load_text_to_speech( + model_dir + .to_str() + .expect("Pocket model path must be valid UTF-8"), + ) + .expect("load Pocket model"); + let style = load_voice_style(voice_path).expect("load selected voice"); + let samples = engine + .synth_chunk(text, "en", &style, 1) + .expect("synthesize preview"); + let stats = PcmStats::analyze(&samples, SAMPLE_RATE); + assert!( + stats.is_non_silent(), + "generated PCM must be non-silent: {stats:?}" + ); + assert!( + stats.duration_seconds > 0.2, + "generated PCM is unexpectedly short: {stats:?}" + ); + (samples, stats) +} + +#[test] +#[ignore = "requires BUZZ_POCKET_MODEL_DIR and runs the installed Pocket ONNX model"] +fn objective_import_synthesis_delete_and_mary_fallback() { + let model_dir = required_path("BUZZ_POCKET_MODEL_DIR"); + let temp = tempfile::tempdir().expect("temporary voice workspace"); + let source = temp.path().join("Imported Eve.wav"); + fs::copy(checked_in_voice(), &source).expect("copy checked-in voice fixture"); + + let library_root = temp.path().join("library"); + let library = PocketVoiceLibrary::new(&library_root); + let imported = library.import_path(&source).expect("import valid WAV"); + assert_eq!( + library.find(&imported.key).expect("read selection"), + Some(imported.clone()) + ); + + drop(library); + let relaunched = PocketVoiceLibrary::new(&library_root); + let selected = relaunched + .find(&imported.key) + .expect("reload persisted selection") + .expect("selected imported voice survived relaunch"); + let imported_path = relaunched + .resolve_file(&selected) + .expect("resolve persisted imported voice"); + let (imported_pcm, imported_stats) = synthesize(&model_dir, &imported_path, PREVIEW_TEXT); + + let evidence = evidence_dir(); + fs::create_dir_all(&evidence).expect("create evidence directory"); + let imported_wav = evidence.join("imported-preview.wav"); + write_pcm16_wav(&imported_wav, &imported_pcm, SAMPLE_RATE) + .expect("write imported preview evidence"); + + relaunched + .delete(&imported.key) + .expect("delete imported voice"); + assert_eq!( + relaunched.find(&imported.key).expect("reload after delete"), + None + ); + + let mary_path = model_dir.join(format!("{DEFAULT_VOICE}.{VOICE_FILE_EXT}")); + assert_eq!( + mary_path.file_name().and_then(|name| name.to_str()), + Some("reference_sample.wav"), + "fallback must remain the deterministic Mary reference" + ); + let (mary_pcm, mary_stats) = synthesize(&model_dir, &mary_path, PREVIEW_TEXT); + let mary_wav = evidence.join("mary-fallback-preview.wav"); + write_pcm16_wav(&mary_wav, &mary_pcm, SAMPLE_RATE) + .expect("write Mary fallback preview evidence"); + + println!( + "{}", + serde_json::json!({ + "importedKey": imported.key, + "persistence": "reloaded", + "afterDelete": "pocket:mary", + "importedPreview": { + "path": imported_wav, + "samples": imported_stats.sample_count, + "sampleRate": imported_stats.sample_rate, + "durationSeconds": imported_stats.duration_seconds, + "peak": imported_stats.peak, + "rms": imported_stats.rms, + "nonSilentSamples": imported_stats.non_silent_samples, + }, + "maryFallbackPreview": { + "path": mary_wav, + "samples": mary_stats.sample_count, + "sampleRate": mary_stats.sample_rate, + "durationSeconds": mary_stats.duration_seconds, + "peak": mary_stats.peak, + "rms": mary_stats.rms, + "nonSilentSamples": mary_stats.non_silent_samples, + } + }) + ); +} diff --git a/deploy/charts/buzz/README.md b/deploy/charts/buzz/README.md index a7c4bcf63b..b2778df28b 100644 --- a/deploy/charts/buzz/README.md +++ b/deploy/charts/buzz/README.md @@ -52,6 +52,48 @@ See: The chart fails at `helm install` / `helm template` time with a clear message if any of these are missing or malformed (see `templates/_validate.tpl`). +## S3 URL addressing + +Buzz uses one URL style for both media and Git/CAS object-store requests: + +| `s3.addressingStyle` | Request shape | Use for | +|---|---|---| +| `path` (default) | `https://endpoint/bucket/key` | Bundled MinIO and endpoints whose DNS does not resolve bucket subdomains | +| `virtual` | `https://bucket.endpoint/key` | AWS-style providers and new Railway Storage Buckets | + +The chart always renders `s3.addressingStyle` as +`BUZZ_S3_ADDRESSING_STYLE`. It renders `s3.region` as `BUZZ_S3_REGION` only +when explicitly set, preserving the relay's existing `AWS_REGION` fallback for +upgrades. Only `path` and `virtual` addressing styles are accepted; invalid +values fail chart rendering and relay startup. The bundled MinIO quickstart +deliberately keeps `path` because its Service DNS resolves one endpoint +hostname, not arbitrary `.` names. + +For a Railway Storage Bucket, map its variables to chart values in the service +or generated Helm configuration: + +```yaml +s3: + endpoint: "${{Object Storage.ENDPOINT}}" + bucket: "${{Object Storage.BUCKET}}" + region: "${{Object Storage.REGION}}" + addressingStyle: virtual +``` + +Store `BUZZ_S3_ACCESS_KEY=${{Object Storage.ACCESS_KEY_ID}}` and +`BUZZ_S3_SECRET_KEY=${{Object Storage.SECRET_ACCESS_KEY}}` in the Secret named by +`secrets.existingSecret`. Railway's Credentials tab is authoritative for older +buckets, which may still require `path`. The setting changes request routing and +SigV4 signing, so do not put the bucket into `s3.endpoint`; pass Railway's base +`ENDPOINT` and `BUCKET` separately. + +Object storage is contacted during relay startup only when +`BUZZ_GIT_CONFORMANCE_PROBE` is enabled (the relay default). A probe failure is +startup-fatal, so Kubernetes readiness never opens. If an operator explicitly +disables that probe through `relay.extraEnv`, `/_readiness` does not test object +storage; configuration is still parsed strictly, but reachability and addressing +errors surface on the first storage operation. + ## Relay Pod extensions The chart exposes narrow extension points for init containers, volumes, relay diff --git a/deploy/charts/buzz/examples/argocd-app.yaml b/deploy/charts/buzz/examples/argocd-app.yaml index 7e90f64bd7..a29a6919b7 100644 --- a/deploy/charts/buzz/examples/argocd-app.yaml +++ b/deploy/charts/buzz/examples/argocd-app.yaml @@ -16,9 +16,14 @@ metadata: spec: project: default source: - repoURL: oci://ghcr.io/block/buzz/charts - chart: buzz - targetRevision: 0.1.0 + # Argo CD >= 3.1 native OCI sources: repoURL must be the FULL chart + # artifact path — with the `repoURL: …/charts` + `chart: buzz` split + # form, the `chart` field is ignored for oci:// URLs and the fetch + # fails with a 403 (`repository:block/buzz/charts:pull` denied). The + # spec validator still requires `path`; use "." for OCI sources. + repoURL: oci://ghcr.io/block/buzz/charts/buzz + path: . + targetRevision: 0.1.7 helm: releaseName: buzz values: | @@ -41,6 +46,8 @@ spec: s3: endpoint: "https://s3.us-east-1.amazonaws.com" bucket: "buzz-media" + region: "us-east-1" + addressingStyle: virtual # accessKey / secretKey live in buzz-secrets persistence: diff --git a/deploy/charts/buzz/examples/flux-helmrelease.yaml b/deploy/charts/buzz/examples/flux-helmrelease.yaml index 16754c0fcb..09a6bfeb6a 100644 --- a/deploy/charts/buzz/examples/flux-helmrelease.yaml +++ b/deploy/charts/buzz/examples/flux-helmrelease.yaml @@ -41,6 +41,8 @@ spec: s3: endpoint: "https://s3.us-east-1.amazonaws.com" bucket: "buzz-media" + region: "us-east-1" + addressingStyle: virtual persistence: git: diff --git a/deploy/charts/buzz/templates/_validate.tpl b/deploy/charts/buzz/templates/_validate.tpl index 946424f9a3..aa7f7ac13c 100644 --- a/deploy/charts/buzz/templates/_validate.tpl +++ b/deploy/charts/buzz/templates/_validate.tpl @@ -75,10 +75,12 @@ surface at template time regardless of which manifest helm renders first. {{- fail "Postgres source missing: enable postgresql.enabled=true, set externalPostgresql.url, or provide secrets.existingSecret with key DATABASE_URL." -}} {{- end -}} -{{/* S3 / object-storage source must exist somewhere (relay hard-fails its - startup conformance probe without a reachable bucket). */}} +{{/* S3 / object-storage source must exist somewhere. With the default + BUZZ_GIT_CONFORMANCE_PROBE behavior, an unreachable bucket is detected + before the relay opens its listener; operators can explicitly disable that + startup gate. */}} {{- if not (or .Values.minio.enabled .Values.s3.endpoint .Values.secrets.existingSecret) -}} - {{- fail "S3/object-storage source missing: enable minio.enabled=true (quickstart in-cluster), set s3.endpoint + s3.bucket + credentials, or provide secrets.existingSecret with keys BUZZ_S3_ACCESS_KEY + BUZZ_S3_SECRET_KEY. The relay runs a startup S3 conformance probe and exits if storage is unreachable." -}} + {{- fail "S3/object-storage source missing: enable minio.enabled=true (quickstart in-cluster), set s3.endpoint + s3.bucket + credentials, or provide secrets.existingSecret with keys BUZZ_S3_ACCESS_KEY + BUZZ_S3_SECRET_KEY. By default the relay runs a startup S3 conformance probe and exits if storage is unreachable; disabling BUZZ_GIT_CONFORMANCE_PROBE also removes that startup storage check." -}} {{- end -}} {{- end -}} diff --git a/deploy/charts/buzz/templates/deployment.yaml b/deploy/charts/buzz/templates/deployment.yaml index bf2df4c2c8..67a93138c5 100644 --- a/deploy/charts/buzz/templates/deployment.yaml +++ b/deploy/charts/buzz/templates/deployment.yaml @@ -170,6 +170,10 @@ spec: - { name: BUZZ_S3_ENDPOINT, value: {{ $s3Endpoint | quote }} } {{- end }} - { name: BUZZ_S3_BUCKET, value: {{ .Values.s3.bucket | quote }} } + {{- if .Values.s3.region }} + - { name: BUZZ_S3_REGION, value: {{ .Values.s3.region | quote }} } + {{- end }} + - { name: BUZZ_S3_ADDRESSING_STYLE, value: {{ .Values.s3.addressingStyle | quote }} } # ── Secrets (from chart-managed or existing) ───────────── - name: BUZZ_RELAY_PRIVATE_KEY diff --git a/deploy/charts/buzz/tests/render_test.yaml b/deploy/charts/buzz/tests/render_test.yaml index 3e044f5d7c..cf08210781 100644 --- a/deploy/charts/buzz/tests/render_test.yaml +++ b/deploy/charts/buzz/tests/render_test.yaml @@ -30,6 +30,18 @@ tests: path: kind value: Service template: templates/service.yaml + - notContains: + path: spec.template.spec.containers[0].env + content: + name: BUZZ_S3_REGION + any: true + template: templates/deployment.yaml + - contains: + path: spec.template.spec.containers[0].env + content: + name: BUZZ_S3_ADDRESSING_STYLE + value: "path" + template: templates/deployment.yaml - contains: path: spec.template.spec.containers[0].env content: @@ -47,6 +59,32 @@ tests: value: "true" template: templates/deployment.yaml + - it: renders virtual-hosted S3 addressing for providers that require it + set: + relayUrl: wss://buzz.example.com + ownerPubkey: "0000000000000000000000000000000000000000000000000000000000000000" + externalPostgresql.url: postgres://u:p@h:5432/d + externalRedis.url: redis://h:6379 + s3.endpoint: https://storage.railway.app + s3.bucket: buzz-media-example + s3.region: auto + s3.addressingStyle: virtual + s3.accessKey: a + s3.secretKey: s + asserts: + - contains: + path: spec.template.spec.containers[0].env + content: + name: BUZZ_S3_REGION + value: "auto" + template: templates/deployment.yaml + - contains: + path: spec.template.spec.containers[0].env + content: + name: BUZZ_S3_ADDRESSING_STYLE + value: "virtual" + template: templates/deployment.yaml + - it: lets an explicit value opt out of media read auth for dev/public deployments set: relayUrl: wss://buzz.example.com diff --git a/deploy/charts/buzz/tests/validation_test.yaml b/deploy/charts/buzz/tests/validation_test.yaml index f0a3869795..a5a0050a86 100644 --- a/deploy/charts/buzz/tests/validation_test.yaml +++ b/deploy/charts/buzz/tests/validation_test.yaml @@ -58,6 +58,17 @@ tests: - failedTemplate: errorPattern: "Postgres source missing" + - it: rejects an invalid S3 addressing style + set: + relayUrl: wss://buzz.example.com + ownerPubkey: "0000000000000000000000000000000000000000000000000000000000000000" + externalPostgresql.url: postgres://u:p@h:5432/d + s3.endpoint: http://minio:9000 + s3.addressingStyle: auto + asserts: + - failedTemplate: + errorPattern: "s3.addressingStyle: s3.addressingStyle must be one of the following:.*path.*virtual" + - it: fails when S3/object-storage source is missing set: relayUrl: wss://buzz.example.com diff --git a/deploy/charts/buzz/values.schema.json b/deploy/charts/buzz/values.schema.json index 53bb29bb60..9cb6a02c9b 100644 --- a/deploy/charts/buzz/values.schema.json +++ b/deploy/charts/buzz/values.schema.json @@ -198,6 +198,15 @@ "properties": { "endpoint": { "type": "string", "pattern": "^(https?://.+)?$" }, "bucket": { "type": "string", "minLength": 1 }, + "region": { + "type": "string", + "description": "Optional S3 region used for SigV4 signing. When empty, BUZZ_S3_REGION is omitted so the relay can use AWS_REGION or its own default." + }, + "addressingStyle": { + "type": "string", + "enum": ["path", "virtual"], + "description": "S3 URL style shared by media and Git/CAS clients. Defaults to path for bundled MinIO compatibility." + }, "accessKey": { "type": "string" }, "secretKey": { "type": "string" } } diff --git a/deploy/charts/buzz/values.yaml b/deploy/charts/buzz/values.yaml index 8ac5086e27..810f8a9658 100644 --- a/deploy/charts/buzz/values.yaml +++ b/deploy/charts/buzz/values.yaml @@ -338,6 +338,12 @@ externalRedis: s3: endpoint: "" bucket: "buzz-media" + # Optional SigV4 signing region. Leave empty to preserve the relay's + # AWS_REGION fallback; set the provider's credential value when needed. + region: "" + # path: https://endpoint/bucket/key (bundled MinIO-compatible default) + # virtual: https://bucket.endpoint/key (standard S3; required by new Railway buckets) + addressingStyle: path accessKey: "" secretKey: "" diff --git a/deploy/compose/.env.example b/deploy/compose/.env.example index 838824c17f..f6ab4fcab9 100644 --- a/deploy/compose/.env.example +++ b/deploy/compose/.env.example @@ -33,6 +33,8 @@ REDIS_PASSWORD=CHANGE_ME_RANDOM_PASSWORD BUZZ_S3_ACCESS_KEY=CHANGE_ME_RANDOM_ACCESS_KEY BUZZ_S3_SECRET_KEY=CHANGE_ME_RANDOM_SECRET_KEY BUZZ_S3_BUCKET=buzz-media +# Bundled MinIO uses path-style URLs; deploy/compose/compose.yml pins this. +BUZZ_S3_ADDRESSING_STYLE=path # Optional host ports. Base compose publishes the relay directly on BUZZ_HTTP_PORT. BUZZ_HTTP_PORT=3000 diff --git a/deploy/compose/README.md b/deploy/compose/README.md index 0de524fb5b..bb0e63fe15 100644 --- a/deploy/compose/README.md +++ b/deploy/compose/README.md @@ -38,6 +38,11 @@ keypair. migrations. - The stack uses Postgres, Redis, MinIO, and a git data volume because those are real Buzz dependencies today. Minimal mode can simplify this later. +- The bundled Compose stack fixes the relay endpoint to `http://minio:9000` and + `BUZZ_S3_ADDRESSING_STYLE=path`: Docker DNS resolves `minio`, not + `.minio`. It is not configurable for an external S3 provider through + `.env`; use the Helm chart or a custom Compose configuration for providers + such as new Railway Storage Buckets that require `virtual` addressing. Run `./run.sh backup-hint` for the backup checklist. diff --git a/deploy/compose/compose.yml b/deploy/compose/compose.yml index bc3c27501e..15337c92a2 100644 --- a/deploy/compose/compose.yml +++ b/deploy/compose/compose.yml @@ -12,6 +12,8 @@ services: DATABASE_URL: postgres://${POSTGRES_USER:-buzz}:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-buzz} REDIS_URL: redis://:${REDIS_PASSWORD:?set REDIS_PASSWORD}@redis:6379 BUZZ_S3_ENDPOINT: http://minio:9000 + # Docker DNS resolves `minio`, not arbitrary `.minio` hosts. + BUZZ_S3_ADDRESSING_STYLE: path BUZZ_S3_ACCESS_KEY: ${BUZZ_S3_ACCESS_KEY:?set BUZZ_S3_ACCESS_KEY} BUZZ_S3_SECRET_KEY: ${BUZZ_S3_SECRET_KEY:?set BUZZ_S3_SECRET_KEY} BUZZ_S3_BUCKET: ${BUZZ_S3_BUCKET:-buzz-media} diff --git a/desktop/package.json b/desktop/package.json index adac095a47..e8145f5468 100644 --- a/desktop/package.json +++ b/desktop/package.json @@ -1,7 +1,7 @@ { "name": "buzz", "private": true, - "version": "0.5.0", + "version": "0.5.3", "type": "module", "scripts": { "dev": "vite", diff --git a/desktop/playwright.config.ts b/desktop/playwright.config.ts index 459fa75743..7ce7f48389 100644 --- a/desktop/playwright.config.ts +++ b/desktop/playwright.config.ts @@ -34,6 +34,7 @@ export default defineConfig({ "**/hosted-communities-settings-screenshots.spec.ts", "**/invites-settings-screenshots.spec.ts", "**/messaging.spec.ts", + "**/message-feedback-snapshots.spec.ts", "**/custom-emoji.spec.ts", "**/profile-custom-emoji-status.spec.ts", "**/custom-emoji-ui.spec.ts", @@ -48,6 +49,7 @@ export default defineConfig({ "**/activity-scope-label-screenshots.spec.ts", "**/welcome-agent-modal-screenshots.spec.ts", "**/local-archive-screenshots.spec.ts", + "**/voice-settings.spec.ts", "**/agent-readiness-screenshots.spec.ts", "**/agent-error-state-screenshots.spec.ts", "**/edit-agent.spec.ts", @@ -94,6 +96,7 @@ export default defineConfig({ "**/cold-switch-longtask.perf.ts", "**/timeline-no-shift.spec.ts", "**/human-edit-agent-content.spec.ts", + "**/empty-edit-delete.spec.ts", "**/reaction-order.spec.ts", "**/reaction-names.spec.ts", "**/inbox-reactions.spec.ts", @@ -118,15 +121,18 @@ export default defineConfig({ "**/nostr-bind.spec.ts", "**/mobile-pairing-qr.spec.ts", "**/profile-nsec-reveal.spec.ts", + "**/profile-backup-settings.spec.ts", "**/signout-confirmation.spec.ts", "**/agent-provider-dropdowns.spec.ts", "**/agent-lifecycle-feedback.spec.ts", + "**/agent-access-warning.spec.ts", "**/inbox-live-update.spec.ts", "**/mesh-compute.spec.ts", "**/observer-archive-policy.spec.ts", "**/harness-management.spec.ts", "**/harness-catalog-screenshots.spec.ts", "**/inline-custom-harness.spec.ts", + "**/huddle-transcription.spec.ts", ], use: { ...devices["Desktop Chrome"], diff --git a/desktop/public/harness-logos/CREDITS.md b/desktop/public/harness-logos/CREDITS.md index de4e003c3f..716c43e1ae 100644 --- a/desktop/public/harness-logos/CREDITS.md +++ b/desktop/public/harness-logos/CREDITS.md @@ -9,6 +9,7 @@ license permits redistribution. | File | Upstream | Commit | License | Source path | Modifications | |---|---|---|---|---|---| +| `devin.svg` | [Cognition Devin documentation](https://docs.devin.ai/cli) | Retrieved 2026-07-27 | Cognition trademark; nominative use to identify the Devin harness | Official documentation `logo/favicon.svg` | Added the official black mark to a white square canvas so it remains legible in both app themes | | `hermes.png` | [NousResearch/hermes-agent](https://github.com/NousResearch/hermes-agent) | `6ad632b` | MIT © 2025 Nous Research | `website/static/img/logo.png` | Cropped the baked-in border frame, padded to square, resized to 64×64, quantised to a 16-colour palette | | `openclaw.svg` | [openclaw/openclaw](https://github.com/openclaw/openclaw) | `b06f40a` | MIT © 2026 OpenClaw Foundation | `ui/public/favicon.svg` | Removed the SMIL animation elements (renders the upstream rest pose statically — verified pixel-identical to the upstream frame at t=0); minified paths | | `omp.svg` | [can1357/oh-my-pi](https://github.com/can1357/oh-my-pi) | `667111575ebba136dadfd6989379e7f67e0d40d9` | MIT © 2025 Mario Zechner; © 2025–2026 Can Bölük | `assets/icon.svg` | None | diff --git a/desktop/public/harness-logos/devin.svg b/desktop/public/harness-logos/devin.svg new file mode 100644 index 0000000000..e760797d68 --- /dev/null +++ b/desktop/public/harness-logos/devin.svg @@ -0,0 +1,4 @@ + + + + diff --git a/desktop/scripts/build-release-config.mjs b/desktop/scripts/build-release-config.mjs index 389d18aec5..d1cd8181eb 100644 --- a/desktop/scripts/build-release-config.mjs +++ b/desktop/scripts/build-release-config.mjs @@ -52,6 +52,15 @@ const releaseConfig = { }, }; +// Tauri applies --config after platform-specific config using RFC 7396. +// Any externalBin value here would therefore replace the platform sidecar list, +// while null would silently delete it. This delta must never own that key. +if (Object.hasOwn(releaseConfig.bundle, "externalBin")) { + throw new Error( + "Release config must not define bundle.externalBin; sidecars are platform-specific", + ); +} + console.log(`Updater enabled -> ${updaterEndpoint}`); writeFileSync(outputConfigPath, `${JSON.stringify(releaseConfig, null, 2)}\n`); diff --git a/desktop/scripts/check-pubkey-truncation.mjs b/desktop/scripts/check-pubkey-truncation.mjs index 95e56fb282..d65db13545 100644 --- a/desktop/scripts/check-pubkey-truncation.mjs +++ b/desktop/scripts/check-pubkey-truncation.mjs @@ -18,12 +18,10 @@ const rules = [ // Non-display uses: array windows over pubkey lists, color/initials // derivation where the value is never presented as an identity. const overrides = new Set([ - // ProfileAvatar fallback label — decorative glyphs inside an avatar disc. - "src/features/huddle/components/ParticipantList.tsx:92", // HexAvatar: 6-char badge + hue derivation inside a color-coded disc, // clearly decorative (paired with a full truncatePubkey aria-label). - "src/features/huddle/components/ParticipantList.tsx:143", - "src/features/huddle/components/ParticipantList.tsx:144", + "src/features/huddle/components/ParticipantList.tsx:150", + "src/features/huddle/components/ParticipantList.tsx:151", // clientId (not a pubkey) sliced in a debug log next to the real thing. "src/features/channels/readState/readStateManager.ts:338", // Array windows (first N pubkeys), not string truncation. diff --git a/desktop/scripts/texture-card/generate-card-texture.mjs b/desktop/scripts/texture-card/generate-card-texture.mjs index 75cc24e744..57ebc61a9b 100644 --- a/desktop/scripts/texture-card/generate-card-texture.mjs +++ b/desktop/scripts/texture-card/generate-card-texture.mjs @@ -12,83 +12,116 @@ import path from "node:path"; import { fileURLToPath } from "node:url"; const HERE = path.dirname(fileURLToPath(import.meta.url)); -const OUTPUT = path.resolve( - HERE, - "../../src/shared/ui/assets/card-texture.png", -); - -// CSS-pixel source geometry. Screenshotting at DPR 2 produces a crisp asset. -const CARD_SIZE = 640; -const OUTSET = 96; -const CAPTURE_SIZE = CARD_SIZE + OUTSET * 2; +const OUTPUT_DIRECTORY = path.resolve(HERE, "../../src/shared/ui/assets"); const DPR = 2; // Approved texture parameters, archived from the former runtime SVG filter. -const BLUR = 66; -const DILATE = Math.round(BLUR * 0.85); const THRESHOLD_BIAS = 0.302; const SLOPE = 8; const FREQUENCY = 0.999; const OCTAVES = 3; const SEED = 5315; -await mkdir(path.dirname(OUTPUT), { recursive: true }); +const TEXTURES = [ + { + filename: "card-texture.png", + color: "white", + cardSize: 640, + outset: 96, + blur: 66, + innerBand: 112, + }, + { + filename: "card-texture-dark.png", + color: "#171b21", + cardSize: 640, + outset: 96, + blur: 66, + innerBand: 112, + }, + { + filename: "card-texture-compact.png", + color: "white", + cardSize: 320, + outset: 24, + blur: 24, + innerBand: 44, + }, + { + filename: "card-texture-dark-compact.png", + color: "#171b21", + cardSize: 320, + outset: 24, + blur: 24, + innerBand: 44, + }, +]; + +await mkdir(OUTPUT_DIRECTORY, { recursive: true }); const browser = await chromium.launch(); try { - const page = await browser.newPage({ - deviceScaleFactor: DPR, - viewport: { height: CAPTURE_SIZE, width: CAPTURE_SIZE }, - }); + for (const texture of TEXTURES) { + const captureSize = texture.cardSize + texture.outset * 2; + const dilate = Math.round(texture.blur * 0.85); + const output = path.join(OUTPUT_DIRECTORY, texture.filename); + const page = await browser.newPage({ + deviceScaleFactor: DPR, + viewport: { height: captureSize, width: captureSize }, + }); - await page.setContent(` - -
- - -
`); + await page.setContent(` + +
+ + +
`); - await page.locator("#stage").screenshot({ - omitBackground: true, - path: OUTPUT, - }); + await page.locator("#stage").screenshot({ + omitBackground: true, + path: output, + }); + await page.close(); + + console.log(`Generated ${output}`); + console.log(`Asset: ${captureSize * DPR}×${captureSize * DPR}px @${DPR}x`); + console.log( + `Runtime slice: ${(texture.outset + texture.innerBand) * DPR}px; outset: ${texture.outset}px`, + ); + } } finally { await browser.close(); } - -console.log(`Generated ${OUTPUT}`); -console.log(`Asset: ${CAPTURE_SIZE * DPR}×${CAPTURE_SIZE * DPR}px @${DPR}x`); -console.log(`Runtime slice: ${(OUTSET + 112) * DPR}px; outset: ${OUTSET}px`); diff --git a/desktop/src-tauri/Cargo.lock b/desktop/src-tauri/Cargo.lock index 66553ef595..9feecbee01 100644 --- a/desktop/src-tauri/Cargo.lock +++ b/desktop/src-tauri/Cargo.lock @@ -75,6 +75,20 @@ dependencies = [ "subtle", ] +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if 1.0.4", + "getrandom 0.3.4", + "once_cell", + "serde", + "version_check", + "zerocopy", +] + [[package]] name = "aho-corasick" version = "1.1.4" @@ -694,6 +708,12 @@ version = "1.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fd307490d624467aa6f74b0eabb77633d1f758a7b25f12bceb0b22e08d9726f6" +[[package]] +name = "base64" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e1b586273c5702936fe7b7d6896644d8be71e6314cfe09d3167c95f712589e8" + [[package]] name = "base64" version = "0.21.7" @@ -718,6 +738,12 @@ version = "0.11.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32637268377fc7b10a8c6d51de3e7fba1ce5dd371a96e342b34e6078db558e7f" +[[package]] +name = "beef" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3a8241f3ebb85c056b509d4327ad0358fbbba6ffb340bf388f26350aeda225b1" + [[package]] name = "bip39" version = "2.2.2" @@ -1010,7 +1036,7 @@ dependencies = [ [[package]] name = "buzz-desktop" -version = "0.5.0" +version = "0.5.3" dependencies = [ "anyhow", "arboard", @@ -1018,11 +1044,13 @@ dependencies = [ "audioadapter-buffers", "axum", "base64 0.22.1", + "block2", "buzz-agent", "buzz-core", "buzz-media", "buzz-persona", "buzz-sdk", + "buzz-voice", "bytes", "bzip2 0.6.1", "chrono", @@ -1032,6 +1060,7 @@ dependencies = [ "ed25519-dalek", "flate2", "futures-util", + "getrandom 0.2.17", "hex", "image", "infer", @@ -1047,7 +1076,9 @@ dependencies = [ "neteq", "nostr", "notify-rust", + "objc2", "objc2-app-kit", + "objc2-foundation", "opus", "plist", "png 0.18.1", @@ -1076,6 +1107,7 @@ dependencies = [ "tauri-plugin-single-instance", "tauri-plugin-updater", "tauri-plugin-window-state", + "tauri-utils", "tempfile", "tokio", "tokio-tungstenite 0.29.0", @@ -1085,6 +1117,7 @@ dependencies = [ "url", "user-idle", "uuid", + "webkit2gtk", "window-vibrancy", "windows-sys 0.61.2", "zeroize", @@ -1143,6 +1176,24 @@ dependencies = [ "uuid", ] +[[package]] +name = "buzz-voice" +version = "0.1.0" +dependencies = [ + "atomic-write-file", + "hex", + "ort", + "ort-sys", + "rand 0.10.2", + "sentencepiece-model", + "serde", + "serde_json", + "sha2 0.11.0", + "sherpa-onnx", + "symphonia", + "tokenizers", +] + [[package]] name = "by_address" version = "1.2.1" @@ -1508,7 +1559,7 @@ version = "3.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34" dependencies = [ - "windows-sys 0.48.0", + "windows-sys 0.61.2", ] [[package]] @@ -1545,6 +1596,7 @@ dependencies = [ "itoa", "rustversion", "ryu", + "serde", "static_assertions", ] @@ -1745,7 +1797,7 @@ dependencies = [ "wasm-bindgen", "wasm-bindgen-futures", "web-sys", - "windows 0.61.3", + "windows 0.62.2", ] [[package]] @@ -1796,6 +1848,16 @@ dependencies = [ "crossbeam-utils", ] +[[package]] +name = "crossbeam-deque" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5181e0de7b61eb03a81e347d6dd8797bae9da5146707b51077e2d71a54ec0ceb" +dependencies = [ + "crossbeam-epoch", + "crossbeam-utils", +] + [[package]] name = "crossbeam-epoch" version = "0.9.20" @@ -2120,6 +2182,15 @@ dependencies = [ "syn 2.0.118", ] +[[package]] +name = "dary_heap" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b1e3a325bc115f096c8b77bbf027a7c2592230e70be2d985be950d3d5e60ebe" +dependencies = [ + "serde", +] + [[package]] name = "dasp_sample" version = "0.11.0" @@ -2149,7 +2220,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ccc2776f0c61eca1ca32528f85548abd1a4be8fb53d1b21c013e4f18da1e7090" dependencies = [ "data-encoding", - "syn 1.0.109", + "syn 2.0.118", ] [[package]] @@ -2617,6 +2688,12 @@ version = "3.3.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "dea2df4cf52843e0452895c455a1a2cfbb842a1e7329671acf418fdc53ed4c59" +[[package]] +name = "esaxx-rs" +version = "0.1.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d817e038c30374a4bcb22f94d0a8a0e216958d4c3dcde369b1439fec4bdda6e6" + [[package]] name = "euclid" version = "0.22.14" @@ -2675,6 +2752,17 @@ dependencies = [ "regex", ] +[[package]] +name = "fancy-regex" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6e24cb5a94bcae1e5408b0effca5cd7172ea3c5755049c5f3af4cd283a165298" +dependencies = [ + "bit-set 0.8.0", + "regex-automata", + "regex-syntax", +] + [[package]] name = "fast-srgb8" version = "1.0.0" @@ -3099,8 +3187,8 @@ dependencies = [ "libc", "log", "rustversion", - "windows-link 0.1.3", - "windows-result 0.3.4", + "windows-link 0.2.1", + "windows-result 0.4.1", ] [[package]] @@ -3845,7 +3933,7 @@ dependencies = [ "tokio", "tower-service", "tracing", - "windows-registry 0.5.3", + "windows-registry 0.6.1", ] [[package]] @@ -3860,7 +3948,7 @@ dependencies = [ "js-sys", "log", "wasm-bindgen", - "windows-core 0.61.2", + "windows-core 0.62.2", ] [[package]] @@ -4746,6 +4834,39 @@ version = "0.4.33" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad" +[[package]] +name = "logos" +version = "0.14.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7251356ef8cb7aec833ddf598c6cb24d17b689d20b993f9d11a3d764e34e6458" +dependencies = [ + "logos-derive", +] + +[[package]] +name = "logos-codegen" +version = "0.14.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "59f80069600c0d66734f5ff52cc42f2dabd6b29d205f333d61fd7832e9e9963f" +dependencies = [ + "beef", + "fnv", + "lazy_static", + "proc-macro2", + "quote", + "regex-syntax", + "syn 2.0.118", +] + +[[package]] +name = "logos-derive" +version = "0.14.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24fb722b06a9dc12adb0963ed585f19fc61dc5413e6a9be9422ef92c091e731d" +dependencies = [ + "logos-codegen", +] + [[package]] name = "loom" version = "0.7.2" @@ -4855,6 +4976,22 @@ dependencies = [ "libc", ] +[[package]] +name = "macro_rules_attribute" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b3ae8f6d608c795738406608304d30a2dfbdc8e58e44f7ba43236da5208ded3c" +dependencies = [ + "macro_rules_attribute-proc_macro", + "pastey", +] + +[[package]] +name = "macro_rules_attribute-proc_macro" +version = "0.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc04a4c58212d57930a24bf47d3fa87485264a3a054e9c10e042eb373573ad3c" + [[package]] name = "markup5ever" version = "0.38.0" @@ -4881,6 +5018,16 @@ version = "0.8.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3" +[[package]] +name = "matrixmultiply" +version = "0.3.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f607c237553f086e7043417a51df26b2eb899d3caff94e6a67592ff992fedc7" +dependencies = [ + "autocfg", + "rawpointer", +] + [[package]] name = "maybe-async" version = "0.2.11" @@ -4936,8 +5083,8 @@ dependencies = [ [[package]] name = "mesh-llm-api-client" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "hex", "mesh-llm-client", @@ -4946,8 +5093,8 @@ dependencies = [ [[package]] name = "mesh-llm-api-server" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "mesh-llm-api-client", @@ -4957,13 +5104,13 @@ dependencies = [ [[package]] name = "mesh-llm-build-info" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" [[package]] name = "mesh-llm-client" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "async-trait", @@ -4980,7 +5127,7 @@ dependencies = [ "mesh-llm-types", "model-artifact", "nostr-sdk", - "prost", + "prost 0.14.4", "rand 0.10.2", "rustls", "serde", @@ -4994,8 +5141,8 @@ dependencies = [ [[package]] name = "mesh-llm-config" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "dirs", @@ -5010,8 +5157,8 @@ dependencies = [ [[package]] name = "mesh-llm-embedded-runtime" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "mesh-llm-host-runtime", @@ -5020,8 +5167,8 @@ dependencies = [ [[package]] name = "mesh-llm-events" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "clap", @@ -5032,8 +5179,8 @@ dependencies = [ [[package]] name = "mesh-llm-gpu-bench" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "cc", @@ -5045,8 +5192,8 @@ dependencies = [ [[package]] name = "mesh-llm-guardrails" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "serde", "serde_json", @@ -5054,16 +5201,16 @@ dependencies = [ [[package]] name = "mesh-llm-hardware-profile" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "mesh-llm-native-runtime", ] [[package]] name = "mesh-llm-host-runtime" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "argon2", @@ -5085,7 +5232,6 @@ dependencies = [ "http", "http-body-util", "httparse", - "if-addrs", "iroh", "json5", "keyring", @@ -5119,7 +5265,7 @@ dependencies = [ "opentelemetry", "opentelemetry-otlp", "opentelemetry_sdk", - "prost", + "prost 0.14.4", "rand 0.10.2", "regex-lite", "reqwest 0.12.28", @@ -5133,6 +5279,7 @@ dependencies = [ "serde_yaml", "sha2 0.10.9", "skippy-coordinator", + "skippy-ffi", "skippy-protocol", "skippy-runtime", "skippy-server", @@ -5155,8 +5302,8 @@ dependencies = [ [[package]] name = "mesh-llm-identity" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "argon2", "base64 0.22.1", @@ -5177,8 +5324,8 @@ dependencies = [ [[package]] name = "mesh-llm-native-runtime" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "serde", @@ -5188,8 +5335,8 @@ dependencies = [ [[package]] name = "mesh-llm-node" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "mesh-llm-types", @@ -5202,13 +5349,13 @@ dependencies = [ [[package]] name = "mesh-llm-plugin" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "async-trait", - "prost", - "prost-build", + "prost 0.14.4", + "prost-build 0.14.4", "protoc-bin-vendored", "rmcp", "schemars 1.2.1", @@ -5219,8 +5366,8 @@ dependencies = [ [[package]] name = "mesh-llm-plugin-manager" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "dirs", @@ -5230,6 +5377,7 @@ dependencies = [ "reqwest 0.12.28", "serde", "serde_json", + "sha2 0.10.9", "tar", "tempfile", "zip 2.4.2", @@ -5237,29 +5385,29 @@ dependencies = [ [[package]] name = "mesh-llm-protocol" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "hex", "iroh", - "prost", + "prost 0.14.4", "serde_json", "sha2 0.10.9", ] [[package]] name = "mesh-llm-routing" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "iroh", ] [[package]] name = "mesh-llm-runtime-install" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "dirs", @@ -5281,8 +5429,8 @@ dependencies = [ [[package]] name = "mesh-llm-sdk" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "mesh-llm-api-client", @@ -5296,8 +5444,8 @@ dependencies = [ [[package]] name = "mesh-llm-skills" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "dirs", @@ -5307,8 +5455,8 @@ dependencies = [ [[package]] name = "mesh-llm-system" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "chrono", @@ -5330,8 +5478,8 @@ dependencies = [ [[package]] name = "mesh-llm-types" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "hex", "serde", @@ -5341,13 +5489,13 @@ dependencies = [ [[package]] name = "mesh-llm-ui" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" [[package]] name = "mesh-mixture-of-agents" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "async-trait", "mesh-llm-guardrails", @@ -5358,6 +5506,28 @@ dependencies = [ "tracing", ] +[[package]] +name = "miette" +version = "7.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f98efec8807c63c752b5bd61f862c165c115b0a35685bdcfd9238c7aeb592b7" +dependencies = [ + "cfg-if 1.0.4", + "miette-derive", + "unicode-width 0.1.14", +] + +[[package]] +name = "miette-derive" +version = "7.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db5b29714e950dbb20d5e6f74f9dcec4edbcc1067bb7f8ed198c097b8c1a818b" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + [[package]] name = "mime" version = "0.3.17" @@ -5419,8 +5589,8 @@ dependencies = [ [[package]] name = "model-artifact" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "async-trait", @@ -5430,8 +5600,8 @@ dependencies = [ [[package]] name = "model-hf" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "async-trait", @@ -5448,8 +5618,8 @@ dependencies = [ [[package]] name = "model-package" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "bytes", @@ -5468,16 +5638,16 @@ dependencies = [ [[package]] name = "model-ref" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "serde", ] [[package]] name = "model-resolver" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "model-artifact", @@ -5503,6 +5673,28 @@ dependencies = [ "uuid", ] +[[package]] +name = "monostate" +version = "0.1.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3341a273f6c9d5bef1908f17b7267bbab0e95c9bf69a0d4dcf8e9e1b2c76ef67" +dependencies = [ + "monostate-impl", + "serde", + "serde_core", +] + +[[package]] +name = "monostate-impl" +version = "0.1.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e4db6d5580af57bf992f59068d4ea26fd518574ff48d7639b255a36f9de6e7e9" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + [[package]] name = "more-asserts" version = "0.3.1" @@ -5630,6 +5822,21 @@ dependencies = [ "tempfile", ] +[[package]] +name = "ndarray" +version = "0.17.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "520080814a7a6b4a6e9070823bb24b4531daac8c4627e08ba5de8c5ef2f2752d" +dependencies = [ + "matrixmultiply", + "num-complex", + "num-integer", + "num-traits", + "portable-atomic", + "portable-atomic-util", + "rawpointer", +] + [[package]] name = "ndk" version = "0.9.0" @@ -5917,9 +6124,9 @@ dependencies = [ [[package]] name = "nostr" -version = "0.44.6" +version = "0.44.7" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e826dd648489de2c5b293920e20b92932ef820302007c1987c758d4d06eeb2cf" +checksum = "c7d3d987ea7078dc36947cde532637c472a229426702e4331dd7667325378bd9" dependencies = [ "base64 0.22.1", "bech32", @@ -5961,9 +6168,9 @@ dependencies = [ [[package]] name = "nostr-relay-pool" -version = "0.44.1" +version = "0.44.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "91b2c039df4f96c4bf7dae52a74fd5516ad6dda83a11c0c69dea91b5255a4f37" +checksum = "c85c54d6ca9aae4ae2bf19a7663ba9db5f45f783f1d24aff55f006386b8b99a1" dependencies = [ "async-utility", "async-wsocket", @@ -6131,7 +6338,7 @@ version = "0.7.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "680998035259dcfcafe653688bf2aa6d3e2dc05e98be6ab46afb089dc84f1df8" dependencies = [ - "proc-macro-crate 1.3.1", + "proc-macro-crate 3.5.0", "proc-macro2", "quote", "syn 2.0.118", @@ -6505,8 +6712,8 @@ dependencies = [ [[package]] name = "openai-frontend" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "async-trait", "axum", @@ -6611,7 +6818,7 @@ dependencies = [ "opentelemetry-http", "opentelemetry-proto", "opentelemetry_sdk", - "prost", + "prost 0.14.4", "reqwest 0.12.28", "thiserror 2.0.18", ] @@ -6626,7 +6833,7 @@ dependencies = [ "const-hex", "opentelemetry", "opentelemetry_sdk", - "prost", + "prost 0.14.4", "serde", "serde_json", "tonic", @@ -6692,6 +6899,24 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "ort" +version = "2.0.0-rc.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7de3af33d24a745ffb8fab904b13478438d1cd52868e6f17735ef6e1f8bf133" +dependencies = [ + "ndarray", + "ort-sys", + "smallvec", + "tracing", +] + +[[package]] +name = "ort-sys" +version = "2.0.0-rc.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7b497d21a8b6fbb4b5a544f8fadb77e801a09ae0add9e411d31c6f89e3c1e90" + [[package]] name = "os_pipe" version = "1.2.3" @@ -6699,7 +6924,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7d8fae84b431384b68627d0f9b3b1245fcf9f46f6c0e3dc902e9dce64edd1967" dependencies = [ "libc", - "windows-sys 0.45.0", + "windows-sys 0.61.2", ] [[package]] @@ -6919,6 +7144,16 @@ dependencies = [ "pest", ] +[[package]] +name = "petgraph" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3672b37090dbd86368a4145bc067582552b29c27377cad4e0a306c97f9bd7772" +dependencies = [ + "fixedbitset 0.5.7", + "indexmap 2.14.0", +] + [[package]] name = "petgraph" version = "0.8.3" @@ -7183,6 +7418,15 @@ dependencies = [ "serde", ] +[[package]] +name = "portable-atomic-util" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a106d1259c23fac8e543272398ae0e3c0b8d33c88ed73d0cc71b0f1d902618" +dependencies = [ + "portable-atomic", +] + [[package]] name = "portmapper" version = "0.19.1" @@ -7393,6 +7637,16 @@ dependencies = [ "unarray", ] +[[package]] +name = "prost" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2796faa41db3ec313a31f7624d9286acf277b52de526150b7e69f3debf891ee5" +dependencies = [ + "bytes", + "prost-derive 0.13.5", +] + [[package]] name = "prost" version = "0.14.4" @@ -7400,7 +7654,27 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "528ac67416ff8646872a3c02cad9cc4ee5dc9f9540c9b10771855c95cb2e5ae1" dependencies = [ "bytes", - "prost-derive", + "prost-derive 0.14.4", +] + +[[package]] +name = "prost-build" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "be769465445e8c1474e9c5dac2018218498557af32d9ed057325ec9a41ae81bf" +dependencies = [ + "heck 0.5.0", + "itertools", + "log", + "multimap", + "once_cell", + "petgraph 0.7.1", + "prettyplease", + "prost 0.13.5", + "prost-types 0.13.5", + "regex", + "syn 2.0.118", + "tempfile", ] [[package]] @@ -7409,19 +7683,32 @@ version = "0.14.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "03da047801ff44bb6a4d407d4860c05fd70bb81714e6b2f3812603d5b145b042" dependencies = [ - "heck 0.4.1", + "heck 0.5.0", "itertools", "log", "multimap", - "petgraph", + "petgraph 0.8.3", "prettyplease", - "prost", - "prost-types", + "prost 0.14.4", + "prost-types 0.14.4", "regex", "syn 2.0.118", "tempfile", ] +[[package]] +name = "prost-derive" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a56d757972c98b346a9b766e3f02746cde6dd1cd1d1d563472929fdd74bec4d" +dependencies = [ + "anyhow", + "itertools", + "proc-macro2", + "quote", + "syn 2.0.118", +] + [[package]] name = "prost-derive" version = "0.14.4" @@ -7435,13 +7722,35 @@ dependencies = [ "syn 2.0.118", ] +[[package]] +name = "prost-reflect" +version = "0.14.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b5edd582b62f5cde844716e66d92565d7faf7ab1445c8cebce6e00fba83ddb2" +dependencies = [ + "logos", + "miette", + "once_cell", + "prost 0.13.5", + "prost-types 0.13.5", +] + +[[package]] +name = "prost-types" +version = "0.13.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52c2c1bf36ddb1a1c396b3601a3cec27c2462e45f07c386894ec3ccf5332bd16" +dependencies = [ + "prost 0.13.5", +] + [[package]] name = "prost-types" version = "0.14.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f94967dc7688f3054c7fac87473ffae4cc4c3904800e2d9f5b857246d8963b0a" dependencies = [ - "prost", + "prost 0.14.4", ] [[package]] @@ -7508,6 +7817,33 @@ version = "3.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "95067976aca6421a523e491fce939a3e65249bac4b977adee0ee9771568e8aa3" +[[package]] +name = "protox" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6f352af331bf637b8ecc720f7c87bf903d2571fa2e14a66e9b2558846864b54a" +dependencies = [ + "bytes", + "miette", + "prost 0.13.5", + "prost-reflect", + "prost-types 0.13.5", + "protox-parse", + "thiserror 1.0.69", +] + +[[package]] +name = "protox-parse" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a3a462d115462c080ae000c29a47f0b3985737e5d3a995fcdbcaa5c782068dde" +dependencies = [ + "logos", + "miette", + "prost-types 0.13.5", + "thiserror 1.0.69", +] + [[package]] name = "pxfm" version = "0.1.30" @@ -7756,7 +8092,7 @@ dependencies = [ "thiserror 2.0.18", "unicode-segmentation", "unicode-truncate", - "unicode-width", + "unicode-width 0.2.2", ] [[package]] @@ -7819,7 +8155,7 @@ dependencies = [ "strum", "time", "unicode-segmentation", - "unicode-width", + "unicode-width 0.2.2", ] [[package]] @@ -7828,6 +8164,43 @@ version = "0.6.2" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "20675572f6f24e9e76ef639bc5552774ed45f1c30e2951e1e99c59888861c539" +[[package]] +name = "rawpointer" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "60a357793950651c4ed0f3f52338f53b2f809f32d83a07f72909fa13e4c6c1e3" + +[[package]] +name = "rayon" +version = "1.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d" +dependencies = [ + "either", + "rayon-core", +] + +[[package]] +name = "rayon-cond" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2964d0cf57a3e7a06e8183d14a8b527195c706b7983549cd5462d5aa3747438f" +dependencies = [ + "either", + "itertools", + "rayon", +] + +[[package]] +name = "rayon-core" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91" +dependencies = [ + "crossbeam-deque", + "crossbeam-utils", +] + [[package]] name = "realfft" version = "3.5.0" @@ -8640,6 +9013,18 @@ version = "0.6.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "cd0b0ec5f1c1ca621c432a25813d8d60c88abe6d3e08a3eb9cf37d97a0fe3d73" +[[package]] +name = "sentencepiece-model" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40b87bf750a8322c3236d7aa63c1f4a6862187d00d2d8b038e1dfe263bfe43ec" +dependencies = [ + "miette", + "prost 0.13.5", + "prost-build 0.13.5", + "protox", +] + [[package]] name = "serde" version = "1.0.228" @@ -9038,8 +9423,8 @@ checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649" [[package]] name = "skippy-cache" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "blake3", @@ -9048,40 +9433,40 @@ dependencies = [ [[package]] name = "skippy-coordinator" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "thiserror 2.0.18", ] [[package]] name = "skippy-ffi" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "libloading 0.8.9", ] [[package]] name = "skippy-metrics" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" [[package]] name = "skippy-protocol" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ - "prost", - "prost-build", + "prost 0.14.4", + "prost-build 0.14.4", "protoc-bin-vendored", "serde", ] [[package]] name = "skippy-runtime" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "anyhow", "libc", @@ -9094,9 +9479,10 @@ dependencies = [ [[package]] name = "skippy-server" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ + "ahash", "anyhow", "async-trait", "axum", @@ -9122,8 +9508,8 @@ dependencies = [ [[package]] name = "skippy-topology" -version = "0.73.1" -source = "git+https://github.com/Mesh-LLM/mesh-llm.git?rev=f455d493a2ae82baf2a326e2d0fda351433b4b30#f455d493a2ae82baf2a326e2d0fda351433b4b30" +version = "0.74.0" +source = "git+https://github.com/Mesh-LLM/mesh-llm.git?tag=v0.74.0#e60b2fe43aa05271569fbeff2a457133aef456a1" dependencies = [ "serde", "serde_json", @@ -9252,6 +9638,18 @@ dependencies = [ "der", ] +[[package]] +name = "spm_precompiled" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5851699c4033c63636f7ea4cf7b7c1f1bf06d0cc03cfb42e711de5a5c46cf326" +dependencies = [ + "base64 0.13.1", + "nom 7.1.3", + "serde", + "unicode-segmentation", +] + [[package]] name = "sse-stream" version = "0.2.4" @@ -9386,6 +9784,7 @@ dependencies = [ "symphonia-bundle-flac", "symphonia-bundle-mp3", "symphonia-codec-aac", + "symphonia-codec-alac", "symphonia-codec-pcm", "symphonia-codec-vorbis", "symphonia-core", @@ -9430,6 +9829,16 @@ dependencies = [ "symphonia-core", ] +[[package]] +name = "symphonia-codec-alac" +version = "0.5.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8413fa754942ac16a73634c9dfd1500ed5c61430956b33728567f667fdd393ab" +dependencies = [ + "log", + "symphonia-core", +] + [[package]] name = "symphonia-codec-pcm" version = "0.5.5" @@ -9633,7 +10042,7 @@ version = "1.4.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fce91f2f0ec87dff7e6bcbbeb267439aa1188703003c6055193c821487400432" dependencies = [ - "unicode-width", + "unicode-width 0.2.2", ] [[package]] @@ -10150,7 +10559,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" dependencies = [ "fastrand", - "getrandom 0.3.4", + "getrandom 0.4.3", "once_cell", "rustix 1.1.4", "windows-sys 0.61.2", @@ -10208,7 +10617,7 @@ dependencies = [ "anyhow", "base64 0.22.1", "bitflags 2.13.0", - "fancy-regex", + "fancy-regex 0.11.0", "filedescriptor", "finl_unicode", "fixedbitset 0.4.2", @@ -10371,6 +10780,39 @@ version = "0.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" +[[package]] +name = "tokenizers" +version = "0.22.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b238e22d44a15349529690fb07bd645cf58149a1b1e44d6cb5bd1641ff1a6223" +dependencies = [ + "ahash", + "aho-corasick", + "compact_str 0.9.1", + "dary_heap", + "derive_builder", + "esaxx-rs", + "fancy-regex 0.14.0", + "getrandom 0.3.4", + "itertools", + "log", + "macro_rules_attribute", + "monostate", + "paste", + "rand 0.9.4", + "rayon", + "rayon-cond", + "regex", + "regex-syntax", + "serde", + "serde_json", + "spm_precompiled", + "thiserror 2.0.18", + "unicode-normalization-alignments", + "unicode-segmentation", + "unicode_categories", +] + [[package]] name = "tokio" version = "1.52.3" @@ -10704,7 +11146,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "50849f68853be452acf590cde0b146665b8d507b3b8af17261df47e02c209ea0" dependencies = [ "bytes", - "prost", + "prost 0.14.4", "tonic", ] @@ -10885,7 +11327,7 @@ checksum = "b8765b90061cba6c22b5831f675da109ae5561588290f9fa2317adab2714d5a6" dependencies = [ "memchr", "nom 8.0.0", - "petgraph", + "petgraph 0.8.3", ] [[package]] @@ -11056,6 +11498,15 @@ dependencies = [ "tinyvec", ] +[[package]] +name = "unicode-normalization-alignments" +version = "0.1.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43f613e4fa046e69818dd287fdc4bc78175ff20331479dab6e1b0f98d57062de" +dependencies = [ + "smallvec", +] + [[package]] name = "unicode-segmentation" version = "1.13.3" @@ -11070,9 +11521,15 @@ checksum = "16b380a1238663e5f8a691f9039c73e1cdae598a30e9855f541d29b08b53e9a5" dependencies = [ "itertools", "unicode-segmentation", - "unicode-width", + "unicode-width 0.2.2", ] +[[package]] +name = "unicode-width" +version = "0.1.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7dd6e30e90baa6f72411720665d41d89b9a3d039dc45b8faea1ddd07f617f6af" + [[package]] name = "unicode-width" version = "0.2.2" @@ -11085,6 +11542,12 @@ version = "0.2.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853" +[[package]] +name = "unicode_categories" +version = "0.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39ec24b3121d976906ece63c9daad25b85969647682eee313cb5779fdd69e14e" + [[package]] name = "universal-hash" version = "0.5.1" @@ -11786,7 +12249,7 @@ version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" dependencies = [ - "windows-sys 0.48.0", + "windows-sys 0.61.2", ] [[package]] @@ -12415,8 +12878,8 @@ dependencies = [ "log", "serde", "thiserror 2.0.18", - "windows 0.61.3", - "windows-core 0.61.2", + "windows 0.62.2", + "windows-core 0.62.2", ] [[package]] diff --git a/desktop/src-tauri/Cargo.toml b/desktop/src-tauri/Cargo.toml index 324218a49d..fd58f27878 100644 --- a/desktop/src-tauri/Cargo.toml +++ b/desktop/src-tauri/Cargo.toml @@ -2,7 +2,7 @@ [package] name = "buzz-desktop" -version = "0.5.0" +version = "0.5.3" description = "Buzz desktop app" authors = ["you"] edition = "2021" @@ -40,9 +40,16 @@ keyring = { version = "3.6.3", default-features = false, features = ["sync-secre # connection is dropped, which the plugin does immediately. Default features # keep the pure-Rust zbus backend, matching the plugin (no libdbus needed). notify-rust = "4" +# Enable getUserMedia in the WebKitGTK webview (see src/linux_media.rs). Pinned +# to the exact version wry links so both resolve to one webkit2gtk-sys and we +# don't get duplicate symbols; bump in lockstep with wry. +webkit2gtk = { version = "=2.0.2", features = ["v2_22"] } [target.'cfg(target_os = "macos")'.dependencies] -objc2-app-kit = { version = "0.3.2", default-features = false, features = ["NSHapticFeedback"] } +block2 = { version = "0.6", default-features = false, features = ["std"] } +objc2 = { version = "0.6.4", default-features = false } +objc2-app-kit = { version = "0.3.2", default-features = false, features = ["NSEvent", "NSHapticFeedback", "NSMenu", "NSMenuItem", "NSStatusItem", "block2"] } +objc2-foundation = { version = "0.3.2", default-features = false, features = ["NSProcessInfo", "NSString"] } keyring = { version = "3.6.3", default-features = false, features = ["apple-native", "vendored"], optional = true } security-framework = { version = "3.7.0", features = ["OSX_10_15"] } window-vibrancy = "0.6" @@ -58,7 +65,7 @@ user-idle = { version = "0.6", default-features = false } atomic-write-file = "0.3" anyhow = "1" dirs = "6" -tauri = { version = "2", features = ["macos-private-api"] } +tauri = { version = "2", features = ["macos-private-api", "tray-icon"] } tauri-plugin-deep-link = "2" tauri-plugin-opener = "2" tauri-plugin-single-instance = { version = "2", features = ["deep-link"] } @@ -80,7 +87,10 @@ serde = { version = "1", features = ["derive"] } serde_json = "1" serde_yaml = "0.9" toml = "0.8" -nostr = { version = "0.44", features = ["nip44"] } +nostr = { version = "0.44", features = ["nip44", "nip49"] } +# OS-entropy source for backup passphrase generation (already in the tree as a +# transitive dependency; pinned here for direct use). +getrandom = "0.2" zeroize = "1" reqwest = { version = "0.13", features = ["json", "query", "stream", "blocking"] } rustls = { version = "0.23", default-features = false, features = ["aws_lc_rs", "std"] } @@ -89,15 +99,16 @@ buzz_core_pkg = { package = "buzz-core", path = "../../crates/buzz-core" } buzz_persona_pkg = { package = "buzz-persona", path = "../../crates/buzz-persona" } buzz_sdk_pkg = { package = "buzz-sdk", path = "../../crates/buzz-sdk" } buzz_agent_pkg = { package = "buzz-agent", path = "../../crates/buzz-agent" } +buzz_voice_pkg = { package = "buzz-voice", path = "../../crates/buzz-voice" } iroh = { version = "1.0.2", optional = true } -mesh-llm-sdk = { git = "https://github.com/Mesh-LLM/mesh-llm.git", rev = "f455d493a2ae82baf2a326e2d0fda351433b4b30", package = "mesh-llm-sdk", default-features = false, features = ["client", "serving"], optional = true } -mesh-llm-host-runtime = { git = "https://github.com/Mesh-LLM/mesh-llm.git", rev = "f455d493a2ae82baf2a326e2d0fda351433b4b30", package = "mesh-llm-host-runtime", default-features = false, features = ["dynamic-native-runtime"], optional = true } +mesh-llm-sdk = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-sdk", default-features = false, features = ["client", "serving"], optional = true } +mesh-llm-host-runtime = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-host-runtime", default-features = false, features = ["dynamic-native-runtime"], optional = true } # Model catalog + hardware survey for the Share-compute model picker (same # diagnose pattern as mesh-console). Lib name of mesh-llm-client is mesh_client. -mesh-llm-client = { git = "https://github.com/Mesh-LLM/mesh-llm.git", rev = "f455d493a2ae82baf2a326e2d0fda351433b4b30", package = "mesh-llm-client", optional = true } -mesh-llm-node = { git = "https://github.com/Mesh-LLM/mesh-llm.git", rev = "f455d493a2ae82baf2a326e2d0fda351433b4b30", package = "mesh-llm-node", optional = true } -mesh-llm-system = { git = "https://github.com/Mesh-LLM/mesh-llm.git", rev = "f455d493a2ae82baf2a326e2d0fda351433b4b30", package = "mesh-llm-system", optional = true } -mesh-llm-events = { git = "https://github.com/Mesh-LLM/mesh-llm.git", rev = "f455d493a2ae82baf2a326e2d0fda351433b4b30", package = "mesh-llm-events", optional = true } +mesh-llm-client = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-client", optional = true } +mesh-llm-node = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-node", optional = true } +mesh-llm-system = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-system", optional = true } +mesh-llm-events = { git = "https://github.com/Mesh-LLM/mesh-llm.git", tag = "v0.74.0", package = "mesh-llm-events", optional = true } base64 = "0.22" sha2 = "0.11" tar = "0.4" @@ -128,6 +139,7 @@ strip-ansi-escapes = "0.2" tracing = "0.1" [dev-dependencies] +tauri-utils = "2" # `test-util` enables tokio's paused-clock (`start_paused`) so the relay # admission gate tests can assert exact wait durations without real sleeps. tokio = { version = "1", features = ["test-util"] } diff --git a/desktop/src-tauri/assets/card_template.png b/desktop/src-tauri/assets/card_template.png new file mode 100644 index 0000000000..2225d1d442 Binary files /dev/null and b/desktop/src-tauri/assets/card_template.png differ diff --git a/desktop/src-tauri/examples/pocket_bench.rs b/desktop/src-tauri/examples/pocket_bench.rs deleted file mode 100644 index b4f5635a95..0000000000 --- a/desktop/src-tauri/examples/pocket_bench.rs +++ /dev/null @@ -1,116 +0,0 @@ -//! Cold-vs-warm latency bench for Pocket TTS. -//! -//! This duplicates the small config-building snippet from `huddle::pocket` so it -//! doesn't depend on changing module visibility for a one-off dev tool. -//! Keep in sync with `huddle::pocket::load_text_to_speech`. -//! -//! Run with the model files in a directory (defaults to /tmp/pocket-tts-bench): -//! cargo run --release --example pocket_bench -//! cargo run --release --example pocket_bench /path/to/pocket-tts - -use std::path::PathBuf; -use std::time::Instant; - -use sherpa_onnx::{ - self, GenerationConfig, OfflineTts, OfflineTtsConfig, OfflineTtsModelConfig, - OfflineTtsPocketModelConfig, Wave, -}; - -const SAMPLE_RATE: u32 = 24_000; -const TEST_TEXT: &str = - "Hello, this is a test of the new Pocket TTS engine running on sherpa-onnx."; - -fn main() { - let model_dir = std::env::args() - .nth(1) - .unwrap_or_else(|| "/tmp/pocket-tts-bench".to_string()); - println!("Model dir: {model_dir}"); - - let dir = PathBuf::from(&model_dir); - let p = |name: &str| dir.join(name).to_string_lossy().into_owned(); - - let t0 = Instant::now(); - let cfg = OfflineTtsConfig { - model: OfflineTtsModelConfig { - pocket: OfflineTtsPocketModelConfig { - lm_main: Some(p("lm_main.int8.onnx")), - lm_flow: Some(p("lm_flow.int8.onnx")), - encoder: Some(p("encoder.onnx")), - decoder: Some(p("decoder.int8.onnx")), - text_conditioner: Some(p("text_conditioner.onnx")), - vocab_json: Some(p("vocab.json")), - token_scores_json: Some(p("token_scores.json")), - voice_embedding_cache_capacity: 16, - }, - num_threads: 1, - debug: false, - ..Default::default() - }, - ..Default::default() - }; - let engine = OfflineTts::create(&cfg).expect("engine create"); - let load_ms = t0.elapsed().as_secs_f32() * 1000.0; - println!("Engine load: {load_ms:.1} ms"); - - let t0 = Instant::now(); - let voice_path = dir.join("reference_sample.wav"); - let wave = Wave::read(voice_path.to_str().unwrap()).expect("voice WAV"); - let samples = wave.samples().to_vec(); - let sr = wave.sample_rate(); - let voice_ms = t0.elapsed().as_secs_f32() * 1000.0; - println!("Voice load: {voice_ms:.1} ms"); - - let gen = || GenerationConfig { - speed: 1.05, - num_steps: 1, - silence_scale: 1.0, // production setting (huddle::pocket::SYNTH_SILENCE_SCALE) - reference_audio: Some(samples.clone()), - reference_sample_rate: sr, - ..Default::default() - }; - - let t0 = Instant::now(); - let cold = engine - .generate_with_config(TEST_TEXT, &gen(), None:: bool>) - .expect("cold synth"); - let cold_ms = t0.elapsed().as_secs_f32() * 1000.0; - let cold_audio_ms = (cold.samples().len() as f32 / SAMPLE_RATE as f32) * 1000.0; - let cold_rtf_x = cold_audio_ms / cold_ms; - println!( - "Cold synth: {cold_ms:.1} ms → {cold_audio_ms:.1} ms audio → {cold_rtf_x:.2}× realtime" - ); - - let t0 = Instant::now(); - let warm = engine - .generate_with_config(TEST_TEXT, &gen(), None:: bool>) - .expect("warm synth"); - let warm_ms = t0.elapsed().as_secs_f32() * 1000.0; - let warm_audio_ms = (warm.samples().len() as f32 / SAMPLE_RATE as f32) * 1000.0; - let warm_rtf_x = warm_audio_ms / warm_ms; - println!( - "Warm synth: {warm_ms:.1} ms → {warm_audio_ms:.1} ms audio → {warm_rtf_x:.2}× realtime" - ); - - let out_path = "/tmp/pocket_bench_out.wav"; - let ok = sherpa_onnx::write(out_path, warm.samples(), SAMPLE_RATE as i32); - println!( - "Wrote {} ({} samples, ok={ok})", - out_path, - warm.samples().len() - ); - - let delta_ms = cold_ms - warm_ms; - let delta_pct = (delta_ms / warm_ms) * 100.0; - println!(); - println!("Cold/warm delta: {delta_ms:+.1} ms ({delta_pct:+.1}%)"); - println!( - "Decision: warmup {}.", - if delta_ms > 200.0 { - "RECOMMENDED — significant cold-call penalty" - } else if delta_ms > 50.0 { - "OPTIONAL — small cold-call penalty" - } else { - "UNNECESSARY — cold and warm essentially equal" - } - ); -} diff --git a/desktop/src-tauri/examples/pocket_clip_probe.rs b/desktop/src-tauri/examples/pocket_clip_probe.rs deleted file mode 100644 index ad8657599f..0000000000 --- a/desktop/src-tauri/examples/pocket_clip_probe.rs +++ /dev/null @@ -1,122 +0,0 @@ -//! Clipping probe for any fixed playback gain applied after Pocket TTS synth. -//! -//! Synthesises a spread of sentences (short/long, calm/energetic) and reports -//! the raw peak of each, the post-gain peak, and the fraction of samples that -//! would hit a ±1.0 clamp — i.e. how much a fixed gain would flat-top the -//! waveform ("blown out" distortion). -//! -//! History: the production pipeline briefly shipped a fixed 9.3× gain -//! calibrated on a single bench utterance that peaked at 0.076. This probe -//! showed real output peaks at 0.4–0.97, so that gain clipped 13–34% of all -//! samples (the 2026-06-12 "blown out" report). Production now applies no -//! gain — run this probe before reintroducing one. -//! -//! Run with model files in ~/.buzz/models/pocket-tts (override with arg 1): -//! cargo run --release --example pocket_clip_probe - -use std::path::PathBuf; - -use sherpa_onnx::{ - self, GenerationConfig, OfflineTts, OfflineTtsConfig, OfflineTtsModelConfig, - OfflineTtsPocketModelConfig, Wave, -}; - -/// Candidate gain under test (the regressed production value). -const GAIN: f32 = 9.3; - -const PROMPTS: &[&str] = &[ - "Hello, this is a test of the new Pocket TTS engine running on sherpa-onnx.", - "Yep, I can hear you.", - "Absolutely! That sounds fantastic, let's do it right now!", - "The quick brown fox jumps over the lazy dog near the riverbank.", - "I found three problems in the code: a race condition, a memory leak, and an off-by-one error in the loop bounds.", - "No.", - "Warning! The build failed because seventeen tests crashed unexpectedly!", - "Sure, I can walk you through the whole pipeline step by step whenever you're ready.", -]; - -fn main() { - let model_dir = std::env::args().nth(1).unwrap_or_else(|| { - dirs::home_dir() - .expect("home dir") - .join(".buzz/models/pocket-tts") - .to_string_lossy() - .into_owned() - }); - eprintln!("Model dir: {model_dir}"); - - let dir = PathBuf::from(&model_dir); - let p = |name: &str| dir.join(name).to_string_lossy().into_owned(); - - let cfg = OfflineTtsConfig { - model: OfflineTtsModelConfig { - pocket: OfflineTtsPocketModelConfig { - lm_main: Some(p("lm_main.int8.onnx")), - lm_flow: Some(p("lm_flow.int8.onnx")), - encoder: Some(p("encoder.onnx")), - decoder: Some(p("decoder.int8.onnx")), - text_conditioner: Some(p("text_conditioner.onnx")), - vocab_json: Some(p("vocab.json")), - token_scores_json: Some(p("token_scores.json")), - voice_embedding_cache_capacity: 16, - }, - num_threads: 1, - debug: false, - ..Default::default() - }, - ..Default::default() - }; - let engine = OfflineTts::create(&cfg).expect("engine create"); - - let voice_path = dir.join("reference_sample.wav"); - let wave = Wave::read(voice_path.to_str().unwrap()).expect("voice WAV"); - let voice_samples = wave.samples().to_vec(); - let voice_sr = wave.sample_rate(); - - let gen = || GenerationConfig { - speed: 1.05, - num_steps: 1, - silence_scale: 1.0, - reference_audio: Some(voice_samples.clone()), - reference_sample_rate: voice_sr, - ..Default::default() - }; - - let _ = engine.generate_with_config("warmup.", &gen(), None:: bool>); - - println!( - "{:<46} | {:>8} | {:>9} | {:>9} | {:>10}", - "prompt", "raw peak", "raw RMS", "post-gain", "% clipped" - ); - println!("{}", "-".repeat(95)); - - let mut worst_clip = 0.0f32; - for prompt in PROMPTS { - let out = engine - .generate_with_config(prompt, &gen(), None:: bool>) - .expect("synth"); - let samples = out.samples(); - - let peak = samples.iter().fold(0.0f32, |m, s| m.max(s.abs())); - let rms = (samples.iter().map(|s| s * s).sum::() / samples.len() as f32).sqrt(); - let post = peak * GAIN; - let clipped = samples.iter().filter(|s| s.abs() * GAIN > 1.0).count(); - let clip_pct = 100.0 * clipped as f32 / samples.len() as f32; - worst_clip = worst_clip.max(clip_pct); - - let label: String = prompt.chars().take(44).collect(); - println!("{label:<46} | {peak:>8.4} | {rms:>9.4} | {post:>9.3} | {clip_pct:>9.3}%"); - } - - println!(); - println!( - "Verdict: worst-case clipped fraction {worst_clip:.3}% — {}", - if worst_clip > 0.1 { - "AUDIBLE DISTORTION LIKELY (gain too hot)" - } else if worst_clip > 0.0 { - "marginal — occasional transient clipping" - } else { - "no clipping at this gain" - } - ); -} diff --git a/desktop/src-tauri/examples/pocket_onset_probe.rs b/desktop/src-tauri/examples/pocket_onset_probe.rs deleted file mode 100644 index 05b4d0193c..0000000000 --- a/desktop/src-tauri/examples/pocket_onset_probe.rs +++ /dev/null @@ -1,149 +0,0 @@ -//! Onset-attenuation probe for Pocket TTS. -//! -//! Synthesises a handful of short sentences and dumps per-sentence onset -//! statistics (samples[0], 1ms/5ms/20ms peak + RMS) so we can decide whether -//! the production `apply_fades` 8 ms fade-in is masking real audio. -//! -//! Also writes the raw (un-faded, un-normalised) audio of each sentence to -//! /tmp so they can be inspected in Audacity / aplay without rodio in the -//! loop. -//! -//! Run with model files in /tmp/pocket-tts-bench (override with arg 1): -//! cargo run --release --example pocket_onset_probe -//! cargo run --release --example pocket_onset_probe /path/to/pocket-tts - -use std::path::PathBuf; - -use sherpa_onnx::{ - self, GenerationConfig, OfflineTts, OfflineTtsConfig, OfflineTtsModelConfig, - OfflineTtsPocketModelConfig, Wave, -}; - -const SAMPLE_RATE: u32 = 24_000; - -/// Test prompts chosen to span different onsets: -/// - palatal glide 'Y' (soft onset) -/// - voiceless fricative 'H' (very soft onset) -/// - labio-velar glide 'W' (medium onset) -/// - voiceless stop 'T' (hard onset) -const PROMPTS: &[&str] = &[ - "Yep, I can hear you.", - "Hello there friend.", - "What can I help with?", - "Try this experiment now.", -]; - -fn main() { - let model_dir = std::env::args() - .nth(1) - .unwrap_or_else(|| "/tmp/pocket-tts-bench".to_string()); - eprintln!("Model dir: {model_dir}"); - - let dir = PathBuf::from(&model_dir); - let p = |name: &str| dir.join(name).to_string_lossy().into_owned(); - - let cfg = OfflineTtsConfig { - model: OfflineTtsModelConfig { - pocket: OfflineTtsPocketModelConfig { - lm_main: Some(p("lm_main.int8.onnx")), - lm_flow: Some(p("lm_flow.int8.onnx")), - encoder: Some(p("encoder.onnx")), - decoder: Some(p("decoder.int8.onnx")), - text_conditioner: Some(p("text_conditioner.onnx")), - vocab_json: Some(p("vocab.json")), - token_scores_json: Some(p("token_scores.json")), - voice_embedding_cache_capacity: 16, - }, - num_threads: 1, - debug: false, - ..Default::default() - }, - ..Default::default() - }; - let engine = OfflineTts::create(&cfg).expect("engine create"); - - let voice_path = dir.join("reference_sample.wav"); - let wave = Wave::read(voice_path.to_str().unwrap()).expect("voice WAV"); - let voice_samples = wave.samples().to_vec(); - let voice_sr = wave.sample_rate(); - - // Warmup so we're not measuring cold-call jitter. - { - let cfg = GenerationConfig { - speed: 1.05, - num_steps: 1, - silence_scale: 1.0, // production setting (huddle::pocket::SYNTH_SILENCE_SCALE) - reference_audio: Some(voice_samples.clone()), - reference_sample_rate: voice_sr, - ..Default::default() - }; - let _ = engine.generate_with_config("warmup.", &cfg, None:: bool>); - } - - println!( - "{:<28} | {:>10} | {:>10} {:>10} | {:>10} {:>10} | {:>10} {:>10}", - "prompt", - "samples[0]", - "peak@1ms", - "rms@1ms", - "peak@5ms", - "rms@5ms", - "peak@20ms", - "rms@20ms" - ); - println!("{}", "-".repeat(120)); - - for prompt in PROMPTS { - // Mirror the production prompt-prep (capitalise + terminal punctuation). - // These prompts already have it, so this is just to match what - // sherpa-onnx sees in production. - let cfg = GenerationConfig { - speed: 1.05, - num_steps: 1, - silence_scale: 1.0, // production setting (huddle::pocket::SYNTH_SILENCE_SCALE) - reference_audio: Some(voice_samples.clone()), - reference_sample_rate: voice_sr, - ..Default::default() - }; - let out = engine - .generate_with_config(prompt, &cfg, None:: bool>) - .expect("synth"); - let samples = out.samples(); - - let n_1ms = (SAMPLE_RATE as f32 * 0.001) as usize; - let n_5ms = (SAMPLE_RATE as f32 * 0.005) as usize; - let n_20ms = (SAMPLE_RATE as f32 * 0.020) as usize; - - let stats = |range: &[f32]| -> (f32, f32) { - if range.is_empty() { - return (0.0, 0.0); - } - let peak = range.iter().fold(0.0_f32, |a, &x| a.max(x.abs())); - let sumsq: f32 = range.iter().map(|x| x * x).sum(); - let rms = (sumsq / range.len() as f32).sqrt(); - (peak, rms) - }; - - let first = samples.first().copied().unwrap_or(0.0); - let (p1, r1) = stats(&samples[..n_1ms.min(samples.len())]); - let (p5, r5) = stats(&samples[..n_5ms.min(samples.len())]); - let (p20, r20) = stats(&samples[..n_20ms.min(samples.len())]); - - println!( - "{:<28} | {:>10.6} | {:>10.6} {:>10.6} | {:>10.6} {:>10.6} | {:>10.6} {:>10.6}", - prompt, first, p1, r1, p5, r5, p20, r20 - ); - - let safe: String = prompt - .chars() - .map(|c| if c.is_ascii_alphanumeric() { c } else { '_' }) - .collect(); - let out_path = format!("/tmp/pocket_onset_{}.wav", &safe[..safe.len().min(24)]); - let _ = sherpa_onnx::write(&out_path, samples, SAMPLE_RATE as i32); - eprintln!( - " → wrote {out_path} ({} samples = {:.3} s)", - samples.len(), - samples.len() as f32 / SAMPLE_RATE as f32 - ); - } -} diff --git a/desktop/src-tauri/examples/pocket_quality_ab.rs b/desktop/src-tauri/examples/pocket_quality_ab.rs deleted file mode 100644 index 0c31f1c910..0000000000 --- a/desktop/src-tauri/examples/pocket_quality_ab.rs +++ /dev/null @@ -1,519 +0,0 @@ -//! Reproducible blind Pocket TTS quality corpus generator. -//! -//! Renders Buzz's production prompt preparation and post-processing across: -//! INT8/FP32 × per-sentence/grouped generation. The generated filenames are -//! deterministically blinded; keep `key.json` away from listeners until their -//! scoring sheet is complete. -//! -//! Usage: -//! cargo run --release --example pocket_quality_ab -- \ -//! [--idle-minutes N --only ITEM] -//! -//! The optional idle run intentionally creates one engine per condition, warms -//! all four, sleeps once, and then makes each clip the first generation after -//! dormancy. It requires `--only` because only the first synthesis after an -//! uninterrupted idle is a valid post-idle observation. Run each 5/15-minute -//! item as a separate process. - -// Importing the production module also brings in runtime-only helpers that this -// standalone corpus generator deliberately does not call. -#![allow(dead_code)] - -#[path = "../src/huddle/pocket.rs"] -mod production_pocket; -#[path = "../src/huddle/preprocessing.rs"] -mod production_preprocessing; - -use std::collections::HashMap; -use std::fs; -use std::path::{Path, PathBuf}; -use std::time::{Duration, Instant}; - -use serde::Serialize; -use sha2::{Digest, Sha256}; -use sherpa_onnx::{GenerationConfig, OfflineTts, OfflineTtsConfig, Wave}; - -use production_pocket::{prepare_pocket_prompt, SAMPLE_RATE}; -use production_preprocessing::{preprocess_for_tts, split_sentences}; - -const NUM_STEPS: i32 = 1; -const SILENCE_SCALE: f32 = 1.0; -const INTER_SENTENCE_SILENCE_SAMPLES: usize = SAMPLE_RATE as usize / 10; -const LEAD_IN_SAMPLES: usize = SAMPLE_RATE as usize / 50; -const FADE_OUT_SAMPLES: usize = SAMPLE_RATE as usize * 8 / 1000; -const TARGET_RMS_DBFS: f32 = -23.0; -const BLINDING_SEED: &str = "pocket-quality-2026-07-21-v1"; - -const CORPUS: &[CorpusItem] = &[ - CorpusItem { id: "short_one_word", kind: "short", text: "Yep." }, - CorpusItem { id: "short_four_words", kind: "short", text: "Sounds good to me." }, - CorpusItem { - id: "multi_relay_review", - kind: "multi-sentence", - text: "I looked at the relay code this morning. The lease logic is solid. There's one race in the worker claim path, though. I'll write it up and send you a patch.", - }, - CorpusItem { - id: "multi_community_size", - kind: "multi-sentence", - text: "Great question. The answer is it depends on the community size. For small ones, keep it simple.", - }, - CorpusItem { - id: "mixed_agent_message", - kind: "mixed", - text: "That's 42 open PRs right now — mostly small. I'll triage them after lunch.", - }, -]; - -#[derive(Clone, Copy)] -struct CorpusItem { - id: &'static str, - kind: &'static str, - text: &'static str, -} - -#[derive(Clone, Copy, Debug, Serialize)] -#[serde(rename_all = "snake_case")] -enum Precision { - Int8, - Fp32, -} - -#[derive(Clone, Copy, Debug, Serialize)] -#[serde(rename_all = "snake_case")] -enum Chunking { - PerSentence, - Grouped, -} - -#[derive(Clone, Copy, Debug)] -struct Condition { - precision: Precision, - chunking: Chunking, -} - -const CONDITIONS: [Condition; 4] = [ - Condition { - precision: Precision::Int8, - chunking: Chunking::PerSentence, - }, - Condition { - precision: Precision::Int8, - chunking: Chunking::Grouped, - }, - Condition { - precision: Precision::Fp32, - chunking: Chunking::PerSentence, - }, - Condition { - precision: Precision::Fp32, - chunking: Chunking::Grouped, - }, -]; - -#[derive(Serialize)] -struct KeyFile { - warning: &'static str, - blinding_seed: &'static str, - target_rms_dbfs: f32, - items: Vec, -} - -#[derive(Serialize)] -struct KeyItem { - id: String, - kind: String, - text: String, - clips: Vec, -} - -#[derive(Serialize)] -struct KeyClip { - file: String, - precision: Precision, - chunking: Chunking, - cold_start: bool, - idle_minutes: Option, - synthesis_ms: u128, - audio_seconds: f32, -} - -struct Voice { - samples: Vec, - sample_rate: i32, -} - -struct Engine { - inner: OfflineTts, - voice: Voice, -} - -fn main() -> Result<(), String> { - let mut args = std::env::args().skip(1); - let int8_dir = required_path(args.next(), "INT8 model directory")?; - let fp32_dir = required_path(args.next(), "FP32 model directory")?; - let output_dir = required_path(args.next(), "output directory")?; - let mut idle_minutes = None; - let mut only_item = None; - while let Some(arg) = args.next() { - match arg.as_str() { - "--idle-minutes" => { - idle_minutes = Some( - args.next() - .ok_or("--idle-minutes requires a value")? - .parse::() - .map_err(|e| format!("invalid idle minutes: {e}"))?, - ); - } - "--only" => only_item = Some(args.next().ok_or("--only requires an item ID")?), - _ => return Err(format!("unknown argument: {arg}")), - } - } - - if idle_minutes.is_some() && only_item.is_none() { - return Err("--idle-minutes requires --only so every clip is first-after-idle".into()); - } - if let Some(ref requested) = only_item { - if !CORPUS.iter().any(|item| item.id == requested) { - return Err(format!("unknown corpus item for --only: {requested}")); - } - } - - validate_model_dir(&int8_dir, Precision::Int8)?; - validate_model_dir(&fp32_dir, Precision::Fp32)?; - fs::create_dir_all(&output_dir).map_err(|e| e.to_string())?; - - let mut engines = Vec::with_capacity(CONDITIONS.len()); - for condition in CONDITIONS { - let dir = match condition.precision { - Precision::Int8 => &int8_dir, - Precision::Fp32 => &fp32_dir, - }; - let engine = load_engine(dir, condition.precision)?; - // Production warms once before serving a real utterance. Cold cases use - // separate fresh engines below and deliberately skip this call. - synth_chunks(&engine, &["warmup".to_string()])?; - engines.push(engine); - } - - if let Some(minutes) = idle_minutes { - eprintln!("All four warmed engines idle for {minutes} minute(s)…"); - std::thread::sleep(Duration::from_secs(minutes * 60)); - } - - let mut key_items = Vec::new(); - for item in CORPUS { - if only_item - .as_deref() - .is_some_and(|requested| requested != item.id) - { - continue; - } - let preprocessed = preprocess_for_tts(item.text); - let per_sentence: Vec = split_sentences(&preprocessed) - .into_iter() - .filter(|s| !s.trim().is_empty()) - .collect(); - // These corpus texts are deliberately below the upstream ~50-token - // grouping target, so grouped mode is one exact generate() call. - let grouped = vec![per_sentence.join(" ")]; - let item_dir = output_dir.join(item.id); - fs::create_dir_all(&item_dir).map_err(|e| e.to_string())?; - let clip_order = blinded_order(item.id); - let mut clips = Vec::new(); - - let mut rendered = Vec::new(); - for (condition_index, engine) in engines.iter().enumerate() { - let condition = CONDITIONS[condition_index]; - let chunks = match condition.chunking { - Chunking::PerSentence => &per_sentence, - Chunking::Grouped => &grouped, - }; - let started = Instant::now(); - let audio = synth_chunks(engine, chunks)?; - rendered.push(( - condition_index, - condition, - audio, - started.elapsed().as_millis(), - )); - } - loudness_match_item(&mut rendered); - for (condition_index, condition, audio, synth_ms) in rendered { - let clip_number = clip_order[condition_index] + 1; - let file_name = format!("clip{clip_number}.wav"); - write_wav(&item_dir.join(&file_name), &audio)?; - clips.push(KeyClip { - file: format!("{}/{file_name}", item.id), - precision: condition.precision, - chunking: condition.chunking, - cold_start: false, - idle_minutes, - synthesis_ms: synth_ms, - audio_seconds: audio.len() as f32 / SAMPLE_RATE as f32, - }); - } - clips.sort_by(|a, b| a.file.cmp(&b.file)); - key_items.push(KeyItem { - id: item.id.to_string(), - kind: item.kind.to_string(), - text: item.text.to_string(), - clips, - }); - } - - // Explicit fresh-engine cold-start clips for the two highest-signal texts. - // Idle runs intentionally omit them: they happen after the post-idle clips - // and add no valid idle observation. - for item in if idle_minutes.is_none() { CORPUS } else { &[] } { - if !matches!(item.id, "short_one_word" | "multi_relay_review") { - continue; - } - if only_item - .as_deref() - .is_some_and(|requested| requested != item.id) - { - continue; - } - let cold_id = format!("cold_{}", item.id); - let preprocessed = preprocess_for_tts(item.text); - let sentences: Vec = split_sentences(&preprocessed) - .into_iter() - .filter(|s| !s.trim().is_empty()) - .collect(); - let grouped = vec![sentences.join(" ")]; - let item_dir = output_dir.join(&cold_id); - fs::create_dir_all(&item_dir).map_err(|e| e.to_string())?; - let clip_order = blinded_order(&cold_id); - let mut clips = Vec::new(); - let mut rendered = Vec::new(); - for (condition_index, condition) in CONDITIONS.iter().copied().enumerate() { - let dir = match condition.precision { - Precision::Int8 => &int8_dir, - Precision::Fp32 => &fp32_dir, - }; - let engine = load_engine(dir, condition.precision)?; - let chunks = match condition.chunking { - Chunking::PerSentence => &sentences, - Chunking::Grouped => &grouped, - }; - let started = Instant::now(); - let audio = synth_chunks(&engine, chunks)?; - rendered.push(( - condition_index, - condition, - audio, - started.elapsed().as_millis(), - )); - } - loudness_match_item(&mut rendered); - for (condition_index, condition, audio, synth_ms) in rendered { - let clip_number = clip_order[condition_index] + 1; - let file_name = format!("clip{clip_number}.wav"); - write_wav(&item_dir.join(&file_name), &audio)?; - clips.push(KeyClip { - file: format!("{cold_id}/{file_name}"), - precision: condition.precision, - chunking: condition.chunking, - cold_start: true, - idle_minutes: None, - synthesis_ms: synth_ms, - audio_seconds: audio.len() as f32 / SAMPLE_RATE as f32, - }); - } - clips.sort_by(|a, b| a.file.cmp(&b.file)); - key_items.push(KeyItem { - id: cold_id, - kind: "cold-start".to_string(), - text: item.text.to_string(), - clips, - }); - } - - let key = KeyFile { - warning: "DO NOT OPEN UNTIL LISTENING SCORES ARE FINAL", - blinding_seed: BLINDING_SEED, - target_rms_dbfs: TARGET_RMS_DBFS, - items: key_items, - }; - fs::write( - output_dir.join("key.json"), - serde_json::to_vec_pretty(&key).map_err(|e| e.to_string())?, - ) - .map_err(|e| e.to_string())?; - write_scoring_sheet(&output_dir, &key)?; - println!("Wrote blind corpus to {}", output_dir.display()); - println!("Give listeners the WAV folders and SCORING.md; withhold key.json."); - Ok(()) -} - -fn required_path(value: Option, label: &str) -> Result { - value - .map(PathBuf::from) - .ok_or_else(|| format!("missing {label}")) -} - -fn model_file(precision: Precision, base: &str) -> String { - match precision { - Precision::Int8 => format!("{base}.int8.onnx"), - Precision::Fp32 => format!("{base}.onnx"), - } -} - -fn validate_model_dir(dir: &Path, precision: Precision) -> Result<(), String> { - for file in [ - model_file(precision, "lm_main"), - model_file(precision, "lm_flow"), - "encoder.onnx".into(), - model_file(precision, "decoder"), - "text_conditioner.onnx".into(), - "vocab.json".into(), - "token_scores.json".into(), - "reference_sample.wav".into(), - ] { - if !dir.join(&file).is_file() { - return Err(format!("missing {}", dir.join(file).display())); - } - } - Ok(()) -} - -fn load_engine(dir: &Path, precision: Precision) -> Result { - let p = |name: &str| dir.join(name).to_string_lossy().into_owned(); - let mut cfg = OfflineTtsConfig::default(); - cfg.model.pocket.lm_main = Some(p(&model_file(precision, "lm_main"))); - cfg.model.pocket.lm_flow = Some(p(&model_file(precision, "lm_flow"))); - cfg.model.pocket.encoder = Some(p("encoder.onnx")); - cfg.model.pocket.decoder = Some(p(&model_file(precision, "decoder"))); - cfg.model.pocket.text_conditioner = Some(p("text_conditioner.onnx")); - cfg.model.pocket.vocab_json = Some(p("vocab.json")); - cfg.model.pocket.token_scores_json = Some(p("token_scores.json")); - cfg.model.pocket.voice_embedding_cache_capacity = 16; - cfg.model.num_threads = 1; - cfg.model.debug = false; - let inner = - OfflineTts::create(&cfg).ok_or_else(|| format!("failed to create {precision:?} engine"))?; - let wave = - Wave::read(&p("reference_sample.wav")).ok_or("failed to read reference_sample.wav")?; - Ok(Engine { - inner, - voice: Voice { - samples: wave.samples().to_vec(), - sample_rate: wave.sample_rate(), - }, - }) -} - -fn synth_chunks(engine: &Engine, chunks: &[String]) -> Result, String> { - let mut out = Vec::new(); - for chunk in chunks { - let prepared = prepare_pocket_prompt(chunk).ok_or("empty prepared prompt")?; - let extra = prepared.max_frames.map(|max_frames| { - HashMap::from([( - "max_frames".to_string(), - serde_json::Value::from(max_frames), - )]) - }); - let cfg = GenerationConfig { - num_steps: NUM_STEPS, - silence_scale: SILENCE_SCALE, - reference_audio: Some(engine.voice.samples.clone()), - reference_sample_rate: engine.voice.sample_rate, - extra, - ..Default::default() - }; - let audio = engine - .inner - .generate_with_config(&prepared.text, &cfg, None:: bool>) - .ok_or_else(|| format!("synthesis failed for {chunk:?}"))?; - let mut samples: Vec = audio.samples().iter().map(|s| s.clamp(-1.0, 1.0)).collect(); - apply_fade_out(&mut samples); - out.extend(std::iter::repeat_n(0.0, LEAD_IN_SAMPLES)); - out.extend(samples); - out.extend(std::iter::repeat_n( - 0.0, - INTER_SENTENCE_SILENCE_SAMPLES - LEAD_IN_SAMPLES, - )); - } - Ok(out) -} - -fn apply_fade_out(samples: &mut [f32]) { - let fade = FADE_OUT_SAMPLES.min(samples.len() / 2); - for i in 0..fade { - samples[samples.len() - 1 - i] *= i as f32 / fade as f32; - } -} - -fn active_rms(samples: &[f32]) -> Option { - let (sum_squares, count) = samples - .iter() - .filter(|sample| sample.abs() > 1.0e-4) - .fold((0.0_f32, 0_usize), |(sum, count), sample| { - (sum + sample * sample, count + 1) - }); - (count > 0).then(|| (sum_squares / count as f32).sqrt()) -} - -/// Attenuate every clip in one comparison set to the quietest active-speech RMS. -/// This removes the louder-is-better confound without normalizing dynamics or -/// claiming standards-compliant integrated LUFS. The dBFS value is a ceiling. -fn loudness_match_item(rendered: &mut [(usize, Condition, Vec, u128)]) { - let ceiling = 10.0_f32.powf(TARGET_RMS_DBFS / 20.0); - let target = rendered - .iter() - .filter_map(|(_, _, samples, _)| active_rms(samples)) - .fold(ceiling, f32::min); - for (_, _, samples, _) in rendered { - let Some(rms) = active_rms(samples) else { - continue; - }; - let gain = (target / rms).min(1.0); - for sample in samples { - *sample *= gain; - } - } -} - -fn blinded_order(item_id: &str) -> [usize; 4] { - let mut keyed: Vec<(usize, Vec)> = (0..4) - .map(|index| { - let digest = Sha256::digest(format!("{BLINDING_SEED}:{item_id}:{index}")); - (index, digest.to_vec()) - }) - .collect(); - keyed.sort_by(|a, b| a.1.cmp(&b.1)); - let mut condition_to_clip = [0; 4]; - for (clip, (condition, _)) in keyed.into_iter().enumerate() { - condition_to_clip[condition] = clip; - } - condition_to_clip -} - -fn write_wav(path: &Path, samples: &[f32]) -> Result<(), String> { - let path = path - .to_str() - .ok_or_else(|| format!("non-UTF8 path: {}", path.display()))?; - if sherpa_onnx::write(path, samples, SAMPLE_RATE as i32) { - Ok(()) - } else { - Err(format!("failed to write {path}")) - } -} - -fn write_scoring_sheet(output_dir: &Path, key: &KeyFile) -> Result<(), String> { - let mut sheet = String::from("# Pocket TTS blind listening sheet\n\nDo not open `key.json` until this sheet is complete. Rank best to worst; ties are allowed.\n\n"); - for item in &key.items { - sheet.push_str(&format!( - "## {} ({})\n\n> {}\n\n", - item.id, item.kind, item.text - )); - sheet.push_str("Rank: `____ > ____ > ____ > ____`\n\n| Clip | seam | onset | garble | robotic | timbre | truncate | note |\n|---|---|---|---|---|---|---|---|\n"); - for clip in 1..=4 { - sheet.push_str(&format!( - "| clip{clip} | [ ] | [ ] | [ ] | [ ] | [ ] | [ ] | |\n" - )); - } - sheet.push('\n'); - } - fs::write(output_dir.join("SCORING.md"), sheet).map_err(|e| e.to_string()) -} diff --git a/desktop/src-tauri/resources/pocket-voices/NOTICE.md b/desktop/src-tauri/resources/pocket-voices/NOTICE.md new file mode 100644 index 0000000000..9cc515dea3 --- /dev/null +++ b/desktop/src-tauri/resources/pocket-voices/NOTICE.md @@ -0,0 +1,35 @@ +# Pocket TTS English VCTK presets + +Buzz exposes Kyutai's twelve official English VCTK Pocket presets. The WAV +bytes are unchanged from `kyutai/tts-voices` revision +`323332d33f997de8394f24a193e1a76df720e01a`; only local filenames differ. + +| Voice | Upstream asset | SHA-256 | +| --- | --- | --- | +| Anna | `vctk/p228_023_enhanced.wav` | `0a6de25cf12bf1540beb85979f306a92be81fecc051c547c5395e7e5237a3856` | +| Vera | `vctk/p229_023_enhanced.wav` | `309cf91a895830f15842b398f69a4962cb1f7e0bfab10e25dd27838e826c204b` | +| Fantine | `vctk/p244_023_enhanced.wav` | `5f07d4e2a3f20a15572aae885156b43ef3fc12ef3812996fd135680d9956448b` | +| Charles | `vctk/p254_023_enhanced.wav` | `6b681a429198f16e378d53bccb08d06939da7b00144a7696111d4f8f76be7756` | +| Paul | `vctk/p259_023_enhanced.wav` | `7aba504fe0b3b16478b69eb27ce6007e3cb42b0c1915b5f1c6a6024ae37d679b` | +| Eponine | `vctk/p262_023_enhanced.wav` | `a13c27fb47627b05223691a0ef2974358a18c886e6c2f9d2762ff1d02c20926b` | +| Azelma | `vctk/p303_023_enhanced.wav` | `60e3d26cdf2efdec5df712152c839928f4d5522821e6554ae11fd96c57ab1026` | +| George | `vctk/p315_023_enhanced.wav` | `29a41f93bf5236e5b21501091d7774c255d5f3d4e62fa4f9fdf0a92a793c84ae` | +| Mary | `vctk/p333_023_enhanced.wav` | `a35b0468382218e9f37a9a7494d1e4b74deaf18d7ced22265b4e325bb55c183f` | +| Jane | `vctk/p339_023_enhanced.wav` | `2f12e7f155eb3118f55425394f1b049e5b1b67bdc9b3932c8ba4521420aeb84a` | +| Michael | `vctk/p360_023_enhanced.wav` | `b6743e9195e5e3fd34fe9d1633ae93f7ffab787b249e45f6467d7d6f7a6ee6ad` | +| Eve | `vctk/p361_023_enhanced.wav` | `396e7cbd066b0f3fb6d67fa26e7904076958239d736d4390f15b5fe88feb14cd` | + +Mary is already installed as the Pocket model's `reference_sample.wav`, so it +is not duplicated in this resource directory. + +Source repository: +https://huggingface.co/kyutai/tts-voices/tree/323332d33f997de8394f24a193e1a76df720e01a/vctk + +The original recordings are from the Voice Cloning Toolkit (VCTK) corpus, +licensed CC BY 4.0: +https://datashare.ed.ac.uk/handle/10283/3443 + +The recordings were enhanced by ai-coustics: +https://ai-coustics.com/ + +Neither Kyutai, the VCTK speakers, nor ai-coustics endorses Buzz. diff --git a/desktop/src-tauri/resources/pocket-voices/anna.wav b/desktop/src-tauri/resources/pocket-voices/anna.wav new file mode 100644 index 0000000000..79d60697ff Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/anna.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/azelma.wav b/desktop/src-tauri/resources/pocket-voices/azelma.wav new file mode 100644 index 0000000000..e9d0c00b3f Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/azelma.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/charles.wav b/desktop/src-tauri/resources/pocket-voices/charles.wav new file mode 100644 index 0000000000..2170975545 Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/charles.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/eponine.wav b/desktop/src-tauri/resources/pocket-voices/eponine.wav new file mode 100644 index 0000000000..bded6f4f09 Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/eponine.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/eve.wav b/desktop/src-tauri/resources/pocket-voices/eve.wav new file mode 100644 index 0000000000..216665ff13 Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/eve.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/fantine.wav b/desktop/src-tauri/resources/pocket-voices/fantine.wav new file mode 100644 index 0000000000..28c2b1140d Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/fantine.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/george.wav b/desktop/src-tauri/resources/pocket-voices/george.wav new file mode 100644 index 0000000000..739d5bc7a5 Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/george.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/jane.wav b/desktop/src-tauri/resources/pocket-voices/jane.wav new file mode 100644 index 0000000000..3c9890473b Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/jane.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/michael.wav b/desktop/src-tauri/resources/pocket-voices/michael.wav new file mode 100644 index 0000000000..861da085c7 Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/michael.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/paul.wav b/desktop/src-tauri/resources/pocket-voices/paul.wav new file mode 100644 index 0000000000..bfde50fdd9 Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/paul.wav differ diff --git a/desktop/src-tauri/resources/pocket-voices/vera.wav b/desktop/src-tauri/resources/pocket-voices/vera.wav new file mode 100644 index 0000000000..e4fce84ce3 Binary files /dev/null and b/desktop/src-tauri/resources/pocket-voices/vera.wav differ diff --git a/desktop/src-tauri/src/app_state.rs b/desktop/src-tauri/src/app_state.rs index 94d162e620..fc90e6ab14 100644 --- a/desktop/src-tauri/src/app_state.rs +++ b/desktop/src-tauri/src/app_state.rs @@ -2,7 +2,7 @@ use std::{ collections::HashMap, io::Write, sync::{ - atomic::{AtomicBool, AtomicU16}, + atomic::{AtomicBool, AtomicU16, AtomicU8}, Arc, Mutex, }, }; @@ -13,10 +13,15 @@ use tauri::{AppHandle, Manager}; use tokio::sync::Mutex as AsyncMutex; use crate::huddle::HuddleState; +pub(crate) use crate::identity_storage::{IdentityStorage, RecoveryState, ResolvedIdentity}; use crate::managed_agents::config_bridge::SessionConfigCache; use crate::managed_agents::{ManagedAgentPairRuntime, ManagedAgentRuntimeKey}; + pub struct AppState { pub keys: Mutex, + /// Durable backend holding `keys`. Updated after the key write and before + /// recovery flags are cleared so `get_identity` reports a consistent state. + pub(crate) identity_storage: AtomicU8, pub http_client: reqwest::Client, /// A no-redirect client for authenticated relay media fetches (download, /// clipboard copy, snapshot, editor). Every caller pre-validates the URL @@ -48,15 +53,13 @@ pub struct AppState { pub channel_templates_store_lock: Mutex<()>, pub managed_agent_processes: Mutex>, pub huddle_state: Mutex, + pub huddle_audio: crate::huddle::tts_settings::HuddleAudioSettingsState, /// Tauri app handle — stored after setup so huddle commands can emit /// `huddle-state-changed` events without needing the handle threaded /// through every call site. /// /// Set once during `setup()` in `lib.rs`; never cleared. pub app_handle: Mutex>, - /// Selected audio output device name. `None` = system default. - /// Used by `connect_audio_relay` and TTS pipeline when opening sinks. - pub audio_output_device: Mutex>, /// Port of the localhost media streaming proxy (set during setup). pub media_proxy_port: AtomicU16, /// Set when identity resolution detected a "keyring-locked" state: the @@ -178,19 +181,20 @@ pub fn build_media_fetch_client() -> reqwest::Result { pub fn build_app_state() -> AppState { // Env var takes precedence (dev/CI). If absent, resolve_persisted_identity() // in setup() will replace the ephemeral placeholder with a persisted key. - let keys = match identity_from_env() { + let (keys, identity_storage) = match identity_from_env() { Some(keys) => { eprintln!( "buzz-desktop: configured identity pubkey {}", keys.public_key().to_hex() ); - keys + (keys, IdentityStorage::Environment) } - None => Keys::generate(), + None => (Keys::generate(), IdentityStorage::Ephemeral), }; AppState { keys: Mutex::new(keys), + identity_storage: AtomicU8::new(identity_storage as u8), http_client: reqwest::Client::builder() .resolve("localhost", std::net::SocketAddr::from(([127, 0, 0, 1], 0))) .pool_idle_timeout(std::time::Duration::from_secs(10)) @@ -213,8 +217,8 @@ pub fn build_app_state() -> AppState { managed_agent_processes: Mutex::new(HashMap::new()), session_config_cache: Mutex::new(HashMap::new()), huddle_state: Mutex::new(HuddleState::default()), + huddle_audio: Default::default(), app_handle: Mutex::new(None), - audio_output_device: Mutex::new(None), media_proxy_port: AtomicU16::new(0), prevent_sleep: Arc::new(Mutex::new( crate::prevent_sleep::PreventSleepState::default(), @@ -366,9 +370,13 @@ pub fn resolve_persisted_identity(app: &AppHandle, state: &AppState) -> Result<( std::fs::create_dir_all(&data_dir).map_err(|e| format!("create app data dir: {e}"))?; let resolved = load_or_create_identity(&data_dir)?; - // Write keys before setting the recovery flags (Release) so any thread - // that reads a flag as false with Acquire is guaranteed to see the keys. - *state.keys.lock().map_err(|e| e.to_string())? = resolved.keys; + // Write keys and storage before setting the recovery flags (Release) so + // any thread that reads a flag as false with Acquire sees consistent data. + { + let mut active_keys = state.keys.lock().map_err(|e| e.to_string())?; + *active_keys = resolved.keys; + state.set_identity_storage(resolved.storage); + } state.identity_lost.store( resolved.recovery == RecoveryState::Lost, std::sync::atomic::Ordering::Release, @@ -394,26 +402,6 @@ const IDENTITY_KEY_NAME: &str = "identity"; /// keyring is merely unreachable (the key IS in the keyring, must NOT generate). const MIGRATION_MARKER_NAME: &str = "identity.migrated"; -/// Recovery state produced by identity resolution. `None` means the app has -/// a real, usable identity. `Lost` means the keyring was reachable-but-empty -/// despite a prior successful migration — the key vanished externally. `KeyringLocked` -/// means the keyring is unreachable this boot but was used in the past -/// (marker present, no file) — the key still exists but is temporarily -/// inaccessible. Both non-`None` variants boot with an ephemeral key; the -/// frontend shows a different recovery screen for each. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -enum RecoveryState { - None, - Lost, - KeyringLocked, -} - -/// The output of identity resolution. -struct ResolvedIdentity { - keys: Keys, - recovery: RecoveryState, -} - /// The keyring operations the identity resolution flow needs. Abstracted so the /// corrupt-keyring recovery decision ([`recover_from_keyring`]) can be /// unit-tested against a fake without touching the live OS keyring. @@ -465,6 +453,7 @@ fn load_or_create_identity(data_dir: &std::path::Path) -> Result Result<(), String> { +) -> Result { match persist_identity_to_keyring(store, keys, legacy_path, data_dir) { - Ok(()) => Ok(()), + Ok(()) => Ok(IdentityStorage::SystemKeyring), Err(e) => { eprintln!( "buzz-desktop: keyring write failed during import ({e}), \ falling back to identity.key" ); - save_key_file(legacy_path, keys) + save_key_file(legacy_path, keys)?; + Ok(IdentityStorage::LocalFile) } } } @@ -892,7 +895,7 @@ pub(crate) fn persist_imported_identity( keys: &Keys, legacy_path: &std::path::Path, data_dir: &std::path::Path, -) -> Result<(), String> { +) -> Result { persist_imported_identity_impl(store, keys, legacy_path, data_dir) } @@ -920,15 +923,6 @@ fn write_migration_marker(marker_path: &std::path::Path) -> Result<(), String> { .map_err(|e| format!("commit migration marker: {e}")) } -/// Which backend [`store_key_preferring_keyring`] wrote to. The caller writes -/// the migration marker only after a keyring success — on the file-fallback arm -/// the key is on disk and a marker would wrongly trip the next Unreachable boot -/// into failing closed. -enum PersistBackend { - Keyring, - File, -} - /// Generate a fresh identity, persist it through the store, return it. /// /// On a keyring-backed persist no file is written, so a later @@ -940,9 +934,10 @@ fn generate_and_persist( store: &impl IdentityKeyStore, legacy_path: &std::path::Path, data_dir: &std::path::Path, -) -> Result { +) -> Result<(Keys, IdentityStorage), String> { let keys = Keys::generate(); - if let PersistBackend::Keyring = store_key_preferring_keyring(store, &keys, legacy_path)? { + let storage = store_key_preferring_keyring(store, &keys, legacy_path)?; + if storage == IdentityStorage::SystemKeyring { let marker_path = migration_marker_path(data_dir); if let Err(e) = write_migration_marker(&marker_path) { eprintln!( @@ -956,7 +951,7 @@ fn generate_and_persist( "buzz-desktop: generated and saved identity pubkey {}", keys.public_key().to_hex() ); - Ok(keys) + Ok((keys, storage)) } /// Persist `keys` through the store, silently falling back to the `0o600` file @@ -968,17 +963,17 @@ fn store_key_preferring_keyring( store: &impl IdentityKeyStore, keys: &Keys, legacy_path: &std::path::Path, -) -> Result { +) -> Result { let nsec = keys .secret_key() .to_bech32() .map_err(|e| format!("encode nsec: {e}"))?; match store.store(IDENTITY_KEY_NAME, &nsec) { - Ok(()) => Ok(PersistBackend::Keyring), + Ok(()) => Ok(IdentityStorage::SystemKeyring), Err(keyring_err) => { eprintln!("buzz-desktop: keyring write failed ({keyring_err}), using file fallback"); save_key_file(legacy_path, keys)?; - Ok(PersistBackend::File) + Ok(IdentityStorage::LocalFile) } } } diff --git a/desktop/src-tauri/src/app_state_tests.rs b/desktop/src-tauri/src/app_state_tests.rs index 485dfaea15..751bcf22e5 100644 --- a/desktop/src-tauri/src/app_state_tests.rs +++ b/desktop/src-tauri/src/app_state_tests.rs @@ -484,7 +484,7 @@ fn fresh_keyring_generate_writes_marker() { let resolved = resolve_identity_with_store(&store, &legacy_path, dir.path()).unwrap(); // The key was stored in the keyring (not the file), and the marker marks it. - assert!(!legacy_path.exists()); + assert!(!legacy_path.exists() && resolved.storage == IdentityStorage::SystemKeyring); assert!(migration_marker_path(dir.path()).exists()); assert_eq!( store @@ -541,7 +541,10 @@ fn fresh_generate_keyring_failure_falls_back_to_file_without_marker() { let from_file = load_key_file(&legacy_path).unwrap(); assert_key_eq(&resolved.keys, &from_file); // No marker: the file is the authoritative store, not the keyring. - assert!(!migration_marker_path(dir.path()).exists()); + assert!( + !migration_marker_path(dir.path()).exists() + && resolved.storage == IdentityStorage::LocalFile + ); } // ── New tests for the three defects fixed in this PR ───────────────────── @@ -786,10 +789,7 @@ fn persist_imported_identity_falls_back_to_file_on_keyring_failure() { let result = persist_imported_identity_impl(&store, &imported_keys, &legacy_path, dir.path()); // The policy core handles the keyring failure — Ok, not Err. - assert!( - result.is_ok(), - "must not propagate keyring failure when file fallback succeeds" - ); + assert_eq!(result.unwrap(), IdentityStorage::LocalFile); // Key is recoverable from the file on next boot. let from_file = load_key_file(&legacy_path).unwrap(); diff --git a/desktop/src-tauri/src/archive/mod.rs b/desktop/src-tauri/src/archive/mod.rs index a65b126a4d..42c6812674 100644 --- a/desktop/src-tauri/src/archive/mod.rs +++ b/desktop/src-tauri/src/archive/mod.rs @@ -483,21 +483,23 @@ pub async fn list_save_subscriptions( /// Does NOT purge already-archived event data — retention is decoupled in v1. /// GC of orphaned event rows happens in P4 purge commands, not here. #[tauri::command] -pub fn delete_save_subscription( +pub async fn delete_save_subscription( state: State<'_, AppState>, scope_type: ScopeType, scope_value: String, ) -> Result { let identity_pk = identity_pubkey(&state)?; let relay_url = relay_ws_url_with_override(&state); - let conn = open_db()?; - store::delete_save_subscription( - &conn, - &identity_pk, - &relay_url, - scope_type.as_str(), - &scope_value, - ) + run_archive_db_task(move |conn| { + store::delete_save_subscription( + conn, + &identity_pk, + &relay_url, + scope_type.as_str(), + &scope_value, + ) + }) + .await } // ── read_archived_events ───────────────────────────────────────────────────── @@ -516,7 +518,7 @@ pub fn delete_save_subscription( /// newest-first order. Compound cursor `(before_created_at, before_id)` works /// identically to `read_archived_events`. #[tauri::command] -pub fn read_archived_observer_events_for_channel( +pub async fn read_archived_observer_events_for_channel( state: State<'_, AppState>, channel_id: String, before_created_at: Option, @@ -525,16 +527,18 @@ pub fn read_archived_observer_events_for_channel( ) -> Result, String> { let identity_pk = identity_pubkey(&state)?; let relay_url = relay_ws_url_with_override(&state); - let conn = open_db()?; - store::read_archived_observer_events_for_channel( - &conn, - &identity_pk, - &relay_url, - &channel_id, - before_created_at, - before_id.as_deref(), - limit.unwrap_or(DEFAULT_READ_LIMIT), - ) + run_archive_db_task(move |conn| { + store::read_archived_observer_events_for_channel( + conn, + &identity_pk, + &relay_url, + &channel_id, + before_created_at, + before_id.as_deref(), + limit.unwrap_or(DEFAULT_READ_LIMIT), + ) + }) + .await } // ── index_observer_channel_id ───────────────────────────────────────────────── @@ -548,24 +552,26 @@ pub fn read_archived_observer_events_for_channel( /// /// Idempotent: rows that are already indexed are left unchanged. #[tauri::command] -pub fn index_observer_channel_id( +pub async fn index_observer_channel_id( state: State<'_, AppState>, entries: Vec, ) -> Result<(), String> { let identity_pk = identity_pubkey(&state)?; let relay_url = relay_ws_url_with_override(&state); - let conn = open_db()?; - for entry in &entries { - store::upsert_observer_channel_index( - &conn, - &identity_pk, - &relay_url, - &entry.event_id, - entry.channel_id.as_deref(), - entry.created_at, - )?; - } - Ok(()) + run_archive_db_task(move |conn| { + for entry in &entries { + store::upsert_observer_channel_index( + conn, + &identity_pk, + &relay_url, + &entry.event_id, + entry.channel_id.as_deref(), + entry.created_at, + )?; + } + Ok(()) + }) + .await } /// A single (event_id, channel_id?, created_at) record used by @@ -591,21 +597,23 @@ pub struct ObserverChannelIndexEntry { /// Together these constitute the one-shot idempotent backfill required by the /// Slice 1 acceptance criteria (Thufir Pass 4). #[tauri::command] -pub fn read_unindexed_observer_rows( +pub async fn read_unindexed_observer_rows( state: State<'_, AppState>, ) -> Result, String> { let identity_pk = identity_pubkey(&state)?; let relay_url = relay_ws_url_with_override(&state); - let conn = open_db()?; - let rows = store::read_unindexed_observer_rows(&conn, &identity_pk, &relay_url)?; - Ok(rows - .into_iter() - .map(|(id, raw_json, created_at)| RawObserverRow { - id, - raw_json, - created_at, - }) - .collect()) + run_archive_db_task(move |conn| { + let rows = store::read_unindexed_observer_rows(conn, &identity_pk, &relay_url)?; + Ok(rows + .into_iter() + .map(|(id, raw_json, created_at)| RawObserverRow { + id, + raw_json, + created_at, + }) + .collect()) + }) + .await } /// Wire type returned by `read_unindexed_observer_rows`. diff --git a/desktop/src-tauri/src/assets/eff_short_wordlist_2_0.txt b/desktop/src-tauri/src/assets/eff_short_wordlist_2_0.txt new file mode 100644 index 0000000000..9ac732fe36 --- /dev/null +++ b/desktop/src-tauri/src/assets/eff_short_wordlist_2_0.txt @@ -0,0 +1,1296 @@ +aardvark +abandoned +abbreviate +abdomen +abhorrence +abiding +abnormal +abrasion +absorbing +abundant +abyss +academy +accountant +acetone +achiness +acid +acoustics +acquire +acrobat +actress +acuteness +aerosol +aesthetic +affidavit +afloat +afraid +aftershave +again +agency +aggressor +aghast +agitate +agnostic +agonizing +agreeing +aidless +aimlessly +ajar +alarmclock +albatross +alchemy +alfalfa +algae +aliens +alkaline +almanac +alongside +alphabet +already +also +altitude +aluminum +always +amazingly +ambulance +amendment +amiable +ammunition +amnesty +amoeba +amplifier +amuser +anagram +anchor +android +anesthesia +angelfish +animal +anklet +announcer +anonymous +answer +antelope +anxiety +anyplace +aorta +apartment +apnea +apostrophe +apple +apricot +aquamarine +arachnid +arbitrate +ardently +arena +argument +aristocrat +armchair +aromatic +arrowhead +arsonist +artichoke +asbestos +ascend +aseptic +ashamed +asinine +asleep +asocial +asparagus +astronaut +asymmetric +atlas +atmosphere +atom +atrocious +attic +atypical +auctioneer +auditorium +augmented +auspicious +automobile +auxiliary +avalanche +avenue +aviator +avocado +awareness +awhile +awkward +awning +awoke +axially +azalea +babbling +backpack +badass +bagpipe +bakery +balancing +bamboo +banana +barracuda +basket +bathrobe +bazooka +blade +blender +blimp +blouse +blurred +boatyard +bobcat +body +bogusness +bohemian +boiler +bonnet +boots +borough +bossiness +bottle +bouquet +boxlike +breath +briefcase +broom +brushes +bubblegum +buckle +buddhist +buffalo +bullfrog +bunny +busboy +buzzard +cabin +cactus +cadillac +cafeteria +cage +cahoots +cajoling +cakewalk +calculator +camera +canister +capsule +carrot +cashew +cathedral +caucasian +caviar +ceasefire +cedar +celery +cement +census +ceramics +cesspool +chalkboard +cheesecake +chimney +chlorine +chopsticks +chrome +chute +cilantro +cinnamon +circle +cityscape +civilian +clay +clergyman +clipboard +clock +clubhouse +coathanger +cobweb +coconut +codeword +coexistent +coffeecake +cognitive +cohabitate +collarbone +computer +confetti +copier +cornea +cosmetics +cotton +couch +coverless +coyote +coziness +crawfish +crewmember +crib +croissant +crumble +crystal +cubical +cucumber +cuddly +cufflink +cuisine +culprit +cup +curry +cushion +cuticle +cybernetic +cyclist +cylinder +cymbal +cynicism +cypress +cytoplasm +dachshund +daffodil +dagger +dairy +dalmatian +dandelion +dartboard +dastardly +datebook +daughter +dawn +daytime +dazzler +dealer +debris +decal +dedicate +deepness +defrost +degree +dehydrator +deliverer +democrat +dentist +deodorant +depot +deranged +desktop +detergent +device +dexterity +diamond +dibs +dictionary +diffuser +digit +dilated +dimple +dinnerware +dioxide +diploma +directory +dishcloth +ditto +dividers +dizziness +doctor +dodge +doll +dominoes +donut +doorstep +dorsal +double +downstairs +dozed +drainpipe +dresser +driftwood +droppings +drum +dryer +dubiously +duckling +duffel +dugout +dumpster +duplex +durable +dustpan +dutiful +duvet +dwarfism +dwelling +dwindling +dynamite +dyslexia +eagerness +earlobe +easel +eavesdrop +ebook +eccentric +echoless +eclipse +ecosystem +ecstasy +edged +editor +educator +eelworm +eerie +effects +eggnog +egomaniac +ejection +elastic +elbow +elderly +elephant +elfishly +eliminator +elk +elliptical +elongated +elsewhere +elusive +elves +emancipate +embroidery +emcee +emerald +emission +emoticon +emperor +emulate +enactment +enchilada +endorphin +energy +enforcer +engine +enhance +enigmatic +enjoyably +enlarged +enormous +enquirer +enrollment +ensemble +entryway +enunciate +envoy +enzyme +epidemic +equipment +erasable +ergonomic +erratic +eruption +escalator +eskimo +esophagus +espresso +essay +estrogen +etching +eternal +ethics +etiquette +eucalyptus +eulogy +euphemism +euthanize +evacuation +evergreen +evidence +evolution +exam +excerpt +exerciser +exfoliate +exhale +exist +exorcist +explode +exquisite +exterior +exuberant +fabric +factory +faded +failsafe +falcon +family +fanfare +fasten +faucet +favorite +feasibly +february +federal +feedback +feigned +feline +femur +fence +ferret +festival +fettuccine +feudalist +feverish +fiberglass +fictitious +fiddle +figurine +fillet +finalist +fiscally +fixture +flashlight +fleshiness +flight +florist +flypaper +foamless +focus +foggy +folksong +fondue +footpath +fossil +fountain +fox +fragment +freeway +fridge +frosting +fruit +fryingpan +gadget +gainfully +gallstone +gamekeeper +gangway +garlic +gaslight +gathering +gauntlet +gearbox +gecko +gem +generator +geographer +gerbil +gesture +getaway +geyser +ghoulishly +gibberish +giddiness +giftshop +gigabyte +gimmick +giraffe +giveaway +gizmo +glasses +gleeful +glisten +glove +glucose +glycerin +gnarly +gnomish +goatskin +goggles +goldfish +gong +gooey +gorgeous +gosling +gothic +gourmet +governor +grape +greyhound +grill +groundhog +grumbling +guacamole +guerrilla +guitar +gullible +gumdrop +gurgling +gusto +gutless +gymnast +gynecology +gyration +habitat +hacking +haggard +haiku +halogen +hamburger +handgun +happiness +hardhat +hastily +hatchling +haughty +hazelnut +headband +hedgehog +hefty +heinously +helmet +hemoglobin +henceforth +herbs +hesitation +hexagon +hubcap +huddling +huff +hugeness +hullabaloo +human +hunter +hurricane +hushing +hyacinth +hybrid +hydrant +hygienist +hypnotist +ibuprofen +icepack +icing +iconic +identical +idiocy +idly +igloo +ignition +iguana +illuminate +imaging +imbecile +imitator +immigrant +imprint +iodine +ionosphere +ipad +iphone +iridescent +irksome +iron +irrigation +island +isotope +issueless +italicize +itemizer +itinerary +itunes +ivory +jabbering +jackrabbit +jaguar +jailhouse +jalapeno +jamboree +janitor +jarring +jasmine +jaundice +jawbreaker +jaywalker +jazz +jealous +jeep +jelly +jeopardize +jersey +jetski +jezebel +jiffy +jigsaw +jingling +jobholder +jockstrap +jogging +john +joinable +jokingly +journal +jovial +joystick +jubilant +judiciary +juggle +juice +jujitsu +jukebox +jumpiness +junkyard +juror +justifying +juvenile +kabob +kamikaze +kangaroo +karate +kayak +keepsake +kennel +kerosene +ketchup +khaki +kickstand +kilogram +kimono +kingdom +kiosk +kissing +kite +kleenex +knapsack +kneecap +knickers +koala +krypton +laboratory +ladder +lakefront +lantern +laptop +laryngitis +lasagna +latch +laundry +lavender +laxative +lazybones +lecturer +leftover +leggings +leisure +lemon +length +leopard +leprechaun +lettuce +leukemia +levers +lewdness +liability +library +licorice +lifeboat +lightbulb +likewise +lilac +limousine +lint +lioness +lipstick +liquid +listless +litter +liverwurst +lizard +llama +luau +lubricant +lucidity +ludicrous +luggage +lukewarm +lullaby +lumberjack +lunchbox +luridness +luscious +luxurious +lyrics +macaroni +maestro +magazine +mahogany +maimed +majority +makeover +malformed +mammal +mango +mapmaker +marbles +massager +matchstick +maverick +maximum +mayonnaise +moaning +mobilize +moccasin +modify +moisture +molecule +momentum +monastery +moonshine +mortuary +mosquito +motorcycle +mousetrap +movie +mower +mozzarella +muckiness +mudflow +mugshot +mule +mummy +mundane +muppet +mural +mustard +mutation +myriad +myspace +myth +nail +namesake +nanosecond +napkin +narrator +nastiness +natives +nautically +navigate +nearest +nebula +nectar +nefarious +negotiator +neither +nemesis +neoliberal +nephew +nervously +nest +netting +neuron +nevermore +nextdoor +nicotine +niece +nimbleness +nintendo +nirvana +nuclear +nugget +nuisance +nullify +numbing +nuptials +nursery +nutcracker +nylon +oasis +oat +obediently +obituary +object +obliterate +obnoxious +observer +obtain +obvious +occupation +oceanic +octopus +ocular +office +oftentimes +oiliness +ointment +older +olympics +omissible +omnivorous +oncoming +onion +onlooker +onstage +onward +onyx +oomph +opaquely +opera +opium +opossum +opponent +optical +opulently +oscillator +osmosis +ostrich +otherwise +ought +outhouse +ovation +oven +owlish +oxford +oxidize +oxygen +oyster +ozone +pacemaker +padlock +pageant +pajamas +palm +pamphlet +pantyhose +paprika +parakeet +passport +patio +pauper +pavement +payphone +pebble +peculiarly +pedometer +pegboard +pelican +penguin +peony +pepperoni +peroxide +pesticide +petroleum +pewter +pharmacy +pheasant +phonebook +phrasing +physician +plank +pledge +plotted +plug +plywood +pneumonia +podiatrist +poetic +pogo +poison +poking +policeman +poncho +popcorn +porcupine +postcard +poultry +powerboat +prairie +pretzel +princess +propeller +prune +pry +pseudo +psychopath +publisher +pucker +pueblo +pulley +pumpkin +punchbowl +puppy +purse +pushup +putt +puzzle +pyramid +python +quarters +quesadilla +quilt +quote +racoon +radish +ragweed +railroad +rampantly +rancidity +rarity +raspberry +ravishing +rearrange +rebuilt +receipt +reentry +refinery +register +rehydrate +reimburse +rejoicing +rekindle +relic +remote +renovator +reopen +reporter +request +rerun +reservoir +retriever +reunion +revolver +rewrite +rhapsody +rhetoric +rhino +rhubarb +rhyme +ribbon +riches +ridden +rigidness +rimmed +riptide +riskily +ritzy +riverboat +roamer +robe +rocket +romancer +ropelike +rotisserie +roundtable +royal +rubber +rudderless +rugby +ruined +rulebook +rummage +running +rupture +rustproof +sabotage +sacrifice +saddlebag +saffron +sainthood +saltshaker +samurai +sandworm +sapphire +sardine +sassy +satchel +sauna +savage +saxophone +scarf +scenario +schoolbook +scientist +scooter +scrapbook +sculpture +scythe +secretary +sedative +segregator +seismology +selected +semicolon +senator +septum +sequence +serpent +sesame +settler +severely +shack +shelf +shirt +shovel +shrimp +shuttle +shyness +siamese +sibling +siesta +silicon +simmering +singles +sisterhood +sitcom +sixfold +sizable +skateboard +skeleton +skies +skulk +skylight +slapping +sled +slingshot +sloth +slumbering +smartphone +smelliness +smitten +smokestack +smudge +snapshot +sneezing +sniff +snowsuit +snugness +speakers +sphinx +spider +splashing +sponge +sprout +spur +spyglass +squirrel +statue +steamboat +stingray +stopwatch +strawberry +student +stylus +suave +subway +suction +suds +suffocate +sugar +suitcase +sulphur +superstore +surfer +sushi +swan +sweatshirt +swimwear +sword +sycamore +syllable +symphony +synagogue +syringes +systemize +tablespoon +taco +tadpole +taekwondo +tagalong +takeout +tallness +tamale +tanned +tapestry +tarantula +tastebud +tattoo +tavern +thaw +theater +thimble +thorn +throat +thumb +thwarting +tiara +tidbit +tiebreaker +tiger +timid +tinsel +tiptoeing +tirade +tissue +tractor +tree +tripod +trousers +trucks +tryout +tubeless +tuesday +tugboat +tulip +tumbleweed +tupperware +turtle +tusk +tutorial +tuxedo +tweezers +twins +tyrannical +ultrasound +umbrella +umpire +unarmored +unbuttoned +uncle +underwear +unevenness +unflavored +ungloved +unhinge +unicycle +unjustly +unknown +unlocking +unmarked +unnoticed +unopened +unpaved +unquenched +unroll +unscrewing +untied +unusual +unveiled +unwrinkled +unyielding +unzip +upbeat +upcountry +update +upfront +upgrade +upholstery +upkeep +upload +uppercut +upright +upstairs +uptown +upwind +uranium +urban +urchin +urethane +urgent +urologist +username +usher +utensil +utility +utmost +utopia +utterance +vacuum +vagrancy +valuables +vanquished +vaporizer +varied +vaseline +vegetable +vehicle +velcro +vendor +vertebrae +vestibule +veteran +vexingly +vicinity +videogame +viewfinder +vigilante +village +vinegar +violin +viperfish +virus +visor +vitamins +vivacious +vixen +vocalist +vogue +voicemail +volleyball +voucher +voyage +vulnerable +waffle +wagon +wakeup +walrus +wanderer +wasp +water +waving +wheat +whisper +wholesaler +wick +widow +wielder +wifeless +wikipedia +wildcat +windmill +wipeout +wired +wishbone +wizardry +wobbliness +wolverine +womb +woolworker +workbasket +wound +wrangle +wreckage +wristwatch +wrongdoing +xerox +xylophone +yacht +yahoo +yard +yearbook +yesterday +yiddish +yield +yo-yo +yodel +yogurt +yuppie +zealot +zebra +zeppelin +zestfully +zigzagged +zillion +zipping +zirconium +zodiac +zombie +zookeeper +zucchini diff --git a/desktop/src-tauri/src/commands/agent_discovery.rs b/desktop/src-tauri/src/commands/agent_discovery.rs index d6429e0454..cbbf4ce351 100644 --- a/desktop/src-tauri/src/commands/agent_discovery.rs +++ b/desktop/src-tauri/src/commands/agent_discovery.rs @@ -236,10 +236,12 @@ pub async fn install_acp_runtime( // returns (Guard impl Drop) — so Phase 2's restart path runs outside // the guard and cannot re-enter the mutex. let runtime_id_clone = runtime_id.clone(); - let install_result = - tokio::task::spawn_blocking(move || install_acp_runtime_blocking(&runtime_id_clone)) - .await - .map_err(|e| format!("install task panicked: {e}"))??; + let app_clone = app.clone(); + let install_result = tokio::task::spawn_blocking(move || { + install_acp_runtime_blocking(&runtime_id_clone, &app_clone) + }) + .await + .map_err(|e| format!("install task panicked: {e}"))??; if !install_result.success { return Ok(install_result); @@ -259,12 +261,21 @@ pub async fn install_acp_runtime( steps: install_result.steps, restarted_count, failed_restart_count, + log_path: install_result.log_path, }) } /// Err(_) = infrastructure failure (panic, concurrency guard). /// Ok({success: false}) = an install step failed (stderr captured in steps). -fn install_acp_runtime_blocking(runtime_id: &str) -> Result { +/// +/// The reporter is built here rather than by the caller so this run's log +/// session starts only once the concurrency guard is held and the runtime id is +/// resolved to its canonical catalog form: a rejected install must not rotate a +/// running one's log, and the log filename is derived from that id. +fn install_acp_runtime_blocking( + runtime_id: &str, + app: &tauri::AppHandle, +) -> Result { // Re-fetch the login-shell PATH so a Node.js installation that happened // after app launch (or after a previous failed install) is visible to this // run and to the subsequent discover_acp_providers call. @@ -297,6 +308,8 @@ fn install_acp_runtime_blocking(runtime_id: &str) -> Result Result Result Result command, Ok(None) => cmd.to_string(), Err(step) => { - steps.push(*step); - return Ok(InstallRuntimeResult { - success: false, - steps, - restarted_count: 0, - failed_restart_count: 0, - }); + reporter.record_step(&mut steps, *step); + return Ok(reporter.failed(steps)); } }; - let mut result = run_install_command_with_retry("adapter", &planned); + let mut result = run_install_command_with_retry("adapter", &planned, &reporter); if !result.success && result.hint.is_none() && is_npm_global_install(cmd) { result.hint = npm_eacces_hint(&result.stderr, cmd); } let success = result.success; steps.push(result); if !success { - return Ok(InstallRuntimeResult { - success: false, - steps, - restarted_count: 0, - failed_restart_count: 0, - }); + return Ok(reporter.failed(steps)); } } } - post_install_verification::run(runtime_id, &mut steps); + post_install_verification::run(runtime_id, &mut steps, &reporter); Ok(InstallRuntimeResult { success: steps.iter().all(|step| step.success), steps, restarted_count: 0, failed_restart_count: 0, + log_path: reporter.log_path(), }) } @@ -1016,8 +1010,11 @@ fn build_install_command(command: &str) -> Result } // ── install command execution ───────────────────────────────────────────────── +mod install_capture; mod install_exec; +mod install_report; use install_exec::run_install_command_with_retry; +use install_report::InstallReporter; // ── managed Node/npm runtime ────────────────────────────────────────────────── mod managed_node; diff --git a/desktop/src-tauri/src/commands/agent_discovery/install_capture.rs b/desktop/src-tauri/src/commands/agent_discovery/install_capture.rs new file mode 100644 index 0000000000..903b68715a --- /dev/null +++ b/desktop/src-tauri/src/commands/agent_discovery/install_capture.rs @@ -0,0 +1,319 @@ +//! Bounded capture of an install command's output. +//! +//! One drain per stream feeds a [`Capture`], which holds two independently +//! bounded views of the same bytes: a small one sized for an error toast and a +//! large one sized for the install log file. Both are shared with the draining +//! reader rather than returned by it, so whatever arrived before a stall is +//! readable at the ceiling — exactly when the output matters most. + +use std::collections::VecDeque; +use std::io::Read; +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; + +/// How much of each end a capture keeps, and how it names what was cut. +#[derive(Clone, Copy)] +struct Caps { + head: usize, + tail: usize, + marker: fn(usize) -> String, +} + +/// Sized for a UI error message: enough to identify the failure, small enough +/// to read in a toast. +const UI_CAPS: Caps = Caps { + head: 512, + tail: 1024, + marker: |omitted| format!("... ({omitted} bytes omitted) ..."), +}; + +/// Sized for the log file, where the budget is disk rather than screen. At this +/// cap a real install log is complete in practice; the marker names the cases +/// where it is not, so the file never implies completeness it does not have. +const LOG_CAPS: Caps = Caps { + head: 128 * 1024, + tail: 128 * 1024, + marker: |omitted| format!("... [{omitted} bytes omitted at cap] ..."), +}; + +/// Bounded capture of one stream: its first `head` bytes, its last `tail` +/// bytes, and the total byte count. Output of any size costs a fixed amount of +/// memory, so an installer that prints megabytes cannot grow the process. +struct BoundedOutput { + head: Vec, + tail: VecDeque, + total: usize, + caps: Caps, +} + +type SharedOutput = Arc>; + +impl BoundedOutput { + fn shared(caps: Caps) -> SharedOutput { + Arc::new(Mutex::new(Self { + head: Vec::new(), + tail: VecDeque::new(), + total: 0, + caps, + })) + } + + /// Absorb one read. Chunk boundaries are irrelevant to the result: the head + /// fills first, the remainder rolls through the tail window. + fn push(&mut self, chunk: &[u8]) { + self.total += chunk.len(); + let head_room = self + .caps + .head + .saturating_sub(self.head.len()) + .min(chunk.len()); + let (head_part, tail_part) = chunk.split_at(head_room); + self.head.extend_from_slice(head_part); + self.tail.extend(tail_part); + while self.tail.len() > self.caps.tail { + self.tail.pop_front(); + } + } + + fn render(&self) -> String { + let tail: Vec = self.tail.iter().copied().collect(); + if self.total <= self.caps.head + self.caps.tail { + // Nothing was dropped, so head followed by tail is the whole stream. + let mut whole = self.head.clone(); + whole.extend_from_slice(&tail); + return decode(&whole); + } + // Both ends are cut at arbitrary byte offsets, so trim any partial + // character rather than emitting replacement chars, then drop the + // partial *token* each cut left behind. The marker counts every dropped + // byte, including both trims. + let head = erode_head(utf8_prefix(&self.head)); + let tail = erode_tail(utf8_suffix(&tail)); + let omitted = self.total - head.len() - tail.len(); + format!( + "{}\n{}\n{}", + decode(head), + (self.caps.marker)(omitted), + decode(tail) + ) + } +} + +/// The two bounded views of one stream, filled by a single drain. +pub(super) struct Capture { + ui: SharedOutput, + log: SharedOutput, +} + +impl Capture { + pub(super) fn new() -> Self { + Self { + ui: BoundedOutput::shared(UI_CAPS), + log: BoundedOutput::shared(LOG_CAPS), + } + } + + /// What the UI shows for this stream. + pub(super) fn ui(&self) -> String { + render(&self.ui) + } + + /// What the install log records for this stream. + pub(super) fn log(&self) -> String { + render(&self.log) + } + + fn push(&self, chunk: &[u8]) { + for sink in [&self.ui, &self.log] { + if let Ok(mut sink) = sink.lock() { + sink.push(chunk); + } + } + } +} + +/// Called with each complete line an install prints, for the live output line +/// in the UI. Shared across both drain threads of one attempt. +pub(super) type LineObserver = Arc; + +/// Render a capture even if its drain thread panicked mid-write — a poisoned +/// lock must not cost the diagnostics. +fn render(sink: &SharedOutput) -> String { + sink.lock().unwrap_or_else(|p| p.into_inner()).render() +} + +/// Read `pipe` to EOF, feeding fixed-size chunks into `capture` and each +/// complete line to `observer`. Read errors end the drain — a broken pipe means +/// the child is gone and there is nothing left to capture. +pub(super) fn drain_into(mut pipe: impl Read, capture: &Capture, observer: Option<&LineObserver>) { + let mut chunk = [0u8; 8192]; + let mut lines = LineSplitter::default(); + loop { + match pipe.read(&mut chunk) { + Ok(0) => return, + Ok(n) => { + capture.push(&chunk[..n]); + if let Some(observe) = observer { + lines.feed(&chunk[..n], |line| observe(line)); + } + } + Err(e) if e.kind() == std::io::ErrorKind::Interrupted => continue, + Err(_) => return, + } + } +} + +/// Reassembles lines from arbitrary read chunks. A partial trailing line is +/// held until its newline arrives, so an observer only ever sees complete +/// lines. The buffer is capped: a program that prints megabytes without a +/// newline must not grow it without bound. +#[derive(Default)] +struct LineSplitter { + partial: Vec, +} + +impl LineSplitter { + /// Longest line reassembled. Beyond this the excess is dropped, since the + /// consumer displays a single truncated line anyway. + const MAX_LINE: usize = 4096; + + fn feed(&mut self, chunk: &[u8], mut emit: impl FnMut(&str)) { + for byte in chunk { + if *byte == b'\n' { + let line = String::from_utf8_lossy(&self.partial).trim().to_string(); + self.partial.clear(); + if !line.is_empty() { + emit(&line); + } + } else if self.partial.len() < Self::MAX_LINE { + self.partial.push(*byte); + } + } + } +} + +/// Rate limiter for the live output line: at most one event per +/// `min_interval`. +/// +/// A line arriving inside the window is *held* rather than dropped, and the +/// newest held line replaces any older one. Dropping was wrong at two points: +/// the last line of an attempt — typically the failure that caused the retry — +/// vanished if it landed inside the window, and so did a new attempt's first +/// line when it arrived within 250ms of the previous attempt's last. +pub(super) struct Throttle { + min_interval: Duration, + state: Mutex, +} + +#[derive(Default)] +struct ThrottleState { + last_emitted: Option, + pending: Option, +} + +impl Throttle { + pub(super) fn new(min_interval: Duration) -> Self { + Self { + min_interval, + state: Mutex::new(ThrottleState::default()), + } + } + + /// Offer one line. `Some` means emit it now; `None` means it is held as the + /// newest pending line, to be emitted by [`Throttle::take_pending`] or + /// replaced by a line that supersedes it. + pub(super) fn offer(&self, line: &str, now: Instant) -> Option { + let Ok(mut state) = self.state.lock() else { + return None; + }; + if state + .last_emitted + .is_some_and(|prev| now.duration_since(prev) < self.min_interval) + { + state.pending = Some(line.to_string()); + return None; + } + state.last_emitted = Some(now); + // Emitting a newer line makes the held one obsolete: the display shows + // one line, and it must be the latest. + state.pending = None; + Some(line.to_string()) + } + + /// Take the held line, if the window closed on one. + pub(super) fn take_pending(&self) -> Option { + self.state.lock().ok()?.pending.take() + } + + /// Open the window for a new attempt, so its first line is emitted + /// immediately instead of waiting out the previous attempt's window. + pub(super) fn restart(&self) { + if let Ok(mut state) = self.state.lock() { + *state = ThrottleState::default(); + } + } +} + +fn decode(bytes: &[u8]) -> String { + String::from_utf8_lossy(bytes).into_owned() +} + +/// Drop a trailing partial UTF-8 sequence, keeping mid-stream invalid bytes for +/// the lossy decode to mark. +fn utf8_prefix(bytes: &[u8]) -> &[u8] { + match std::str::from_utf8(bytes) { + Ok(_) => bytes, + Err(e) if e.error_len().is_none() => &bytes[..e.valid_up_to()], + Err(_) => bytes, + } +} + +/// Drop leading UTF-8 continuation bytes — at most three can precede a +/// character start. +fn utf8_suffix(bytes: &[u8]) -> &[u8] { + let start = bytes + .iter() + .take(3) + .take_while(|b| *b & 0b1100_0000 == 0b1000_0000) + .count(); + &bytes[start..] +} + +/// How far a cut edge looks for a token boundary. Sized past any credential +/// shape worth protecting (an `nsec1` key is 63 bytes, registry tokens are +/// shorter) and short enough that erosion costs a token rather than a chunk of +/// output. A cut inside a longer whitespace-free run is left alone: erasing +/// kilobytes of a single-token stream would cost more diagnostics than the +/// fragment could leak. +const MAX_ERODED_TOKEN: usize = 256; + +/// Drop the partial token a head cut left at its end. +/// +/// Redaction runs on the rendered text and matches whole tokens: a prefixed +/// secret up to the next whitespace, or an exact env value. A cut through the +/// middle of a secret leaves a fragment that matches neither and therefore +/// survives scrubbing, so the fragment is removed here instead — at the cut, +/// where it is still identifiable as partial. The omitted-byte marker counts +/// what this drops. +fn erode_head(bytes: &[u8]) -> &[u8] { + let window = bytes.len().saturating_sub(MAX_ERODED_TOKEN); + match bytes[window..].iter().rposition(u8::is_ascii_whitespace) { + Some(last) => &bytes[..=window + last], + None => bytes, + } +} + +/// Drop the partial token a tail cut left at its start — the direction that +/// matters most, since a fragment there has lost the `nsec1`-style prefix the +/// scrubber keys on. See [`erode_head`]. +fn erode_tail(bytes: &[u8]) -> &[u8] { + let window = MAX_ERODED_TOKEN.min(bytes.len()); + match bytes[..window].iter().position(u8::is_ascii_whitespace) { + Some(first) => &bytes[first..], + None => bytes, + } +} + +#[cfg(test)] +#[path = "install_capture_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/commands/agent_discovery/install_capture_tests.rs b/desktop/src-tauri/src/commands/agent_discovery/install_capture_tests.rs new file mode 100644 index 0000000000..8830f355df --- /dev/null +++ b/desktop/src-tauri/src/commands/agent_discovery/install_capture_tests.rs @@ -0,0 +1,437 @@ +use super::*; + +/// Feed `chunks` through a capture in order. +fn capture_of(chunks: &[&[u8]]) -> Capture { + let capture = Capture::new(); + for chunk in chunks { + capture.push(chunk); + } + capture +} + +/// Render what the UI would show for a stream of `chunks`. +fn ui(chunks: &[&[u8]]) -> String { + capture_of(chunks).ui() +} + +// ── bounded capture ────────────────────────────────────────────────────────── + +/// Output within the cap is passed through byte-for-byte — no marker, no loss. +#[test] +fn test_capture_leaves_short_output_untouched() { + let short = "a".repeat(1536); + + assert_eq!(ui(&[short.as_bytes()]), short); +} + +/// Over the cap, both ends survive and the middle is replaced by a marker +/// naming the omitted byte count — the head keeps the command's opening +/// context and the tail keeps the error that usually trails. +#[test] +fn test_capture_over_cap_keeps_head_and_tail_with_marker() { + let input = format!( + "{}{}{}", + "H".repeat(512), + "M".repeat(4000), + "T".repeat(1024) + ); + + let out = ui(&[input.as_bytes()]); + + assert!(out.starts_with(&"H".repeat(512))); + assert!(out.ends_with(&"T".repeat(1024))); + assert!( + out.contains("... (4000 bytes omitted) ..."), + "marker must name the omitted byte count, got: {out}" + ); +} + +/// The rendered result depends only on the byte stream, not on how the reads +/// happened to split it — a real drain sees arbitrary chunk sizes. +#[test] +fn test_capture_is_independent_of_chunk_boundaries() { + let input = "x".repeat(9000); + let one_shot = ui(&[input.as_bytes()]); + + let chunked: Vec<&[u8]> = input.as_bytes().chunks(7).collect(); + + assert_eq!(ui(&chunked), one_shot); +} + +/// Truncation must not split a multi-byte character. Both cut points land +/// mid-codepoint here; the partial bytes are dropped rather than decoded into +/// replacement chars. +#[test] +fn test_capture_does_not_split_multibyte_characters() { + // "é" is 2 bytes, so every candidate cut index lands mid-character. + let input = "é".repeat(4000); + + let out = ui(&[input.as_bytes()]); + + assert!(out.contains("bytes omitted"), "input must exceed the cap"); + assert!(!out.contains('\u{fffd}'), "no replacement chars: {out}"); +} + +/// Memory stays flat regardless of how much the installer prints: the rendered +/// UI result of a 4MiB stream is no larger than that of a 6KiB one. +#[test] +fn test_capture_of_huge_output_stays_bounded() { + let chunk = vec![b'z'; 8192]; + + let capture = Capture::new(); + for _ in 0..512 { + capture.push(&chunk); + } + + let out = capture.ui(); + assert!( + out.len() < 2048, + "4MiB of output must render bounded, got {} bytes", + out.len() + ); + assert!(out.contains("bytes omitted")); +} + +// ── the log view is separately bounded ─────────────────────────────────────── + +/// The log view holds output the UI view had to cut. A toast is capped for +/// readability; the log file's budget is disk, and "Full log: {path}" has to +/// point at more than the toast already showed. +#[test] +fn test_log_view_keeps_output_the_ui_view_truncates() { + let input = format!("start{}end", "m".repeat(64 * 1024)); + + let capture = capture_of(&[input.as_bytes()]); + + assert!( + capture.ui().contains("bytes omitted"), + "64KiB must exceed the UI cap" + ); + assert_eq!( + capture.log(), + input, + "the same output must be complete in the log view" + ); +} + +/// Even the log view is bounded — a runaway installer cannot fill the disk — +/// and when it does cut, the record says so inline at the cap rather than +/// implying completeness. +#[test] +fn test_log_view_is_bounded_and_marks_its_cap() { + let head = "H".repeat(128 * 1024); + let middle = "M".repeat(5000); + let tail = "T".repeat(128 * 1024); + let input = format!("{head}{middle}{tail}"); + + let capture = capture_of(&[input.as_bytes()]); + + let out = capture.log(); + assert!( + out.len() < 300 * 1024, + "output past the log cap must render bounded, got {} bytes", + out.len() + ); + assert!(out.starts_with(&head), "the log head must survive intact"); + assert!(out.ends_with(&tail), "the log tail must survive intact"); + assert!( + out.contains("... [5000 bytes omitted at cap] ..."), + "a cut log record must name the cap inline, got the middle: {}", + &out[128 * 1024..(128 * 1024 + 64).min(out.len())] + ); +} + +/// The two views mark their cuts differently on purpose: the toast reads as +/// prose, the log record reads as a machine-scannable annotation. +#[test] +fn test_ui_and_log_views_use_their_own_cap_markers() { + let input = "x".repeat(300 * 1024); + + let capture = capture_of(&[input.as_bytes()]); + + assert!( + capture.ui().contains("bytes omitted) ..."), + "the UI marker reads as prose: {}", + capture.ui() + ); + assert!(capture.log().contains("bytes omitted at cap] ...")); +} + +// ── line observation ───────────────────────────────────────────────────────── + +/// Collect the lines a drain over `chunks` reports. +fn observed_lines(chunks: &[&[u8]]) -> Vec { + let seen: Arc>> = Arc::new(Mutex::new(Vec::new())); + let observer: LineObserver = { + let seen = Arc::clone(&seen); + Arc::new(move |line: &str| seen.lock().unwrap().push(line.to_string())) + }; + let bytes: Vec = chunks.concat(); + + drain_into(bytes.as_slice(), &Capture::new(), Some(&observer)); + + let observed = seen.lock().unwrap().clone(); + observed +} + +/// The observer sees complete lines, reassembled across the read boundaries +/// that split them — a live output line must never show half a word. +#[test] +fn test_observer_reassembles_lines_split_across_reads() { + let lines = observed_lines(&[b"downloa", b"ding 40%\nunpack", b"ing\n"]); + + assert_eq!(lines, vec!["downloading 40%", "unpacking"]); +} + +/// A trailing line with no newline is never reported: it may still be growing, +/// and showing a half-line as if complete is worse than showing the previous +/// one. +#[test] +fn test_observer_withholds_a_line_that_has_no_newline_yet() { + let lines = observed_lines(&[b"complete\n", b"still-writing"]); + + assert_eq!(lines, vec!["complete"]); +} + +/// Blank lines carry nothing to display; progress output is full of them. +#[test] +fn test_observer_skips_blank_lines() { + let lines = observed_lines(&[b"a\n\n \nb\n"]); + + assert_eq!(lines, vec!["a", "b"]); +} + +/// A pathological line with no newline must not grow the buffer without bound. +#[test] +fn test_observer_caps_a_pathologically_long_line() { + let huge = "x".repeat(100_000); + + let lines = observed_lines(&[huge.as_bytes(), b"\n"]); + + assert_eq!(lines.len(), 1); + assert!( + lines[0].len() <= LineSplitter::MAX_LINE, + "line must be capped, got {} bytes", + lines[0].len() + ); +} + +/// A drain with no observer still captures — the log and UI views do not +/// depend on anyone watching. +#[test] +fn test_drain_captures_without_an_observer() { + let capture = Capture::new(); + + drain_into(b"hello\n".as_slice(), &capture, None); + + assert_eq!(capture.ui(), "hello\n"); +} + +// ── throttle ───────────────────────────────────────────────────────────────── + +/// The first line goes out immediately, and one arriving inside the window is +/// *held* rather than dropped: it becomes the pending line, so the newest output +/// survives the rate limit instead of vanishing. +#[test] +fn test_throttle_emits_the_first_line_and_holds_the_next_in_window() { + let throttle = Throttle::new(Duration::from_millis(250)); + let start = Instant::now(); + + assert_eq!(throttle.offer("first", start), Some("first".to_string())); + assert_eq!( + throttle.offer("second", start + Duration::from_millis(100)), + None + ); + assert_eq!( + throttle.take_pending(), + Some("second".to_string()), + "the line inside the window must be retained, not dropped" + ); +} + +/// A burst inside one window collapses to its newest line: the display shows a +/// single line, so an older held line has no value once a newer one exists. +#[test] +fn test_throttle_keeps_only_the_newest_held_line() { + let throttle = Throttle::new(Duration::from_millis(250)); + let start = Instant::now(); + throttle.offer("emitted", start); + + throttle.offer("held-then-superseded", start + Duration::from_millis(50)); + throttle.offer("newest", start + Duration::from_millis(100)); + + assert_eq!(throttle.take_pending(), Some("newest".to_string())); +} + +/// Once the window passes, emission resumes and nothing is left pending — the +/// emitted line *is* the newest, so holding it too would emit it twice. +#[test] +fn test_throttle_emits_again_after_the_window_and_clears_the_held_line() { + let throttle = Throttle::new(Duration::from_millis(250)); + let start = Instant::now(); + throttle.offer("first", start); + throttle.offer("held", start + Duration::from_millis(50)); + + assert_eq!( + throttle.offer("later", start + Duration::from_millis(300)), + Some("later".to_string()) + ); + + assert_eq!( + throttle.take_pending(), + None, + "a line emitted after the window supersedes the held one" + ); +} + +/// The window is measured from the last *emitted* line, not the last offer: a +/// stream of held lines must not extend the silence. +#[test] +fn test_throttle_window_runs_from_the_last_emission() { + let throttle = Throttle::new(Duration::from_millis(250)); + let start = Instant::now(); + throttle.offer("first", start); + + assert_eq!( + throttle.offer("held", start + Duration::from_millis(200)), + None + ); + + assert_eq!( + throttle.offer("next", start + Duration::from_millis(260)), + Some("next".to_string()), + "a held line must not restart the window" + ); +} + +/// A pending line is taken once. Taking it twice would re-emit a line the +/// display already shows. +#[test] +fn test_throttle_yields_a_held_line_only_once() { + let throttle = Throttle::new(Duration::from_millis(250)); + let start = Instant::now(); + throttle.offer("first", start); + throttle.offer("held", start + Duration::from_millis(50)); + + assert_eq!(throttle.take_pending(), Some("held".to_string())); + + assert_eq!(throttle.take_pending(), None); +} + +/// Restarting opens the window immediately, which is what lets a new attempt's +/// first line go out even when it arrives inside the previous attempt's window. +/// It also discards a held line: that line belongs to the attempt that just +/// ended, and the new attempt is about to clear the display. +#[test] +fn test_throttle_restart_opens_the_window_and_discards_the_held_line() { + let throttle = Throttle::new(Duration::from_millis(250)); + let start = Instant::now(); + throttle.offer("previous attempt", start); + throttle.offer("held", start + Duration::from_millis(10)); + + throttle.restart(); + + assert_eq!(throttle.take_pending(), None); + assert_eq!( + throttle.offer("new attempt", start + Duration::from_millis(20)), + Some("new attempt".to_string()) + ); +} + +// ── cut-edge erosion ───────────────────────────────────────────────────────── + +/// A secret cut in half by the head cap must not survive as a fragment. +/// Redaction matches whole tokens — a prefixed secret up to the next whitespace +/// — so `nsec1qqq…` cut mid-value would still be scrubbed, but the *tail* of +/// that same value, having lost its prefix, would not be. Both cut edges drop +/// their partial token for that reason. +#[test] +fn test_capture_drops_the_partial_token_at_each_cut_edge() { + // Positioned so the head cap lands inside the first secret and the tail cap + // inside the second. + let head_secret = "nsec1headsecretvalue"; + let tail_secret = "nsec1tailsecretvalue"; + let input = format!( + "{} {head_secret} {} {tail_secret} {}", + "h".repeat(500), + "m".repeat(4000), + "t".repeat(1010) + ); + + let out = ui(&[input.as_bytes()]); + + assert!(out.contains("bytes omitted"), "input must exceed the cap"); + for fragment in ["nsec1head", "secretvalue"] { + assert!( + !out.contains(fragment), + "a fragment of a cut token must not survive: {out}" + ); + } +} + +/// Erosion stops at the nearest whitespace, so it costs one partial token and +/// not the surrounding output — the head's earlier lines and the tail's later +/// ones are what make a truncated capture readable. +#[test] +fn test_capture_erosion_keeps_the_complete_tokens_around_the_cut() { + let input = format!( + "opening line +{} +cut-here-head{}cut-here-tail +{} +closing line +", + "h".repeat(480), + "m".repeat(4000), + "t".repeat(980) + ); + + let out = ui(&[input.as_bytes()]); + + assert!(out.starts_with("opening line\n"), "got: {out}"); + assert!(out.ends_with("closing line\n"), "got: {out}"); +} + +/// A cut inside a whitespace-free run longer than the erosion window is left +/// intact. Erosion is bounded on purpose: erasing kilobytes of a single-token +/// stream — `npm` progress bars and base64 payloads both look like this — would +/// cost more diagnostics than a fragment of one could leak. +#[test] +fn test_capture_of_one_giant_token_keeps_its_cut_edges() { + let input = "x".repeat(4000); + + let out = ui(&[input.as_bytes()]); + + assert!(out.starts_with(&"x".repeat(512)), "got: {out}"); + assert!(out.ends_with(&"x".repeat(1024)), "got: {out}"); +} + +/// The marker's byte count stays honest across erosion: what it names as omitted +/// must equal the input minus what is actually shown, or a reader cannot trust +/// the file to say how much is missing. +#[test] +fn test_capture_marker_counts_the_bytes_erosion_dropped() { + let input = format!( + "{} {} {}", + "h".repeat(600), + "m".repeat(4000), + "t".repeat(1100) + ); + + let out = ui(&[input.as_bytes()]); + + let (head, rest) = out.split_once('\n').expect("a marker line"); + let (marker, tail) = rest.split_once('\n').expect("a marker line"); + let omitted: usize = marker + .trim_start_matches("... (") + .split_once(' ') + .expect("a byte count") + .0 + .parse() + .expect("a byte count"); + assert_eq!( + head.len() + omitted + tail.len(), + input.len(), + "shown + omitted must account for every input byte" + ); +} diff --git a/desktop/src-tauri/src/commands/agent_discovery/install_exec.rs b/desktop/src-tauri/src/commands/agent_discovery/install_exec.rs index 63163ceadc..3b94f2ef8a 100644 --- a/desktop/src-tauri/src/commands/agent_discovery/install_exec.rs +++ b/desktop/src-tauri/src/commands/agent_discovery/install_exec.rs @@ -5,13 +5,46 @@ //! `install_powershell_command`, `build_install_command`); this module owns //! only what happens once a `Command` exists. -use std::io::Read; +use std::sync::Arc; +use std::time::{Duration, Instant}; +use super::install_capture::{drain_into, Capture, LineObserver}; +use super::install_report::{InstallOutcome, InstallReporter}; use crate::managed_agents::InstallStepResult; /// Maximum number of attempts for a transient-looking install command. const INSTALL_MAX_ATTEMPTS: u32 = 3; +/// Absolute wall-clock ceiling for a single install command. +/// +/// This is a ceiling, not an inactivity timeout: nothing observable +/// distinguishes a hung installer from one silently transferring a large +/// artifact (the Goose step downloads a ~79MB release asset with no progress +/// output, and npm at its default log level prints only at the end), so silence +/// alone never kills an install. The previous 300s wall killed +/// slow-but-working installs — Windows Defender scanning every file npm +/// extracts pushes past it routinely (#2401). +/// +/// The cost of a larger ceiling: skipping onboarding does not cancel a running +/// install and a per-runtime guard rejects a second one, so this is also the +/// longest a user who skipped a genuinely *hung* install waits before Install +/// works again in Settings. User-facing cancellation is the product-level fix. +const INSTALL_TIMEOUT: Duration = Duration::from_secs(900); + +/// How long the group gets to exit on SIGTERM before the ceiling escalates to +/// SIGKILL. +#[cfg(unix)] +const TERM_GRACE: Duration = Duration::from_secs(1); + +/// How long the ceiling waits after killing the install's process group — +/// applied separately to reaping the killed child and to the output drains +/// finishing. The kill closes the pipe write ends, so both normally complete +/// within microseconds; the bound covers the cases where they don't (a process +/// that escaped the group and still holds a pipe, or a termination that failed +/// outright). Neither may hold the install — nor the per-runtime concurrency +/// guard behind it — open past the ceiling. +const POST_KILL_GRACE: Duration = Duration::from_secs(2); + /// Run an install command, retrying transient failures with backoff. /// /// Runtime installs pull artifacts over the network — Goose's `curl … | bash` @@ -22,10 +55,24 @@ const INSTALL_MAX_ATTEMPTS: u32 = 3; /// `INSTALL_MAX_ATTEMPTS` times. Failures with no exit code — a timeout or a /// shell that never spawned — are not retried, since re-running them just costs /// the user more time without a plausible path to success. -pub(super) fn run_install_command_with_retry(step: &str, command: &str) -> InstallStepResult { +/// +/// Every attempt is recorded through `reporter`, so the install log holds the +/// full retry history even though the UI only ever sees the last attempt. +pub(super) fn run_install_command_with_retry( + step: &str, + command: &str, + reporter: &InstallReporter, +) -> InstallStepResult { run_install_with_retry( INSTALL_MAX_ATTEMPTS, - |_attempt| run_install_command(step, command), + |attempt| { + // Before the command spawns, so the previous attempt's last line + // stops being displayed for the whole backoff rather than until the + // new attempt happens to print something. + reporter.start_attempt(); + let outcome = run_install_command(step, command, reporter.line_observer()); + reporter.record_attempt(attempt, outcome) + }, std::thread::sleep, ) } @@ -92,11 +139,15 @@ fn prepare_install_command(command: &str) -> Result InstallStepResult { +fn run_install_command( + step: &str, + command: &str, + observer: Option, +) -> InstallOutcome { let mut cmd = match prepare_install_command(command) { Ok(cmd) => cmd, Err(hint) => { - return InstallStepResult { + return InstallOutcome::synthesized(InstallStepResult { step: step.to_string(), command: command.to_string(), success: false, @@ -104,11 +155,11 @@ fn run_install_command(step: &str, command: &str) -> InstallStepResult { stderr: "no suitable shell found for install commands".to_string(), exit_code: None, hint: Some(hint), - }; + }); } }; - let mut child = match cmd + let child = match cmd .stdin(std::process::Stdio::null()) .stdout(std::process::Stdio::piped()) .stderr(std::process::Stdio::piped()) @@ -116,7 +167,7 @@ fn run_install_command(step: &str, command: &str) -> InstallStepResult { { Ok(child) => child, Err(e) => { - return InstallStepResult { + return InstallOutcome::synthesized(InstallStepResult { step: step.to_string(), command: command.to_string(), success: false, @@ -124,146 +175,312 @@ fn run_install_command(step: &str, command: &str) -> InstallStepResult { stderr: format!("failed to spawn shell: {e}"), exit_code: None, hint: None, - }; + }); } }; - // Drain stdout/stderr on background threads to prevent pipe buffer deadlock. + await_install_child(step, command, child, INSTALL_TIMEOUT, observer) +} + +/// Drain a spawned install child's output into bounded buffers and wait for it +/// to exit, killing it at `timeout`. +/// +/// Split from the spawn so the timing-sensitive half is testable without a real +/// login shell: shell startup alone can outlast a short test ceiling on a +/// loaded machine. Production always passes [`INSTALL_TIMEOUT`]. +fn await_install_child( + step: &str, + command: &str, + mut child: std::process::Child, + timeout: Duration, + observer: Option, +) -> InstallOutcome { + // Drain stdout/stderr on background threads to prevent pipe buffer + // deadlock. Each drain feeds a bounded capture the main thread can read at + // any time, so a timeout can still surface whatever the install printed + // before it stalled. + let stdout_capture = Arc::new(Capture::new()); + let stderr_capture = Arc::new(Capture::new()); let stdout_pipe = child.stdout.take(); let stderr_pipe = child.stderr.take(); - let stdout_thread = std::thread::spawn(move || { - let mut buf = String::new(); - if let Some(mut pipe) = stdout_pipe { - let _ = pipe.read_to_string(&mut buf); + // One event stream carries every input the ceiling waits on, so the exit + // and the drains are governed by the same deadline instead of the exit + // releasing the drains from it. + let (events_tx, events) = std::sync::mpsc::channel(); + + std::thread::spawn({ + let (capture, done, observer) = ( + Arc::clone(&stdout_capture), + events_tx.clone(), + observer.clone(), + ); + move || { + if let Some(pipe) = stdout_pipe { + drain_into(pipe, &capture, observer.as_ref()); + } + let _ = done.send(Settled::Drained); } - buf }); - let stderr_thread = std::thread::spawn(move || { - let mut buf = String::new(); - if let Some(mut pipe) = stderr_pipe { - let _ = pipe.read_to_string(&mut buf); + std::thread::spawn({ + let (capture, done) = (Arc::clone(&stderr_capture), events_tx.clone()); + move || { + if let Some(pipe) = stderr_pipe { + drain_into(pipe, &capture, observer.as_ref()); + } + let _ = done.send(Settled::Drained); } - buf }); // Save the PID before moving `child` into the wait thread so we can // kill the process on timeout. let child_pid = child.id(); - let (tx, rx) = std::sync::mpsc::channel(); - let wait_thread = std::thread::spawn(move || { - let status = child.wait(); - let _ = tx.send(status); + std::thread::spawn(move || { + let _ = events_tx.send(Settled::Exited(child.wait())); }); - // 5-minute timeout for install commands. - let deadline = std::time::Instant::now() + std::time::Duration::from_secs(300); - loop { - let remaining = deadline.saturating_duration_since(std::time::Instant::now()); - if remaining.is_zero() { - // Timeout: kill the child process via its PID, then join all - // threads so nothing leaks. - #[cfg(unix)] - unsafe { - libc::kill(child_pid as i32, libc::SIGTERM); - } - #[cfg(windows)] - { - let _ = crate::managed_agents::taskkill_tree(child_pid); - } - drop(rx); - let _ = wait_thread.join(); - let _ = stdout_thread.join(); - let _ = stderr_thread.join(); - return InstallStepResult { + // No thread is ever joined. Each sends its one event before exiting, so a + // join after a complete settle would add nothing — and a join before one + // would reintroduce the unbounded wait this loop exists to prevent. + let mut settle = Settle::default(); + let ended = settle.collect(&events, Instant::now() + timeout); + if ended == Collected::Deadline { + // Ceiling reached: kill the install's whole process group — the install + // shell is a session leader (`setsid` in its `pre_exec`), so signalling + // only the leader would leave descendants running and holding the + // output pipes open. + // + // Whether the leader had already exited decides the verdict. If it had, + // only a descendant was holding a drain open: the install genuinely + // finished and its real status stands. If it had not, the install itself + // was still running and this is a timeout — the status the kill produces + // moments later describes the kill, not the install, so it is discarded. + let install_finished = settle.status.is_some(); + terminate_install_group(child_pid); + // Reaping the child and finishing the drains share one bound. Both + // normally complete within microseconds of the kill, which closes the + // pipes; when they don't — a process that escaped the group still + // holding a pipe, or a termination that failed outright — waiting would + // defeat the very ceiling that fired and keep the per-runtime install + // guard behind it closed. Stragglers are detached instead; the captures + // are read under the lock either way. + settle.collect(&events, Instant::now() + POST_KILL_GRACE); + if !install_finished { + return failed_with_capture( + step, + command, + timeout_message(timeout), + &stdout_capture, + &stderr_capture, + ); + } + } + + match settle.status { + Some(Ok(status)) => InstallOutcome { + step: InstallStepResult { step: step.to_string(), command: command.to_string(), - success: false, - stdout: String::new(), - stderr: "install command timed out after 5 minutes".to_string(), - exit_code: None, + success: status.success(), + stdout: stdout_capture.ui(), + stderr: stderr_capture.ui(), + exit_code: status.code(), hint: None, - }; - } + }, + log_stdout: stdout_capture.log(), + log_stderr: stderr_capture.log(), + }, + Some(Err(e)) => failed_with_capture( + step, + command, + format!("failed to check process status: {e}"), + &stdout_capture, + &stderr_capture, + ), + // Every sender is gone without an exit ever arriving. + None => failed_with_capture( + step, + command, + "internal error: install wait ended without a status".to_string(), + &stdout_capture, + &stderr_capture, + ), + } +} - match rx.recv_timeout(std::time::Duration::from_millis(200).min(remaining)) { - Ok(Ok(status)) => { - let _ = wait_thread.join(); - let stdout = stdout_thread.join().unwrap_or_default(); - let stderr_raw = stderr_thread.join().unwrap_or_default(); - return InstallStepResult { - step: step.to_string(), - command: command.to_string(), - success: status.success(), - stdout: truncate_output(stdout), - stderr: truncate_output(stderr_raw), - exit_code: status.code(), - hint: None, - }; - } - Ok(Err(e)) => { - let _ = wait_thread.join(); - let _ = stdout_thread.join(); - let _ = stderr_thread.join(); - return InstallStepResult { - step: step.to_string(), - command: command.to_string(), - success: false, - stdout: String::new(), - stderr: format!("failed to check process status: {e}"), - exit_code: None, - hint: None, - }; - } - Err(std::sync::mpsc::RecvTimeoutError::Timeout) => { - // Still running; loop and check deadline again. - continue; - } - Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => { - // wait_thread dropped sender without sending — shouldn't happen. - let _ = wait_thread.join(); - let _ = stdout_thread.join(); - let _ = stderr_thread.join(); - return InstallStepResult { - step: step.to_string(), - command: command.to_string(), - success: false, - stdout: String::new(), - stderr: "internal error: wait thread disconnected".to_string(), - exit_code: None, - hint: None, - }; +/// One input the ceiling waits on. +enum Settled { + Exited(std::io::Result), + Drained, +} + +/// How a bounded [`Settle::collect`] ended. +#[derive(PartialEq, Debug)] +enum Collected { + /// The child exited and both drains reached EOF. + Complete, + /// The deadline passed first. + Deadline, + /// Every sender is gone — a thread died without reporting. + Disconnected, +} + +/// What the install has settled so far: the child's exit status once it is +/// known, and how many of the two drains have reached EOF. +/// +/// Collecting is resumable, so the ceiling can fold more events into the same +/// state under a second, post-kill deadline. +#[derive(Default)] +struct Settle { + status: Option>, + drained: usize, +} + +impl Settle { + const DRAINS: usize = 2; + + fn is_complete(&self) -> bool { + self.status.is_some() && self.drained >= Self::DRAINS + } + + /// Fold events until the install has fully settled or `deadline` passes. + /// + /// The exit and the drains share one deadline deliberately: a shell can exit + /// while a descendant it left behind still holds the inherited output pipes, + /// and waiting on those drains outside the deadline would let such a + /// descendant outlast the ceiling — holding the per-runtime install guard, + /// which is the very failure the ceiling exists to prevent. + fn collect( + &mut self, + events: &std::sync::mpsc::Receiver, + deadline: Instant, + ) -> Collected { + while !self.is_complete() { + let remaining = deadline.saturating_duration_since(Instant::now()); + match events.recv_timeout(remaining) { + Ok(Settled::Exited(status)) => self.status = Some(status), + Ok(Settled::Drained) => self.drained += 1, + Err(std::sync::mpsc::RecvTimeoutError::Timeout) => return Collected::Deadline, + Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => { + return Collected::Disconnected + } } } + Collected::Complete } } -/// Cap output to head + tail to avoid flooding the UI with large error dumps, -/// while preserving the most useful parts of the output. -fn truncate_output(s: String) -> String { - const HEAD: usize = 512; - const TAIL: usize = 1024; - const LIMIT: usize = HEAD + TAIL; - if s.len() <= LIMIT { - return s; - } - let head_end = floor_char_boundary(&s, HEAD); - let tail_start = floor_char_boundary(&s, s.len().saturating_sub(TAIL)); - let omitted = tail_start - head_end; - format!( - "{}\n... ({omitted} bytes omitted) ...\n{}", - &s[..head_end], - &s[tail_start..] - ) +/// Kill the install's process group, escalating on the *tree's* liveness. +/// +/// The install ceiling owns this rather than reusing +/// `managed_agents::terminate_process`, which escalates to SIGKILL only while +/// the group *leader* is still running: a descendant that ignores SIGTERM +/// outlives the leader, keeps the output pipes open, and never receives the +/// group SIGKILL. The ceiling's contract is that nothing survives it, and the +/// shared helper's escalation is load-bearing for the agent stop/restore paths, +/// so the stricter rule lives here instead of changing it for them. +/// +/// Nothing is returned: every outcome — including a signal that could not be +/// delivered at all — has the same handling, the bounded waits at the call +/// site. +#[cfg(unix)] +fn terminate_install_group(pid: u32) { + signal_install_tree(pid, libc::SIGTERM); + let deadline = Instant::now() + TERM_GRACE; + while install_tree_is_alive(pid) { + if Instant::now() >= deadline { + signal_install_tree(pid, libc::SIGKILL); + return; + } + std::thread::sleep(Duration::from_millis(50)); + } +} + +/// Signal every process in `pid`'s group, falling back to the leader alone when +/// the group cannot be signalled — the leader may have changed groups, or macOS +/// may refuse one member — since killing the install shell beats killing +/// nothing. +#[cfg(unix)] +fn signal_install_tree(pid: u32, signal: i32) { + if unsafe { libc::kill(-(pid as i32), signal) } != 0 { + unsafe { libc::kill(pid as i32, signal) }; + } +} + +/// Whether anything the ceiling aimed at is still running: a member of the +/// process group, or the leader itself. +#[cfg(unix)] +fn install_tree_is_alive(pid: u32) -> bool { + signal_reaches(-(pid as i32)) || signal_reaches(pid as i32) +} + +/// `kill(target, 0)` distinguishes "nothing there" (`ESRCH`) from every other +/// outcome. Anything ambiguous — notably `EPERM` for a member we may not +/// signal — counts as alive, so an unclear answer escalates rather than +/// declaring the tree dead. +#[cfg(unix)] +fn signal_reaches(target: i32) -> bool { + if unsafe { libc::kill(target, 0) } == 0 { + return true; + } + std::io::Error::last_os_error().raw_os_error() != Some(libc::ESRCH) +} + +/// Windows has no process groups on this path: `terminate_process` runs +/// `taskkill /T /F`, which is already tree-wide and unconditional, so there is +/// no escalation to get wrong. +#[cfg(not(unix))] +fn terminate_install_group(pid: u32) { + let _ = crate::managed_agents::terminate_process(pid); } -fn floor_char_boundary(s: &str, mut index: usize) -> usize { - index = index.min(s.len()); - while index > 0 && !s.is_char_boundary(index) { - index -= 1; +/// A failure carrying whatever the drains captured, with `reason` leading +/// stderr so the surfaced message names the failure before the install's own +/// output. +fn failed_with_capture( + step: &str, + command: &str, + reason: String, + stdout: &Capture, + stderr: &Capture, +) -> InstallOutcome { + InstallOutcome { + step: InstallStepResult { + step: step.to_string(), + command: command.to_string(), + success: false, + stdout: stdout.ui(), + stderr: lead_with_reason(&reason, stderr.ui()), + exit_code: None, + hint: None, + }, + log_stdout: stdout.log(), + log_stderr: lead_with_reason(&reason, stderr.log()), } - index +} + +/// Put `reason` ahead of the install's own stderr, so the surfaced message names +/// the failure before the output. An empty capture leaves the reason alone, +/// without a dangling separator. +fn lead_with_reason(reason: &str, captured: String) -> String { + if captured.is_empty() { + reason.to_string() + } else { + format!("{reason}\n{captured}") + } +} + +/// Name the limit that fired and its value, so a ceiling kill is +/// distinguishable from the installer's own failure. +fn timeout_message(timeout: Duration) -> String { + let secs = timeout.as_secs(); + let limit = if secs >= 60 { + format!("{}-minute", secs / 60) + } else { + format!("{secs}-second") + }; + format!("install command exceeded the {limit} ceiling and was terminated") } #[cfg(test)] @@ -410,48 +627,304 @@ mod tests { assert_eq!(cmd.get_current_dir(), Some(expected.as_path())); } - // ── output truncation ───────────────────────────────────────────────────── + // ── install ceiling ─────────────────────────────────────────────────────── + + /// The ceiling is Will's ruling: 15 minutes, and the error names the limit + /// that fired so a ceiling kill is not mistaken for the installer's own + /// failure. + #[test] + fn test_ceiling_is_fifteen_minutes_and_error_names_it() { + assert_eq!(INSTALL_TIMEOUT, Duration::from_secs(900)); + assert!( + timeout_message(INSTALL_TIMEOUT).contains("15-minute"), + "got: {}", + timeout_message(INSTALL_TIMEOUT) + ); + } + + /// Spawn `script` under `sh` as a process-group leader with piped output — + /// the same shape [`run_install_command`] hands to + /// [`await_install_child`], minus the login shell whose own startup can + /// outlast a short test ceiling. + #[cfg(unix)] + fn spawn_group_leader(script: &str) -> std::process::Child { + use std::os::unix::process::CommandExt; + + let mut cmd = std::process::Command::new("/bin/sh"); + cmd.arg("-c").arg(script); + unsafe { + cmd.pre_exec(|| { + libc::setsid(); + Ok(()) + }); + } + cmd.stdin(std::process::Stdio::null()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .spawn() + .expect("sh must spawn") + } + + /// A command killed by the ceiling must surface what it printed before + /// stalling — that partial output is the only evidence of where the install + /// got stuck — and must stay unretryable, since re-running a hang just + /// costs the user another ceiling. + #[cfg(unix)] + #[test] + fn test_ceiling_returns_captured_output_and_stays_unretryable() { + let child = spawn_group_leader("echo out-before-hang; echo err-before-hang >&2; sleep 60"); + + let started = Instant::now(); + let outcome = await_install_child("cli", "install", child, Duration::from_secs(5), None); + let result = &outcome.step; + + assert!(!result.success); + assert_eq!(result.exit_code, None, "a killed command has no exit code"); + assert!( + !install_failure_is_retryable(result), + "a ceiling kill must not be retried" + ); + assert!( + result.stdout.contains("out-before-hang"), + "stdout captured before the stall must survive, got: {:?}", + result.stdout + ); + assert!( + result.stderr.contains("5-second ceiling"), + "stderr must name the ceiling that actually fired, got: {:?}", + result.stderr + ); + assert!( + result.stderr.contains("err-before-hang"), + "stderr captured before the stall must survive, got: {:?}", + result.stderr + ); + assert!( + started.elapsed() < Duration::from_secs(30), + "the ceiling must not wait on the hung command's own exit" + ); + assert!( + outcome.log_stderr.contains("err-before-hang"), + "the log record of a ceiling kill must carry the output too, got: {:?}", + outcome.log_stderr + ); + } + + /// A failure whose stream captured nothing surfaces the reason alone — no + /// dangling separator from an empty capture. + #[test] + fn test_failure_with_no_captured_output_reports_only_the_reason() { + let result = failed_with_capture( + "cli", + "curl … | bash", + "boom".to_string(), + &Capture::new(), + &Capture::new(), + ) + .step; + + assert_eq!(result.stdout, ""); + assert_eq!(result.stderr, "boom"); + } + + // ── post-kill settle bound ──────────────────────────────────────────────── + + /// A sender that never arrives — the shape of a failed termination, whose + /// child is never reaped — must not extend the wait past its deadline. + #[test] + fn test_settling_on_a_message_that_never_arrives_stops_at_the_deadline() { + let (_tx, events) = std::sync::mpsc::channel::(); + + let started = Instant::now(); + let ended = Settle::default().collect(&events, started + Duration::from_millis(200)); + + assert_eq!(ended, Collected::Deadline); + assert!( + started.elapsed() < Duration::from_secs(1), + "the wait must end at its deadline, took {:?}", + started.elapsed() + ); + } + + /// An exit alone is not a settle: the drains are inputs to the same wait, so + /// a shell that exited while a descendant holds a pipe still hits the + /// deadline instead of being released from it. + #[test] + fn test_exit_without_drains_still_hits_the_deadline() { + let (tx, events) = std::sync::mpsc::channel(); + tx.send(Settled::Exited(Ok(exit_status_zero()))).unwrap(); + + let started = Instant::now(); + let mut settle = Settle::default(); + let ended = settle.collect(&events, started + Duration::from_millis(200)); + + assert_eq!( + ended, + Collected::Deadline, + "a leader exit must not complete the settle while a drain is outstanding" + ); + assert!(settle.status.is_some(), "the exit status must be retained"); + assert!(started.elapsed() < Duration::from_secs(1)); + } - /// Output within the cap is passed through byte-for-byte — no marker, no loss. + /// The settle completes only when the exit and both drains have arrived, and + /// it is resumable: state folded under the first deadline carries into the + /// post-kill one. #[test] - fn test_truncate_output_leaves_short_output_untouched() { - let short = "a".repeat(1536); + fn test_settle_completes_on_exit_plus_both_drains_and_resumes() { + let (tx, events) = std::sync::mpsc::channel(); + tx.send(Settled::Drained).unwrap(); + + let mut settle = Settle::default(); + assert_eq!( + settle.collect(&events, Instant::now() + Duration::from_millis(50)), + Collected::Deadline + ); + + tx.send(Settled::Exited(Ok(exit_status_zero()))).unwrap(); + tx.send(Settled::Drained).unwrap(); + + assert_eq!( + settle.collect(&events, Instant::now() + Duration::from_secs(5)), + Collected::Complete, + "the second collect must build on the first's state, not restart it" + ); + } - assert_eq!(truncate_output(short.clone()), short); + /// Exit status of a trivially successful command, for driving `Settle` + /// without a real install. + fn exit_status_zero() -> std::process::ExitStatus { + std::process::Command::new("true") + .status() + .expect("run `true`") } - /// Over the cap, both ends survive and the middle is replaced by a marker - /// naming the omitted byte count — the head keeps the command's opening - /// context and the tail keeps the error that usually trails. + /// A shell can exit while a descendant it left behind still holds the + /// inherited output pipes. If the exit released the drains from the + /// deadline, that descendant would hold the install — and the per-runtime + /// concurrency guard behind it — open indefinitely, which is exactly the + /// failure the ceiling exists to prevent. The leader here exits in + /// milliseconds; only the descendant outlives the ceiling. + #[cfg(unix)] #[test] - fn test_truncate_output_keeps_head_and_tail_with_marker() { - let input = format!( - "{}{}{}", - "H".repeat(512), - "M".repeat(4000), - "T".repeat(1024) + fn test_promptly_exited_leader_with_a_pipe_holding_descendant_still_obeys_the_ceiling() { + let dir = tempfile::tempdir().expect("tempdir"); + let pidfile = dir.path().join("lingering.pid"); + let child = spawn_group_leader(&format!( + "sh -c 'echo $$ > {pid}; sleep 120' & exit 3", + pid = pidfile.display() + )); + + let started = Instant::now(); + let outcome = await_install_child("cli", "install", child, Duration::from_secs(2), None); + + assert!( + started.elapsed() < Duration::from_secs(30), + "a descendant holding the pipe must not outlast the ceiling, took {:?}", + started.elapsed() ); + assert_eq!( + outcome.step.exit_code, + Some(3), + "the leader's real status outranks the ceiling's verdict once it is known" + ); + // The deadline must still reach the kill on this path: a leader exit that + // skipped termination would leave the descendant running with the pipes + // open, which is the defect itself rather than a detail of it. + let pid = recorded_pid(&pidfile); + assert!( + await_death(pid), + "descendant {pid} survived — a leader exit must not skip the ceiling's kill" + ); + } - let out = truncate_output(input); + /// Wait up to 3s for `pid` to disappear. + #[cfg(unix)] + fn await_death(pid: u32) -> bool { + for _ in 0..30 { + if !crate::managed_agents::process_is_running(pid) { + return true; + } + std::thread::sleep(Duration::from_millis(100)); + } + false + } + + /// Read the pid a test descendant recorded for itself. + #[cfg(unix)] + fn recorded_pid(pidfile: &std::path::Path) -> u32 { + for _ in 0..50 { + if let Ok(text) = std::fs::read_to_string(pidfile) { + if let Ok(pid) = text.trim().parse() { + return pid; + } + } + std::thread::sleep(Duration::from_millis(100)); + } + panic!("the descendant never recorded its pid at {pidfile:?}"); + } + + /// The install shell is a process-group leader, and its descendants inherit + /// the output pipes. Killing only the leader leaves them running and the + /// drains blocked on a pipe nobody will close, so the ceiling kills the + /// whole group. + #[cfg(unix)] + #[test] + fn test_ceiling_kills_descendants_holding_the_output_pipe() { + let dir = tempfile::tempdir().expect("tempdir"); + let pidfile = dir.path().join("descendant.pid"); + let child = spawn_group_leader(&format!( + "sh -c 'echo $$ > {pid}; sleep 60' & echo leader-up; sleep 60", + pid = pidfile.display() + )); + + let started = Instant::now(); + let outcome = await_install_child("cli", "install", child, Duration::from_secs(5), None); + let result = &outcome.step; - assert!(out.starts_with(&"H".repeat(512))); - assert!(out.ends_with(&"T".repeat(1024))); + assert!(!result.success); assert!( - out.contains("... (4000 bytes omitted) ..."), - "marker must name the omitted byte count, got: {out}" + started.elapsed() < Duration::from_secs(30), + "the drains must not block on a descendant's inherited pipe" + ); + + let pid = recorded_pid(&pidfile); + assert!( + await_death(pid), + "descendant {pid} survived the ceiling kill — the group was not signalled" ); } - /// Truncation must not split a multi-byte character. Cutting mid-codepoint - /// would panic on the slice; the boundary floor prevents it. + /// Escalation must key off the group, not the leader: a descendant that + /// ignores SIGTERM outlives the leader, and if SIGKILL is skipped because + /// the leader is gone it keeps running with the output pipes open — past the + /// ceiling, and past the concurrency guard that blocks the next install. + #[cfg(unix)] #[test] - fn test_truncate_output_does_not_split_multibyte_characters() { - // "é" is 2 bytes, so every candidate cut index lands mid-character. - let input = "é".repeat(4000); + fn test_ceiling_kills_sigterm_ignoring_descendant() { + let dir = tempfile::tempdir().expect("tempdir"); + let pidfile = dir.path().join("stubborn.pid"); + // An ignored disposition survives exec, so the descendant's own `sleep` + // ignores SIGTERM too — nothing in that subtree dies without SIGKILL. + let child = spawn_group_leader(&format!( + "sh -c 'trap \"\" TERM; echo $$ > {pid}; sleep 60' & echo leader-up; sleep 60", + pid = pidfile.display() + )); + + let started = Instant::now(); + let outcome = await_install_child("cli", "install", child, Duration::from_secs(5), None); + let result = &outcome.step; - let out = truncate_output(input); + assert!(!result.success); + assert!( + started.elapsed() < Duration::from_secs(30), + "a SIGTERM-ignoring descendant must not hold the ceiling open" + ); - assert!(out.contains("bytes omitted"), "input must exceed the cap"); - assert!(!out.contains('\u{fffd}'), "no replacement chars: {out}"); + let pid = recorded_pid(&pidfile); + assert!( + await_death(pid), + "SIGTERM-ignoring descendant {pid} survived — escalation followed the leader, not the group" + ); } } diff --git a/desktop/src-tauri/src/commands/agent_discovery/install_report.rs b/desktop/src-tauri/src/commands/agent_discovery/install_report.rs new file mode 100644 index 0000000000..24bcd3456a --- /dev/null +++ b/desktop/src-tauri/src/commands/agent_discovery/install_report.rs @@ -0,0 +1,595 @@ +//! Where an install's output goes: the install log file (complete history) and +//! the live output line in the UI (current progress). +//! +//! Both destinations hang off the same drain seam in +//! [`super::install_capture`], and both are best-effort: an install must never +//! fail because a log write or an event emit did. +//! +//! [`InstallReporter`] owns two explicit lifecycles, because both the log and +//! the live line are meaningless without a notion of "this run": +//! +//! * a **log session**, started once per run, which keeps the previous run's +//! file as `.1` and writes this run's header; and +//! * a **live-event sequence**, monotonic across the whole install, which is +//! what lets the UI drop a superseded line. A per-command retry number +//! cannot do that job — it restarts at 1 for every step. + +use std::io::Write; +use std::path::{Path, PathBuf}; +use std::sync::atomic::{AtomicU64, Ordering}; +use std::sync::{Arc, Mutex, RwLock}; +use std::time::{Duration, Instant}; + +use serde::Serialize; + +use super::install_capture::{LineObserver, Throttle}; +use crate::managed_agents::{InstallRuntimeResult, InstallStepResult}; + +/// One install command's result: what the UI shows, and the log-scale copy of +/// the same output for the log file. +pub(super) struct InstallOutcome { + pub(super) step: InstallStepResult, + pub(super) log_stdout: String, + pub(super) log_stderr: String, +} + +impl InstallOutcome { + /// A step Buzz synthesized rather than ran — a failed prerequisite, or the + /// post-install verification. Its own message is the whole record. + pub(super) fn synthesized(step: InstallStepResult) -> Self { + Self { + log_stdout: step.stdout.clone(), + log_stderr: step.stderr.clone(), + step, + } + } +} + +/// Payload of the `acp-install-output` event. +/// +/// `seq` is monotonic across the entire install, so the UI can drop a line a +/// later step or attempt has already superseded. The retry number cannot serve +/// as that key: it restarts at 1 for every step, so a step that succeeded on +/// attempt 2 would make the next step's attempt-1 output look stale and freeze +/// the display. +/// +/// `line: None` is the *start signal*: an attempt is beginning and the displayed +/// line must clear now. It is emitted unthrottled, because the point is that +/// stale output stops being shown before the new work prints anything. +#[derive(Serialize, Clone, Debug)] +pub(super) struct InstallOutputEvent { + pub(super) runtime_id: String, + pub(super) seq: u64, + pub(super) line: Option, +} + +/// Emits one live output event. Boxed rather than holding an `AppHandle` so the +/// reporter is constructible — and assertable — without a Tauri app. +type EmitEvent = Arc; + +/// Literal secret values scrubbed out of everything this module publishes, in +/// addition to the shapes [`crate::managed_agents::redact_secrets_with`] +/// recognises on its own. +type Secrets = Arc>; + +/// At most four live-output events per second. Coalescing *holds* the newest +/// line rather than dropping it: a burst that ends just before the window +/// closes would otherwise leave the display showing a line the install had +/// already moved past. +const LIVE_LINE_INTERVAL: Duration = Duration::from_millis(250); + +pub(super) struct InstallReporter { + log: Option, + /// `None` when nothing is listening, which is also what makes + /// [`InstallReporter::line_observer`] `None` — the drain then skips line + /// reassembly entirely instead of doing it for no one. + live: Option, + secrets: Secrets, +} + +impl Drop for InstallReporter { + /// Serialise deactivation against in-flight publications: take the + /// exclusive lifecycle write lock, mark the run as inactive, and return — + /// all before the per-runtime concurrency guard releases. + /// + /// Every drain thread holds the shared read guard from its admission check + /// through its `(self.emit)(...)` call, so the write lock here blocks until + /// every in-flight publication has finished. After this returns, `active` + /// is `false` under an exclusive write, and any thread that attempts a new + /// `offer` will read `false` under a read lock and return without emitting. + /// + /// The ordering guarantee: `reporter` is declared after `_guard` in + /// `install_acp_runtime_blocking` (line 311 vs line 306), so Rust drops + /// `reporter` first in reverse-declaration order — deactivation completes + /// before the runtime guard releases and a new install can start. + fn drop(&mut self) { + if let Some(live) = &self.live { + if let Ok(mut active) = live.lifecycle.write() { + *active = false; + } + } + } +} + +impl InstallReporter { + /// The reporter a real install run uses: it starts this run's log session + /// and emits live output events through `app`. + /// + /// `runtime_id` must already be the canonical id from the runtime catalog — + /// the log path is built from it, so resolving it first is what keeps a raw + /// command argument out of a filename. + /// + /// A log that cannot be resolved or opened degrades to no log rather than + /// failing the install: a user with a broken app-data directory still needs + /// the install itself to work. + pub(super) fn for_run(app: &tauri::AppHandle, runtime_id: &str) -> Self { + // Read from the app's own package info rather than the frontend's + // `getVersion` plugin call: the header is written on the Rust side, and + // this cannot fail or be mocked out from under the log. + let app_version = app.package_info().version.to_string(); + let log = crate::managed_agents::storage::install_log_path(app, runtime_id) + .ok() + .and_then(|path| InstallLog::start(&path, runtime_id, &app_version)); + let app = app.clone(); + let emit: EmitEvent = Arc::new(move |event| { + use tauri::Emitter; + let _ = app.emit("acp-install-output", event); + }); + Self::new(runtime_id, log, Some(emit)) + } + + fn new(runtime_id: &str, log: Option, emit: Option) -> Self { + // Snapshot the environment's secrets once, at construction: the install + // inherits this environment, so anything it echoes came from here. + Self::with_secrets(runtime_id, log, emit, env_secret_values()) + } + + /// The reporter over an explicit secret set, which is what makes the + /// scrubbing assertable: a test can name a proxy credential without + /// exporting a real `HTTPS_PROXY` into the process every HTTP client in the + /// suite would then read. + fn with_secrets( + runtime_id: &str, + log: Option, + emit: Option, + secrets: Vec, + ) -> Self { + let secrets: Secrets = Arc::new(secrets); + let live = emit.map(|emit| Live { + runtime_id: Arc::from(runtime_id), + emit, + throttle: Arc::new(Throttle::new(LIVE_LINE_INTERVAL)), + seq: Arc::new(AtomicU64::new(0)), + lifecycle: Arc::new(RwLock::new(true)), + secrets: Arc::clone(&secrets), + }); + Self { log, live, secrets } + } + + /// The log file to point the user at, or `None` when this run has no log — + /// the failure message then omits the pointer rather than naming a file + /// that does not exist. + pub(super) fn log_path(&self) -> Option { + Some(self.log.as_ref()?.path.display().to_string()) + } + + /// A failed install carrying the steps recorded so far and the log holding + /// their full history. Every early return in the install shapes its result + /// here, so none can forget the log pointer the failure message needs. + pub(super) fn failed(&self, steps: Vec) -> InstallRuntimeResult { + InstallRuntimeResult { + success: false, + steps, + restarted_count: 0, + failed_restart_count: 0, + log_path: self.log_path(), + } + } + + /// Mark the start of one executed attempt: clear whatever line the previous + /// attempt left on screen, and start this attempt's clock. + /// + /// The clear is emitted unthrottled and reopens the rate window, so the new + /// attempt's first line cannot be swallowed by the previous attempt's. + /// Without this signal the prior attempt's last line — typically the failure + /// that caused the retry — sits under the spinner through the backoff and + /// through a silent next attempt. + pub(super) fn start_attempt(&self) { + if let Some(log) = &self.log { + log.mark_attempt_start(); + } + if let Some(live) = &self.live { + live.throttle.restart(); + live.publish(None); + } + } + + /// Observer for one attempt's drains, or `None` when nothing is listening. + pub(super) fn line_observer(&self) -> Option { + let live = self.live.clone()?; + Some(Arc::new(move |line: &str| live.offer(line))) + } + + /// Record one executed attempt of a step, returning the step with secrets + /// scrubbed out of the output the UI will render. + /// + /// The scrub happens here rather than at the construction sites because + /// every executed step reaches the caller through this function — the + /// timeout path, the status-check failure, and the ordinary exit all build + /// their `InstallStepResult` straight from the captures. + pub(super) fn record_attempt( + &self, + attempt: u32, + outcome: InstallOutcome, + ) -> InstallStepResult { + // The drains are finished, so a line the throttle is still holding is + // this attempt's last and nothing is coming to replace it. + if let Some(live) = &self.live { + live.flush_pending(); + } + self.write_record(Some(attempt), &outcome); + self.redacted_step(outcome.step) + } + + /// Push a synthesized step onto `steps` and record it. Routing every step + /// through here is what keeps the log complete: a step that reaches the UI + /// without passing this function is invisible in the file. + pub(super) fn record_step(&self, steps: &mut Vec, step: InstallStepResult) { + self.write_record(None, &InstallOutcome::synthesized(step.clone())); + steps.push(self.redacted_step(step)); + } + + /// Scrub the frontend-visible fields of a step. The failure message the UI + /// builds renders `stderr`/`stdout` and the hint verbatim, so they need the + /// same scrubbing as the log record and the live line — the log is not the + /// only place an install's output is read. + fn redacted_step(&self, mut step: InstallStepResult) -> InstallStepResult { + step.command = redact(&step.command, &self.secrets); + step.stdout = redact(&step.stdout, &self.secrets); + step.stderr = redact(&step.stderr, &self.secrets); + step.hint = step.hint.map(|hint| redact(&hint, &self.secrets)); + step + } + + /// Append one record. Best-effort by contract: a full disk or a revoked + /// permission degrades the diagnostics, it does not fail the install. + fn write_record(&self, attempt: Option, outcome: &InstallOutcome) { + let Some(log) = &self.log else { + return; + }; + log.append(&render_record( + attempt, + log.take_attempt_elapsed(), + outcome, + &self.secrets, + )); + } +} + +/// The shared half of the reporter — everything a drain thread's observer needs, +/// owned rather than borrowed so an observer can outlive the call that made it. +#[derive(Clone)] +struct Live { + runtime_id: Arc, + emit: EmitEvent, + throttle: Arc, + seq: Arc, + /// Lifecycle lock: `true` while the run is active, `false` once + /// `InstallReporter` has been dropped. + /// + /// Drain threads hold a **shared read guard** from the admission check + /// through the `(self.emit)(...)` call, making the admit-and-publish pair + /// atomic with respect to deactivation. `InstallReporter::drop` takes the + /// **exclusive write guard** and sets the value to `false`; this blocks + /// until every in-flight publication finishes, then prevents any new + /// publications from starting. The write lock is held only for the flag + /// store and is released before the per-runtime concurrency guard drops, + /// so its duration is bounded by the time a single `emit` call takes — + /// microseconds to low milliseconds for the Tauri IPC broadcast. + lifecycle: Arc>, + secrets: Secrets, +} + +impl Live { + /// Offer one drained line to the rate limiter, emitting it if the window is + /// open and holding it as the newest pending line if not. + /// + /// The read guard is held from the admission check through the emit call so + /// that `InstallReporter::drop`'s write lock must wait for any in-flight + /// publication to complete before deactivating. This makes the + /// check-then-emit pair atomic with respect to shutdown. + fn offer(&self, line: &str) { + let Ok(guard) = self.lifecycle.read() else { + return; + }; + if !*guard { + return; + } + if let Some(line) = self.throttle.offer(line, Instant::now()) { + self.publish_under_guard(line); + } + // `guard` drops here, releasing the read lock after publication. + } + + fn flush_pending(&self) { + let Ok(guard) = self.lifecycle.read() else { + return; + }; + if !*guard { + return; + } + if let Some(line) = self.throttle.take_pending() { + self.publish_under_guard(line); + } + // `guard` drops here, releasing the read lock after publication. + } + + /// Emit `line` now, bypassing the rate window and the lifecycle lock. + /// + /// Only called from `InstallReporter` methods that run on the reporter + /// itself (never from detached drain threads), so no lifecycle guard is + /// needed — the reporter is alive by definition when its own methods run. + fn publish(&self, line: Option) { + (self.emit)(InstallOutputEvent { + runtime_id: self.runtime_id.to_string(), + seq: self.seq.fetch_add(1, Ordering::Relaxed), + line: line.map(|line| redact(&line, &self.secrets)), + }); + } + + /// Publish `line` while already holding a read guard on `lifecycle`. The + /// caller is responsible for checking `active` before calling this. + fn publish_under_guard(&self, line: String) { + (self.emit)(InstallOutputEvent { + runtime_id: self.runtime_id.to_string(), + seq: self.seq.fetch_add(1, Ordering::Relaxed), + line: Some(redact(&line, &self.secrets)), + }); + } +} + +/// This run's log file: one session, opened once, appended to per record. +struct InstallLog { + path: PathBuf, + /// When the attempt currently running started, so its record can name its + /// own duration. A 15-minute ceiling is only diagnosable if the file says + /// how long each attempt actually took. + attempt_start: Mutex>, +} + +impl InstallLog { + /// Start this run's session, or `None` if the file cannot be opened. + /// + /// Rotation happens here, once per run, rather than per record: a run either + /// gets its own file or it gets no log at all, so two runs are never + /// interleaved in one file. + /// + /// The header identifies the environment the run happened in, not just the + /// run: a Windows install failure and a macOS one on the same runtime are + /// different bugs, and a stale app version explains a failure that no longer + /// reproduces. + fn start(path: &Path, runtime_id: &str, app_version: &str) -> Option { + let mut file = crate::managed_agents::storage::start_install_log_session(path).ok()?; + let _ = file.write_all( + format!( + "=== install run runtime={runtime_id} app={app_version} os={} started={}\n", + std::env::consts::OS, + chrono::Utc::now().to_rfc3339() + ) + .as_bytes(), + ); + Some(Self { + path: path.to_path_buf(), + attempt_start: Mutex::new(None), + }) + } + + fn mark_attempt_start(&self) { + if let Ok(mut start) = self.attempt_start.lock() { + *start = Some(Instant::now()); + } + } + + /// How long the attempt being recorded ran, consumed so a later record + /// cannot reuse it. `None` for a synthesized step, which never ran. + fn take_attempt_elapsed(&self) -> Option { + Some(self.attempt_start.lock().ok()?.take()?.elapsed()) + } + + fn append(&self, record: &str) { + if let Ok(mut file) = crate::managed_agents::storage::open_install_log_file(&self.path) { + let _ = file.write_all(record.as_bytes()); + } + } +} + +/// One self-contained record. Each is capped independently by the log-scale +/// capture that produced it, so an early attempt that printed megabytes cannot +/// push a later attempt — or the verification step that explains the failure — +/// out of the file. +fn render_record( + attempt: Option, + elapsed: Option, + outcome: &InstallOutcome, + secrets: &Secrets, +) -> String { + let step = &outcome.step; + let attempt = attempt.map_or_else(|| "-".to_string(), |n| n.to_string()); + let exit = step + .exit_code + .map_or_else(|| "none".to_string(), |code| code.to_string()); + let elapsed = elapsed.map_or_else( + || "-".to_string(), + |elapsed| format!("{:.1}s", elapsed.as_secs_f64()), + ); + let mut record = format!( + "=== {} step={} attempt={attempt} success={} exit={exit} elapsed={elapsed}\n$ {}\n", + chrono::Utc::now().to_rfc3339(), + step.step, + step.success, + redact(&step.command, secrets), + ); + for (label, text) in [ + ("stdout", &outcome.log_stdout), + ("stderr", &outcome.log_stderr), + ] { + if !text.trim().is_empty() { + record.push_str(&format!("--- {label} ---\n{}\n", redact(text, secrets))); + } + } + if let Some(hint) = &step.hint { + record.push_str(&format!("--- hint ---\n{}\n", redact(hint, secrets))); + } + record +} + +/// Scrub secrets before anything reaches disk or the UI. The log is written +/// unattended and the live line is rendered verbatim, so scrubbing happens at +/// the write, not at the read. +fn redact(text: &str, secrets: &Secrets) -> String { + let extras: Vec<&str> = secrets.iter().map(String::as_str).collect(); + crate::managed_agents::redact_secrets_with(text, &extras) +} + +/// Values of environment variables whose *name* marks them as secret. +/// +/// An install inherits Buzz's environment and installers echo it back — npm +/// prints the resolved registry config on an auth failure, and a shell that +/// traces its commands prints every expansion. Without this, only the +/// hard-coded key shapes would be scrubbed, so a plain `NPM_TOKEN` or +/// `ANTHROPIC_API_KEY` would land in the file in clear text. +fn env_secret_values() -> Vec { + secret_values_from(std::env::vars()) +} + +/// The secret-bearing part of each variable that carries one. +/// +/// Split from [`env_secret_values`] so the classification is assertable without +/// mutating the process environment — setting a real `HTTPS_PROXY` in a test +/// would be read by every HTTP client the rest of the suite builds. +/// +/// Three kinds of variable are recognised, because they need different +/// treatment: +/// +/// * **URL-valued variables**, where only the userinfo is the credential; +/// * **exactly named credentials**, whose whole value is the secret; and +/// * **name-marked secrets**, matched by a marker substring. +fn secret_values_from(vars: impl IntoIterator) -> Vec { + vars.into_iter() + .filter_map(|(name, value)| { + let name = name.to_ascii_uppercase(); + if URL_CREDENTIAL_VAR_NAMES.contains(&name.as_str()) { + // Only the userinfo, so the endpoint itself stays named in the + // record: an install that fails against a proxy or a private + // registry is diagnosable only if the log still says which one + // it went through, and the host and port are not the secret. + // Redacting the whole value would erase that while protecting + // nothing more. + return url_userinfo(&value).map(str::to_string); + } + if SECRET_VAR_NAMES.contains(&name.as_str()) { + // Deliberately not subject to the 8-byte floor below: an exact + // name is a fact, not the guess the marker rule makes, so there + // is nothing for a floor to protect against. npm's one-time + // password is six digits and is a credential at that length — + // short, but still above the four-byte minimum + // [`crate::managed_agents::redact_secrets_with`] applies, so it + // survives to be scrubbed. + return (!value.is_empty()).then_some(value); + } + // A value under 8 bytes is more likely a flag like `true` or a + // version than a credential, and scrubbing those makes ordinary + // output unreadable. + (value.len() >= 8 && name_marks_secret(&name)).then_some(value) + }) + .collect() +} + +/// Variables whose value is a URL that may embed a credential in its userinfo. +/// +/// npm's own `npm_config_*` aliases are here too: npm resolves them ahead of +/// the conventional names and echoes the result from `npm config list`, and +/// none of these names carries a marker [`name_marks_secret`] would catch. +/// Matching is case-insensitive because the caller uppercases the name first, +/// which is what npm's lowercase spelling needs. +const URL_CREDENTIAL_VAR_NAMES: &[&str] = &[ + "HTTP_PROXY", + "HTTPS_PROXY", + "ALL_PROXY", + "NPM_CONFIG_PROXY", + "NPM_CONFIG_HTTPS_PROXY", + "NPM_CONFIG_REGISTRY", +]; + +/// Variables whose whole value is a credential, recognised by exact name. +/// +/// These are npm's supported credential settings, whose names carry no marker +/// [`name_marks_secret`] would catch. They are listed exactly rather than +/// matched on `KEY` or `AUTH` substrings: those occur throughout an ordinary +/// environment, and scrubbing on them would delete unrelated values from the +/// whole log. +/// +/// * `NPM_CONFIG_KEY` — the PEM client key used to reach a registry. +/// * `NPM_CONFIG__AUTH` — the base64 basic-auth blob (npm's own double +/// underscore, matching the `_auth` setting). +/// * `NPM_CONFIG_OTP` — the registry one-time password. +const SECRET_VAR_NAMES: &[&str] = &["NPM_CONFIG_KEY", "NPM_CONFIG__AUTH", "NPM_CONFIG_OTP"]; + +/// Whether an environment variable's name marks its value as a credential. +/// +/// Keyed on the name because a secret's *value* has no reliable shape. The +/// markers avoid substrings that occur in non-secret names: `AUTH` is left out +/// because it matches `GIT_AUTHOR_NAME`, whose value is a person's name, and +/// personal access tokens match on `_PAT` as a *suffix* rather than a substring +/// — `contains("_PAT")` would match every `*_PATH` variable on the system and +/// scrub directory names out of the whole log. +fn name_marks_secret(name: &str) -> bool { + const SECRET_NAME_MARKERS: &[&str] = &[ + "TOKEN", + "SECRET", + "PASSWORD", + "PASSWD", + "APIKEY", + "API_KEY", + "PRIVATE_KEY", + "ACCESS_KEY", + "CREDENTIAL", + ]; + name.ends_with("_PAT") + || SECRET_NAME_MARKERS + .iter() + .any(|marker| name.contains(marker)) +} + +/// The `user:password` credential embedded in a URL, if it has one. +/// +/// Parsed rather than pattern-matched so a URL with no credential — the common +/// case — contributes nothing to scrub. The last `@` in the +/// authority separates userinfo from host, so a password containing an +/// encoded `@` still splits correctly. +/// +/// A bare username with no password is not treated as a credential: it is not +/// secret on its own, and scrubbing it would erase every occurrence of a word +/// like `user` from the whole record. +fn url_userinfo(value: &str) -> Option<&str> { + let authority = value + .split_once("://")? + .1 + .split(['/', '?', '#']) + .next() + .unwrap_or_default(); + let userinfo = authority.rsplit_once('@')?.0; + userinfo.contains(':').then_some(userinfo) +} + +#[cfg(test)] +#[path = "install_report_test_support.rs"] +mod test_support; + +#[cfg(test)] +#[path = "install_report_tests.rs"] +mod tests; + +#[cfg(test)] +#[path = "install_report_redaction_tests.rs"] +mod redaction_tests; diff --git a/desktop/src-tauri/src/commands/agent_discovery/install_report_redaction_tests.rs b/desktop/src-tauri/src/commands/agent_discovery/install_report_redaction_tests.rs new file mode 100644 index 0000000000..ce97559616 --- /dev/null +++ b/desktop/src-tauri/src/commands/agent_discovery/install_report_redaction_tests.rs @@ -0,0 +1,480 @@ +use super::test_support::*; +use super::*; + +// ── redaction ──────────────────────────────────────────────────────────────── + +/// Secrets that an installer echoed must not land on disk. The log is written +/// unattended, so scrubbing happens at the write, not at the read. +#[test] +fn test_log_redacts_secrets_before_writing() { + let h = harness(); + let leak = "npm ERR! token nsec1qqqqqqqqqqsecretvalue failed"; + + h.reporter.record_attempt(1, outcome("cli", false, leak)); + + let log = h.log_contents(); + assert!(!log.contains("nsec1qqqqqqqqqqsecretvalue"), "got: {log}"); + assert!(log.contains("[REDACTED]"), "got: {log}"); +} + +/// The environment's own secrets are scrubbed too, by *name* rather than shape. +/// An install inherits Buzz's environment and installers echo it back — npm +/// prints its resolved config on an auth failure — and a token with no +/// recognizable prefix would otherwise reach the file verbatim. +#[test] +fn test_log_redacts_an_environment_secret_with_no_recognizable_prefix() { + let secret = "0e8f31c5a4b7d296e5f1a"; + // Set before the reporter is built: the snapshot is taken at construction. + std::env::set_var("BUZZ_TEST_REGISTRY_TOKEN", secret); + let h = harness(); + std::env::remove_var("BUZZ_TEST_REGISTRY_TOKEN"); + + h.reporter.record_attempt( + 1, + outcome("cli", false, &format!("npm ERR! _authToken={secret}")), + ); + + let log = h.log_contents(); + assert!(!log.contains(secret), "got: {log}"); + assert!(log.contains("[REDACTED]"), "got: {log}"); +} + +/// A live line carries the same scrubbing as the log record. The line is +/// rendered verbatim in the UI, so a leak there is as visible as one on disk. +#[test] +fn test_a_live_line_is_redacted_before_it_is_emitted() { + let h = harness(); + + let observer = h.reporter.line_observer().expect("an observer"); + observer("fetching with token nsec1qqqqqqqqqqleaked"); + + let lines = h.lines(); + assert_eq!(lines.len(), 1); + let line = lines[0].clone().expect("a line, not a clear signal"); + assert!(!line.contains("nsec1qqqqqqqqqqleaked"), "got: {line}"); + assert!(line.contains("[REDACTED]"), "got: {line}"); +} + +// ── proxy and PAT credentials ──────────────────────────────────────────────── + +/// A proxy URL's password is a credential, but the proxy itself is diagnostic +/// information: an install that fails behind a proxy is only debuggable if the +/// record still says which proxy it went through. So the userinfo is scrubbed +/// and the host is kept. +#[test] +fn test_proxy_userinfo_is_secret_but_the_proxy_host_is_not() { + let secrets = secret_values_from([( + "HTTPS_PROXY".to_string(), + "http://corpuser:hunter2pass@proxy.example:8080".to_string(), + )]); + + assert_eq!(secrets, vec!["corpuser:hunter2pass"]); +} + +/// A proxy with no credential contributes nothing — scrubbing a bare host would +/// erase the proxy's name from every record while protecting nothing. A bare +/// username is not a credential either, and scrubbing it would delete every +/// occurrence of that word from the log. +#[test] +fn test_a_proxy_without_credentials_contributes_no_secret() { + let secrets = secret_values_from([ + ( + "HTTP_PROXY".to_string(), + "http://proxy.example:8080".to_string(), + ), + ( + "ALL_PROXY".to_string(), + "socks5://10.0.0.1:1080".to_string(), + ), + ( + "HTTPS_PROXY".to_string(), + "http://user@proxy.example:8080".to_string(), + ), + ]); + + assert!(secrets.is_empty(), "got: {secrets:?}"); +} + +/// npm reads its own `npm_config_*` aliases in preference to the conventional +/// proxy variables and prints the resolved value back, so a credential set only +/// under an alias would otherwise never enter the scrub list. npm spells them +/// in lowercase, so both cases have to classify. +#[test] +fn test_npm_proxy_aliases_are_classified_in_either_case() { + let secrets = secret_values_from([ + ( + "npm_config_proxy".to_string(), + "http://corpuser:lowerplain@proxy.example:8080".to_string(), + ), + ( + "NPM_CONFIG_PROXY".to_string(), + "http://corpuser:upperplain@proxy.example:8080".to_string(), + ), + ( + "npm_config_https_proxy".to_string(), + "http://corpuser:lowertls@proxy.example:8080".to_string(), + ), + ( + "NPM_CONFIG_HTTPS_PROXY".to_string(), + "http://corpuser:uppertls@proxy.example:8080".to_string(), + ), + ]); + + assert_eq!( + secrets, + vec![ + "corpuser:lowerplain", + "corpuser:upperplain", + "corpuser:lowertls", + "corpuser:uppertls", + ] + ); +} + +/// The alias carries the same userinfo-only policy as the conventional names: +/// a credential-less alias contributes nothing, so the proxy stays named in the +/// record. +#[test] +fn test_an_npm_proxy_alias_without_credentials_contributes_no_secret() { + let secrets = secret_values_from([ + ( + "npm_config_proxy".to_string(), + "http://proxy.example:8080".to_string(), + ), + ( + "NPM_CONFIG_HTTPS_PROXY".to_string(), + "http://user@proxy.example:8080".to_string(), + ), + ]); + + assert!(secrets.is_empty(), "got: {secrets:?}"); +} + +/// The classifier and the reporter have to agree: an alias credential that +/// classifies but never reaches the scrub list still leaks. This drives the +/// reporter with exactly what the classifier produced for an alias, and asserts +/// the log and the returned step both come back clean. +#[test] +fn test_an_npm_alias_credential_is_redacted_from_the_log_and_the_returned_step() { + let password = "hunter2pass"; + let h = harness_with_secrets(secret_values_from([( + "npm_config_proxy".to_string(), + format!("http://corpuser:{password}@proxy.example:8080"), + )])); + + let returned = h.reporter.record_attempt( + 1, + InstallOutcome { + step: InstallStepResult { + stderr: format!( + "npm ERR! proxy=http://corpuser:{password}@proxy.example:8080 tunneling failed" + ), + ..step("cli", false, "") + }, + log_stdout: String::new(), + log_stderr: format!( + "npm config: proxy = http://corpuser:{password}@proxy.example:8080" + ), + }, + ); + + let log = h.log_contents(); + assert!(!log.contains(password), "log leaked the password: {log}"); + assert!( + log.contains("proxy.example"), + "the proxy host is diagnostic and must survive: {log}" + ); + assert!( + !returned.stderr.contains(password), + "the returned step leaked the password: {}", + returned.stderr + ); +} + +// ── npm's own credential settings ──────────────────────────────────────────── + +/// npm accepts every one of its settings as an `npm_config_*` variable, so a +/// registry client key, a basic-auth blob or a one-time password can arrive +/// under a name that carries no marker. Their whole value is the credential — +/// unlike a proxy, none of it is diagnostic — and npm spells them in lowercase. +#[test] +fn test_npm_credential_configs_are_secret_in_either_case() { + let secrets = secret_values_from([ + ( + "npm_config_key".to_string(), + "-----BEGIN PRIVATE KEY-----lowerkey".to_string(), + ), + ( + "NPM_CONFIG_KEY".to_string(), + "-----BEGIN PRIVATE KEY-----upperkey".to_string(), + ), + ("npm_config__auth".to_string(), "bG93ZXJhdXRo".to_string()), + ("NPM_CONFIG__AUTH".to_string(), "dXBwZXJhdXRo".to_string()), + ("npm_config_otp".to_string(), "618243".to_string()), + ("NPM_CONFIG_OTP".to_string(), "907154".to_string()), + ]); + + assert_eq!( + secrets, + vec![ + "-----BEGIN PRIVATE KEY-----lowerkey", + "-----BEGIN PRIVATE KEY-----upperkey", + "bG93ZXJhdXRo", + "dXBwZXJhdXRo", + "618243", + "907154", + ] + ); +} + +/// An unset-but-exported credential is empty, and an empty needle would match +/// everywhere. The name being exact does not make a blank value a secret. +#[test] +fn test_an_empty_npm_credential_config_contributes_no_secret() { + let secrets = secret_values_from([ + ("NPM_CONFIG_KEY".to_string(), String::new()), + ("npm_config_otp".to_string(), String::new()), + ]); + + assert!(secrets.is_empty(), "got: {secrets:?}"); +} + +/// A private registry's URL follows the proxy policy rather than the whole-value +/// one: which registry an install talked to is exactly what a 401 or an ETIMEDOUT +/// has to be read against, so only the userinfo is the secret. +#[test] +fn test_npm_registry_userinfo_is_secret_but_the_registry_host_is_not() { + let secrets = secret_values_from([( + "npm_config_registry".to_string(), + "https://builder:hunter2pass@registry.example/api/npm/".to_string(), + )]); + + assert_eq!(secrets, vec!["builder:hunter2pass"]); +} + +/// The public registry — and any private one reached with a token header rather +/// than URL credentials — contributes nothing, so the registry stays named in +/// the record. +#[test] +fn test_an_npm_registry_without_credentials_contributes_no_secret() { + let secrets = secret_values_from([ + ( + "npm_config_registry".to_string(), + "https://registry.npmjs.org/".to_string(), + ), + ( + "NPM_CONFIG_REGISTRY".to_string(), + "https://builder@registry.example/api/npm/".to_string(), + ), + ]); + + assert!(secrets.is_empty(), "got: {secrets:?}"); +} + +/// The credential settings are matched by exact name, never by a `KEY` or +/// `AUTH` substring. Those occur throughout an ordinary environment on values +/// that are paths, agent sockets and people's names, and scrubbing them would +/// delete unrelated text from every record. +#[test] +fn test_key_and_auth_inside_a_variable_name_do_not_make_it_secret() { + let secrets = secret_values_from([ + ( + "SSH_AUTH_SOCK".to_string(), + "/tmp/ssh-agent.socket".to_string(), + ), + ("GIT_AUTHOR_NAME".to_string(), "Ada Lovelace".to_string()), + ( + "KEYCHAIN".to_string(), + "/Users/dev/Library/login.keychain".to_string(), + ), + ( + "NPM_CONFIG_KEYFILE".to_string(), + "/Users/dev/.npm/client.pem".to_string(), + ), + ]); + + assert!(secrets.is_empty(), "got: {secrets:?}"); +} + +/// The wiring, not just the classification: npm prints its resolved config on an +/// auth failure, so each of these has to be gone from the log and from the step +/// the frontend renders. The one-time password is the interesting one — at six +/// digits it is far shorter than any other secret here, and a value under four +/// bytes is dropped by the shared redactor rather than scrubbed. +#[test] +fn test_npm_credential_configs_are_redacted_from_the_log_and_the_returned_step() { + let client_key = "-----BEGIN PRIVATE KEY-----MIIEvQIBADAN"; + let auth = "YnVpbGRlcjpodW50ZXIycGFzcw=="; + let otp = "618243"; + let registry_password = "hunter2pass"; + let h = harness_with_secrets(secret_values_from([ + ("npm_config_key".to_string(), client_key.to_string()), + ("npm_config__auth".to_string(), auth.to_string()), + ("npm_config_otp".to_string(), otp.to_string()), + ( + "npm_config_registry".to_string(), + format!("https://builder:{registry_password}@registry.example/api/npm/"), + ), + ])); + + let returned = h.reporter.record_attempt( + 1, + InstallOutcome { + step: InstallStepResult { + stderr: format!("npm ERR! 401 otp={otp} _auth={auth}"), + ..step("cli", false, "") + }, + log_stdout: format!("npm config: key = {client_key}"), + log_stderr: format!( + "npm config: registry = https://builder:{registry_password}@registry.example/api/npm/" + ), + }, + ); + + let log = h.log_contents(); + for secret in [client_key, auth, otp, registry_password] { + assert!(!log.contains(secret), "log leaked {secret}: {log}"); + } + assert!( + log.contains("registry.example"), + "the registry host is diagnostic and must survive: {log}" + ); + assert!( + !returned.stderr.contains(otp) && !returned.stderr.contains(auth), + "the returned step leaked a credential: {}", + returned.stderr + ); +} + +/// `*_PATH` variables must not be mistaken for personal access tokens. A +/// `contains("_PAT")` rule would match `PATH` itself and scrub every directory +/// name out of the log, which is why the rule matches `_PAT` as a suffix. +#[test] +fn test_a_path_variable_is_not_treated_as_a_personal_access_token() { + let secrets = secret_values_from([ + ("PATH".to_string(), "/usr/local/bin:/usr/bin".to_string()), + ("GOPATH".to_string(), "/home/user/go".to_string()), + ( + "CARGO_HOME_PATH".to_string(), + "/home/user/.cargo".to_string(), + ), + ]); + + assert!(secrets.is_empty(), "got: {secrets:?}"); +} + +/// Variables named as personal access tokens are secret by name, whatever shape +/// their value has. +#[test] +fn test_pat_named_variables_are_secret() { + let secrets = secret_values_from([ + ( + "GITHUB_PAT".to_string(), + "ghp_abcdefghij0123456789".to_string(), + ), + ( + "GH_PAT".to_string(), + "github_pat_abcdefghij0123".to_string(), + ), + ]); + + assert_eq!(secrets.len(), 2, "got: {secrets:?}"); +} + +/// The whole point of the widening: a proxy password and a PAT that the +/// installer echoed reach neither the log nor the live line. +/// +/// Both are checked through the real reporter rather than the classifier, so +/// this covers the wiring — a classifier that recognises a secret the reporter +/// never consults would still leak. +#[test] +fn test_proxy_and_pat_credentials_are_redacted_from_the_log_and_the_live_line() { + let proxy_password = "hunter2pass"; + let pat = "ghp_abcdefghij0123456789"; + // The classifier's own tests cover recognising these under their real + // variable names; injecting the resulting secrets here keeps a live + // `HTTPS_PROXY` out of the process the rest of the suite shares. + let h = harness_with_secrets(vec![format!("corpuser:{proxy_password}"), pat.to_string()]); + + h.reporter.record_attempt( + 1, + outcome( + "cli", + false, + &format!( + "npm ERR! proxy=http://corpuser:{proxy_password}@proxy.example authToken={pat}" + ), + ), + ); + let observer = h.reporter.line_observer().expect("an observer"); + observer(&format!("cloning https://{pat}@github.com/org/repo")); + + let log = h.log_contents(); + assert!( + !log.contains(proxy_password), + "log leaked the proxy password: {log}" + ); + assert!(!log.contains(pat), "log leaked the PAT: {log}"); + assert!( + log.contains("proxy.example"), + "the proxy host is diagnostic and must survive: {log}" + ); + + let line = h.lines().into_iter().flatten().next().expect("a live line"); + assert!(!line.contains(pat), "live line leaked the PAT: {line}"); + assert!(line.contains("[REDACTED]"), "got: {line}"); +} + +/// The third surface: the step returned to the frontend. `getInstallErrorMessage` +/// renders the failing step's stderr verbatim, so a secret that the log and the +/// live line both scrub would still reach the user through the error dialog. +#[test] +fn test_a_returned_step_is_redacted_before_the_frontend_renders_it() { + let pat = "ghp_abcdefghij0123456789"; + let h = harness_with_secrets(vec![pat.to_string()]); + + let returned = h.reporter.record_attempt( + 1, + InstallOutcome { + step: InstallStepResult { + stdout: format!("configuring remote with {pat}"), + stderr: format!("fatal: authentication failed for token {pat}"), + hint: Some(format!("check that {pat} has the repo scope")), + ..step("cli", false, "") + }, + log_stdout: String::new(), + log_stderr: String::new(), + }, + ); + + assert!(!returned.stdout.contains(pat), "got: {}", returned.stdout); + assert!(!returned.stderr.contains(pat), "got: {}", returned.stderr); + let hint = returned.hint.expect("a hint"); + assert!(!hint.contains(pat), "got: {hint}"); +} + +/// A synthesized step reaches the frontend through the other funnel, and needs +/// the same scrubbing — the managed-node prerequisite failures are built this +/// way and carry whatever the underlying command printed. +#[test] +fn test_a_synthesized_step_is_redacted_before_it_reaches_the_caller() { + let pat = "ghp_abcdefghij0123456789"; + let h = harness_with_secrets(vec![pat.to_string()]); + let mut steps = Vec::new(); + + h.reporter.record_step( + &mut steps, + InstallStepResult { + stderr: format!("npm ERR! 401 with {pat}"), + ..step("adapter", false, "") + }, + ); + + assert_eq!(steps.len(), 1); + assert!(!steps[0].stderr.contains(pat), "got: {}", steps[0].stderr); + assert!( + steps[0].stderr.contains("[REDACTED]"), + "got: {}", + steps[0].stderr + ); +} diff --git a/desktop/src-tauri/src/commands/agent_discovery/install_report_test_support.rs b/desktop/src-tauri/src/commands/agent_discovery/install_report_test_support.rs new file mode 100644 index 0000000000..17b2789560 --- /dev/null +++ b/desktop/src-tauri/src/commands/agent_discovery/install_report_test_support.rs @@ -0,0 +1,106 @@ +//! Shared harness for the install-report test modules. +//! +//! The tests are split by concern — redaction in +//! [`super::install_report_redaction_tests`], everything else in +//! [`super::install_report_tests`] — and both drive the reporter through the +//! same harness, so it lives here rather than in either of them. + +use super::*; +use std::sync::Mutex; + +/// Stands in for the real `app.package_info().version`, which needs a Tauri app. +pub(crate) const TEST_APP_VERSION: &str = "9.9.9"; + +/// A reporter with a started log session in a temp dir, and the emitted events +/// captured. +pub(crate) struct Harness { + /// Kept alive so the log outlives the harness; a test that reuses the + /// directory for a second run takes it. + pub(crate) _dir: tempfile::TempDir, + pub(crate) log: PathBuf, + pub(crate) reporter: InstallReporter, + pub(crate) events: Arc>>, +} + +pub(crate) fn harness() -> Harness { + harness_at(None) +} + +/// A harness whose log lives in `dir`, or in a fresh temp dir when `dir` is +/// `None`. Passing a directory lets a test seed a previous run's file first. +pub(crate) fn harness_at(dir: Option) -> Harness { + harness_inner(dir, None) +} + +/// A harness whose reporter scrubs exactly `secrets`, so a proxy or PAT +/// credential can be asserted without exporting it into the process. +pub(crate) fn harness_with_secrets(secrets: Vec) -> Harness { + harness_inner(None, Some(secrets)) +} + +fn harness_inner(dir: Option, secrets: Option>) -> Harness { + let dir = dir.unwrap_or_else(|| tempfile::tempdir().expect("tempdir")); + let log = dir.path().join("install-goose.log"); + let events: Arc>> = Arc::new(Mutex::new(Vec::new())); + let emit: EmitEvent = { + let events = Arc::clone(&events); + Arc::new(move |event| events.lock().unwrap().push(event)) + }; + let started = InstallLog::start(&log, "goose", TEST_APP_VERSION); + Harness { + reporter: match secrets { + Some(secrets) => InstallReporter::with_secrets("goose", started, Some(emit), secrets), + None => InstallReporter::new("goose", started, Some(emit)), + }, + _dir: dir, + log, + events, + } +} + +/// A reporter with no log file and nothing listening — the degraded shape. +pub(crate) fn silent_reporter() -> InstallReporter { + InstallReporter::new("goose", None, None) +} + +pub(crate) fn step(name: &str, success: bool, stderr: &str) -> InstallStepResult { + InstallStepResult { + step: name.to_string(), + command: "curl … | bash".to_string(), + success, + stdout: String::new(), + stderr: stderr.to_string(), + exit_code: Some(if success { 0 } else { 1 }), + hint: None, + } +} + +/// An executed attempt whose log copy differs from the UI copy — the real shape, +/// since the two views are capped differently. +pub(crate) fn outcome(name: &str, success: bool, log_stdout: &str) -> InstallOutcome { + InstallOutcome { + step: step(name, success, ""), + log_stdout: log_stdout.to_string(), + log_stderr: String::new(), + } +} + +impl Harness { + pub(crate) fn log_contents(&self) -> String { + std::fs::read_to_string(&self.log).unwrap_or_default() + } + + /// The emitted lines in order, with a clear signal rendered as `None`. + pub(crate) fn lines(&self) -> Vec> { + self.events + .lock() + .unwrap() + .iter() + .map(|e| e.line.clone()) + .collect() + } + + pub(crate) fn events(&self) -> Vec { + self.events.lock().unwrap().clone() + } +} diff --git a/desktop/src-tauri/src/commands/agent_discovery/install_report_tests.rs b/desktop/src-tauri/src/commands/agent_discovery/install_report_tests.rs new file mode 100644 index 0000000000..c286b618b6 --- /dev/null +++ b/desktop/src-tauri/src/commands/agent_discovery/install_report_tests.rs @@ -0,0 +1,564 @@ +use super::test_support::*; +use super::*; +use crate::commands::agent_discovery::install_capture::{drain_into, Capture}; + +// ── the log records history the UI does not keep ───────────────────────────── + +/// Every attempt is recorded, not just the one the UI surfaces. Reproducing an +/// install failure means seeing whether attempts 1 and 2 failed the same way. +#[test] +fn test_log_records_every_attempt_not_only_the_last() { + let h = harness(); + + h.reporter + .record_attempt(1, outcome("cli", false, "attempt-one-output")); + h.reporter + .record_attempt(2, outcome("cli", false, "attempt-two-output")); + + let log = h.log_contents(); + assert!(log.contains("attempt-one-output"), "got: {log}"); + assert!(log.contains("attempt-two-output"), "got: {log}"); + assert!( + log.contains("attempt=1") && log.contains("attempt=2"), + "got: {log}" + ); +} + +/// A first attempt that printed a huge amount must not push later records out of +/// the file. Records are capped individually by the log-scale capture that +/// produced them, so the run's total is bounded by steps × attempts × cap rather +/// than by one runaway attempt. +/// +/// The flood goes through a real [`Capture`] rather than straight into the +/// record, so this exercises the cap that actually bounds a record. +#[test] +fn test_first_attempt_overflow_does_not_erase_later_records() { + let h = harness(); + // Through the real drain, so the record is bounded by the cap that bounds a + // production record rather than by a string this test chose. + let capture = Capture::new(); + drain_into(vec![b'F'; 4 * 1024 * 1024].as_slice(), &capture, None); + + h.reporter + .record_attempt(1, outcome("cli", false, &capture.log())); + h.reporter + .record_attempt(2, outcome("cli", false, "second-attempt-detail")); + h.reporter.record_step( + &mut Vec::new(), + step("verify", false, "verification-detail"), + ); + + let log = h.log_contents(); + assert!( + log.contains("bytes omitted at cap"), + "the flooded record must be marked as cut" + ); + assert!( + log.contains("second-attempt-detail"), + "a later attempt must survive an earlier flood" + ); + assert!( + log.contains("verification-detail"), + "the synthesized step explaining the failure must survive too" + ); + assert!( + log.len() < 4 * 1024 * 1024, + "4MiB of first-attempt output must not reach the file, got {} bytes", + log.len() + ); +} + +/// A step Buzz synthesizes — a failed prerequisite, or post-install +/// verification — reaches the log as well as the UI. `record_step` is the only +/// path that guarantees this, which is why callers use it instead of +/// `steps.push`. +#[test] +fn test_recording_a_synthesized_step_logs_it_and_keeps_it_for_the_ui() { + let h = harness(); + let mut steps = Vec::new(); + + h.reporter + .record_step(&mut steps, step("verify", false, "still-not-usable")); + + assert_eq!(steps.len(), 1, "the UI must still receive the step"); + assert!(h.log_contents().contains("still-not-usable")); +} + +// ── one file per run ───────────────────────────────────────────────────────── + +/// A run opens with a header naming the runtime and the environment the run +/// happened in, so a file holding one run of several steps is identifiable as +/// that run rather than a stream of records — and a failure report says which +/// app version and OS produced it without a second round trip to the user. +#[test] +fn test_a_run_opens_with_a_header_naming_the_runtime_app_version_and_os() { + let h = harness(); + let log = h.log_contents(); + + assert!( + log.starts_with(&format!( + "=== install run runtime=goose app={TEST_APP_VERSION} os={} started=", + std::env::consts::OS + )), + "got: {log}" + ); +} + +/// A new run does not append to the previous run's file: it starts a fresh one +/// and keeps the previous as `.1`. Reading a log has to mean reading one run — +/// records accumulated across runs are indistinguishable from retries within +/// one. +#[test] +fn test_a_new_run_starts_a_fresh_file_and_keeps_the_previous_as_dot_one() { + let first = harness(); + first + .reporter + .record_attempt(1, outcome("cli", false, "previous-run-output")); + let previous = first.log.clone(); + let dir = first._dir; + drop(first.reporter); + + let second = harness_at(Some(dir)); + second + .reporter + .record_attempt(1, outcome("cli", false, "current-run-output")); + + let log = second.log_contents(); + assert!(log.contains("current-run-output"), "got: {log}"); + assert!( + !log.contains("previous-run-output"), + "the new run's file must not carry the previous run's records: {log}" + ); + let rotated = std::fs::read_to_string(previous.with_extension("log.1")).expect("read .1"); + assert!( + rotated.contains("previous-run-output"), + "the previous run must remain readable as .1: {rotated}" + ); +} + +/// Each executed attempt records how long it ran. A 15-minute ceiling is only +/// diagnosable if the file says which attempt consumed the time. +#[test] +fn test_an_executed_attempt_records_its_own_duration() { + let h = harness(); + + h.reporter.start_attempt(); + h.reporter.record_attempt(1, outcome("cli", true, "done")); + h.reporter + .record_step(&mut Vec::new(), step("verify", true, "")); + + let log = h.log_contents(); + assert!( + log.contains("attempt=1") && log.contains("elapsed=0."), + "an executed attempt must carry its duration: {log}" + ); + assert!( + log.contains("attempt=- ") && log.contains("elapsed=-"), + "a synthesized step never ran, so it has no duration: {log}" + ); +} + +// ── the log pointer ────────────────────────────────────────────────────────── + +/// The path is available as soon as the run's session opens, because the file +/// exists from that moment — the header is already in it. A failure before any +/// step ran still points the user at a real file. +#[test] +fn test_log_path_is_available_from_the_start_of_the_run() { + let h = harness(); + + assert_eq!(h.reporter.log_path(), Some(h.log.display().to_string())); +} + +/// A reporter with no log — an unresolvable app-data directory — records +/// nothing and reports no path, but must not panic or fail the install. +#[test] +fn test_reporter_without_a_log_records_nothing_and_reports_no_path() { + let reporter = silent_reporter(); + let mut steps = Vec::new(); + + reporter.start_attempt(); + reporter.record_attempt(1, outcome("cli", false, "output")); + reporter.record_step(&mut steps, step("verify", false, "detail")); + + assert_eq!(reporter.log_path(), None); + assert_eq!(steps.len(), 1, "the UI path is unaffected by a missing log"); +} + +/// A log path inside a directory that no longer exists cannot open a session, so +/// the run degrades to no log rather than failing. +#[test] +fn test_an_unopenable_log_degrades_to_no_log() { + let path = PathBuf::from("/nonexistent-dir-for-test/install-goose.log"); + + assert!(InstallLog::start(&path, "goose", TEST_APP_VERSION).is_none()); +} + +// ── live output line ───────────────────────────────────────────────────────── + +/// Lines carry an install-wide monotonic sequence number, so the UI can order +/// them across steps and attempts — which a per-step retry number cannot do. +#[test] +fn test_emitted_lines_carry_their_runtime_and_a_monotonic_sequence() { + let h = harness(); + + let observer = h.reporter.line_observer().expect("an observer"); + observer("downloading"); + h.reporter.start_attempt(); + + let events = h.events(); + assert_eq!(events.len(), 2); + assert!(events.iter().all(|e| e.runtime_id == "goose")); + assert_eq!(events[0].line.as_deref(), Some("downloading")); + assert_eq!(events[0].seq, 0); + assert_eq!( + events[1].seq, 1, + "the clear signal takes the next sequence number, so it cannot be \ + mistaken for a stale event" + ); +} + +/// Starting an attempt clears the display first: the previous attempt's last +/// line is typically the failure that caused the retry, and leaving it under the +/// spinner through the backoff shows the user the past as if it were current. +#[test] +fn test_starting_an_attempt_clears_the_displayed_line() { + let h = harness(); + let observer = h.reporter.line_observer().expect("an observer"); + observer("download failed"); + + h.reporter.start_attempt(); + + assert_eq!( + h.lines(), + vec![Some("download failed".to_string()), None], + "the attempt boundary must emit a clear" + ); +} + +/// The clear is not rate-limited, and it reopens the window: a new attempt's +/// first line goes out immediately even if it arrives inside the previous +/// attempt's window. This is the case the throttle used to swallow entirely. +#[test] +fn test_a_new_attempts_first_line_is_emitted_even_inside_the_previous_window() { + let h = harness(); + let observer = h.reporter.line_observer().expect("an observer"); + observer("attempt one failed"); + + // No wait: the previous line was emitted microseconds ago, so this is well + // inside the 250ms window. + h.reporter.start_attempt(); + observer("attempt two starting"); + + assert_eq!( + h.lines(), + vec![ + Some("attempt one failed".to_string()), + None, + Some("attempt two starting".to_string()), + ] + ); +} + +/// A burst inside the window coalesces to one event, and the line it emits is +/// the *newest* — the display shows current progress, not the line that happened +/// to arrive when the window opened. +#[test] +fn test_a_burst_coalesces_to_the_newest_line_not_the_first() { + let h = harness(); + let observer = h.reporter.line_observer().expect("an observer"); + + observer("one"); + observer("two"); + observer("three"); + // Ends the attempt, which is when a held line is known to be the last. + h.reporter.record_attempt(1, outcome("cli", true, "done")); + + assert_eq!( + h.lines(), + vec![Some("one".to_string()), Some("three".to_string())], + "the held line must be the newest, and it must not be lost" + ); +} + +/// The throttle is per install, not per stream: stdout and stderr of one attempt +/// share one window, so an install printing on both does not double the event +/// rate. +#[test] +fn test_both_streams_of_one_attempt_share_the_rate_window() { + let h = harness(); + let stdout = h.reporter.line_observer().expect("an observer"); + let stderr = h.reporter.line_observer().expect("an observer"); + + stdout("progress"); + stderr("warning"); + h.reporter.record_attempt(1, outcome("cli", true, "done")); + + assert_eq!( + h.lines(), + vec![Some("progress".to_string()), Some("warning".to_string())], + "the second stream's line is held, not emitted immediately, and not lost" + ); +} + +/// Nothing listening means no observer at all, so the drain skips line +/// reassembly entirely rather than doing the work and discarding it. +#[test] +fn test_no_observer_when_nothing_is_listening() { + assert!(silent_reporter().line_observer().is_none()); +} + +// ── late-drain deactivation ────────────────────────────────────────────────── + +/// After the reporter is dropped (run settled), a drain thread that still holds +/// a cloned observer must not be able to emit. The lifecycle lock is shared by +/// reference with every `line_observer` clone, so taking the exclusive write +/// lock in `Drop` — which waits out any in-flight read guards — ensures no +/// late event can reach the listener. +/// +/// The test resets the throttle via `start_attempt` before the late emit, so +/// the late line would be emitted unconditionally without the lifecycle lock. +#[test] +fn test_a_detached_observer_cannot_emit_after_the_reporter_is_dropped() { + let h = harness(); + + // Simulate a drain thread: `line_observer` clones `Live` (owned, not + // borrowed), so the closure outlives the reporter. + let observer = h.reporter.line_observer().expect("an observer"); + observer("before-settle"); + + assert_eq!( + h.lines(), + vec![Some("before-settle".to_string())], + "a live observer must emit before the reporter drops" + ); + + // Open a fresh throttle window so the next offer would emit immediately — + // simulating the drain arriving after the 250ms rate window closed. + h.reporter.start_attempt(); + let events = Arc::clone(&h.events); + + // Drop the reporter — takes the exclusive lifecycle write lock, waits for + // any in-flight publications, then deactivates. + drop(h); + + // A late offer after drop must be blocked by the deactivated lifecycle. + observer("late-drain-after-settle"); + + let lines: Vec> = events + .lock() + .unwrap() + .iter() + .map(|e| e.line.clone()) + .collect(); + assert_eq!( + lines, + // before-settle + the clear from start_attempt, no late line + vec![Some("before-settle".to_string()), None], + "a detached observer must not emit after the reporter is dropped" + ); +} + +/// Deterministic concurrency pin: proves that reporter deactivation cannot +/// complete while a drain thread is in-flight between admission and publication. +/// +/// The lifecycle `RwLock` enforces this: drain threads hold a shared read guard +/// for the entire (check → emit) span, so `InstallReporter::drop`'s write lock +/// blocks until every admitted publication finishes. +/// +/// **How this test fails on the old atomic shape** (`dc6421ac4`): on that shape, +/// `offer` loads `active` once as a plain atomic read and then calls `publish` +/// independently. There is no shared lock, so `drop` (an atomic store) can +/// complete while the thread is between the load and the emit — the test +/// exposes this by asserting the write lock CANNOT be acquired while a reader +/// holds a read guard. On the atomic shape the `lifecycle` field does not exist, +/// so the entire serialisation contract is absent and the pin fails. +#[test] +fn test_deactivation_blocks_until_in_flight_publication_completes() { + // Build a reporter and grab a Live clone that represents a drain thread. + let h = harness(); + let live_clone = { + // Extract the `Live` from a `line_observer` closure by temporarily + // building a second observer and using the lifecycle Arc directly. + h.reporter.line_observer().expect("observer"); + // Clone the inner lifecycle from the reporter's `Live` via a + // white-box path: the test module is a child of install_report and + // can access private fields. + h.reporter + .live + .as_ref() + .expect("live exists") + .lifecycle + .clone() + }; + + // Phase 1: acquire the shared read guard (admission). + let guard = live_clone.read().expect("lifecycle read"); + assert!(*guard, "lifecycle must be active at admission"); + + // Phase 2: while the read guard is held, a write lock must be blocked. + // This is the core serialisation invariant: `Drop` cannot complete until + // every admitted reader releases its guard. + assert!( + live_clone.try_write().is_err(), + "a write lock must not be acquirable while a read guard is held — \ + Drop must block while a publication is in-flight" + ); + + // Phase 3: release the read guard (publication completed). + drop(guard); + + // Phase 4: write lock is now available, and Drop can set active=false. + let mut write = live_clone.write().expect("lifecycle write"); + *write = false; + drop(write); + + // Phase 5: a new reader after deactivation sees active=false and returns. + let guard2 = live_clone.read().expect("lifecycle read"); + assert!( + !*guard2, + "lifecycle must be inactive after deactivation — new admissions rejected" + ); +} + +/// Shared-consumer assertion: drive a potential late run-1 event AND run-2's +/// events through the same `nextInstallOutputLine`-equivalent reducer and assert +/// that run 2's output replaces — not revives — any stale run-1 state. +/// +/// The frontend reduces events into a single consumer that resets to `null` +/// when `isInstalling` goes false (i.e., when the run settles). This test +/// models that reset and verifies the full cross-run contract: +/// +/// 1. Run 1 emits normally; the late drain is silenced by the lifecycle lock +/// (no event with a high run-1 `seq` ever reaches the shared sink). +/// 2. At run-1 settlement the consumer resets to `null`, exactly as the +/// frontend hook does when `isInstalling` becomes false. +/// 3. Run 2 starts, emits its `seq=0` clear and first line. With a null-state +/// consumer the reducer accepts both immediately — even if a late run-1 +/// event HAD arrived (it didn't), the reset would have cleared its seq. +/// +/// **Why this test fails on the old atomic shape**: without the lifecycle lock, +/// `obs1("late-run-one-drain")` emits a high-seq run-1 event into the shared +/// sink. That event arrives AFTER the consumer reset (its seq is accepted from +/// a null state), leaving `{ seq: N, line: "late-run-one-drain" }` in the +/// consumer. Run 2 then emits `seq=0` and `seq=1`, both ≤ N, so they are +/// rejected and the final state stays on run 1's output. +#[test] +fn test_run2_output_replaces_stale_run1_state_through_shared_consumer() { + // One shared event sink for all events from both runs, simulating the + // permanent frontend listener that receives all `acp-install-output` events. + let all_events: Arc>> = Arc::new(Mutex::new(Vec::new())); + // Track where run 1 settles so the consumer reset can be applied at the + // correct boundary in the fold below. + let run1_settle_len: Arc> = Arc::new(Mutex::new(0)); + let runtime_id = "goose"; + + // ── Run 1 ────────────────────────────────────────────────────────────── + let dir1 = tempfile::tempdir().expect("tempdir"); + let log1 = dir1.path().join("install-goose.log"); + let emit1: EmitEvent = { + let sink = Arc::clone(&all_events); + Arc::new(move |event| sink.lock().unwrap().push(event)) + }; + let reporter1 = InstallReporter::new( + runtime_id, + InstallLog::start(&log1, runtime_id, "1.0.0"), + Some(emit1), + ); + + reporter1.start_attempt(); + let obs1 = reporter1.line_observer().expect("observer"); + obs1("run-one-output"); + reporter1.record_attempt(1, outcome("cli", true, "done")); + + // Drop reporter1 — deactivates obs1 via the exclusive lifecycle write lock, + // which blocks until any in-flight read guard (publication) has released. + drop(reporter1); + + // Record the sink length at settlement — this is the point where the + // frontend resets its consumer state to null (isInstalling = false). + *run1_settle_len.lock().unwrap() = all_events.lock().unwrap().len(); + + // A late drain from run 1 arrives after settlement. With the lifecycle + // lock this is silenced. Without the lock (old atomic shape) it would + // reach the sink with a high seq, poisoning the null-reset consumer before + // run 2 can emit its restarted seq=0. + // Reset the throttle so the offer would emit unconditionally if the lock + // were absent — this makes the mutation meaningful. + { + // We need a fresh throttle window. Simulate by directly restarting + // via a new reporter to touch the shared Live (not possible after drop), + // so instead just call offer — the throttle holds the last emit time + // from flush_pending, which was microseconds ago. Give the window time + // to expire so the offer fires immediately on the old atomic shape. + std::thread::sleep(Duration::from_millis(300)); + } + obs1("late-run-one-drain"); + + // ── Run 2 ────────────────────────────────────────────────────────────── + let dir2 = tempfile::tempdir().expect("tempdir"); + let log2 = dir2.path().join("install-goose.log"); + let emit2: EmitEvent = { + let sink = Arc::clone(&all_events); + Arc::new(move |event| sink.lock().unwrap().push(event)) + }; + let reporter2 = InstallReporter::new( + runtime_id, + InstallLog::start(&log2, runtime_id, "1.0.0"), + Some(emit2), + ); + + reporter2.start_attempt(); + let obs2 = reporter2.line_observer().expect("observer"); + obs2("run-two-first-line"); + reporter2.record_attempt(1, outcome("cli", true, "done")); + + // ── Shared-consumer fold ──────────────────────────────────────────────── + // Fold all emitted events through the nextInstallOutputLine reducer logic. + // At the run-1 settlement boundary, reset consumer to null — exactly as + // the frontend hook does when isInstalling becomes false. + struct State { + seq: u64, + line: Option, + } + let settle_at = *run1_settle_len.lock().unwrap(); + let events = all_events.lock().unwrap().clone(); + let mut consumer: Option = None; + for (i, event) in events.iter().enumerate() { + // Simulate frontend reset at run-1 settlement boundary. + if i == settle_at { + consumer = None; + } + if event.runtime_id != runtime_id { + continue; + } + if let Some(ref c) = consumer { + if event.seq <= c.seq { + continue; // reject stale / out-of-order + } + } + consumer = Some(State { + seq: event.seq, + line: event.line.clone(), + }); + } + + // The final consumer state must be run 2's first line. + // + // With the lifecycle lock: obs1("late-run-one-drain") emits nothing, + // so after the null reset only run-2 events arrive — run 2 wins cleanly. + // + // Without the lifecycle lock (old atomic shape): obs1 emits the late line + // with a high seq into the already-reset (null) consumer, consumer becomes + // { seq: N, line: "late-run-one-drain" }. Run 2's seq=0/1 are both ≤ N + // and are rejected, leaving the final state on run 1's stale output. + let final_line = consumer + .as_ref() + .and_then(|s| s.line.as_deref()) + .unwrap_or(""); + assert_eq!( + final_line, "run-two-first-line", + "run 2 must replace — not revive — stale run-1 state through the shared consumer; \ + got: {final_line:?}" + ); +} diff --git a/desktop/src-tauri/src/commands/agent_discovery/post_install_verification.rs b/desktop/src-tauri/src/commands/agent_discovery/post_install_verification.rs index 3155104b56..535d4c9ecb 100644 --- a/desktop/src-tauri/src/commands/agent_discovery/post_install_verification.rs +++ b/desktop/src-tauri/src/commands/agent_discovery/post_install_verification.rs @@ -1,13 +1,19 @@ use crate::managed_agents::{AcpAvailabilityStatus, InstallStepResult}; -pub(super) fn run(runtime_id: &str, steps: &mut Vec) { +use super::install_report::InstallReporter; + +pub(super) fn run( + runtime_id: &str, + steps: &mut Vec, + reporter: &InstallReporter, +) { // Observe PATH changes and binaries added after Buzz launched. crate::managed_agents::refresh_login_shell_path(); crate::managed_agents::clear_resolve_cache(); let availability = crate::managed_agents::discover_acp_runtime_availability(runtime_id); if let Some(failure) = failure(runtime_id, availability) { - steps.push(failure); + reporter.record_step(steps, failure); } } diff --git a/desktop/src-tauri/src/commands/agent_models.rs b/desktop/src-tauri/src/commands/agent_models.rs index ca1fe9bdf6..7ce03b140b 100644 --- a/desktop/src-tauri/src/commands/agent_models.rs +++ b/desktop/src-tauri/src/commands/agent_models.rs @@ -99,6 +99,17 @@ pub async fn get_agent_models( // so a build-provided provider still gets live discovery. let effective_provider = effective_discovery_provider(saved_provider.as_deref(), provider_env_var, &merged_env); + if let Some(models) = discover_openrouter_models( + &state.http_client, + &effective_provider, + &merged_env, + persisted_model.clone(), + ) + .await? + { + return Ok(models); + } + if let Some(models) = discover_openai_compatible_models( &state.http_client, &effective_provider, @@ -154,69 +165,11 @@ fn model_discovery_error(pubkey: &str, error: &str) -> String { ) } -/// Everything `get_agent_models` needs from the record + context, resolved in -/// one pure step so the linked-agent regression test can bind the exact values -/// the command consumes. -#[derive(Debug, PartialEq, Eq)] -struct AgentModelDiscoveryConfig { - /// Effective harness command (descriptor-resolved), for `resolve_command`. - command: String, - /// Effective harness args (descriptor-resolved). - args: Vec, - /// Model from the authoritative resolver spawn uses — linked instances - /// read their definition, never stale `record.model` bytes. - model: Option, - /// Provider from the same authoritative resolver — never stale - /// `record.provider` bytes for linked instances. - provider: Option, - /// The runtime's provider env var (e.g. `GOOSE_PROVIDER`), so discovery - /// can recover the provider from the env when the resolver yields none. - /// `None` for runtimes that do not take a provider, or an unknown command. - provider_env_var: Option<&'static str>, - /// The descriptor's fully layered env (definition/persona/global/agent). - env: BTreeMap, -} - -/// Resolve the model-discovery config for a saved agent — the descriptor-backed -/// successor to the old `saved_agent_model_discovery_config`. -/// -/// Command/args/env come from `resolve_effective_harness_descriptor` (the same -/// resolver as `spawn_agent_child`); model/provider come from -/// `resolve_effective_model_provider` (#1968's definition-authoritative -/// contract) — linked instances read their definition, never a stale -/// materialized `record.model`/`record.provider`, so discovery cannot query a -/// provider this agent will not actually launch with. Definition-less -/// instances keep their own record values, matching spawn's -/// `resolve_definition_less` arm. When the resolver yields no provider, -/// `effective_discovery_provider` recovers the provider the agent will -/// actually launch with from the runtime's own provider env var, read out of -/// the descriptor env (which already layers definition/persona/global values -/// the same way spawn does). -/// -/// Returns `Err("DANGLING_HARNESS_ID:")` from the descriptor resolver when -/// the harness id no longer exists; the caller routes it through -/// `model_discovery_error`. -fn agent_model_discovery_config( - record: &crate::managed_agents::ManagedAgentRecord, - personas: &[crate::managed_agents::AgentDefinition], - global: &crate::managed_agents::GlobalAgentConfig, -) -> Result { - let descriptor = - crate::managed_agents::resolve_effective_harness_descriptor(record, personas, global)?; - let (model, provider) = - crate::managed_agents::resolve_effective_model_provider(record, personas, global); - let provider_env_var = - known_acp_runtime(&descriptor.command).and_then(|meta| meta.provider_env_var); - - Ok(AgentModelDiscoveryConfig { - command: descriptor.command, - args: descriptor.args, - model, - provider, - provider_env_var, - env: descriptor.env, - }) -} +#[path = "agent_models_discovery_config.rs"] +mod discovery_config; +use discovery_config::{ + agent_model_discovery_config, draft_agent_model_discovery_env, AgentModelDiscoveryConfig, +}; #[derive(Debug, Deserialize)] #[serde(rename_all = "camelCase")] @@ -269,31 +222,12 @@ pub async fn discover_agent_models( .unwrap_or_else(|| agent_command.to_string()); let runtime_meta = known_acp_runtime(agent_command); - let mut derived_env = BTreeMap::new(); - if let Some(meta) = runtime_meta { - let provider = input - .provider - .as_deref() - .map(str::trim) - .filter(|value| !value.is_empty()); - if !meta.provider_locked { - if let (Some(env_key), Some(provider)) = (meta.provider_env_var, provider) { - derived_env.insert(env_key.to_string(), provider.to_string()); - } - } - } - // Layer definition_env below user env_vars so user overrides always win. - // Reserved keys are stripped, matching the same filter applied at spawn. - let mut filtered_definition_env = BTreeMap::new(); - for (key, value) in &input.definition_env { - if !crate::managed_agents::is_reserved_env_key(key) { - filtered_definition_env.insert(key.clone(), value.clone()); - } - } - // Merge: derived (metadata) → definition env → user env_vars. - let merged_with_def = - crate::managed_agents::merged_user_env(&derived_env, &filtered_definition_env); - let merged_env = crate::managed_agents::merged_user_env(&merged_with_def, &input.env_vars); + let merged_env = draft_agent_model_discovery_env( + agent_command, + input.provider.as_deref(), + &input.definition_env, + &input.env_vars, + ); let merged_env = discovery_env_with_baked_floor(merged_env); // Recover a build-provided provider when the form has none, so the create // dialog discovers live models instead of falling through to the subprocess. @@ -348,6 +282,13 @@ pub async fn discover_agent_models( return Err("Buzz shared compute is not available in this build".to_string()); } + if let Some(models) = + discover_openrouter_models(&state.http_client, &effective_provider, &merged_env, None) + .await? + { + return Ok(models); + } + if let Some(models) = discover_openai_compatible_models( &state.http_client, &effective_provider, @@ -388,6 +329,15 @@ struct OpenAiModelListItem { created: Option, } +#[path = "agent_models_openrouter.rs"] +mod openrouter; +use openrouter::discover_openrouter_models; +#[cfg(test)] +use openrouter::{ + filter_openrouter_models, is_openrouter_provider, openrouter_models_url, + OpenRouterModelListItem, OpenRouterModelListResponse, +}; + fn is_openai_compatible_provider(provider: Option<&str>) -> bool { matches!( provider diff --git a/desktop/src-tauri/src/commands/agent_models_discovery_config.rs b/desktop/src-tauri/src/commands/agent_models_discovery_config.rs new file mode 100644 index 0000000000..e43f09495b --- /dev/null +++ b/desktop/src-tauri/src/commands/agent_models_discovery_config.rs @@ -0,0 +1,112 @@ +//! Model-discovery configuration resolution for the agent-models commands. +//! +//! Two entry points, one per call shape: [`agent_model_discovery_config`] +//! resolves a *saved* agent through the same descriptor/model resolvers spawn +//! uses, and [`draft_agent_model_discovery_env`] derives the env for an unsaved +//! form. Both are pure so the regression tests can bind the exact values the +//! commands consume. +//! +//! Included from `agent_models.rs` via `#[path]`, so `super::*` resolves +//! against that module (the `agent_models_tests.rs` convention). + +use std::collections::BTreeMap; + +use crate::managed_agents::known_acp_runtime; + +/// Everything `get_agent_models` needs from the record + context, resolved in +/// one pure step so the linked-agent regression test can bind the exact values +/// the command consumes. +#[derive(Debug, PartialEq, Eq)] +pub(super) struct AgentModelDiscoveryConfig { + /// Effective harness command (descriptor-resolved), for `resolve_command`. + pub(super) command: String, + /// Effective harness args (descriptor-resolved). + pub(super) args: Vec, + /// Model from the authoritative resolver spawn uses — linked instances + /// read their definition, never stale `record.model` bytes. + pub(super) model: Option, + /// Provider from the same authoritative resolver — never stale + /// `record.provider` bytes for linked instances. + pub(super) provider: Option, + /// The runtime's provider env var (e.g. `GOOSE_PROVIDER`), so discovery + /// can recover the provider from the env when the resolver yields none. + /// `None` for runtimes that do not take a provider, or an unknown command. + pub(super) provider_env_var: Option<&'static str>, + /// The descriptor's fully layered env (definition/persona/global/agent). + pub(super) env: BTreeMap, +} + +/// Resolve the model-discovery config for a saved agent — the descriptor-backed +/// successor to the old `saved_agent_model_discovery_config`. +/// +/// Command/args/env come from `resolve_effective_harness_descriptor` (the same +/// resolver as `spawn_agent_child`); model/provider come from +/// `resolve_effective_model_provider` (#1968's definition-authoritative +/// contract) — linked instances read their definition, never a stale +/// materialized `record.model`/`record.provider`, so discovery cannot query a +/// provider this agent will not actually launch with. Definition-less +/// instances keep their own record values, matching spawn's +/// `resolve_definition_less` arm. When the resolver yields no provider, +/// `effective_discovery_provider` recovers the provider the agent will +/// actually launch with from the runtime's own provider env var, read out of +/// the descriptor env (which already layers definition/persona/global values +/// the same way spawn does). +/// +/// Returns `Err("DANGLING_HARNESS_ID:")` from the descriptor resolver when +/// the harness id no longer exists; the caller routes it through +/// `model_discovery_error`. +pub(super) fn agent_model_discovery_config( + record: &crate::managed_agents::ManagedAgentRecord, + personas: &[crate::managed_agents::AgentDefinition], + global: &crate::managed_agents::GlobalAgentConfig, +) -> Result { + let descriptor = + crate::managed_agents::resolve_effective_harness_descriptor(record, personas, global)?; + let (model, provider) = + crate::managed_agents::resolve_effective_model_provider(record, personas, global); + let provider_env_var = + known_acp_runtime(&descriptor.command).and_then(|meta| meta.provider_env_var); + + Ok(AgentModelDiscoveryConfig { + command: descriptor.command, + args: descriptor.args, + model, + provider, + provider_env_var, + env: descriptor.env, + }) +} + +/// Derive the discovery env for an unsaved ("draft") agent configuration. +/// +/// Mirrors the layering `agent_model_discovery_config` takes from the harness +/// descriptor, but sources the provider from form input: runtime-derived +/// provider env var → definition env → user env vars, so user overrides always +/// win. Extracted so the draft path has the same tested seam as the saved one. +pub(super) fn draft_agent_model_discovery_env( + agent_command: &str, + provider: Option<&str>, + definition_env: &BTreeMap, + env_vars: &BTreeMap, +) -> BTreeMap { + let mut derived_env = BTreeMap::new(); + if let Some(meta) = known_acp_runtime(agent_command) { + let provider = provider.map(str::trim).filter(|value| !value.is_empty()); + if !meta.provider_locked { + if let (Some(env_key), Some(provider)) = (meta.provider_env_var, provider) { + derived_env.insert(env_key.to_string(), provider.to_string()); + } + } + } + // Reserved keys are stripped from definition env, matching the same filter + // applied at spawn. + let mut filtered_definition_env = BTreeMap::new(); + for (key, value) in definition_env { + if !crate::managed_agents::is_reserved_env_key(key) { + filtered_definition_env.insert(key.clone(), value.clone()); + } + } + let merged_with_def = + crate::managed_agents::merged_user_env(&derived_env, &filtered_definition_env); + crate::managed_agents::merged_user_env(&merged_with_def, env_vars) +} diff --git a/desktop/src-tauri/src/commands/agent_models_openrouter.rs b/desktop/src-tauri/src/commands/agent_models_openrouter.rs new file mode 100644 index 0000000000..be6dd2cf26 --- /dev/null +++ b/desktop/src-tauri/src/commands/agent_models_openrouter.rs @@ -0,0 +1,112 @@ +use std::collections::BTreeMap; + +use serde::Deserialize; + +use crate::managed_agents::{AgentModelInfo, AgentModelsResponse}; + +#[cfg(test)] +use super::env_value; +use super::{env_or_process_value, redaction_env_with_value, DiscoveryProvider}; + +#[derive(Debug, Deserialize)] +#[cfg_attr(test, derive(Clone))] +pub(super) struct OpenRouterModelListResponse { + pub data: Vec, +} + +#[derive(Debug, Deserialize)] +#[cfg_attr(test, derive(Clone))] +pub(super) struct OpenRouterModelListItem { + pub id: String, + #[serde(default)] + pub supported_parameters: Vec, +} + +pub(super) fn is_openrouter_provider(provider: Option<&str>) -> bool { + matches!( + provider + .map(str::trim) + .map(str::to_ascii_lowercase) + .as_deref(), + Some("openrouter") + ) +} + +#[cfg(test)] +pub(super) fn openrouter_models_url(env: &BTreeMap) -> String { + let base_url = env_value(env, "OPENROUTER_BASE_URL") + .unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string()); + format!("{}/models", base_url.trim_end_matches('/')) +} + +fn openrouter_models_url_for_discovery(env: &BTreeMap) -> String { + let base_url = env_or_process_value(env, "OPENROUTER_BASE_URL") + .unwrap_or_else(|| "https://openrouter.ai/api/v1".to_string()); + format!("{}/models", base_url.trim_end_matches('/')) +} + +pub(super) async fn discover_openrouter_models( + client: &reqwest::Client, + provider: &DiscoveryProvider, + env: &BTreeMap, + selected_model: Option, +) -> Result, String> { + if !is_openrouter_provider(provider.as_deref()) { + return Ok(None); + } + + let api_key = match provider.required_env(env, "OPENROUTER_API_KEY")? { + Some(api_key) => api_key, + None => return Ok(None), + }; + let redaction_env = redaction_env_with_value(env, "OPENROUTER_API_KEY", &api_key); + let url = openrouter_models_url_for_discovery(env); + let response = client + .get(&url) + .bearer_auth(&api_key) + .send() + .await + .map_err(|error| format!("OpenRouter model discovery request failed: {error}"))?; + let status = response.status(); + if !status.is_success() { + let body = response.text().await.unwrap_or_default(); + let body = crate::managed_agents::redact_env_values_in(&body, &redaction_env); + return Err(format!("OpenRouter model discovery HTTP {status}: {body}")); + } + + let response = response + .json::() + .await + .map_err(|error| format!("OpenRouter model discovery response parse failed: {error}"))?; + + filter_openrouter_models(response, selected_model) +} + +pub(super) fn filter_openrouter_models( + response: OpenRouterModelListResponse, + selected_model: Option, +) -> Result, String> { + let models: Vec = response + .data + .into_iter() + .filter(|m| m.supported_parameters.iter().any(|p| p == "tools")) + .map(|m| AgentModelInfo { + id: m.id.clone(), + name: Some(m.id), + description: None, + }) + .collect(); + + if models.is_empty() { + return Err("OpenRouter model discovery returned no tools-capable models".to_string()); + } + + Ok(Some(AgentModelsResponse { + agent_name: "openrouter".to_string(), + agent_version: "models-api".to_string(), + models, + agent_default_model: None, + selected_model, + supports_switching: true, + })) +} diff --git a/desktop/src-tauri/src/commands/agent_models_tests.rs b/desktop/src-tauri/src/commands/agent_models_tests.rs index b65f240900..14c981d730 100644 --- a/desktop/src-tauri/src/commands/agent_models_tests.rs +++ b/desktop/src-tauri/src/commands/agent_models_tests.rs @@ -590,3 +590,294 @@ fn model_discovery_error_converts_dangling_sentinel_to_sentence() { let plain = model_discovery_error("agent-pk", "plain failure"); assert_eq!(plain, "cannot discover models for agent-pk: plain failure"); } + +// --------------------------------------------------------------------------- +// OpenRouter provider +// --------------------------------------------------------------------------- + +#[test] +fn is_openrouter_provider_matches() { + assert!(is_openrouter_provider(Some("openrouter"))); + assert!(is_openrouter_provider(Some(" OpenRouter "))); + assert!(!is_openrouter_provider(Some("openai"))); + assert!(!is_openrouter_provider(Some("anthropic"))); + assert!(!is_openrouter_provider(None)); +} + +#[test] +fn openrouter_models_url_uses_default_base_url() { + assert_eq!( + openrouter_models_url(&BTreeMap::new()), + "https://openrouter.ai/api/v1/models" + ); +} + +#[test] +fn openrouter_models_url_respects_custom_base_url() { + let env = BTreeMap::from([( + "OPENROUTER_BASE_URL".to_string(), + "https://eu.openrouter.ai/api/v1".to_string(), + )]); + assert_eq!( + openrouter_models_url(&env), + "https://eu.openrouter.ai/api/v1/models" + ); +} + +#[test] +fn openrouter_models_url_strips_trailing_slash() { + let env = BTreeMap::from([( + "OPENROUTER_BASE_URL".to_string(), + "https://proxy.example.com/api/v1/".to_string(), + )]); + assert_eq!( + openrouter_models_url(&env), + "https://proxy.example.com/api/v1/models" + ); +} + +#[test] +fn openrouter_filter_keeps_tools_capable_models() { + let response = OpenRouterModelListResponse { + data: vec![ + OpenRouterModelListItem { + id: "anthropic/claude-opus-4-7".to_string(), + supported_parameters: vec!["tools".to_string(), "reasoning".to_string()], + }, + OpenRouterModelListItem { + id: "openai/gpt-5.5-pro".to_string(), + supported_parameters: vec!["tools".to_string()], + }, + OpenRouterModelListItem { + id: "meta-llama/llama-no-tools".to_string(), + supported_parameters: vec!["temperature".to_string()], + }, + ], + }; + let result = filter_openrouter_models(response, None).unwrap().unwrap(); + let ids: Vec<_> = result.models.iter().map(|m| m.id.as_str()).collect(); + assert_eq!(ids, vec!["anthropic/claude-opus-4-7", "openai/gpt-5.5-pro"]); +} + +#[test] +fn openrouter_filter_excludes_absent_supported_parameters() { + let response: OpenRouterModelListResponse = + serde_json::from_str(r#"{"data": [{"id": "model-no-params"}]}"#).unwrap(); + assert!( + response.data[0].supported_parameters.is_empty(), + "absent supported_parameters must default to empty vec" + ); + let result = filter_openrouter_models(response, None); + assert!( + result.is_err(), + "models with no supported_parameters must be excluded" + ); + assert!( + result.unwrap_err().contains("no tools-capable models"), + "error must indicate no tools-capable models" + ); +} + +#[test] +fn openrouter_filter_excludes_empty_supported_parameters() { + let response = OpenRouterModelListResponse { + data: vec![OpenRouterModelListItem { + id: "model-empty-params".to_string(), + supported_parameters: Vec::new(), + }], + }; + let result = filter_openrouter_models(response, None); + assert!(result.is_err()); + assert!(result.unwrap_err().contains("no tools-capable models")); +} + +#[test] +fn openrouter_filter_empty_result_returns_error() { + let response = OpenRouterModelListResponse { data: Vec::new() }; + let result = filter_openrouter_models(response, None); + assert!(result.is_err()); + assert!(result.unwrap_err().contains("no tools-capable models")); +} + +#[test] +fn openrouter_filter_preserves_selected_model() { + let response = OpenRouterModelListResponse { + data: vec![OpenRouterModelListItem { + id: "openai/gpt-5.5-pro".to_string(), + supported_parameters: vec!["tools".to_string()], + }], + }; + let result = filter_openrouter_models(response, Some("openai/gpt-5.5-pro".to_string())) + .unwrap() + .unwrap(); + assert_eq!(result.selected_model.as_deref(), Some("openai/gpt-5.5-pro")); +} + +#[test] +fn openrouter_credential_redaction_env_records_key() { + let env = BTreeMap::from([( + "OPENROUTER_API_KEY".to_string(), + "sk-or-v1-secret-key-12345".to_string(), + )]); + let redaction = + redaction_env_with_value(&env, "OPENROUTER_API_KEY", "sk-or-v1-secret-key-12345"); + assert_eq!( + redaction.get("OPENROUTER_API_KEY").map(String::as_str), + Some("sk-or-v1-secret-key-12345"), + "redaction env must record the API key for error body redaction" + ); +} + +#[test] +fn openrouter_saved_agent_model_discovery_resolves_provider() { + let record: crate::managed_agents::ManagedAgentRecord = serde_json::from_str( + r#"{ + "pubkey": "abcd1234", + "name": "test-agent", + "private_key_nsec": "nsec1fake", + "relay_url": "wss://localhost:3000", + "acp_command": "buzz-acp", + "agent_command": "buzz-agent", + "agent_command_override": "buzz-agent", + "agent_args": [], + "mcp_command": "", + "turn_timeout_seconds": 320, + "system_prompt": null, + "model": "anthropic/claude-sonnet-4", + "provider": "openrouter", + "env_vars": { + "OPENROUTER_API_KEY": "sk-or-test-key", + "BUZZ_PRIVATE_KEY": "must-not-leak" + }, + "created_at": "2026-01-01T00:00:00Z", + "updated_at": "2026-01-01T00:00:00Z", + "last_started_at": null, + "last_stopped_at": null, + "last_exit_code": null, + "last_error": null + }"#, + ) + .expect("sample openrouter managed agent record"); + + let discovery = agent_model_discovery_config( + &record, + &[], + &crate::managed_agents::GlobalAgentConfig::default(), + ) + .expect("discovery config should resolve for an openrouter record"); + assert_eq!(discovery.provider.as_deref(), Some("openrouter")); + assert_eq!( + discovery.model.as_deref(), + Some("anthropic/claude-sonnet-4") + ); + assert_eq!( + discovery.env.get("OPENROUTER_API_KEY").map(String::as_str), + Some("sk-or-test-key") + ); + assert!(!discovery.env.contains_key("BUZZ_PRIVATE_KEY")); +} + +/// B5/T4: unsaved-agent ("draft") discovery mirrors the saved-agent path — +/// `draft_agent_model_discovery_env` must derive the provider env var from +/// form input the same way `agent_model_discovery_config` derives it from a +/// persisted record's harness descriptor, and preserve caller-supplied env +/// (including the OpenRouter API key) unmodified. +#[test] +fn openrouter_draft_agent_model_discovery_derives_provider_env() { + let env_vars = BTreeMap::from([( + "OPENROUTER_API_KEY".to_string(), + "sk-or-draft-key".to_string(), + )]); + + let merged = draft_agent_model_discovery_env( + "buzz-agent", + Some("openrouter"), + &BTreeMap::new(), + &env_vars, + ); + + assert_eq!( + merged.get("BUZZ_AGENT_PROVIDER").map(String::as_str), + Some("openrouter"), + "provider env var must be derived from form input for a known ACP runtime" + ); + assert_eq!( + merged.get("OPENROUTER_API_KEY").map(String::as_str), + Some("sk-or-draft-key"), + "caller-supplied env vars must survive the merge" + ); +} + +#[test] +fn draft_agent_model_discovery_env_omits_provider_when_absent() { + let merged = + draft_agent_model_discovery_env("buzz-agent", None, &BTreeMap::new(), &BTreeMap::new()); + assert!( + !merged.contains_key("BUZZ_AGENT_PROVIDER"), + "no provider must be derived when the caller supplies none" + ); +} + +/// The three-tier precedence this merge exists to preserve: main's inline +/// `derived → definition_env → env_vars` layering was folded into +/// `draft_agent_model_discovery_env`, so pin the order at every collision +/// boundary rather than trusting the two single-tier tests above. +/// +/// `SHARED` collides across all three tiers, so the user value proves the +/// full chain; the pairwise keys prove each adjacent boundary independently +/// (a merge that dropped only the middle tier would still satisfy `SHARED`). +/// `BUZZ_PRIVATE_KEY` proves a reserved key cannot ride in on a harness +/// definition, which is the tier a user never types. +#[test] +fn draft_agent_model_discovery_env_layers_all_three_tiers_in_order() { + // Tier 2 (middle): harness definition env — overlays the runtime-derived + // floor, loses to user env. + let definition_env = BTreeMap::from([ + ("SHARED".to_string(), "from-definition".to_string()), + // Collides with tier 1: `buzz-agent`'s own provider env var, which the + // `provider` argument derives below. + ("BUZZ_AGENT_PROVIDER".to_string(), "openai".to_string()), + ("USER_OVER_DEF".to_string(), "from-definition".to_string()), + ("DEFINITION_ONLY".to_string(), "from-definition".to_string()), + // Reserved: must never reach the child, even from a definition. + ("BUZZ_PRIVATE_KEY".to_string(), "must-not-leak".to_string()), + ]); + // Tier 3 (top): user-entered env — wins over everything. + let env_vars = BTreeMap::from([ + ("SHARED".to_string(), "from-user".to_string()), + ("USER_OVER_DEF".to_string(), "from-user".to_string()), + ("USER_ONLY".to_string(), "from-user".to_string()), + ]); + + // Tier 1 (floor): `Some("openrouter")` derives BUZZ_AGENT_PROVIDER. + let merged = draft_agent_model_discovery_env( + "buzz-agent", + Some("openrouter"), + &definition_env, + &env_vars, + ); + + let expected: &[(&str, Option<&str>)] = &[ + // Collides in all three tiers — the top tier wins. + ("SHARED", Some("from-user")), + // Tier 2 over tier 1: the definition's value survives, proving the + // derived provider is the floor and not layered on top. + ("BUZZ_AGENT_PROVIDER", Some("openai")), + // Tier 3 over tier 2. + ("USER_OVER_DEF", Some("from-user")), + // Single-tier keys pass through untouched. + ("DEFINITION_ONLY", Some("from-definition")), + ("USER_ONLY", Some("from-user")), + // Reserved keys never survive the definition tier. Doubly enforced — + // the explicit `is_reserved_env_key` filter here and `merged_user_env`'s + // own `retain` — so this pins the contract, not either mechanism. + ("BUZZ_PRIVATE_KEY", None), + ]; + for (key, want) in expected { + assert_eq!( + merged.get(*key).map(String::as_str), + *want, + "env key `{key}` must resolve to {want:?} after three-tier layering" + ); + } +} diff --git a/desktop/src-tauri/src/commands/agents.rs b/desktop/src-tauri/src/commands/agents.rs index 0758fc3aac..3b114b0474 100644 --- a/desktop/src-tauri/src/commands/agents.rs +++ b/desktop/src-tauri/src/commands/agents.rs @@ -1362,7 +1362,7 @@ use deploy::build_deploy_payload; #[cfg(test)] use deploy::deploy_payload_json; #[cfg(test)] -pub(crate) use deploy::resolve_deploy_model_provider; +use deploy::{ensure_remote_provider_supported, resolve_deploy_model_provider}; #[path = "agents_profile.rs"] mod profile; diff --git a/desktop/src-tauri/src/commands/agents_deploy.rs b/desktop/src-tauri/src/commands/agents_deploy.rs index af785711d5..9bb0f6230d 100644 --- a/desktop/src-tauri/src/commands/agents_deploy.rs +++ b/desktop/src-tauri/src/commands/agents_deploy.rs @@ -1,6 +1,8 @@ //! Provider deploy payload construction, split from `agents.rs` (file-size -//! guard). `build_deploy_payload` gathers live state; `deploy_payload_json` -//! is the pure serialization half so payload completeness stays testable. +//! guard). The launch block is derived from the same effective descriptor and +//! policy helpers as local spawn so remote execution does not reimplement them. + +use std::collections::BTreeMap; use tauri::AppHandle; @@ -13,17 +15,6 @@ use crate::{ }; /// Resolve the deploy-specific structured model/provider for a managed agent. -/// -/// Delegates to the single effective-config resolver which enforces -/// definition-authoritative semantics for linked instances: -/// - **Linked:** definition → global. Stale record bytes are never consulted. -/// - **Definition-less:** instance → global. -/// - **Orphaned:** returns `(None, None)` — spawn is blocked elsewhere. -/// -/// Both local spawn and deploy now use the same resolver, so they can never -/// disagree on what model/provider an agent runs with. -/// -/// Exported `pub(crate)` for unit testing. #[cfg(test)] pub(crate) fn resolve_deploy_model_provider( record: &ManagedAgentRecord, @@ -36,58 +27,116 @@ pub(crate) fn resolve_deploy_model_provider( .unwrap_or((None, None)) } -/// Build the standard agent JSON payload for provider deploy calls. -/// -/// Like local spawn, provider deploy re-reads live persona env vars and -/// structured model/provider so remote agents receive current credentials -/// and the same authoritative values that local spawn derives from -/// `runtime_metadata_env_vars`. The only field still pinned is -/// `agent_command`/`agent_args` — those were captured at create time. -/// The only read-time resolution is `relay_url`: a blank pin resolves to -/// the active workspace relay here, matching the create-path contract. +/// Serialize the portable launch contract shared with provider-backed agents. /// -/// Fails closed when the private key is unavailable (keyring outage leaves -/// it empty after hydration): without this guard a provider deploy would -/// serialize `"private_key_nsec": ""` and launch the agent with no -/// identity — the same hazard the local spawn path refuses via -/// `spawn_key_refusal`. +/// `descriptor.env` is the authoritative six-layer environment. Policy values +/// are deliberately separate because providers apply them below that layered +/// environment, preserving the local spawn's power-user override semantics. +pub(super) fn build_launch_block( + record: &ManagedAgentRecord, + descriptor: &crate::managed_agents::readiness::EffectiveHarnessDescriptor, + teams: &[crate::managed_agents::TeamRecord], + effective_prompt: Option<&str>, + effective_model: Option<&str>, + owner_pubkey: &str, +) -> serde_json::Value { + use crate::managed_agents::{known_acp_runtime, resolve_session_title, SESSION_TITLE_ENV_VAR}; + + let runtime = known_acp_runtime(&descriptor.command); + let mut policy_env = BTreeMap::new(); + + if let Some(runtime) = runtime { + policy_env.extend( + runtime + .default_env + .iter() + .map(|(key, value)| ((*key).to_string(), (*value).to_string())), + ); + if runtime.mcp_hooks { + policy_env.insert("MCP_HOOK_SERVERS".into(), "*".into()); + } + } + policy_env.insert("BUZZ_ACP_RELAY_OBSERVER".into(), "true".into()); + policy_env.insert("BUZZ_ACP_LAZY_POOL".into(), "true".into()); + policy_env.insert("BUZZ_ACP_AGENTS".into(), record.parallelism.to_string()); + + if let Some(value) = effective_prompt { + policy_env.insert("BUZZ_ACP_SYSTEM_PROMPT".into(), value.to_string()); + } + if let Some(value) = effective_model { + policy_env.insert("BUZZ_ACP_MODEL".into(), value.to_string()); + } + if let Some(value) = record.idle_timeout_seconds { + policy_env.insert("BUZZ_ACP_IDLE_TIMEOUT".into(), value.to_string()); + } + if let Some(value) = record.max_turn_duration_seconds { + policy_env.insert("BUZZ_ACP_MAX_TURN_DURATION".into(), value.to_string()); + } + if let Some(value) = resolve_session_title(record.display_name.as_deref(), &record.name) { + policy_env.insert(SESSION_TITLE_ENV_VAR.into(), value); + } + if let Some(value) = + crate::managed_agents::spawn_hash::effective_team_instructions(record, teams) + { + policy_env.insert("BUZZ_ACP_TEAM_INSTRUCTIONS".into(), value); + } + + serde_json::json!({ + "command": descriptor.command, + "args": descriptor.args, + "env": descriptor.env, + "policy_env": policy_env, + "owner_pubkey": owner_pubkey, + }) +} + +pub(super) fn ensure_remote_provider_supported(provider: Option<&str>) -> Result<(), String> { + if provider.map(str::trim) == Some(crate::managed_agents::RELAY_MESH_PROVIDER_ID) { + return Err( + "shared-compute agents cannot be deployed remotely because the mesh endpoint is local to the desktop" + .to_string(), + ); + } + Ok(()) +} + +/// Build the standard agent JSON payload for provider deploy calls. pub(super) fn build_deploy_payload( app: &AppHandle, state: &AppState, record: &ManagedAgentRecord, ) -> Result { - // Fails closed when the private key is unavailable — same guard as local - // spawn. Without this, a keyring outage would serialize `"private_key_nsec": ""` - // and launch the agent with no identity. if let Some(err) = crate::managed_agents::spawn_key_refusal(record) { return Err(err); } - // Merge global + persona + agent env_vars for provider deploy — the same - // live-persona-under-overrides semantics as local spawn. Global env vars - // are the lowest user-settable layer: global < persona < agent (last-wins - // on key collision). Without this, provider-backed agents wouldn't receive - // credentials saved on the persona or the agent itself. - let global_config = crate::managed_agents::load_global_agent_config(app).unwrap_or_default(); - let global_env = global_config.env_vars.clone(); - let persona_env = - crate::managed_agents::resolve_persona_env(app, record.persona_id.as_deref())?; - // Merge: global < persona (persona wins over global). - let global_persona_merged = crate::managed_agents::merged_user_env(&global_env, &persona_env); - // Merge: global+persona < agent (agent wins over everything). - let merged_env = - crate::managed_agents::merged_user_env(&global_persona_merged, &record.env_vars); - + let global = crate::managed_agents::load_global_agent_config(app).unwrap_or_default(); let personas = load_personas(app).unwrap_or_default(); - let cfg = crate::managed_agents::effective_config::resolve_effective_config( - record, - &personas, - &global_config, + let teams = crate::managed_agents::load_teams(app).unwrap_or_default(); + let persona_env = + crate::managed_agents::live_persona_env(&personas, record.persona_id.as_deref()); + let global_persona_env = crate::managed_agents::merged_user_env(&global.env_vars, &persona_env); + let merged_user_env = + crate::managed_agents::merged_user_env(&global_persona_env, &record.env_vars); + let effective = crate::managed_agents::effective_config::resolve_effective_config( + record, &personas, &global, ) .require_resolved()?; - let effective_model = cfg.model.value; - let effective_provider = cfg.provider.value; - let effective_prompt = cfg.system_prompt.value; + + ensure_remote_provider_supported(effective.provider.value.as_deref())?; + + let descriptor = + crate::managed_agents::resolve_effective_harness_descriptor(record, &personas, &global) + .map_err(|error| crate::managed_agents::user_facing_harness_error(&error))?; + let owner_pubkey = super::workspace_owner_hex(state)?; + let launch = build_launch_block( + record, + &descriptor, + &teams, + effective.system_prompt.value.as_deref(), + effective.model.value.as_deref(), + &owner_pubkey, + ); Ok(deploy_payload_json( record, @@ -95,23 +144,24 @@ pub(super) fn build_deploy_payload( &record.relay_url, &relay_ws_url_with_override(state), ), - effective_model, - effective_provider, - effective_prompt, - merged_env, + effective.model.value, + effective.provider.value, + effective.system_prompt.value, + merged_user_env, + launch, )) } -/// Pure serialization half of [`build_deploy_payload`] — every field the -/// provider harness receives is deliberately listed here, so payload -/// completeness is testable without an `AppHandle`. +/// Pure serialization half of [`build_deploy_payload`]. Legacy top-level fields +/// remain for display/bookkeeping; providers execute the resolved `launch` block. pub(super) fn deploy_payload_json( record: &ManagedAgentRecord, relay_url: String, effective_model: Option, effective_provider: Option, effective_prompt: Option, - merged_env: std::collections::BTreeMap, + merged_env: BTreeMap, + launch: serde_json::Value, ) -> serde_json::Value { serde_json::json!({ "name": &record.name, @@ -130,5 +180,81 @@ pub(super) fn deploy_payload_json( "respond_to": record.respond_to, "respond_to_allowlist": &record.respond_to_allowlist, "env_vars": merged_env, + "launch": launch, }) } + +#[cfg(test)] +mod tests { + use super::*; + use crate::managed_agents::{readiness::EffectiveHarnessDescriptor, RespondTo, TeamRecord}; + + fn record() -> ManagedAgentRecord { + serde_json::from_value(serde_json::json!({ + "pubkey": "abcd1234", + "name": "agent-handle", + "display_name": "Agent\u{0000} Name", + "private_key_nsec": "nsec1fake", + "relay_url": "wss://relay.example", + "acp_command": "buzz-acp", + "agent_command": "goose", + "agent_args": [], + "mcp_command": "", + "turn_timeout_seconds": 320, + "idle_timeout_seconds": 17, + "max_turn_duration_seconds": 23, + "parallelism": 4, + "respond_to": RespondTo::OwnerOnly, + "respond_to_allowlist": [], + "team_id": "team-1", + "created_at": "2026-01-01T00:00:00Z", + "updated_at": "2026-01-01T00:00:00Z" + })) + .unwrap() + } + + #[test] + fn launch_block_preserves_descriptor_and_spawn_policy() { + let record = record(); + let descriptor = EffectiveHarnessDescriptor { + command: "goose".into(), + args: vec!["acp".into()], + env: BTreeMap::from([ + ("GOOSE_MODE".into(), "custom".into()), + ("SECRET_FROM_PERSONA".into(), "secret".into()), + ]), + }; + let teams: Vec = serde_json::from_value(serde_json::json!([{ + "id": "team-1", "name": "Team", "instructions": "Coordinate", "persona_ids": [], "created_at": "2026-01-01T00:00:00Z", "updated_at": "2026-01-01T00:00:00Z" + }])).unwrap(); + + let launch = build_launch_block( + &record, + &descriptor, + &teams, + Some("prompt"), + Some("model"), + "owner-hex", + ); + + assert_eq!(launch["command"], "goose"); + assert_eq!(launch["args"], serde_json::json!(["acp"])); + assert_eq!(launch["env"]["GOOSE_MODE"], "custom"); + // policy_env is applied first, so this default remains separate from + // the descriptor value that wins in launch.env. + assert_eq!(launch["policy_env"]["GOOSE_MODE"], "auto"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_LAZY_POOL"], "true"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_RELAY_OBSERVER"], "true"); + assert_eq!( + launch["policy_env"]["BUZZ_ACP_TEAM_INSTRUCTIONS"], + "Coordinate" + ); + assert_eq!(launch["policy_env"]["BUZZ_ACP_SESSION_TITLE"], "Agent Name"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_SYSTEM_PROMPT"], "prompt"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_MODEL"], "model"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_IDLE_TIMEOUT"], "17"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_MAX_TURN_DURATION"], "23"); + assert_eq!(launch["policy_env"]["BUZZ_ACP_AGENTS"], "4"); + assert_eq!(launch["owner_pubkey"], "owner-hex"); + } +} diff --git a/desktop/src-tauri/src/commands/agents_tests.rs b/desktop/src-tauri/src/commands/agents_tests.rs index 03389d1d18..df135298c4 100644 --- a/desktop/src-tauri/src/commands/agents_tests.rs +++ b/desktop/src-tauri/src/commands/agents_tests.rs @@ -263,6 +263,19 @@ fn normalize_relay_mesh_trims_and_preserves_valid_config() { ); } +#[test] +fn deploy_refuses_resolved_relay_mesh_provider_with_padding() { + let record = bare_agent_record(Some("p1"), None, None); + let personas = vec![persona_record("p1", None, Some(" relay-mesh "))]; + let global = crate::managed_agents::GlobalAgentConfig::default(); + + let (_, provider) = resolve_deploy_model_provider(&record, &personas, &global); + let error = ensure_remote_provider_supported(provider.as_deref()) + .expect_err("resolved shared-compute provider must not deploy remotely"); + + assert!(error.contains("cannot be deployed remotely"), "{error}"); +} + #[test] fn created_avatar_prefers_explicit_input() { let resolved = resolve_created_avatar_url( @@ -398,50 +411,93 @@ fn legacy_avatar_empty_when_nothing_resolves() { // ── Provider deploy payload completeness ───────────────────────────────────── -/// Regression (PR #1667 review, Thufir): the provider deploy payload must -/// carry every behavioral field the local spawn path applies — a field -/// missing here silently strips it from provider-backed agents. +/// The shared provider fixture is the contract arbiter: it must be the exact +/// richest deploy request produced by the real desktop serializers. #[test] -fn deploy_payload_carries_the_full_behavioral_quad() { - let allow = "a".repeat(64); - let record: ManagedAgentRecord = serde_json::from_str(&format!( - r#"{{ - "pubkey": "abcd1234", - "name": "test-agent", - "private_key_nsec": "nsec1fake", - "relay_url": "wss://localhost:3000", - "acp_command": "buzz-acp", - "agent_command": "goose", - "agent_args": [], - "mcp_command": "", - "turn_timeout_seconds": 320, - "system_prompt": null, - "parallelism": 4, - "respond_to": "allowlist", - "respond_to_allowlist": ["{allow}"], - "created_at": "2026-01-01T00:00:00Z", - "updated_at": "2026-01-01T00:00:00Z", - "last_started_at": null, - "last_stopped_at": null, - "last_exit_code": null, - "last_error": null - }}"# - )) - .expect("sample record"); - - let payload = deploy_payload_json( +fn deploy_payload_matches_the_shared_full_launch_fixture() { + let fixture_path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join( + "../../crates/buzz-backend-kubernetes/tests/fixtures/provider-wire/deploy-full-launch.request.json", + ); + let fixture: serde_json::Value = serde_json::from_str( + &std::fs::read_to_string(&fixture_path) + .unwrap_or_else(|error| panic!("read {}: {error}", fixture_path.display())), + ) + .expect("parse shared provider fixture"); + let record: ManagedAgentRecord = serde_json::from_value(serde_json::json!({ + "pubkey": "abcd1234", + "name": "worker", + "private_key_nsec": "nsec1vl029mgpspedva04g90vltkh6fvh240zqtv9k0t9af8935ke9laqsnlfe5", + "relay_url": "wss://localhost:3000", + "auth_tag": "tag-1", + "acp_command": "buzz-acp", + "agent_command": "goose", + "runtime": "goose", + "model": "gpt-5", + "provider": "openai", + "env_vars": {"USER_KEY": "user-value"}, + "agent_args": [], + "mcp_command": "", + "turn_timeout_seconds": 300, + "system_prompt": null, + "idle_timeout_seconds": null, + "max_turn_duration_seconds": null, + "parallelism": 10, + "respond_to": "allowlist", + "respond_to_allowlist": ["aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"], + "created_at": "2026-01-01T00:00:00Z", + "updated_at": "2026-01-01T00:00:00Z" + })) + .expect("fixture source record"); + let descriptor = crate::managed_agents::resolve_effective_harness_descriptor( + &record, + &[], + &crate::managed_agents::GlobalAgentConfig::default(), + ) + .expect("resolve fixture source record descriptor"); + let launch = super::deploy::build_launch_block( &record, - "wss://relay.example".to_string(), - Some("gpt-x".to_string()), - Some("openai".to_string()), + &descriptor, + &[], None, - std::collections::BTreeMap::new(), + Some("gpt-5"), + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + ); + let agent = deploy_payload_json( + &record, + "wss://relay.example".into(), + Some("gpt-5".into()), + Some("openai".into()), + None, + std::collections::BTreeMap::from([("USER_KEY".into(), "user-value".into())]), + launch, ); - assert_eq!(payload["parallelism"], 4); - assert_eq!(payload["respond_to"], "allowlist"); - assert_eq!(payload["respond_to_allowlist"][0], "a".repeat(64)); - assert_eq!(payload["model"], "gpt-x"); - assert_eq!(payload["provider"], "openai"); - assert_eq!(payload["relay_url"], "wss://relay.example"); + assert_eq!( + agent, fixture["agent"], + "desktop payload drifted from the shared provider fixture" + ); +} + +#[test] +fn tauri_platform_configs_bundle_kubernetes_only_on_supported_hosts() { + use tauri_utils::{config::parse::read_from, platform::Target}; + + let config_root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")); + for (target, expected) in [ + (Target::MacOS, true), + (Target::Linux, true), + (Target::Windows, false), + ] { + let (config, paths) = read_from(target, config_root).expect("read Tauri config"); + let external_bins = config["bundle"]["externalBin"] + .as_array() + .expect("bundle.externalBin array"); + let has_kubernetes = external_bins + .iter() + .any(|value| value == "binaries/buzz-backend-kubernetes"); + assert_eq!( + has_kubernetes, expected, + "unexpected Kubernetes externalBin for {target}; merged {paths:?}" + ); + } } diff --git a/desktop/src-tauri/src/commands/export_util.rs b/desktop/src-tauri/src/commands/export_util.rs index 806f58d739..ded14679c1 100644 --- a/desktop/src-tauri/src/commands/export_util.rs +++ b/desktop/src-tauri/src/commands/export_util.rs @@ -1,16 +1,14 @@ use tauri::AppHandle; use tauri_plugin_dialog::DialogExt; -/// Show a save-file dialog with a custom filter and write `data` to the chosen -/// path. Returns `Ok(true)` when the file was written, `Ok(false)` when the -/// user cancelled the dialog. -pub async fn save_bytes_with_dialog( +/// Show a save-file dialog with a custom filter and return the chosen path, +/// or `None` when the user cancelled. Selection only — no write. +pub async fn pick_save_path( app: &AppHandle, suggested_filename: &str, filter_name: &str, extensions: &[&str], - data: &[u8], -) -> Result { +) -> Result, String> { let (tx, rx) = tokio::sync::oneshot::channel(); app.dialog() .file() @@ -23,12 +21,34 @@ pub async fn save_bytes_with_dialog( let selected = rx.await.map_err(|_| "dialog cancelled".to_string())?; let file_path = match selected { Some(p) => p, - None => return Ok(false), + None => return Ok(None), }; let dest = file_path .as_path() .ok_or_else(|| "Save dialog returned an invalid path".to_string())?; + Ok(Some(dest.to_path_buf())) +} + +/// Show a save-file dialog with a custom filter and write `data` to the chosen +/// path. Returns `Ok(true)` when the file was written, `Ok(false)` when the +/// user cancelled the dialog. +/// +/// NOT for secrets: the write is plain `std::fs::write` (no atomic commit, no +/// 0o600). Secret exports go through `pick_save_path` + +/// `key_backup::write_backup_file`. +pub async fn save_bytes_with_dialog( + app: &AppHandle, + suggested_filename: &str, + filter_name: &str, + extensions: &[&str], + data: &[u8], +) -> Result { + let dest = match pick_save_path(app, suggested_filename, filter_name, extensions).await? { + Some(p) => p, + None => return Ok(false), + }; + std::fs::write(dest, data).map_err(|e| format!("Failed to write file: {e}"))?; Ok(true) diff --git a/desktop/src-tauri/src/commands/identity.rs b/desktop/src-tauri/src/commands/identity.rs index 33783c05a5..33ecf3cfca 100644 --- a/desktop/src-tauri/src/commands/identity.rs +++ b/desktop/src-tauri/src/commands/identity.rs @@ -43,6 +43,7 @@ pub fn get_identity(state: State<'_, AppState>) -> Result Ok(IdentityInfo { pubkey: pubkey_hex, display_name, + storage: state.identity_storage().as_str().to_string(), lost, locked, reset_failed, @@ -135,23 +136,29 @@ pub async fn sign_event( } #[tauri::command] -pub fn decrypt_observer_event( +pub async fn decrypt_observer_event( event_json: String, state: State<'_, AppState>, ) -> Result { let keys = state.signing_keys()?; - let event = Event::from_json(event_json).map_err(|error| format!("invalid event: {error}"))?; - // Defense-in-depth: verify event ID and signature before decrypting. - if !event.verify_id() { - return Err("observer event has invalid ID".into()); - } - if !event.verify_signature() { - return Err("observer event has invalid signature".into()); - } + tauri::async_runtime::spawn_blocking(move || { + let event = + Event::from_json(event_json).map_err(|error| format!("invalid event: {error}"))?; + + // Defense-in-depth: verify event ID and signature before decrypting. + if !event.verify_id() { + return Err("observer event has invalid ID".into()); + } + if !event.verify_signature() { + return Err("observer event has invalid signature".into()); + } - buzz_core_pkg::observer::decrypt_observer_payload(&keys, &event) - .map_err(|error| format!("decrypt observer event failed: {error}")) + buzz_core_pkg::observer::decrypt_observer_payload(&keys, &event) + .map_err(|error| format!("decrypt observer event failed: {error}")) + }) + .await + .map_err(|e| format!("spawn_blocking failed: {e}"))? } #[tauri::command] @@ -188,14 +195,157 @@ pub fn get_nsec(state: State<'_, AppState>) -> Result { .map_err(|error| format!("encode nsec: {error}")) } +/// Generate a passphrase for a new encrypted backup (EFF short wordlist, OS +/// entropy). `words` is clamped to the range allowed by `key_backup`; +/// `separator` joins the words (defaults to a space). +#[tauri::command] +pub fn generate_backup_passphrase( + words: Option, + separator: Option, +) -> Result { + crate::key_backup::generate_passphrase( + words.map_or(crate::key_backup::DEFAULT_PASSPHRASE_WORDS, |w| w as usize), + separator.as_deref().unwrap_or(" "), + ) +} + +/// Core of [`create_ncryptsec_backup`], factored so tests can drive it with a +/// bare `AppState` + temp dir (and a fast scrypt tier) without an `AppHandle`. +pub(crate) fn create_backup_with_log_n( + state: &AppState, + password: &str, + log_n: u8, +) -> Result { + if password.chars().count() < crate::key_backup::MIN_PASSPHRASE_LEN { + return Err(format!( + "passphrase must be at least {} characters", + crate::key_backup::MIN_PASSPHRASE_LEN + )); + } + + // Serialize against import_identity/persist_current_identity: the blob + // must be derived from — and persisted for — one stable identity. Also + // caps KDF concurrency at one. + let _mutation_guard = state.identity_mutation.lock().map_err(|e| e.to_string())?; + + // Recovery mode (lost/locked) → Err, same gate as signing. + let keys = state.signing_keys()?; + + crate::key_backup::create_backup_blob(&keys, password, log_n) +} + +/// Create a NIP-49 backup of the live identity in memory. +/// +/// Encrypts under `password`, decrypt-verifies the fresh blob against the live +/// pubkey, and returns the `ncryptsec1…` string for the native save flow. The +/// body runs under `identity_mutation`, so identity changes cannot race the KDF. +#[tauri::command] +pub async fn create_ncryptsec_backup( + password: String, + app_handle: tauri::AppHandle, +) -> Result { + tokio::task::spawn_blocking(move || { + let password = zeroize::Zeroizing::new(password); + let state = app_handle.state::(); + create_backup_with_log_n(&state, &password, crate::key_backup::BACKUP_LOG_N) + }) + .await + .map_err(|e| format!("spawn_blocking failed: {e}"))? +} + +#[derive(Debug, serde::Serialize)] +#[serde(rename_all = "camelCase")] +pub struct BackupVerification { + pub pubkey: String, + pub npub: String, + pub matches_current_identity: bool, +} + +fn verify_ncryptsec_backup_inner( + state: &AppState, + ncryptsec: &str, + password: &str, +) -> Result { + let keys = crate::key_backup::decrypt_ncryptsec(ncryptsec, password)?; + let pubkey = keys.public_key(); + let current = state.signing_keys()?.public_key(); + Ok(BackupVerification { + pubkey: pubkey.to_hex(), + npub: pubkey + .to_bech32() + .map_err(|e| format!("encode backup identity: {e}"))?, + matches_current_identity: pubkey == current, + }) +} + +/// Decrypt and validate a NIP-49 backup without exposing its secret key. +#[tauri::command] +pub async fn verify_ncryptsec_backup( + ncryptsec: String, + password: String, + app_handle: tauri::AppHandle, +) -> Result { + tokio::task::spawn_blocking(move || { + let password = zeroize::Zeroizing::new(password); + let state = app_handle.state::(); + verify_ncryptsec_backup_inner(&state, &ncryptsec, &password) + }) + .await + .map_err(|e| format!("spawn_blocking failed: {e}"))? +} + +/// Save a portable copy of an `ncryptsec1…` backup to a user-chosen path. +/// +/// The input must parse as a structurally valid NIP-49 payload. The dialog is +/// selection-only; the write uses secret-file semantics (atomic + 0o600). +/// Never mutates canonical app state. Returns the chosen path, or `None` when +/// the user cancelled. +#[tauri::command] +pub async fn save_ncryptsec_copy( + ncryptsec: String, + app_handle: tauri::AppHandle, +) -> Result, String> { + // Reject anything that is not a valid encrypted-key blob — this command + // must not become a generic file writer. + crate::key_backup::parse_ncryptsec(&ncryptsec)?; + let normalized = ncryptsec.trim().to_string(); + + let dest = match crate::commands::export_util::pick_save_path( + &app_handle, + crate::key_backup::BACKUP_FILE_NAME, + "Password-protected key backup", + &["ncryptsec"], + ) + .await? + { + Some(p) => p, + None => return Ok(None), + }; + + let dest_for_write = dest.clone(); + tokio::task::spawn_blocking(move || { + crate::key_backup::write_backup_file(&dest_for_write, &normalized) + }) + .await + .map_err(|e| format!("spawn_blocking failed: {e}"))??; + + Ok(Some(dest.display().to_string())) +} + #[tauri::command] pub async fn import_identity( nsec: String, + password: Option, app_handle: tauri::AppHandle, ) -> Result { tokio::task::spawn_blocking(move || { - let trimmed = nsec.trim(); - let keys = Keys::parse(trimmed).map_err(|e| format!("Invalid private key: {e}"))?; + // NIP-49 backups require a passphrase and decrypt entirely in Rust. + // Raw nsec/hex input follows the existing parser path unchanged. + let password = password.map(zeroize::Zeroizing::new); + let keys = crate::key_backup::recover_keys_from_input( + &nsec, + password.as_ref().map(|value| value.as_str()), + )?; // Serialize against persist_current_identity: hold this guard for the // full function body so a concurrent stale persist can't overwrite @@ -210,30 +360,14 @@ pub async fn import_identity( std::fs::create_dir_all(&data_dir).map_err(|e| format!("create app data dir: {e}"))?; let key_path = data_dir.join("identity.key"); - // Persist into the OS keyring first (store → read-back verify → marker → - // delete file). Falls back to the 0o600 file when the keyring is - // unavailable; returns Err only when both backends fail. - let store = crate::secret_store::SecretStore::shared(crate::app_state::keyring_service()); - crate::app_state::persist_imported_identity(store, &keys, &key_path, &data_dir)?; - - // Update in-memory keys BEFORE clearing recovery flags. The Release - // stores below pair with Acquire loads in get_identity: a reader - // observing false is guaranteed to see the updated keys. - let pubkey = keys.public_key(); - *state.keys.lock().map_err(|e| e.to_string())? = keys; - - // Clear both recovery flags — an import is valid in either lost or - // keyring-locked state and resolves both. In the locked case the - // keyring is unreachable, so persist_imported_identity already fell - // back to identity.key; on the next Unreachable boot the file is - // loaded directly and when the keyring returns the adoption path - // picks it up. - state - .identity_lost - .store(false, std::sync::atomic::Ordering::Release); - state - .keyring_locked - .store(false, std::sync::atomic::Ordering::Release); + let (pubkey, storage) = commit_imported_identity(&state, &data_dir, keys, |keys| { + // Persist into the OS keyring first (store → read-back verify → + // marker → delete file). Falls back to the 0o600 file when the + // keyring is unavailable; returns Err only when both backends fail. + let store = + crate::secret_store::SecretStore::shared(crate::app_state::keyring_service()); + crate::app_state::persist_imported_identity(store, keys, &key_path, &data_dir) + })?; let pubkey_hex = pubkey.to_hex(); let display_name = truncated_display_name(&pubkey)?; @@ -243,6 +377,7 @@ pub async fn import_identity( Ok(IdentityInfo { pubkey: pubkey_hex, display_name, + storage: storage.as_str().to_string(), lost: false, locked: false, reset_failed: false, @@ -252,6 +387,69 @@ pub async fn import_identity( .map_err(|e| format!("spawn_blocking failed: {e}"))? } +/// Commit an imported identity: durably persist, swap in-memory keys, clear +/// recovery flags, then remove the previous identity's stale app-managed +/// backup. Caller must hold `state.identity_mutation`. +/// +/// Ordering is the contract: +/// +/// 1. `persist` runs FIRST. If it fails (`Err` from both keyring and file +/// fallback), nothing has changed — the previous identity stays live in +/// memory AND its valid canonical `identity.ncryptsec` stays on disk. +/// 2. Only after durable persistence do we swap `state.keys` and clear the +/// recovery flags. +/// 3. Stale-backup cleanup runs LAST and is deliberately best-effort: at that +/// point the import is durably committed, so reporting a cleanup failure +/// as a command `Err` would claim a half-applied import that actually +/// succeeded. The leftover blob is still passphrase-encrypted and is +/// replaced by the next backup creation; we log and move on. +fn commit_imported_identity( + state: &AppState, + data_dir: &std::path::Path, + keys: nostr::Keys, + persist: impl FnOnce(&nostr::Keys) -> Result, +) -> Result<(nostr::PublicKey, crate::app_state::IdentityStorage), String> { + // Capture the previous pubkey up front for post-commit cleanup. + let previous_pubkey = state.keys.lock().map_err(|e| e.to_string())?.public_key(); + + let storage = persist(&keys)?; + + // Update in-memory keys BEFORE clearing recovery flags. The Release + // stores below pair with Acquire loads in get_identity: a reader + // observing false is guaranteed to see the updated keys. + let pubkey = keys.public_key(); + { + let mut active_keys = state.keys.lock().map_err(|e| e.to_string())?; + *active_keys = keys; + state.set_identity_storage(storage); + } + + // Clear both recovery flags — an import is valid in either lost or + // keyring-locked state and resolves both. In the locked case the + // keyring is unreachable, so the persist step already fell back to + // identity.key; on the next Unreachable boot the file is loaded + // directly and when the keyring returns the adoption path picks it up. + state + .identity_lost + .store(false, std::sync::atomic::Ordering::Release); + state + .keyring_locked + .store(false, std::sync::atomic::Ordering::Release); + + // Importing a different identity invalidates the app-managed backup: it + // encrypts the previous key and must not linger mislabeled. Best-effort + // per the ordering contract above. + if let Err(e) = crate::key_backup::cleanup_stale_backup(&previous_pubkey, &pubkey, data_dir) { + eprintln!( + "buzz-desktop: import committed, but stale key backup cleanup failed: {e}; \ + the leftover identity.ncryptsec encrypts the PREVIOUS key and will be \ + replaced by the next backup creation" + ); + } + + Ok((pubkey, storage)) +} + /// Make the current ephemeral identity durable by persisting it to the OS /// keyring (or falling back to identity.key). This is called when the user /// chooses to start a new identity instead of re-importing their previous one @@ -295,11 +493,12 @@ pub async fn persist_current_identity( let key_path = data_dir.join("identity.key"); let store = crate::secret_store::SecretStore::shared(crate::app_state::keyring_service()); - crate::app_state::persist_imported_identity(store, &keys, &key_path, &data_dir)?; + let storage = + crate::app_state::persist_imported_identity(store, &keys, &key_path, &data_dir)?; - // Keys are already the live identity — only clear identity_lost. - // Release pairs with Acquire in get_identity so readers see - // consistent state. + // Keys are already the live identity. Record where the durable write + // landed before clearing identity_lost. + state.set_identity_storage(storage); state .identity_lost .store(false, std::sync::atomic::Ordering::Release); @@ -311,6 +510,7 @@ pub async fn persist_current_identity( Ok(IdentityInfo { pubkey: pubkey_hex, display_name, + storage: storage.as_str().to_string(), lost: false, locked: false, reset_failed: false, @@ -583,3 +783,7 @@ mod nostr_identity_binding_tests { assert_eq!(error, "expires_at is expired"); } } + +#[cfg(test)] +#[path = "identity_key_backup_tests.rs"] +mod identity_key_backup_tests; diff --git a/desktop/src-tauri/src/commands/identity_key_backup_tests.rs b/desktop/src-tauri/src/commands/identity_key_backup_tests.rs new file mode 100644 index 0000000000..c36af66879 --- /dev/null +++ b/desktop/src-tauri/src/commands/identity_key_backup_tests.rs @@ -0,0 +1,139 @@ +use super::{create_backup_with_log_n, verify_ncryptsec_backup_inner}; +use crate::app_state::build_app_state; +use nostr::{Keys, ToBech32}; + +/// Fast scrypt tier for tests; production uses BACKUP_LOG_N (18), covered +/// once in key_backup_tests::round_trip_at_production_cost. +const FAST_LOG_N: u8 = 16; +const PASSWORD: &str = "correct horse battery"; + +#[test] +fn verification_returns_only_public_identity_and_match_status() { + let state = build_app_state(); + let backup = create_backup_with_log_n(&state, PASSWORD, FAST_LOG_N).unwrap(); + let result = verify_ncryptsec_backup_inner(&state, &backup, PASSWORD).unwrap(); + assert_eq!( + result.pubkey, + state.keys.lock().unwrap().public_key().to_hex() + ); + assert!(result.npub.starts_with("npub1")); + assert!(result.matches_current_identity); +} + +#[test] +fn verification_reports_valid_backup_for_a_different_identity() { + let state = build_app_state(); + let other = Keys::generate(); + let backup = crate::key_backup::create_backup_blob(&other, PASSWORD, FAST_LOG_N).unwrap(); + let result = verify_ncryptsec_backup_inner(&state, &backup, PASSWORD).unwrap(); + assert_eq!(result.pubkey, other.public_key().to_hex()); + assert!(!result.matches_current_identity); +} + +#[test] +fn verification_rejects_wrong_password() { + let state = build_app_state(); + let backup = + crate::key_backup::create_backup_blob(&Keys::generate(), PASSWORD, FAST_LOG_N).unwrap(); + assert_eq!( + verify_ncryptsec_backup_inner(&state, &backup, "wrong password").unwrap_err(), + "wrong backup password or damaged key backup" + ); +} + +#[test] +fn verification_accepts_maximum_supported_kdf_cost() { + let state = build_app_state(); + let backup = crate::key_backup::create_backup_blob( + &Keys::generate(), + PASSWORD, + crate::key_backup::MAX_VERIFY_LOG_N, + ) + .unwrap(); + verify_ncryptsec_backup_inner(&state, &backup, PASSWORD).unwrap(); +} + +#[test] +fn verification_rejects_unsupported_kdf_cost_before_decryption() { + let state = build_app_state(); + let supported = + crate::key_backup::create_backup_blob(&Keys::generate(), PASSWORD, FAST_LOG_N).unwrap(); + let encrypted = crate::key_backup::parse_ncryptsec(&supported).unwrap(); + let mut payload = encrypted.as_vec(); + payload[1] = crate::key_backup::MAX_VERIFY_LOG_N + 1; + let unsupported = nostr::nips::nip49::EncryptedSecretKey::from_slice(&payload) + .unwrap() + .to_bech32() + .unwrap(); + + let err = verify_ncryptsec_backup_inner(&state, &unsupported, PASSWORD).unwrap_err(); + assert_eq!( + err, + format!( + "unsupported backup KDF cost: log_n {} exceeds maximum {}", + crate::key_backup::MAX_VERIFY_LOG_N + 1, + crate::key_backup::MAX_VERIFY_LOG_N + ) + ); +} + +#[test] +fn rejects_short_passphrase() { + let state = build_app_state(); + let err = create_backup_with_log_n(&state, "short", FAST_LOG_N).unwrap_err(); + assert!(err.contains("at least"), "{err}"); +} + +#[test] +fn recovery_mode_blocks_backup_creation() { + let state = build_app_state(); + + state + .identity_lost + .store(true, std::sync::atomic::Ordering::Release); + assert!( + create_backup_with_log_n(&state, PASSWORD, FAST_LOG_N).is_err(), + "lost identity must not be backed up" + ); + state + .identity_lost + .store(false, std::sync::atomic::Ordering::Release); + + state + .keyring_locked + .store(true, std::sync::atomic::Ordering::Release); + assert!( + create_backup_with_log_n(&state, PASSWORD, FAST_LOG_N).is_err(), + "locked keyring must not be backed up" + ); +} + +/// Concurrent identity changes serialize with backup creation. +#[test] +fn concurrent_identity_swap_vs_backup_is_serialized() { + let state = std::sync::Arc::new(build_app_state()); + let key_a = state.keys.lock().unwrap().clone(); + let key_b = Keys::generate(); + + let swapper = { + let state = state.clone(); + let key_b = key_b.clone(); + std::thread::spawn(move || { + // Mirrors import_identity's locking: mutation guard held + // across the key swap. + let _guard = state.identity_mutation.lock().unwrap(); + *state.keys.lock().unwrap() = key_b; + }) + }; + + let backup = create_backup_with_log_n(&state, PASSWORD, FAST_LOG_N).unwrap(); + swapper.join().unwrap(); + + let recovered = crate::key_backup::decrypt_ncryptsec(&backup, PASSWORD) + .unwrap() + .public_key(); + assert!( + recovered == key_a.public_key() || recovered == key_b.public_key(), + "backup must match one coherent identity" + ); +} diff --git a/desktop/src-tauri/src/commands/media.rs b/desktop/src-tauri/src/commands/media.rs index bf8692ff70..ed3b340238 100644 --- a/desktop/src-tauri/src/commands/media.rs +++ b/desktop/src-tauri/src/commands/media.rs @@ -411,7 +411,7 @@ fn should_retry_legacy_upload(status: reqwest::StatusCode) -> bool { } async fn send_upload_attempt( - state: &State<'_, AppState>, + state: &AppState, url: String, auth_header: &str, mime: &str, @@ -455,10 +455,22 @@ async fn send_upload_attempt( response.map_err(|error| classify_request_error(&error)) } +pub(crate) async fn upload_image_bytes( + body: Vec, + state: &AppState, +) -> Result { + let mime = detect_and_validate_mime(&body)?; + if !mime.starts_with("image/") { + return Err("profile avatar must be an image".to_string()); + } + let body = sanitize_image_for_upload(body, &mime)?; + do_upload(body, &mime, state, None).await +} + async fn do_upload( body: Vec, mime: &str, - state: &State<'_, AppState>, + state: &AppState, progress: Option<(tauri::AppHandle, String)>, ) -> Result { let sha256 = hex::encode(Sha256::digest(&body)); @@ -559,7 +571,7 @@ pub async fn upload_media( /// files from ever leaving the client on image-only surfaces. async fn process_picked_path( path: std::path::PathBuf, - state: &State<'_, AppState>, + state: &AppState, images_only: bool, ) -> Result { // Pin the inode by opening the fd BEFORE spawn_blocking. This prevents a diff --git a/desktop/src-tauri/src/commands/media_download.rs b/desktop/src-tauri/src/commands/media_download.rs index d3b1a9499d..7bc94da25d 100644 --- a/desktop/src-tauri/src/commands/media_download.rs +++ b/desktop/src-tauri/src/commands/media_download.rs @@ -8,7 +8,8 @@ use crate::commands::export_util::save_bytes_with_dialog; use crate::commands::media::{detect_and_validate_mime, mint_media_get_auth, sanitize_filename}; use crate::commands::{ personas::{ - decode_snapshot_from_bytes, MAX_SNAPSHOT_JSON_BYTES, MAX_SNAPSHOT_PNG_BYTES, PNG_MAGIC, + parse_snapshot_payload_from_bytes, MAX_SNAPSHOT_JSON_BYTES, MAX_SNAPSHOT_PNG_BYTES, + PNG_MAGIC, }, team_snapshot::{ decode_team_snapshot_from_bytes, MAX_TEAM_SNAPSHOT_JSON_BYTES, MAX_TEAM_SNAPSHOT_PNG_BYTES, @@ -505,10 +506,12 @@ pub async fn fetch_snapshot_bytes( // 4. Bytes must parse as the snapshot type selected by the filename. // Team parsing rejects retired flat JSON and persona-pack ZIP inputs - // before anything reaches the frontend. + // before anything reaches the frontend. Agent kinds accept both plain + // manifests and structurally valid locked (encrypted) card envelopes — + // transit validation never decrypts; unlock happens at import time. match kind { SnapshotFileKind::AgentJson | SnapshotFileKind::AgentPng => { - decode_snapshot_from_bytes(&bytes) + parse_snapshot_payload_from_bytes(&bytes) .map_err(|e| format!("invalid agent snapshot: {e}"))?; } SnapshotFileKind::TeamJson | SnapshotFileKind::TeamPng => { diff --git a/desktop/src-tauri/src/commands/media_snapshot_png.rs b/desktop/src-tauri/src/commands/media_snapshot_png.rs index 734d8f5dc8..bcaec6a592 100644 --- a/desktop/src-tauri/src/commands/media_snapshot_png.rs +++ b/desktop/src-tauri/src/commands/media_snapshot_png.rs @@ -204,6 +204,7 @@ mod tests { s3_secret_key: String::new(), s3_bucket: String::new(), s3_region: "us-east-1".to_string(), + s3_addressing_style: buzz_media_pkg::S3AddressingStyle::Path, max_image_bytes: 50 * 1024 * 1024, max_gif_bytes: 10 * 1024 * 1024, max_video_bytes: 524_288_000, diff --git a/desktop/src-tauri/src/commands/mesh_llm.rs b/desktop/src-tauri/src/commands/mesh_llm.rs index 305c54a203..528ca38767 100644 --- a/desktop/src-tauri/src/commands/mesh_llm.rs +++ b/desktop/src-tauri/src/commands/mesh_llm.rs @@ -5,12 +5,35 @@ use tauri::{AppHandle, Manager, State}; use crate::{app_state::AppState, mesh_llm, relay}; -#[derive(Debug, serde::Deserialize, serde::Serialize)] +#[derive(Clone, Debug, serde::Deserialize, serde::Serialize)] #[serde(rename_all = "camelCase")] struct MeshSharingConfig { enabled: bool, + /// A fresh Share Compute request that must cross a process boundary before + /// it can start. Consumed before startup so an interrupted download is not + /// resumed on a later launch. + #[serde(default)] + start_on_next_launch: bool, model_id: String, max_vram_gb: Option, + /// Community relay where Share Compute was explicitly enabled. Older + /// configs predate community binding and restore against the active relay. + #[serde(default)] + relay_url: Option, +} + +fn pending_new_start_checkpoint(config: &MeshSharingConfig) -> MeshSharingConfig { + let mut checkpoint = config.clone(); + checkpoint.enabled = false; + checkpoint.start_on_next_launch = false; + checkpoint +} + +fn one_shot_restart_checkpoint(config: &MeshSharingConfig) -> MeshSharingConfig { + let mut checkpoint = config.clone(); + checkpoint.enabled = false; + checkpoint.start_on_next_launch = true; + checkpoint } fn mesh_sharing_config_path(app: &AppHandle) -> Result { @@ -89,8 +112,10 @@ fn sharing_config_from_request( .ok_or_else(|| "modelId is required for serve mode".to_string())?; Ok(MeshSharingConfig { enabled: true, + start_on_next_launch: false, model_id: model_id.to_string(), max_vram_gb: request.max_vram_gb, + relay_url: request.relay_url.clone(), }) } @@ -117,7 +142,7 @@ fn restart_to_share( app: &AppHandle, config: &MeshSharingConfig, ) -> CmdResult { - save_mesh_sharing_config(app, config)?; + save_mesh_sharing_config(app, &one_shot_restart_checkpoint(config))?; let status = restarting_share_status(config); app.request_restart(); Ok(status) @@ -133,7 +158,7 @@ fn buzz_mesh_name_for_relay(relay_url: &str) -> String { format!("buzz-community-{}", &digest[..32]) } -fn buzz_mesh_name(state: &AppState) -> String { +pub(super) fn buzz_mesh_name(state: &AppState) -> String { buzz_mesh_name_for_relay(&relay::relay_ws_url_with_override(state)) } @@ -150,8 +175,13 @@ fn advance_mesh_status_cursor( Ok(cursor) } -async fn query_mesh_discovery_events(state: &AppState) -> Result, String> { - let mut events = relay::query_relay(state, &[mesh_llm::relay_membership_filter()]).await?; +async fn query_mesh_discovery_events_at( + state: &AppState, + relay_url: &str, +) -> Result, String> { + let api_base_url = relay::relay_http_base_url(relay_url); + let mut events = + relay::query_relay_at(state, &api_base_url, &[mesh_llm::relay_membership_filter()]).await?; let member_pubkeys = mesh_llm::current_member_pubkeys(&events); if member_pubkeys.is_empty() { // Distinguish "relay returned a membership snapshot listing zero @@ -172,7 +202,7 @@ async fn query_mesh_discovery_events(state: &AppState) -> Result = None; loop { - let page = relay::query_relay(state, &[status_filter.clone()]).await?; + let page = relay::query_relay_at(state, &api_base_url, &[status_filter.clone()]).await?; let done = page.len() < mesh_llm::MESH_STATUS_PAGE_SIZE; if !done { let cursor = advance_mesh_status_cursor(&mut status_filter, &page)?; @@ -188,6 +218,10 @@ async fn query_mesh_discovery_events(state: &AppState) -> Result Result, String> { + query_mesh_discovery_events_at(state, &relay::relay_ws_url_with_override(state)).await +} + /// Resolve the admission roster by intersecting member-signed mesh status /// reporters with the current NIP-43 direct-member list. /// @@ -201,6 +235,14 @@ pub(crate) async fn resolve_trusted_owner_ids(state: &AppState) -> Result Result, String> { + let events = query_mesh_discovery_events_at(state, relay_url).await?; + Ok(mesh_llm::owner_ids_from_events(&events)) +} + /// Resolve the roster for an initial node *start*, failing closed to self-only /// (an empty roster) when the relay query fails. This is safe only at start: /// there is no established allowlist to preserve yet. The periodic @@ -244,10 +286,11 @@ fn buzz_mesh_join_targets( /// Resolve the validated member endpoint this runtime should join to enter the /// existing Buzz community mesh. `Ok(None)` means this machine is the first /// live serving member (or is itself the shared bootstrap contact). -pub(crate) async fn resolve_buzz_mesh_join_targets( +pub(crate) async fn resolve_buzz_mesh_join_targets_at( state: &AppState, + relay_url: &str, ) -> Result, String> { - let events = query_mesh_discovery_events(state).await?; + let events = query_mesh_discovery_events_at(state, relay_url).await?; let self_owner_id = mesh_llm::ensure_owner_identity() .map_err(|error| format!("failed to load mesh owner identity: {error}"))? .owner_id; @@ -261,8 +304,11 @@ pub(crate) async fn resolve_buzz_mesh_join_targets( /// snapshot. A node start used to repeat the full membership + status query /// for each value, making Share Compute startup both slower and more exposed /// to inconsistent snapshots. -async fn resolve_buzz_mesh_startup(state: &AppState) -> (Vec, Option) { - match query_mesh_discovery_events(state).await { +async fn resolve_buzz_mesh_startup_at( + state: &AppState, + relay_url: &str, +) -> (Vec, Option) { + match query_mesh_discovery_events_at(state, relay_url).await { Ok(events) => { let trusted_owner_ids = mesh_llm::owner_ids_from_events(&events); let join_token = mesh_llm::ensure_owner_identity() @@ -291,33 +337,66 @@ async fn resolve_buzz_mesh_startup(state: &AppState) -> (Vec, Option CmdResult<()> { - let Some(config) = load_mesh_sharing_config(app)? else { + let Some(mut config) = load_mesh_sharing_config(app)? else { return Ok(()); }; - if !config.enabled || config.model_id.trim().is_empty() { + if (!config.enabled && !config.start_on_next_launch) || config.model_id.trim().is_empty() { return Ok(()); } + config.model_id = mesh_llm::canonical_curated_model_id(&config.model_id).to_string(); if state.mesh_llm_runtime.lock().await.is_some() { return Ok(()); } - let (trusted_owner_ids, join_token) = resolve_buzz_mesh_startup(state).await; + let relay_url = config + .relay_url + .clone() + .unwrap_or_else(|| relay::relay_ws_url_with_override(state)); + let (trusted_owner_ids, join_token) = resolve_buzz_mesh_startup_at(state, &relay_url).await; let mut runtime = state.mesh_llm_runtime.lock().await; if runtime.is_some() { return Ok(()); } + if config.start_on_next_launch { + // Consume a role-switch request before doing any potentially long model + // work. If Buzz exits during that work, the next launch stays stopped. + config = pending_new_start_checkpoint(&config); + save_mesh_sharing_config(app, &config)?; + } + // This is restoration of a previously inference-ready serving node. Keep + // the enabled checkpoint armed while restoring so a transient startup + // failure does not silently turn Share Compute off. let request = mesh_llm::StartMeshNodeRequest { mode: mesh_llm::MeshNodeMode::Serve, - model_id: Some(config.model_id), + model_id: Some(config.model_id.clone()), max_vram_gb: config.max_vram_gb, join_token, - mesh_name: Some(buzz_mesh_name(state)), + mesh_name: Some(buzz_mesh_name_for_relay(&relay_url)), + relay_url: Some(relay_url), trusted_owner_ids: Some(trusted_owner_ids), }; let started = mesh_llm::DesktopMeshRuntime::start(request) .await .map_err(|error| format!("failed to restore Share Compute: {error:#}"))?; + // Install the restored runtime immediately: it is tracked by AppState from + // here on, so it can never be orphaned. Restoring a previously + // inference-ready node still has to load ~tens of GB of weights and may + // download package layers after the ports bind, and the readiness probe + // itself serializes behind any first inference. None of that is a failed + // restore — stopping the node and reporting failure (the old behaviour) + // tore down a node that was simply still warming up. The checkpoint stays + // armed (`enabled`), so a genuinely broken restore is retried next launch + // rather than silently turning Share Compute off. *runtime = Some(started); + config.enabled = true; + config.start_on_next_launch = false; + save_mesh_sharing_config(app, &config)?; drop(runtime); + if let Err(error) = wait_for_mesh_inference(&config.model_id).await { + eprintln!( + "buzz-mesh: restored node is not inference-ready yet ({error}); \ + leaving it to warm up without tearing it down" + ); + } mesh_llm::publish_current_status_once(app, "restore").await; Ok(()) } @@ -328,6 +407,11 @@ pub async fn mesh_start_node( state: State<'_, AppState>, mut request: mesh_llm::StartMeshNodeRequest, ) -> CmdResult { + let relay_url = relay::relay_ws_url_with_override(&state); + request.relay_url = Some(relay_url.clone()); + if let Some(model_id) = request.model_id.as_mut() { + *model_id = mesh_llm::canonical_curated_model_id(model_id).to_string(); + } let sharing_config = if request.mode == mesh_llm::MeshNodeMode::Serve { Some(sharing_config_from_request(&request)?) } else { @@ -362,13 +446,14 @@ pub async fn mesh_start_node( // Frontend requests never carry a roster. Resolve it and the bootstrap // endpoint from one snapshot so UI startup does not repeat relay probes. if request.trusted_owner_ids.is_none() || request.join_token.is_none() { - let (trusted_owner_ids, join_token) = resolve_buzz_mesh_startup(&state).await; + let (trusted_owner_ids, join_token) = + resolve_buzz_mesh_startup_at(&state, &relay_url).await; request.trusted_owner_ids.get_or_insert(trusted_owner_ids); if request.join_token.is_none() { request.join_token = join_token; } } - request.mesh_name = Some(buzz_mesh_name(&state)); + request.mesh_name = Some(buzz_mesh_name_for_relay(&relay_url)); let mut runtime = state.mesh_llm_runtime.lock().await; let plan = match runtime.as_ref() { @@ -386,6 +471,15 @@ pub async fn mesh_start_node( return Err("mesh node is already running".to_string()); } + if let Some(config) = sharing_config.as_ref() { + // Persist a DISARMED checkpoint to cover the window of the potentially + // long `start()` below: if Buzz exits before the runtime is installed + // and tracked, the next launch stays stopped rather than trying to + // restore a node that never came up. The enabled config is armed right + // after install succeeds. + save_mesh_sharing_config(&app, &pending_new_start_checkpoint(config))?; + } + let started = mesh_llm::DesktopMeshRuntime::start(request) .await .map_err(|error| format!("{error:#}"))?; @@ -409,10 +503,28 @@ pub async fn mesh_start_node( )); } }; + // Install (track) the runtime BEFORE probing readiness so it can never be + // orphaned. A readiness timeout is not death: mesh binds its ports before + // weights finish loading / layers finish downloading, and serializes all + // ingress HTTP (this probe included) behind any in-flight turn — a cold + // start can take minutes. The old code stopped the node and restarted the + // app on that timeout, turning startup latency into a restart loop. *runtime = Some(started); drop(runtime); if let Some(config) = sharing_config.as_ref() { + // Installed + tracked == Share Compute is on, so persist the enabled + // config now (mirroring restore), not gated on the probe. Gating it + // meant a slow first start served fine but came back OFF next launch. + // Safe: neither the watchdog (evicts only a closed port) nor restore + // (leaves a warming node alone) can loop a slow-but-alive node, and an + // unstartable config fails earlier in `start()`. Probe is informational. save_mesh_sharing_config(&app, config)?; + if let Err(error) = wait_for_mesh_inference(&config.model_id).await { + eprintln!( + "buzz-mesh: node started but inference is not ready yet ({error}); \ + leaving it to warm up (Share Compute stays armed for next launch)" + ); + } } mesh_llm::publish_current_status_once(&app, "start").await; Ok(status) @@ -612,6 +724,7 @@ pub(crate) async fn ensure_client_node_for_model( max_vram_gb: None, join_token: Some(join_token.clone()), mesh_name: Some(buzz_mesh_name(state)), + relay_url: Some(relay::relay_ws_url_with_override(state)), trusted_owner_ids: Some(resolve_trusted_owner_ids_or_self_only(state).await), }; let mut runtime = state.mesh_llm_runtime.lock().await; @@ -753,6 +866,18 @@ pub(crate) async fn ensure_relay_mesh_for_record( } } } + + // A persisted Share Compute configuration is authoritative about this + // machine's role. If no runtime is currently tracked (for example after a + // clean process restart), restore the serving node instead of treating an + // agent request as permission to replace it with a client node. + if load_mesh_sharing_config(app)? + .is_some_and(|config| config.enabled && !config.model_id.trim().is_empty()) + { + restore_mesh_sharing(app, &state).await?; + return wait_for_mesh_inference(model_id).await; + } + let target = match resolve_mesh_bootstrap_target(&state, model_id).await { Ok(Some(target)) => target, Ok(None) => { @@ -768,15 +893,9 @@ pub(crate) async fn ensure_relay_mesh_for_record( } }; - // Serve→Client re-arm transition (micspiral review #3, intentional-by-design): - // if the dead ingress belonged to a *serve* node with running consumer - // agents, this re-arms it as a Client (`MeshNodeMode::Client`). That is the - // correct/safe recovery here — config-backed serve restoration is - // `restore_mesh_sharing`'s job (`MeshNodeMode::Serve`), and - // `ensure_client_node_for_model` reuses any live runtime of *either* mode - // (the router resolves per-request), so it only cold-starts a Client when - // there is genuinely no runtime. Falling back to Client if a serve node - // crashed under local pressure is a desirable fail-safe, not a regression. + // No serving configuration exists, so this is a genuine consumer-only + // start. A configured serving machine is restored above and never reaches + // this client fallback. ensure_client_node_for_model(&state, model_id, Some(target.endpoint_addr)).await?; wait_for_mesh_inference(model_id).await } @@ -792,14 +911,17 @@ pub async fn mesh_stop_node( // role under the lock and, when it's a consume session, leave it running // and return its live status unchanged. The frontend also guards this, but // status can be stale between polls, so the backend is authoritative. - let taken = { + let (taken, bound_relay_url) = { let mut guard = state.mesh_llm_runtime.lock().await; if let Some(runtime) = guard.as_ref() { if !share_stop_should_teardown(runtime.mode()) { return runtime.status().await.map_err(|error| error.to_string()); } } - guard.take() + let bound_relay_url = guard + .as_ref() + .and_then(|runtime| runtime.start_request().relay_url.clone()); + (guard.take(), bound_relay_url) }; if let Some(runtime) = taken { runtime.stop().await.map_err(|error| error.to_string())?; @@ -808,11 +930,13 @@ pub async fn mesh_stop_node( &app, &MeshSharingConfig { enabled: false, + start_on_next_launch: false, model_id: String::new(), max_vram_gb: None, + relay_url: None, }, )?; - mesh_llm::publish_stopped_status_once(&app, "stop").await; + mesh_llm::publish_stopped_status_once_at(&app, bound_relay_url.as_deref(), "stop").await; Ok(mesh_llm::stopped_status()) } diff --git a/desktop/src-tauri/src/commands/mesh_llm_tests.rs b/desktop/src-tauri/src/commands/mesh_llm_tests.rs index ccc5287d62..26eb1f5fba 100644 --- a/desktop/src-tauri/src/commands/mesh_llm_tests.rs +++ b/desktop/src-tauri/src/commands/mesh_llm_tests.rs @@ -110,6 +110,74 @@ fn buzz_mesh_name_is_stable_and_does_not_expose_the_relay() { assert!(!first.contains("example")); } +#[test] +fn sharing_config_keeps_the_community_where_sharing_was_enabled() { + let request = mesh_llm::StartMeshNodeRequest { + mode: mesh_llm::MeshNodeMode::Serve, + model_id: Some("test-model".to_string()), + max_vram_gb: Some(24), + join_token: None, + mesh_name: Some("buzz-community-test".to_string()), + relay_url: Some("wss://community.example".to_string()), + trusted_owner_ids: Some(Vec::new()), + }; + + let config = sharing_config_from_request(&request).expect("valid sharing config"); + assert_eq!(config.relay_url.as_deref(), Some("wss://community.example")); +} + +#[test] +fn legacy_sharing_config_without_community_binding_still_loads() { + let config: MeshSharingConfig = serde_json::from_value(serde_json::json!({ + "enabled": true, + "modelId": "test-model", + "maxVramGb": null + })) + .expect("legacy sharing config"); + + assert_eq!(config.relay_url, None); + assert!(!config.start_on_next_launch); +} + +#[test] +fn new_start_checkpoint_prevents_incomplete_download_restore() { + let config = MeshSharingConfig { + enabled: true, + start_on_next_launch: false, + model_id: "test-model".to_string(), + max_vram_gb: Some(24), + relay_url: Some("wss://community.example".to_string()), + }; + + let checkpoint = pending_new_start_checkpoint(&config); + assert!(!checkpoint.enabled); + assert!(!checkpoint.start_on_next_launch); + assert_eq!(checkpoint.model_id, config.model_id); + assert_eq!(checkpoint.max_vram_gb, config.max_vram_gb); + assert_eq!(checkpoint.relay_url, config.relay_url); +} + +#[test] +fn role_switch_checkpoint_starts_exactly_once_after_restart() { + let config = MeshSharingConfig { + enabled: true, + start_on_next_launch: false, + model_id: "test-model".to_string(), + max_vram_gb: Some(24), + relay_url: Some("wss://community.example".to_string()), + }; + + let restart = one_shot_restart_checkpoint(&config); + assert!(!restart.enabled); + assert!(restart.start_on_next_launch); + + let consumed = pending_new_start_checkpoint(&restart); + assert!(!consumed.enabled); + assert!(!consumed.start_on_next_launch); + assert_eq!(consumed.model_id, config.model_id); + assert_eq!(consumed.relay_url, config.relay_url); +} + #[test] fn readiness_failure_is_catalog_sync_when_model_never_visible() { assert_eq!( @@ -345,6 +413,7 @@ fn ensure_serve_runtime_serves_other_model() { max_vram_gb: None, join_token: None, mesh_name: None, + relay_url: None, trusted_owner_ids: None, }) .await diff --git a/desktop/src-tauri/src/commands/mod.rs b/desktop/src-tauri/src/commands/mod.rs index 1c89ee4f77..66ef7ef17b 100644 --- a/desktop/src-tauri/src/commands/mod.rs +++ b/desktop/src-tauri/src/commands/mod.rs @@ -44,6 +44,7 @@ mod project_git; mod project_git_branches; mod project_git_diff; mod project_git_exec; +mod project_git_merge_error; mod project_git_push; mod project_git_workflow; mod project_repo_paths; diff --git a/desktop/src-tauri/src/commands/personas/card.rs b/desktop/src-tauri/src/commands/personas/card.rs new file mode 100644 index 0000000000..c516db1736 --- /dev/null +++ b/desktop/src-tauri/src/commands/personas/card.rs @@ -0,0 +1,972 @@ +//! `mint_agent_card` / `save_agent_card` Tauri commands — Agent Trading Cards. +//! +//! Mints a collectible trading-card PNG for an agent via one OpenAI Responses +//! API call (designer model + native `image_generation` tool), then embeds the +//! agent's `buzz_agent_snapshot` manifest through the existing snapshot +//! encoder so the card IS an importable `.agent.png`. +//! +//! Boundary rules (agreed with Wren, buzz-agent-trading-cards thread): +//! - Snapshot construction/injection reuses `agent_snapshot.rs` — cards +//! inherit manifest-v1 behavior, exclusions, and size checks. No card-only +//! wire format exists. +//! - Memory inclusion is opt-in and shares the export flow's semantics: the +//! same three levels (`none`/`core`/`everything`), the same owner-gated +//! `get_agent_memory` fetch, and a memory source DERIVED from the resolved +//! instance (never caller-supplied), so cross-agent memory pairing is +//! structurally impossible. The default is `none`; the encoder still +//! rejects `none` + entries. +//! - The 10 MiB `.agent.png` ceiling is enforced on the FINAL bytes (after +//! resize + chunk injection) via `validate_snapshot_encode_size`. +//! - Round-trip verification decodes the final bytes and compares the logical +//! manifest before anything is returned to the frontend. +//! - The API key is resolved through the same env layering the agent runtime +//! uses (global config < persona < agent record) and never leaves Rust. +//! It is never logged. + +use base64::{engine::general_purpose::STANDARD, Engine as _}; +use serde::{Deserialize, Serialize}; +use tauri::{AppHandle, State}; + +use super::super::export_util::save_bytes_with_dialog; +use super::snapshot::{ + memory_entries_from_listing, parse_memory_level, resolve_from_lists, + validate_snapshot_encode_size, +}; +use crate::{ + app_state::AppState, + commands::engrams::get_agent_memory, + managed_agents::{ + agent_snapshot::{ + build_snapshot, decode_avatar_data_url, decode_snapshot_png, encode_snapshot_png, + extract_chunk_payload_png, MemoryLevel, + }, + agent_snapshot_envelope::{ + decrypt_envelope, encode_locked_snapshot_png, parse_chunk_payload, ChunkPayload, + }, + load_agent_definitions, load_global_agent_config, load_managed_agents, load_personas, + save_global_agent_config, validate_global_config, + }, +}; + +/// The Buzz card frame template — Tyler's gold-honeycomb base. Generation +/// input only: it never participates in the snapshot manifest, PNG chunk, +/// import decoder, or attachment validation. Embedded at compile time for +/// deterministic packaging (see `card_template_decodes` test). +const CARD_TEMPLATE_PNG: &[u8] = include_bytes!("../../../assets/card_template.png"); + +/// Designer model driving copy + art direction. +const DESIGNER_MODEL: &str = "gpt-5.6-sol"; +/// Image model invoked natively via the Responses `image_generation` tool. +const IMAGE_MODEL: &str = "gpt-image-2"; +/// Final card width in pixels (2:3 portrait → 1500x2250). +const CARD_WIDTH: u32 = 1500; +/// Longest edge for the real avatar inlined into an unlocked card's manifest. +/// Kind:0 pictures render small; 512px keeps the doubly-base64-encoded +/// manifest chunk modest next to the 1500-wide card body. +const MANIFEST_AVATAR_MAX_DIM: u32 = 512; +/// Upper bound for a fetched avatar (pre-resize input to the model). +const MAX_AVATAR_FETCH_BYTES: usize = 10 * 1024 * 1024; +/// One mint is a single long API call (~2–3 minutes observed). +const MINT_TIMEOUT_SECS: u64 = 600; + +/// Error prefix the frontend matches to route the user to provider settings +/// instead of showing a raw failure. +pub(crate) const NO_KEY_ERROR_PREFIX: &str = "NO_OPENAI_KEY:"; + +/// Wire shape returned by `mint_agent_card`. +#[derive(Debug, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct MintedCard { + /// Final `.agent.png` bytes (chunk-injected, round-trip verified), + /// base64-encoded for the IPC boundary. + pub card_png_base64: String, + /// Suggested filename, e.g. `eva.agent.png`. + pub file_name: String, + /// Designer commentary emitted alongside the image (may be empty). + pub designer_notes: String, + /// True when the embedded snapshot is NIP-44-encrypted to the + /// (owner, agent) pair — only their nsecs can import this card. + pub locked: bool, + /// How much memory is embedded in the card's snapshot ("none"/"core"/ + /// "everything"). The viewer's import disclosure depends on this. + pub memory_level: MemoryLevel, +} + +// ── Card archive ────────────────────────────────────────────────────────────── + +/// Sidecar metadata for one archived card PNG. Stored as `.json` next +/// to `.agent.png` in the cards dir — two plain files per mint, no +/// shared index to corrupt. Listing scans sidecars; a card whose PNG is +/// missing is skipped rather than failing the whole list. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct ArchivedCardMeta { + /// Unique on-disk PNG file name within the cards dir. + pub stored_file_name: String, + /// Suggested save-as name, e.g. `eva.agent.png`. + pub file_name: String, + /// The id the card was minted for (instance pubkey or definition slug). + pub agent_id: String, + pub agent_name: String, + pub designer_notes: String, + pub locked: bool, + /// Memory embedded in this card's snapshot. Defaults to `None` when the + /// sidecar predates the field — every pre-field mint was minted with + /// `MemoryLevel::None` (it was structural), so the default is honest. + #[serde(default)] + pub memory_level: MemoryLevel, + /// ISO-8601 mint timestamp. + pub minted_at: String, + /// Small JPEG preview for gallery grids, base64. Populated by + /// `list_agent_cards` from the sidecar thumb file — never stored in the + /// JSON sidecar itself. + #[serde(default, skip_deserializing)] + pub thumb_jpeg_base64: Option, +} + +fn cards_dir(app: &AppHandle) -> Result { + let dir = crate::managed_agents::managed_agents_base_dir(app)?.join("cards"); + std::fs::create_dir_all(&dir).map_err(|e| format!("failed to create cards dir: {e}"))?; + Ok(dir) +} + +/// Persist a freshly minted card to the archive. Failures are surfaced to the +/// caller (which logs and continues) — an archive write must never fail a +/// mint the user already paid for. +fn archive_minted_card( + app: &AppHandle, + agent_id: &str, + agent_name: &str, + card: &MintedCard, + bytes: &[u8], +) -> Result { + let dir = cards_dir(app)?; + let stem = format!( + "{}-{}", + crate::util::slugify(agent_name, "agent", 50), + uuid::Uuid::new_v4() + ); + let stored_file_name = format!("{stem}.agent.png"); + let meta = ArchivedCardMeta { + stored_file_name: stored_file_name.clone(), + file_name: card.file_name.clone(), + agent_id: agent_id.to_string(), + agent_name: agent_name.to_string(), + designer_notes: card.designer_notes.clone(), + locked: card.locked, + memory_level: card.memory_level, + minted_at: crate::util::now_iso(), + thumb_jpeg_base64: None, + }; + // PNG first, sidecar second: a crash between the two leaves an orphaned + // PNG (invisible to the list), never a sidecar pointing at nothing. + std::fs::write(dir.join(&stored_file_name), bytes) + .map_err(|e| format!("failed to write archived card: {e}"))?; + let meta_json = serde_json::to_string_pretty(&meta) + .map_err(|e| format!("failed to serialize card metadata: {e}"))?; + std::fs::write(dir.join(format!("{stem}.json")), meta_json) + .map_err(|e| format!("failed to write card metadata: {e}"))?; + // Thumb last and best-effort: the gallery grid falls back to lazy + // full-card loading for a card whose thumb is missing. + if let Ok(thumb) = encode_card_thumb(bytes) { + let _ = std::fs::write(dir.join(format!("{stem}.thumb.jpg")), thumb); + } + Ok(meta) +} + +/// Downscale card PNG bytes to a small JPEG for gallery grids. The full card +/// is ~1500x2250 PNG (megabytes); shipping that per card over IPC just to +/// draw a grid tile is waste. +fn encode_card_thumb(bytes: &[u8]) -> Result, String> { + const THUMB_WIDTH: u32 = 300; + let img = image::load_from_memory(bytes).map_err(|e| format!("thumb decode: {e}"))?; + let scale = THUMB_WIDTH as f64 / img.width() as f64; + let thumb = img.resize( + THUMB_WIDTH, + (img.height() as f64 * scale).round().max(1.0) as u32, + image::imageops::FilterType::Triangle, + ); + let mut out = Vec::new(); + // JPEG has no alpha; cards are opaque, so flatten unconditionally. + let rgb = image::DynamicImage::ImageRgb8(thumb.to_rgb8()); + rgb.write_to( + &mut std::io::Cursor::new(&mut out), + image::ImageFormat::Jpeg, + ) + .map_err(|e| format!("thumb encode: {e}"))?; + Ok(out) +} + +/// Reject any archive file name that could escape the cards dir or name a +/// non-archive file. Archive names are generated by `archive_minted_card` +/// (slug + UUID), so a strict shape check loses nothing legitimate. +fn validate_archive_file_name(stored_file_name: &str) -> Result<(), String> { + let valid = stored_file_name.ends_with(".agent.png") + && !stored_file_name.contains(['/', '\\']) + && !stored_file_name.contains(".."); + if !valid { + return Err("Invalid archived card file name.".to_string()); + } + Ok(()) +} + +/// List all archived cards, newest first. +#[tauri::command] +pub fn list_agent_cards(app: AppHandle) -> Result, String> { + let dir = cards_dir(&app)?; + let entries = std::fs::read_dir(&dir).map_err(|e| format!("failed to read cards dir: {e}"))?; + let mut cards = Vec::new(); + for entry in entries.flatten() { + let path = entry.path(); + if path.extension().and_then(|e| e.to_str()) != Some("json") { + continue; + } + let Ok(content) = std::fs::read_to_string(&path) else { + continue; + }; + let Ok(meta) = serde_json::from_str::(&content) else { + // A malformed sidecar hides one card, never the archive. + eprintln!( + "buzz-desktop: card-archive: skipping malformed sidecar {}", + path.display() + ); + continue; + }; + let mut meta = meta; + if validate_archive_file_name(&meta.stored_file_name).is_ok() + && dir.join(&meta.stored_file_name).is_file() + { + // Attach the pre-rendered grid thumb when present (best-effort). + if let Some(stem) = path.file_stem().and_then(|s| s.to_str()) { + meta.thumb_jpeg_base64 = std::fs::read(dir.join(format!("{stem}.thumb.jpg"))) + .ok() + .map(|b| STANDARD.encode(&b)); + } + cards.push(meta); + } + } + // ISO-8601 sorts lexicographically; newest first. + cards.sort_by(|a, b| b.minted_at.cmp(&a.minted_at)); + Ok(cards) +} + +/// Load one archived card's PNG bytes as base64, keyed by its stored file +/// name (as returned by `list_agent_cards`). +#[tauri::command] +pub fn load_agent_card(stored_file_name: String, app: AppHandle) -> Result { + validate_archive_file_name(&stored_file_name)?; + let bytes = std::fs::read(cards_dir(&app)?.join(&stored_file_name)) + .map_err(|e| format!("failed to read archived card: {e}"))?; + Ok(STANDARD.encode(&bytes)) +} + +// ── Key resolution ──────────────────────────────────────────────────────────── + +/// Pure layering: global env < persona env < agent record env, then the +/// process environment as a development fallback. Returns the first +/// non-empty value for `key`. +pub(crate) fn resolve_env_from_layers( + key: &str, + global_env: &std::collections::BTreeMap, + persona_env: &std::collections::BTreeMap, + record_env: &std::collections::BTreeMap, + process_value: Option, +) -> Option { + for layer in [record_env, persona_env, global_env] { + if let Some(v) = layer.get(key) { + let v = v.trim(); + if !v.is_empty() { + return Some(v.to_string()); + } + } + } + process_value.filter(|k| !k.trim().is_empty()) +} + +/// The Responses endpoint to post mints to. `OPENAI_BASE_URL` (same env +/// layering as the key) overrides the default host, supporting endpoints and +/// proxies that speak the OpenAI Responses shape with Bearer auth. Azure +/// OpenAI is NOT covered by this override alone — it uses its own URL scheme +/// and `api-key` auth header, which would need a real driver. +pub(crate) fn responses_url(base_url: Option) -> String { + let base = base_url.unwrap_or_else(|| "https://api.openai.com/v1".to_string()); + format!("{}/responses", base.trim_end_matches('/')) +} + +// ── Prompt construction ─────────────────────────────────────────────────────── + +/// Build the designer instructions. Pure so tests can pin the contract: +/// style-match-the-avatar is DEFAULT behavior; owner directions (art AND +/// card text) take primacy over those style defaults, but never over the +/// fixed contract (frame identity, geometry, text fidelity). +pub(crate) fn build_card_instructions( + agent_name: &str, + persona_notes: &str, + style_notes: &str, +) -> String { + let owner_directions = if style_notes.trim().is_empty() { + String::new() + } else { + format!( + "\nOWNER'S DIRECTIONS — these override the default art-style and copy guidance \ + below wherever they conflict (they cannot change the frame, layout, or \ + text-fidelity requirements). The owner may direct the art, the card text \ + (type line, ability, flavor), or both:\n{style_notes}\n" + ) + }; + format!( + r#"You are designing one premium collectible trading card for the Buzz agent "{agent_name}". + +Input image 1 is the official Buzz card frame template (gold honeycomb border, dark interior, name banner top, hex badge top-right, text box lower third). Input image 2 is the agent's avatar — study its exact art style: medium, pixel grid if any, palette, shading, background motifs. + +Persona notes for the card copy: +{persona_notes} +{owner_directions} +First, write professional trading-card copy at Magic: The Gathering editorial quality: +- a type line (e.g. "Legendary Agent — Team Lead"), +- ONE keyworded ability: short bolded ability name + one sentence of crisp rules text written like real MTG rules (present tense, precise, no fluff), +- ONE italic flavor-text line, evocative and short, the kind that gets quoted. +Where the owner's directions specify card text, use their wording within the 220-character text-box limit below (edited only for spelling; if their text exceeds the limit, condense it minimally while keeping their words and intent); invent copy only for the parts they left open. +Keep total text-box copy under 220 characters so it renders cleanly. + +Then generate the finished card with the image tool, exactly 1024x1536 portrait: +- The frame must follow input image 1 faithfully: same gold honeycomb border, same layout, honey drip detail. +- Default art style: match input image 2's art style EXACTLY — same medium, same pixel density if pixel art, same palette, same background honeycomb-lattice sky. It must look like the same artist drew a larger scene: the character in a confident pose, conjuring glowing golden hexagons. The owner's directions above override any of this default styling where they conflict. +- Name banner: "{agent_name}" plus the type line beneath it in smaller type. +- Text box: the ability name in bold, rules text in regular, then the flavor line in italics, cleanly typeset like a real MTG card — professional kerning, no misspellings, hyphenate nothing. +- Top-right hex badge: one small emblem of your choice, no text. +Render all text with perfect fidelity."# + ) +} + +/// Encode raw image bytes as a `data:image/png;base64,` URL, downscaling to +/// `max_dim` on the longest edge so request payloads stay small. +fn image_data_url(bytes: &[u8], max_dim: u32) -> Result { + Ok(format!( + "data:image/png;base64,{}", + STANDARD.encode(png_bytes_resized(bytes, max_dim)?) + )) +} + +/// Re-encode an image as PNG, downscaling so neither side exceeds `max_dim`. +fn png_bytes_resized(bytes: &[u8], max_dim: u32) -> Result, String> { + let img = image::load_from_memory(bytes).map_err(|e| format!("Failed to decode image: {e}"))?; + let img = if img.width().max(img.height()) > max_dim { + img.resize(max_dim, max_dim, image::imageops::FilterType::Lanczos3) + } else { + img + }; + let mut png = Vec::new(); + img.write_to(&mut std::io::Cursor::new(&mut png), image::ImageFormat::Png) + .map_err(|e| format!("Failed to encode image: {e}"))?; + Ok(png) +} + +// ── Response parsing ────────────────────────────────────────────────────────── + +/// Extract the generated image (base64) and any designer text from a +/// Responses API payload. Pure for testability. +pub(crate) fn extract_card_output(resp: &serde_json::Value) -> Result<(String, String), String> { + let output = resp + .get("output") + .and_then(|o| o.as_array()) + .ok_or_else(|| "Responses payload has no output array".to_string())?; + + let mut image_b64 = None; + let mut notes = Vec::new(); + for item in output { + match item.get("type").and_then(|t| t.as_str()) { + Some("image_generation_call") => { + if let Some(result) = item.get("result").and_then(|r| r.as_str()) { + image_b64 = Some(result.to_string()); + } + } + Some("message") => { + if let Some(content) = item.get("content").and_then(|c| c.as_array()) { + for c in content { + if c.get("type").and_then(|t| t.as_str()) == Some("output_text") { + if let Some(text) = c.get("text").and_then(|t| t.as_str()) { + notes.push(text.to_string()); + } + } + } + } + } + _ => {} + } + } + + let image_b64 = image_b64.ok_or_else(|| { + let types: Vec<&str> = output + .iter() + .filter_map(|i| i.get("type").and_then(|t| t.as_str())) + .collect(); + format!("No image in Responses output (item types: {types:?})") + })?; + Ok((image_b64, notes.join("\n"))) +} + +// ── Commands ────────────────────────────────────────────────────────────────── + +/// Save an `OPENAI_API_KEY` into the global Agent Defaults env for card +/// minting — a narrow seam with deliberately different semantics from the +/// general `set_global_agent_config`: +/// +/// - **No agent restarts.** The general command stops/restarts every running +/// local agent whose effective env changes, because agent env is baked at +/// spawn time. The mint command re-reads the config from disk on every +/// mint, so minting needs no restart — and a card setup must never disrupt +/// running agents as a side effect. Agents pick the key up naturally on +/// their next (re)start. +/// - **Read-modify-write of the latest on-disk config.** The config is +/// re-read immediately before the single-key insert + write (under the +/// managed-agents store lock, which serializes it against the other card +/// and agent-store commands), so a settings save that landed after this +/// dialog opened is not clobbered with a stale dialog-open snapshot. +/// (The general settings editor performs its own whole-config write; as +/// today, the last writer wins between the two surfaces.) +/// +/// Standard global-config validation still applies (POSIX key shape, +/// reserved-key reject, size caps) — this is not a validation bypass. +#[tauri::command] +pub fn card_mint_save_openai_key( + key: String, + app: AppHandle, + state: State<'_, AppState>, +) -> Result<(), String> { + let key = key.trim().to_string(); + if key.is_empty() { + return Err("API key cannot be empty.".to_string()); + } + + let _store_guard = state + .managed_agents_store_lock + .lock() + .map_err(|e| e.to_string())?; + + let mut config = load_global_agent_config(&app)?; + config.env_vars.insert("OPENAI_API_KEY".to_string(), key); + validate_global_config(&config)?; + save_global_agent_config(&app, &config) +} + +/// Report whether an OpenAI key would resolve for a card mint of agent `id`, +/// using exactly the same env layering as `mint_agent_card`. Lets the mint +/// dialog offer inline key setup BEFORE the user commits to a mint, instead +/// of failing after the fact. Never returns the key itself. +#[tauri::command] +pub fn card_mint_key_status( + id: String, + app: AppHandle, + state: State<'_, AppState>, +) -> Result { + let _store_guard = state + .managed_agents_store_lock + .lock() + .map_err(|e| e.to_string())?; + + let instances = load_managed_agents(&app)?; + let definitions = load_agent_definitions(&app)?; + let (record, _) = resolve_from_lists(&id, &instances, &definitions)?; + + let global = load_global_agent_config(&app).unwrap_or_default(); + let personas = load_personas(&app).unwrap_or_default(); + let persona_env = record + .persona_id + .as_deref() + .and_then(|pid| personas.iter().find(|p| p.id == pid)) + .map(|p| p.env_vars.clone()) + .unwrap_or_default(); + + Ok(resolve_env_from_layers( + "OPENAI_API_KEY", + &global.env_vars, + &persona_env, + &record.env_vars, + std::env::var("OPENAI_API_KEY").ok(), + ) + .is_some()) +} + +/// Mint a trading card for the agent identified by `id` (instance pubkey, +/// instance slug, or definition slug — same resolution as snapshot export). +/// +/// When `lock` is true the embedded manifest is NIP-44-encrypted to the +/// (owner, agent) pair per the locked-envelope contract — this requires a +/// linked agent instance (the second key endpoint); bare definitions cannot +/// be locked. +/// +/// When `memory_level` is `"core"` or `"everything"`, the owner's decrypted +/// memory for the agent is embedded in the manifest — same levels and fetch +/// as snapshot export. The memory source is always the resolved instance +/// itself (derived, never caller-supplied), so it requires a linked instance; +/// bare definitions can only mint `"none"` (the default). +/// +/// Returns the final, chunk-injected, round-trip-verified `.agent.png` bytes. +/// Reroll = call again; the command holds no session state. +#[tauri::command] +pub async fn mint_agent_card( + id: String, + style_notes: Option, + lock: Option, + memory_level: Option, + app: AppHandle, + state: State<'_, AppState>, +) -> Result { + let lock = lock.unwrap_or(false); + let memory_level = parse_memory_level(memory_level.as_deref().unwrap_or(""))?; + // ── Resolve the record + API key under lock ────────────────────────────── + let (mut record, is_definition, api_key, base_url) = { + let _store_guard = state + .managed_agents_store_lock + .lock() + .map_err(|e| e.to_string())?; + + let instances = load_managed_agents(&app)?; + let definitions = load_agent_definitions(&app)?; + let (record, is_definition) = + resolve_from_lists(&id, &instances, &definitions).map(|(r, d)| (r.clone(), d))?; + + let global = load_global_agent_config(&app).unwrap_or_default(); + let personas = load_personas(&app).unwrap_or_default(); + let persona_env = record + .persona_id + .as_deref() + .and_then(|pid| personas.iter().find(|p| p.id == pid)) + .map(|p| p.env_vars.clone()) + .unwrap_or_default(); + + let api_key = resolve_env_from_layers( + "OPENAI_API_KEY", + &global.env_vars, + &persona_env, + &record.env_vars, + std::env::var("OPENAI_API_KEY").ok(), + ) + .ok_or_else(|| { + format!( + "{NO_KEY_ERROR_PREFIX} No OPENAI_API_KEY found. Add one in the agent's \ + environment variables or global agent settings to mint cards." + ) + })?; + let base_url = resolve_env_from_layers( + "OPENAI_BASE_URL", + &global.env_vars, + &persona_env, + &record.env_vars, + std::env::var("OPENAI_BASE_URL").ok(), + ); + + (record, is_definition, api_key, base_url) + }; + + // ── Locking needs its two exact key endpoints up front, BEFORE the + // API spend: the owner identity secret and the agent instance pubkey. + let lock_keys = if lock { + if is_definition { + return Err( + "Locked cards need a linked agent instance — this persona has never been \ + started, so there is no agent key to lock to." + .to_string(), + ); + } + let owner_keys = state.signing_keys()?; + // Same canonical check the envelope decoder enforces (incl. curve + // validation) — a non-point record pubkey must fail BEFORE the API + // spend, not at post-mint encryption. + let agent_pubkey = crate::managed_agents::agent_snapshot_envelope::parse_canonical_pubkey( + "agentPubkey", + &record.pubkey, + ) + .map_err(|_| { + "Agent record has an invalid pubkey (not a canonical x-only key).".to_string() + })?; + if owner_keys.public_key() == agent_pubkey { + return Err("Cannot lock a card to itself: owner and agent keys match.".to_string()); + } + Some((owner_keys, agent_pubkey)) + } else { + None + }; + + // ── Memory needs a keyed instance, resolved up front BEFORE the API + // spend — the memory source is always the resolved instance itself + // (derived, never caller-supplied), so cross-agent pairing cannot be + // expressed. A failed fetch fails the mint here, not after payment. + let memory_entries = if memory_level == MemoryLevel::None { + Vec::new() + } else { + if is_definition { + return Err( + "Cards with memory need a linked agent instance — this persona has never \ + been started, so there is no agent memory to include." + .to_string(), + ); + } + let listing = get_agent_memory(record.pubkey.clone(), app.clone(), state.clone()).await?; + memory_entries_from_listing(listing, memory_level) + }; + + let display_name = record + .display_name + .clone() + .unwrap_or_else(|| record.name.clone()); + + // ── Prefer the agent's own kind:0 profile picture ──────────────────────── + // The record's `avatar_url` is a stale presentation snapshot: with + // agent-managed profiles the agent updates its own kind:0 `picture` and + // desktop reconciliation is disabled (`agent_settings.rs`), so the relay + // profile — not the local record — is the live source of truth for how the + // agent looks. Definitions have no keypair and thus no kind:0; they keep + // the record's avatar. A relay error fails the mint here, BEFORE the API + // spend (same fail-early rule as the key/memory guards above) — minting + // with the wrong face wastes the spend it was supposed to protect. + if !is_definition { + let relay_url = crate::relay::effective_agent_relay_url( + &record.relay_url, + &crate::relay::relay_ws_url_with_override(&state), + ); + let profile = crate::relay::query_agent_profile(&state, &relay_url, &record.pubkey) + .await + .map_err(|e| format!("Could not read the agent's profile for its avatar: {e}"))?; + record.avatar_url = preferred_avatar_url( + profile.and_then(|info| info.picture), + record.avatar_url.take(), + ); + } + + // ── Resolve avatar bytes (data URL, else fetch) ────────────────────────── + let avatar_bytes = match record.avatar_url.as_deref() { + Some(url) if url.starts_with("data:") => decode_avatar_data_url(url) + .ok_or_else(|| "Agent avatar data URL could not be decoded.".to_string())?, + Some(url) if url.starts_with("http://") || url.starts_with("https://") => { + // Relay-hosted avatars (kind:0 pictures under the relay's /media/) + // may require Blossom get-auth (`require_media_get_auth`). Mint the + // header ONLY for same-origin URLs so the token never leaves the + // relay (same contract as `media_download.rs`). + let relay_base = crate::relay::relay_api_base_url_with_override(&state); + let auth = is_same_origin(url, &relay_base) + .then(|| crate::commands::media::mint_media_get_auth(&state, &relay_base)) + .flatten(); + fetch_avatar(url, auth.as_deref()).await? + } + _ => { + return Err( + "Agent has no avatar image. Set an avatar before minting a card.".to_string(), + ) + } + }; + + // ── Build the manifest now (with any requested memory) so a broken agent + // fails before we spend minutes on the API call. ─────────────────────── + let manifest_avatar = manifest_avatar_bytes( + lock_keys.is_some(), + &avatar_bytes, + record.avatar_url.as_deref(), + )?; + let snapshot = build_snapshot( + &record, + memory_level, + memory_entries, + manifest_avatar.as_deref(), + ); + + // ── One Responses API call ─────────────────────────────────────────────── + // For locked mints, prove the manifest (including any embedded memory) + // fits the NIP-44 plaintext cap BEFORE spending minutes on the API call + // (same fail-early rule as the memory guard above). + if lock_keys.is_some() { + let json_len = + crate::managed_agents::agent_snapshot::encode_snapshot_json(&snapshot)?.len(); + if json_len > buzz_core_pkg::engram::NIP44_PLAINTEXT_MAX { + let hint = if memory_level == MemoryLevel::None { + "Reduce the avatar size or mint an unlocked card." + } else { + "Include less memory, reduce the avatar size, or mint an unlocked card." + }; + return Err(format!( + "Agent manifest is too large to lock ({json_len} bytes; the encrypted \ + format caps at {}). {hint}", + buzz_core_pkg::engram::NIP44_PLAINTEXT_MAX + )); + } + } + let instructions = build_card_instructions( + &display_name, + snapshot.definition.system_prompt.as_deref().unwrap_or(""), + style_notes.as_deref().unwrap_or(""), + ); + let body = serde_json::json!({ + "model": DESIGNER_MODEL, + "reasoning": {"effort": "high"}, + "instructions": "You are a senior TCG card designer and MTG rules editor.", + "input": [{ + "role": "user", + "content": [ + {"type": "input_text", "text": instructions}, + {"type": "input_image", "image_url": image_data_url(CARD_TEMPLATE_PNG, 1024)?}, + {"type": "input_image", "image_url": image_data_url(&avatar_bytes, 1024)?}, + ], + }], + "tools": [{ + "type": "image_generation", + "model": IMAGE_MODEL, + "quality": "high", + "size": "1024x1536", + "output_format": "png", + }], + "tool_choice": "required", + }); + + let client = reqwest::Client::builder() + .timeout(std::time::Duration::from_secs(MINT_TIMEOUT_SECS)) + .build() + .map_err(|e| format!("Failed to build HTTP client: {e}"))?; + let resp = client + .post(responses_url(base_url)) + .bearer_auth(&api_key) + .json(&body) + .send() + .await + .map_err(|e| format!("Card mint request failed: {e}"))?; + + let status = resp.status(); + let payload: serde_json::Value = resp + .json() + .await + .map_err(|e| format!("Card mint response was not JSON: {e}"))?; + if !status.is_success() { + // Never echo the request (it embeds nothing secret, but keep the + // failure surface small); the OpenAI error body is safe to surface. + let detail = payload + .get("error") + .and_then(|e| e.get("message")) + .and_then(|m| m.as_str()) + .unwrap_or("unknown error"); + return Err(format!("Card mint failed (HTTP {status}): {detail}")); + } + + let (image_b64, designer_notes) = extract_card_output(&payload)?; + let raw_card = STANDARD + .decode(image_b64.as_bytes()) + .map_err(|e| format!("Generated image was not valid base64: {e}"))?; + + // ── Resize to 1500-wide, inject chunk via the existing encoder ────────── + let card_img = image::load_from_memory(&raw_card) + .map_err(|e| format!("Generated image could not be decoded: {e}"))?; + let scale = CARD_WIDTH as f64 / card_img.width() as f64; + let card_img = card_img.resize( + CARD_WIDTH, + (card_img.height() as f64 * scale).round() as u32, + image::imageops::FilterType::Lanczos3, + ); + let mut card_png = Vec::new(); + card_img + .write_to( + &mut std::io::Cursor::new(&mut card_png), + image::ImageFormat::Png, + ) + .map_err(|e| format!("Failed to encode card PNG: {e}"))?; + + let final_bytes = match &lock_keys { + None => encode_snapshot_png(&snapshot, Some(&card_png)) + .map_err(|e| format!("Failed to embed agent snapshot in card: {e}"))?, + Some((owner_keys, agent_pubkey)) => { + encode_locked_snapshot_png(&snapshot, owner_keys, agent_pubkey, Some(&card_png)) + .map_err(|e| format!("Failed to embed locked agent snapshot in card: {e}"))? + } + }; + + // ── Verify: size ceiling + round-trip on the FINAL bytes ──────────────── + // Locked cards: extract the actual chunk, parse the envelope, decrypt + // with the owner key, then compare the logical manifest (ciphertext is + // nondeterministic — never compare bytes). + validate_snapshot_encode_size(final_bytes.len(), true)?; + let decoded = match &lock_keys { + None => decode_snapshot_png(&final_bytes) + .map_err(|e| format!("Card failed round-trip verification: {e}"))?, + Some((owner_keys, _)) => { + let payload = extract_chunk_payload_png(&final_bytes) + .map_err(|e| format!("Card failed round-trip verification: {e}"))?; + match parse_chunk_payload(&payload) + .map_err(|e| format!("Card failed round-trip verification: {e}"))? + { + ChunkPayload::Locked(envelope) => { + decrypt_envelope(&envelope, owner_keys.secret_key()) + .map_err(|e| format!("Card failed round-trip verification: {e}"))? + } + ChunkPayload::Plain(_) => { + return Err( + "Card round-trip verification failed: expected a locked envelope." + .to_string(), + ) + } + } + } + }; + if decoded != snapshot { + return Err("Card round-trip verification failed: manifest mismatch.".to_string()); + } + + let slug = crate::util::slugify(&display_name, "agent", 50); + let minted = MintedCard { + card_png_base64: STANDARD.encode(&final_bytes), + file_name: format!("{slug}.agent.png"), + designer_notes, + locked: lock_keys.is_some(), + memory_level, + }; + + // Archive best-effort: the mint is already paid for and verified, so a + // failed archive write logs and continues — it never fails the mint. + if let Err(e) = archive_minted_card(&app, &id, &display_name, &minted, &final_bytes) { + eprintln!("buzz-desktop: card-archive: failed to archive minted card: {e}"); + } + + Ok(minted) +} + +/// The avatar the mint should use: the agent's kind:0 `picture` when one is +/// published and non-blank, else the local record's `avatar_url`. +/// +/// Pure so the precedence is unit-testable without a relay: a blank or +/// whitespace-only `picture` must NOT shadow a real record avatar. +fn preferred_avatar_url( + kind0_picture: Option, + record_avatar_url: Option, +) -> Option { + kind0_picture + .filter(|p| !p.trim().is_empty()) + .or(record_avatar_url) +} + +/// The avatar bytes the card manifest should inline. +/// +/// Unlocked cards must carry the agent's REAL avatar inline: the PNG body is +/// the generated card artwork, and the importer only adopts the body as the +/// avatar when the manifest carries no inline bytes (`import.rs`) — without +/// these bytes an imported agent would wear the card as its face. Downscaled +/// to [`MANIFEST_AVATAR_MAX_DIM`] so the manifest tEXt chunk stays small. +/// +/// Locked cards keep the data-URL-only behavior: the whole manifest must fit +/// the NIP-44 plaintext cap (65 KB), which cannot carry inline pixels, and a +/// locked envelope never reaches the import body override anyway. +fn manifest_avatar_bytes( + locked: bool, + avatar_bytes: &[u8], + record_avatar_url: Option<&str>, +) -> Result>, String> { + if locked { + return Ok(decode_avatar_data_url(record_avatar_url.unwrap_or(""))); + } + png_bytes_resized(avatar_bytes, MANIFEST_AVATAR_MAX_DIM) + .map(Some) + .map_err(|e| format!("Failed to inline the agent avatar into the card manifest: {e}")) +} + +/// True when `url` shares an origin (scheme, host, port) with `relay_base`. +/// +/// Gate for attaching the minted media get-auth header — the token must never +/// be sent to a non-relay origin (same contract as `validate_download_url` in +/// `media_download.rs`, but non-fatal: a foreign origin just fetches +/// unauthenticated instead of failing the mint). +fn is_same_origin(url: &str, relay_base: &str) -> bool { + match (url::Url::parse(url), url::Url::parse(relay_base)) { + (Ok(u), Ok(b)) => u.origin() == b.origin(), + _ => false, + } +} + +/// Fetch an avatar over HTTP with a hard size cap. +/// +/// `auth` is an optional pre-minted Blossom get-auth header value, attached +/// verbatim — the caller is responsible for only supplying it for +/// relay-origin URLs. Redirects are not followed when auth is present +/// (redirect-hop guard, same rule as `media_download.rs`). +/// +/// The cap bounds network and memory, not just the final buffer: the +/// Content-Length header is checked before any body bytes are read, and the +/// body is streamed with a running count so a missing or dishonest header +/// still cannot exceed the cap (same contract as `media_download.rs`). +async fn fetch_avatar(url: &str, auth: Option<&str>) -> Result, String> { + use futures_util::StreamExt; + + let mut builder = reqwest::Client::builder().timeout(std::time::Duration::from_secs(30)); + if auth.is_some() { + // Never let a relay 3xx forward the auth header across origins. + builder = builder.redirect(reqwest::redirect::Policy::none()); + } + let client = builder + .build() + .map_err(|e| format!("Failed to build HTTP client: {e}"))?; + let mut req = client.get(url); + if let Some(auth) = auth { + req = req.header("authorization", auth); + } + let resp = req + .send() + .await + .map_err(|e| format!("Failed to fetch agent avatar: {e}"))?; + if !resp.status().is_success() { + return Err(format!("Avatar fetch failed: HTTP {}", resp.status())); + } + + if let Some(content_length) = resp.content_length() { + if content_length > MAX_AVATAR_FETCH_BYTES as u64 { + return Err("Agent avatar is too large to use as card input.".to_string()); + } + } + + let mut bytes = Vec::new(); + let mut stream = resp.bytes_stream(); + while let Some(chunk) = stream.next().await { + let chunk = chunk.map_err(|e| format!("Failed to read avatar bytes: {e}"))?; + append_within_avatar_cap(&mut bytes, &chunk)?; + } + Ok(bytes) +} + +/// Append a body chunk to the avatar buffer, rejecting before the append if +/// the total would cross `MAX_AVATAR_FETCH_BYTES`. Split out so the cap +/// boundary is unit-testable without an HTTP server. +fn append_within_avatar_cap(buf: &mut Vec, chunk: &[u8]) -> Result<(), String> { + if buf.len() + chunk.len() > MAX_AVATAR_FETCH_BYTES { + return Err("Agent avatar is too large to use as card input.".to_string()); + } + buf.extend_from_slice(chunk); + Ok(()) +} + +/// Save previously minted card bytes to disk via the OS save dialog. +/// +/// Re-validates the bytes (chunk parses as a plain manifest or a +/// structurally valid locked envelope, size within the import ceiling) so a +/// corrupted preview can never be written as a `.agent.png`. No decryption +/// happens here — the mint already round-trip-verified with the real key. +#[tauri::command] +pub async fn save_agent_card( + card_png_base64: String, + file_name: String, + app: AppHandle, +) -> Result { + let bytes = STANDARD + .decode(card_png_base64.as_bytes()) + .map_err(|e| format!("Card bytes were not valid base64: {e}"))?; + validate_snapshot_encode_size(bytes.len(), true)?; + let payload = extract_chunk_payload_png(&bytes) + .map_err(|e| format!("Refusing to save: card failed snapshot validation: {e}"))?; + parse_chunk_payload(&payload) + .map_err(|e| format!("Refusing to save: card failed snapshot validation: {e}"))?; + + let safe_name = if file_name.ends_with(".agent.png") && !file_name.contains(['/', '\\']) { + file_name + } else { + "card.agent.png".to_string() + }; + save_bytes_with_dialog(&app, &safe_name, "Agent card", &["png"], &bytes).await +} + +// ── Tests ───────────────────────────────────────────────────────────────────── + +#[cfg(test)] +mod tests; diff --git a/desktop/src-tauri/src/commands/personas/card/tests.rs b/desktop/src-tauri/src/commands/personas/card/tests.rs new file mode 100644 index 0000000000..ca69c43866 --- /dev/null +++ b/desktop/src-tauri/src/commands/personas/card/tests.rs @@ -0,0 +1,324 @@ +//! Unit tests for `card.rs` — split into a child module file so the parent +//! stays under the 1000-line gate (same layout as `snapshot/tests.rs`). + +use super::*; +use std::collections::BTreeMap; + +#[test] +fn archive_file_name_validation_rejects_escapes() { + assert!(validate_archive_file_name("eva-1234.agent.png").is_ok()); + for bad in [ + "../escape.agent.png", + "sub/dir.agent.png", + "sub\\dir.agent.png", + "not-a-card.png", + "plain.json", + "", + ] { + assert!( + validate_archive_file_name(bad).is_err(), + "expected rejection: {bad:?}" + ); + } +} + +#[test] +fn card_template_decodes_with_expected_shape() { + // The embedded template is generation input only, but a corrupt or + // accidentally swapped asset should fail the build's test gate, not a + // user's first mint. + let img = image::load_from_memory(CARD_TEMPLATE_PNG).expect("template must decode"); + // 2:3-ish portrait frame. + assert!(img.height() > img.width(), "template must be portrait"); + assert!(img.width() >= 512, "template unexpectedly small"); +} + +#[test] +fn key_resolution_layering_record_wins() { + let mut global = BTreeMap::new(); + global.insert("OPENAI_API_KEY".to_string(), "global".to_string()); + let mut persona = BTreeMap::new(); + persona.insert("OPENAI_API_KEY".to_string(), "persona".to_string()); + let mut record = BTreeMap::new(); + record.insert("OPENAI_API_KEY".to_string(), "record".to_string()); + + assert_eq!( + resolve_env_from_layers("OPENAI_API_KEY", &global, &persona, &record, None).as_deref(), + Some("record") + ); + record.clear(); + assert_eq!( + resolve_env_from_layers("OPENAI_API_KEY", &global, &persona, &record, None).as_deref(), + Some("persona") + ); + persona.clear(); + assert_eq!( + resolve_env_from_layers("OPENAI_API_KEY", &global, &persona, &record, None).as_deref(), + Some("global") + ); + global.clear(); + assert_eq!( + resolve_env_from_layers( + "OPENAI_API_KEY", + &global, + &persona, + &record, + Some("process".to_string()) + ) + .as_deref(), + Some("process") + ); + assert!(resolve_env_from_layers("OPENAI_API_KEY", &global, &persona, &record, None).is_none()); +} + +#[test] +fn key_resolution_skips_blank_values() { + let mut record = BTreeMap::new(); + record.insert("OPENAI_API_KEY".to_string(), " ".to_string()); + let mut persona = BTreeMap::new(); + persona.insert("OPENAI_API_KEY".to_string(), "persona".to_string()); + assert_eq!( + resolve_env_from_layers("OPENAI_API_KEY", &BTreeMap::new(), &persona, &record, None) + .as_deref(), + Some("persona") + ); +} + +#[test] +fn responses_url_default_and_override() { + assert_eq!(responses_url(None), "https://api.openai.com/v1/responses"); + // Trailing slashes must not produce a double-slash path. + assert_eq!( + responses_url(Some("https://proxy.example/v1/".to_string())), + "https://proxy.example/v1/responses" + ); + assert_eq!( + responses_url(Some("https://proxy.example/v1".to_string())), + "https://proxy.example/v1/responses" + ); +} + +#[test] +fn instructions_pin_style_match_default_and_owner_primacy() { + let base = build_card_instructions("Eva", "leads the team", ""); + assert!(base.contains("match input image 2's art style EXACTLY")); + assert!(base.contains("\"Eva\"")); + assert!(!base.contains("OWNER'S DIRECTIONS")); + + let directed = build_card_instructions("Eva", "leads the team", "make it stormy"); + // Owner directions take primacy over style defaults... + assert!(directed.contains("OWNER'S DIRECTIONS")); + assert!(directed.contains("make it stormy")); + assert!(directed.contains("override the default art-style and copy guidance")); + // ...but the fixed contract survives: frame, style anchor (as an + // overridable default), and text-fidelity requirements stay present. + assert!(directed.contains("match input image 2's art style EXACTLY")); + assert!(directed.contains("cannot change the frame, layout, or")); + assert!(directed.contains("Render all text with perfect fidelity")); + // Card-text direction is an explicitly named capability, and the + // owner-wording rule acknowledges the fixed 220-char text-box limit + // (no mutually impossible "verbatim" vs "under 220 chars" pair). + assert!(directed.contains("card text")); + assert!(directed.contains("use their wording within the 220-character text-box limit")); +} + +#[test] +fn extract_card_output_happy_path_and_missing_image() { + let ok = serde_json::json!({ + "output": [ + {"type": "reasoning"}, + {"type": "image_generation_call", "result": "aW1n"}, + {"type": "message", "content": [ + {"type": "output_text", "text": "notes here"} + ]} + ] + }); + let (img, notes) = extract_card_output(&ok).unwrap(); + assert_eq!(img, "aW1n"); + assert_eq!(notes, "notes here"); + + let missing = serde_json::json!({"output": [{"type": "message", "content": []}]}); + let err = extract_card_output(&missing).unwrap_err(); + assert!(err.contains("No image"), "{err}"); + + let no_output = serde_json::json!({}); + assert!(extract_card_output(&no_output).is_err()); +} + +#[test] +fn kind0_picture_wins_over_record_avatar_unless_blank() { + let some = |s: &str| Some(s.to_string()); + // Published picture wins. + assert_eq!( + preferred_avatar_url(some("https://relay/media/k0.png"), some("data:image/png;x")), + some("https://relay/media/k0.png") + ); + // No profile / no picture: record avatar survives. + assert_eq!( + preferred_avatar_url(None, some("data:image/png;x")), + some("data:image/png;x") + ); + // Blank or whitespace picture must not shadow a real avatar. + assert_eq!( + preferred_avatar_url(some(""), some("data:image/png;x")), + some("data:image/png;x") + ); + assert_eq!( + preferred_avatar_url(some(" "), some("data:image/png;x")), + some("data:image/png;x") + ); + // Nothing anywhere: None (caller surfaces the "no avatar" error). + assert_eq!(preferred_avatar_url(None, None), None); +} + +#[test] +fn unlocked_manifest_inlines_real_avatar_bytes_downscaled() { + // 700px source (over MANIFEST_AVATAR_MAX_DIM) in a solid color. + let avatar = image::DynamicImage::ImageRgba8(image::RgbaImage::from_pixel( + 700, + 700, + image::Rgba([9, 120, 33, 255]), + )); + let mut avatar_png = std::io::Cursor::new(Vec::new()); + avatar + .write_to(&mut avatar_png, image::ImageFormat::Png) + .unwrap(); + + let inlined = manifest_avatar_bytes(false, avatar_png.get_ref(), None) + .unwrap() + .expect("unlocked mints must inline the real avatar"); + let img = image::load_from_memory(&inlined).unwrap(); + assert_eq!( + (img.width(), img.height()), + (MANIFEST_AVATAR_MAX_DIM, MANIFEST_AVATAR_MAX_DIM) + ); + assert_eq!(img.to_rgba8().get_pixel(0, 0).0, [9, 120, 33, 255]); + + // Undecodable avatar bytes fail the mint (pre-spend), never silently + // produce a card whose import would wear the artwork as a face. + assert!(manifest_avatar_bytes(false, b"not a png", None).is_err()); +} + +#[test] +fn locked_manifest_keeps_data_url_only_avatar() { + // Locked mints must not inline fetched bytes (NIP-44 cap): only a record + // data URL carries over, exactly as before. + let unused = [0u8; 4]; + assert_eq!( + manifest_avatar_bytes(true, &unused, Some("data:image/png;base64,aGk=")) + .unwrap() + .as_deref(), + Some(b"hi".as_slice()) + ); + assert_eq!( + manifest_avatar_bytes(true, &unused, Some("https://relay/media/a.png")).unwrap(), + None + ); + assert_eq!(manifest_avatar_bytes(true, &unused, None).unwrap(), None); +} + +#[test] +fn media_get_auth_gate_is_same_origin_only() { + // The minted Blossom get-auth header may only travel to the relay's own + // origin — scheme, host, and port all count (same contract as + // `validate_download_url` in `media_download.rs`). + let relay = "https://relay.example.com"; + assert!(is_same_origin( + "https://relay.example.com/media/abc.png", + relay + )); + // Different host, scheme, or port: no auth. + assert!(!is_same_origin( + "https://evil.example.com/media/abc.png", + relay + )); + assert!(!is_same_origin( + "http://relay.example.com/media/abc.png", + relay + )); + assert!(!is_same_origin( + "https://relay.example.com:8443/media/abc.png", + relay + )); + // Unparseable inputs fail closed. + assert!(!is_same_origin("not a url", relay)); + assert!(!is_same_origin( + "https://relay.example.com/x", + "also not a url" + )); + // Explicit port on both sides matches. + assert!(is_same_origin( + "http://localhost:3100/media/abc.png", + "http://localhost:3100" + )); +} + +#[test] +fn avatar_cap_rejects_before_appending_crossing_chunk() { + // The streaming accumulator must reject a chunk that would cross the + // cap BEFORE buffering it — this is what bounds memory when + // Content-Length is missing or dishonest. + let mut buf = vec![0u8; MAX_AVATAR_FETCH_BYTES - 1]; + assert!(append_within_avatar_cap(&mut buf, &[0u8]).is_ok()); + assert_eq!(buf.len(), MAX_AVATAR_FETCH_BYTES); + // Exactly at the cap: one more byte must fail and not grow the buffer. + assert!(append_within_avatar_cap(&mut buf, &[0u8]).is_err()); + assert_eq!(buf.len(), MAX_AVATAR_FETCH_BYTES); + + // A single oversized chunk is rejected outright. + let mut fresh = Vec::new(); + let oversized = vec![0u8; MAX_AVATAR_FETCH_BYTES + 1]; + assert!(append_within_avatar_cap(&mut fresh, &oversized).is_err()); + assert!(fresh.is_empty()); +} + +#[test] +fn save_rejects_plain_png_without_snapshot_chunk() { + // A plain PNG (no buzz_agent_snapshot chunk) must not be saveable as + // a card. Exercise the same validation the command runs. + let img = image::DynamicImage::new_rgba8(4, 4); + let mut png = Vec::new(); + img.write_to(&mut std::io::Cursor::new(&mut png), image::ImageFormat::Png) + .unwrap(); + assert!(decode_snapshot_png(&png).is_err()); +} + +#[test] +fn archived_sidecar_without_memory_level_defaults_to_none() { + // Every mint before the memory option existed embedded MemoryLevel::None + // structurally, so old sidecars (no memoryLevel field) must deserialize + // to None — the gallery's disclosure depends on this being honest. + let legacy = r#"{ + "storedFileName": "eva-1234.agent.png", + "fileName": "eva.agent.png", + "agentId": "abc", + "agentName": "Eva", + "designerNotes": "", + "locked": false, + "mintedAt": "2026-07-28T00:00:00Z" + }"#; + let meta: ArchivedCardMeta = serde_json::from_str(legacy).unwrap(); + assert_eq!(meta.memory_level, MemoryLevel::None); + + let with_level = legacy.replace( + "\"locked\": false,", + "\"locked\": false, \"memoryLevel\": \"everything\",", + ); + let meta: ArchivedCardMeta = serde_json::from_str(&with_level).unwrap(); + assert_eq!(meta.memory_level, MemoryLevel::Everything); +} + +#[test] +fn minted_card_serializes_memory_level_snake_case_value() { + // The TS layer narrows on the exact wire strings "none"/"core"/ + // "everything" — pin the serde representation the frontend will see. + let minted = MintedCard { + card_png_base64: String::new(), + file_name: "eva.agent.png".to_string(), + designer_notes: String::new(), + locked: false, + memory_level: MemoryLevel::Core, + }; + let json = serde_json::to_value(&minted).unwrap(); + assert_eq!(json["memoryLevel"], "core"); +} diff --git a/desktop/src-tauri/src/commands/personas/mod.rs b/desktop/src-tauri/src/commands/personas/mod.rs index 66f7296a25..0cd7ad0324 100644 --- a/desktop/src-tauri/src/commands/personas/mod.rs +++ b/desktop/src-tauri/src/commands/personas/mod.rs @@ -306,11 +306,14 @@ pub async fn set_persona_active( } pub(crate) const PNG_MAGIC: [u8; 4] = [0x89, 0x50, 0x4E, 0x47]; +mod card; mod snapshot; -pub use snapshot::encode_agent_snapshot_for_send; -pub use snapshot::export_agent_snapshot; +pub use card::*; +#[cfg(test)] +pub(crate) use snapshot::import::decode_snapshot_from_bytes; pub(crate) use snapshot::import::{ - decode_snapshot_from_bytes, resolve_snapshot_import_behavior, MAX_SNAPSHOT_JSON_BYTES, + parse_snapshot_payload_from_bytes, resolve_snapshot_import_behavior, MAX_SNAPSHOT_JSON_BYTES, MAX_SNAPSHOT_PNG_BYTES, }; pub use snapshot::{confirm_agent_snapshot_import, preview_agent_snapshot_import}; +pub use snapshot::{encode_agent_snapshot_for_send, export_agent_snapshot}; diff --git a/desktop/src-tauri/src/commands/personas/pending.rs b/desktop/src-tauri/src/commands/personas/pending.rs index a4003329bc..cab5fababc 100644 --- a/desktop/src-tauri/src/commands/personas/pending.rs +++ b/desktop/src-tauri/src/commands/personas/pending.rs @@ -75,11 +75,11 @@ pub(super) fn prepare_persona_publication( } fn retained_persona_is_shared(row: Option<&RetainedEvent>) -> bool { - use buzz_core_pkg::kind::persona_event_is_shared; + use buzz_core_pkg::kind::event_is_shared; use nostr::JsonUtil; row.and_then(|retained| nostr::Event::from_json(&retained.raw_event).ok()) - .is_some_and(|event| persona_event_is_shared(&event)) + .is_some_and(|event| event_is_shared(&event)) } /// Project each persona's catalog visibility from the active relay+owner diff --git a/desktop/src-tauri/src/commands/personas/snapshot.rs b/desktop/src-tauri/src/commands/personas/snapshot.rs index a3ba731875..e7bd1597e6 100644 --- a/desktop/src-tauri/src/commands/personas/snapshot.rs +++ b/desktop/src-tauri/src/commands/personas/snapshot.rs @@ -142,7 +142,7 @@ pub(crate) fn validate_snapshot_encode_size(bytes_len: usize, is_png: bool) -> R } /// Parse a `memory_level` string to `MemoryLevel`. -fn parse_memory_level(s: &str) -> Result { +pub(crate) fn parse_memory_level(s: &str) -> Result { match s { "none" | "" => Ok(MemoryLevel::None), "core" => Ok(MemoryLevel::Core), @@ -153,6 +153,32 @@ fn parse_memory_level(s: &str) -> Result { } } +/// Flatten an owner-decrypted memory listing into manifest entries for +/// `memory_level`: `Core` takes the core entry only; `Everything` appends all +/// `mem/*` entries after it. Pure so both the export and card-mint paths share +/// (and tests can pin) the level → entries selection. +pub(crate) fn memory_entries_from_listing( + listing: crate::commands::engrams::AgentMemoryListing, + memory_level: MemoryLevel, +) -> Vec { + let mut entries = Vec::new(); + if let Some(core) = listing.core { + entries.push(AgentSnapshotMemoryEntry { + slug: core.slug, + body: core.body, + }); + } + if memory_level == MemoryLevel::Everything { + for mem in listing.memories { + entries.push(AgentSnapshotMemoryEntry { + slug: mem.slug, + body: mem.body, + }); + } + } + entries +} + /// Parse a `format` string to a PNG flag. fn parse_format_is_png(s: &str) -> Result { match s { @@ -164,6 +190,33 @@ fn parse_format_is_png(s: &str) -> Result { } } +fn materialize_portable_runtime_defaults( + record: &mut ManagedAgentRecord, + global: &crate::managed_agents::GlobalAgentConfig, +) { + if record + .model + .as_deref() + .is_none_or(|value| value.trim().is_empty()) + { + record.model = global.model.clone(); + } + if record + .provider + .as_deref() + .is_none_or(|value| value.trim().is_empty()) + { + record.provider = global.provider.clone(); + } + if record + .runtime + .as_deref() + .is_none_or(|value| value.trim().is_empty()) + { + record.runtime = global.preferred_runtime.clone(); + } +} + /// Shared production encoding path. /// /// Resolves the agent definition, validates inputs, fetches optional memory, @@ -196,6 +249,13 @@ pub(crate) async fn materialize_snapshot_bytes( let definitions = load_agent_definitions(&app)?; let (def_record, is_definition) = resolve_from_lists(&id, &instances, &definitions) .map(|(r, is_def)| (r.clone(), is_def))?; + let mut def_record = def_record; + // A snapshot is a verbatim portable copy of the effective runtime, + // provider, and model configuration, not a pointer to the sender's + // machine-wide defaults. This does not translate or substitute values + // for a different recipient setup. + let global = crate::managed_agents::load_global_agent_config(&app).unwrap_or_default(); + materialize_portable_runtime_defaults(&mut def_record, &global); let memory_pubkey = if memory_level != MemoryLevel::None { let mpk = memory_source_pubkey.as_deref().unwrap_or(""); @@ -233,22 +293,7 @@ pub(crate) async fn materialize_snapshot_bytes( // ── Fetch memory ───────────────────────────────────────────────────────── let memory_entries: Vec = if let Some(pubkey) = memory_pubkey { let listing = get_agent_memory(pubkey, app.clone(), state).await?; - let mut entries = Vec::new(); - if let Some(core) = listing.core { - entries.push(AgentSnapshotMemoryEntry { - slug: core.slug, - body: core.body, - }); - } - if memory_level == MemoryLevel::Everything { - for mem in listing.memories { - entries.push(AgentSnapshotMemoryEntry { - slug: mem.slug, - body: mem.body, - }); - } - } - entries + memory_entries_from_listing(listing, memory_level) } else { Vec::new() }; @@ -400,6 +445,8 @@ pub async fn encode_agent_snapshot_for_send( }) } +#[cfg(test)] +mod fidelity_tests; #[cfg(test)] mod tests; diff --git a/desktop/src-tauri/src/commands/personas/snapshot/fidelity_tests.rs b/desktop/src-tauri/src/commands/personas/snapshot/fidelity_tests.rs new file mode 100644 index 0000000000..b769d74d7b --- /dev/null +++ b/desktop/src-tauri/src/commands/personas/snapshot/fidelity_tests.rs @@ -0,0 +1,271 @@ +use super::import::decode_snapshot_from_bytes; +use super::*; +use crate::managed_agents::{ + agent_snapshot::{ + AgentSnapshot, AgentSnapshotDefinition, AgentSnapshotMemory, AgentSnapshotProfile, + FORMAT_DISCRIMINATOR, FORMAT_VERSION, + }, + BackendKind, ManagedAgentRecord, RespondTo, +}; +use std::collections::BTreeMap; + +fn make_definition(slug: &str) -> ManagedAgentRecord { + ManagedAgentRecord { + pubkey: String::new(), + slug: Some(slug.to_string()), + name: slug.to_string(), + display_name: None, + persona_id: None, + private_key_nsec: String::new(), + auth_tag: None, + relay_url: String::new(), + avatar_url: None, + acp_command: String::new(), + agent_command: String::new(), + agent_command_override: None, + agent_args: vec![], + mcp_command: String::new(), + turn_timeout_seconds: 0, + idle_timeout_seconds: None, + max_turn_duration_seconds: None, + parallelism: 1, + system_prompt: None, + model: None, + provider: None, + persona_source_version: None, + env_vars: BTreeMap::new(), + start_on_app_launch: false, + auto_restart_on_config_change: false, + runtime_pid: None, + backend: BackendKind::Local, + backend_agent_id: None, + provider_binary_path: None, + team_id: None, + persona_team_dir: None, + persona_name_in_team: None, + created_at: String::new(), + updated_at: String::new(), + last_started_at: None, + last_stopped_at: None, + last_exit_code: None, + last_error: None, + last_error_code: None, + respond_to: RespondTo::default(), + respond_to_allowlist: vec![], + runtime: None, + name_pool: vec![], + is_builtin: false, + is_active: false, + shared: false, + source_team: None, + source_team_persona_slug: None, + catalog_source: None, + definition_respond_to: None, + definition_respond_to_allowlist: vec![], + definition_parallelism: None, + relay_mesh: None, + } +} + +/// Build a minimal valid AgentSnapshot for import tests. +fn make_snapshot( + memory_level: MemoryLevel, + entries: Vec, +) -> AgentSnapshot { + AgentSnapshot { + format: FORMAT_DISCRIMINATOR.to_string(), + version: FORMAT_VERSION, + definition: AgentSnapshotDefinition { + name: "Test Agent".to_string(), + source_is_builtin: false, + system_prompt: Some("You are helpful.".to_string()), + runtime: None, + model: None, + provider: None, + parallelism: None, + respond_to: None, + respond_to_allowlist: vec![], + name_pool: vec![], + idle_timeout_seconds: None, + max_turn_duration_seconds: None, + }, + profile: AgentSnapshotProfile { + display_name: "Test Agent".to_string(), + about: None, + avatar_data_url: None, + avatar_url: None, + }, + memory: AgentSnapshotMemory { + level: memory_level, + entries, + }, + } +} + +// ── Portable effective configuration ───────────────────────────────────── + +#[test] +fn inherited_runtime_provider_and_model_are_materialized_for_export() { + let mut record = make_definition("wren"); + let global = crate::managed_agents::GlobalAgentConfig { + preferred_runtime: Some("goose".to_string()), + provider: Some("databricks_v2".to_string()), + model: Some("databricks-gpt-5-6-sol".to_string()), + ..Default::default() + }; + + materialize_portable_runtime_defaults(&mut record, &global); + + assert_eq!(record.runtime.as_deref(), Some("goose")); + assert_eq!(record.provider.as_deref(), Some("databricks_v2")); + assert_eq!(record.model.as_deref(), Some("databricks-gpt-5-6-sol")); +} + +#[test] +fn explicit_runtime_provider_and_model_win_over_global_defaults() { + let mut record = make_definition("wren"); + record.runtime = Some("claude".to_string()); + record.provider = Some("anthropic".to_string()); + record.model = Some("claude-opus-5".to_string()); + let global = crate::managed_agents::GlobalAgentConfig { + preferred_runtime: Some("goose".to_string()), + provider: Some("databricks_v2".to_string()), + model: Some("databricks-gpt-5-6-sol".to_string()), + ..Default::default() + }; + + materialize_portable_runtime_defaults(&mut record, &global); + + assert_eq!(record.runtime.as_deref(), Some("claude")); + assert_eq!(record.provider.as_deref(), Some("anthropic")); + assert_eq!(record.model.as_deref(), Some("claude-opus-5")); +} + +/// PNG image-body avatar overrides manifest avatar fields and all definition +/// config survives the exact production decoder. +#[test] +fn import_png_body_avatar_and_full_model_round_trip() { + use crate::managed_agents::agent_snapshot::{decode_avatar_data_url, encode_snapshot_png}; + + let mut snapshot = make_snapshot(MemoryLevel::None, vec![]); + snapshot.definition.runtime = Some("goose".to_string()); + snapshot.definition.model = Some("databricks-gpt-5-6-sol".to_string()); + snapshot.definition.provider = Some("databricks_v2".to_string()); + snapshot.profile.avatar_data_url = None; + snapshot.profile.avatar_url = Some("https://sender.invalid/avatar.png".to_string()); + + let avatar = image::DynamicImage::ImageRgba8(image::RgbaImage::from_pixel( + 4, + 3, + image::Rgba([23, 91, 177, 255]), + )); + let mut avatar_png = std::io::Cursor::new(Vec::new()); + avatar + .write_to(&mut avatar_png, image::ImageFormat::Png) + .unwrap(); + let png_bytes = encode_snapshot_png(&snapshot, Some(avatar_png.get_ref())).unwrap(); + + let decoded = decode_snapshot_from_bytes(&png_bytes).unwrap(); + assert_eq!(decoded.definition.runtime.as_deref(), Some("goose")); + assert_eq!( + decoded.definition.model.as_deref(), + Some("databricks-gpt-5-6-sol") + ); + assert_eq!( + decoded.definition.provider.as_deref(), + Some("databricks_v2") + ); + assert_eq!( + decoded.profile.avatar_url.as_deref(), + Some("https://sender.invalid/avatar.png") + ); + + let avatar_data_url = decoded + .profile + .avatar_data_url + .as_deref() + .expect("PNG image body must become the effective portable avatar"); + let avatar_bytes = decode_avatar_data_url(avatar_data_url).unwrap(); + let imported_avatar = image::load_from_memory(&avatar_bytes).unwrap(); + assert_eq!((imported_avatar.width(), imported_avatar.height()), (4, 3)); + assert_eq!( + imported_avatar.to_rgba8().get_pixel(0, 0).0, + [23, 91, 177, 255] + ); +} + +/// The transparent 1×1 no-avatar card must not override a manifest fallback. +#[test] +fn import_png_placeholder_keeps_manifest_avatar_fallback() { + use crate::managed_agents::agent_snapshot::encode_snapshot_png; + + let mut snapshot = make_snapshot(MemoryLevel::None, vec![]); + snapshot.profile.avatar_data_url = None; + snapshot.profile.avatar_url = Some("https://example.com/avatar.png".to_string()); + let png_bytes = encode_snapshot_png(&snapshot, None).unwrap(); + + let decoded = decode_snapshot_from_bytes(&png_bytes).unwrap(); + assert!(decoded.profile.avatar_data_url.is_none()); + assert_eq!(decoded.profile.avatar_url, snapshot.profile.avatar_url); +} + +/// An unlocked trading card imports the agent's REAL avatar, never the card. +/// +/// Mint-shaped input: the PNG body is the generated card artwork, while the +/// manifest inlines the source avatar (`manifest_avatar_bytes` in `card.rs`). +/// The #3578 body-wins override must not fire when the manifest already +/// carries inline avatar bytes — otherwise the imported agent publishes the +/// 1500-wide card as its kind:0 picture. +#[test] +fn import_unlocked_card_uses_manifest_avatar_not_card_artwork() { + use crate::managed_agents::agent_snapshot::{decode_avatar_data_url, encode_snapshot_png}; + use base64::{engine::general_purpose::STANDARD, Engine as _}; + + // The real avatar: 4×3 solid blue, inlined in the manifest at mint time. + let real_avatar = image::DynamicImage::ImageRgba8(image::RgbaImage::from_pixel( + 4, + 3, + image::Rgba([23, 91, 177, 255]), + )); + let mut real_avatar_png = std::io::Cursor::new(Vec::new()); + real_avatar + .write_to(&mut real_avatar_png, image::ImageFormat::Png) + .unwrap(); + + let mut snapshot = make_snapshot(MemoryLevel::None, vec![]); + snapshot.profile.avatar_data_url = Some(format!( + "data:image/png;base64,{}", + STANDARD.encode(real_avatar_png.get_ref()) + )); + snapshot.profile.avatar_url = Some("https://relay.example/media/live-kind0.png".to_string()); + + // The card artwork: a distinct 1500×2250 solid red "trading card" as the + // PNG body — the exact dimensions the minter encodes for unlocked cards. + // Size matters: 2250px exceeds `snapshot_avatar`'s 2048px decode limit, + // so reaching the body override here wouldn't just import the wrong + // face — it would fail the import outright. + let card_art = image::DynamicImage::ImageRgba8(image::RgbaImage::from_pixel( + 1500, + 2250, + image::Rgba([200, 16, 16, 255]), + )); + let mut card_png = std::io::Cursor::new(Vec::new()); + card_art + .write_to(&mut card_png, image::ImageFormat::Png) + .unwrap(); + let file_bytes = encode_snapshot_png(&snapshot, Some(card_png.get_ref())).unwrap(); + + // Production import decode: the effective avatar must be the real one. + let decoded = decode_snapshot_from_bytes(&file_bytes).unwrap(); + let avatar_bytes = + decode_avatar_data_url(decoded.profile.avatar_data_url.as_deref().unwrap()).unwrap(); + let imported = image::load_from_memory(&avatar_bytes).unwrap(); + assert_eq!( + (imported.width(), imported.height()), + (4, 3), + "imported avatar must be the source avatar, not the card artwork" + ); + assert_eq!(imported.to_rgba8().get_pixel(0, 0).0, [23, 91, 177, 255]); + // The live kind:0 URL fallback survives untouched. + assert_eq!(decoded.profile.avatar_url, snapshot.profile.avatar_url); +} diff --git a/desktop/src-tauri/src/commands/personas/snapshot/import.rs b/desktop/src-tauri/src/commands/personas/snapshot/import.rs index 9d7d238918..d7f0323304 100644 --- a/desktop/src-tauri/src/commands/personas/snapshot/import.rs +++ b/desktop/src-tauri/src/commands/personas/snapshot/import.rs @@ -13,7 +13,11 @@ use tauri::{AppHandle, Emitter, State}; use crate::{ app_state::AppState, managed_agents::{ - agent_snapshot::{decode_snapshot_json, decode_snapshot_png, AgentSnapshot, MemoryLevel}, + agent_snapshot::{extract_chunk_payload_png, AgentSnapshot, MemoryLevel}, + agent_snapshot_envelope::{ + decrypt_envelope, parse_chunk_payload, resolve_unlock_secret, ChunkPayload, + LOCKED_CARD_REFUSAL, + }, load_managed_agents, load_personas, save_managed_agents, save_personas, AgentDefinition, ManagedAgentRecord, RespondTo, }, @@ -72,6 +76,16 @@ pub struct AgentSnapshotImportPreview { pub has_source_allowlist: bool, /// Number of source allowlist entries. pub source_allowlist_count: usize, + /// Full source allowlist entries, surfaced before import so hidden access + /// configuration is never reduced to a count. + pub source_allowlist: Vec, + /// Pretty-printed, validated manifest exactly as decoded from the file. + /// The UI makes this available before confirmation for full payload review. + pub manifest_json: String, + /// True when the snapshot came from a locked (encrypted) card that this + /// machine successfully unlocked. Cards that cannot be unlocked never + /// reach a preview — they fail closed with the locked-card refusal. + pub locked: bool, } /// The confirmation request sent from the UI after the user reviews the preview. @@ -210,42 +224,130 @@ const PNG_MAGIC: [u8; 4] = [0x89, 0x50, 0x4e, 0x47]; /// /// **Size cap:** PNG inputs over 10 MiB and JSON inputs over 5 MiB are rejected /// before allocation to avoid avoidable large-input work. -pub(crate) fn decode_snapshot_from_bytes( - file_bytes: &[u8], -) -> Result { - if file_bytes.len() >= 4 && file_bytes[..4] == PNG_MAGIC { +/// +/// **Locked cards:** a structurally valid locked envelope parses successfully +/// as `ChunkPayload::Locked` — no decryption happens here. Callers that can +/// unlock go through [`decode_snapshot_for_import`]; callers that only need +/// transit validation (e.g. `fetch_snapshot_bytes`) accept `Locked` as-is. +pub(crate) fn parse_snapshot_payload_from_bytes(file_bytes: &[u8]) -> Result { + let payload: ChunkPayload = if file_bytes.len() >= 4 && file_bytes[..4] == PNG_MAGIC { if file_bytes.len() > MAX_SNAPSHOT_PNG_BYTES { return Err(format!( "Snapshot file is too large ({} MiB). PNG snapshots must be under 10 MiB.", file_bytes.len() / (1024 * 1024) )); } - let snapshot = decode_snapshot_png(file_bytes)?; - if snapshot.memory.level == MemoryLevel::None && !snapshot.memory.entries.is_empty() { - return Err( - "Snapshot is malformed: memory.level is 'none' but entries are present." - .to_string(), - ); + let chunk_json = extract_chunk_payload_png(file_bytes)?; + let mut payload = parse_chunk_payload(&chunk_json)?; + // The PNG image body is the portable avatar. It deliberately wins over + // a manifest avatar *URL*, which may only be reachable by the sender. + // A 1×1 export placeholder leaves the manifest fallback intact. + // Inline manifest avatar *bytes* are authoritative and never + // overridden: trading cards supply the generated card artwork as the + // PNG body and carry the agent's real avatar inline — adopting the + // body there would import the card as the agent's face. + // Locked envelopes stay opaque here — there is no manifest to override + // until the unlock path decrypts one. + if let ChunkPayload::Plain(snapshot) = &mut payload { + if snapshot.profile.avatar_data_url.is_none() { + if let Some(avatar_data_url) = + crate::managed_agents::snapshot_avatar::snapshot_png_avatar_data_url( + file_bytes, + )? + { + snapshot.profile.avatar_data_url = Some(avatar_data_url); + } + } } - return Ok(snapshot); - } - // JSON path — apply size cap before serde allocation. - if file_bytes.len() > MAX_SNAPSHOT_JSON_BYTES { - return Err(format!( - "Snapshot file is too large ({} MiB). JSON snapshots must be under 5 MiB.", - file_bytes.len() / (1024 * 1024) - )); - } - let snapshot = decode_snapshot_json(file_bytes)?; + payload + } else { + // JSON path — apply size cap before serde allocation. + if file_bytes.len() > MAX_SNAPSHOT_JSON_BYTES { + return Err(format!( + "Snapshot file is too large ({} MiB). JSON snapshots must be under 5 MiB.", + file_bytes.len() / (1024 * 1024) + )); + } + parse_chunk_payload(file_bytes)? + }; // Consistency check: none + non-empty entries is always malformed, - // regardless of format. Mirrors the PNG path above so the rule is - // enforced at decode time for both formats. - if !snapshot.memory.entries.is_empty() && snapshot.memory.level == MemoryLevel::None { + // regardless of enclosing format. Enforced at decode time for plain + // payloads here, and after decryption for locked ones (see + // `enforce_memory_consistency` callers). + if let ChunkPayload::Plain(snapshot) = &payload { + enforce_memory_consistency(snapshot)?; + } + Ok(payload) +} + +/// The shared malformed-memory guard: `memory.level == none` with non-empty +/// entries is always rejected before any write. +fn enforce_memory_consistency( + snapshot: &crate::managed_agents::agent_snapshot::AgentSnapshot, +) -> Result<(), String> { + if snapshot.memory.level == MemoryLevel::None && !snapshot.memory.entries.is_empty() { return Err( "Snapshot is malformed: memory.level is 'none' but entries are present.".to_string(), ); } - Ok(snapshot) + Ok(()) +} + +/// Decode a plain snapshot from raw bytes, refusing locked cards. +/// +/// Test-only convenience: production call sites either unlock through +/// [`decode_snapshot_for_import`] or validate structurally through +/// [`parse_snapshot_payload_from_bytes`]. +#[cfg(test)] +pub(crate) fn decode_snapshot_from_bytes( + file_bytes: &[u8], +) -> Result { + match parse_snapshot_payload_from_bytes(file_bytes)? { + ChunkPayload::Plain(snapshot) => Ok(*snapshot), + ChunkPayload::Locked(_) => Err(LOCKED_CARD_REFUSAL.to_string()), + } +} + +/// Decode a snapshot for import, unlocking locked cards when — and only +/// when — this machine holds one of the envelope's two exact key endpoints +/// (the owner identity or the named local agent record). +/// +/// Returns the decoded manifest and whether it came from a locked envelope. +/// When neither endpoint exists, fails closed with the locked-card refusal — +/// never partial plaintext, never crypto details. +pub(crate) fn decode_snapshot_for_import( + file_bytes: &[u8], + owner_keys: Option<&nostr::Keys>, + records: &[ManagedAgentRecord], +) -> Result<(crate::managed_agents::agent_snapshot::AgentSnapshot, bool), String> { + match parse_snapshot_payload_from_bytes(file_bytes)? { + ChunkPayload::Plain(snapshot) => Ok((*snapshot, false)), + ChunkPayload::Locked(envelope) => { + let secret = resolve_unlock_secret(&envelope, owner_keys, records) + .ok_or_else(|| LOCKED_CARD_REFUSAL.to_string())?; + let snapshot = decrypt_envelope(&envelope, &secret)?; + enforce_memory_consistency(&snapshot)?; + Ok((snapshot, true)) + } + } +} + +async fn materialize_import_avatar( + avatar_data_url: Option<&str>, + avatar_url: Option<&str>, + upload: F, +) -> Result, String> +where + F: FnOnce(Vec) -> Fut, + Fut: std::future::Future>, +{ + let Some(avatar_data_url) = avatar_data_url else { + return Ok(avatar_url.map(str::to_string)); + }; + let avatar_bytes = + crate::managed_agents::agent_snapshot::decode_avatar_data_url(avatar_data_url) + .ok_or_else(|| "Snapshot avatar data is malformed.".to_string())?; + upload(avatar_bytes).await.map(Some) } // ── `preview_agent_snapshot_import` ────────────────────────────────────────── @@ -257,19 +359,38 @@ pub(crate) fn decode_snapshot_from_bytes( /// `.agent.png` file. The format is sniffed from the content, not the /// extension, so an incorrectly-named file is handled correctly. /// +/// Locked cards are unlocked here when this machine holds one of the +/// envelope's two exact key endpoints; a card that cannot be unlocked fails +/// with the locked-card refusal (shown directly to the user), never a +/// partial preview. Identity-recovery mode is tolerated: owner keys are +/// simply unavailable, so only the agent-record endpoint can unlock. +/// /// Returns an `AgentSnapshotImportPreview` or a descriptive error. Errors -/// represent irrecoverable failures (corrupt / unsupported file) and are -/// shown directly to the user. +/// represent irrecoverable failures (corrupt / unsupported / locked-to- +/// someone-else file) and are shown directly to the user. #[tauri::command] pub async fn preview_agent_snapshot_import( file_bytes: Vec, file_name: String, + app: AppHandle, + state: State<'_, AppState>, ) -> Result { + // Key material + records are gathered up front (cheap, lock-scoped) so + // the blocking decode below owns plain data. + let owner_keys = state.signing_keys().ok(); + let records = { + let _store_guard = state + .managed_agents_store_lock + .lock() + .map_err(|e| e.to_string())?; + load_managed_agents(&app)? + }; tokio::task::spawn_blocking(move || { reject_legacy_persona_filename(&file_name)?; - let snapshot = decode_snapshot_from_bytes(&file_bytes)?; + let (snapshot, locked) = + decode_snapshot_for_import(&file_bytes, owner_keys.as_ref(), &records)?; - Ok(build_agent_snapshot_import_preview(&snapshot)) + build_agent_snapshot_import_preview(&snapshot, locked) }) .await .map_err(|e| format!("spawn_blocking failed: {e}"))? @@ -277,7 +398,8 @@ pub async fn preview_agent_snapshot_import( pub(crate) fn build_agent_snapshot_import_preview( snapshot: &AgentSnapshot, -) -> AgentSnapshotImportPreview { + locked: bool, +) -> Result { let memory_level = match snapshot.memory.level { MemoryLevel::None => "none", MemoryLevel::Core => "core", @@ -285,7 +407,11 @@ pub(crate) fn build_agent_snapshot_import_preview( } .to_string(); - AgentSnapshotImportPreview { + let manifest_json = serde_json::to_string_pretty(snapshot) + .map_err(|e| format!("failed to render snapshot manifest: {e}"))?; + let source_allowlist = snapshot.definition.respond_to_allowlist.clone(); + + Ok(AgentSnapshotImportPreview { display_name: snapshot.profile.display_name.clone(), is_builtin: snapshot.definition.source_is_builtin, model: snapshot.definition.model.clone(), @@ -299,9 +425,12 @@ pub(crate) fn build_agent_snapshot_import_preview( .or_else(|| snapshot.profile.avatar_url.clone()), memory_level, memory_entry_count: snapshot.memory.entries.len(), - source_allowlist_count: snapshot.definition.respond_to_allowlist.len(), - has_source_allowlist: !snapshot.definition.respond_to_allowlist.is_empty(), - } + source_allowlist_count: source_allowlist.len(), + has_source_allowlist: !source_allowlist.is_empty(), + source_allowlist, + manifest_json, + locked, + }) } // ── `confirm_agent_snapshot_import` ────────────────────────────────────────── @@ -329,8 +458,20 @@ pub async fn confirm_agent_snapshot_import( app: AppHandle, state: State<'_, AppState>, ) -> Result { - // ── Phase 1: validate (no I/O) ─────────────────────────────────────────── - let snapshot = decode_snapshot_from_bytes(&input.file_bytes)?; + // ── Phase 1: validate (no writes) ──────────────────────────────────────── + // Locked cards unlock only via this machine's exact key endpoints; + // anything else fails closed here, before key generation. + let snapshot = { + let owner_keys = state.signing_keys().ok(); + let records = { + let _store_guard = state + .managed_agents_store_lock + .lock() + .map_err(|e| e.to_string())?; + load_managed_agents(&app)? + }; + decode_snapshot_for_import(&input.file_bytes, owner_keys.as_ref(), &records)?.0 + }; let display_name = snapshot.profile.display_name.trim().to_string(); if display_name.is_empty() { @@ -346,12 +487,21 @@ pub async fn confirm_agent_snapshot_import( )?; let minted_parallelism = minted.parallelism; - // Effective avatar: data URL wins; URL fallback when data URL is absent. - let effective_avatar: Option = snapshot - .profile - .avatar_data_url - .clone() - .or_else(|| snapshot.profile.avatar_url.clone()); + // Profile metadata must contain a hosted URL. Inline avatar data can be far + // larger than the relay's kind:0 content limit, so upload imported pixels + // before minting or persisting the new agent. Failing here keeps import + // atomic instead of creating an agent whose profile can never publish. + let effective_avatar = materialize_import_avatar( + snapshot.profile.avatar_data_url.as_deref(), + snapshot.profile.avatar_url.as_deref(), + |avatar_bytes| async { + crate::commands::media::upload_image_bytes(avatar_bytes, &state) + .await + .map(|descriptor| descriptor.url) + .map_err(|error| format!("Could not upload the imported avatar: {error}")) + }, + ) + .await?; // Wire-format string for the persona definition's respond_to field. // Omit when it is the default (owner-only) to keep definitions clean. @@ -650,7 +800,7 @@ fn retain_agent_pending(app: &AppHandle, state: &AppState, record: &ManagedAgent /// POST a pre-built signed engram event to the relay, authenticating as the /// new agent. -async fn submit_engram_event( +pub(crate) async fn submit_engram_event( state: &AppState, agent_keys: &nostr::Keys, event_json: &[u8], @@ -660,6 +810,8 @@ async fn submit_engram_event( use crate::relay::build_nip98_auth_header_for_keys; use reqwest::Method; + crate::egress_guard::assert_no_key_backup_bytes(event_json, "persona snapshot engram submit")?; + // Wait before signing: the relay enforces NIP-98 freshness (±60s) and the // gate may hold for up to MAX_HINT_SECONDS (300s). Building auth before the // wait produces a stale `created_at` that the relay will reject. @@ -703,3 +855,143 @@ async fn submit_engram_event( } Ok(()) } + +// ── NIP-49 egress guard: boundary 7 (persona snapshot engram submit) ───────── + +#[cfg(test)] +mod egress_guard_tests { + use super::submit_engram_event; + + const NCRYPTSEC: &str = "ncryptsec1qgg9947rlpvqu76pj5ecreduf9jxhselq2nae2kghhvd5g7dgjtcxfqtd67p9m0w57lspw8gsq6yphnm8623nsl8xn9j4jdzz84zm3frztj3z7s35vpzmqf6ksu8r89qk5z2zxfmu5gv8th8wclt0h4p"; + + /// An engram body carrying an ncryptsec must be rejected by the guard + /// before any network I/O (the target port is a discard address; a guard + /// error — not a connection error — proves the abort ordering). + #[tokio::test] + async fn blocks_ncryptsec_before_network() { + let state = crate::app_state::build_app_state(); + let keys = nostr::Keys::generate(); + let body = format!("{{\"content\":\"{NCRYPTSEC}\"}}"); + let err = submit_engram_event( + &state, + &keys, + body.as_bytes(), + "http://127.0.0.1:9/events", + None, + ) + .await + .unwrap_err(); + assert!(err.contains("key-backup material"), "{err}"); + } +} + +#[cfg(test)] +mod import_avatar_tests { + use super::materialize_import_avatar; + use std::cell::Cell; + + #[tokio::test] + async fn inline_avatar_is_uploaded_and_replaced_with_hosted_url() { + let uploaded = Cell::new(false); + let result = materialize_import_avatar( + Some("data:image/png;base64,iVBORw0KGgo="), + Some("https://sender.invalid/avatar.png"), + |bytes| { + uploaded.set(true); + async move { + assert_eq!(bytes, b"\x89PNG\r\n\x1a\n"); + Ok("https://relay.example/media/avatar.png".to_string()) + } + }, + ) + .await + .unwrap(); + + assert!(uploaded.get()); + assert_eq!( + result.as_deref(), + Some("https://relay.example/media/avatar.png") + ); + } + + #[tokio::test] + async fn hosted_avatar_skips_upload() { + let result = + materialize_import_avatar(None, Some("https://sender.example/avatar.png"), |_| async { + panic!("hosted avatars must not be uploaded") + }) + .await + .unwrap(); + + assert_eq!(result.as_deref(), Some("https://sender.example/avatar.png")); + } + + #[tokio::test] + async fn relay_sized_inline_avatar_becomes_bounded_signed_profile() { + use base64::{engine::general_purpose::STANDARD, Engine}; + use image::ImageEncoder; + use nostr::JsonUtil; + + let mut pixels = vec![0_u8; 512 * 512 * 4]; + let mut seed = 0x1234_5678_u32; + for byte in &mut pixels { + seed ^= seed << 13; + seed ^= seed >> 17; + seed ^= seed << 5; + *byte = seed as u8; + } + let mut source = Vec::new(); + image::codecs::png::PngEncoder::new(&mut source) + .write_image(&pixels, 512, 512, image::ExtendedColorType::Rgba8) + .unwrap(); + assert!(source.len() > 256 * 1024); + let data_url = format!("data:image/png;base64,{}", STANDARD.encode(&source)); + assert!(data_url.len() > 256 * 1024); + + let avatar = materialize_import_avatar(Some(&data_url), None, |bytes| async move { + let mime = crate::commands::media::detect_and_validate_mime(&bytes)?; + assert_eq!(mime, "image/png"); + let sanitized = crate::commands::media::sanitize_image_for_upload(bytes, &mime)?; + image::load_from_memory(&sanitized).map_err(|error| error.to_string())?; + Ok("https://relay.example/media/avatar.png".to_string()) + }) + .await + .unwrap() + .unwrap(); + + let event = + crate::events::build_profile(Some("Imported agent"), None, Some(&avatar), None, None) + .unwrap() + .sign_with_keys(&nostr::Keys::generate()) + .unwrap(); + assert!(event.content.len() < 64 * 1024); + assert!(!event.content.contains("data:image/")); + assert!(event + .content + .contains("https://relay.example/media/avatar.png")); + assert!(event.as_json().len() < 256 * 1024); + } + + #[tokio::test] + async fn upload_failure_aborts_avatar_materialization() { + let result = materialize_import_avatar( + Some("data:image/png;base64,iVBORw0KGgo="), + None, + |_| async { Err("relay upload failed".to_string()) }, + ) + .await; + + assert_eq!(result.unwrap_err(), "relay upload failed"); + } + + #[tokio::test] + async fn malformed_inline_avatar_fails_before_upload() { + let result = + materialize_import_avatar(Some("data:image/png;base64,not-base64!"), None, |_| async { + panic!("malformed avatars must not be uploaded") + }) + .await; + + assert_eq!(result.unwrap_err(), "Snapshot avatar data is malformed."); + } +} diff --git a/desktop/src-tauri/src/commands/personas/snapshot/tests.rs b/desktop/src-tauri/src/commands/personas/snapshot/tests.rs index 4289310280..c453b09a9d 100644 --- a/desktop/src-tauri/src/commands/personas/snapshot/tests.rs +++ b/desktop/src-tauri/src/commands/personas/snapshot/tests.rs @@ -564,7 +564,7 @@ fn import_preview_includes_exported_definition_metadata() { let bytes = crate::managed_agents::agent_snapshot::encode_snapshot_json(&snapshot).unwrap(); let decoded = decode_snapshot_from_bytes(&bytes).unwrap(); - let preview = build_agent_snapshot_import_preview(&decoded); + let preview = build_agent_snapshot_import_preview(&decoded, false).unwrap(); assert!(preview.is_builtin); assert_eq!(preview.model.as_deref(), Some("claude-opus-4-5")); @@ -949,51 +949,14 @@ fn test_parse_format_is_png_invalid_returns_error() { } // ── Export: validate_snapshot_encode_size ──────────────────────────────────── -// -// Tests call `validate_snapshot_encode_size` directly so they prove the exact -// production guard — not a manual reconstruction. Removing or reversing the -// check in production code will cause these tests to fail. -/// JSON: boundary-1 passes, boundary is the last legal byte count. -#[test] -fn validate_encode_size_json_at_boundary_minus_1_passes() { - assert!(super::validate_snapshot_encode_size(MAX_SNAPSHOT_JSON_BYTES - 1, false).is_ok()); -} +#[path = "tests_memory_entries.rs"] +mod memory_entries; -/// JSON: exactly at the boundary is the last accepted size. -#[test] -fn validate_encode_size_json_at_boundary_passes() { - assert!(super::validate_snapshot_encode_size(MAX_SNAPSHOT_JSON_BYTES, false).is_ok()); -} +#[path = "tests_encode_size.rs"] +mod encode_size; -/// JSON: boundary+1 is rejected. -#[test] -fn validate_encode_size_json_over_boundary_is_rejected() { - let err = super::validate_snapshot_encode_size(MAX_SNAPSHOT_JSON_BYTES + 1, false).unwrap_err(); - assert!( - err.contains("size limit"), - "error must mention size limit, got: {err}" - ); -} +// ── Import: decode_snapshot_for_import (locked cards) ───────────────────── -/// PNG: boundary-1 passes. -#[test] -fn validate_encode_size_png_at_boundary_minus_1_passes() { - assert!(super::validate_snapshot_encode_size(MAX_SNAPSHOT_PNG_BYTES - 1, true).is_ok()); -} - -/// PNG: exactly at the boundary passes. -#[test] -fn validate_encode_size_png_at_boundary_passes() { - assert!(super::validate_snapshot_encode_size(MAX_SNAPSHOT_PNG_BYTES, true).is_ok()); -} - -/// PNG: boundary+1 is rejected. -#[test] -fn validate_encode_size_png_over_boundary_is_rejected() { - let err = super::validate_snapshot_encode_size(MAX_SNAPSHOT_PNG_BYTES + 1, true).unwrap_err(); - assert!( - err.contains("size limit"), - "error must mention size limit, got: {err}" - ); -} +#[path = "tests_locked.rs"] +mod locked_import; diff --git a/desktop/src-tauri/src/commands/personas/snapshot/tests_encode_size.rs b/desktop/src-tauri/src/commands/personas/snapshot/tests_encode_size.rs new file mode 100644 index 0000000000..36eaa99716 --- /dev/null +++ b/desktop/src-tauri/src/commands/personas/snapshot/tests_encode_size.rs @@ -0,0 +1,55 @@ +//! Export-size guard tests for `validate_snapshot_encode_size`. +//! +//! Kept in a sibling file so `snapshot/tests.rs` stays under the +//! 1000-line gate; `#[path]`-included from there as a child module, +//! so `super::*` still resolves to the shared test imports. +//! +//! Tests call `validate_snapshot_encode_size` directly so they prove the +//! exact production guard — not a manual reconstruction. Removing or +//! reversing the check in production code will cause these tests to fail. + +use super::*; + +/// JSON: boundary-1 passes, boundary is the last legal byte count. +#[test] +fn validate_encode_size_json_at_boundary_minus_1_passes() { + assert!(validate_snapshot_encode_size(MAX_SNAPSHOT_JSON_BYTES - 1, false).is_ok()); +} + +/// JSON: exactly at the boundary is the last accepted size. +#[test] +fn validate_encode_size_json_at_boundary_passes() { + assert!(validate_snapshot_encode_size(MAX_SNAPSHOT_JSON_BYTES, false).is_ok()); +} + +/// JSON: boundary+1 is rejected. +#[test] +fn validate_encode_size_json_over_boundary_is_rejected() { + let err = validate_snapshot_encode_size(MAX_SNAPSHOT_JSON_BYTES + 1, false).unwrap_err(); + assert!( + err.contains("size limit"), + "error must mention size limit, got: {err}" + ); +} + +/// PNG: boundary-1 passes. +#[test] +fn validate_encode_size_png_at_boundary_minus_1_passes() { + assert!(validate_snapshot_encode_size(MAX_SNAPSHOT_PNG_BYTES - 1, true).is_ok()); +} + +/// PNG: exactly at the boundary passes. +#[test] +fn validate_encode_size_png_at_boundary_passes() { + assert!(validate_snapshot_encode_size(MAX_SNAPSHOT_PNG_BYTES, true).is_ok()); +} + +/// PNG: boundary+1 is rejected. +#[test] +fn validate_encode_size_png_over_boundary_is_rejected() { + let err = validate_snapshot_encode_size(MAX_SNAPSHOT_PNG_BYTES + 1, true).unwrap_err(); + assert!( + err.contains("size limit"), + "error must mention size limit, got: {err}" + ); +} diff --git a/desktop/src-tauri/src/commands/personas/snapshot/tests_locked.rs b/desktop/src-tauri/src/commands/personas/snapshot/tests_locked.rs new file mode 100644 index 0000000000..296444f78d --- /dev/null +++ b/desktop/src-tauri/src/commands/personas/snapshot/tests_locked.rs @@ -0,0 +1,129 @@ +//! Locked-card import tests for `decode_snapshot_for_import`. +//! +//! Kept in a sibling file so `snapshot/tests.rs` stays under the +//! 1000-line gate; `#[path]`-included from there as a child module, +//! so `super::*` still resolves to the shared test helpers. + +use super::*; +use crate::commands::personas::snapshot::import::{ + decode_snapshot_for_import, parse_snapshot_payload_from_bytes, +}; +use crate::managed_agents::agent_snapshot_envelope::{ + encode_locked_snapshot_png, encrypt_snapshot_envelope, ChunkPayload, LOCKED_CARD_REFUSAL, +}; + +/// Build a keyed instance record holding real key material, so the +/// agent-endpoint unlock path resolves exactly as production does. +fn record_for(agent: &nostr::Keys) -> ManagedAgentRecord { + ManagedAgentRecord { + pubkey: agent.public_key().to_hex(), + slug: None, + persona_id: Some("locked-test".to_string()), + private_key_nsec: nostr::ToBech32::to_bech32(agent.secret_key()).unwrap(), + ..make_definition("") + } +} + +fn locked_png(owner: &nostr::Keys, agent: &nostr::Keys) -> (AgentSnapshot, Vec) { + let snapshot = make_snapshot(MemoryLevel::None, vec![]); + let png = encode_locked_snapshot_png(&snapshot, owner, &agent.public_key(), None).unwrap(); + (snapshot, png) +} + +/// Owner identity key unlocks a locked card; `locked` is reported true. +#[test] +fn owner_endpoint_unlocks_locked_png() { + let (owner, agent) = (nostr::Keys::generate(), nostr::Keys::generate()); + let (snapshot, png) = locked_png(&owner, &agent); + let (decoded, locked) = decode_snapshot_for_import(&png, Some(&owner), &[]).unwrap(); + assert_eq!(decoded, snapshot); + assert!(locked); +} + +/// A local managed-agent record holding the agent nsec unlocks the card +/// even when the owner identity does not match (e.g. re-import on the +/// agent's own machine under a different owner identity). +#[test] +fn agent_record_endpoint_unlocks_locked_png() { + let (owner, agent) = (nostr::Keys::generate(), nostr::Keys::generate()); + let (snapshot, png) = locked_png(&owner, &agent); + let other_identity = nostr::Keys::generate(); + let records = vec![record_for(&agent)]; + let (decoded, locked) = + decode_snapshot_for_import(&png, Some(&other_identity), &records).unwrap(); + assert_eq!(decoded, snapshot); + assert!(locked); +} + +/// No matching endpoint → only the locked-card refusal, nothing else. +#[test] +fn stranger_fails_closed_with_refusal_only() { + let (owner, agent) = (nostr::Keys::generate(), nostr::Keys::generate()); + let (_snapshot, png) = locked_png(&owner, &agent); + let stranger = nostr::Keys::generate(); + let unrelated_record = record_for(&nostr::Keys::generate()); + let err = decode_snapshot_for_import(&png, Some(&stranger), &[unrelated_record]).unwrap_err(); + assert_eq!(err, LOCKED_CARD_REFUSAL); + // And with no key material at all. + let err = decode_snapshot_for_import(&png, None, &[]).unwrap_err(); + assert_eq!(err, LOCKED_CARD_REFUSAL); +} + +/// Plain snapshots pass through unchanged with `locked == false`, with or +/// without key material in scope. +#[test] +fn plain_snapshot_passes_through_unlocked() { + use crate::managed_agents::agent_snapshot::encode_snapshot_png; + let snapshot = make_snapshot(MemoryLevel::None, vec![]); + let png = encode_snapshot_png(&snapshot, None).unwrap(); + let owner = nostr::Keys::generate(); + let (decoded, locked) = decode_snapshot_for_import(&png, Some(&owner), &[]).unwrap(); + assert_eq!(decoded, snapshot); + assert!(!locked); + let (decoded, locked) = decode_snapshot_for_import(&png, None, &[]).unwrap(); + assert_eq!(decoded, snapshot); + assert!(!locked); +} + +/// The memory-consistency guard fires AFTER decryption too: a locked +/// envelope whose plaintext declares level none + non-empty entries is +/// rejected even for a legitimate endpoint. +#[test] +fn decrypted_manifest_memory_consistency_enforced() { + let (owner, agent) = (nostr::Keys::generate(), nostr::Keys::generate()); + let malformed = make_snapshot( + MemoryLevel::None, + vec![AgentSnapshotMemoryEntry { + slug: "core".to_string(), + body: "leaked".to_string(), + }], + ); + // encrypt_snapshot_envelope does not guard memory consistency (the + // PNG encoder does), so this constructs the malicious payload. + let envelope = encrypt_snapshot_envelope(&malformed, &owner, &agent.public_key()).unwrap(); + let json = serde_json::to_vec(&envelope).unwrap(); + let err = decode_snapshot_for_import(&json, Some(&owner), &[]).unwrap_err(); + assert!( + err.contains("'none' but entries are present"), + "post-decrypt consistency guard must fire, got: {err}" + ); +} + +/// Transit validation (`fetch_snapshot_bytes` path) accepts a locked PNG +/// without any key material — structural validation only, no decryption. +#[test] +fn transit_validation_accepts_locked_png_without_keys() { + let (owner, agent) = (nostr::Keys::generate(), nostr::Keys::generate()); + let (_snapshot, png) = locked_png(&owner, &agent); + let payload = parse_snapshot_payload_from_bytes(&png).unwrap(); + assert!(matches!(payload, ChunkPayload::Locked(_))); +} + +/// The keyless plain decoder refuses locked cards with the refusal. +#[test] +fn plain_decoder_refuses_locked_cards() { + let (owner, agent) = (nostr::Keys::generate(), nostr::Keys::generate()); + let (_snapshot, png) = locked_png(&owner, &agent); + let err = decode_snapshot_from_bytes(&png).unwrap_err(); + assert_eq!(err, LOCKED_CARD_REFUSAL); +} diff --git a/desktop/src-tauri/src/commands/personas/snapshot/tests_memory_entries.rs b/desktop/src-tauri/src/commands/personas/snapshot/tests_memory_entries.rs new file mode 100644 index 0000000000..b17efa1ad1 --- /dev/null +++ b/desktop/src-tauri/src/commands/personas/snapshot/tests_memory_entries.rs @@ -0,0 +1,55 @@ +//! Tests for `memory_entries_from_listing` — the shared level → entries +//! selection used by both snapshot export and card minting. Split from +//! `tests.rs` to keep that file under the 1000-line gate; `#[path]`-included +//! from there as a child module, so `super::*` resolves to `tests`'s parent +//! scope re-exports. + +use super::*; + +fn listing_fixture() -> crate::commands::engrams::AgentMemoryListing { + let entry = |slug: &str, body: &str| crate::commands::engrams::EngramEntry { + slug: slug.to_string(), + body: body.to_string(), + event_id: "e".repeat(64), + created_at: 1, + outgoing_refs: vec![], + }; + crate::commands::engrams::AgentMemoryListing { + core: Some(entry("core", "core body")), + memories: vec![entry("mem/a", "a body"), entry("mem/b", "b body")], + truncated: false, + fetched_at: 1, + } +} + +#[test] +fn memory_entries_core_takes_core_only() { + let entries = memory_entries_from_listing(listing_fixture(), MemoryLevel::Core); + assert_eq!(entries.len(), 1); + assert_eq!(entries[0].slug, "core"); + assert_eq!(entries[0].body, "core body"); +} + +#[test] +fn memory_entries_everything_appends_mem_entries_after_core() { + let entries = memory_entries_from_listing(listing_fixture(), MemoryLevel::Everything); + assert_eq!( + entries.iter().map(|e| e.slug.as_str()).collect::>(), + vec!["core", "mem/a", "mem/b"] + ); +} + +#[test] +fn memory_entries_missing_core_still_yields_mem_entries_for_everything() { + let mut listing = listing_fixture(); + listing.core = None; + let entries = memory_entries_from_listing(listing, MemoryLevel::Everything); + assert_eq!( + entries.iter().map(|e| e.slug.as_str()).collect::>(), + vec!["mem/a", "mem/b"] + ); + + let mut core_only = listing_fixture(); + core_only.core = None; + assert!(memory_entries_from_listing(core_only, MemoryLevel::Core).is_empty()); +} diff --git a/desktop/src-tauri/src/commands/project_git_merge_error.rs b/desktop/src-tauri/src/commands/project_git_merge_error.rs new file mode 100644 index 0000000000..460ce83fa3 --- /dev/null +++ b/desktop/src-tauri/src/commands/project_git_merge_error.rs @@ -0,0 +1,152 @@ +//! Structured pull-request merge failures returned across the Tauri boundary. + +use serde::Serialize; + +/// Machine-readable recovery metadata for a failed pull-request merge. +#[derive(Debug, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct ProjectPullRequestMergeRecovery { + action: String, + target_branch: String, + source_branch: String, +} + +/// Structured pull-request merge failure returned across the Tauri boundary. +#[derive(Debug, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct ProjectPullRequestMergeError { + code: String, + message: String, + recovery: Option, +} + +impl ProjectPullRequestMergeError { + pub(crate) fn new(code: &str, message: impl Into) -> Self { + Self { + code: code.to_string(), + message: message.into(), + recovery: None, + } + } + + fn conflict(target_branch: String, source_branch: String) -> Self { + Self { + code: "merge_conflict".to_string(), + message: "Pull request has merge conflicts.".to_string(), + recovery: Some(ProjectPullRequestMergeRecovery { + action: "open_terminal".to_string(), + target_branch, + source_branch, + }), + } + } +} + +impl From for ProjectPullRequestMergeError { + fn from(message: String) -> Self { + // Relay push-policy denial for a repo with no `buzz-channel` binding. + // The stable token is declared in `buzz-core::git_perms` + // (GIT_NO_CHANNEL_BINDING_TOKEN); the relay guarantees the denial body + // starts with it. Push failures reach this conversion as raw + // stderr/`remote:` text, so match the token anywhere in the message. + if message.contains(buzz_core_pkg::git_perms::GIT_NO_CHANNEL_BINDING_TOKEN) { + return Self::new( + buzz_core_pkg::git_perms::GIT_NO_CHANNEL_BINDING_TOKEN, + "This repository is not bound to a channel, so the relay cannot \ + authorize pushes. Bind it with: buzz repos bind --id \ + --channel ", + ); + } + Self::new("merge_failed", message) + } +} + +pub(crate) fn classify_merge_error( + message: String, + has_conflicts: bool, + target_branch: &str, + source_branch: &str, +) -> ProjectPullRequestMergeError { + if has_conflicts { + ProjectPullRequestMergeError::conflict(target_branch.to_string(), source_branch.to_string()) + } else { + ProjectPullRequestMergeError::new( + "merge_failed", + format!("Pull request merge failed: {message}"), + ) + } +} + +#[cfg(test)] +mod tests { + use super::{classify_merge_error, ProjectPullRequestMergeError}; + + #[test] + fn merge_conflict_error_has_stable_recovery_metadata() { + let error = + ProjectPullRequestMergeError::conflict("main".to_string(), "feature/demo".to_string()); + + assert_eq!(error.code, "merge_conflict"); + assert_eq!(error.message, "Pull request has merge conflicts."); + let recovery = error.recovery.expect("conflict recovery"); + assert_eq!(recovery.action, "open_terminal"); + assert_eq!(recovery.target_branch, "main"); + assert_eq!(recovery.source_branch, "feature/demo"); + } + + #[test] + fn merge_conflict_error_serializes_for_tauri_clients() { + let error = + ProjectPullRequestMergeError::conflict("main".to_string(), "feature/demo".to_string()); + let value = serde_json::to_value(error).expect("serialize merge conflict"); + + assert_eq!(value["code"], "merge_conflict"); + assert_eq!(value["recovery"]["targetBranch"], "main"); + assert_eq!(value["recovery"]["sourceBranch"], "feature/demo"); + } + + #[test] + fn merge_error_classification_only_recovers_conflicts() { + let conflict = classify_merge_error( + "CONFLICT (content): Merge conflict in src/main.rs".to_string(), + true, + "main", + "feature/demo", + ); + assert_eq!(conflict.code, "merge_conflict"); + assert!(conflict.recovery.is_some()); + + let other = classify_merge_error( + "fatal: refusing to merge unrelated histories".to_string(), + false, + "main", + "feature/demo", + ); + assert_eq!(other.code, "merge_failed"); + assert!(other.recovery.is_none()); + } + + #[test] + fn no_channel_binding_denial_converts_to_structured_code() { + // The relay's push-policy denial arrives as raw git stderr with + // `remote:` framing; the stable token must be recognized wherever it + // sits in the message. + let remote_stderr = format!( + "remote: {}\nerror: failed to push some refs", + buzz_core_pkg::git_perms::GIT_NO_CHANNEL_BINDING_BODY + ); + let error = ProjectPullRequestMergeError::from(remote_stderr); + + assert_eq!( + error.code, + buzz_core_pkg::git_perms::GIT_NO_CHANNEL_BINDING_TOKEN + ); + assert!(error.message.contains("buzz repos bind")); + assert!(error.recovery.is_none()); + + // Unrelated push failures keep the generic code and original text. + let generic = ProjectPullRequestMergeError::from("connection reset".to_string()); + assert_eq!(generic.code, "merge_failed"); + assert_eq!(generic.message, "connection reset"); + } +} diff --git a/desktop/src-tauri/src/commands/project_git_workflow.rs b/desktop/src-tauri/src/commands/project_git_workflow.rs index 39832feb10..624bbf4dfc 100644 --- a/desktop/src-tauri/src/commands/project_git_workflow.rs +++ b/desktop/src-tauri/src/commands/project_git_workflow.rs @@ -32,67 +32,10 @@ pub struct ProjectRepoMergeResult { pub status_publication_error: Option, } -/// Machine-readable recovery metadata for a failed pull-request merge. -#[derive(Debug, Serialize)] -#[serde(rename_all = "camelCase")] -pub struct ProjectPullRequestMergeRecovery { - action: String, - target_branch: String, - source_branch: String, -} - -/// Structured pull-request merge failure returned across the Tauri boundary. -#[derive(Debug, Serialize)] -#[serde(rename_all = "camelCase")] -pub struct ProjectPullRequestMergeError { - code: String, - message: String, - recovery: Option, -} - -impl ProjectPullRequestMergeError { - fn new(code: &str, message: impl Into) -> Self { - Self { - code: code.to_string(), - message: message.into(), - recovery: None, - } - } - - fn conflict(target_branch: String, source_branch: String) -> Self { - Self { - code: "merge_conflict".to_string(), - message: "Pull request has merge conflicts.".to_string(), - recovery: Some(ProjectPullRequestMergeRecovery { - action: "open_terminal".to_string(), - target_branch, - source_branch, - }), - } - } -} - -impl From for ProjectPullRequestMergeError { - fn from(message: String) -> Self { - Self::new("merge_failed", message) - } -} - -fn classify_merge_error( - message: String, - has_conflicts: bool, - target_branch: &str, - source_branch: &str, -) -> ProjectPullRequestMergeError { - if has_conflicts { - ProjectPullRequestMergeError::conflict(target_branch.to_string(), source_branch.to_string()) - } else { - ProjectPullRequestMergeError::new( - "merge_failed", - format!("Pull request merge failed: {message}"), - ) - } -} +/// Machine-readable pull-request merge failure types live in +/// [`super::project_git_merge_error`]; re-imported here for the merge +/// workflow below. +use super::project_git_merge_error::{classify_merge_error, ProjectPullRequestMergeError}; struct ProjectRepoMergeGitResult { message: String, @@ -739,8 +682,8 @@ pub async fn merge_project_pull_request( mod tests { use super::{ align_unborn_head_branch, build_merged_status_event, build_pull_request_status_event, - build_review_request_event, classify_merge_error, normalize_commit, same_repository, - validate_merge_status_metadata, ProjectPullRequestMergeError, + build_review_request_event, normalize_commit, same_repository, + validate_merge_status_metadata, }; use crate::commands::project_git_exec::{build_test_git_auth_config, run_git}; use nostr::{Event, JsonUtil, Keys, Timestamp}; @@ -785,51 +728,6 @@ mod tests { )); } - #[test] - fn merge_conflict_error_has_stable_recovery_metadata() { - let error = - ProjectPullRequestMergeError::conflict("main".to_string(), "feature/demo".to_string()); - - assert_eq!(error.code, "merge_conflict"); - assert_eq!(error.message, "Pull request has merge conflicts."); - let recovery = error.recovery.expect("conflict recovery"); - assert_eq!(recovery.action, "open_terminal"); - assert_eq!(recovery.target_branch, "main"); - assert_eq!(recovery.source_branch, "feature/demo"); - } - - #[test] - fn merge_conflict_error_serializes_for_tauri_clients() { - let error = - ProjectPullRequestMergeError::conflict("main".to_string(), "feature/demo".to_string()); - let value = serde_json::to_value(error).expect("serialize merge conflict"); - - assert_eq!(value["code"], "merge_conflict"); - assert_eq!(value["recovery"]["targetBranch"], "main"); - assert_eq!(value["recovery"]["sourceBranch"], "feature/demo"); - } - - #[test] - fn merge_error_classification_only_recovers_conflicts() { - let conflict = classify_merge_error( - "CONFLICT (content): Merge conflict in src/main.rs".to_string(), - true, - "main", - "feature/demo", - ); - assert_eq!(conflict.code, "merge_conflict"); - assert!(conflict.recovery.is_some()); - - let other = classify_merge_error( - "fatal: refusing to merge unrelated histories".to_string(), - false, - "main", - "feature/demo", - ); - assert_eq!(other.code, "merge_failed"); - assert!(other.recovery.is_none()); - } - #[test] fn merged_status_is_signed_by_repository_owner() { let keys = Keys::generate(); diff --git a/desktop/src-tauri/src/commands/team_snapshot.rs b/desktop/src-tauri/src/commands/team_snapshot.rs index 91a0126f58..97cd11933d 100644 --- a/desktop/src-tauri/src/commands/team_snapshot.rs +++ b/desktop/src-tauri/src/commands/team_snapshot.rs @@ -895,7 +895,7 @@ fn retain_agent_pending(app: &AppHandle, state: &AppState, record: &ManagedAgent /// POST a pre-built signed engram event to the relay, authenticating as the /// new agent. Mirrors the same helper in `snapshot::import`. -async fn submit_engram_event( +pub(crate) async fn submit_engram_event( state: &AppState, agent_keys: &nostr::Keys, event_json: &[u8], @@ -905,6 +905,8 @@ async fn submit_engram_event( use crate::relay::build_nip98_auth_header_for_keys; use reqwest::Method; + crate::egress_guard::assert_no_key_backup_bytes(event_json, "team snapshot engram submit")?; + // Wait before signing: the relay enforces NIP-98 freshness (±60s) and the // gate may hold for up to MAX_HINT_SECONDS (300s). Building auth before the // wait produces a stale `created_at` that the relay will reject. diff --git a/desktop/src-tauri/src/commands/team_snapshot/tests.rs b/desktop/src-tauri/src/commands/team_snapshot/tests.rs index 0616411307..c9a6d8812a 100644 --- a/desktop/src-tauri/src/commands/team_snapshot/tests.rs +++ b/desktop/src-tauri/src/commands/team_snapshot/tests.rs @@ -733,3 +733,31 @@ fn full_rollback_at_teams_boundary_absent_agents_store() { assert!(!teams_path.exists()); assert_eq!(errors.len(), 1, "only the teams-write error"); } + +// ── NIP-49 egress guard: boundary 6 (team snapshot engram submit) ──────────── + +mod egress_guard_boundary { + use super::super::submit_engram_event; + + const NCRYPTSEC: &str = "ncryptsec1qgg9947rlpvqu76pj5ecreduf9jxhselq2nae2kghhvd5g7dgjtcxfqtd67p9m0w57lspw8gsq6yphnm8623nsl8xn9j4jdzz84zm3frztj3z7s35vpzmqf6ksu8r89qk5z2zxfmu5gv8th8wclt0h4p"; + + /// An engram body carrying an ncryptsec must be rejected by the guard + /// before any network I/O (the target port is a discard address; a guard + /// error — not a connection error — proves the abort ordering). + #[tokio::test] + async fn blocks_ncryptsec_before_network() { + let state = crate::app_state::build_app_state(); + let keys = nostr::Keys::generate(); + let body = format!("{{\"content\":\"{NCRYPTSEC}\"}}"); + let err = submit_engram_event( + &state, + &keys, + body.as_bytes(), + "http://127.0.0.1:9/events", + None, + ) + .await + .unwrap_err(); + assert!(err.contains("key-backup material"), "{err}"); + } +} diff --git a/desktop/src-tauri/src/egress_guard.rs b/desktop/src-tauri/src/egress_guard.rs new file mode 100644 index 0000000000..db58ddafa0 --- /dev/null +++ b/desktop/src-tauri/src/egress_guard.rs @@ -0,0 +1,58 @@ +//! Relay egress guard for NIP-49 key-backup material. +//! +//! The local `ncryptsec` backup (see [`crate::key_backup`]) must NEVER be +//! transmitted to a relay. This module enforces that contract at runtime, +//! fail-closed, at every relay-bound egress boundary: +//! +//! | # | Boundary | Site | +//! |---|----------|------| +//! | 1 | `submit_signed_event_at_with_keys` (funnel for `submit_event*`) | `relay/submit.rs` | +//! | 2 | `sync_managed_agent_profile` | `relay.rs` | +//! | 3 | pre-signed path into the boundary-1 funnel | `relay/submit.rs` | +//! | 4 | `submit_signed_event_with_keys` | `relay.rs` | +//! | 5 | huddle STT publisher | `huddle/pipeline.rs` | +//! | 6 | `submit_engram_event` (team snapshot) | `commands/team_snapshot.rs` | +//! | 7 | `submit_engram_event` (persona import) | `commands/personas/snapshot/import.rs` | +//! | 8 | native websocket send loop (all webview relay WS) | `native_websocket.rs` | +//! +//! The inventory-completeness test in `egress_guard_tests.rs` asserts that +//! every `/events` URL-construction site in the tree calls this guard, so a +//! new submission path fails the build until it is wired. +//! +//! Scope: `ncryptsec1` only. The raw `nsec` intentionally transits the +//! NIP-44-encrypted pairing session (NIP-AB payload_type "nsec"); guarding it +//! here would break pairing. Raw-key DLP is separate policy work. + +/// Bech32 HRP of NIP-49 encrypted secret keys. +const NCRYPTSEC_PREFIX: &str = "ncryptsec1"; +/// Bech32 also permits an ALL-UPPERCASE encoding of the same payload +/// (BIP-173); an uppercased valid backup decodes identically, so the guard +/// must reject it too. Mixed case is invalid bech32 and cannot decode — a +/// substring matching either all-lower or all-upper prefix covers every +/// decodable form. +const NCRYPTSEC_PREFIX_UPPER: &str = "NCRYPTSEC1"; + +/// Reject `text` if it contains NIP-49 key-backup material. +/// +/// Returns `Err` when an `ncryptsec1…` (or uppercase `NCRYPTSEC1…`) +/// substring is present. Callers MUST abort the network operation on `Err` — +/// this is a fail-closed guard, not a warning. +pub fn assert_no_key_backup(text: &str, context: &'static str) -> Result<(), String> { + if text.contains(NCRYPTSEC_PREFIX) || text.contains(NCRYPTSEC_PREFIX_UPPER) { + return Err(format!( + "blocked {context}: payload contains NIP-49 key-backup material \ + (ncryptsec); the local key backup must never be transmitted to a relay" + )); + } + Ok(()) +} + +/// Byte-slice variant for callers that hold serialized bodies. +pub fn assert_no_key_backup_bytes(body: &[u8], context: &'static str) -> Result<(), String> { + // ncryptsec is ASCII bech32; a UTF-8-lossy view preserves any occurrence. + assert_no_key_backup(&String::from_utf8_lossy(body), context) +} + +#[cfg(test)] +#[path = "egress_guard_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/egress_guard_tests.rs b/desktop/src-tauri/src/egress_guard_tests.rs new file mode 100644 index 0000000000..f487c8ce16 --- /dev/null +++ b/desktop/src-tauri/src/egress_guard_tests.rs @@ -0,0 +1,446 @@ +use super::*; + +/// NIP-49 spec vector — a real ncryptsec blob for injection payloads. +const NCRYPTSEC: &str = "ncryptsec1qgg9947rlpvqu76pj5ecreduf9jxhselq2nae2kghhvd5g7dgjtcxfqtd67p9m0w57lspw8gsq6yphnm8623nsl8xn9j4jdzz84zm3frztj3z7s35vpzmqf6ksu8r89qk5z2zxfmu5gv8th8wclt0h4p"; + +fn assert_guard_error(err: &str) { + assert!( + err.contains("key-backup material"), + "expected the egress-guard error, got: {err}" + ); +} + +// ── Guard unit behavior ─────────────────────────────────────────────────────── + +#[test] +fn rejects_ncryptsec_anywhere_in_text() { + assert_guard_error(&assert_no_key_backup(NCRYPTSEC, "test").unwrap_err()); + assert_guard_error( + &assert_no_key_backup( + &format!("{{\"content\":\"my backup: {NCRYPTSEC}\"}}"), + "test", + ) + .unwrap_err(), + ); +} + +/// Bech32 permits an all-uppercase encoding of the same payload — an +/// uppercased valid backup must not bypass the guard (text and bytes). +/// Mixed case is invalid bech32 (cannot decode) and is deliberately not +/// blocked. +#[test] +fn rejects_uppercase_ncryptsec() { + let upper = NCRYPTSEC.to_ascii_uppercase(); + assert_guard_error(&assert_no_key_backup(&upper, "test").unwrap_err()); + assert_guard_error(&assert_no_key_backup_bytes(upper.as_bytes(), "test").unwrap_err()); + // Mixed case cannot decode; not blocked. + assert!(assert_no_key_backup("nCrYpTsEc1qgg9947r", "test").is_ok()); +} + +#[test] +fn passes_clean_payloads_including_raw_nsec() { + assert!(assert_no_key_backup("hello world", "test").is_ok()); + assert!(assert_no_key_backup("", "test").is_ok()); + // Scope is ncryptsec1 ONLY: raw nsec intentionally transits the encrypted + // pairing session and must NOT be blocked (plan D4 / pairing.rs). + let nsec = nostr::ToBech32::to_bech32(nostr::Keys::generate().secret_key()).unwrap(); + assert!(assert_no_key_backup(&nsec, "test").is_ok()); + // Near-miss prefixes are not blocked. + assert!(assert_no_key_backup("ncryptsec", "test").is_ok()); +} + +#[test] +fn byte_variant_matches_text_variant() { + assert_guard_error(&assert_no_key_backup_bytes(NCRYPTSEC.as_bytes(), "test").unwrap_err()); + assert!(assert_no_key_backup_bytes(b"clean body", "test").is_ok()); + // Invalid UTF-8 around an intact ncryptsec substring must still trip the + // guard (from_utf8_lossy preserves the ASCII run). + let mut body = vec![0xff, 0xfe]; + body.extend_from_slice(NCRYPTSEC.as_bytes()); + body.push(0xff); + assert_guard_error(&assert_no_key_backup_bytes(&body, "test").unwrap_err()); +} + +#[test] +fn error_names_the_boundary_context() { + let err = assert_no_key_backup(NCRYPTSEC, "huddle STT publish").unwrap_err(); + assert!(err.contains("huddle STT publish"), "{err}"); +} + +// ── Runtime injection per boundary ──────────────────────────────────────────── +// +// Each test drives the real production function with an ncryptsec-bearing +// payload and asserts the guard aborts the operation before any network I/O +// (no listener exists at the target address; a distinctive guard error — not +// a connection error — proves the abort happened first). +// +// Boundaries 6 and 7 (`submit_engram_event` twins) are module-private inside +// `commands`; their injection tests live next to them: +// - commands/team_snapshot/tests.rs::egress_guard_boundary +// - commands/personas/snapshot/import.rs::egress_guard_tests + +/// Boundary 1: `relay/submit.rs` `submit_event_at_with_keys` (the funnel for +/// all `submit_event*` variants). +#[tokio::test] +async fn boundary_submit_event_at_with_keys_blocks_ncryptsec() { + let state = crate::app_state::build_app_state(); + let keys = nostr::Keys::generate(); + let builder = nostr::EventBuilder::new(nostr::Kind::Custom(9), NCRYPTSEC); + let err = crate::relay::submit_event_at_with_keys( + builder, + &state, + "http://127.0.0.1:9", // discard port — must never be reached + &keys, + ) + .await + .unwrap_err(); + assert_guard_error(&err); +} + +/// Boundary 2: `relay.rs` `sync_managed_agent_profile` (agent kind:0 profile). +#[tokio::test] +async fn boundary_sync_managed_agent_profile_blocks_ncryptsec() { + let state = crate::app_state::build_app_state(); + let keys = nostr::Keys::generate(); + let err = crate::relay::sync_managed_agent_profile( + &state, + "ws://127.0.0.1:9", + &keys, + &format!("agent {NCRYPTSEC}"), + None, + None, + ) + .await + .unwrap_err(); + assert_guard_error(&err); +} + +/// Boundary 3: `relay/submit.rs` `submit_signed_event_at_with_keys` — the +/// pre-signed entry into the boundary-1 funnel (main's submit refactor +/// replaced `relay.rs` `submit_signed_event` with this scoped form). +#[tokio::test] +async fn boundary_submit_signed_event_at_with_keys_blocks_ncryptsec() { + let state = crate::app_state::build_app_state(); + let keys = nostr::Keys::generate(); + let event = nostr::EventBuilder::new(nostr::Kind::Custom(9), NCRYPTSEC) + .sign_with_keys(&keys) + .unwrap(); + let err = crate::relay::submit_signed_event_at_with_keys( + &event, + &state, + "http://127.0.0.1:9", // discard port — must never be reached + &keys, + ) + .await + .unwrap_err(); + assert_guard_error(&err); +} + +/// Boundary 4: `relay.rs` `submit_signed_event_with_keys`. +#[tokio::test] +async fn boundary_submit_signed_event_with_keys_blocks_ncryptsec() { + let state = crate::app_state::build_app_state(); + *state.relay_url_override.lock().unwrap() = Some("ws://127.0.0.1:9".to_string()); + let keys = nostr::Keys::generate(); + let event = nostr::EventBuilder::new(nostr::Kind::Custom(9), NCRYPTSEC) + .sign_with_keys(&keys) + .unwrap(); + let err = crate::relay::submit_signed_event_with_keys(&event, &state, &keys, None) + .await + .unwrap_err(); + assert_guard_error(&err); +} + +/// Boundary 5: huddle STT publisher (`huddle/pipeline.rs`). +#[test] +fn boundary_huddle_stt_blocks_ncryptsec() { + let keys = nostr::Keys::generate(); + let channel = uuid::Uuid::new_v4(); + let builder = + crate::events::build_message(channel, NCRYPTSEC, None, &[], &[], &[], &[]).unwrap(); + let err = crate::huddle::pipeline::sign_and_guard_stt_body(builder, &keys).unwrap_err(); + assert_guard_error(&err); + + // Clean transcripts pass through the same seam. + let builder = + crate::events::build_message(channel, "hello huddle", None, &[], &[], &[], &[]).unwrap(); + assert!(crate::huddle::pipeline::sign_and_guard_stt_body(builder, &keys).is_ok()); +} + +/// Boundary 8: native websocket send loop — the single choke point for all +/// webview-originated relay websocket frames. +#[tokio::test] +async fn boundary_native_websocket_blocks_ncryptsec() { + let manager = crate::native_websocket::WebSocketManager::default(); + // Text frame: guard fires before the connection lookup, so no connection + // is needed — and the error must be the guard's, not "not found". + let err = crate::native_websocket::send_message( + &manager, + 1, + crate::native_websocket::WebSocketMessage::Text(format!( + "[\"EVENT\",{{\"content\":\"{NCRYPTSEC}\"}}]" + )), + ) + .await + .unwrap_err(); + assert_guard_error(&err); + + // Binary frame variant. + let err = crate::native_websocket::send_message( + &manager, + 1, + crate::native_websocket::WebSocketMessage::Binary(NCRYPTSEC.as_bytes().to_vec()), + ) + .await + .unwrap_err(); + assert_guard_error(&err); + + // Clean frames fall through to normal handling ("connection not found" + // here — the guard did not reject them). + let err = crate::native_websocket::send_message( + &manager, + 1, + crate::native_websocket::WebSocketMessage::Text("[\"REQ\",\"sub\",{}]".to_string()), + ) + .await + .unwrap_err(); + assert!(err.contains("not found"), "{err}"); +} + +// ── Structural tripwires ────────────────────────────────────────────────────── + +fn src_rust_files() -> Vec { + fn walk(dir: &std::path::Path, out: &mut Vec) { + for entry in std::fs::read_dir(dir).unwrap() { + let path = entry.unwrap().path(); + if path.is_dir() { + walk(&path, out); + } else if path.extension().and_then(|e| e.to_str()) == Some("rs") { + out.push(path); + } + } + } + let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); + let mut out = Vec::new(); + walk(&root, &mut out); + out +} + +/// Site-granular `/events` inventory: `(file suffix, expected non-comment +/// `/events` occurrences, expected guard call sites — full-path calls into +/// the egress-guard module)`. +/// +/// Every entry pairs the URL-construction count with the guard-call count for +/// that file, so BOTH of these fail the scan (not just a brand-new file): +/// - adding an unguarded ninth `/events` site inside an already-listed file +/// (count goes up without a matching table update), and +/// - removing/refactoring away a guard call while its egress site remains. +/// +/// Updating a row here is the deliberate act that must accompany wiring the +/// guard + adding an injection test for the new site. +const EVENTS_INVENTORY: &[(&str, usize, usize)] = &[ + // Production egress boundaries (see egress_guard.rs table): + ("src/relay.rs", 2, 2), // boundaries 2, 4 + ("src/relay/submit.rs", 1, 1), // boundaries 1 + 3 (shared funnel) + ("src/huddle/pipeline.rs", 1, 1), // boundary 5 + ("src/commands/team_snapshot.rs", 1, 1), // boundary 6 + ("src/commands/personas/snapshot/import.rs", 2, 1), // boundary 7 + its in-file injection-test fixture URL + ("src/native_websocket.rs", 0, 2), // boundary 8 (WS frames; no events URL) + // Test-only fixtures — no production egress, no guard: + ("src/relay_admission.rs", 1, 0), + ("src/archive/mod_tests.rs", 1, 0), + ("src/managed_agents/persona_events/tests.rs", 1, 0), + ("src/commands/team_snapshot/tests.rs", 1, 0), + // Mock-relay route in its in-file tests; production publish goes through + // the guarded boundary-1 funnel (`submit_signed_event_at_with_keys`). + ("src/commands/personas/sharing.rs", 1, 0), +]; + +// Needles are assembled at runtime so this scan file itself contains no +// contiguous match and needs no self-referential inventory row. +fn events_needle() -> String { + ["/ev", "ents"].concat() +} +fn guard_needle() -> String { + ["egress_guard::", "assert_no_key_backup"].concat() +} + +/// Pure scan core over `(relative path, content)` pairs. Returns violations; +/// empty means every file matches its inventory row exactly (files absent +/// from the table are expected to have zero `/events` sites and zero guard +/// calls). +fn events_inventory_violations(files: &[(String, String)]) -> Vec { + let events = events_needle(); + let guard = guard_needle(); + let mut violations = Vec::new(); + + for (rel, content) in files { + let expected = EVENTS_INVENTORY + .iter() + .find(|(suffix, _, _)| rel.ends_with(suffix)) + .map(|&(_, e, g)| (e, g)) + .unwrap_or((0, 0)); + + let mut event_sites = Vec::new(); + for (i, line) in content.lines().enumerate() { + if line.trim_start().starts_with("//") { + continue; // doc/comment mentions + } + if line.contains(&events) { + event_sites.push(format!(" {rel}:{}: {}", i + 1, line.trim())); + } + } + let guard_count = content.matches(&guard).count(); + + if (event_sites.len(), guard_count) != expected { + violations.push(format!( + "{rel}: found {} events-URL site(s) + {} guard call(s), inventory \ + expects {} + {}. Sites found:\n{}", + event_sites.len(), + guard_count, + expected.0, + expected.1, + if event_sites.is_empty() { + " (none)".to_string() + } else { + event_sites.join("\n") + }, + )); + } + } + violations +} + +fn read_src_files() -> Vec<(String, String)> { + let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")); + src_rust_files() + .into_iter() + .map(|path| { + let rel = path + .strip_prefix(root) + .unwrap() + .to_string_lossy() + .replace('\\', "/"); + let content = std::fs::read_to_string(&path).unwrap(); + (rel, content) + }) + .collect() +} + +/// Inventory completeness: every `/events` URL-construction site in +/// `desktop/src-tauri/src` must match the site-granular inventory above. A +/// future ninth submission path — in a NEW file or an ALREADY-LISTED one — +/// fails this test until its guard is wired, its injection test exists, and +/// its inventory row is updated. +#[test] +fn events_url_inventory_is_fully_guarded() { + let violations = events_inventory_violations(&read_src_files()); + assert!( + violations.is_empty(), + "events-URL egress inventory drift — wire crate::egress_guard, add an \ + injection test, then update EVENTS_INVENTORY:\n{}", + violations.join("\n") + ); +} + +/// Mutation-style proof of the tripwire's guarantee: an unguarded ninth +/// `/events` site added to an already-inventoried file (relay.rs) is caught. +#[test] +fn inventory_scan_catches_new_site_in_allowlisted_file() { + let mut files = read_src_files(); + let relay = files + .iter_mut() + .find(|(rel, _)| rel.ends_with("src/relay.rs")) + .expect("relay.rs must be in the scan set"); + relay.1.push_str(&format!( + "\nfn sneaky_ninth_site(base: &str) -> String {{ format!(\"{{base}}{}\") }}\n", + events_needle() + )); + let violations = events_inventory_violations(&files); + assert!( + violations.iter().any(|v| v.contains("src/relay.rs")), + "an unguarded ninth events-URL site in relay.rs must trip the scan: {violations:?}" + ); +} + +/// The pairing also fires in reverse: a guard call deleted while its egress +/// site remains is caught. +#[test] +fn inventory_scan_catches_removed_guard_call() { + let mut files = read_src_files(); + let relay = files + .iter_mut() + .find(|(rel, _)| rel.ends_with("src/relay.rs")) + .expect("relay.rs must be in the scan set"); + relay.1 = relay.1.replacen(&guard_needle(), "removed_guard", 1); + let violations = events_inventory_violations(&files); + assert!( + violations.iter().any(|v| v.contains("src/relay.rs")), + "a removed guard call in relay.rs must trip the scan: {violations:?}" + ); +} + +/// A brand-new file with an `/events` site (no inventory row) is caught. +#[test] +fn inventory_scan_catches_new_unlisted_file() { + let mut files = read_src_files(); + files.push(( + "src/brand_new_egress.rs".to_string(), + format!("let url = format!(\"{{}}{}\", base);", events_needle()), + )); + let violations = events_inventory_violations(&files); + assert!( + violations + .iter() + .any(|v| v.contains("src/brand_new_egress.rs")), + "{violations:?}" + ); +} + +/// Source allowlist: NIP-49 material handling is confined to the identity / +/// backup / import / guard files. Anything else touching ncryptsec or the +/// nip49 codec is structural drift. +#[test] +fn ncryptsec_handling_is_confined_to_allowlisted_files() { + let allowlist: &[&str] = &[ + "src/key_backup.rs", + "src/key_backup_tests.rs", + "src/egress_guard.rs", + "src/egress_guard_tests.rs", + "src/commands/identity.rs", + "src/commands/identity_key_backup_tests.rs", + "src/lib.rs", // module registration + invoke handler + // boundary wiring (guard call sites name the module, not the codec): + "src/relay.rs", + "src/relay/submit.rs", + "src/huddle/pipeline.rs", + "src/commands/team_snapshot.rs", + "src/commands/team_snapshot/tests.rs", + "src/commands/personas/snapshot/import.rs", + "src/native_websocket.rs", + ]; + + let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")); + let mut violations = Vec::new(); + for path in src_rust_files() { + let rel = path + .strip_prefix(root) + .unwrap() + .to_string_lossy() + .replace('\\', "/"); + if allowlist.iter().any(|a| rel.ends_with(a)) { + continue; + } + let content = std::fs::read_to_string(&path).unwrap(); + for needle in ["ncryptsec", "EncryptedSecretKey", "nip49"] { + if content.contains(needle) { + violations.push(format!("{rel}: contains {needle:?}")); + } + } + } + assert!( + violations.is_empty(), + "NIP-49 material outside allowlisted files:\n{}", + violations.join("\n") + ); +} diff --git a/desktop/src-tauri/src/event_sync.rs b/desktop/src-tauri/src/event_sync.rs index d9fe6acdb9..ee8e0d8b10 100644 --- a/desktop/src-tauri/src/event_sync.rs +++ b/desktop/src-tauri/src/event_sync.rs @@ -165,7 +165,7 @@ fn migrate_personas_in_dir_at( scoped_record.shared = existing .as_ref() .and_then(|row| nostr::Event::from_json(&row.raw_event).ok()) - .is_some_and(|event| buzz_core_pkg::kind::persona_event_is_shared(&event)); + .is_some_and(|event| buzz_core_pkg::kind::event_is_shared(&event)); let event = build_persona_event(&scoped_record) .map_err(|e| format!("failed to build event for '{}': {e}", record.display_name))? .custom_created_at(monotonic_created_at( diff --git a/desktop/src-tauri/src/huddle/agent_tts_routing.rs b/desktop/src-tauri/src/huddle/agent_tts_routing.rs new file mode 100644 index 0000000000..2ee3ec0d41 --- /dev/null +++ b/desktop/src-tauri/src/huddle/agent_tts_routing.rs @@ -0,0 +1,56 @@ +use super::HuddlePhase; + +#[derive(Debug, PartialEq, Eq)] +pub(super) enum AgentTtsRuntimeGate { + Disabled, + Inactive, + NeedsPipeline, + Ready, +} + +pub(super) fn classify_agent_tts_runtime( + enabled: bool, + phase: &HuddlePhase, + has_pipeline: bool, +) -> AgentTtsRuntimeGate { + if !enabled { + AgentTtsRuntimeGate::Disabled + } else if !matches!(phase, HuddlePhase::Connected | HuddlePhase::Active) { + AgentTtsRuntimeGate::Inactive + } else if has_pipeline { + AgentTtsRuntimeGate::Ready + } else { + AgentTtsRuntimeGate::NeedsPipeline + } +} + +/// Maximum text length accepted for TTS synthesis. +/// ~2000 chars is 1–2 minutes of speech. Longer messages are truncated. +pub(super) const MAX_TTS_TEXT_LEN: usize = 2000; + +pub(super) fn normalize_agent_tts_text(text: String) -> String { + if text.chars().count() > MAX_TTS_TEXT_LEN { + let mut truncated: String = text.chars().take(MAX_TTS_TEXT_LEN).collect(); + truncated.push_str("... message truncated."); + truncated + } else { + text + } +} + +pub(super) async fn enqueue_agent_tts_text( + route_id: u64, + text: String, + enqueue: F, +) -> Result<(), String> +where + F: FnOnce(u64, String) -> Result<(), String> + Send + 'static, +{ + tokio::task::spawn_blocking(move || enqueue(route_id, text)) + .await + .map_err(|error| format!("TTS enqueue task failed: {error}"))? +} + +#[cfg(test)] +#[path = "agent_tts_routing_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/huddle/agent_tts_routing_tests.rs b/desktop/src-tauri/src/huddle/agent_tts_routing_tests.rs new file mode 100644 index 0000000000..cb550d7005 --- /dev/null +++ b/desktop/src-tauri/src/huddle/agent_tts_routing_tests.rs @@ -0,0 +1,57 @@ +use super::{ + classify_agent_tts_runtime, enqueue_agent_tts_text, normalize_agent_tts_text, + AgentTtsRuntimeGate, MAX_TTS_TEXT_LEN, +}; +use crate::huddle::HuddlePhase; + +#[tokio::test] +async fn assistant_plain_text_routes_unchanged_into_voice_pipeline_boundary() { + let (sender, receiver) = std::sync::mpsc::channel(); + let text = "A newly submitted assistant reply.".to_string(); + let route_id = 42; + + enqueue_agent_tts_text(route_id, text.clone(), move |route_id, queued| { + sender + .send((route_id, queued)) + .map_err(|error| error.to_string()) + }) + .await + .expect("route assistant text"); + + assert_eq!( + receiver.recv().expect("queued text"), + (route_id, text), + "route correlation must survive the native queue boundary" + ); +} + +#[test] +fn disabled_is_the_only_intentional_runtime_no_op() { + assert_eq!( + classify_agent_tts_runtime(false, &HuddlePhase::Connected, false), + AgentTtsRuntimeGate::Disabled + ); + assert_eq!( + classify_agent_tts_runtime(true, &HuddlePhase::Idle, false), + AgentTtsRuntimeGate::Inactive + ); + assert_eq!( + classify_agent_tts_runtime(true, &HuddlePhase::Connected, false), + AgentTtsRuntimeGate::NeedsPipeline + ); + assert_eq!( + classify_agent_tts_runtime(true, &HuddlePhase::Connected, true), + AgentTtsRuntimeGate::Ready + ); +} + +#[test] +fn assistant_text_truncation_is_unicode_safe_before_voice_routing() { + let input = "🦀".repeat(MAX_TTS_TEXT_LEN + 1); + let output = normalize_agent_tts_text(input); + assert_eq!( + output.chars().count(), + MAX_TTS_TEXT_LEN + "... message truncated.".chars().count() + ); + assert!(output.ends_with("... message truncated.")); +} diff --git a/desktop/src-tauri/src/huddle/agents.rs b/desktop/src-tauri/src/huddle/agents.rs index 02c4045410..2de22f99d8 100644 --- a/desktop/src-tauri/src/huddle/agents.rs +++ b/desktop/src-tauri/src/huddle/agents.rs @@ -2,7 +2,8 @@ //! //! Mental model: //! add_agent_to_huddle → kind:9000 to ephemeral channel -//! → kind:9000 to parent channel (best-effort) +//! → preserve existing parent membership, or +//! kind:9000 to parent channel (best-effort) //! //! ACP spawning is NOT needed here: the running agent process auto-subscribes //! when it receives the kind:9000 membership notification. Huddle-specific @@ -11,7 +12,10 @@ use serde::Serialize; use uuid::Uuid; -use crate::{app_state::AppState, events, relay::submit_event}; +use crate::{ + app_state::AppState, events, huddle::relay_api::fetch_channel_members_with_roles, + relay::submit_event, +}; // ── Constants ───────────────────────────────────────────────────────────────── @@ -61,8 +65,9 @@ with the next one. /// The field exists for forward compatibility with future batch-add operations /// where partial success may be meaningful. /// -/// `parent_added` reflects whether the parent-channel add succeeded; -/// `parent_error` carries the error string when it didn't. +/// `parent_added` reflects whether the parent already contained the agent or +/// the parent-channel add succeeded; `parent_error` carries the error string +/// when neither condition could be confirmed. #[derive(Debug, Serialize)] pub struct AgentAddResult { /// Always `true` — invariant guaranteed by [`add_agent_to_huddle`]. @@ -91,17 +96,33 @@ pub async fn add_agent_to_huddle( let add_eph = events::build_add_member(ephemeral_channel_id, agent_pubkey, Some("bot"))?; submit_event(add_eph, state).await?; - // 2. Add agent to parent channel — so agent has full context. - // Best-effort: capture the error but don't propagate it. - let (parent_added, parent_error) = { + // 2. Preserve any active parent membership, regardless of role. Rewriting + // an existing DM member as `bot` is both unnecessary and forbidden for + // non-admins. Otherwise add the agent so it has full context. + // Best-effort: capture a real error but don't propagate it. + let parent_channel_id_string = parent_channel_id.to_string(); + let parent_already_contains_agent = + fetch_channel_members_with_roles(&parent_channel_id_string, state) + .await + .is_ok_and(|members| contains_member(&members, agent_pubkey)); + + let (parent_added, parent_error) = if parent_already_contains_agent { + (true, None) + } else { let add_parent = events::build_add_member(parent_channel_id, agent_pubkey, Some("bot"))?; match submit_event(add_parent, state).await { Ok(_) => (true, None), Err(e) => { - eprintln!( - "buzz-desktop: add agent to parent channel failed (may already be member): {e}" - ); - (false, Some(e)) + let active_after_error = + fetch_channel_members_with_roles(&parent_channel_id_string, state) + .await + .is_ok_and(|members| contains_member(&members, agent_pubkey)); + if active_after_error { + (true, None) + } else { + eprintln!("buzz-desktop: add agent to parent channel failed: {e}"); + (false, Some(e)) + } } } }; @@ -112,3 +133,26 @@ pub async fn add_agent_to_huddle( parent_error, }) } + +fn contains_member(members: &[(String, Option)], pubkey: &str) -> bool { + members + .iter() + .any(|(member_pubkey, _)| member_pubkey.eq_ignore_ascii_case(pubkey)) +} + +#[cfg(test)] +mod tests { + use super::contains_member; + + #[test] + fn existing_parent_membership_is_preserved_regardless_of_role() { + let members = vec![ + ("agent-member".to_owned(), Some("member".to_owned())), + ("agent-bot".to_owned(), Some("bot".to_owned())), + ]; + + assert!(contains_member(&members, "AGENT-MEMBER")); + assert!(contains_member(&members, "agent-bot")); + assert!(!contains_member(&members, "missing")); + } +} diff --git a/desktop/src-tauri/src/huddle/audio_output.rs b/desktop/src-tauri/src/huddle/audio_output.rs index dbd09353db..34dec53094 100644 --- a/desktop/src-tauri/src/huddle/audio_output.rs +++ b/desktop/src-tauri/src/huddle/audio_output.rs @@ -39,7 +39,8 @@ fn list_audio_output_devices_blocking() -> Result, String #[tauri::command] pub fn set_audio_output_device(name: String, state: State<'_, AppState>) -> Result<(), String> { let mut guard = state - .audio_output_device + .huddle_audio + .output_device .lock() .map_err(|e| e.to_string())?; *guard = if name.is_empty() { None } else { Some(name) }; @@ -50,7 +51,8 @@ pub fn set_audio_output_device(name: String, state: State<'_, AppState>) -> Resu #[tauri::command] pub fn get_audio_output_device(state: State<'_, AppState>) -> Result { let guard = state - .audio_output_device + .huddle_audio + .output_device .lock() .map_err(|e| e.to_string())?; Ok(guard.clone().unwrap_or_default()) diff --git a/desktop/src-tauri/src/huddle/mod.rs b/desktop/src-tauri/src/huddle/mod.rs index a815bf2d06..03264f80f4 100644 --- a/desktop/src-tauri/src/huddle/mod.rs +++ b/desktop/src-tauri/src/huddle/mod.rs @@ -23,6 +23,7 @@ //! takes `stt_pipeline`/`tts_pipeline` out of the lock, then calls `shutdown()` //! and drops them outside the lock (thread joins can block ~200ms). +mod agent_tts_routing; pub mod agents; pub mod audio_output; pub mod jitter; @@ -37,6 +38,9 @@ pub mod state; pub mod stt; pub mod transcription; pub mod tts; +pub mod tts_settings; +mod tts_voice_import; +mod tts_voice_registry; pub mod wire; // ── Shared utilities ────────────────────────────────────────────────────────── @@ -63,16 +67,25 @@ pub(super) fn drain_until_shutdown( pub use state::{HuddleJoinInfo, HuddlePhase, HuddleState, VoiceInputMode}; pub use transcription::{set_huddle_transcription_enabled, start_stt_pipeline}; +pub use tts_settings::set_tts_enabled; // ── Imports ─────────────────────────────────────────────────────────────────── -use std::sync::{atomic::Ordering, Arc}; +use std::sync::atomic::Ordering; use tauri::State; use uuid::Uuid; use crate::{app_state::AppState, events, relay::submit_event}; -use pipeline::{maybe_start_stt_pipeline, maybe_start_tts_pipeline, post_connect_setup}; +use agent_tts_routing::{ + classify_agent_tts_runtime, enqueue_agent_tts_text, normalize_agent_tts_text, + AgentTtsRuntimeGate, +}; +pub use pipeline::check_pipeline_hotstart; +use pipeline::{ + await_inflight_tts_start, maybe_start_stt_pipeline, maybe_start_tts_pipeline, + post_connect_setup, start_auto_enabled_transcription, PostConnectOutcome, +}; use relay_api::{ count_human_members, fetch_channel_members, parse_channel_uuid, validate_pubkey_hex, MAX_HUDDLE_AGENTS, @@ -186,7 +199,7 @@ pub async fn start_huddle( }; // Transition to Creating. - { + let huddle_generation = { let mut hs = state.huddle()?; if hs.phase != HuddlePhase::Idle { return Err(format!( @@ -194,9 +207,11 @@ pub async fn start_huddle( hs.phase )); } + let generation = hs.begin_huddle_lifetime(); hs.phase = HuddlePhase::Creating; hs.parent_channel_id = Some(parent_channel_id.clone()); - } + generation + }; let ephemeral_uuid = Uuid::new_v4(); let ephemeral_channel_id = ephemeral_uuid.to_string(); @@ -259,27 +274,33 @@ pub async fn start_huddle( match result { Ok(successful_agents) => { // 5. Store active state. - { + let committed = { let mut hs = state.huddle()?; - hs.phase = HuddlePhase::Connected; - hs.is_creator = true; - hs.ephemeral_channel_id = Some(ephemeral_channel_id.clone()); - // Only store agents that were successfully enrolled. - *hs.agent_pubkeys.lock().unwrap_or_else(|e| e.into_inner()) = - successful_agents.clone(); - // Include the current user + successfully enrolled agents as participants. - // Use successful_agents (not member_pubkeys) so failed enrollments - // are not reflected in the participant list. - let own_pubkey = state - .keys - .lock() - .map(|k| k.public_key().to_hex()) - .unwrap_or_default(); - let mut participants = successful_agents.clone(); - if !own_pubkey.is_empty() && !participants.contains(&own_pubkey) { - participants.insert(0, own_pubkey); + if !hs.owns_huddle_lifetime(huddle_generation, HuddlePhase::Creating) { + false + } else { + hs.phase = HuddlePhase::Connected; + hs.is_creator = true; + hs.ephemeral_channel_id = Some(ephemeral_channel_id.clone()); + *hs.agent_pubkeys.lock().unwrap_or_else(|e| e.into_inner()) = + successful_agents.clone(); + hs.maybe_auto_enable_transcription_for_agents(); + let own_pubkey = state + .keys + .lock() + .map(|k| k.public_key().to_hex()) + .unwrap_or_default(); + let mut participants = successful_agents.clone(); + if !own_pubkey.is_empty() && !participants.contains(&own_pubkey) { + participants.insert(0, own_pubkey); + } + hs.participants = participants; + true } - hs.participants = participants; + }; + if !committed { + emit_end_and_archive(&parent_channel_id, &ephemeral_channel_id, &state).await; + return Err("huddle start was superseded".to_owned()); } // 6. Notify frontend of state change. @@ -287,16 +308,30 @@ pub async fn start_huddle( // 7. Hydrate members, download models, start pipelines (incl. audio relay). // Audio relay failure is fatal — no point in a huddle without audio. - if let Err(e) = post_connect_setup(&state, &ephemeral_channel_id).await { - // Rollback: audio relay failed after state was committed. - // Publish the terminal lifecycle event before archiving so - // other clients do not reconstruct a phantom active huddle. - emit_end_and_archive(&parent_channel_id, &ephemeral_channel_id, &state).await; - if let Ok(mut hs) = state.huddle_state.lock() { - hs.reset_preserving_generation(); + match post_connect_setup(&state, &ephemeral_channel_id, huddle_generation).await { + Ok(PostConnectOutcome::Ready) => {} + Ok(PostConnectOutcome::Stale) => { + return Err("huddle start was superseded".to_owned()); + } + Err(e) => { + // Roll back only if this failed setup still owns the active + // huddle. A stale failure must not tear down its replacement. + let still_current = state + .huddle() + .map(|hs| hs.is_current_huddle(&ephemeral_channel_id, huddle_generation)) + .unwrap_or(false); + if still_current { + emit_end_and_archive(&parent_channel_id, &ephemeral_channel_id, &state) + .await; + if let Ok(mut hs) = state.huddle_state.lock() { + if hs.is_current_huddle(&ephemeral_channel_id, huddle_generation) { + hs.reset_preserving_generation(); + } + } + state.emit_huddle_state_changed(); + } + return Err(e); } - state.emit_huddle_state_changed(); - return Err(e); } Ok(HuddleJoinInfo { @@ -314,11 +349,11 @@ pub async fn start_huddle( } } } - // Reset state to Idle so the user can retry. - // Preserve session_generation so in-flight transcription tasks - // from a prior session still see a stale generation and exit. + // Reset only if this failed attempt still owns the Creating state. if let Ok(mut hs) = state.huddle_state.lock() { - hs.reset_preserving_generation(); + if hs.owns_huddle_lifetime(huddle_generation, HuddlePhase::Creating) { + hs.reset_preserving_generation(); + } } Err(e) } @@ -340,7 +375,7 @@ pub async fn join_huddle( state: State<'_, AppState>, ) -> Result { // Transition to Connecting. - { + let huddle_generation = { let mut hs = state.huddle()?; if hs.phase != HuddlePhase::Idle { return Err(format!( @@ -348,10 +383,12 @@ pub async fn join_huddle( hs.phase )); } + let generation = hs.begin_huddle_lifetime(); hs.phase = HuddlePhase::Connecting; hs.parent_channel_id = Some(parent_channel_id.clone()); hs.ephemeral_channel_id = Some(ephemeral_channel_id.clone()); - } + generation + }; // Seed participant list with own pubkey as a fallback until relay responds. let own_pubkey = state @@ -360,12 +397,20 @@ pub async fn join_huddle( .map(|k| k.public_key().to_hex()) .unwrap_or_default(); - { + let committed = { let mut hs = state.huddle()?; - hs.phase = HuddlePhase::Connected; - if !own_pubkey.is_empty() { - hs.participants = vec![own_pubkey]; + if !hs.owns_huddle_lifetime(huddle_generation, HuddlePhase::Connecting) { + false + } else { + hs.phase = HuddlePhase::Connected; + if !own_pubkey.is_empty() { + hs.participants = vec![own_pubkey]; + } + true } + }; + if !committed { + return Err("huddle join was superseded".to_owned()); } // Notify frontend of state change. @@ -373,15 +418,25 @@ pub async fn join_huddle( // Hydrate members, download models, start pipelines (incl. audio relay). // Audio relay failure is fatal — no point in a huddle without audio. - if let Err(e) = post_connect_setup(&state, &ephemeral_channel_id).await { - // Rollback: audio relay failed after state was committed. - // Reset state to Idle so the user can retry. The ephemeral channel - // has a TTL and will expire — no manual archive needed for joiners. - if let Ok(mut hs) = state.huddle_state.lock() { - hs.reset_preserving_generation(); + match post_connect_setup(&state, &ephemeral_channel_id, huddle_generation).await { + Ok(PostConnectOutcome::Ready) => {} + Ok(PostConnectOutcome::Stale) => { + return Err("huddle join was superseded".to_owned()); + } + Err(e) => { + // Reset only the huddle lifetime that failed. + let mut did_reset = false; + if let Ok(mut hs) = state.huddle_state.lock() { + if hs.is_current_huddle(&ephemeral_channel_id, huddle_generation) { + hs.reset_preserving_generation(); + did_reset = true; + } + } + if did_reset { + state.emit_huddle_state_changed(); + } + return Err(e); } - state.emit_huddle_state_changed(); - return Err(e); } Ok(HuddleJoinInfo { @@ -675,123 +730,6 @@ pub fn push_audio_pcm( } } -/// Hot-start: check if voice models just finished downloading during an active -/// huddle and start the corresponding pipelines. -/// -/// Called by the frontend on a timer or after model status changes. No-op if -/// the huddle is not active or pipelines are already running. -#[tauri::command] -pub async fn check_pipeline_hotstart(state: State<'_, AppState>) -> Result<(), String> { - let (is_active, ephemeral_channel_id) = { - let hs = state.huddle()?; - ( - matches!(hs.phase, HuddlePhase::Connected | HuddlePhase::Active), - hs.ephemeral_channel_id.clone(), - ) - }; - - if !is_active { - return Ok(()); - } - - // Detect dead pipelines: if the worker thread has exited (init failure or crash), - // clear the pipeline handle so hot-start can retry on the next cycle. - { - let mut hs = state.huddle()?; - if let Some(ref p) = hs.stt_pipeline { - if p.is_finished() { - hs.stt_pipeline = None; - } - } - if let Some(ref p) = hs.tts_pipeline { - if p.is_finished() { - hs.tts_pipeline = None; - } - } - } - // Re-read after potential cleanup. - let (has_stt, has_tts, transcription_enabled) = { - let hs = state.huddle()?; - ( - hs.stt_pipeline.is_some(), - hs.tts_pipeline.is_some(), - hs.transcription_enabled, - ) - }; - - // Check if models just became ready (one-shot flags). - let stt_ready = models::global_model_manager() - .map(|m| m.take_stt_ready()) - .unwrap_or(false); - let tts_ready = models::global_model_manager() - .map(|m| m.take_tts_ready()) - .unwrap_or(false); - - // Start TTS first (so STT can capture tts_cancel). - if !has_tts && (tts_ready || models::is_tts_ready()) { - if let Err(e) = maybe_start_tts_pipeline(&state).await { - eprintln!("buzz-desktop: TTS hotstart failed: {e}"); - } - } - - if transcription_enabled && !has_stt && (stt_ready || models::is_stt_ready()) { - if let Some(eph_id) = &ephemeral_channel_id { - if let Err(e) = maybe_start_stt_pipeline(&state, eph_id).await { - eprintln!("buzz-desktop: STT hotstart failed: {e}"); - } - } - } - - // Periodically refresh agent_pubkeys from relay membership. - // This catches mid-huddle agent additions/removals by other participants, - // keeping STT p-tags authoritative throughout the session. - // Throttled to every 15 s (not on every 5 s hotstart poll). - // - // NOTE: The frontend ALSO polls agent membership independently (every 10 s - // via get_huddle_agent_pubkeys). This is intentional — the two polls have - // different failure semantics: - // - Rust (here): preserves stale list on failure (STT p-tags should not - // disappear on a transient network blip). - // - React (HuddleContext.tsx): clears list on failure (TTS authorization - // must fail-closed — never speak from a stale agent list). - // - // On Ok: always replace (even with empty — agents may have been removed). - // On Err: preserve the existing list (transient failure shouldn't zero it). - if let Some(eph_id) = &ephemeral_channel_id { - let should_refresh = { - let hs = state.huddle()?; - match hs.last_agent_refresh { - None => true, - Some(t) => t.elapsed() >= std::time::Duration::from_secs(15), - } - }; - if should_refresh { - // Fetch agents (for STT p-tags) and all members (for participant list). - // Sequential — tokio::join! requires the `macros` feature. - // Only update the throttle timestamp when at least one fetch succeeds, - // so transient failures retry immediately on the next poll cycle. - // Fetch both lists before acquiring the lock — no lock held across await. - let fresh_agents = fetch_channel_members(eph_id, Some("bot"), &state) - .await - .ok(); - let fresh_members = fetch_channel_members(eph_id, None, &state).await.ok(); - - if fresh_agents.is_some() || fresh_members.is_some() { - let mut hs = state.huddle()?; - if let Some(agents) = fresh_agents { - *hs.agent_pubkeys.lock().unwrap_or_else(|e| e.into_inner()) = agents; - } - if let Some(members) = fresh_members { - hs.participants = members; - } - hs.last_agent_refresh = Some(std::time::Instant::now()); - } - } - } - - Ok(()) -} - /// Trigger a background download of voice models (Parakeet STT + Pocket TTS). /// /// Returns immediately — downloads run in tokio background tasks. @@ -817,91 +755,90 @@ pub fn get_model_status(_state: State<'_, AppState>) -> Result) -> Result<(), String> { - let old_pipeline = { - let mut hs = state.huddle()?; - hs.tts_enabled = enabled; - if !enabled { - hs.tts_pipeline.take() // Take out of lock. - } else { - None - } - }; - // Shut down outside the lock — thread join happens here. - if let Some(ref pipeline) = old_pipeline { - pipeline.shutdown(); - } - drop(old_pipeline); - - if enabled { - // Re-start TTS pipeline if models are available and huddle is active. - let phase = { - let hs = state.huddle()?; - hs.phase.clone() - }; - if matches!(phase, HuddlePhase::Connected | HuddlePhase::Active) { - if let Err(e) = maybe_start_tts_pipeline(&state).await { - eprintln!("buzz-desktop: TTS pipeline restart failed: {e}"); - } - } - } - - Ok(()) -} - /// Speak an agent message via TTS. /// -/// Maximum text length accepted for TTS synthesis. -/// ~2000 chars ≈ 1–2 minutes of speech. Longer messages are truncated. -const MAX_TTS_TEXT_LEN: usize = 2000; - -/// Called by the WebView when it receives an incoming agent kind:9 message. +/// Called by the WebView when it receives an eligible live agent message. /// Lazily starts the TTS pipeline if models are ready but the pipeline hasn't /// been created yet (e.g. models finished downloading after huddle started). /// -/// No-op if TTS is disabled or models aren't ready. +/// Disabled is the only intentional no-op. Enabled-but-unavailable speech +/// returns an error so the caller cannot mistake a dropped message for success. #[tauri::command] -pub async fn speak_agent_message(text: String, state: State<'_, AppState>) -> Result<(), String> { +pub async fn speak_agent_message( + text: String, + route_id: u64, + state: State<'_, AppState>, +) -> Result<(), String> { + eprintln!("buzz-desktop: tts stage=invoke status=started route_id={route_id}"); // Truncate oversized messages — agents shouldn't monologue in a voice huddle. // Use char count (not byte length) to avoid panicking on multi-byte UTF-8. - let text = if text.chars().count() > MAX_TTS_TEXT_LEN { - let mut truncated: String = text.chars().take(MAX_TTS_TEXT_LEN).collect(); - truncated.push_str("... message truncated."); - truncated - } else { - text - }; + let text = normalize_agent_tts_text(text); let needs_pipeline = { - let hs = state.huddle()?; - hs.tts_enabled - && hs.tts_pipeline.is_none() - && matches!(hs.phase, HuddlePhase::Connected | HuddlePhase::Active) + let mut hs = state.huddle()?; + if hs + .tts_pipeline + .as_ref() + .is_some_and(|pipeline| pipeline.is_finished()) + { + hs.tts_pipeline = None; + } + match classify_agent_tts_runtime(hs.tts_enabled, &hs.phase, hs.tts_pipeline.is_some()) { + AgentTtsRuntimeGate::Disabled => { + eprintln!( + "buzz-desktop: tts stage=invoke status=no_op reason=disabled route_id={route_id}" + ); + return Ok(()); + } + AgentTtsRuntimeGate::Inactive => { + eprintln!( + "buzz-desktop: tts stage=invoke status=failed reason=inactive_huddle route_id={route_id}" + ); + return Err( + "Agent text to speech is unavailable outside an active huddle".to_string(), + ); + } + AgentTtsRuntimeGate::NeedsPipeline => true, + AgentTtsRuntimeGate::Ready => false, + } }; // Lazy-start: models may have finished downloading after the huddle began. if needs_pipeline { - if let Err(e) = maybe_start_tts_pipeline(&state).await { - eprintln!("buzz-desktop: TTS lazy-start failed: {e}"); - } + maybe_start_tts_pipeline(&state).await.inspect_err(|_| { + eprintln!( + "buzz-desktop: tts stage=invoke status=failed reason=startup_failed route_id={route_id}" + ); + })?; + await_inflight_tts_start(&state).await.inspect_err(|_| { + eprintln!( + "buzz-desktop: tts stage=invoke status=failed reason=startup_timeout route_id={route_id}" + ); + })?; } - let hs = state.huddle()?; - if hs.tts_enabled { - if let Some(ref pipeline) = hs.tts_pipeline { - pipeline.speak(text)?; - } - } - Ok(()) + let sender = { + let hs = state.huddle()?; + hs.tts_pipeline + .as_ref() + .map(|pipeline| pipeline.text_sender()) + }; + let Some(sender) = sender else { + eprintln!( + "buzz-desktop: tts stage=invoke status=failed reason=unavailable route_id={route_id}" + ); + return Err("Agent text to speech is enabled but its audio pipeline is unavailable".into()); + }; + enqueue_agent_tts_text(route_id, text, move |route_id, text| { + sender + .send(route_id, text) + .map_err(|error| format!("TTS queue closed while waiting to enqueue: {error}")) + }) + .await + .inspect(|_| eprintln!("buzz-desktop: tts stage=queue status=accepted route_id={route_id}")) + .inspect_err(|_| { + eprintln!("buzz-desktop: tts stage=queue status=failed reason=closed route_id={route_id}") + }) } /// Add an agent to the active huddle. @@ -924,7 +861,7 @@ pub async fn add_agent_to_huddle( ) -> Result { validate_pubkey_hex(&agent_pubkey)?; - let (eph_id, parent_id) = { + let (eph_id, parent_id, huddle_generation) = { let hs = state.huddle()?; if !matches!(hs.phase, HuddlePhase::Connected | HuddlePhase::Active) { return Err("no active huddle".to_string()); @@ -948,7 +885,7 @@ pub async fn add_agent_to_huddle( .clone() .ok_or("no ephemeral channel")?; let parent = hs.parent_channel_id.clone().ok_or("no parent channel")?; - (eph, parent) + (eph, parent, hs.huddle_generation) }; let eph_uuid = Uuid::parse_str(&eph_id).map_err(|e| e.to_string())?; @@ -957,29 +894,30 @@ pub async fn add_agent_to_huddle( // Returns Err only if the ephemeral add fails — parent failure is in the result. let result = agents::add_agent_to_huddle(eph_uuid, parent_uuid, &agent_pubkey, &state).await?; - // Ephemeral add succeeded — safe to register for p-tagging. - // Clone the Arc first so we can drop the outer HuddleState lock before - // acquiring the inner pubkeys lock (avoids the E0597 borrow-checker error). - { - let agent_pubkeys_arc = { - let hs = state.huddle()?; - Arc::clone(&hs.agent_pubkeys) - }; - let mut pubkeys = agent_pubkeys_arc.lock().unwrap_or_else(|e| e.into_inner()); + // Ephemeral add succeeded — register it only if this is still the huddle + // that initiated the relay operation. + let transcription_auto_enabled = { + let mut hs = state.huddle()?; + if !hs.is_current_huddle(&eph_id, huddle_generation) { + return Ok(result); + } + let mut pubkeys = hs.agent_pubkeys.lock().unwrap_or_else(|e| e.into_inner()); if !pubkeys.contains(&agent_pubkey) { pubkeys.push(agent_pubkey.clone()); } - } + drop(pubkeys); + if !hs.participants.contains(&agent_pubkey) { + hs.participants.push(agent_pubkey.clone()); + } + hs.maybe_auto_enable_transcription_for_agents() + }; // No guidelines re-post needed — the agent sees the original kind:48106 // guidelines via EOSE replay when it subscribes to the ephemeral channel. - - // Also add the agent to the visible participants list. - { - let mut hs = state.huddle()?; - if !hs.participants.contains(&agent_pubkey) { - hs.participants.push(agent_pubkey); - } + if transcription_auto_enabled { + start_auto_enabled_transcription(&state, &eph_id).await; + } else { + state.emit_huddle_state_changed(); } Ok(result) diff --git a/desktop/src-tauri/src/huddle/models.rs b/desktop/src-tauri/src/huddle/models.rs index 169ddf66c0..f9f7065769 100644 --- a/desktop/src-tauri/src/huddle/models.rs +++ b/desktop/src-tauri/src/huddle/models.rs @@ -24,6 +24,14 @@ use std::sync::{Arc, Mutex, OnceLock}; use serde::{Deserialize, Serialize}; use sha2::{Digest, Sha256}; +use super::pocket::{ + april_model_info, PocketModelArtifact, APRIL_BUNDLE_ID, APRIL_MODEL_ID, APRIL_MODEL_REVISION, +}; +use super::tts_voice_registry::POCKET_VOICES; + +#[path = "models_voice_upgrade.rs"] +mod voice_upgrade; + // ── Integrity verification ──────────────────────────────────────────────────── // // All model artifacts are verified against pinned SHA-256 hashes before @@ -38,19 +46,15 @@ use sha2::{Digest, Sha256}; /// Computed from a known-good download. Update when upgrading model versions. const STT_ARCHIVE_SHA256: &str = "17f945007b52ccd8b7200ffc7c5652e9e8e961dfdf479cefcabd06cf5703630b"; -/// HuggingFace base URL for the sherpa-onnx Pocket TTS fp32 repackage. -/// -/// Pinned to commit 96d1e53ce3311ca6c2c6a35e2062d36b4cec6fa3 -/// (2026-02-10) for reproducible downloads. -/// -/// fp32 (not int8): a direct same-runtime A/B (k2-fsa/sherpa-onnx#3172) -/// found the ONNX int8 quantization audibly degraded Pocket TTS output and -/// that fp32 "significantly improved quality even at 1 step". The runtime -/// bundle grows from ~189 MB to ~473 MB; encoder, text conditioner, both -/// JSON tables, and LICENSE are byte-identical between the two repos — only -/// the three quantized sessions (lm_main, lm_flow, decoder) change. -const POCKET_HF_BASE: &str = - "https://huggingface.co/csukuangfj2/sherpa-onnx-pocket-tts-2026-01-26/resolve/96d1e53ce3311ca6c2c6a35e2062d36b4cec6fa3"; +fn pocket_artifact_url(filename: &str) -> String { + format!( + "https://huggingface.co/{APRIL_MODEL_ID}/resolve/{APRIL_MODEL_REVISION}/onnx/{APRIL_BUNDLE_ID}/{filename}" + ) +} + +fn pocket_license_url() -> String { + format!("https://huggingface.co/{APRIL_MODEL_ID}/resolve/{APRIL_MODEL_REVISION}/onnx/LICENSE") +} /// Reference voice WAV: "Mary (f, conversation)" from the Kyutai TTS demo /// voice set — VCTK speaker p333, ai-coustics-enhanced. Pinned to @@ -64,20 +68,19 @@ const POCKET_HF_BASE: &str = const POCKET_REFERENCE_WAV_URL: &str = "https://huggingface.co/kyutai/tts-voices/resolve/323332d33f997de8394f24a193e1a76df720e01a/vctk/p333_023_enhanced.wav"; -/// SHA-256 hashes for individual Pocket TTS model files. -/// Computed from known-good pinned downloads. Update when upgrading model versions. -#[rustfmt::skip] -const TTS_FILE_HASHES: &[(&str, &str)] = &[ - ("decoder.onnx", "f267880fde6c58b17b0a8f3647eaf8dcfad321f833f32d583ebc2fb2d1a15f10"), - ("encoder.onnx", "e8f2f6d301ffb96e398b138a7dc6d3038622d236044636b73d920bab85890260"), - ("lm_flow.onnx", "79c013a554a54e63319c33c0cc8830cbbedc9b7e448ae7e26f7923ae11f9873e"), - ("lm_main.onnx", "255d1a9263c5abdf36034abfc19c11d21cc5f40f0f87d8361288e972cbd5c578"), - ("text_conditioner.onnx", "0b84e837d7bfaf2c896627b03e3f080320309f37f4fc7df7698c644f7ba5e6b1"), - ("vocab.json", "6fb646346cf931016f70c4921aab0900ce7a304b893cb02135c74e294abfea01"), - ("token_scores.json", "5be2f278caf9b9800741f0fd82bff677f4943ec764c356f907213434b622d958"), - ("LICENSE", "fe7b4ce83b8381cc5b216bbb4af73c570688d1b819c73bbaed8ca401f4677cd6"), - ("reference_sample.wav", "a35b0468382218e9f37a9a7494d1e4b74deaf18d7ced22265b4e325bb55c183f"), -]; +const TTS_LICENSE_ARTIFACT: PocketModelArtifact = PocketModelArtifact { + filename: "LICENSE", + sha256: "fe7b4ce83b8381cc5b216bbb4af73c570688d1b819c73bbaed8ca401f4677cd6", + size_bytes: 18_655, + quantized: false, +}; + +const TTS_REFERENCE_ARTIFACT: PocketModelArtifact = PocketModelArtifact { + filename: "reference_sample.wav", + sha256: "a35b0468382218e9f37a9a7494d1e4b74deaf18d7ced22265b4e325bb55c183f", + size_bytes: 639_084, + quantized: false, +}; // ── Model versioning ────────────────────────────────────────────────────────── // @@ -92,15 +95,8 @@ const TTS_FILE_HASHES: &[(&str, &str)] = &[ /// honest (each version tag identifies one specific set of model bytes). const STT_MODEL_VERSION: &str = "2"; -/// Model manifest version for Pocket TTS. Increment when upgrading model files. -/// Bumped "1" → "2" when the bundled reference voice changed from KevinAHM's -/// anonymous 16 kHz sample to Mary (VCTK p333, 32 kHz, ai-coustics-enhanced) -/// from kyutai/tts-voices. The hash mismatch on `reference_sample.wav` would -/// fail readiness on its own, but the manifest bump makes the re-download -/// reason explicit and skips the failing-then-re-fetching transient state. -/// Bumped "2" → "3" for the int8 → fp32 model swap (see `POCKET_HF_BASE`): -/// existing int8 installs must re-download the suffixless fp32 sessions. -const TTS_MODEL_VERSION: &str = "3"; +/// Identifies the April INT8 asset set plus the official VCTK presets. +const TTS_MODEL_VERSION: &str = "5"; /// Filename for the version manifest written alongside model files. const MANIFEST_FILENAME: &str = ".buzz-model-manifest"; @@ -110,9 +106,9 @@ const MANIFEST_FILENAME: &str = ".buzz-model-manifest"; /// Maximum expected STT archive size (200 MB — actual is ~100 MB). const MAX_STT_DOWNLOAD_BYTES: u64 = 200 * 1024 * 1024; -/// Maximum expected Pocket TTS file size (400 MB per file — largest is -/// `lm_main.onnx` at ~303 MB fp32). -const MAX_TTS_FILE_BYTES: u64 = 400 * 1024 * 1024; +/// Maximum expected Pocket TTS file size. The largest pinned INT8 artifact is +/// `flow_lm_main_int8.onnx` at 76,341,079 bytes. +const MAX_TTS_FILE_BYTES: u64 = 100 * 1024 * 1024; /// NVIDIA Parakeet TDT-CTC 110M (English, int8) — packaged for sherpa-onnx by /// k2-fsa. Single ONNX file (CTC head) + tokens.txt. Avg WER ~7.5% across @@ -168,50 +164,29 @@ const TTS_MODEL_DIR_NAME: &str = "pocket-tts"; /// Attribution sidecar written next to the Pocket TTS model files. const TTS_LICENSE_FILE_NAME: &str = "MODEL_LICENSE.txt"; -/// CC-BY-4.0 §3(a)(1) attribution block for Pocket TTS, its ONNX packaging, -/// and the bundled reference voice WAV. -const TTS_LICENSE_TEXT: &str = "\ -Pocket TTS -© Kyutai. - -Licensed under the Creative Commons Attribution 4.0 International License -(CC-BY-4.0). License text: https://creativecommons.org/licenses/by/4.0/ - -Original model by Kyutai: https://huggingface.co/kyutai/pocket-tts -Paper: Charles, Roebel, et al., Pocket TTS (arXiv:2509.06926). -Mimi neural codec by Kyutai is bundled as part of the model. - -ONNX export by KevinAHM: https://huggingface.co/KevinAHM/pocket-tts-onnx -Sherpa-onnx repackage by csukuangfj / k2-fsa: -https://huggingface.co/csukuangfj2/sherpa-onnx-pocket-tts-2026-01-26 - -Bundled reference voice (reference_sample.wav): -\"Mary (f, conversation)\" preset from the Kyutai TTS demo voice catalogue -(https://kyutai.org/tts), distributed via -https://huggingface.co/kyutai/tts-voices as `vctk/p333_023_enhanced.wav`. -Original recording from the Voice Cloning Toolkit (VCTK) corpus, speaker p333: -https://datashare.ed.ac.uk/handle/10283/3443 (CC-BY-4.0). -Recording enhancement (denoise/dereverb) by ai-coustics: -https://ai-coustics.com/ - -Buzz ships all ONNX/model artifacts and the reference voice WAV unmodified, -renamed only by placement in the local model directory. - -Provided \"AS IS\", without warranty of any kind, express or implied. See the -license text for full warranty disclaimer. -"; - /// All files that must be present for Pocket TTS to be considered ready. const TTS_EXPECTED_FILES: &[&str] = &[ - "decoder.onnx", - "encoder.onnx", - "lm_flow.onnx", - "lm_main.onnx", + "bundle.json", + "bos_before_voice.npy", + "flow_lm_main_int8.onnx", + "flow_lm_flow_int8.onnx", + "mimi_decoder_int8.onnx", + "mimi_encoder.onnx", "text_conditioner.onnx", - "vocab.json", - "token_scores.json", + "tokenizer.model", "LICENSE", "reference_sample.wav", + "anna.wav", + "vera.wav", + "fantine.wav", + "charles.wav", + "paul.wav", + "eponine.wav", + "azelma.wav", + "george.wav", + "jane.wav", + "michael.wav", + "eve.wav", TTS_LICENSE_FILE_NAME, ]; @@ -404,6 +379,7 @@ struct ModelSlot { dir_name: &'static str, // subdir under ~/.buzz/models/ expected_files: &'static [&'static str], // files required for "ready" version: &'static str, // manifest version; increment to force re-download + expected_size: fn(&str) -> Option, status: Arc>, just_ready: Arc, // fires once when download completes } @@ -418,11 +394,17 @@ impl ModelSlot { dir_name, expected_files, version, + expected_size: |_| None, status: Arc::new(Mutex::new(ModelStatus::NotDownloaded)), just_ready: Arc::new(AtomicBool::new(false)), } } + fn with_expected_sizes(mut self, expected_size: fn(&str) -> Option) -> Self { + self.expected_size = expected_size; + self + } + fn model_dir(&self, models_dir: &Path) -> PathBuf { models_dir.join(self.dir_name) } @@ -432,7 +414,17 @@ impl ModelSlot { std::fs::read_to_string(dir.join(MANIFEST_FILENAME)) .map(|v| v.trim() == self.version) .unwrap_or(false) - && self.expected_files.iter().all(|f| dir.join(f).is_file()) + && self.expected_files.iter().all(|filename| { + let path = dir.join(filename); + path.is_file() + && (self.expected_size)(filename) + .map(|expected| { + path.metadata() + .map(|metadata| metadata.len() == expected) + .unwrap_or(false) + }) + .unwrap_or(true) + }) } fn dir_if_ready(&self, models_dir: &Path) -> Option { @@ -453,6 +445,39 @@ impl ModelSlot { self.just_ready.swap(false, Ordering::AcqRel) } + /// Recover or clean up the backup left by an interrupted atomic install. + fn recover_interrupted_install(&self, models_dir: &Path) { + let final_dir = self.model_dir(models_dir); + let backup_dir = final_dir.with_extension("old"); + if !backup_dir.exists() { + return; + } + if self.is_ready(models_dir) { + if let Err(error) = std::fs::remove_dir_all(&backup_dir) { + eprintln!( + "buzz-desktop: could not remove stale {} backup: {error}", + self.dir_name + ); + } + return; + } + if final_dir.exists() { + if let Err(error) = std::fs::remove_dir_all(&final_dir) { + eprintln!( + "buzz-desktop: could not remove incomplete {} install: {error}", + self.dir_name + ); + return; + } + } + if let Err(error) = std::fs::rename(&backup_dir, &final_dir) { + eprintln!( + "buzz-desktop: could not restore interrupted {} install: {error}", + self.dir_name + ); + } + } + /// Spawn a background download task if not already ready or downloading. fn start_download( &self, @@ -511,6 +536,9 @@ impl ModelSlot { )); } + std::fs::write(source_dir.join(MANIFEST_FILENAME), self.version) + .map_err(|e| format!("write model manifest: {e}"))?; + let final_dir = self.model_dir(models_dir); let backup_dir = final_dir.with_extension("old"); @@ -529,8 +557,6 @@ impl ModelSlot { return Err(format!("install new model: {e}")); } - std::fs::write(final_dir.join(MANIFEST_FILENAME), self.version) - .map_err(|e| format!("write model manifest: {e}"))?; let _ = tokio::fs::remove_dir_all(&backup_dir).await; if let Some(extra) = temp_cleanup { let _ = tokio::fs::remove_dir_all(extra).await; @@ -542,6 +568,25 @@ impl ModelSlot { } } +fn tts_expected_size(filename: &str) -> Option { + april_model_info() + .artifacts + .iter() + .find(|artifact| artifact.filename == filename) + .map(|artifact| artifact.size_bytes) + .or_else(|| { + [TTS_LICENSE_ARTIFACT, TTS_REFERENCE_ARTIFACT] + .iter() + .find(|artifact| artifact.filename == filename) + .map(|artifact| artifact.size_bytes) + }) +} + +fn tts_model_slot() -> ModelSlot { + ModelSlot::new(TTS_MODEL_DIR_NAME, TTS_EXPECTED_FILES, TTS_MODEL_VERSION) + .with_expected_sizes(tts_expected_size) +} + // ── ModelManager ────────────────────────────────────────────────────────────── /// Manages download and location of STT/TTS model files. @@ -561,11 +606,13 @@ impl ModelManager { /// Returns `None` if the home directory cannot be resolved. pub fn new() -> Option { let models_dir = dirs::home_dir()?.join(".buzz").join("models"); - Some(Self { + let manager = Self { models_dir, stt: ModelSlot::new(STT_MODEL_DIR_NAME, STT_EXPECTED_FILES, STT_MODEL_VERSION), - tts: ModelSlot::new(TTS_MODEL_DIR_NAME, TTS_EXPECTED_FILES, TTS_MODEL_VERSION), - }) + tts: tts_model_slot(), + }; + manager.tts.recover_interrupted_install(&manager.models_dir); + Some(manager) } // ── STT accessors ──────────────────────────────────────────────────────── @@ -638,8 +685,11 @@ impl ModelManager { } } - /// Start a background Pocket TTS download (~189 MB). No-op if already ready or downloading. + /// Start a background Pocket TTS download. No-op if already ready or downloading. pub fn start_tts_download(&self, http_client: reqwest::Client) { + if let Err(error) = voice_upgrade::install_vctk_presets_into_v4_model(&self.models_dir) { + eprintln!("buzz-desktop: could not upgrade existing Pocket voices in place: {error}"); + } let manager = self.clone(); self.tts.start_download( &self.models_dir, @@ -754,10 +804,10 @@ impl ModelManager { /// Download and verify the Pocket TTS model files from HuggingFace. /// /// Downloads files into `~/.buzz/models/pocket-tts/`: - /// - five ONNX sessions (Pocket TTS + Mimi codec) - /// - `vocab.json` / `token_scores.json` for sherpa-onnx text conditioning + /// - five ONNX sessions selected by the April INT8 bundle + /// - bundle metadata, SentencePiece tokenizer, and learned voice BOS /// - upstream `LICENSE` plus Buzz's `MODEL_LICENSE.txt` attribution sidecar - /// - `reference_sample.wav` as the bundled default voice + /// - `reference_sample.wav` plus the embedded official VCTK presets /// /// Files are written to a temp directory first, then moved atomically. async fn download_tts_model(&self, http_client: reqwest::Client) -> Result<(), String> { @@ -768,24 +818,18 @@ impl ModelManager { let temp_dir = self.models_dir.join("pocket-tts.tmp"); fresh_temp_dir(&temp_dir).await?; - let model_files = [ - "decoder.onnx", - "encoder.onnx", - "lm_flow.onnx", - "lm_main.onnx", - "text_conditioner.onnx", - "vocab.json", - "token_scores.json", - "LICENSE", - ]; - let mut downloads: Vec<(String, &'static str)> = model_files + let mut downloads: Vec<(String, PocketModelArtifact)> = april_model_info() + .artifacts .iter() - .map(|filename| (format!("{POCKET_HF_BASE}/{filename}"), *filename)) + .copied() + .map(|artifact| (pocket_artifact_url(artifact.filename), artifact)) .collect(); - downloads.push((POCKET_REFERENCE_WAV_URL.to_string(), "reference_sample.wav")); + downloads.push((pocket_license_url(), TTS_LICENSE_ARTIFACT)); + downloads.push((POCKET_REFERENCE_WAV_URL.to_string(), TTS_REFERENCE_ARTIFACT)); let total_files = downloads.len() as u32; - for (i, (url, filename)) in downloads.iter().enumerate() { + for (i, (url, artifact)) in downloads.iter().enumerate() { + let filename = artifact.filename; eprintln!("buzz-desktop: downloading Pocket TTS {filename} from {url}"); let response = fetch_url(&http_client, url, filename) @@ -822,16 +866,19 @@ impl ModelManager { })?; eprintln!("buzz-desktop: downloaded {bytes} bytes ({filename}), wrote to disk"); - let expected = TTS_FILE_HASHES - .iter() - .find(|(n, _)| *n == *filename) - .map(|(_, hash)| *hash) - .ok_or_else(|| format!("missing expected hash for Pocket TTS file: {filename}"))?; + if bytes != artifact.size_bytes { + let _ = tokio::fs::remove_dir_all(&temp_dir).await; + return Err(format!( + "Pocket TTS {filename} size check failed: expected {} bytes, got {bytes}", + artifact.size_bytes + )); + } let actual = sha256_file(&dest).await?; - if actual != expected { + if actual != artifact.sha256 { let _ = tokio::fs::remove_dir_all(&temp_dir).await; return Err(format!( - "Pocket TTS {filename} integrity check failed: expected {expected}, got {actual}" + "Pocket TTS {filename} integrity check failed: expected {}, got {actual}", + artifact.sha256 )); } @@ -842,9 +889,20 @@ impl ModelManager { }); } - tokio::fs::write(temp_dir.join(TTS_LICENSE_FILE_NAME), TTS_LICENSE_TEXT) - .await - .map_err(|e| format!("write TTS model license sidecar: {e}"))?; + tokio::fs::write( + temp_dir.join(TTS_LICENSE_FILE_NAME), + voice_upgrade::TTS_LICENSE_TEXT, + ) + .await + .map_err(|e| format!("write TTS model license sidecar: {e}"))?; + for voice in POCKET_VOICES { + let Some(bytes) = voice.bytes else { + continue; + }; + tokio::fs::write(temp_dir.join(voice.reference_file), bytes) + .await + .map_err(|e| format!("install bundled {} voice: {e}", voice.display_name))?; + } self.tts.set_status(ModelStatus::Downloading { progress_percent: 90, @@ -931,24 +989,5 @@ pub fn is_tts_ready() -> bool { } #[cfg(test)] -mod tests { - use super::*; - - #[test] - fn tts_readiness_requires_license_sidecar() { - let temp = tempfile::tempdir().expect("tempdir"); - let slot = ModelSlot::new(TTS_MODEL_DIR_NAME, TTS_EXPECTED_FILES, TTS_MODEL_VERSION); - let model_dir = temp.path().join(TTS_MODEL_DIR_NAME); - std::fs::create_dir_all(&model_dir).expect("create model dir"); - - for file in TTS_EXPECTED_FILES { - std::fs::write(model_dir.join(file), b"test").expect("write expected file"); - } - std::fs::write(model_dir.join(MANIFEST_FILENAME), TTS_MODEL_VERSION).expect("manifest"); - - assert!(slot.is_ready(temp.path())); - - std::fs::remove_file(model_dir.join(TTS_LICENSE_FILE_NAME)).expect("remove sidecar"); - assert!(!slot.is_ready(temp.path())); - } -} +#[path = "models_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/huddle/models_tests.rs b/desktop/src-tauri/src/huddle/models_tests.rs new file mode 100644 index 0000000000..699ffbe459 --- /dev/null +++ b/desktop/src-tauri/src/huddle/models_tests.rs @@ -0,0 +1,146 @@ +use super::*; + +fn create_ready_model_dir(root: &Path) -> PathBuf { + let model_dir = root.join(TTS_MODEL_DIR_NAME); + std::fs::create_dir_all(&model_dir).expect("create model dir"); + for file in TTS_EXPECTED_FILES { + let path = model_dir.join(file); + let handle = std::fs::File::create(path).expect("create expected file"); + if let Some(size) = tts_expected_size(file) { + handle.set_len(size).expect("size expected file"); + } else { + std::fs::write(model_dir.join(file), b"test").expect("write expected file"); + } + } + std::fs::write(model_dir.join(MANIFEST_FILENAME), TTS_MODEL_VERSION).expect("manifest"); + model_dir +} + +#[test] +fn expected_files_match_april_int8_metadata() { + let mut expected = april_model_info() + .artifacts + .iter() + .map(|artifact| artifact.filename) + .chain([TTS_LICENSE_ARTIFACT.filename, TTS_LICENSE_FILE_NAME]) + .chain(POCKET_VOICES.iter().map(|voice| voice.reference_file)) + .collect::>(); + expected.sort_unstable(); + let mut actual = TTS_EXPECTED_FILES.to_vec(); + actual.sort_unstable(); + + assert_eq!(actual, expected); + assert!(!actual.contains(&"flow_lm_main.onnx")); + assert!(!actual.contains(&"flow_lm_flow.onnx")); + assert!(!actual.contains(&"mimi_decoder.onnx")); + assert!(!actual.contains(&"marius.wav")); +} + +#[test] +fn tts_readiness_requires_license_sidecar() { + let temp = tempfile::tempdir().expect("tempdir"); + let slot = tts_model_slot(); + let model_dir = create_ready_model_dir(temp.path()); + + assert!(slot.is_ready(temp.path())); + + std::fs::remove_file(model_dir.join(TTS_LICENSE_FILE_NAME)).expect("remove sidecar"); + assert!(!slot.is_ready(temp.path())); +} + +#[test] +fn tts_readiness_rejects_truncated_pinned_artifact() { + let temp = tempfile::tempdir().expect("tempdir"); + let slot = tts_model_slot(); + let model_dir = create_ready_model_dir(temp.path()); + let artifact = april_model_info().artifacts[0]; + + std::fs::OpenOptions::new() + .write(true) + .open(model_dir.join(artifact.filename)) + .expect("open artifact") + .set_len(artifact.size_bytes - 1) + .expect("truncate artifact"); + + assert!(!slot.is_ready(temp.path())); +} + +#[test] +fn january_cache_is_not_ready_for_april_int8() { + let temp = tempfile::tempdir().expect("tempdir"); + let slot = tts_model_slot(); + let model_dir = temp.path().join(TTS_MODEL_DIR_NAME); + std::fs::create_dir_all(&model_dir).expect("create model dir"); + for file in [ + "decoder.onnx", + "encoder.onnx", + "lm_flow.onnx", + "lm_main.onnx", + "text_conditioner.onnx", + "vocab.json", + "token_scores.json", + "LICENSE", + "reference_sample.wav", + TTS_LICENSE_FILE_NAME, + ] { + std::fs::write(model_dir.join(file), b"january").expect("write January file"); + } + std::fs::write(model_dir.join(MANIFEST_FILENAME), "3").expect("manifest"); + + assert!(!slot.is_ready(temp.path())); +} + +#[test] +fn interrupted_install_restores_backup_when_destination_is_missing() { + let temp = tempfile::tempdir().expect("tempdir"); + let slot = tts_model_slot(); + let backup_dir = temp.path().join("pocket-tts.old"); + std::fs::create_dir_all(&backup_dir).expect("create backup"); + std::fs::write(backup_dir.join("sentinel"), b"previous").expect("write sentinel"); + + slot.recover_interrupted_install(temp.path()); + + assert_eq!( + std::fs::read(temp.path().join(TTS_MODEL_DIR_NAME).join("sentinel")) + .expect("restored sentinel"), + b"previous" + ); + assert!(!backup_dir.exists()); +} + +#[test] +fn interrupted_install_replaces_incomplete_destination_with_backup() { + let temp = tempfile::tempdir().expect("tempdir"); + let slot = tts_model_slot(); + let model_dir = temp.path().join(TTS_MODEL_DIR_NAME); + let backup_dir = temp.path().join("pocket-tts.old"); + std::fs::create_dir_all(&model_dir).expect("create incomplete destination"); + std::fs::write(model_dir.join("incomplete"), b"april").expect("write incomplete file"); + std::fs::create_dir_all(&backup_dir).expect("create backup"); + std::fs::write(backup_dir.join("sentinel"), b"previous").expect("write sentinel"); + + slot.recover_interrupted_install(temp.path()); + + assert_eq!( + std::fs::read(model_dir.join("sentinel")).expect("restored sentinel"), + b"previous" + ); + assert!(!model_dir.join("incomplete").exists()); + assert!(!backup_dir.exists()); +} + +#[test] +fn ready_destination_removes_stale_backup() { + let temp = tempfile::tempdir().expect("tempdir"); + let slot = tts_model_slot(); + let model_dir = create_ready_model_dir(temp.path()); + let backup_dir = temp.path().join("pocket-tts.old"); + std::fs::create_dir_all(&backup_dir).expect("create backup"); + std::fs::write(backup_dir.join("sentinel"), b"previous").expect("write sentinel"); + + slot.recover_interrupted_install(temp.path()); + + assert!(slot.is_ready(temp.path())); + assert!(model_dir.exists()); + assert!(!backup_dir.exists()); +} diff --git a/desktop/src-tauri/src/huddle/models_voice_upgrade.rs b/desktop/src-tauri/src/huddle/models_voice_upgrade.rs new file mode 100644 index 0000000000..5233e02615 --- /dev/null +++ b/desktop/src-tauri/src/huddle/models_voice_upgrade.rs @@ -0,0 +1,128 @@ +use super::*; +use crate::huddle::tts_voice_registry::POCKET_VOICES; + +const PRESET_VOICE_TTS_MODEL_VERSION: &str = "4"; + +/// Attribution written beside every installed Pocket model and voice asset. +pub(super) const TTS_LICENSE_TEXT: &str = "\ +Pocket TTS +© Kyutai. + +Licensed under the Creative Commons Attribution 4.0 International License +(CC-BY-4.0). License text: https://creativecommons.org/licenses/by/4.0/ + +Original model by Kyutai: https://huggingface.co/kyutai/pocket-tts +Paper: Charles, Roebel, et al., Pocket TTS (arXiv:2509.06926). +Mimi neural codec by Kyutai is bundled as part of the model. + +April 2026 ONNX export by KevinAHM: +https://huggingface.co/KevinAHM/pocket-tts-onnx +Pinned revision: 58a6d00cf13d239b6748cb0769f35c580a8f606c + +Bundled English VCTK presets: Anna (p228), Vera (p229), Fantine (p244), +Charles (p254), Paul (p259), Eponine (p262), Azelma (p303), George (p315), +Mary (p333), Jane (p339), Michael (p360), and Eve (p361). These exact, +ai-coustics-enhanced WAVs come from Kyutai's tts-voices repository at revision +323332d33f997de8394f24a193e1a76df720e01a. +Source: https://huggingface.co/kyutai/tts-voices/tree/323332d33f997de8394f24a193e1a76df720e01a/vctk +Original recordings: Voice Cloning Toolkit (VCTK) corpus, +https://datashare.ed.ac.uk/handle/10283/3443 (CC-BY-4.0). +Enhancement (denoise/dereverb): ai-coustics, https://ai-coustics.com/ + +Buzz ships the ONNX/model artifacts and voice WAVs unmodified, renamed only +by placement in the local model directory. + +Provided \"AS IS\", without warranty of any kind, express or implied. See the +license text for full warranty disclaimer. +"; + +fn is_embedded_voice_file(filename: &str) -> bool { + POCKET_VOICES + .iter() + .any(|voice| voice.bytes.is_some() && voice.reference_file == filename) +} + +/// Add the official VCTK presets to an otherwise-ready v4 install. +/// +/// Model artifacts and Mary already exist in v4. The manifest is written last, +/// so interruption leaves v4 intact and the next launch retries. +pub(super) fn install_vctk_presets_into_v4_model(models_dir: &Path) -> Result<(), String> { + let model_dir = models_dir.join(TTS_MODEL_DIR_NAME); + let manifest_path = model_dir.join(MANIFEST_FILENAME); + let version = match std::fs::read_to_string(&manifest_path) { + Ok(version) => version, + Err(_) => return Ok(()), + }; + if version.trim() != PRESET_VOICE_TTS_MODEL_VERSION { + return Ok(()); + } + if !TTS_EXPECTED_FILES + .iter() + .filter(|filename| !is_embedded_voice_file(filename)) + .all(|filename| model_dir.join(filename).is_file()) + { + return Ok(()); + } + + for voice in POCKET_VOICES { + let Some(bytes) = voice.bytes else { + continue; + }; + std::fs::write(model_dir.join(voice.reference_file), bytes) + .map_err(|error| format!("write bundled {} voice: {error}", voice.display_name))?; + } + let retired_marius = model_dir.join("marius.wav"); + if retired_marius.is_file() { + std::fs::remove_file(retired_marius) + .map_err(|error| format!("remove retired Marius voice: {error}"))?; + } + std::fs::write(model_dir.join(TTS_LICENSE_FILE_NAME), TTS_LICENSE_TEXT) + .map_err(|error| format!("update Pocket voice notice: {error}"))?; + std::fs::write(manifest_path, TTS_MODEL_VERSION) + .map_err(|error| format!("update Pocket model manifest: {error}")) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn v4_install_adds_presets_without_redownloading_models() { + let temp = tempfile::tempdir().expect("tempdir"); + let model_dir = temp.path().join(TTS_MODEL_DIR_NAME); + std::fs::create_dir_all(&model_dir).expect("create model dir"); + for file in TTS_EXPECTED_FILES + .iter() + .filter(|filename| !is_embedded_voice_file(filename)) + { + std::fs::write(model_dir.join(file), b"existing").expect("write prior file"); + } + std::fs::write( + model_dir.join(MANIFEST_FILENAME), + PRESET_VOICE_TTS_MODEL_VERSION, + ) + .expect("write prior manifest"); + std::fs::write(model_dir.join("marius.wav"), b"retired").expect("write retired voice"); + + install_vctk_presets_into_v4_model(temp.path()).expect("in-place upgrade"); + + for voice in POCKET_VOICES { + if let Some(bytes) = voice.bytes { + assert_eq!( + std::fs::read(model_dir.join(voice.reference_file)) + .expect("bundled voice installed"), + bytes + ); + } + } + assert_eq!( + std::fs::read_to_string(model_dir.join(MANIFEST_FILENAME)).expect("updated manifest"), + TTS_MODEL_VERSION + ); + assert!(!model_dir.join("marius.wav").exists()); + assert!( + ModelSlot::new(TTS_MODEL_DIR_NAME, TTS_EXPECTED_FILES, TTS_MODEL_VERSION) + .is_ready(temp.path()) + ); + } +} diff --git a/desktop/src-tauri/src/huddle/pipeline.rs b/desktop/src-tauri/src/huddle/pipeline.rs index ceccedd8b6..fba5464a69 100644 --- a/desktop/src-tauri/src/huddle/pipeline.rs +++ b/desktop/src-tauri/src/huddle/pipeline.rs @@ -3,12 +3,16 @@ //! Handles starting, hot-starting, and spawning transcription tasks for //! the voice pipelines. Extracted from mod.rs to keep the command layer thin. -use std::sync::{ - atomic::{AtomicU64, Ordering}, - Arc, Mutex, +use std::{ + sync::{ + atomic::{AtomicU64, Ordering}, + Arc, Mutex, + }, + time::Duration, }; use nostr::JsonUtil; +use tauri::State; use uuid::Uuid; use crate::app_state::AppState; @@ -16,59 +20,228 @@ use crate::events; use super::models; use super::relay_api::{self, fetch_channel_members, parse_channel_uuid}; -use super::state::{HuddlePhase, VoiceInputMode}; +use super::state::{HuddlePhase, HuddleState, VoiceInputMode}; use super::stt; use super::tts; +pub(crate) enum PostConnectOutcome { + Ready, + Stale, +} + +/// Hot-start: check if voice models just finished downloading during an active +/// huddle and start the corresponding pipelines. +/// +/// Called by the frontend on a timer or after model status changes. No-op if +/// the huddle is not active or pipelines are already running. +#[tauri::command] +pub async fn check_pipeline_hotstart(state: State<'_, AppState>) -> Result<(), String> { + let (is_active, ephemeral_channel_id, huddle_generation) = { + let hs = state.huddle()?; + ( + matches!(hs.phase, HuddlePhase::Connected | HuddlePhase::Active), + hs.ephemeral_channel_id.clone(), + hs.huddle_generation, + ) + }; + + if !is_active { + return Ok(()); + } + + // Detect dead pipelines: if the worker thread has exited (init failure or crash), + // clear the pipeline handle so hot-start can retry on the next cycle. + { + let mut hs = state.huddle()?; + if let Some(ref p) = hs.stt_pipeline { + if p.is_finished() { + hs.stt_pipeline = None; + } + } + if let Some(ref p) = hs.tts_pipeline { + if p.is_finished() { + hs.tts_pipeline = None; + } + } + } + // Re-read after potential cleanup. + let (has_stt, has_tts, transcription_enabled) = { + let hs = state.huddle()?; + ( + hs.stt_pipeline.is_some(), + hs.tts_pipeline.is_some(), + hs.transcription_enabled, + ) + }; + + // Check if models just became ready (one-shot flags). + let stt_ready = models::global_model_manager() + .map(|m| m.take_stt_ready()) + .unwrap_or(false); + let tts_ready = models::global_model_manager() + .map(|m| m.take_tts_ready()) + .unwrap_or(false); + + // Start TTS first (so STT can capture tts_cancel). + if !has_tts && (tts_ready || models::is_tts_ready()) { + if let Err(e) = maybe_start_tts_pipeline(&state).await { + eprintln!("buzz-desktop: TTS hotstart failed: {e}"); + } + } + if transcription_enabled && !has_stt && (stt_ready || models::is_stt_ready()) { + if let Some(eph_id) = &ephemeral_channel_id { + if let Err(e) = maybe_start_stt_pipeline(&state, eph_id).await { + eprintln!("buzz-desktop: STT hotstart failed: {e}"); + } + } + } + + // Periodically refresh agent membership from the relay. + // This catches mid-huddle additions/removals by other participants, keeps + // STT p-tags authoritative, and auto-enables transcription when the first + // agent appears unless the user has already chosen a transcription state. + // Throttled independently from the more frequent hotstart poll. + // + // NOTE: The frontend ALSO polls agent membership independently via + // get_huddle_agent_pubkeys. This is intentional — the two polls have + // different failure semantics: + // - Rust (here): preserves stale list on failure (STT p-tags should not + // disappear on a transient network blip). + // - React (HuddleContext.tsx): clears list on failure (TTS authorization + // must fail-closed — never speak from a stale agent list). + // + // On Ok: always replace (even with empty — agents may have been removed). + // On Err: preserve the existing list (transient failure shouldn't zero it). + if let Some(eph_id) = &ephemeral_channel_id { + let should_refresh = { + let hs = state.huddle()?; + match hs.last_agent_refresh { + None => true, + Some(t) => t.elapsed() >= std::time::Duration::from_secs(15), + } + }; + if should_refresh { + // Fetch agents (for STT p-tags) before all members (for participant + // list) so relay membership queries remain ordered. + // Only update the throttle timestamp when at least one fetch succeeds, + // so transient failures retry immediately on the next poll cycle. + // Fetch both lists before acquiring the lock — no lock held across await. + let fresh_agents = fetch_channel_members(eph_id, Some("bot"), &state) + .await + .ok(); + let fresh_members = fetch_channel_members(eph_id, None, &state).await.ok(); + let transcription_auto_enabled = if fresh_agents.is_some() || fresh_members.is_some() { + let mut hs = state.huddle()?; + if !hs.is_current_huddle(eph_id, huddle_generation) { + return Ok(()); + } + if let Some(agents) = fresh_agents { + *hs.agent_pubkeys.lock().unwrap_or_else(|e| e.into_inner()) = agents; + } + if let Some(members) = fresh_members { + hs.participants = members; + } + hs.last_agent_refresh = Some(std::time::Instant::now()); + hs.maybe_auto_enable_transcription_for_agents() + } else { + false + }; + if transcription_auto_enabled { + start_auto_enabled_transcription(&state, eph_id).await; + } + } + } + + Ok(()) +} + pub(crate) async fn post_connect_setup( state: &AppState, ephemeral_channel_id: &str, -) -> Result<(), String> { + huddle_generation: u64, +) -> Result { + { + let hs = state.huddle()?; + if !hs.is_current_huddle(ephemeral_channel_id, huddle_generation) { + return Ok(PostConnectOutcome::Stale); + } + } + // Hydrate agent pubkeys and participants from relay in parallel // (authoritative — overrides local guesses). let (agents_result, all_members_result) = tokio::join!( fetch_channel_members(ephemeral_channel_id, Some("bot"), state), fetch_channel_members(ephemeral_channel_id, None, state), ); - if let Ok(agents) = agents_result { - let hs = state.huddle()?; - *hs.agent_pubkeys.lock().unwrap_or_else(|e| e.into_inner()) = agents; - } - - if let Ok(all_members) = all_members_result { - if !all_members.is_empty() { - let mut hs = state.huddle()?; - hs.participants = all_members; + let transcription_auto_enabled = { + let mut hs = state.huddle()?; + if !hs.is_current_huddle(ephemeral_channel_id, huddle_generation) { + return Ok(PostConnectOutcome::Stale); + } + if let Ok(agents) = agents_result { + *hs.agent_pubkeys.lock().unwrap_or_else(|e| e.into_inner()) = agents; + } + if let Ok(all_members) = all_members_result { + if !all_members.is_empty() { + hs.participants = all_members; + } } + hs.maybe_auto_enable_transcription_for_agents() + }; + + if transcription_auto_enabled { + state.emit_huddle_state_changed(); } - // Prepare TTS for agent voice. STT is transcript-specific and starts only - // when transcription is explicitly enabled. + // Prepare voice models. Agent presence may have auto-enabled transcription; + // explicit user choices remain authoritative. if let Some(mgr) = models::global_model_manager() { mgr.start_tts_download(state.http_client.clone()); + if state.huddle()?.transcription_enabled { + mgr.start_stt_download(state.http_client.clone()); + } } // Connect audio relay WebSocket (Opus encode/decode pipeline). // This is the core audio path — failure is fatal for the huddle. let parent_id = { let hs = state.huddle()?; + if !hs.is_current_huddle(ephemeral_channel_id, huddle_generation) { + return Ok(PostConnectOutcome::Stale); + } hs.parent_channel_id.clone() }; - let (cancel, pcm_tx) = - relay_api::connect_audio_relay(ephemeral_channel_id, parent_id.as_deref(), state).await?; + let audio_result = + relay_api::connect_audio_relay(ephemeral_channel_id, parent_id.as_deref(), state).await; { let mut hs = state.huddle()?; + if !hs.is_current_huddle(ephemeral_channel_id, huddle_generation) { + if let Ok((cancel, _)) = audio_result { + cancel.cancel(); + } + return Ok(PostConnectOutcome::Stale); + } + let (cancel, pcm_tx) = audio_result?; hs.audio_ws_cancel = Some(cancel); hs.audio_relay_pcm_tx = Some(pcm_tx); } - // Start TTS immediately. STT/transcript posting is opt-in and starts only - // after the user explicitly enables transcription. + // Start TTS immediately, then STT when transcription is enabled either by + // the user or by authoritative agent membership. + if !state + .huddle()? + .is_current_huddle(ephemeral_channel_id, huddle_generation) + { + return Ok(PostConnectOutcome::Stale); + } if let Err(e) = maybe_start_tts_pipeline(state).await { eprintln!("buzz-desktop: TTS pipeline failed to start: {e}"); } + if let Err(e) = maybe_start_stt_pipeline(state, ephemeral_channel_id).await { + eprintln!("buzz-desktop: STT pipeline failed to start: {e}"); + } - Ok(()) + Ok(PostConnectOutcome::Ready) } /// Attempt to start the STT pipeline if models are present. @@ -83,12 +256,16 @@ pub(crate) async fn maybe_start_stt_pipeline( state: &AppState, ephemeral_channel_id: &str, ) -> Result { - { + let huddle_generation = { let hs = state.huddle()?; - if !hs.transcription_enabled { + if !hs.transcription_enabled + || !matches!(hs.phase, HuddlePhase::Connected | HuddlePhase::Active) + || hs.ephemeral_channel_id.as_deref() != Some(ephemeral_channel_id) + { return Ok(false); } - } + hs.huddle_generation + }; if !models::is_stt_ready() { return Ok(false); // Models not downloaded yet — voice-only mode. @@ -97,21 +274,29 @@ pub(crate) async fn maybe_start_stt_pipeline( let channel_uuid = parse_channel_uuid(ephemeral_channel_id)?; - // Atomically claim the construction slot (mirrors tts_starting pattern). - { - let hs = state.huddle()?; - if hs.stt_starting.swap(true, Ordering::AcqRel) { - return Ok(false); // Another caller is already constructing. - } - } - - // Grab shared flags, agent pubkeys, and session generation from HuddleState. + // Atomically claim construction and grab shared state under one lock. // If replacing an existing pipeline, bump generation first so the old // transcription task's next POST sees a stale generation and exits. // Take the old pipeline OUT of the lock before dropping — Drop joins // the worker thread (~200ms) and must not block under the mutex. - let (tts_active, tts_cancel, agent_pubkeys_arc, session_gen, ptt_active_for_stt, old_stt) = { + let ( + tts_active, + tts_cancel, + agent_pubkeys_arc, + session_gen, + expected_generation, + stt_starting, + ptt_active_for_stt, + old_stt, + ) = { let mut hs = state.huddle()?; + if !hs.is_current_huddle(ephemeral_channel_id, huddle_generation) { + return Ok(false); + } + if hs.stt_starting.swap(true, Ordering::AcqRel) { + return Ok(false); + } + let stt_starting = Arc::clone(&hs.stt_starting); // Invalidate any existing transcription task before replacing the pipeline. if hs.stt_pipeline.is_some() { hs.session_generation.fetch_add(1, Ordering::Release); @@ -130,6 +315,8 @@ pub(crate) async fn maybe_start_stt_pipeline( Some(Arc::clone(&hs.tts_cancel)), Arc::clone(&hs.agent_pubkeys), Arc::clone(&hs.session_generation), + hs.session_generation.load(Ordering::Acquire), + stt_starting, ptt, old, ) @@ -144,13 +331,11 @@ pub(crate) async fn maybe_start_stt_pipeline( let (pipeline, text_rx) = match constructed { Ok(Ok(p)) => p, Ok(Err(e)) => { - let hs = state.huddle()?; - hs.stt_starting.store(false, Ordering::Release); + stt_starting.store(false, Ordering::Release); return Err(e); } Err(e) => { - let hs = state.huddle()?; - hs.stt_starting.store(false, Ordering::Release); + stt_starting.store(false, Ordering::Release); return Err(format!("spawn_blocking failed: {e}")); } }; @@ -158,10 +343,14 @@ pub(crate) async fn maybe_start_stt_pipeline( { let mut hs = state.huddle()?; - hs.stt_starting.store(false, Ordering::Release); + stt_starting.store(false, Ordering::Release); // Phase check: huddle may have been torn down during construction. if !hs.transcription_enabled - || !matches!(hs.phase, HuddlePhase::Connected | HuddlePhase::Active) + || !hs.is_current_transcription_generation( + ephemeral_channel_id, + huddle_generation, + expected_generation, + ) { return Ok(false); } @@ -172,6 +361,17 @@ pub(crate) async fn maybe_start_stt_pipeline( Ok(true) } +/// Start STT after agent presence automatically enables transcription. +pub(crate) async fn start_auto_enabled_transcription(state: &AppState, ephemeral_channel_id: &str) { + if let Some(manager) = models::global_model_manager() { + manager.start_stt_download(state.http_client.clone()); + } + if let Err(error) = maybe_start_stt_pipeline(state, ephemeral_channel_id).await { + eprintln!("buzz-desktop: auto-enabled STT failed to start: {error}"); + } + state.emit_huddle_state_changed(); +} + /// Attempt to start the TTS pipeline if TTS models are present and TTS is enabled. /// /// Returns `Ok(true)` if the pipeline was started, `Ok(false)` if preconditions @@ -192,10 +392,44 @@ pub(crate) async fn maybe_start_tts_pipeline(state: &AppState) -> Result return Ok(false), }; + // Avoid resolving and hashing imported voice files on every hot-start poll + // when TTS is already disabled or running. The guarded claim below repeats + // these checks after the fallible work to close the race. + { + let huddle = state.huddle()?; + if huddle.tts_pipeline.is_some() || !huddle.tts_enabled { + return Ok(false); + } + } + + // Resolve all fallible construction inputs before claiming the sentinel so + // an unreadable optional voice registry cannot wedge future start attempts. + let output_device = state + .huddle_audio + .output_device + .lock() + .unwrap_or_else(|e| e.into_inner()) + .clone(); + let app = state + .app_handle + .lock() + .map_err(|error| format!("app handle lock poisoned: {error}"))? + .clone(); + let voice_preferences = state + .huddle_audio + .tts + .lock() + .map_err(|error| format!("text-to-speech settings lock poisoned: {error}")) + .map(|settings| settings.voice_preferences.clone())?; + let initial_voice = match app { + Some(app) => super::tts_settings::pocket_voice_reference(&app, &voice_preferences)?, + None => super::tts_settings::bundled_pocket_voice_reference(&voice_preferences), + }; + // Atomically check preconditions and claim the construction slot. // The sentinel prevents a second caller from starting construction // while we're building outside the lock. - let (tts_active, tts_cancel) = { + let (tts_active, tts_cancel, tts_starting) = { let hs = state.huddle()?; if hs.tts_pipeline.is_some() { return Ok(false); @@ -206,18 +440,25 @@ pub(crate) async fn maybe_start_tts_pipeline(state: &AppState) -> Result Result Result<(), String> { + let starting = { + let huddle = state.huddle()?; + Arc::clone(&huddle.tts_starting) + }; + tokio::time::timeout(Duration::from_secs(15), async { + while starting.load(Ordering::Acquire) { + tokio::time::sleep(Duration::from_millis(10)).await; } - hs.tts_pipeline = Some(pipeline); + }) + .await + .map_err(|_| "TTS pipeline startup did not finish before timeout".to_string())?; + // The owner clears the sentinel while holding the huddle lock, before it + // publishes. Reacquiring that lock ensures publication is visible before + // the losing caller looks up the sender. + drop(state.huddle()?); + Ok(()) +} + +struct TtsStartingGuard(Arc); + +impl Drop for TtsStartingGuard { + fn drop(&mut self) { + self.0.store(false, Ordering::Release); } +} +/// Publish a constructed TTS pipeline against the latest settings. +/// +/// Construction happens outside locks and can overlap a voice change or OFF +/// transition. Holding the huddle lock while re-reading settings gives either +/// transition a safe ordering: it updates the installed pipeline afterward, +/// or this finalizer observes the new setting before publishing. +fn finalize_tts_pipeline_start( + state: &AppState, + publish: impl FnOnce(&str, &mut HuddleState), +) -> Result { + let mut huddle = state.huddle()?; + huddle.tts_starting.store(false, Ordering::Release); + if !huddle.tts_enabled + || !matches!(huddle.phase, HuddlePhase::Connected | HuddlePhase::Active) + || huddle.tts_pipeline.is_some() + { + return Ok(false); + } + let app = state + .app_handle + .lock() + .map_err(|error| format!("app handle lock poisoned: {error}"))? + .clone(); + let preferences = state + .huddle_audio + .tts + .lock() + .map_err(|error| format!("text-to-speech settings lock poisoned: {error}")) + .map(|settings| settings.voice_preferences.clone())?; + let voice = match app { + Some(app) => super::tts_settings::pocket_voice_reference(&app, &preferences)?, + None => super::tts_settings::bundled_pocket_voice_reference(&preferences), + }; + publish(&voice, &mut huddle); Ok(true) } +fn should_reselect_constructed_voice(constructed_voice: &str, latest_voice: &str) -> bool { + constructed_voice != latest_voice +} + +/// Sign an STT transcript event and produce the guarded POST body. +/// +/// Factored out of the transcription loop so egress boundary 5 (huddle STT) +/// has a directly testable seam: the NIP-49 egress guard runs here, before +/// any bytes can reach the network. +pub(crate) fn sign_and_guard_stt_body( + builder: nostr::EventBuilder, + keys: &nostr::Keys, +) -> Result, String> { + let event = builder + .sign_with_keys(keys) + .map_err(|e| format!("sign event: {e}"))?; + let body_bytes = event.as_json().into_bytes(); + crate::egress_guard::assert_no_key_backup_bytes(&body_bytes, "huddle STT publish")?; + Ok(body_bytes) +} + /// Spawn a tokio task that reads text_rx and posts kind:9 events. /// /// Fix 1: `agent_pubkeys_arc` is an `Arc>>` cloned from @@ -310,14 +632,13 @@ pub(crate) fn spawn_transcription_task( // the kind event and build NIP-98 auth after the wait so both // timestamps are fresh — single clean order: wait → sign → auth → send. crate::relay_admission::wait_for_rate_limit().await; - let event = match builder.sign_with_keys(&keys) { - Ok(e) => e, + let body_bytes = match sign_and_guard_stt_body(builder, &keys) { + Ok(b) => b, Err(e) => { - eprintln!("buzz-desktop: STT sign event: {e}"); + eprintln!("buzz-desktop: STT publish: {e}"); continue; } }; - let body_bytes = event.as_json().into_bytes(); let url = format!("{relay_base_url}/events"); let auth_header = match crate::relay::build_nip98_auth_header_for_keys( &keys, @@ -357,3 +678,148 @@ pub(crate) fn spawn_transcription_task( } }); } + +#[cfg(test)] +mod tts_start_race_tests { + use std::sync::{ + atomic::{AtomicBool, Ordering}, + Arc, Barrier, Mutex, + }; + use std::time::Duration; + + use crate::app_state::build_app_state; + + use super::{ + await_inflight_tts_start, finalize_tts_pipeline_start, should_reselect_constructed_voice, + HuddlePhase, + }; + + #[tokio::test] + async fn a_losing_starter_observes_publication_before_resuming() { + let state = Arc::new(build_app_state()); + { + let mut huddle = state.huddle().expect("huddle state"); + huddle.phase = HuddlePhase::Active; + huddle.tts_enabled = true; + huddle.tts_starting.store(true, Ordering::Release); + } + let published = Arc::new(AtomicBool::new(false)); + let owner_state = Arc::clone(&state); + let owner_published = Arc::clone(&published); + let owner = std::thread::spawn(move || { + std::thread::sleep(Duration::from_millis(20)); + finalize_tts_pipeline_start(&owner_state, |_, _| { + owner_published.store(true, Ordering::Release); + }) + }); + + await_inflight_tts_start(&state) + .await + .expect("wait for pipeline owner"); + assert!(published.load(Ordering::Acquire)); + assert!(owner.join().expect("pipeline owner").expect("finalize")); + } + + #[test] + fn constructor_fallback_survives_unchanged_preference_at_publication() { + let selected_voice = Mutex::new(super::super::pocket::DEFAULT_VOICE.to_string()); + let constructed_voice = "eve"; + let latest_voice = "eve"; + + if should_reselect_constructed_voice(constructed_voice, latest_voice) { + *selected_voice.lock().expect("selected voice") = latest_voice.to_string(); + } + + assert_eq!( + selected_voice.lock().expect("selected voice").as_str(), + super::super::pocket::DEFAULT_VOICE + ); + } + + #[test] + fn construction_reconciles_a_voice_selected_while_starting() { + let state = Arc::new(build_app_state()); + { + let mut huddle = state.huddle().expect("huddle state"); + huddle.phase = HuddlePhase::Active; + huddle.tts_enabled = true; + huddle.tts_starting.store(true, Ordering::Release); + } + + let constructed = Arc::new(Barrier::new(2)); + let publish = Arc::new(Barrier::new(2)); + let selected_voice = Arc::new(Mutex::new(None)); + let worker_state = Arc::clone(&state); + let worker_constructed = Arc::clone(&constructed); + let worker_publish = Arc::clone(&publish); + let worker_voice = Arc::clone(&selected_voice); + let worker = std::thread::spawn(move || { + worker_constructed.wait(); + worker_publish.wait(); + finalize_tts_pipeline_start(&worker_state, |voice, _| { + *worker_voice.lock().expect("selected voice") = Some(voice.to_string()); + }) + }); + + constructed.wait(); + assert!(state + .huddle() + .expect("huddle state") + .tts_starting + .load(Ordering::Acquire)); + state + .huddle_audio + .tts + .lock() + .expect("text-to-speech settings") + .voice_preferences = vec!["pocket:eve".to_string()]; + publish.wait(); + + assert!(worker.join().expect("starter thread").expect("finalize")); + assert_eq!( + *selected_voice.lock().expect("selected voice"), + Some("eve".to_string()) + ); + } + + #[test] + fn construction_is_discarded_when_disabled_while_starting() { + let state = Arc::new(build_app_state()); + { + let mut huddle = state.huddle().expect("huddle state"); + huddle.phase = HuddlePhase::Active; + huddle.tts_enabled = true; + huddle.tts_starting.store(true, Ordering::Release); + } + + let constructed = Arc::new(Barrier::new(2)); + let publish = Arc::new(Barrier::new(2)); + let did_publish = Arc::new(Mutex::new(false)); + let worker_state = Arc::clone(&state); + let worker_constructed = Arc::clone(&constructed); + let worker_publish = Arc::clone(&publish); + let worker_did_publish = Arc::clone(&did_publish); + let worker = std::thread::spawn(move || { + worker_constructed.wait(); + worker_publish.wait(); + finalize_tts_pipeline_start(&worker_state, |_, _| { + *worker_did_publish.lock().expect("publish flag") = true; + }) + }); + + constructed.wait(); + { + let mut huddle = state.huddle().expect("huddle state"); + huddle.tts_enabled = false; + } + publish.wait(); + + assert!(!worker.join().expect("starter thread").expect("finalize")); + assert!(!*did_publish.lock().expect("publish flag")); + assert!(!state + .huddle() + .expect("huddle state") + .tts_starting + .load(Ordering::Acquire)); + } +} diff --git a/desktop/src-tauri/src/huddle/pocket.rs b/desktop/src-tauri/src/huddle/pocket.rs index ee1faf928a..fd407103d1 100644 --- a/desktop/src-tauri/src/huddle/pocket.rs +++ b/desktop/src-tauri/src/huddle/pocket.rs @@ -1,654 +1,4 @@ -//! Pocket TTS engine wrapper around sherpa-onnx's `OfflineTts`. -//! -//! Pocket TTS is a small (~473 MB fp32 ONNX) zero-shot voice-cloning TTS -//! model from Kyutai. It runs quickly on CPU via sherpa-onnx, replacing the -//! previous Kokoro-82M engine that also required an espeak-free but -//! lexicon-heavy G2P pipeline (Misaki + CMUdict). -//! -//! Full-precision fp32 sessions, not the ~189 MB int8 quantization we -//! originally shipped: a direct same-runtime A/B (k2-fsa/sherpa-onnx#3172) -//! found the int8 ONNX export audibly degraded output quality, and fp32 -//! "significantly improved quality even at 1 step". -//! -//! ## Attribution -//! -//! - **Model**: Kyutai *Pocket TTS* — Charles, Roebel, et al., 2026. -//! arXiv:2509.06926. Original repository: . -//! Licensed CC-BY-4.0. -//! - **Mimi neural codec**: Kyutai, bundled in the same release. CC-BY-4.0. -//! - **ONNX export**: KevinAHM — -//! . CC-BY-4.0. -//! - **sherpa-onnx repackage**: csukuangfj / k2-fsa — -//! . -//! Repackages KevinAHM's export with the file layout sherpa-onnx's -//! `OfflineTtsPocketModelConfig` expects. CC-BY-4.0. -//! - **Reference voice WAV** (`reference_sample.wav`): the "Mary -//! (f, conversation)" preset from the Kyutai TTS demo -//! (), which maps to `vctk/p333_023_enhanced.wav` -//! in . CC-BY-4.0, base recording -//! from the VCTK corpus, enhanced by ai-coustics. -//! -//! Buzz ships these files unmodified; see the on-disk `MODEL_LICENSE.txt` -//! sidecar written by `huddle::models` during install for the canonical -//! CC-BY-4.0 §3(a)(1) attribution block. -//! -//! ## Engine-module contract (see `huddle::tts`) -//! -//! `pocket.rs` exposes a fixed surface used by `tts.rs`. Mirroring this -//! contract is what lets the TTS pipeline stay engine-agnostic: -//! -//! - `SAMPLE_RATE: u32` — engine output sample rate in Hz. -//! - `DEFAULT_VOICE: &str` — default voice name (without extension). -//! - `VOICE_FILE_EXT: &str` — extension for per-voice files on disk. -//! - `load_text_to_speech(model_dir)` → `Result` -//! - `load_voice_style(path)` → `Result` -//! - `Engine::synth_chunk(&self, text, lang, &VoiceStyle, steps)` -//! → `Result, String>` -//! -//! `lang` and `steps` are accepted for API compatibility with the previous -//! Kokoro engine but are unused — Pocket TTS does its own language ID from -//! the input text and is not a diffusion model (consistency LM, one step). -//! There is no speed knob: sherpa-onnx's `GenerationConfig.speed` is only -//! read by some model families (vits), never by the Pocket impl -//! (`offline-tts-pocket-impl.h` — zero references), and upstream pocket-tts -//! has no speed parameter either. - -use std::collections::HashMap; -use std::path::{Path, PathBuf}; - -use sherpa_onnx::{GenerationConfig, OfflineTts, OfflineTtsConfig, Wave}; - -// ── Engine-module contract: public consts ───────────────────────────────────── - -/// Pocket TTS emits 24 kHz mono PCM. Matches the previous Kokoro output rate, -/// so the rodio sink and inter-sentence silence buffer in `tts.rs` remain valid. -pub const SAMPLE_RATE: u32 = 24_000; - -/// Name (without extension) of the bundled reference voice. The model directory -/// is expected to contain `.` after install. -pub const DEFAULT_VOICE: &str = "reference_sample"; - -/// Voice files for Pocket TTS are reference audio (WAV). Distinct from the -/// Kokoro `.bin` style vectors — the model conditions on raw waveform samples, -/// not a precomputed embedding, so the extension change is honest. -pub const VOICE_FILE_EXT: &str = "wav"; - -// ── Tuning ──────────────────────────────────────────────────────────────────── - -/// Single-threaded ONNX execution for predictable CPU contention with the STT -/// pipeline. Matches `STT_NUM_THREADS` in `stt.rs`; raise only if a benchmark -/// argues for it. -const TTS_NUM_THREADS: i32 = 1; - -/// LRU cache size for cloned voice embeddings inside the sherpa-onnx engine. -/// We bind to one voice per pipeline today, but the upstream example uses 16 -/// and the cost is negligible — keep room for future multi-voice support. -const VOICE_EMBEDDING_CACHE_CAPACITY: i32 = 16; - -/// Pocket TTS is a consistency-based LM. Generation quality saturates at one -/// denoising step — the upstream `GenerationConfig` default of 5 multiplies -/// synthesis time by ~5× with no audible benefit on this model. -const SYNTH_NUM_STEPS: i32 = 1; - -/// Leave the generated audio's silences untouched (1.0 is the identity). -/// -/// sherpa-onnx's `ScaleSilence` (`offline-tts.cc`) is *not* pre/post padding -/// control: it finds every interior silence run ≥ 0.2 s (|s| ≤ 0.01) and -/// multiplies its length by this factor. The previous value of 0.0 — set -/// under the mistaken belief it disabled lead-in/lead-out padding — deleted -/// every natural pause inside an utterance: clause breaks, breaths, the gap -/// after a comma. Words slammed together and endings cut abruptly. The -/// reference Pocket TTS pipeline does not post-process silence at all; -/// 1.0 restores parity. -const SYNTH_SILENCE_SCALE: f32 = 1.0; - -/// sherpa-onnx upstream default for `max_frames` (LM steps), in -/// `offline-tts-pocket-impl.h:Generate`. 500 steps ≈ 40 s of audio at the -/// Mimi 12.5 Hz frame rate. Referenced only by the regression test below; -/// production code path never raises (or even reads) this value — we just -/// leave sherpa-onnx's own default in place by not setting the override. -#[cfg(test)] -const SHERPA_ONNX_MAX_FRAMES_DEFAULT: i32 = 500; - -/// Tight `max_frames` we ask for on short, padded prompts to bound the -/// original "monster breathing" runaway. 100 LM steps ≈ 8 s of audio — -/// roomy for any one-to-four-word utterance the user is likely to elicit -/// while still well short of the 40 s upstream default. Chosen with slack so -/// we never *truncate* a legitimate short reply. -const SHORT_PROMPT_MAX_FRAMES: i32 = 100; - -/// Word-count threshold (inclusive) below which we pad the prompt with -/// leading spaces and cap `max_frames` tighter than the upstream default. -/// Matches upstream `pocket_tts.models.tts_model.prepare_text_prompt`. Above -/// this threshold we leave sherpa-onnx's own defaults in place — overriding -/// them caused the "first 'yep' is just static" regression seen on -/// 2026-05-18, where dropping `frames_after_eos` below the upstream default -/// of 3 clipped the leading audio of multi-clause sentences. -const SHORT_PROMPT_WORD_THRESHOLD: usize = 4; - -/// Number of leading spaces prepended to short prompts. The upstream Python -/// uses exactly 8 — keep parity rather than tuning blindly. -/// -/// This is upstream's *only* mitigation for the FlowLM cold-start smear on -/// short utterances (kyutai-labs/pocket-tts #91, #70): the autoregressive -/// generation has a 2–3 step "settle" period where the first phoneme can be -/// smeared. A previous revision added a sacrificial `". . "` prefix plus an -/// amplitude-threshold trim to strip the rendered prefix from the output — -/// but the trim's absolute threshold (0.02 against raw peaks of ~0.076) sat -/// in soft-onset territory and could eat real word starts, and its tuning -/// was calibrated against `silence_scale = 0.0` audio. Deleted in favour of -/// upstream parity: accept the occasional smeared first syllable rather -/// than risk trimming real speech. -const SHORT_PROMPT_PAD_SPACES: usize = 8; - -/// sherpa-onnx's documented `frames_after_eos` default. We deliberately do -/// *not* override this knob — the previous attempt to bump it for short -/// inputs and lower it for long inputs lowered it below the upstream default -/// of 3, which clipped the leading audio of multi-clause sentences (the -/// "first 'yep' is static" regression). The constant exists only for the -/// regression test below. Source: `offline-tts-pocket-impl.h:Generate`. -#[cfg(test)] -const SHERPA_ONNX_FRAMES_AFTER_EOS_DEFAULT: i32 = 3; - -// ── ONNX file names (five Pocket TTS sessions plus two JSON tables) ─────────── - -const FILE_LM_MAIN: &str = "lm_main.onnx"; -const FILE_LM_FLOW: &str = "lm_flow.onnx"; -const FILE_ENCODER: &str = "encoder.onnx"; -const FILE_DECODER: &str = "decoder.onnx"; -const FILE_TEXT_COND: &str = "text_conditioner.onnx"; -const FILE_VOCAB: &str = "vocab.json"; -const FILE_TOKEN_SCORES: &str = "token_scores.json"; - -// ── Voice style ─────────────────────────────────────────────────────────────── - -/// Loaded reference voice — normalised f32 PCM samples plus their sample rate. -/// -/// Pocket TTS takes a reference waveform per generation call (not a -/// precomputed style embedding), so we keep the samples in memory and clone -/// the small `Vec` into each `GenerationConfig` rather than re-reading the -/// WAV from disk on every sentence. -#[derive(Debug, Clone)] -pub struct VoiceStyle { - samples: Vec, - sample_rate: i32, -} - -/// Load a reference voice WAV from disk. -/// -/// Accepts any sample rate sherpa-onnx's `Wave::read` can decode — Pocket TTS -/// resamples internally using `reference_sample_rate`. The bundled -/// `reference_sample.wav` ("Mary" — VCTK p333, enhanced) is 32 kHz mono. -pub fn load_voice_style(path: &Path) -> Result { - let path_str = path - .to_str() - .ok_or_else(|| format!("voice path is not valid UTF-8: {}", path.display()))?; - let wave = Wave::read(path_str) - .ok_or_else(|| format!("could not read voice WAV at {}", path.display()))?; - let samples = wave.samples().to_vec(); - if samples.is_empty() { - return Err(format!("voice WAV is empty: {}", path.display())); - } - Ok(VoiceStyle { - samples, - sample_rate: wave.sample_rate(), - }) -} - -// ── Engine ──────────────────────────────────────────────────────────────────── - -/// Pocket TTS engine handle. Cheap to construct (one `OfflineTts::create` -/// call). Owned by the TTS worker thread for the lifetime of a huddle session. -/// -/// `OfflineTts` does not implement `Debug`, so we don't derive it here — the -/// pipeline only needs to move the engine into the worker thread and call -/// `synth_chunk` on it, never to print it. -pub struct PocketTts { - inner: OfflineTts, -} - -/// Build the Pocket TTS engine from the model directory installed by -/// `huddle::models`. Returns `Err` if any expected ONNX or JSON file is -/// missing — readiness is normally enforced by `is_tts_ready` upstream, but -/// the check is repeated here so a manually-modified model dir produces a -/// clear error string instead of an opaque sherpa-onnx `None`. -pub fn load_text_to_speech(model_dir: &str) -> Result { - let dir = PathBuf::from(model_dir); - for name in [ - FILE_LM_MAIN, - FILE_LM_FLOW, - FILE_ENCODER, - FILE_DECODER, - FILE_TEXT_COND, - FILE_VOCAB, - FILE_TOKEN_SCORES, - ] { - let p = dir.join(name); - if !p.is_file() { - return Err(format!("missing Pocket TTS file: {}", p.display())); - } - } - - let to_str = |name: &str| -> String { dir.join(name).to_string_lossy().into_owned() }; - - // Build the config by mutating defaults — mirrors `stt.rs` and stays - // resilient if sherpa-onnx adds unrelated model-family fields. - let mut cfg = OfflineTtsConfig::default(); - cfg.model.pocket.lm_main = Some(to_str(FILE_LM_MAIN)); - cfg.model.pocket.lm_flow = Some(to_str(FILE_LM_FLOW)); - cfg.model.pocket.encoder = Some(to_str(FILE_ENCODER)); - cfg.model.pocket.decoder = Some(to_str(FILE_DECODER)); - cfg.model.pocket.text_conditioner = Some(to_str(FILE_TEXT_COND)); - cfg.model.pocket.vocab_json = Some(to_str(FILE_VOCAB)); - cfg.model.pocket.token_scores_json = Some(to_str(FILE_TOKEN_SCORES)); - cfg.model.pocket.voice_embedding_cache_capacity = VOICE_EMBEDDING_CACHE_CAPACITY; - cfg.model.num_threads = TTS_NUM_THREADS; - // Explicit — defaults are not part of the API contract, and noisy debug - // logging in release builds would be expensive on every synthesized chunk. - cfg.model.debug = false; - - let inner = OfflineTts::create(&cfg) - .ok_or_else(|| "OfflineTts::create returned None for Pocket TTS".to_string())?; - Ok(PocketTts { inner }) -} - -// ── Prompt preparation ──────────────────────────────────────────────────────── - -/// Result of [`prepare_pocket_prompt`]: a synthesizer-ready prompt plus the -/// per-call generation overrides derived from the original text. -/// -/// `None` for either override means "leave sherpa-onnx's documented default -/// in place". The pipeline only sets `max_frames` (and only for short -/// padded inputs) so it can bound the original "monster breathing" runaway -/// without disturbing the rest of the LM sampling envelope. -#[derive(Debug, Clone, PartialEq)] -pub(crate) struct PreparedPrompt { - /// Text to hand to `OfflineTts::generate_with_config`. Capitalized, - /// punctuation-terminated, and (for short inputs) left-padded with - /// spaces — upstream's mitigation for the FlowLM cold-start smear. - pub text: String, - /// Value to pass via `GenerationConfig.extra["max_frames"]`, or `None` to - /// keep the upstream default of 500 LM steps. We only override on short - /// padded prompts where we have a tight expectation on output length. - pub max_frames: Option, -} - -/// Mirror of the *text-preparation* half of upstream -/// `pocket_tts.models.tts_model.prepare_text_prompt`. Sherpa-onnx's C++ -/// Pocket TTS impl does not run these preparation steps, so short / -/// unpunctuated / lowercase inputs can trigger up to 40 s of runaway -/// generation when the EOS logit never crosses its threshold. We replicate -/// the upstream Python recipe here: -/// -/// 1. Collapse interior whitespace (already done by `preprocess_for_tts`, but -/// cheap to re-check after sentence splitting). -/// 2. Capitalize the first letter. -/// 3. Append `.` if the text doesn't end in punctuation. -/// 4. If fewer than five words, prepend `SHORT_PROMPT_PAD_SPACES` spaces -/// (upstream's cold-start mitigation — see the constant's docstring) and -/// return a tight [`SHORT_PROMPT_MAX_FRAMES`] cap so the LM can't run -/// away if EOS still doesn't fire. -/// -/// We do **not** override `frames_after_eos` — sherpa-onnx's default of 3 -/// is what we want. An earlier version set it to 1 on long inputs, which -/// clipped the leading audio of multi-clause sentences ("first 'yep' is -/// just static" regression). Tests `prepare_prompt_never_lowers_frames_…` -/// lock this in. -/// -/// Returns `None` only if the input is empty after trimming — caller should -/// skip synthesis in that case. -pub(crate) fn prepare_pocket_prompt(input: &str) -> Option { - let trimmed = input.trim(); - if trimmed.is_empty() { - return None; - } - - // Collapse stray double-spaces / embedded newlines that may slip past - // `preprocess_for_tts` when sentences are spliced back together. - let mut cleaned = String::with_capacity(trimmed.len()); - let mut last_was_space = false; - for ch in trimmed.chars() { - let is_ws = ch.is_whitespace(); - if is_ws { - if !last_was_space { - cleaned.push(' '); - } - last_was_space = true; - } else { - cleaned.push(ch); - last_was_space = false; - } - } - - // Capitalize first character. Uses `to_uppercase` (multi-codepoint safe). - let first = cleaned.chars().next().expect("cleaned non-empty above"); - if first.is_lowercase() { - let upper: String = first.to_uppercase().collect(); - let mut iter = cleaned.chars(); - iter.next(); - cleaned = upper + iter.as_str(); - } - - // Ensure terminal punctuation. Anything not in `.!?;:,` gets a period. - // The upstream Python only checks `isalnum` → period, but for our agent - // text we already may end in `!` `?` `.` etc. — treat any of those as OK. - let last = cleaned - .chars() - .next_back() - .expect("cleaned non-empty above"); - if !matches!(last, '.' | '!' | '?' | ';' | ':' | ',') { - cleaned.push('.'); - } - - // Word count of the *cleaned but not padded* text — padding is whitespace - // only and would just lie to the threshold check below. - let word_count = cleaned.split_whitespace().count(); - - let (final_text, max_frames) = if word_count <= SHORT_PROMPT_WORD_THRESHOLD { - let mut padded = String::with_capacity(cleaned.len() + SHORT_PROMPT_PAD_SPACES); - for _ in 0..SHORT_PROMPT_PAD_SPACES { - padded.push(' '); - } - padded.push_str(&cleaned); - (padded, Some(SHORT_PROMPT_MAX_FRAMES)) - } else { - // For everything ≥5 words, fall back to upstream defaults. Overriding - // these is what caused the "first 'yep' is static" regression — the - // upstream LM has been tuned for `frames_after_eos = 3` and - // `max_frames = 500`, and there's no clear win in second-guessing. - (cleaned, None) - }; - - Some(PreparedPrompt { - text: final_text, - max_frames, - }) -} - -/// Build the `GenerationConfig.extra` HashMap from a [`PreparedPrompt`]. -/// -/// Centralised so the regression test below can assert that we **never** -/// emit a `frames_after_eos` override — the previous attempt to override -/// that knob (setting it to 1 for ≥5-word inputs) clipped the leading -/// audio of multi-clause sentences (the "first 'yep' is static" bug on -/// 2026-05-18). The upstream sherpa-onnx default of 3 is what we want, and -/// the right way to keep it is to not set it at all. -fn build_generation_extra(prepared: &PreparedPrompt) -> Option> { - prepared.max_frames.map(|mf| { - let mut h: HashMap = HashMap::with_capacity(1); - h.insert("max_frames".to_string(), serde_json::Value::from(mf)); - h - }) -} - -impl PocketTts { - /// Synthesise `text` with the given reference voice. - /// - /// `_lang` and `_steps` are accepted for API compatibility with the - /// previous Kokoro engine. Pocket TTS infers language from the input text - /// directly and is a one-step consistency model. Returns an empty buffer - /// for whitespace-only input. - pub fn synth_chunk( - &self, - text: &str, - _lang: &str, - style: &VoiceStyle, - _steps: usize, - ) -> Result, String> { - // Mirror upstream pocket-tts prompt prep — without this short or - // unpunctuated inputs can cause the LM's EOS logit to never trip, - // producing up to 40 s of "monster breathing" garbage on the first - // utterance. See `prepare_pocket_prompt` for the full recipe. - let prepared = match prepare_pocket_prompt(text) { - Some(p) => p, - None => return Ok(Vec::new()), - }; - - // Per-call generation hints sherpa-onnx forwards to - // `offline-tts-pocket-impl.h`. We only override `max_frames`, and - // only for short padded prompts where we have a tight expectation - // on output length — that bounds the original runaway without - // disturbing the rest of the LM sampling envelope. See - // `prepare_pocket_prompt` docs for the regression history. - let extra = build_generation_extra(&prepared); - - let cfg = GenerationConfig { - num_steps: SYNTH_NUM_STEPS, - silence_scale: SYNTH_SILENCE_SCALE, - reference_audio: Some(style.samples.clone()), - reference_sample_rate: style.sample_rate, - extra, - // `speed` stays at its default: the Pocket impl never reads it - // (see the engine-contract note in the module docs). - ..Default::default() - }; - - // No progress callback — synthesis is fast enough that returning the - // whole buffer at once keeps the lookahead pipelining in `tts.rs` - // simple. `None:: bool>` pins the callback type for the - // `generate_with_config` generic parameter. - let audio = self - .inner - .generate_with_config(&prepared.text, &cfg, None:: bool>) - .ok_or_else(|| { - format!( - "Pocket TTS synthesis failed for text ({} chars)", - prepared.text.len() - ) - })?; - - let sample_rate = audio.sample_rate(); - if sample_rate != SAMPLE_RATE as i32 { - eprintln!( - "buzz-desktop: Pocket TTS returned unexpected sample rate {sample_rate}Hz \ - (expected {SAMPLE_RATE}Hz); playback speed may be wrong" - ); - } - - Ok(audio.samples().to_vec()) - } -} - -#[cfg(test)] -mod tests { - use super::*; - - // ── prepare_pocket_prompt ──────────────────────────────────────────────── - - #[test] - fn prepare_prompt_returns_none_for_empty_input() { - assert!(prepare_pocket_prompt("").is_none()); - assert!(prepare_pocket_prompt(" ").is_none()); - assert!(prepare_pocket_prompt("\n\t ").is_none()); - } - - /// Helper: the exact leading sequence prepended to every short prompt — - /// 8 spaces of padding (upstream's cold-start mitigation). - /// Centralising this keeps the assertions readable. - fn short_prefix() -> String { - " ".repeat(SHORT_PROMPT_PAD_SPACES) - } - - #[test] - fn prepare_prompt_pads_and_capitalizes_one_word() { - // The "yep" case Tyler hit in production — bare lowercase one-word - // utterance with no punctuation. Must be padded with the short-prompt - // space pad, capitalized, terminated, with a tight `max_frames` cap - // to bound runaway gen. - let out = prepare_pocket_prompt("yep").expect("non-empty"); - assert_eq!(out.text, format!("{}Yep.", short_prefix())); - assert_eq!(out.max_frames, Some(SHORT_PROMPT_MAX_FRAMES)); - const { - assert!( - SHORT_PROMPT_MAX_FRAMES < SHERPA_ONNX_MAX_FRAMES_DEFAULT, - "short cap must be tighter than the upstream default" - ); - } - } - - #[test] - fn prepare_prompt_preserves_existing_punctuation() { - let out = prepare_pocket_prompt("yes!").expect("non-empty"); - assert_eq!(out.text, format!("{}Yes!", short_prefix())); // exclamation kept - let out = prepare_pocket_prompt("really?").expect("non-empty"); - assert_eq!(out.text, format!("{}Really?", short_prefix())); - } - - #[test] - fn prepare_prompt_threshold_is_inclusive_at_four_words() { - // 4 words = short (padded + tight max_frames); 5 words = long - // (no padding, no overrides — upstream defaults stand). - let four = prepare_pocket_prompt("one two three four").expect("non-empty"); - assert_eq!( - four.text, - format!("{}One two three four.", short_prefix()), - "four-word input should get exactly the space pad" - ); - assert_eq!(four.max_frames, Some(SHORT_PROMPT_MAX_FRAMES)); - - let five = prepare_pocket_prompt("one two three four five").expect("non-empty"); - assert!( - !five.text.starts_with(' '), - "five-word input should NOT be padded" - ); - assert_eq!( - five.max_frames, None, - "long inputs must leave sherpa-onnx's max_frames default in place" - ); - } - - #[test] - fn prepare_prompt_does_not_pad_long_text() { - let long = "This is a longer sentence that the model should handle just fine."; - let out = prepare_pocket_prompt(long).expect("non-empty"); - assert!(!out.text.starts_with(' ')); - assert_eq!(out.max_frames, None); - assert!(out.text.ends_with('.')); - } - - #[test] - fn prepare_prompt_collapses_whitespace() { - let out = prepare_pocket_prompt("Hello world\n\nfriend").expect("non-empty"); - // 3 words → short → padded. Interior whitespace collapsed. - assert_eq!(out.text, format!("{}Hello world friend.", short_prefix())); - } - - #[test] - fn prepare_prompt_does_not_double_capitalize_already_uppercase() { - let out = prepare_pocket_prompt("HELLO there").expect("non-empty"); - assert_eq!(out.text, format!("{}HELLO there.", short_prefix())); - } - - #[test] - fn prepare_prompt_handles_non_ascii_first_letter() { - // Cyrillic lowercase 'д' → uppercase 'Д'. Must not panic / produce - // mojibake. - let out = prepare_pocket_prompt("дa").expect("non-empty"); - assert!(out.text.contains("Дa.")); - } - - /// REGRESSION GUARD: short prompts must receive *only* whitespace - /// padding — no sacrificial text. A previous revision prepended a - /// `". . "` cold-start absorber and trimmed the rendered audio back out - /// with an amplitude threshold that could eat soft word onsets. If - /// non-whitespace ever reappears in the pad, the synth output will - /// contain audio for text the user never wrote. - #[test] - fn prepare_prompt_pad_is_whitespace_only() { - let out = prepare_pocket_prompt("I'm happy.").expect("non-empty"); - let pad_len = out.text.len() - "I'm happy.".len(); - assert!( - out.text[..pad_len].chars().all(|c| c == ' '), - "short-prompt pad must be spaces only, got {:?}", - &out.text[..pad_len] - ); - assert_eq!(out.text, format!("{}I'm happy.", short_prefix())); - } - - // ── build_generation_extra ─────────────────────────────────────────────── - // - // These tests pin down a behaviour we've now regressed twice on: - // 1) Not padding/punctuating short inputs → 40 s of "monster breathing" - // (pre-773a2a1). - // 2) Setting `frames_after_eos = 1` on long inputs → clipped leading - // audio of multi-clause sentences, e.g. "Yep, I can hear you. …" - // came out as a static burst (the 773a2a1 regression Tyler hit on - // 2026-05-18 ~14:30 UTC). - // - // The contract we enforce going forward: we **only** override - // `max_frames`, and only for ≤4-word inputs. Every other knob is left - // at sherpa-onnx's documented default (notably `frames_after_eos = 3`). - - #[test] - fn build_extra_short_prompt_sets_only_max_frames() { - let prepared = prepare_pocket_prompt("yep").expect("non-empty"); - let extra = build_generation_extra(&prepared).expect("short prompts get extra"); - // Exactly one key — `max_frames` — and nothing else. - assert_eq!(extra.len(), 1, "extra has unexpected keys: {extra:?}"); - assert_eq!( - extra.get("max_frames"), - Some(&serde_json::Value::from(SHORT_PROMPT_MAX_FRAMES)) - ); - assert!( - !extra.contains_key("frames_after_eos"), - "frames_after_eos must never be set — upstream default of {SHERPA_ONNX_FRAMES_AFTER_EOS_DEFAULT} is what we want" - ); - } - - #[test] - fn build_extra_long_prompt_is_none() { - // ≥5 words: no extras at all. This is the key fix for the "first - // 'yep' in 'Yep, I can hear you. …' is static" regression — we - // were previously forcing `frames_after_eos = 1` on this path. - let prepared = prepare_pocket_prompt("Yep, I can hear you.").expect("non-empty"); - assert_eq!( - build_generation_extra(&prepared), - None, - "long prompts must not override any LM knob" - ); - } - - #[test] - fn build_extra_never_lowers_frames_after_eos_for_any_word_count() { - // Sweep a range of prompt lengths and assert the `extra` map (when - // present) never carries a `frames_after_eos` override that's lower - // than the upstream sherpa-onnx default. Implemented as a structural - // check — we just never set the key — but worth a property test in - // case someone reintroduces the override in the future. - let prompts: &[&str] = &[ - "hi", - "hi there", - "yes please", - "one two three four", - "one two three four five", - "a slightly longer reply, hopefully fine", - "This is a multi-clause sentence. It has two parts.", - "really really really really really long prompt with lots of words just to be sure", - ]; - for &p in prompts { - let prepared = prepare_pocket_prompt(p).expect("non-empty"); - if let Some(extra) = build_generation_extra(&prepared) { - if let Some(v) = extra.get("frames_after_eos") { - let n = v.as_i64().expect("frames_after_eos should be int"); - assert!( - n >= SHERPA_ONNX_FRAMES_AFTER_EOS_DEFAULT as i64, - "prompt {p:?} set frames_after_eos={n}, below upstream default of {SHERPA_ONNX_FRAMES_AFTER_EOS_DEFAULT}" - ); - } - } - } - } - - #[test] - fn short_prompt_max_frames_is_below_upstream_default() { - // Sanity: the override only ever *lowers* the cap, never raises it. - const { - assert!(SHORT_PROMPT_MAX_FRAMES < SHERPA_ONNX_MAX_FRAMES_DEFAULT); - } - // …and is still large enough for a one-to-four-word reply. At Mimi's - // 12.5 Hz frame rate, 100 frames = 8 s, which is roomy. - const { - assert!(SHORT_PROMPT_MAX_FRAMES >= 50, "would risk truncation"); - } - } -} +pub use buzz_voice_pkg::pocket::*; +pub(crate) use buzz_voice_pkg::{ + april_model_info, PocketModelArtifact, APRIL_BUNDLE_ID, APRIL_MODEL_ID, APRIL_MODEL_REVISION, +}; diff --git a/desktop/src-tauri/src/huddle/relay_api.rs b/desktop/src-tauri/src/huddle/relay_api.rs index eb3fea92d5..3f2aa76a56 100644 --- a/desktop/src-tauri/src/huddle/relay_api.rs +++ b/desktop/src-tauri/src/huddle/relay_api.rs @@ -164,7 +164,8 @@ pub(crate) async fn connect_audio_relay( let cancel_clone = cancel.clone(); let (pcm_tx, pcm_rx) = tokio::sync::mpsc::channel::>(50); let output_device_name = state - .audio_output_device + .huddle_audio + .output_device .lock() .unwrap_or_else(|e| e.into_inner()) .clone(); diff --git a/desktop/src-tauri/src/huddle/state.rs b/desktop/src-tauri/src/huddle/state.rs index 876c2d688b..37eb3533f6 100644 --- a/desktop/src-tauri/src/huddle/state.rs +++ b/desktop/src-tauri/src/huddle/state.rs @@ -5,7 +5,7 @@ use serde::{Deserialize, Serialize}; use std::sync::{ - atomic::{AtomicBool, AtomicU64}, + atomic::{AtomicBool, AtomicU64, Ordering}, Arc, Mutex, }; @@ -80,6 +80,15 @@ pub struct HuddleState { pub tts_enabled: bool, /// Whether STT transcript posting is enabled for this huddle. pub transcription_enabled: bool, + /// Whether the user has explicitly used the transcription control in this + /// huddle. Agent presence may auto-enable transcription only while this is + /// false, so membership refreshes never undo an explicit user choice. + /// + /// This is backend-only session state: keeping it in `HuddleState` makes it + /// survive frontend remounts and audio reconnects, while huddle teardown + /// resets it for the next session. + #[serde(skip)] + pub transcription_user_controlled: bool, /// Shared flag: true while TTS is playing audio. /// Shared with the STT pipeline for barge-in / echo gating. #[serde(skip)] @@ -103,6 +112,10 @@ pub struct HuddleState { /// Used to throttle the refresh in check_pipeline_hotstart to every 15 s. #[serde(skip)] pub last_agent_refresh: Option, + /// Monotonic identity for a local huddle lifetime. Unlike transcript + /// generation, this changes only when a new start/join attempt begins. + #[serde(skip)] + pub huddle_generation: u64, /// Session generation — incremented on every teardown. The transcription /// task captures this at spawn time and checks before each POST. If the /// generation has changed, the task silently drops the transcript. @@ -157,11 +170,13 @@ impl Clone for HuddleState { is_creator: self.is_creator, tts_enabled: self.tts_enabled, transcription_enabled: self.transcription_enabled, + transcription_user_controlled: self.transcription_user_controlled, tts_active: Arc::clone(&self.tts_active), tts_cancel: Arc::clone(&self.tts_cancel), tts_starting: Arc::clone(&self.tts_starting), stt_starting: Arc::clone(&self.stt_starting), last_agent_refresh: self.last_agent_refresh, + huddle_generation: self.huddle_generation, session_generation: Arc::clone(&self.session_generation), voice_input_mode: self.voice_input_mode.clone(), ptt_active: Arc::clone(&self.ptt_active), @@ -184,11 +199,13 @@ impl Default for HuddleState { is_creator: false, tts_enabled: true, transcription_enabled: false, + transcription_user_controlled: false, tts_active: Arc::new(AtomicBool::new(false)), tts_cancel: Arc::new(AtomicBool::new(false)), tts_starting: Arc::new(AtomicBool::new(false)), stt_starting: Arc::new(AtomicBool::new(false)), last_agent_refresh: None, + huddle_generation: 0, session_generation: Arc::new(AtomicU64::new(0)), voice_input_mode: VoiceInputMode::default(), ptt_active: Arc::new(AtomicBool::new(false)), @@ -197,13 +214,247 @@ impl Default for HuddleState { } impl HuddleState { + /// Begin a new local huddle lifetime and return its identity. + pub(crate) fn begin_huddle_lifetime(&mut self) -> u64 { + self.huddle_generation = self.huddle_generation.wrapping_add(1); + self.huddle_generation + } + + pub(crate) fn owns_huddle_lifetime(&self, huddle_generation: u64, phase: HuddlePhase) -> bool { + self.huddle_generation == huddle_generation && self.phase == phase + } + + /// Whether an async result still belongs to the active huddle that + /// initiated it. The channel id is the huddle-session identity; transcript + /// generation changes within the same huddle must not invalidate it. + pub(crate) fn is_current_huddle( + &self, + ephemeral_channel_id: &str, + huddle_generation: u64, + ) -> bool { + matches!(self.phase, HuddlePhase::Connected | HuddlePhase::Active) + && self.ephemeral_channel_id.as_deref() == Some(ephemeral_channel_id) + && self.huddle_generation == huddle_generation + } + + /// Whether an STT construction still belongs to the current transcript + /// generation within the active huddle. + pub(crate) fn is_current_transcription_generation( + &self, + ephemeral_channel_id: &str, + huddle_generation: u64, + session_generation: u64, + ) -> bool { + self.is_current_huddle(ephemeral_channel_id, huddle_generation) + && self.session_generation.load(Ordering::Acquire) == session_generation + } + + /// Invalidate in-flight transcription work and give the next constructor a + /// fresh sentinel that stale constructors cannot clear. + pub(crate) fn invalidate_transcription_pipeline(&mut self) { + self.session_generation.fetch_add(1, Ordering::Release); + self.stt_starting = Arc::new(AtomicBool::new(false)); + } + + /// Record an explicit transcription choice made through the existing user + /// control. Later agent membership refreshes must preserve this choice. + pub(crate) fn set_transcription_enabled_by_user(&mut self, enabled: bool) { + self.transcription_enabled = enabled; + self.transcription_user_controlled = true; + } + + /// Enable transcription when an agent is present and the user has not + /// explicitly chosen a transcription state for this huddle. + /// + /// Returns true only for the transition from disabled to enabled, allowing + /// callers to start models/pipelines and emit state exactly once. Removing + /// the last agent deliberately leaves the current state unchanged. + pub(crate) fn maybe_auto_enable_transcription_for_agents(&mut self) -> bool { + let has_agent = !self + .agent_pubkeys + .lock() + .unwrap_or_else(|e| e.into_inner()) + .is_empty(); + if has_agent && !self.transcription_user_controlled && !self.transcription_enabled { + self.transcription_enabled = true; + return true; + } + false + } + /// Reset to default state while preserving the session generation counter. /// Used by start_huddle rollback, join_huddle rollback, and teardown_huddle /// to invalidate in-flight transcription tasks without losing the generation. pub(crate) fn reset_preserving_generation(&mut self) { let gen = Arc::clone(&self.session_generation); + let huddle_generation = self.huddle_generation; + let tts_enabled = self.tts_enabled; *self = Self::default(); self.session_generation = gen; + self.huddle_generation = huddle_generation; + self.tts_enabled = tts_enabled; + } +} + +#[cfg(test)] +mod tests { + use std::sync::atomic::Ordering; + + use super::HuddleState; + + fn set_agents(state: &HuddleState, agents: &[&str]) { + *state + .agent_pubkeys + .lock() + .unwrap_or_else(|error| error.into_inner()) = + agents.iter().map(|agent| (*agent).to_owned()).collect(); + } + + #[test] + fn first_agent_auto_enables_transcription_once() { + let mut state = HuddleState::default(); + set_agents(&state, &["agent"]); + + assert!(state.maybe_auto_enable_transcription_for_agents()); + assert!(state.transcription_enabled); + assert!(!state.maybe_auto_enable_transcription_for_agents()); + } + + #[test] + fn explicit_user_disable_is_not_undone_by_agent_presence() { + let mut state = HuddleState::default(); + set_agents(&state, &["agent"]); + assert!(state.maybe_auto_enable_transcription_for_agents()); + + state.set_transcription_enabled_by_user(false); + + assert!(!state.maybe_auto_enable_transcription_for_agents()); + assert!(!state.transcription_enabled); + } + + #[test] + fn last_agent_leaving_preserves_current_transcription_state() { + let mut state = HuddleState::default(); + set_agents(&state, &["agent"]); + assert!(state.maybe_auto_enable_transcription_for_agents()); + + set_agents(&state, &[]); + + assert!(!state.maybe_auto_enable_transcription_for_agents()); + assert!(state.transcription_enabled); + } + + #[test] + fn clone_preserves_user_control_across_frontend_state_reads() { + let mut state = HuddleState::default(); + state.set_transcription_enabled_by_user(false); + + let mut clone = state.clone(); + set_agents(&clone, &["agent"]); + + assert!(clone.transcription_user_controlled); + assert!(!clone.maybe_auto_enable_transcription_for_agents()); + } + + #[test] + fn stale_huddle_identity_is_rejected_after_replacement() { + let mut state = HuddleState { + phase: super::HuddlePhase::Active, + ephemeral_channel_id: Some("huddle-a".to_owned()), + ..HuddleState::default() + }; + let huddle_generation = state.begin_huddle_lifetime(); + let generation = state.session_generation.load(Ordering::Acquire); + assert!(state.is_current_huddle("huddle-a", huddle_generation)); + assert!(state.is_current_transcription_generation( + "huddle-a", + huddle_generation, + generation + )); + + state.session_generation.fetch_add(1, Ordering::Release); + assert!(state.is_current_huddle("huddle-a", huddle_generation)); + assert!(!state.is_current_transcription_generation( + "huddle-a", + huddle_generation, + generation + )); + + state.ephemeral_channel_id = Some("huddle-b".to_owned()); + + assert!(!state.is_current_huddle("huddle-a", huddle_generation)); + } + + #[test] + fn same_channel_rejoin_gets_a_new_huddle_lifetime() { + let mut state = HuddleState { + phase: super::HuddlePhase::Active, + ephemeral_channel_id: Some("huddle".to_owned()), + ..HuddleState::default() + }; + let first_generation = state.begin_huddle_lifetime(); + state.reset_preserving_generation(); + state.phase = super::HuddlePhase::Active; + state.ephemeral_channel_id = Some("huddle".to_owned()); + let second_generation = state.begin_huddle_lifetime(); + + assert_ne!(first_generation, second_generation); + assert!(!state.is_current_huddle("huddle", first_generation)); + assert!(state.is_current_huddle("huddle", second_generation)); + } + + #[test] + fn superseded_create_cannot_commit_or_reset_replacement_lifetime() { + let mut state = HuddleState::default(); + let first_generation = state.begin_huddle_lifetime(); + state.phase = super::HuddlePhase::Creating; + assert!(state.owns_huddle_lifetime(first_generation, super::HuddlePhase::Creating)); + + state.reset_preserving_generation(); + let replacement_generation = state.begin_huddle_lifetime(); + state.phase = super::HuddlePhase::Creating; + + assert!(!state.owns_huddle_lifetime(first_generation, super::HuddlePhase::Creating)); + assert!(state.owns_huddle_lifetime(replacement_generation, super::HuddlePhase::Creating)); + } + + #[test] + fn superseded_join_cannot_commit_replacement_lifetime() { + let mut state = HuddleState::default(); + let first_generation = state.begin_huddle_lifetime(); + state.phase = super::HuddlePhase::Connecting; + + state.reset_preserving_generation(); + let replacement_generation = state.begin_huddle_lifetime(); + state.phase = super::HuddlePhase::Connecting; + + assert!(!state.owns_huddle_lifetime(first_generation, super::HuddlePhase::Connecting)); + assert!(state.owns_huddle_lifetime(replacement_generation, super::HuddlePhase::Connecting)); + } + + #[test] + fn teardown_preserves_installation_global_tts_preference() { + let mut state = HuddleState { + tts_enabled: false, + phase: super::HuddlePhase::Active, + ..HuddleState::default() + }; + state.reset_preserving_generation(); + assert!(!state.tts_enabled); + assert_eq!(state.phase, super::HuddlePhase::Idle); + } + + #[test] + fn stale_constructor_cannot_clear_replacement_sentinel() { + let mut state = HuddleState::default(); + let stale_sentinel = std::sync::Arc::clone(&state.stt_starting); + stale_sentinel.store(true, Ordering::Release); + + state.invalidate_transcription_pipeline(); + state.stt_starting.store(true, Ordering::Release); + stale_sentinel.store(false, Ordering::Release); + + assert!(state.stt_starting.load(Ordering::Acquire)); } } diff --git a/desktop/src-tauri/src/huddle/transcription.rs b/desktop/src-tauri/src/huddle/transcription.rs index 0d752c1de7..5962f57cf4 100644 --- a/desktop/src-tauri/src/huddle/transcription.rs +++ b/desktop/src-tauri/src/huddle/transcription.rs @@ -1,5 +1,3 @@ -use std::sync::atomic::Ordering; - use tauri::State; use crate::app_state::AppState; @@ -15,10 +13,12 @@ use super::{models, pipeline::maybe_start_stt_pipeline}; pub async fn start_stt_pipeline(state: State<'_, AppState>) -> Result<(), String> { let ephemeral_channel_id = { let mut hs = state.huddle()?; - hs.transcription_enabled = true; - hs.ephemeral_channel_id + let ephemeral_channel_id = hs + .ephemeral_channel_id .clone() - .ok_or("no active huddle — start or join a huddle first")? + .ok_or("no active huddle — start or join a huddle first")?; + hs.set_transcription_enabled_by_user(true); + ephemeral_channel_id }; match maybe_start_stt_pipeline(&state, &ephemeral_channel_id).await { @@ -41,14 +41,17 @@ pub async fn set_huddle_transcription_enabled( ) -> Result<(), String> { let (ephemeral_channel_id, old_stt) = { let mut hs = state.huddle()?; - hs.transcription_enabled = enabled; + let ephemeral_channel_id = hs + .ephemeral_channel_id + .clone() + .ok_or("no active huddle — start or join a huddle first")?; + hs.set_transcription_enabled_by_user(enabled); if enabled { - (hs.ephemeral_channel_id.clone(), None) + (ephemeral_channel_id, None) } else { - hs.session_generation.fetch_add(1, Ordering::Release); - hs.stt_starting.store(false, Ordering::Release); - (hs.ephemeral_channel_id.clone(), hs.stt_pipeline.take()) + hs.invalidate_transcription_pipeline(); + (ephemeral_channel_id, hs.stt_pipeline.take()) } }; @@ -58,12 +61,10 @@ pub async fn set_huddle_transcription_enabled( drop(old_stt); if enabled { - let eph_id = - ephemeral_channel_id.ok_or("no active huddle — start or join a huddle first")?; if let Some(manager) = models::global_model_manager() { manager.start_stt_download(state.http_client.clone()); } - if let Err(e) = maybe_start_stt_pipeline(&state, &eph_id).await { + if let Err(e) = maybe_start_stt_pipeline(&state, &ephemeral_channel_id).await { eprintln!("buzz-desktop: STT transcript start failed: {e}"); } } diff --git a/desktop/src-tauri/src/huddle/tts.rs b/desktop/src-tauri/src/huddle/tts.rs index 63a435cd8e..c03589f9fe 100644 --- a/desktop/src-tauri/src/huddle/tts.rs +++ b/desktop/src-tauri/src/huddle/tts.rs @@ -35,10 +35,11 @@ //! can gate microphone input while the agent is speaking. use std::{ + collections::VecDeque, num::NonZero, path::PathBuf, sync::{ - atomic::{AtomicBool, Ordering}, + atomic::{AtomicBool, AtomicU64, Ordering}, mpsc::{self, SyncSender}, Arc, Mutex, MutexGuard, PoisonError, }, @@ -46,9 +47,21 @@ use std::{ time::Duration, }; -use super::pocket::{load_text_to_speech, load_voice_style, SAMPLE_RATE, VOICE_FILE_EXT}; +use super::pocket::{ + load_text_to_speech, load_voice_style, DEFAULT_VOICE, SAMPLE_RATE, VOICE_FILE_EXT, +}; use super::preprocessing::{preprocess_for_tts, split_sentences}; +#[path = "tts_voice_transition.rs"] +mod voice_transition; +use voice_transition::*; +#[path = "tts_startup.rs"] +mod startup; +use startup::await_worker_startup; +#[path = "tts_audio.rs"] +mod audio; +use audio::*; + // ── Constants ───────────────────────────────────────────────────────────────── /// Maximum number of queued text items. @@ -56,15 +69,15 @@ use super::preprocessing::{preprocess_for_tts, split_sentences}; /// TTS can play it. Excess items are dropped with a warning. const TEXT_QUEUE_DEPTH: usize = 8; -/// How long the worker waits on the text channel before checking the shutdown flag. +/// How long the worker waits before checking the shutdown flag. const RECV_TIMEOUT: Duration = Duration::from_millis(100); - /// Poll interval of the barge-in monitor thread. Bounds flag-to-silence /// latency: a cancel is noticed within one tick, and rodio's internal /// `periodic_access` wrapper stops the in-flight source within a further /// ~5 ms — so playing audio dies ~15 ms after the flag is set, even while /// the worker is blocked inside `synth_chunk`. const MONITOR_TICK: Duration = Duration::from_millis(10); +const AUDIO_PRIME_TIMEOUT: Duration = Duration::from_secs(2); /// Pocket TTS is a one-step consistency model, not diffusion. Kept for API compat. const SYNTH_STEPS: usize = 1; @@ -73,9 +86,8 @@ const SYNTH_STEPS: usize = 1; /// /// Applied only at the *end* of each synthesised sentence to eliminate the /// click that would otherwise occur when a non-zero waveform terminates -/// abruptly. **No fade-in is applied** — see `apply_fade_out` for the -/// rationale and `examples/pocket_onset_probe.rs` for the measurement that -/// motivated removing the leading fade. +/// abruptly. **No fade-in is applied** — see `apply_fade_out` for why preserving +/// the leading waveform is important. const FADE_OUT_SAMPLES: usize = (SAMPLE_RATE as f64 * 0.008) as usize; /// Length of the zero-sample cushion prepended before each synthesized @@ -101,19 +113,17 @@ const SENTENCE_LEAD_IN_SAMPLES: usize = (SAMPLE_RATE as f64 * 0.020) as usize; /// names chunk stitching as the reliability lever). Our previous /// sentence-per-call path created ~2–4× more seams than upstream. /// -/// We don't ship the SentencePiece tokenizer, so 50 tokens is approximated -/// with a character budget. The bundled 4k-entry vocab averages ~4 chars per -/// token, but usage-weighted English text leans on short common tokens, so -/// the effective ratio is ~2–4 chars/token and 200 chars ≈ 60–100 tokens — -/// modestly above upstream's 50, deliberately: erring large means fewer -/// seams, and even ~100 tokens is far below the model's 500-LM-step (~40 s) -/// ceiling. Do not shrink this budget to chase an exact 50-token match. +/// This character budget performs only coarse sentence packing. The April +/// engine applies its SentencePiece tokenizer afterward and refines every +/// result at the bundle's exact 50-token boundary. const MAX_CHUNK_CHARS: usize = 200; /// Silence inserted between sentences by the TTS pipeline (seconds). /// Injected as a silent buffer between each synthesized sentence chunk. const INTER_SENTENCE_SILENCE: f32 = 0.1; +type WorkerControlState = (Arc, Arc, WorkerCancelSignals); + // ── Public pipeline handle ──────────────────────────────────────────────────── /// Handle to the running TTS pipeline. @@ -122,7 +132,7 @@ const INTER_SENTENCE_SILENCE: f32 = 0.1; #[derive(Debug)] pub struct TtsPipeline { /// Send preprocessed text into the pipeline. - text_tx: SyncSender, + text_tx: SyncSender, /// `true` while the agent is speaking. Shared with the STT pipeline for gating. #[allow(dead_code)] pub tts_active: Arc, @@ -132,38 +142,25 @@ pub struct TtsPipeline { /// Kept alive here so the Arc isn't dropped — the worker holds a clone. #[allow(dead_code)] cancel: Arc, - /// Voice name (e.g. "reference_sample"). Stored for future voice-switching support. - #[allow(dead_code)] - voice: String, + /// Internal cancellation used only for voice changes. Kept separate so a + /// concurrent human barge-in always clears every queued message. + voice_cancel: Arc, + /// Selected manifest voice. The worker reloads only the lightweight style + /// when this changes; the warmed Pocket engine and audio player stay alive. + voice: Arc>, + /// Tags messages so a voice change drops only pre-change queue entries. + voice_generation: Arc, + /// Completed after the worker drains pre-change text and installs the new style. + voice_change_ack: VoiceChangeAck, /// Worker thread handle — taken on drop to join cleanly. thread: Option>, } impl TtsPipeline { - /// Spawn the TTS pipeline thread using the default voice. - /// - /// `model_dir` must contain the Pocket TTS files declared by `huddle::models` - /// (the five ONNX sessions, the two JSON tables, and `.wav`). - /// - /// `tts_active` is set to `true` while audio is playing and `false` when idle. - /// Pass the same `Arc` to the STT pipeline to gate microphone input. + /// Spawn the TTS pipeline thread with a manifest-backed voice name. /// - /// `cancel` is the shared barge-in flag from `HuddleState.tts_cancel`. Pass the - /// same `Arc` to the STT pipeline so both sides reference the same flag for the - /// entire huddle session — no stale references after pipeline restarts. - pub fn new( - model_dir: PathBuf, - tts_active: Arc, - cancel: Arc, - output_device: Option, - ) -> Result { - use super::pocket::DEFAULT_VOICE; - Self::new_with_voice(model_dir, tts_active, cancel, DEFAULT_VOICE, output_device) - } - - /// Spawn the TTS pipeline thread with a specific voice name. Today only the - /// bundled default voice (see `pocket::DEFAULT_VOICE`) is shipped; other - /// names will surface a clear error from `load_voice_style`. + /// `cancel` is shared with STT for barge-in. The same handle survives voice + /// changes so the warmed Pocket engine is retained. pub fn new_with_voice( model_dir: PathBuf, tts_active: Arc, @@ -171,37 +168,56 @@ impl TtsPipeline { voice: &str, output_device: Option, ) -> Result { - let (text_tx, text_rx) = mpsc::sync_channel::(TEXT_QUEUE_DEPTH); + let (text_tx, text_rx) = mpsc::sync_channel::(TEXT_QUEUE_DEPTH); let shutdown = Arc::new(AtomicBool::new(false)); // cancel is passed in from HuddleState.tts_cancel — shared with STT for barge-in. let shutdown_worker = Arc::clone(&shutdown); let cancel_worker = Arc::clone(&cancel); + let voice_cancel = Arc::new(AtomicBool::new(false)); + let worker_voice_cancel = Arc::clone(&voice_cancel); let tts_active_worker = Arc::clone(&tts_active); - let voice_name = voice.to_string(); + let voice = Arc::new(Mutex::new(voice.to_string())); + let voice_worker = Arc::clone(&voice); + let voice_generation = Arc::new(AtomicU64::new(1)); + let worker_voice_generation = Arc::clone(&voice_generation); + let voice_change_ack = Arc::new(Mutex::new(None)); + let worker_voice_change_ack = Arc::clone(&voice_change_ack); let model_dir_worker = model_dir.clone(); + let (startup_tx, startup_rx) = mpsc::sync_channel(1); let handle = thread::Builder::new() .name("tts-worker".into()) .spawn(move || { tts_worker( model_dir_worker, - voice_name, + ( + voice_worker, + worker_voice_generation, + worker_voice_change_ack, + ), text_rx, - tts_active_worker, - shutdown_worker, - cancel_worker, + ( + tts_active_worker, + shutdown_worker, + (cancel_worker, worker_voice_cancel), + ), output_device, + startup_tx, ) }) .map_err(|e| format!("failed to spawn tts-worker thread: {e}"))?; + let handle = await_worker_startup(handle, startup_rx)?; Ok(Self { text_tx, tts_active, shutdown, cancel, - voice: voice.to_string(), + voice_cancel, + voice, + voice_generation, + voice_change_ack, thread: Some(handle), }) } @@ -211,14 +227,59 @@ impl TtsPipeline { /// Non-blocking. Returns `Err` if the queue is full (bounded at /// `TEXT_QUEUE_DEPTH`) — caller may log and discard. pub fn speak(&self, text: String) -> Result<(), String> { - self.text_tx.try_send(text).map_err(|e| { - eprintln!("buzz-desktop: TTS queue saturated, dropping message: {e}"); - format!("TTS queue full, dropping: {e}") - }) + self.text_tx + .try_send(QueuedText { + generation: self.voice_generation.load(Ordering::Acquire), + route_id: 0, + text, + }) + .map_err(|e| { + eprintln!("buzz-desktop: TTS queue saturated, dropping message: {e}"); + format!("TTS queue full, dropping: {e}") + }) + } + + /// Clone the bounded queue sender so callers can apply backpressure without + /// holding the huddle mutex. Disabling TTS drops the receiver and unblocks + /// any waiting sender while the shared cancellation flag stops playback. + pub(crate) fn text_sender(&self) -> TtsTextSender { + TtsTextSender { + text_tx: self.text_tx.clone(), + generation: self.voice_generation.load(Ordering::Acquire), + } + } + + /// Select a bundled Pocket voice for subsequent speech. + /// + /// Current playback and queued text are cancelled immediately so content + /// cannot continue in the old voice. The worker keeps its warmed inference + /// engine and reloads only the reference style before the next utterance. + pub fn select_voice(&self, voice: &str) -> Option> { + let acknowledged = begin_voice_change( + &self.voice, + &self.voice_generation, + &self.voice_cancel, + &self.voice_change_ack, + voice, + ); + if acknowledged.is_some() { + eprintln!("buzz-desktop: tts stage=cancellation reason=voice_switch route_id=0"); + } + acknowledged + } + + /// Reconcile the voice of a pipeline that has not been published yet. + /// + /// No caller can enqueue text before publication, so raising the shared + /// cancellation flag here would create a race that could discard the first + /// message queued immediately after installation. + pub(crate) fn select_voice_before_publish(&self, voice: &str) { + *self.voice.lock().unwrap_or_else(|error| error.into_inner()) = voice.to_string(); } /// Signal the worker thread to stop. pub fn shutdown(&self) { + eprintln!("buzz-desktop: tts stage=cancellation reason=shutdown route_id=0"); self.shutdown.store(true, Ordering::Release); } @@ -244,40 +305,52 @@ impl Drop for TtsPipeline { fn tts_worker( model_dir: PathBuf, - voice_name: String, - text_rx: mpsc::Receiver, - tts_active: Arc, - shutdown: Arc, - cancel: Arc, + voice_state: WorkerVoiceState, + text_rx: mpsc::Receiver, + control_state: WorkerControlState, output_device: Option, + startup_tx: mpsc::SyncSender>, ) { + let (selected_voice, voice_generation, voice_change_ack) = voice_state; + let (tts_active, shutdown, cancel_signals) = control_state; + let (cancel, voice_cancel) = cancel_signals; // ── 1. Initialise TTS engine ────────────────────────────────────────────── let model_dir_str = model_dir.to_string_lossy().to_string(); let engine = match load_text_to_speech(&model_dir_str) { Ok(e) => e, Err(e) => { - eprintln!( - "buzz-desktop: TTS engine init failed (model_dir={}): {e}. TTS disabled.", - model_dir.display() - ); - drain_until_shutdown(text_rx, &shutdown); + let error = format!("TTS engine initialization failed: {e}"); + eprintln!("buzz-desktop: tts stage=startup status=failed reason=engine_load"); + let _ = startup_tx.send(Err(error)); return; } }; // ── 2. Load voice style ─────────────────────────────────────────────────── - let voice_path = model_dir.join(format!("{voice_name}.{VOICE_FILE_EXT}")); - let style = match load_voice_style(&voice_path) { + let requested_voice = selected_voice + .lock() + .unwrap_or_else(|error| error.into_inner()) + .clone(); + let mut voice_name = DEFAULT_VOICE.to_string(); + let fallback_path = model_dir.join(format!("{DEFAULT_VOICE}.{VOICE_FILE_EXT}")); + let mut style = match load_voice_style(&fallback_path) { Ok(s) => s, Err(e) => { - eprintln!( - "buzz-desktop: TTS voice style load failed ({voice_name}): {e}. TTS disabled." - ); - drain_until_shutdown(text_rx, &shutdown); + let error = format!("TTS voice style initialization failed: {e}"); + eprintln!("buzz-desktop: tts stage=startup status=failed reason=fallback_voice_style"); + let _ = startup_tx.send(Err(error)); return; } }; + if requested_voice != DEFAULT_VOICE + && !reconcile_selected_voice(&model_dir, &selected_voice, &mut voice_name, &mut style) + { + let _ = startup_tx.send(Err( + "TTS selected voice and Mary fallback are unavailable".to_string() + )); + return; + } // ── 2b. Warmup inference ───────────────────────────────────────────────── // The first ONNX inference on any session is significantly slower than @@ -285,15 +358,10 @@ fn tts_worker( // pool allocation, and graph-specific caches. Run a short dummy synthesis // and discard the output so the first real utterance runs at warm-session speed. { - let t = std::time::Instant::now(); match engine.synth_chunk("warmup", "en", &style, SYNTH_STEPS) { - Ok(_) => eprintln!( - "buzz-desktop: TTS warmup completed in {:.0}ms", - t.elapsed().as_millis() - ), - Err(e) => eprintln!( - "buzz-desktop: TTS warmup failed after {:.0}ms: {e} — first utterance may be slow", - t.elapsed().as_millis() + Ok(_) => eprintln!("buzz-desktop: tts stage=warmup status=ready"), + Err(_) => eprintln!( + "buzz-desktop: tts stage=warmup status=failed reason=inference first_utterance_may_be_slow=true" ), } } @@ -306,8 +374,9 @@ fn tts_worker( { Ok(h) => h, Err(e) => { - eprintln!("buzz-desktop: TTS audio output failed: {e}. TTS disabled."); - drain_until_shutdown(text_rx, &shutdown); + let error = format!("TTS audio output initialization failed: {e}"); + eprintln!("buzz-desktop: tts stage=startup status=failed reason=output_open"); + let _ = startup_tx.send(Err(error)); return; } }; @@ -315,14 +384,14 @@ fn tts_worker( let channels = match NonZero::new(1u16) { Some(c) => c, None => { - eprintln!("buzz-desktop: TTS channel count invariant violated"); + let _ = startup_tx.send(Err("TTS channel count invariant violated".to_string())); return; } }; let rate = match NonZero::new(SAMPLE_RATE) { Some(r) => r, None => { - eprintln!("buzz-desktop: TTS sample rate invariant violated"); + let _ = startup_tx.send(Err("TTS sample rate invariant violated".to_string())); return; } }; @@ -346,10 +415,22 @@ fn tts_worker( player.append(SamplesBuffer::new(channels, rate, silence)); // Wait for the silent buffer to drain — this ensures the output stream // is fully initialized before the first real utterance. + let deadline = std::time::Instant::now() + AUDIO_PRIME_TIMEOUT; while !player.empty() { + if std::time::Instant::now() >= deadline { + eprintln!("buzz-desktop: tts stage=startup status=failed reason=output_prime"); + let _ = startup_tx.send(Err( + "TTS audio output did not become ready before timeout".to_string(), + )); + return; + } thread::sleep(Duration::from_millis(10)); } } + if startup_tx.send(Ok(())).is_err() { + return; + } + eprintln!("buzz-desktop: tts stage=startup status=ready"); // ── 3b. Barge-in monitor thread ─────────────────────────────────────────── // @@ -377,6 +458,7 @@ fn tts_worker( let monitor = { let player = Arc::clone(&player); let cancel = Arc::clone(&cancel); + let voice_cancel = Arc::clone(&voice_cancel); let tts_active = Arc::clone(&tts_active); let stop = Arc::clone(&monitor_stop); let player_ops = Arc::clone(&player_ops); @@ -384,12 +466,12 @@ fn tts_worker( .name("tts-barge-in-monitor".into()) .spawn(move || { while !stop.load(Ordering::Acquire) { - if cancel.load(Ordering::Acquire) { + if cancel.load(Ordering::Acquire) || voice_cancel.load(Ordering::Acquire) { let _ops = lock_player_ops(&player_ops); // Re-check under the lock: the worker may have // consumed this cancel (and appended fresh audio) // between the load above and the lock acquisition. - if cancel.load(Ordering::Acquire) { + if cancel.load(Ordering::Acquire) || voice_cancel.load(Ordering::Acquire) { // clear() pauses the persistent player; play() // un-pauses (see handle_cancel_or_shutdown). // Idempotent — safe to repeat every tick until @@ -423,13 +505,45 @@ fn tts_worker( // idle branch below uses it to decide when to drop `tts_active` and to // arm a fresh lead-in cushion for the next utterance. let mut first_append = true; + let mut last_route_id = 0; + let mut deferred_text = VecDeque::new(); + let append_audio = |prepared: PreparedModelAudio, route_id: u64| { + let _ops = lock_player_ops(&player_ops); + if cancel.load(Ordering::Acquire) + || voice_cancel.load(Ordering::Acquire) + || shutdown.load(Ordering::Acquire) + { + let reason = if shutdown.load(Ordering::Acquire) { + "shutdown" + } else if cancel.load(Ordering::Acquire) { + "barge_in" + } else { + "voice_switch" + }; + eprintln!( + "buzz-desktop: tts stage=synthesis status=cancelled reason={reason} route_id={route_id}" + ); + return false; + } + player.append(SamplesBuffer::new(channels, rate, prepared.buffer)); + eprintln!( + "buzz-desktop: tts stage=player status=append_accepted route_id={route_id} chunk_index={} sample_count={}", + prepared.chunk_index, prepared.sample_count + ); + // Set this only after append so STT remains open during synthesis. + tts_active.store(true, Ordering::Release); + true + }; loop { + let mut no_current_text = None; if handle_cancel_or_shutdown( - &cancel, + (&cancel, &voice_cancel), &shutdown, &tts_active, - &text_rx, + (&text_rx, &mut deferred_text, &mut no_current_text), + &voice_change_ack, + None, Some((&player, &player_ops)), ) { if shutdown.load(Ordering::Acquire) { @@ -441,28 +555,47 @@ fn tts_worker( continue; } - let raw_text = match text_rx.recv_timeout(RECV_TIMEOUT) { - Ok(t) => t, - Err(mpsc::RecvTimeoutError::Timeout) => { - // Nothing queued. If playback has also finished, the agent - // has gone quiet — release the mic gate and reset the - // lead-in so the next utterance gets a fresh cushion. - if player.empty() && !first_append { - tts_active.store(false, Ordering::Release); - first_append = true; + // Voice changes cancel the old utterance/queue and are observed here, + // before receiving subsequent text. A bad bundled asset falls back to + // Mary without discarding the already-warmed Pocket engine. + let voice_ready = + reconcile_selected_voice(&model_dir, &selected_voice, &mut voice_name, &mut style); + acknowledge_voice_change(&voice_change_ack, &voice_cancel); + if !voice_ready { + continue; + } + + let mut queued_text = Some(match deferred_text.pop_front() { + Some(text) => text, + None => match text_rx.recv_timeout(RECV_TIMEOUT) { + Ok(text) => text, + Err(mpsc::RecvTimeoutError::Timeout) => { + // Nothing queued. If playback has also finished, the agent + // has gone quiet — release the mic gate and reset the + // lead-in so the next utterance gets a fresh cushion. + if player.empty() && !first_append { + tts_active.store(false, Ordering::Release); + eprintln!( + "buzz-desktop: tts stage=player status=drained route_id={last_route_id}" + ); + first_append = true; + } + continue; } - continue; - } - Err(mpsc::RecvTimeoutError::Disconnected) => break, - }; + Err(mpsc::RecvTimeoutError::Disconnected) => break, + }, + }); // Check cancel again after unblocking — a cancel may have arrived // while we were waiting. + let pending_route_id = queued_text.as_ref().map(|queued| queued.route_id); if handle_cancel_or_shutdown( - &cancel, + (&cancel, &voice_cancel), &shutdown, &tts_active, - &text_rx, + (&text_rx, &mut deferred_text, &mut queued_text), + &voice_change_ack, + pending_route_id, Some((&player, &player_ops)), ) { if shutdown.load(Ordering::Acquire) { @@ -471,6 +604,30 @@ fn tts_worker( first_append = true; continue; } + let Some(queued_text) = queued_text else { + continue; + }; + if queued_text.generation < voice_generation.load(Ordering::Acquire) { + eprintln!( + "buzz-desktop: tts stage=queue status=dropped reason=voice_switch route_id={}", + queued_text.route_id + ); + continue; + } + let raw_text = queued_text.text; + let route_id = queued_text.route_id; + eprintln!("buzz-desktop: tts stage=synthesis status=started route_id={route_id}"); + + // The selected voice can change while this worker is blocked in + // recv_timeout. Reconcile again after receipt so the first message + // queued after an unpublished pipeline is installed cannot use the + // voice captured when construction began. + if !reconcile_selected_voice(&model_dir, &selected_voice, &mut voice_name, &mut style) { + eprintln!( + "buzz-desktop: tts stage=synthesis status=failed reason=voice_unavailable route_id={route_id}" + ); + continue; + } // If playback already drained while we were waiting for this item, // the agent is silent — release the mic gate BEFORE preprocessing/ @@ -482,37 +639,53 @@ fn tts_worker( // stays set across items.) if player.empty() && !first_append { tts_active.store(false, Ordering::Release); + eprintln!("buzz-desktop: tts stage=player status=drained route_id={last_route_id}"); first_append = true; } // Preprocess text. let text = preprocess_for_tts(&raw_text); if text.is_empty() { + eprintln!( + "buzz-desktop: tts stage=synthesis status=empty reason=preprocess route_id={route_id}" + ); continue; } // Split into sentences, then group into synthesis chunks: the first // sentence stays alone (fast time-to-first-audio), the rest pack - // greedily up to MAX_CHUNK_CHARS. Each chunk is one `generate()` - // call; playback of chunk N overlaps synthesis of chunk N+1 - // (lookahead pipelining). Grouping matches upstream's ~50-token - // chunking and halves the exposed prosody seams on multi-sentence - // replies — see MAX_CHUNK_CHARS. + // greedily up to MAX_CHUNK_CHARS. Playback of each model unit overlaps + // synthesis of the next one. The Pocket engine applies its exact + // 50-token split; keeping those units within one playback chunk avoids + // adding fades and pauses at token-only boundaries. let sentences: Vec = split_sentences(&text) .into_iter() .filter(|s| !s.trim().is_empty()) .collect(); let chunks = group_sentences_into_chunks(&sentences, MAX_CHUNK_CHARS); + if chunks.is_empty() { + eprintln!( + "buzz-desktop: tts stage=synthesis status=empty reason=no_chunks route_id={route_id}" + ); + continue; + } - for chunk in &chunks { + let mut synthesis_outcome = "completed"; + let mut appended_audio = false; + let mut model_unit_index = 0_usize; + 'playback_chunks: for chunk in &chunks { + let mut no_current_text = None; if handle_cancel_or_shutdown( - &cancel, + (&cancel, &voice_cancel), &shutdown, &tts_active, - &text_rx, + (&text_rx, &mut deferred_text, &mut no_current_text), + &voice_change_ack, + Some(route_id), Some((&player, &player_ops)), ) { first_append = true; + synthesis_outcome = "cancelled"; break; } @@ -521,53 +694,113 @@ fn tts_worker( continue; } - match engine.synth_chunk(text, "en", &style, SYNTH_STEPS) { - Ok(samples) if !samples.is_empty() => { - let mut audio = clamp_to_full_scale(samples); - // Fade-out only — fading-in would attenuate the consonant - // onset (see `apply_fade_out` docstring + the - // 2026-05-18 "first little sound is missing" regression). - apply_fade_out(&mut audio); - - // Build one contiguous buffer per synthesized sentence: - // lead-in cushion + audio + trailing gap. Keeping this as - // a single rodio source preserves the original queue/drain - // semantics (one append per sentence) while still giving - // every chunk a quiet device warm-up window. - let buf = - build_sentence_append_buffer(&mut first_append, audio, silence_buf_len); - - // Check-and-append under `player_ops`, serialized with - // the monitor: a barge-in may have arrived during - // synthesis (the blocking window the monitor thread - // exists for). Don't append the now-stale sentence — the - // human interrupted; speaking it anyway would talk over - // them. Holding the lock for the check + append means the - // monitor can never clear between our check passing and - // the buffer landing. The flag is deliberately NOT - // consumed here: the loop-top handle_cancel_or_shutdown - // does the full consume (drain queue, reset lead-in) on - // the next iteration. - let _ops = lock_player_ops(&player_ops); - if cancel.load(Ordering::Acquire) { - // Nothing appended; the loop-top consume re-arms - // `first_append` (the flag is still set — the worker - // is its only consumer). + let model_chunks = match engine.split_text_into_chunks(text) { + Ok(model_chunks) => model_chunks, + Err(_) => { + eprintln!( + "buzz-desktop: tts stage=synthesis status=failed reason=chunking route_id={route_id}" + ); + synthesis_outcome = "failed"; + break 'playback_chunks; + } + }; + if model_chunks.is_empty() { + eprintln!( + "buzz-desktop: tts stage=synthesis status=empty reason=no_chunks route_id={route_id}" + ); + continue; + } + let mut playback_audio = PlaybackChunkAudio::new(); + for model_chunk in &model_chunks { + let chunk_index = model_unit_index; + model_unit_index += 1; + let mut no_current_text = None; + if handle_cancel_or_shutdown( + (&cancel, &voice_cancel), + &shutdown, + &tts_active, + (&text_rx, &mut deferred_text, &mut no_current_text), + &voice_change_ack, + Some(route_id), + Some((&player, &player_ops)), + ) { + first_append = true; + synthesis_outcome = "cancelled"; + break 'playback_chunks; + } + + let synthesis = engine.synth_chunk(model_chunk, "en", &style, SYNTH_STEPS); + if cancel.load(Ordering::Acquire) + || voice_cancel.load(Ordering::Acquire) + || shutdown.load(Ordering::Acquire) + { + let reason = if shutdown.load(Ordering::Acquire) { + "shutdown" + } else if cancel.load(Ordering::Acquire) { + "barge_in" + } else { + "voice_switch" + }; + eprintln!( + "buzz-desktop: tts stage=synthesis status=cancelled reason={reason} route_id={route_id}" + ); + // The monitor already stopped any queued playback. Discard + // synthesis that completed after cancellation so stale audio + // never reaches the player, while keeping buzz-voice's + // extracted April engine API unchanged. + first_append = true; + synthesis_outcome = "cancelled"; + break 'playback_chunks; + } + match synthesis { + Ok(samples) if !samples.is_empty() => { + if let Some(prepared) = playback_audio.push( + samples, + chunk_index, + &mut first_append, + silence_buf_len, + player.empty(), + ) { + if !append_audio(prepared, route_id) { + first_append = true; + synthesis_outcome = "cancelled"; + break 'playback_chunks; + } + appended_audio = true; + last_route_id = route_id; + } + } + Ok(_) => { + eprintln!( + "buzz-desktop: tts stage=synthesis status=empty route_id={route_id} chunk_index={chunk_index}" + ); + } + Err(_) => { + eprintln!( + "buzz-desktop: tts stage=synthesis status=failed reason=inference route_id={route_id} chunk_index={chunk_index}" + ); + synthesis_outcome = "failed"; break; } - player.append(SamplesBuffer::new(channels, rate, buf)); - // NOTE: tts_active is set AFTER player.append(), not - // before. Setting it before synthesis would cause STT to - // discard user speech during the synthesis window as - // "echo" even though no audio is actually playing yet. - // See crossfire review C3. - tts_active.store(true, Ordering::Release); } - Ok(_) => {} - Err(e) => { - eprintln!("buzz-desktop: TTS synth failed: {e}"); + } + if let Some(prepared) = + playback_audio.finish(&mut first_append, silence_buf_len, player.empty()) + { + if !append_audio(prepared, route_id) { + first_append = true; + synthesis_outcome = "cancelled"; + break 'playback_chunks; } + appended_audio = true; + last_route_id = route_id; } + if synthesis_outcome == "failed" { + break 'playback_chunks; + } + } + if synthesis_outcome == "completed" && appended_audio { + eprintln!("buzz-desktop: tts stage=synthesis status=completed route_id={route_id}"); } if shutdown.load(Ordering::Acquire) { @@ -582,6 +815,7 @@ fn tts_worker( let _ = handle.join(); } + finish_voice_change_ack(&voice_change_ack); tts_active.store(false, Ordering::Release); } @@ -595,13 +829,21 @@ fn tts_worker( /// it is serialized with the monitor's stale-branch re-check (see the monitor /// block in `tts_worker`). fn handle_cancel_or_shutdown( - cancel: &AtomicBool, + cancel_signals: CancelSignals<'_>, shutdown: &AtomicBool, tts_active: &AtomicBool, - text_rx: &mpsc::Receiver, + text_state: CancelTextState<'_>, + voice_change_ack: &VoiceChangeAck, + active_route_id: Option, player: Option<(&rodio::Player, &Mutex<()>)>, ) -> bool { + let (cancel, voice_cancel) = cancel_signals; + let (text_rx, deferred_text, current_text) = text_state; if shutdown.load(Ordering::Acquire) { + eprintln!( + "buzz-desktop: tts stage=cancellation reason=shutdown route_id={}", + active_route_id.unwrap_or(0) + ); if let Some((p, ops)) = player { let _ops = lock_player_ops(ops); p.clear(); @@ -609,7 +851,29 @@ fn handle_cancel_or_shutdown( tts_active.store(false, Ordering::Release); return true; } - if cancel.load(Ordering::Acquire) { + if cancel.load(Ordering::Acquire) || voice_cancel.load(Ordering::Acquire) { + // Serialize with begin_voice_change so the generation boundary and + // cancel consumption are observed as one transition. + let pending_voice_change = voice_change_ack + .lock() + .unwrap_or_else(|error| error.into_inner()); + // Consume at the serialization point. A later barge-in remains true + // for the next pass instead of being overwritten after queue cleanup. + let barge_in = cancel.swap(false, Ordering::AcqRel); + voice_cancel.store(false, Ordering::Release); + eprintln!( + "buzz-desktop: tts stage=cancellation reason={} route_id={}", + if barge_in { "barge_in" } else { "voice_switch" }, + active_route_id.unwrap_or(0) + ); + let preserve_generation = (!barge_in) + .then(|| { + pending_voice_change + .as_ref() + .map(|pending| pending.generation) + }) + .flatten(); + retain_cancelled_text(deferred_text, current_text, text_rx, preserve_generation); if let Some((p, ops)) = player { let _ops = lock_player_ops(ops); // `Player::clear()` removes queued sources AND pauses the player @@ -622,11 +886,6 @@ fn handle_cancel_or_shutdown( // Consume the flag under the lock: once released with // `cancel == false`, the monitor's stale branch no-ops instead // of clearing the fresh post-cancel utterance. - while text_rx.try_recv().is_ok() {} - cancel.store(false, Ordering::Release); - } else { - while text_rx.try_recv().is_ok() {} - cancel.store(false, Ordering::Release); } tts_active.store(false, Ordering::Release); return true; @@ -644,135 +903,11 @@ fn lock_player_ops(ops: &Mutex<()>) -> MutexGuard<'_, ()> { ops.lock().unwrap_or_else(PoisonError::into_inner) } -/// Hard-clamp samples to ±1.0 full scale. -/// -/// No gain is applied: Pocket TTS already emits speech-level audio -/// (peaks 0.4–0.97, RMS ≈ −20 dBFS across varied sentences — measured by -/// `examples/pocket_clip_probe`), matching the kyutai reference pipeline, -/// which applies no output scaling. Two earlier gain stages were both -/// regressions against that baseline: per-sentence peak normalization caused -/// level pumping between sentences, and the fixed 9.3× gain that replaced it -/// was calibrated on a single anomalously-quiet bench utterance (peak 0.076) -/// and clipped 13–34% of samples on real speech ("blown out", 2026-06-12). -/// The clamp alone remains as the safety net against outlier transients. -fn clamp_to_full_scale(samples: Vec) -> Vec { - samples.into_iter().map(|s| s.clamp(-1.0, 1.0)).collect() -} - -/// Apply a short linear fade-out at the *end* of `samples`. -/// -/// Uses `FADE_OUT_SAMPLES` (8 ms) or half the buffer length, whichever is -/// smaller. Eliminates the click that occurs when a non-zero waveform -/// terminates abruptly at a sentence boundary. -/// -/// # Why no fade-in -/// -/// An earlier revision (pre 2026-05) symmetrically faded *in* over the same -/// 8 ms window. That swallowed the leading consonant attack on every -/// sentence — Pocket TTS produces real audio energy inside the first -/// millisecond (RMS ≈ 0.02, peak ≈ 0.03 measured across four prompts in -/// `examples/pocket_onset_probe.rs`), and a linear 0→1 ramp over 192 samples -/// scales those onset samples by ≤50 % for the first ~4 ms. The result was -/// the "first little sound or two is missing" regression heard on -/// 2026-05-18. -/// -/// The first sample of Pocket output measures ≈ 0.0018 (≈ −54 dBFS) — well -/// below the threshold at which a DC-jump would be audible as a click — so -/// no fade-in is needed. The OS audio device gets its quiet ramp-up window -/// from `SENTENCE_LEAD_IN_SAMPLES` instead, inserted as pure silence before -/// each sentence buffer. -fn apply_fade_out(samples: &mut [f32]) { - let len = samples.len(); - let fade = FADE_OUT_SAMPLES.min(len / 2); - for i in 0..fade { - samples[len - 1 - i] *= i as f32 / fade as f32; - } -} - -/// Build the single buffer appended to the rodio `Player` for one synthesised -/// sentence. -/// -/// Every sentence chunk gets a short lead-in pad immediately before its audio. -/// This matters for chunks that start with soft first phonemes (`I'm`, `I've`): -/// the synthesized buffer can begin with speech within the first millisecond, -/// so the playback layer must provide the device/mixer cushion. -/// To keep the audible gap unchanged, the trailing silence after this chunk is -/// shortened by the same amount (`silence_buf_len - SENTENCE_LEAD_IN_SAMPLES`): -/// sentence N contributes 80 ms of post-speech silence and sentence N+1 -/// contributes the remaining 20 ms of pre-speech cushion. -/// -/// The lead-in, audio, and trailing silence are concatenated into one -/// `SamplesBuffer` before appending. This keeps rodio's queue shape at one -/// tracked source per synthesized sentence, avoiding source-boundary/drain -/// regressions from enqueueing the lead-in, audio, and tail as separate sounds. -/// -/// `first_append` is flipped on the first call after the player goes idle. -/// The worker uses it in the idle branch of the main loop to distinguish -/// "never queued anything since last drain" from "drained after speaking", -/// which controls when `tts_active` is released and the lead-in re-armed. -fn build_sentence_append_buffer( - first_append: &mut bool, - audio: Vec, - silence_buf_len: usize, -) -> Vec { - if *first_append { - *first_append = false; - } - - let trailing_silence_len = silence_buf_len.saturating_sub(SENTENCE_LEAD_IN_SAMPLES); - let mut buf = Vec::with_capacity(SENTENCE_LEAD_IN_SAMPLES + audio.len() + trailing_silence_len); - buf.extend(std::iter::repeat_n(0.0_f32, SENTENCE_LEAD_IN_SAMPLES)); - buf.extend(audio); - buf.extend(std::iter::repeat_n(0.0_f32, trailing_silence_len)); - buf -} - -/// Group sentences into synthesis chunks. -/// -/// The first sentence always stands alone — it is what the listener hears -/// first, and synthesizing it by itself keeps time-to-first-audio at the -/// single-sentence cost. Subsequent sentences pack greedily: a sentence -/// joins the current chunk while the combined length stays within -/// `max_chars`; otherwise it starts a new chunk. A single sentence longer -/// than `max_chars` becomes its own chunk unsplit — Pocket TTS handles long -/// single sentences fine (the ceiling is the 500-LM-step default), it's the -/// *seams* we're minimizing. -/// -/// Sentences within a chunk are joined with a single space; sentence-ending -/// punctuation is preserved by `split_sentences`, so the model sees natural -/// multi-sentence prose — the same shape upstream's ~50-token chunker feeds it. -fn group_sentences_into_chunks(sentences: &[String], max_chars: usize) -> Vec { - let mut chunks: Vec = Vec::new(); - for (i, sentence) in sentences.iter().enumerate() { - let sentence = sentence.trim(); - if sentence.is_empty() { - continue; - } - if i == 0 || chunks.is_empty() { - chunks.push(sentence.to_string()); - continue; - } - // Never merge into the first chunk — it's the latency-critical one. - let can_merge = chunks.len() > 1 - && chunks - .last() - .is_some_and(|c| c.len() + 1 + sentence.len() <= max_chars); - if can_merge { - let last = chunks.last_mut().expect("non-empty checked above"); - last.push(' '); - last.push_str(sentence); - } else { - chunks.push(sentence.to_string()); - } - } - chunks -} - -// drain_until_shutdown lives in super (huddle/mod.rs) — shared with stt.rs. -use super::drain_until_shutdown; - // ── Tests ───────────────────────────────────────────────────────────────────── #[cfg(test)] #[path = "tts_tests.rs"] mod tests; +#[cfg(test)] +#[path = "tts_voice_selection_tests.rs"] +mod voice_selection_tests; diff --git a/desktop/src-tauri/src/huddle/tts_audio.rs b/desktop/src-tauri/src/huddle/tts_audio.rs new file mode 100644 index 0000000000..58300b7497 --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_audio.rs @@ -0,0 +1,235 @@ +use super::{FADE_OUT_SAMPLES, SENTENCE_LEAD_IN_SAMPLES}; + +pub(super) struct PreparedModelAudio { + pub(super) buffer: Vec, + pub(super) sample_count: usize, + pub(super) chunk_index: usize, +} + +/// Holds one synthesized model unit so playback-boundary decoration is based +/// on the first and last unit that actually produced audio. +pub(super) struct PlaybackChunkAudio { + pending: Option<(Vec, usize)>, + appended: bool, +} + +impl PlaybackChunkAudio { + pub(super) fn new() -> Self { + Self { + pending: None, + appended: false, + } + } + + pub(super) fn push( + &mut self, + samples: Vec, + chunk_index: usize, + first_append: &mut bool, + silence_buf_len: usize, + playback_idle: bool, + ) -> Option { + if samples.is_empty() { + return None; + } + let previous = self.pending.replace((samples, chunk_index))?; + let prepared = prepare_model_audio( + previous, + first_append, + silence_buf_len, + !self.appended || playback_idle, + false, + ); + self.appended = true; + Some(prepared) + } + + pub(super) fn finish( + &mut self, + first_append: &mut bool, + silence_buf_len: usize, + playback_idle: bool, + ) -> Option { + let pending = self.pending.take()?; + Some(prepare_model_audio( + pending, + first_append, + silence_buf_len, + !self.appended || playback_idle, + true, + )) + } +} + +fn prepare_model_audio( + (samples, chunk_index): (Vec, usize), + first_append: &mut bool, + silence_buf_len: usize, + starts_playback_chunk: bool, + ends_playback_chunk: bool, +) -> PreparedModelAudio { + let sample_count = samples.len(); + let mut audio = clamp_to_full_scale(samples); + if ends_playback_chunk { + apply_fade_out(&mut audio); + } + PreparedModelAudio { + buffer: build_sentence_append_buffer( + first_append, + audio, + silence_buf_len, + starts_playback_chunk, + ends_playback_chunk, + ), + sample_count, + chunk_index, + } +} + +/// Hard-clamp samples to ±1.0 full scale. +pub(super) fn clamp_to_full_scale(samples: Vec) -> Vec { + samples.into_iter().map(|s| s.clamp(-1.0, 1.0)).collect() +} + +/// Apply a short linear fade-out to avoid a discontinuity at playback boundaries. +pub(super) fn apply_fade_out(samples: &mut [f32]) { + let len = samples.len(); + let fade = FADE_OUT_SAMPLES.min(len / 2); + for i in 0..fade { + samples[len - 1 - i] *= i as f32 / fade as f32; + } +} + +pub(super) fn build_sentence_append_buffer( + first_append: &mut bool, + audio: Vec, + silence_buf_len: usize, + starts_playback_chunk: bool, + ends_playback_chunk: bool, +) -> Vec { + if *first_append { + *first_append = false; + } + + let lead_in_len = if starts_playback_chunk { + SENTENCE_LEAD_IN_SAMPLES + } else { + 0 + }; + let trailing_silence_len = if ends_playback_chunk { + silence_buf_len.saturating_sub(SENTENCE_LEAD_IN_SAMPLES) + } else { + 0 + }; + let mut buffer = Vec::with_capacity(lead_in_len + audio.len() + trailing_silence_len); + buffer.extend(std::iter::repeat_n(0.0_f32, lead_in_len)); + buffer.extend(audio); + buffer.extend(std::iter::repeat_n(0.0_f32, trailing_silence_len)); + buffer +} + +pub(super) fn group_sentences_into_chunks(sentences: &[String], max_chars: usize) -> Vec { + let mut chunks: Vec = Vec::new(); + for (index, sentence) in sentences.iter().enumerate() { + let sentence = sentence.trim(); + if sentence.is_empty() { + continue; + } + if index == 0 || chunks.is_empty() { + chunks.push(sentence.to_string()); + continue; + } + let can_merge = chunks.len() > 1 + && chunks + .last() + .is_some_and(|chunk| chunk.len() + 1 + sentence.len() <= max_chars); + if can_merge { + if let Some(last) = chunks.last_mut() { + last.push(' '); + last.push_str(sentence); + } + } else { + chunks.push(sentence.to_string()); + } + } + chunks +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn multi_unit_audio_decorates_only_outer_playback_boundaries() { + let mut chunk = PlaybackChunkAudio::new(); + let mut first_append = true; + let silence = SENTENCE_LEAD_IN_SAMPLES + 100; + + assert!(chunk + .push(vec![0.4; 16], 0, &mut first_append, silence, false) + .is_none()); + let first = chunk + .push(vec![0.5; 16], 1, &mut first_append, silence, false) + .expect("first ready model unit"); + assert_eq!(first.buffer.len(), SENTENCE_LEAD_IN_SAMPLES + 16); + assert!(first.buffer[..SENTENCE_LEAD_IN_SAMPLES] + .iter() + .all(|sample| *sample == 0.0)); + assert_eq!(first.buffer[SENTENCE_LEAD_IN_SAMPLES], 0.4); + + let last = chunk + .finish(&mut first_append, silence, false) + .expect("last ready model unit"); + assert_eq!(last.buffer.len(), 16 + 100); + assert_eq!(last.buffer.last(), Some(&0.0)); + } + + #[test] + fn empty_edge_units_do_not_steal_lead_in_or_trailing_boundary() { + let mut chunk = PlaybackChunkAudio::new(); + let mut first_append = true; + let silence = SENTENCE_LEAD_IN_SAMPLES + 100; + + assert!(chunk + .push(Vec::new(), 0, &mut first_append, silence, false) + .is_none()); + assert!(chunk + .push(vec![0.5; 16], 1, &mut first_append, silence, false) + .is_none()); + assert!(chunk + .push(Vec::new(), 2, &mut first_append, silence, false) + .is_none()); + + let only = chunk + .finish(&mut first_append, silence, false) + .expect("only audible model unit"); + assert_eq!(only.buffer.len(), SENTENCE_LEAD_IN_SAMPLES + 16 + 100); + assert!(only.buffer[..SENTENCE_LEAD_IN_SAMPLES] + .iter() + .all(|sample| *sample == 0.0)); + assert_eq!(only.buffer.last(), Some(&0.0)); + } + + #[test] + fn playback_underrun_rearms_the_onset_cushion() { + let mut chunk = PlaybackChunkAudio::new(); + let mut first_append = true; + let silence = SENTENCE_LEAD_IN_SAMPLES + 100; + + assert!(chunk + .push(vec![0.4; 16], 0, &mut first_append, silence, false) + .is_none()); + let first = chunk + .push(vec![0.5; 16], 1, &mut first_append, silence, false) + .expect("first model unit"); + assert_eq!(first.buffer.len(), SENTENCE_LEAD_IN_SAMPLES + 16); + + let after_underrun = chunk + .push(vec![0.6; 16], 2, &mut first_append, silence, true) + .expect("model unit after underrun"); + assert_eq!(after_underrun.buffer.len(), SENTENCE_LEAD_IN_SAMPLES + 16); + assert!(after_underrun.buffer[..SENTENCE_LEAD_IN_SAMPLES] + .iter() + .all(|sample| *sample == 0.0)); + } +} diff --git a/desktop/src-tauri/src/huddle/tts_settings.rs b/desktop/src-tauri/src/huddle/tts_settings.rs new file mode 100644 index 0000000000..1b378af823 --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_settings.rs @@ -0,0 +1,971 @@ +//! Installation-global text-to-speech preferences and the local voice registry. +//! +//! Voice keys are backend-qualified (`pocket:mary`, `siri:aaron`) and +//! preferences are ordered. A client resolves the first compatible entry for +//! its one active playback backend. The same [`VoicePreferences`] value can be +//! embedded in installation-global settings or future agent identity without a +//! schema change. Availability is intentionally client-local. + +use std::{ + path::{Path, PathBuf}, + sync::{Arc, Mutex}, + time::Duration, +}; + +use serde::{Deserialize, Serialize}; +use tauri::{AppHandle, Manager, State}; + +use crate::{app_state::AppState, managed_agents::storage::atomic_write_json_restricted}; + +use super::{ + models, + pocket::DEFAULT_VOICE, + tts_voice_registry::{source_url, MARY_VOICE_KEY, POCKET_VOICES}, + HuddlePhase, HuddleState, +}; + +const SETTINGS_FILE: &str = "tts-settings.json"; +const CURRENT_VERSION: u32 = 1; +const VOICE_CHANGE_ACK_TIMEOUT: Duration = Duration::from_secs(5); +pub const POCKET_BACKEND_ID: &str = "pocket"; + +type VoiceChangeWait = ( + Arc, + tokio::sync::oneshot::Receiver<()>, +); + +const VOICE_AVAILABILITY_BUNDLED: &str = "bundled"; +const VOICE_AVAILABILITY_INSTALLED: &str = "installed"; + +/// Installation-global huddle audio and speech preferences. +#[derive(Default)] +pub struct HuddleAudioSettingsState { + pub tts: Mutex, + pub tts_load_error: Mutex>, + pub tts_transition: tokio::sync::Mutex<()>, + /// Selected huddle output device. `None` uses the system default. + pub output_device: Mutex>, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[serde(rename_all = "camelCase")] +pub struct VoiceRegistryEntry { + /// Stable identity, never derived from or merged by the display name. + /// + /// Built-ins use `backend:slug`. Future imports use + /// `pocket:imported:` so two clips with the same + /// editable label remain distinct. + pub key: String, + pub display_name: String, + pub backend: String, + pub backend_name: String, + /// Client-local state: bundled, installed, downloadable, or unavailable. + pub availability: String, + pub fallback_key: Option, + pub reference_file: Option, + pub provenance: VoiceProvenance, +} + +#[derive(Debug, Clone, Serialize, PartialEq, Eq)] +#[serde(rename_all = "camelCase")] +pub struct VoiceProvenance { + pub source: String, + pub content_hash: Option, + pub license: Option, + pub source_url: Option, +} + +/// Ordered, backend-qualified preferences shared by global and agent settings. +/// +/// Unknown but well-formed keys remain persisted because a different client +/// may have that backend installed. Resolution is always local. +pub type VoicePreferences = Vec; + +/// Bundled Pocket voices available without local imports. +pub fn bundled_voice_registry() -> Vec { + POCKET_VOICES + .iter() + .map(|voice| VoiceRegistryEntry { + key: voice.key.to_string(), + display_name: voice.display_name.to_string(), + backend: POCKET_BACKEND_ID.to_string(), + backend_name: "Pocket TTS".to_string(), + availability: VOICE_AVAILABILITY_BUNDLED.to_string(), + fallback_key: (voice.key != MARY_VOICE_KEY).then(|| MARY_VOICE_KEY.to_string()), + reference_file: Some(voice.reference_file.to_string()), + provenance: VoiceProvenance { + source: "bundled".to_string(), + content_hash: Some(voice.sha256.to_string()), + license: Some("CC-BY-4.0".to_string()), + source_url: Some(source_url(voice)), + }, + }) + .collect() +} + +/// Cross-backend registry of bundled and locally installed voices. +pub fn voice_registry(app: &AppHandle) -> Vec { + let mut registry = bundled_voice_registry(); + match super::tts_voice_import::load_registry(app) { + Ok(imported) => registry.extend(imported.into_iter().map(|voice| VoiceRegistryEntry { + key: voice.key, + display_name: voice.display_name, + backend: POCKET_BACKEND_ID.to_string(), + backend_name: "Pocket TTS".to_string(), + availability: VOICE_AVAILABILITY_INSTALLED.to_string(), + fallback_key: Some(MARY_VOICE_KEY.to_string()), + reference_file: Some(voice.file_name), + provenance: VoiceProvenance { + source: "local import".to_string(), + content_hash: Some(voice.content_hash), + license: None, + source_url: None, + }, + })), + Err(error) => { + eprintln!( + "buzz-desktop: {error}; imported Pocket voices are unavailable for this session" + ); + } + } + registry +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)] +#[serde(rename_all = "camelCase")] +pub struct TtsSettings { + pub version: u32, + pub agent_text_to_speech: bool, + pub voice_preferences: VoicePreferences, +} + +impl Default for TtsSettings { + fn default() -> Self { + Self { + version: CURRENT_VERSION, + agent_text_to_speech: true, + voice_preferences: vec![MARY_VOICE_KEY.to_string()], + } + } +} + +pub fn voice_by_key(app: &AppHandle, key: &str) -> Option { + voice_registry(app) + .into_iter() + .find(|voice| voice.key == key) +} + +fn is_qualified_voice_key(key: &str) -> bool { + key.split_once(':') + .is_some_and(|(backend, voice)| !backend.is_empty() && !voice.is_empty()) +} + +fn is_locally_available(availability: &str) -> bool { + matches!( + availability, + VOICE_AVAILABILITY_BUNDLED | VOICE_AVAILABILITY_INSTALLED + ) +} + +#[cfg(test)] +pub fn resolve_voice_for_backend( + preferences: &[String], + backend: &str, +) -> Result { + resolve_voice_for_backend_in_registry(preferences, backend, &bundled_voice_registry()) +} + +fn resolve_voice_for_backend_in_registry( + preferences: &[String], + backend: &str, + registry: &[VoiceRegistryEntry], +) -> Result { + preferences + .iter() + .filter_map(|key| registry.iter().find(|voice| voice.key == *key)) + .find(|voice| voice.backend == backend && is_locally_available(voice.availability.as_str())) + .or_else(|| { + registry.iter().find(|voice| { + voice.backend == backend + && voice.fallback_key.is_none() + && is_locally_available(voice.availability.as_str()) + }) + }) + .cloned() + .ok_or_else(|| format!("No locally available fallback voice for backend {backend}")) +} + +pub fn bundled_pocket_voice_reference(preferences: &[String]) -> String { + resolve_voice_for_backend_in_registry(preferences, POCKET_BACKEND_ID, &bundled_voice_registry()) + .ok() + .and_then(|voice| voice.reference_file) + .and_then(|file| file.strip_suffix(".wav").map(str::to_string)) + .unwrap_or_else(|| DEFAULT_VOICE.to_string()) +} + +pub fn pocket_voice_reference(app: &AppHandle, preferences: &[String]) -> Result { + let registry = voice_registry(app); + let voice = resolve_voice_for_backend_in_registry(preferences, POCKET_BACKEND_ID, ®istry)?; + if voice.key.starts_with("pocket:imported:") { + let imported = super::tts_voice_import::load_registry(app)? + .into_iter() + .find(|candidate| candidate.key == voice.key) + .ok_or_else(|| format!("Imported voice {} is unavailable", voice.display_name))?; + return super::tts_voice_import::resolve_file(app, &imported) + .map(|path| path.to_string_lossy().into_owned()); + } + Ok(voice + .reference_file + .and_then(|file| file.strip_suffix(".wav").map(str::to_string)) + .unwrap_or_else(|| DEFAULT_VOICE.to_string())) +} + +pub(crate) fn settings_path(app: &AppHandle) -> Result { + app.path() + .app_data_dir() + .map(|dir| dir.join(SETTINGS_FILE)) + .map_err(|error| format!("could not locate Buzz settings storage: {error}")) +} + +pub(crate) fn load_from_path(path: &Path) -> Result { + if !path.exists() { + return Ok(TtsSettings::default()); + } + let bytes = std::fs::read(path) + .map_err(|error| format!("could not read text-to-speech settings: {error}"))?; + let value: serde_json::Value = serde_json::from_slice(&bytes) + .map_err(|error| format!("text-to-speech settings are not valid JSON: {error}"))?; + + // Unversioned settings are incompatible with the V1 schema. Use + // deterministic V1 defaults rather than interpreting ambiguous fields. + if value.get("version").is_none() { + return Ok(TtsSettings::default()); + } + + let version = value + .get("version") + .and_then(serde_json::Value::as_u64) + .ok_or("text-to-speech settings version is invalid")?; + if version > u64::from(CURRENT_VERSION) { + return Err(format!( + "text-to-speech settings version {version} is newer than this Buzz build supports" + )); + } + + // Legacy V1 settings may contain one bare Pocket `voiceId`. Preserve the + // toggle and qualify it into the ordered cross-backend preference schema. + if value.get("voicePreferences").is_none() { + let legacy_voice = value + .get("voiceId") + .or_else(|| value.get("voice_id")) + .and_then(serde_json::Value::as_str) + .unwrap_or("mary"); + let voice_key = if is_qualified_voice_key(legacy_voice) { + legacy_voice.to_string() + } else { + format!("{POCKET_BACKEND_ID}:{legacy_voice}") + }; + return Ok(TtsSettings { + version: CURRENT_VERSION, + agent_text_to_speech: value + .get("agentTextToSpeech") + .and_then(serde_json::Value::as_bool) + .unwrap_or(true), + voice_preferences: vec![voice_key], + }); + } + + let mut settings: TtsSettings = serde_json::from_value(value) + .map_err(|error| format!("text-to-speech settings are invalid: {error}"))?; + settings.version = CURRENT_VERSION; + if settings.voice_preferences.is_empty() + || settings + .voice_preferences + .iter() + .any(|key| !is_qualified_voice_key(key)) + { + settings.voice_preferences = TtsSettings::default().voice_preferences; + } + Ok(settings) +} + +pub(crate) fn save_to_path(path: &Path, settings: &TtsSettings) -> Result<(), String> { + if settings.voice_preferences.is_empty() { + return Err("At least one voice preference is required".to_string()); + } + if let Some(key) = settings + .voice_preferences + .iter() + .find(|key| !is_qualified_voice_key(key)) + { + return Err(format!( + "Voice preference keys must be backend-qualified: {key}" + )); + } + let payload = serde_json::to_vec_pretty(settings) + .map_err(|error| format!("could not encode text-to-speech settings: {error}"))?; + atomic_write_json_restricted(path, &payload) + .map_err(|error| format!("could not save text-to-speech settings: {error}")) +} + +pub fn load_for_app(app: &AppHandle) -> (TtsSettings, Option) { + let result = settings_path(app).and_then(|path| load_from_path(&path)); + match result { + Ok(settings) => (settings, None), + Err(error) => { + eprintln!("buzz-desktop: {error}; preserving the file and using Mary for this session"); + (TtsSettings::default(), Some(error)) + } + } +} + +#[tauri::command] +pub fn get_tts_settings(state: State<'_, AppState>) -> Result { + if let Some(error) = state + .huddle_audio + .tts_load_error + .lock() + .map_err(|lock_error| format!("text-to-speech settings lock poisoned: {lock_error}"))? + .clone() + { + return Err(format!( + "Voice settings could not be loaded and were left unchanged: {error}" + )); + } + state + .huddle_audio + .tts + .lock() + .map(|settings| settings.clone()) + .map_err(|error| format!("text-to-speech settings lock poisoned: {error}")) +} + +#[tauri::command] +pub fn list_voice_registry(app: AppHandle) -> Vec { + voice_registry(&app) +} + +fn ensure_settings_writable(state: &AppState) -> Result<(), String> { + if let Some(error) = state + .huddle_audio + .tts_load_error + .lock() + .map_err(|lock_error| format!("text-to-speech settings lock poisoned: {lock_error}"))? + .as_ref() + { + return Err(format!( + "Voice settings were not saved because the existing file could not be loaded: {error}" + )); + } + Ok(()) +} + +fn cancel_huddle_speech( + huddle: &mut super::HuddleState, +) -> Option> { + huddle.tts_enabled = false; + huddle + .tts_cancel + .store(true, std::sync::atomic::Ordering::Release); + huddle.tts_pipeline.take() +} + +fn disable_tts_runtime(state: &AppState) -> Result<(), String> { + let old_pipeline = { + let mut huddle = state.huddle()?; + cancel_huddle_speech(&mut huddle) + }; + if let Some(ref pipeline) = old_pipeline { + pipeline.shutdown(); + } + drop(old_pipeline); + state.emit_huddle_state_changed(); + Ok(()) +} + +fn commit_effective_off(state: &AppState) -> Result<(), String> { + state + .huddle_audio + .tts + .lock() + .map_err(|error| format!("text-to-speech settings lock poisoned: {error}"))? + .agent_text_to_speech = false; + Ok(()) +} + +fn enable_tts_runtime(huddle: &mut HuddleState, voice: &str) -> Option { + huddle.tts_enabled = true; + // OFF removes the pipeline. Clear a prior cancellation only when enabling + // a fresh pipeline; an idempotent ON write must not erase a voice + // transition that the existing worker still needs to drain. + prepare_enable_cancel(&huddle.tts_cancel, huddle.tts_pipeline.is_some()); + huddle.tts_pipeline.as_ref().and_then(|pipeline| { + pipeline + .select_voice(voice) + .map(|acknowledged| (Arc::clone(pipeline), acknowledged)) + }) +} + +fn prepare_enable_cancel(cancel: &std::sync::atomic::AtomicBool, has_pipeline: bool) { + if !has_pipeline { + cancel.store(false, std::sync::atomic::Ordering::Release); + } +} + +async fn apply_tts_settings( + settings: TtsSettings, + app: &AppHandle, + state: &AppState, +) -> Result, String> { + if settings.version != CURRENT_VERSION { + return Err(format!( + "Unsupported text-to-speech settings version: {}", + settings.version + )); + } + + // OFF is safety-sensitive: stop current and queued speech before any disk + // I/O, and never resume it merely because persistence fails. + if !settings.agent_text_to_speech { + disable_tts_runtime(state)?; + commit_effective_off(state)?; + } + + ensure_settings_writable(state)?; + save_to_path(&settings_path(app)?, &settings)?; + + *state + .huddle_audio + .tts + .lock() + .map_err(|error| format!("text-to-speech settings lock poisoned: {error}"))? = + settings.clone(); + + let mut voice_change_wait = None; + if settings.agent_text_to_speech { + let (active, voice_change_ack) = { + let mut huddle = state.huddle()?; + let voice_reference = pocket_voice_reference(app, &settings.voice_preferences)?; + let voice_change_ack = enable_tts_runtime(&mut huddle, &voice_reference); + ( + matches!(huddle.phase, HuddlePhase::Connected | HuddlePhase::Active), + voice_change_ack, + ) + }; + voice_change_wait = voice_change_ack; + if active { + if let Err(error) = super::pipeline::maybe_start_tts_pipeline(state).await { + eprintln!("buzz-desktop: could not hot-start text to speech: {error}"); + } + } + state.emit_huddle_state_changed(); + } + Ok(voice_change_wait) +} + +fn current_settings(state: &AppState) -> Result { + state + .huddle_audio + .tts + .lock() + .map_err(|error| format!("text-to-speech settings lock poisoned: {error}")) + .map(|settings| settings.clone()) +} + +async fn finish_voice_change(voice_change: Option) -> Result<(), String> { + let Some((pipeline, acknowledged)) = voice_change else { + return Ok(()); + }; + wait_for_voice_change_ack(acknowledged, VOICE_CHANGE_ACK_TIMEOUT, || { + pipeline.is_finished() + }) + .await +} + +async fn finish_durable_voice_change(voice_change: Option) { + if let Err(error) = finish_voice_change(voice_change).await { + eprintln!( + "buzz-desktop: tts stage=voice_switch status=delayed reason=ack_timeout error={error}" + ); + } +} + +async fn wait_for_voice_change_ack( + mut acknowledged: tokio::sync::oneshot::Receiver<()>, + timeout: Duration, + mut worker_is_finished: impl FnMut() -> bool, +) -> Result<(), String> { + let deadline = tokio::time::sleep(timeout); + tokio::pin!(deadline); + loop { + tokio::select! { + _ = &mut acknowledged => return Ok(()), + _ = &mut deadline => { + return Err( + "Pocket TTS is still finishing the previous voice. Turn Agent text to speech off and try again." + .to_string(), + ); + } + _ = tokio::time::sleep(Duration::from_millis(25)) => { + if worker_is_finished() { + return Ok(()); + } + } + } + } +} + +/// Compatibility command for the huddle speaker button. It updates the same +/// installation-global preference as Settings; there is no per-huddle override. +#[tauri::command] +pub async fn set_tts_enabled( + enabled: bool, + app: AppHandle, + state: State<'_, AppState>, +) -> Result { + let transition = state.huddle_audio.tts_transition.lock().await; + let mut settings = state + .huddle_audio + .tts + .lock() + .map_err(|error| format!("text-to-speech settings lock poisoned: {error}"))? + .clone(); + settings.agent_text_to_speech = enabled; + let voice_change = apply_tts_settings(settings, &app, &state).await?; + drop(transition); + finish_voice_change(voice_change).await?; + current_settings(&state) +} + +fn settings_with_pocket_voice( + settings: TtsSettings, + voice_key: &str, + app: &AppHandle, +) -> Result { + settings_with_pocket_voice_from_registry(settings, voice_key, &voice_registry(app)) +} + +fn settings_with_pocket_voice_from_registry( + mut settings: TtsSettings, + voice_key: &str, + registry: &[VoiceRegistryEntry], +) -> Result { + let voice = registry + .iter() + .find(|voice| voice.key == voice_key) + .ok_or_else(|| format!("Unknown voice: {voice_key}"))?; + if voice.backend != POCKET_BACKEND_ID || !is_locally_available(&voice.availability) { + return Err("The selected Pocket voice is not available on this device".to_string()); + } + let first_pocket_index = settings + .voice_preferences + .iter() + .position(|key| key.starts_with("pocket:")); + settings + .voice_preferences + .retain(|key| !key.starts_with("pocket:")); + let insert_at = first_pocket_index + .unwrap_or(settings.voice_preferences.len()) + .min(settings.voice_preferences.len()); + settings + .voice_preferences + .insert(insert_at, voice_key.to_string()); + Ok(settings) +} + +#[tauri::command] +pub async fn set_pocket_voice( + voice_key: String, + app: AppHandle, + state: State<'_, AppState>, +) -> Result { + let transition = state.huddle_audio.tts_transition.lock().await; + let settings = state + .huddle_audio + .tts + .lock() + .map_err(|error| format!("text-to-speech settings lock poisoned: {error}"))? + .clone(); + let settings = settings_with_pocket_voice(settings, &voice_key, &app)?; + let voice_change = apply_tts_settings(settings, &app, &state).await?; + drop(transition); + finish_durable_voice_change(voice_change).await; + current_settings(&state) +} + +#[tauri::command] +pub async fn preview_pocket_voice( + voice_key: String, + app: AppHandle, + state: State<'_, AppState>, +) -> Result<(), String> { + let voice = + voice_by_key(&app, &voice_key).ok_or_else(|| format!("Unknown voice: {voice_key}"))?; + if voice.backend != POCKET_BACKEND_ID { + return Err("Only Pocket voices can be previewed in this build".to_string()); + } + if !models::is_tts_ready() { + return Err("Voice files are still downloading. Try preview again shortly.".to_string()); + } + let model_dir = models::tts_model_dir().ok_or("Pocket voice files are unavailable")?; + let output_device = state + .huddle_audio + .output_device + .lock() + .unwrap_or_else(|error| error.into_inner()) + .clone(); + let voice_name = pocket_voice_reference(&app, std::slice::from_ref(&voice_key))?; + tokio::task::spawn_blocking(move || { + let active = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false)); + let cancel = std::sync::Arc::new(std::sync::atomic::AtomicBool::new(false)); + let pipeline = super::tts::TtsPipeline::new_with_voice( + model_dir, + active.clone(), + cancel, + &voice_name, + output_device, + )?; + pipeline.speak("Hello! This is how I’ll read agent responses.".to_string())?; + let started = std::time::Instant::now(); + let mut heard_audio = false; + while started.elapsed() < std::time::Duration::from_secs(30) { + let is_active = active.load(std::sync::atomic::Ordering::Acquire); + heard_audio |= is_active; + if heard_audio && !is_active { + return Ok(()); + } + std::thread::sleep(std::time::Duration::from_millis(25)); + } + Err("Voice preview timed out. Check your audio output and try again.".to_string()) + }) + .await + .map_err(|error| format!("Voice preview task failed: {error}"))? +} + +#[derive(Debug, Clone, Serialize)] +#[serde(rename_all = "camelCase")] +pub struct TtsVoiceMutation { + pub settings: TtsSettings, + pub registry: Vec, +} + +#[tauri::command] +pub async fn import_pocket_voice( + app: AppHandle, + state: State<'_, AppState>, +) -> Result, String> { + let Some(imported) = super::tts_voice_import::pick_and_import(&app).await? else { + return Ok(None); + }; + let transition = state.huddle_audio.tts_transition.lock().await; + let settings = current_settings(&state)?; + let settings = settings_with_pocket_voice(settings, &imported.key, &app)?; + let voice_change = apply_tts_settings(settings, &app, &state).await?; + drop(transition); + finish_durable_voice_change(voice_change).await; + Ok(Some(TtsVoiceMutation { + settings: current_settings(&state)?, + registry: voice_registry(&app), + })) +} + +#[tauri::command] +pub async fn delete_pocket_voice( + voice_key: String, + app: AppHandle, + state: State<'_, AppState>, +) -> Result { + if !voice_key.starts_with("pocket:imported:") { + return Err("Bundled voices cannot be deleted".to_string()); + } + if voice_by_key(&app, &voice_key).is_none() { + return Err(format!("Unknown imported voice: {voice_key}")); + } + + let transition = state.huddle_audio.tts_transition.lock().await; + let current = current_settings(&state)?; + let selected = resolve_voice_for_backend_in_registry( + ¤t.voice_preferences, + POCKET_BACKEND_ID, + &voice_registry(&app), + ) + .is_ok_and(|voice| voice.key == voice_key); + let voice_change = if selected { + let fallback = settings_with_pocket_voice(current, MARY_VOICE_KEY, &app)?; + apply_tts_settings(fallback, &app, &state).await? + } else { + None + }; + drop(transition); + finish_durable_voice_change(voice_change).await; + super::tts_voice_import::delete(&app, &voice_key)?; + Ok(TtsVoiceMutation { + settings: current_settings(&state)?, + registry: voice_registry(&app), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + const EVE_VOICE_KEY: &str = "pocket:eve"; + + #[tokio::test] + async fn stalled_voice_change_returns_an_actionable_error() { + let (_keep_pending, acknowledged) = tokio::sync::oneshot::channel(); + + let error = wait_for_voice_change_ack(acknowledged, Duration::from_millis(1), || false) + .await + .expect_err("stalled worker should time out"); + + assert!(error.contains("Turn Agent text to speech off")); + } + + #[test] + fn idempotent_enable_preserves_an_existing_pipeline_cancel() { + let cancel = std::sync::atomic::AtomicBool::new(true); + prepare_enable_cancel(&cancel, true); + assert!(cancel.load(std::sync::atomic::Ordering::Acquire)); + prepare_enable_cancel(&cancel, false); + assert!(!cancel.load(std::sync::atomic::Ordering::Acquire)); + } + + #[test] + fn defaults_are_backwards_compatible_and_use_mary() { + assert_eq!( + TtsSettings::default(), + TtsSettings { + version: 1, + agent_text_to_speech: true, + voice_preferences: vec!["pocket:mary".to_string()], + } + ); + } + + #[test] + fn registry_has_all_official_english_vctk_presets() { + assert_eq!( + bundled_voice_registry() + .iter() + .map(|voice| { + ( + voice.key.as_str(), + voice.display_name.as_str(), + voice.reference_file.as_deref(), + ) + }) + .collect::>(), + vec![ + ("pocket:anna", "Anna", Some("anna.wav")), + ("pocket:vera", "Vera", Some("vera.wav")), + ("pocket:fantine", "Fantine", Some("fantine.wav")), + ("pocket:charles", "Charles", Some("charles.wav")), + ("pocket:paul", "Paul", Some("paul.wav")), + ("pocket:eponine", "Eponine", Some("eponine.wav")), + ("pocket:azelma", "Azelma", Some("azelma.wav")), + ("pocket:george", "George", Some("george.wav")), + ("pocket:mary", "Mary", Some("reference_sample.wav")), + ("pocket:jane", "Jane", Some("jane.wav")), + ("pocket:michael", "Michael", Some("michael.wav")), + ("pocket:eve", "Eve", Some("eve.wav")), + ] + ); + } + + #[test] + fn local_backend_resolution_uses_first_compatible_preference() { + let preferences = vec![ + "siri:aaron".to_string(), + EVE_VOICE_KEY.to_string(), + MARY_VOICE_KEY.to_string(), + "kokoro:af_heart".to_string(), + ]; + assert_eq!( + resolve_voice_for_backend(&preferences, POCKET_BACKEND_ID) + .expect("Pocket fallback") + .key, + EVE_VOICE_KEY + ); + } + + #[test] + fn unsupported_or_missing_preferences_fall_back_to_backend_default() { + let preferences = vec![ + "siri:aaron".to_string(), + "pocket:imported:deadbeef".to_string(), + ]; + assert_eq!( + resolve_voice_for_backend(&preferences, POCKET_BACKEND_ID) + .expect("Pocket fallback") + .key, + MARY_VOICE_KEY + ); + } + + #[test] + fn identity_is_qualified_key_not_display_label() { + assert!(is_qualified_voice_key("pocket:imported:audio-content-hash")); + assert_ne!(MARY_VOICE_KEY, EVE_VOICE_KEY); + let mut registry = bundled_voice_registry(); + registry[0].display_name = "Jim".to_string(); + registry[1].display_name = "Jim".to_string(); + assert_eq!(registry[0].display_name, registry[1].display_name); + assert_ne!(registry[0].key, registry[1].key); + assert_eq!( + registry + .iter() + .map(|voice| voice.key.as_str()) + .collect::>() + .len(), + registry.len() + ); + } + + #[test] + fn bundled_vctk_assets_match_the_registry_manifest() { + for voice in POCKET_VOICES { + let Some(bytes) = voice.bytes else { + continue; + }; + assert_eq!(&bytes[0..4], b"RIFF", "{}", voice.display_name); + assert_eq!(&bytes[8..12], b"WAVE", "{}", voice.display_name); + assert_eq!( + hex::encode(::digest(bytes)), + voice.sha256, + "{}", + voice.display_name + ); + } + } + + #[test] + fn migrates_unversioned_experiment_settings_to_v1_defaults() { + let dir = tempfile::tempdir().expect("temp dir"); + let path = dir.path().join(SETTINGS_FILE); + std::fs::write(&path, r#"{"voice":"legacy-experiment"}"#).expect("fixture write"); + assert_eq!( + load_from_path(&path).expect("migration"), + TtsSettings::default() + ); + } + + #[test] + fn migrates_bare_pocket_voice_id_to_qualified_preferences() { + let dir = tempfile::tempdir().expect("temp dir"); + let path = dir.path().join(SETTINGS_FILE); + std::fs::write( + &path, + r#"{"version":1,"agentTextToSpeech":false,"voiceId":"eve"}"#, + ) + .expect("fixture write"); + assert_eq!( + load_from_path(&path).expect("migration"), + TtsSettings { + version: 1, + agent_text_to_speech: false, + voice_preferences: vec![EVE_VOICE_KEY.to_string()], + } + ); + } + + #[test] + fn unknown_qualified_preferences_are_preserved_for_other_clients() { + let dir = tempfile::tempdir().expect("temp dir"); + let path = dir.path().join(SETTINGS_FILE); + std::fs::write( + &path, + r#"{"version":1,"agentTextToSpeech":false,"voicePreferences":["siri:aaron","pocket:imported:abc123"]}"#, + ) + .expect("fixture write"); + let settings = load_from_path(&path).expect("load"); + assert!(!settings.agent_text_to_speech); + assert_eq!( + settings.voice_preferences, + vec!["siri:aaron", "pocket:imported:abc123"] + ); + } + + #[test] + fn rejects_future_schema_versions_clearly() { + let dir = tempfile::tempdir().expect("temp dir"); + let path = dir.path().join(SETTINGS_FILE); + std::fs::write( + &path, + r#"{"version":99,"agentTextToSpeech":true,"voicePreferences":["pocket:mary"]}"#, + ) + .expect("fixture write"); + assert!(load_from_path(&path) + .expect_err("future version should fail") + .contains("newer than this Buzz build supports")); + } + + #[test] + fn disabling_cancels_runtime_before_persistence_can_fail() { + let mut huddle = super::super::HuddleState { + tts_enabled: true, + ..super::super::HuddleState::default() + }; + assert!(!huddle.tts_cancel.load(std::sync::atomic::Ordering::Acquire)); + assert!(cancel_huddle_speech(&mut huddle).is_none()); + assert!(!huddle.tts_enabled); + assert!(huddle.tts_cancel.load(std::sync::atomic::Ordering::Acquire)); + } + + #[test] + fn pocket_voice_update_preserves_the_latest_toggle_and_other_backends() { + let current = TtsSettings { + agent_text_to_speech: false, + voice_preferences: vec!["siri:aaron".to_string(), MARY_VOICE_KEY.to_string()], + ..TtsSettings::default() + }; + let updated = settings_with_pocket_voice_from_registry( + current, + EVE_VOICE_KEY, + &bundled_voice_registry(), + ) + .expect("available voice"); + assert!(!updated.agent_text_to_speech); + assert_eq!(updated.voice_preferences, vec!["siri:aaron", EVE_VOICE_KEY]); + } + + #[test] + fn failed_off_persistence_cannot_be_undone_by_a_later_voice_update() { + let state = crate::app_state::build_app_state(); + commit_effective_off(&state).expect("commit effective OFF state"); + + // This models the next command after the OFF save fails: it must merge + // from effective memory state, not the stale last-persisted ON value. + let current = state.huddle_audio.tts.lock().expect("settings").clone(); + let voice_update = settings_with_pocket_voice_from_registry( + current, + EVE_VOICE_KEY, + &bundled_voice_registry(), + ) + .expect("available voice"); + assert!(!voice_update.agent_text_to_speech); + } + + #[test] + fn failed_disabled_voice_save_does_not_change_the_remembered_voice() { + let state = crate::app_state::build_app_state(); + state + .huddle_audio + .tts + .lock() + .expect("settings") + .agent_text_to_speech = false; + let current = state.huddle_audio.tts.lock().expect("settings").clone(); + let unsaved = settings_with_pocket_voice_from_registry( + current, + EVE_VOICE_KEY, + &bundled_voice_registry(), + ) + .expect("available voice"); + + // This is the only pre-persistence mutation for an OFF candidate. + commit_effective_off(&state).expect("commit effective OFF state"); + let remembered = state.huddle_audio.tts.lock().expect("settings").clone(); + assert_eq!(remembered.voice_preferences, vec![MARY_VOICE_KEY]); + assert_eq!(unsaved.voice_preferences, vec![EVE_VOICE_KEY]); + } +} diff --git a/desktop/src-tauri/src/huddle/tts_startup.rs b/desktop/src-tauri/src/huddle/tts_startup.rs new file mode 100644 index 0000000000..2cb50401a9 --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_startup.rs @@ -0,0 +1,24 @@ +use std::{sync::mpsc, thread}; + +pub(super) fn await_worker_startup( + handle: thread::JoinHandle<()>, + startup_rx: mpsc::Receiver>, +) -> Result, String> { + match startup_rx.recv() { + Ok(Ok(())) => Ok(handle), + Ok(Err(error)) => { + let _ = handle.join(); + Err(error) + } + Err(error) => { + let _ = handle.join(); + Err(format!( + "TTS worker exited before reporting readiness: {error}" + )) + } + } +} + +#[cfg(test)] +#[path = "tts_startup_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/huddle/tts_startup_tests.rs b/desktop/src-tauri/src/huddle/tts_startup_tests.rs new file mode 100644 index 0000000000..cf688c2db8 --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_startup_tests.rs @@ -0,0 +1,44 @@ +use super::*; +use std::sync::{ + atomic::{AtomicBool, Ordering}, + Arc, +}; + +#[test] +fn startup_failure_is_returned_after_worker_exit() { + let (tx, rx) = mpsc::sync_channel(1); + let exited = Arc::new(AtomicBool::new(false)); + let exited_worker = Arc::clone(&exited); + let handle = std::thread::spawn(move || { + tx.send(Err("output unavailable".to_string())) + .expect("startup receiver"); + exited_worker.store(true, Ordering::Release); + }); + + assert_eq!( + await_worker_startup(handle, rx).expect_err("startup must fail"), + "output unavailable" + ); + assert!(exited.load(Ordering::Acquire)); +} + +#[test] +fn worker_exit_before_readiness_is_a_startup_error() { + let (tx, rx) = mpsc::sync_channel::>(1); + let handle = std::thread::spawn(move || drop(tx)); + + assert!(await_worker_startup(handle, rx) + .expect_err("closed startup channel must fail") + .contains("before reporting readiness")); +} + +#[test] +fn ready_ack_precedes_pipeline_publication_boundary() { + let (tx, rx) = mpsc::sync_channel(1); + let handle = std::thread::spawn(move || { + tx.send(Ok(())).expect("startup receiver"); + }); + + let handle = await_worker_startup(handle, rx).expect("ready worker"); + handle.join().expect("worker exits"); +} diff --git a/desktop/src-tauri/src/huddle/tts_tests.rs b/desktop/src-tauri/src/huddle/tts_tests.rs index 7887f8bbdb..1908b096b1 100644 --- a/desktop/src-tauri/src/huddle/tts_tests.rs +++ b/desktop/src-tauri/src/huddle/tts_tests.rs @@ -9,6 +9,9 @@ use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::mpsc; use std::sync::{Arc, Mutex}; +#[path = "tts_tests/token_split.rs"] +mod token_split; + // ── Remote interrupt tracker ────────────────────────────────────────────── // // Models the per-peer frame counting logic in the recv task of @@ -785,16 +788,6 @@ fn apply_fade_out_single_sample() { assert_eq!(samples[0], 1.0); } -/// Sanity-check the per-sentence cushion length: 20 ms at 24 kHz must -/// land at exactly 480 samples. This is a const computation, so the -/// real value of this test is documenting *why* 20 ms was chosen — it -/// covers a typical CoreAudio buffer turnover (256–1024 samples) -/// without being audible as user-facing latency. -#[test] -fn sentence_lead_in_is_sane() { - assert_eq!(SENTENCE_LEAD_IN_SAMPLES, 480, "20 ms × 24 kHz"); -} - // ── build_sentence_append_buffer tests ─────────────────────────────────── /// REGRESSION: every chunk needs an onset cushion; synthesized chunks @@ -812,6 +805,8 @@ fn lead_in_pad_is_present_for_every_sentence_chunk() { &mut first, vec![0.5_f32; SENTENCE_AUDIO_LEN], SILENCE_BUF_LEN, + true, + true, ); assert_eq!(buf.len(), SENTENCE_AUDIO_LEN + SILENCE_BUF_LEN); @@ -840,11 +835,11 @@ fn lead_in_pad_is_present_for_every_sentence_chunk() { #[test] fn build_sentence_append_buffer_flips_first_append() { let mut first = true; - let _ = build_sentence_append_buffer(&mut first, vec![0.5; 100], 2400); + let _ = build_sentence_append_buffer(&mut first, vec![0.5; 100], 2400, true, true); assert!(!first, "first call must flip the flag"); // Subsequent call: still has a per-sentence lead-in, flag stays false. - let buf = build_sentence_append_buffer(&mut first, vec![0.5; 100], 2400); + let buf = build_sentence_append_buffer(&mut first, vec![0.5; 100], 2400, true, true); assert!(buf[..SENTENCE_LEAD_IN_SAMPLES].iter().all(|&s| s == 0.0)); assert!(!first); } @@ -853,7 +848,7 @@ fn build_sentence_append_buffer_flips_first_append() { #[test] fn first_sentence_leading_silence_is_exactly_lead_in() { let mut first = true; - let buf = build_sentence_append_buffer(&mut first, vec![0.5; 100], 2400); + let buf = build_sentence_append_buffer(&mut first, vec![0.5; 100], 2400, true, true); assert!(buf[..SENTENCE_LEAD_IN_SAMPLES].iter().all(|&s| s == 0.0)); assert_eq!(buf[SENTENCE_LEAD_IN_SAMPLES], 0.5); } @@ -863,8 +858,10 @@ fn first_sentence_leading_silence_is_exactly_lead_in() { fn sentence_gap_budget_is_preserved() { let mut first = true; let silence_buf_len = 2400; - let first_buf = build_sentence_append_buffer(&mut first, vec![0.5; 100], silence_buf_len); - let second_buf = build_sentence_append_buffer(&mut first, vec![0.5; 100], silence_buf_len); + let first_buf = + build_sentence_append_buffer(&mut first, vec![0.5; 100], silence_buf_len, true, true); + let second_buf = + build_sentence_append_buffer(&mut first, vec![0.5; 100], silence_buf_len, true, true); let first_tail = &first_buf[SENTENCE_LEAD_IN_SAMPLES + 100..]; let second_lead = &second_buf[..SENTENCE_LEAD_IN_SAMPLES]; @@ -877,7 +874,7 @@ fn sentence_gap_budget_is_preserved() { #[test] fn sentence_append_buffer_is_one_contiguous_source() { let mut first = true; - let buf = build_sentence_append_buffer(&mut first, vec![0.5; 100], 2400); + let buf = build_sentence_append_buffer(&mut first, vec![0.5; 100], 2400, true, true); assert_eq!(buf.len(), 2400 + 100); assert!(buf[..SENTENCE_LEAD_IN_SAMPLES].iter().all(|&s| s == 0.0)); @@ -949,9 +946,8 @@ fn chunk_grouping_packs_up_to_budget_then_spills() { assert_eq!(chunks[2], d); } -/// A single sentence longer than the budget is passed through unsplit — -/// long single sentences are fine (the LM cap bounds runaway); only seams -/// are being minimized. +/// A single sentence longer than the coarse budget is passed through here; +/// the loaded April engine subsequently enforces its exact 50-token limit. #[test] fn chunk_grouping_oversized_sentence_passes_through() { let long = "word ".repeat(60).trim_end().to_string() + "."; diff --git a/desktop/src-tauri/src/huddle/tts_tests/token_split.rs b/desktop/src-tauri/src/huddle/tts_tests/token_split.rs new file mode 100644 index 0000000000..b9249c9afc --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_tests/token_split.rs @@ -0,0 +1,24 @@ +use super::*; + +/// The onset cushion covers 20 ms at the production sample rate. +#[test] +fn sentence_lead_in_is_sane() { + assert_eq!(SENTENCE_LEAD_IN_SAMPLES, 480, "20 ms × 24 kHz"); +} + +/// Model-token splits remain contiguous: only the playback chunk as a whole +/// receives its onset cushion and trailing sentence gap. +#[test] +fn token_split_units_do_not_add_sentence_boundary_padding() { + let mut first = true; + let silence_buf_len = 2400; + let first_unit = + build_sentence_append_buffer(&mut first, vec![0.5; 100], silence_buf_len, true, false); + let last_unit = + build_sentence_append_buffer(&mut first, vec![0.25; 100], silence_buf_len, false, true); + + assert_eq!(first_unit.len(), SENTENCE_LEAD_IN_SAMPLES + 100); + assert_eq!(first_unit.last(), Some(&0.5)); + assert_eq!(last_unit.first(), Some(&0.25)); + assert_eq!(first_unit.len() + last_unit.len(), 200 + silence_buf_len); +} diff --git a/desktop/src-tauri/src/huddle/tts_voice_import.rs b/desktop/src-tauri/src/huddle/tts_voice_import.rs new file mode 100644 index 0000000000..cdcc7761e2 --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_voice_import.rs @@ -0,0 +1,59 @@ +//! Tauri native-picker adapter for the reusable local Pocket voice library. + +use std::path::PathBuf; + +use buzz_voice_pkg::imported::{ImportedVoice, PocketVoiceLibrary}; +use tauri::{AppHandle, Manager}; + +pub fn voices_dir(app: &AppHandle) -> Result { + app.path() + .app_data_dir() + .map(|path| path.join("tts").join("pocket-voices")) + .map_err(|error| format!("could not locate local voice storage: {error}")) +} + +fn library(app: &AppHandle) -> Result { + voices_dir(app).map(PocketVoiceLibrary::new) +} + +pub fn load_registry(app: &AppHandle) -> Result, String> { + library(app)?.load() +} + +pub fn resolve_file(app: &AppHandle, voice: &ImportedVoice) -> Result { + library(app)?.resolve_file(voice) +} + +pub async fn pick_and_import(app: &AppHandle) -> Result, String> { + use tauri_plugin_dialog::DialogExt; + + let (sender, receiver) = tokio::sync::oneshot::channel(); + app.dialog() + .file() + .add_filter( + "Audio", + &["wav", "m4a", "mp3", "flac", "ogg", "oga", "aif", "aiff"], + ) + .pick_file(move |path| { + let _ = sender.send(path); + }); + let Some(file_path) = receiver + .await + .map_err(|_| "voice picker closed unexpectedly".to_string())? + else { + return Ok(None); + }; + let path = file_path + .as_path() + .ok_or("the selected voice path is invalid")? + .to_path_buf(); + let voice_library = library(app)?; + tokio::task::spawn_blocking(move || voice_library.import_path(&path)) + .await + .map_err(|error| format!("voice import task failed: {error}"))? + .map(Some) +} + +pub fn delete(app: &AppHandle, key: &str) -> Result<(), String> { + library(app)?.delete(key) +} diff --git a/desktop/src-tauri/src/huddle/tts_voice_registry.rs b/desktop/src-tauri/src/huddle/tts_voice_registry.rs new file mode 100644 index 0000000000..bdfbd7677b --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_voice_registry.rs @@ -0,0 +1,129 @@ +//! Built-in Pocket voice identities and immutable asset metadata. +//! +//! Stable keys identify audio, not display labels. Future imported voices use +//! `pocket:imported:` and may share editable labels. + +pub(super) const MARY_VOICE_KEY: &str = "pocket:mary"; +pub(super) const VCTK_REVISION: &str = "323332d33f997de8394f24a193e1a76df720e01a"; + +pub(super) struct PocketVoiceSpec { + pub key: &'static str, + pub display_name: &'static str, + pub reference_file: &'static str, + pub upstream_file: &'static str, + pub sha256: &'static str, + pub bytes: Option<&'static [u8]>, +} + +macro_rules! bundled_voice { + ($key:literal, $name:literal, $file:literal, $upstream:literal, $hash:literal) => { + PocketVoiceSpec { + key: $key, + display_name: $name, + reference_file: concat!($file, ".wav"), + upstream_file: concat!("vctk/", $upstream), + sha256: $hash, + bytes: Some(include_bytes!(concat!( + "../../resources/pocket-voices/", + $file, + ".wav" + ))), + } + }; +} + +/// Official English Pocket presets, in the order published by Kyutai. +pub(super) static POCKET_VOICES: &[PocketVoiceSpec] = &[ + bundled_voice!( + "pocket:anna", + "Anna", + "anna", + "p228_023_enhanced.wav", + "0a6de25cf12bf1540beb85979f306a92be81fecc051c547c5395e7e5237a3856" + ), + bundled_voice!( + "pocket:vera", + "Vera", + "vera", + "p229_023_enhanced.wav", + "309cf91a895830f15842b398f69a4962cb1f7e0bfab10e25dd27838e826c204b" + ), + bundled_voice!( + "pocket:fantine", + "Fantine", + "fantine", + "p244_023_enhanced.wav", + "5f07d4e2a3f20a15572aae885156b43ef3fc12ef3812996fd135680d9956448b" + ), + bundled_voice!( + "pocket:charles", + "Charles", + "charles", + "p254_023_enhanced.wav", + "6b681a429198f16e378d53bccb08d06939da7b00144a7696111d4f8f76be7756" + ), + bundled_voice!( + "pocket:paul", + "Paul", + "paul", + "p259_023_enhanced.wav", + "7aba504fe0b3b16478b69eb27ce6007e3cb42b0c1915b5f1c6a6024ae37d679b" + ), + bundled_voice!( + "pocket:eponine", + "Eponine", + "eponine", + "p262_023_enhanced.wav", + "a13c27fb47627b05223691a0ef2974358a18c886e6c2f9d2762ff1d02c20926b" + ), + bundled_voice!( + "pocket:azelma", + "Azelma", + "azelma", + "p303_023_enhanced.wav", + "60e3d26cdf2efdec5df712152c839928f4d5522821e6554ae11fd96c57ab1026" + ), + bundled_voice!( + "pocket:george", + "George", + "george", + "p315_023_enhanced.wav", + "29a41f93bf5236e5b21501091d7774c255d5f3d4e62fa4f9fdf0a92a793c84ae" + ), + PocketVoiceSpec { + key: MARY_VOICE_KEY, + display_name: "Mary", + reference_file: "reference_sample.wav", + upstream_file: "vctk/p333_023_enhanced.wav", + sha256: "a35b0468382218e9f37a9a7494d1e4b74deaf18d7ced22265b4e325bb55c183f", + bytes: None, + }, + bundled_voice!( + "pocket:jane", + "Jane", + "jane", + "p339_023_enhanced.wav", + "2f12e7f155eb3118f55425394f1b049e5b1b67bdc9b3932c8ba4521420aeb84a" + ), + bundled_voice!( + "pocket:michael", + "Michael", + "michael", + "p360_023_enhanced.wav", + "b6743e9195e5e3fd34fe9d1633ae93f7ffab787b249e45f6467d7d6f7a6ee6ad" + ), + bundled_voice!( + "pocket:eve", + "Eve", + "eve", + "p361_023_enhanced.wav", + "396e7cbd066b0f3fb6d67fa26e7904076958239d736d4390f15b5fe88feb14cd" + ), +]; + +pub(super) fn source_url(voice: &PocketVoiceSpec) -> String { + format!( + "https://huggingface.co/kyutai/tts-voices/blob/{VCTK_REVISION}/{}", + voice.upstream_file + ) +} diff --git a/desktop/src-tauri/src/huddle/tts_voice_selection_tests.rs b/desktop/src-tauri/src/huddle/tts_voice_selection_tests.rs new file mode 100644 index 0000000000..45662c9921 --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_voice_selection_tests.rs @@ -0,0 +1,385 @@ +use super::*; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::Arc; + +fn inert_pipeline(cancel: Arc) -> TtsPipeline { + let (text_tx, text_rx) = std::sync::mpsc::sync_channel(TEXT_QUEUE_DEPTH); + let shutdown = Arc::new(AtomicBool::new(false)); + let worker_shutdown = Arc::clone(&shutdown); + let thread = std::thread::spawn(move || { + while !worker_shutdown.load(Ordering::Acquire) { + let _ = text_rx.recv_timeout(RECV_TIMEOUT); + } + }); + TtsPipeline { + text_tx, + tts_active: Arc::new(AtomicBool::new(false)), + shutdown, + cancel, + voice_cancel: Arc::new(AtomicBool::new(false)), + voice: Arc::new(std::sync::Mutex::new("reference_sample".to_string())), + voice_generation: Arc::new(AtomicU64::new(1)), + voice_change_ack: Arc::new(std::sync::Mutex::new(None)), + thread: Some(thread), + } +} + +#[test] +fn selecting_a_voice_raises_only_the_internal_cancel_and_retains_the_engine_handle() { + let cancel = Arc::new(AtomicBool::new(false)); + let pipeline = inert_pipeline(Arc::clone(&cancel)); + + let _acknowledged = pipeline.select_voice("eve"); + + assert!(!cancel.load(Ordering::Acquire)); + assert!(pipeline.voice_cancel.load(Ordering::Acquire)); + assert_eq!( + pipeline + .voice + .lock() + .unwrap_or_else(|error| error.into_inner()) + .as_str(), + "eve" + ); +} + +#[test] +fn reconciling_an_unpublished_pipeline_does_not_cancel_its_first_message() { + let cancel = Arc::new(AtomicBool::new(false)); + let pipeline = inert_pipeline(Arc::clone(&cancel)); + + pipeline.select_voice_before_publish("eve"); + + assert!(!cancel.load(Ordering::Acquire)); + assert_eq!( + pipeline + .voice + .lock() + .unwrap_or_else(|error| error.into_inner()) + .as_str(), + "eve" + ); +} + +#[test] +fn received_text_reconciles_a_voice_changed_while_the_worker_was_waiting() { + let model_dir = tempfile::tempdir().expect("temp model dir"); + let bundled_voice = + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("resources/pocket-voices/eve.wav"); + std::fs::copy( + &bundled_voice, + model_dir.path().join("reference_sample.wav"), + ) + .expect("Mary test voice"); + std::fs::copy(&bundled_voice, model_dir.path().join("eve.wav")).expect("Eve test voice"); + + let selected_voice = Arc::new(std::sync::Mutex::new("reference_sample".to_string())); + let mut style = + load_voice_style(&model_dir.path().join("reference_sample.wav")).expect("initial style"); + let waiting = Arc::new(std::sync::Barrier::new(2)); + let (text_tx, text_rx) = std::sync::mpsc::channel(); + let worker_voice = Arc::clone(&selected_voice); + let worker_waiting = Arc::clone(&waiting); + let worker_model_dir = model_dir.path().to_path_buf(); + let worker = std::thread::spawn(move || { + let mut voice_name = "reference_sample".to_string(); + worker_waiting.wait(); + let text = text_rx.recv().expect("first queued text"); + assert!(reconcile_selected_voice( + &worker_model_dir, + &worker_voice, + &mut voice_name, + &mut style, + )); + (text, voice_name) + }); + + waiting.wait(); + *selected_voice.lock().expect("selected voice") = "eve".to_string(); + text_tx + .send("first message".to_string()) + .expect("queue first message"); + + assert_eq!( + worker.join().expect("worker"), + ("first message".to_string(), "eve".to_string()) + ); +} + +#[test] +fn corrupt_selected_voice_falls_back_to_mary() { + let model_dir = tempfile::tempdir().expect("temp model dir"); + let bundled_voice = + std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("resources/pocket-voices/eve.wav"); + std::fs::copy(bundled_voice, model_dir.path().join("reference_sample.wav")) + .expect("Mary test voice"); + std::fs::write(model_dir.path().join("eve.wav"), b"not a wave") + .expect("corrupt selected voice"); + + let selected_voice = std::sync::Mutex::new("eve".to_string()); + let mut voice_name = "reference_sample".to_string(); + let mut style = + load_voice_style(&model_dir.path().join("reference_sample.wav")).expect("Mary style"); + + assert!(reconcile_selected_voice( + model_dir.path(), + &selected_voice, + &mut voice_name, + &mut style, + )); + assert_eq!(voice_name, DEFAULT_VOICE); + assert_eq!( + selected_voice + .lock() + .unwrap_or_else(|error| error.into_inner()) + .as_str(), + DEFAULT_VOICE + ); +} + +#[test] +fn an_in_hand_post_change_message_survives_cancellation() { + let selected_voice = Arc::new(std::sync::Mutex::new("reference_sample".to_string())); + let voice_generation = AtomicU64::new(1); + let barge_in = AtomicBool::new(false); + let voice_cancel = Arc::new(AtomicBool::new(false)); + let voice_change_ack = Arc::new(std::sync::Mutex::new(None)); + let (text_tx, text_rx) = std::sync::mpsc::sync_channel(1); + let mut acknowledged = begin_voice_change( + &selected_voice, + &voice_generation, + &voice_cancel, + &voice_change_ack, + "eve", + ) + .expect("voice changed"); + assert!(voice_cancel.load(Ordering::Acquire)); + assert!(matches!( + acknowledged.try_recv(), + Err(tokio::sync::oneshot::error::TryRecvError::Empty) + )); + acknowledge_voice_change(&voice_change_ack, &voice_cancel); + assert!(matches!( + acknowledged.try_recv(), + Err(tokio::sync::oneshot::error::TryRecvError::Empty) + )); + text_tx + .send(QueuedText { + generation: voice_generation.load(Ordering::Acquire), + route_id: 1, + text: "new message".to_string(), + }) + .expect("new message"); + let mut current_text = Some(text_rx.recv().expect("in-hand new message")); + + let shutdown = AtomicBool::new(false); + let active = AtomicBool::new(true); + let mut deferred_text = VecDeque::from([ + QueuedText { + generation: 1, + route_id: 2, + text: "old message".to_string(), + }, + QueuedText { + generation: voice_generation.load(Ordering::Acquire), + route_id: 3, + text: "later new message".to_string(), + }, + ]); + assert!(handle_cancel_or_shutdown( + (&barge_in, &voice_cancel), + &shutdown, + &active, + (&text_rx, &mut deferred_text, &mut current_text), + &voice_change_ack, + None, + None, + )); + acknowledge_voice_change(&voice_change_ack, &voice_cancel); + acknowledged.blocking_recv().expect("voice change ack"); + + assert_eq!( + deferred_text + .pop_front() + .expect("preserved post-change message") + .text, + "new message" + ); + assert_eq!( + deferred_text + .pop_front() + .expect("later post-change message") + .text, + "later new message" + ); + assert!(text_rx.try_recv().is_err()); +} + +#[test] +fn superseding_voice_change_removes_earlier_deferred_messages() { + let selected_voice = std::sync::Mutex::new("reference_sample".to_string()); + let voice_generation = AtomicU64::new(1); + let barge_in = AtomicBool::new(false); + let voice_cancel = AtomicBool::new(false); + let voice_change_ack = Arc::new(std::sync::Mutex::new(None)); + let (_text_tx, text_rx) = std::sync::mpsc::channel(); + let shutdown = AtomicBool::new(false); + let active = AtomicBool::new(true); + let mut deferred_text = VecDeque::new(); + let mut current_text = None; + + let first = begin_voice_change( + &selected_voice, + &voice_generation, + &voice_cancel, + &voice_change_ack, + "eve", + ) + .expect("first voice change"); + deferred_text.push_back(QueuedText { + generation: voice_generation.load(Ordering::Acquire), + route_id: 4, + text: "message for Eve".to_string(), + }); + assert!(handle_cancel_or_shutdown( + (&barge_in, &voice_cancel), + &shutdown, + &active, + (&text_rx, &mut deferred_text, &mut current_text), + &voice_change_ack, + None, + None, + )); + acknowledge_voice_change(&voice_change_ack, &voice_cancel); + first.blocking_recv().expect("first acknowledgement"); + + let _second = begin_voice_change( + &selected_voice, + &voice_generation, + &voice_cancel, + &voice_change_ack, + "reference_sample", + ) + .expect("second voice change"); + assert!(handle_cancel_or_shutdown( + (&barge_in, &voice_cancel), + &shutdown, + &active, + (&text_rx, &mut deferred_text, &mut current_text), + &voice_change_ack, + None, + None, + )); + + assert!(deferred_text.is_empty()); +} + +#[test] +fn barge_in_clears_deferred_voice_change_messages() { + let barge_in = AtomicBool::new(true); + let voice_cancel = AtomicBool::new(false); + let shutdown = AtomicBool::new(false); + let active = AtomicBool::new(true); + let voice_change_ack = Arc::new(std::sync::Mutex::new(None)); + let (_text_tx, text_rx) = std::sync::mpsc::channel(); + let mut deferred_text = VecDeque::from([QueuedText { + generation: 2, + route_id: 5, + text: "deferred message".to_string(), + }]); + let mut current_text = None; + + assert!(handle_cancel_or_shutdown( + (&barge_in, &voice_cancel), + &shutdown, + &active, + (&text_rx, &mut deferred_text, &mut current_text), + &voice_change_ack, + None, + None, + )); + + assert!(deferred_text.is_empty()); +} + +#[test] +fn barge_in_during_a_voice_change_clears_post_change_messages() { + let selected_voice = std::sync::Mutex::new("reference_sample".to_string()); + let voice_generation = AtomicU64::new(1); + let barge_in = AtomicBool::new(false); + let voice_cancel = AtomicBool::new(false); + let voice_change_ack = Arc::new(std::sync::Mutex::new(None)); + let (_text_tx, text_rx) = std::sync::mpsc::channel(); + let shutdown = AtomicBool::new(false); + let active = AtomicBool::new(true); + let mut deferred_text = VecDeque::new(); + let mut current_text = None; + + let _acknowledged = begin_voice_change( + &selected_voice, + &voice_generation, + &voice_cancel, + &voice_change_ack, + "eve", + ) + .expect("voice change"); + deferred_text.push_back(QueuedText { + generation: voice_generation.load(Ordering::Acquire), + route_id: 6, + text: "post-change message".to_string(), + }); + barge_in.store(true, Ordering::Release); + + assert!(handle_cancel_or_shutdown( + (&barge_in, &voice_cancel), + &shutdown, + &active, + (&text_rx, &mut deferred_text, &mut current_text), + &voice_change_ack, + None, + None, + )); + assert!(deferred_text.is_empty()); +} + +#[test] +fn a_sender_captured_before_voice_change_is_stale_even_if_it_sends_after_drain() { + let selected_voice = std::sync::Mutex::new("reference_sample".to_string()); + let voice_generation = Arc::new(AtomicU64::new(1)); + let barge_in = AtomicBool::new(false); + let voice_cancel = AtomicBool::new(false); + let voice_change_ack = Arc::new(std::sync::Mutex::new(None)); + let (text_tx, text_rx) = std::sync::mpsc::sync_channel(1); + let old_sender = TtsTextSender { + text_tx, + generation: voice_generation.load(Ordering::Acquire), + }; + let shutdown = AtomicBool::new(false); + let active = AtomicBool::new(true); + let mut deferred_text = VecDeque::new(); + let mut current_text = None; + + let _acknowledged = begin_voice_change( + &selected_voice, + &voice_generation, + &voice_cancel, + &voice_change_ack, + "eve", + ) + .expect("voice change"); + assert!(handle_cancel_or_shutdown( + (&barge_in, &voice_cancel), + &shutdown, + &active, + (&text_rx, &mut deferred_text, &mut current_text), + &voice_change_ack, + None, + None, + )); + old_sender + .send(7, "late old message".to_string()) + .expect("late send"); + let late = text_rx.recv().expect("late queued text"); + + assert!(late.generation < voice_generation.load(Ordering::Acquire)); +} diff --git a/desktop/src-tauri/src/huddle/tts_voice_transition.rs b/desktop/src-tauri/src/huddle/tts_voice_transition.rs new file mode 100644 index 0000000000..81b33672d3 --- /dev/null +++ b/desktop/src-tauri/src/huddle/tts_voice_transition.rs @@ -0,0 +1,206 @@ +use std::{ + collections::VecDeque, + path::Path, + sync::{ + atomic::{AtomicBool, AtomicU64, Ordering}, + mpsc::{self, SyncSender}, + Arc, Mutex, + }, +}; + +use crate::huddle::pocket::{load_voice_style, VoiceStyle, DEFAULT_VOICE, VOICE_FILE_EXT}; + +#[derive(Debug)] +pub(super) struct PendingVoiceChange { + pub(super) generation: u64, + acknowledged: tokio::sync::oneshot::Sender<()>, +} + +pub(super) type VoiceChangeAck = Arc>>; +pub(super) type WorkerVoiceState = (Arc>, Arc, VoiceChangeAck); +pub(super) type WorkerCancelSignals = (Arc, Arc); +pub(super) type CancelTextState<'a> = ( + &'a mpsc::Receiver, + &'a mut VecDeque, + &'a mut Option, +); +pub(super) type CancelSignals<'a> = (&'a AtomicBool, &'a AtomicBool); + +#[derive(Debug)] +pub(super) struct QueuedText { + pub(super) generation: u64, + pub(super) route_id: u64, + pub(super) text: String, +} + +#[derive(Clone, Debug)] +pub(crate) struct TtsTextSender { + pub(super) text_tx: SyncSender, + pub(super) generation: u64, +} + +impl TtsTextSender { + pub(crate) fn send(&self, route_id: u64, text: String) -> Result<(), String> { + self.text_tx + .send(QueuedText { + generation: self.generation, + route_id, + text, + }) + .map_err(|error| error.to_string()) + } +} + +pub(super) fn begin_voice_change( + selected_voice: &Mutex, + voice_generation: &AtomicU64, + voice_cancel: &AtomicBool, + voice_change_ack: &VoiceChangeAck, + voice: &str, +) -> Option> { + let mut pending_ack = voice_change_ack + .lock() + .unwrap_or_else(|error| error.into_inner()); + let mut selected = selected_voice + .lock() + .unwrap_or_else(|error| error.into_inner()); + if selected.as_str() == voice { + return None; + } + + let (sender, receiver) = tokio::sync::oneshot::channel(); + voice_cancel.store(true, Ordering::Release); + let generation = voice_generation.fetch_add(1, Ordering::AcqRel) + 1; + if let Some(superseded) = pending_ack.replace(PendingVoiceChange { + generation, + acknowledged: sender, + }) { + let _ = superseded.acknowledged.send(()); + } + *selected = voice.to_string(); + Some(receiver) +} + +pub(super) fn acknowledge_voice_change( + voice_change_ack: &VoiceChangeAck, + voice_cancel: &AtomicBool, +) { + let mut pending_ack = voice_change_ack + .lock() + .unwrap_or_else(|error| error.into_inner()); + if voice_cancel.load(Ordering::Acquire) { + return; + } + if let Some(pending) = pending_ack.take() { + let _ = pending.acknowledged.send(()); + } +} + +pub(super) fn finish_voice_change_ack(voice_change_ack: &VoiceChangeAck) { + if let Some(pending) = voice_change_ack + .lock() + .unwrap_or_else(|error| error.into_inner()) + .take() + { + let _ = pending.acknowledged.send(()); + } +} + +pub(super) fn reconcile_selected_voice( + model_dir: &Path, + selected_voice: &Mutex, + voice_name: &mut String, + style: &mut VoiceStyle, +) -> bool { + let requested_voice = selected_voice + .lock() + .unwrap_or_else(|error| error.into_inner()) + .clone(); + if requested_voice == *voice_name { + return true; + } + + let requested_path = voice_path(model_dir, &requested_voice); + match load_voice_style(&requested_path) { + Ok(requested_style) => { + *style = requested_style; + *voice_name = requested_voice; + true + } + Err(_) => { + eprintln!("buzz-desktop: tts stage=voice_switch status=fallback reason=voice_style"); + let fallback_path = model_dir.join(format!("{DEFAULT_VOICE}.{VOICE_FILE_EXT}")); + match load_voice_style(&fallback_path) { + Ok(fallback_style) => { + *style = fallback_style; + *voice_name = DEFAULT_VOICE.to_string(); + *selected_voice + .lock() + .unwrap_or_else(|lock_error| lock_error.into_inner()) = + DEFAULT_VOICE.to_string(); + true + } + Err(_) => { + eprintln!( + "buzz-desktop: tts stage=voice_switch status=failed reason=fallback_voice_style" + ); + false + } + } + } + } +} + +pub(super) fn voice_path(model_dir: &Path, voice: &str) -> std::path::PathBuf { + let path = Path::new(voice); + if path.is_absolute() { + path.to_path_buf() + } else { + model_dir.join(format!("{voice}.{VOICE_FILE_EXT}")) + } +} + +pub(super) fn retain_cancelled_text( + deferred_text: &mut VecDeque, + current_text: &mut Option, + text_rx: &mpsc::Receiver, + preserve_generation: Option, +) { + if let Some(generation) = preserve_generation { + deferred_text.retain(|text| { + let preserve = text.generation >= generation; + if !preserve { + log_cancelled_route(text.route_id, "voice_switch"); + } + preserve + }); + if let Some(text) = current_text.take() { + if text.generation >= generation { + deferred_text.push_front(text); + } else { + log_cancelled_route(text.route_id, "voice_switch"); + } + } + while let Ok(text) = text_rx.try_recv() { + if text.generation >= generation { + deferred_text.push_back(text); + } else { + log_cancelled_route(text.route_id, "voice_switch"); + } + } + } else { + for text in deferred_text.drain(..) { + log_cancelled_route(text.route_id, "barge_in"); + } + if let Some(text) = current_text.take() { + log_cancelled_route(text.route_id, "barge_in"); + } + while let Ok(text) = text_rx.try_recv() { + log_cancelled_route(text.route_id, "barge_in"); + } + } +} + +fn log_cancelled_route(route_id: u64, reason: &str) { + eprintln!("buzz-desktop: tts stage=queue status=dropped reason={reason} route_id={route_id}"); +} diff --git a/desktop/src-tauri/src/identity_storage.rs b/desktop/src-tauri/src/identity_storage.rs new file mode 100644 index 0000000000..b39c1a0331 --- /dev/null +++ b/desktop/src-tauri/src/identity_storage.rs @@ -0,0 +1,62 @@ +use nostr::Keys; + +use crate::app_state::AppState; + +/// Durable location of the active human identity. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[repr(u8)] +pub(crate) enum IdentityStorage { + Ephemeral = 0, + SystemKeyring = 1, + LocalFile = 2, + Environment = 3, +} + +impl IdentityStorage { + pub(crate) fn as_str(self) -> &'static str { + match self { + Self::Ephemeral => "ephemeral", + Self::SystemKeyring => "system-keyring", + Self::LocalFile => "local-file", + Self::Environment => "environment", + } + } + + fn from_u8(value: u8) -> Self { + match value { + 1 => Self::SystemKeyring, + 2 => Self::LocalFile, + 3 => Self::Environment, + _ => Self::Ephemeral, + } + } +} + +impl AppState { + pub(crate) fn identity_storage(&self) -> IdentityStorage { + IdentityStorage::from_u8( + self.identity_storage + .load(std::sync::atomic::Ordering::Acquire), + ) + } + + pub(crate) fn set_identity_storage(&self, storage: IdentityStorage) { + self.identity_storage + .store(storage as u8, std::sync::atomic::Ordering::Release); + } +} + +/// Recovery state produced by identity resolution. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum RecoveryState { + None, + Lost, + KeyringLocked, +} + +/// Identity and persistence metadata produced by startup resolution. +pub(crate) struct ResolvedIdentity { + pub(crate) keys: Keys, + pub(crate) recovery: RecoveryState, + pub(crate) storage: IdentityStorage, +} diff --git a/desktop/src-tauri/src/key_backup.rs b/desktop/src-tauri/src/key_backup.rs new file mode 100644 index 0000000000..f97bf95a67 --- /dev/null +++ b/desktop/src-tauri/src/key_backup.rs @@ -0,0 +1,234 @@ +//! NIP-49 encrypted local key backup. +//! +//! Creates a password-encrypted `ncryptsec` backup of the user's identity key +//! for a user-selected local file. The blob is **local-only by contract**: it must +//! never be transmitted to a relay on any path. That contract is enforced at +//! runtime by [`crate::egress_guard`] (wired into every relay event-body +//! constructor and the native websocket send loop) and structurally by the +//! source-allowlist scan in this module's tests. +//! +//! Creation decrypt-verifies the fresh blob against the live identity before +//! returning it. Portable copies use atomic, owner-only file writes. + +use nostr::nips::nip49::{EncryptedSecretKey, KeySecurity}; +use nostr::{FromBech32, Keys, ToBech32}; + +/// Bech32 prefix of NIP-49 encrypted secret keys. Import routing is +/// case-insensitive because bech32 permits all-uppercase encodings. +pub const NCRYPTSEC_HRP: &str = "ncryptsec1"; + +/// scrypt cost for new backups (2^18 — Gossip's desktop default, ~256 MiB). +/// The blob self-describes its cost, so this can be raised later without +/// breaking existing backups. +pub const BACKUP_LOG_N: u8 = 18; + +/// Highest scrypt cost accepted when decrypting an untrusted backup. +/// +/// NIP-49 intentionally leaves `log_n` client-selected. Capping it at the tier +/// Buzz itself emits keeps generated and upstream-compatible lower-cost backups +/// readable without allowing a crafted payload to request unbounded memory +/// before password authentication. +pub const MAX_VERIFY_LOG_N: u8 = BACKUP_LOG_N; + +/// Filename of the app-managed canonical backup inside the app data dir. +pub const BACKUP_FILE_NAME: &str = "identity.ncryptsec"; + +/// Default number of words in a generated backup passphrase. Three words +/// from a 1296-word list ≈ 31 bits of entropy before the scrypt work factor. +pub const DEFAULT_PASSPHRASE_WORDS: usize = 3; + +/// Bounds for the generator's word-count control. At the lower bound a draw +/// can fall below [`MIN_PASSPHRASE_LEN`] (three 3-char words), so +/// [`generate_passphrase`] re-draws until the phrase meets the minimum. +pub const MIN_PASSPHRASE_WORDS: usize = 3; +pub const MAX_PASSPHRASE_WORDS: usize = 10; + +/// EFF short wordlist 2.0 (1296 words, one per line). +const WORDLIST: &str = include_str!("assets/eff_short_wordlist_2_0.txt"); + +/// Minimum length for a user-chosen passphrase. +pub const MIN_PASSPHRASE_LEN: usize = 12; + +/// Encrypt the identity secret key under `password` and verify the result. +/// +/// Returns the bech32 `ncryptsec1…` string. The fresh blob is decrypted and +/// its derived pubkey compared to the live identity **before** returning, so +/// a returned blob is always provably recoverable with the same password. +pub fn create_backup_blob(keys: &Keys, password: &str, log_n: u8) -> Result { + let secret_key = keys.secret_key(); + + let encrypted = EncryptedSecretKey::new(secret_key, password, log_n, KeySecurity::Unknown) + .map_err(|e| format!("encrypt key backup: {e}"))?; + + let ncryptsec = encrypted + .to_bech32() + .map_err(|e| format!("encode ncryptsec: {e}"))?; + + // Integrity check: decrypt the fresh blob and confirm it recovers the + // exact live identity. A corrupted or mis-encrypted blob must never be + // shown to the user as a "backup". This is the second, deliberate KDF + // invocation of the one-artifact-per-action contract. + verify_backup_blob(&ncryptsec, password, &keys.public_key())?; + + Ok(ncryptsec) +} + +/// Decrypt `ncryptsec` with `password` and assert it recovers a key whose +/// public key equals `expected_pubkey`. +pub fn verify_backup_blob( + ncryptsec: &str, + password: &str, + expected_pubkey: &nostr::PublicKey, +) -> Result<(), String> { + let encrypted = parse_ncryptsec(ncryptsec)?; + let recovered = encrypted + .decrypt(password) + .map_err(|e| format!("verify key backup (decrypt): {e}"))?; + let recovered_keys = Keys::new(recovered); + if recovered_keys.public_key() != *expected_pubkey { + return Err("verify key backup: decrypted key does not match identity".to_string()); + } + Ok(()) +} + +/// Parse a bech32 `ncryptsec1…` string, rejecting anything that is not a +/// structurally valid NIP-49 payload. +pub fn parse_ncryptsec(input: &str) -> Result { + EncryptedSecretKey::from_bech32(input.trim()).map_err(|e| format!("invalid ncryptsec: {e}")) +} + +/// Decrypt an `ncryptsec1…` string with `password` into identity keys. +pub fn decrypt_ncryptsec(input: &str, password: &str) -> Result { + let encrypted = parse_ncryptsec(input)?; + let log_n = encrypted.log_n(); + if log_n > MAX_VERIFY_LOG_N { + return Err(format!( + "unsupported backup KDF cost: log_n {log_n} exceeds maximum {MAX_VERIFY_LOG_N}" + )); + } + let secret_key = encrypted + .decrypt(password) + .map_err(|_| "wrong backup password or damaged key backup".to_string())?; + Ok(Keys::new(secret_key)) +} + +/// Recover identity keys from either an encrypted NIP-49 backup or the raw +/// nsec/hex formats accepted before encrypted imports were added. +pub fn recover_keys_from_input(input: &str, password: Option<&str>) -> Result { + let trimmed = input.trim(); + let is_ncryptsec = trimmed + .get(..NCRYPTSEC_HRP.len()) + .is_some_and(|prefix| prefix.eq_ignore_ascii_case(NCRYPTSEC_HRP)); + + if is_ncryptsec { + let password = password.ok_or_else(|| "key backup requires a password".to_string())?; + decrypt_ncryptsec(trimmed, password) + } else { + Keys::parse(trimmed).map_err(|e| format!("Invalid private key: {e}")) + } +} + +/// Path of the canonical app-managed backup file. +pub fn backup_file_path(data_dir: &std::path::Path) -> std::path::PathBuf { + data_dir.join(BACKUP_FILE_NAME) +} + +/// Atomically write `ncryptsec` to `path` with owner-only permissions, then +/// reread and byte-compare. Same crash-safety pattern as +/// `app_state::save_key_file`. +pub fn write_backup_file(path: &std::path::Path, ncryptsec: &str) -> Result<(), String> { + use atomic_write_file::AtomicWriteFile; + use std::io::Write; + + let mut file = AtomicWriteFile::open(path) + .map_err(|e| format!("open backup file for atomic write: {e}"))?; + + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + file.set_permissions(std::fs::Permissions::from_mode(0o600)) + .map_err(|e| format!("set backup file permissions: {e}"))?; + } + + file.write_all(ncryptsec.as_bytes()) + .map_err(|e| format!("write backup file: {e}"))?; + file.commit() + .map_err(|e| format!("commit backup file: {e}"))?; + + // Reread and byte-compare: only report success for bytes that are + // actually on disk. + let on_disk = std::fs::read_to_string(path).map_err(|e| format!("reread backup file: {e}"))?; + if on_disk != ncryptsec { + return Err("backup file verification failed: on-disk bytes differ".to_string()); + } + + Ok(()) +} + +/// Delete the app-managed backup if present. Missing files are already clean. +pub fn delete_backup_file(data_dir: &std::path::Path) -> Result<(), String> { + let path = backup_file_path(data_dir); + match std::fs::remove_file(path) { + Ok(()) => Ok(()), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()), + Err(e) => Err(format!("delete stale backup file: {e}")), + } +} + +/// Remove the app-managed backup only when an import changes identities. +pub fn cleanup_stale_backup( + previous: &nostr::PublicKey, + new: &nostr::PublicKey, + data_dir: &std::path::Path, +) -> Result<(), String> { + if previous != new { + delete_backup_file(data_dir)?; + } + Ok(()) +} + +/// Generate a passphrase of `word_count` EFF short-wordlist words joined by +/// `separator`, using OS entropy. +/// +/// `word_count` is clamped to `MIN_PASSPHRASE_WORDS..=MAX_PASSPHRASE_WORDS`. +/// Because a low-word-count draw can land under [`MIN_PASSPHRASE_LEN`] +/// (e.g. three 3-char words), whole phrases below the minimum are rejected +/// and re-drawn — the result always passes the same length gate applied to +/// user-chosen passphrases. Uses rejection sampling for a uniform +/// distribution over the 1296 words. +pub fn generate_passphrase(word_count: usize, separator: &str) -> Result { + let word_count = word_count.clamp(MIN_PASSPHRASE_WORDS, MAX_PASSPHRASE_WORDS); + let words: Vec<&str> = WORDLIST.lines().filter(|l| !l.is_empty()).collect(); + if words.len() != 1296 { + return Err(format!( + "wordlist corrupted: expected 1296 words, found {}", + words.len() + )); + } + + // At 3 words the under-length probability per draw is small, so a few + // attempts always suffice; the cap only guards against a logic bug + // becoming an infinite loop. + for _ in 0..128 { + let mut chosen: Vec<&str> = Vec::with_capacity(word_count); + while chosen.len() < word_count { + let mut buf = [0u8; 2]; + getrandom::getrandom(&mut buf).map_err(|e| format!("entropy source: {e}"))?; + let value = u16::from_le_bytes(buf); + // Rejection sampling: accept only values below the largest + // multiple of 1296 that fits in u16 (65536 - 65536 % 1296 = 64800). + if value < 64800 { + chosen.push(words[(value as usize) % 1296]); + } + } + let phrase = chosen.join(separator); + if phrase.chars().count() >= MIN_PASSPHRASE_LEN { + return Ok(phrase); + } + } + Err("could not generate a passphrase meeting the minimum length".to_string()) +} + +#[cfg(test)] +#[path = "key_backup_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/key_backup_tests.rs b/desktop/src-tauri/src/key_backup_tests.rs new file mode 100644 index 0000000000..b9713201e1 --- /dev/null +++ b/desktop/src-tauri/src/key_backup_tests.rs @@ -0,0 +1,230 @@ +use super::*; + +/// NIP-49 spec vector (same as rust-nostr's upstream test): decrypts with +/// password "nostr" at our call sites. +const SPEC_NCRYPTSEC: &str = "ncryptsec1qgg9947rlpvqu76pj5ecreduf9jxhselq2nae2kghhvd5g7dgjtcxfqtd67p9m0w57lspw8gsq6yphnm8623nsl8xn9j4jdzz84zm3frztj3z7s35vpzmqf6ksu8r89qk5z2zxfmu5gv8th8wclt0h4p"; +const SPEC_SECRET_HEX: &str = "3501454135014541350145413501453fefb02227e449e57cf4d3a3ce05378683"; + +/// Fast scrypt tier for tests. log_n 18 is exercised once in +/// `round_trip_at_production_cost`. +const FAST_LOG_N: u8 = 16; + +// ── Codec ───────────────────────────────────────────────────────────────────── + +#[test] +fn spec_vector_decrypts_at_our_call_site() { + let keys = decrypt_ncryptsec(SPEC_NCRYPTSEC, "nostr").unwrap(); + assert_eq!(keys.secret_key().to_secret_hex(), SPEC_SECRET_HEX); +} + +#[test] +fn round_trip_fast_tier() { + let keys = Keys::generate(); + let blob = create_backup_blob(&keys, "correct horse battery", FAST_LOG_N).unwrap(); + assert!(blob.starts_with("ncryptsec1")); + let recovered = decrypt_ncryptsec(&blob, "correct horse battery").unwrap(); + assert_eq!(recovered.public_key(), keys.public_key()); +} + +#[test] +fn round_trip_at_production_cost() { + // One log_n 18 round trip: proves the production constant works end to + // end (slow — several seconds — but deliberate; see plan D5). + let keys = Keys::generate(); + let blob = create_backup_blob(&keys, "production cost tier check", BACKUP_LOG_N).unwrap(); + let recovered = decrypt_ncryptsec(&blob, "production cost tier check").unwrap(); + assert_eq!(recovered.public_key(), keys.public_key()); +} + +#[test] +fn wrong_password_is_a_friendly_error() { + let keys = Keys::generate(); + let blob = create_backup_blob(&keys, "right password", FAST_LOG_N).unwrap(); + let err = decrypt_ncryptsec(&blob, "wrong password").unwrap_err(); + assert_eq!(err, "wrong backup password or damaged key backup"); +} + +#[test] +fn nfkc_cross_form_passphrase_round_trips() { + // "é" composed (U+00E9) vs decomposed (e + U+0301): NIP-49 mandates NFKC + // normalization, so a passphrase entered in either form must decrypt. + let keys = Keys::generate(); + let composed = "caf\u{00e9} passphrase"; + let decomposed = "cafe\u{0301} passphrase"; + assert_ne!(composed, decomposed); + let blob = create_backup_blob(&keys, composed, FAST_LOG_N).unwrap(); + let recovered = decrypt_ncryptsec(&blob, decomposed).unwrap(); + assert_eq!(recovered.public_key(), keys.public_key()); +} + +#[test] +fn parse_rejects_garbage_and_wrong_hrp() { + assert!(parse_ncryptsec("garbage").is_err()); + assert!(parse_ncryptsec("").is_err()); + // Valid bech32, wrong HRP (an nsec is not an encrypted backup). + let nsec = Keys::generate().secret_key().to_bech32().unwrap(); + assert!(parse_ncryptsec(&nsec).is_err()); + // Truncated blob. + assert!(parse_ncryptsec(&SPEC_NCRYPTSEC[..SPEC_NCRYPTSEC.len() - 10]).is_err()); +} + +#[test] +fn verify_backup_blob_catches_pubkey_mismatch() { + // Corrupted-blob simulation: the blob decrypts fine but recovers a key + // that is not the live identity — verification must fail. + let other = Keys::generate(); + let blob = create_backup_blob(&other, "some password", FAST_LOG_N).unwrap(); + let live = Keys::generate(); + let err = verify_backup_blob(&blob, "some password", &live.public_key()).unwrap_err(); + assert!(err.contains("does not match identity"), "{err}"); +} + +// ── Import key recovery ─────────────────────────────────────────────────────── + +#[test] +fn recover_keys_ncryptsec_happy_path() { + let keys = recover_keys_from_input(&format!(" {SPEC_NCRYPTSEC}\n"), Some("nostr")).unwrap(); + assert_eq!(keys.secret_key().to_secret_hex(), SPEC_SECRET_HEX); +} + +#[test] +fn recover_keys_ncryptsec_requires_password() { + let err = recover_keys_from_input(SPEC_NCRYPTSEC, None).unwrap_err(); + assert_eq!(err, "key backup requires a password"); +} + +#[test] +fn recover_keys_ncryptsec_wrong_password() { + let err = recover_keys_from_input(SPEC_NCRYPTSEC, Some("wrong")).unwrap_err(); + assert_eq!(err, "wrong backup password or damaged key backup"); +} + +#[test] +fn recover_keys_uppercase_ncryptsec_classifies_as_encrypted() { + let upper = SPEC_NCRYPTSEC.to_ascii_uppercase(); + assert_eq!( + recover_keys_from_input(&upper, None).unwrap_err(), + "key backup requires a password" + ); + let keys = recover_keys_from_input(&upper, Some("nostr")).unwrap(); + assert_eq!(keys.secret_key().to_secret_hex(), SPEC_SECRET_HEX); + + let mut mixed = SPEC_NCRYPTSEC.to_string(); + mixed.replace_range(0..1, "N"); + let err = recover_keys_from_input(&mixed, Some("nostr")).unwrap_err(); + assert!(err.contains("invalid ncryptsec"), "{err}"); +} + +#[test] +fn recover_keys_raw_nsec_path_unchanged() { + let keys = Keys::generate(); + let nsec = keys.secret_key().to_bech32().unwrap(); + let recovered = recover_keys_from_input(&nsec, Some("ignored")).unwrap(); + assert_eq!(recovered.public_key(), keys.public_key()); + let recovered = recover_keys_from_input(&nsec, None).unwrap(); + assert_eq!(recovered.public_key(), keys.public_key()); + assert!(recover_keys_from_input("garbage", None).is_err()); +} + +// ── File lifecycle ──────────────────────────────────────────────────────────── + +#[test] +fn write_backup_file_persists_0600_and_verifies() { + let dir = tempfile::tempdir().unwrap(); + let path = backup_file_path(dir.path()); + write_backup_file(&path, SPEC_NCRYPTSEC).unwrap(); + + let on_disk = std::fs::read_to_string(&path).unwrap(); + assert_eq!(on_disk, SPEC_NCRYPTSEC); + + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + let mode = std::fs::metadata(&path).unwrap().permissions().mode(); + assert_eq!(mode & 0o777, 0o600, "backup file must be owner-only"); + } +} + +#[test] +fn write_backup_file_overwrites_atomically() { + let dir = tempfile::tempdir().unwrap(); + let path = backup_file_path(dir.path()); + write_backup_file(&path, "ncryptsec1old").unwrap(); + write_backup_file(&path, SPEC_NCRYPTSEC).unwrap(); + assert_eq!(std::fs::read_to_string(&path).unwrap(), SPEC_NCRYPTSEC); + // No leftover temp files from the atomic write. + let entries: Vec<_> = std::fs::read_dir(dir.path()) + .unwrap() + .map(|e| e.unwrap().file_name()) + .collect(); + assert_eq!(entries, vec![std::ffi::OsString::from(BACKUP_FILE_NAME)]); +} + +#[test] +fn delete_backup_file_is_idempotent() { + let dir = tempfile::tempdir().unwrap(); + delete_backup_file(dir.path()).unwrap(); + let path = backup_file_path(dir.path()); + write_backup_file(&path, SPEC_NCRYPTSEC).unwrap(); + delete_backup_file(dir.path()).unwrap(); + assert!(!path.exists()); +} + +#[test] +fn cleanup_stale_backup_removes_only_on_identity_change() { + let dir = tempfile::tempdir().unwrap(); + let path = backup_file_path(dir.path()); + let a = Keys::generate().public_key(); + let b = Keys::generate().public_key(); + + write_backup_file(&path, SPEC_NCRYPTSEC).unwrap(); + cleanup_stale_backup(&a, &a, dir.path()).unwrap(); + assert!(path.exists(), "same identity must keep the backup"); + + cleanup_stale_backup(&a, &b, dir.path()).unwrap(); + assert!( + !path.exists(), + "identity change must remove the stale backup" + ); +} + +#[test] +fn generated_passphrase_respects_word_count_and_separator() { + let words: std::collections::HashSet<&str> = + WORDLIST.lines().filter(|l| !l.is_empty()).collect(); + assert_eq!(words.len(), 1296, "EFF short wordlist 2.0 has 1296 words"); + + for (count, separator) in [(3, "-"), (4, "-"), (6, " "), (5, "."), (10, "")] { + let phrase = generate_passphrase(count, separator).unwrap(); + if separator.is_empty() { + // No separator to split on; length gate below still applies. + } else { + let parts: Vec<&str> = phrase.split(separator).collect(); + assert_eq!(parts.len(), count); + for w in &parts { + assert!(words.contains(w), "unknown word {w:?}"); + } + } + assert!(phrase.chars().count() >= MIN_PASSPHRASE_LEN); + } +} + +#[test] +fn generated_passphrase_clamps_word_count() { + // Below the floor: clamped up to MIN_PASSPHRASE_WORDS, never shorter. + let phrase = generate_passphrase(1, "-").unwrap(); + assert_eq!(phrase.split('-').count(), MIN_PASSPHRASE_WORDS); + // Above the ceiling: clamped down to MAX_PASSPHRASE_WORDS. + let phrase = generate_passphrase(50, "-").unwrap(); + assert_eq!(phrase.split('-').count(), MAX_PASSPHRASE_WORDS); +} + +#[test] +fn generated_passphrases_are_not_repeated() { + // 3 words × ~10.3 bits each — a collision across 8 draws would indicate a + // broken entropy source, not bad luck. + let mut seen = std::collections::HashSet::new(); + for _ in 0..8 { + assert!(seen.insert(generate_passphrase(DEFAULT_PASSPHRASE_WORDS, "-").unwrap())); + } +} diff --git a/desktop/src-tauri/src/lib.rs b/desktop/src-tauri/src/lib.rs index 35f4eae866..c4b733e3e0 100644 --- a/desktop/src-tauri/src/lib.rs +++ b/desktop/src-tauri/src/lib.rs @@ -4,9 +4,13 @@ mod archive; mod builderlab; mod commands; mod deep_link; +mod egress_guard; mod event_sync; mod events; mod huddle; +mod identity_storage; +mod key_backup; +mod linux_media; mod managed_agents; mod media_proxy; #[cfg(feature = "mesh-llm")] @@ -28,6 +32,8 @@ mod reset; mod secret_store; mod shutdown; mod templates; +#[cfg(target_os = "macos")] +mod tray_menu; mod util; #[cfg(target_os = "linux")] pub mod webkit_rendering; @@ -59,14 +65,13 @@ use mesh_llm_stubs::*; #[cfg(all(feature = "mesh-llm", target_os = "macos"))] use shutdown::{hard_exit_after_mesh_shutdown, relaunch_after_mesh_shutdown}; use shutdown::{is_restart_request, shut_down_app}; -use std::sync::{ - atomic::{AtomicBool, Ordering}, - Arc, -}; -#[cfg(target_os = "macos")] -use tauri::Listener; +use std::sync::{atomic::AtomicBool, atomic::Ordering, Arc}; use tauri::{Emitter, Manager, RunEvent}; +#[cfg(target_os = "macos")] +use tauri::{Listener, WindowEvent}; use tauri_plugin_window_state::StateFlags; +#[cfg(target_os = "macos")] +use tray_menu::show_main_window; #[cfg(target_os = "macos")] const INITIAL_RENDER_READY_EVENT: &str = "initial-render-ready"; @@ -195,6 +200,11 @@ pub fn run() { return; } + // Linux/WebKitGTK needs media-stream settings and a + // permission-request handler for getUserMedia; no-op + // on macOS/Windows. + linux_media::enable_media_capture(&webview); + // macOS applies the restored geometry asynchronously. Wait // for several identical outer bounds and for React to // commit the startup surface before revealing it. @@ -360,6 +370,8 @@ pub fn run() { .manage(commands::pairing::PairingHandle::new()) .setup(move |app| { let app_handle = app.handle().clone(); + #[cfg(target_os = "macos")] + tray_menu::init(&app_handle)?; // ── Phase 2: boot-time sentinel wipe ────────────────────────────── // Must run before migrations and identity resolution so the wipe @@ -452,6 +464,18 @@ pub fn run() { *guard = Some(app_handle.clone()); } + let (tts_settings, tts_settings_load_error) = + huddle::tts_settings::load_for_app(&app_handle); + if let Ok(mut guard) = state.huddle_audio.tts.lock() { + *guard = tts_settings.clone(); + } + if let Ok(mut guard) = state.huddle_audio.tts_load_error.lock() { + *guard = tts_settings_load_error; + } + if let Ok(mut huddle) = state.huddle_state.lock() { + huddle.tts_enabled = tts_settings.agent_text_to_speech; + } + // Bring up the runtime-owned shared-compute coordinator before // saved agents are restored. Its lifetime is tied to the app, not // a UI mount; it publishes discovery and reconciles membership for @@ -657,6 +681,10 @@ pub fn run() { title_bar_double_click, get_identity, get_nsec, + generate_backup_passphrase, + create_ncryptsec_backup, + verify_ncryptsec_backup, + save_ncryptsec_copy, import_identity, persist_current_identity, get_profile, @@ -820,6 +848,12 @@ pub fn run() { update_team, delete_team, export_agent_snapshot, + card_mint_key_status, + card_mint_save_openai_key, + mint_agent_card, + save_agent_card, + list_agent_cards, + load_agent_card, preview_agent_snapshot_import, confirm_agent_snapshot_import, encode_agent_snapshot_for_send, @@ -858,6 +892,12 @@ pub fn run() { download_voice_models, get_model_status, set_tts_enabled, + huddle::tts_settings::get_tts_settings, + huddle::tts_settings::list_voice_registry, + huddle::tts_settings::set_pocket_voice, + huddle::tts_settings::preview_pocket_voice, + huddle::tts_settings::import_pocket_voice, + huddle::tts_settings::delete_pocket_voice, speak_agent_message, add_agent_to_huddle, check_pipeline_hotstart, @@ -895,6 +935,14 @@ pub fn run() { archive::read_unindexed_observer_rows, is_auto_update_supported, set_window_vibrancy, + #[cfg(target_os = "macos")] + tray_menu::clear_tray_agent_activity, + #[cfg(target_os = "macos")] + tray_menu::requeue_tray_actions, + #[cfg(target_os = "macos")] + tray_menu::take_tray_actions, + #[cfg(target_os = "macos")] + tray_menu::update_tray_agent_activity, ]) .build(tauri::generate_context!()) .expect("error while building tauri application"); @@ -907,6 +955,22 @@ pub fn run() { let run_shutdown_done = Arc::clone(&shutdown_done); let restart_requested = Arc::new(AtomicBool::new(false)); app.run(move |app_handle, event| match event { + #[cfg(target_os = "macos")] + RunEvent::Reopen { .. } => show_main_window(app_handle), + #[cfg(target_os = "macos")] + RunEvent::WindowEvent { + label, + event: WindowEvent::CloseRequested { api, .. }, + .. + } if label == "main" => { + // Keep the webview alive so Buzz can be reopened from its tray menu. + api.prevent_close(); + if let Some(window) = app_handle.get_webview_window("main") { + if let Err(error) = window.hide() { + eprintln!("buzz-desktop: failed to hide main window: {error}"); + } + } + } RunEvent::ExitRequested { code, .. } => { if is_restart_request(code) { restart_requested.store(true, Ordering::SeqCst); diff --git a/desktop/src-tauri/src/linux_media.rs b/desktop/src-tauri/src/linux_media.rs new file mode 100644 index 0000000000..240e2f8a77 --- /dev/null +++ b/desktop/src-tauri/src/linux_media.rs @@ -0,0 +1,146 @@ +//! Linux-only: enable media capture (`getUserMedia`) in the WebKitGTK webview. +//! +//! On macOS (WKWebView) and Windows (WebView2) the media-permission prompt is +//! routed to the OS automatically, so microphone/camera capture "just works". +//! WebKitGTK is different on two counts, and both must be handled or capture +//! fails on Linux only: +//! +//! * `enable-media-stream` is **off by default**, so `navigator.mediaDevices` +//! never exposes a working `getUserMedia`; and +//! * the default `permission-request` handler **denies every request**, so even +//! with media-stream on, the call rejects with `NotAllowedError`. +//! +//! This module reaches the underlying `webkit2gtk::WebView` via +//! [`tauri::Webview::with_webview`], enables media-stream, and installs a +//! `permission-request` handler that is **deny-by-default**: a `UserMedia` +//! request is allowed only when it comes from a trusted app origin and asks for +//! an audio and/or video device. Tauri does not restrict navigation by default, +//! so without the origin check any document that ended up in this webview would +//! inherit silent mic/camera access for the process lifetime. +//! +//! Buzz's AppImage pins `GDK_BACKEND=x11` (see [`crate::webkit_rendering`]), +//! which is the backend WebKitGTK media capture is reliable on. + +/// The origin Tauri serves the packaged app from on Linux. +/// Consumed only by linux-gated [`enable_media_capture`]; kept compiling on all +/// platforms so the unit tests run everywhere. +#[cfg_attr(not(target_os = "linux"), allow(dead_code))] +const PROD_ORIGIN: &str = "tauri://localhost"; + +/// The Vite dev-server origin (`devUrl` in `tauri.conf.json`, `strictPort` +/// 1420 in `vite.config.ts`). Only trusted in debug builds. +#[cfg(debug_assertions)] +#[cfg_attr(not(target_os = "linux"), allow(dead_code))] +const DEV_ORIGIN: &str = "http://localhost:1420"; + +/// Whether `uri` (the webview's current document URI) is a trusted app origin +/// allowed to use mic/camera. Matches the origin exactly or as a path prefix so +/// `tauri://localhost.evil.com` and `http://localhost:14200` do not slip +/// through. Pure and platform-independent so it can be unit-tested everywhere. +#[cfg_attr(not(target_os = "linux"), allow(dead_code))] +fn is_trusted_media_origin(uri: &str) -> bool { + fn matches(uri: &str, origin: &str) -> bool { + uri == origin + || uri + .strip_prefix(origin) + .is_some_and(|rest| rest.starts_with('/')) + } + + if matches(uri, PROD_ORIGIN) { + return true; + } + #[cfg(debug_assertions)] + if matches(uri, DEV_ORIGIN) { + return true; + } + false +} + +/// Enable microphone/camera capture for `webview` if it is running on +/// WebKitGTK. A no-op on every non-Linux target, so callers can invoke it +/// unconditionally from shared startup code. +#[cfg(target_os = "linux")] +pub fn enable_media_capture(webview: &tauri::Webview) { + use webkit2gtk::{ + glib::prelude::Cast, PermissionRequestExt, SettingsExt, UserMediaPermissionRequest, + UserMediaPermissionRequestExt, WebViewExt, + }; + + // `with_webview` runs the closure on the UI thread, which GTK calls + // require. It errors only if the platform webview is unavailable. + let result = webview.with_webview(|platform_webview| { + // On Linux this is the underlying `webkit2gtk::WebView`. + let webview = platform_webview.inner(); + + if let Some(settings) = WebViewExt::settings(&webview) { + settings.set_enable_media_stream(true); + } + + // Deny-by-default: allow only mic/camera requests from a trusted app + // origin; deny everything else (still returning `true` so WebKit's + // auto-deny default does not also run). Non-`UserMedia` requests return + // `false` and keep their default handling. + webview.connect_permission_request(|wv, request| { + let Some(request) = request.downcast_ref::() else { + return false; + }; + + let uri = wv.uri().map(|u| u.to_string()).unwrap_or_default(); + let for_device = request.is_for_audio_device() || request.is_for_video_device(); + + if for_device && is_trusted_media_origin(&uri) { + request.allow(); + } else { + request.deny(); + } + true + }); + }); + + if let Err(error) = result { + eprintln!("buzz-desktop: could not enable WebKitGTK media capture: {error}"); + } +} + +/// No-op stub so shared startup code can call [`enable_media_capture`] on every +/// platform. macOS and Windows route media permissions through the OS. +#[cfg(not(target_os = "linux"))] +pub fn enable_media_capture(_webview: &tauri::Webview) {} + +#[cfg(test)] +mod tests { + use super::is_trusted_media_origin; + + #[test] + fn allows_production_app_origin() { + assert!(is_trusted_media_origin("tauri://localhost")); + assert!(is_trusted_media_origin( + "tauri://localhost/channels/general" + )); + } + + #[test] + fn denies_untrusted_origins() { + assert!(!is_trusted_media_origin("")); + assert!(!is_trusted_media_origin("https://evil.example.com")); + // Prefix look-alikes must not slip through. + assert!(!is_trusted_media_origin("tauri://localhost.evil.com")); + assert!(!is_trusted_media_origin("tauri://localhostfoo")); + } + + #[cfg(debug_assertions)] + #[test] + fn allows_dev_origin_in_debug_only() { + assert!(is_trusted_media_origin("http://localhost:1420")); + assert!(is_trusted_media_origin("http://localhost:1420/")); + // A different localhost port is still untrusted. + assert!(!is_trusted_media_origin("http://localhost:14200")); + assert!(!is_trusted_media_origin("http://localhost:3000")); + } + + #[cfg(not(debug_assertions))] + #[test] + fn denies_dev_origin_in_release() { + assert!(!is_trusted_media_origin("http://localhost:1420")); + } +} diff --git a/desktop/src-tauri/src/managed_agents/agent_snapshot.rs b/desktop/src-tauri/src/managed_agents/agent_snapshot.rs index 16a0d35b23..7c08e7095f 100644 --- a/desktop/src-tauri/src/managed_agents/agent_snapshot.rs +++ b/desktop/src-tauri/src/managed_agents/agent_snapshot.rs @@ -306,9 +306,22 @@ pub fn encode_snapshot_png( ); } - // Manifest → JSON → base64 for the tEXt chunk payload. + // Manifest → JSON for the tEXt chunk payload. The payload/PNG composition + // is shared with the locked-card encoder in `agent_snapshot_envelope`; + // plain cards remain byte-identical to the pre-envelope encoder. let json_bytes = encode_snapshot_json(snapshot)?; - let chunk_text = STANDARD.encode(&json_bytes); + encode_chunk_payload_png(&json_bytes, avatar_bytes) +} + +/// Encode arbitrary chunk-payload JSON (plain manifest or locked envelope) +/// into a PNG carrying it base64-encoded in the `buzz_agent_snapshot` tEXt +/// chunk. Shared by the plain encoder above and +/// `agent_snapshot_envelope::encode_locked_snapshot_png`. +pub(crate) fn encode_chunk_payload_png( + json_bytes: &[u8], + avatar_bytes: Option<&[u8]>, +) -> Result, String> { + let chunk_text = STANDARD.encode(json_bytes); // Use the avatar as the PNG image body, transcoding decodable non-PNG // avatars. Fall back to a minimal 1×1 transparent placeholder only when @@ -334,8 +347,11 @@ pub fn encode_snapshot_png( Ok(png_bytes) } -/// Decode a manifest from a `.agent.png` tEXt chunk. -pub fn decode_snapshot_png(png_bytes: &[u8]) -> Result { +/// Extract and base64-decode the raw `buzz_agent_snapshot` chunk payload +/// (JSON bytes) from a PNG, without interpreting it. The payload may be a +/// plain manifest or a locked envelope — callers dispatch on the parsed +/// `format` via `agent_snapshot_envelope::parse_chunk_payload`. +pub(crate) fn extract_chunk_payload_png(png_bytes: &[u8]) -> Result, String> { let decoder = Decoder::new(Cursor::new(png_bytes)); let reader = decoder .read_info() @@ -349,10 +365,18 @@ pub fn decode_snapshot_png(png_bytes: &[u8]) -> Result { .map(|c| c.text.as_str()) .ok_or_else(|| "PNG does not contain a buzz_agent_snapshot tEXt chunk".to_string())?; - let json_bytes = STANDARD + STANDARD .decode(chunk_text.trim()) - .map_err(|e| format!("Invalid base64 in PNG chunk: {e}"))?; + .map_err(|e| format!("Invalid base64 in PNG chunk: {e}")) +} +/// Decode a manifest from a `.agent.png` tEXt chunk. +/// +/// Plain snapshots only — a locked (encrypted) chunk payload fails here with +/// the manifest format error. Import paths that must handle locked cards go +/// through `agent_snapshot_envelope::parse_chunk_payload` instead. +pub fn decode_snapshot_png(png_bytes: &[u8]) -> Result { + let json_bytes = extract_chunk_payload_png(png_bytes)?; decode_snapshot_json(&json_bytes) } @@ -473,527 +497,5 @@ fn inject_text_chunk(png_bytes: &[u8], keyword: &str, text: &str) -> Result ManagedAgentRecord { - ManagedAgentRecord { - pubkey: "deadbeef".to_string(), - name: "Test Agent".to_string(), - display_name: Some("Test Agent Display".to_string()), - persona_id: Some("SENTINEL_PERSONA_ID".to_string()), // MUST NOT appear in snapshot - team_id: Some("SENTINEL_TEAM_ID".to_string()), // MUST NOT appear in snapshot - private_key_nsec: "nsec1secret".to_string(), // MUST NOT appear in snapshot - auth_tag: Some("auth-tag-secret".to_string()), // MUST NOT appear in snapshot - relay_url: "wss://relay.example.com".to_string(), // MUST NOT appear in snapshot - avatar_url: Some("https://example.com/avatar.png".to_string()), - acp_command: "/usr/local/bin/acp".to_string(), // MUST NOT appear in snapshot - agent_command: "goose".to_string(), // MUST NOT appear in snapshot - agent_command_override: Some("goose-override".to_string()), // MUST NOT appear - agent_args: vec!["--arg".to_string()], // MUST NOT appear in snapshot - mcp_command: "mcp-server".to_string(), // MUST NOT appear in snapshot - turn_timeout_seconds: 120, // deprecated, MUST NOT appear - idle_timeout_seconds: Some(30), - max_turn_duration_seconds: Some(600), - parallelism: 2, - system_prompt: Some("You are a test agent.".to_string()), - model: Some("claude-opus-4".to_string()), - provider: Some("anthropic".to_string()), - persona_source_version: Some("v1.0".to_string()), // MUST NOT appear - env_vars: { - let mut m = BTreeMap::new(); - m.insert("API_KEY".to_string(), "secret123".to_string()); // MUST NOT appear - m - }, - start_on_app_launch: true, - auto_restart_on_config_change: true, - runtime_pid: Some(12345), // MUST NOT appear - backend: BackendKind::Provider { - // MUST NOT appear — carries a provider secret - id: "SENTINEL_BACKEND_ID".to_string(), - config: serde_json::json!({"api_key": "SENTINEL_BACKEND_SECRET"}), - }, - backend_agent_id: Some("SENTINEL_BACKEND_AGENT_ID".to_string()), // MUST NOT appear - provider_binary_path: Some("/usr/bin/SENTINEL_PROVIDER_BINARY".to_string()), // MUST NOT appear - persona_team_dir: Some(std::path::PathBuf::from("SENTINEL_TEAM_DIR")), // MUST NOT appear - persona_name_in_team: Some("SENTINEL_NAME_IN_TEAM".to_string()), // MUST NOT appear - created_at: "2024-01-01T00:00:00Z".to_string(), - updated_at: "2024-01-02T00:00:00Z".to_string(), - last_started_at: Some("2024-01-03T00:00:00Z".to_string()), // MUST NOT appear - last_stopped_at: None, - last_exit_code: Some(0), // MUST NOT appear - last_error: Some("SENTINEL_LAST_ERROR".to_string()), // MUST NOT appear - last_error_code: Some(42), // MUST NOT appear - respond_to: RespondTo::default(), - respond_to_allowlist: vec!["pubkey1hex".to_string()], - slug: Some("test-agent".to_string()), - runtime: Some("goose".to_string()), - name_pool: vec!["Alice".to_string(), "Bob".to_string()], - is_builtin: false, - is_active: true, - shared: false, - source_team: Some("team-id-123".to_string()), // MUST NOT appear - source_team_persona_slug: Some("lep".to_string()), // MUST NOT appear - definition_respond_to: Some("allowlist".to_string()), - catalog_source: None, - definition_respond_to_allowlist: vec!["abc123def".to_string()], - definition_parallelism: Some(4), - relay_mesh: None, - } - } - - // ── Round-trip tests ────────────────────────────────────────────────────── - - #[test] - fn json_round_trip_config_only() { - let record = minimal_record(); - let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); - let bytes = encode_snapshot_json(&snapshot).unwrap(); - let parsed = decode_snapshot_json(&bytes).unwrap(); - assert_eq!(parsed, snapshot); - } - - #[test] - fn json_round_trip_with_memory() { - let record = minimal_record(); - let entries = vec![ - AgentSnapshotMemoryEntry { - slug: "core".to_string(), - body: "I am a test agent.".to_string(), - }, - AgentSnapshotMemoryEntry { - slug: "mem/research".to_string(), - body: "Some research notes.".to_string(), - }, - ]; - let snapshot = build_snapshot(&record, MemoryLevel::Everything, entries, None); - let bytes = encode_snapshot_json(&snapshot).unwrap(); - let parsed = decode_snapshot_json(&bytes).unwrap(); - assert_eq!(parsed, snapshot); - } - - #[test] - fn png_round_trip_no_memory() { - let record = minimal_record(); - let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); - let png_bytes = encode_snapshot_png(&snapshot, None).unwrap(); - let parsed = decode_snapshot_png(&png_bytes).unwrap(); - assert_eq!(parsed.definition.name, snapshot.definition.name); - assert_eq!(parsed.profile.display_name, snapshot.profile.display_name); - assert_eq!(parsed.memory.level, MemoryLevel::None); - } - - #[test] - fn png_round_trip_with_avatar_png() { - // Build a minimal PNG avatar. - let avatar = make_png_with_text("dummy", "value").unwrap(); - let record = minimal_record(); - let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], Some(&avatar)); - // Avatar should be inlined as a data URL. - assert!(snapshot - .profile - .avatar_data_url - .as_deref() - .unwrap_or("") - .starts_with("data:image/png;base64,")); - - let png_bytes = encode_snapshot_png(&snapshot, Some(&avatar)).unwrap(); - let parsed = decode_snapshot_png(&png_bytes).unwrap(); - assert_eq!(parsed.definition.name, snapshot.definition.name); - } - - #[test] - fn png_snapshot_transcodes_jpeg_avatar_into_image_body() { - let avatar = image::DynamicImage::ImageRgb8(image::RgbImage::from_pixel( - 3, - 2, - image::Rgb([0x12, 0x34, 0x56]), - )); - let mut jpeg_bytes = Vec::new(); - avatar - .write_to(&mut Cursor::new(&mut jpeg_bytes), image::ImageFormat::Jpeg) - .unwrap(); - - let snapshot = build_snapshot( - &minimal_record(), - MemoryLevel::None, - vec![], - Some(&jpeg_bytes), - ); - let png_bytes = encode_snapshot_png(&snapshot, Some(&jpeg_bytes)).unwrap(); - let decoder = Decoder::new(Cursor::new(png_bytes)); - let reader = decoder.read_info().unwrap(); - - assert_eq!((reader.info().width, reader.info().height), (3, 2)); - } - - // ── PNG memory parity ───────────────────────────────────────────────────── - - #[test] - fn png_round_trip_with_core_memory() { - let record = minimal_record(); - let entries = vec![AgentSnapshotMemoryEntry { - slug: "core".to_string(), - body: "remember this".to_string(), - }]; - let snapshot = build_snapshot(&record, MemoryLevel::Core, entries, None); - - let png_bytes = encode_snapshot_png(&snapshot, None).unwrap(); - let parsed = decode_snapshot_png(&png_bytes).unwrap(); - - assert_eq!(parsed.memory, snapshot.memory); - } - - #[test] - fn png_round_trip_with_everything_memory() { - let record = minimal_record(); - let entries = vec![ - AgentSnapshotMemoryEntry { - slug: "core".to_string(), - body: "remember this".to_string(), - }, - AgentSnapshotMemoryEntry { - slug: "mem/notes".to_string(), - body: "private notes".to_string(), - }, - ]; - let snapshot = build_snapshot(&record, MemoryLevel::Everything, entries, None); - - let png_bytes = encode_snapshot_png(&snapshot, None).unwrap(); - let parsed = decode_snapshot_png(&png_bytes).unwrap(); - - assert_eq!(parsed.memory, snapshot.memory); - } - - #[test] - fn png_export_with_no_memory_succeeds() { - let record = minimal_record(); - let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); - assert!(encode_snapshot_png(&snapshot, None).is_ok()); - } - - #[test] - fn png_export_rejects_none_level_with_nonempty_entries() { - // Inconsistent state: level == None but entries is non-empty. - // The encoder must reject this to prevent a memory-leak bypass. - let record = minimal_record(); - let entries = vec![AgentSnapshotMemoryEntry { - slug: "core".to_string(), - body: "leaked memory".to_string(), - }]; - // Build with entries, then override level to None in the struct. - let mut snapshot = build_snapshot(&record, MemoryLevel::Core, entries, None); - snapshot.memory.level = MemoryLevel::None; // force inconsistency - let result = encode_snapshot_png(&snapshot, None); - assert!( - result.is_err(), - "PNG encoder must reject level=None with non-empty entries" - ); - assert!( - result - .unwrap_err() - .contains("memory.level 'none' and non-empty memory entries"), - "Error must explain the malformed memory state" - ); - } - - // ── Secret exclusion tests ──────────────────────────────────────────────── - // - // These tests assert that every field in the exclusion list is absent from - // the serialized snapshot. We serialize to JSON and assert the key is NOT - // present. - - fn snapshot_json_string(record: &ManagedAgentRecord) -> String { - let snapshot = build_snapshot(record, MemoryLevel::None, vec![], None); - let bytes = encode_snapshot_json(&snapshot).unwrap(); - String::from_utf8(bytes).unwrap() - } - - #[test] - fn secret_exclusion_private_key_nsec_absent() { - let record = minimal_record(); - let json = snapshot_json_string(&record); - assert!( - !json.contains("nsec1secret"), - "nsec must not appear in snapshot" - ); - assert!( - !json.contains("privateKeyNsec") && !json.contains("private_key_nsec"), - "privateKeyNsec field must not appear in snapshot" - ); - } - - #[test] - fn secret_exclusion_auth_tag_absent() { - let record = minimal_record(); - let json = snapshot_json_string(&record); - assert!( - !json.contains("auth-tag-secret"), - "auth_tag value must not appear in snapshot" - ); - assert!( - !json.contains("authTag") && !json.contains("auth_tag"), - "authTag field must not appear in snapshot" - ); - } - - #[test] - fn secret_exclusion_env_vars_absent() { - let record = minimal_record(); - let json = snapshot_json_string(&record); - assert!( - !json.contains("API_KEY") && !json.contains("secret123"), - "env_vars content must not appear in snapshot" - ); - assert!( - !json.contains("envVars") && !json.contains("env_vars"), - "envVars field must not appear in snapshot" - ); - } - - #[test] - fn secret_exclusion_relay_url_absent() { - let record = minimal_record(); - let json = snapshot_json_string(&record); - assert!( - !json.contains("wss://relay.example.com"), - "relay_url value must not appear in snapshot" - ); - assert!( - !json.contains("relayUrl") && !json.contains("relay_url"), - "relayUrl field must not appear in snapshot" - ); - } - - #[test] - fn snapshot_omits_removed_mcp_toolsets_config() { - let record = minimal_record(); - let json = snapshot_json_string(&record); - assert!( - !json.contains("mcpToolsets") && !json.contains("mcp_toolsets"), - "removed MCP toolsets config must not re-enter snapshots" - ); - } - - #[test] - fn secret_exclusion_machine_commands_absent() { - let record = minimal_record(); - let json = snapshot_json_string(&record); - // acp_command / agent_command / agent_command_override / agent_args / mcp_command - assert!( - !json.contains("/usr/local/bin/acp"), - "acp_command path must not appear" - ); - assert!( - !json.contains("acpCommand") && !json.contains("acp_command"), - "acpCommand field must not appear" - ); - assert!( - !json.contains("agentCommand") && !json.contains("agent_command"), - "agentCommand field must not appear" - ); - assert!( - !json.contains("mcpCommand") && !json.contains("mcp_command"), - "mcpCommand field must not appear" - ); - } - - #[test] - fn secret_exclusion_runtime_state_absent() { - let record = minimal_record(); - let json = snapshot_json_string(&record); - assert!( - !json.contains("runtimePid") && !json.contains("runtime_pid"), - "runtimePid must not appear" - ); - assert!( - !json.contains("backendAgentId") && !json.contains("backend_agent_id"), - "backendAgentId must not appear" - ); - assert!( - !json.contains("SENTINEL_BACKEND_AGENT_ID"), - "backendAgentId value must not appear" - ); - assert!( - !json.contains("providerBinaryPath") && !json.contains("provider_binary_path"), - "providerBinaryPath must not appear" - ); - assert!( - !json.contains("SENTINEL_PROVIDER_BINARY"), - "providerBinaryPath value must not appear" - ); - assert!( - !json.contains("lastStartedAt") && !json.contains("last_started_at"), - "lastStartedAt must not appear" - ); - assert!( - !json.contains("lastExitCode") && !json.contains("last_exit_code"), - "lastExitCode must not appear" - ); - // backend blob — neither the type tag nor provider secret must leak. - assert!( - !json.contains("\"backend\"") && !json.contains("backend"), - "backend field must not appear" - ); - assert!( - !json.contains("SENTINEL_BACKEND_ID") && !json.contains("SENTINEL_BACKEND_SECRET"), - "backend config values must not appear" - ); - // last_error / last_error_code - assert!( - !json.contains("lastError") && !json.contains("last_error"), - "lastError must not appear" - ); - assert!( - !json.contains("SENTINEL_LAST_ERROR"), - "lastError value must not appear" - ); - assert!( - !json.contains("lastErrorCode") && !json.contains("last_error_code"), - "lastErrorCode must not appear" - ); - } - - #[test] - fn secret_exclusion_lineage_ids_absent() { - let record = minimal_record(); - let json = snapshot_json_string(&record); - assert!( - !json.contains("team-id-123"), - "source_team value must not appear" - ); - assert!( - !json.contains("sourceTeam") && !json.contains("source_team"), - "sourceTeam field must not appear" - ); - assert!( - !json.contains("sourceTeamPersonaSlug"), - "sourceTeamPersonaSlug must not appear" - ); - assert!( - !json.contains("personaSourceVersion") && !json.contains("persona_source_version"), - "personaSourceVersion must not appear" - ); - // personaId - assert!( - !json.contains("personaId") && !json.contains("persona_id"), - "personaId field must not appear" - ); - assert!( - !json.contains("SENTINEL_PERSONA_ID"), - "personaId value must not appear" - ); - // teamId - assert!( - !json.contains("teamId") && !json.contains("team_id"), - "teamId field must not appear" - ); - assert!( - !json.contains("SENTINEL_TEAM_ID"), - "teamId value must not appear" - ); - // personaTeamDir - assert!( - !json.contains("personaTeamDir") && !json.contains("persona_team_dir"), - "personaTeamDir field must not appear" - ); - assert!( - !json.contains("SENTINEL_TEAM_DIR"), - "personaTeamDir value must not appear" - ); - // personaNameInTeam - assert!( - !json.contains("personaNameInTeam") && !json.contains("persona_name_in_team"), - "personaNameInTeam field must not appear" - ); - assert!( - !json.contains("SENTINEL_NAME_IN_TEAM"), - "personaNameInTeam value must not appear" - ); - } - - // ── Definition field presence tests ────────────────────────────────────── - - #[test] - fn definition_fields_present_in_snapshot() { - let record = minimal_record(); - let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); - - assert_eq!(snapshot.definition.name, "Test Agent Display"); - assert!(!snapshot.definition.source_is_builtin); - assert_eq!( - snapshot.definition.system_prompt.as_deref(), - Some("You are a test agent.") - ); - assert_eq!(snapshot.definition.runtime.as_deref(), Some("goose")); - assert_eq!(snapshot.definition.model.as_deref(), Some("claude-opus-4")); - assert_eq!(snapshot.definition.provider.as_deref(), Some("anthropic")); - assert_eq!(snapshot.definition.name_pool, vec!["Alice", "Bob"]); - // definition_respond_to maps to respond_to in the snapshot definition - assert_eq!(snapshot.definition.respond_to.as_deref(), Some("allowlist")); - // definition_respond_to_allowlist should be included - assert!(!snapshot.definition.respond_to_allowlist.is_empty()); - } - - #[test] - fn profile_fields_present_in_snapshot() { - let record = minimal_record(); - let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); - assert_eq!(snapshot.profile.display_name, "Test Agent Display"); - // No bytes → should fall back to avatar_url - assert_eq!( - snapshot.profile.avatar_url.as_deref(), - Some("https://example.com/avatar.png") - ); - assert!(snapshot.profile.avatar_data_url.is_none()); - } - - #[test] - fn avatar_inlined_when_under_size_limit() { - let record = minimal_record(); - let small_png = make_png_with_text("k", "v").unwrap(); - let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], Some(&small_png)); - assert!(snapshot.profile.avatar_data_url.is_some()); - assert!(snapshot.profile.avatar_url.is_none()); - } - - #[test] - fn avatar_url_fallback_when_over_size_limit() { - let mut record = minimal_record(); - record.avatar_url = Some("https://example.com/big.png".to_string()); - // Synthesize oversized avatar bytes (> 2 MB) — just a large zeroed vec. - let big_bytes = vec![0u8; MAX_AVATAR_INLINE_BYTES + 1]; - let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], Some(&big_bytes)); - assert!(snapshot.profile.avatar_data_url.is_none()); - assert_eq!( - snapshot.profile.avatar_url.as_deref(), - Some("https://example.com/big.png") - ); - } - - // ── Format/version validation ───────────────────────────────────────────── - - #[test] - fn invalid_format_discriminator_is_rejected() { - let mut snapshot = build_snapshot(&minimal_record(), MemoryLevel::None, vec![], None); - snapshot.format = "not-a-buzz-snapshot".to_string(); - let bytes = serde_json::to_vec(&snapshot).unwrap(); - let result = decode_snapshot_json(&bytes); - assert!(result.is_err()); - assert!(result.unwrap_err().contains("Unsupported snapshot format")); - } - - #[test] - fn unsupported_version_is_rejected() { - let mut snapshot = build_snapshot(&minimal_record(), MemoryLevel::None, vec![], None); - snapshot.version = 99; - let bytes = serde_json::to_vec(&snapshot).unwrap(); - let result = decode_snapshot_json(&bytes); - assert!(result.is_err()); - assert!(result.unwrap_err().contains("Unsupported snapshot version")); - } -} +#[path = "agent_snapshot_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/managed_agents/agent_snapshot_envelope.rs b/desktop/src-tauri/src/managed_agents/agent_snapshot_envelope.rs new file mode 100644 index 0000000000..8508c27073 --- /dev/null +++ b/desktop/src-tauri/src/managed_agents/agent_snapshot_envelope.rs @@ -0,0 +1,638 @@ +//! Locked (encrypted) agent-card envelope — NIP-44 v2 over the snapshot manifest. +//! +//! A locked card carries the same `buzz_agent_snapshot` tEXt chunk as a plain +//! card, but the chunk JSON is a typed outer envelope whose ciphertext +//! decrypts to the ordinary manifest. The NIP-44 v2 conversation key is +//! symmetric over the (owner, agent) pair, so BOTH the owner's and the +//! agent's nsec decrypt the card — nobody else's does (NIP-AE's scheme). +//! +//! Wire contract (agreed with Wren, buzz-agent-trading-cards thread): +//! - Plain cards keep today's exact bytes; detection dispatches once on the +//! exact `format` discriminator and rejects unknown versions/schemes +//! rather than falling through to manifest parsing. +//! - Key lookup is exact-endpoint only: the owner identity key when its +//! pubkey equals `ownerPubkey`, or a hydrated local managed-agent record +//! whose record pubkey AND derived-secret pubkey equal `agentPubkey`. +//! No trial decryption; anything else fails closed as locked. +//! - Caps beyond the outer 10 MiB PNG gate: 65,535-byte NIP-44 plaintext +//! limit on the serialized manifest BEFORE encryption; envelope JSON and +//! ciphertext are capped before serde/base64/decrypt work; decrypted bytes +//! are capped before snapshot parsing. +//! - Decrypt/auth failures return only the locked-card refusal — never +//! partial plaintext or crypto details. + +use buzz_core_pkg::engram::NIP44_PLAINTEXT_MAX; +use nostr::nips::nip44::{self, Version}; +use nostr::{Keys, PublicKey, SecretKey}; +use serde::{Deserialize, Serialize}; + +use super::agent_snapshot::{ + decode_snapshot_json, encode_chunk_payload_png, encode_snapshot_json, AgentSnapshot, + MemoryLevel, FORMAT_DISCRIMINATOR, +}; +use super::types::ManagedAgentRecord; + +/// Discriminator for the locked envelope. Distinct from the plain manifest's +/// `buzz-agent-snapshot` so detection never guesses. +pub const LOCKED_FORMAT: &str = "buzz-agent-snapshot-encrypted"; +/// Envelope schema version this module produces and accepts. +pub const LOCKED_VERSION: u32 = 1; +/// Encryption scheme identifier this module produces and accepts. +pub const LOCKED_SCHEME: &str = "nip44-v2"; + +/// A max-size NIP-44 v2 payload (1 version + 32 nonce + 2 len + 65,536 +/// padded + 32 MAC = 65,603 bytes) base64-encodes to 87,472 chars. +/// Anything larger is rejected before base64/decrypt work. +pub const MAX_LOCKED_CIPHERTEXT_BYTES: usize = 90_000; +/// Envelope JSON = ciphertext + two pubkeys + fixed keys. Rejected before +/// typed deserialization. +pub const MAX_LOCKED_ENVELOPE_JSON_BYTES: usize = MAX_LOCKED_CIPHERTEXT_BYTES + 1024; + +/// The only error a failed unlock may surface. Deliberately says nothing +/// about which key was tried or why decryption failed. +pub const LOCKED_CARD_REFUSAL: &str = + "This card is locked to its owner and agent. Only they can import it."; + +// ── Envelope types ──────────────────────────────────────────────────────────── + +/// Typed outer envelope stored (base64 JSON) in the `buzz_agent_snapshot` +/// chunk of a locked card. +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "camelCase")] +pub struct LockedSnapshotEnvelope { + /// Always [`LOCKED_FORMAT`]. + pub format: String, + /// Always [`LOCKED_VERSION`]. + pub version: u32, + pub encryption: LockedEncryption, +} + +#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)] +#[serde(rename_all = "camelCase")] +pub struct LockedEncryption { + /// Always [`LOCKED_SCHEME`]. + pub scheme: String, + /// Owner identity pubkey (64 lowercase hex). Plaintext so a decryptor + /// knows which counterparty to pair with. + pub owner_pubkey: String, + /// Agent instance pubkey (64 lowercase hex). + pub agent_pubkey: String, + /// NIP-44 v2 ciphertext (base64) of the plain manifest JSON. + pub ciphertext: String, +} + +/// Result of parsing a chunk payload: either today's plain manifest or a +/// validated locked envelope. The plain manifest is boxed because it may +/// inline a multi-KB avatar data URL, dwarfing the envelope variant. +#[derive(Debug)] +pub enum ChunkPayload { + Plain(Box), + Locked(LockedSnapshotEnvelope), +} + +/// Minimal probe used to read the `format` discriminator without building a +/// full JSON tree for large plain manifests. +#[derive(Deserialize)] +struct FormatProbe { + #[serde(default)] + format: Option, +} + +// ── Validation ──────────────────────────────────────────────────────────────── + +/// Canonical pubkey check: exactly 64 lowercase hex chars that parse as a +/// valid x-only pubkey. Lowercase is required so string comparisons against +/// record pubkeys (always `to_hex()` output) stay sound. Curve validation is +/// explicit: nostr's `PublicKey::from_hex` only decodes 32 bytes and defers +/// lift-x validation to `xonly()`, so a non-point like `"f" * 64` would +/// otherwise pass structurally and fail only at decrypt time. +pub(crate) fn parse_canonical_pubkey(field: &str, value: &str) -> Result { + if value.len() != 64 + || !value + .chars() + .all(|c| c.is_ascii_digit() || ('a'..='f').contains(&c)) + { + return Err(format!( + "Locked card envelope has a malformed {field} (expected 64 lowercase hex chars)." + )); + } + let pubkey = PublicKey::from_hex(value) + .map_err(|_| format!("Locked card envelope has an invalid {field}."))?; + pubkey + .xonly() + .map_err(|_| format!("Locked card envelope has an invalid {field} (not a curve point)."))?; + Ok(pubkey) +} + +/// Structural validation of a locked envelope: exact version + scheme, +/// canonical pubkeys, distinct endpoints, bounded ciphertext. Does no +/// key lookup or crypto. +pub fn validate_envelope( + envelope: &LockedSnapshotEnvelope, +) -> Result<(PublicKey, PublicKey), String> { + if envelope.format != LOCKED_FORMAT { + return Err(format!( + "Unsupported locked card format: {:?} (expected {LOCKED_FORMAT:?})", + envelope.format + )); + } + if envelope.version != LOCKED_VERSION { + return Err(format!( + "Unsupported locked card envelope version: {} (expected {LOCKED_VERSION})", + envelope.version + )); + } + if envelope.encryption.scheme != LOCKED_SCHEME { + return Err(format!( + "Unsupported locked card encryption scheme: {:?} (expected {LOCKED_SCHEME:?})", + envelope.encryption.scheme + )); + } + let owner = parse_canonical_pubkey("ownerPubkey", &envelope.encryption.owner_pubkey)?; + let agent = parse_canonical_pubkey("agentPubkey", &envelope.encryption.agent_pubkey)?; + if owner == agent { + return Err("Locked card envelope owner and agent pubkeys must differ.".to_string()); + } + if envelope.encryption.ciphertext.len() > MAX_LOCKED_CIPHERTEXT_BYTES { + return Err("Locked card ciphertext exceeds the maximum size.".to_string()); + } + if envelope.encryption.ciphertext.is_empty() { + return Err("Locked card ciphertext is empty.".to_string()); + } + Ok((owner, agent)) +} + +// ── Dispatch ────────────────────────────────────────────────────────────────── + +/// Parse a raw chunk payload (JSON bytes from `extract_chunk_payload_png` or +/// an `.agent.json` file) and dispatch on the exact `format` discriminator. +/// +/// - `buzz-agent-snapshot` → full plain-manifest decode + validation. +/// - `buzz-agent-snapshot-encrypted` → size caps, typed envelope parse, +/// structural validation. No decryption happens here. +/// - anything else (including missing `format`) → error, never a fall-through. +pub fn parse_chunk_payload(json_bytes: &[u8]) -> Result { + let probe: FormatProbe = + serde_json::from_slice(json_bytes).map_err(|e| format!("Invalid snapshot JSON: {e}"))?; + match probe.format.as_deref() { + Some(f) if f == FORMAT_DISCRIMINATOR => Ok(ChunkPayload::Plain(Box::new( + decode_snapshot_json(json_bytes)?, + ))), + Some(f) if f == LOCKED_FORMAT => { + // Cap the envelope JSON before typed deserialization; a locked + // envelope is small by construction (unlike plain manifests, + // which may inline a multi-MB avatar). + if json_bytes.len() > MAX_LOCKED_ENVELOPE_JSON_BYTES { + return Err("Locked card envelope exceeds the maximum size.".to_string()); + } + let envelope: LockedSnapshotEnvelope = serde_json::from_slice(json_bytes) + .map_err(|e| format!("Invalid locked card envelope: {e}"))?; + validate_envelope(&envelope)?; + Ok(ChunkPayload::Locked(envelope)) + } + Some(other) => Err(format!("Unsupported snapshot format: {other:?}")), + None => Err("Snapshot payload has no format discriminator.".to_string()), + } +} + +// ── Encrypt ─────────────────────────────────────────────────────────────────── + +/// Encrypt a snapshot manifest into a locked envelope under the NIP-44 v2 +/// conversation key for (owner secret, agent pubkey). +/// +/// Fails clearly (never silently truncates) when the serialized manifest +/// exceeds the NIP-44 plaintext limit. +pub fn encrypt_snapshot_envelope( + snapshot: &AgentSnapshot, + owner_keys: &Keys, + agent_pubkey: &PublicKey, +) -> Result { + let json_bytes = encode_snapshot_json(snapshot)?; + if json_bytes.len() > NIP44_PLAINTEXT_MAX { + return Err(format!( + "Agent manifest is too large to lock ({} bytes; the encrypted \ + format caps at {NIP44_PLAINTEXT_MAX}). Reduce the avatar size \ + or mint an unlocked card.", + json_bytes.len() + )); + } + let plaintext = std::str::from_utf8(&json_bytes) + .map_err(|e| format!("Manifest JSON was not UTF-8: {e}"))?; + let ciphertext = nip44::encrypt( + owner_keys.secret_key(), + agent_pubkey, + plaintext, + Version::V2, + ) + .map_err(|e| format!("Failed to encrypt card manifest: {e}"))?; + + Ok(LockedSnapshotEnvelope { + format: LOCKED_FORMAT.to_string(), + version: LOCKED_VERSION, + encryption: LockedEncryption { + scheme: LOCKED_SCHEME.to_string(), + owner_pubkey: owner_keys.public_key().to_hex(), + agent_pubkey: agent_pubkey.to_hex(), + ciphertext, + }, + }) +} + +/// Encode a snapshot into a LOCKED `.agent.png`: encrypt the manifest into +/// the envelope, then compose the PNG through the same chunk encoder plain +/// cards use. Mirrors `encode_snapshot_png`'s structural memory guard. +pub fn encode_locked_snapshot_png( + snapshot: &AgentSnapshot, + owner_keys: &Keys, + agent_pubkey: &PublicKey, + avatar_bytes: Option<&[u8]>, +) -> Result, String> { + if snapshot.memory.level == MemoryLevel::None && !snapshot.memory.entries.is_empty() { + return Err( + "Cannot write a snapshot with memory.level 'none' and non-empty memory entries." + .to_string(), + ); + } + let envelope = encrypt_snapshot_envelope(snapshot, owner_keys, agent_pubkey)?; + let envelope_json = serde_json::to_vec(&envelope) + .map_err(|e| format!("Failed to serialize locked card envelope: {e}"))?; + encode_chunk_payload_png(&envelope_json, avatar_bytes) +} + +// ── Decrypt ─────────────────────────────────────────────────────────────────── + +/// Exact-endpoint key resolution (no trial decryption): +/// - the owner identity secret, only when its pubkey equals `ownerPubkey`; +/// - a hydrated local managed-agent record whose record pubkey AND +/// derived-secret pubkey both equal `agentPubkey`. +/// +/// Returns `None` when neither exact endpoint exists — callers fail closed +/// with [`LOCKED_CARD_REFUSAL`]. +pub fn resolve_unlock_secret( + envelope: &LockedSnapshotEnvelope, + owner_keys: Option<&Keys>, + records: &[ManagedAgentRecord], +) -> Option { + if let Some(keys) = owner_keys { + if keys.public_key().to_hex() == envelope.encryption.owner_pubkey { + return Some(keys.secret_key().clone()); + } + } + let record = records + .iter() + .find(|r| r.pubkey == envelope.encryption.agent_pubkey)?; + let agent_keys = Keys::parse(record.private_key_nsec.trim()).ok()?; + if agent_keys.public_key().to_hex() != envelope.encryption.agent_pubkey { + return None; + } + Some(agent_keys.secret_key().clone()) +} + +/// Decrypt a validated envelope with `my_secret`, which must be one of the +/// envelope's two exact endpoints (its derived pubkey selects the +/// counterparty). Returns the decoded, validated snapshot manifest. +/// +/// Every auth/crypto failure maps to [`LOCKED_CARD_REFUSAL`] — nothing about +/// the failure mode leaks. Manifest decode errors after a successful decrypt +/// are surfaced normally (the caller proved key possession). +pub fn decrypt_envelope( + envelope: &LockedSnapshotEnvelope, + my_secret: &SecretKey, +) -> Result { + let (owner_pub, agent_pub) = validate_envelope(envelope)?; + let my_pub = Keys::new(my_secret.clone()).public_key(); + let counterparty = if my_pub == owner_pub { + agent_pub + } else if my_pub == agent_pub { + owner_pub + } else { + return Err(LOCKED_CARD_REFUSAL.to_string()); + }; + + let plaintext = nip44::decrypt(my_secret, &counterparty, &envelope.encryption.ciphertext) + .map_err(|_| LOCKED_CARD_REFUSAL.to_string())?; + if plaintext.len() > NIP44_PLAINTEXT_MAX { + return Err(LOCKED_CARD_REFUSAL.to_string()); + } + decode_snapshot_json(plaintext.as_bytes()) +} + +// ── Tests ───────────────────────────────────────────────────────────────────── + +#[cfg(test)] +mod tests { + use super::*; + use crate::managed_agents::agent_snapshot::{ + extract_chunk_payload_png, AgentSnapshotDefinition, AgentSnapshotMemory, + AgentSnapshotProfile, FORMAT_VERSION, + }; + + fn sample_snapshot() -> AgentSnapshot { + AgentSnapshot { + format: FORMAT_DISCRIMINATOR.to_string(), + version: FORMAT_VERSION, + definition: AgentSnapshotDefinition { + name: "Locked Test".to_string(), + system_prompt: Some("You are a locked test agent.".to_string()), + runtime: None, + model: None, + provider: None, + parallelism: Some(1), + respond_to: None, + respond_to_allowlist: Vec::new(), + name_pool: Vec::new(), + idle_timeout_seconds: None, + max_turn_duration_seconds: None, + source_is_builtin: false, + }, + profile: AgentSnapshotProfile { + display_name: "Locked Test".to_string(), + about: None, + avatar_data_url: None, + avatar_url: None, + }, + memory: AgentSnapshotMemory { + level: MemoryLevel::None, + entries: Vec::new(), + }, + } + } + + fn owner_agent_keys() -> (Keys, Keys) { + (Keys::generate(), Keys::generate()) + } + + /// Minimal hydrated record for endpoint-resolution tests. Only the + /// pubkey/nsec pair matters here. + fn record_with_keys(pubkey: String, private_key_nsec: String) -> ManagedAgentRecord { + ManagedAgentRecord { + pubkey, + name: "Locked Test".to_string(), + persona_id: None, + private_key_nsec, + auth_tag: None, + relay_url: "ws://localhost:3000".to_string(), + avatar_url: None, + acp_command: "buzz-acp".to_string(), + agent_command: "goose".to_string(), + agent_args: vec![], + mcp_command: String::new(), + turn_timeout_seconds: 300, + idle_timeout_seconds: None, + max_turn_duration_seconds: None, + parallelism: 1, + system_prompt: None, + model: None, + env_vars: std::collections::BTreeMap::new(), + start_on_app_launch: false, + auto_restart_on_config_change: true, + runtime_pid: None, + backend: crate::managed_agents::types::BackendKind::Local, + backend_agent_id: None, + provider_binary_path: None, + team_id: None, + persona_team_dir: None, + persona_name_in_team: None, + created_at: String::new(), + updated_at: String::new(), + last_started_at: None, + last_stopped_at: None, + last_exit_code: None, + last_error: None, + last_error_code: None, + respond_to: crate::managed_agents::types::RespondTo::OwnerOnly, + respond_to_allowlist: vec![], + display_name: None, + slug: None, + runtime: None, + name_pool: Vec::new(), + is_builtin: false, + is_active: true, + shared: false, + source_team: None, + source_team_persona_slug: None, + catalog_source: None, + definition_respond_to: None, + definition_respond_to_allowlist: Vec::new(), + definition_parallelism: None, + relay_mesh: None, + agent_command_override: None, + persona_source_version: None, + provider: None, + } + } + + fn locked_envelope() -> (LockedSnapshotEnvelope, Keys, Keys) { + let (owner, agent) = owner_agent_keys(); + let env = + encrypt_snapshot_envelope(&sample_snapshot(), &owner, &agent.public_key()).unwrap(); + (env, owner, agent) + } + + #[test] + fn owner_secret_decrypts() { + let (env, owner, _agent) = locked_envelope(); + let decoded = decrypt_envelope(&env, owner.secret_key()).unwrap(); + assert_eq!(decoded, sample_snapshot()); + } + + #[test] + fn agent_secret_decrypts() { + let (env, _owner, agent) = locked_envelope(); + let decoded = decrypt_envelope(&env, agent.secret_key()).unwrap(); + assert_eq!(decoded, sample_snapshot()); + } + + #[test] + fn unrelated_key_fails_closed_with_refusal_only() { + let (env, _owner, _agent) = locked_envelope(); + let stranger = Keys::generate(); + let err = decrypt_envelope(&env, stranger.secret_key()).unwrap_err(); + assert_eq!(err, LOCKED_CARD_REFUSAL); + } + + #[test] + fn tampered_ciphertext_fails_with_refusal_only() { + let (mut env, owner, _agent) = locked_envelope(); + // Flip a character mid-ciphertext (keep valid base64 alphabet). + let mid = env.encryption.ciphertext.len() / 2; + let mut bytes = env.encryption.ciphertext.into_bytes(); + bytes[mid] = if bytes[mid] == b'A' { b'B' } else { b'A' }; + env.encryption.ciphertext = String::from_utf8(bytes).unwrap(); + let err = decrypt_envelope(&env, owner.secret_key()).unwrap_err(); + assert_eq!(err, LOCKED_CARD_REFUSAL); + } + + #[test] + fn swapped_pubkeys_fail_closed_at_endpoint_resolution() { + let (mut env, owner, agent) = locked_envelope(); + std::mem::swap( + &mut env.encryption.owner_pubkey, + &mut env.encryption.agent_pubkey, + ); + // The NIP-44 conversation key is symmetric over the pair, so a swap + // cannot grant a stranger anything — but it desyncs the routing + // hints, and exact-endpoint resolution fails closed rather than + // guessing: the owner identity no longer matches `ownerPubkey`, and + // no local record holds the pubkey now in `agentPubkey`. + assert!(resolve_unlock_secret(&env, Some(&owner), &[]).is_none()); + let nsec = nostr::ToBech32::to_bech32(agent.secret_key()).unwrap(); + let record = record_with_keys(agent.public_key().to_hex(), nsec); + assert!(resolve_unlock_secret(&env, None, std::slice::from_ref(&record)).is_none()); + } + + #[test] + fn mislabeled_pubkey_fails_decryption_with_refusal_only() { + // Replacing `agentPubkey` with a third party's key makes the owner + // derive the wrong conversation key — the NIP-44 MAC fails and only + // the refusal surfaces. + let (mut env, owner, _agent) = locked_envelope(); + env.encryption.agent_pubkey = Keys::generate().public_key().to_hex(); + let err = decrypt_envelope(&env, owner.secret_key()).unwrap_err(); + assert_eq!(err, LOCKED_CARD_REFUSAL); + } + + #[test] + fn malformed_pubkeys_rejected_structurally() { + let (env, _owner, _agent) = locked_envelope(); + + let mut short = env.clone(); + short.encryption.owner_pubkey = "abc123".to_string(); + assert!(validate_envelope(&short).unwrap_err().contains("malformed")); + + let mut upper = env.clone(); + upper.encryption.agent_pubkey = upper.encryption.agent_pubkey.to_uppercase(); + assert!(validate_envelope(&upper).unwrap_err().contains("malformed")); + + // A 64-hex string that is not a curve point (lift-x fails for + // x = p-1... all-f) must be rejected STRUCTURALLY — before any key + // lookup or decrypt work — per the wire contract. + let mut not_a_point = env.clone(); + not_a_point.encryption.agent_pubkey = "f".repeat(64); + assert!(validate_envelope(¬_a_point) + .unwrap_err() + .contains("not a curve point")); + + let mut same = env; + same.encryption.agent_pubkey = same.encryption.owner_pubkey.clone(); + assert!(validate_envelope(&same).unwrap_err().contains("differ")); + } + + #[test] + fn unknown_format_version_scheme_rejected() { + let (env, ..) = locked_envelope(); + + let mut bad_version = env.clone(); + bad_version.version = 2; + assert!(validate_envelope(&bad_version) + .unwrap_err() + .contains("version")); + + let mut bad_scheme = env.clone(); + bad_scheme.encryption.scheme = "nip44-v3".to_string(); + assert!(validate_envelope(&bad_scheme) + .unwrap_err() + .contains("scheme")); + + // Unknown top-level format never falls through to manifest parsing. + let unknown = serde_json::json!({"format": "buzz-agent-snapshot-v9", "version": 1}); + let err = parse_chunk_payload(unknown.to_string().as_bytes()).unwrap_err(); + assert!(err.contains("Unsupported snapshot format"), "{err}"); + + let missing = serde_json::json!({"version": 1}); + let err = parse_chunk_payload(missing.to_string().as_bytes()).unwrap_err(); + assert!(err.contains("no format discriminator"), "{err}"); + } + + #[test] + fn plaintext_cap_enforced_before_encryption() { + let (owner, agent) = owner_agent_keys(); + let mut snapshot = sample_snapshot(); + // Inflate the manifest beyond the NIP-44 plaintext limit. + snapshot.definition.system_prompt = Some("x".repeat(NIP44_PLAINTEXT_MAX)); + let err = encrypt_snapshot_envelope(&snapshot, &owner, &agent.public_key()).unwrap_err(); + assert!(err.contains("too large to lock"), "{err}"); + } + + #[test] + fn ciphertext_and_envelope_caps_enforced_before_crypto() { + let (mut env, ..) = locked_envelope(); + env.encryption.ciphertext = "A".repeat(MAX_LOCKED_CIPHERTEXT_BYTES + 1); + assert!(validate_envelope(&env) + .unwrap_err() + .contains("maximum size")); + + // Oversized envelope JSON is rejected before typed deserialization. + let huge = format!( + r#"{{"format":"{LOCKED_FORMAT}","version":1,"pad":"{}","encryption":{{}}}}"#, + "p".repeat(MAX_LOCKED_ENVELOPE_JSON_BYTES) + ); + let err = parse_chunk_payload(huge.as_bytes()).unwrap_err(); + assert!(err.contains("maximum size"), "{err}"); + } + + #[test] + fn locked_png_round_trips_through_chunk_and_decrypt() { + let (owner, agent) = owner_agent_keys(); + let snapshot = sample_snapshot(); + let png = encode_locked_snapshot_png(&snapshot, &owner, &agent.public_key(), None).unwrap(); + + let payload = extract_chunk_payload_png(&png).unwrap(); + let ChunkPayload::Locked(env) = parse_chunk_payload(&payload).unwrap() else { + panic!("locked PNG must parse as a locked envelope"); + }; + // Both endpoints decrypt to the same logical manifest (compare + // manifests, never ciphertext — the NIP-44 nonce is random). + assert_eq!( + decrypt_envelope(&env, owner.secret_key()).unwrap(), + snapshot + ); + assert_eq!( + decrypt_envelope(&env, agent.secret_key()).unwrap(), + snapshot + ); + } + + #[test] + fn plain_manifest_dispatches_to_plain() { + let json = encode_snapshot_json(&sample_snapshot()).unwrap(); + let ChunkPayload::Plain(decoded) = parse_chunk_payload(&json).unwrap() else { + panic!("plain manifest must parse as Plain"); + }; + assert_eq!(*decoded, sample_snapshot()); + } + + #[test] + fn resolve_unlock_secret_owner_exact_endpoint() { + let (env, owner, _agent) = locked_envelope(); + let secret = resolve_unlock_secret(&env, Some(&owner), &[]).unwrap(); + assert_eq!(&secret, owner.secret_key()); + + // A different identity key is NOT tried. + let other = Keys::generate(); + assert!(resolve_unlock_secret(&env, Some(&other), &[]).is_none()); + assert!(resolve_unlock_secret(&env, None, &[]).is_none()); + } + + #[test] + fn resolve_unlock_secret_agent_requires_record_and_derived_match() { + let (env, _owner, agent) = locked_envelope(); + let nsec = nostr::ToBech32::to_bech32(agent.secret_key()).unwrap(); + + let record = record_with_keys(agent.public_key().to_hex(), nsec); + let secret = resolve_unlock_secret(&env, None, std::slice::from_ref(&record)).unwrap(); + assert_eq!(&secret, agent.secret_key()); + + // Record pubkey matches but the stored secret derives a DIFFERENT + // pubkey → refused (no trial decryption on mismatched material). + let mut forged = record.clone(); + forged.private_key_nsec = + nostr::ToBech32::to_bech32(Keys::generate().secret_key()).unwrap(); + assert!(resolve_unlock_secret(&env, None, std::slice::from_ref(&forged)).is_none()); + + // Record for some other agent → not an endpoint. + let mut unrelated = record; + unrelated.pubkey = Keys::generate().public_key().to_hex(); + assert!(resolve_unlock_secret(&env, None, std::slice::from_ref(&unrelated)).is_none()); + } +} diff --git a/desktop/src-tauri/src/managed_agents/agent_snapshot_tests.rs b/desktop/src-tauri/src/managed_agents/agent_snapshot_tests.rs new file mode 100644 index 0000000000..b4492418e5 --- /dev/null +++ b/desktop/src-tauri/src/managed_agents/agent_snapshot_tests.rs @@ -0,0 +1,599 @@ +//! Unit tests for `managed_agents/agent_snapshot.rs`. +//! +//! Kept in a sibling file so `agent_snapshot.rs` stays under the +//! 1000-line gate; `#[path]`-included from there. + +use super::*; +use crate::managed_agents::types::{BackendKind, ManagedAgentRecord, RespondTo}; +use std::collections::BTreeMap; + +/// Build a minimal `ManagedAgentRecord` for testing. Only the fields +/// relevant to snapshot export are filled; the rest use defaults. +fn minimal_record() -> ManagedAgentRecord { + ManagedAgentRecord { + pubkey: "deadbeef".to_string(), + name: "Test Agent".to_string(), + display_name: Some("Test Agent Display".to_string()), + persona_id: Some("SENTINEL_PERSONA_ID".to_string()), // MUST NOT appear in snapshot + team_id: Some("SENTINEL_TEAM_ID".to_string()), // MUST NOT appear in snapshot + private_key_nsec: "nsec1secret".to_string(), // MUST NOT appear in snapshot + auth_tag: Some("auth-tag-secret".to_string()), // MUST NOT appear in snapshot + relay_url: "wss://relay.example.com".to_string(), // MUST NOT appear in snapshot + avatar_url: Some("https://example.com/avatar.png".to_string()), + acp_command: "/usr/local/bin/acp".to_string(), // MUST NOT appear in snapshot + agent_command: "goose".to_string(), // MUST NOT appear in snapshot + agent_command_override: Some("goose-override".to_string()), // MUST NOT appear + agent_args: vec!["--arg".to_string()], // MUST NOT appear in snapshot + mcp_command: "mcp-server".to_string(), // MUST NOT appear in snapshot + turn_timeout_seconds: 120, // deprecated, MUST NOT appear + idle_timeout_seconds: Some(30), + max_turn_duration_seconds: Some(600), + parallelism: 2, + system_prompt: Some("You are a test agent.".to_string()), + model: Some("claude-opus-4".to_string()), + provider: Some("anthropic".to_string()), + persona_source_version: Some("v1.0".to_string()), // MUST NOT appear + env_vars: { + let mut m = BTreeMap::new(); + m.insert("API_KEY".to_string(), "secret123".to_string()); // MUST NOT appear + m + }, + start_on_app_launch: true, + auto_restart_on_config_change: true, + runtime_pid: Some(12345), // MUST NOT appear + backend: BackendKind::Provider { + // MUST NOT appear — carries a provider secret + id: "SENTINEL_BACKEND_ID".to_string(), + config: serde_json::json!({"api_key": "SENTINEL_BACKEND_SECRET"}), + }, + backend_agent_id: Some("SENTINEL_BACKEND_AGENT_ID".to_string()), // MUST NOT appear + provider_binary_path: Some("/usr/bin/SENTINEL_PROVIDER_BINARY".to_string()), // MUST NOT appear + persona_team_dir: Some(std::path::PathBuf::from("SENTINEL_TEAM_DIR")), // MUST NOT appear + persona_name_in_team: Some("SENTINEL_NAME_IN_TEAM".to_string()), // MUST NOT appear + created_at: "2024-01-01T00:00:00Z".to_string(), + updated_at: "2024-01-02T00:00:00Z".to_string(), + last_started_at: Some("2024-01-03T00:00:00Z".to_string()), // MUST NOT appear + last_stopped_at: None, + last_exit_code: Some(0), // MUST NOT appear + last_error: Some("SENTINEL_LAST_ERROR".to_string()), // MUST NOT appear + last_error_code: Some(42), // MUST NOT appear + respond_to: RespondTo::default(), + respond_to_allowlist: vec!["pubkey1hex".to_string()], + slug: Some("test-agent".to_string()), + runtime: Some("goose".to_string()), + name_pool: vec!["Alice".to_string(), "Bob".to_string()], + is_builtin: false, + is_active: true, + shared: false, + source_team: Some("team-id-123".to_string()), // MUST NOT appear + source_team_persona_slug: Some("lep".to_string()), // MUST NOT appear + definition_respond_to: Some("allowlist".to_string()), + catalog_source: None, + definition_respond_to_allowlist: vec!["abc123def".to_string()], + definition_parallelism: Some(4), + relay_mesh: None, + } +} + +// ── Round-trip tests ────────────────────────────────────────────────────── + +#[test] +fn json_round_trip_config_only() { + let record = minimal_record(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); + let bytes = encode_snapshot_json(&snapshot).unwrap(); + let parsed = decode_snapshot_json(&bytes).unwrap(); + assert_eq!(parsed, snapshot); +} + +#[test] +fn json_round_trip_with_memory() { + let record = minimal_record(); + let entries = vec![ + AgentSnapshotMemoryEntry { + slug: "core".to_string(), + body: "I am a test agent.".to_string(), + }, + AgentSnapshotMemoryEntry { + slug: "mem/research".to_string(), + body: "Some research notes.".to_string(), + }, + ]; + let snapshot = build_snapshot(&record, MemoryLevel::Everything, entries, None); + let bytes = encode_snapshot_json(&snapshot).unwrap(); + let parsed = decode_snapshot_json(&bytes).unwrap(); + assert_eq!(parsed, snapshot); +} + +#[test] +fn png_round_trip_no_memory() { + let record = minimal_record(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); + let png_bytes = encode_snapshot_png(&snapshot, None).unwrap(); + let parsed = decode_snapshot_png(&png_bytes).unwrap(); + assert_eq!(parsed.definition.name, snapshot.definition.name); + assert_eq!(parsed.profile.display_name, snapshot.profile.display_name); + assert_eq!(parsed.memory.level, MemoryLevel::None); +} + +#[test] +fn png_round_trip_with_avatar_png() { + // Build a minimal PNG avatar. + let avatar = make_png_with_text("dummy", "value").unwrap(); + let record = minimal_record(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], Some(&avatar)); + // Avatar should be inlined as a data URL. + assert!(snapshot + .profile + .avatar_data_url + .as_deref() + .unwrap_or("") + .starts_with("data:image/png;base64,")); + + let png_bytes = encode_snapshot_png(&snapshot, Some(&avatar)).unwrap(); + let parsed = decode_snapshot_png(&png_bytes).unwrap(); + assert_eq!(parsed.definition.name, snapshot.definition.name); +} + +/// Plain-card byte compatibility: `encode_snapshot_png` was refactored +/// through the shared `encode_chunk_payload_png` when locked cards were +/// added. Plain cards must emit byte-identical PNGs to the pre-envelope +/// encoder. This vector reimplements the legacy encoder body verbatim and +/// asserts equality on all three composition paths: placeholder (no avatar), +/// PNG-avatar (where tEXt chunk injection ordering matters), and +/// JPEG-avatar transcode. +#[test] +fn plain_encoder_bytes_identical_to_pre_envelope_encoder() { + // Verbatim pre-refactor `encode_snapshot_png` body (post memory guard). + fn legacy_encode( + snapshot: &AgentSnapshot, + avatar_bytes: Option<&[u8]>, + ) -> Result, String> { + let json_bytes = encode_snapshot_json(snapshot)?; + let chunk_text = STANDARD.encode(&json_bytes); + let png_bytes = match avatar_bytes.filter(|bytes| !bytes.is_empty()) { + Some(bytes) => { + let encoded_avatar = if bytes.starts_with(b"\x89PNG") { + inject_text_chunk(bytes, PNG_CHUNK_KEYWORD, &chunk_text).or_else(|_| { + transcode_avatar_to_png_with_text(bytes, PNG_CHUNK_KEYWORD, &chunk_text) + }) + } else { + transcode_avatar_to_png_with_text(bytes, PNG_CHUNK_KEYWORD, &chunk_text) + }; + match encoded_avatar { + Ok(png_bytes) => png_bytes, + Err(_) => make_png_with_text(PNG_CHUNK_KEYWORD, &chunk_text)?, + } + } + None => make_png_with_text(PNG_CHUNK_KEYWORD, &chunk_text)?, + }; + Ok(png_bytes) + } + + let record = minimal_record(); + + // Placeholder path (no avatar). + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); + assert_eq!( + encode_snapshot_png(&snapshot, None).unwrap(), + legacy_encode(&snapshot, None).unwrap(), + "placeholder-path plain PNG bytes must match the pre-envelope encoder" + ); + + // PNG-avatar path: chunk injected into the avatar image body. + let avatar = make_png_with_text("dummy", "value").unwrap(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], Some(&avatar)); + assert_eq!( + encode_snapshot_png(&snapshot, Some(&avatar)).unwrap(), + legacy_encode(&snapshot, Some(&avatar)).unwrap(), + "avatar-path plain PNG bytes must match the pre-envelope encoder" + ); + + // JPEG-avatar path: transcode-to-PNG composition. + let jpeg_avatar = image::DynamicImage::ImageRgb8(image::RgbImage::from_pixel( + 4, + 4, + image::Rgb([0x10, 0x20, 0x30]), + )); + let mut jpeg_bytes = Vec::new(); + jpeg_avatar + .write_to(&mut Cursor::new(&mut jpeg_bytes), image::ImageFormat::Jpeg) + .unwrap(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], Some(&jpeg_bytes)); + assert_eq!( + encode_snapshot_png(&snapshot, Some(&jpeg_bytes)).unwrap(), + legacy_encode(&snapshot, Some(&jpeg_bytes)).unwrap(), + "transcode-path plain PNG bytes must match the pre-envelope encoder" + ); +} + +#[test] +fn png_snapshot_transcodes_jpeg_avatar_into_image_body() { + let avatar = image::DynamicImage::ImageRgb8(image::RgbImage::from_pixel( + 3, + 2, + image::Rgb([0x12, 0x34, 0x56]), + )); + let mut jpeg_bytes = Vec::new(); + avatar + .write_to(&mut Cursor::new(&mut jpeg_bytes), image::ImageFormat::Jpeg) + .unwrap(); + + let snapshot = build_snapshot( + &minimal_record(), + MemoryLevel::None, + vec![], + Some(&jpeg_bytes), + ); + let png_bytes = encode_snapshot_png(&snapshot, Some(&jpeg_bytes)).unwrap(); + let decoder = Decoder::new(Cursor::new(png_bytes)); + let reader = decoder.read_info().unwrap(); + + assert_eq!((reader.info().width, reader.info().height), (3, 2)); +} + +// ── PNG memory parity ───────────────────────────────────────────────────── + +#[test] +fn png_round_trip_with_core_memory() { + let record = minimal_record(); + let entries = vec![AgentSnapshotMemoryEntry { + slug: "core".to_string(), + body: "remember this".to_string(), + }]; + let snapshot = build_snapshot(&record, MemoryLevel::Core, entries, None); + + let png_bytes = encode_snapshot_png(&snapshot, None).unwrap(); + let parsed = decode_snapshot_png(&png_bytes).unwrap(); + + assert_eq!(parsed.memory, snapshot.memory); +} + +#[test] +fn png_round_trip_with_everything_memory() { + let record = minimal_record(); + let entries = vec![ + AgentSnapshotMemoryEntry { + slug: "core".to_string(), + body: "remember this".to_string(), + }, + AgentSnapshotMemoryEntry { + slug: "mem/notes".to_string(), + body: "private notes".to_string(), + }, + ]; + let snapshot = build_snapshot(&record, MemoryLevel::Everything, entries, None); + + let png_bytes = encode_snapshot_png(&snapshot, None).unwrap(); + let parsed = decode_snapshot_png(&png_bytes).unwrap(); + + assert_eq!(parsed.memory, snapshot.memory); +} + +#[test] +fn png_export_with_no_memory_succeeds() { + let record = minimal_record(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); + assert!(encode_snapshot_png(&snapshot, None).is_ok()); +} + +#[test] +fn png_export_rejects_none_level_with_nonempty_entries() { + // Inconsistent state: level == None but entries is non-empty. + // The encoder must reject this to prevent a memory-leak bypass. + let record = minimal_record(); + let entries = vec![AgentSnapshotMemoryEntry { + slug: "core".to_string(), + body: "leaked memory".to_string(), + }]; + // Build with entries, then override level to None in the struct. + let mut snapshot = build_snapshot(&record, MemoryLevel::Core, entries, None); + snapshot.memory.level = MemoryLevel::None; // force inconsistency + let result = encode_snapshot_png(&snapshot, None); + assert!( + result.is_err(), + "PNG encoder must reject level=None with non-empty entries" + ); + assert!( + result + .unwrap_err() + .contains("memory.level 'none' and non-empty memory entries"), + "Error must explain the malformed memory state" + ); +} + +// ── Secret exclusion tests ──────────────────────────────────────────────── +// +// These tests assert that every field in the exclusion list is absent from +// the serialized snapshot. We serialize to JSON and assert the key is NOT +// present. + +fn snapshot_json_string(record: &ManagedAgentRecord) -> String { + let snapshot = build_snapshot(record, MemoryLevel::None, vec![], None); + let bytes = encode_snapshot_json(&snapshot).unwrap(); + String::from_utf8(bytes).unwrap() +} + +#[test] +fn secret_exclusion_private_key_nsec_absent() { + let record = minimal_record(); + let json = snapshot_json_string(&record); + assert!( + !json.contains("nsec1secret"), + "nsec must not appear in snapshot" + ); + assert!( + !json.contains("privateKeyNsec") && !json.contains("private_key_nsec"), + "privateKeyNsec field must not appear in snapshot" + ); +} + +#[test] +fn secret_exclusion_auth_tag_absent() { + let record = minimal_record(); + let json = snapshot_json_string(&record); + assert!( + !json.contains("auth-tag-secret"), + "auth_tag value must not appear in snapshot" + ); + assert!( + !json.contains("authTag") && !json.contains("auth_tag"), + "authTag field must not appear in snapshot" + ); +} + +#[test] +fn secret_exclusion_env_vars_absent() { + let record = minimal_record(); + let json = snapshot_json_string(&record); + assert!( + !json.contains("API_KEY") && !json.contains("secret123"), + "env_vars content must not appear in snapshot" + ); + assert!( + !json.contains("envVars") && !json.contains("env_vars"), + "envVars field must not appear in snapshot" + ); +} + +#[test] +fn secret_exclusion_relay_url_absent() { + let record = minimal_record(); + let json = snapshot_json_string(&record); + assert!( + !json.contains("wss://relay.example.com"), + "relay_url value must not appear in snapshot" + ); + assert!( + !json.contains("relayUrl") && !json.contains("relay_url"), + "relayUrl field must not appear in snapshot" + ); +} + +#[test] +fn snapshot_omits_removed_mcp_toolsets_config() { + let record = minimal_record(); + let json = snapshot_json_string(&record); + assert!( + !json.contains("mcpToolsets") && !json.contains("mcp_toolsets"), + "removed MCP toolsets config must not re-enter snapshots" + ); +} + +#[test] +fn secret_exclusion_machine_commands_absent() { + let record = minimal_record(); + let json = snapshot_json_string(&record); + // acp_command / agent_command / agent_command_override / agent_args / mcp_command + assert!( + !json.contains("/usr/local/bin/acp"), + "acp_command path must not appear" + ); + assert!( + !json.contains("acpCommand") && !json.contains("acp_command"), + "acpCommand field must not appear" + ); + assert!( + !json.contains("agentCommand") && !json.contains("agent_command"), + "agentCommand field must not appear" + ); + assert!( + !json.contains("mcpCommand") && !json.contains("mcp_command"), + "mcpCommand field must not appear" + ); +} + +#[test] +fn secret_exclusion_runtime_state_absent() { + let record = minimal_record(); + let json = snapshot_json_string(&record); + assert!( + !json.contains("runtimePid") && !json.contains("runtime_pid"), + "runtimePid must not appear" + ); + assert!( + !json.contains("backendAgentId") && !json.contains("backend_agent_id"), + "backendAgentId must not appear" + ); + assert!( + !json.contains("SENTINEL_BACKEND_AGENT_ID"), + "backendAgentId value must not appear" + ); + assert!( + !json.contains("providerBinaryPath") && !json.contains("provider_binary_path"), + "providerBinaryPath must not appear" + ); + assert!( + !json.contains("SENTINEL_PROVIDER_BINARY"), + "providerBinaryPath value must not appear" + ); + assert!( + !json.contains("lastStartedAt") && !json.contains("last_started_at"), + "lastStartedAt must not appear" + ); + assert!( + !json.contains("lastExitCode") && !json.contains("last_exit_code"), + "lastExitCode must not appear" + ); + // backend blob — neither the type tag nor provider secret must leak. + assert!( + !json.contains("\"backend\"") && !json.contains("backend"), + "backend field must not appear" + ); + assert!( + !json.contains("SENTINEL_BACKEND_ID") && !json.contains("SENTINEL_BACKEND_SECRET"), + "backend config values must not appear" + ); + // last_error / last_error_code + assert!( + !json.contains("lastError") && !json.contains("last_error"), + "lastError must not appear" + ); + assert!( + !json.contains("SENTINEL_LAST_ERROR"), + "lastError value must not appear" + ); + assert!( + !json.contains("lastErrorCode") && !json.contains("last_error_code"), + "lastErrorCode must not appear" + ); +} + +#[test] +fn secret_exclusion_lineage_ids_absent() { + let record = minimal_record(); + let json = snapshot_json_string(&record); + assert!( + !json.contains("team-id-123"), + "source_team value must not appear" + ); + assert!( + !json.contains("sourceTeam") && !json.contains("source_team"), + "sourceTeam field must not appear" + ); + assert!( + !json.contains("sourceTeamPersonaSlug"), + "sourceTeamPersonaSlug must not appear" + ); + assert!( + !json.contains("personaSourceVersion") && !json.contains("persona_source_version"), + "personaSourceVersion must not appear" + ); + // personaId + assert!( + !json.contains("personaId") && !json.contains("persona_id"), + "personaId field must not appear" + ); + assert!( + !json.contains("SENTINEL_PERSONA_ID"), + "personaId value must not appear" + ); + // teamId + assert!( + !json.contains("teamId") && !json.contains("team_id"), + "teamId field must not appear" + ); + assert!( + !json.contains("SENTINEL_TEAM_ID"), + "teamId value must not appear" + ); + // personaTeamDir + assert!( + !json.contains("personaTeamDir") && !json.contains("persona_team_dir"), + "personaTeamDir field must not appear" + ); + assert!( + !json.contains("SENTINEL_TEAM_DIR"), + "personaTeamDir value must not appear" + ); + // personaNameInTeam + assert!( + !json.contains("personaNameInTeam") && !json.contains("persona_name_in_team"), + "personaNameInTeam field must not appear" + ); + assert!( + !json.contains("SENTINEL_NAME_IN_TEAM"), + "personaNameInTeam value must not appear" + ); +} + +// ── Definition field presence tests ────────────────────────────────────── + +#[test] +fn definition_fields_present_in_snapshot() { + let record = minimal_record(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); + + assert_eq!(snapshot.definition.name, "Test Agent Display"); + assert!(!snapshot.definition.source_is_builtin); + assert_eq!( + snapshot.definition.system_prompt.as_deref(), + Some("You are a test agent.") + ); + assert_eq!(snapshot.definition.runtime.as_deref(), Some("goose")); + assert_eq!(snapshot.definition.model.as_deref(), Some("claude-opus-4")); + assert_eq!(snapshot.definition.provider.as_deref(), Some("anthropic")); + assert_eq!(snapshot.definition.name_pool, vec!["Alice", "Bob"]); + // definition_respond_to maps to respond_to in the snapshot definition + assert_eq!(snapshot.definition.respond_to.as_deref(), Some("allowlist")); + // definition_respond_to_allowlist should be included + assert!(!snapshot.definition.respond_to_allowlist.is_empty()); +} + +#[test] +fn profile_fields_present_in_snapshot() { + let record = minimal_record(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], None); + assert_eq!(snapshot.profile.display_name, "Test Agent Display"); + // No bytes → should fall back to avatar_url + assert_eq!( + snapshot.profile.avatar_url.as_deref(), + Some("https://example.com/avatar.png") + ); + assert!(snapshot.profile.avatar_data_url.is_none()); +} + +#[test] +fn avatar_inlined_when_under_size_limit() { + let record = minimal_record(); + let small_png = make_png_with_text("k", "v").unwrap(); + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], Some(&small_png)); + assert!(snapshot.profile.avatar_data_url.is_some()); + assert!(snapshot.profile.avatar_url.is_none()); +} + +#[test] +fn avatar_url_fallback_when_over_size_limit() { + let mut record = minimal_record(); + record.avatar_url = Some("https://example.com/big.png".to_string()); + // Synthesize oversized avatar bytes (> 2 MB) — just a large zeroed vec. + let big_bytes = vec![0u8; MAX_AVATAR_INLINE_BYTES + 1]; + let snapshot = build_snapshot(&record, MemoryLevel::None, vec![], Some(&big_bytes)); + assert!(snapshot.profile.avatar_data_url.is_none()); + assert_eq!( + snapshot.profile.avatar_url.as_deref(), + Some("https://example.com/big.png") + ); +} + +// ── Format/version validation ───────────────────────────────────────────── + +#[test] +fn invalid_format_discriminator_is_rejected() { + let mut snapshot = build_snapshot(&minimal_record(), MemoryLevel::None, vec![], None); + snapshot.format = "not-a-buzz-snapshot".to_string(); + let bytes = serde_json::to_vec(&snapshot).unwrap(); + let result = decode_snapshot_json(&bytes); + assert!(result.is_err()); + assert!(result.unwrap_err().contains("Unsupported snapshot format")); +} + +#[test] +fn unsupported_version_is_rejected() { + let mut snapshot = build_snapshot(&minimal_record(), MemoryLevel::None, vec![], None); + snapshot.version = 99; + let bytes = serde_json::to_vec(&snapshot).unwrap(); + let result = decode_snapshot_json(&bytes); + assert!(result.is_err()); + assert!(result.unwrap_err().contains("Unsupported snapshot version")); +} diff --git a/desktop/src-tauri/src/managed_agents/backend.rs b/desktop/src-tauri/src/managed_agents/backend.rs index 2a72af92d7..84dd7e99da 100644 --- a/desktop/src-tauri/src/managed_agents/backend.rs +++ b/desktop/src-tauri/src/managed_agents/backend.rs @@ -1,3 +1,4 @@ +use sha2::{Digest, Sha256}; use std::io::{BufReader, Read, Write}; use std::path::{Path, PathBuf}; use std::sync::mpsc; @@ -7,6 +8,61 @@ const STDERR_CAP: usize = 65536; /// Provider responses should be small JSON objects. Cap stdout to prevent a /// buggy or malicious provider from OOM-ing the desktop process. const STDOUT_CAP: usize = 1_048_576; // 1 MB +const PROVIDER_PROTOCOL_VERSION: u64 = 1; + +fn validate_provider_info(info: &serde_json::Value) -> Result<(), String> { + let object = info + .as_object() + .ok_or_else(|| "provider info response must be a JSON object".to_string())?; + let actual_version = object + .get("protocol_version") + .and_then(serde_json::Value::as_u64); + if actual_version != Some(PROVIDER_PROTOCOL_VERSION) { + return Err(match actual_version { + Some(version) => format!( + "unsupported provider protocol version {version}; desktop requires {PROVIDER_PROTOCOL_VERSION}" + ), + None => "provider info response missing integer protocol_version".to_string(), + }); + } + if object.get("ok") != Some(&serde_json::Value::Bool(true)) { + return Err("provider info response must contain ok: true".to_string()); + } + for field in ["name", "version", "description"] { + if object + .get(field) + .is_none_or(|value| value.as_str().is_none_or(str::is_empty)) + { + return Err(format!( + "provider info response missing non-empty string {field}" + )); + } + } + if !object + .get("config_schema") + .is_some_and(serde_json::Value::is_object) + { + return Err("provider info response missing object config_schema".to_string()); + } + + const FIELDS: &[&str] = &[ + "ok", + "name", + "version", + "protocol_version", + "description", + "config_schema", + ]; + if let Some(field) = object + .keys() + .find(|field| !FIELDS.contains(&field.as_str())) + { + return Err(format!( + "provider info response contains unknown field {field}" + )); + } + Ok(()) +} /// Invoke a provider binary: write JSON to stdin, read JSON from stdout. /// @@ -286,7 +342,7 @@ fn redact_secrets(s: &str) -> String { /// (would match every short token in normal log output). Entries are /// applied in decreasing length order so superstrings get scrubbed before /// substrings — protects against partial overlap leaks. -fn redact_secrets_with(s: &str, extras: &[&str]) -> String { +pub(crate) fn redact_secrets_with(s: &str, extras: &[&str]) -> String { let mut result = s.to_string(); // Extras: longest first to avoid partial-overlap leaks. We use @@ -305,9 +361,23 @@ fn redact_secrets_with(s: &str, extras: &[&str]) -> String { // Then prefix-based scrubbing. This loop *can* re-scan because each // replacement shortens the buffer past the matched prefix — the - // replacement marker `[REDACTED]` does not contain `nsec1` or - // `sprt_tok_`, so progress is guaranteed. - for prefix in &["nsec1", "sprt_tok_"] { + // replacement marker `[REDACTED]` contains none of these prefixes, so + // progress is guaranteed. Any prefix added here must preserve that. + // + // GitHub tokens are recognised by shape as well as by variable name: a + // token reaches output from outside our environment too — embedded in a + // git remote URL an installer echoes, say — where no name-based rule can + // see it. + for prefix in &[ + "nsec1", + "sprt_tok_", + "ghp_", + "gho_", + "ghu_", + "ghs_", + "ghr_", + "github_pat_", + ] { while let Some(pos) = result.find(prefix) { let end = result[pos..] .find(|c: char| c.is_whitespace() || c == '"' || c == '\'') @@ -319,23 +389,29 @@ fn redact_secrets_with(s: &str, extras: &[&str]) -> String { result } -/// Collect string values from `request["agent"]["env_vars"]` (if present) -/// to feed into [`redact_secrets_with`]. Returns an empty Vec if the -/// request shape doesn't match, which is fine — falls back to the default -/// prefix-based scrubbing. +/// Collect string values from every environment map a deploy request can +/// carry. Providers may echo any of these values in diagnostics, including +/// definition/baked values that exist only in the resolved launch block. fn env_secrets_from_request(request: &serde_json::Value) -> Vec { - request - .get("agent") - .and_then(|a| a.get("env_vars")) - .and_then(|e| e.as_object()) - .map(|obj| { - obj.values() - .filter_map(|v| v.as_str()) - .filter(|s| !s.is_empty()) - .map(String::from) - .collect() - }) - .unwrap_or_default() + let agent = request.get("agent"); + let maps = [ + agent.and_then(|value| value.get("env_vars")), + agent + .and_then(|value| value.get("launch")) + .and_then(|value| value.get("env")), + agent + .and_then(|value| value.get("launch")) + .and_then(|value| value.get("policy_env")), + ]; + + maps.into_iter() + .flatten() + .filter_map(serde_json::Value::as_object) + .flat_map(|map| map.values()) + .filter_map(serde_json::Value::as_str) + .filter(|value| !value.is_empty()) + .map(String::from) + .collect() } /// Public-in-crate helper: redact every non-empty value from `env` (plus @@ -354,22 +430,102 @@ pub(crate) fn redact_env_values_in( redact_secrets_with(s, &values) } -/// Deploy an agent via provider binary. Returns the provider-assigned agent_id. -/// -/// `request_id` is included for provider-side logging/correlation but is not -/// validated in the response — the stdin→stdout exchange is 1:1 per process. +/// Copy a resolved provider into a private staging directory while hashing +/// exactly the bytes copied. The staged file becomes non-writable before either +/// invocation, closing the path replacement and in-place rewrite races. +fn stage_provider( + binary: &Path, +) -> Result<(tempfile::TempDir, PathBuf, String, std::fs::File), String> { + let directory = tempfile::Builder::new() + .prefix("buzz-provider-") + .tempdir() + .map_err(|error| format!("failed to create provider staging directory: {error}"))?; + let suffix = if cfg!(windows) { ".exe" } else { "" }; + let staged_path = directory.path().join(format!("provider{suffix}")); + let mut source = std::fs::File::open(binary) + .map_err(|error| format!("failed to open provider for staging: {error}"))?; + let mut staged = std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&staged_path) + .map_err(|error| format!("failed to create staged provider: {error}"))?; + let mut hasher = Sha256::new(); + let mut buffer = [0_u8; 64 * 1024]; + loop { + let count = source + .read(&mut buffer) + .map_err(|error| format!("failed to read provider for staging: {error}"))?; + if count == 0 { + break; + } + staged + .write_all(&buffer[..count]) + .map_err(|error| format!("failed to write staged provider: {error}"))?; + hasher.update(&buffer[..count]); + } + staged + .sync_all() + .map_err(|error| format!("failed to sync staged provider: {error}"))?; + + let mut permissions = staged + .metadata() + .map_err(|error| format!("failed to inspect staged provider: {error}"))? + .permissions(); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + permissions.set_mode(0o500); + } + #[cfg(not(unix))] + permissions.set_readonly(true); + std::fs::set_permissions(&staged_path, permissions) + .map_err(|error| format!("failed to protect staged provider: {error}"))?; + drop(staged); + + #[cfg(windows)] + let execution_guard = { + use std::os::windows::fs::OpenOptionsExt; + std::fs::OpenOptions::new() + .read(true) + // Permit CreateProcess to read the image while denying replacement, + // writes, and deletion until both invocations finish. + .share_mode(windows_sys::Win32::Storage::FileSystem::FILE_SHARE_READ) + .open(&staged_path) + }; + #[cfg(not(windows))] + let execution_guard = std::fs::File::open(&staged_path); + let execution_guard = execution_guard + .map_err(|error| format!("failed to lock staged provider for execution: {error}"))?; + Ok(( + directory, + staged_path, + hex::encode(hasher.finalize()), + execution_guard, + )) +} + +/// Deploy through one immutable staged copy: negotiate protocol v1 before the +/// secret-bearing request, then invoke deploy on those exact same bytes. pub fn provider_deploy( binary: &Path, agent: &serde_json::Value, provider_config: &serde_json::Value, ) -> Result { + let (_directory, staged, _digest, _execution_guard) = stage_provider(binary)?; + let info_request = serde_json::json!({ + "op": "info", + "request_id": uuid::Uuid::new_v4().to_string(), + }); + let info = invoke_provider(&staged, &info_request, Duration::from_secs(10))?; + validate_provider_info(&info)?; + let request = serde_json::json!({ "op": "deploy", "request_id": uuid::Uuid::new_v4().to_string(), "agent": agent, "provider_config": provider_config, }); - let resp = invoke_provider(binary, &request, Duration::from_secs(600))?; + let resp = invoke_provider(&staged, &request, Duration::from_secs(600))?; resp["agent_id"] .as_str() .map(String::from) @@ -409,6 +565,24 @@ pub fn validate_provider_config(config: &serde_json::Value) -> Result<(), String Ok(()) } +/// Derive a provider id from the filename Tauri stages at runtime. Tauri +/// removes its target-triple suffix while copying an external binary, but on +/// Windows leaves the executable/script extension, which is not part of the +/// provider id. +fn provider_id_from_filename(name: &str) -> Option<&str> { + let raw = name.strip_prefix("buzz-backend-")?; + let id = [".exe", ".bat", ".cmd"] + .into_iter() + .find_map(|extension| { + raw.get(raw.len().saturating_sub(extension.len())..) + .filter(|suffix| suffix.eq_ignore_ascii_case(extension)) + .map(|_| &raw[..raw.len() - extension.len()]) + }) + .unwrap_or(raw); + + (!id.is_empty()).then_some(id) +} + /// Enumerate PATH for buzz-backend-* executables. Returns (id, path) pairs. /// Only includes files that are executable. Does NOT execute any binaries. /// @@ -450,10 +624,12 @@ pub fn discover_provider_candidates() -> Vec<(String, PathBuf)> { }; for entry in entries.flatten() { let name = entry.file_name().to_string_lossy().to_string(); - if let Some(id) = name.strip_prefix(prefix) { - if !id.is_empty() && !seen.contains(&name) && is_executable(&entry.path()) { - seen.insert(name.clone()); - results.push((id.to_string(), entry.path())); + if name.starts_with(prefix) { + if let Some(id) = provider_id_from_filename(&name) { + if !seen.contains(&name) && is_executable(&entry.path()) { + seen.insert(name.clone()); + results.push((id.to_string(), entry.path())); + } } } } @@ -524,180 +700,5 @@ pub struct BackendProviderInfo { } #[cfg(test)] -mod tests { - use super::*; - - #[test] - fn redact_secrets_replaces_nsec() { - let s = "key=nsec1abc123def456 other"; - let r = redact_secrets(s); - assert!(r.contains("[REDACTED]")); - assert!(!r.contains("nsec1abc123def456")); - } - - #[test] - fn redact_secrets_replaces_token() { - let s = r#"{"token":"sprt_tok_xyz789"}"#; - let r = redact_secrets(s); - assert!(r.contains("[REDACTED]")); - assert!(!r.contains("sprt_tok_xyz789")); - } - - #[test] - fn redact_secrets_with_extras_scrubs_user_env_values() { - // If a provider echoes back a user-supplied API key in its error - // output, the desktop must not surface that secret unredacted via - // `last_error`. We scrub the literal values that came from the - // request's `agent.env_vars`. - let secret = "sk-ant-api03-abc123def456"; - let stderr = format!("auth failed with key {secret} on host api.anthropic.com"); - let r = redact_secrets_with(&stderr, &[secret]); - assert!(r.contains("[REDACTED]")); - assert!(!r.contains(secret)); - } - - #[test] - fn redact_secrets_with_extras_skips_short_values() { - // Don't scrub values shorter than 4 chars — too noisy. - let r = redact_secrets_with("error code: 42", &["42"]); - assert!(r.contains("42")); - } - - #[test] - fn redact_secrets_with_extras_terminates_when_value_substring_of_marker() { - // Regression: an earlier impl used `while let Some(pos) = find(value)` - // which never terminates if the user's env value is a substring of - // the replacement marker `[REDACTED]` — each replacement - // reintroduces the same text. Now uses `str::replace` (single-pass). - for value in ["REDACTED", "EDACTE", "REDA", "ACTED"] { - let r = redact_secrets_with(&format!("leak={value}"), &[value]); - assert!(r.contains("[REDACTED]")); - } - } - - #[test] - fn redact_secrets_with_extras_handles_overlapping_secrets() { - // Longer entries get scrubbed first so the substring "abc12" isn't - // matched before "abc123" is consumed. - let s = "key1=abc123 key2=abc12"; - let r = redact_secrets_with(s, &["abc12", "abc123"]); - assert!(!r.contains("abc123")); - assert!(!r.contains("abc12 ")); - } - - #[test] - fn env_secrets_from_request_extracts_string_values() { - let req = serde_json::json!({ - "op": "deploy", - "agent": { - "env_vars": { - "ANTHROPIC_API_KEY": "sk-ant-test", - "EMPTY": "", - "NUMERIC": 42, - }, - }, - }); - let secrets = env_secrets_from_request(&req); - assert!(secrets.iter().any(|v| v == "sk-ant-test")); - // Empty and non-string values are filtered out. - assert_eq!(secrets.len(), 1); - } - - #[test] - fn env_secrets_from_request_handles_missing_shape() { - assert!(env_secrets_from_request(&serde_json::json!({})).is_empty()); - assert!(env_secrets_from_request(&serde_json::json!({"agent": {}})).is_empty()); - assert!( - env_secrets_from_request(&serde_json::json!({"agent": {"env_vars": null}})).is_empty() - ); - } - - #[test] - fn redact_env_values_in_scrubs_map_values() { - let mut env = std::collections::BTreeMap::new(); - env.insert("ANTHROPIC_API_KEY".to_string(), "sk-ant-real".to_string()); - env.insert("EMPTY".to_string(), String::new()); - let stderr = "auth=sk-ant-real failed; other context"; - let r = redact_env_values_in(stderr, &env); - assert!(!r.contains("sk-ant-real")); - assert!(r.contains("[REDACTED]")); - } - - #[test] - fn validate_provider_config_rejects_secret_key() { - let cfg = serde_json::json!({"api_key": "val"}); - assert!(validate_provider_config(&cfg).is_err()); - } - - #[test] - fn validate_provider_config_rejects_nested() { - let cfg = serde_json::json!({"region": {"us": "east"}}); - assert!(validate_provider_config(&cfg).is_err()); - } - - #[test] - fn validate_provider_config_accepts_scalars() { - let cfg = serde_json::json!({"region": "us-east-1", "tier": "standard"}); - assert!(validate_provider_config(&cfg).is_ok()); - } - - #[test] - fn validate_provider_config_allows_key_as_substring() { - // "keyboard", "monkey" contain "key" as substring but not as a word segment. - let cfg = serde_json::json!({"keyboard_layout": "us", "monkey_wrench": "tight"}); - assert!(validate_provider_config(&cfg).is_ok()); - } - - #[test] - fn validate_provider_config_rejects_camel_case_secrets() { - assert!(validate_provider_config(&serde_json::json!({"apiKey": "val"})).is_err()); - assert!(validate_provider_config(&serde_json::json!({"accessToken": "val"})).is_err()); - assert!(validate_provider_config(&serde_json::json!({"clientSecret": "val"})).is_err()); - // ALL-CAPS variants - assert!(validate_provider_config(&serde_json::json!({"apiKEY": "val"})).is_err()); - assert!(validate_provider_config(&serde_json::json!({"accessTOKEN": "val"})).is_err()); - } - - #[test] - fn split_config_key_handles_all_styles() { - assert_eq!(split_config_key("apiKey"), vec!["api", "key"]); - assert_eq!(split_config_key("access_token"), vec!["access", "token"]); - assert_eq!(split_config_key("keyboard"), vec!["keyboard"]); - assert_eq!(split_config_key("client-secret"), vec!["client", "secret"]); - // Acronym runs stay together - assert_eq!(split_config_key("APIKey"), vec!["api", "key"]); - assert_eq!(split_config_key("apiKEY"), vec!["api", "key"]); - assert_eq!(split_config_key("accessTOKEN"), vec!["access", "token"]); - assert_eq!(split_config_key("MyAPIKey"), vec!["my", "api", "key"]); - } - - #[test] - fn resolve_provider_binary_rejects_invalid_ids() { - // Path traversal - assert!(resolve_provider_binary("../evil").is_err()); - // Empty - assert!(resolve_provider_binary("").is_err()); - // Uppercase - assert!(resolve_provider_binary("MyProvider").is_err()); - // Spaces - assert!(resolve_provider_binary("my provider").is_err()); - // Shell metacharacters - assert!(resolve_provider_binary("foo;rm -rf /").is_err()); - // Valid format but not on PATH — should fail with "not found" - assert!(resolve_provider_binary("nonexistent-test-id-12345").is_err()); - } - - #[test] - fn resolve_provider_binary_accepts_valid_id_format() { - // Valid ID format should pass validation. If the binary happens to - // exist on PATH, Ok is returned; otherwise Err contains "not found" - // (not "invalid provider ID"). Either outcome proves validation passed. - match resolve_provider_binary("zzz-nonexistent-test-provider") { - Ok(_) => {} // unlikely but fine — binary exists - Err(e) => assert!( - e.contains("not found"), - "expected 'not found' error, got: {e}" - ), - } - } -} +#[path = "backend_tests.rs"] +mod tests; diff --git a/desktop/src-tauri/src/managed_agents/backend_tests.rs b/desktop/src-tauri/src/managed_agents/backend_tests.rs new file mode 100644 index 0000000000..ce1f81466f --- /dev/null +++ b/desktop/src-tauri/src/managed_agents/backend_tests.rs @@ -0,0 +1,452 @@ +use super::*; + +#[test] +fn redact_secrets_replaces_nsec() { + let s = "key=nsec1abc123def456 other"; + let r = redact_secrets(s); + assert!(r.contains("[REDACTED]")); + assert!(!r.contains("nsec1abc123def456")); +} + +#[test] +fn redact_secrets_replaces_token() { + let s = r#"{"token":"sprt_tok_xyz789"}"#; + let r = redact_secrets(s); + assert!(r.contains("[REDACTED]")); + assert!(!r.contains("sprt_tok_xyz789")); +} + +#[test] +fn redact_secrets_with_extras_scrubs_user_env_values() { + // If a provider echoes back a user-supplied API key in its error + // output, the desktop must not surface that secret unredacted via + // `last_error`. We scrub the literal values that came from the + // request's `agent.env_vars`. + let secret = "sk-ant-api03-abc123def456"; + let stderr = format!("auth failed with key {secret} on host api.anthropic.com"); + let r = redact_secrets_with(&stderr, &[secret]); + assert!(r.contains("[REDACTED]")); + assert!(!r.contains(secret)); +} + +#[test] +fn redact_secrets_with_extras_skips_short_values() { + // Don't scrub values shorter than 4 chars — too noisy. + let r = redact_secrets_with("error code: 42", &["42"]); + assert!(r.contains("42")); +} + +/// GitHub tokens are recognised by shape, so one that never passed through +/// our environment — embedded in a remote URL an installer echoes — is +/// still scrubbed. The scan runs to the next whitespace or quote, so the +/// rest of the URL goes with it; over-redaction is the safe direction. +#[test] +fn redact_secrets_with_scrubs_github_token_prefixes() { + for token in [ + "ghp_abcdefghij0123456789", + "gho_abcdefghij0123456789", + "ghu_abcdefghij0123456789", + "ghs_abcdefghij0123456789", + "ghr_abcdefghij0123456789", + "github_pat_abcdefghij0123456789", + ] { + let r = redact_secrets_with(&format!("cloning https://{token}@github.com/o/r now"), &[]); + assert!(!r.contains(token), "leaked {token}: {r}"); + assert!(r.contains("[REDACTED]"), "got: {r}"); + assert!(r.contains("cloning"), "scan must stop at whitespace: {r}"); + assert!(r.ends_with(" now"), "scan must stop at whitespace: {r}"); + } +} + +#[test] +fn redact_secrets_with_extras_terminates_when_value_substring_of_marker() { + // Regression: an earlier impl used `while let Some(pos) = find(value)` + // which never terminates if the user's env value is a substring of + // the replacement marker `[REDACTED]` — each replacement + // reintroduces the same text. Now uses `str::replace` (single-pass). + for value in ["REDACTED", "EDACTE", "REDA", "ACTED"] { + let r = redact_secrets_with(&format!("leak={value}"), &[value]); + assert!(r.contains("[REDACTED]")); + } +} + +#[test] +fn redact_secrets_with_extras_handles_overlapping_secrets() { + // Longer entries get scrubbed first so the substring "abc12" isn't + // matched before "abc123" is consumed. + let s = "key1=abc123 key2=abc12"; + let r = redact_secrets_with(s, &["abc12", "abc123"]); + assert!(!r.contains("abc123")); + assert!(!r.contains("abc12 ")); +} + +#[test] +fn env_secrets_from_request_extracts_string_values() { + let req = serde_json::json!({ + "op": "deploy", + "agent": { + "env_vars": { + "ANTHROPIC_API_KEY": "sk-ant-test", + "EMPTY": "", + "NUMERIC": 42, + }, + }, + }); + let secrets = env_secrets_from_request(&req); + assert!(secrets.iter().any(|v| v == "sk-ant-test")); + // Empty and non-string values are filtered out. + assert_eq!(secrets.len(), 1); +} + +#[test] +fn env_secrets_from_request_handles_missing_shape() { + assert!(env_secrets_from_request(&serde_json::json!({})).is_empty()); + assert!(env_secrets_from_request(&serde_json::json!({"agent": {}})).is_empty()); + assert!(env_secrets_from_request(&serde_json::json!({"agent": {"env_vars": null}})).is_empty()); +} + +#[test] +fn redact_env_values_in_scrubs_map_values() { + let mut env = std::collections::BTreeMap::new(); + env.insert("ANTHROPIC_API_KEY".to_string(), "sk-ant-real".to_string()); + env.insert("EMPTY".to_string(), String::new()); + let stderr = "auth=sk-ant-real failed; other context"; + let r = redact_env_values_in(stderr, &env); + assert!(!r.contains("sk-ant-real")); + assert!(r.contains("[REDACTED]")); +} + +#[test] +fn env_secrets_from_request_includes_resolved_launch_maps() { + let req = serde_json::json!({ + "agent": { + "env_vars": {"LEGACY": "legacy-secret"}, + "launch": { + "env": {"PERSONA": "persona-secret"}, + "policy_env": {"POLICY": "policy-secret"} + } + } + }); + let secrets = env_secrets_from_request(&req); + assert_eq!(secrets.len(), 3); + for secret in ["legacy-secret", "persona-secret", "policy-secret"] { + assert!(secrets.iter().any(|candidate| candidate == secret)); + } +} + +#[cfg(unix)] +fn write_test_provider(path: &Path, body: &str) { + use std::os::unix::fs::PermissionsExt; + std::fs::write(path, format!("#!/bin/sh\nset -eu\n{body}\n")).unwrap(); + std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o700)).unwrap(); +} + +#[cfg(unix)] +#[test] +fn provider_deploy_negotiates_and_deploys_the_same_staged_bytes() { + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + let log = directory.path().join("invocations"); + let body = format!( + r#"read request +printf '%s\n' "$0" >> '{}' +case "$request" in + *\"op\":\"info\"*) printf '%s\n' '{{"ok":true,"name":"test","version":"1.0.0","protocol_version":1,"description":"test provider","config_schema":{{}}}}' ;; + *\"op\":\"deploy\"*) printf '%s\n' '{{"ok":true,"agent_id":"remote-1"}}' ;; +esac"#, + log.display() + ); + write_test_provider(&provider, &body); + + let id = provider_deploy(&provider, &serde_json::json!({}), &serde_json::json!({})) + .expect("staged deploy"); + assert_eq!(id, "remote-1"); + let paths: Vec<_> = std::fs::read_to_string(log) + .unwrap() + .lines() + .map(str::to_owned) + .collect(); + assert_eq!(paths.len(), 2); + assert_eq!(paths[0], paths[1]); + assert_ne!(Path::new(&paths[0]), provider); + assert!( + !Path::new(&paths[0]).exists(), + "staging directory must be deleted" + ); +} + +#[cfg(unix)] +fn replacement_provider() -> &'static str { + r#"#!/bin/sh +set -eu +read request +case "$request" in + *\"op\":\"info\"*) printf '%s\n' '{"ok":true,"name":"replacement","version":"9.9.9","protocol_version":1,"description":"replacement provider","config_schema":{}}' ;; + *\"op\":\"deploy\"*) printf '%s\n' '{"ok":true,"agent_id":"replacement-bytes-ran"}' ;; +esac +"# +} + +#[cfg(unix)] +#[test] +fn provider_deploy_uses_staged_bytes_after_same_inode_source_rewrite() { + use std::os::unix::fs::MetadataExt; + + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + let replacement = directory.path().join("replacement"); + std::fs::write(&replacement, replacement_provider()).unwrap(); + let body = format!( + r#"read request +case "$request" in + *\"op\":\"info\"*) + cat '{}' > '{}' + chmod 700 '{}' + printf '%s\n' '{{"ok":true,"name":"original","version":"1.0.0","protocol_version":1,"description":"original provider","config_schema":{{}}}}' + ;; + *\"op\":\"deploy\"*) printf '%s\n' '{{"ok":true,"agent_id":"original-staged-bytes"}}' ;; +esac"#, + replacement.display(), + provider.display(), + provider.display(), + ); + write_test_provider(&provider, &body); + let inode_before = std::fs::metadata(&provider).unwrap().ino(); + + let id = provider_deploy(&provider, &serde_json::json!({}), &serde_json::json!({})) + .expect("deploy from immutable staged copy"); + + assert_eq!(id, "original-staged-bytes"); + assert_eq!( + std::fs::metadata(&provider).unwrap().ino(), + inode_before, + "test must rewrite the source binary in place" + ); + assert_eq!( + std::fs::read_to_string(&provider).unwrap(), + replacement_provider(), + "source pathname must contain replacement bytes before deploy" + ); +} + +#[cfg(unix)] +#[test] +fn provider_deploy_uses_staged_bytes_after_source_pathname_replacement() { + use std::os::unix::fs::{MetadataExt, PermissionsExt}; + + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + let replacement = directory.path().join("replacement"); + std::fs::write(&replacement, replacement_provider()).unwrap(); + std::fs::set_permissions(&replacement, std::fs::Permissions::from_mode(0o700)).unwrap(); + let body = format!( + r#"read request +case "$request" in + *\"op\":\"info\"*) + mv '{}' '{}' + printf '%s\n' '{{"ok":true,"name":"original","version":"1.0.0","protocol_version":1,"description":"original provider","config_schema":{{}}}}' + ;; + *\"op\":\"deploy\"*) printf '%s\n' '{{"ok":true,"agent_id":"original-staged-bytes"}}' ;; +esac"#, + replacement.display(), + provider.display(), + ); + write_test_provider(&provider, &body); + let inode_before = std::fs::metadata(&provider).unwrap().ino(); + + let id = provider_deploy(&provider, &serde_json::json!({}), &serde_json::json!({})) + .expect("deploy from immutable staged copy"); + + assert_eq!(id, "original-staged-bytes"); + assert_ne!( + std::fs::metadata(&provider).unwrap().ino(), + inode_before, + "test must replace the source pathname with a different inode" + ); + assert_eq!( + std::fs::read_to_string(&provider).unwrap(), + replacement_provider(), + "source pathname must contain replacement bytes before deploy" + ); +} + +#[cfg(unix)] +#[test] +fn provider_deploy_refuses_mismatch_before_sending_agent_secret() { + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + let marker = directory.path().join("deploy-received"); + let body = format!( + r#"read request +case "$request" in + *\"op\":\"info\"*) printf '%s\n' '{{"ok":true,"name":"test","version":"2.0.0","protocol_version":2,"description":"test provider","config_schema":{{}}}}' ;; + *\"op\":\"deploy\"*) touch '{}'; printf '%s\n' '{{"ok":true,"agent_id":"bad"}}' ;; +esac"#, + marker.display() + ); + write_test_provider(&provider, &body); + + let error = provider_deploy( + &provider, + &serde_json::json!({"private_key_nsec": "nsec1must-not-cross"}), + &serde_json::json!({}), + ) + .unwrap_err(); + assert!(error.contains("protocol version 2"), "{error}"); + assert!(!marker.exists()); + assert!(!error.contains("nsec1must-not-cross")); +} + +#[cfg(unix)] +#[test] +fn provider_deploy_requires_an_explicit_integer_protocol_version() { + let directory = tempfile::tempdir().unwrap(); + let provider = directory.path().join("provider"); + write_test_provider( + &provider, + r#"read request +printf '%s\n' '{"ok":true,"version":"1.0.0"}'"#, + ); + + let error = + provider_deploy(&provider, &serde_json::json!({}), &serde_json::json!({})).unwrap_err(); + assert!( + error.contains("missing integer protocol_version"), + "{error}" + ); +} + +#[test] +fn provider_info_requires_the_complete_flat_wire_shape() { + let complete = serde_json::json!({ + "ok": true, + "name": "kubernetes", + "version": "1.0.0", + "protocol_version": 1, + "description": "Kubernetes provider", + "config_schema": {} + }); + assert!(validate_provider_info(&complete).is_ok()); + + let mut missing = complete.clone(); + missing.as_object_mut().unwrap().remove("config_schema"); + assert!(validate_provider_info(&missing) + .unwrap_err() + .contains("config_schema")); + + let mut nested = complete; + nested.as_object_mut().unwrap().insert( + "provider".into(), + serde_json::json!({"protocol_version": 1}), + ); + assert!(validate_provider_info(&nested) + .unwrap_err() + .contains("unknown field provider")); +} + +#[test] +fn validate_provider_config_rejects_secret_key() { + let cfg = serde_json::json!({"api_key": "val"}); + assert!(validate_provider_config(&cfg).is_err()); +} + +#[test] +fn validate_provider_config_rejects_nested() { + let cfg = serde_json::json!({"region": {"us": "east"}}); + assert!(validate_provider_config(&cfg).is_err()); +} + +#[test] +fn validate_provider_config_accepts_scalars() { + let cfg = serde_json::json!({"region": "us-east-1", "tier": "standard"}); + assert!(validate_provider_config(&cfg).is_ok()); +} + +#[test] +fn validate_provider_config_allows_key_as_substring() { + // "keyboard", "monkey" contain "key" as substring but not as a word segment. + let cfg = serde_json::json!({"keyboard_layout": "us", "monkey_wrench": "tight"}); + assert!(validate_provider_config(&cfg).is_ok()); +} + +#[test] +fn validate_provider_config_rejects_camel_case_secrets() { + assert!(validate_provider_config(&serde_json::json!({"apiKey": "val"})).is_err()); + assert!(validate_provider_config(&serde_json::json!({"accessToken": "val"})).is_err()); + assert!(validate_provider_config(&serde_json::json!({"clientSecret": "val"})).is_err()); + // ALL-CAPS variants + assert!(validate_provider_config(&serde_json::json!({"apiKEY": "val"})).is_err()); + assert!(validate_provider_config(&serde_json::json!({"accessTOKEN": "val"})).is_err()); +} + +#[test] +fn split_config_key_handles_all_styles() { + assert_eq!(split_config_key("apiKey"), vec!["api", "key"]); + assert_eq!(split_config_key("access_token"), vec!["access", "token"]); + assert_eq!(split_config_key("keyboard"), vec!["keyboard"]); + assert_eq!(split_config_key("client-secret"), vec!["client", "secret"]); + // Acronym runs stay together + assert_eq!(split_config_key("APIKey"), vec!["api", "key"]); + assert_eq!(split_config_key("apiKEY"), vec!["api", "key"]); + assert_eq!(split_config_key("accessTOKEN"), vec!["access", "token"]); + assert_eq!(split_config_key("MyAPIKey"), vec!["my", "api", "key"]); +} + +#[test] +fn provider_filename_strips_the_windows_extension() { + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes.exe"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes.EXE"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes.bat"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-kubernetes.CMD"), + Some("kubernetes") + ); + assert_eq!( + provider_id_from_filename("buzz-backend-my-provider"), + Some("my-provider") + ); + assert_eq!(provider_id_from_filename("other"), None); +} + +#[test] +fn resolve_provider_binary_rejects_invalid_ids() { + // Path traversal + assert!(resolve_provider_binary("../evil").is_err()); + // Empty + assert!(resolve_provider_binary("").is_err()); + // Uppercase + assert!(resolve_provider_binary("MyProvider").is_err()); + // Spaces + assert!(resolve_provider_binary("my provider").is_err()); + // Shell metacharacters + assert!(resolve_provider_binary("foo;rm -rf /").is_err()); + // Valid format but not on PATH — should fail with "not found" + assert!(resolve_provider_binary("nonexistent-test-id-12345").is_err()); +} + +#[test] +fn resolve_provider_binary_accepts_valid_id_format() { + // Valid ID format should pass validation. If the binary happens to + // exist on PATH, Ok is returned; otherwise Err contains "not found" + // (not "invalid provider ID"). Either outcome proves validation passed. + match resolve_provider_binary("zzz-nonexistent-test-provider") { + Ok(_) => {} // unlikely but fine — binary exists + Err(e) => assert!( + e.contains("not found"), + "expected 'not found' error, got: {e}" + ), + } +} diff --git a/desktop/src-tauri/src/managed_agents/discovery.rs b/desktop/src-tauri/src/managed_agents/discovery.rs index eecbf4de3e..8d1b8a5013 100644 --- a/desktop/src-tauri/src/managed_agents/discovery.rs +++ b/desktop/src-tauri/src/managed_agents/discovery.rs @@ -10,8 +10,11 @@ use crate::managed_agents::{ HarnessSource, }; +mod presets; mod runtime_metadata; +use presets::{preset_catalog_entry, PRESET_HARNESSES}; +pub(crate) use presets::{preset_harness_definitions, preset_harness_ids}; pub(crate) use runtime_metadata::KnownAcpRuntime; const GOOSE_AVATAR_URL: &str = "https://goose-docs.ai/img/logo_dark.png"; @@ -19,7 +22,6 @@ const CLAUDE_CODE_AVATAR_URL: &str = "https://anthropic.gallerycdn.vsassets.io/e const CODEX_AVATAR_URL: &str = "https://openai.gallerycdn.vsassets.io/extensions/openai/chatgpt/26.5313.41514/1773706730621/Microsoft.VisualStudio.Services.Icons.Default"; const BUZZ_AGENT_AVATAR_URL: &str = "https://raw.githubusercontent.com/block/buzz/refs/heads/main/crates/buzz-agent/buzz-agent.png"; - fn common_binary_paths() -> &'static [PathBuf] { static PATHS: OnceLock> = OnceLock::new(); PATHS.get_or_init(|| { @@ -41,6 +43,7 @@ fn common_binary_paths() -> &'static [PathBuf] { home.join(".local/bin"), home.join(".volta/bin"), home.join(".asdf/shims"), + home.join(".bun/bin"), ]); } // Windows well-known dirs for npm global shims and standalone installer targets. @@ -1436,225 +1439,6 @@ pub(crate) fn discover_acp_runtime_availability(runtime_id: &str) -> Option, -} - -/// Build the catalog entry for one preset harness through an injectable -/// resolver — the seam the preset loop consumes and tests bind. -/// -/// Availability consumes only the adapter-missing arm of the builtin -/// predicate: adapter presence alone decides `Available` (exactly today's -/// behavior — an `amp-acp` without `amp` stays selectable), and -/// `underlying_cli` is consulted only when the adapter is absent, to -/// distinguish `AdapterMissing` (vendor CLI present) from `NotInstalled` -/// (neither found). See the `underlying_cli` field doc for why the full -/// `classify_runtime` predicate is deliberately not used here. -fn preset_catalog_entry( - def: &PresetHarness, - resolve: impl Fn(&str) -> Option, -) -> AcpRuntimeCatalogEntry { - let (availability, command, binary_path) = match resolve(def.command) { - Some(path) => ( - AcpAvailabilityStatus::Available, - Some(def.command.to_string()), - Some(path.display().to_string()), - ), - None => { - let underlying_cli_found = def - .underlying_cli - .map(|cli| resolve(cli).is_some()) - .unwrap_or(false); - if underlying_cli_found { - (AcpAvailabilityStatus::AdapterMissing, None, None) - } else { - (AcpAvailabilityStatus::NotInstalled, None, None) - } - } - }; - let underlying_cli_path = def - .underlying_cli - .and_then(resolve) - .map(|p| p.display().to_string()); - - let default_args = normalize_agent_args( - def.command, - def.args.iter().map(|s| s.to_string()).collect(), - ); - - AcpRuntimeCatalogEntry { - id: def.id.to_string(), - label: def.label.to_string(), - // No remote URL — all preset icons are bundled assets. - avatar_url: String::new(), - availability, - command, - binary_path, - default_args, - mcp_command: None, - model_env_var: None, - provider_env_var: None, - thinking_env_var: None, - install_hint: def.install_hint.to_string(), - install_instructions_url: def.install_instructions_url.to_string(), - can_auto_install: false, - // Kept false even for adapter presets: presets carry one flat - // install_hint (the adapter's), so the requiresExternalCli - // "CLI is missing" wording would pair the wrong noun with it. - // The builtin path, with per-availability hints, is the only - // consumer of the true case. - requires_external_cli: false, - underlying_cli_path, - node_required: false, - auth_status: AuthStatus::NotApplicable, - login_hint: None, - source: HarnessSource::Preset, - // Preset entries have static, non-editable env; definition_env is empty. - definition_env: Default::default(), - } -} - -const PRESET_HARNESSES: &[PresetHarness] = &[ - PresetHarness { - id: "cursor", - label: "Cursor", - command: "cursor-agent", - args: &["acp"], - install_instructions_url: "https://cursor.com/downloads", - install_hint: "Buzz talks to Cursor through the cursor-agent CLI's ACP mode.", - underlying_cli: None, - }, - PresetHarness { - id: "omp", - label: "Oh My Pi", - command: "omp", - args: &["acp"], - install_instructions_url: "https://github.com/can1357/oh-my-pi", - install_hint: "Buzz talks to Oh My Pi through its CLI's ACP mode (omp acp).", - underlying_cli: None, - }, - PresetHarness { - id: "grok", - label: "Grok Build", - command: "grok", - args: &["agent", "--always-approve", "stdio"], - install_instructions_url: "https://build.x.ai/docs", - install_hint: "Buzz talks to Grok Build through its CLI's agent stdio mode.", - underlying_cli: None, - }, - PresetHarness { - id: "opencode", - label: "OpenCode", - command: "opencode", - args: &["acp"], - install_instructions_url: "https://opencode.ai/docs", - install_hint: "Buzz talks to OpenCode through its CLI's ACP mode (opencode acp).", - underlying_cli: None, - }, - PresetHarness { - id: "kimi", - label: "Kimi Code", - command: "kimi", - args: &["acp"], - install_instructions_url: "https://kimi.ai/download", - install_hint: "Buzz talks to Kimi Code through its CLI's ACP mode (kimi acp).", - underlying_cli: None, - }, - PresetHarness { - id: "amp", - label: "Amp", - command: "amp-acp", - args: &[], - install_instructions_url: "https://github.com/tao12345666333/amp-acp", - install_hint: "Buzz talks to the Amp CLI through the amp-acp adapter. Follow the setup guide to install the adapter so the amp-acp command is on your PATH.", - underlying_cli: Some("amp"), - }, - PresetHarness { - id: "hermes", - label: "Hermes Agent", - command: "hermes-acp", - args: &[], - install_instructions_url: "https://hermes-agent.nousresearch.com", - install_hint: "Buzz talks to Hermes Agent through its hermes-acp command.", - underlying_cli: None, - }, - PresetHarness { - id: "openclaw", - label: "OpenClaw", - command: "openclaw", - args: &["acp"], - install_instructions_url: "https://docs.openclaw.ai/start/getting-started", - install_hint: "Buzz talks to OpenClaw through its ACP mode (openclaw acp), which relies on the OpenClaw Gateway daemon. Follow the setup guide to install both.\n\n\ - ⚠️ Execution-locus note: `openclaw acp` runs tools inside the \ - OpenClaw Gateway daemon, not in the Desktop process. \ - Desktop-injected BUZZ_* env vars are visible to the `openclaw` \ - harness process itself, but do NOT automatically reach the \ - Gateway's execution environment. If your tools or agent logic \ - needs BUZZ_* credentials at execution time, set them on the \ - Gateway's own environment separately.", - underlying_cli: None, - }, -]; - -/// Return the static preset harness definitions as `HarnessDefinition` values. -/// -/// Used by `warm_harness_registry_from_dir` to seed the loaded-harness registry -/// at startup before the frontend triggers a full discovery run. -pub(crate) fn preset_harness_definitions( -) -> Vec { - PRESET_HARNESSES - .iter() - .map( - |p| crate::managed_agents::custom_harnesses::HarnessDefinition { - id: p.id.to_string(), - label: p.label.to_string(), - command: p.command.to_string(), - args: p.args.iter().map(|s| s.to_string()).collect(), - env: std::collections::BTreeMap::new(), - install_instructions_url: p.install_instructions_url.to_string(), - install_hint: p.install_hint.to_string(), - }, - ) - .collect() -} - -/// Return the static slice of preset harness IDs. -/// -/// Used by `check_id_collision` in `custom_harnesses` to derive the reserved-ID -/// set from the single source of truth (`PRESET_HARNESSES`) rather than a -/// hand-maintained copy. Adding a preset automatically reserves its ID. -pub(crate) fn preset_harness_ids() -> &'static [&'static str] { - // `PRESET_HARNESSES` is `'static`; we project its `id` fields. - // Computed once via OnceLock to avoid repeated allocations on hot paths. - use std::sync::OnceLock; - static IDS: OnceLock> = OnceLock::new(); - IDS.get_or_init(|| PRESET_HARNESSES.iter().map(|p| p.id).collect()) - .as_slice() -} - /// Discover all ACP runtimes, optionally merging user-defined custom harnesses /// from `custom_harnesses_dir`. /// diff --git a/desktop/src-tauri/src/managed_agents/discovery/presets.rs b/desktop/src-tauri/src/managed_agents/discovery/presets.rs new file mode 100644 index 0000000000..3622b21c4a --- /dev/null +++ b/desktop/src-tauri/src/managed_agents/discovery/presets.rs @@ -0,0 +1,335 @@ +use std::path::PathBuf; +use std::sync::OnceLock; + +use crate::managed_agents::{ + AcpAvailabilityStatus, AcpRuntimeCatalogEntry, AuthStatus, HarnessSource, +}; + +use super::normalize_agent_args; + +/// Static data for a well-known tier-2 ACP harness. +pub(super) struct PresetHarness { + pub(super) id: &'static str, + label: &'static str, + command: &'static str, + args: &'static [&'static str], + install_instructions_url: &'static str, + install_hint: &'static str, + /// Vendor CLI the ACP command wraps, when the preset is an adapter. + /// + /// Consulted only when the adapter is absent, so `AdapterMissing` replaces + /// `NotInstalled` when the CLI is present but the adapter is not. `None` + /// when the command is itself the vendor CLI. + underlying_cli: Option<&'static str>, +} + +/// Build one preset catalog entry through an injectable command resolver. +pub(super) fn preset_catalog_entry( + def: &PresetHarness, + resolve: impl Fn(&str) -> Option, +) -> AcpRuntimeCatalogEntry { + let (availability, command, binary_path) = match resolve(def.command) { + Some(path) => ( + AcpAvailabilityStatus::Available, + Some(def.command.to_string()), + Some(path.display().to_string()), + ), + None => { + let underlying_cli_found = def + .underlying_cli + .map(|cli| resolve(cli).is_some()) + .unwrap_or(false); + if underlying_cli_found { + (AcpAvailabilityStatus::AdapterMissing, None, None) + } else { + (AcpAvailabilityStatus::NotInstalled, None, None) + } + } + }; + let underlying_cli_path = def + .underlying_cli + .and_then(resolve) + .map(|path| path.display().to_string()); + + AcpRuntimeCatalogEntry { + id: def.id.to_string(), + label: def.label.to_string(), + // No remote URL — all preset icons are bundled assets. + avatar_url: String::new(), + availability, + command, + binary_path, + default_args: normalize_agent_args( + def.command, + def.args.iter().map(|arg| arg.to_string()).collect(), + ), + mcp_command: None, + model_env_var: None, + provider_env_var: None, + thinking_env_var: None, + install_hint: def.install_hint.to_string(), + install_instructions_url: def.install_instructions_url.to_string(), + can_auto_install: false, + // Presets carry one flat install hint, so builtin external-CLI copy + // would name the wrong missing component for adapter presets. + requires_external_cli: false, + underlying_cli_path, + node_required: false, + auth_status: AuthStatus::NotApplicable, + login_hint: None, + source: HarnessSource::Preset, + definition_env: Default::default(), + } +} + +pub(super) const PRESET_HARNESSES: &[PresetHarness] = &[ + PresetHarness { + id: "devin", + label: "Devin", + command: "devin", + args: &["acp"], + install_instructions_url: "https://docs.devin.ai/cli", + install_hint: "Buzz talks to Devin through the official Devin CLI's ACP mode (devin acp).", + underlying_cli: None, + }, + PresetHarness { + id: "cursor", + label: "Cursor", + command: "cursor-agent", + args: &["acp"], + install_instructions_url: "https://cursor.com/downloads", + install_hint: "Buzz talks to Cursor through the cursor-agent CLI's ACP mode.", + underlying_cli: None, + }, + PresetHarness { + id: "omp", + label: "Oh My Pi", + command: "omp", + args: &["acp"], + install_instructions_url: "https://omp.sh/", + install_hint: "Buzz talks to Oh My Pi through its CLI's ACP mode (omp acp).", + underlying_cli: None, + }, + PresetHarness { + id: "grok", + label: "Grok Build", + command: "grok", + args: &["agent", "--always-approve", "stdio"], + install_instructions_url: "https://build.x.ai/docs", + install_hint: "Buzz talks to Grok Build through its CLI's agent stdio mode.", + underlying_cli: None, + }, + PresetHarness { + id: "opencode", + label: "OpenCode", + command: "opencode", + args: &["acp"], + install_instructions_url: "https://opencode.ai/docs", + install_hint: "Buzz talks to OpenCode through its CLI's ACP mode (opencode acp).", + underlying_cli: None, + }, + PresetHarness { + id: "kimi", + label: "Kimi Code", + command: "kimi", + args: &["acp"], + install_instructions_url: "https://kimi.ai/download", + install_hint: "Buzz talks to Kimi Code through its CLI's ACP mode (kimi acp).", + underlying_cli: None, + }, + PresetHarness { + id: "amp", + label: "Amp", + command: "amp-acp", + args: &[], + install_instructions_url: "https://github.com/tao12345666333/amp-acp", + install_hint: "Buzz talks to the Amp CLI through the amp-acp adapter. Follow the setup guide to install the adapter so the amp-acp command is on your PATH.", + underlying_cli: Some("amp"), + }, + PresetHarness { + id: "hermes", + label: "Hermes Agent", + command: "hermes-acp", + args: &[], + install_instructions_url: "https://hermes-agent.nousresearch.com", + install_hint: "Buzz talks to Hermes Agent through its hermes-acp command.", + underlying_cli: None, + }, + PresetHarness { + id: "openclaw", + label: "OpenClaw", + command: "openclaw", + args: &["acp"], + install_instructions_url: "https://docs.openclaw.ai/start/getting-started", + install_hint: "Buzz talks to OpenClaw through its ACP mode (openclaw acp), which relies on the OpenClaw Gateway daemon. Follow the setup guide to install both.\n\n\ + ⚠️ Execution-locus note: `openclaw acp` runs tools inside the \ + OpenClaw Gateway daemon, not in the Desktop process. \ + Desktop-injected BUZZ_* env vars are visible to the `openclaw` \ + harness process itself, but do NOT automatically reach the \ + Gateway's execution environment. If your tools or agent logic \ + needs BUZZ_* credentials at execution time, set them on the \ + Gateway's own environment separately.", + underlying_cli: None, + }, +]; + +/// Return preset definitions for the spawn/readiness registry. +pub(crate) fn preset_harness_definitions( +) -> Vec { + PRESET_HARNESSES + .iter() + .map( + |preset| crate::managed_agents::custom_harnesses::HarnessDefinition { + id: preset.id.to_string(), + label: preset.label.to_string(), + command: preset.command.to_string(), + args: preset.args.iter().map(|arg| arg.to_string()).collect(), + env: Default::default(), + install_instructions_url: preset.install_instructions_url.to_string(), + install_hint: preset.install_hint.to_string(), + }, + ) + .collect() +} + +/// Return preset IDs from the catalog's single source of truth. +pub(crate) fn preset_harness_ids() -> &'static [&'static str] { + static IDS: OnceLock> = OnceLock::new(); + IDS.get_or_init(|| PRESET_HARNESSES.iter().map(|preset| preset.id).collect()) + .as_slice() +} + +#[cfg(test)] +mod tests { + use std::path::PathBuf; + + use crate::managed_agents::{AcpAvailabilityStatus, AuthStatus, HarnessSource}; + + use super::{preset_catalog_entry, PresetHarness, PRESET_HARNESSES}; + + /// Amp-shaped preset: an ACP adapter wrapping a separately installed CLI. + const ADAPTER_PRESET: PresetHarness = PresetHarness { + id: "amp-test", + label: "Amp Test", + command: "amp-acp", + args: &[], + install_instructions_url: "https://example.com/install", + install_hint: "Install the amp-acp npm adapter.", + underlying_cli: Some("amp"), + }; + + #[test] + fn devin_preset_uses_official_native_acp_invocation() { + let preset = PRESET_HARNESSES + .iter() + .find(|preset| preset.id == "devin") + .expect("Devin preset should be present"); + + assert_eq!(preset.label, "Devin"); + assert_eq!(preset.command, "devin"); + assert_eq!(preset.args, &["acp"]); + assert_eq!(preset.underlying_cli, None); + assert_eq!(preset.install_instructions_url, "https://docs.devin.ai/cli"); + + let entry = preset_catalog_entry(preset, |command| { + (command == "devin").then(|| PathBuf::from("/usr/local/bin/devin")) + }); + assert_eq!(entry.availability, AcpAvailabilityStatus::Available); + assert_eq!(entry.command.as_deref(), Some("devin")); + assert_eq!(entry.default_args, vec!["acp"]); + assert_eq!(entry.binary_path.as_deref(), Some("/usr/local/bin/devin")); + assert_eq!(entry.auth_status, AuthStatus::NotApplicable); + assert_eq!(entry.source, HarnessSource::Preset); + + let missing_entry = preset_catalog_entry(preset, |_| None); + assert_eq!( + missing_entry.availability, + AcpAvailabilityStatus::NotInstalled + ); + assert!(missing_entry.command.is_none()); + assert_eq!(missing_entry.default_args, vec!["acp"]); + } + + #[test] + fn devin_preset_is_exposed_in_the_runtime_catalog() { + use crate::managed_agents::custom_harnesses::registry_test_lock; + + // Discovery touches process-global command-resolution and the loaded + // harness registry. Serialize with the other discovery tests. + let _path_guard = crate::managed_agents::lock_path_mutex(); + let _registry_guard = registry_test_lock(); + + let entry = super::super::discover_acp_runtimes_from(None) + .into_iter() + .find(|entry| entry.id == "devin") + .expect("Devin preset should appear in the runtime catalog"); + + assert_eq!(entry.label, "Devin"); + assert_eq!(entry.default_args, vec!["acp"]); + assert_eq!(entry.install_instructions_url, "https://docs.devin.ai/cli"); + assert_eq!(entry.source, HarnessSource::Preset); + } + + #[test] + fn adapter_missing_when_underlying_cli_present() { + let entry = preset_catalog_entry(&ADAPTER_PRESET, |command| { + (command == "amp").then(|| PathBuf::from("/usr/local/bin/amp")) + }); + assert_eq!(entry.availability, AcpAvailabilityStatus::AdapterMissing); + assert!(entry.command.is_none()); + assert!(entry.binary_path.is_none()); + assert_eq!( + entry.underlying_cli_path.as_deref(), + Some("/usr/local/bin/amp") + ); + assert!(!entry.requires_external_cli); + assert_eq!(entry.install_hint, "Install the amp-acp npm adapter."); + } + + #[test] + fn not_installed_when_adapter_and_cli_are_missing() { + let entry = preset_catalog_entry(&ADAPTER_PRESET, |_| None); + assert_eq!(entry.availability, AcpAvailabilityStatus::NotInstalled); + assert!(entry.underlying_cli_path.is_none()); + assert!(!entry.requires_external_cli); + } + + #[test] + fn available_when_adapter_and_cli_are_present() { + let entry = preset_catalog_entry(&ADAPTER_PRESET, |command| match command { + "amp-acp" => Some(PathBuf::from("/usr/local/bin/amp-acp")), + "amp" => Some(PathBuf::from("/usr/local/bin/amp")), + _ => None, + }); + assert_eq!(entry.availability, AcpAvailabilityStatus::Available); + assert_eq!(entry.command.as_deref(), Some("amp-acp")); + assert_eq!(entry.binary_path.as_deref(), Some("/usr/local/bin/amp-acp")); + assert_eq!( + entry.underlying_cli_path.as_deref(), + Some("/usr/local/bin/amp") + ); + } + + #[test] + fn adapter_presence_is_enough_for_availability() { + let entry = preset_catalog_entry(&ADAPTER_PRESET, |command| { + (command == "amp-acp").then(|| PathBuf::from("/usr/local/bin/amp-acp")) + }); + assert_eq!(entry.availability, AcpAvailabilityStatus::Available); + assert_eq!(entry.command.as_deref(), Some("amp-acp")); + assert_eq!(entry.binary_path.as_deref(), Some("/usr/local/bin/amp-acp")); + assert!(entry.underlying_cli_path.is_none()); + } + + #[test] + fn preset_without_underlying_cli_stays_simple() { + let preset = PresetHarness { + underlying_cli: None, + ..ADAPTER_PRESET + }; + let entry = preset_catalog_entry(&preset, |_| None); + assert_eq!(entry.availability, AcpAvailabilityStatus::NotInstalled); + assert!(!entry.requires_external_cli); + assert!(entry.underlying_cli_path.is_none()); + } +} diff --git a/desktop/src-tauri/src/managed_agents/discovery/tests.rs b/desktop/src-tauri/src/managed_agents/discovery/tests.rs index 1b587dca0e..6fe6a77521 100644 --- a/desktop/src-tauri/src/managed_agents/discovery/tests.rs +++ b/desktop/src-tauri/src/managed_agents/discovery/tests.rs @@ -6,9 +6,9 @@ use super::{ codex_adapter_is_outdated, create_time_agent_command_override, default_agent_command, effective_agent_command, find_nvm_default_bin, find_via_login_shell, is_login_shell_path_uninit, is_safe_nvm_tag, managed_agent_avatar_url, normalize_agent_args, - parse_semver_tag, preset_catalog_entry, probe_codex_acp_version, record_agent_command, - refresh_login_shell_path, try_record_agent_command, PresetHarness, BUZZ_AGENT_AVATAR_URL, - CLAUDE_CODE_AVATAR_URL, CODEX_AVATAR_URL, GOOSE_AVATAR_URL, + parse_semver_tag, probe_codex_acp_version, record_agent_command, refresh_login_shell_path, + try_record_agent_command, BUZZ_AGENT_AVATAR_URL, CLAUDE_CODE_AVATAR_URL, CODEX_AVATAR_URL, + GOOSE_AVATAR_URL, }; use crate::managed_agents::AcpAvailabilityStatus; @@ -187,90 +187,6 @@ fn classifies_cli_missing_when_adapter_found_but_cli_absent() { assert_eq!(path.as_deref(), Some("/opt/homebrew/bin/codex-acp")); } -/// Amp-shaped preset: an ACP adapter (`amp-acp`) wrapping a separately -/// installed vendor CLI (`amp`). -const ADAPTER_PRESET: PresetHarness = PresetHarness { - id: "amp-test", - label: "Amp Test", - command: "amp-acp", - args: &[], - install_instructions_url: "https://example.com/install", - install_hint: "Install the amp-acp npm adapter.", - underlying_cli: Some("amp"), -}; - -#[test] -fn preset_entry_adapter_missing_when_underlying_cli_present() { - // Vendor CLI resolves, adapter does not — the state Tyler's Amp - // hand-test hit. Must NOT degrade to the misleading NotInstalled. - let entry = preset_catalog_entry(&ADAPTER_PRESET, |cmd| { - (cmd == "amp").then(|| PathBuf::from("/usr/local/bin/amp")) - }); - assert_eq!(entry.availability, AcpAvailabilityStatus::AdapterMissing); - assert!(entry.command.is_none()); - assert!(entry.binary_path.is_none()); - assert_eq!( - entry.underlying_cli_path.as_deref(), - Some("/usr/local/bin/amp") - ); - assert!(!entry.requires_external_cli); - assert_eq!(entry.install_hint, "Install the amp-acp npm adapter."); -} - -#[test] -fn preset_entry_not_installed_when_both_missing() { - let entry = preset_catalog_entry(&ADAPTER_PRESET, |_| None); - assert_eq!(entry.availability, AcpAvailabilityStatus::NotInstalled); - assert!(entry.underlying_cli_path.is_none()); - assert!(!entry.requires_external_cli); -} - -#[test] -fn preset_entry_available_when_adapter_and_cli_present() { - let entry = preset_catalog_entry(&ADAPTER_PRESET, |cmd| match cmd { - "amp-acp" => Some(PathBuf::from("/usr/local/bin/amp-acp")), - "amp" => Some(PathBuf::from("/usr/local/bin/amp")), - _ => None, - }); - assert_eq!(entry.availability, AcpAvailabilityStatus::Available); - assert_eq!(entry.command.as_deref(), Some("amp-acp")); - assert_eq!(entry.binary_path.as_deref(), Some("/usr/local/bin/amp-acp")); - assert_eq!( - entry.underlying_cli_path.as_deref(), - Some("/usr/local/bin/amp") - ); -} - -#[test] -fn preset_entry_stays_available_when_adapter_present_but_cli_absent() { - // Wren's regression guard: today an `amp-acp` install without `amp` - // is Available and selectable. Feeding underlying_cli through the - // FULL classify_runtime predicate would flip this to CliMissing - // (unselectable, with backwards install copy) — the adapter-missing - // arm is the only one presets consume. - let entry = preset_catalog_entry(&ADAPTER_PRESET, |cmd| { - (cmd == "amp-acp").then(|| PathBuf::from("/usr/local/bin/amp-acp")) - }); - assert_eq!(entry.availability, AcpAvailabilityStatus::Available); - assert_eq!(entry.command.as_deref(), Some("amp-acp")); - assert_eq!(entry.binary_path.as_deref(), Some("/usr/local/bin/amp-acp")); - assert!(entry.underlying_cli_path.is_none()); -} - -#[test] -fn preset_entry_without_underlying_cli_stays_simple() { - // Most presets: the command IS the vendor CLI. No external-CLI flag, - // absent command means plain NotInstalled. - let preset = PresetHarness { - underlying_cli: None, - ..ADAPTER_PRESET - }; - let entry = preset_catalog_entry(&preset, |_| None); - assert_eq!(entry.availability, AcpAvailabilityStatus::NotInstalled); - assert!(!entry.requires_external_cli); - assert!(entry.underlying_cli_path.is_none()); -} - fn persona_with_runtime(id: &str, runtime: Option<&str>) -> crate::managed_agents::AgentDefinition { crate::managed_agents::AgentDefinition { id: id.to_string(), diff --git a/desktop/src-tauri/src/managed_agents/env_vars.rs b/desktop/src-tauri/src/managed_agents/env_vars.rs index 592a5cbbd9..1653371e7f 100644 --- a/desktop/src-tauri/src/managed_agents/env_vars.rs +++ b/desktop/src-tauri/src/managed_agents/env_vars.rs @@ -77,6 +77,10 @@ pub(crate) const RESERVED_ENV_KEYS: &[&str] = &[ "BUZZ_ACP_RESPOND_TO", "BUZZ_ACP_RESPOND_TO_ALLOWLIST", "BUZZ_ACP_AGENT_OWNER", + // Remote lifetime/presence policy: user env must not disable the + // desktop/provider-owned bounds while the saved record still promises them. + "BUZZ_ACP_EXIT_AFTER_INACTIVITY", + "BUZZ_ACP_NO_PRESENCE", // Readiness handoff: desktop is the ONLY readiness source. A saved or // ambient env var must not be able to forge setup mode (NotReady) on a // Ready agent or suppress it (empty/stale payload) on a NotReady one. @@ -307,28 +311,5 @@ pub(crate) fn live_persona_env( .unwrap_or_default() } -/// Resolve live env_vars for a linked persona, loading personas from disk. -/// -/// Returns the persona's `env_vars` map if a persona_id is provided and found; -/// returns an empty map if no persona is linked. Errors if the linked persona -/// is missing. Used by the provider deploy path, which has no pre-loaded -/// persona slice. -pub(crate) fn resolve_persona_env( - app: &tauri::AppHandle, - persona_id: Option<&str>, -) -> Result, String> { - let Some(pid) = persona_id else { - return Ok(std::collections::BTreeMap::new()); - }; - let personas = super::load_personas(app).map_err(|e| { - format!("failed to load personas while resolving env for persona `{pid}`: {e}") - })?; - let persona = personas - .into_iter() - .find(|p| p.id == pid) - .ok_or_else(|| format!("persona `{pid}` not found while resolving env"))?; - Ok(persona.env_vars) -} - #[cfg(test)] mod tests; diff --git a/desktop/src-tauri/src/managed_agents/env_vars/tests.rs b/desktop/src-tauri/src/managed_agents/env_vars/tests.rs index cf57b12546..534c2e0835 100644 --- a/desktop/src-tauri/src/managed_agents/env_vars/tests.rs +++ b/desktop/src-tauri/src/managed_agents/env_vars/tests.rs @@ -158,6 +158,15 @@ fn reserved_keys_include_respond_to_gate() { } } +#[test] +fn reserved_keys_include_remote_lifetime_policy() { + for key in ["BUZZ_ACP_EXIT_AFTER_INACTIVITY", "BUZZ_ACP_NO_PRESENCE"] { + assert!(is_reserved_env_key(key), "{key} should be reserved"); + let agent = map(&[(key, "0")]); + assert!(merged_user_env(&BTreeMap::new(), &agent).is_empty()); + } +} + #[test] fn reserved_keys_include_code_execution_surface() { // The agent/MCP command + args are what Buzz actually exec's. diff --git a/desktop/src-tauri/src/managed_agents/mod.rs b/desktop/src-tauri/src/managed_agents/mod.rs index b0e86f8edb..772d707f27 100644 --- a/desktop/src-tauri/src/managed_agents/mod.rs +++ b/desktop/src-tauri/src/managed_agents/mod.rs @@ -1,6 +1,7 @@ mod agent_env; pub(crate) mod agent_events; pub(crate) mod agent_snapshot; +pub(crate) mod agent_snapshot_envelope; pub(crate) mod team_snapshot; pub(crate) use agent_env::{ baked_build_env, build_buzz_agent_provider_defaults, discovery_env_with_baked_floor, @@ -29,6 +30,7 @@ pub mod retention; mod runtime; mod runtime_commands; mod runtime_types; +pub(crate) mod snapshot_avatar; pub(crate) mod spawn_hash; pub(crate) mod storage; pub(crate) mod team_events; diff --git a/desktop/src-tauri/src/managed_agents/nest.rs b/desktop/src-tauri/src/managed_agents/nest.rs index e13ab2baad..c8f008836d 100644 --- a/desktop/src-tauri/src/managed_agents/nest.rs +++ b/desktop/src-tauri/src/managed_agents/nest.rs @@ -50,7 +50,7 @@ const NEST_AGENTS_VERSION: u32 = 4; /// Template content version for SKILL.md. /// Bump this when changing `nest_skill.md` to trigger refresh on existing installs. -const NEST_SKILL_VERSION: u32 = 4; +const NEST_SKILL_VERSION: u32 = 5; const BEGIN_MARKER: &str = ""; diff --git a/desktop/src-tauri/src/managed_agents/nest/tests.rs b/desktop/src-tauri/src/managed_agents/nest/tests.rs index d2c415e725..031b049a49 100644 --- a/desktop/src-tauri/src/managed_agents/nest/tests.rs +++ b/desktop/src-tauri/src/managed_agents/nest/tests.rs @@ -29,6 +29,18 @@ fn init_nest_dir_prod_sets_buzz() { } } +#[test] +fn nest_skill_contains_safe_mention_workflow() { + assert!(BUZZ_CLI_SKILL_MD.contains("--mention ")); + assert!(BUZZ_CLI_SKILL_MD.contains("every presentation-only name that should notify")); + assert!(BUZZ_CLI_SKILL_MD + .contains("permits unresolved or ambiguous `@Name` text as presentation-only")); + assert!(BUZZ_CLI_SKILL_MD.contains("signed event's `mention_pubkeys`")); + assert!(BUZZ_CLI_SKILL_MD.contains("no follow-up verification command is needed")); + assert!(BUZZ_CLI_SKILL_MD.contains("Add membership separately only when authorized")); + assert!(BUZZ_CLI_SKILL_MD.contains("never changes membership automatically")); +} + #[test] fn ensure_nest_creates_all_dirs_and_agents_md() { let tmp = tempfile::tempdir().unwrap(); diff --git a/desktop/src-tauri/src/managed_agents/nest_skill.md b/desktop/src-tauri/src/managed_agents/nest_skill.md index fefdfa77fa..79a5ea301d 100644 --- a/desktop/src-tauri/src/managed_agents/nest_skill.md +++ b/desktop/src-tauri/src/managed_agents/nest_skill.md @@ -87,14 +87,11 @@ Write commands are unaffected. `--format json` (default) returns full fields. ## Communication Patterns -**Mentions that notify:** Use `@Name` directly in message content — the CLI auto-resolves channel members by name and adds the required p-tags. No `--mention` flag exists or is needed. `nostr:npub1…` inline references are also auto-resolved to p-tags without needing a flag. +**Mentions that notify:** Keep readable `@Name` text in message content and, when intended pubkeys are known, pass the identities in the same send with repeatable `--mention `. Any explicit identity (`--mention` or `nostr:npub...`) permits unresolved or ambiguous `@Name` text as presentation-only; uniquely resolved member names still add recipients. Include a pubkey for every presentation-only name that should notify. The CLI reports the signed event's `mention_pubkeys`; no follow-up verification command is needed. Without explicit identities, names resolve against current channel members. An unresolved/ambiguous name or non-member target stops before publishing. Add membership separately only when authorized, then retry; sending never changes membership automatically. ```bash -# ✅ Correct — notification delivered automatically -buzz messages send --channel --content "@Alice check this" - -# Multiple mentions — same pattern -buzz messages send --channel --content "@Alice @Bob review please" +buzz messages send --channel \ + --content "@Alice check this" --mention ``` ## DM Management diff --git a/desktop/src-tauri/src/managed_agents/persona_events.rs b/desktop/src-tauri/src/managed_agents/persona_events.rs index ea61a811db..6afc18a501 100644 --- a/desktop/src-tauri/src/managed_agents/persona_events.rs +++ b/desktop/src-tauri/src/managed_agents/persona_events.rs @@ -5,7 +5,7 @@ use std::collections::BTreeMap; -use buzz_core_pkg::kind::{persona_event_is_shared, KIND_PERSONA}; +use buzz_core_pkg::kind::{event_is_shared, KIND_PERSONA}; use nostr::{EventBuilder, Kind, Tag}; use serde::{Deserialize, Serialize}; @@ -192,7 +192,7 @@ pub fn persona_from_event(event: &nostr::Event) -> Result Vec { } Some("anthropic") => Some("ANTHROPIC_MODEL"), Some("openai") | Some("openai-compat") => Some("OPENAI_COMPAT_MODEL"), + Some("openrouter") => Some("OPENROUTER_MODEL"), _ => None, }; let model_present = effective @@ -523,6 +524,12 @@ fn buzz_agent_requirements(effective: &EffectiveAgentEnv) -> Vec { key: "DATABRICKS_HOST".to_string(), }); } + Some("openrouter") + if env_key_missing("OPENROUTER_API_KEY") => { + missing.push(Requirement::EnvKey { + key: "OPENROUTER_API_KEY".to_string(), + }); + } _ => { // Unknown provider or no provider yet — only the NormalizedField // requirement above captures this gap. @@ -630,6 +637,13 @@ fn goose_requirements( key: "DATABRICKS_HOST".to_string(), }); } + Some("openrouter") + if env_key_missing("OPENROUTER_API_KEY") && !file_key_present("OPENROUTER_API_KEY") => + { + missing.push(Requirement::EnvKey { + key: "OPENROUTER_API_KEY".to_string(), + }); + } _ => {} } @@ -1668,195 +1682,62 @@ mod tests { field: "model".to_string() })); } -} - -// ── goose file-config–aware requirement tests ───────────────────────────── -// -// These tests call `goose_requirements` directly, injecting a synthetic -// `RuntimeFileConfig` so there is no disk I/O and tests are deterministic. - -#[cfg(test)] -mod goose_file_config_tests { - use std::collections::BTreeMap; - - use super::*; - use crate::managed_agents::config_bridge::RuntimeFileConfig; - - fn empty_env() -> EffectiveAgentEnv { - EffectiveAgentEnv { - env: BTreeMap::new(), - config_file_path: Some("~/.config/goose/config.yaml"), - effective_command: "goose".to_string(), - } - } - fn env_with(pairs: &[(&str, &str)]) -> EffectiveAgentEnv { - EffectiveAgentEnv { - env: pairs - .iter() - .map(|(k, v)| (k.to_string(), v.to_string())) - .collect(), - config_file_path: Some("~/.config/goose/config.yaml"), - effective_command: "goose".to_string(), - } - } - - fn databricks_file_config() -> RuntimeFileConfig { - let mut extra = BTreeMap::new(); - extra.insert( - "DATABRICKS_HOST".to_string(), - "https://dbc.example.com".to_string(), - ); - RuntimeFileConfig { - provider: Some("databricks_v2".to_string()), - model: Some("goose-claude-4-6-opus".to_string()), - extra, - ..Default::default() - } - } + // ── OpenRouter readiness ───────────────────────────────────────────── #[test] - fn goose_file_config_silences_databricks_host_requirement() { - // File has provider, model, and DATABRICKS_HOST — all requirements silenced. - let env = empty_env(); - let cfg = databricks_file_config(); - let result = goose_requirements(&env, Some(&cfg)); - assert!( - result.is_empty(), - "all requirements should be silenced by goose file config; \ - got: {:?}", - result - ); - } - - #[test] - fn goose_env_empty_file_absent_still_not_ready() { - // No env, no file config → provider and model both required. - let env = empty_env(); - let result = goose_requirements(&env, None); - assert!( - result.contains(&Requirement::NormalizedField { - field: "provider".to_string() - }), - "provider must be required when absent from both env and file" - ); - assert!( - result.contains(&Requirement::NormalizedField { - field: "model".to_string() - }), - "model must be required when absent from both env and file" - ); - } - - #[test] - fn goose_file_config_silences_provider_and_model_but_not_anthropic_key() { - // File has provider=anthropic and model, but ANTHROPIC_API_KEY is not - // in the file's `extra` map — it must still be required. - let cfg = RuntimeFileConfig { - provider: Some("anthropic".to_string()), - model: Some("claude-opus-4-5".to_string()), - extra: BTreeMap::new(), - ..Default::default() - }; - let env = empty_env(); - let result = goose_requirements(&env, Some(&cfg)); - // Provider and model silenced. - assert!( - !result.contains(&Requirement::NormalizedField { - field: "provider".to_string() - }), - "provider silenced by file config" - ); - assert!( - !result.contains(&Requirement::NormalizedField { - field: "model".to_string() - }), - "model silenced by file config" + fn buzz_agent_openrouter_with_all_fields_is_ready() { + let env = make_env( + "buzz-agent", + env_with(&[ + ("BUZZ_AGENT_PROVIDER", "openrouter"), + ("BUZZ_AGENT_MODEL", "anthropic/claude-sonnet-4"), + ("OPENROUTER_API_KEY", "sk-or-test-key"), + ]), ); - // ANTHROPIC_API_KEY not in file extra → still required. + let result = agent_readiness(&env); assert!( - result.contains(&Requirement::EnvKey { - key: "ANTHROPIC_API_KEY".to_string() - }), - "ANTHROPIC_API_KEY must remain required when not in file extra" + result.is_ready(), + "openrouter with all fields should be ready" ); } #[test] - fn goose_env_provider_wins_over_file_provider_for_cred_check() { - // Env has GOOSE_PROVIDER=anthropic (different from file's databricks_v2). - // The env provider must win for credential checking. - let env = env_with(&[ - ("GOOSE_PROVIDER", "anthropic"), - ("GOOSE_MODEL", "claude-opus-4-5"), - ]); - let cfg = databricks_file_config(); // has provider=databricks_v2 - let result = goose_requirements(&env, Some(&cfg)); - // anthropic requires ANTHROPIC_API_KEY, not DATABRICKS_HOST. - assert!( - result.contains(&Requirement::EnvKey { - key: "ANTHROPIC_API_KEY".to_string() - }), - "env provider=anthropic must require ANTHROPIC_API_KEY" - ); - assert!( - !result.contains(&Requirement::EnvKey { - key: "DATABRICKS_HOST".to_string() - }), - "env provider=anthropic must NOT require DATABRICKS_HOST" + fn buzz_agent_openrouter_missing_key_returns_not_ready() { + let env = make_env( + "buzz-agent", + env_with(&[ + ("BUZZ_AGENT_PROVIDER", "openrouter"), + ("BUZZ_AGENT_MODEL", "anthropic/claude-sonnet-4"), + ]), ); + let result = agent_readiness(&env); + assert!(!result.is_ready()); + assert!(result.requirements().contains(&Requirement::EnvKey { + key: "OPENROUTER_API_KEY".to_string() + })); } #[test] - fn goose_flat_databricks_host_in_file_config_silences_requirement() { - // Will's typical goose config: flat DATABRICKS_HOST at the top level, - // no active_provider — provider inferred as "databricks". - // The parser must store extra["DATABRICKS_HOST"] = value (canonical key), - // and goose_requirements must then silence the DATABRICKS_HOST requirement. - let mut extra = BTreeMap::new(); - extra.insert( - "DATABRICKS_HOST".to_string(), - "https://block.cloud.databricks.com".to_string(), - ); - let cfg = RuntimeFileConfig { - provider: Some("databricks".to_string()), - model: Some("goose-claude-4-5".to_string()), - extra, - ..Default::default() - }; - let env = empty_env(); - let result = goose_requirements(&env, Some(&cfg)); - // All requirements silenced — provider (file), model (file), DATABRICKS_HOST (file). - assert!( - result.is_empty(), - "flat DATABRICKS_HOST in file config must silence all requirements; \ - got: {:?}", - result + fn buzz_agent_openrouter_with_provider_model_fallback_is_ready() { + let env = make_env( + "buzz-agent", + env_with(&[ + ("BUZZ_AGENT_PROVIDER", "openrouter"), + ("OPENROUTER_MODEL", "google/gemini-2.5-flash"), + ("OPENROUTER_API_KEY", "sk-or-test-key"), + ]), ); - } - - #[test] - fn goose_goose_provider_databricks_flat_host_silences_databricks_host() { - // GOOSE_PROVIDER=databricks (not active_provider) + flat DATABRICKS_HOST. - // The parser canonicalizes to extra["DATABRICKS_HOST"]; readiness must silence it. - let mut extra = BTreeMap::new(); - extra.insert( - "DATABRICKS_HOST".to_string(), - "https://dbc.example.com".to_string(), - ); - let cfg = RuntimeFileConfig { - provider: Some("databricks".to_string()), - model: Some("some-model".to_string()), - extra, - ..Default::default() - }; - let env = empty_env(); - let result = goose_requirements(&env, Some(&cfg)); + let result = agent_readiness(&env); assert!( - !result.contains(&Requirement::EnvKey { - key: "DATABRICKS_HOST".to_string() - }), - "DATABRICKS_HOST must be silenced when canonical key is in file extra" + result.is_ready(), + "OPENROUTER_MODEL fallback should satisfy model requirement" ); } } + +// Goose file-config-aware requirement tests live in a sibling file so this +// module stays under the desktop file-size ratchet. +#[cfg(test)] +#[path = "readiness_goose_file_config_tests.rs"] +mod goose_file_config_tests; diff --git a/desktop/src-tauri/src/managed_agents/readiness_goose_file_config_tests.rs b/desktop/src-tauri/src/managed_agents/readiness_goose_file_config_tests.rs new file mode 100644 index 0000000000..46d0e4c7a7 --- /dev/null +++ b/desktop/src-tauri/src/managed_agents/readiness_goose_file_config_tests.rs @@ -0,0 +1,190 @@ +//! Goose file-config-aware requirement tests. +//! +//! These tests call `goose_requirements` directly, injecting a synthetic +//! `RuntimeFileConfig` so there is no disk I/O and tests are deterministic. +//! +//! Included from `readiness.rs` via `#[path]`; `super::*` therefore resolves +//! against that module, matching the `storage_tests.rs` convention. + +use std::collections::BTreeMap; + +use super::*; +use crate::managed_agents::config_bridge::RuntimeFileConfig; + +fn empty_env() -> EffectiveAgentEnv { + EffectiveAgentEnv { + env: BTreeMap::new(), + config_file_path: Some("~/.config/goose/config.yaml"), + effective_command: "goose".to_string(), + } +} + +fn env_with(pairs: &[(&str, &str)]) -> EffectiveAgentEnv { + EffectiveAgentEnv { + env: pairs + .iter() + .map(|(k, v)| (k.to_string(), v.to_string())) + .collect(), + config_file_path: Some("~/.config/goose/config.yaml"), + effective_command: "goose".to_string(), + } +} + +fn databricks_file_config() -> RuntimeFileConfig { + let mut extra = BTreeMap::new(); + extra.insert( + "DATABRICKS_HOST".to_string(), + "https://dbc.example.com".to_string(), + ); + RuntimeFileConfig { + provider: Some("databricks_v2".to_string()), + model: Some("goose-claude-4-6-opus".to_string()), + extra, + ..Default::default() + } +} + +#[test] +fn goose_file_config_silences_databricks_host_requirement() { + // File has provider, model, and DATABRICKS_HOST — all requirements silenced. + let env = empty_env(); + let cfg = databricks_file_config(); + let result = goose_requirements(&env, Some(&cfg)); + assert!( + result.is_empty(), + "all requirements should be silenced by goose file config; \ + got: {:?}", + result + ); +} + +#[test] +fn goose_env_empty_file_absent_still_not_ready() { + // No env, no file config → provider and model both required. + let env = empty_env(); + let result = goose_requirements(&env, None); + assert!( + result.contains(&Requirement::NormalizedField { + field: "provider".to_string() + }), + "provider must be required when absent from both env and file" + ); + assert!( + result.contains(&Requirement::NormalizedField { + field: "model".to_string() + }), + "model must be required when absent from both env and file" + ); +} + +#[test] +fn goose_file_config_silences_provider_and_model_but_not_anthropic_key() { + // File has provider=anthropic and model, but ANTHROPIC_API_KEY is not + // in the file's `extra` map — it must still be required. + let cfg = RuntimeFileConfig { + provider: Some("anthropic".to_string()), + model: Some("claude-opus-4-5".to_string()), + extra: BTreeMap::new(), + ..Default::default() + }; + let env = empty_env(); + let result = goose_requirements(&env, Some(&cfg)); + // Provider and model silenced. + assert!( + !result.contains(&Requirement::NormalizedField { + field: "provider".to_string() + }), + "provider silenced by file config" + ); + assert!( + !result.contains(&Requirement::NormalizedField { + field: "model".to_string() + }), + "model silenced by file config" + ); + // ANTHROPIC_API_KEY not in file extra → still required. + assert!( + result.contains(&Requirement::EnvKey { + key: "ANTHROPIC_API_KEY".to_string() + }), + "ANTHROPIC_API_KEY must remain required when not in file extra" + ); +} + +#[test] +fn goose_env_provider_wins_over_file_provider_for_cred_check() { + // Env has GOOSE_PROVIDER=anthropic (different from file's databricks_v2). + // The env provider must win for credential checking. + let env = env_with(&[ + ("GOOSE_PROVIDER", "anthropic"), + ("GOOSE_MODEL", "claude-opus-4-5"), + ]); + let cfg = databricks_file_config(); // has provider=databricks_v2 + let result = goose_requirements(&env, Some(&cfg)); + // anthropic requires ANTHROPIC_API_KEY, not DATABRICKS_HOST. + assert!( + result.contains(&Requirement::EnvKey { + key: "ANTHROPIC_API_KEY".to_string() + }), + "env provider=anthropic must require ANTHROPIC_API_KEY" + ); + assert!( + !result.contains(&Requirement::EnvKey { + key: "DATABRICKS_HOST".to_string() + }), + "env provider=anthropic must NOT require DATABRICKS_HOST" + ); +} + +#[test] +fn goose_flat_databricks_host_in_file_config_silences_requirement() { + // Will's typical goose config: flat DATABRICKS_HOST at the top level, + // no active_provider — provider inferred as "databricks". + // The parser must store extra["DATABRICKS_HOST"] = value (canonical key), + // and goose_requirements must then silence the DATABRICKS_HOST requirement. + let mut extra = BTreeMap::new(); + extra.insert( + "DATABRICKS_HOST".to_string(), + "https://block.cloud.databricks.com".to_string(), + ); + let cfg = RuntimeFileConfig { + provider: Some("databricks".to_string()), + model: Some("goose-claude-4-5".to_string()), + extra, + ..Default::default() + }; + let env = empty_env(); + let result = goose_requirements(&env, Some(&cfg)); + // All requirements silenced — provider (file), model (file), DATABRICKS_HOST (file). + assert!( + result.is_empty(), + "flat DATABRICKS_HOST in file config must silence all requirements; \ + got: {:?}", + result + ); +} + +#[test] +fn goose_goose_provider_databricks_flat_host_silences_databricks_host() { + // GOOSE_PROVIDER=databricks (not active_provider) + flat DATABRICKS_HOST. + // The parser canonicalizes to extra["DATABRICKS_HOST"]; readiness must silence it. + let mut extra = BTreeMap::new(); + extra.insert( + "DATABRICKS_HOST".to_string(), + "https://dbc.example.com".to_string(), + ); + let cfg = RuntimeFileConfig { + provider: Some("databricks".to_string()), + model: Some("some-model".to_string()), + extra, + ..Default::default() + }; + let env = empty_env(); + let result = goose_requirements(&env, Some(&cfg)); + assert!( + !result.contains(&Requirement::EnvKey { + key: "DATABRICKS_HOST".to_string() + }), + "DATABRICKS_HOST must be silenced when canonical key is in file extra" + ); +} diff --git a/desktop/src-tauri/src/managed_agents/relay_mesh.rs b/desktop/src-tauri/src/managed_agents/relay_mesh.rs index 7a6f5b094a..5c246feedc 100644 --- a/desktop/src-tauri/src/managed_agents/relay_mesh.rs +++ b/desktop/src-tauri/src/managed_agents/relay_mesh.rs @@ -42,15 +42,66 @@ pub fn apply_relay_mesh_env( RELAY_MESH_PREFER_MESH_FOR_AUTO_ENV.to_string(), "1".to_string(), ); - // Keep the requested response inside smaller local-model context windows, - // and spend that budget on an answer/tool call instead of hidden reasoning. - // Without both settings Qwen3 either fails the router's fit check at the - // agent default (32K) or can consume a tight cap before serializing a tool. - env.insert( - "BUZZ_AGENT_MAX_OUTPUT_TOKENS".to_string(), - "4096".to_string(), - ); - env.insert("BUZZ_AGENT_THINKING_EFFORT".to_string(), "none".to_string()); + // Keep the requested response inside smaller local-model context windows. + // These are defaults, not policy: the effective agent/persona/global env + // may deliberately choose a smaller cap or a different effort. This function + // runs after those layers during readiness, so never clobber their values. + insert_default_if_unset(env, "BUZZ_AGENT_MAX_OUTPUT_TOKENS", "4096"); + // Mesh agents run on small local models, which are the ones most likely to + // do the work and then end the turn without publishing it — the failure the + // reply guard exists to catch. Everywhere else it stays opt-in and unset. + // A default, not policy: an explicit `0` from the agent/persona/global env + // survives (see `insert_default_if_unset`, and the copy-forward list in + // `relay_mesh_process_env` that preserves it through the spawn path). + insert_default_if_unset(env, "BUZZ_AGENT_REQUIRE_REPLY", "1"); + // Deliberately no BUZZ_AGENT_THINKING_EFFORT default: mesh translates + // `reasoning_effort` into the chat template's `enable_thinking` flag, so any + // value we pick overrides each model's own template default — and the right + // value is model-specific. Measured with the real prompt and toolset: + // gemma-4-E4B delivers 0/8 at `none` but 6/6 with the field absent, while + // Qwen3-8B delivers 8/8 either way and burns ~4x the output tokens once + // thinking is on (121 -> ~470), risking the 4096 cap. Omitting the field + // lets every model use its own default; explicit agent/persona/global + // values still apply. +} + +#[cfg(feature = "mesh-llm")] +fn insert_default_if_unset( + env: &mut std::collections::BTreeMap, + key: &str, + value: &str, +) { + if env.get(key).is_none_or(|current| current.trim().is_empty()) { + env.insert(key.to_string(), value.to_string()); + } +} + +/// Build the final Mesh-specific process overrides from the already-resolved +/// harness environment. Only user-owned generation controls are seeded: the +/// derived provider/base URL/model values remain authoritative, and unrelated +/// credentials (notably `OPENAI_API_KEY`) must not be copied back after the +/// spawn path removes them. +#[cfg(feature = "mesh-llm")] +pub fn relay_mesh_process_env( + effective_env: &std::collections::BTreeMap, + model: &str, +) -> std::collections::BTreeMap { + let mut env = std::collections::BTreeMap::new(); + for key in [ + "BUZZ_AGENT_MAX_OUTPUT_TOKENS", + "BUZZ_AGENT_THINKING_EFFORT", + // Must be copied forward for the user's value to survive: this map is + // written onto the command *after* the layered user env, so a key absent + // here is re-defaulted by `apply_relay_mesh_env` below and an explicit + // `BUZZ_AGENT_REQUIRE_REPLY=0` would be silently overridden back to `1`. + "BUZZ_AGENT_REQUIRE_REPLY", + ] { + if let Some(value) = effective_env.get(key) { + env.insert(key.to_string(), value.clone()); + } + } + apply_relay_mesh_env(&mut env, Some(RELAY_MESH_PROVIDER_ID), Some(model)); + env } #[cfg(all(test, feature = "mesh-llm"))] @@ -60,7 +111,7 @@ mod tests { use super::*; #[test] - fn native_provider_uses_context_safe_non_reasoning_budget() { + fn native_provider_uses_context_safe_tool_calling_budget() { let mut env = BTreeMap::new(); apply_relay_mesh_env( &mut env, @@ -72,14 +123,135 @@ mod tests { env.get("BUZZ_AGENT_MAX_OUTPUT_TOKENS").map(String::as_str), Some("4096") ); + // Must stay unset: any value we pick overrides the model's own chat + // template default, and the right value is model-specific ("none" + // stops gemma tool-calling; enabling thinking makes Qwen3 burn ~4x the + // output budget). + assert_eq!(env.get("BUZZ_AGENT_THINKING_EFFORT"), None); + assert_eq!( + env.get(RELAY_MESH_PREFER_MESH_FOR_AUTO_ENV) + .map(String::as_str), + Some("1") + ); + } + + #[test] + fn native_provider_preserves_explicit_generation_controls() { + let mut env = BTreeMap::from([ + ( + "BUZZ_AGENT_MAX_OUTPUT_TOKENS".to_string(), + "2048".to_string(), + ), + ("BUZZ_AGENT_THINKING_EFFORT".to_string(), "high".to_string()), + ]); + apply_relay_mesh_env( + &mut env, + Some(RELAY_MESH_PROVIDER_ID), + Some(RELAY_MESH_AUTO_MODEL_ID), + ); + + assert_eq!( + env.get("BUZZ_AGENT_MAX_OUTPUT_TOKENS").map(String::as_str), + Some("2048") + ); assert_eq!( env.get("BUZZ_AGENT_THINKING_EFFORT").map(String::as_str), - Some("none") + Some("high") + ); + } + + #[test] + fn native_provider_enables_reply_guard_by_default() { + let mut env = BTreeMap::new(); + apply_relay_mesh_env( + &mut env, + Some(RELAY_MESH_PROVIDER_ID), + Some(RELAY_MESH_AUTO_MODEL_ID), ); + assert_eq!( - env.get(RELAY_MESH_PREFER_MESH_FOR_AUTO_ENV) - .map(String::as_str), + env.get("BUZZ_AGENT_REQUIRE_REPLY").map(String::as_str), + Some("1"), + "mesh agents opt into the reply guard automatically" + ); + } + + #[test] + fn native_provider_preserves_explicit_reply_guard_opt_out() { + let mut env = BTreeMap::from([("BUZZ_AGENT_REQUIRE_REPLY".to_string(), "0".to_string())]); + apply_relay_mesh_env( + &mut env, + Some(RELAY_MESH_PROVIDER_ID), + Some(RELAY_MESH_AUTO_MODEL_ID), + ); + + assert_eq!( + env.get("BUZZ_AGENT_REQUIRE_REPLY").map(String::as_str), + Some("0"), + "an explicit opt-out is a user decision, not a value to re-default" + ); + } + + #[test] + fn non_mesh_provider_leaves_reply_guard_unset() { + let mut env = BTreeMap::new(); + apply_relay_mesh_env(&mut env, Some("anthropic"), Some("claude-haiku-4.5")); + + assert_eq!( + env.get("BUZZ_AGENT_REQUIRE_REPLY"), + None, + "the guard stays opt-in everywhere except mesh" + ); + assert!(env.is_empty(), "non-mesh providers get no mesh env at all"); + } + + /// The spawn path writes this map onto the command *after* the layered user + /// env, so an explicit opt-out only survives if it is copied forward. Without + /// the copy-forward, `apply_relay_mesh_env` re-defaults it to `1` here and + /// silently overrides the user at spawn while readiness still shows `0`. + #[test] + fn process_env_preserves_explicit_reply_guard_opt_out() { + let effective_env = + BTreeMap::from([("BUZZ_AGENT_REQUIRE_REPLY".to_string(), "0".to_string())]); + + let env = relay_mesh_process_env(&effective_env, "Gemma-4"); + + assert_eq!( + env.get("BUZZ_AGENT_REQUIRE_REPLY").map(String::as_str), + Some("0") + ); + } + + #[test] + fn process_env_enables_reply_guard_when_user_is_silent() { + let env = relay_mesh_process_env(&BTreeMap::new(), "Gemma-4"); + + assert_eq!( + env.get("BUZZ_AGENT_REQUIRE_REPLY").map(String::as_str), Some("1") ); } + + #[test] + fn process_env_seeds_controls_without_restoring_unrelated_credentials() { + let effective_env = BTreeMap::from([ + ( + "BUZZ_AGENT_MAX_OUTPUT_TOKENS".to_string(), + "1024".to_string(), + ), + ("OPENAI_API_KEY".to_string(), "must-not-leak".to_string()), + ]); + + let env = relay_mesh_process_env(&effective_env, "Gemma-4"); + + assert_eq!( + env.get("BUZZ_AGENT_MAX_OUTPUT_TOKENS").map(String::as_str), + Some("1024") + ); + assert_eq!( + env.get("OPENAI_COMPAT_MODEL").map(String::as_str), + Some("Gemma-4") + ); + assert!(!env.contains_key("OPENAI_API_KEY")); + } } diff --git a/desktop/src-tauri/src/managed_agents/runtime.rs b/desktop/src-tauri/src/managed_agents/runtime.rs index f3b4cb67fd..37927961ed 100644 --- a/desktop/src-tauri/src/managed_agents/runtime.rs +++ b/desktop/src-tauri/src/managed_agents/runtime.rs @@ -869,12 +869,7 @@ pub fn spawn_agent_child( // uses the same trim semantics as the preflight callers. #[cfg(feature = "mesh-llm")] if let Some(ref mesh_model_id) = mesh_model_id { - let mut mesh_env = std::collections::BTreeMap::new(); - super::apply_relay_mesh_env( - &mut mesh_env, - Some(super::RELAY_MESH_PROVIDER_ID), - Some(mesh_model_id.as_str()), - ); + let mesh_env = super::relay_mesh_process_env(&descriptor.env, mesh_model_id); command.env_remove("OPENAI_API_KEY"); for (key, value) in mesh_env { command.env(key, value); diff --git a/desktop/src-tauri/src/managed_agents/snapshot_avatar.rs b/desktop/src-tauri/src/managed_agents/snapshot_avatar.rs new file mode 100644 index 0000000000..a1044b31f4 --- /dev/null +++ b/desktop/src-tauri/src/managed_agents/snapshot_avatar.rs @@ -0,0 +1,42 @@ +use base64::{engine::general_purpose::STANDARD, Engine as _}; +use image::ImageDecoder; +use std::io::Cursor; + +const MAX_AVATAR_INLINE_BYTES: usize = 2 * 1024 * 1024; +const MAX_AVATAR_DIMENSION: u32 = 2048; +const MAX_AVATAR_DECODE_ALLOC: u64 = 32 * 1024 * 1024; + +/// Materialize a snapshot PNG's visible pixels as a bounded portable avatar. +/// The exact transparent 1×1 no-avatar placeholder and images that cannot fit +/// the persisted inline-avatar budget leave the manifest fallback intact. +pub(crate) fn snapshot_png_avatar_data_url(png_bytes: &[u8]) -> Result, String> { + let reader = image::ImageReader::with_format(Cursor::new(png_bytes), image::ImageFormat::Png); + let mut decoder = reader + .into_decoder() + .map_err(|e| format!("Failed to decode snapshot avatar: {e}"))?; + let mut limits = image::Limits::default(); + limits.max_image_width = Some(MAX_AVATAR_DIMENSION); + limits.max_image_height = Some(MAX_AVATAR_DIMENSION); + limits.max_alloc = Some(MAX_AVATAR_DECODE_ALLOC); + decoder + .set_limits(limits) + .map_err(|e| format!("Snapshot avatar exceeds safe decoding limits: {e}"))?; + let (width, height) = decoder.dimensions(); + let image = image::DynamicImage::from_decoder(decoder) + .map_err(|e| format!("Failed to decode snapshot avatar: {e}"))?; + if width == 1 && height == 1 && image.to_rgba8().get_pixel(0, 0).0 == [0, 0, 0, 0] { + return Ok(None); + } + + let mut clean_png = Vec::new(); + image + .write_to(&mut Cursor::new(&mut clean_png), image::ImageFormat::Png) + .map_err(|e| format!("Failed to encode snapshot avatar: {e}"))?; + if clean_png.len() > MAX_AVATAR_INLINE_BYTES { + return Ok(None); + } + Ok(Some(format!( + "data:image/png;base64,{}", + STANDARD.encode(clean_png) + ))) +} diff --git a/desktop/src-tauri/src/managed_agents/storage.rs b/desktop/src-tauri/src/managed_agents/storage.rs index f6f89ed898..652bb9b9ea 100644 --- a/desktop/src-tauri/src/managed_agents/storage.rs +++ b/desktop/src-tauri/src/managed_agents/storage.rs @@ -52,6 +52,34 @@ fn managed_agents_logs_dir(app: &AppHandle) -> Result { Ok(dir) } +/// Install-log path for `runtime_id`, alongside the agent logs. +pub fn install_log_path(app: &AppHandle, runtime_id: &str) -> Result { + Ok(managed_agents_logs_dir(app)?.join(install_log_filename(runtime_id)?)) +} + +/// Filename for a runtime's install log, or an error for an id that must not +/// become one. +/// +/// The id is validated rather than trusted: ids reach this from user-defined +/// custom harnesses as well as the catalog, and a `../` or a separator in one +/// would place the log outside the logs directory. Rejecting beats sanitizing — +/// a rejected id means no log, while a rewritten one could collide with another +/// runtime's. +fn install_log_filename(runtime_id: &str) -> Result { + if runtime_id.is_empty() || !runtime_id.chars().all(is_safe_id_char) { + return Err(format!( + "unsafe runtime id for a log filename: {runtime_id}" + )); + } + Ok(format!("install-{runtime_id}.log")) +} + +/// Characters allowed in a runtime id used as a filename. Excludes `/`, `\`, +/// `:` and `.`, so no id can traverse or escape the logs directory. +fn is_safe_id_char(c: char) -> bool { + c.is_ascii_alphanumeric() || c == '-' || c == '_' +} + pub fn managed_agent_log_path(app: &AppHandle, pubkey: &str) -> Result { Ok(managed_agents_logs_dir(app)?.join(format!("{pubkey}.log"))) } @@ -632,6 +660,62 @@ pub(crate) fn open_log_file(path: &Path) -> Result { .map_err(|error| format!("failed to open log file {}: {error}", path.display())) } +/// Start a new install-log session at `path`: keep the previous run as +/// `.1` and return a freshly created, empty current file. +/// +/// Rotating per *run* rather than by size is what bounds this file. A run +/// writes one record per executed attempt, each capped by the log-scale +/// capture, so one run's file is bounded by steps × attempts × cap and the +/// history on disk is bounded at two runs. Size-triggered rotation could not +/// promise either: it never replaced an existing `.1`, and on Windows — +/// where rename does not replace its destination — it stopped working +/// altogether once `.1` existed, leaving the current file to grow. +/// +/// The old `.1` is therefore *removed* before the rename rather than renamed +/// over. Every step is best-effort: a rotation that fails must not cost the +/// user the install, so the session continues with a truncated current file. +pub(crate) fn start_install_log_session(path: &Path) -> Result { + if path.exists() { + let mut previous = path.as_os_str().to_owned(); + previous.push(".1"); + let previous = PathBuf::from(previous); + let _ = fs::remove_file(&previous); + let _ = fs::rename(path, &previous); + } + open_install_log(path, /* truncate */ true) +} + +/// Open an install log for appending one more record to the current session. +pub(crate) fn open_install_log_file(path: &Path) -> Result { + open_install_log(path, /* truncate */ false) +} + +/// Open an install log owner-only. +/// +/// The mode is set *in the create* rather than chmod'd afterwards, so the file +/// is never briefly group/world-readable. Install output can carry registry +/// tokens and proxy credentials echoed by a failing installer, so the window +/// matters even though it is short. An existing file's mode is left as-is — +/// `OpenOptions::mode` only applies on creation, and silently re-tightening a +/// file the user relaxed is not this function's call to make. +fn open_install_log(path: &Path, truncate: bool) -> Result { + let mut options = OpenOptions::new(); + options.create(true); + if truncate { + options.write(true).truncate(true); + } else { + options.append(true); + } + #[cfg(unix)] + { + use std::os::unix::fs::OpenOptionsExt; + options.mode(0o600); + } + options + .open(path) + .map_err(|error| format!("failed to open log file {}: {error}", path.display())) +} + pub(crate) fn append_log_marker(path: &Path, message: &str) -> Result<(), String> { let mut file = open_log_file(path)?; writeln!(file, "{message}").map_err(|error| format!("failed to write log marker: {error}")) diff --git a/desktop/src-tauri/src/managed_agents/storage_tests.rs b/desktop/src-tauri/src/managed_agents/storage_tests.rs index 73567bb915..9943c6b3ac 100644 --- a/desktop/src-tauri/src/managed_agents/storage_tests.rs +++ b/desktop/src-tauri/src/managed_agents/storage_tests.rs @@ -698,3 +698,135 @@ fn try_delete_agent_key_returns_result() { // team_snapshot::tests::rollback_aggregates_multiple_errors. let _: fn(&str) -> Result<(), String> = super::try_delete_agent_key; } + +// ── install logs ───────────────────────────────────────────────────────────── + +/// Install output can carry registry tokens and proxy credentials a failing +/// installer echoed, and the file is written unattended. `0o600` must come from +/// the create itself: a post-write `chmod` leaves a window where the umask +/// decides, and a crash inside it leaves the log readable to other local users. +#[cfg(unix)] +#[test] +fn install_log_is_created_owner_only_without_post_write_chmod() { + use std::os::unix::fs::PermissionsExt; + + let dir = tempfile::tempdir().expect("temp dir"); + let path = dir.path().join("install-goose.log"); + + let mut file = super::open_install_log_file(&path).expect("open install log"); + file.write_all(b"npm ERR!\n").expect("write"); + + let mode = std::fs::metadata(&path) + .expect("metadata") + .permissions() + .mode() + & 0o777; + assert_eq!(mode, 0o600, "install logs must be owner-only"); +} + +/// A run starts a new current file and keeps the previous run as `.1`, so the +/// two runs are never mixed and the history on disk stays bounded at two. +#[test] +fn install_log_session_keeps_the_previous_run_as_dot_one() { + let dir = tempfile::tempdir().expect("temp dir"); + let path = dir.path().join("install-goose.log"); + + let mut first = super::start_install_log_session(&path).expect("first session"); + first.write_all(b"run-one\n").expect("write"); + let mut second = super::start_install_log_session(&path).expect("second session"); + second.write_all(b"run-two\n").expect("write"); + + assert_eq!( + std::fs::read_to_string(&path).expect("read current"), + "run-two\n", + "the current file must hold only the newest run" + ); + assert_eq!( + std::fs::read_to_string(dir.path().join("install-goose.log.1")).expect("read .1"), + "run-one\n", + "the previous run must be preserved as .1" + ); +} + +/// The third run must still rotate when `.1` already exists. Windows `rename` +/// does not replace its destination, so a rename-only rotation silently stops +/// working here and leaves the current file to grow across every later run — +/// the old `.1` is removed first precisely so this cannot happen. Runs on the +/// Windows target too: this is the path that fails there. +#[test] +fn install_log_session_replaces_an_existing_dot_one() { + let dir = tempfile::tempdir().expect("temp dir"); + let path = dir.path().join("install-goose.log"); + let rotated = dir.path().join("install-goose.log.1"); + // Seed the state a rename-only rotation cannot get out of: both files exist. + std::fs::write(&path, b"previous-run\n").expect("seed current"); + std::fs::write(&rotated, b"ancient-run\n").expect("seed .1"); + + let mut file = super::start_install_log_session(&path).expect("session"); + file.write_all(b"fresh-run\n").expect("write"); + + assert_eq!( + std::fs::read_to_string(&path).expect("read current"), + "fresh-run\n", + "the current file must restart even when .1 was already present" + ); + assert_eq!( + std::fs::read_to_string(&rotated).expect("read .1"), + "previous-run\n", + ".1 must be replaced by the run that just ended, not kept" + ); +} + +/// Records written after the session starts append to it — a run's later +/// records must not erase its earlier ones. +#[test] +fn install_log_appends_within_a_session() { + let dir = tempfile::tempdir().expect("temp dir"); + let path = dir.path().join("install-goose.log"); + + let mut session = super::start_install_log_session(&path).expect("session"); + session.write_all(b"header\n").expect("write"); + for record in ["first\n", "second\n"] { + let mut file = super::open_install_log_file(&path).expect("open install log"); + file.write_all(record.as_bytes()).expect("write"); + } + + assert_eq!( + std::fs::read_to_string(&path).expect("read back"), + "header\nfirst\nsecond\n" + ); +} + +/// A runtime id becomes part of a filename. Ids reach this from user-defined +/// custom harnesses as well as the catalog, so anything that could traverse or +/// escape the logs directory is rejected rather than sanitized — a rejected id +/// simply means no log, while a silently rewritten one could collide with +/// another runtime's log. +#[test] +fn install_log_filename_rejects_ids_that_would_escape_the_logs_dir() { + for id in [ + "../../etc/passwd", + "goose/../../evil", + "sub/dir", + "back\\slash", + "with.dot", + "", + ] { + assert!( + super::install_log_filename(id).is_err(), + "id {id:?} must not be accepted as a filename component" + ); + } +} + +/// Ordinary catalog and custom-harness ids are accepted — the guard must not +/// reject the ids it exists to serve. +#[test] +fn install_log_filename_accepts_ordinary_runtime_ids() { + for id in ["goose", "claude-code", "buzz_agent", "codex2"] { + assert_eq!( + super::install_log_filename(id).expect("id must be usable in a log filename"), + format!("install-{id}.log") + ); + } +} diff --git a/desktop/src-tauri/src/managed_agents/types.rs b/desktop/src-tauri/src/managed_agents/types.rs index 3d8e0ed02b..fcd8b13fc9 100644 --- a/desktop/src-tauri/src/managed_agents/types.rs +++ b/desktop/src-tauri/src/managed_agents/types.rs @@ -587,7 +587,8 @@ pub struct ManagedAgentLogResponse { pub enum AcpAvailabilityStatus { Available, AdapterMissing, - /// Adapter binary is present but is from the deprecated package (< 1.0). Reinstall required. + /// Adapter binary is present but unsupported — either the deprecated + /// package or a version below the supported floor. Reinstall required. AdapterOutdated, CliMissing, NotInstalled, @@ -701,6 +702,10 @@ pub struct InstallRuntimeResult { /// Number of agents whose stop succeeded but respawn failed. /// Mirrors `GlobalAgentConfigSaveResult.failed_restart_count`. pub failed_restart_count: u32, + /// Install log file for this run, when one was written. The UI surfaces it + /// on failure so a user can read the full retry history instead of only the + /// last step's truncated output. `None` when no log could be opened. + pub log_path: Option, } #[derive(Debug, Clone, Serialize)] diff --git a/desktop/src-tauri/src/mesh_llm/catalog.rs b/desktop/src-tauri/src/mesh_llm/catalog.rs index 385971cb86..1a11fcfcd1 100644 --- a/desktop/src-tauri/src/mesh_llm/catalog.rs +++ b/desktop/src-tauri/src/mesh_llm/catalog.rs @@ -19,12 +19,14 @@ use mesh_llm_system::vram::{format_rated_capacity, rated_capacity_gb}; /// The large pick is resolved through mesh-llm's remote catalog /// (huggingface.co/datasets/meshllm/catalog), so it does not need to exist in /// the compiled `MODEL_CATALOG`; the entry is synthesized below. -const CURATED_LARGE: &str = "gemma-4-26B-A4B-it-UD-Q4_K_M"; +const CURATED_LARGE: &str = "unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M"; +const CURATED_LARGE_ALIAS: &str = "gemma-4-26B-A4B-it-UD-Q4_K_M"; const CURATED_LARGE_SIZE: &str = "17GB"; const CURATED_LARGE_FILE: &str = "gemma-4-26B-A4B-it-UD-Q4_K_M.gguf"; const CURATED_LARGE_DESCRIPTION: &str = "Gemma 4 26B MoE (4B active) — Buzz default for 64GB+ machines"; -const CURATED_SMALL: &str = "Gemma-4-E4B-it-Q4_K_M"; +const CURATED_SMALL: &str = "unsloth/gemma-4-E4B-it-GGUF:Q4_K_M"; +const CURATED_SMALL_ALIAS: &str = "Gemma-4-E4B-it-Q4_K_M"; /// Rated-capacity boundary between the two curated tiers, in GB (marketing /// capacity — a "64GB" Mac rates as 64 even though usable AI memory is less). const CURATED_LARGE_MIN_RATED_GB: u64 = 64; @@ -37,6 +39,16 @@ fn buzz_recommended_model(rated_gb: Option) -> &'static str { } } +/// Convert Buzz's pre-0.74 curated package aliases into the canonical model +/// ids advertised and accepted by Mesh's OpenAI ingress. +pub(crate) fn canonical_curated_model_id(model_id: &str) -> &str { + match model_id.trim() { + CURATED_SMALL_ALIAS => CURATED_SMALL, + CURATED_LARGE_ALIAS => CURATED_LARGE, + other => other, + } +} + /// How a model sits inside this machine's usable AI memory. /// Mirrors mesh-llm's private `fit_code_for_size_label` thresholds. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)] @@ -146,12 +158,13 @@ fn build_catalog( .filter(|m| !is_draft_only(&m.name)) .map(|m| { let size_gb = parse_size_gb(&m.size); + let name = canonical_curated_model_id(&m.name).to_string(); MeshCatalogEntry { fit: fit_code(size_gb, vram_gb), - installed: is_installed(&m.file, &m.name), + installed: is_installed(&m.file, &name) || is_installed(&m.file, &m.name), recommended: false, curated: false, - name: m.name.clone(), + name, size: m.size.clone(), size_gb, description: m.description.clone(), @@ -166,7 +179,8 @@ fn build_catalog( let size_gb = parse_size_gb(CURATED_LARGE_SIZE); entries.push(MeshCatalogEntry { fit: fit_code(size_gb, vram_gb), - installed: is_installed(CURATED_LARGE_FILE, CURATED_LARGE), + installed: is_installed(CURATED_LARGE_FILE, CURATED_LARGE) + || is_installed(CURATED_LARGE_FILE, CURATED_LARGE_ALIAS), recommended: false, curated: false, name: CURATED_LARGE.to_string(), @@ -256,6 +270,8 @@ mod tests { #[test] fn recommendation_follows_buzz_curated_tiers() { + assert_eq!(CURATED_SMALL, "unsloth/gemma-4-E4B-it-GGUF:Q4_K_M"); + assert_eq!(CURATED_LARGE, "unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M"); // 64GB+ rated machines get the large curated pick. let large = build_catalog(None, 64_000_000_000, 64.0, &[]); assert_eq!(large.recommended.as_deref(), Some(CURATED_LARGE)); @@ -269,6 +285,22 @@ mod tests { assert_eq!(tiny.recommended.as_deref(), Some(CURATED_SMALL)); } + #[test] + fn curated_package_aliases_migrate_to_openai_model_ids() { + assert_eq!( + canonical_curated_model_id(CURATED_SMALL_ALIAS), + CURATED_SMALL + ); + assert_eq!( + canonical_curated_model_id(CURATED_LARGE_ALIAS), + CURATED_LARGE + ); + assert_eq!( + canonical_curated_model_id("other/model:Q4"), + "other/model:Q4" + ); + } + #[test] fn curated_picks_lead_the_catalog() { let catalog = build_catalog(None, 96_000_000_000, 96.0, &[]); diff --git a/desktop/src-tauri/src/mesh_llm/coordinator.rs b/desktop/src-tauri/src/mesh_llm/coordinator.rs index 1e279353b6..066fa46373 100644 --- a/desktop/src-tauri/src/mesh_llm/coordinator.rs +++ b/desktop/src-tauri/src/mesh_llm/coordinator.rs @@ -132,7 +132,7 @@ pub async fn start_coordinator(app: AppHandle) { /// MeshLLM establishes the encrypted peer transport itself. async fn reconcile_buzz_mesh_join(app: &AppHandle) -> Result<(), String> { let state = app.state::(); - let peer_ids = { + let (peer_ids, relay_url) = { let runtime = state.mesh_llm_runtime.lock().await; let Some(runtime) = runtime.as_ref() else { return Ok(()); @@ -141,10 +141,16 @@ async fn reconcile_buzz_mesh_join(app: &AppHandle) -> Result<(), String> { .status_report_payload() .await .map_err(|error| error.to_string())?; - visible_peer_ids(&payload) + let relay_url = runtime + .start_request() + .relay_url + .clone() + .unwrap_or_else(|| crate::relay::relay_ws_url_with_override(&state)); + (visible_peer_ids(&payload), relay_url) }; - let targets = crate::commands::mesh_llm::resolve_buzz_mesh_join_targets(&state).await?; + let targets = + crate::commands::mesh_llm::resolve_buzz_mesh_join_targets_at(&state, &relay_url).await?; let Some(target) = targets .into_iter() .find(|target| !target_is_visible(target, &peer_ids)) @@ -201,8 +207,13 @@ fn target_is_visible(target: &crate::mesh_llm::MeshServeTarget, peer_ids: &[Stri enum RosterReconcileAction { /// Keep the running allowlist untouched (no-op, or a failure we ride out). Keep, - /// Restart the node with a freshly resolved roster. - Restart(Vec), + /// Restart Buzz so MeshLLM is rebuilt with a freshly resolved roster. + /// + /// MeshLLM's native listeners are process-owned in practice: stopping and + /// starting the embedded runtime in one process can terminate Buzz or race + /// ports 9337/3131. The process boundary is therefore part of the safety + /// contract, not an implementation detail. + RestartProcess, /// Observed a *shrink* (or empty) once. Hold the current allowlist and /// require the same reduced roster on the next poll before tearing down, /// so a single transient short-read never drops a member mid-inference. @@ -224,9 +235,9 @@ fn roster_shrinks(current: &[String], fresh: &[String]) -> bool { /// Rules: /// - query failed (`Err`) → `Keep` (never de-admit on a relay blip) /// - resolved roster == current → `Keep` (no-op) -/// - grows (only additions) → `Restart` immediately (fast admission) +/// - grows (only additions) → `RestartProcess` immediately (fast admission) /// - shrinks/empties, first observation → `AwaitConfirm` (hold, re-check next poll) -/// - shrinks/empties, confirmed → `Restart` (same reduced roster twice) +/// - shrinks/empties, confirmed → `RestartProcess` (same reduced roster twice) fn roster_reconcile_action( current_owners: &[String], pending_shrink: Option<&[String]>, @@ -248,13 +259,13 @@ fn roster_reconcile_action( // Growth (pure additions) is safe to apply immediately. if !roster_shrinks(current_owners, &fresh) { - return RosterReconcileAction::Restart(fresh); + return RosterReconcileAction::RestartProcess; } // A shrink (including down to empty) must be confirmed across two // consecutive polls with the *same* reduced roster before we tear down. match pending_shrink { - Some(pending) if pending == fresh => RosterReconcileAction::Restart(fresh), + Some(pending) if pending == fresh => RosterReconcileAction::RestartProcess, _ => RosterReconcileAction::AwaitConfirm(fresh), } } @@ -283,8 +294,13 @@ async fn reconcile_roster( // other member on a transient relay blip (the flapping restart loop). Keep // the current allowlist and try again on the next poll. A shrink is held // for one extra poll (hysteresis) so a single short-read never tears down. - let query = crate::commands::mesh_llm::resolve_trusted_owner_ids(&state).await; - let fresh = match roster_reconcile_action(current_owners, pending_shrink.as_deref(), query) { + let relay_url = current_request + .relay_url + .as_deref() + .map(str::to_owned) + .unwrap_or_else(|| crate::relay::relay_ws_url_with_override(&state)); + let query = crate::commands::mesh_llm::resolve_trusted_owner_ids_at(&state, &relay_url).await; + match roster_reconcile_action(current_owners, pending_shrink.as_deref(), query) { RosterReconcileAction::Keep => { *pending_shrink = None; return Ok(()); @@ -294,34 +310,12 @@ async fn reconcile_roster( *pending_shrink = Some(reduced); return Ok(()); } - RosterReconcileAction::Restart(fresh) => { + RosterReconcileAction::RestartProcess => { *pending_shrink = None; - fresh } - }; + } - let mut request = current_request.clone(); - request.trusted_owner_ids = Some(fresh); - // Bootstrap endpoints are live device state, not configuration. The - // endpoint used at the previous start may belong to the member that just - // left or to a device whose iroh identity rotated while offline. Resolve a - // fresh validated peer for this restart; starting isolated is safe because - // the join watcher will converge it when a member next publishes. - request.join_token = match crate::commands::mesh_llm::resolve_buzz_mesh_join_targets(&state) - .await - { - Ok(targets) => targets - .into_iter() - .next() - .map(|target| target.endpoint_addr), - Err(error) => { - eprintln!( - "buzz-mesh: could not refresh bootstrap endpoint for roster restart; starting isolated: {error}" - ); - None - } - }; - let mut guard = state.mesh_llm_runtime.lock().await; + let guard = state.mesh_llm_runtime.lock().await; let startup_pending = match guard.as_ref() { Some(runtime) => runtime.is_starting().await, None => false, @@ -342,24 +336,14 @@ async fn reconcile_roster( // snapshot. return Ok(()); } - let Some(running) = guard.take() else { + if guard.is_none() { return Ok(()); - }; - eprintln!("buzz-mesh: membership roster changed; restarting mesh node with fresh allowlist"); - if let Err(error) = running.stop().await { - drop(guard); - eprintln!( - "buzz-mesh: stopping mesh node for roster restart failed; restarting Buzz instead of racing the occupied ingress: {error}" - ); - app.request_restart(); - return Err(format!( - "mesh node shutdown failed during roster change: {error}" - )); } - let replacement = crate::mesh_llm::DesktopMeshRuntime::start(request) - .await - .map_err(|error| format!("mesh node restart after roster change failed: {error:#}"))?; - *guard = Some(replacement); + drop(guard); + eprintln!( + "buzz-mesh: membership roster changed; restarting Buzz to rebuild MeshLLM with the fresh community allowlist" + ); + app.request_restart(); Ok(()) } @@ -377,11 +361,15 @@ pub(crate) async fn publish_current_status_once(app: &AppHandle, reason: &str) { } } -pub(crate) async fn publish_stopped_status_once(app: &AppHandle, reason: &str) { +pub(crate) async fn publish_stopped_status_once_at( + app: &AppHandle, + relay_url: Option<&str>, + reason: &str, +) { let state = app.state::(); match tokio::time::timeout( STATUS_PUBLISH_TIMEOUT, - publish_stopped_status_for_state(&state), + publish_stopped_status_for_state(&state, relay_url), ) .await { @@ -396,26 +384,43 @@ pub(crate) async fn publish_stopped_status_once(app: &AppHandle, reason: &str) { async fn publish_current_status_for_state(state: &AppState) -> Result<(), String> { let identity = super::ensure_owner_identity() .map_err(|error| format!("failed to load mesh owner identity: {error}"))?; - let mut payload = { + let (mut payload, relay_url) = { let runtime = state.mesh_llm_runtime.lock().await; match runtime.as_ref() { - Some(runtime) => runtime - .status_report_payload() - .await - .map_err(|error| error.to_string())?, - None => stopped_status_payload(&identity), + Some(runtime) => { + let payload = runtime + .status_report_payload() + .await + .map_err(|error| error.to_string())?; + let relay_url = runtime + .start_request() + .relay_url + .clone() + .unwrap_or_else(|| crate::relay::relay_ws_url_with_override(state)); + (payload, relay_url) + } + None => ( + stopped_status_payload(&identity), + crate::relay::relay_ws_url_with_override(state), + ), } }; bind_payload_to_member(state, &identity, &mut payload)?; - publish_status_report(state, payload).await + publish_status_report_at(state, &relay_url, payload).await } -async fn publish_stopped_status_for_state(state: &AppState) -> Result<(), String> { +async fn publish_stopped_status_for_state( + state: &AppState, + relay_url: Option<&str>, +) -> Result<(), String> { let identity = super::ensure_owner_identity() .map_err(|error| format!("failed to load mesh owner identity: {error}"))?; let mut payload = stopped_status_payload(&identity); bind_payload_to_member(state, &identity, &mut payload)?; - publish_status_report(state, payload).await + let relay_url = relay_url + .map(str::to_owned) + .unwrap_or_else(|| crate::relay::relay_ws_url_with_override(state)); + publish_status_report_at(state, &relay_url, payload).await } fn stopped_status_payload(identity: &super::identity::OwnerIdentity) -> serde_json::Value { @@ -469,13 +474,21 @@ pub(crate) fn build_status_report_event( .tags([d, k])) } -pub(crate) async fn publish_status_report( +async fn publish_status_report_at( state: &AppState, + relay_url: &str, payload: serde_json::Value, ) -> Result<(), String> { - crate::relay::submit_event(build_status_report_event(payload)?, state) - .await - .map(|_| ()) + let api_base_url = crate::relay::relay_http_base_url(relay_url); + let keys = state.signing_keys()?; + crate::relay::submit_event_at_with_keys( + build_status_report_event(payload)?, + state, + &api_base_url, + &keys, + ) + .await + .map(|_| ()) } #[cfg(test)] @@ -554,11 +567,11 @@ mod tests { // Growth (pure additions) applies immediately — fast admission is fine. #[test] - fn roster_growth_restarts_immediately() { + fn roster_growth_requests_process_restart_immediately() { let current = vec!["owner-a".to_string()]; let fresh = vec!["owner-a".to_string(), "owner-c".to_string()]; - let action = roster_reconcile_action(¤t, None, Ok(fresh.clone())); - assert_eq!(action, RosterReconcileAction::Restart(fresh)); + let action = roster_reconcile_action(¤t, None, Ok(fresh)); + assert_eq!(action, RosterReconcileAction::RestartProcess); } // A shrink is NOT applied on first observation — it must be confirmed. @@ -572,11 +585,11 @@ mod tests { // The same reduced roster on two consecutive polls confirms the shrink. #[test] - fn roster_shrink_restarts_once_confirmed() { + fn roster_shrink_requests_process_restart_once_confirmed() { let current = vec!["owner-a".to_string(), "owner-b".to_string()]; let reduced = vec!["owner-a".to_string()]; let action = roster_reconcile_action(¤t, Some(&reduced), Ok(reduced.clone())); - assert_eq!(action, RosterReconcileAction::Restart(reduced)); + assert_eq!(action, RosterReconcileAction::RestartProcess); } // A shrink that changes between polls is not confirmed — it re-holds with @@ -600,7 +613,7 @@ mod tests { assert_eq!(first, RosterReconcileAction::AwaitConfirm(Vec::new())); let empty: Vec = Vec::new(); let confirmed = roster_reconcile_action(¤t, Some(&empty), Ok(Vec::new())); - assert_eq!(confirmed, RosterReconcileAction::Restart(Vec::new())); + assert_eq!(confirmed, RosterReconcileAction::RestartProcess); } // A shrink followed by recovery to the full roster cancels the teardown. diff --git a/desktop/src-tauri/src/mesh_llm/mod.rs b/desktop/src-tauri/src/mesh_llm/mod.rs index 6e3ab4b28b..e206c53886 100644 --- a/desktop/src-tauri/src/mesh_llm/mod.rs +++ b/desktop/src-tauri/src/mesh_llm/mod.rs @@ -1,7 +1,7 @@ use std::collections::BTreeMap; mod coordinator; -pub(crate) use coordinator::{publish_current_status_once, publish_stopped_status_once}; +pub(crate) use coordinator::{publish_current_status_once, publish_stopped_status_once_at}; pub use coordinator::{start_coordinator, MeshCoordinator, KIND_BUZZ_MESH_MEMBER_STATUS}; mod discovery; @@ -14,6 +14,7 @@ pub(crate) use discovery::{ use discovery::{device_name_from_status, endpoint_id_from_status, enrich_status_payload_identity}; mod catalog; +pub(crate) use catalog::canonical_curated_model_id; pub use catalog::{model_catalog, MeshModelCatalog}; mod identity; @@ -200,6 +201,11 @@ pub struct StartMeshNodeRequest { /// accepted from the frontend and contains no relay address. #[serde(default, skip_deserializing)] pub mesh_name: Option, + /// Relay this runtime's community membership and discovery are bound to. + /// Injected by the backend when sharing starts and retained across UI + /// workspace switches; moving a share requires an explicit stop/start. + #[serde(default, skip_deserializing)] + pub relay_url: Option, /// Mesh owner ids admitted to this node (the member roster from /// member-signed discovery notes). `None` = caller did not resolve a roster /// (tests, direct invocations): the node runs without allowlist @@ -308,17 +314,20 @@ pub const MESH_WORKER_STACK_SIZE: usize = 8 * 1024 * 1024; /// before the node starts. Without this the download happens *inside* /// `serve::start()` where the UI can only show a frozen "starting…" state. /// Already-installed models return immediately from the cache scan. -async fn ensure_model_downloaded(model: &str) -> anyhow::Result<()> { - let model_owned = model.to_string(); - let installed = tokio::task::spawn_blocking(move || { +async fn model_is_installed(model: &str) -> bool { + let model_owned = model.replace("@main", ""); + tokio::task::spawn_blocking(move || { let cache = mesh_llm_node::models::default_huggingface_cache_dir(); mesh_llm_node::models::scan_installed_models(cache) .iter() - .any(|m| m.model_ref.contains(&model_owned)) + .any(|m| m.model_ref.replace("@main", "").contains(&model_owned)) }) .await - .unwrap_or(false); - if installed { + .unwrap_or(false) +} + +async fn ensure_model_downloaded(model: &str) -> anyhow::Result<()> { + if model_is_installed(model).await { return Ok(()); } mesh_llm_host_runtime::models::download_model_ref_with_progress_details(model, true) diff --git a/desktop/src-tauri/src/mesh_llm/mod_tests.rs b/desktop/src-tauri/src/mesh_llm/mod_tests.rs index 0b726c264f..557cd040fa 100644 --- a/desktop/src-tauri/src/mesh_llm/mod_tests.rs +++ b/desktop/src-tauri/src/mesh_llm/mod_tests.rs @@ -12,6 +12,7 @@ fn pending_client_runtime( max_vram_gb: None, join_token: Some("initial-token".to_string()), mesh_name: None, + relay_url: None, trusted_owner_ids: None, }; super::DesktopMeshRuntime { diff --git a/desktop/src-tauri/src/mesh_llm/recovery.rs b/desktop/src-tauri/src/mesh_llm/recovery.rs index 809fab8993..7933fd291e 100644 --- a/desktop/src-tauri/src/mesh_llm/recovery.rs +++ b/desktop/src-tauri/src/mesh_llm/recovery.rs @@ -149,34 +149,60 @@ fn should_evict_after_probe( probe: MeshIngressProbe, consecutive: u32, ) -> bool { - urgency == MeshRecoveryUrgency::Foreground && probe == MeshIngressProbe::PortClosed - || consecutive >= DEAD_PROBE_EVICT_THRESHOLD + // Only a CLOSED port is evidence of death. A bound-but-HTTP-unresponsive + // port ("Unhealthy") is a BUSY node, not a dead one: mesh serializes all + // HTTP on the ingress — including the `/v1/models` liveness probe — behind + // in-flight inference, so a large-prompt turn on a big model leaves the + // control plane unresponsive for the whole turn (measured ~27s on a + // gemma-4-26B node) while TCP-connect keeps answering in ~0ms. Model load + // and package-layer download are unresponsive in exactly the same way. + // Evicting on any of those turns ordinary backpressure into a destructive + // whole-app restart loop, which is the regression this fixes. A genuinely + // wedged bound port cannot be distinguished from a busy one without a + // lock-free health endpoint on the ingress (tracked upstream in mesh-llm); + // until that exists we never evict a bound port and rely solely on the + // unambiguous closed-port signal. + match probe { + MeshIngressProbe::Live | MeshIngressProbe::Unhealthy => false, + MeshIngressProbe::PortClosed => { + urgency == MeshRecoveryUrgency::Foreground || consecutive >= DEAD_PROBE_EVICT_THRESHOLD + } + } +} + +fn requires_process_restart( + mode: crate::mesh_llm::MeshNodeMode, + startup_in_progress: bool, +) -> bool { + startup_in_progress || mode == crate::mesh_llm::MeshNodeMode::Serve } -/// Probe and, when justified, remove one stale runtime. A closed port is -/// decisive for a foreground agent start; watchdog and ambiguous/unhealthy -/// ports require consecutive failures to avoid restarting on a transient load -/// spike. +/// Probe and, when justified, remove one stale runtime. Only a CLOSED port is +/// treated as death: a foreground agent start evicts immediately, the watchdog +/// after a short consecutive-failure streak. A bound-but-unresponsive +/// ("Unhealthy") port is never evicted — it is a busy or still-loading node, +/// not a dead one (see `should_evict_after_probe`). pub(crate) async fn recover_stale_mesh_runtime( state: &AppState, urgency: MeshRecoveryUrgency, ) -> MeshRuntimeRecovery { - let (candidate_id, startup_in_progress) = match state.mesh_llm_runtime.lock().await.as_ref() { - Some(runtime) => (runtime.id(), runtime.is_starting().await), - None => { - state.mesh_recovery.reset_probe_streak(); - // A cancelled SDK startup can outlive its Buzz-side task briefly - // because the embedded runtime runs on its own thread. Never start - // a replacement merely because the tracked handle is gone: first - // prove the old ingress is either still useful or has released the - // port. This closes the port-conflict loop in #2304. - return match probe_mesh_ingress().await { - MeshIngressProbe::Live => MeshRuntimeRecovery::Live, - MeshIngressProbe::PortClosed => MeshRuntimeRecovery::Absent, - MeshIngressProbe::Unhealthy => MeshRuntimeRecovery::ReleasePending, - }; - } - }; + let (candidate_id, startup_in_progress, candidate_mode) = + match state.mesh_llm_runtime.lock().await.as_ref() { + Some(runtime) => (runtime.id(), runtime.is_starting().await, runtime.mode()), + None => { + state.mesh_recovery.reset_probe_streak(); + // A cancelled SDK startup can outlive its Buzz-side task briefly + // because the embedded runtime runs on its own thread. Never start + // a replacement merely because the tracked handle is gone: first + // prove the old ingress is either still useful or has released the + // port. This closes the port-conflict loop in #2304. + return match probe_mesh_ingress().await { + MeshIngressProbe::Live => MeshRuntimeRecovery::Live, + MeshIngressProbe::PortClosed => MeshRuntimeRecovery::Absent, + MeshIngressProbe::Unhealthy => MeshRuntimeRecovery::ReleasePending, + }; + } + }; let probe = probe_mesh_ingress().await; if probe == MeshIngressProbe::Live { state.mesh_recovery.reset_probe_streak(); @@ -196,12 +222,13 @@ pub(crate) async fn recover_stale_mesh_runtime( return MeshRuntimeRecovery::Debouncing; } - // The pinned SDK does not yield its control handle until the management - // API is ready. Dropping its still-pending start future would detach the - // embedded runtime thread without sending a shutdown request, so Buzz must - // not evict it and race a replacement onto the same ports. A controlled - // app relaunch is the only process-owned cleanup boundary in this state. - if startup_in_progress { + // Never replace a serving runtime in-process. Its native listeners and + // model host are process-owned; stopping it here and then cold-starting a + // client silently disables Share Compute and can race ports 9337/3131. + // Pending client startups have the same ownership problem because the SDK + // has not yielded a shutdown handle yet. In both cases, process restart is + // the only boundary that preserves the configured role safely. + if requires_process_restart(candidate_mode, startup_in_progress) { state.mesh_recovery.reset_probe_streak(); return MeshRuntimeRecovery::RestartRequired; } @@ -241,6 +268,12 @@ pub(crate) async fn recover_stale_mesh_runtime( pub(crate) async fn rearm_relay_mesh_for_running_agents(app: &AppHandle) -> Result<(), String> { let state = app.state::(); let _rearm_guard = state.mesh_recovery.rearm_lock.lock().await; + let runtime_mode = state + .mesh_llm_runtime + .lock() + .await + .as_ref() + .map(|runtime| runtime.mode()); let recovery = recover_stale_mesh_runtime(&state, MeshRecoveryUrgency::Watchdog).await; let active_pubkeys = active_managed_agent_pubkeys(&state); // Mesh participation is resolved through the same definition-authoritative @@ -254,6 +287,13 @@ pub(crate) async fn rearm_relay_mesh_for_running_agents(app: &AppHandle) -> Resu | MeshRuntimeRecovery::Debouncing | MeshRuntimeRecovery::Replaced => return Ok(()), MeshRuntimeRecovery::RestartRequired => { + if runtime_mode == Some(crate::mesh_llm::MeshNodeMode::Serve) { + eprintln!( + "buzz-mesh: serving ingress failed; restarting Buzz to restore Share Compute without changing roles" + ); + app.request_restart(); + return Ok(()); + } let records = crate::managed_agents::load_managed_agents(app).unwrap_or_default(); if !records.iter().any(|record| { running_relay_mesh_model_id(record, &active_pubkeys, &personas, &global).is_some() @@ -410,6 +450,7 @@ mod tests { name_pool: Vec::new(), is_builtin: false, is_active: true, + shared: false, source_team: None, source_team_persona_slug: None, catalog_source: None, @@ -467,6 +508,89 @@ mod tests { )); } + #[test] + fn watchdog_closed_port_still_evicts_after_consecutive_streak() { + // A genuinely dead listener (crashed / released its port) must still be + // reclaimed — the closed-port signal is unchanged by this fix. + assert!(!should_evict_after_probe( + MeshRecoveryUrgency::Watchdog, + MeshIngressProbe::PortClosed, + 1 + )); + assert!(should_evict_after_probe( + MeshRecoveryUrgency::Watchdog, + MeshIngressProbe::PortClosed, + DEAD_PROBE_EVICT_THRESHOLD + )); + } + + #[test] + fn busy_or_loading_bound_port_is_never_evicted() { + // The regression this fixes: mesh serializes all ingress HTTP (incl. + // the `/v1/models` liveness probe) behind in-flight inference, so a + // large-prompt turn, a model load, or a layer download leaves the port + // bound-but-unresponsive ("Unhealthy"). No probe streak, and no + // urgency, may evict such a node — doing so restarts a node that is + // alive and working. + for consecutive in [1, 2, 5, 100] { + for urgency in [ + MeshRecoveryUrgency::Watchdog, + MeshRecoveryUrgency::Foreground, + ] { + assert!( + !should_evict_after_probe(urgency, MeshIngressProbe::Unhealthy, consecutive), + "a bound-but-busy port must never evict (urgency={urgency:?}, \ + consecutive={consecutive})" + ); + } + } + } + + #[test] + fn long_model_load_never_reaches_the_restart_path() { + // Pins the exact false positive this fix removes. A big model stays + // bound-but-unresponsive for MINUTES while it loads weights and + // downloads package layers, so the watchdog sees an unbroken run of + // `Unhealthy` probes. Walk ~5 minutes of watchdog passes at its 15s + // base interval and assert the eviction gate stays shut the whole way + // — for a serve node, one `true` here is a whole-app restart. + let state = MeshRecoveryState::default(); + let runtime_id = 42; + let passes = (5 * 60) / 15; + + for pass in 1..=passes { + let consecutive = state.record_dead_probe(runtime_id); + // The streak really does climb — the non-eviction below is the + // rule refusing to act, not the counter quietly resetting. + assert_eq!( + consecutive, pass, + "probe streak should keep climbing across a long load" + ); + for urgency in [ + MeshRecoveryUrgency::Watchdog, + MeshRecoveryUrgency::Foreground, + ] { + assert!( + !should_evict_after_probe(urgency, MeshIngressProbe::Unhealthy, consecutive), + "a still-loading node must never be evicted \ + (urgency={urgency:?}, minute={}, streak={consecutive})", + pass * 15 / 60 + ); + } + } + + // Sanity: the streak blew far past the threshold that used to evict, + // so the old logic WOULD have restarted this healthy loading node. + assert!( + passes >= DEAD_PROBE_EVICT_THRESHOLD, + "test must exceed the old eviction threshold to be meaningful" + ); + + // Once the load finishes and the ingress answers, the streak clears. + state.reset_probe_streak(); + assert_eq!(state.record_dead_probe(runtime_id), 1); + } + #[test] fn probe_streak_is_scoped_to_runtime_identity() { let state = MeshRecoveryState::default(); @@ -481,6 +605,22 @@ mod tests { assert!(STALE_STOP_TIMEOUT <= Duration::from_secs(15)); } + #[test] + fn failed_serving_runtime_requires_process_restart_instead_of_client_fallback() { + assert!(requires_process_restart( + crate::mesh_llm::MeshNodeMode::Serve, + false + )); + assert!(requires_process_restart( + crate::mesh_llm::MeshNodeMode::Client, + true + )); + assert!(!requires_process_restart( + crate::mesh_llm::MeshNodeMode::Client, + false + )); + } + #[test] fn only_running_relay_mesh_agents_trigger_rearm() { let personas: Vec = Vec::new(); @@ -531,4 +671,39 @@ mod tests { ); assert!(!"user note: shared compute config".starts_with(MESH_REARM_ERROR_SENTINEL)); } + + // Black-box proof of the classification the eviction rule stands on. A + // mesh node busy in inference (or loading, or downloading) keeps its + // ingress TCP port accepting connections in ~0ms while HTTP does not answer + // within the probe timeout — measured directly against a gemma-4-26B node: + // a concurrent `/v1/models` took ~27s, queued behind one in-flight turn. + // This stands up exactly that shape — a listener that accepts then never + // replies — and asserts the probe reads `Unhealthy` (busy), the verdict + // `should_evict_after_probe` now refuses to evict on. If this regressed to + // `PortClosed`, a busy node would again be misread as dead and restarted. + #[tokio::test] + async fn bound_but_stalled_http_classifies_as_unhealthy_not_closed() { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0") + .await + .expect("bind ephemeral port"); + let port = listener.local_addr().unwrap().port(); + // Accept and hold connections open without ever writing a response — + // the wire-level equivalent of a node serializing HTTP behind a turn. + let accept_task = tokio::spawn(async move { + let mut held = Vec::new(); + while let Ok((stream, _)) = listener.accept().await { + held.push(stream); // keep the socket open, never respond + } + }); + + let probe = probe_mesh_ingress_at(&format!("http://127.0.0.1:{port}/v1")).await; + accept_task.abort(); + + assert_eq!( + probe, + MeshIngressProbe::Unhealthy, + "a TCP-bound port that stalls HTTP (a busy/loading node) must read \ + Unhealthy, never PortClosed — the eviction fix depends on this" + ); + } } diff --git a/desktop/src-tauri/src/models.rs b/desktop/src-tauri/src/models.rs index 1d9747bc20..3f04d3d7a1 100644 --- a/desktop/src-tauri/src/models.rs +++ b/desktop/src-tauri/src/models.rs @@ -6,6 +6,8 @@ use serde::{Deserialize, Deserializer, Serialize}; pub struct IdentityInfo { pub pubkey: String, pub display_name: String, + /// Durable location of the active identity key. + pub storage: String, /// True when the app booted with an ephemeral key because the OS keyring /// was empty despite a prior successful migration (key was externally /// deleted). The frontend routes to the nsec re-import step when true. diff --git a/desktop/src-tauri/src/mouse_nav.rs b/desktop/src-tauri/src/mouse_nav.rs new file mode 100644 index 0000000000..cd729f7304 --- /dev/null +++ b/desktop/src-tauri/src/mouse_nav.rs @@ -0,0 +1,142 @@ +//! Native macOS handler for back/forward navigation inputs (mouse X1/X2 +//! buttons and horizontal swipe gestures). +//! +//! WKWebView never delivers these inputs to the web content layer, so a DOM +//! listener can't see them (Safari itself handles them natively in the app +//! layer, not in the page). This module installs an NSEvent local monitor +//! and emits a `mouse-nav` Tauri event that `useBackForwardControls` acts on +//! in the frontend. Two event shapes map to navigation: +//! +//! - `otherMouseUp` with button 3/4 — mice whose X1/X2 buttons reach the app +//! as plain mouse buttons. +//! - `swipe` with a horizontal delta — AppKit's page-swipe gesture +//! (`swipeWithEvent:`): `deltaX > 0` is back, `deltaX < 0` is forward. +//! Sent by mouse drivers that synthesize a page-swipe gesture for the +//! back/forward buttons instead of button-3/4 events (the hardware this +//! was verified on). Stock Apple trackpad and Magic Mouse swipes arrive +//! as phased scroll-wheel events instead, which this module does not +//! handle — that path (`ScrollWheel` + `trackSwipeEventWithOptions:`, +//! which also needs scroll-edge detection) is a follow-up. +//! +//! Compiled macOS-only (via `tray_menu`). Non-macOS X1/X2 behavior is left +//! to the underlying webview. + +/// Maps an `otherMouseUp` button number to a navigation direction. +/// Buttons 3 and 4 are X1 (back) and X2 (forward). +fn direction_for_button(button: isize) -> Option<&'static str> { + match button { + 3 => Some("back"), + 4 => Some("forward"), + _ => None, + } +} + +/// Maps a swipe gesture's horizontal delta to a navigation direction, +/// following the AppKit `swipeWithEvent:` convention: positive is back, +/// negative is forward. A swipe arrives as a begin/end pair and only the +/// end event carries the direction, so `deltaX == 0` maps to `None`. +fn direction_for_swipe(delta_x: f64) -> Option<&'static str> { + if delta_x > 0.0 { + Some("back") + } else if delta_x < 0.0 { + Some("forward") + } else { + None + } +} + +pub fn init(app_handle: &tauri::AppHandle) { + use block2::RcBlock; + use objc2_app_kit::{NSEvent, NSEventMask, NSEventType}; + use tauri::Emitter; + + let app = app_handle.clone(); + let block = RcBlock::new(move |event: std::ptr::NonNull| -> *mut NSEvent { + // SAFETY: the monitor hands us a valid NSEvent for the matched mask. + let ev = unsafe { event.as_ref() }; + + match ev.r#type() { + NSEventType::OtherMouseUp => { + if let Some(direction) = direction_for_button(ev.buttonNumber()) { + // Emit to the main window explicitly instead of + // broadcasting (`emit`) so navigation stays scoped if + // multi-window ever lands. "main" is the default label + // for the single configured window (see deep_link.rs). + let _ = app.emit_to("main", "mouse-nav", direction); + // Swallow the release: nothing downstream should also act + // on it. The matching press deliberately passes through: + // WKWebView never delivers X1/X2 to the page, so the + // unmatched down is inert, and swallowing presses risks + // interfering with AppKit behaviors keyed off mouse-down. + return std::ptr::null_mut(); + } + } + NSEventType::Swipe => { + if let Some(direction) = direction_for_swipe(ev.deltaX()) { + let _ = app.emit_to("main", "mouse-nav", direction); + } + // Pass swipes through: nothing else navigates on them, and + // swallowing mid-gesture events could confuse AppKit's + // gesture tracking. + } + _ => {} + } + + event.as_ptr() + }); + + // SAFETY: the block returns either null or the pointer it was given, both + // valid per the monitor contract. The returned monitor token is + // deliberately leaked: the monitor must live for the whole app lifetime. + let monitor = unsafe { + NSEvent::addLocalMonitorForEventsMatchingMask_handler( + NSEventMask::OtherMouseUp | NSEventMask::Swipe, + &block, + ) + }; + + if let Some(monitor) = monitor { + std::mem::forget(monitor); + } else { + eprintln!("buzz-desktop: mouse-nav: failed to install NSEvent monitor"); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn button_3_is_back() { + assert_eq!(direction_for_button(3), Some("back")); + } + + #[test] + fn button_4_is_forward() { + assert_eq!(direction_for_button(4), Some("forward")); + } + + #[test] + fn other_buttons_do_not_navigate() { + for button in [0, 1, 2, 5, -1] { + assert_eq!(direction_for_button(button), None); + } + } + + #[test] + fn positive_swipe_delta_is_back() { + assert_eq!(direction_for_swipe(1.0), Some("back")); + assert_eq!(direction_for_swipe(0.5), Some("back")); + } + + #[test] + fn negative_swipe_delta_is_forward() { + assert_eq!(direction_for_swipe(-1.0), Some("forward")); + assert_eq!(direction_for_swipe(-0.5), Some("forward")); + } + + #[test] + fn zero_delta_swipe_begin_event_is_ignored() { + assert_eq!(direction_for_swipe(0.0), None); + } +} diff --git a/desktop/src-tauri/src/native_websocket.rs b/desktop/src-tauri/src/native_websocket.rs index c0cf2e76f1..128f2df79d 100644 --- a/desktop/src-tauri/src/native_websocket.rs +++ b/desktop/src-tauri/src/native_websocket.rs @@ -24,7 +24,7 @@ type Id = u32; #[derive(Debug, Deserialize)] #[serde(tag = "type", content = "data")] -enum WebSocketMessage { +pub(crate) enum WebSocketMessage { Text(String), Binary(Vec), Ping(Vec), @@ -33,7 +33,7 @@ enum WebSocketMessage { } #[derive(Debug, Deserialize)] -struct CloseFramePayload { +pub(crate) struct CloseFramePayload { code: u16, reason: String, } @@ -82,7 +82,7 @@ struct ConnectionHandle { } #[derive(Clone)] -struct WebSocketManager { +pub(crate) struct WebSocketManager { connections: Arc>>>, connect_cancel: Arc>, } @@ -182,11 +182,23 @@ async fn connect( open_connection(manager.inner(), &url, on_message).await } -async fn send_message( +pub(crate) async fn send_message( manager: &WebSocketManager, id: Id, message: WebSocketMessage, ) -> Result<(), String> { + // Egress guard: the NIP-49 local key backup must never reach a relay. + // This is the single choke point for all webview-originated websocket + // frames (see `crate::egress_guard`). + match &message { + WebSocketMessage::Text(text) => { + crate::egress_guard::assert_no_key_backup(text, "websocket text frame")? + } + WebSocketMessage::Binary(bytes) => { + crate::egress_guard::assert_no_key_backup_bytes(bytes, "websocket binary frame")? + } + _ => {} + } let handle = manager .connections .lock() diff --git a/desktop/src-tauri/src/relay.rs b/desktop/src-tauri/src/relay.rs index f896695624..71aa21c413 100644 --- a/desktop/src-tauri/src/relay.rs +++ b/desktop/src-tauri/src/relay.rs @@ -450,6 +450,7 @@ pub async fn sync_managed_agent_profile( let event = build_profile_event(agent_keys, display_name, avatar_url, auth_tag)?; let event_json = event.as_json(); let body_bytes = event_json.into_bytes(); + crate::egress_guard::assert_no_key_backup_bytes(&body_bytes, "agent profile sync")?; let url = format!("{}/events", relay_http_base_url(relay_url)); let auth = build_nip98_auth_header_for_keys(agent_keys, &Method::POST, &url, &body_bytes)?; @@ -566,6 +567,7 @@ pub async fn submit_signed_event_with_keys( crate::relay_admission::wait_for_rate_limit().await; let url = format!("{}/events", relay_api_base_url_with_override(state)); let body_bytes = event.as_json().into_bytes(); + crate::egress_guard::assert_no_key_backup_bytes(&body_bytes, "signed event submit (keys)")?; let auth_header = build_nip98_auth_header_for_keys(keys, &Method::POST, &url, &body_bytes)?; let mut request = state diff --git a/desktop/src-tauri/src/relay/submit.rs b/desktop/src-tauri/src/relay/submit.rs index 2a42d86c2b..eaad29d3b1 100644 --- a/desktop/src-tauri/src/relay/submit.rs +++ b/desktop/src-tauri/src/relay/submit.rs @@ -25,6 +25,7 @@ pub async fn submit_signed_event_at_with_keys( crate::relay_admission::wait_for_rate_limit().await; let url = format!("{}/events", api_base_url.trim_end_matches('/')); let body_bytes = event.as_json().into_bytes(); + crate::egress_guard::assert_no_key_backup_bytes(&body_bytes, "relay event submit")?; let auth_header = build_nip98_auth_header_for_keys(keys, &Method::POST, &url, &body_bytes)?; let response = state diff --git a/desktop/src-tauri/src/reset.rs b/desktop/src-tauri/src/reset.rs index d2e35e6839..18ddd80eb8 100644 --- a/desktop/src-tauri/src/reset.rs +++ b/desktop/src-tauri/src/reset.rs @@ -463,6 +463,26 @@ mod tests { assert_eq!(kc.delete_calls.get(), 1, "keychain deleted once"); } + // ── NIP-49: the boot wipe destroys the app-managed key backup ───────────── + + #[test] + fn test_wipe_removes_app_managed_key_backup() { + let tmp = TempDir::new().unwrap(); + let app_data = make_app_data(&tmp); + let backup = crate::key_backup::backup_file_path(&app_data); + std::fs::write(&backup, b"encrypted-backup-bytes").unwrap(); + + write_sentinel(&app_data).unwrap(); + let kc = FakeKeychain::ok(); + let outcome = run_boot_reset_with_keychain(make_ctx(&app_data, &kc, false)); + + assert!(outcome.completed); + assert!( + !backup.exists(), + "sign-out wipe must destroy the app-managed key backup" + ); + } + // ── Test 3: keychain failure keeps sentinel ──────────────────────────────── #[test] diff --git a/desktop/src-tauri/src/tray_menu.rs b/desktop/src-tauri/src/tray_menu.rs new file mode 100644 index 0000000000..6dcef0ecf7 --- /dev/null +++ b/desktop/src-tauri/src/tray_menu.rs @@ -0,0 +1,646 @@ +//! Native system-tray menu for the desktop app. +//! +//! The webview owns the live agent-turn state. It sends the small display +//! projection here so the native menu can remain useful while Buzz is hidden. + +// Mouse back/forward (X1/X2 buttons and swipe) is also macOS-only native I/O; +// group it here so both platform-layer init paths share one call site in lib.rs. +#[path = "mouse_nav.rs"] +pub(crate) mod mouse_nav; + +use std::{ + sync::{Mutex, OnceLock}, + time::{Duration, Instant}, +}; + +#[cfg(target_os = "macos")] +use objc2::MainThreadMarker; +#[cfg(target_os = "macos")] +use objc2_foundation::{NSProcessInfo, NSString}; +use serde::{Deserialize, Serialize}; +use tauri::{ + image::Image, + menu::{Menu, MenuItem, PredefinedMenuItem}, + tray::{TrayIcon, TrayIconBuilder}, + AppHandle, Emitter, Manager, Runtime, +}; + +const TRAY_ID: &str = "buzz-tray"; +const OPEN_BUZZ_ID: &str = "tray-open-buzz"; +const NEW_CHANNEL_ID: &str = "tray-new-channel"; +const QUIT_ID: &str = "tray-quit"; +const OPEN_CHANNEL_PREFIX: &str = "tray-open-channel:"; +const OPEN_CHANNEL_ACTIVITY_SEPARATOR: char = '|'; +#[cfg(target_os = "macos")] +const TRAY_MENU_MINIMUM_WIDTH: f64 = 320.0; + +static PREVIEW_STARTED_AT: OnceLock = OnceLock::new(); + +/// A local-only menu preview for demonstrating the working-agent section +/// without connecting to a relay. It is deliberately unavailable in release +/// builds and must be explicitly enabled when launching the debug app. +fn preview_activities() -> Option> { + if !cfg!(debug_assertions) || std::env::var("BUZZ_TRAY_MENU_DEMO").ok().as_deref() != Some("1") + { + return None; + } + + let preview_elapsed = PREVIEW_STARTED_AT.get_or_init(Instant::now).elapsed(); + + Some(vec![ + TrayAgentActivity { + activity_id: "tray-preview-planning-scout".into(), + agent_name: "Scout".into(), + channel_id: "tray-preview-planning".into(), + channel_name: "planning".into(), + elapsed: format_elapsed(Duration::from_secs(192) + preview_elapsed), + }, + TrayAgentActivity { + activity_id: "tray-preview-planning-builder".into(), + agent_name: "Builder".into(), + channel_id: "tray-preview-planning".into(), + channel_name: "planning".into(), + elapsed: format_elapsed(Duration::from_secs(68) + preview_elapsed), + }, + TrayAgentActivity { + activity_id: "tray-preview-mobile-reviewer".into(), + agent_name: "Reviewer".into(), + channel_id: "tray-preview-mobile".into(), + channel_name: "mobile".into(), + elapsed: format_elapsed(Duration::from_secs(31) + preview_elapsed), + }, + ]) +} + +fn preview_recent_activities() -> Option> { + if !cfg!(debug_assertions) || std::env::var("BUZZ_TRAY_MENU_DEMO").ok().as_deref() != Some("1") + { + return None; + } + + Some(vec![TrayAgentActivity { + activity_id: "recent:tray-preview-design-architect".into(), + agent_name: "Architect".into(), + channel_id: "tray-preview-design".into(), + channel_name: "design".into(), + elapsed: "4m 25s".into(), + }]) +} + +fn format_elapsed(elapsed: Duration) -> String { + let total_seconds = elapsed.as_secs(); + if total_seconds < 60 { + return format!("{total_seconds}s"); + } + + let seconds = total_seconds % 60; + let total_minutes = total_seconds / 60; + if total_minutes < 60 { + return format!("{total_minutes}m {seconds}s"); + } + + let minutes = total_minutes % 60; + let hours = total_minutes / 60; + format!("{hours}h {minutes}m {seconds}s") +} + +/// Builds the standalone Buzz bee as a transparent, macOS template image. +/// +/// The app icon includes a rounded square, which is useful for the Dock but +/// looks out of place beside the monochrome menu-bar icons. Keeping this +/// vector-derived mask here also lets macOS tint it correctly in light and +/// dark menu bars without a separate bitmap asset. +fn tray_bee_icon() -> Image<'static> { + const WIDTH: u32 = 64; + const HEIGHT: u32 = 43; + const SAMPLES_PER_AXIS: u32 = 4; + const BEE_WIDTH: f32 = 466.0; + const BEE_HEIGHT: f32 = 309.0; + + fn circle_contains(x: f32, y: f32, center_x: f32, center_y: f32, radius: f32) -> bool { + let delta_x = x - center_x; + let delta_y = y - center_y; + delta_x * delta_x + delta_y * delta_y <= radius * radius + } + + fn rounded_rect_contains( + x: f32, + y: f32, + left: f32, + top: f32, + width: f32, + height: f32, + radius: f32, + ) -> bool { + let right = left + width; + let bottom = top + height; + let closest_x = x.clamp(left + radius, right - radius); + let closest_y = y.clamp(top + radius, bottom - radius); + let delta_x = x - closest_x; + let delta_y = y - closest_y; + delta_x * delta_x + delta_y * delta_y <= radius * radius + } + + fn bee_contains(x: f32, y: f32) -> bool { + let silhouette = circle_contains(x, y, 91.7, 154.5, 91.7) + || circle_contains(x, y, 374.3, 154.5, 91.7) + || rounded_rect_contains(x, y, 128.0, 0.0, 210.0, 309.0, 34.0); + let cutout = circle_contains(x, y, 193.3, 84.4, 27.0) + || circle_contains(x, y, 276.0, 84.4, 27.0) + || rounded_rect_contains(x, y, 166.3, 157.2, 136.9, 38.3, 5.0) + || rounded_rect_contains(x, y, 166.9, 235.1, 136.2, 37.6, 5.0); + + silhouette && !cutout + } + + let mut rgba = vec![0; (WIDTH * HEIGHT * 4) as usize]; + let samples = SAMPLES_PER_AXIS * SAMPLES_PER_AXIS; + + for pixel_y in 0..HEIGHT { + for pixel_x in 0..WIDTH { + let mut covered_samples = 0; + for sample_y in 0..SAMPLES_PER_AXIS { + for sample_x in 0..SAMPLES_PER_AXIS { + let x = (pixel_x as f32 + (sample_x as f32 + 0.5) / SAMPLES_PER_AXIS as f32) + / WIDTH as f32 + * BEE_WIDTH; + let y = (pixel_y as f32 + (sample_y as f32 + 0.5) / SAMPLES_PER_AXIS as f32) + / HEIGHT as f32 + * BEE_HEIGHT; + if bee_contains(x, y) { + covered_samples += 1; + } + } + } + + let index = ((pixel_y * WIDTH + pixel_x) * 4) as usize; + rgba[index + 3] = (covered_samples * u8::MAX as u32 / samples) as u8; + } + } + + Image::new_owned(rgba, WIDTH, HEIGHT) +} + +/// A running agent and its current channel. +#[derive(Debug, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct TrayAgentActivity { + activity_id: String, + agent_name: String, + channel_id: String, + channel_name: String, + elapsed: String, +} + +struct TrayActivityMenuItem { + activity_id: String, + channel_id: String, + agent_item: MenuItem, +} + +struct TrayActionQueue { + community_generation: u64, + pending_actions: Vec, +} + +struct TrayMenuState { + activity_items: Mutex>>, + action_queue: Mutex, +} + +#[derive(Debug, Deserialize, Serialize, PartialEq, Eq)] +#[serde(rename_all = "camelCase", tag = "kind")] +pub enum TrayAction { + NewChannel, + OpenChannel { + channel_id: String, + community_generation: u64, + }, +} + +pub(crate) fn show_main_window(app: &AppHandle) { + let Some(window) = app.get_webview_window("main") else { + return; + }; + if let Err(error) = window.unminimize() { + eprintln!("buzz-desktop: failed to restore main window from tray: {error}"); + return; + } + if let Err(error) = window.show() { + eprintln!("buzz-desktop: failed to show main window from tray: {error}"); + return; + } + if let Err(error) = window.set_focus() { + eprintln!("buzz-desktop: failed to focus main window from tray: {error}"); + } +} + +fn queue_tray_action(app: &AppHandle, mut action: TrayAction) { + let state = app.state::>(); + let Ok(mut queue) = state.action_queue.lock() else { + eprintln!("buzz-desktop: tray action queue is unavailable"); + return; + }; + if let TrayAction::OpenChannel { + community_generation, + .. + } = &mut action + { + *community_generation = queue.community_generation; + } + queue.pending_actions.push(action); + drop(queue); + + if let Err(error) = app.emit("tray-action-available", ()) { + eprintln!("buzz-desktop: failed to notify frontend of tray action: {error}"); + } +} + +fn append_separator(app: &AppHandle, menu: &Menu) -> tauri::Result<()> { + menu.append(&PredefinedMenuItem::separator(app)?) +} + +fn agent_item_label(activity: &TrayAgentActivity) -> String { + let primary = format!("{} · {}", activity.agent_name, activity.elapsed); + + #[cfg(target_os = "macos")] + { + if supports_menu_item_subtitles() { + primary + } else { + format!("{primary} — #{}", activity.channel_name) + } + } + + #[cfg(not(target_os = "macos"))] + { + format!("{primary} — #{}", activity.channel_name) + } +} + +#[cfg(target_os = "macos")] +fn supports_menu_item_subtitles() -> bool { + static SUPPORTS_SUBTITLES: OnceLock = OnceLock::new(); + *SUPPORTS_SUBTITLES.get_or_init(|| { + NSProcessInfo::processInfo() + .operatingSystemVersion() + .majorVersion + >= 14 + }) +} + +fn channel_item_id(activity: &TrayAgentActivity) -> String { + format!( + "{OPEN_CHANNEL_PREFIX}{}{OPEN_CHANNEL_ACTIVITY_SEPARATOR}{}", + activity.channel_id, activity.activity_id + ) +} + +fn build_menu( + app: &AppHandle, + activities: &[TrayAgentActivity], + recent_activities: &[TrayAgentActivity], +) -> tauri::Result<(Menu, Vec>)> { + let menu = Menu::new(app)?; + let mut activity_items = + Vec::with_capacity(activities.len().saturating_add(recent_activities.len())); + + let running = MenuItem::new(app, "Running", false, None::<&str>)?; + menu.append(&running)?; + + if activities.is_empty() { + let empty = MenuItem::new(app, "No agents are running", false, None::<&str>)?; + menu.append(&empty)?; + } else { + append_activity_items(app, &menu, activities, &mut activity_items)?; + } + + if !recent_activities.is_empty() { + append_separator(app, &menu)?; + let recent = MenuItem::new(app, "Recent", false, None::<&str>)?; + menu.append(&recent)?; + append_activity_items(app, &menu, recent_activities, &mut activity_items)?; + } + + append_separator(app, &menu)?; + menu.append(&MenuItem::with_id( + app, + NEW_CHANNEL_ID, + "New Channel", + true, + None::<&str>, + )?)?; + append_separator(app, &menu)?; + menu.append(&MenuItem::with_id( + app, + OPEN_BUZZ_ID, + "Open Buzz", + true, + None::<&str>, + )?)?; + append_separator(app, &menu)?; + menu.append(&MenuItem::with_id( + app, + QUIT_ID, + "Quit Buzz", + true, + None::<&str>, + )?)?; + + Ok((menu, activity_items)) +} + +fn append_activity_items( + app: &AppHandle, + menu: &Menu, + activities: &[TrayAgentActivity], + activity_items: &mut Vec>, +) -> tauri::Result<()> { + for activity in activities { + let agent_item = MenuItem::with_id( + app, + channel_item_id(activity), + agent_item_label(activity), + true, + None::<&str>, + )?; + menu.append(&agent_item)?; + activity_items.push(TrayActivityMenuItem { + activity_id: activity.activity_id.clone(), + channel_id: activity.channel_id.clone(), + agent_item, + }); + } + + Ok(()) +} + +#[cfg(target_os = "macos")] +fn apply_activity_presentation( + tray: &TrayIcon, + activities: &[TrayAgentActivity], + recent_activities: &[TrayAgentActivity], +) -> Result<(), String> { + if !supports_menu_item_subtitles() { + return Ok(()); + } + + let subtitles = activities + .iter() + .chain(recent_activities) + .map(|activity| format!("#{}", activity.channel_name)) + .collect::>(); + let running_count = activities.len(); + let recent_count = recent_activities.len(); + + tray.with_inner_tray_icon(move |inner| { + let Some(status_item) = inner.ns_status_item() else { + return; + }; + let Some(main_thread) = MainThreadMarker::new() else { + return; + }; + let Some(menu) = status_item.menu(main_thread) else { + return; + }; + menu.setMinimumWidth(TRAY_MENU_MINIMUM_WIDTH); + + let mut item_index = 1; + for subtitle in subtitles.iter().take(running_count) { + if let Some(item) = menu.itemAtIndex(item_index) { + let subtitle = NSString::from_str(subtitle); + item.setSubtitle(Some(&subtitle)); + } + item_index += 1; + } + + if running_count == 0 { + item_index += 1; + } + + if recent_count > 0 { + // The separator and Recent heading precede the completed rows. + item_index += 2; + for subtitle in subtitles.iter().skip(running_count) { + if let Some(item) = menu.itemAtIndex(item_index) { + let subtitle = NSString::from_str(subtitle); + item.setSubtitle(Some(&subtitle)); + } + item_index += 1; + } + } + }) + .map_err(|error| error.to_string()) +} + +#[cfg(not(target_os = "macos"))] +fn apply_activity_presentation( + _tray: &TrayIcon, + _activities: &[TrayAgentActivity], + _recent_activities: &[TrayAgentActivity], +) -> Result<(), String> { + Ok(()) +} + +fn handle_menu_event(app: &AppHandle, id: &str) { + match id { + OPEN_BUZZ_ID => show_main_window(app), + NEW_CHANNEL_ID => { + show_main_window(app); + queue_tray_action(app, TrayAction::NewChannel); + } + QUIT_ID => app.exit(0), + _ => { + let Some(channel_id) = id.strip_prefix(OPEN_CHANNEL_PREFIX) else { + return; + }; + show_main_window(app); + let channel_id = channel_id + .split_once(OPEN_CHANNEL_ACTIVITY_SEPARATOR) + .map(|(channel_id, _)| channel_id) + .unwrap_or(channel_id); + queue_tray_action( + app, + TrayAction::OpenChannel { + channel_id: channel_id.into(), + community_generation: 0, + }, + ); + } + } +} + +/// Installs the persistent Buzz tray icon with the initial empty activity menu. +pub fn init(app: &AppHandle) -> tauri::Result<()> { + let preview_activities = preview_activities(); + let preview_recent_activities = preview_recent_activities(); + let activities = preview_activities.as_deref().unwrap_or(&[]); + let recent_activities = preview_recent_activities.as_deref().unwrap_or(&[]); + let (menu, activity_items) = build_menu(app, activities, recent_activities)?; + app.manage(TrayMenuState { + activity_items: Mutex::new(activity_items), + action_queue: Mutex::new(TrayActionQueue { + community_generation: 0, + pending_actions: Vec::new(), + }), + }); + let tray = TrayIconBuilder::with_id(TRAY_ID) + .menu(&menu) + .icon(tray_bee_icon()) + .icon_as_template(true) + .on_menu_event(|app, event| handle_menu_event(app, event.id.as_ref())) + .build(app)?; + if let Err(error) = apply_activity_presentation(&tray, activities, recent_activities) { + eprintln!("buzz-desktop: failed to apply tray menu presentation: {error}"); + } + mouse_nav::init(app); + Ok(()) +} + +/// Drains actions selected from the tray while the frontend was unavailable. +#[tauri::command] +pub fn take_tray_actions(app: AppHandle) -> Result, String> { + let state = app.state::>(); + let mut queue = state + .action_queue + .lock() + .map_err(|_| "Buzz tray action queue is unavailable".to_string())?; + Ok(std::mem::take(&mut queue.pending_actions)) +} + +fn requeue_actions(queue: &mut TrayActionQueue, mut actions: Vec) { + actions.retain(|action| match action { + TrayAction::NewChannel => true, + TrayAction::OpenChannel { + community_generation, + .. + } => *community_generation == queue.community_generation, + }); + actions.append(&mut queue.pending_actions); + queue.pending_actions = actions; +} + +/// Restores actions that were drained as the frontend unmounted. Channel +/// actions from a previous community generation are discarded. +#[tauri::command] +pub fn requeue_tray_actions( + app: AppHandle, + actions: Vec, +) -> Result<(), String> { + let state = app.state::>(); + let mut queue = state + .action_queue + .lock() + .map_err(|_| "Buzz tray action queue is unavailable".to_string())?; + requeue_actions(&mut queue, actions); + drop(queue); + app.emit("tray-action-available", ()) + .map_err(|error| error.to_string()) +} + +/// Clears community-scoped agent activity and queued channel navigation from +/// the native tray menu. +#[tauri::command] +pub fn clear_tray_agent_activity(app: AppHandle) -> Result<(), String> { + let state = app.state::>(); + let mut queue = state + .action_queue + .lock() + .map_err(|_| "Buzz tray action queue is unavailable".to_string())?; + queue.community_generation = queue.community_generation.wrapping_add(1); + queue + .pending_actions + .retain(|action| matches!(action, TrayAction::NewChannel)); + drop(queue); + + update_tray_agent_activity(app, Vec::new(), Vec::new()) +} + +/// Replaces the native menu's activity section with the current live work. +#[tauri::command] +pub fn update_tray_agent_activity( + app: AppHandle, + activities: Vec, + recent_activities: Vec, +) -> Result<(), String> { + let preview_activities = preview_activities(); + let preview_recent_activities = preview_recent_activities(); + let activities = preview_activities.as_deref().unwrap_or(&activities); + let recent_activities = preview_recent_activities + .as_deref() + .unwrap_or(&recent_activities); + let state = app.state::>(); + let mut activity_items = state + .activity_items + .lock() + .map_err(|_| "Buzz tray menu state is unavailable".to_string())?; + + if activity_items.len() == activities.len().saturating_add(recent_activities.len()) + && activity_items + .iter() + .zip(activities.iter().chain(recent_activities)) + .all(|(item, activity)| { + item.activity_id == activity.activity_id && item.channel_id == activity.channel_id + }) + { + for (item, activity) in activity_items + .iter() + .zip(activities.iter().chain(recent_activities)) + { + item.agent_item + .set_text(agent_item_label(activity)) + .map_err(|error| error.to_string())?; + } + let tray = app + .tray_by_id(TRAY_ID) + .ok_or_else(|| "Buzz tray icon is not available".to_string())?; + apply_activity_presentation(&tray, activities, recent_activities)?; + return Ok(()); + } + + let (menu, next_activity_items) = + build_menu(&app, activities, recent_activities).map_err(|error| error.to_string())?; + let tray = app + .tray_by_id(TRAY_ID) + .ok_or_else(|| "Buzz tray icon is not available".to_string())?; + tray.set_menu(Some(menu)) + .map_err(|error| error.to_string())?; + apply_activity_presentation(&tray, activities, recent_activities)?; + *activity_items = next_activity_items; + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::{requeue_actions, TrayAction, TrayActionQueue}; + + #[test] + fn stale_channel_actions_are_not_requeued_after_community_change() { + let mut queue = TrayActionQueue { + community_generation: 2, + pending_actions: Vec::new(), + }; + + requeue_actions( + &mut queue, + vec![TrayAction::OpenChannel { + channel_id: "old-channel".into(), + community_generation: 1, + }], + ); + + assert!(queue.pending_actions.is_empty()); + } + + #[test] + fn new_channel_actions_survive_community_change() { + let mut queue = TrayActionQueue { + community_generation: 2, + pending_actions: Vec::new(), + }; + + requeue_actions(&mut queue, vec![TrayAction::NewChannel]); + + assert_eq!(queue.pending_actions, vec![TrayAction::NewChannel]); + } +} diff --git a/desktop/src-tauri/tauri.conf.json b/desktop/src-tauri/tauri.conf.json index 7a480c4c18..93326b9968 100644 --- a/desktop/src-tauri/tauri.conf.json +++ b/desktop/src-tauri/tauri.conf.json @@ -1,7 +1,7 @@ { "$schema": "https://schema.tauri.app/config/2", "productName": "Buzz", - "version": "0.5.0", + "version": "0.5.3", "identifier": "xyz.block.buzz.app", "build": { "beforeDevCommand": { @@ -55,6 +55,7 @@ "externalBin": [ "binaries/buzz-acp", "binaries/buzz-agent", + "binaries/buzz-backend-kubernetes", "binaries/buzz-dev-mcp", "binaries/git-credential-nostr", "binaries/buzz" diff --git a/desktop/src-tauri/tauri.windows.conf.json b/desktop/src-tauri/tauri.windows.conf.json new file mode 100644 index 0000000000..abf09b5bfd --- /dev/null +++ b/desktop/src-tauri/tauri.windows.conf.json @@ -0,0 +1,11 @@ +{ + "bundle": { + "externalBin": [ + "binaries/buzz-acp", + "binaries/buzz-agent", + "binaries/buzz-dev-mcp", + "binaries/git-credential-nostr", + "binaries/buzz" + ] + } +} diff --git a/desktop/src/app/App.tsx b/desktop/src/app/App.tsx index 90b5bf5ebc..44618f2c72 100644 --- a/desktop/src/app/App.tsx +++ b/desktop/src/app/App.tsx @@ -56,6 +56,7 @@ import { WelcomeSetup } from "@/features/communities/ui/WelcomeSetup"; import { CommunityApplyErrorScreen } from "@/features/communities/ui/CommunityApplyErrorScreen"; import { CommunityChangeOverlay } from "@/features/communities/ui/CommunityChangeOverlay"; import { setAvatarProfileSyncQueryClient } from "@/features/profile/avatarProfileSync"; +import { EncryptedBackupProvider } from "@/features/settings/EncryptedBackupProvider"; import { createBuzzQueryClient } from "@/shared/api/queryClient"; import { isSharedIdentity as isSharedIdentityCmd } from "@/shared/api/tauri"; import { getProfile } from "@/shared/api/tauriProfiles"; @@ -270,9 +271,18 @@ function AppReady({ } return ( - - - + + void router.navigate({ + to: "/settings", + search: { section: "profile" }, + }) + } + > + + + + ); } diff --git a/desktop/src/app/AppHuddleBar.tsx b/desktop/src/app/AppHuddleBar.tsx new file mode 100644 index 0000000000..9fa12d513f --- /dev/null +++ b/desktop/src/app/AppHuddleBar.tsx @@ -0,0 +1,25 @@ +import type * as React from "react"; + +import { HuddleBar } from "@/features/huddle"; + +import { AppProfilePanelProvider } from "@/app/AppProfilePanelProvider"; + +type AppHuddleBarProps = Pick< + React.ComponentProps, + "onOpenThread" | "onVisibilityChange" +>; + +export function AppHuddleBar({ + onOpenThread, + onVisibilityChange, +}: AppHuddleBarProps) { + return ( + + + + ); +} diff --git a/desktop/src/app/AppProfilePanelProvider.tsx b/desktop/src/app/AppProfilePanelProvider.tsx new file mode 100644 index 0000000000..213acec498 --- /dev/null +++ b/desktop/src/app/AppProfilePanelProvider.tsx @@ -0,0 +1,22 @@ +import * as React from "react"; + +import { useAppNavigation } from "@/app/navigation/useAppNavigation"; +import { ProfilePanelProvider } from "@/shared/context/ProfilePanelContext"; + +export function AppProfilePanelProvider({ + children, +}: Readonly<{ children: React.ReactNode }>) { + const { goProfile } = useAppNavigation(); + const handleOpenProfilePanel = React.useCallback( + (pubkey: string) => { + void goProfile(pubkey); + }, + [goProfile], + ); + + return ( + + {children} + + ); +} diff --git a/desktop/src/app/AppShell.tsx b/desktop/src/app/AppShell.tsx index 75f57257cc..4eb0a42bbe 100644 --- a/desktop/src/app/AppShell.tsx +++ b/desktop/src/app/AppShell.tsx @@ -63,7 +63,8 @@ import { type SettingsSection, isSettingsSection, } from "@/features/settings/ui/SettingsPanels"; -import { HuddleBar, HuddleProvider } from "@/features/huddle"; +import { HuddleProvider } from "@/features/huddle"; +import { AppHuddleBar } from "@/app/AppHuddleBar"; import { useDueReminderBadgeCount } from "@/features/reminders/hooks"; import { RemindMeLaterProvider } from "@/features/reminders/ui/RemindMeLaterProvider"; import { useReminderNotifications } from "@/features/reminders/useReminderNotifications"; @@ -96,7 +97,8 @@ import { useMessageDeepLinks } from "@/shared/useMessageDeepLinks"; import { SidebarInset, SidebarProvider } from "@/shared/ui/sidebar"; import { RelayConnectionOverlay } from "@/app/RelayConnectionOverlay"; import { useSidebarRelayConnectionCard } from "@/features/sidebar/ui/useSidebarRelayConnectionCard"; - +import { AppShellTrayMenu } from "@/app/useAppShellTrayMenu"; +import { AppProfilePanelProvider } from "@/app/AppProfilePanelProvider"; const LazySettingsScreen = React.lazy(async () => { const module = await import("@/features/settings/ui/SettingsScreen"); return { default: module.SettingsScreen }; @@ -159,7 +161,6 @@ export function AppShell() { ? locationSearchSection : DEFAULT_SETTINGS_SECTION; const startupReady = useDeferredStartup(); - const identityQuery = useIdentityQuery(); const { mutedChannelIds, muteChannel, unmuteChannel } = useChannelMutes( identityQuery.data?.pubkey, @@ -231,6 +232,7 @@ export function AppShell() { const relayConnectionCard = useSidebarRelayConnectionCard( channelsErrorMessage, communitiesHook.activeCommunity?.relayUrl, + `${communitiesHook.activeCommunity?.id ?? "none"}-${communitiesHook.reinitKey}`, ); const memberChannels = React.useMemo( () => channels.filter((channel) => channel.isMember), @@ -301,7 +303,6 @@ export function AppShell() { ? (channels.find((channel) => channel.id === targetChannelId) ?? null) : null; }, [channels, managedChannelId, selectedChannelId]); - const { handleChannelNotification, handleDmNotification, @@ -516,7 +517,6 @@ export function AppShell() { }, [applyAgents, applyCanvas, createChannelMutation, goChannel], ); - const handleCreateForum = React.useCallback( async ({ description, @@ -584,7 +584,6 @@ export function AppShell() { }, [goHome, hideDmMutation, selectedChannelId], ); - const handleOpenSettings = React.useCallback( (section: SettingsSection = DEFAULT_SETTINGS_SECTION) => { setIsChannelManagementOpen(false); @@ -592,12 +591,10 @@ export function AppShell() { }, [goSettings], ); - const handleCloseSettings = React.useCallback( () => closeSettings(), [closeSettings], ); - // Section switches rewrite the settings entry rather than stacking one // history entry per section, so back always exits settings in one step. const handleSettingsSectionChange = React.useCallback( @@ -613,20 +610,13 @@ export function AppShell() { }, [openSearchHit], ); - useAppShellLifecycleEffects({ homeBadgeCountExcludingHighPriority, unreadChannelIds, unreadChannelNotificationCount, }); - // Dispatch `buzz://message` deep links into the router. useMessageDeepLinks(); - - const handleOpenNewDm = React.useCallback( - () => void goNewMessage(), - [goNewMessage], - ); const handleOpenCreateChannel = React.useCallback( () => setIsCreateChannelOpen(true), [], @@ -659,7 +649,7 @@ export function AppShell() { if (key === "k" && event.shiftKey) { event.preventDefault(); - handleOpenNewDm(); + void goNewMessage(); return; } @@ -688,9 +678,9 @@ export function AppShell() { }; }, [ handleOpenBrowseChannels, - handleOpenNewDm, handleOpenCreateChannel, handleOpenSearch, + goNewMessage, goHome, settingsOpen, ]); @@ -706,9 +696,13 @@ export function AppShell() { markChannelRead, selectedView, }); - return ( + ) : null} - {!settingsOpen ? ( - - ) : null} - {settingsOpen ? ( -
- - + {!settingsOpen ? ( + + ) : null} + {settingsOpen ? ( +
+ + + +
+ ) : ( +
+ { + const id = communitiesHook.addCommunity({ + ...community, + pubkey: + community.pubkey ?? + identityQuery.data?.pubkey, + }); + handleSwitchCommunity(id); + }} + onAddCommunityOpenChange={ + addCommunityDialog.onOpenChange } - notificationSettings={notificationSettings.settings} - onClose={handleCloseSettings} - onSectionChange={handleSettingsSectionChange} - onSetDesktopNotificationsEnabled={ - notificationSettings.setDesktopEnabled + onNewMessage={goNewMessage} + onBackgroundClick={requestFocusedThreadClose} + onCreateChannelOpenChange={setIsCreateChannelOpen} + onOpenAddCommunity={addCommunityDialog.openDialog} + onSendFeedback={() => setIsSendFeedbackOpen(true)} + onUpdateCommunity={communitiesHook.updateCommunity} + onRemoveCommunity={(id) => + void handleRemoveCommunity(id) } - onSetHomeBadgeEnabled={ - notificationSettings.setHomeBadgeEnabled + onSwitchCommunity={handleSwitchCommunity} + onCreateAgent={() => requestOpenCreateAgent()} + selfPresenceStatus={presenceSession.currentStatus} + communities={communitiesHook.communities} + onCreateChannel={handleCreateChannel} + onCreateForum={handleCreateForum} + onHideDm={handleHideDm} + onMarkAllChannelsRead={markAllChannelsRead} + onMarkChannelRead={markChannelRead} + onMarkChannelUnread={markChannelUnread} + onBrowseChannels={handleOpenBrowseChannels} + onOpenDm={async ({ pubkeys }) => { + const directMessage = + await openDmMutation.mutateAsync({ + pubkeys, + }); + await goChannel(directMessage.id); + }} + onSelectAgents={() => void goAgents()} + onSelectChannel={(channelId) => + void goChannel(channelId) } - onSetSlotAlertsEnabled={ - notificationSettings.setSlotAlertsEnabled + onOpenSearchResult={handleOpenSearchResult} + searchChannels={channels} + searchFocusRequest={searchFocusRequest} + onSelectHome={() => void goHome()} + onSelectProjects={() => void goProjects()} + onSelectPulse={() => void goPulse()} + onSelectSettings={handleOpenSettings} + onSelectWorkflows={() => void goWorkflows()} + onSetPresenceStatus={(status) => + presenceSession.setStatus(status) } - onSetNotifyWhileViewing={ - notificationSettings.setNotifyWhileViewing + onSetUserStatus={(text, emoji) => + setUserStatusMutation.mutate({ text, emoji }) } - onSetAllSlotAlertsEnabled={ - notificationSettings.setAllSlotAlertsEnabled + onClearUserStatus={() => + setUserStatusMutation.mutate({ + text: "", + emoji: "", + }) } - onSetSoundForSlot={ - notificationSettings.setSoundForSlot + profile={profileQuery.data} + selfUserStatus={ + deferredPubkey + ? (selfStatusQuery.data?.[ + deferredPubkey.toLowerCase() + ] ?? undefined) + : undefined } - section={settingsSection} + selectedChannelId={selectedChannelId} + selectedView={selectedView} + unreadChannelIds={unreadChannelIds} + unreadChannelCounts={unreadChannelCounts} + mutedChannelIds={mutedChannelIds} + onMuteChannel={muteChannel} + onUnmuteChannel={unmuteChannel} + starredChannelIds={starredChannelIds} + onStarChannel={starChannel} + onUnstarChannel={unstarChannel} /> - -
- ) : ( -
- { - const id = communitiesHook.addCommunity({ - ...community, - pubkey: - community.pubkey ?? identityQuery.data?.pubkey, - }); - handleSwitchCommunity(id); - }} - onAddCommunityOpenChange={ - addCommunityDialog.onOpenChange - } - onNewMessage={handleOpenNewDm} - onBackgroundClick={requestFocusedThreadClose} - onCreateChannelOpenChange={setIsCreateChannelOpen} - onOpenAddCommunity={addCommunityDialog.openDialog} - onSendFeedback={() => setIsSendFeedbackOpen(true)} - onUpdateCommunity={communitiesHook.updateCommunity} - onRemoveCommunity={(id) => - void handleRemoveCommunity(id) - } - onSwitchCommunity={handleSwitchCommunity} - onCreateAgent={() => requestOpenCreateAgent()} - selfPresenceStatus={presenceSession.currentStatus} - communities={communitiesHook.communities} - onCreateChannel={handleCreateChannel} - onCreateForum={handleCreateForum} - onHideDm={handleHideDm} - onMarkAllChannelsRead={markAllChannelsRead} - onMarkChannelRead={markChannelRead} - onMarkChannelUnread={markChannelUnread} - onBrowseChannels={handleOpenBrowseChannels} - onOpenDm={async ({ pubkeys }) => { - const directMessage = - await openDmMutation.mutateAsync({ - pubkeys, - }); - await goChannel(directMessage.id); - }} - onSelectAgents={() => void goAgents()} - onSelectChannel={(channelId) => - void goChannel(channelId) - } - onOpenSearchResult={handleOpenSearchResult} - searchChannels={channels} - searchFocusRequest={searchFocusRequest} - onSelectHome={() => void goHome()} - onSelectProjects={() => void goProjects()} - onSelectPulse={() => void goPulse()} - onSelectSettings={handleOpenSettings} - onSelectWorkflows={() => void goWorkflows()} - onSetPresenceStatus={(status) => - presenceSession.setStatus(status) - } - onSetUserStatus={(text, emoji) => - setUserStatusMutation.mutate({ text, emoji }) - } - onClearUserStatus={() => - setUserStatusMutation.mutate({ - text: "", - emoji: "", - }) - } - profile={profileQuery.data} - selfUserStatus={ - deferredPubkey - ? (selfStatusQuery.data?.[ - deferredPubkey.toLowerCase() - ] ?? undefined) - : undefined + + + + + + + + +
+ )} + + + { + setIsChannelManagementOpen(open); + if (!open) { + setManagedChannelId(null); } - selectedChannelId={selectedChannelId} - selectedView={selectedView} - unreadChannelIds={unreadChannelIds} - unreadChannelCounts={unreadChannelCounts} - mutedChannelIds={mutedChannelIds} - onMuteChannel={muteChannel} - onUnmuteChannel={unmuteChannel} - starredChannelIds={starredChannelIds} - onStarChannel={starChannel} - onUnstarChannel={unstarChannel} - /> - - - - - - - - -
- )} - - - { - setIsChannelManagementOpen(open); - if (!open) { + }} + onDeleteActiveChannel={() => { + setIsChannelManagementOpen(false); setManagedChannelId(null); - } - }} - onDeleteActiveChannel={() => { - setIsChannelManagementOpen(false); - setManagedChannelId(null); - void goHome({ replace: true }); - }} - onSelectChannel={(channelId) => { - void goChannel(channelId); - }} - /> - + void goHome({ replace: true }); + }} + onSelectChannel={(channelId) => { + void goChannel(channelId); + }} + /> + +
- { void goChannel(channelId, { messageId, diff --git a/desktop/src/app/navigation/backForwardChords.test.mjs b/desktop/src/app/navigation/backForwardChords.test.mjs new file mode 100644 index 0000000000..cb60203d04 --- /dev/null +++ b/desktop/src/app/navigation/backForwardChords.test.mjs @@ -0,0 +1,139 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { matchBackForwardChord } from "./backForwardChords.ts"; + +function chord(overrides = {}) { + return { + altKey: false, + code: "", + ctrlKey: false, + key: "", + metaKey: false, + shiftKey: false, + ...overrides, + }; +} + +// ── macOS: ⌘[ / ⌘] ─────────────────────────────────────────────────────────── + +test("mac: ⌘[ matches back", () => { + assert.equal( + matchBackForwardChord(chord({ key: "[", metaKey: true }), true), + "back", + ); +}); + +test("mac: ⌘] matches forward", () => { + assert.equal( + matchBackForwardChord(chord({ key: "]", metaKey: true }), true), + "forward", + ); +}); + +test("mac: matches by code for non-US layouts", () => { + assert.equal( + matchBackForwardChord( + chord({ code: "BracketLeft", key: "Dead", metaKey: true }), + true, + ), + "back", + ); + assert.equal( + matchBackForwardChord( + chord({ code: "BracketRight", key: "Dead", metaKey: true }), + true, + ), + "forward", + ); +}); + +test("mac: requires meta", () => { + assert.equal(matchBackForwardChord(chord({ key: "[" }), true), null); +}); + +test("mac: rejects extra modifiers", () => { + for (const extra of [ + { altKey: true }, + { ctrlKey: true }, + { shiftKey: true }, + ]) { + assert.equal( + matchBackForwardChord(chord({ key: "[", metaKey: true, ...extra }), true), + null, + ); + } +}); + +test("mac: Alt+arrows do not match (that is the win/linux chord)", () => { + assert.equal( + matchBackForwardChord(chord({ altKey: true, key: "ArrowLeft" }), true), + null, + ); +}); + +test("mac: ⌘←/⌘→ never match — they are line start/end in text editing", () => { + // Deliberately unbound: editable targets must keep receiving ⌘←/⌘→ so + // line-start/line-end editing still works. Only ⌘[ / ⌘] navigate. + assert.equal( + matchBackForwardChord(chord({ key: "ArrowLeft", metaKey: true }), true), + null, + ); + assert.equal( + matchBackForwardChord(chord({ key: "ArrowRight", metaKey: true }), true), + null, + ); +}); + +// ── Windows/Linux: Alt+← / Alt+→ ───────────────────────────────────────────── + +test("win/linux: Alt+ArrowLeft matches back", () => { + assert.equal( + matchBackForwardChord(chord({ altKey: true, key: "ArrowLeft" }), false), + "back", + ); +}); + +test("win/linux: Alt+ArrowRight matches forward", () => { + assert.equal( + matchBackForwardChord(chord({ altKey: true, key: "ArrowRight" }), false), + "forward", + ); +}); + +test("win/linux: requires alt", () => { + assert.equal(matchBackForwardChord(chord({ key: "ArrowLeft" }), false), null); +}); + +test("win/linux: rejects extra modifiers", () => { + for (const extra of [ + { ctrlKey: true }, + { metaKey: true }, + { shiftKey: true }, + ]) { + assert.equal( + matchBackForwardChord( + chord({ altKey: true, key: "ArrowLeft", ...extra }), + false, + ), + null, + ); + } +}); + +test("win/linux: ⌘[ does not match (that is the mac chord)", () => { + assert.equal( + matchBackForwardChord(chord({ key: "[", metaKey: true }), false), + null, + ); +}); + +// ── Non-chord keys never match ──────────────────────────────────────────────── + +test("plain bracket / arrow keys without the platform modifier never match", () => { + for (const isMac of [true, false]) { + for (const key of ["[", "]", "ArrowLeft", "ArrowRight", "a", "Enter"]) { + assert.equal(matchBackForwardChord(chord({ key }), isMac), null); + } + } +}); diff --git a/desktop/src/app/navigation/backForwardChords.ts b/desktop/src/app/navigation/backForwardChords.ts new file mode 100644 index 0000000000..ac81f52ea7 --- /dev/null +++ b/desktop/src/app/navigation/backForwardChords.ts @@ -0,0 +1,51 @@ +/** + * Global back/forward navigation chords. + * + * macOS: ⌘[ / ⌘] — matching Safari, Chrome, Finder, and Slack. + * Windows/Linux: Alt+← / Alt+→ — matching browsers and Slack. + * + * Kept pure (no DOM access) so chord matching can be unit tested; the + * window listener wiring lives in `useBackForwardControls`. + */ + +export type BackForwardDirection = "back" | "forward"; + +export type BackForwardChordEvent = Pick< + KeyboardEvent, + "altKey" | "code" | "ctrlKey" | "key" | "metaKey" | "shiftKey" +>; + +export function matchBackForwardChord( + event: BackForwardChordEvent, + isMac: boolean, +): BackForwardDirection | null { + if (isMac) { + if (!event.metaKey || event.ctrlKey || event.altKey || event.shiftKey) { + return null; + } + + if (event.key === "[" || event.code === "BracketLeft") { + return "back"; + } + + if (event.key === "]" || event.code === "BracketRight") { + return "forward"; + } + + return null; + } + + if (!event.altKey || event.metaKey || event.ctrlKey || event.shiftKey) { + return null; + } + + if (event.key === "ArrowLeft") { + return "back"; + } + + if (event.key === "ArrowRight") { + return "forward"; + } + + return null; +} diff --git a/desktop/src/app/navigation/useAppNavigation.ts b/desktop/src/app/navigation/useAppNavigation.ts index f928970610..d19ac03120 100644 --- a/desktop/src/app/navigation/useAppNavigation.ts +++ b/desktop/src/app/navigation/useAppNavigation.ts @@ -79,6 +79,18 @@ export function useAppNavigation() { [commitNavigation], ); + const goProfile = React.useCallback( + (pubkey: string, behavior?: NavigationBehavior) => + commitNavigation( + { + to: "/pulse", + search: { profile: pubkey }, + }, + behavior, + ), + [commitNavigation], + ); + const goProjects = React.useCallback( (behavior?: NavigationBehavior) => commitNavigation( @@ -303,6 +315,7 @@ export function useAppNavigation() { goProject, goProjects, goPulse, + goProfile, goSettings, goWorkflow, goWorkflows, diff --git a/desktop/src/app/navigation/useBackForwardControls.ts b/desktop/src/app/navigation/useBackForwardControls.ts index 7f7d84f6d7..e5513247d5 100644 --- a/desktop/src/app/navigation/useBackForwardControls.ts +++ b/desktop/src/app/navigation/useBackForwardControls.ts @@ -4,7 +4,10 @@ import { useRouter, useRouterState, } from "@tanstack/react-router"; +import { isTauri } from "@tauri-apps/api/core"; +import { listen } from "@tauri-apps/api/event"; +import { matchBackForwardChord } from "@/app/navigation/backForwardChords"; import { isMacPlatform } from "@/shared/lib/platform"; import { trimMapToSize } from "@/shared/lib/trimMapToSize"; @@ -14,19 +17,6 @@ type RouterHistoryState = { key?: string; }; -function isEditableTarget(target: EventTarget | null): boolean { - if (!(target instanceof HTMLElement)) { - return false; - } - - return ( - target.isContentEditable || - target.closest( - 'input, textarea, select, [contenteditable=""], [contenteditable="true"]', - ) !== null - ); -} - export function useBackForwardControls() { const router = useRouter(); const canGoBack = useCanGoBack(); @@ -81,42 +71,34 @@ export function useBackForwardControls() { }, [canGoForward, router.history]); const handleKeyDown = React.useEffectEvent((event: KeyboardEvent) => { - if (isEditableTarget(event.target)) { + // Note: the chords deliberately fire even when focus is inside an + // editable element. The composer autofocuses on every channel switch + // (`useComposerAutofocus`), so in steady state focus almost always + // lives in a contenteditable — an editable-target guard here made the + // shortcuts effectively dead (#3775). Safe because neither ⌘[ / ⌘] + // (macOS) nor Alt+←/→ (Windows/Linux) carry text-editing semantics, + // and the TipTap editor binds no conflicting shortcuts. + const direction = matchBackForwardChord(event, isMacPlatform()); + + if (direction === "back") { + event.preventDefault(); + goBack(); return; } - const isMac = isMacPlatform(); - const isBackShortcut = isMac - ? event.metaKey && - !event.ctrlKey && - !event.altKey && - !event.shiftKey && - (event.key === "[" || event.code === "BracketLeft") - : event.altKey && - !event.metaKey && - !event.ctrlKey && - !event.shiftKey && - event.key === "ArrowLeft"; - const isForwardShortcut = isMac - ? event.metaKey && - !event.ctrlKey && - !event.altKey && - !event.shiftKey && - (event.key === "]" || event.code === "BracketRight") - : event.altKey && - !event.metaKey && - !event.ctrlKey && - !event.shiftKey && - event.key === "ArrowRight"; - - if (isBackShortcut) { + if (direction === "forward") { event.preventDefault(); + goForward(); + } + }); + + const handleMouseNav = React.useEffectEvent((direction: string) => { + if (direction === "back") { goBack(); return; } - if (isForwardShortcut) { - event.preventDefault(); + if (direction === "forward") { goForward(); } }); @@ -128,6 +110,23 @@ export function useBackForwardControls() { }; }, []); + // macOS: WKWebView never delivers X1/X2 button events or horizontal + // swipe gestures to the DOM, so the native layer catches them + // (`mouse_nav.rs`) and forwards them as a Tauri event. + React.useEffect(() => { + if (!isTauri()) { + return; + } + + const unlistenPromise = listen("mouse-nav", (event) => { + handleMouseNav(event.payload); + }); + + return () => { + void unlistenPromise.then((unlisten) => unlisten()); + }; + }, []); + return { canGoBack, canGoForward, diff --git a/desktop/src/app/useAppShellTrayMenu.tsx b/desktop/src/app/useAppShellTrayMenu.tsx new file mode 100644 index 0000000000..e4b9774848 --- /dev/null +++ b/desktop/src/app/useAppShellTrayMenu.tsx @@ -0,0 +1,41 @@ +import type { Channel } from "@/shared/api/types"; +import { isMacPlatform } from "@/shared/lib/platform"; + +import { useTrayMenu } from "@/app/useTrayMenu"; + +/** Keeps the ticking native tray menu outside AppShell's render cycle. */ +export function AppShellTrayMenu({ + channels, + goChannel, + openCreateChannel, +}: { + channels: Channel[]; + goChannel: (channelId: string) => Promise; + openCreateChannel: () => void; +}) { + if (!isMacPlatform()) return null; + return ( + + ); +} + +function MacAppShellTrayMenu({ + channels, + goChannel, + openCreateChannel, +}: { + channels: Channel[]; + goChannel: (channelId: string) => Promise; + openCreateChannel: () => void; +}): null { + useTrayMenu({ + channels, + goChannel, + openCreateChannel, + }); + return null; +} diff --git a/desktop/src/app/useTrayMenu.ts b/desktop/src/app/useTrayMenu.ts new file mode 100644 index 0000000000..355c8e5d4f --- /dev/null +++ b/desktop/src/app/useTrayMenu.ts @@ -0,0 +1,160 @@ +import * as React from "react"; +import { isTauri, invoke } from "@tauri-apps/api/core"; +import { listen } from "@tauri-apps/api/event"; + +import { + getActiveTurnsForAgent, + useActiveAgentTurnsByChannel, +} from "@/features/agents/activeAgentTurnsStore"; +import { + useManagedAgentsQuery, + useRelayAgentsQuery, +} from "@/features/agents/hooks"; +import { normalizePubkey, truncatePubkey } from "@/shared/lib/pubkey"; +import { useNow } from "@/shared/lib/useNow"; +import { formatElapsed } from "@/features/agents/ui/agentSessionUtils"; +import type { Channel } from "@/shared/api/types"; + +type TrayAgentActivity = { + activityId: string; + agentName: string; + channelId: string; + channelName: string; + elapsed: string; +}; + +type TrayAction = + | { kind: "newChannel" } + | { kind: "openChannel"; channelId: string }; + +const MAX_RECENT_TRAY_ACTIVITIES = 5; + +/** + * Keeps Buzz's native tray menu synchronized with active agent turns and + * forwards its navigation actions into the React app. + */ +export function useTrayMenu({ + channels, + goChannel, + openCreateChannel, +}: { + channels: Channel[]; + goChannel: (channelId: string) => Promise; + openCreateChannel: () => void; +}): void { + const activeTurns = useActiveAgentTurnsByChannel(); + const now = useNow(1000); + const managedAgents = useManagedAgentsQuery().data; + const relayAgents = useRelayAgentsQuery().data; + const previousActivitiesRef = React.useRef( + new Map(), + ); + const [recentActivities, setRecentActivities] = React.useState< + TrayAgentActivity[] + >([]); + + const activities = React.useMemo(() => { + const channelNames = new Map( + channels.map((channel) => [channel.id, channel.name]), + ); + const agentNames = new Map(); + for (const agent of [...(managedAgents ?? []), ...(relayAgents ?? [])]) { + agentNames.set(normalizePubkey(agent.pubkey), agent.name); + } + + return activeTurns.flatMap((channelTurn) => + channelTurn.agentPubkeys.map((pubkey) => { + const agentTurn = getActiveTurnsForAgent(pubkey).find( + (turn) => turn.channelId === channelTurn.channelId, + ); + + return { + activityId: `${channelTurn.channelId}:${normalizePubkey(pubkey)}`, + agentName: + agentNames.get(normalizePubkey(pubkey)) ?? + `Agent ${truncatePubkey(pubkey)}`, + channelId: channelTurn.channelId, + channelName: + channelNames.get(channelTurn.channelId) ?? "Unknown channel", + elapsed: formatElapsed( + now - (agentTurn?.anchorAt ?? channelTurn.anchorAt), + ), + }; + }), + ); + }, [activeTurns, channels, managedAgents, now, relayAgents]); + + React.useEffect(() => { + const currentActivities = new Map( + activities.map((activity) => [activity.activityId, activity]), + ); + const completedActivities = [...previousActivitiesRef.current.entries()] + .filter(([activityId]) => !currentActivities.has(activityId)) + .map(([, activity]) => ({ + ...activity, + activityId: `recent:${activity.activityId}:${Date.now()}`, + })); + + if (completedActivities.length > 0) { + setRecentActivities((current) => + [...completedActivities, ...current].slice( + 0, + MAX_RECENT_TRAY_ACTIVITIES, + ), + ); + } + previousActivitiesRef.current = currentActivities; + }, [activities]); + + React.useEffect(() => { + if (!isTauri()) return; + void invoke("update_tray_agent_activity", { + activities, + recentActivities, + }).catch((error) => { + console.error("Failed to update the macOS tray menu", error); + }); + }, [activities, recentActivities]); + + React.useEffect(() => { + if (!isTauri()) return; + + let disposed = false; + let unlisten: (() => void) | undefined; + + const handlePendingActions = async () => { + if (disposed) return; + const actions = await invoke("take_tray_actions"); + if (disposed) { + if (actions.length > 0) { + await invoke("requeue_tray_actions", { actions }); + } + return; + } + for (const action of actions) { + if (action.kind === "newChannel") { + openCreateChannel(); + } else { + void goChannel(action.channelId); + } + } + }; + + void (async () => { + const nextUnlisten = await listen("tray-action-available", () => { + void handlePendingActions(); + }); + if (disposed) { + nextUnlisten(); + return; + } + unlisten = nextUnlisten; + await handlePendingActions(); + })(); + + return () => { + disposed = true; + unlisten?.(); + }; + }, [goChannel, openCreateChannel]); +} diff --git a/desktop/src/features/agents/AGENTS.md b/desktop/src/features/agents/AGENTS.md index 35ad4a63af..d9222c7032 100644 --- a/desktop/src/features/agents/AGENTS.md +++ b/desktop/src/features/agents/AGENTS.md @@ -114,6 +114,39 @@ with a TypeScript lookup table or an id comparison in a component. published or removed. A queued update must stay visibly queued, and the catalog itself must render only relay-confirmed publications — never an optimistic local persona. +11. **Shared agent access names the consequence where it is selected.** The + shared respond-to field shows a persistent warning whenever `anyone` **or** + `allowlist` is selected — both hand the host's access to someone other than + the owner, so both disclose it and only the audience phrase differs. This + covers persona-backed create and edit surfaces. Keep that disclosure in + the shared field instead of adding surface-specific flags. It renders + directly below the selector for `anyone` but *after* the people picker for + `allowlist`, so it never sits between the user and the selection they came + to make. The copy leads with the audience ("Anyone can use this agent to + access…") so it reads as a warning rather than an explanation, and stays one + sentence — don't split the mechanism into a second sentence. Both the machine + and the stakes it names come from `lib/agentAccessWarning.ts`, keyed on an + optional `runLocation`: instance surfaces resolve it from + `ManagedAgent.backend` via `runLocationForBackend`, and the create flow from + `WhereToRunDraft.runOn` via `runLocationForRunOn`. `AgentDialog` is the one + place that resolves it for dialog surfaces and publishes it through + `ui/AgentRunLocationContext.tsx`; the field reads that context and lets an + explicit `runLocation` prop win. Do **not** thread the value as a prop + through `AgentDefinitionDialog` / `AgentInstanceEditDialog` — both are + already over the 1000-line ceiling, and neither uses the value itself. + Surfaces rendered outside `AgentDialog` (e.g. `EditRespondToDialog`) pass the + prop directly. Local names "your + computer, including files, accounts, and connected tools"; remote names "the + server it runs on, including any accounts and tools available there" — + deliberately *not* the owner's files, which aren't theirs to describe on a + host they don't own. **An unknown location falls back to the local wording — + never hedge with "computer or server".** A remote host requires an + installed `buzz-backend-*` provider, and without one `WhereToRunSection` + never renders, so "server" would name a concept the owner has never been + shown; when it *is* remote they picked that host from the selector + themselves. Never synthesize a run location a surface doesn't have. Don't + expose `respond-to`, `allowlist`, Nostr, or harness jargon in primary UI + copy. ## The tests that enforce this @@ -128,6 +161,12 @@ with a TypeScript lookup table or an id comparison in a component. `isCacheableDiscoveryResponse`, `deriveModelDiscoveryPending`, `isSuccessfulEmptyDiscovery`. If the "reopen to retry" copy becomes inert again, these tests will catch it. +- `ui/respondToFieldContract.test.mjs` — plain-language mode labels, the + persistent warning contract for shared agent access, and its two render + positions (after the people picker for `allowlist`). +- `lib/agentAccessWarning.test.mjs` — every mode × run-location copy variant + plus both resolvers, including unknown-reads-as-local and + blank-`runOn`-is-not-a-provider. - `desktop/tests/e2e/onboarding-agent-defaults.spec.ts` — onboarding behavior acceptance coverage for readiness, failure states, defaults, navigation, successful-empty vs failed optional-model discovery, and persistence races. diff --git a/desktop/src/features/agents/cardMintStore.test.mjs b/desktop/src/features/agents/cardMintStore.test.mjs new file mode 100644 index 0000000000..eb9c0f07b0 --- /dev/null +++ b/desktop/src/features/agents/cardMintStore.test.mjs @@ -0,0 +1,170 @@ +import assert from "node:assert/strict"; +import { beforeEach, describe, it } from "node:test"; + +// cardMintStore drives the non-blocking mint flow: the dialog dispatches a +// job and closes; the composer chip, completion toast, and viewer all read +// this store. These tests exercise the job lifecycle with an injected mintFn +// (never the real Tauri command). +// +// The store imports sonner (toast); calling toast outside a mounted +// only queues, so no DOM is required. + +import { + dismissCardMintJob, + getCardGalleryOpen, + getCardMintJobs, + getCardViewerState, + closeCardViewer, + openCardViewer, + resetCardMintStore, + runCardMintJob, + setCardGalleryOpen, + subscribeCardMintStore, + viewMintedCardJob, +} from "./cardMintStore.ts"; + +const CARD = { + cardPngBase64: "aGVsbG8=", + fileName: "eva.agent.png", + designerNotes: "notes", + locked: false, + memoryLevel: "none", +}; + +const INPUT = { agentId: "agent-1", agentName: "Eva" }; + +describe("cardMintStore", () => { + beforeEach(() => { + resetCardMintStore(); + }); + + it("forwards the mint input — including memoryLevel — to mintFn", async () => { + const seen = []; + await runCardMintJob( + { ...INPUT, styleNotes: "stormy", lock: true, memoryLevel: "core" }, + (...args) => { + seen.push(args); + return Promise.resolve({ ...CARD, memoryLevel: "core" }); + }, + ); + assert.deepEqual(seen, [["agent-1", "stormy", true, "core"]]); + + // Omitted memoryLevel stays undefined so Rust applies its "none" default. + await runCardMintJob(INPUT, (...args) => { + seen.push(args); + return Promise.resolve(CARD); + }); + assert.deepEqual(seen[1], ["agent-1", undefined, undefined, undefined]); + }); + + it("tracks a successful mint through minting → done", async () => { + let resolveMint; + const pending = new Promise((resolve) => { + resolveMint = resolve; + }); + const run = runCardMintJob(INPUT, () => pending); + + let jobs = getCardMintJobs(); + assert.equal(jobs.length, 1); + assert.equal(jobs[0].phase, "minting"); + assert.equal(jobs[0].input.agentName, "Eva"); + assert.equal(jobs[0].card, null); + + resolveMint(CARD); + await run; + + jobs = getCardMintJobs(); + assert.equal(jobs.length, 1); + assert.equal(jobs[0].phase, "done"); + assert.deepEqual(jobs[0].card, CARD); + assert.equal(jobs[0].error, null); + }); + + it("records the error message on a failed mint", async () => { + await runCardMintJob(INPUT, () => Promise.reject(new Error("boom"))); + const jobs = getCardMintJobs(); + assert.equal(jobs.length, 1); + assert.equal(jobs[0].phase, "error"); + assert.equal(jobs[0].error, "boom"); + assert.equal(jobs[0].card, null); + }); + + it("strips the NO_OPENAI_KEY wire prefix from mint errors", async () => { + await runCardMintJob(INPUT, () => + Promise.reject(new Error("NO_OPENAI_KEY: No OPENAI_API_KEY found.")), + ); + assert.equal(getCardMintJobs()[0].error, "No OPENAI_API_KEY found."); + }); + + it("viewMintedCardJob moves a done job into the viewer and clears the chip", async () => { + await runCardMintJob(INPUT, () => Promise.resolve(CARD)); + const jobId = getCardMintJobs()[0].jobId; + + viewMintedCardJob(jobId); + + assert.equal(getCardMintJobs().length, 0); + const viewer = getCardViewerState(); + assert.ok(viewer); + assert.equal(viewer.agentName, "Eva"); + assert.deepEqual(viewer.card, CARD); + // Fresh mints keep their input so the viewer can reroll. + assert.deepEqual(viewer.remint, INPUT); + }); + + it("viewMintedCardJob ignores jobs that are still minting", () => { + let resolveMint; + void runCardMintJob( + INPUT, + () => new Promise((resolve) => (resolveMint = resolve)), + ); + const jobId = getCardMintJobs()[0].jobId; + + viewMintedCardJob(jobId); + + assert.equal(getCardMintJobs().length, 1); + assert.equal(getCardViewerState(), null); + resolveMint(CARD); // avoid a dangling promise + }); + + it("dismissCardMintJob removes only the named job", async () => { + await runCardMintJob(INPUT, () => Promise.reject(new Error("a"))); + await runCardMintJob({ agentId: "agent-2", agentName: "Wren" }, () => + Promise.reject(new Error("b")), + ); + const [first, second] = getCardMintJobs(); + + dismissCardMintJob(first.jobId); + + const jobs = getCardMintJobs(); + assert.equal(jobs.length, 1); + assert.equal(jobs[0].jobId, second.jobId); + }); + + it("openCardViewer/closeCardViewer manage archive views without remint", () => { + openCardViewer({ card: CARD, agentName: "Eva", remint: null }); + assert.equal(getCardViewerState()?.remint, null); + closeCardViewer(); + assert.equal(getCardViewerState(), null); + }); + + it("gallery open flag toggles and notifies subscribers", () => { + let notified = 0; + const unsubscribe = subscribeCardMintStore(() => { + notified += 1; + }); + setCardGalleryOpen(true); + assert.equal(getCardGalleryOpen(), true); + setCardGalleryOpen(true); // no-op must not notify + setCardGalleryOpen(false); + assert.equal(getCardGalleryOpen(), false); + assert.equal(notified, 2); + unsubscribe(); + }); + + it("concurrent jobs keep distinct snapshots (referential updates)", async () => { + const before = getCardMintJobs(); + await runCardMintJob(INPUT, () => Promise.resolve(CARD)); + const after = getCardMintJobs(); + assert.notEqual(before, after, "snapshot identity must change on update"); + }); +}); diff --git a/desktop/src/features/agents/cardMintStore.ts b/desktop/src/features/agents/cardMintStore.ts new file mode 100644 index 0000000000..0e4746d445 --- /dev/null +++ b/desktop/src/features/agents/cardMintStore.ts @@ -0,0 +1,227 @@ +import * as React from "react"; +import { toast } from "sonner"; + +import { + mintAgentCard, + NO_OPENAI_KEY_PREFIX, + type MintedAgentCard, + type SnapshotMemoryLevel, +} from "@/shared/api/tauriPersonas"; + +/** + * Module store for agent-card mints (`useSyncExternalStore` pattern, same as + * `avatarPresentationStore`). + * + * A mint is one stateless ~2–3 minute Rust call. Owning the in-flight promise + * here — instead of inside the mint dialog — is what makes the dialog + * non-blocking: it dispatches and closes, the composer activity rail shows a + * live "Minting card…" chip, and completion lands as a clickable toast plus a + * persistent "card ready" chip, none of which need the dialog mounted. + */ + +/** Everything needed to run (or re-run) one mint. */ +export type CardMintInput = { + agentId: string; + agentName: string; + styleNotes?: string; + lock?: boolean; + /** Memory to embed in the card's snapshot. Omitted = "none". */ + memoryLevel?: SnapshotMemoryLevel; +}; + +export type CardMintJob = { + jobId: string; + input: CardMintInput; + phase: "minting" | "done" | "error"; + /** Populated when phase is "done". */ + card: MintedAgentCard | null; + /** Populated when phase is "error". */ + error: string | null; + startedAt: number; +}; + +/** Card content shown by the global viewer dialog. */ +export type CardViewerState = { + card: MintedAgentCard; + agentName: string; + /** + * Present when the card can be rerolled (fresh mints carry their input; + * archive views do not — the original style notes are gone). + */ + remint: CardMintInput | null; + /** + * Monotonic per-open sequence, assigned by the store. The viewer keys its + * content on this so switching cards remounts (resetting recipients/menu + * state) — card bytes can't serve as the key because every card PNG shares + * the same header prefix and dimensions. + */ + viewerSeq: number; +}; + +let jobs: CardMintJob[] = []; +let viewer: CardViewerState | null = null; +let galleryOpen = false; +const listeners = new Set<() => void>(); +let nextJobId = 1; +let nextViewerSeq = 1; + +function emitChange(): void { + for (const listener of listeners) listener(); +} + +function updateJob(jobId: string, patch: Partial): void { + jobs = jobs.map((job) => (job.jobId === jobId ? { ...job, ...patch } : job)); + emitChange(); +} + +/** + * Run one mint as a background job. `mintFn` is injectable for tests; the + * public `startCardMint` binds the real Tauri command. + */ +export async function runCardMintJob( + input: CardMintInput, + mintFn: ( + id: string, + styleNotes?: string, + lock?: boolean, + memoryLevel?: SnapshotMemoryLevel, + ) => Promise, +): Promise { + const jobId = `card-mint-${nextJobId++}`; + jobs = [ + ...jobs, + { + jobId, + input, + phase: "minting", + card: null, + error: null, + startedAt: Date.now(), + }, + ]; + emitChange(); + + try { + const card = await mintFn( + input.agentId, + input.styleNotes, + input.lock, + input.memoryLevel, + ); + updateJob(jobId, { phase: "done", card }); + toast.success(`${input.agentName}'s card is ready`, { + action: { + label: "View card", + onClick: () => viewMintedCardJob(jobId), + }, + duration: 10_000, + }); + } catch (error) { + let message = error instanceof Error ? error.message : "Card mint failed."; + if (message.startsWith(NO_OPENAI_KEY_PREFIX)) { + // The dialog pre-checks the key, so this only happens when the key was + // removed between dialog-open and mint. The dialog's key-setup panel is + // long gone — surface a plain instruction instead of the wire prefix. + message = message.slice(NO_OPENAI_KEY_PREFIX.length).trim(); + } + updateJob(jobId, { phase: "error", error: message }); + toast.error(`Minting ${input.agentName}'s card failed`, { + description: message, + }); + } +} + +/** Start a mint in the background. Fire-and-forget; state flows via the store. */ +export function startCardMint(input: CardMintInput): void { + void runCardMintJob(input, mintAgentCard); +} + +/** Open the finished card of a job in the viewer and clear its rail chip. */ +export function viewMintedCardJob(jobId: string): void { + const job = jobs.find((candidate) => candidate.jobId === jobId); + if (job?.phase !== "done" || !job.card) return; + viewer = { + card: job.card, + agentName: job.input.agentName, + remint: job.input, + viewerSeq: nextViewerSeq++, + }; + jobs = jobs.filter((candidate) => candidate.jobId !== jobId); + emitChange(); +} + +/** Remove a job chip (used for error dismissal). */ +export function dismissCardMintJob(jobId: string): void { + const next = jobs.filter((candidate) => candidate.jobId !== jobId); + if (next.length === jobs.length) return; + jobs = next; + emitChange(); +} + +/** Open the viewer on an arbitrary card (e.g. one loaded from the archive). */ +export function openCardViewer( + state: Omit, +): void { + viewer = { ...state, viewerSeq: nextViewerSeq++ }; + emitChange(); +} + +export function closeCardViewer(): void { + if (!viewer) return; + viewer = null; + emitChange(); +} + +export function setCardGalleryOpen(open: boolean): void { + if (galleryOpen === open) return; + galleryOpen = open; + emitChange(); +} + +export function resetCardMintStore(): void { + jobs = []; + viewer = null; + galleryOpen = false; + emitChange(); +} + +export function getCardMintJobs(): CardMintJob[] { + return jobs; +} + +export function getCardViewerState(): CardViewerState | null { + return viewer; +} + +export function getCardGalleryOpen(): boolean { + return galleryOpen; +} + +export function subscribeCardMintStore(listener: () => void): () => void { + listeners.add(listener); + return () => listeners.delete(listener); +} + +export function useCardMintJobs(): CardMintJob[] { + return React.useSyncExternalStore( + subscribeCardMintStore, + getCardMintJobs, + getCardMintJobs, + ); +} + +export function useCardViewerState(): CardViewerState | null { + return React.useSyncExternalStore( + subscribeCardMintStore, + getCardViewerState, + getCardViewerState, + ); +} + +export function useCardGalleryOpen(): boolean { + return React.useSyncExternalStore( + subscribeCardMintStore, + getCardGalleryOpen, + getCardGalleryOpen, + ); +} diff --git a/desktop/src/features/agents/lib/agentAccessWarning.test.mjs b/desktop/src/features/agents/lib/agentAccessWarning.test.mjs new file mode 100644 index 0000000000..0a3d13ce7b --- /dev/null +++ b/desktop/src/features/agents/lib/agentAccessWarning.test.mjs @@ -0,0 +1,89 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + agentAccessWarningText, + runLocationForBackend, + runLocationForRunOn, +} from "./agentAccessWarning.ts"; + +test("only the modes that share access warn", () => { + assert.equal(agentAccessWarningText("owner-only", "local"), null); + assert.ok(agentAccessWarningText("anyone", "local")); + assert.ok(agentAccessWarningText("allowlist", "local")); +}); + +test("a local agent names this computer and what is reachable on it", () => { + assert.equal( + agentAccessWarningText("anyone", "local"), + "Anyone can use this agent to access your computer, including files, accounts, and connected tools.", + ); + assert.equal( + agentAccessWarningText("allowlist", "local"), + "Selected people can use this agent to access your computer, including files, accounts, and connected tools.", + ); +}); + +test("a provider-backed agent names the server, and not the owner's files", () => { + // A remote host's files aren't the owner's to describe, so the tail narrows + // to the accounts and tools provisioned there. + assert.equal( + agentAccessWarningText("anyone", "remote"), + "Anyone can use this agent to access the server it runs on, including any accounts and tools available there.", + ); + assert.equal( + agentAccessWarningText("allowlist", "remote"), + "Selected people can use this agent to access the server it runs on, including any accounts and tools available there.", + ); + assert.doesNotMatch( + agentAccessWarningText("anyone", "remote"), + /your computer/, + ); +}); + +test("an unknown run location reads as local, not as a hedge", () => { + // "computer or server" names a concept most owners have never been shown: + // the Run on selector only renders when a buzz-backend-* provider exists. + for (const unknown of [undefined, null]) { + assert.equal( + agentAccessWarningText("anyone", unknown), + "Anyone can use this agent to access your computer, including files, accounts, and connected tools.", + ); + } +}); + +test("every variant leads with the audience and stays jargon-free", () => { + for (const mode of ["anyone", "allowlist"]) { + for (const runLocation of [null, "local", "remote"]) { + const text = agentAccessWarningText(mode, runLocation); + assert.match( + text, + /^(Anyone|Selected people) can use this agent to access/, + ); + assert.doesNotMatch(text, /respond-to|allowlist|pubkey|Nostr|harness/i); + } + } +}); + +test("runLocationForBackend maps the backend union", () => { + assert.equal(runLocationForBackend({ type: "local" }), "local"); + assert.equal( + runLocationForBackend({ type: "provider", id: "blox", config: {} }), + "remote", + ); + assert.equal(runLocationForBackend(null), null); + assert.equal(runLocationForBackend(undefined), null); +}); + +test("runLocationForRunOn treats a provider id as remote", () => { + assert.equal(runLocationForRunOn("local"), "local"); + assert.equal(runLocationForRunOn("blox"), "remote"); +}); + +test("runLocationForRunOn treats a blank value as unknown", () => { + // `runOn` is typed `"local" | string`, so a blank must not read as a + // provider id and produce the server wording. + assert.equal(runLocationForRunOn(""), null); + assert.equal(runLocationForRunOn(null), null); + assert.equal(runLocationForRunOn(undefined), null); +}); diff --git a/desktop/src/features/agents/lib/agentAccessWarning.ts b/desktop/src/features/agents/lib/agentAccessWarning.ts new file mode 100644 index 0000000000..d98058f5bf --- /dev/null +++ b/desktop/src/features/agents/lib/agentAccessWarning.ts @@ -0,0 +1,63 @@ +import type { ManagedAgentBackend, RespondToMode } from "@/shared/api/types"; + +/** + * Where an agent's process runs, as far as the calling surface can tell. + * + * Deliberately coarser than `ManagedAgentBackend`: the warning copy only needs + * to know "this machine" vs "somewhere else", so surfaces resolve their own + * backend shape down to this before handing it over. `null` means the surface + * genuinely cannot tell — see `agentAccessWarningText` for how that is + * treated. + */ +export type AgentRunLocation = "local" | "remote"; + +/** Resolve a running agent's backend record. `null` when the backend is unknown. */ +export function runLocationForBackend( + backend: ManagedAgentBackend | null | undefined, +): AgentRunLocation | null { + if (!backend) return null; + return backend.type === "local" ? "local" : "remote"; +} + +/** + * Resolve the create flow's `WhereToRunDraft.runOn`, which is `"local"` or a + * discovered provider id. An empty string is treated as unknown rather than as + * a provider, since `runOn` is typed `"local" | string`. + */ +export function runLocationForRunOn( + runOn: string | null | undefined, +): AgentRunLocation | null { + if (!runOn) return null; + return runOn === "local" ? "local" : "remote"; +} + +/** + * Copy for the shared-access warning in the respond-to field, or `null` for + * modes that share nothing. + * + * Both `anyone` and `allowlist` hand the host's access to someone other than + * the owner, so both warn; only the audience phrase differs. + * + * An unknown run location falls back to the same "your computer" wording as + * `local` rather than hedging with "computer or server". A remote host is only + * reachable when a `buzz-backend-*` provider binary is installed — without one + * `WhereToRunSection`'s "Run on" selector never renders and every agent is + * local — so hedging would name a concept most owners have never been shown. + * When it *is* remote the owner picked that host from the selector + * deliberately, so naming a server is meaningful there. + */ +export function agentAccessWarningText( + mode: RespondToMode, + runLocation?: AgentRunLocation | null, +): string | null { + if (mode !== "anyone" && mode !== "allowlist") return null; + const audience = mode === "anyone" ? "Anyone" : "Selected people"; + // The two locations differ in more than the noun: a local agent reaches the + // owner's own files, while a remote host's files aren't theirs to describe — + // only the accounts and tools provisioned there. + const target = + runLocation === "remote" + ? "the server it runs on, including any accounts and tools available there" + : "your computer, including files, accounts, and connected tools"; + return `${audience} can use this agent to access ${target}.`; +} diff --git a/desktop/src/features/agents/lib/agentCardGalleryState.test.mjs b/desktop/src/features/agents/lib/agentCardGalleryState.test.mjs new file mode 100644 index 0000000000..ebc7caa8f4 --- /dev/null +++ b/desktop/src/features/agents/lib/agentCardGalleryState.test.mjs @@ -0,0 +1,96 @@ +import assert from "node:assert/strict"; +import { describe, it } from "node:test"; + +// Pins the gallery's query-state boundary: a rejected `list_agent_cards` +// MUST surface as an error state — never as "No cards yet". `data` falls +// back to `[]` at the call site, so an emptiness-first branch would render +// a permissions/IPC failure as a false empty archive of paid cards. + +import { agentCardGalleryViewState } from "./agentCardGalleryState.ts"; + +describe("agentCardGalleryViewState", () => { + it("maps a rejected query to an error state with the message, not empty", () => { + const state = agentCardGalleryViewState({ + isLoading: false, + isError: true, + error: new Error("cards directory unreadable"), + data: undefined, + }); + assert.deepEqual(state, { + kind: "error", + message: "cards directory unreadable", + }); + }); + + it("error wins even when data has fallen back to an empty array", () => { + // react-query keeps `data: undefined` on first failure, but the call + // site coalesces to []. Guard the exact shape that caused the bug. + const state = agentCardGalleryViewState({ + isLoading: false, + isError: true, + error: new Error("ipc failure"), + data: [], + }); + assert.equal(state.kind, "error"); + }); + + it("stringifies non-Error rejections", () => { + const state = agentCardGalleryViewState({ + isLoading: false, + isError: true, + error: "permission denied", + data: undefined, + }); + assert.deepEqual(state, { kind: "error", message: "permission denied" }); + }); + + it("null/undefined rejections still produce a message", () => { + const state = agentCardGalleryViewState({ + isLoading: false, + isError: true, + error: null, + data: undefined, + }); + assert.deepEqual(state, { kind: "error", message: "unknown error" }); + }); + + it("loading while fetching", () => { + const state = agentCardGalleryViewState({ + isLoading: true, + isError: false, + error: null, + data: undefined, + }); + assert.deepEqual(state, { kind: "loading" }); + }); + + it("no data yet (not loading, not error) stays loading, not empty", () => { + const state = agentCardGalleryViewState({ + isLoading: false, + isError: false, + error: null, + data: undefined, + }); + assert.deepEqual(state, { kind: "loading" }); + }); + + it("resolved and empty is the only path to the empty state", () => { + const state = agentCardGalleryViewState({ + isLoading: false, + isError: false, + error: null, + data: [], + }); + assert.deepEqual(state, { kind: "empty" }); + }); + + it("resolved with cards renders cards", () => { + const state = agentCardGalleryViewState({ + isLoading: false, + isError: false, + error: null, + data: [{ storedFileName: "a.png" }], + }); + assert.deepEqual(state, { kind: "cards" }); + }); +}); diff --git a/desktop/src/features/agents/lib/agentCardGalleryState.ts b/desktop/src/features/agents/lib/agentCardGalleryState.ts new file mode 100644 index 0000000000..8f6cf4269e --- /dev/null +++ b/desktop/src/features/agents/lib/agentCardGalleryState.ts @@ -0,0 +1,35 @@ +/** + * View-state selection for the minted-card gallery. + * + * Kept as a pure function so the error boundary is testable without a DOM: + * a rejected `list_agent_cards` query MUST surface as an error state, never + * as an empty archive — "No cards yet" on a permissions/IPC failure tells + * someone their paid, persisted cards do not exist. + */ + +export type AgentCardGalleryViewState = + | { kind: "loading" } + | { kind: "error"; message: string } + | { kind: "empty" } + | { kind: "cards" }; + +export function agentCardGalleryViewState(query: { + isLoading: boolean; + isError: boolean; + error: unknown; + data: readonly unknown[] | undefined; +}): AgentCardGalleryViewState { + // Error wins over everything: `data` falls back to `[]` at the call site, + // so checking emptiness first would render a failure as a false empty. + if (query.isError) { + const message = + query.error instanceof Error + ? query.error.message + : String(query.error ?? "unknown error"); + return { kind: "error", message }; + } + if (query.isLoading || query.data === undefined) { + return { kind: "loading" }; + } + return query.data.length === 0 ? { kind: "empty" } : { kind: "cards" }; +} diff --git a/desktop/src/features/agents/lib/personaCatalogRelay.test.mjs b/desktop/src/features/agents/lib/personaCatalogRelay.test.mjs index 24f6959b1c..fbaf1f5274 100644 --- a/desktop/src/features/agents/lib/personaCatalogRelay.test.mjs +++ b/desktop/src/features/agents/lib/personaCatalogRelay.test.mjs @@ -205,6 +205,30 @@ test("test_non_svg_data_avatar_is_rejected", () => { assert.equal(catalogAvatarUrl("data:image/png,%89PNG"), null); }); +test("test_legacy_inline_raster_avatar_survives_the_catalog", () => { + for (const mime of ["png", "jpeg", "gif", "webp"]) { + const avatar = `data:image/${mime};base64,iVBORw0KGgo=`; + assert.equal(catalogAvatarUrl(avatar), avatar); + } +}); + +test("test_inline_raster_avatar_rejects_unbounded_or_malformed_payloads", () => { + const prefix = "data:image/png;base64,"; + const payloadLength = 256 * 1_024 - prefix.length; + const validPayloadLength = payloadLength - (payloadLength % 4); + const withinCap = `${prefix}${"a".repeat(validPayloadLength - 2)}==`; + assert.ok(withinCap.length <= 256 * 1_024); + assert.equal(catalogAvatarUrl(withinCap), withinCap); + assert.equal( + catalogAvatarUrl( + `${withinCap}${"a".repeat(256 * 1_024 - withinCap.length + 1)}`, + ), + null, + ); + assert.equal(catalogAvatarUrl("data:image/png;base64,not base64"), null); + assert.equal(catalogAvatarUrl("data:image/bmp;base64,aA=="), null); +}); + test("test_oversized_inline_svg_avatar_is_rejected", () => { const withinCap = `data:image/svg+xml,${"a".repeat(8_192 - "data:image/svg+xml,".length)}`; assert.equal(withinCap.length, 8_192); diff --git a/desktop/src/features/agents/lib/personaCatalogRelay.ts b/desktop/src/features/agents/lib/personaCatalogRelay.ts index c85a976ba6..02c3f8e202 100644 --- a/desktop/src/features/agents/lib/personaCatalogRelay.ts +++ b/desktop/src/features/agents/lib/personaCatalogRelay.ts @@ -93,6 +93,16 @@ function isSafeHttpUrl(value: unknown): value is string { const INLINE_SVG_AVATAR_PREFIX = "data:image/svg+xml,"; const MAX_INLINE_SVG_AVATAR_LENGTH = 8_192; +/** + * Shared persona heads can carry an uploaded avatar as an inline raster. Keep + * those self-contained images renderable without accepting arbitrary `data:` + * URLs: only the raster MIME types browsers decode in ``, strict base64 + * shape, and a bound no larger than the relay's event-content ceiling. + */ +const MAX_INLINE_RASTER_AVATAR_LENGTH = 256 * 1_024; +const INLINE_RASTER_AVATAR_RE = + /^data:image\/(?:png|jpeg|gif|webp);base64,([A-Za-z0-9+/]+={0,2})$/u; + function isInlineSvgAvatar(value: unknown): value is string { return ( typeof value === "string" && @@ -101,6 +111,17 @@ function isInlineSvgAvatar(value: unknown): value is string { ); } +function isInlineRasterAvatar(value: unknown): value is string { + if ( + typeof value !== "string" || + value.length > MAX_INLINE_RASTER_AVATAR_LENGTH + ) { + return false; + } + const match = INLINE_RASTER_AVATAR_RE.exec(value); + return match !== null && (match[1]?.length ?? 0) % 4 === 0; +} + function optionalString(value: unknown): string | null { return typeof value === "string" && value.trim().length > 0 ? value : null; } @@ -121,7 +142,9 @@ function parsePersonaContent(event: RelayEvent): CatalogAgentProjection | null { } const avatarUrl = - isSafeHttpUrl(parsed.avatar_url) || isInlineSvgAvatar(parsed.avatar_url) + isSafeHttpUrl(parsed.avatar_url) || + isInlineSvgAvatar(parsed.avatar_url) || + isInlineRasterAvatar(parsed.avatar_url) ? parsed.avatar_url : null; const namePool = Array.isArray(parsed.name_pool) diff --git a/desktop/src/features/agents/lib/useInstallOutputLine.test.mjs b/desktop/src/features/agents/lib/useInstallOutputLine.test.mjs new file mode 100644 index 0000000000..d65583e00d --- /dev/null +++ b/desktop/src/features/agents/lib/useInstallOutputLine.test.mjs @@ -0,0 +1,82 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { nextInstallOutputLine } from "./useInstallOutputLine.ts"; + +function event(runtimeId, seq, line) { + return { runtime_id: runtimeId, seq, line }; +} + +test("nextInstallOutputLine: adopts the first line for the watched runtime", () => { + assert.deepEqual( + nextInstallOutputLine(null, event("goose", 0, "downloading"), "goose"), + { seq: 0, line: "downloading" }, + ); +}); + +test("nextInstallOutputLine: a later line replaces the current one", () => { + const current = { seq: 4, line: "downloading" }; + + assert.deepEqual( + nextInstallOutputLine(current, event("goose", 5, "unpacking"), "goose"), + { seq: 5, line: "unpacking" }, + ); +}); + +test("nextInstallOutputLine: ignores a line from another runtime", () => { + const current = { seq: 1, line: "downloading" }; + + assert.equal( + nextInstallOutputLine(current, event("codex", 2, "other work"), "goose"), + current, + ); +}); + +test("nextInstallOutputLine: ignores an out-of-order line", () => { + const current = { seq: 7, line: "retrying" }; + + assert.equal( + nextInstallOutputLine(current, event("goose", 6, "stale line"), "goose"), + current, + ); +}); + +test("nextInstallOutputLine: ignores a replay of the current sequence number", () => { + const current = { seq: 7, line: "retrying" }; + + assert.equal( + nextInstallOutputLine(current, event("goose", 7, "duplicate"), "goose"), + current, + ); +}); + +test("nextInstallOutputLine: a null line clears the display", () => { + const current = { seq: 3, line: "download failed" }; + + assert.deepEqual( + nextInstallOutputLine(current, event("goose", 4, null), "goose"), + { seq: 4, line: null }, + ); +}); + +test("nextInstallOutputLine: a later step's first line is adopted after a higher attempt", () => { + // The seq is install-wide: step 2 attempt 1 always follows step 1 attempt 2, + // which is exactly what an attempt-keyed comparison got wrong. + const current = { seq: 9, line: "step one, attempt two" }; + + assert.deepEqual( + nextInstallOutputLine( + current, + event("goose", 10, "step two, attempt one"), + "goose", + ), + { seq: 10, line: "step two, attempt one" }, + ); +}); + +test("nextInstallOutputLine: a first event mid-install is adopted", () => { + assert.deepEqual( + nextInstallOutputLine(null, event("goose", 42, "downloading"), "goose"), + { seq: 42, line: "downloading" }, + ); +}); diff --git a/desktop/src/features/agents/lib/useInstallOutputLine.ts b/desktop/src/features/agents/lib/useInstallOutputLine.ts new file mode 100644 index 0000000000..9f50843fc7 --- /dev/null +++ b/desktop/src/features/agents/lib/useInstallOutputLine.ts @@ -0,0 +1,108 @@ +import * as React from "react"; +import { listen } from "@tauri-apps/api/event"; + +/** Mirror of the Rust `InstallOutputEvent` payload (install_report.rs). */ +export type InstallOutputEvent = { + runtime_id: string; + /** Monotonic across the whole install, not per step or per attempt. */ + seq: number; + /** Null is the start signal: clear the displayed line now. */ + line: string | null; +}; + +/** The line being shown, and the sequence number that produced it. */ +export type InstallOutputState = { + seq: number; + line: string | null; +}; + +/** + * Fold one event into the displayed line. + * + * Events from another runtime are ignored — every install card listens to the + * same channel. So is an out-of-order event: emission is monotonic in `seq`, so + * a lower one has already been superseded. That matters at a retry boundary, + * where a line emitted just as the next attempt starts would otherwise sit + * under the spinner showing the failure the user already had. + * + * The ordering key is the install-wide `seq` rather than the attempt number, + * which restarts at 1 for every step: keyed on attempt, a step that succeeded on + * attempt 2 would make the next step's attempt-1 output look stale and freeze + * the display for the rest of the install. + */ +export function nextInstallOutputLine( + current: InstallOutputState | null, + event: InstallOutputEvent, + runtimeId: string, +): InstallOutputState | null { + if (event.runtime_id !== runtimeId) return current; + if (current && event.seq <= current.seq) return current; + return { seq: event.seq, line: event.line }; +} + +/** + * The install command's most recent output line for `runtimeId`, or null when + * nothing is being shown — the install is not running, nothing has printed yet, + * or the backend cleared the line because a new attempt is starting. + * + * An install runs for up to 15 minutes with no other feedback than a spinner; + * this turns that wait into observable progress. The backend throttles + * emission, so this re-renders a few times a second at most. + * + * Pass `isInstalling` so the line clears when the install settles — a finished + * install must not leave its last line under a fresh Install button. + */ +export function useInstallOutputLine( + runtimeId: string, + isInstalling: boolean, +): string | null { + const [state, setState] = React.useState(null); + + // Subscribed for this runtime's whole lifetime, not just while installing. + // The install command is invoked from the click handler, so the backend can + // emit the attempt-start clear and the first line before React commits + // `isInstalling` — and there is no replay, so a subscription that waited for + // that commit would lose those events permanently. A fast command's entire + // output is exactly what fits in that window. + React.useEffect(() => { + let cancelled = false; + let unlisten: (() => void) | null = null; + (async () => { + try { + const stop = await listen( + "acp-install-output", + (event) => { + if (cancelled) return; + setState((current) => + nextInstallOutputLine(current, event.payload, runtimeId), + ); + }, + ); + if (cancelled) { + stop(); + } else { + unlisten = stop; + } + } catch { + // Event system unavailable (web/e2e) — the spinner shows alone. + } + })(); + return () => { + cancelled = true; + unlisten?.(); + }; + }, [runtimeId]); + + // `seq` is monotonic within one install and restarts at 0 for the next, so + // state must not outlive the run that produced it: a retained higher `seq` + // would make every event of the following install look superseded. Settling + // is the run boundary, so it is where the ordering key resets. + React.useEffect(() => { + if (!isInstalling) setState(null); + }, [isInstalling]); + + // Events that arrive after the install settles — a drain flushing its last + // line — must not reappear under a fresh Install button, so the line is + // reported only while the install is running. + return (isInstalling ? state?.line : null) ?? null; +} diff --git a/desktop/src/features/agents/ui/AgentCardMintDialog.tsx b/desktop/src/features/agents/ui/AgentCardMintDialog.tsx new file mode 100644 index 0000000000..219de4aefa --- /dev/null +++ b/desktop/src/features/agents/ui/AgentCardMintDialog.tsx @@ -0,0 +1,364 @@ +import * as React from "react"; +import { + AlertCircle, + Brain, + ExternalLink, + GalleryVerticalEnd, + KeyRound, + Lock, + Sparkles, +} from "lucide-react"; +import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query"; +import { openUrl } from "@tauri-apps/plugin-opener"; +import { toast } from "sonner"; + +import { + setCardGalleryOpen, + startCardMint, +} from "@/features/agents/cardMintStore"; +import { globalAgentConfigQueryKey } from "@/features/agents/useGlobalAgentConfig"; +import { + cardMintKeyStatus, + cardMintSaveOpenaiKey, + type SnapshotMemoryLevel, +} from "@/shared/api/tauriPersonas"; +import { Button } from "@/shared/ui/button"; +import { + Dialog, + DialogContent, + DialogDescription, + DialogHeader, + DialogTitle, +} from "@/shared/ui/dialog"; +import { Input } from "@/shared/ui/input"; +import { Switch } from "@/shared/ui/switch"; +import { Textarea } from "@/shared/ui/textarea"; +import { SnapshotOptionMenu } from "./SnapshotOptionMenu"; + +const OPENAI_KEYS_URL = "https://platform.openai.com/api-keys"; + +/** Same three levels as snapshot export; "Agent only" is the safe default. */ +const MEMORY_LEVELS: { value: SnapshotMemoryLevel; label: string }[] = [ + { value: "none", label: "Agent only" }, + { value: "core", label: "Agent + core memory" }, + { value: "everything", label: "Agent + all memories" }, +]; + +/** + * The free alternative, as an action: ordinary snapshot export shares the + * same importable agent without card art or API spend. Rendered in both the + * key-setup panel and the normal pre-mint form (the cost disclosure and its + * escape hatch must be visible BEFORE any spend, not only during onboarding). + */ +function FreeSharePathRow({ + disabled, + onExportInstead, +}: { + disabled: boolean; + onExportInstead?: () => void; +}) { + return ( +
+

+ Don’t want to spend money? Ordinary export shares the same importable + agent — free, just without the card art. +

+ {onExportInstead ? ( + + ) : null} +
+ ); +} + +/** + * Mint-a-trading-card dialog — the pre-mint half only: key setup (when + * needed) → optional style notes → "Mint card". Minting itself runs as a + * background job in `cardMintStore`: this dialog dispatches and closes, the + * composer activity rail shows live status, and the finished card opens in + * the global `AgentCardViewerDialog` (preview, reroll, save, share). + * + * The saved PNG carries the agent's `buzz_agent_snapshot` chunk, so sharing + * the card shares an importable agent (fresh identity, never secrets; memory + * only when the owner opts in below — plaintext unless the card is locked). + * All snapshot construction and verification happens in Rust. + */ +export function AgentCardMintDialog({ + agentId, + agentName, + canLock, + onExportInstead, + onOpenChange, +}: { + /** Instance pubkey or definition slug — same resolution as snapshot export. */ + agentId: string; + agentName: string; + /** + * True when the agent has a linked instance (a keypair to lock to). + * Locking is disabled — with an explanation — for bare definitions. + */ + canLock: boolean; + /** + * Free alternative: close this dialog and open the ordinary snapshot + * export flow (no API spend). Omitted = the action is not rendered. + */ + onExportInstead?: () => void; + onOpenChange: (open: boolean) => void; +}) { + const [styleNotes, setStyleNotes] = React.useState(""); + const [lockCard, setLockCard] = React.useState(false); + const [memoryLevel, setMemoryLevel] = + React.useState("none"); + const [keyDraft, setKeyDraft] = React.useState(""); + + const queryClient = useQueryClient(); + + const effectiveLock = canLock && lockCard; + // Embedded memory is plaintext in an unlocked card — and unlocked cards + // are meant to be shared. A locked card encrypts the whole manifest to the + // (owner, agent) pair, so the plaintext warning would be false there. + const showMemoryWarning = memoryLevel !== "none" && !effectiveLock; + + // Whether a key already resolves through the agent's env layering. While + // unknown (loading/error) we show the normal mint form — the mint itself + // still fails cleanly if no key exists. + const keyStatusQuery = useQuery({ + queryKey: ["cardMintKeyStatus", agentId], + queryFn: () => cardMintKeyStatus(agentId), + }); + const needsKey = keyStatusQuery.data === false; + + // Save the pasted key into the global Agent Defaults env — the same single + // source of truth every agent inherits. Narrow Rust seam: validated + // single-key merge, never restarts running agents (the mint re-reads + // config per call, so no restart is needed for minting). + const saveKeyMutation = useMutation({ + mutationFn: (key: string) => cardMintSaveOpenaiKey(key), + onSuccess: () => { + queryClient.setQueryData(["cardMintKeyStatus", agentId], true); + // The Agent Defaults editor caches the whole config — refetch it so a + // later-opened settings view shows the key we just wrote. + void queryClient.invalidateQueries({ + queryKey: globalAgentConfigQueryKey, + }); + setKeyDraft(""); + toast.success( + "API key saved to your agent defaults. Running agents pick it up on their next restart.", + ); + }, + onError: (error) => + toast.error(typeof error === "string" ? error : "Couldn't save the key."), + }); + + function beginMint() { + // Dispatch to the background store and close: the composer rail shows + // "Minting card…" and the completion toast opens the viewer. + startCardMint({ + agentId, + agentName, + styleNotes: styleNotes.trim() || undefined, + lock: effectiveLock, + memoryLevel: canLock ? memoryLevel : "none", + }); + onOpenChange(false); + } + + return ( + + + + + + {`Create ${agentName}'s card`} + + + Mint a collectible trading card that doubles as a shareable, + importable copy of this agent. + + + + {needsKey ? ( +
+
+ + + One-time setup: OpenAI API key + +

+ Minting a card costs money — it generates the art and card text + through the OpenAI API with your key (typically well under a + dollar per mint, billed by OpenAI). The key is saved to your + agent defaults, so you only do this once. +

+ + setKeyDraft(e.target.value)} + placeholder="sk-…" + type="password" + value={keyDraft} + /> +
+ +
+ +
+
+ ) : ( +
+
+