feat(desktop): workspace-scoped agent definition store - #4485
Draft
wpfleger96 wants to merge 83 commits into
Draft
feat(desktop): workspace-scoped agent definition store#4485wpfleger96 wants to merge 83 commits into
wpfleger96 wants to merge 83 commits into
Conversation
Introduce the `WorkspaceAgentScope` type and the scaffolding for a
four-stage workspace transition state machine (Phase 1 foundation).
## Scope model (managed_agents/scope.rs)
- `WorkspaceAgentScope { scope_id, relay_url, owner_pubkey, definitions_dir,
generation }` — the single scope authority for a workspace's agent
definition store. Immutable; callers capture one scope at operation
entry and thread it through `_at(scope)` APIs.
- `derive_scope_id(relay_url, owner_pubkey)` — the canonical sha256
derivation, byte-identical to the retention DB derivation. Both
subsystems now go through one shared helper so "same scope" can never
disagree between definitions and retention.
- `next_scope_generation()` / `current_scope_generation()` — global
monotonic counter incremented on every scope change or identity-import
clear. Long-running operations read at entry and revalidate before
commit; a stale commit aborts.
- `WorkspaceApplyResult { applied, degraded }` — typed result for the
four-stage transition machine (prepare / drain / commit / post-commit).
- Scoped layout: `agents/scopes/<scope_id>/{managed-agents.json,
teams.json, global-agent-config.json}`.
## AppState additions (app_state.rs)
- `identity_mutation: AsyncMutex<()>` (was `Mutex<()>`) — Layer 1 async
lock; callers may `.await` while holding it. Converted so the workspace
transition machine can hold it across awaits without blocking the
executor.
- `workspace_transition: AsyncMutex<()>` — serializes workspace
transitions (`apply_workspace` and live identity import). Lock order:
identity_mutation → workspace_transition → Mesh rearm → mesh_llm_runtime.
- `active_agent_scope: Mutex<Option<WorkspaceAgentScope>>` — `None` from
boot until the first successful `apply_workspace`. Every agent command
fails closed on `None`; there is NO fallback to the legacy unscoped root.
## Retention parity (retention.rs)
- `scoped_retention_db_path` now delegates to `derive_scope_id` instead
of inlining its own sha256, making the hash provably identical.
- `scope_for_arrival` / `arrival_retention_scope` extended to match on
both relay AND owner pubkey. An in-flight old-owner event on the same
relay can no longer land in the new owner's active store after an
identity switch.
## Inbound reconcile (commands/personas/inbound.rs)
- Both `arrival_retention_scope` call sites pass the event's pubkey as
the owner dimension, closing the identity-switch cross-contamination gap.
## Caller updates
- `identity.rs`: three `identity_mutation.lock().map_err()` callers
converted to `.blocking_lock()` (Tokio async mutex's sync-context
variant, safe from `spawn_blocking` threads).
- `identity_key_backup_tests.rs`: test thread mirror updated to match.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…chokepoints ## Scoped storage APIs Add path-based `_at(definitions_dir)` variants alongside every `app: &AppHandle` storage chokepoint. These are the primary API for long-running operations that have captured a `WorkspaceAgentScope` at their entry point: - `storage.rs`: `load_agent_store_at`, `load_managed_agents_at`, `load_agent_definitions_at`, `save_managed_agents_at`, `save_agent_definitions_at`, `managed_agents_store_path_at`; the internal `write_agent_store` now delegates to `write_agent_store_to_path` which is shared with the new scoped write path. - `teams.rs`: `teams_store_path_at`, `load_teams_at`, `save_teams_at`. - `global_config/mod.rs`: `global_config_path_at`, `load_global_agent_config_at`, `save_global_agent_config_at`; the load path is factored into `load_global_agent_config_from_path`. ## AppState scope helpers - `capture_active_scope()` — snapshot of current `Option<WorkspaceAgentScope>`. Callers crossing `.await` or thread boundaries capture at entry. - `commit_active_scope(scope)` — infallible commit-stage setter (Layer 2). - `clear_active_scope()` — clear + generation bump for identity import drain and prepare-stage rollback. ## Event-sync retarget `run_event_sync`, `spawn_event_sync`, `migrate_personas_to_events`, `migrate_teams_to_events`, and `reconcile_agents_to_events` all gain a `definitions_dir: &Path` / `PathBuf` parameter. They no longer resolve the base dir from `AppHandle` — the caller passes the scoped definitions dir directly, closing the bypass that read from the legacy unscoped root. ## apply_workspace scope commit After applying relay + keys, `apply_workspace` derives a `WorkspaceAgentScope` from the effective (relay, owner) pair and commits it via `commit_active_scope`. The immediately following `spawn_event_sync` call reads the committed scope via `capture_active_scope()`, so event sync for this apply uses the scoped definitions dir. A legacy-root fallback is preserved during the Phase 1→2 transition period for pre-apply boot callers. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…mantics Three storage chokepoints (managed_agents_store_path, teams_store_path, global_config_path) now route through capture_active_scope() and fail with a clear error when no workspace scope is active. There is no fallback to the legacy unscoped root — returning a legacy path would recreate split-brain storage. apply_workspace acquires the workspace_transition lock (Layer 1 async serialization) before entering spawn_blocking so scope transitions are serialized against concurrent import_identity calls. import_identity implements both scope modes per v4 plan: - No-active-scope path (recovery/onboarding): persist identity, clear scope, bump generation. No scope is derived or claimed; the next apply_workspace performs adoption. - Live-active path (membership-denied flow): drain managed-agent runtimes (delegates to shutdown_managed_agents), persist identity, clear scope, bump generation. Drain failures are logged but non-fatal; the frontend's re-apply restores agents. Both paths bump the scope generation so in-flight operations see a new generation and abort their commits. The fallback relay can never claim legacy data — claims are only written inside apply_workspace's prepare stage. All 2112 tests pass. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Convert restore.rs, shutdown.rs, and list_managed_agent_runtimes to use a single captured workspace scope per logical operation rather than re-resolving the active scope on every load/save call. restore.rs: - backfill_persona_snapshots captures scope at entry; uses load_managed_agents_at / save_managed_agents_at / load_personas_at throughout the single store-lock epoch. - restore_managed_agents_on_launch captures scope at function entry and clones definitions_dir; all three phases (A: collect, B: spawn, C: write-back) use the same captured path, preventing a concurrent workspace switch from writing Phase C results into the wrong scope. - persist_restore_error receives definitions_dir explicitly. - Both functions return Err (with a clear message) when no scope is active, keeping the fail-closed invariant. shutdown.rs: - When no workspace scope is active (boot before apply_workspace, or after import_identity cleared the scope) skip load_managed_agents and drain only from the in-memory runtime map. Prevents the shutdown path from panicking with 'no active workspace scope'. - record_idx: Option<usize> on AgentToStop distinguishes runtimes with a backing record from those drained without one. - save_managed_agents only called when records were actually loaded. runtime_commands.rs: - list_managed_agent_runtimes captures scope at function entry and uses load_personas_at / load_global_agent_config_at / load_managed_agents_at / save_managed_agents_at so all reads in one poll see the same scope, even if a workspace switch races between the pre-lock and in-lock loads. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…m ledger Implements the Phase 2 scope initialization pipeline: - scope_init.rs: staged directory install with durable manifest (AdoptedLegacy | LegacyClaimedByOther | FreshNoLegacy), atomic rename, separate _ready marker, crash-safe restart semantics - Canonical family claim ledger: reads retention.db's retention_migrations table first (pre-existing claims win); falls back to agents/legacy-claim.json when no retention.db exists - Legacy adoption: copies managed-agents.json, teams.json, global-agent-config.json, and personas.json (when present) into a sibling ._staging directory, then renames atomically - apply_workspace: Prepare stage now calls ensure_scope_ready before the Layer-2 commit epoch; a failed prepare leaves the old scope active and untouched - 6 new unit tests cover FreshNoLegacy, AdoptedLegacy, second-scope LegacyClaimedByOther, idempotent re-init, staging cleanup on retry, and retention.db claim taking precedence over first-activation order All 2118 tests pass. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ped dev-sync Completes Phase 2 of the workspace-scoped agent store: - scope_init.rs: replace the run_scoped_migrations no-op with the full ordered migration pipeline (fold, strip, refresh-avatars, backfill, detach, reconcile-names, reconcile-mcp, databricks-v1-to-v2, materialize) running against the scope directory after staged install. Ordering mirrors migration.rs::run_boot_migrations_inner's load-bearing order. - migration submodules: expose fold_personas_in_dir, strip_baked_team_ instructions_in_dir, backfill_standalone_agents_in_dir, detach_directory_ backed_teams_in_dir, materialize_runtimes_in_file as pub(crate) - migration.rs: add _at(definitions_dir) wrapper functions for reconcile_ provider_mcp_commands, reconcile_databricks_v1_to_v2, refresh_builtin_ agent_avatars, reconcile_legacy_command_names, materialize_agent_runtimes re-export the dir-level helpers under the crate's migration module - SHARED_AGENT_FILES: emptied; legacy unscoped files no longer symlinked across worktrees (they live under agents/scopes/ now) - SHARED_AGENT_DIRS: add agents/scopes so all scoped stores are shared across dev worktrees without requiring knowledge of the dynamic scope ID - migration_tests.rs: rewrite 8 sync tests to match the new SHARED_AGENT_DIRS layout; add scope-dir-based write-through and seed-up tests All 2118 tests pass. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Phase 3 of workspace-scoped agent store:
3a: reconcile_managed_agent_runtimes loses its `communities` parameter.
The backend derives the sole target relay from the captured active scope;
cross-scope fan-out is no longer representable at the API level.
- runtime_commands.rs: capture active scope relay; remove communities Vec
- runtime_types.rs: remove ManagedAgentCommunityTarget struct
- tauriManagedAgents.ts: reconcileManagedAgentRuntimes() takes no args
- managedAgentRuntimeHooks.ts: bootstrapManagedAgentRuntimePairs calls
parameterless reconcile; drop communities list construction
- useManagedAgentRuntimeReconciliation.ts: rewritten to track a single
activeCommunityKey instead of per-relay state; simplified retry logic
- AppShell.tsx: pass `${activeCommunity?.id}-${reinitKey}` as the key
3b: Mesh relay-match reuse rule + fail-closed serve preflight + watchdog.
- mesh_llm.rs: ensure_relay_mesh_for_record captures scope relay at entry;
a live runtime is only reused when its relay matches the scope relay;
serve-mode mismatch fails closed with a precise 'Share Compute is
currently pinned to <relay>' error; client-mode mismatch falls through
to re-arm; drain_mesh_client_if_stale drains a client whose relay
differs from the incoming workspace relay (Layer-1 async, non-fatal).
- recovery.rs: rearm_relay_mesh_for_running_agents captures one scope per
pass; Live early-return only taken on relay match; serve-mode Live
mismatch skips the pass (machine-level pinning).
- personas.rs: add scoped load_personas_at / save_personas_at variants.
3c: Drain journal + compensation + apply_workspace rewrite.
- runtime_commands.rs: DrainJournalEntry struct, drain_scope_runtimes
(snapshot journal + stop all live runtimes, returns stopped/remaining/
first_error), compensate_drain (restart exactly the stopped entries).
- workspace.rs: apply_workspace return type changed from () to
WorkspaceApplyResult. Layer-1 async drains the Mesh client before
spawn_blocking. Drain stage acquires managed_agent_runtime_transition,
calls drain_scope_runtimes; on failure calls compensate_drain and
returns applied:false. Per-transition restore replaces the launch-only
managed_agent_restore_pending one-shot. Post-commit failures (event
sync, restore) surface as degraded entries on WorkspaceApplyResult.
3d: Scope-tagged runtime map entries.
- runtime_types.rs: ManagedAgentPairRuntime gains scope_id: Option<String>
- starting() constructor takes scope_id; captured from active scope at
spawn time in runtime_commands.rs, restore.rs, and runtime.rs.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…Phase 3e)
Surfaces the degraded-apply result from the backend all the way to the UI:
tauri.ts:
- Add WorkspaceApplyResult interface { applied: boolean; degraded: string[] }
- applyCommunity() now returns Promise<WorkspaceApplyResult> instead of
Promise<void>; passes typed result through from apply_workspace command.
useCommunityInit.ts:
- Consumes applyResult from applyCommunity instead of discarding void.
- applied: false (drain-failed) → park on the loading gate so the user
can retry via a workspace switch; the specific degradation messages are
shown as the error.
- applied: true with degraded entries → console.warn (informational;
workspace IS active; post-commit step failed gracefully).
- Existing catch block still handles genuine Tauri errors (poisoned lock,
invalid nsec, etc.) unchanged.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
ManagedAgentPairRuntime::starting() now takes a scope_id argument. Update the test helper that constructs a fake PairRuntime to pass None. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…rnal extraction Extract execute_drain_journal() as a pure inner function that takes the runtime HashMap directly — no AppHandle needed — enabling deterministic unit testing of the drain/compensate logic without a Tauri mock app. Move the test block from runtime_commands.rs to the sibling runtime_commands_tests.rs (following the storage_tests.rs pattern) to keep the main file under the 1000-line size gate. New tests: - test_drain_empty_map_returns_success - test_drain_exited_process_counts_as_stopped_and_clears_map - test_drain_scope_id_propagates_from_runtime_starting - test_drain_missing_key_treated_as_already_stopped - test_drain_cleanup_fn_called_for_each_stopped_entry - test_workspace_apply_result_drain_failed_returns_applied_false - test_workspace_apply_result_degradation_accumulates All 2125 existing tests continue to pass. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…y tests Add 13 missing Phase 4 unit tests covering the full v4 test matrix: Scope model (scope.rs): - test_generation_staleness_detected_after_scope_change — stale-commit detection: captured generation G diverges from current after a switch - test_scope_switch_a_to_b_to_a_advances_generation — A→B→A round-trip produces strictly increasing generations; relay fields correct at each step - test_rapid_scope_switch_a_b_c_all_stale_after_c — rapid A→B→C: both A and B stale relative to C; counter order A < B < C - test_switch_during_restore_detected_by_generation_check — mid-flight switch detected by generation check without spawning threads Identity/scope lifecycle (app_state_tests.rs): - test_import_before_first_apply_leaves_scope_none — import when scope=None does not derive/claim any scope; only bumps generation - test_live_import_with_active_scope_clears_scope_and_bumps_generation — live import clears scope and advances generation; commands fail closed - test_fallback_relay_never_claims_during_identity_import — identity import operations (clear + bump) never touch the filesystem claim ledger - test_prepare_failure_leaves_old_scope_intact — old scope unchanged when commit_active_scope is never called (prepare error path) - test_inactive_runtime_exit_after_scope_cleared_is_safe — scope=None after clear is safe for runtime-exit observers Crash boundaries (scope_init.rs): - test_crash_after_claim_before_staging_resumes_correctly — fallback claim exists, no staging: full staged install runs, legacy adopted - test_crash_during_staging_copy_is_cleaned_on_retry — stale staging with partial content is cleaned; final file comes from legacy source - test_crash_after_staging_manifest_before_rename_resumes_correctly — staging with manifest but no rename: cleaned and re-run - test_crash_after_rename_before_ready_resumes_migrations — target exists with manifest but no _ready: skip re-staging, resume migrations, preserve post-crash writes Also fixes ensure_scope_ready to implement the plan's "installed-but-not-Ready resumes migrations" contract: when the target directory already has a manifest (rename completed), skip install_staged and go straight to migrations + ready marker, preserving any post-crash inbound/interactive writes. Mesh relay-scope (mesh_llm_tests.rs): - test_serve_pinned_relay_mismatch_fails_closed — relay mismatch detection + fail-closed error prefix verified against the exact code path - test_client_relay_mismatch_is_not_fail_closed — client mismatch falls through (treat as absent), not the serve fail-closed error - test_watchdog_scope_relay_check_uses_normalized_comparison — relay normalization consistency including trailing-slash and whitespace edge cases All 2138 tests pass (was 2125). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Nine files were over the gate limit after the workspace-scoped store implementation. Trims/extractions to get all under limit: - app_state_tests.rs: move scope lifecycle tests to app_state_scope_tests.rs - app_state.rs: move pending_owned_channels methods to identity_storage.rs - AppShell.tsx: inline reconciliation key as String(reinitKey) (1 line vs 3) - tauri.ts: type alias ApplyWorkspaceResult + biome-ignore format to keep the applyCommunity body under the limit - runtime.rs: restructure scope capture to save a net line - storage.rs: trim doc comments on _at variants to single-liners - mesh_llm.rs: make scope_impl pub(crate) mod; fold scope relay capture inside check_mesh_runtime_relay_scope; remove verbose comments - migration.rs: extract scoped migration helpers to migration_scope.rs via include!(); trim SHARED_AGENT_FILES/DIRS block comments - migration_tests.rs: trim explanatory comments to save net 33 lines Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…mmit (C3) Lock architecture fix: `managed_agent_runtime_transition` is now held from journal creation through the end of the commit swap so no concurrent start/reconcile can insert a runtime in the gap between drain and scope publication. All fallible commit guards (relay_url_override, keys, active_agent_scope) are acquired BEFORE any field is mutated. A lock-poison failure after drain runs compensation and returns `applied: false` — never a half-committed state. The prior code dropped the transition guard at the end of the drain block (inner scope) while the adjacent comment claimed "the commit below also holds it" — the comment was false. Removes that false claim. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…d migrations, atomic drain, import drain protocol, boot migration strip) C1: Remove #[allow(dead_code)] from WorkspaceAgentScope owner_pubkey and generation fields; add validate_scope_generation() production helper; restore Phase B uses captured scope relay (not live relay_ws_url_with_override); Phase C validates generation before acquiring store lock and terminates stale-spawn children. C2: run_scoped_migrations returns Result<(), String> propagating first failure; ensure_scope_ready withholds _ready on Err; added Step 10 JSON validation gate; fixed crash-resume test fixture to use valid JSON; added migration-failure test that verifies no _ready on corrupt input, then repair+retry writes _ready. C3: (Already committed as 3325363.) Transition lock held continuously from drain through commit. C4: drain_managed_agent_runtimes_for_import returns Result<Vec<DrainJournalEntry>>; import_identity acquires managed_agent_runtime_transition lock for live-active path; drain failure compensates and returns Err before identity persist; persist failure compensates stopped entries and returns Err; removed commit_active_scope from identity_storage.rs. C5: run_boot_migrations_inner stripped of all definition-touching steps (now in scoped pipeline); backfill_persona_snapshots_at added and called in prepare stage; legacy retention migration moved to prepare stage; try_regenerate_nest removed from lib.rs boot (now post-commit in workspace.rs); managed_agent_restore_pending field and write removed from AppState and lib.rs. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ecovery from captured WorkspaceAgentScope active_retention_scope now derives relay and owner from capture_active_scope() rather than reading relay_ws_url_with_override + signing_keys() independently. Returns Err when no active scope exists (fail closed) or when signing keys pubkey disagrees with scope owner (defensive guard). rearm_relay_mesh_for_running_agents captures both relay and definitions_dir from the active scope at function entry. All store reads (load_managed_agents, load_personas, load_global_agent_config) and error-persist writes now use _at(definitions_dir) so they target the captured scope's store throughout the recovery pass, not whichever scope happens to be active when each helper runs. persist_mesh_last_error and clear_mesh_last_error_if_set refactored to _at() variants that take an explicit definitions_dir rather than resolving through the live active scope. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…pShell composite key
spawn_event_sync returns Result<(), String> so dispatch failure can be captured
by the workspace-apply post-commit section rather than being silently ignored.
The value is always Ok(()) since tauri::async_runtime::spawn is infallible; this
establishes the typed interface for future signaling.
try_regenerate_nest returns Result<(), String> instead of swallowing errors.
All fire-and-forget callers updated to .ok() to explicitly discard the Result.
apply_workspace post-commit:
- try_regenerate_nest moved out of the spawn_blocking closure into the async
post-commit section so its Result can populate the degraded vec.
- spawn_event_sync Result captured; dispatch failure pushed to degraded.
- Nest failure reported as 'nest context regeneration failed: ...' degradation.
useCommunityInit.ts: post-commit degraded items now emit a toast.warning (8 s)
via sonner so the user sees partial failures. Previously only console.warn.
AppShell.tsx: useManagedAgentRuntimeReconciliation key changed from
String(reinitKey) to `${activeCommunity?.id}-${reinitKey}`. A same-relay
identity swap (new communityId, unchanged reinitKey) now correctly re-triggers
runtime reconciliation. Destructured activeCommunity and reinitKey from
communitiesHook and updated two other call sites for consistency.
dead pub use exports in migration.rs removed (fold, backfill, detach, strip,
materialize — all now accessed only through scoped _in_dir/_at variants).
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
C4 removed commit_active_scope from identity_storage.rs (no longer called in production after the inline commit). app_state_scope_tests.rs uses it as a test helper to set up a live scope without running the full apply_workspace pipeline. Re-add it under #[cfg(test)] so tests continue to compile. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ncile key Without the '?? "none"' fallback, a null activeCommunity?.id produces the string "undefined-N" rather than "none-N". Both are valid change signals, but the explicit fallback matches the existing pattern at line 233 of the same file and satisfies Thufir's pass-2 finding C7. The line expands to 3 lines after biome formatting (88 chars), landing AppShell.tsx at exactly 1000 gate-counted lines — still within the 1000-line ratchet (gate condition is > 1000). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
After C5 stripped the per-scope definition migrations from the global boot path, the app-level wrapper functions (fold, materialize, backfill, detach, team_suffix, refresh_builtin_agent_avatars, reconcile_*) became unused. Their scoped _at()/_in_dir() variants are what the pipeline calls. Add #[allow(dead_code)] with a rationale comment to each wrapper rather than deleting them — the wrappers document the prior call shape and serve as reference for future integration. Also drop the spurious let _ = binding on remove_agent_runtime_receipt (returns (), not Result) flagged by clippy::let-unit-value. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Adds test_two_workspace_relay_partition to the managed-agent e2e suite. The test models workspace A and workspace B as two distinct owner keypairs on the same relay and verifies in both directions: 1. Owner A's NIP-33 author-scoped subscription returns only A's definition, not B's (workspace B content never leaks into A's view). 2. Owner B's subscription is symmetric — returns only B's definition. 3. Cross-scope queries prove NIP-33 (kind, author, d-tag) scoping: two owners publishing under the same d-tag get distinct relay coordinates that cannot collide or bleed across. This is the relay-level half of the live two-workspace leak probe required by the workspace-scoped agent definition store (PR #4485, plan v4 Phase 4). The filesystem-level half is covered by scope_id unit tests confirming that distinct (relay_url, owner_pubkey) pairs always produce distinct scope_id directories under agents/scopes/<scope_id>/. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
C1 — Production agents-root contract: - scope_init.rs already had the correct base_dir contract (no extra 'agents' join); added production-shaped adoption test that mirrors the exact managed_agents_base_dir semantics to prevent regression. C2/C3 — Deadlock removal + store lock: - workspace.rs: drop rt_transition + store lock BEFORE compensate_drain on all three commit-guard failure paths; hold store lock from drain through commit so concurrent store writes cannot interleave. - identity.rs: drain returns Err((stopped, msg)); compensate uses the real stopped slice, not []; locks dropped before compensate_drain. C4 — Captured-scope completion: - confirm_team_snapshot_import and confirm_agent_snapshot_import: both now capture scope at entry, use _at() APIs throughout, validate generation before first write, resolve RetentionScope from captured. - Mesh recovery (recovery.rs): capture full WorkspaceAgentScope at entry; validate generation before each write to definitions_dir. - Restore missing-record stale-child: when find_managed_agent_mut fails for a spawned child (record deleted between Phase B and C), terminate the child and remove its receipt instead of leaking the process. C5 — Pre-Ready family in scope initializer: - ensure_scope_ready gains owner_pubkey parameter. - New run_pre_ready_family: runs legacy retention migration and persona snapshot backfill before writing _ready so a crash leaves the scope in a retryable state, not permanently marked Ready with incomplete data. - workspace.rs guards remain for pre-existing Ready scopes (idempotent). C6 — Delete dead boot-migration wrappers: - Deleted backfill_standalone_agents, detach_directory_backed_teams, and strip_baked_team_instructions (the #[allow(dead_code)]-suppressed app-level wrappers); their _in_dir equivalents are the authoritative scoped pipeline entry points. - Removed tests for the deleted functions from migration_command_tests.rs. C7 — Structured degradation reporting: - spawn_event_sync return type changed from Result<(), String> to (): the dispatch cannot fail; the false Result contract is removed. - workspace.rs restore spawn now emits workspace-degraded Tauri event when restore_managed_agents_on_launch returns Err, making restore failures observable to the UI instead of silently logged. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…tions - backfill.rs: remove blank line between two consecutive doc comment blocks - detach.rs: merge orphaned step-list doc comment into function doc comment - migration.rs: remove blank line after doc comment before private fn - migration_tests.rs, migration_command_tests.rs, migration_avatar_tests.rs, migration_databricks_tests.rs: add .unwrap() to calls that now return Result after C5 migration fallibility changes Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Contributor
|
workspace-scoped agents/stores is the right cut for the cross-relay leak. the two-owner e2e note in the test comment helped me follow the nip-33 half. |
…y, captured scope, lock-aware compensation
Eight corrections from the resumed loop (fresh 3-pass budget, Option A ruling):
1. Option A Mesh: delete pre-prepare drain_mesh_client_if_stale and rollback
restore_mesh_sharing (compensated a drain that no longer happens). Replace
with fail_if_client_mesh_active preflight in both apply_workspace and
identity import. Journaled Mesh recipe deferred as tracked follow-up.
2. Versioned _ready: scope_is_ready now reads marker content and compares
against READY_MARKER_VERSION ("v1"); old unversioned markers return false
and force re-run through run_pre_ready_family. Delete log-only post-ready
best-effort guards (backfill + retention migration) from apply_workspace.
Add test: old marker -> pipeline re-runs -> version advances.
3. Snapshot outbound phases use captured scope relay: both
confirm_agent_snapshot_import (Phase 3b profile) and
confirm_team_snapshot_import (Phases 4/5 profile + memory) now use
captured_scope.relay_url instead of relay_ws_url_with_override.
Add test proving outbound relay is captured-scope, not live-state.
4. Generation checks atomic with writes: global_agent_config Phase 1 validates
scope generation inside the store lock before writing config; Phase 2
(restart_local_agent_on_config_change) validates under lock before stop.
collect_restart_candidates renamed to collect_restart_candidates_at with
definitions_dir parameter. Mesh recovery helpers (persist_mesh_last_error_at,
clear_mesh_last_error_if_set_at) take captured_scope and validate generation
inside the store lock.
5. Lock-aware compensation gate: AtomicBool
managed_agent_drain_compensation_in_progress added to AppState.
compensate_drain sets it true (Release) before restarting entries, false
after. start_pair loads it (Acquire) before taking the transition lock and
returns Err if set. Closes the drop-then-compensate interleave window
without recursive locking. Add deterministic partial-drain test.
6. Pre-scope migrations deleted: migrate_agent_keys_to_dev_service (AppHandle
variant) removed from storage.rs. Pre-scope calls removed from
run_boot_migrations_inner. Scoped variants in run_pre_ready_family are
authoritative.
7. Degradation wired to UI: workspace-degraded Tauri event listener added to
useNestNotifications.ts (toast.error with payload as description). False
comment about emit_workspace_degradation removed from event_sync.rs.
backfill_persona_snapshots_at (dead lock-taking wrapper) deleted.
8. e2e test docs: test_two_workspace_relay_partition comment corrected --
Direction 3 asserts len==1 (B's event), not zero. Explicit note added that
this test does not cover desktop workspaces or substitute for the live probe.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…t.rs) Move scope_init tests to scope_init_tests.rs via #[path] include to bring scope_init.rs under the 1000-line ratchet (603 lines after extraction). Move test_outbound_relay_uses_captured_scope_not_live_state from import_avatar_tests (in import.rs) to the adjacent tests.rs to bring import.rs under the 1000-line limit (999 lines after move). Both files previously crossed the limit after the resume-pass corrections added the versioned-ready test block and the captured-relay test. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…h stop, scope, CI
Item 1 — Compensation primitive: replace AtomicBool gate with lock-owning
compensate_drain that takes the caller's already-held rt_transition guard
by value, re-acquires only the store lock, validates captured scope generation,
then restores journal entries via start_pair_under_held_locks. Split
start_pair_under_held_locks out of start_pair so both the normal and
compensation paths share the spawn-and-register body. execute_drain_journal
refactored to accept an injectable stop_fn via drain_journal_with_stop;
execute_drain_journal_with_stop_fn exposed for test injection. Tests: live
SIGKILL drain, structural lock-release proof, deterministic partial-failure
test with injected stop error covering stopped prefix and remaining tail.
Item 2 — Client stop + start serialization: mesh_stop_client Tauri command
in mesh_llm_scope.rs stops only a client-mode runtime; serve/absent are
no-ops. Client start (ensure_relay_mesh_for_record) acquires
workspace_transition through runtime installation to serialize against
apply_workspace, which holds workspace_transition from before the Option A
preflight through commit. fail_if_client_mesh_active preflight runs under
workspace_transition so no new client can start in the check→commit gap.
UI: 'Stop using shared compute' button in MeshComputeSettingsCard shown when
isConsuming; calls new meshStopClient() in tauriMesh.ts. e2eBridge mock for
mesh_stop_client added.
Item 3 — Fallible migrations + atomic marker: rename_provider_to_runtime_in_personas
propagates Result; migrate_agent_keys_to_dev_service_at returns Result and
propagates from copy_agent_keys_between_stores. run_scoped_migrations uses ?
on persona-provider step. _ready marker written via temp+rename (atomic).
dev-key migration skipped in unit-test builds (#[cfg(not(test))]) to avoid
macOS Keychain dialogs. Tests: old-marker upgrade (no scope deletion), partial
migration failure withholds v1 until repair succeeds.
Item 4 — Global-config captured respawn: Phase 2 restart validates captured
scope generation under store lock before stop, and again before respawn via
start_local_agent_pairs_with_preflight_at (new captured variant using
definitions_dir). persist_last_error validates generation under store lock.
Item 5 — Snapshot imports captured operation context: both confirm_agent_snapshot_import
and confirm_team_snapshot_import capture owner keys at entry, verify against
captured_scope.owner_pubkey immediately, thread captured keys through all
mint/retention/engram phases. Re-verify owner key under store lock before
Phase 3a write. Outbound profile/memory phases use captured_scope.relay_url.
Item 6 — CI red: rustfmt applied (agents_scoped.rs, import.rs); clippy
needless_borrow at team_snapshot.rs:793 fixed; e2eBridge apply_workspace mock
returns { applied: true, degraded: [] } in both immediate and delayed branches;
mesh_stop_client mock case added. Stale 3-line doc fragment removed from
mesh_llm.rs; visibility of re-exported agents_scoped fns bumped to pub(crate).
Item 7 — Listener behavioral test: useNestNotifications.test.mjs exercises
workspace-degraded toast payload, unlisten cleanup, and boundary payloads
without requiring a real Tauri runtime. Doc comment updated: event-sync dispatch
failure does not emit workspace-degraded (shutdown-time, no toast surface).
Item 8 — Dead code (minor): backfill_persona_snapshots AppHandle shim removed;
stale scope_init.rs:388-393 comment corrected; make_base_dir test helper removed.
File-size gate: mesh_llm.rs (999), import.rs (999), team_snapshot.rs (999).
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Resolves two conflicts against main:
AppShell.tsx: HEAD had const { activeCommunity, reinitKey } = communitiesHook
for the composite workspace key; origin/main added useHuddlePresentation()
destructuring from the Huddle redesign (#4281). Resolution keeps both: the
composite key is required for useManagedAgentRuntimeReconciliation, and the
Huddle hooks are needed for the new Huddle UI.
MeshComputeSettingsCard.tsx: HEAD had the Stop using shared compute affordance
plus the legacy inline model section; origin/main (#3735) replaced the inline
model section with the MeshModelPicker component. Resolution keeps the Stop
button block and adopts the MeshModelPicker layout, discarding the replaced
inline model controls.
Also corrects the false comment at runtime_commands_tests.rs:342-345 that
claimed compensate_drain is covered by the desktop integration test suite.
The compensation round-trip requires an AppHandle; the codebase has no
tauri::test harness and no AppHandle mock. The honest coverage statement is:
drain-prefix contract proven by unit test, restart path integration-only.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The applied-but-blocked test additions in the branch-tip commit left three assertions over the line-width limit, which fails desktop-tauri-fmt-check. Reformatting them clears Rust Lint for subsequent phase pushes. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
|
The Desktop failure is unrelated to this branch. The compiled-flag job reran the full Tauri suite and generated_passphrase_respects_word_count_and_separator drew a hyphenated EFF word, so a three-word phrase produced four parts when the test split on hyphens. The other 2,448 tests passed. A rerun should clear this check. The flaky assertion belongs in a separate small fix rather than another change to this already large branch. |
The Phase 0 seam made every persona writer merge-preserving; Phase 1 lands the data model that seam was built for: `library.json` and the scope-local crash journals, plus their quarantine-preserving IO. No operation mutates them yet — share, edit, materialize, delete, and deploy land in Phases 2-5. `library.rs` models the versioned document envelope, `SharedDefinition` (the allowlisted content a share may carry — credentials, identity, `env_vars`, activation, and projection metadata are structurally absent), `LibraryEntry` with its per-scope `ProjectionState` machine, verified `IdentityBinding`s, and the permanent `DeferredArchive`/`RemovalManifest` retirement markers. `load_library` classifies per §2.1: an absent file is a valid empty library, whole-document corruption is preserved as `.invalid` and blocks all mutation, an unknown version is read-only fail, and a single malformed / semantically invalid / identity-colliding entry is quarantined raw while healthy siblings stay usable. Identity collisions on `library_id`, a live origin, or a non-terminal `(scope, slug)` claim group-quarantine every collider — never first-wins. `library/journals.rs` adds the scope-local `pending-agent-keys.json` and `deploy-intents.json`, which share the owning workspace's failure domain rather than the library's so a broken `library.json` never blocks a workspace-local create or deploy. Their failure domains are asymmetric: a pending-keys failure degrades only that scope's create/import path, while a deploy-intents failure — unknown version, syntax error, or a duplicate-pubkey mutex violation validated on read — fails the scope's destructive and deploy paths closed. `apply_shared_definition` is the sole writer of shared content onto a scoped keyless record, assigning exactly the shared slots plus revision/timestamp and mirroring `into_agent_record` so a populated record is byte-identical to a freshly projected one. `ManagedAgentRecord` gains `last_completed_deploy_attempt_id`, the deploy-provenance stamp that forms an inseparable pair with `backend_agent_id` in `copy_runtime_state`. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Interim review of Phases 0+1 found one CRITICAL and three IMPORTANT defects plus one MINOR; all are resolved here so Phase 2 can resume. F1 (CRITICAL): cross-owner binding aliasing. The document identity index now maps each bound agent_pubkey to its owner and group-quarantines every entry when one pubkey is bound under two owners; same-owner reuse across entries stays healthy (§2.5). Per-entry validation could not see this document-wide alias of a process-global keyring identity. F2 (IMPORTANT): deploy-intent routing is now validated by validate_provider_config on read (fail -> Unreadable) and at the save_deploy_intents writer boundary, so a malformed or secret-bearing row is never exposed as authoritative routing (§2.1). F3 (IMPORTANT): the Phase-0 routing seam is present. A raw-record-by-slug lookup plus a typed MutationRoute decision route delete/inbound/import; merge_preserving_definitions fails closed when a plain save would delete or edit the shared slots of a library-projected record, while an unchanged projected re-pass rides through intact (§2.7). F4 (IMPORTANT): the P14-I2 provenance regressions are added — legacy byte-compat round-trip, a concurrent deploy-success pair-churn rollback, and a non-None deploy stamp surviving apply_definition_view/into_agent_record. F5 (MINOR): deferred_archives gains encapsulated upsert_deferred_archive (SET semantics on (scope_id, agent_pubkey)) and a deferred_archive_obligations read view that collapses legacy duplicate rows to one obligation (§2.3). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
F1: canonical identity indexing — binding validation parses every owner and agent pubkey through a canonical-encoding gate (64 lowercase hex + curve-point check, mirroring parse_canonical_pubkey), rejecting any non-canonical spelling into the quarantine ladder. This makes the document-wide raw-string identity index sound: a cross-owner alias can no longer evade the check by re-casing its hex. F3: command-boundary preflight — raw_record_by_slug and MutationRoute gain live consumers. delete_persona and snapshot/team import route on for_slug; both inbound arms route on for_persona_d_tag (their real match key, derived from source_team_persona_slug). Each consults the RAW keyless store BEFORE any destructive effect, so a library-projected target fails closed with zero side effects until §3 wires the state machine. The merge seam now compares a SharedSlotFingerprint over the SharedDefinition allowlist instead of the whole record, so scope-local edits (is_active, env_vars, timestamps) on a projected record ride the plain path. F4: the legacy byte-compat test drives the real store save->load->save path and asserts a byte-exact fixpoint against the pinned-head baseline, plus key-absence — the assertion now matches its name. F5: deferred_archives is private; upsert_deferred_archive and deferred_archive_obligations are the only access outside the module. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…e rule Phase 2 (revised) of the cross-workspace agent library: the read side of the definition-instance relation resolver and the inbound kind:30177 canonical-linkage rule, with the interim fail-closed new-link posture. No instance-removal/insertion coordinator (that is Phase 4b). Relation resolver: MutationRoute::for_linked_definition resolves an instance's persona_id against the raw keyless definition store to classify its linkage (persona_id IS the linked definition's slug). This is the one canonical join every library mechanism uses; nothing re-derives the persona_id join ad hoc. Inbound 30177 canonical linkage: apply_inbound_managed_agent now consults the resolver and freezes two linkage-authorship cases, applying only safe per-instance fields (name, parallelism, respond_to, respond_to_allowlist): - OwnedByLibrary: the matched instance is linked to a projected definition and the event would clear or re-point persona_id. - InadmissibleNewLink: the event would newly link a plain instance to a projected definition (only the Phase-4b coordinator may admit). A frozen linkage re-retains the local record at a monotonically newer created_at (via retain_agent_record) so the relay head converges back to the library-authoritative linkage, mirroring the 30175 rule. Plain-to-plain relinks and no-op events apply exactly as at head. The round-2 projected persona preflights stay intact in front of this. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…e and retryable The §2.8 canonical-linkage freeze path retained the inbound event as the head before attempting the corrective library-authoritative re-retain, then swallowed a failed re-retain to stderr and returned Ok. That left the non-authoritative head retained with no recovery: replay is dead because the same event re-arriving is Skipped at the equal-created_at guard before the convergence branch runs. Propagate the corrective re-retain failure (converge_frozen_linkage) so the command cannot report success over a divergent head. The durable retry owner is the boot-time reconcile_agents_to_events pass, which re-diffs the still-authoritative on-disk record against the retained head every launch and re-queues the corrective row at a monotonic bump. Surface the freeze reason as a typed InboundReconcileOutcome (reconcile_inbound_persona_event now returns it) instead of stderr-only, and consume it in usePersonaSync. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…mmand seam The §2.8 corrective-retain failure path was only proven at the extracted helper. No test crossed the real `?` propagation site, so weakening it to a swallow left the whole suite green — the pass-1 defect, reintroducible undetected. The boot-reconcile requeue was proven only from synthetically seeded state, not the state a failed command actually leaves. Extract the retain -> apply -> §2.8 convergence body of the blocking command into `apply_inbound_upsert_in_scope`, generic over `tauri::Runtime`, so a mock app can drive it. No behavior change: the wrapper resolves the arrival scope and the §2.7 preflight, then delegates; the corrective `?` moves inside the seam unchanged. Add one integration fixture that drives the real seam against a MockRuntime app, an active workspace scope, and a retention DB with a conditional trigger that permits the inbound `pending_sync = 0` write and rejects the corrective `pending_sync = 1` write. It asserts the command returns `Err`, the frozen linkage persists to disk with the safe field applied, the hostile inbound head stays retained, and the real boot reconcile against that same state restores the authoritative projection at a bumped `created_at` with `pending_sync = 1`. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The process-global session_config_cache was keyed by {pubkey, relay_url}
with no owner, scope, or generation, and put_agent_session_config gated
only on a same-pubkey record in the then-active store. A delayed
session_config_captured frame from workspace A could survive an A->B
drain, repopulate B's colliding key, and surface A's live session as
B's. Lifecycle frames already carried the missing capability
(startNonce + tracked-live-runtime check); session-config frames did not.
Move the cache onto ManagedAgentPairRuntime (session_config), making an
ownerless entry unrepresentable and destroyed atomically with the runtime
on drain/removal/exit-prune. session_config_captured now carries
startNonce; no-nonce old-harness frames are dropped with no fallback.
Admission requires the frame's exact {pubkey, relay_url, start_nonce} to
match a still-live tracked runtime whose scope_id equals the current exact
scope, validated under the runtime-map lock immediately before the sole
mutation; the store read is demoted to enrichment. get_agent_config_surface
consumes the cache only through the same still-current capability.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Phase-3 chunk 1 of the cross-workspace agent library ($2.5). Two keyring-free pure functions with no production callers yet — the crash-safe mint protocol and their insertion/removal transaction wiring land in Phase 4b per the resequencing ruling: - key_archive_protected(agent_pubkey): the deletion-safety predicate (P13-C1, widened by P17-C1). Scans RAW library.json entries so a quarantined entry that ever bound the pubkey still protects it; over-protecting only leaves a secret resident, while mis-deleting a live global key is irreversible. - select_binding_seed: deterministic seed selection (P3-I2) — earliest created_at, ties broken by lowest pubkey; None only when zero instances exist. Tests live in a new binding_tests submodule to keep the Phase-1 tests.rs suite clear of the 1000-line file ratchet. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
put_managed_agent_runtime_lifecycle_for validated pair key, start_nonce, and process liveness but never required the tracked runtime's scope_id to equal the current active scope — the same cross-scope leak class killed on the sibling put_agent_session_config (P23-C1). A same-pair/same-nonce lifecycle frame from a drained workspace was accepted and mutated the runtime after a workspace rotation. Capture the active scope before the runtime-map lock and reject any frame whose runtime.scope_id differs, mirroring the sibling command's shape. Add a capability test proving a stale-scope frame is rejected with the runtime lifecycle unmutated (no status emit). Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The runtime lifecycle scope-match fix pushed runtime_commands.rs past the 1000-line file-size ratchet. Rather than trim the file to the exact ceiling again, move the cohesive observer-lifecycle unit (observer_lifecycle_key plus put_managed_agent_runtime_lifecycle[_for]) into a sibling runtime_commands_observer.rs, #[path]-included like the existing seams module. Mechanical move only — no behavior change. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Phase-3 §2.5 crash-safe mint protocol construction half (P4-I3, P8-C2). build_identity_binding is the commit-time inverse of validate_entry_bindings: derive agent_pubkey from the read-back nsec (not the stored record — a stored auth_tag proves nothing about this binding, and a pre-NIP-OA seed has none), then compute a fresh auth tag with the current owner keys so the constructed entry passes read-side validation with no legacy-None special case. Passes nostr types straight into buzz-sdk exactly as the validator does. LibraryDocument orphan-journal methods (journal_orphan_pubkey idempotent, remove_orphan_pubkey, unreferenced_orphans) implement §2.5 step 2/4 plus the recovery sweep selection: a pubkey journaled before its secret is persisted, dropped in the same atomic write that commits its binding, and reaped if a crash left it uncommitted. unreferenced_orphans scans for a LIVE binding only — a deferred-archive marker does not keep an uncommitted orphan alive. Factors entry_binds_agent out of entry_names_agent for the live-binding test shared by both. Keyring I/O and full transaction wiring remain DEFER-to-4b. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The §2.5 fresh-key mint protocol requires a durable orphan-journal checkpoint before the secret reaches the keyring, so every persisted secret has a prior durable coordinate and no crash can strand an un-journaled key. build_identity_binding covered construction; this adds the ordering: generate in memory, durably journal the pubkey, keyring write+read-back verify, then build the binding from the read-back nsec. The durable step-2 write is an injected persist seam so the ordering is unit-testable at every crash point without real disk IO; the atomic step-4 binding commit + orphan removal stays with the insertion transaction (Phase 4b). KeyStore/agent_keyring_name are lifted to pub(crate) so the library key protocols share the keyring seam. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The mint orchestrator leaves a durable orphan row whenever a crash lands before the atomic step-4 commit; nothing reclaimed the dangling keyring secret. Add reap_unreferenced_orphans: for each orphan with no live binding, delete its keyring entry THEN drop its journal row, persisting the trimmed document once. Delete-before-drop and drop-only-on-success make the sweep idempotent across recovery points, and a backend delete failure keeps that row for a later retry. store_all only merges, so a new KeyStore::delete seam (delegating to SecretStore::delete, absent-entry = Ok) is required; the fakes mirror that contract. Also folds in Paul's read-back-miss mint crash point: load returning None after a verified write yields Err, no binding, and a surviving orphan row for the reap. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The §3.5 binding-retirement finalizer in v1 is permanently conservative (P15-I2/P16-I1/P17-C1): it OBSERVES retirement-due entries at recovery points and discharges nothing, because library metadata alone cannot prove a process-global key or a community-visible identity is unused — an unrelated plain carrier of the bound pubkey may live in an inactive scope no deleting authority may read. The keyring entry, deferred rows, and tombstoned binding record persist as permanent journaled markers. Add LibraryEntry::is_retirement_due (derived condition: at least one projection, all terminal, a binding key or deferred row still names it) and LoadedLibrary::retirement_due_entries (the pure observer selector). The recovery-point wiring is Phase 4b. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
KeyStore::write_and_verify confirmed a write by calling load(), which returns the in-process cache that store() itself just advanced — proving the cache was updated, not that the OS keyring durably holds the value. mint_bound_identity relies on this before returning a commit-ready binding, so a backend that acks a write without persisting it could let Phase 4b commit an identity binding whose only secret dies with the process, violating the §2.5 invariant that no binding exists with an unverified key. Route the production confirmation through SecretStore::verify_stored_raw, which bypasses the cache and reads the OS backend directly — the same primitive the identity path already uses. The other caller (migrate_inline_key) is upgraded for free; its Ok/Err contract is unchanged, only strengthened to mean durably-verified. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…A C1) Route every owner and managed-agent relay send through the owner_identity_egress admission layer so no code path can publish under an owner identity without a rate-limit-witnessed EgressLease. Privatize AppState.keys behind latch-gated checked accessors (signing_keys/current_pubkey), thread the EgressLease witness through all six explicit-key funnels plus the four git-workflow and four managed-agent sink sites, and rewrite the huddle STT task to re-resolve keys per send with a mid-huddle recovery-latch break. The C5 P25/P28 coordinator (journal + three-valued commit outcome + Indeterminate latch) and its egress-drain wiring stay deferred: each unreachable item carries a per-item allow(dead_code) naming C5 as its consumer, with semantics pinned by the owner_identity_egress unit tests until C5 makes them reachable from production paths. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…_egress Managed-agent sink #4 was labeled a reaction publish; the actual site is send_managed_agent_channel_message (the managed-agent channel message send). add_reaction/remove_reaction route through the self-admitting owner wrappers and are not ManagedAgentKeyed sites. Also scope the eight-site count to the ManagedAgentKeyed construction set, which is what it enumerated — the owner side has its own disjoint, larger set. Resolves the P29-C1 MINOR from C1 close. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Add OwnerIdentityCapability<P> over the existing egress registry: a generation-stamped, registry-tracked handle for authority that outlives the bounded lease that derived it. Two policies land: Session (authenticated connections — the huddle audio socket, later the frontend relay WS) and Bearer (pre-minted Blossom headers, threaded in a follow-up). Each capability is registered with its revocation handle (the session's cancellation token; the bearer's registry id) so the C5 coordinator barrier only invokes what C2 registered — it never retrofits the registry schema. admit_exercise() validates BOTH current egress admission AND capability_generation == current identity-persistence generation immediately before each transmission, so a stale capability sends zero bytes. Issuance runs under a bounded lease (the signing that derives the capability is an ordinary leased operation). The huddle audio socket is threaded: the NIP-42 auth signs under a bounded lease (dropped before the joined-await), the session capability is registered with the connection's cancel token, and the send task validates it before every frame batch — a frame cannot ride the established peer after an identity transition supersedes it. C2 builds substrate only: the coordinator revocation barrier (revoke_durable_capabilities_before) and drain wiring defer to C5 with the egress drain, gated behind the same generation bump C5 introduces. Per-item allow(dead_code) with the C5-consumer comment; C5's zero-allow confirmation extends to these. generation never bumps until C5, so this is behavior-preserving. 8 new unit tests (2438 lib pass): generation-stamp, exercise admits when live+current, stale-capability zero-bytes controls (generation bump, drain, latch) for both kinds, barrier revokes old-generation only, registration- completeness + deregister-on-drop, and a no-transition control. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…auth (C2b) Bring the owner-derived Blossom bearers into the durable-capability world so a header minted under identity A cannot be attached after a transition to B. mint_media_get_auth and the do_upload t=upload mint now sign under a bounded egress lease (issuance is an ordinary leased operation, spec L4569-4570) and return/hold an OwnerIdentityCapability<BearerPolicy> registered with its revocation handle. The four get-auth attach sites (media_download, personas::card, media_proxy x2) and the upload dispatch validate the bearer via admit_exercise() immediately before the HTTP send — a stale capability attaches nothing (get-auth stays fail-open) or uploads zero bytes. mint_media_get_auth becomes async; the ripple is a mechanical .await through its four already-async callers. Removes the register_owner_bearer allow(dead_code) now that C2b consumes it. Substrate coverage is unchanged: the stale-bearer zero-bytes and registration-completeness controls already pin the exercise behavior these guards depend on. 2438 lib tests pass, clippy + fmt clean. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…x round) Addresses Paul's four C2 findings before C3. F1 (barrier bypass): register_owner_session/register_owner_bearer re-read the current generation at registration, so a capability derived from a losing identity could be stamped with the winning generation and survive the C5 barrier. Both now take &OwnerIdentityEgressLease as a compile-enforced witness and stamp lease.generation() via a shared register_durable() helper — a bump between admission and registration leaves a stale stamp that the first admit_exercise refuses (fail-closed). Huddle registers the session while the auth lease is still held (before the joined await, not spanning it). Corrects the DurableRegistry "same lock" and media.rs "no bump can slip" doc claims. Adds a red-then-green control: admit under gen N, drain bumps to N+1, register, exercise refuses. F3 (doctrine): the upload legacy retry re-attached the signed header without a second admit_exercise. Revalidate before the retry dispatch — two transmissions, two validations. F4 (scope): create_auth_event (frontend relay WS) now signs under a per-send bounded lease, gating reconnection against an in-flight drain. Frontend session REGISTRATION (capability + native-WS teardown) is explicitly deferred to C6/C7 with the frontend identity store. F2 (gates): split six files under the desktop file-size ratchet — never trimmed. owner_identity_egress.rs → directory module with the durable substrate in durable.rs; relay.rs/media.rs/identity.rs move test modules to sibling _tests.rs files; card.rs extracts the card-archive cluster to card/archive.rs and messages.rs the feed-item projection to messages/feed_item.rs. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Extend the durable owner-identity capability registry with ArtifactPolicy
(NIP-AP, P30/P31/P32-C1): a generation-stamped owner-key-derived value that
leaves its producing lease and is applied later. Unlike Session/Bearer,
an artifact owns no side-effecting teardown, so its only revocation is the
exercise-time / application-site generation compare — the transition bump
IS the invalidation (shape (ii), acked by Paul). revoke_durable_capabilities_before
performs no per-entry artifact work; the barrier reaches artifacts via the bump.
The seven commands/identity producers admit a bounded lease before the owner-key
sign/encrypt/decrypt and return the value wrapped as StampedArtifact<T>
({value, artifact:{id,generation}}), stamping the issuing lease's generation
(the F1 lesson). The wire shape is locked on both sides; a serialization test
pins it so C6/C7 inherits a stable contract.
The 7 TS adapters unwrap .value at the boundary with a named C6/C7 deferral —
outward threading to application sites + the generation-compare lands in C6/C7
coupled to the code that reads the stamp (supersedes condition 2 of the
stamp-at-boundary ruling, revised on the measured 87-file transitive radius).
StampedArtifact type and the four identity adapters split into small modules to
respect the desktop file-size ratchet.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Compose main's post-C3 changes with the workspace-scoped agent store
(WSA) refactor, keeping BOTH sides' semantics. Non-mechanical resolutions
(per Paul's cluster ruling, thread eb3246b2):
- Pollen rename → scoped-pipeline step 1.5 (NOT global pre-scope): the
live store on HEAD is scopes/<id>/managed-agents.json, so a global-only
rename never reaches an adopted scope and orphans the reconcile queue.
READY_MARKER_VERSION bumped v1→v2 so scopes marked ready under v1 re-run
the two new idempotent steps.
- Team-membership repair → scoped step 4.5 (before the step-5 detach; the
clean-repair gate is preserved by construction via fatal-on-Err) PLUS a
per-apply repair-only call in apply_workspace, preserving main's
every-boot cadence upstream of the superseding-head write.
- Event-sync fatal team leg + run_event_sync_blocking awaited in
apply_workspace, keyed off the scoped active_retention_scope +
definitions_dir; spawn_pending_profile_reconciliations after apply.
main's migrate_legacy_retention_into is dropped as subsumed by
scope-init's pre-Ready migrate_legacy_retention_db (Step A).
- inbound.rs apply composed HEAD's §2.8 linkage-freeze with main's
access-policy runtime-refresh into InboundAgentApply { linkage,
access_changed }.
- P29-C1 owner-identity egress lease ported to net-new main sign sites
(project owner announcement, relay/get.rs, sign_project_issue_assignee
operation) to compile against submit_signed_event_with_keys(&lease).
Merge fallout resolved: #5682's local-spawn idle-pool-sleep env
(idle_pool_sleep_env / IDLE_POOL_SLEEP_SECS) is subsumed — its only call
site was the local direct-env block HEAD's ff16a80 deleted, and the
remote-deploy policy_env path never carried it; the orphaned symbols are
dropped. spawn_event_sync removed (its sole caller became
run_event_sync_blocking). mesh_llm_tests.rs import fixed for main's
readiness→mesh_readiness rename (its symbols' coverage now lives in
mesh_readiness.rs's own inline tests). Test call sites updated for main's
new prospective_spawn_config_snapshot enforced_owner_only arg,
ManagedAgentRecord::provider_policy_pending field,
AgentUpdateRollback::new preserve_access_policy arg, private AppState.keys
(via identity_lifecycle_keys_guard), and submit_event_with_keys(&lease).
File-size ratchet (base 978e585): four files crossed the 1000-line cap
after the merge and were split, never trimmed. team_snapshot.rs →
team_snapshot/retain.rs (retain_agent_pending); personas/inbound.rs →
inbound/tombstone.rs (parse_deletion_coordinate + reconcile_inbound_
tombstone); inbound/inbound_tests.rs → inbound/team_tests.rs (kind:30176
team-inbound tests); managed_agents/retention.rs → retention/tests.rs
(inline test module).
Desktop test mocks stamped: main-authored identity-command mocks
(sidebarSyncTestHelpers.mjs installTauriMock; communityThemeSync.test.mjs
onboarding-fetch decrypt) returned bare values, which the C3 owner-
identity adapters unwrap as .value → undefined. Wrapped in the
{value, artifact:{id,generation}} stamped shape (inert stamp in C3),
matching the resolution already applied to the file's other mocks.
Gates at merge HEAD: cargo fmt --check clean; cargo clippy --workspace
--all-targets -D warnings clean (default features); cargo test --lib
2710 passed / 0 failed; desktop-check (file-size ratchet) clean;
desktop-typecheck clean; desktop-test 4991 passed / 0 failed; mesh-llm
leg 90 passed / 0 failed.
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
get_nsec and create_ncryptsec_backup admit an owner-identity egress lease BEFORE reading/deriving the secret and return their value as a StampedArtifact carrying the issuing lease's generation. The NIP-49 constructor sits outside the sign/encrypt method sweep, so the export class is stamped constructor-agnostically. Adapters unwrap .value; the reveal/copy generation-compare defers to C6/C7. Closed-world enumeration and the ncryptsec source allowlist updated in the same commit. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
The production adapters unwrap .value from the StampedArtifact wire shape, but the Playwright bridge mocked sign_event, create_auth_event, nip44_encrypt_to_self, nip44_decrypt_from_self, and sign_nostr_identity_binding with bare returns. At runtime the app under test hit JSON.parse(undefined) at every sign/encrypt site, cascading into membership-subscribe, read-state, and badge failures across the smoke suite. Add a local stamped() helper and route every owner-identity artifact mock (including the C4 nsec/backup cases) through it. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
h2 0.4.14 is vulnerable to unbounded memory growth from empty DATA frames (RUSTSEC-2026-0258, patched in 0.4.16). cargo-deny fails the Security gate on the advisory ingest. Bump is lockfile-only; 224 other dependencies unchanged. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…nator Phone-recovery identity swap (`import_recovered_identity`) bypassed the managed-agent runtime drain, active-scope clear, and scope-generation bump that `import_identity` performs (P25-C1): it acquired only `identity_mutation` and committed via `commit_imported_identity` directly. A recovery in an active workspace left stale agent runtimes bound to the prior identity. Extract the shared `run_identity_transition` coordinator so both callers route through the same lock-ordering and drain path. The pre-commit boundary is generalized to a `commit_under_fence` primitive: it locks the supplied fence, runs the validity check, and runs the durable commit under the same held guard, so a racing supersession can neither interleave nor slip between check and commit (P26-C1). Normal import passes no fence and an always-Ok check; recovery passes the pairing `generation_fence` and the task-currency check. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
… module The C5 coordinator additions (`run_identity_transition`, `commit_under_fence`) pushed `commands/identity.rs` past the 1000-line desktop file-size ratchet. Move the coordinator cluster — the drain helper, `run_identity_transition`, `commit_under_fence`, and `import_identity_blocking` — into a `#[path]` sibling `identity_transition.rs`, mirroring the `agents_scoped.rs` split. The two `pub(crate)` entry points are re-exported so external call paths (`import_identity`, phone recovery, the fence tests) are unchanged. No behavior change. Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…ed-agent-store * origin/main: fix(desktop): eliminate mounted-view CPU burn — compositor-safe shimmer, observer append fast path, poll-tick disk reads (#6198) Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com> # Conflicts: # desktop/src-tauri/src/commands/agents.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR partitions the agent definition store by workspace, closing a two-directional leak where definitions (managed agents, teams, global config) and their runtime connections were shared across relays.
Previously a single unscoped store lived at
agents/and everyapply_workspaceevent-synced all records to the newly selected relay, while the runtime reconciler fanned agents into every configured community. After an identity switch, in-flight old-owner inbound events could write into the new owner's store.What this PR does
WorkspaceAgentScope { scope_id, relay_url, owner_pubkey, definitions_dir, generation }as the single scope authority;scope_idis byte-identical to the retention DB's sha256 derivation via a shared helperapplyCommunity()anduseCommunityInit.tsAdoptedLegacy | LegacyClaimedByOther | FreshNoLegacymanifest, atomic rename, versioned_readymarker v1) that uses the existingretention.dbclaim as the canonical ownership ledger so retention and definition adoption can never diverge_readymarker: an unversioned or v0 marker forces re-run throughrun_pre_ready_family(retention migration + persona backfill) before advancing to v1, so scopes created by earlier defective pipeline iterations are repaired on next activation. Marker written via temp+rename (atomic)scope_for_arrivalto match(relay, owner)so in-flight old-owner inbound events cannot land in the new owner's store after an identity switchworkspace_transitionthrough runtime installation so switches cannot race a concurrent client start. Serve-mode runtimes are machine-level and never block a switch. The journaled Mesh recipe is tracked as a follow-up below.start_on_app_launchagents on every activation; enforce relay-match reuse onensure_relay_mesh_for_recordwith a fail-closed preflight error when a serve-mode runtime belongs to another relaycommunitiesparameter fromreconcile_managed_agent_runtimes; both frontend callers now invoke a parameterless command whose relay is derived server-side from the active scope, making cross-scope fan-out unrepresentableset_global_agent_config) captures scope at entry; Phase 1 validates generation inside the store lock before writing; Phase 2 per-agent restart validates under lock before stop — all I/O targets the captureddefinitions_dirviastart_local_agent_pairs_with_preflight_atconfirm_agent_snapshot_import,confirm_team_snapshot_import) capture owner keys at entry, verify pubkey against captured scope, and thread them through all mint/retention/engram phases. Re-verify under store lock before Phase 3a write. All outbound profile/memory publication usescaptured_scope.relay_urlcompensate_draintakes the caller's already-heldmanaged_agent_runtime_transitionguard by value, re-acquires only the store lock, validates captured scope generation, then restores journal entries viastart_pair_under_held_locks— closes the drop-then-compensate interleave window without recursive locking or an AtomicBool gatemigrate_persona_provider_to_runtimeruns as step 0 ofrun_scoped_migrations(returnsResult, propagates errors);migrate_agent_keys_to_dev_serviceruns inrun_pre_ready_family(debug non-test builds only, returnsResult). Pre-scope calls inrun_boot_migrations_innerremoved.workspace-degradedTauri event wired touseNestNotifications.ts(toast.error with cleanup + behavioral test); restore spawn emits the event on failure;spawn_event_syncreturn type corrected to()(fire-and-forget; dispatch failure during runtime shutdown has no toast surface)Deferred follow-up (tracked here, no separate issue)
Journaled Mesh recipe: client-mode Mesh runtimes have no persisted restart recipe (restore only knows Serve mode). The correct end-state is to include the Mesh client in the drain journal with ownership and restart data, coordinate drain via
rearm_lock → mesh_llm_runtimein the drain stage after prepare, and compensate synchronously on failure. This requires building the restart recipe for consumer-mode clients first. Option A (fail-closed switch while a client runtime is active, with user-accessible stop command) is the interim behavior for this PR.