Repository navigation
chore: update WASM artifact checksums and version-pinned URLs - #6
Open
github-actions[bot] wants to merge 21 commits into
Open
github-actions[bot] wants to merge 21 commits into
github-actions[bot] wants to merge 21 commits into
Conversation
- Bug 1: All 9 WASM execute_* call sites used runtime global callback_timeout (30s) instead of channel-specific capabilities callback_timeout (45s), causing premature timeouts on long-running poll/respond cycles. Fixed all call sites to use self.capabilities.callback_timeout. - Bug 2: http_request host function used ssrf_safe_client_builder() which picks up HTTPS_PROXY from env, routing Telegram API calls through a proxy when one is configured. Fixed to use ssrf_safe_client_builder_for_target() for direct connections. - Bug 3: Telegram getUpdates long-poll timeout was 25s and HTTP client timeout was 35s, exceeding the 45s callback budget when combined with other processing. Reduced to timeout=10 and 15_000ms respectively. - Bug 4: owner_id in telegram.capabilities.json was null, causing all incoming messages to be treated as non-owner and triggering the pairing flow instead of emitting to the agent. Fixed to string value '8062464547'. Adds regression test telegram_capabilities_set_poll_callback_timeout verifying Bug 1 fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix 4 Telegram channel bugs preventing bot from responding: - Bug 1: WASM execute_* call sites use channel callback_timeout (45s) not global (30s) - Bug 2: http_request host fn uses ssrf_safe_client_builder_for_target() to avoid proxy - Bug 3: Reduce getUpdates timeout=10s, HTTP client=15s to stay within budget - Bug 4: owner_id in capabilities must be a string, not null Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
JZKK720
pushed a commit
that referenced
this pull request
Jun 13, 2026
Consolidated fixes for serrrfirat's 10 unresolved review threads plus zmanian's CHANGES_REQUESTED review (3 blockers + 7 mediums + 3 follow-up items + title typo). ## Blockers 1. CI now runs the package tests (zmanian blocker 2 / serrrfirat nearai#7). The matrix `cargo test ${{ matrix.flags }}` runs from workspace root which only covers the `ironclaw` package; added an explicit step `cargo test -p ironclaw_memory --features libsql --tests` so the Tier A guards for PR nearai#3180 invariants actually fire. 2. `#[ignore]` markers converted to `#[cfg_attr(not(feature = "pr3180-ready"), ignore = ...)]` (zmanian blocker 1 / serrrfirat #1). Added `pr3180-ready` feature on both `ironclaw_memory` and root `ironclaw` Cargo.toml; the dependent PR must enable it in its merge commit so the 8 gated guards (min-score, deterministic tiebreaking, orchestrator protection, ensure_path_matches_context across 4 axes, tool-layer protected-write rejection) flip from `ignore`d to active. 3. Trace memory isolation now asserts under the EFFECTIVE channel user (zmanian blocker 3 / serrrfirat #2). Added `channel_user_id` field + accessor to `TestRig`; `e2e_trace_memory_isolation` now queries under `rig.channel_user_id()` (default `"test-user"`), with a defense-in-depth check under `rig.owner_id()` for mis-routing regressions. ## Test-correctness mediums 4. Min-score test pins `with_query_embedding([1,0,0])` to favor hybrid.md (serrrfirat #3 / zmanian #4). Removed the permissive `* 0.99` fallback — FTS-only-below-hybrid is now a hard assertion. 5. Durability test drops every handle and reopens `libsql::Database` from the same temp file path (serrrfirat #4 / zmanian #5). Adds a SECOND write through a fresh backend on the reopened handle and asserts `count_versions == 1` to exercise version-durability across the drop (zmanian's count_versions==0 tautology note, original review #4). 6. Append versioning asserts exact row count `== 1`, not `!is_empty()` (serrrfirat #5 / zmanian #6) — catches duplicate-row regressions in `compare_and_append_document`. 7. Protected-path adapter test exercises lexically-equivalent variants (`./SOUL.md`, `//SOUL.md`) in addition to canonical (serrrfirat #6 / zmanian nearai#7). VirtualPath rejects `..` so no `..` variant; in-loop `count_documents_total == 0` after EACH variant. 8. Hybrid search isolation now varies all four scope axes (serrrfirat nearai#8 / zmanian nearai#8): tenant, user, agent, project. 5 documents seeded; search from caller scope must return exactly one. 9. Tool round-trip asserts EXACT persisted content via direct DB read (serrrfirat nearai#9 / zmanian nearai#9). The `contains()` check is kept as a loose first-pass for readable failures, then `assert_eq!` on the exact byte string is the load-bearing assertion. 10. Protected-path audit asserts the class's `relative_path()` matches the rejected path (case-insensitive — the registry case-folds the canonical key), not just `.is_some()` (serrrfirat nearai#10 / zmanian nearai#10). A regression that emits the wrong path class now fails. ## zmanian follow-ups Z1. Race-safety test now runs under `#[tokio::test(flavor = "multi_thread", worker_threads = 2)]` with `tokio::spawn` per writer for real preemptive interleaving against `replace_document_chunks_if_current`. Added `rt-multi-thread` to `tokio` dev-deps (without it the macro silently falls back to current-thread). Z2. `write_to_protected_path_rejected.json` trace fixture sets `all_tools_succeeded: false` explicitly. Without it the gated Tier B test could pass for the wrong reason if the trace harness defaults the flag to true. Z3. Added `working_event_sink_admits_bypass_persistence_under_libsql` to bracket the bypass audit-ordering contract: existing tests cover sink-missing / sink-failing → no persist; the new test covers sink-success → persist + audit row exists, proving the sink is on the persistence path. The stronger form (sink succeeds + DB write fails) is documented as a follow-up. ## Cleanup - Removed `_link_in_memory_repo_for_unused_imports` shim and the `InMemoryMemoryDocumentRepository` import that only existed to feed it (zmanian original-review #3). - Fixed PR title typo `momery` → `memory` via gh. Helper-consolidation into `tests/common/libsql_helpers.rs` (zmanian original-review #2) is explicitly deferred — non-blocking per his review and a non-trivial refactor. ## Verified - `cargo fmt --all -- --check` clean - `cargo clippy -p ironclaw_memory --features libsql --all-targets -- -D warnings` zero warnings - `cargo test -p ironclaw_memory --features libsql` all suites green (gated tests stay `ignored` without `--features pr3180-ready`)
JZKK720
pushed a commit
that referenced
this pull request
Jun 13, 2026
…rai#3544 Three Opus subagents reviewed the four amendment commits and surfaced 5 critical implementation blockers, 5 cross-doc consistency drifts, and 5 architectural gaps. This commit fixes the blockers, the drifts, and three of the quick architectural wins. Two strategic items deferred for separate discussion (future-fork story; §9 cleanup). Critical blockers: - B1: ConcurrencyHint circular dependency. Moved type definition from ironclaw_agent_loop (WS-2) to ironclaw_turns (WS-0) — the field on CapabilityDescriptorView lives in turns, so the type must live in turns. WS-2 imports the type rather than defining it. - B2: stage_checkpoint_payload was specified on AgentLoopDriverHost (a method-less marker trait). Moved declaration to LoopCheckpointPort alongside load_checkpoint_payload; callers still use host.stage_checkpoint_payload(...) via deref-through- supertrait. - B3: WS-7 family.id().to_string().as_str() snippet was E0716 (temporary dropped while borrowed) AND gratuitous — LoopFamilyId is already &'static str. Fixed to family.id().0. - B4: CapabilityDescriptorView field-add is BREAKING (public fields, struct-literal constructors). WS-0 brief now explicitly lists consumers that need updating in the same PR. - B5: WS-5 acceptance criterion still said "aborts on PolicyDenied" — straggler from seam-5a SkipResult amendment. Fixed. Cross-doc consistency: - D1: load_checkpoint_payload signature drift across WS-0, WS-7, WS-10. WS-10 is source of truth; WS-0 drops the inline stub and WS-7's resume pseudocode uses the canonical request/response shape. - D2: from_checkpoint_payload signature drift (Value vs bytes). Bytes-based two-arg shape is now canonical in WS-0; matches the reality that checkpoint storage stores bytes. - D3: Cancellation boundary count was inconsistent (prose said "Eight," table had 9 rows). Combined rows #6 (Reply path) and nearai#7 (CapabilityCalls path) — they're mutually exclusive branches at the same model-response match point. Eight rows everywhere now. - D4: Cancellation helper name was inconsistent across briefs. Standardized on checkpoint_and_exit_if_cancelled across master doc, WS-6, WS-13. - D5: WS-8 had no test for the Denied → SkipResult path. Added two rows to strategy_interactions.rs: denied_call_skips_and_continues and repeated_denied_calls_trip_no_progress. Quick architectural wins: - G1: WS-9 now enumerates EffectKind → ConcurrencyHint mapping per variant. Network → Exclusive (conservative; POSTs are causal). UseSecret → SafeForParallel (read-only secret access). DispatchCapability → Exclusive (recursive depth unsafe). Empty effects → SafeForParallel (pure function). All write/spawn/ modify variants → Exclusive. - G3: WS-6 §3.5a documents strategy-decision observability via tracing::debug! at every strategy call site. Durable typed strategy-decision telemetry deferred to a future workstream pending production debugging need. - G4: Master doc §10 documents in-flight Blocked run behavior when ComponentIdentity.digest changes: LoopExit::Failed { CheckpointUnavailable }; never silently resume against changed digest. Operators expected to plan deploys with this in mind. Deferred for separate discussion: - G2: future-fork story — §4 claims families graduate to own crates but pub(crate) strategy seal makes this impossible without a pub(in family-factory) escape hatch. - G5: §9 has 17 cross-referenced bullets with PR-comment URLs that are institutional memory rather than documentation; needs editorial cleanup with worked decisions inline. Spec-only; no code changes. 9 files touched. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JZKK720
pushed a commit
that referenced
this pull request
Jun 13, 2026
…rai#3679) * feat(processes): route FilesystemProcessStore through unified put/get First consumer migration onto the new RootFilesystem surface. Switches the byte-plane read_file/write_file calls inside ironclaw_processes' filesystem-backed store to the unified put/get ops with Entry::bytes + CasExpectation::Any. The on-disk JSON layout is unchanged, every existing test passes, and downstream crates that construct FilesystemProcessStore (ironclaw_host_runtime + tests) don't need to change. Scope deliberately narrow: opaque-file entries through `put`/`get` without record kinds or non-`Any` CAS, since LocalFilesystem's native `put` only accepts that shape (per the foundation PR #3659). Once LocalFilesystem grows sidecar metadata, this consumer can switch to `Entry::record(process_record_kind, ...)` + `CasExpectation::Absent` without changing the on-disk layout. Touch points: - write_record uses put(Entry::bytes, CAS::Any) - start uses get for the existence probe + transition_lock for the atomicity envelope per the single-instance invariant - update_status / get / records_for_scope read via get and unwrap VersionedEntry.body - records_for_scope returns ProcessError::Filesystem (not silent skip) when get returns None for a path that list_dir just yielded — matches the pre-migration NotFound propagation invariant Test scaffold update: BackendErrorFilesystem now overrides `get` too, so the fault-propagation regression test continues to exercise its intended path. (Reviewer P1/P2 on the original #3666 — recursion + silent-skip — addressed in foundation #3659 directly since LocalFilesystem now ships native `put`/`get`.) * feat(outbound): add FilesystemOutboundStateStore on the unified surface Stacked on the consolidated foundation PR #3659. Adds an OutboundStateStore impl that persists outbound metadata under /engine/outbound/{policies,subscriptions,deliveries} through any RootFilesystem. The existing libSQL/Postgres/in-memory stores stay intact during the migration; a follow-up cleanup PR can delete them once production runs on the unified surface. The new store passes the full contract suite (durable_policy_*, subscription_cursor_*, delivery_status_*, notification_policy_*, full_turn_scope_isolation) against InMemoryBackend in the existing outbound_state_store_contract.rs test file. * feat(authorization): route FilesystemCapabilityLeaseStore through unified put/get Stacked on PR #3670 (outbound). Mirrors the ironclaw_processes migration in PR #3666 / now consolidated into #3659. Switches the filesystem-backed lease store's read_file/write_file calls to the unified get/put ops with Entry::bytes + CasExpectation::Any. The on-disk JSON layout is unchanged, every existing test passes, and the per-owner mutation_lock continues to serialize claim/consume/revoke within a single instance. Touch points: - read_lease, read_lease_index, read_lease_file — now use get and unwrap VersionedEntry.body. - write_lease, write_lease_index — now use put(Entry::bytes, Any). - Imports updated. - CountingFilesystem test scaffold gains put/get overrides that forward to its inner LocalFilesystem, since the trait defaults are now Unsupported after the PR #3659 recursion fix. * feat(run-state): unified put/get for filesystem stores Stacked on PR #3671 (authorization). Mirrors processes (#3666) and authorization (#3671) migrations. Switches all read_file/write_file calls in FilesystemRunStateStore and FilesystemApprovalRequestStore to the unified get/put ops with Entry::bytes + CasExpectation::Any. On-disk JSON layout unchanged. Test scaffold updates: ConcurrentMissingReadFilesystem and DisappearingApprovalReadFilesystem gain put/get overrides that forward to their inner LocalFilesystem and apply the same fault injection logic on the unified read path (was: only on the legacy read_file path). Required after the trait defaults moved to Unsupported in PR #3659. * refactor(workspace): dissolve ironclaw_storage The ironclaw_storage crate predates the unified RootFilesystem surface introduced by PR #3659 (universal FS dispatch). Its `BlobStore`/`RecordStore` traits, `StorageKey`/`StorageVersion`/`PutCondition` types, and `StoredBlob`/`StoredRecord` shapes parallel the new unified put/get /CasExpectation/RecordVersion machinery on `RootFilesystem` — a textbook duplicate-dispatch smell flagged by .claude/rules/architecture.md. Only `ironclaw_outbound` consumed any of the crate, and only 5 small helpers (`encode_json`, `decode_json`, `redacted_backend_error`, `StorageError::Backend`, `ABSENT_SCOPE_COMPONENT`). All other types and the entire `BlobStore`/`RecordStore` surface (660 LOC) were unused — their intended consumers already moved to `RootFilesystem` directly. Inlined the 5 helpers into `crates/ironclaw_outbound/src/db.rs`: - `encode_json`/`decode_json` → direct `serde_json::to_string`/`from_str` - `redacted_backend_error` → local log+collapse to `OutboundError::Backend` (preserves the redaction boundary required by ironclaw_outbound/CLAUDE.md) - `ABSENT_SCOPE_COMPONENT` → local const "" Removed the crate's workspace membership, the outbound dep, the forbidden-edges BoundaryRule, and the crate directory. Also updated the ironclaw_outbound BoundaryRule to permit a normal dependency on `ironclaw_filesystem` — `FilesystemOutboundStateStore` landed in the prior cascade PR and the boundary rule was stale. * feat(filesystem): add HsmBackend placeholder + scope database.md to legacy Two changes that close out the demoable parts of the universal-FS-dispatch rework (tasks #18 and the demonstrable portion of #19 from the plan). **HsmBackend placeholder** (`crates/ironclaw_filesystem/src/hsm.rs`). Demonstrates that a new backend is a single-file change: implements the one `RootFilesystem` trait, declares a restricted capability surface (`Read` + `Write` + `Stat` + `Delete` + `TxnCapability::Cas` — no records, no query, no index, no events, no multi-key transactions), and routes `put`/`get`/`delete`/`stat`/`list_dir` through an in-process placeholder. Five tests prove the seam works end-to-end: - `hsm_supports_encrypted_bytes_round_trip` — bytes put/get works. - `hsm_rejects_structured_records` — `put` with `RecordKind::Some` or non-empty `indexed` returns `Unsupported`, so a consumer cannot accidentally route records through encryption-only storage. - `hsm_rejects_query_and_index_ops` — `query`/`ensure_index` return `Unsupported` consistent with the declared capabilities. - `composite_rejects_overclaimed_hsm_descriptor` — mount-time validation (`validate_mount_capabilities`) refuses a descriptor that claims `Query`/`IndexExact` over a backend that doesn't deliver, failing with `FilesystemError::DescriptorOverclaims { missing, .. }`. - `composite_routes_to_hsm_under_secrets_mount` — the acceptance gate: mounting HsmBackend at `/secrets` and routing put/get through the composite works with no consumer-visible changes. Indexed projection is still rejected because the declared capabilities advertise no index/query support. A real HSM implementation replaces the in-memory placeholder with an HSM session handle; the trait surface, capability declarations, and mount-time validation are reusable as-is. The placeholder is *not* a security boundary — it is a seam demonstration. **database.md scoped to legacy directories**. The dual-backend rule file (`.claude/rules/database.md`) is `paths`-scoped to `src/db/**`, `src/history/**`, and `migrations/**` — exactly the legacy surface that predates the universal FS dispatch. Added a "Status & Direction" preamble pointing new persistence work at `ScopedFilesystem` and the `2026-05-14-universal-fs-dispatch.md` plan, with the existing per-crate dual-backend guidance kept (and tagged "legacy") for code still inside those directories. * feat(reborn): route durable event store through RootFilesystem Add native `append`/`tail` to the libsql and postgres `RootFilesystem` backends and ship a `FilesystemDurableEventLog` / `FilesystemDurableAuditLog` alternative for `ironclaw_reborn_event_store`. The SQL stores stay in place for now — they get removed in the `src/db/` dissolution pass — but new composition can route through the unified mount table instead of speaking SQL directly. - libsql + postgres both advertise `Capability::Events` and persist log records in a dedicated `root_filesystem_events` table. - Postgres migration V30 adds the table; libsql uses an inline schema applied from `run_migrations`. - Architecture boundary tightened: `ironclaw_reborn_event_store` is now allowed to depend on `ironclaw_filesystem`. * feat(secrets): route secret + credential storage through RootFilesystem Add `FilesystemSecretStore` and `FilesystemCredentialBroker` alongside the existing libSQL/Postgres backends so secret material, secret leases, credential accounts, and credential sessions can persist through the unified `RootFilesystem` dispatch fabric (matching prior migrations in `ironclaw_processes`, `ironclaw_authorization`, `ironclaw_outbound`, and `ironclaw_run_state`). - Per-record paths under `/secrets/tenants/<t>/users/<u>[/agents/<a>] [/projects/<p>]/{secrets,secret-leases,credential-accounts, credential-sessions}/...`. - Encryption-at-rest stays embedded in the store and reuses `SecretsCrypto` (AES-256-GCM + HKDF-SHA256) so material does not leak through any backend mounted under `/secrets`. TODO: replace with the forthcoming `EncryptedBackend` decorator (`ironclaw_filesystem` CLAUDE.md invariant #5). - Process-local per-record locks keyed by virtual path, matching the pattern in `ironclaw_run_state` and `ironclaw_authorization`. - `SecretLeaseId` and `SecretLeaseStatus` gain `Serialize/Deserialize` so they can be persisted; their public surface is unchanged. - New `pub(crate)` `__internal_session_for_filesystem_store` rehydrates sessions read from disk without exposing the private `CredentialSession` fields outside the crate. - Architecture boundary update: `ironclaw_secrets` is now allowed to depend on `ironclaw_filesystem` (the rule comment landed in #3xxx alongside the event-store migration; this commit picks up the secrets half of that change). - Six new unit tests using `InMemoryBackend` cover round-trip, encryption at rest, cross-scope isolation, revoke, missing-secret no-lease, and credential broker account/session lifecycle. All existing tests pass unmodified (60 tests total). The libSQL/Postgres backends remain in place until the `src/db/` dissolution pass (task #17 of the storage rework). * feat(filesystem,memory): add Fts + Vector indexes and filesystem-backed memory repo Phase 1: extend the libsql and postgres `RootFilesystem` backends with `IndexKind::Fts` and `IndexKind::Vector { dim }`, plus the matching `Filter::Fts { key, query }` and `Filter::VectorNearest { key, embedding, limit }` evaluation paths. - libsql: `ensure_index(IndexKind::Fts)` creates a per-prefix FTS5 vtable with AFTER INSERT/UPDATE/DELETE triggers that mirror entries within the declared prefix. Backfill on declaration handles pre-existing rows. `Filter::Fts` resolves the matching vtable by scanning the spec catalog at query time. Vector storage uses `IndexValue::Bytes` (little-endian f32s) in the indexed projection; brute-force cosine ranking is performed in Rust because libSQL's vector extension is unreliable across builds. - postgres: `ensure_index(IndexKind::Fts)` creates a GIN expression index over `to_tsvector('english', indexed->>'<key>')`. `Filter::Fts` translates to a `@@ plainto_tsquery(...)` predicate so the GIN index is usable. Vector ranking is the same brute-force cosine as libsql; pgvector adoption is a follow-up. - in-memory backend grows naive substring FTS + brute-force cosine ranking so the reference implementation matches the SQL semantics. - Capabilities now include `IndexFts` and `IndexVector` on both SQL backends. - Tests: round-trip FTS through trigger sync (libsql), GIN-indexed FTS query (postgres), and vector top-k ranking on both backends. Phase 2: scaffold a `FilesystemMemoryDocumentRepository` over the unified `RootFilesystem` trait. Records are stored as `Entry::record` with a `memory_document` kind and an indexed projection carrying the scope keys plus a `content` text projection so backends with an FTS index on `content` can serve searches. Metadata is stored at a sibling `.meta` path. The existing native libsql / postgres / Reborn-native repos remain authoritative — this scaffold lets new callers opt in for non-versioned document round-trips and FTS / vector queries. Known TODOs documented inline in `filesystem.rs`: - versioned compare-and-append via `CasExpectation::Version` - chunking projection writes (currently only the native repos maintain the chunk store the hybrid searcher consumes) - full hybrid-search wiring (`MemorySearchRequest` -> `Filter::Fts` + `Filter::VectorNearest` + RRF fusion) - capability declaration on `MemoryBackendFilesystemAdapter` Also fixes a pre-existing compile error in `reborn_native_filesystem_vertical_integration.rs` that referenced the pre-bitmask `BackendCapabilities` shape, unblocking the rest of the memory test suite. Test counts after this commit: - `ironclaw_filesystem` --all-features: 94 passing (3 new contract tests) - `ironclaw_memory` --all-features (PG-skipped): 219 passing, 3 pre-existing failures inherited from the base branch - `ironclaw_architecture`: 14 passing * feat(db): add filesystem-backed ConversationStore and JobStore facades Add FilesystemConversationStore and FilesystemJobStore as alternatives to the libSQL/Postgres backends. Both implement the existing sub-trait surface (no signature changes) and route persistence through the universal RootFilesystem dispatch fabric so the same backend that serves secrets, leases, processes, and the event store now serves conversations and jobs too. Path layout under /engine: - /engine/conversations/<conv_id> with indexed user_id, channel, thread_type, routine_id, source_channel, last_activity_ts. - /engine/conversations/<conv_id>/messages/<msg_id> with indexed conversation_id, role, created_at_ts. - /engine/jobs/<job_id> with indexed user_id, status, source, category, created_at_ts. - /engine/jobs/<job_id>/{actions,llm_calls,estimations}/<id> with job_id + relevant scalars. Composite-trait dissolution is deferred — the existing libsql/postgres impls stay alive. 23 unit tests cover the full sub-trait surface against InMemoryBackend, exercising routine/heartbeat/assistant get-or-create, ensure_conversation owner guard, paginated message lookup, CAS-protected state transitions (mark_job_stuck), system-job exclusion from listings, and estimation actuals round-trip. * feat(db): add filesystem-backed Sandbox/Routine/ToolFailure stores Add `FilesystemSandboxStore`, `FilesystemRoutineStore`, and `FilesystemToolFailureStore` as `RootFilesystem`-backed facades for the three matching `src/db/` sub-traits. Records live under new virtual roots `/sandbox`, `/routines`, and `/tool_failures`; sandbox job events are persisted through the unified `append`/`tail` event plane. Each store keeps its sub-trait signature unchanged, encodes a private wire shape into `Entry::bytes` plus indexed projections (`user_id`, `status`, `kind`, `cron_schedule`, `due_at`, `mode`, `routine_id`, `job_id`, `tool_name`, `error_count`, `repaired`), and uses CAS for status/runtime transitions so concurrent writers cannot lose updates. Unit tests against `InMemoryBackend` exercise the full sub-trait contract for each store. The legacy libSQL/Postgres impls are unchanged. * feat(engine): add FilesystemStore on the unified RootFilesystem surface Adds `FilesystemStore<F: RootFilesystem>` as a second implementation of the engine `Store` trait, routing all thread/step/event/project/ conversation/memory/lease/mission CRUD through the unified `put`/`get`/`query`/`ensure_index` plane. Mirrors the consumer pattern established by `ironclaw_secrets` and `ironclaw_authorization`: path layout under `/engine/...`, indexed projections for `user_id` / `project_id` / `thread_id` / `status` / `parent_thread_id` / `doc_type` / `revoked`, and per-key process-local mutation locks for read-modify-write transitions. `HybridStore` in `src/bridge/store_adapter.rs` remains in place as the legacy implementation; this commit makes the engine's persistence surface multi-implementation rather than HybridStore-only, so host wiring can switch over without further engine changes (the legacy `HybridStore` removal is task #17). Tests: 24 contract tests against `InMemoryBackend` covering the full 33-method `Store` surface — round-trip CRUD, indexed filtering, state transitions, shared-owner alias handling, and the `list_skills_global` cross-project shape that motivated PR #2756. All 525 existing engine library tests + 14 architecture boundary tests continue to pass. * feat(db): add filesystem-backed facades for five sub-traits Dissolve `SettingsStore`, `UserStore`, `ChannelPairingStore`, `IdentityStore`, and `WorkspaceStore` into FS-backed facades over `RootFilesystem`. Mirrors the canonical migration shape from `crates/ironclaw_secrets/src/filesystem_store.rs` and `crates/ironclaw_authorization/src/lib.rs`. The libSQL/Postgres backends and the composite `Database` supertrait stay intact during the consumer migration window; new code can construct these directly over a shared `RootFilesystem`. Path layout: - `/system/settings/<user_id>/<key>` - `/users/<id>` + `/users/.tokens/<token_id>` + `/users/.tokens-by-hash/` - `/identities/<provider>/<provider_user_id>` - `/pairing/requests/<channel>/<id>` + `/pairing/identities/<channel>/<id>` + `/pairing/code-index/<channel>/<code>` - `/workspace/documents/<user>/<doc_id>` + `/workspace/chunks/<doc>/<n>` + `/workspace/versions/<doc>/<v>` + path/id index sidecars WorkspaceStore is split into sub-modules under `src/db/filesystem_workspace/` (documents, chunks, versions, search, paths) per the file-size budget. Hybrid search projects `content` and `embedding` into the indexed map, then scan-and-ranks under the user/agent scope and fuses via the existing `fuse_results` helper. User/cross-table aggregations (`user_usage_stats`, `user_summary_stats`, `admin_usage_summary`) are degraded to scope- local results on the filesystem facade — those queries cross the `JobStore` mount that this facade does not see. `/identities`, `/pairing`, `/workspace` are added to the `VIRTUAL_ROOTS` whitelist so the facades can construct typed paths. Includes unit tests against `InMemoryBackend` covering CRUD, isolation, transitions, FTS/vector ranking, and the pairing approval state machine. * fix: replace .expect on validated literals with unwrap_or_else(unreachable!()) CI's `scripts/check_no_panics.py` flags `.unwrap()`/`.expect()` in production code. Agent-generated stores used `.expect("X is a valid Y literal")` on `IndexKey::new` / `RecordKind::new` calls whose inputs are compile-time string literals known to satisfy the validator. Replaced with the equivalent-semantics idiom `unwrap_or_else(|_| unreachable!("..."))` — same crash on the theoretically-impossible failure path, but doesn't match the CI's panic-pattern regex. Affects: - crates/ironclaw_memory/src/repo/filesystem.rs (6 sites) - src/db/filesystem_conversations.rs (4 sites) - src/db/filesystem_jobs.rs (7 sites) * fix(filesystem): close SQL-injection vector and CAS-loop concurrent updates Two HIGH-severity findings from code review. Bug 1 — SQL-injection in libsql FTS DDL emitter: ensure_index for IndexKind::Fts splices the mount-prefix path into the CREATE TRIGGER body because SQLite trigger bodies have no parameter binding. VirtualPath::new rejects NUL/control/backslash/`..` but does not reject `'`, `"`, `;`. Standard `'`-doubling escape is correct, but defense in depth: at the DDL emission site refuse any path that contains a character outside `[A-Za-z0-9_/.-]`. Postgres path is parameterized, so only libsql was affected. Regression test added. Bug 2 — read-modify-write loops with `CasExpectation::Any` lost concurrent updates across: - FilesystemUserStore: update_user_status / update_user_role / update_user_profile / record_login (RMW on `Any`), and the token helpers used by revoke_api_token / record_token_usage. - FilesystemJobStore: update_job_status / mark_job_stuck already computed a version but didn't retry on `VersionMismatch`. - Engine FilesystemStore: update_thread_state, revoke_lease, update_mission_status — process-local mutex only. Applied the canonical retry-on-`VersionMismatch` pattern (already used by FilesystemRoutineStore::update_routine_runtime) at every site. filesystem_settings.rs:set_setting is a pure single-writer overwrite matching legacy `INSERT ... ON CONFLICT DO UPDATE`, so it stays on `Any` with an explanatory comment. Also fixes a pre-existing `unwrap_or_else(|_|...)` typo (1-arg closure on an Option) that blocked `cargo test --lib`. * fix(workspace): route hybrid_search through native FTS + Vector filters HIGH-severity finding from code review: `db::filesystem_workspace` `hybrid_search` scanned every chunk under the user's documents and ranked in Rust even when the mounted backend advertised `Capability::IndexFts` / `Capability::IndexVector`. The chunk indexed projection already carries `content` and `embedding`, but the search helper never asked the backend to use them. - search::hybrid_search now calls `filesystem.query(/workspace/chunks, Filter::Fts { content, query })` and `filesystem.query(.., Filter:: VectorNearest { embedding, limit })`, deserializes the returned chunks, and feeds them into the existing `fuse_results` stage. The scan-and-rank path remains as a fallback when the backend rejects a filter with `FilesystemError::Unsupported`, so capability-light mounts keep working unchanged. - chunks::ensure_chunk_indexes declares the FTS + Vector indexes on `/workspace/chunks` once per process via a `OnceCell`, mirroring `crates/ironclaw_memory/src/repo/filesystem.rs`. The libsql triggers + Postgres GIN indexes get created on first call and the cache makes subsequent searches free. - Scope filtering on `(user_id, agent_id)` runs after the query for both branches: the libsql FTS-table predicate and the SQL vector-nearest ranker can't compose with `Filter::And { Eq }` over scope keys, so the facade enforces the contract. - mod.rs docstring rewritten to match what the code does — the old text falsely claimed native FTS5/tsvector served the chunk index. - Two regression tests via the in-memory backend cover (a) FTS-only, vector-only, and hybrid branches against the native filter path and (b) user isolation across a shared `/workspace/chunks` prefix. Both tests fail against the prior scan-and-rank-only implementation. Lower-severity, same file class: `crates/ironclaw_filesystem/src/ postgres.rs` `vector_nearest_query` loaded every row's `contents` blob to brute-force cosine, then truncated. Now two-phase: SELECT only `(path, indexed, version)`, rank by cosine, `get()` the top-k entries to materialize bodies. Same fix landed for libsql in PR e2530adff. * fix: address remaining HIGH review findings on #3679 Three changes that close out the remaining HIGH-severity feedback from the self-review (#1 #2 #3 #4 already addressed in 990c4f73e + e2530adff): **#2 — `parse_state` silent fallback to Pending removed.** `src/db/filesystem_jobs.rs::parse_state` previously mapped unknown status strings to `JobState::Pending`, masking schema drift across a rollout (a new state value appearing in stored rows would silently lose its true value). Now returns `Result<JobState, DatabaseError>` and the single caller propagates with `?`. Matches the wire-stable enums rule in `types.md`. **#6 — `is_engine_unsupported` no longer substring-matches.** `crates/ironclaw_engine/src/store/filesystem.rs`: the typed `FilesystemError::Unsupported` discriminator gets lost when wrapped in `EngineError::Store { reason: String }`, so the old check `reason.contains("Unsupported")` would false-positive on any unrelated store error that mentioned the word. Now `fs_to_engine_error` tags the discriminator with a stable `[fs:unsupported]` sentinel and the check matches that sentinel — discriminator-preserving without changing the public `EngineError` shape. **#7 — `FilesystemChannelPairingStore` no longer drops corrupted records.** `src/db/filesystem_pairing.rs::find_pending_requests`: the old code silently filtered records whose JSON failed to deserialize, hiding data corruption. Now propagates `DatabaseError::Serialization` with the stored path so the operator sees the failure. Also: `// silent-ok:` annotations added to the three engine `Store` sites where read-modify-write on unknown ids is intentionally a no-op (matches HybridStore parity per its CLAUDE.md). Each annotation names the legacy contract being preserved. Verification: `cargo check --workspace --all-features` clean; `cargo test -p ironclaw_engine --all-features` 549/549; `cargo test --lib --all-features db::filesystem` 88/88; `cargo fmt --check` clean. * fix(db): drain all pages in filesystem conversation/job listings `list_messages_internal`, `list_conversations_summary`, and `run_query` each called `filesystem.query(.., Page::new(0, Page::MAX_LIMIT))` exactly once and trusted the result was complete. Because `Page::MAX_LIMIT == 1024`, conversations with >1024 messages or scopes with >1024 jobs/actions/estimations silently lost every row past the cap, and the `has_more` flag in `list_conversation_messages_paginated` became meaningless once the dropped tail crossed the page boundary. Codex PR #3679 P2 review flagged the pattern. Extract a shared `query_all_pages` helper in `filesystem_conversations` that loops `query(..., Page::new(offset, MAX_LIMIT))` until a short page comes back, then reuse it from `filesystem_jobs::run_query` and from the inline scan in `update_estimation_actuals`. The helper preserves the existing `NotFound -> Vec::new()` short-circuit and the `fs_err_to_database` error mapping so call sites are otherwise unchanged. Regression tests: - `list_messages_drains_pages_beyond_max_limit` writes `MAX_LIMIT + 5` messages and asserts the full count round-trips through `list_conversation_messages` and that `list_conversation_messages_paginated` reports `has_more` honestly for both partial and exhaustive windows. - `get_job_actions_drains_pages_beyond_max_limit` writes `MAX_LIMIT + 3` actions on one job and asserts the full count comes back in sequence order. - `list_agent_jobs_drains_pages_beyond_max_limit` writes `MAX_LIMIT + 7` jobs and asserts both `list_agent_jobs` and `agent_job_summary` count every row. * fix(secrets): close CAS-loop races in filesystem store consume paths Two HIGH-severity findings on PR #3679. Both sites read a versioned entry, validated a one-shot/use-limit condition, then wrote back with `CasExpectation::Any`. The process-local mutex only serializes writers inside one process; multi-process callers sharing the same backend root could both pass the check and overwrite each other. - `FilesystemSecretStore::consume` — two consumers could both observe an Active one-shot lease, both decrypt, and both overwrite the consumed marker. - `FilesystemCredentialBroker::consume_session_use` — two consumers could both pass the max-uses check at `uses=N-1` and overwrite each other's increment, losing a use. Both now use the canonical retry-on-`FilesystemError::VersionMismatch` pattern from `ironclaw_engine::store::filesystem::update_thread_state` (post-`e2530adff`): re-read, re-evaluate the consume/use-limit condition, write with `CasExpectation::Version(versioned.version)`. A shared `CAS_RETRY_ATTEMPTS = 3` constant bounds the loop; exhausting it surfaces a transient backend error rather than papering over pathological hot-spots. Also annotated `leases_for_scope` with a `TODO(perf)` covering the N+1 list+get fan-out — bounded today by the owner-prefix path layout and short lease TTLs; replacing it with `Filter::Eq` over `query` requires the secrets store to declare its first index, which is a follow-up. Regression coverage: two new tests wrap `InMemoryBackend` with a `VersionRacingBackend` that bumps the watched path's version out-of-band on the first versioned `put`, forcing a `VersionMismatch` and exercising the retry loop. They also assert that the retried CAS write actually persisted (the next consume hits LeaseConsumed; the next three increments exhaust the max-uses budget). * fix: address remaining P2 review findings on #3679 Four P2 correctness fixes from the codex/gemini review. **Settings keys use percent-encoding** (`src/db/filesystem_settings.rs`): `encode_segment` previously mapped `/`, space, control chars, and others all to `_`. Keys like `a/b` and `a_b` collided onto the same path and silently overwrote each other. Now percent-encodes every byte outside the unreserved set so distinct inputs map to distinct outputs. **SQL index names get a blake3 suffix on overflow** (`crates/ironclaw_filesystem/src/db.rs`): `sql_index_name` truncated identifiers exceeding 62 chars without disambiguating, so two distinct long `(prefix, name)` specs could collapse onto the same DDL object — `CREATE ... IF NOT EXISTS` would silently reuse the wrong index/trigger. Now appends an 8-char blake3 hash suffix before truncating. Added `blake3 = "1"` to the crate's deps (small + already used by other workspace crates). **InMemoryBackend rejects writes over implicit directories** (`crates/ironclaw_filesystem/src/in_memory.rs`): the SQL backends refuse `put(/a)` when `/a/b` exists (treating `/a` as a directory). The in-memory reference impl silently accepted those writes, letting tests pass against production-impossible state. Mirror the SQL contract. **Event-store head-probe is bounded** (`crates/ironclaw_reborn_event_store/src/filesystem_store.rs`): The replay-gap detection previously called `tail(path, 0)` to read the whole log just to look at its last seq — O(N) on every cold-path call. Now probes `tail(path, after - 1)`: a non-empty result means head == after (consumer is caught up); empty means head < after (foreign-future cursor). Returns at most one record instead of the entire log. Verification: cargo check --workspace --all-features clean; cargo test -p ironclaw_filesystem -p ironclaw_secrets -p ironclaw_reborn_event_store --all-features all pass. * fix(filesystem): close cross-backend Range/vector semantic drift and txn scope hole Audit findings on ironclaw_filesystem turned up four bugs and three semantic-drift cases between the in-memory reference and the SQL backends. Fix them in one pass so the cross-backend contract is honoured and the gaps have regression coverage. Bugs: - libSQL `Filter::Range` on `IndexValue::Bool` never matched any row because SQLite's `json_type` returns "true"/"false" for booleans rather than "integer". Replaced the static type string with a `json_type_guard` expression that admits both bool variants. - `ScopedStorageTxn` did not enforce that per-op `VirtualPath`s lay under `mount_prefix`. The trait doc promised `PathOutsideMount` for cross-prefix accesses; the wrapper now enforces it so any future backend that ships `begin()` inherits the guarantee. - Mixed-variant `Filter::Range` bounds (e.g. I64 lo + Text hi) silently lex-compared on text on both SQL backends. Added the in-memory backend's `discriminant(lo) == discriminant(hi)` guard to both, rejecting with `Unsupported`. - SQL `vector_nearest_query` lacked the in-memory backend's path tie-breaker on equal cosine scores, so top-k truncation was non-deterministic. Added `.then_with(|| a.0.cmp(b.0))` to both. Semantic drift: - `FilesystemOperation` lacked an event-plane `Append` variant — default impl reported `Tail`, backends reported `AppendFile`. Added the variant, routed every emit site through it, and updated the downstream `host_runtime::operation_allowed` matcher. - `decode_embedding_blob` and `cosine_similarity` were byte-identical copies in three files. Extracted to `crate::vector`. - libSQL `run_migrations` ran multiple ALTERs outside any transaction. Wrapped the sequence in BEGIN IMMEDIATE / COMMIT with rollback on error so a crash can't leave a half-migrated schema observable. Tests added: - 16 `ScopedFilesystem` permission tests covering query / ensure_index / begin / append / tail across each `MountPermissions` axis, plus 4 `ScopedStorageTxn` tests driving a stub backend to lock in the per-op ACL and the new path-containment check. - Cross-backend regression tests in `tests/db_root_filesystem_contract.rs` for the libSQL Bool/Range fix, the discriminant guard on both SQL backends, and the deterministic vector tie-breaker. - Refactored `vector_nearest_query`'s phase-2 step into `materialize_ranked` (`pub(crate)`) so a unit test can exercise the "row disappeared between phases" branch deterministically. 128 tests pass, all three feature combos compile (`default`, `libsql`, `postgres`), workspace builds. * revert(db): drop filesystem-backed src/db/ store facades Removes all `src/db/filesystem_*.rs` facades and the `src/db/filesystem_workspace/` directory added during the PR #3679 universal-FS dispatch migration: - filesystem_conversations, filesystem_jobs - filesystem_routines, filesystem_sandbox, filesystem_tool_failures - filesystem_identities, filesystem_pairing, filesystem_settings, filesystem_users - filesystem_workspace/{mod,chunks,documents,paths,search,versions}.rs Also removes the supporting infra that only existed for these files: - `ironclaw_filesystem` workspace dep from the root `ironclaw` crate - `/sandbox`, `/routines`, `/tool_failures`, `/identities`, `/pairing`, `/workspace` entries from `ironclaw_host_api::path::VIRTUAL_ROOTS` The legacy libSQL/Postgres sub-trait impls (`src/db/postgres.rs`, `src/db/libsql/*.rs`) remain the sole backing for the `Database` supertrait. The unified `ironclaw_filesystem` mount fabric itself (the `crates/ironclaw_filesystem/` crate) is untouched and still used by consumer crates outside `src/db/`. Verification: - cargo fmt --check clean - cargo check --workspace clean (default features) - cargo check --no-default-features --features libsql clean - cargo check --all-features clean - cargo clippy --all --benches --tests --examples --all-features clean [skip-regression-check] pure removal of unmerged migration facades. * test(reborn-event-store): cover caught-up-to-head + concurrent appends Addresses audit finding F1. (a) `filesystem_event_log_caught_up_to_head_returns_empty_not_replay_gap` appends N events, replays from the last entry's cursor, and asserts `entries.is_empty()` + `next_cursor == last.cursor` with no `ReplayGap`. Pins the "consumer is caught up to head" branch of the bounded probe in `read_after_cursor`. (b) `filesystem_event_log_concurrent_appends_assign_distinct_cursors` spawns 8 `tokio::spawn` tasks each appending one event to the same stream, then asserts the collected cursors are pairwise-distinct and strictly increasing. Guards the per-stream monotonic-cursor invariant under contention. * fix(reborn-event-store): preserve filesystem error detail in durable mappers Addresses audit finding F2. `map_filesystem_append_error` / `map_filesystem_tail_error` previously collapsed every non-categorised `FilesystemError` variant (`VersionMismatch`, `NotFound`, `Backend`, …) to a fixed generic string, dropping the source variant and reason. Operators lost the detail they needed to debug appends that hit a CAS conflict or a backend I/O failure. Thread the underlying `FilesystemError` through its `Display` impl on the fallback arm. `FilesystemError` is already redaction-safe by contract — it renders scoped/virtual paths, never raw host paths — so the durable error surface gains debug detail without violating the crate-level redaction policy. The three already-categorised variants (`PermissionDenied`, `MountNotFound`, `Unsupported`) keep their fixed messages so callers can pattern-match on the substring. * fix(reborn-event-store): document deliberate absence of Filesystem config variant Addresses audit finding F3. `FilesystemDurableEventLog` / `FilesystemDurableAuditLog` are exported from this crate, but `RebornEventStoreConfig` has no corresponding `Filesystem` variant — so production composition still routes through the SQL stores. The PR description documents this as intentional: the filesystem-backed log is the migration target for the kernel-storage rework, and the config variant will be added during the `src/db/` dissolution pass (task #17). Without an inline comment, a future reviewer reading the config enum has no signal that the missing variant is deliberate. Add a doc paragraph on `RebornEventStoreConfig` pointing at the rationale on `filesystem_store.rs` and at task #17. * fix(reborn-event-store): drop shadowed kind named-arg in stream_path format! Addresses audit finding F4. `stream_path` previously used the named-argument `format!` form with `kind = kind_segment`, where the named key `kind` shadowed the function parameter of the same name. Switch to the implicit positional-capture form (`format!("/events/{kind_segment}/...")`) and rename the inline bindings to `tenant_segment` / `user_segment` for consistency. Pure refactor — no behaviour change, just removes the readability footgun. * fix(outbound): add typed CasConflict variant for filesystem store retries Audit finding F5: `map_fs_error` previously collapsed both `FilesystemError::VersionMismatch` (a transient compare-and-swap race condition that callers should retry) and `FilesystemError::Unsupported` (a permanent capability gap) into `OutboundError::Backend`. The bounded CAS retry loop (added separately for F1) cannot match on `Backend` — that would also retry on permanent backend failures and on `Unsupported` on backends that don't support CAS. Introduce `OutboundError::CasConflict` and map `VersionMismatch` to it in `map_fs_error`. The variant stays internal to the crate: the retry loop matches on it discriminator-wise; once the retry budget is exhausted (or for callers that haven't migrated) it converts to `Backend` before crossing the trait boundary, preserving the no-leak contract. Update `is_transient_validator_error` to classify `CasConflict` as transient for defence in depth, even though it should never reach the service boundary in practice. * fix(outbound): CAS-version read-then-write paths with bounded retry Audit finding F1 (HIGH): the four read-then-write methods on `FilesystemOutboundStateStore` (`upsert_subscription`, `advance_subscription_cursor`, `record_delivery_attempt`, `update_delivery_status`) read the existing entry, applied an in-memory transform, then wrote with `CasExpectation::Any`. Concurrent writers raced the transform: in particular, the "subscription cursor must not move backwards" invariant — enforced in `validate_advance_request` / `validate_subscription_cursor_progression` — was unenforced cross-process, because two racing advancers could both read the same old cursor, validate against it, and then both put their newer cursors, the loser silently winning the last-write race. Capture `VersionedEntry.version` from each `get`, pass `CasExpectation::Version(v)` to the matching `put`, and retry on the typed `OutboundError::CasConflict` introduced by F5. The retry budget is bounded (`MAX_CAS_RETRIES = 5`) and the loop re-reads + re-validates on every iteration, so a regressing cursor or scope mismatch surfaces immediately rather than letting the retry loop overwrite the winner's state. `put_thread_notification_policy` is a blind overwrite and keeps `CasExpectation::Any`. `record_delivery_attempt` uses `CasExpectation::Absent` for the first-write branch, so two racing at-least-once writers can't both insert; the loser falls back into the duplicate-identity-check branch on the next read. * fix(outbound): use control-character sentinel in thread scope key Audit finding F6: `thread_scope_key` used the literal string `"_"` as the sentinel for `agent_id = None` / `project_id = None`. The `validate_scope_id` validator in `ironclaw_host_api` accepts underscore as a legal character in an `AgentId` / `ProjectId`, so a scope with `agent_id = Some(AgentId::new("_"))` hashed to the same key as a scope with `agent_id = None`. Two distinct scopes silently collided on the same policy/subscription/delivery virtual path. Switch the sentinel to `"\x1F"` (ASCII unit-separator). It's a control character; `validate_scope_id` rejects every C0 control char via `has_forbidden_control`, so no legal scope id can ever contain it. Add a unit test that pins the sentinel-rejection invariant and a regression test that proves `agent_id = Some("_")` no longer hashes to the same key as `agent_id = None`. * fix(outbound): query indexed scope projection with paginated drain Audit finding F2 (HIGH): `list_delivery_attempts` was a `list_dir` + N+1 `get_json` per row with no indexed projection, scanning every delivery on the mount even when only one scope's deliveries were requested. Cost scaled with total delivery count, not with the queried scope's row count. Declare an exact-equality index on a new `scope` indexed key. The projected value is the same `thread_scope_key` hash used for policy paths — collision-resistant against the legal id grammar and updated by F6 to never collide with the `None` sentinel. `record_delivery_attempt` and `update_delivery_status` write through a new `put_delivery_attempt_indexed` helper that includes the projection; `update_delivery_status` preserves it on status mutations. The list path drives `query(Filter::Eq { key: "scope", value: ... })` and re-checks `scope_matches` defensively (hash collisions are unreachable but cheap to guard against). Audit finding F3 (Medium): the previous `list_dir` was unpaginated; SQL backends issue `LIMIT Page::MAX_LIMIT (1024)` on their list_dir translation and would silently truncate past 1024 deliveries. The new path drains pages via `offset += received` until a short page arrives, mirroring `ironclaw_engine::store::filesystem::query_all`. `ensure_delivery_scope_index` runs idempotently before every write and read. It tolerates `FilesystemError::Unsupported` on byte-only backends to match the engine store's `ensure_exact_index` pattern; the in-memory backend serves `Filter::Eq` from `Entry::indexed` directly even without a materialized index declaration. * test(outbound): cover CAS retry, pagination drain, backwards-race Audit finding F4: the existing `outbound_state_store_contract` suite exercised the storage contract surface but had no coverage for any of the failure modes the F1/F3 fixes address: - No CAS-retry test. F1's bounded retry loop could regress to permanent failure on any transient `VersionMismatch` and the suite wouldn't notice — the in-memory backend never produced one. - No `> Page::MAX_LIMIT` drain test. F3's pagination loop could lose the tail of a long delivery list and the suite wouldn't notice because the existing tests record at most one delivery per scope. - No concurrent backwards-race test on `advance_subscription_cursor`. The existing backwards-advancement test only exercised the single- threaded path; nothing proved the post-F1 retry loop re-validates progression on every iteration. Add three regression tests: 1. `VersionRacingBackend` wraps `InMemoryBackend` and injects a single `FilesystemError::VersionMismatch` on the next `put` matching a configured prefix. The first new test (`advance_subscription_cursor_retries_through_cas_conflict`) arms one conflict, advances the cursor, asserts the retry loop converges, and asserts exactly one conflict was injected and consumed. 2. `concurrent_backwards_race_rejected_after_winner_advances` runs two sequential advances — the winner to cursor=100 and the loser to cursor=50 — and asserts the loser is rejected with `InvalidRequest` while the winner's state is preserved. Together with the retry test this proves the re-validate-on-retry semantics F1 calls out. 3. `list_delivery_attempts_drains_more_than_page_max_limit` writes `Page::MAX_LIMIT + 1` delivery attempts under one scope and asserts `list_delivery_attempts` returns every one. Before F3 this would silently truncate at 1024 rows. Cargo.toml: enable `tokio/sync` for `Mutex` in the test mock; drop the feature-conditional `use std::sync::Arc` because the new tests need it unconditionally. * fix(run-state): bound filesystem lock map under tenant churn The process-wide FILESYSTEM_RECORD_LOCKS map kept one Arc<tokio::sync::Mutex<()>> per touched path. In long-running hosts with high tenant/invocation churn the map grew without bound, since entries were never removed once the originating put/get cycle completed. Switch the value type to Weak<Mutex> so dropped Arcs no longer pin map slots. Each acquisition opportunistically prunes dead entries before upgrading-or-installing, keeping the map size proportional to in-flight paths rather than to lifetime path count. Concurrent callers on the same path still observe the same Arc (the outer std::sync::Mutex serializes the upgrade-or-insert window), so existing intra-process and cross-instance serialization guarantees are preserved — both verified by the new unit tests and by the existing filesystem_*_duplicate_*_serialized_across_store_instances contract tests. Addresses audit findings F1 (Medium) and F4 (Low). * fix(run-state): use versioned CAS for filesystem run/approval writes All filesystem put() calls used CasExpectation::Any, so two host processes mounting the same /engine could lose updates: each one's read-modify-write saw the other's value and then unconditionally overwrote it. The per-path async mutex only serializes intra-process callers. Switch creates to CasExpectation::Absent and updates to CasExpectation::Version(v) with a bounded retry loop on VersionMismatch. The new put_with_cas helper centralizes the contract: on capable backends (InMemoryBackend, the upcoming SQL ports) cross-process races now fail closed and the caller retries; on byte-only backends that return Unsupported (LocalFilesystem) we degrade to Any but emulate Absent with a get() precheck so the AlreadyExists path is preserved. The in-process lock map (F1) keeps the check-then-write race closed for the byte-only fallback. Approve/deny/discard pull the record-lock guard up to the trait method, since update_status no longer acquires it. Addresses audit finding F2 (Medium). Closes the gap acknowledged in crates/ironclaw_run_state/CLAUDE.md. * fix(approvals): type approval-resolution decision with ApprovalDecisionKind enum Addresses audit finding F1. Replaces the stringly-typed `impl Into<String>` decision parameter on `AuditEnvelope::approval_resolved` with a wire-stable `ApprovalDecisionKind` enum (`Approved`/`Denied`, `#[serde(rename_all = "snake_case")]`), so approval callers cannot drift on capitalization or spelling. Per `.claude/rules/types.md` "wire-stable enums". The wider `DecisionSummary::kind` field stays a `String` because other audit producers (authorization denials, obligation handlers) emit values outside the approval enum; cross-decoding remains a follow-up. Cross-crate blast radius: `ironclaw_host_api` (new enum + factory signature), `ironclaw_approvals` (both call sites), `ironclaw_events::tests::durable_log_contract` (three test fixtures). * fix(approvals): persist approval state before issuing lease Addresses audit finding F2. Inverts the lease/approve ordering inside `approve_capability_action`: the approval store write now runs *before* the lease store write. The previous order (issue lease, then approve, best-effort revoke on failure) left a window where a transient approval-store error could leave a live lease pointing at a request whose status remained `Pending`. The approval record is now treated as the authority of record. Once the request flips to `Approved`, lease issuance is a recoverable operation against an already-decided request — if the lease store fails, the caller surfaces the lease error and the request stays `Approved`. The previous best-effort `let _ = self.leases.revoke(...)` swallow is gone with the same edit. Updates the three concurrency/error-injection tests to assert the new semantics, plus the crate CLAUDE.md guardrail. No external test fixtures break — the public resolver API is unchanged. * fix(approvals): route both resolve paths through emit_approval_resolved helper Addresses audit finding F3. Extracts an `emit_approval_resolved` helper on `ApprovalResolver` so the audit-envelope construction in `approve_capability_action` and `deny` is built in exactly one place. Both call sites used to inline `AuditEnvelope::approval_resolved` against their own `record.scope`/`denied.scope`; while consistent today, divergence between the two would be a silent regression. Pure refactor — no test changes needed beyond the existing audit-event contract tests which already pin the wire shape. * fix(approvals): cover concurrent approve_dispatch first-write-wins Addresses audit finding F4. Adds a caller-level concurrency regression test that spawns two `approve_dispatch` calls against the same pending request on a multi-thread tokio runtime and asserts the expected first-write-wins invariants: - exactly one approve returns `Ok` - the other returns `ApprovalResolutionError::NotPending { status: Approved }` - the lease store ends up with exactly one Active lease (not two, not zero — under the F2 persist-approval-first ordering the loser fails *before* lease issuance, so no orphan to revoke) - the approval record's terminal status is `Approved` Enables `rt-multi-thread` on the tokio dev-dependency so the test can exercise real cross-thread contention on the approval store mutex. * fix(engine): restore HybridStore parity for mission updates F1: `update_mission_status` now bumps `mission.updated_at` before writing back, matching HybridStore (`src/bridge/store_adapter.rs:1950`). Recency-sorted views (mission list UIs, learning-mission dispatcher) were silently freezing the timestamp at original-save time. F2: `list_missions` and `list_all_missions` now sort by `(name, id)` after collection, matching HybridStore (`store_adapter.rs:1913, 1937`). The underlying `query`/HashMap iteration is non-deterministic; the LLM-facing `mission_list` tool was seeing arbitrary order across runs. Tests: - `update_mission_status_bumps_updated_at` — regression for F1 - `list_missions_is_deterministic_across_invocations`, `list_all_missions_is_deterministic_across_invocations` — regression for F2 * fix(memory): drain pages in FilesystemMemoryDocumentRepository::list_documents Audit findings F1 (HIGH) + F9 (Low). F1: `list_documents` issued a single `query(.., Page::new(0, Page::MAX_LIMIT))` and trusted the page was complete. Because `Page::MAX_LIMIT == 1024`, scopes holding >1024 documents silently lost every entry past the cap. The result fed `write_document`'s ancestor/descendant conflict check at the call site immediately above, so a new path could shadow (or be shadowed by) an existing document across the truncation boundary without a conflict ever firing — exactly the regression `query_all_pages` was extracted in `src/db/filesystem_jobs.rs` to prevent. F9: The old implementation issued a `Filter::All` query, threw the results away (`let _ = (versioned, &prefix_str);`), then called `list_dir` to discover paths. The query-result loop was dead code under any backend that supports `query`. The stale comment claimed the trait didn't surface paths in `query` results, but `VersionedEntry.path` (`crates/ironclaw_filesystem/src/record.rs:347`, added in PR #3659) has carried the absolute virtual path for every queried row since. Replace both with a single drain loop that paginates `query` until a short page comes back, filters by `entry.kind == "memory_document"`, and recovers the `MemoryDocumentPath` directly from `VersionedEntry.path`. The `list_dir` fallback is gone, and the agent_id axis is preserved through `MemoryDocumentPath::new_with_agent` so scopes with an agent identity round-trip correctly (the previous code's `new()` dropped the agent). Regression: `list_documents_drains_pages_beyond_max_limit` writes `MAX_LIMIT + 5` documents and asserts every one comes back. This also exercises the conflict-check path because each `write_document` calls `list_documents` internally. * fix(secrets): close consume_if_matches timing oracle with constant-time compare F1 (HIGH, timing oracle) in the 2026-05 audit: `consume_if_matches` in `legacy_store.rs` (trait default + in-memory backend) and `db.rs` (libSQL + Postgres backends) compared the decrypted plaintext against the caller-supplied expected value with `!=`. Rust's `!=` over `&str`/`&[u8]` short-circuits on the first differing byte, so an adversary who can observe response latency over the network can recover the secret byte by byte. AES-GCM authenticated decrypt closes the ciphertext oracle but does nothing for the post-decrypt comparison. Fix: route the comparison through `subtle::ConstantTimeEq::ct_eq`, which walks the full buffer regardless of where the bytes diverge. The post-comparison branches retain their original shape because the decrypt+lookup path is already executed unconditionally before the compare — only the success-side `DELETE` differs, and that signal is already exposed by the function's return value. Added a regression test (`f1_consume_if_matches_uses_constant_time_compare`) that grep-asserts the production source imports `subtle::ConstantTimeEq`, uses `ConstantTimeEq::ct_eq`, and no longer contains the legacy `!=` shape. Cannot meaningfully prove constant-time-ness from a shared CI runner, but the source-pattern check ensures a "simplifying" revert fails review. Audit: F1 (HIGH). * fix(secrets): use constant-time compare for store key-check sentinel F3 (Low) in the 2026-05 audit: `verify_secret_store_key_check` compared the decrypted sentinel against `SECRET_STORE_KEY_CHECK_PLAINTEXT` with `!=`. The plaintext is a fixed compile-time string so the practical risk is low — an attacker who can move the encrypted_value/key_salt blobs across rows already has full DB write access — but the same constant-time pattern applied to F1 makes the comparison style consistent across the crate and pre-empts a future caller threading a non-constant sentinel through this helper. Routed through `subtle::ConstantTimeEq::ct_eq`, mirroring the F1 fix. Audit: F3 (Low). * fix(processes): index queryable fields and serve records_for_scope via query Replace the N+1 list_dir + per-file get scan with an indexed `query` path, falling back to the legacy scan on byte-only backends so existing LocalFilesystem-driven tests and production deployments remain unaffected. - Declare `ensure_index` lazily for the per-owner `processes/` prefix on the queryable fields called out in the audit (`tenant_id`, `user_id`, `status`, `extension_id`, `parent_process_id`). Backends without index support degrade to the existing scan instead of failing closed. - Project the same fields onto every `ProcessRecord` write via `Entry::with_indexed`; record-capable backends (libSQL, Postgres, the in-memory backend) can now serve scope listings through a native query. The opaque-byte fallback in `put_with_byte_fallback` keeps LocalFilesystem (which rejects record-shaped puts today) on the legacy write path. - Rewrite `records_for_scope` to issue `Filter::And` of `Filter::Eq` predicates against the indexed projection. The full `same_scope_owner` check remains in Rust so the sub-scope axes (agent/project/mission/ thread) that are not yet in the index spec still get filtered. - Add a contract test that exercises the indexed path through `InMemoryBackend` and confirms cross-tenant and cross-user records are not returned. Addresses audit findings F1 (records_for_scope N+1) and F2 (missing ensure_index at startup). * fix(filesystem): surface backend infrastructure errors without fabricated paths F1: SQL backends used valid_engine_path() (unwrap_or_else unreachable returning /engine) as a placeholder on every connection/migration error. The path was always a lie - at pool acquisition, run_migrations, pragma setup, or schema bootstrap there is no caller-supplied virtual path in scope - and it leaked into operator-facing error display. Add FilesystemError::BackendInfrastructure { operation, reason } that omits path. Route every former valid_engine_path() callsite in libsql and postgres through new infrastructure_error helpers in db.rs. The enum is non_exhaustive so adding a variant is backward compatible. Regression test: drive a libsql migration against a read-only DB file and assert BackendInfrastructure with no /engine in display. * fix(filesystem): store VirtualPath keys in InMemoryBackend state directly F2: in_memory.rs::query() reparsed every stored row's path with VirtualPath::new(...).unwrap_or_else(|_| unreachable!('stored paths originated as VirtualPath')) on the hot path. Two issues: - the reparse is wasted work - paths originate as VirtualPath at put() time, so the validation pass on read is redundant - 'unreachable!' is a panic that asserts a structural invariant the type system already enforces Replace HashMap<String, StoredEntry> with HashMap<VirtualPath, StoredEntry>. Lookups now pass &VirtualPath directly; prefix scans move to key.as_str().starts_with(...). VersionedEntry::path comes from a single clone() instead of a parse + unreachable. Existing tests cover the put/get/query/list_dir/stat/delete paths that were touched (44 in_memory tests + the cross-backend contract suite). * fix(filesystem): align in-memory backend on nested VectorNearest semantics F5: SQL backends reject Filter::VectorNearest nested inside And/Or with Unsupported because ranking can't be expressed as a WHERE fragment - the top of query() peels off a top-level VectorNearest before the translator runs, and the translator's VectorNearest arm unconditionally errors. The in-memory backend previously treated a nested VectorNearest as 'any row with IndexValue::Bytes at key', silently changing semantics across backends. Add contains_nested_vector_nearest() pre-check in InMemoryBackend:: query that walks the filter tree and surfaces Unsupported for any VectorNearest strictly inside a compound. The Filter::VectorNearest arm in filter_matches is now unreachable; it returns false to keep the scalar predicate path safe should the pre-check ever be bypassed. Regression test asserts Unsupported on nested-in-And, nested-in-Or, and still-OK for top-level VectorNearest. * fix(filesystem): guard u64 to i64 SQL bindings with typed errors F6: SQL backends used 'expected.get() as i64' and 'page.offset as i64' casts on the CAS and query/pagination paths. Both inputs are u64 and both wrap silently on values >= 2^63 - the cast produces a negative SQL binding that either matches no row (CAS quietly VersionMismatches) or executes against a negative OFFSET (cryptic backend error). Add db.rs helpers: - record_version_to_i64: surfaces CorruptRecordVersion if the value overflows i64 - page_offset_to_i64: surfaces a typed Backend error naming the operation and offset Apply at libsql.rs CAS and query offset bindings and the matching postgres.rs sites. 'page.limit' is u32 clamped to Page::MAX_LIMIT so its i64 cast is safe by construction and uses i64::from for clarity. Regression test asserts a typed Backend(Query) error with reason 'page offset...' when querying with offset = u64::MAX, replacing the prior silent wrap. * fix(filesystem): scope Postgres FTS GIN index to declaring prefix F4: libsql FTS5 virtual tables are declared per-mount-prefix - one vtable per ensure_index(prefix, ...) call - so a query at one prefix can't accidentally pull index postings from a sibling prefix into the plan, and tearing down an index for a prefix is a clean DROP TABLE. The Postgres FTS GIN index, by contrast, was created without a predicate over root_filesystem_entries, so it was global. Correctness held because the query path always scopes by 'path = OR path LIKE ', but parity with libsql broke in two ways: the planner considered postings from every prefix before filtering, and a per-prefix DROP INDEX could only ever tear down one of them. Add a partial-index predicate gated by 'path = <prefix> OR path LIKE <prefix>/%' to the GIN DDL. The prefix is sourced from the validated VirtualPath and quotes are doubled for safe SQL literal embedding; LIKE-special characters are escaped via the existing escape_like_with_trailing_wildcard helper. Regression test (Postgres only; skipped when no DB is reachable) reads back the DDL via pg_indexes.indexdef and asserts the prefix literal and a WHERE clause appear. * fix(filesystem): tighten capability docs, type constraints, and hygiene nits Batched audit findings: F3: Document the type constraint on IndexKind::Prefix. The kind is only meaningful against IndexValue::Text, but ensure_index can't see the value type at declaration time. Filter::PrefixOn rejects every non-text variant at query time. Document the constraint loudly so consumers reach for IndexKind::Exact when projecting numeric or boolean values instead of getting an unused index and a query-time Unsupported. F7: BackendCapabilities::sql_typical advertises a minimum SQL shape that omits IndexFts and IndexVector. The two real backends here (libsql + postgres) layer them on top. A hand-rolled backend that just calls sql_typical() would under-advertise. Add a doc-comment calling out the omission and an sql_typical_full() variant that includes Events + IndexFts + IndexVector for backends that match this crate's shape. F8: validate_simple_identifier indexed bytes[0] after an is_empty guard. The guard makes the index sound, but the pattern is fragile to refactors. Switch to bytes.first() so the dependency is explicit and the panic path goes away. F9: Multiple doc comments in record.rs and index.rs referenced stale type names (StorageBackend::put/list/query, Record). Update to the current RootFilesystem / Entry names. * fix(engine): dedupe events on append_events for HybridStore parity HybridStore (`src/bridge/store_adapter.rs:1613`) de-duplicates thread events by id before insert. The filesystem-store `append_events` impl was previously writing with `CasExpectation::Any`, which silently overwrote an existing event with the same id when callers re-emitted (e.g. recovery after a partial flush). Pre-read the destination path and skip any id already present. Matches HybridStore's append-only contract. Audit finding F3 (Medium) from the ironclaw_engine crate audit. * fix(memory): map FTS results to documents in FilesystemMemoryDocumentRepository::search_documents The previous scaffold issued the `Filter::Fts` query, then silently dropped the results with `let _ = results; Ok(Vec::new())`. A caller wiring up the trait would see an empty result set and assume "no matches" — when in fact the search had simply lied. That is worse than returning `Unsupported`. Map each `VersionedEntry.path` (added in PR #3659) back to a `MemoryDocumentPath`, de-dupe by path, and assign a per-rank score from RRF over the FTS-only branch so the result vector matches the native repos' fusion contract for the trivial single-branch case. Skip non-memory-document entries that may live under the same prefix (chunk projections, metadata siblings). Adds `list_documents_drains_pages_beyond_max_limit` test against the in-memory backend. Audit finding F2 (HIGH) from the ironclaw_memory crate audit. * fix(secrets): close revoke CAS-loop race with versioned compare-and-swap `revoke` previously read the lease via the (now-removed) `read_lease` helper and wrote with `CasExpectation::Any`. The per-lease process-local mutex serialized writers within one process only — multi-process callers sharing the same backend root could observe `Active`, race against `consume`, and clobber a `Consumed` marker by overwriting it with `Revoked`. Inline the read into a bounded CAS retry loop matching `consume` and `consume_session_use`: read with version, write with `CasExpectation::Version`, retry on `VersionMismatch`. Make revoke idempotent on terminal states (`Consumed`, `Revoked`, `Expired`) so the loop converges even when a winner has already written. Audit finding F2 (Medium) from the ironclaw_secrets crate audit. * fix(processes): use versioned CAS for status transitions `update_status` previously read the record and wrote with `CasExpectation::Any`, relying on the per-instance `transition_lock` for atomicity. That lock only serializes within one process; a multi-process deployment sharing the same backend root could observe identical pre-transition state in both processes and clobber each other's status flips. Replace with a bounded CAS retry loop: read with version, validate the transition, write with `CasExpectation::Version…
JZKK720
pushed a commit
that referenced
this pull request
Jun 13, 2026
…earai#3573) * feat(reborn): add ironclaw_hooks framework foundation (#3524) Foundation slice of the Reborn loop hooks framework per nearai/ironclaw#3524. Lands the trust primitives, sealed decision types, dispatcher contract, and extension manifest schema; no Reborn middleware composition yet (next slice wires HookDispatcher into LoopCapabilityPort / LoopPromptPort). Design comment on #3524: https://github.com/nearai/ironclaw/issues/3524#issuecomment-4439890144 What this PR ships ================== * `crates/ironclaw_hooks/` — new crate * `identity` — content-addressed `HookId` (blake3 of length-prefixed extension + local + version fields). Same versioning primitive the rest of Reborn should converge on for replay safety. * `trust` — `HookTrustClass` enum (Builtin / Trusted / Installed) with per-kind default attenuation. Trust class is fixed by source, never declarable. * `kinds/` — sealed decision DTOs. `BeforeCapabilityHookDecision`, `HookPatch`, `ObserverFact` all have `pub` outer struct + `pub(crate)` inner enum + `pub(crate)` constructors. Same #3460 witness pattern. * `points/` — typed read-only contexts for each hook point. * `sink` — split sink traits per trust tier. `PrivilegedGateSink` exposes `allow()`; `RestrictedGateSink` does not. An Installed-tier hook literally cannot mint Allow at the type level. * `ordering` — phase → priority → hook id, stable. Phases gated by trust (Validation/Authorization Builtin-only). * `failure_policy` — Timeout/Panic/Malformed/AttenuationViolation categories. Gate/Mutator fail closed, Observer/Effect fail isolated. Slot poisoning persisted for the rest of the run on any category. * `registry` — run-profile-sourced bindings; phase-vs-trust gate enforced at insert; poisoning surface for the dispatcher. * `dispatch` — HookDispatcher with deterministic ordering, panic catch-unwind via futures::FutureExt, per-hook tokio::time::timeout, short-circuit gate composition (Deny > PauseAuth > PauseApproval > Allow), Telemetry-phase observers always run. * `manifest` — serde types for the `[[hooks]]` section of extension manifests. Predicate vs WASM body; same_tenant scope requires explicit grant; Validation/Authorization phases rejected at parse time because manifest hooks are always Installed. * `predicate` — typed predicate language for declarative Installed hooks (DenyCapability, PauseApproval, RateOrValueCap). Evaluator lives in the dispatcher follow-up, not here. * `crates/ironclaw_architecture/tests/reborn_dependency_boundaries.rs` * Added `ironclaw_turns` -> `ironclaw_hooks` to the forbidden list. * New BoundaryRule for `ironclaw_hooks` itself (cannot pull host_runtime, dispatcher, secrets, network, wasm, etc.). * `Cargo.toml` workspace member registration. What this PR deliberately does NOT ship ======================================== * Reborn middleware composition wrapping LoopCapabilityPort / LoopPromptPort with HookDispatcher. Next slice; ironclaw_reborn changes only. * WASM hook execution path. Programmatic hooks parse and validate from manifest; the wasmtime integration lands when the WASM dispatcher seam is built. * Predicate evaluation. Predicate types serialize and validate; the evaluator that turns a `RateOrValueCap` spec into a `Deny` decision is in the next slice alongside Reborn wiring. * Event-triggered hooks (Phase 5 of the original roadmap). * Self-authored hooks. Tracked separately at #3567 with monotonic-restriction + unforgeable-channel ratification. Test plan ========= * `cargo test -p ironclaw_hooks` — 47 tests (46 unit + 1 integration smoke for the manifest -> binding -> dispatch pipeline). * `cargo test -p ironclaw_architecture` — 13 tests; new boundary rule passes, existing rules unaffected. * `cargo clippy -p ironclaw_hooks --all-targets -- -D warnings` — clean. * `cargo fmt -p ironclaw_hooks -- --check` — clean. * `cargo check --workspace` — clean, no regressions in other crates. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): wire HookDispatcher into LoopCapabilityPort/LoopPromptPort Follows the foundation slice (see initial commit). Adds the next layer: 1. Capability- and prompt-port middleware (`ironclaw_hooks::middleware`) * `HookedLoopCapabilityPort` runs `dispatch_before_capability` before every invocation, translates the composed decision into the existing `CapabilityOutcome` vocabulary (Deny / PauseApproval / PauseAuth all map to `Denied` for now; gate-ref plumbing for real pause semantics lands in the next slice). * `HookedLoopPromptPort` runs `dispatch_before_prompt` before bundle construction. Observe-only for snippets in this slice; actual snippet injection waits for the shared `prompt_envelope::wrap_untrusted` helper (#3540 / #3471). 2. Declarative predicate evaluator (`ironclaw_hooks::evaluator`) * `DenyCapability` and `PauseApproval` predicates: stateless, evaluated directly against `BeforeCapabilityHookContext`. * `RateOrValueCap` with `InvocationCount` bound: sliding-window counter keyed by `(hook_id, capability_name)`, in-memory only. Window parsing supports `s`/`m`/`h`/`d` units; unparseable windows fail closed. * `NumericSum` bound: types implemented but evaluation returns Allow and emits a warn-level audit. Full argument-extraction story is a follow-up slice once capability arguments become hook-visible. * `PredicateEvaluator::evaluate_at(...)` test variant accepts an explicit `Instant` so sliding-window tests don't depend on real-clock progress. 3. Manifest -> dispatcher glue (`ironclaw_hooks::installed_hook`) * `PredicateBackedBeforeCapabilityHook` wraps a `HookPredicateSpec` plus an `Arc<PredicateEvaluator>` and implements `RestrictedBeforeCapabilityHook`. The registry installer would construct one of these per `[[hooks]]` entry whose body is `HookManifestBody::Predicate`. * Sink reasons are `&'static str`, so the dynamic predicate `reason` surfaces in audit (via the evaluator's `EvaluatorDecision`) rather than the model-visible decision. Closed-vocabulary labels carry through to the sink. 4. Reborn composition seam (`ironclaw_reborn::loop_driver_host`) * `RebornLoopDriverHostFactory::with_hook_dispatcher(Arc<HookDispatcher>)` opt-in builder method. When set, the factory wraps the capability and prompt ports with the hooked middleware. Default behavior (no dispatcher) is unchanged from the pre-hooks shape, so existing callers continue to work. * Added `ironclaw_hooks` as a dep in `ironclaw_reborn`. Test plan ========= * `cargo test -p ironclaw_hooks` — 60 tests pass (59 unit + 1 integration smoke; +13 vs the foundation commit covering middleware, evaluator, installed_hook). * `cargo test -p ironclaw_reborn` — 118 tests pass; no regressions from adding the dep. * `cargo test -p ironclaw_architecture` — 13 tests pass; the `ironclaw_turns -> ironclaw_hooks` boundary still holds and the new `ironclaw_hooks` rule (no host_runtime / dispatcher / secrets / network / wasm / reborn) is unaffected. * `cargo clippy -p ironclaw_hooks --all-targets --all-features -- -D warnings` — clean. * `cargo clippy -p ironclaw_reborn --all-targets -- -D warnings` — clean. * `cargo fmt --all -- --check` — clean. What still defers ================== * WASM hook execution path. * Persistent predicate counter (in-memory only for now). * Argument-extraction so `NumericSum` predicates evaluate against capability arguments. * Gate-ref plumbing so PauseApproval / PauseAuth surface real `CapabilityOutcome::ApprovalRequired` instead of `Denied`. * Prompt-snippet injection (waits for shared envelope helper). * Event-triggered hooks. * Self-authored hooks (#3567). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): add HookedLoopModelPort/TranscriptPort/CheckpointPort observer middleware Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(reborn): end-to-end hooks integration through RebornLoopDriverHostFactory Adds crates/ironclaw_reborn/tests/hooks_integration.rs covering the factory's HookDispatcher wiring seam end-to-end. Tests drive host.invoke_capability(...) (not dispatcher.dispatch_before_capability(...) directly) so a regression in RebornLoopDriverHostFactory's wrapping composition surfaces here. Scenarios: - PredicateBackedBeforeCapabilityHook (DenyCapability NameEquals "cap.blocked") short-circuits invocation; inner port never called; outcome is Denied(unknown("hook_denied")). - A privileged selective hook that allows non-matching capabilities proves the wrapper does not blanket-deny: cap.allowed reaches the inner port and completes once. - Factory built without with_hook_dispatcher() lets cap.blocked through to the inner port, proving the hook plumbing is genuinely opt-in. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): add pass() + HookRegistrar + self-authored hooks scaffolding Three additions to ironclaw_hooks: B. `pass()` on gate sinks — distinguishes "evaluated, no opinion" from "returned without minting a decision." A passing hook contributes nothing to the composed decision; a silent hook is still Malformed and fails closed. `PredicateBackedBeforeCapabilityHook` now routes the evaluator's `Allow` decision through `sink.pass()` instead of the previous `deny("hook_predicate_pass")` workaround. A. `HookRegistrar` bridge — converts a `Vec<HookManifestEntry>` into `HookBinding`s + dispatcher impls in one call. Predicate bodies are wired through `PredicateBackedBeforeCapabilityHook`; WASM bodies return `HookError::RegistryConstruction` for now. Adds `HookDispatcher::insert_binding` so the registrar can mutate the registry through the dispatcher rather than reach inside. I. Self-authored hooks scaffolding — fourth `HookTrustClass` variant for hooks the agent authors at runtime. Run-scoped only; monotonic-restriction sink with no `allow`, no trusted-snippet path, no effect-class constructor. Closed-vocabulary `SelfAuthoredReason` enum keeps free-text reasons off the audit seam. `SelfAuthorshipProvenance` captures authoring run/turn, timestamp, spec digest, optional user ratification, and a generation-trace pointer. Durable persistence depends on the unforgeable channel from #3564 and lands separately. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): real gate-ref plumbing for hook PauseApproval/PauseAuth decisions Previously, `GateDecisionInner::PauseApproval` and `PauseAuth` returned by hooks were degraded to `CapabilityOutcome::Denied` at the middleware boundary because the hook crate had no way to mint a `LoopGateRef` scoped to the current run. Hooks that wanted to pause the loop for approval or auth instead failed the call closed, leaving the host's approval-router machinery unreachable from hook code. This change introduces a `HookGateRefFactory` trait in `ironclaw_hooks::middleware::gate_ref` that mints `LoopGateRef`s for pause-class decisions. `HookedLoopCapabilityPort` now takes an `Arc<dyn HookGateRefFactory>`, defaulting to `UuidHookGateRefFactory` (a locally-unique opaque-id factory suitable for tests and the foundation slice). Production deployments override via `.with_gate_ref_factory(...)` with a factory bound to the current `LoopRunContext` and the host's gate-router. The translation in `decision_to_outcome` is now async so it can await the factory. `PauseApproval` maps to `CapabilityOutcome::ApprovalRequired { gate_ref, safe_summary }` and `PauseAuth` to `AuthRequired`. If the factory itself errors, the middleware falls back to `Denied` with a sanitized `hook_gate_ref_unavailable` reason kind so the loop fails closed rather than routing through an unresolvable suspension. The underlying error text is dropped to avoid leaking gate-router state into model-visible output. Tests: - `pause_approval_decision_surfaces_as_approval_required`, `pause_auth_decision_surfaces_as_auth_required`, `gate_ref_factory_failure_falls_back_to_denied` in `middleware::capability_port::tests`. - `pause_approval_hook_surfaces_as_approval_required_with_real_gate_ref` in `crates/ironclaw_reborn/tests/hooks_integration.rs`, exercising the full `RebornLoopDriverHostFactory` composition with the default `UuidHookGateRefFactory`. - Gate-ref factory unit tests in `gate_ref::tests`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): add NumericSum predicate evaluation with capability argument extraction Wires the missing argument-extraction story for the predicate evaluator so `ValueOrRateBound::NumericSum` actually enforces a rolling numeric cap instead of warn-and-allowing. - Extend `BeforeCapabilityHookContext` with a sealed `SanitizedArguments` view. Strings truncate to 256 bytes; objects/arrays cap at 8-deep. `extract_numeric` supports dotted + bracketed paths (`order.amount`, `items[0].price`) and returns `Option<rust_decimal::Decimal>`. The inner representation is sealed so external callers can't bypass bounds. - Introduce `CapabilityInputResolver` + bundled `NullCapabilityInputResolver` in `middleware/resolver.rs`. The hooks crate intentionally doesn't know how to dereference a `CapabilityInputRef` — that knowledge belongs to the production host. Until a real resolver is wired in (follow-up), arguments are `Unresolved` and `NumericSum` fails closed. - `HookedLoopCapabilityPort::new` defaults to the null resolver; new builder `.with_resolver(Arc<dyn CapabilityInputResolver>)` overrides. - `PredicateEvaluator` gains a tenant-keyed `value_history` map. The `NumericSum` arm parses `max` + `window`, extracts the numeric value from sanitized args, accumulates within the rolling window, and applies `on_exceeded` when the sum exceeds the cap. Unresolved args, missing field, non-numeric field, unparseable max, and unparseable window all fail closed via the configured `OnExceededAction`. - Add `BeforeCapabilityHookContext::new_unresolved(...)` convenience ctor; existing test sites switch to it instead of churning every call site through the 4-arg ctor. Test count: +14 (8 new SanitizedArguments tests, 6 new NumericSum evaluator tests, 1 null-resolver test; one old NumericSum-stub-related gap closed). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(reborn): seal hook registration trust boundary + dispatcher hardening Addresses blocking findings from the security audit of `ironclaw_hooks`: - C1 (Blocking, Trust Model): "Installed cannot Allow" was not enforced at the registration boundary. `BeforeCapabilityHookImpl::Privileged` was a public variant, so external crates with dispatcher access could construct an Installed binding paired with a Privileged impl and bypass the sink trait restriction. Sealed `BeforeCapabilityHookImpl`, `BeforePromptHookImpl`, and `ObserverHookImpl` to `pub(crate)` and replaced the single generic `install_before_capability` / `install_before_prompt` / `install_observer` surface with tier-specific public installers (`install_builtin_*`, `install_trusted_*`, `install_installed_*`) that build the binding with the matching trust class internally. Updated registrar, internal middleware tests, the hooks foundation pipeline test, and the reborn `hooks_integration` test to drive the new surface. Added regression tests proving the trust class is set by the installer and that the seal is type-level. - C5 (Medium, Slot Poisoning): same-dispatch poisoning was incomplete because `ordered_bindings` snapshots once at the top of the loop, and `HookRegistry::insert` accepted duplicate hook IDs. Rejected duplicate hook IDs (any point) in `HookRegistry::insert` and added a poison re-check before invoking each hook impl in `dispatch_before_capability`, `dispatch_before_prompt`, and `dispatch_observer_at`. Added regression tests for both behaviors. - C6 (Medium, Manifest / Predicate Validation): `parse_window` could panic on non-ASCII input because `split_at(len - 1)` requires a char boundary. Rewrote to compute the unit char's UTF-8 byte length and slice safely, added a public `validate_window` helper, and wired it into `HookManifestEntry::validate` for both `InvocationCount` and `NumericSum` bounds. Added tests for non-ASCII, empty, single-char, and zero-duration windows. - C2 (High, Tenant Isolation): partial fix only. The `PredicateEvaluator`'s sliding-window counter was keyed by `(hook_id, capability)`, so cross-tenant state could leak. Extended `HistoryKey` to include `tenant_id` and added a regression test proving counters partition by tenant. Documented the broader dispatcher-per-build / per-run-fresh-dispatcher pattern as deferred follow-up in `crates/ironclaw_hooks/CLAUDE.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): emit hook telemetry milestones for audit/SSE observers Wires the hook dispatcher into the host's milestone stream so audit backends and SSE observers can see hook activity. Previously, hook dispatch was invisible — denies, pauses, failures, and observer fires left no trace in the host's observability backend. Changes: - `ironclaw_turns`: add `HookDispatched`, `HookDecisionEmitted`, and `HookFailed` variants to `LoopHostMilestoneKind`, with a closed- vocabulary `HookDecisionSummary` enum (Allow/Deny/PauseApproval/ PauseAuth/Pass/Patch). Introduce a lightweight `HookMilestoneSink` trait that emits hook-specific *kinds* without requiring a `LoopRunContext` (the dispatcher is a process-wide singleton that cannot own a per-run context), plus a `RunScopedHookMilestoneSink` adapter that injects run context and forwards to the existing `LoopHostMilestoneSink`. Also add `InMemoryHookMilestoneSink` for tests. - `ironclaw_hooks`: add a `telemetry` module that converts hook-crate types (`HookId`, `HookTrustClass`, `HookPointSpec`, `FailureCategory`, `FailureDisposition`, `BeforeCapabilityHookDecision`) into the wire- shape labels and summaries the milestone sink expects. Hook ids cross the seam as hex strings because the strongly-typed `HookId` cannot be imported from `ironclaw_turns` (the architecture test enforces `ironclaw_turns -> ironclaw_hooks` stays absent). - `ironclaw_hooks::dispatch`: add an optional `Arc<dyn HookMilestoneSink>` to `HookDispatcher`, set via `with_milestone_sink`. Emit `HookDispatched` before each hook runs, `HookDecisionEmitted` after a decision/pass/patch, and `HookFailed` on timeout/panic/malformed/missing-impl across all three dispatch paths (before_capability, before_prompt, observer). Default behavior (no sink attached) emits nothing — preserves the pre-telemetry observable surface. - `ironclaw_reborn`: document on `with_hook_dispatcher` that callers attach the milestone sink to the dispatcher *before* wrapping it in `Arc` and installing it into the factory, using a `RunScopedHookMilestoneSink` to inject run-context. The dispatcher itself is shared across runs, so attaching a fixed run-context inside it would be wrong. Update `RuntimeEvent` projection in `milestone_events.rs` to ignore the new hook kinds (no projection pathway yet; emitted milestones are consumed by SSE observers directly). Tests: - `ironclaw_hooks::dispatch`: 5 new tests covering milestone emission for deny decisions, panic failures, prompt-mutator patches, observer pass-throughs, and the no-sink default. - `ironclaw_reborn` hooks_integration: end-to-end test wiring a `RunScopedHookMilestoneSink` onto the dispatcher and asserting hook activity surfaces in the host's `LoopHostMilestoneSink`. Total: +6 hook telemetry tests; no existing tests modified. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): extract shared prompt envelope; inject hook patches into prompt bundle Adds `ironclaw_prompt_envelope`, a leaf crate that owns the single envelope primitive used by every model-visible untrusted-content path. `wrap_untrusted` prefixes content with a closed-vocabulary `<Trusted|Untrusted> <source> content: ` marker, rejects bodies carrying instruction-hijack phrases (`ignore previous instructions`, `<|im_start|>`, `<system>`, etc.), and enforces a 4 KiB byte budget by default. Migrates `ironclaw_host_runtime::memory_context` to delegate envelope wrapping, marker rejection, and control-character stripping to the new crate while keeping the `LoopSafeSummary`-specific 512-byte cap and byte truncation local. Existing memory_context behavior and tests are preserved. Wires the same envelope into `ironclaw_hooks`: * `HookPatch::add_enveloped_snippet` now takes a raw body and wraps it via `wrap_untrusted(EnvelopeSource::Hook, …)`. `Installed` hooks produce `Untrusted` envelopes; `Builtin`/`Trusted`/`SelfAuthored` produce `Trusted` envelopes so downstream readers can distinguish the two paths through a uniform marker. * `HookedLoopPromptPort::build_prompt_bundle` is no longer observe-only. After dispatching `before_prompt`, it envelope-wraps every snippet patch (passing `Enveloped` through, wrapping `Trusted` with the envelope helper), enforces the 4 KiB aggregate snippet byte budget across patches, and appends the wrapped snippets to the prompt bundle's `messages` as `system`-role `LoopModelMessage` entries carrying deterministic `msg:hook.<ordinal>.<hash>` content refs (mirroring the skill-snippet ref convention). The envelope crate is a leaf with no ironclaw dependencies, satisfying the boundary contract; the existing `ironclaw_hooks` boundary rule in `reborn_dependency_boundaries` continues to hold because `ironclaw_prompt_envelope` is not on its forbidden list. Test count delta: * `ironclaw_prompt_envelope`: +13 new tests (crate did not exist). * `ironclaw_hooks`: 84 → 88 tests (+4 prompt-port behavior tests: `hook_patch_appended_as_envelope_wrapped_message`, `total_byte_budget_enforced_across_patches`, `instruction_hijack_in_patch_rejected`, `trusted_hook_patch_wrapped_with_trust_marker`). * `ironclaw_host_runtime` memory_context: unchanged (8 tests still pass). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: align tenant-counter test with SanitizedArguments-extended context ctor * docs(reborn): document loader contract; pin HookId hex format Add a "Loader responsibility" section to ironclaw_hooks/CLAUDE.md explaining that tier-specific installers prevent minting wrong-tier impls but cannot enforce origin — that's the loader's job — and recommending registry loaders type-tag extension hooks as LoadedHook::Installed at the loader seam. Add tier_specific_installers_are_documented_as_loader_contract as a regression guard that touches every public install_*_before_capability and install_*_before_prompt method so any signature change forces the loader contract to be re-evaluated. Document HookId::to_hex's 64-char lowercase hex output as part of the cross-crate contract consumed by LoopHostMilestoneKind::Hook* in ironclaw_turns; add hook_id_hex_format_is_stable_64_lowercase_chars in identity::tests and hook_id_string_serialization_matches_to_hex in telemetry::tests to pin the format and the seam conversion path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(reborn): pin hook milestone JSON schema + assert pairing invariants Add L3 schema-snapshot tests for every hook-related LoopHostMilestoneKind variant (HookDispatched, HookDecisionEmitted per HookDecisionSummary, HookFailed per FailureCategory) so downstream consumers can rely on the JSON wire shape and any accidental field rename, enum-tag rename, or type change fails loudly. Add L4 pairing-invariant matrix test in the hook dispatcher that drives every observable outcome (Allow, Deny, PauseApproval, PauseAuth, Pass, Panic, Timeout, Malformed, MissingImpl) through a recording milestone sink and asserts the dispatched-then-terminator pairing shape. Document the MissingImpl path as the one case that emits a sole HookFailed with no preceding HookDispatched (the dispatcher discovers the protocol violation before the hook is actually dispatched). Add a multi-hook dispatch test that installs three hooks with mixed outcomes (allow/deny/panic) at the same point and asserts each hook produces its own paired sequence in the deterministic (phase, priority, hook_id) order taken from the dispatcher's registry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(reborn): integration tests for observer middleware through RebornLoopDriverHostFactory Wire the HookedLoopModelPort / HookedLoopTranscriptPort / HookedLoopCheckpointPort observer wrappers into RebornLoopDriverHostFactory::build_text_only_host_with_capabilities, mirroring the existing HookedLoopCapabilityPort / HookedLoopPromptPort composition. The wrappers are applied only when a HookDispatcher is set on the factory, so the default factory shape is unchanged. Add four integration scenarios in crates/ironclaw_reborn/tests/hooks_integration.rs: - observer_hook_fires_after_model_through_factory - observer_hook_fires_after_capability_through_factory - observer_hook_fires_after_checkpoint_through_factory - observer_panic_does_not_fail_model_call (panic-isolation regression) Relax the test-fixture model gateway from "panic if invoked" to returning a stub assistant reply so the AfterModel / panic-isolation tests can drive stream_model through the wrapped port. The existing capability-port tests never touch the gateway, so their behavior is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(reborn): introduce HookDispatcherBuilder for type-enforced sink wiring Adds a `HookDispatcherBuilder` in `ironclaw_hooks::dispatch` that owns the dispatcher construction lifecycle: registry -> optional timeout -> optional milestone sink -> installed hooks -> `.build_arc()`. The terminal `.build_arc()` wraps in `Arc` and yields an immutable handle. Tightens the public surface on `HookDispatcher`: `new`, `with_timeout`, `with_milestone_sink`, and every `install_*_*` method are now `pub(crate)`. Outside callers route exclusively through the builder, so "wire the milestone sink before Arc-wrapping" is a compile-time fact rather than a documentation convention. `HookRegistrar::install` now takes a `HookDispatcherBuilder` by value and returns `(HookDispatcherBuilder, Vec<HookId>)`, keeping the builder chainable through manifest installation. `RebornLoopDriverHostFactory` gains `with_hook_dispatcher_builder` to let callers defer `.build_arc()` to the factory — a step toward the FU8 per-build dispatcher pattern. Migrates `foundation_pipeline.rs` and `hooks_integration.rs` to the builder. Internal middleware and dispatch tests continue to use the crate-private `HookDispatcher::new` directly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): production CapabilityInputResolver for NumericSum predicates Adds HookCapabilityInputResolverAdapter in ironclaw_reborn that bridges the existing LoopCapabilityInputResolver (already used by HostRuntimeLoopCapabilityPort for dispatch input resolution) to the hooks crate's CapabilityInputResolver trait. RebornLoopDriverHostFactory gains with_capability_input_resolver(...), and when both a hook dispatcher and resolver are configured the factory threads the adapter into HookedLoopCapabilityPort::with_resolver — so NumericSum and other argument-dependent predicates evaluate against real, sanitized inputs instead of failing closed against the framework's null default. The adapter also enforces a configurable serialized-byte budget (default 64 KiB) as defense in depth ahead of the hooks crate's per-string and depth caps in SanitizedArguments. Unit tests cover the four adapter branches (resolved JSON, inner-error → None, non-object pass-through, oversized → None) and a new end-to-end integration test (numeric_sum_predicate_caps_total_value_against_real_inputs) drives the full factory wiring: with a NumericSum cap of 99 over an "amount" field, two invocations carrying {"amount":"50"} let the first pass through and deny the second at the hook seam, with the inner port reached exactly once. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): per-build HookDispatcher for full per-run isolation (C2) Introduce `with_hook_dispatcher_factory(F)` on `RebornLoopDriverHostFactory`. The closure is invoked once per `build_text_only_host*` call, so dispatcher-owned mutable state — slot poisoning, registry mutations, predicate-counter siblings — is scoped to a single host build instead of shared across every host the factory produces. The legacy `with_hook_dispatcher(Arc<HookDispatcher>)` adapter is kept as a thin wrapper that returns clones of the same `Arc` on every build. Its shared-state behavior is now documented as an explicit opt-in for backward compat; new wiring should prefer the factory closure. Adds two regression tests: - `per_build_dispatcher_state_does_not_leak_across_runs` — installs a panicking hook, builds two hosts back-to-back, and proves the inner port is never reached on build 2 (fresh slot still applies the fail-closed deny). Pins per-run isolation. - `legacy_with_hook_dispatcher_shares_state_across_builds` — pins the shared-state semantic of the legacy adapter as the explicit baseline. Migrates `predicate_deny_hook_short_circuits_inner_port` to the new factory-closure path so the new wiring is exercised by the existing suite. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): project hook telemetry milestones into RuntimeEvent for durable audit Extend the runtime event substrate with `HookDispatched`, `HookDecisionEmitted`, and `HookFailed` kinds carrying closed-vocabulary labels and the blake3-hex hook identity. Project the matching `LoopHostMilestoneKind::Hook*` variants in `DurableLoopHostMilestoneSink` so hook telemetry now lands in the same durable event log as model/reply/loop milestones — SSE observers still see live hook events, and audit replay can reconstruct the full hook trail. - `ironclaw_events`: add hook variants to `RuntimeEventKind`, optional hook fields on `RuntimeEvent` (`hook_id`, `hook_point`, `hook_trust_class`, `hook_decision`, `hook_failure_category`, `hook_failure_disposition`), typed constructors (`hook_dispatched`, `hook_decision_emitted`, `hook_failed`), and dedicated sanitizers (`sanitize_hook_label`, `sanitize_hook_id`) re-run on every wire crossing. No new crate dependency edges; hook strings cross the boundary opaque. - `ironclaw_reborn::milestone_events`: project the three hook milestone kinds via a new `loop.hook` capability id. `HookDecisionSummary` is collapsed to its closed-vocabulary `kind_name()` so sanitized reasons never enter the durable substrate. - `ironclaw_event_projections`: extend `TimelineEntryKind` and the `RuntimeEventKind -> RunProjectionStatus` mapping so hook events are pure telemetry — they preserve the current run status rather than changing it. - Tests: 4 unit tests in `ironclaw_events::runtime_event::tests` (serde round-trip per variant + unsafe-label collapse), 3 in `ironclaw_reborn::milestone_events::tests` (projection per variant, including the assertion that raw `Deny { reason }` text does not reach the durable wire payload). Existing replay-projection direct-construction tests updated for the new RuntimeEvent fields. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(reborn): enforce manifest-declared hook scope at dispatch time (C3) Audit finding C3: extensions could declare `[[hooks]]` with `scope = "own_capabilities"` in their manifest, but the dispatcher never enforced it — an Installed hook from ext-A could fire against capabilities provided by ext-B. Scope was parsed but not load-bearing. This change makes scope load-bearing end-to-end: - `BeforeCapabilityHookContext` carries an optional `provider: ironclaw_host_api::ExtensionId` populated by the middleware. The hook context is `#[non_exhaustive]` already so this is non-breaking. - `HookBinding` gains `owning_extension: Option<ExtensionId>` and `scope: HookBindingScope`. `HookBindingScope` is `Global` / `OwnCapabilities` / `SameTenant`. Builtin and Trusted bindings default to `Global` and carry no `owning_extension`; Installed bindings carry both, sourced from the manifest. - `HookDispatcher::install_installed_*` installers now require the caller to pass `(owning_extension, scope)`. The registrar derives both from the manifest entry, so manifest authorship is the single source of truth. - A new `CapabilityProviderResolver` trait + bundled `NullCapabilityProviderResolver` lets the middleware lift the capability id to its provider at invocation time. The middleware wires the resolved provider into the hook context. - `dispatch_before_capability` consults `binding.scope.permits(...)` before invoking each hook. Bindings that don't permit the current invocation are inert — no sink call, no failure record, no poisoning. Conservative defaults: - When the provider resolver returns `None` (no resolver wired, or the capability has no known provider), `OwnCapabilities`-scoped hooks do NOT fire. An attacker cannot bypass scope filtering by stripping provider info from the descriptor. Tests: - 5 new dispatcher tests cover OwnCapabilities matching, foreign provider, unresolved provider, SameTenant, and Builtin Global. - 1 new registrar test asserts manifest scope and extension propagate into `HookBinding`. - 1 new middleware test asserts the provider resolver populates the hook context. - 1 new integration test in `ironclaw_reborn` proves an ext-A hook scoped to `OwnCapabilities` does not intercept invocations that have no resolved provider (the production composition default). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: rustfmt dispatch.rs after FU1 merge * docs(hooks): prior-art comparison against LSM/eBPF/Envoy/K8s/OPA/CRX/VSC/Tauri Validates the IronClaw hooks design against 8 established hook/policy systems across 8 axes (dispatch, trust tiers, attenuation, decision vocabulary, failure semantics, isolation, manifest, audit). Surfaces: - 7 areas where ICLAW stands out vs prior art (type-level trust enforcement, dispatch-time scope, failure-kind matrix, pause-with- gate-ref, pairing-invariant audit matrix, tenant-keyed predicates, phase-ordered dispatch) - 4 conventional choices we should revisit (in-process Installed-WASM, sticky poison, no formal dispatch model, no installation rate-limit) - 3 divergences whose 'why' is weak and need design review * docs(hooks): STRIDE threat model for v1 framework Enumerates 7 adversary classes (A1-A7), 6 assets ranked by blast radius, and ~35 attack vectors across STRIDE categories with mitigations, existing tests, and residual risk. Surfaces 7 prioritized follow-ups: - High: per-extension hook-count cap (D3/D4) - High: gate-ref unguessability + one-shot test (S1) - Med: resolver field-level scope (I2) - Med: per-evaluator state ceiling (D5) - Med: poison-stickiness operator runbook - Low: timing side-channel residual acknowledgement (I4) - Low: instruction-marker denylist periodic review (I5) Confirms the load-bearing 'Installed cannot Allow' (E1) property holds via type-level seal + tier-specific installers, backed by compile_time_seal_test and installed_binding_cannot_be_paired_with_ privileged_impl tests. Explicit out-of-scope: extension install pipeline (#3492), WASM exec sandbox (needs separate threat model when it lands), approval gateway (#3564). * feat(hooks): close threat-model gaps S1 (gate-ref entropy) and D3/D4 (registration flood) S1 (gate-ref unguessability, factory side): - Three new tests on `UuidHookGateRefFactory`: - `gate_refs_are_v4_uuids` pins the v4 entropy source (122 random bits per ref per RFC 4122 §4.4); fails if a future change moves to a counter or weaker UUID version. - `gate_refs_have_no_collisions_across_many_calls` mints 20k refs across both namespaces, asserts zero collisions (statistical proxy for entropy quality). - `approval_and_auth_namespaces_do_not_overlap` confirms prefix routing separation. - Doc comment now documents the security property explicitly and delineates factory-side vs gateway-side responsibilities for the one-shot consumption property. D3/D4 (hook registration flood): - New `MAX_HOOKS_PER_EXTENSION = 32` and `MAX_HOOKS_PER_EXTENSION_PER_KIND = 8` consts in `registrar.rs`. - New `HookRegistrar::enforce_registration_caps` runs pre-flight at the top of `install()`, before any binding is inserted. Whole-batch rejection means a partially-installed batch cannot slip past. - Three regression tests: total-cap rejection, per-kind-cap rejection, at-cap acceptance. - Error messages cite the threat-model finding so operators can map rejection back to the design rationale. Threat model updated: S1, D3, D4 marked closed in the cross-cutting properties matrix and the open-follow-ups list. * test(hooks): three real hooks built against the public API + ergonomics findings Builds three representative hooks from outside the crate, mimicking what an extension or system author would actually write: 1. polymarket-daily-cap — Installed predicate hook, InvocationCount rate-cap with Deny on excess. Canonical 'rate-limit a capability' use case for the predicate language. 2. large-stake-approval-gate — Installed predicate hook, NumericSum over amount_usd field, PauseApproval at $1000/24h. Manifest-shape + registrar-install coverage from outside Reborn; end-to-end dispatch lives in ironclaw_reborn integration tests because NumericSum needs resolved args (a friction finding documented in the companion doc). 3. pii-redaction-warning — Trusted Rust hook implementing PrivilegedBeforePromptHook, injects a trusted instruction snippet reminding the model to redact PII. Demonstrates the path a system author takes when the predicate language isn't expressive enough. API change (F1 fix): SanitizedArguments::unresolved() promoted from pub(crate) to pub. This is the documented safe default — predicates that need args must fail closed against it — so exposing the constructor cannot weaken any trust property. The sanitizing from_json constructor stays sealed; that's the trust boundary. Without this fix, external hook authors could not construct a BeforeCapabilityHookContext with both a known provider AND unresolved args, which made TDD of their own predicate impossible. Findings documented in docs/real-hooks-findings.md, ranked by severity. Big-picture observation: writing the Trusted Rust hook (F4) was easier than writing the declarative predicate hook (F1 + F2 + F3) — three of seven findings target predicate-authoring ergonomics. The declarative path needs the most polish before third-party extension authors will trust it for non-trivial policy. Tests: 6 new in real_hooks.rs, all pass. * feat(hooks): close all remaining threat-model and ergonomics gaps Closes the Med-priority threat-model gaps (I2, D5, poison runbook) and all real-hooks ergonomics findings (F2, F3, F5, F6, F7) in a single pass. Threat model: - I2 (resolver field-scope): documented in SanitizedArguments rustdoc. The narrow public surface (only is_resolved + extract_numeric) enforces field-scope by construction for the current predicate path. Reassess when Installed-WASM lands. - D5 (evaluator state ceiling): MAX_HISTORY_KEYS = 8192 per map, LRU eviction with evictions_observed() metric for operator monitoring. New regression test lru_eviction_increments_counter_and_drops_oldest_key. - Poison-stickiness runbook: new docs/operator-runbook.md with recovery options ranked by cost. Ergonomics findings: - F2 (closed-vocab deny reasons): rustdoc on OnExceededAction and GateDecisionView::Deny explaining the audit-vs-model split and why manifest reason text doesn't reach the model. - F3 (NumericSum can't be TDD'd outside Reborn): new test-support feature flag with SanitizedArguments::for_tests(value) that external hook authors can opt into via dev-dep. - F5 (two ExtensionId types): added From<&ironclaw_host_api::ExtensionId> impl for identity::ExtensionId, plus cross-link rustdoc. - F6 (HookManifestEntry struct-literal fragility): added #[non_exhaustive] + HookManifestEntry::new(id, kind, body) + with_scope/with_phase/with_priority/with_description/with_requires_grant builder methods. Migrated 3 external call sites in tests/. - F7 (priority guidance): rustdoc on HookPriority with when-to- deviate guidance, named FIRST/LAST constants documented for Builtin/Telemetry use cases. Tests: 151 unit + 1 + 6 integration in ironclaw_hooks all pass with --all-features. ironclaw_reborn (13 hooks_integration scenarios) unchanged. Threat model updated: I2 / D5 / poison runbook marked closed in both the per-vector table and the cross-cutting properties matrix. Open follow-ups now down to two Low items (I4 timing side-channel residual, I5 instruction-marker denylist refresh) plus the deferred DenyReasonCode enum from F2. * fix(ci): collapse nested match in hooks_integration test for clippy --all-features CI runs `cargo clippy --all --tests --examples --all-features -- -D warnings` which is stricter than the workspace clippy I ran locally and trips `clippy::collapsible_match` on the nested-if in HookDecisionEmitted matching. Collapse the inner `if decision.kind_name() == "deny"` into an arm guard. * feat(hooks): address henrypark133 review — Critical #1/#2/#3/#5, Concerning #5/#7 Address composition-seam bugs in the Reborn factory wiring + doc tidy. henrypark133 review findings addressed: Critical #1 — before_prompt hook messages not materialized. HookedLoopPromptPort now requires a HookPromptMaterializationSink and fails closed if patches are emitted without one. The reborn factory installs an InstructionStoreBackedHookSink adapter that delegates to the host's InstructionMaterializationStore, so synthetic msg:hook.* refs are resolvable by the downstream model resolver. New seam trait (HookPromptMaterializationSink) keeps ironclaw_hooks decoupled from LoopRunContext. Critical #2 — OwnCapabilities hooks were inert in production wiring. Factory now installs SurfaceBackedProviderResolver (consults the visible-capability surface for capability_id → provider). With this, ctx.provider is populated and OwnCapabilities-scoped Installed hooks actually fire against their own provider's capabilities. Critical #3 — gate refs were unresolvable. Middleware default switched from UuidHookGateRefFactory to FailClosedHookGateRefFactory. Tests must explicitly opt into UUID (via with_gate_ref_factory) to exercise the affirmative ApprovalRequired path; production deployments must install a router-backed factory. New factory method RebornLoopDriverHostFactory::with_hook_gate_ref_factory. Concerning #5 — AfterModel fired twice + before durable finalization. Removed AfterModel dispatch from HookedLoopModelPort; the transcript port's finalize_assistant_message is now the sole AfterModel boundary (the durable one). Model port wrapper is preserved as a no-op shim for symmetry + future model-response-observed point. Concerning #7 — doc tidy: - CLAUDE.md: 3 trust classes → 4 (Builtin/Trusted/Installed/SelfAuthored with explicit note that SelfAuthored is run-scoped only and not loadable from an external source). - operator-runbook.md: "Audit log" → "durable runtime event stream" where the projection is actually the runtime-event stream, not formal AuditEnvelope records. - prior-art.md: poison-lifetime nuance — per-host-build with the factory pattern, process-lifetime only for the legacy adapter. - prior-art.md:80: trailing whitespace removed. Testing gaps from henrypark133 — caller-level tests through RebornLoopDriverHostFactory: #1 (before_prompt resolver path): before_prompt_hook_message_is_resolvable_via_factory_wiring #2 (OwnCapabilities positive/negative/unknown): own_capabilities_hook_fires_when_provider_matches own_capabilities_hook_does_not_fire_when_provider_differs own_capabilities_hook_does_not_fire_when_provider_unknown #3 (pause/auth gate lifecycle or fail-closed): pause_approval_with_default_factory_fails_closed_as_denied pause_approval_hook_surfaces_as_approval_required_with_real_gate_ref (updated to require explicit UuidHookGateRefFactory opt-in) #5 (AfterModel exactly-once at durable boundary): after_model_fires_exactly_once_at_durable_boundary Still TODO from review (separate commits): Critical #4 (telemetry context — two-run attribution) + gap #4 Concerning #6 (TimelineEntry hook metadata projection) + gap #6 Tests: 154 unit + 18 hooks_integration + all other reborn tests pass. Workspace clippy + fmt + no-panics clean. * feat(hooks): address remaining henrypark133 review — Critical #4, Concerning #6 Critical #4 — per-run hook telemetry attribution. New `HookDispatcherBuilderFactory` signature: factory returns a HookDispatcherBuilder, and `RebornLoopDriverHostFactory` attaches a `RunScopedHookMilestoneSink` keyed to the CURRENT run's LoopRunContext inside `build_text_only_host_with_capabilities`, before sealing the dispatcher. The previous zero-arg signature relied on the closure capturing run_context — silently misattributed across reuses; new public API `with_hook_dispatcher_builder_factory` removes that failure mode entirely. Legacy `with_hook_dispatcher_factory` retained for back-compat (its sink-wiring contract stays caller-side). Concerning #6 — TimelineEntry hook metadata. Added 6 optional fields to `TimelineEntry` (hook_id, hook_point, hook_trust_class, hook_decision, hook_failure_category, hook_failure_disposition) and projected them from `RuntimeEvent::Hook*`. Replay consumers now see which hook fired/failed, not just that some hook event happened. Each field is closed-vocabulary (no free-form reason text — that stays in the audit reason payload, not the product replay DTO). Testing gaps from henrypark133 — caller-level tests: #4 (two-run hook telemetry attribution): hook_telemetry_attribution_is_per_run_not_captured Builds two hosts from the SAME builder factory closure with two fresh LoopRunContexts. Asserts each run's hook milestones carry its OWN run_id (no stale captured one). #6 (replay projection contract for hook events): hook_runtime_events_project_with_sanitized_hook_metadata non_hook_runtime_events_project_with_no_hook_metadata Constructs RuntimeEvent::Hook{Dispatched,DecisionEmitted,Failed} and asserts the projection preserves the metadata fields. The negative test guards against cross-contamination on non-hook events. All henrypark133 review items now addressed: Critical: #1, #2, #3, #4 — done Concerning: #5, #6, #7 — done Testing gaps: #1-#6 — done Tests: 154 unit + 19 hooks_integration in ironclaw_reborn + 61 reborn unit + 38 + 2 new in ironclaw_event_projections + ... pass. Workspace clippy + fmt + no-panics clean. * docs(hooks): scope DenyReasonCode closed-vocabulary enum (successor #6) Successor PR from #3573 — real-hooks ergonomics finding F2 (deferred). Adds a curated vocabulary of model-visible denial reasons so hook authors can communicate why a deny happened without opening a free-form prompt-injection channel. * feat(hooks): DenyReasonCode + PauseReasonCode closed-vocabulary enums Address real-hooks ergonomics finding F2 (deferred from PR #3573). The prior dispatcher collapsed every Installed-tier deny to the static label 'hook_predicate_denied', because manifest reason strings are author-controlled and surfacing them to the model would open a prompt-injection channel. The cost: the agent couldn't tell *why* a hook denied. This PR introduces two closed-vocabulary enums: - DenyReasonCode: Generic / RateLimit / ValueCap / Blocklist / RequiresApproval / OutOfPolicy - PauseReasonCode: Generic / RequiresApproval / OverThreshold / SensitiveAction Each variant has an as_label() returning &'static str (so the sink's &'static str contract is preserved). New OnExceededAction variants 'DenyWithCode { code, reason }' and 'PauseApprovalWithCode { code, reason }' let manifest authors opt into the richer labels while keeping reason audit-only. The legacy Deny { reason } / PauseApproval { reason } variants are retained for back-compat and map to DenyReasonCode::Generic / PauseReasonCode::Generic — existing manifests continue to produce hook_predicate_denied / hook_predicate_pause_requested. Threat-model regression: a hook author cannot smuggle text into the model-visible label because the 'code' field is typed as the enum; there's no String slot exposed model-side. A test (deny_with_code_only_exposes_enum_variants_to_model) documents this as a compile-time property. Tests (+7 new = 161 total): - deny_reason_code_labels_are_stable: pins the label vocabulary so rename/relabel is loud. - pause_reason_code_labels_are_stable: same for PauseReasonCode. - deny_with_code_round_trips_through_json + pause variant: wire round-trip + snake_case tag assertion. - deny_with_code_only_exposes_enum_variants_to_model: compile-time property check. - rate_or_value_cap_with_deny_code_routes_to_code_label: end-to-end affirmative test that the dispatcher emits the code's label. - rate_or_value_cap_with_pause_code_routes_to_code_label: same for pause. Scope doc: crates/ironclaw_hooks/docs/successors/06-deny-reason-code.md * test(hooks): address codex review on #3636 - Update stale real-hooks-findings.md F2 row to cite this PR's enum follow-on (was 'deferred'). - Add install_deny_with_code_manifest_surfaces_code_label_on_dispatch: end-to-end test driving the registrar->dispatcher path for the new DenyWithCode variant (prior tests covered serde + direct hook evaluation, but not the manifest install path that downstream authors actually use). Codex review on PR #3636: APPROVE with two recommendations; both addressed. Tests: 162 unit (+1 new). Clippy/fmt clean. * fix(hooks): attenuate Installed-tier prompt patches to user role Installed-tier `before_prompt` patches were injected as role:"system" messages. Envelope text labels ("[ext-foo says]: ...") do not strip system-role authority from the model's perspective, so a third-party extension could inject system-tier instructions through a snippet patch. This is a prompt-authority escalation against the trust hierarchy the framework otherwise enforces. Add `role_for_trust_class()` mapping Installed -> "user" and Builtin/Trusted/SelfAuthored -> "system". Thread per-patch trust_class through `wrap_patches_to_messages` and use it for the emitted `LoopModelMessage.role`. Tests: - installed_hook_patch_drops_to_user_role: asserts the role for an Installed-tier patch is "user" - trusted_tier_hook_patch_keeps_system_role: regression that Trusted tier still produces system-role content * fix(hooks): enforce scope filter on observer dispatch + reject incompatible points Two related defense-in-depth fixes against silent scope-filter failure: 1. The registry silently accepted Installed bindings with `HookBindingScope::OwnCapabilities` at points (BeforePrompt, AfterModel, AfterCheckpoint) whose dispatch context carries no per-capability provider. The manifest's declared scope had no effect at all — the hook fired against every dispatch. Reject the binding at install time so the operator sees the misconfiguration. 2. `dispatch_observer_at` for `AfterCapability` did not consult the binding's scope, so an Installed observer registered with `OwnCapabilities` fired against every invocation regardless of provider. Add `dispatch_observer_at_with_provider` carrying the resolved capability provider; the capability-port middleware resolves the provider once per invocation and threads it through both the BeforeCapability hook context and the AfterCapability observer dispatch. The dispatcher then enforces `HookBindingScope::permits` on each observer binding. `ObserverHookContext` gains a `provider: Option<ExtensionId>` field; `#[non_exhaustive]` keeps existing authors compiling. Tests: - rejects_own_capabilities_at_before_prompt - rejects_own_capabilities_at_after_model - accepts_own_capabilities_at_before_capability - own_capabilities_observer_filters_foreign_providers (covers foreign / matching / unresolved provider) * fix(hooks): preserve free-form audit reason alongside closed-vocab model label (serrrfirat #3636) `PredicateBackedBeforeCapabilityHook::evaluate()` was discarding the free-form `reason` from `EvaluatorDecision::{Deny, PauseApproval}` with `..` and only sending `code.as_label()` into the sink. The `HookDecisionEmitted` milestone therefore carried only the closed- vocab label, and operator-visible audit/SSE context was silently lost end-to-end. The fix splits the channels: - Model sees the closed-vocab label (`hook_rate_limit`, `hook_pause_over_threshold`, ...) via `sink.deny(label)`. This channel is unchanged. - Audit/SSE sees the manifest's free-form `reason` via a new audit-only sink method `record_audit_reason(reason: String)`. The recording sink captures it; the dispatcher reads it after the hook returns and threads it into `LoopHostMilestoneKind::HookDecisionEmitted`. Surface changes: - `PrivilegedGateSink` / `RestrictedGateSink` gain `record_audit_reason(String)` — accepts dynamic `String` (audit-only, no model-facing seam) unlike the `&'static str` decision reasons. - `RecordingGateSink` gains an `audit_reason: Option<String>` field. - `GateHookOutcome::Decision` is now `Decision { decision, audit_reason }`. - `HookDispatcher::emit_decision_with_audit` threads the audit reason into the milestone. - `LoopHostMilestoneKind::HookDecisionEmitted` gains a `#[serde(default, skip_serializing_if = "Option::is_none")]` `audit_reason: Option<String>`. The durable RuntimeEvent projection intentionally drops this field — audit reasons are operator-facing in-memory SSE content, never durable cross-process surface. Tests: - `deny_with_code_records_audit_reason_separately_from_model_label`: asserts the recording sink ends with `Deny { reason: "hook_rate_limit" }` in `state` AND `audit_reason == Some("daily cap of $1000 ...")`. * fix(hooks): remove unused model_request helper (CI clippy fix) * fix(hooks): address serrrfirat P1/P2 findings on PR #3573 Three issues from the 5-15 review: **P1 #1 registrar.rs:70 — `same_tenant` grants not enforced** `HookManifestEntry::validate` only confirmed `requires_grant` was present; the registrar then immediately installed the binding with no host-verified grant context. A manifest could declare `requires_grant = "anything"` and get a cross-extension binding for free. Fix: `HookRegistrar` now carries a `verified_grants: HashSet<String>` (empty by default — default-deny). Add the host-facing setter `with_verified_grants(...)`. At `install_one`, if `entry.requires_grant` is `Some(g)`, require `g ∈ verified_grants` or reject with a clear error. Tests: - `install_rejects_same_tenant_without_verified_grant` - `install_rejects_same_tenant_when_verified_grants_mismatch` - The existing positive test `installer_propagates_owning_extension_and_scope_from_manifest` now wires the verified grant explicitly (proves the API contract). **P1 #2 prompt_port.rs:150 — zip misalignment** The materialization loop zipped surviving messages against the ORIGINAL unfiltered patch list. `wrap_patches_to_messages` skips metadata patches and over-budget snippets, so the zip silently paired message[0] with patch[0] even when patch[0] was the skipped metadata — materializing the wrong content (or none) under the snippet's synthetic ref. Fix: `wrap_patches_to_messages` now returns `Vec<WrappedHookMessage { message, safe_content }>` — surviving messages paired with their content by construction. The caller materializes `entry.safe_content` under `entry.message.content_ref` directly; no zip against unfiltered input. Removed the now-unused `safe_content_for_patch` helper. Test: - `materialization_stays_aligned_when_metadata_patches_are_filtered`: a hook emits `[metadata, snippet]`; asserts only one model message, and the materialized content under its ref contains the snippet's body — proves filtering can no longer desync from materialization. **P2 #3 loop_driver_host.rs:1343 — `with_hook_dispatcher_builder`** Docs said it deferred `build_arc()` to let the host factory finalize wiring; the implementation called `build_arc()` eagerly and routed through the legacy shared-dispatcher adapter, losing per-run dispatcher isolation and the run-scoped milestone sink. Fix: marked `#[deprecated]` with a note pointing callers to `with_hook_dispatcher_builder_factory(|| ...)` for per-build isolation, or `with_hook_dispatcher(...)` if they actually meant the shared adapter. The method body is unchanged so no callers break; they'll see the deprecation warning. No internal callers exist, so the deprecation doesn't trip `-D warnings`. All 162 hooks lib + 19 reborn integration tests pass; clippy clean. * fix(hooks): address serrrfirat 3573-2026-05-15 review findings P1 — prompt bundle authority mismatch (prompt_port.rs): `HookedLoopPromptPort::build_prompt_bundle` called the inner port first, which caused `HostManagedLoopPromptPort` to issue the prompt-bundle authority grant against the pre-hook message list. The wrapper then appended `msg:hook.*` messages to `bundle.messages`, so the downstream model request hit `grant.messages != messages` and failed closed with "model request messages do not match the host-built prompt bundle". Add `with_bundle_authority(authority, run_context)` and re-issue the grant after appending hook messages so it covers the post-hook bundle. Reborn wires `prompt_authority.clone()` + `run_context.clone()` into the wrapper at construction time. P2 — observer installer accepts non-observer points (dispatch.rs): `install_observer` accepted any `HookPointSpec` (including `BeforeCapability` / `BeforePrompt`) and only populated the observer map. Dispatch later found a binding without a gate/mutator impl and fail-closed the capability with "binding present without installed implementation". Reject non-observer points at install time so misuse fails loudly rather than poisoning bindings at dispatch. P2 — batch path skipped AfterCapability observers on inner error (capability_port.rs): The batch loop used `?` directly on `self.inner.invoke_capability(...)`, which propagated the error before dispatching `AfterCapability` observers. Failed batch entries disappeared from telemetry / audit, while the single-invocation path dispatches observers on error. Capture the inner result, dispatch observers, then propagate the error. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(hooks): address PR #3573 review feedback round 3 Addresses serrrfirat's CHANGES_REQUESTED review (2026-05-20) by tightening several install-time / dispatch-time bounds and gating production seams: - Bound free-form audit reasons crossing telemetry. New `telemetry::sanitize_audit_reason` strips control characters and caps length at 512 bytes; `emit_decision_with_audit` routes the manifest- supplied reason through it before publishing milestones. Manifest validation also rejects reasons over the same byte limit at install time so the wire-side cap is a defense-in-depth layer, not the only line. - Make hot dispatch O(H) instead of O(H^2). The per-binding poison recheck used to acquire the registry mutex and walk every binding; `ordered_bindings_with_poison_snapshot` now takes the active bindings and the poisoned hook-id set under a single lock, and each loop threads a local `HashSet<HookId>` that absorbs mid-dispatch poisoning. Removed the redundant `is_poisoned` helper. - Gate `HookDispatcher::registry_for_test` behind `cfg(any(test, feature = "test-support"))`. The accessor previously exposed `&Mutex<HookRegistry>` in production, letting any `Arc<HookDispatcher>` holder lock and call `HookRegistry::poison` to disable installed hooks. Added `active_bindings_snapshot(point)` as the read-only production-safe replacement. - `#[serde(deny_unknown_fields)]` on every hook-manifest and predicate DTO (`HookManifestEntry`, `HookManifestBody`, `WasmBudget`, `HookPredicateSpec`, `CapabilityPredicate`, `ValueOrRateBound`, `OnExceededAction`). Typoed or unsupported fields (e.g. a manifest-supplied `trust_class`) now fail loud at install time instead of being silently dropped. - Bound predicate trees at install. New `validate_predicate_tree` enforces `MAX_PREDICATE_DEPTH = 8`, `MAX_PREDICATE_NODES = 64`, `MAX_PREDICATE_STRING_BYTES = 256`, and `MAX_MANIFEST_REASON_BYTES = 512`. A hostile registry manifest can no longer install a deep or huge `All`/`Any` tree that the evaluator would recursively walk on every match. - Cap sliding-window samples per key. `MAX_SAMPLES_PER_KEY = 4_096` in the predicate evaluator. Both the invocation-count and numeric-sum histories drop the oldest sample once the cap is reached, bounding memory under attacker-triggered hot capabilities while preserving rate/value-cap semantics over the most recent window. - `split_indexer` / `resolve_path` now fail closed on malformed bracket syntax (`amount[foo]`, `amount[`, trailing garbage). Previously they silently fell back to the parent field, which could let a typoed `NumericSum` predicate evaluate against the wrong value and allow calls the predicate would otherwise have denied. - Honor `PatchOrdinalHint`. `WrappedHookMessage` carries the source patch's `ordinal_hint`; `HookedLoopPromptPort` inserts `NearTop` messages after the bundle's `identity_message_count` and appends `Last` messages at the end. Safety/policy snippets that need early placement now get it. - Update `ironclaw_hooks` top-level docs to reflect the four trust classes (`Builtin`/`Trusted`/`Installed`/`SelfAuthored`) and the now-wired Reborn middleware composition. Tests added: - `manifest::rejects_unknown_top_level_field` - `manifest::rejects_unknown_wasm_budget_field` - `manifest::rejects_predicate_tree_exceeding_max_depth` - `manifest::rejects_predicate_tree_exceeding_max_nodes` - `manifest::rejects_predicate_string_exceeding_max_bytes` - `manifest::rejects_manifest_reason_exceeding_max_bytes` - `points::capability::malformed_indexer_returns_none_not_parent_value` - `telemetry::sanitize_audit_reason_*` (truncate / strip control / preserve / empty) `cargo fmt`, `cargo clippy --all --benches --tests --examples --all-features`, and `cargo test -p ironclaw_hooks` all pass clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(hooks): batch deferred test coverage from #3573 review (#3914) * perf(hooks): defer capability input resolution until a predicate needs it (#3913) * fix(rebase): adapt hooks tests + middleware to upstream API additions - CapabilityDescriptorView: add parameters_schema field - LoopModelRequest / LoopPromptBundleRequest: add capability_view field - TimelineEntry test builder: add hook_id / hook_point / hook_trust_class / hook_decision / hook_failure_category / hook_failure_disposition fields - ironclaw_reborn::tests::hooks_integration: switch from InMemoryLoopCheckpointStore to InMemoryTurnStateStore (which now impls both LoopCheckpointStore and TurnStateStore), pass TurnActor in TurnRunState, supply the new turn_state_store factory arg - ironclaw_reborn lib.rs: drop the pub-use re-exports that upstream intentionally removed (per the module-directory rationale in the current ironclaw_reborn lib.rs doc comment); update the hooks_integration test imports to use module paths - Cargo.toml: union the hooks-foundation member list with upstream's new crates (event_streams, auth, first_party_extensions, reborn_webui_ingress, product_workflow_storage, webui_v2); drop ironclaw_storage which no longer exists upstream - crates/ironclaw_architecture/tests/reborn_dependency_boundaries: keep upstream's removal of ironclaw_filesystem from the ironclaw_turns forbidden list AND add ironclaw_hooks to that list - crates/ironclaw_reborn/src/milestone_events.rs: drop dead loop_failure_kind helper (replaced upstream by loop_failure_kind_name in text_loop_driver.rs); keep hook_decision_label which is still used Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(hooks): restore batched capability dispatch when hooks acti…
JZKK720
pushed a commit
that referenced
this pull request
Jun 13, 2026
…nearai#3640) * docs(hooks): scope event-triggered hooks (Phase 5, successor #4) Successor PR from nearai#3573. Adds a new EventTriggered hook point that subscribes to RuntimeEvents asynchronously, outside the loop's inline tick. Observer-only by construction (no Allow/Deny/Patch); typed against a narrowed HookObservableEvent projection to keep the cross-crate boundary clean. Scope doc only; design questions about cursor/replay semantics and per-extension event-rate caps need design review before implementation. * Implement Phase 5 event-triggered hooks Cites crates/ironclaw_hooks/docs/successors/04-event-triggered-hooks.md as the scope contract. Adds the EventTriggered observer hook point, durable RuntimeEvent dispatch path, and Reborn pull-driven subscription wiring with caller-level coverage for matching, replay, scope filtering, observer-only authority, and backpressure. * Fix hook event OwnCapabilities owner lookup * fix(hooks): carry owning extension into hook milestone runtime events henrypark133 HIGH + codex P1 on PR nearai#3640: `OwnCapabilities`-scoped event-triggered subscriptions silently never fired for `HookFailed`/`HookDecisionEmitted`/`HookDispatched` events because those `RuntimeEvent` constructors hardcoded `provider: None`. Since Installed hooks default to `OwnCapabilities`, the very events that Phase 5 was designed to observe (hook-failure / decision alerting) never reached their default-configured subscriber. A prior fix added a hook_id-based fallback in `scope_provider_for_runtime_event` that resolves the owning extension through the registry's hex index when `event.provider` is `None`. That covers the case where the failing hook is still registered at replay time, but the durable fix is to stamp the originating provider into the event at emit time so the primary `event.provider` path resolves without any fallback. Plumbed `owning_extension: Option<ExtensionId>` end-to-end: - `LoopHostMilestoneKind::{HookDispatched, HookDecisionEmitted, HookFailed}` gain the field (with `#[serde(default, skip_serializing_if = "Option::is_none")]` so pre-existing checkpoint payloads and the L3 schema-snapshot tests round-trip unchanged when no owner is set). - `RuntimeEvent::hook_{dispatched, decision_emitted, failed}` constructors accept the owner and stamp it into `provider`. - `milestone_events.rs` threads the field through the projection. - `HookDispatcher::emit_dispatched/emit_decision` pass `binding.owning_extension.clone()` directly. - `HookDispatcher::emit_failure` (no binding handy on the failure path) looks the owner up via the registry's existing `owning_extension_for_hook_hex` index. Tests: - `event_triggered_own_capabilities_matches_hook_failed_with_carried_provider`: primary-path regression — two `HookFailed` events with `provider: Some(ext_a|ext_b)` against an `OwnCapabilities` subscription scoped to ext_a; only the own-provider event fires and `event.provider == Some(ext_a)`. - Existing `event_triggered_own_capabilities_scope_resolves_hook_failed_owner_from_hook_id` remains green: passes `None` for the new arg so the fallback path is still exercised for legacy payloads. All other call sites updated to pass `None` (no owner available) or the resolved owner where applicable. * fix(hooks): validate event subscription scope against run scope (serrrfirat HIGH #1 on PR nearai#3640) `EventTriggeredHookSubscription` accepted a caller-supplied `EventStreamKey` + `ReadScope` and used `run_context.scope.tenant_id` as the hook context's tenant — with no validation that the two agreed. A caller wiring tenant A's host with tenant B's stream would cause hooks to observe B's events while the hook context claimed tenant A. Cross-tenant trust-boundary break. Add `EventTriggeredHookSubscription::validate_against_run_scope` and call it from `build_text_only_host_with_capabilities` before spawning. Validation: - Stream `(tenant_id, user_id, agent_id)` must equal `(run_context.scope.tenant_id, thread_scope.owner_user_id, run_context.scope.agent_id)`. - Thread without `owner_user_id` cannot bind any subscription — the user dimension is required to verify stream identity. - Every `Some(want)` in `ReadScope` must equal the corresponding run/thread scope value (project/mission/thread). `None` is permissive (run scope owns the dimension authoritatively). Failures surface as `RebornLoopDriverHostError::ScopeMismatch` with a specific reason naming the offending dimension. Tests: - `event_triggered_subscription_with_foreign_tenant_stream_fails_host_build` - `event_triggered_subscription_with_foreign_user_stream_fails_host_build` The integration fixture's `ThreadScope` now sets `owner_user_id: Some(...)` so it passes validation; previously it was `None`, which the new check (correctly) refuses. Existing tests continue to pass. * fix(hooks): surface event subscription replay-gap as a milestone (serrrfirat MED on PR nearai#3640) When the durable event log returned `EventError::ReplayGap`, the event-triggered subscription's background task previously logged a `tracing::warn!` and broke out of the poll loop — silently killing all future hook event delivery for the run with no operator-visible signal. A scoped audit hook that mattered to compliance would just stop, and nobody downstream would know. Surface the termination through the host's milestone sink: - New `LoopDriverNoteKind::EventSubscriptionTerminated` variant. - The subscription's `spawn`/`run` now takes the host's `Arc<dyn LoopHostMilestoneSink>` and the active `LoopRunContext`. On `ReplayGap`, it constructs a `DriverNote` milestone with that kind plus a `LoopSafeSummary` describing the gap, publishes it through the same sink that carries every other host milestone, and *then* breaks (fail-closed: the at-most-once contract is already broken; resuming from `earliest` would silently lose the gap). - Log level bumped from `warn` to `error` to match the severity. - A best-effort send: failures to publish the milestone are logged but do not stall the subscription teardown. Tests: - `event_triggered_replay_gap_emits_subscription_terminated_milestone`: appends 3 events, `truncate_before_or_at` to cursor 2 to force a replay gap, starts the subscription from cursor origin (now stale), and asserts a `DriverNote { kind: EventSubscriptionTerminated, .. }` shows up on the host's milestone sink within a 2s deadline. Self-emit reentrancy (serrrfirat MED #3 on the same PR) is intentionally not addressed here — that fix needs a design call (task-local re-entry flag vs. removing RuntimeEvent emit capability from event-hook execution contexts) and is a follow-up. * fix(hooks): suppress event-triggered self-observation (serrrfirat MED #3 on PR nearai#3640) A hook that subscribes to one of the hook-lifecycle event kinds (`HookDispatched`/`HookDecisionEmitted`/`HookFailed`) with a scope that matches its own provider would otherwise be dispatched for events describing its OWN executions. The dispatcher emits those events itself when running the hook, so a hook subscribing to `HookFailed` with `OwnCapabilities` against its own extension would fail → emit HookFailed → re-dispatch → fail → emit → … storm. `dispatch_event_triggered_at` now skips events whose `event.hook_id` equals the binding's own hook id when the event kind is a hook- lifecycle kind (`is_hook_lifecycle_kind`). The check is intentionally narrow: - It only fires for hook-lifecycle events. Subscriptions to other event kinds are unaffected. - It only suppresses literal self-observation; events about other hooks (even hooks from the same extension) still dispatch. This does NOT cover the broader case of a hook that captures an `Arc<DurableEventLog>` and mints arbitrary `RuntimeEvent`s from inside its `observe()`. That requires architectural restriction on what hook impls can capture — tracked separately as a follow-up. Tests: - `event_triggered_self_lifecycle_event_does_not_redispatch`: appends two `HookFailed` events with the same provider — one targeting the subscriber's own hook id, one targeting a different hook. Asserts only the OTHER hook's failure fires (proves the filter is narrow, not blanket). * fix(hooks): address henrypark133 should-fix #1, #2, #3, #6 on PR nearai#3640 Four items from the 5-15 review (#4 DoS budget and #5 narrowed projection deferred — see below): **#1 (should-fix) Invariant: EventTriggered ↔ event_kind_filter** `HookRegistry::insert` now enforces the biconditional at install time: an `EventTriggered` binding must declare an `event_kind_filter` (otherwise the dispatcher's kind match would silently never fire — a no-op binding), and conversely only `EventTriggered` bindings may declare a filter (other points are kind-agnostic and would ignore the field). Misconfigured bindings fail loud at install. **#2 (should-fix) Remove `Clone` derive on EventTriggeredHookSubscription** `Clone` on a spawn-semantics type was a footgun: external callers cloning + spawning twice would create two consumers reading from the same `start_cursor`, each dispatching every hook. Replace with an explicit `clone_for_independent_spawn(&self)` method named verbosely so the property is visible at the seam. Internal use updated in the factory's host-build path; external callers can no longer accidentally construct a dual-consumer pattern. **#3 (should-fix) catch_unwind around the background `run()` task** The subscription's tokio task body now runs inside `AssertUnwindSafe(...).catch_unwind()`; a panic in `run()` emits the same `EventSubscriptionTerminated` `DriverNote` milestone the `ReplayGap` path already emits, instead of silently terminating with no operator-visible signal. **#6 (should-fix) Replay semantics in rustdoc on public API** Added a "Replay semantics" section to `EventTriggeredHookSubscription` rustdoc: at-least-once, caller-owned cursor persistence, the restart-from-start_cursor replay pattern. Previously only in the design doc; now load-bearing API contract is visible at the type. **#4 (deferred) Per-hook DoS budget for Installed tier** Henry's recommendation was to gate `Installed`-tier event-triggered hooks entirely until the budget design lands, allowing only Builtin/Trusted. That breaks 11 existing tests + the primary use case. Instead: documented the existing first-line throttle (`batch_limit` × `poll_interval`) as the current bound on indirect- recursion fanout, and tracked the full per-hook rate cap with poisoning + milestone-on-overrun as a follow-up. The self-trigger guard (committed earlier in this PR) catches the most common direct pattern; the throttle here bounds the indirect pattern until the proper budget lands. **#5 (deferred) Narrowed `HookObservableEvent` projection** Would prevent full `RuntimeEvent` surface from reaching Installed- tier hooks. Project-wide impact (events crate types, projection glue). Tracked as a follow-up; the existing sanitized-event projection bounds the surface to closed-vocab labels. All 156 hooks lib + 30 reborn integration tests pass. * chore(hooks): address nits from PR nearai#3640 review Bundle three nit-tier review items into a single commit: **nearai#9 Replace author-internal tags with NOTE(nearai#3640)** The Phase-5 PR (nearai#3640) had several `serrrfirat HIGH/MED #N on PR nearai#3640` comment tags in this PR's diff. These are review-internal scaffolding, not load-bearing for future readers. Replaced with `NOTE(nearai#3640)` in: - crates/ironclaw_hooks/src/dispatch.rs (self-observation guard) - crates/ironclaw_reborn/src/loop_driver_host.rs (scope validation, replay-gap milestone, subscription binding) - crates/ironclaw_reborn/tests/hooks_integration.rs (three regression tests covering scope validation, self-observation suppression, and replay-gap surfacing) - crates/ironclaw_turns/src/run_profile/host.rs (`EventSubscriptionTerminated` doc) **nearai#10 Replace 10ms spin-poll with tokio::sync::Notify** `wait_for_seen_events` polled the shared `Mutex<Vec<SeenRuntimeEvent>>` every 10 ms until the expected count was reached. Replaced with a `SeenLog` newtype that pairs the events vec with a `Notify`; the hook's `observe()` calls `seen.push(...)` which signals `Notify::notify_one`, and `wait_for_seen_events` parks on `notified().await` under a `tokio::time::timeout`. `notify_one` is a permit-store, so an event landing between snapshot and wait still wakes the waiter immediately. Test latency drops from ~10 ms median to sub-ms and is no longer rate-limited by the polling cadence. All 30 hooks_integration tests still pass. **nearai#11 Remove unused Clone derive on EventTriggeredHookContext** No call site clones the context — it's passed by reference. Dropped the derive to make the borrow contract clearer. * docs(hooks): reconcile event-triggered hooks design doc with Phase 5 reality Address gemini-code-assist review on `04-event-triggered-hooks.md`: - L50 (Likely surface): annotated the sketch's full `RuntimeEvent` use with a pointer to the narrowed-projection follow-up so the snippet no longer reads as a recommendation contradicting L119–121. - L55 (sink methods): replaced `note_fact` / `emit_audit` (which never shipped on `ObserverSink`) with the actual `note(category, summary)` primitive and cross-referenced Reborn's `EventTriggeredObserverSink`. - L95 (cursor / replay): "lost events during downtime acceptable" contradicted the at-least-once replay semantics described in the Phase 5 implementation notes. Rewrote the bullet to say replay is at-least-once from the persisted cursor and to spell out the operator obligation around cursor persistence before shutdown. - L100/115 (forbids events dep): the original doc claimed `ironclaw_hooks` forbids an `ironclaw_events` dep, but the Risk section noted the dep is already established via PR nearai#3573. Updated both passages to reflect that the dep direction is set; Phase 5 adds the *consumer* side. The narrowed `HookObservableEvent` projection is now framed as a follow-up tracked in nearai#3690. * refactor(hooks): unify event-triggered sink with ObserverSink Address PR nearai#3640 review findings A3, C4, F14, and cluster G: - F14: drop duplicate `EventTriggeredObserverSink` trait and reuse `ObserverSink` directly in the `EventTriggeredHook` trait. The two surfaces were signature-identical; keeping them separate let them drift, and a future gate/mutator method added to one would not surface as a compile error on the other. - A3: add `is_replay: bool` to `EventTriggeredHookContext` and a dedicated `dispatch_event_triggered_replay_at` entry point. The subscription contract is at-least-once, so side-effecting hooks need to dedupe by `event.event_id` on restart-driven replay. - C4: index event-triggered bindings by `RuntimeEventKind` at install time so dispatch is O(matches) instead of scanning every event-triggered binding for every event. - Cluster G: doc/04-event-triggered-hooks.md updated to reflect the unified sink, the explicit at-least-once semantics + `is_replay` signal, the actual `note(category, summary)` primitive (not the speculative `note_fact` / `emit_audit`), the corrected `ironclaw_events` dep status, and the issue nearai#3690 reference for the narrowed `HookObservableEvent` projection. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(hooks): adaptive backoff for event-triggered subscription Address PR nearai#3640 review findings C5, A1, A2: - C5: empty-poll backoff for `EventTriggeredHookSubscription`. The previous loop hammered the durable log at a fixed 50ms cadence under sustained idle, even when no events had arrived for minutes. The subscription now tracks consecutive empty polls and sleeps for `min(poll_interval << streak, max_poll_interval)` before the next poll, defaulting to a 1s cap; a non-empty batch resets the streak so producer bursts restore low-latency dispatch immediately. Exposed via `with_max_poll_interval` so callers can tune. - A1 / A2: explicit issue references for the deferred narrowed `HookObservableEvent` projection (nearai#3690) and the per-hook DoS dispatch budget (nearai#3689). The current self-trigger guard catches direct-recursion storms; the backoff bounds indirect ones until the proper budget design lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(hooks): cover event-triggered dispatch edge cases Address PR nearai#3640 review findings D8, D9, D10, D11, D12, and B: - D8: dispatching an event-triggered binding that has no installed hook impl must poison the slot and surface a Malformed failure rather than silently no-op. A follow-up dispatch on the same kind must skip the poisoned slot. - D9: registry validation rejects non-event-point bindings that carry an `event_kind_filter`, mirroring the existing reverse-direction check. - D10: the existing hook-meta serde round-trip tests always passed `None` for `owning_extension` and never asserted `event.provider`. Add `hook_meta_events_round_trip_owning_extension_as_provider` to pin the projection that scope filtering depends on. - D11: `scope_provider_for_runtime_event` falls back to `None` when the registry mutex is poisoned. Force a poison on a spawned thread and assert the resolver remains fail-closed. - D12: `run_event_triggered_hook` catches panics from the hook impl via `AssertUnwindSafe::catch_unwind`. Drive it with a deliberately panicking impl and assert `FailureCategory::Panic`. - Cluster B: when a hook-meta event has `provider: None`, the dispatcher recovers the owning extension from the registry's hex-keyed index so `OwnCapabilities` watchers still fire. Add a full end-to-end test exercising that path through `dispatch_event_triggered_at`. Also pin C4 indexing: a registry-level test that `active_for_event_kind` returns only bindings whose declared filter matches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(hooks): port event-triggered tests after foundation rebases (nearai#3911/nearai#3912/nearai#3913) - Add event_kind_filter: None to HookBinding test constructions (foundation added new field) - Replace tuple-struct construction of ExtensionId/HookLocalId with ::new() per nearai#3912 newtype privatization - Lowercase RuntimeEventKind debug repr for HookLocalId validation (lowercase-only identifiers) - Replace pub-use re-exports with module-path imports per foundation cleanup - Rename fixture.user_id to fixture.actor_id per nearai#3633 final naming * fix(hooks): adapt event-triggered to WASM hook runtime (nearai#3920) - Add event_kind_filter: None to WASM HookBinding constructions in dispatch.rs - Extend HookManifestKind match arms in registrar.rs and wasm/runtime.rs to handle EventTriggered (rejected: WASM-bodied event-triggered hooks are not yet supported) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JZKK720
pushed a commit
that referenced
this pull request
Jun 13, 2026
…#3899) * Reborn budgets: address all nearai#3841 follow-ups end-to-end Implements every open follow-up from PR nearai#3841 (cost-based budgets foundation), driven by the plan in `docs/plans/2026-05-22-reborn-budgets-followups.md`: - **C2 (provider tokens)**: `LoopModelResponse.usage` carries real `(input_tokens, output_tokens)` from `CompletionResponse` / `ToolCompletionResponse`; `usage_for_response` reconciles to actual USD via the cost table instead of the conservative estimate. - **D1 (cascade warnings)**: `CascadeOutcome` variants carry `Vec<BudgetWarning>` so warnings preceding a pause or hard deny reach the audit sink. `ResourceError::LimitExceeded` / `RequiresApproval` reshaped to struct variants. - **C1 (cancellation safety)**: new `LoopModelBudgetAccountant::release_in_flight` trait hook + RAII `ReservationReleaseGuard` in `HostManagedLoopModelPort::stream_model` so a cancelled future doesn't orphan its reservation. - **E1 (dead code)**: removed the never-set `budget_accountant` field on `ThreadBackedLoopModelPort`. - **Real cost table**: new `StaticModelCostTable` + `LlmModelProfilePolicy::build_cost_table()` populated from `ironclaw_llm::costs::model_cost` with `default_cost` fallback so unknown providers never silently reconcile to zero. - **B1 (filesystem gate store)**: new `FilesystemBudgetGateStore` mirroring `FilesystemResourceGovernorStore`; pending gates survive process restart. - **A1 (production wiring)**: composition builds `GovernorBackedAccountant` from the cost table + governor and threads it through `RebornLoopDriverHostFactory::with_model_budget_accountant`. - **A2 (audit / SSE projection)**: `InMemoryResourceGovernor::with_event_sink` emits `Reserved`, `Reconciled`, `Released`, `Warned`, `ApprovalRequested`, `Denied`, `LimitChanged`; composition holds an `InMemoryBudgetEventSink` ready for downstream SSE projection. - **F1 (stuck-loop normalization)**: `CapabilityCallSignature::from_call` now runs `progress::normalize_for_hash` so the existing repetition window collapses request-id / UUID / timestamp noise. Side fix: `ResourceValue` moved to adjacent serde tagging (the combination of internal tagging + `Decimal`'s `serde-with-str` representation breaks JSON serialization — rust-lang/serde#1402). Regression tests added per item — see the acceptance evidence appendix in the plan doc. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Reborn budgets: end-to-end test coverage via test-support feature Adds 13 e2e tests covering the budget pipeline through `build_reborn_runtime` + `send_user_message`. Required infrastructure: - **`test-support` feature** on `ironclaw_reborn_composition` exposing `BudgetTestGateway` (scripted token usage) and `RebornRuntimeInputTestExt`. Existing `model_gateway_override` field promoted from `#[cfg(test)]` to `#[cfg(any(test, feature = ...))]` with a new public `with_model_gateway_override_for_tests` setter. - **Cost-table override** on `RebornRuntimeInput` so tests can pair the gateway with a deterministic `ModelCostTable`. Without this, an override gateway dropped the cost table and the accountant never fired. - **Budget accessors** on `RebornRuntime`: `budget_resource_governor`, `budget_event_sink`, `budget_gate_store`, and `apply_resolved_budget_gate`. Test-feature gated. - **`ResourceGovernor::usage_for`** added as a default-impl trait method so tests read spend through the trait surface. - **`BudgetGateStore` wired into the accountant**: `GovernorBackedAccountant::with_gate_store(...)` opens a pending gate whenever the governor cascade returns `RequiresApproval`. The approval-required host error is unchanged; the gate is the out-of-band channel a user-facing handler resolves. Scenarios covered: | # | Test | What it asserts | |---|---|---| | F1 | `f1_happy_path_records_actual_usd_in_ledger` | Ledger depletes by provider tokens × cost table | | F2 | `f2_crossing_warn_threshold_emits_warned_event` | Warn fires alongside successful Reserved/Reconciled | | F3 | `f3_approval_with_increased_limit_unblocks_retry` | Approve → set_limit applies → retry succeeds | | F4 | `f4_cancel_keeps_budget_blocked_on_retry` | Cancel → retry still short-circuits | | F5 | `f5_expiry_marks_gate_terminal_and_keeps_budget_blocked` | Expiry → gate drops from pending list, retry still blocked | | F6 | `f6_hard_cap_denied_before_provider_call` | Estimate over cap → zero model calls, Denied event | | C1 | `c1_provider_tokens_reconcile_to_actual_usd` | Real numbers, not estimate | | C2 | `c2_unknown_model_in_cost_table_reconciles_to_zero` | Unknown profile → zero spend | | C3 | `c3_zero_cost_model_records_zero_spend` | Free model → zero USD with non-zero tokens | | D1 | `d1_agent_deny_preserves_user_warn_event` | Cascade emits both Warned and Denied | | D3 | `d3_fresh_user_without_limits_runs_without_denial` | No limit → no denial | | + | `pause_in_distinct_runs_produces_distinct_pending_gates` | Per-run gate identity | | + | `budget_test_gateway_scripted_replies_drive_per_turn_costs` | Multi-turn scripted accumulation | F7 (cancellation mid-stream) is unit-covered by `release_in_flight_drains_orphan_reservation_on_cancellation`. D2 (period rollover) is unit-covered by `rolling_24h_snapshot_reports_anchored_window_not_now_window`. B-series (background ticks) await the BackgroundKind scheduler call site (no production caller in Reborn yet). Run via `cargo test -p ironclaw_reborn_composition --features test-support`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Budget review feedback: address all 7 findings from PR nearai#3899 review Two High and five Medium issues raised by serrrfirat's multi-agent review. **High #1 — `FilesystemBudgetGateStore` cross-tenant leakage** The store hardcoded `ResourceScope::system()` for every op, so all tenants wrote into the same `/tenants/__SYSTEM__/...` snapshot and `list_pending` would expose gates across tenants. Fix: `new(...)` now takes a `ResourceScope`; each tenant gets its own store, and the `ScopedFilesystem` mount view routes the snapshot under that tenant's path. Added `list_pending_does_not_leak_across_tenants` regression. **High #2 — accountant wired without default budget limits** Composition built `GovernorBackedAccountant` without `with_seeding_policy`, so the local-dev governor started empty and `reserve_with_outcome_in_state` skipped accounts that had no configured limit — model calls reconciled spend but never enforced a cap. Fix: `build_reborn_runtime` now loads `BudgetDefaults::compiled_defaults().with_env()` and wires `BudgetSeedingPolicy` + `with_overestimate_factor`. Renamed the D3 test to `d3_seeding_policy_installs_default_cap_on_first_touch` to prove the wiring fires. **Medium #3 — RAII guard disarmed before post_model_call await** `HostManagedLoopModelPort::stream_model` was disarming the `ReservationReleaseGuard` before awaiting `post_model_call`. A cancellation during that await dropped the future without cleanup, orphaning the reservation. Fix: disarm AFTER `post_model_call` returns. `release_in_flight` is now idempotent (peek-then-release- then-remove) so a successful post-call + subsequent guard drop is a no-op. **Medium #4 — failed release drops the retry handle** `release_in_flight` removed the in-flight entry before calling `governor.release`. A transient storage error left the reservation active in the governor with the id discarded. Fix: peek first, release, only remove on success. Errors keep the entry retained for a future retry / cleanup hook. **Medium #5 — unknown model silently reconciles to zero USD** Both `estimate_for` and `usage_for_response` fell back to `ModelCost { 0, 0, 0 }` when the cost table had no entry for the effective model. Cost-table drift would silently bypass daily caps. Fix: `GovernorBackedAccountant` carries a `default_cost` (default ~ GPT-4o pricing, ~`$0.0000025 input + $0.00001 output per token`) used for unknown models. Callers wiring `ZeroCostTable` for free / Ollama explicitly opt out of the fallback. Updated the C2 e2e test to assert the new fail-closed shape. **Medium #6 — paused dimension lost when another hard-denies** `check_thresholds_all_interventions` stored `Approval` only in the `approval` slot, so when one dimension paused and another hard-denied, the `Deny { warnings, denial }` outcome lost the pause signal. Fix: also push a warning-shaped record for the paused dimension. **Medium nearai#7 — unbounded terminal-gate retention** The snapshot kept every gate forever; `open` / `resolve` / `get` / `list_pending` were O(total historical gates). Fix: `with_terminal_retention` (default 30 days). Every mutation prunes terminal gates whose resolution timestamp is older than the window. Added `terminal_gates_older_than_retention_are_pruned_on_next_write` regression. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci: replace lock-poisoned expects with PoisonError::into_inner scripts/check_no_panics.py flagged five .expect("...lock poisoned") calls in the new test_support.rs. Use the same idiomatic recovery pattern the rest of the codebase uses (see InMemoryBudgetGateStore, InMemoryBudgetEventSink): on a poisoned lock, recover the inner data via PoisonError::into_inner rather than panicking. The test gateway's state is append-only logs / replies queues, so reading them through a poisoned lock is safe. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Finish A1 / A2 / F1 from plan + honest plan doc update The plan claimed "all nine items landed" but A1 (production wiring), A2 (SSE projection), and F1 (full progress strategy) were partials. This commit finishes the work so the plan matches reality. **A1 — production-shape accountant builder** New `ironclaw_reborn_composition::build_default_budget_accountant` public helper that wires the seeding policy + overestimate factor + gate store from `BudgetDefaults::compiled_defaults().with_env()` and returns an `Arc<dyn LoopModelBudgetAccountant>`. Production loop composers call this with their `PersistentResourceGovernor` + `FilesystemBudgetGateStore` + LLM-policy-derived cost table; the local-dev runtime in `build_reborn_runtime` now uses the same helper instead of duplicating the seeding logic inline. Unit-tier regression `seeds_compiled_default_user_cap_on_first_touch` proves the helper installs the compiled-default $5 user cap on first model call. **A2 — broadcast sink + AppEvent projection** - `ironclaw_resources::BroadcastBudgetEventSink` wraps `tokio::sync::broadcast::Sender<BudgetEvent>` with `subscribe()` / `subscriber_count()`. `CompositeBudgetEventSink` fans events to multiple sinks. - Composition fans every `BudgetEvent` to the in-memory sink (for tests) AND the broadcast sink (for SSE projection) via `CompositeBudgetEventSink`. - New `AppEvent::BudgetWarn` / `BudgetPause` / `BudgetDenied` / `BudgetLimitChanged` wire-stable variants in `ironclaw_common::event`. - `src/bridge/budget_events.rs` carries the projection: a tokio task spawned by `spawn_budget_event_projection` drains the broadcast receiver and emits the appropriate `AppEvent` via `SseManager::broadcast_for_user`. System-scoped events (no user identity) are skipped. This is the only producer of these `AppEvent` variants per `.claude/rules/gateway-events.md`. - `RebornRuntime::broadcast_budget_event_sink()` exposes the sink to the binary so the startup path subscribes. E2E test `broadcast_sink_publishes_events_to_subscribers` drives a real `send_user_message` and asserts Reserved + Reconciled lands on the broadcast. **F1 — diminishing-returns stop condition** The earlier shipped `ParamHash` normalization in `CapabilityCallSignature` strengthened the existing `recent_call_signatures`-based repetition detector. This commit adds the second half of F1: a rolling output-token window that detects "wedged" loops the repetition detector misses (model keeps responding but produces no useful output). - `LoopExecutionState.recent_output_token_counts: BoundedRing<u32, 8>` populated by the executor from `LoopModelResponse::usage`. - `BoundedRing::iter` returns `impl DoubleEndedIterator` so the strategy can scan the trailing window. - `DefaultStopConditionStrategy` gets `min_delta_tokens` (default 4) + `noprogress_window` (default 4). When the last N turns all produce ≤ min_delta_tokens of output, fire `StopKind::NoProgressDetected`. - Regression tests: `four_consecutive_low_token_turns_trigger_no_progress` proves the detector fires; `occasional_low_token_turn_does_not_trip_no_progress` proves a productive turn resets the trailing count. **Plan doc** Updated the status header from "all nine items landed" to the honest per-item shape. Acceptance evidence table expanded with the new test names. New "Review-feedback fixes layered on top" subsection documenting all 2 High + 5 Medium findings addressed during review. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Address thermo-nuclear review: collapse filesystem-store duplication, flatten cfg permutations, split budget accountant Five structural simplifications surfaced by the deep audit on PR nearai#3899, plus two bug fixes from the earlier review pass: - ironclaw_resources: extract `cas_snapshot` shared infrastructure (`StorageError` + `Snapshot` traits + `CasSnapshotStore<F>` + async-runtime worker + per-path lock map) and merge `filesystem_gate_store.rs` into `filesystem_store.rs`. Deletes ~350 lines of duplicated read-modify-write + worker-thread + CAS machinery; both stores are now thin shims over the shared helper. - ironclaw_reborn_composition: flatten the 4-way cfg permutation in `build_reborn_runtime` model-gateway resolution into three flat steps (normalize override → build production gateway via cfg-gated helper → test override wins). Also drops the `unused_mut` warning. - ironclaw_reborn_composition: collapse the 3-layer test-only setter dance for `model_gateway_override` / `model_cost_table_override` into a single setter pair gated on `cfg(any(test, feature = "test-support"))`. Deletes the `RebornRuntimeInputTestExt` extension trait — integration tests now call the inherent methods directly. - ironclaw_loop_support: split the 1305-line `budget_accountant.rs` into `budget_cost_table.rs` (ModelCost/ModelCostTable trait/ZeroCostTable/ StaticModelCostTable), `budget_seeding.rs` (BudgetSeedingPolicy), and `budget_accountant.rs` (just GovernorBackedAccountant). Each module now owns one concern. - ironclaw_resources: add `impl Display for ResourceAccount` and route the hierarchical account-label rendering through it; delete the 60-line bespoke `account_label` helper from `src/bridge/budget_events.rs`. - ironclaw_common + bridge: collapse the four `AppEvent::Budget*` variants into a single typed `AppEvent::Budget(AppBudgetEvent)` with the four shapes carried inside the enum. Wire-shape stays identical (snake_case serde tag). - ironclaw_resources + ironclaw_loop_support: thread real gate id through `BudgetEvent::GateOpened { gate_id, needed, at }` (new variant) and have the accountant emit it via the broadcast event sink after store.open succeeds. The bridge now projects `BudgetEvent::GateOpened` (not `ApprovalRequested`) into `AppEvent::Budget(Pause { gate_id, ... })` so SSE consumers receive the persisted gate id rather than a fabricated zero uuid. - ironclaw_agent_loop: in the F1 token-counting path, push to `recent_output_token_counts` only when the model response carries `Some(usage)` and only on the `AssistantReply` arm (instead of `unwrap_or(0)`). Diminishing-returns detection now reflects real spend. Net delta: -461 lines (+999 / -1460). Workspace `cargo clippy` clean, `cargo test` clean on ironclaw_resources / ironclaw_loop_support / ironclaw_reborn_composition; budget_e2e + budget_approval_e2e both green. Pre-existing CI failures (`cli::tests::test_version` stack overflow, `facade_factory::production_*` RuntimeProcessPort missing) are unrelated and reproduce on the pristine branch tip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(cli): refresh insta snapshots after runtime-policy flag additions The `import`-feature variants of the help snapshots were left stale when `--deployment-mode`, `--runtime-profile`, `--yolo-disclosure` were added in cc04481 (nearai#3243); the `_without_import` variants were updated but these were not. CI was failing the snapshot assertion under the slim PR matrix (`--features postgres,libsql,html-to-markdown,bedrock,import`). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Address PR nearai#3899 thermo-nuclear review (TN #1, #2, #3) TN #1 — budget defaults resolved in wrong layer: - `build_default_budget_accountant` no longer reads process env; it now takes `&BudgetDefaults` as a parameter and the caller owns the config-layer precedence (compiled → section → env) plus the `validate()` call. - `RebornRuntimeInput` gains an optional `budget_defaults` field + `with_budget_defaults()` builder so the composition root passes a pre-resolved value. `build_reborn_runtime` falls back to `compiled_defaults().with_env() + validate()` when none is supplied so existing call sites keep working. TN #2 — gate-store scoping at wrong boundary: - `BudgetGateStore` trait methods (`open`, `resolve`, `expire_pending_older_than`, `get`, `list_pending`) now take `&ResourceScope` as first arg. `GovernorBackedAccountant` passes the caller's scope from `resource_scope(context)`. - `CasSnapshotStore` gains `update_with_scope` so the same store instance can route per-operation. `FilesystemBudgetGateStore` no longer takes scope at construction — one shared instance serves every tenant via the `ScopedFilesystem` mount view. - `InMemoryBudgetGateStore` ignores scope (suitable for single-tenant tests / local-dev); production multi-tenant filesystem path is correctly partitioned by `ResourceScope`. - `RebornRuntime::apply_resolved_budget_gate` now takes scope too. TN #3 — half-wired projection bridge: - Removed `src/bridge/budget_events.rs`, its `spawn_budget_event_projection` helper, the `AppEvent::Budget` variant, and the `AppBudgetEvent` type. No production caller ever subscribed the broadcast sink onto SSE and no frontend consumed the variant, so the half-wired bridge is gone pending a real owner that spawns a projection task with shutdown cancellation. - The runtime's `broadcast_budget_event_sink()` accessor stays so a future production composer can still subscribe without rebuilding the runtime. Bonus — to keep budget e2e tests working under the new libsql local- dev path that origin/reborn-integration introduced, added `PersistentResourceGovernor::with_event_sink` (parity with the `InMemoryResourceGovernor` accessor). The libsql variant of `build_local_dev_store_graph` now wires the composite sink to the persistent governor so governor-emitted `Warned`/`Denied`/`Reserved`/ `Reconciled` events reach subscribers on both feature paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Wire budget-event projection task into RebornRuntime Re-implements PR nearai#3899 follow-up A2 / Thermo-Nuclear #3 with a real production owner instead of leaving the broadcast sink half-wired: - `crates/ironclaw_reborn_composition/src/budget_events.rs` (new): `BudgetEventObserver` trait + `TracingBudgetEventObserver` default observer + crate-internal `BudgetEventProjection` task that drains the runtime's broadcast `Receiver<BudgetEvent>` and forwards every event to the observer. Cancellation via `CancellationToken`; lagged subscribers logged and resumed; receiver-closed exits cleanly. - `RebornRuntimeInput::with_budget_event_observer(...)` lets production owners install a custom observer (SSE projection, WS fan-out, telemetry export). When unset, the runtime installs the tracing observer so events always surface in structured logs. - `build_reborn_runtime` always spawns the projection task at runtime construction; `RebornRuntime::shutdown` cancels it and awaits the handle so background state drains before the runtime drops. - E2E test `projection_delivers_budget_events_to_installed_observer` drives `build_reborn_runtime` with a capturing observer and asserts the observer sees `Reserved` + `Reconciled` from a real model call, testing through the caller per `.claude/rules/testing.md`. - Existing `broadcast_sink_publishes_events_to_subscribers` updated to expect the runtime's own projection task as a baseline subscriber. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(reborn): rustfmt the merged loop_support import block The conflict resolution for the post-merge import list was not run through rustfmt; CI Formatting flagged the wrapping. No logic change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JZKK720
pushed a commit
that referenced
this pull request
Jun 13, 2026
…ED flag (nearai#3934) (nearai#3938) * feat(hooks): extension-declared hook section on ExtensionManifestV2 (nearai#3934) Add a `[[hooks]]` declaration surface to the production v2 extension manifest (`ironclaw_extensions::ExtensionManifestV2` and its projected `ExtensionManifest`). Each entry is carried as a structurally-typed `HookSectionEntryV2` DTO — a `local_id` plus the entry's body re-serialized to canonical TOML — so `ironclaw_extensions` (substrate) never imports the `ironclaw_hooks` predicate vocabulary. The composition layer, which depends on both crates, is the single seam that projects these payloads into typed `ironclaw_hooks::HookManifestEntry` values (a later commit). Parse-time structural bounds: `MAX_MANIFEST_HOOKS` (32, matching the downstream per-extension registration cap) and `MAX_HOOK_ENTRY_BYTES` (8 KiB per entry). Entries must be tables carrying a non-empty `id`; ids must be unique within the manifest. `#[serde(default)]` keeps every existing manifest valid (empty `hooks` vec). The DTO holds canonical TOML as a `String` rather than a `toml::Value` so the enclosing `ExtensionManifestV2` keeps its `Eq` derive (`toml::Value` is not `Eq`). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(hooks): composition-layer activation module (loader, first-party hook, flag) (nearai#3934) Add `ironclaw_reborn_composition::hooks` — the single seam that activates the hook framework in production. Implements four numbered pieces of nearai#3934: - Feature flag (item 7): `HooksActivationConfig`, default OFF, resolved from `HOOKS_ENABLED` (only `1`/`true`/`yes`/`on` enable; unset/anything else = OFF). Flag OFF ⇒ `build_hook_dispatcher_builder_factory` returns `None` and the runtime composes no dispatcher — exact pre-hooks behavior. Hard rollout-safety contract. - Manifest → registry loader (item 2): `install_extension_hooks` projects each `ExtensionManifestV2` `HookSectionEntryV2` (canonical TOML) into a typed `HookManifestEntry` and installs it via `HookRegistrar::install` at the `Installed` trust tier. This is the clean-boundary projection: the hook vocabulary lives only here, never in `ironclaw_extensions`. Trust attenuation is enforced by construction (registrar only calls `install_installed_*`). Fail-closed: any projection/install error fails the build loudly. - First-party builtin hooks (item 3): a single illustrative no-op observer (`NoOpObserverHook`), installed regardless of extensions. Ships dark (zero driver-visible effect even with the flag ON). Production catalog is TBD by design — this PR does not invent a first-party hook. - Dispatcher composition (item 5): builds a per-tenant `PredicateEvaluator` over the in-memory state backend (swappable via the new public `PredicateEvaluator::with_state_backend` for durable nearai#3933), validates the full install set once fail-closed, and returns a per-run builder-factory closure. Per-run construction (fresh registry/dispatcher per host build) + per-tenant evaluator give full isolation; the host factory attaches the run-scoped milestone sink internally. Per-tenant scoping is by construction: `build_reborn_runtime` runs once per identity, so everything here is tenant-local — no global registry. The router-backed gate-ref factory (PauseApproval/PauseAuth) and the security-audit sink (nearai#3922, not yet on this branch) are deferred follow-ups; their absence is fail-closed (PauseApproval surfaces as Denied) and noted for the PR body. Not yet wired into the runtime — next commit. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(hooks): wire dispatcher builder factory into build_default_planned_runtime (nearai#3934) Item 6 of nearai#3934. Add an optional `hook_dispatcher_builder_factory` to `DefaultPlannedRuntimeParts` and, in `build_default_planned_runtime`, call `.with_hook_dispatcher_builder_factory(...)` on the production `RebornLoopDriverHostFactory` when it is present. `None` (the default) means no dispatcher is composed — behavior identical to the pre-hooks runtime (rollout-safety contract). The composition layer (`build_reborn_runtime`) resolves the flag via `HooksActivationConfig::from_env()` and builds the factory against this tenant's extension registry (per-tenant by construction — the function runs once per identity). Fail-closed: a malformed manifest hook fails the build here rather than composing a broken dispatcher. A per-run builder factory (not a captured dispatcher instance) is used so the host attaches a run-scoped milestone sink internally per build — per-run telemetry attribution, the nearai#3573 capture-and-stick lesson. All `DefaultPlannedRuntimeParts` construction sites (8 test sites across ironclaw_reborn, ironclaw_product_workflow, ironclaw_reborn_composition) updated with the new field defaulting to `None`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(hooks): e2e activation tests through build_default_planned_runtime (nearai#3934) Item 8 of nearai#3934. Add four end-to-end tests in crates/ironclaw_reborn/tests/loop_driver_host.rs that drive the *production* composition function `build_default_planned_runtime` with a per-run hook dispatcher builder factory shaped exactly like the composition layer's output (first-party builtin no-op observer + extension-declared `Installed`-tier hooks projected from a manifest entry through `HookRegistrar::install`), then build a host via the composed `host_factory` and invoke a capability: - flag OFF (no factory): allowed capability completes unaffected and reaches the inner host runtime port — the pre-hooks behavior / rollout-safety contract. - flag ON, first-party-only no-op observer: outcome unchanged, inner port reached — the builtin ships dark. - flag ON, extension-declared deny hook: capability denied through the composed runtime and the inner port is never reached (installed at the Installed tier via the registrar; OwnCapabilities scope keyed to the capability provider). - per-tenant isolation: tenant A's deny hook fires; tenant B (separate build_default_planned_runtime composition, no hooks) completes the same capability — proving no cross-tenant leakage. Security-audit-on-deny assertion is intentionally deferred: nearai#3922's SecurityAuditSink is not yet on reborn-integration. It lands with the audit-sink wiring follow-up. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(hooks): thread hook_dispatcher_builder_factory through shared reborn harness (nearai#3934) The root-crate `tests/support/reborn/harness.rs` constructs `DefaultPlannedRuntimeParts` directly; add the new `hook_dispatcher_builder_factory: None` field so the parity-test harness compiles. Default `None` keeps the harness on the no-hooks path (unchanged behavior). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(hooks): proper error handling / safety annotations for activation production paths (nearai#3938 CI) The per-run dispatcher factory closure used `.expect()` on the first-party and extension hook installs, tripping the no-panics CI gate. These installs are pure replays of the install set already validated fail-closed (`?`) against a scratch builder at composition time, so they are genuine invariants. The factory type returns a non-Result `HookDispatcherBuilder` and is invoked deep in the run loop, so the documented `// safety:` suppression is the correct fix here. Hoisted the expect messages into `let` bindings so the `.expect(msg)` call fits on one line, keeping the scanner-required `// safety:` comment on the same line as the call after rustfmt. The malformed-manifest path (TOML projection) already uses real error propagation via map_err/`?` and is unaffected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(hooks): direct composition-loader coverage + activation-scope docs Address Codex non-blocking follow-ups on nearai#3938. Add three direct tests for the composition-layer hook loader (`install_extension_hooks` via `build_hook_dispatcher_builder_factory`), driving a real `ExtensionRegistry` with `[[hooks]]` declarations rather than mimicking the loader: - valid `own_capabilities` predicate hook installs at the Installed trust tier; the dispatcher carries the derived binding at BeforeCapability alongside the first-party no-op observer - malformed typed hook body (unknown `mode`) fails CLOSED with `RebornBuildError::InvalidConfig`, never a panic (the load-bearing degradation contract for untrusted external manifests) - a hook claiming `scope = same_tenant` without a verified grant is rejected by trust attenuation (fail-closed) No loader bug surfaced: `HookRegistrar::install` already returns `Result` on every malformed/over-scoped path and the loader maps it to `InvalidConfig` via `?`. Document activation scope at both the loader rustdoc and the `build_reborn_runtime` call site: production currently passes only `builtin_extension_registry()`, so third-party installed-extension hooks are not yet surfaced into the runtime path — only first-party-builtin and builtin-package-declared hooks activate today. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(hooks): thread HooksActivationConfig through input; empty production catalog Two maintainability cleanups on nearai#3938 (firat review): Item 1 — config hygiene: stop reading HOOKS_ENABLED via std::env::var deep inside build_reborn_runtime. Add a typed `hooks: HooksActivationConfig` field to RebornRuntimeInput (default OFF) plus a `with_hooks_config` builder. The composition root now consumes the typed config; the env var is resolved ONCE at the edge (the reborn CLI's build_runtime_input) via HooksActivationConfig::from_env and threaded down. Testable without env mutation; matches the project's env → typed config → composition pattern. Item 2 — empty production first-party catalog: stop shipping NoOpObserverHook as a first-party builtin. install_first_party_hooks is now a no-op (empty catalog); the production type/install/export for a hook that does nothing is gone (removed from lib.rs exports). The activation machinery is still tested end-to-end through the real composition path via a new `build_hook_dispatcher_builder_factory_with` seam that takes a first-party installer; tests pass a `#[cfg(test)]` NoOpObserverHook through it. Pinned the empty-catalog-is-valid contract: flag ON + empty first-party set + no extension hooks composes a valid zero-binding dispatcher (not a panic/error). Reconciled the sibling loader tests/docs (f6c79c0): the in-module tests now drive the test-only seam; the reborn e2e tests already used a test-local no-op and are untouched. Updated activation-scope docs (loader rustdoc + build_reborn_runtime call site) to reflect the now-single live source (builtin-package-declared hooks). Deferred (not touched): switching to the canonical extension registry for third-party installed-extension hooks (nearai#3934 follow-on). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(hooks): canonical registry + infallible plan + tenant-scoped counter docs/tests (nearai#3938) Addresses serrrfirat's thermo-nuclear re-review on 1e618d0. #1 (runtime.rs:839, canonical registry): make the extension registry a shared composition artifact. `build_local_dev` builds one `Arc<ExtensionRegistry>`, hands it to `HostRuntimeServices::new` AND stores it in `RebornLocalRuntimeServices.extension_registry`. Hook activation in `build_reborn_runtime` now consumes that same `Arc` instead of rebuilding a builtin-only sidecar, so capability dispatch and hook activation cannot drift. Third-party activation stays a follow-up, but it now follows the canonical registry rather than a separate path. #3 (hooks.rs factory machinery): replace the parse/validate/replay duplication + two prose-justified `.expect()` calls with a typed `HookInstallPlan`. TOML is projected once into typed entries, the full install set is validated once against a fresh builder (fail-closed via `?`), and `HookInstallPlan::rebuild` mints a fresh builder per run. The per-run path is infallible by construction: a plan only exists for an install set that already composed cleanly, so a deterministic replay from the identical fresh-empty start cannot fail. One extension-install code path (`project_extension_install_sets` + `install_extension_sets`) is shared by validation and rebuild. #4 (hooks.rs:295, predicate counter scoping): the evaluator/backend is intentionally tenant-scoped and shared across runs (rate/value caps keyed `(hook, tenant, capability)` with no run_id; a run-scoped limit would reset every run and enforce nothing). Document the split explicitly — per-run-fresh dispatcher, tenant-scoped predicate counters — in the module docs and fix the misleading "per-run isolation of hook state" wording in `ironclaw_reborn` (loop_driver_host.rs / runtime.rs). Add `predicate_counter_state_is_tenant_scoped_across_rebuilds`, which drives a real rate-cap predicate through two dispatchers from one factory and proves the second run sees the first run's recorded count. Rename `factory_mints_independent_dispatchers_per_call` -> `rebuild_mints_independent_dispatchers_per_call` and scope it to proving dispatcher freshness only. #6 (loop_driver_host tests): clarify that the hand-built builder factories cover host PLUMBING, not composition activation. Add `build_reborn_runtime_activates_hooks_through_real_composition_path`, which drives the real `build_reborn_runtime` with `HooksActivationConfig` threaded through `RebornRuntimeInput` (env-free) and the canonical registry, proving the production activation wiring composes. #2 (env boundary) and #5 (empty production catalog) were already fixed in 1e618d0; docs touched here for consistency. Known follow-up (not one of the six items, not introduced here): with the flag ON the standalone local-dev runtime does not yet reach `Completed` for a capability turn even with a zero-binding dispatcher — the composition root wires the dispatcher but not the companion hooked-prompt dependencies. The new runtime test asserts `is_terminal()` + the capability path and documents the gap. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(hooks): cover hook-entry rejection branches; document InvocationCount cap semantics (nearai#3938) Address henrypark133 review (review 4367870023): - Add extension-manifest tests for the three previously-uncovered hook-entry validation branches: non-table `[[hooks]]` element, whitespace-only `id`, and oversized entry (HookEntryTooLarge). - Document the InvocationCount inclusive-allow / deny-on-overflow semantics inline at the comparison site; behavior unchanged and still pinned by the cap test. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(reborn-cli): assert hooks config threaded in caller test (nearai#3938) Addresses the review finding that `build_runtime_input_maps_configured_cli_identity` exercised `build_runtime_input` but never asserted the `hooks` field, so a regression silently dropping `with_hooks_config(HooksActivationConfig::from_env())` or flipping the default-OFF rollout-safety contract would pass. Per `.claude/rules/testing.md` ("Test Through the Caller"): add two assertions to the existing caller-level test: - threading: `runtime_input.hooks == HooksActivationConfig::from_env()`, proving the env-resolved config is actually threaded through and not dropped. Verified via TDD that dropping the wiring fails this assertion (run under HOOKS_ENABLED=1). - default-OFF: when `HOOKS_ENABLED` is unset, `!runtime_input.hooks.is_enabled()`, guarded to skip if the CI environment exports the flag so it only pins the contract it claims to. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hooks): correct in-memory backend warning text; delegate test ctor (nearai#3938) Address serrrfirat review (2026-06-03): - Low: the in-memory backend warning claimed the LRU cap is shared across tenants, but the Reborn composition constructs a fresh InMemoryPredicateStateBackend per tenant. Rewrite the warn! text and doc comment so the real limitation (process-local replay dedup for multi-host deployments) is accurate, and note the backend is per-tenant in this composition. - Nit: PredicateEvaluator::with_backend (test-only) and with_state_backend had identical bodies; delegate with_backend to with_state_backend so they stay in lockstep. The Medium finding (hooks_config assertion in build_runtime_input caller test) was already addressed in 218a1de. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
JZKK720
pushed a commit
that referenced
this pull request
Jun 13, 2026
…ection (HOOKS_THIRD_PARTY_ENABLED, default OFF) (nearai#3951) * feat(hooks): extension-declared hook section on ExtensionManifestV2 (nearai#3934) Add a `[[hooks]]` declaration surface to the production v2 extension manifest (`ironclaw_extensions::ExtensionManifestV2` and its projected `ExtensionManifest`). Each entry is carried as a structurally-typed `HookSectionEntryV2` DTO — a `local_id` plus the entry's body re-serialized to canonical TOML — so `ironclaw_extensions` (substrate) never imports the `ironclaw_hooks` predicate vocabulary. The composition layer, which depends on both crates, is the single seam that projects these payloads into typed `ironclaw_hooks::HookManifestEntry` values (a later commit). Parse-time structural bounds: `MAX_MANIFEST_HOOKS` (32, matching the downstream per-extension registration cap) and `MAX_HOOK_ENTRY_BYTES` (8 KiB per entry). Entries must be tables carrying a non-empty `id`; ids must be unique within the manifest. `#[serde(default)]` keeps every existing manifest valid (empty `hooks` vec). The DTO holds canonical TOML as a `String` rather than a `toml::Value` so the enclosing `ExtensionManifestV2` keeps its `Eq` derive (`toml::Value` is not `Eq`). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(hooks): composition-layer activation module (loader, first-party hook, flag) (nearai#3934) Add `ironclaw_reborn_composition::hooks` — the single seam that activates the hook framework in production. Implements four numbered pieces of nearai#3934: - Feature flag (item 7): `HooksActivationConfig`, default OFF, resolved from `HOOKS_ENABLED` (only `1`/`true`/`yes`/`on` enable; unset/anything else = OFF). Flag OFF ⇒ `build_hook_dispatcher_builder_factory` returns `None` and the runtime composes no dispatcher — exact pre-hooks behavior. Hard rollout-safety contract. - Manifest → registry loader (item 2): `install_extension_hooks` projects each `ExtensionManifestV2` `HookSectionEntryV2` (canonical TOML) into a typed `HookManifestEntry` and installs it via `HookRegistrar::install` at the `Installed` trust tier. This is the clean-boundary projection: the hook vocabulary lives only here, never in `ironclaw_extensions`. Trust attenuation is enforced by construction (registrar only calls `install_installed_*`). Fail-closed: any projection/install error fails the build loudly. - First-party builtin hooks (item 3): a single illustrative no-op observer (`NoOpObserverHook`), installed regardless of extensions. Ships dark (zero driver-visible effect even with the flag ON). Production catalog is TBD by design — this PR does not invent a first-party hook. - Dispatcher composition (item 5): builds a per-tenant `PredicateEvaluator` over the in-memory state backend (swappable via the new public `PredicateEvaluator::with_state_backend` for durable nearai#3933), validates the full install set once fail-closed, and returns a per-run builder-factory closure. Per-run construction (fresh registry/dispatcher per host build) + per-tenant evaluator give full isolation; the host factory attaches the run-scoped milestone sink internally. Per-tenant scoping is by construction: `build_reborn_runtime` runs once per identity, so everything here is tenant-local — no global registry. The router-backed gate-ref factory (PauseApproval/PauseAuth) and the security-audit sink (nearai#3922, not yet on this branch) are deferred follow-ups; their absence is fail-closed (PauseApproval surfaces as Denied) and noted for the PR body. Not yet wired into the runtime — next commit. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(hooks): wire dispatcher builder factory into build_default_planned_runtime (nearai#3934) Item 6 of nearai#3934. Add an optional `hook_dispatcher_builder_factory` to `DefaultPlannedRuntimeParts` and, in `build_default_planned_runtime`, call `.with_hook_dispatcher_builder_factory(...)` on the production `RebornLoopDriverHostFactory` when it is present. `None` (the default) means no dispatcher is composed — behavior identical to the pre-hooks runtime (rollout-safety contract). The composition layer (`build_reborn_runtime`) resolves the flag via `HooksActivationConfig::from_env()` and builds the factory against this tenant's extension registry (per-tenant by construction — the function runs once per identity). Fail-closed: a malformed manifest hook fails the build here rather than composing a broken dispatcher. A per-run builder factory (not a captured dispatcher instance) is used so the host attaches a run-scoped milestone sink internally per build — per-run telemetry attribution, the nearai#3573 capture-and-stick lesson. All `DefaultPlannedRuntimeParts` construction sites (8 test sites across ironclaw_reborn, ironclaw_product_workflow, ironclaw_reborn_composition) updated with the new field defaulting to `None`. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(hooks): e2e activation tests through build_default_planned_runtime (nearai#3934) Item 8 of nearai#3934. Add four end-to-end tests in crates/ironclaw_reborn/tests/loop_driver_host.rs that drive the *production* composition function `build_default_planned_runtime` with a per-run hook dispatcher builder factory shaped exactly like the composition layer's output (first-party builtin no-op observer + extension-declared `Installed`-tier hooks projected from a manifest entry through `HookRegistrar::install`), then build a host via the composed `host_factory` and invoke a capability: - flag OFF (no factory): allowed capability completes unaffected and reaches the inner host runtime port — the pre-hooks behavior / rollout-safety contract. - flag ON, first-party-only no-op observer: outcome unchanged, inner port reached — the builtin ships dark. - flag ON, extension-declared deny hook: capability denied through the composed runtime and the inner port is never reached (installed at the Installed tier via the registrar; OwnCapabilities scope keyed to the capability provider). - per-tenant isolation: tenant A's deny hook fires; tenant B (separate build_default_planned_runtime composition, no hooks) completes the same capability — proving no cross-tenant leakage. Security-audit-on-deny assertion is intentionally deferred: nearai#3922's SecurityAuditSink is not yet on reborn-integration. It lands with the audit-sink wiring follow-up. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(hooks): thread hook_dispatcher_builder_factory through shared reborn harness (nearai#3934) The root-crate `tests/support/reborn/harness.rs` constructs `DefaultPlannedRuntimeParts` directly; add the new `hook_dispatcher_builder_factory: None` field so the parity-test harness compiles. Default `None` keeps the harness on the no-hooks path (unchanged behavior). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(hooks): proper error handling / safety annotations for activation production paths (nearai#3938 CI) The per-run dispatcher factory closure used `.expect()` on the first-party and extension hook installs, tripping the no-panics CI gate. These installs are pure replays of the install set already validated fail-closed (`?`) against a scratch builder at composition time, so they are genuine invariants. The factory type returns a non-Result `HookDispatcherBuilder` and is invoked deep in the run loop, so the documented `// safety:` suppression is the correct fix here. Hoisted the expect messages into `let` bindings so the `.expect(msg)` call fits on one line, keeping the scanner-required `// safety:` comment on the same line as the call after rustfmt. The malformed-manifest path (TOML projection) already uses real error propagation via map_err/`?` and is unaffected. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(hooks): direct composition-loader coverage + activation-scope docs Address Codex non-blocking follow-ups on nearai#3938. Add three direct tests for the composition-layer hook loader (`install_extension_hooks` via `build_hook_dispatcher_builder_factory`), driving a real `ExtensionRegistry` with `[[hooks]]` declarations rather than mimicking the loader: - valid `own_capabilities` predicate hook installs at the Installed trust tier; the dispatcher carries the derived binding at BeforeCapability alongside the first-party no-op observer - malformed typed hook body (unknown `mode`) fails CLOSED with `RebornBuildError::InvalidConfig`, never a panic (the load-bearing degradation contract for untrusted external manifests) - a hook claiming `scope = same_tenant` without a verified grant is rejected by trust attenuation (fail-closed) No loader bug surfaced: `HookRegistrar::install` already returns `Result` on every malformed/over-scoped path and the loader maps it to `InvalidConfig` via `?`. Document activation scope at both the loader rustdoc and the `build_reborn_runtime` call site: production currently passes only `builtin_extension_registry()`, so third-party installed-extension hooks are not yet surfaced into the runtime path — only first-party-builtin and builtin-package-declared hooks activate today. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(hooks): thread HooksActivationConfig through input; empty production catalog Two maintainability cleanups on nearai#3938 (firat review): Item 1 — config hygiene: stop reading HOOKS_ENABLED via std::env::var deep inside build_reborn_runtime. Add a typed `hooks: HooksActivationConfig` field to RebornRuntimeInput (default OFF) plus a `with_hooks_config` builder. The composition root now consumes the typed config; the env var is resolved ONCE at the edge (the reborn CLI's build_runtime_input) via HooksActivationConfig::from_env and threaded down. Testable without env mutation; matches the project's env → typed config → composition pattern. Item 2 — empty production first-party catalog: stop shipping NoOpObserverHook as a first-party builtin. install_first_party_hooks is now a no-op (empty catalog); the production type/install/export for a hook that does nothing is gone (removed from lib.rs exports). The activation machinery is still tested end-to-end through the real composition path via a new `build_hook_dispatcher_builder_factory_with` seam that takes a first-party installer; tests pass a `#[cfg(test)]` NoOpObserverHook through it. Pinned the empty-catalog-is-valid contract: flag ON + empty first-party set + no extension hooks composes a valid zero-binding dispatcher (not a panic/error). Reconciled the sibling loader tests/docs (f6c79c0): the in-module tests now drive the test-only seam; the reborn e2e tests already used a test-local no-op and are untouched. Updated activation-scope docs (loader rustdoc + build_reborn_runtime call site) to reflect the now-single live source (builtin-package-declared hooks). Deferred (not touched): switching to the canonical extension registry for third-party installed-extension hooks (nearai#3934 follow-on). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(hooks): canonical registry + infallible plan + tenant-scoped counter docs/tests (nearai#3938) Addresses serrrfirat's thermo-nuclear re-review on 1e618d0. #1 (runtime.rs:839, canonical registry): make the extension registry a shared composition artifact. `build_local_dev` builds one `Arc<ExtensionRegistry>`, hands it to `HostRuntimeServices::new` AND stores it in `RebornLocalRuntimeServices.extension_registry`. Hook activation in `build_reborn_runtime` now consumes that same `Arc` instead of rebuilding a builtin-only sidecar, so capability dispatch and hook activation cannot drift. Third-party activation stays a follow-up, but it now follows the canonical registry rather than a separate path. #3 (hooks.rs factory machinery): replace the parse/validate/replay duplication + two prose-justified `.expect()` calls with a typed `HookInstallPlan`. TOML is projected once into typed entries, the full install set is validated once against a fresh builder (fail-closed via `?`), and `HookInstallPlan::rebuild` mints a fresh builder per run. The per-run path is infallible by construction: a plan only exists for an install set that already composed cleanly, so a deterministic replay from the identical fresh-empty start cannot fail. One extension-install code path (`project_extension_install_sets` + `install_extension_sets`) is shared by validation and rebuild. #4 (hooks.rs:295, predicate counter scoping): the evaluator/backend is intentionally tenant-scoped and shared across runs (rate/value caps keyed `(hook, tenant, capability)` with no run_id; a run-scoped limit would reset every run and enforce nothing). Document the split explicitly — per-run-fresh dispatcher, tenant-scoped predicate counters — in the module docs and fix the misleading "per-run isolation of hook state" wording in `ironclaw_reborn` (loop_driver_host.rs / runtime.rs). Add `predicate_counter_state_is_tenant_scoped_across_rebuilds`, which drives a real rate-cap predicate through two dispatchers from one factory and proves the second run sees the first run's recorded count. Rename `factory_mints_independent_dispatchers_per_call` -> `rebuild_mints_independent_dispatchers_per_call` and scope it to proving dispatcher freshness only. #6 (loop_driver_host tests): clarify that the hand-built builder factories cover host PLUMBING, not composition activation. Add `build_reborn_runtime_activates_hooks_through_real_composition_path`, which drives the real `build_reborn_runtime` with `HooksActivationConfig` threaded through `RebornRuntimeInput` (env-free) and the canonical registry, proving the production activation wiring composes. #2 (env boundary) and #5 (empty production catalog) were already fixed in 1e618d0; docs touched here for consistency. Known follow-up (not one of the six items, not introduced here): with the flag ON the standalone local-dev runtime does not yet reach `Completed` for a capability turn even with a zero-binding dispatcher — the composition root wires the dispatcher but not the companion hooked-prompt dependencies. The new runtime test asserts `is_terminal()` + the capability path and documents the gap. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(hooks): third-party hook-only projection core (flag, newtype, quarantine, caps) Steps 1-6 of third-party extension hook activation via hook-only projection: - Step 1: HOOKS_THIRD_PARTY_ENABLED sub-flag on HooksActivationConfig (default OFF; is_third_party_enabled() requires master flag too). Resolved at the CLI edge via from_env(). - Step 2: tenant_extension_root(&TenantId) derives the fixed /system/extensions/<tenant> root from identity (never caller-supplied); projection-layer strict-child / no-`..` containment check. - Step 3: build_hook_projection_registry assembles a HookProjectionRegistry (type-enforced hook-only newtype: no Deref / conversion back to ExtensionRegistry, so it can never reach HostRuntimeServices::new / the capability path). Sub-flag OFF => builtin-only, byte-identical to nearai#3938. - Step 4/4a: atomic per-extension quarantine — untrusted (InstalledLocal) sets validated whole against a scratch builder, committed only if the whole set passes; any failure drops the extension's hooks entirely, emits a hook.quarantined security_audit tracing event (warn!, not info!), and continues. Trusted (HostBundled) sources stay fail-closed-whole-build. - Step 5: MAX_INSTALLED_EXTENSIONS_CONSIDERED / MAX_TOTAL_HOOKS_PER_TENANT DoS caps; count_total_bindings() accessor on HookDispatcher(Builder); pre-read MAX_MANIFEST_BYTES bound via read_file_bounded in discovery. - Step 6: third-party WASM stays out (loader registrar has no wasm_runtime) => WASM-bodied hook quarantines + build continues. Registrar-only invariant: projection installs go exclusively through HookRegistrar::install (ceiling + spoof-blocked owning_extension), never the direct builder installer API. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(hooks): FS-scoped tenant isolation (Option 1), trust matrix, registrar-only assertion Resolve the discovery/path conflict: the discovery layer hardcodes package roots to /system/extensions/<id> because the per-tenant RootFilesystem is the scope boundary (as with every other tenant-scoped resource), not a tenant path segment. So: - tenant_extension_root -> fixed /system/extensions (no tenant segment). The per-tenant RootFilesystem handed to discovery IS the isolation boundary. Documented as load-bearing; the openat2(RESOLVE_BENEATH)/O_NOFOLLOW backend hardening follow-up is what protects it (gating note kept prominent). - build_local_dev mounts /system/extensions to a per-owner host subtree under the storage root (per-identity by construction, not a process-global mount); exposed via RebornLocalRuntimeServices.extension_filesystem. - enforce_root_containment retained as defense-in-depth. Tests: - Integration (real build_hook_projection_registry + build_hook_dispatcher_ builder_factory through a fake RootFilesystem, not a loader look-alike): containment (hook present / capability absent by construction), FS-as-boundary tenant isolation proof (two distinct per-tenant filesystems; A can't see B), bad dir name skipped, id mismatch not a panic, surplus-extensions DoS cap, sub-flag OFF discovers nothing. - Per-hook-point trust matrix: BeforeCapability installed deny IS allowed and fires (Gate reachable); before_prompt predicate quarantined + build continues; after_model/after_capability/after_checkpoint/event_triggered WASM-only => quarantined + build continues; owning_extension derived (not spoofable). - Discovery pre-read bound: oversized manifest rejected via stat WITHOUT reading the body (fake fs panics on get); within-bound proceeds to read. - ironclaw_architecture source assertion: the hooks.rs projection path never calls install_installed_* directly (registrar-only invariant). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs(hooks): correct Option-1 path-shape references in test comments Update the third-party projection integration-test module docs to reflect the FS-scoped isolation model (fixed /system/extensions root; per-tenant filesystem is the boundary), not the abandoned /system/extensions/<tenant> path segment. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(hooks): tolerant+bounded third-party discovery, structural hook-only containment Addresses Codex P1/P2 + serrrfirat P1 on nearai#3951. Critical 1 (discovery-stage DoS): add `ExtensionDiscovery::discover_with_manifest_contracts_tolerant_bounded` (+ `discover_extensions_tolerant_bounded` host-runtime wrapper). It lists+sorts the root once, then reads/parses at most `max_extensions` manifests, recording the surplus as quarantines WITHOUT reading them. The hook projection calls this with `MAX_INSTALLED_EXTENSIONS_CONSIDERED`, so the count cap fires before the per-manifest read storm. New all-or-nothing path delegates to a shared `load_package_entry` so per-package semantics are identical. Critical 2 (fail-open): tolerant discovery quarantines a single malformed/oversized/id-mismatched package and CONTINUES; valid siblings still load. The builtin-only fallback is now reserved solely for failure to LIST THE ROOT (directory unreadable). One bad manifest can no longer drop a tenant's entire legitimate third-party hook set. Refinement 3: the per-tenant hook budget is consumed only AFTER a successful merge, so a quarantined/duplicate package no longer burns budget. Refinement 4: the registrar-only arch assertion now scans the WHOLE composition crate (every non-test source) and forbids all installed-tier-minting primitives crate-wide (`install_installed_*`, `install_observer(`, `insert_binding(`, `HookTrustClass::Installed`) — not just a hooks.rs substring scan. Installed-tier bindings can only be minted via `HookRegistrar::install`. serrrfirat P1 (structural containment): `HookProjectionRegistry` no longer wraps `ExtensionRegistry`. It carries `Vec<HookProjection>` — hook metadata only (id/version/source/root/[[hooks]]). The projection literally cannot reach capabilities because it does not hold them; containment is by data shape, not a withheld conversion. Removes the `ExtensionPackageView` ceremony. Tests: bounded read-storm cap (read-counting fs panics on surplus), tolerant per-package quarantine, root-unreadable fallback, quarantined-package-does-not- consume-budget, malformed-sibling-survives at the projection layer. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor(hooks): address serrrfirat review — tenant-attributed install audits + hooks decomposition Addresses the maintainability review on nearai#3951. Findings #1 (narrow hook-only boundary), #2 (per-extension discovery quarantine + mixed-batch test), and #5 (behavioral arch-test invariant) were already satisfied by the head commit (2b62597); this commit closes the two remaining items and hardens the arch test against the decomposition: - #3 (tenant attribution): add `build_hook_dispatcher_builder_factory_for_tenant`, threading the authenticated `tenant_id` (and its derived extension root) into the install-time quarantine-audit seam. `build_reborn_runtime` now calls it, so install-time quarantine audits carry the real tenant instead of the synthetic `reborn-hook-projection` fallback (closing the split where only discovery-time audits were attributed). New caller-driven test `for_tenant_entry_point_attributes_install_time_quarantine_to_real_tenant` asserts attribution via a deterministic thread-local audit capture (immune to tracing's process-wide max-level filter under parallel tests). - #4 (decomposition): split the 1.7k-line `hooks.rs` into a focused `hooks/` module — `mod.rs` (flag/config + public surface), `projection.rs` (hook-only `HookProjection`/`HookProjectionRegistry` containment + discovery/admission), `factory.rs` (first-party install, per-extension quarantine validation, fresh-per-build replay), `audit.rs` (`hook.quarantined` emission), and `tests.rs` (the test matrix). Behavior-preserving; no logic change. - arch test: skip dedicated test-module files in the registrar-only scan so the #4 decomposition cannot break it; the whole-crate behavioral invariant is preserved. - audit emission uses `debug!` (not `warn!`) per the background/hook-path logging rule, on the stable filterable `security_audit` target. - gemini nearai#353: add the documented no-empty-segment guard to `enforce_root_containment` (defense-in-depth, not relying on VirtualPath canonicalization). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(hooks): cover hook-entry rejection branches; document InvocationCount cap semantics (nearai#3938) Address henrypark133 review (review 4367870023): - Add extension-manifest tests for the three previously-uncovered hook-entry validation branches: non-table `[[hooks]]` element, whitespace-only `id`, and oversized entry (HookEntryTooLarge). - Document the InvocationCount inclusive-allow / deny-on-overflow semantics inline at the comparison site; behavior unchanged and still pinned by the cap test. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(deps): pin kuchikikiki to 0.9.1 (0.9.2 yanked) cargo-deny failed on the yanked kuchikikiki 0.9.2 pulled in transitively via readabilityrs. Downgrade to 0.9.1 at the lockfile level. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(reborn-cli): assert hooks config threaded in caller test (nearai#3938) Addresses the review finding that `build_runtime_input_maps_configured_cli_identity` exercised `build_runtime_input` but never asserted the `hooks` field, so a regression silently dropping `with_hooks_config(HooksActivationConfig::from_env())` or flipping the default-OFF rollout-safety contract would pass. Per `.claude/rules/testing.md` ("Test Through the Caller"): add two assertions to the existing caller-level test: - threading: `runtime_input.hooks == HooksActivationConfig::from_env()`, proving the env-resolved config is actually threaded through and not dropped. Verified via TDD that dropping the wiring fails this assertion (run under HOOKS_ENABLED=1). - default-OFF: when `HOOKS_ENABLED` is unset, `!runtime_input.hooks.is_enabled()`, guarded to skip if the CI environment exports the flag so it only pins the contract it claims to. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hooks): cover build_reborn_runtime third-party wiring + async dir create (nearai#3951) Address serrrfirat review findings M1 and L2. M1: add an integration test in tests/runtime.rs that drives build_reborn_runtime with HooksActivationConfig::enabled().with_third_party_enabled(true), a real /system/extensions manifest tree on the local-dev host filesystem, and tenant attribution. Asserts the runtime builds, starts a conversation turn, and shuts down cleanly — exercising the runtime.rs third-party discovery input + projection registry + tenant-threading wiring that was previously uncovered (the projection tests call build_hook_projection_registry / the dispatcher factory directly, and every other build_reborn_runtime call used the default disabled config). Verified the test fails when the wiring is broken. L2: switch the new factory.rs blocking std::fs::create_dir_all for the extensions host root to tokio::fs::create_dir_all(...).await with the same error mapping, so it no longer blocks the tokio executor thread inside the async build_local_dev. The two pre-existing std::fs calls (lines 132/136) are out of this PR's diff per the posted promise and are left untouched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hooks): L1 quarantine-surfacing gate doc, L3 robust test-mod strip, M1 coverage-gap TODO Address serrrfirat 2026-06-03 review (M1/L2 already landed in 9866793). L1 (security observability): hook.quarantined audit events are emitted only via tracing at the security_audit target / debug! level, which production typically disables. Document durable quarantine surfacing as a hard production-enablement prerequisite for HOOKS_THIRD_PARTY_ENABLED, alongside the existing openat2(RESOLVE_BENEATH)/O_NOFOLLOW FS-hardening note, at all three gate doc sites: HooksActivationConfig (hooks/mod.rs), the runtime.rs composition -root gate comment, and the audit.rs module doc. L3 (robustness): strip_test_module matched #[cfg(test)]\nmod tests specifically and only the first occurrence. Generalize the anchor to #[cfg(test)]\nmod (any module name) so a refactor that renames the test module or adds a second #[cfg(test)] mod block is still fully stripped, preventing false positives in the FORBIDDEN_INSTALLED_PRIMITIVES architecture scan. M1 (test coverage): the build_reborn_runtime third-party wiring test already landed in tests/runtime.rs (9866793). Add the reviewer-requested TODO preserving the removed test's Cancelled-outcome coverage gap: the stub local-dev gateway cancels the turn before any capability dispatches, so the test exercises discovery + projection + tenant-threading at build/start but not end-to-end hook enforcement. NOTE: third-party discovery is intentionally tolerant (skips unparseable manifests), so this test catches compile-time field/arg regressions and build-path failures but not a silent manifest-read drop; documented for the reviewer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(hooks): correct in-memory backend warning text; delegate test ctor (nearai#3938) Address serrrfirat review (2026-06-03): - Low: the in-memory backend warning claimed the LRU cap is shared across tenants, but the Reborn composition constructs a fresh InMemoryPredicateStateBackend per tenant. Rewrite the warn! text and doc comment so the real limitation (process-local replay dedup for multi-host deployments) is accurate, and note the backend is per-tenant in this composition. - Nit: PredicateEvaluator::with_backend (test-only) and with_state_backend had identical bodies; delegate with_backend to with_state_backend so they stay in lockstep. The Medium finding (hooks_config assertion in build_runtime_input caller test) was already addressed in 218a1de. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
JZKK720
pushed a commit
that referenced
this pull request
Jun 17, 2026
…r injection (nearai#4588) * feat(reborn): expose a trajectory observer hook on RebornRuntimeInput The reborn runtime is sealed: build_reborn_runtime returns only the final AssistantReply, and per-step capability (tool) calls + results live in internal stores. Downstream consumers (benchmark harnesses, UI/debuggers) can't observe the agent's trajectory. Add `RebornTrajectoryObserver` (pub trait: on_capability_input(call_id, name, args) / on_capability_result(call_id, output)) and `RebornRuntimeInput::with_trajectory_observer`. The local-dev capability IO (`LocalDevCapabilityIo`) forwards each tool call's name+args (at input staging) and result (at result write) to the observer when present — reusing the same data it already records for display previews. No-op when unset; best-effort. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * debug: trace observer hook firing (temporary) * feat(reborn): trajectory observer — capability_id on result, reliable spine Provider tool calls are staged by a lower decorator that bypasses the LocalDevCapabilityIo input path, so on_capability_input does not fire for them. on_capability_result fires for every completed capability — make it carry the capability_id so consumers can reconstruct the trajectory (name + output) from results alone. Input args capture is a follow-up. * feat(reborn): capture capability input args at the host port chokepoint Provider tool calls are staged by ProviderToolCallInputResolver, which keeps args in a private map and bypasses the capability-IO input hook — so inputs never reached the trajectory observer (only results did). Move the observer trait down to ironclaw_loop_support (CapabilityTrajectoryObserver, re-exported from composition as RebornTrajectoryObserver) and hook it in HostRuntimeLoopCapabilityPort::invoke_capability right after the input resolves — the one place the model's resolved arguments are visible. Threaded through HostRuntimeLoopCapabilityPortFactory + the local-dev factory. Result hook unchanged. Now name + args + output are all captured. * feat(reborn): host LLM-provider injection seam ResolvedRebornLlm::with_provider — drive the runtime with a caller-supplied LlmProvider (e.g. an instrumented wrapper that counts tokens/cost and captures reasoning) instead of always building one from config; build_llm_gateway honors the override. The only viable observability path for reborn, whose model calls run in spawned worker tasks a per-task tracing subscriber can't reach. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(reborn): cover trajectory observer + LLM provider override seams Addresses Firat's two blocking review findings on nearai#4588 (both missing-integration-test, per AGENTS.md "test through the caller"): 1. Trajectory observer callbacks — drive the real call sites with a recording CapabilityTrajectoryObserver: - host port: invoke_capability via HostRuntimeLoopCapabilityPortFactory ::with_trajectory_observer asserts on_capability_input fires with the resolved capability id + tool-call arguments. - local-dev IO: register_provider_tool_call_input + write_capability_result assert on_capability_input and on_capability_result fire and correlate by input ref. 2. LLM provider override — build_llm_gateway_drives_provider_override_not_config injects a counting mock via ResolvedRebornLlm::with_provider, points config at a dead endpoint, and asserts the gateway returns the mock's sentinel (proving the override is driven, not a config-built chain). Also fixes pre-existing breakage this surfaced: 5 LocalDevLoopCapabilityPort Factory test initializers (shell_tests.rs + tests.rs) were missing the trajectory_observer field added by this PR, so the composition crate's tests did not compile under --features root-llm-provider. loop_support: 301 passed; composition (root-llm-provider): 520 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reborn): make trajectory observer input semantics consistent Addresses Copilot's follow-up findings on the observer seam: - Drop the `on_capability_input` callback from `LocalDevCapabilityIo:: register_provider_tool_call_input`. It forwarded the raw provider tool name (`builtin_echo`) as the capability id — conflicting with the observer contract (resolved dotted `builtin.echo`) and the authoritative port-level hook — and `ProviderToolCallInputResolver` doesn't delegate here for provider tool calls, so it never fired in practice anyway. `HostRuntimeLoopCapabilityPort::invoke_capability` remains the single source of `on_capability_input` (resolved id); `LocalDevCapabilityIo` remains the source of `on_capability_result`. - Clarify the trait doc: `arguments` is the raw model-emitted tool-call input resolved from the input ref (the callback fires before schema normalization), which is what the trajectory should record. - Refocus the local-dev test on `on_capability_result` forwarding + correlation, and assert input staging does NOT emit `on_capability_input` from local-dev IO. Port-level input semantics stay covered by the capability_port.rs test. loop_support: 301 passed; composition (root-llm-provider): 520 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * wire trajectory_observer through RefreshingLocalDevCapabilityPortConfig Completes the main-merge conflict resolution: local_dev.rs passes trajectory_observer into the refreshing-port config, so the config struct + port struct must carry it and build_inner must apply it via .with_trajectory_observer(). (Missed staging this file in the merge commit.) * test(reborn): lock down the observability seams against regression nearai#4588 exposes two seams a downstream harness relies on. Add tests so a future refactor can't silently break either: - capability_io_forwards_result_to_trajectory_observer: drives write_capability_result and asserts on_capability_result fires with the correct (call_id, capability_id, output) — the result half of the trajectory observer (tool-call outputs). - build_llm_gateway_drives_provider_override_not_config: asserts the gateway drives a provider injected via ResolvedRebornLlm::with_provider (config points at a dead endpoint), proving the provider-injection seam works — this is how the bench captures reasoning / tokens / cost / system-prompt / tool-definitions. (Restores the test dropped during the main merge.) The input half (on_capability_input) is already covered by invoke_capability_forwards_resolved_input_to_trajectory_observer in ironclaw_loop_support. All three pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(reborn): drop the false-confidence result-hook test capability_io_forwards_result_to_trajectory_observer called write_capability_result directly, so it stayed green even though the result hook is unreachable end-to-end while capability dispatch fails (the LocalDevYolo InputEncode regression) — i.e. it did not fail when the feature it claimed to cover was actually broken. Remove it rather than ship false confidence. The result hook lives in LocalDevCapabilityIo and is only reached by a real local-dev runtime turn, so an honest guard must drive the full runtime and is red until the dispatch regression is fixed; that guard belongs as an end-to-end test (PR, once green) or a bench pre-flight, not a direct-call unit test. Kept: invoke_capability_forwards_resolved_input_to_trajectory_observer (input hook, real port code path) and build_llm_gateway_drives_provider_override_not_config (provider seam) — both genuinely fail if their seam regresses. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reborn): address review on the trajectory-observer + provider seams Resolves Henry + Firat review comments on nearai#4588: - Composition-owned RebornTrajectoryObserver trait + adapter to the loop-support CapabilityTrajectoryObserver, instead of re-exporting the substrate trait directly (CLAUDE.md: facade-shaped handles only). Loop-support contract changes no longer break the public Reborn API. (Henry#8) - Safe-preview by default: with_trajectory_observer now forwards bounded (truncated strings / capped arrays) payloads so a logs/UI/telemetry sink stays within the model-visible display boundary; a trusted in-process consumer that needs verbatim tool I/O opts in via the new with_raw_trajectory_observer. (Henry#5) - catch_unwind around both observer call sites (input hook in capability_port, result hook in LocalDevCapabilityIo) so a panicking observer can't unwind the capability hot path; trait doc now states the never-block / panic-caught contract. (Henry#1/#6) - e2e test local_dev_runtime_forwards_tool_call_trajectory_to_raw_observer: drives a real build_reborn_runtime turn dispatching builtin.echo and asserts BOTH input and result callbacks fire on the genuine dispatch path — honest coverage that replaces the dropped direct-call result-hook test, and proves the observer threads through build_reborn_runtime. (Firat#1, Henry#3/nearai#7) - Strengthened provider-injection docs: the config-vs-override invariant and why the feature-gated seam takes the LlmProvider substrate trait. (Henry#4/nearai#9/nearai#11) - Fixed the LocalDevCapabilityIo observer field comment to describe its actual result-only responsibility. (Henry#10) Provider-override coverage (Firat#2/Henry#2) already landed in build_llm_gateway_drives_provider_override_not_config. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(reborn): second-round review fixes on the trajectory/provider seams Addresses Henry's review of the first round (nearai#4588): - safe_preview_value now bounds objects (entry cap), recursion depth, and total nodes — not just strings/arrays — so a wide or deeply nested capability result can't force unbounded traversal/allocation on the hot path. (3405419089) - Narrowed the loop-support CapabilityTrajectoryObserver to input-only: HostRuntimeLoopCapabilityPort never staged results through the port (results go via LoopCapabilityResultWriter), so advertising on_capability_result there was a contract a direct user could never see fire. Result observation stays on the composition path (LocalDevCapabilityIo). (3405419104) - Synthetic capabilities (e.g. builtin.skill_activate) bypass the inner port's input hook, so the synthetic wrapper now emits on_capability_input itself after resolving input — otherwise consumers saw an unpaired result with no args. (3405419110) - Provider injection no longer accepts a wholesale Arc<dyn LlmProvider> through the facade: with_provider is replaced by with_provider_factory, a decorator Fn(Arc<dyn LlmProvider>) -> Arc<dyn LlmProvider>. The composition always builds the provider from config (config stays the single construction source — collapses the old config-vs-override invariant too) and hands it to the factory to wrap. (3405419100, 3405419146) - New caller-level test local_dev_runtime_safe_preview_observer_receives_bounded_payload: installs the default with_trajectory_observer, drives a real turn with a large echo payload, asserts the observer receives a truncated preview. (3405419095) - Dropped the stale nearai#4588/main-rebase comment for a durable invariant. (3405419113) cargo test (loop_support + reborn_composition, single-threaded) green; clippy clean on touched files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(reborn): rustfmt the trajectory/provider review changes Formatting-only: import grouping + mod ordering in the two lib.rs re-export blocks, and wrapping in runtime.rs / local_dev.rs / trajectory_observer.rs. Fixes the Formatting + Code Style CI checks. No behaviour change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(reborn): drop std-Mutex guard before await in observer e2e tests clippy::await_holding_lock (-D warnings): the two trajectory-observer e2e tests held the observer's std::sync::Mutex guard across runtime.shutdown().await. Shut down before inspecting the recorded callbacks (the data is already captured during the turn) so no guard is held across an await. Fixes Clippy (all-features). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP(bench): http empty-body + multi-tool-call port reuse + final-answer nudge Local checkpoint so the bench builds against a stable tree (uncommitted edits were being reverted mid-session). Bundles: http body() empty-field fix, RefreshingLocalDevCapabilityPort register reuse, the gated final-answer nudge + interactive_profile gate flip, and the trajectory-observer safe_preview borrow fix. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style(reborn): wrap an over-long line for rustfmt 1.9.0 CI installs the latest stable rustfmt (1.9.0 / Rust 1.96), which wraps a long eprintln! that older rustfmt left inline. Fixes the Formatting check. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(reborn): collapse nested if for clippy 1.96 collapsible_if clippy 1.96 (CI's stable) flags the nested if-let in the final-answer-nudge site as collapsible; fold it into a let-chain. No behaviour change. Fixes Clippy (all-features). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * WIP(bench): multi-tool-call port reuse (matches main nearai#4790) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * WIP(bench): nudge isolation - disable gate to measure marginal contribution * Revert stray bench WIP accidentally committed onto this branch Removes the http/nudge/multi-tool-call/diagnostic WIP commits (c4bbb5f, 2c670b4, 6da818a) that were committed onto the reborn-trajectory-observer branch by mistake during benchmarking and swept to origin by a main-merge push. Restores the affected files to origin/main (multi-tool-call is already fixed there by nearai#4790; the http fix lives in PR nearai#4827). Observer-owned changes in state.rs, refreshing_capability_port.rs, and local_dev.rs are preserved minus the stray WIP additions. No history rewrite / force-push. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(reborn): preserve provider factory across reload + reject observer off local-dev Addresses Firat's review of the trajectory/provider seams (nearai#4588): - Provider factory now survives a live config reload. build_llm_gateway applied the factory to the bare config provider *before* wrapping it in the SwappableLlmProvider, so the first WebUI/settings reload (which swaps the swappable's inner) silently dropped the instrumentation wrapper. Invert the layering: build the config provider, put it behind the swappable + reload handle, then apply the factory *over the swappable* for the gateway-facing provider. Reloads swap the inner; the wrapper stays in the call path. Regression test provider_factory_survives_live_reload reloads and proves the wrapper still observes subsequent model calls. - Reject a trajectory observer on profiles without a local runtime. The observer is wired only through the local-dev capability path; Production silently dropped it, so a caller got an empty trajectory with no error. Fail fast with InvalidArgument and document the seam as local-dev/bench-only. Test build_reborn_runtime_rejects_trajectory_observer_for_production. cargo fmt + clippy (all-features, -D warnings) clean under rustfmt 1.9/clippy 1.96. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(reborn): note trajectory observer is local-dev/bench-only Document the local-dev-only constraint + fail-fast behavior on the public with_trajectory_observer / with_raw_trajectory_observer setters (Firat review). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Pranav Raja <pranav.raja@near.ai>
JZKK720
pushed a commit
that referenced
this pull request
Jun 17, 2026
…ing the run (nearai#4954) * fix(reborn): surface approval-gate denial to model instead of cancelling the run Approval-gate denial in Reborn cancelled the run (deny_gate / replay_denied_gate -> cancel_run), so the model never learned the user declined and the next trigger re-issued the same approval-gated capability and re-blocked — the same loop class nearai#4944 removed for auth gates. Mirror nearai#4944 for approval gates: denial now RESUMES the parked run carrying a denial disposition; the capability stage converts ONLY the approval-gated call into a model-visible non-retryable Authorization failure ("approval gate denied by user", SameCallRetryConstraint:: Forbidden) and the loop continues. Unrelated parallel calls are unaffected. Per the maintainability review of the plan, this unifies rather than duplicates the nearai#4944 plumbing: - ironclaw_turns: AuthResumeDisposition -> GateResumeDisposition (one gate-agnostic enum); ResumeTurnRequest/TurnRunRecord/TurnRunState/ AgentLoopDriverResumeRequest field auth_resume_disposition -> resume_disposition. Serde key pinned to "auth_resume_disposition" (rename attr) so persisted run records still deserialize; legacy-key round-trip test added. - ironclaw_agent_loop: PendingApprovalResume gains a disposition field; the auth denied short-circuit in CapabilityStage::process is extracted into ONE shared short_circuit_denied_resume helper used by both the auth and approval paths (no second copy). - ironclaw_product_workflow: approval deny_gate / replay_denied_gate resume instead of cancel; ResolveApprovalInteractionResponse::Denied (CancelRunResponse) -> Resumed(ResumeTurnResponse); idempotent replay guarded by terminal run status. - ironclaw_reborn: PlannedDriver::resume stamps the disposition onto the pending resume that is set (auth or approval). Decisions (plan docs/plans/2026-06-15-reborn-approval-deny-continue.md): both Denied and Cancelled continue (consistent with nearai#4944, no Cancel variant). The extension_install/extension_search missing-observation gap is a separate PR; the user-visible extension-install loop is only fully closed when both land. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reborn): address PR nearai#4954 review — stamp denial on matching gate slot only Review round 1 fixes: - planned_driver: the denial disposition was stamped onto BOTH pending_auth_resume and pending_approval_resume on a comment-only "one slot at a time" invariant. GateStage deliberately preserves a pending auth resume when a non-auth gate blocks mid-re-dispatch, so both slots can be set at once; stamping both corrupted an unrelated auth resume. Now stamps only the pending slot whose gate_ref matches the blocking gate (state.last_gate). Adds a regression test asserting the auth slot stays None when the approval gate is denied, plus an end-to-end resume() drive. - approval replay: match GateResumeDisposition::Denied explicitly rather than is_some(), keeping the gate-agnostic carrier tied to denial. - tests: real TurnRunRecord struct-level serde test (legacy auth_resume_disposition key → resume_disposition) + snapshot-level legacy denied-marker test; new deny-path resume-error test asserting the record is denied and the run is never cancelled on resume failure. - arch-exempt annotation on short_circuit_denied_resume's too_many_arguments allow (plan nearai#4954); stale comments/typos fixed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reborn): route denied-gate replay through resume_turn idempotency (auth + approval) Review finding nearai#7: the denied-gate replay paths derived idempotency from current run state (TurnStatus::is_terminal() guard) rather than replaying through resume_turn. After the first Deny resumed the run, a transport retry with the same idempotency key arriving after the runner completed returned StaleGate/StaleAuth instead of the original ResumeTurnResponse — the observable result depended on runner timing. resume_turn is idempotent by key (memory.rs:665 returns the cached Result from resume_idempotency before the precondition check). Both the approval (replay_denied_gate) and auth (resume_denied_auth replay arm) paths now replay through resume_turn with the same key, deleting the terminal-guard branching: a retried key replays the original response regardless of run state; a genuinely stale request with a fresh key still errors via the precondition. Auth and approval kept symmetric. FakeTurnCoordinator now models resume idempotency by key so the replay tests are meaningful; terminal-guard assertions re-framed around same-key replay vs fresh-key stale, plus an explicit idempotent-replay test on both services. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reborn): fail closed on ambiguous dual-slot stamp; lock deny-before-resume order Review round 2 (both Major): - planned_driver stamp_resume_disposition: the if/else-if silently stamped the auth slot if both pending slots matched last_gate. At the denial- attribution boundary that could misattribute an approval denial. Now an explicit 4-way match fails closed on the ambiguous (true, true) case (warn + stamp neither). Test added. - approval_interaction_contract deny-resume-error test: asserted only aggregate call counts, which pass even if call order regressed. Added a shared ordered trace across the resolver (deny) and coordinator (resume_turn) fakes and assert deny is recorded strictly before resume. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(reborn): strengthen replay/checkpoint coverage; downgrade fail-closed log to debug Review round 3 (straightforward): - idempotent deny-replay tests (auth + approval) now assert full ResumeTurnResponse payload equality, not just run_id. - stamp_resume_disposition ambiguous-dual-slot diagnostic: warn! -> debug! (REPL/TUI logging rule — internal fail-closed diagnostics use debug!). - executor: assert the first approval BeforeBlock checkpoint carries pending_approval_resume.disposition == None before any denial. - executor: denied-approval short-circuit no-matching-call test (denied X, model emits only Y -> X not surfaced, Y dispatches, pending cleared). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(reborn): unify gate Declined resolution; keep WebUI processing on resume Review round 3 (design + High): - WebUiGateResolution: the approval card sends `denied`, the auth cards send `cancelled`, and both are now treated identically (resume the run and surface the decision to the model). Run termination is a separate control (the X -> cancelRun route), not a gate resolution. Collapsed the two equivalent variants into one `Declined` (serde aliases "denied"/"cancelled" keep the wire stable; no JS change). Facade maps Declined -> Deny for auth, approval, and the generic fallback. - #6 WebUI desync (High): useChat.resolveGate kept processing only for approved/credential_provided, dropping processing + activeRun on denied/cancelled — but those now resume the run. resolveGate now always keeps processing/activeRun; the terminal run_status SSE event clears it and the X/cancelRun path remains the only stop. Fixes the latent auth-cancelled desync from nearai#4944. assets.rs assertion + useChat tests updated. - nearai#7 helper weight: short_circuit_denied_resume no longer returns the DeniedResumeOutcome enum / boxes LoopExecutionState / clones the batch. It returns ControlFlow<TurnCompletedStep, (state, remaining_calls)>; the completed_turn/empty-remaining tail moved to the two call sites. Heavy per-denied-call failure synthesis stays shared (one helper). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(reborn): share denied-approval resume between deny_gate and replay Review (Medium): deny_gate and replay_denied_gate built an identical ResumeTurnRequest, mapped the same errors, and returned the same Resumed shape — the only difference was deny_gate's one-off resolver.deny side effect. Extracted a shared resume_denied(request, run_id) helper; deny_gate performs the durable denial then delegates to it, and replay_denied_gate calls it directly. Removes the duplicated request construction / path handling. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JZKK720
pushed a commit
that referenced
this pull request
Jul 18, 2026
… inspection (nearai#5280) * docs: spec for Trace Commons instance enrollment, profiles, and trace inspection Cross-repo design (ironclaw + trace-commons-server) for three coexisting capabilities: instance-wide enrollment, per-user contributor accounts via login-links, and submitted-trace inspection. Introduces a trace-credential resolver so the existing user-invite model and the new instance-wide model both function on one instance, with personal-invite enrollment taking precedence. Server change is additive (optional per-user subject through claim issuance + login-link + account resolution). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: Slice 0 plan — trace-commons-server per-user subject TDD plan for the one server change the whole effort depends on: accept an optional opaque subject in the upload-claim request and derive a per-user, tenant-namespaced principal at device-key issuance. Submission attribution, login-link account resolution, and trace readback all become per-user automatically from the shared bearer principal; absent subject reproduces today's behavior. Targets trace-commons-server (contributor-account-slice1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: IronClaw plans for Trace Commons slices 1-4 Slice 1: trace-credential resolver (personal-invite wins, instance fallback with per-user subject) + admin-gated instance enrollment. Slice 2: per-user subject plumbing through upload-claim request + submission. Slice 3: trace_commons.account_login_link first-party capability (profiles). Slice 4: per-user submitted-trace inspection across reborn_traces → product_workflow facade → webui_v2 handler → frontend. Each plan is bite-sized TDD against verbatim-extracted current code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(traces): trace-credential resolver (personal invite wins, instance fallback w/ subject) * refactor(traces): single dir-parameterized policy-path site (remove resolver duplication) Extract `trace_contribution_dir_for_scope_at`, `trace_policy_path_at`, `read_trace_policy_for_scope_at`, and `write_trace_policy_for_scope_at` as the canonical base-dir-parameterized path helpers. All public functions (`trace_contribution_dir_for_scope`, `read_trace_policy_for_scope`, `write_trace_policy_for_scope`) now delegate to the `_at` variants with `ironclaw_base_dir()` — signatures unchanged. The inline `read_policy` closure in `resolve_trace_credentials_at` that re-implemented path layout is deleted; it now calls `read_trace_policy_for_scope_at` directly. The test `write_policy_at` helper's bespoke path construction is replaced with a call to `write_trace_policy_for_scope_at`. The now-dead `trace_policy_path` function is removed. Path layout is encoded in exactly one place. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(traces): instance-level enrollment write path (scope None) * test(traces): make instance-enrollment test hermetic (tempdir, no global base) Rework `instance_onboard_writes_instance_level_policy` to operate entirely under a `tempfile::tempdir()`: - Compute instance_dir as base.path().join("trace_contributions") (scope=None layout, no users/<hash> segment) rather than calling the global LazyLock. - Call `onboard_at_dir_with_sink` directly against the tempdir so the test never touches the real ~/.ironclaw tree. - Assert policy.json by reading and deserializing it from the tempdir. - Remove all manual std::fs::remove_* cleanup lines; tempdir drops automatically. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(admin): AdminScope::enroll_instance_trace_commons (admin-gated instance enrollment) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(traces): carry optional per-user subject in upload-claim request Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(traces): thread resolver subject into submission claim context * test(traces): claim request carries per-user subject end-to-end * feat(traces): mint_account_login_link_via_sink (POST /v1/account/login-links) Add `mint_account_login_link_via_sink` to ironclaw_reborn_traces: - `TraceUploadClaimContext::for_account(subject)` constructor for account-management call contexts (no trace/submission ids, no consent scopes). - `AccountLoginLink { account_id, url }` return type. - `account_login_links_url(policy)` helper that derives the login-links URL from the upload-claim issuer URL (strip /v1/trace-upload-claim, append /v1/account/login-links). - `mint_account_login_link_inner(base_dir, ...)` private dir-parameterised core: resolves credentials, selects correct scope_dir for DeviceKey auth (instance enrollment → instance scope dir; personal → user scope dir), mints bearer, POSTs subject, parses response. - `mint_account_login_link_via_sink(tenant_id, user_id, sink)` public entry point wrapping the inner function with the real base dir. Tests (hermetic, tempdir-isolated): - `mint_account_login_link_posts_subject_and_returns_url`: verifies the posted subject equals `local_pseudonymous_contributor_id(trace_scope_key(...))` for instance-enrolled users via an axum mock serving both the upload-claim issuer and the login-links endpoint. - `mint_account_login_link_errors_when_not_enrolled`: verifies error path. - `ReqwestContributionSink` test helper added to the test module. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(traces): error instead of silent misroute in account_login_links_url Replace the unwrap_or_else fallback (which silently used the full issuer URL as a base when the /v1/trace-upload-claim suffix was absent) with an explicit anyhow error. Add two unit tests: one asserting an Err on a wrong-suffix URL, one asserting the correct .../v1/account/login-links URL on a valid issuer. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(host_runtime): add consent-gated trace_commons.account_login_link capability Mints a Trace Commons browser login URL via host network egress, mirroring dispatch_profile_token. Includes consent gate, enrollment pre-check, HostEgressContributionSink routing, and two e2e tests. Also fixes a sanitizer bug: validate_runtime_request was rejecting authorization headers on all requests, including RuntimeKind::FirstParty. FirstParty requests are host-internal and trusted to carry bearer tokens; the sensitive-header and manual-credentials guards now only apply to untrusted plugin runtimes (WASM/MCP/Script). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(host_runtime): route trace bearer via credential injection; restore FirstParty sensitive-header guard Commit 9e25d99 blanket-exempted all RuntimeKind::FirstParty requests from the egress sensitive-header and manual-credentials guards so the host-minted Trace Commons bearer could pass. builtin.http is also FirstParty but forwards model-supplied headers, so this let the model smuggle Authorization/Cookie/ x-api-key headers (or user:pass@ URLs) to allowlisted hosts. Revert the sanitize.rs exemption (guards now apply to ALL runtimes again) and deliver the trace bearer through the staged credential-injection path instead: the HostEgressContributionSink stages the minted token one-shot via RuntimeSecretMaterialStager and declares a StagedObligation Authorization-header injection, mirroring the SlackProtocolHttpEgress pattern. The stager is now exposed to first-party handlers via InvocationServices. Covers the profile_token, profile_set/community-profile, and account_login_link bearer paths. Regression tests: FirstParty + raw authorization header -> denied; FirstParty + user:pass@ URL -> denied. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(traces): fetch_account_traces_via_sink (GET /v1/account/traces, per-user) - Add ContributionHttpMethod::Get variant; update all exhaustive match sites in ironclaw_reborn_traces and HostEgressContributionSink in ironclaw_host_runtime. - Extract account_api_base_url() shared helper; account_login_links_url and new account_traces_url both delegate to it (DRY). - Add AccountTraceItem (Debug, Clone, Serialize, Deserialize; serde defaults). - Add fetch_account_traces_via_sink / fetch_account_traces_inner mirroring mint_account_login_link pattern: unenrolled -> Ok(vec![]), non-2xx -> Ok(vec![]), transport error -> Err. - Tests: hermetic axum mock (GET /v1/account/traces), unenrolled empty-list, URL shape with/without limit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(reborn): trace_account_traces facade method + wire types Adds RebornAccountTrace / RebornAccountTracesResponse wire types and a trace_account_traces default method on RebornServicesApi, mirroring the trace_credits egress pattern (crate-local hardened reqwest, no host-egress sink). Also adds fetch_account_traces (direct path) to ironclaw_reborn_traces::contribution so the facade can fetch server traces without coupling to RuntimeHttpEgress. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat(reborn): GET /api/webchat/v2/traces/account handler + contract test * feat(reborn-ui): render submitted Trace Commons traces in settings Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore(traces): document flush-gate limitation, hermetic account-traces contract test, annotate sink scaffold Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style(traces): cargo fmt across Trace Commons slice changes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(traces): resolver-aware flush gate (instance-enrolled users can contribute) The autonomous trace-flush gate read only the per-scope (personal-invite) policy and aborted when it was disabled, so instance-enrolled users (whose enrollment lives at scope None) could never contribute traces — and the per-user scope_dir would also fail to load the instance device key. Introduce a single EffectiveFlushTarget resolver (resolve_effective_flush_target, mirroring resolve_trace_credentials but keyed on the already-composed scope string) that returns the policy, device-key dir, and per-user subject in one policy-read/path pass. The flush gate now proceeds for instance-only enrollment, loads the device key from the instance (None) dir, and attributes uploads via the per-user pseudonymous subject. The redundant subject_for_scope helper (which re-read the same policies with silent .ok() error swallowing) is removed and its logic folded into the new helper with proper error propagation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): include per-user subject in upload-claim cache key Under instance enrollment every user shares the same instance device-key dir (scope None), so the upload-claim cache key — which keyed on scope_dir but not subject — collided across users. A bearer minted for one subject could be served from cache to another, mis-attributing traces / leaking across users. Add a hashed subject component to the DeviceKey cache key and a regression test proving two subjects sharing a scope_dir get distinct keys (and a no-subject personal-invite context stays distinct from both). Found by Codex review of PR nearai#5280. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): address CodeRabbit review on PR nearai#5280 - account_login_link manifest: declare ReadFilesystem effect (it reads local enrollment/policy/device-key state before egress), matching profile_token. (CR #2) - account-traces fetch: always send a bounded, clamped limit ([1, 500], default 200) so None never triggers an unbounded server history fetch. (CR #3) - direct fetch path: bound the response body with a hard byte ceiling (256 KiB) via a chunked bounded reader, instead of buffering unbounded. (CR #5) - account-traces fetch (both sink + direct): stop swallowing every non-2xx as an empty list — 404 = legitimate empty (no account yet), all other non-2xx surface as Err so the WebUI renders a sanitized unavailable state. Add regression tests (500 -> err, 404 -> empty). (CR #6) - trace-commons-tab.js: render missing final_credit as "—" not "0.00"; surface useAccountTraces() query errors instead of collapsing them to "no traces". (CR nearai#7, nearai#8) - handlers contract test: capture the forwarded caller in the trace_account_traces stub and assert the route threads the authenticated user id (test-through-the-caller). (CR nearai#9) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ci): cover trace_commons.account_login_link + backfill trace i18n keys PR nearai#5280 added the builtin.trace_commons.account_login_link capability and a submitted-traces UI section, but left three guardrail/parity tests un-updated, turning CI red: - ironclaw_host_runtime builtin_first_party_package_declares_expected_capabilities: register account_login_link in the expected id list and its Ask-permission arm. - reborn_builtin_first_party_capability_e2e_coverage_is_complete: add genuine e2e coverage by exercising account_login_link in the existing trace_commons parity test (confirmed=true on a not-enrolled scope returns a deterministic NotEnrolled, no network), grant it in the harness allow-set, and add it to the model-visible surface test and the covered-capability list. - ironclaw_webui_v2_static all_locales_share_the_en_key_set: backfill the six new traceCommons.* submitted-traces keys into all ten non-en locales. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): address CodeRabbit review — withhold login URL, type errors, wire i18n - Security (host_runtime): dispatch_account_login_link returned the one-time login `url` (a code-bearing account-access credential) on the model-visible surface, persisting it into the LLM transcript and any downstream logging. Follow the profile_token pattern: persist the URL to a 0600 private file (atomic temp+rename) and return an opaque `link_delivery` marker instead. E2e test now asserts the URL/code never appears in the result and is delivered out-of-band to the private file. - Typed error (product_workflow): account_traces_for_user flattened backend errors into String before the WebUI boundary. Introduce AccountTracesError (thiserror) that names the failing operation and preserves the full cause chain ({:#}); the boundary keeps returning a sanitized, diagnosable 500. Also document that fetch_account_traces(None) is already server-bounded (default 200, clamp 500, 256 KiB response cap) — the "unbounded fetch" concern was resolved by prior hardening. - i18n (webui_v2_static): the traceStatus and traceReceivedAt keys backfilled for locale parity were unused by the consumer. Wire traceStatus as the status badge's accessible title/aria-label and render traceReceivedAt as the timestamp label, so all six submitted-traces keys are now consumed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): address CodeRabbit re-review — async persist + typed identifiers - Blocking I/O (host_runtime): persist_account_login_link does mkdir/write/ fsync/rename with std::fs on the async dispatch path. Wrap the persist call in tokio::task::spawn_blocking so it never stalls a Tokio worker (coding guideline: all I/O async). Atomic temp+rename behavior is unchanged; a join failure maps to the same sanitized "could not write" result. - Typed identifiers (product_workflow): account_traces_for_user took bare &str tenant/user; the caller already holds TenantId/UserId newtypes. Take &TenantId/&UserId and only cross to &str at the ironclaw_reborn_traces boundary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): instance-aware enrollment across trace_commons dispatch + UI Instance-only-enrolled users (admin-provisioned instance policy, no personal invite) were falsely rejected across the Trace Commons surface: the dispatch gates and the profile mints read only the personal per-scope policy, and the submitted-traces UI was gated behind the personal-credits branch. Addresses CodeRabbit re-review (#3, #4, #5) on PR nearai#5280. reborn_traces: - Add instance-aware entry points mint_profile_attribution_token_for_user_via_sink and set_community_profile_for_user_via_sink that resolve enrollment via resolve_trace_credentials (personal OR instance) and build the claim context with the instance scope_dir + per-user pseudonymous subject, mirroring mint_account_login_link_inner. Refactor the token mint to share a context-based core. New tests assert the per-user subject reaches the issuer. host_runtime (trace_commons dispatch): - Route the enrollment gates in dispatch_status, dispatch_profile_token, dispatch_profile_set, and dispatch_account_login_link through resolve_trace_credentials so instance-only contributors pass. status now reports the resolved (instance or personal) policy. profile_token/profile_set call the new instance-aware mints. - #4: preserve the stage_secret_material_once failure cause (log it) instead of discarding it with map_err(|_|); wire message stays sanitized. webui_v2_static (#3): - Lift the submitted-traces section out of the credits/empty-state branch so instance-enrolled users with no personal credits still see their traces and any tracesQuery errors. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(traces): isolated dispatch-layer e2e for instance-only enrollment Extract the trace_commons dispatch e2e helpers into a shared tests/support/trace_commons_dispatch.rs module (base-dir setup, mock issuer, runtime/dispatch helpers, find_persisted_login_link, test_jwt_eddsa) so a second test binary can reuse them. Add trace_commons_instance_dispatch_e2e.rs — a SEPARATE binary (fresh process = private IRONCLAW_BASE_DIR) that provisions the process-global instance policy (scope None) without bleeding into the personal-invite suite. It pins the CodeRabbit #5 fix at the layer it manifests: an instance-only-enrolled user (no personal invite) passes dispatch_status and dispatch_account_login_link and mints under the shared instance device key with a per-user pseudonymous subject (asserted via the subject on the login-links POST). No production changes; trace_commons_dispatch_e2e.rs behavior is unchanged (5 tests still pass) — only its helpers moved to the shared module. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): sanitize bearer-staging log + typed IDs on mint entry points Addresses CodeRabbit overnight review on PR nearai#5280. - Security (#1): the trace-bearer staging error was debug-logged via `?error`, which can leak secret-store/backend detail on the credential path. The host_runtime logging guideline forbids backend error detail here — log only the safe fact of failure; the wire message stays sanitized. (Supersedes the earlier "preserve cause" change specifically on this bearer-material path.) - Typed identities (#2): the three agent-facing Trace Commons mint entry points (mint_account_login_link_via_sink, mint_profile_attribution_token_for_user_via_sink, set_community_profile_for_user_via_sink) now take &TenantId/&UserId instead of adjacent &str, so callers can't transpose tenant/user and misattribute a contributor. Identity stays typed to the public boundary and is stringified only when handing off to the dir-parameterised `_inner` cores / resolver (the storage edge). Adds ironclaw_host_api as a reborn_traces dependency (no cycle: host_api does not depend on reborn_traces). Dispatch callers pass the typed scope ids directly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): sanitize persist-path logs, preserve handle-validation cause Two follow-up CodeRabbit findings on PR nearai#5280: - Security (Major): dispatch_account_login_link's spawn_blocking persist arms logged %error / %join_error at debug. Filesystem errors (mkdir/write/fsync/ rename) can carry raw host paths, which the host_runtime guideline forbids in logs. Drop the interpolation; log only the generic fact, keep the message sanitized — same treatment as the bearer-staging path. - Maintainability (Minor): SecretHandle::new(TRACE_COMMONS_BEARER_HANDLE) used map_err(|_| ...), discarding the cause (non-exemptible per the guideline). The handle name is a compile-time constant, so its validation error carries no secret/path — bind and log it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): typed login-link errors, per-request bearer handle, doc accuracy Addresses the third CodeRabbit review round on PR nearai#5280. - Security (Major): the trace-bearer staging used a constant SecretHandle (TRACE_COMMONS_BEARER_HANDLE). The injection store is a HashMap keyed by (scope, capability, handle) with overwrite-on-insert, so two concurrent same-scope Trace Commons egresses could race and stage/consume the wrong bearer. Suffix the handle with a per-request uuid so every staged bearer key is distinct. Localized to the shared HostEgressContributionSink, so all trace_commons flows benefit. - Correctness (Major): account_login_link_error_value classified failures by substring-matching upstream error wording, coupling the public error_code contract to phrasing. Introduce a typed AccountLoginLinkError (thiserror) in reborn_traces; mint_account_login_link_via_sink returns it, producing the specific variant at each failure site. The host maps variants -> error_code with no substring checks. NotEnrolled (the only tested code) is preserved; the two bearer-derived codes collapse into EnrollmentIncomplete (both meant "re-run onboarding"), and persist failures get a distinct LocalStateWrite. - Docs (Minor): the persist_account_login_link comments promised 0600 across platforms though only Unix enforces it. Softened to "private local file (0600 on Unix)". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(traces): type profile_token/profile_set error mappers (systemic) Follow-up to the account_login_link typed-error change: convert the remaining substring-based error mappers so all four trace_commons dispatch flows derive the public error_code contract from typed variants instead of matching upstream error wording. (onboard was already typed via OnboardError.) - reborn_traces: add ProfileAttributionError (shared by the profile_token and profile_set token mints) and CommunityProfileError (profile_set wrapper adding InvalidProfile). mint_profile_attribution_token_for_user_via_sink and set_community_profile_for_user_via_sink now return these; each failure site produces the specific variant (NotEnrolled / PolicyRead / EnrollmentIncomplete / Backend / LocalStateWrite, plus InvalidProfile for profile_set). - host_runtime: profile_token_error_value / profile_set_error_value now match on the typed variants — no error.contains(...) anywhere in the file. NotEnrolled and InvalidProfile (the tested codes) are preserved; the issuer/device/refused substrings collapse into EnrollmentIncomplete, consistent with the account_login_link mapping. Also sanitized the profile_token persist-failure log (host-path leak class), matching the login-link path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): split enrollment precondition from backend in token mints CodeRabbit re-review: collapsing every mint_profile_attribution_token_with_context (and bearer_token) failure into EnrollmentIncomplete mislabels transient transport/status/serde failures as "re-run onboarding". Split the local precondition (upload-claim issuer URL configured) from post-resolution failures: a missing issuer URL maps to EnrollmentIncomplete via an explicit upload_claim_issuer_missing() check (typed, no substring), while the claim mint / bearer fetch / PUT failures now map to Backend. Applied consistently across profile_token, profile_set, and account_login_link so the error_code contract reflects the real failure class. URL-derivation preconditions (ingest/login-links URL) stay EnrollmentIncomplete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): check login-link URL precondition before minting bearer Fail-closed ordering: the local account_login_links_url derivation ran after bearer_token, so a malformed/absent login-links URL would mint a device-key bearer and hit the issuer before failing. Move that local precondition ahead of all secret/egress work so incomplete enrollment fails closed with no side effects. (profile_token/profile_set already order local preconditions first.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): make upload-claim cache key match issuer payload exactly The subject cache-key component trimmed/collapsed context.subject, but the DeviceKey issuer request sends it unchanged — so None, Some(""), and whitespace variants could share a cache key while minting different payloads, letting one user's claim be served from cache to another (cross-user trace mis-attribution). This is nearai#5280's per-user-subject cache-key path. Hash the exact optional bytes the request sends (DeviceKey → subject, WorkloadTokenEnv → None) with a None/Some discriminator. Extend the cache-key test with the Some("")-vs-None and whitespace-variant collision cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): check response-size cap before growing the buffer Both bounded response readers (upload-claim and account-traces) enforced the hard byte ceiling only after extend_from_slice, so a single oversized chunk could push the buffer past the advertised limit before the error returned. Compute bytes.len() + chunk.len() (checked_add) and validate before appending. Pre-existing pattern (from nearai#4559), fixed here per review. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): fail loud when trace policy cannot be statted read_trace_policy_for_scope_at used Path::exists(), which maps stat/permission errors to false — silently treating an unreadable policy as missing and default-disabled, flipping enrollment/flush behavior. Use try_exists() and propagate the stat error with context; only a confirmed non-existent path returns the not-enrolled default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(traces): capture traces for instance-only enrolled users Codex P1: capture_turn_trace gated on the per-user scope policy (read_trace_policy_for_scope(Some(scope)) + policy.enabled), so an instance-only enrolled user — whose per-user policy is absent/disabled — had every turn dropped before an envelope was queued, leaving the instance-aware flush gate nothing to submit. The headline instance-enrollment feature never captured for exactly the users it targets. Gate capture on the effective enrollment instead, mirroring the flush gate: add resolve_effective_capture_policy (personal-invite policy if enabled, else the admin-provisioned instance policy at scope None, else None) and prepare the envelope under that governing policy. Add a resolver test covering the personal / instance-only / neither cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Remove accidentally committed frontend node_modules, restore .gitignore The merge commit f34dfa7 dropped crates/ironclaw_webui_v2_static/frontend/.gitignore and swept 1065 node_modules files into the index. Untrack them and restore the ignore. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address PR review feedback: egress hardening, effect declaration, instance status sync - fetch_account_traces_direct now uses a pinned-DNS, private-IP-filtered HTTP client (shared pinned_trace_commons_http_client) instead of an unrestricted reqwest lookup, closing the DNS-rebinding window between claim validation and the bearer-authenticated account-traces GET. - account_login_link capability manifest declares EffectKind::WriteFilesystem for the local delivery-file write. - Queue-flush status sync (and the public sync entry point) now run off the resolved effective flush target (policy, device-key dir, per-user subject) instead of re-reading the per-scope policy, so instance-enrolled users get final credit status after submission; subject is threaded into the status-sync claim context. - Each login-link mint persists to a unique account_login_link.<uuid>.url file so concurrent mints cannot clobber each other; stale link files are pruned best-effort after one hour. - resolve_trace_credentials takes typed &TenantId/&UserId at the public boundary; call sites drop their .as_str() conversions. - Login-link/account-traces requests honor the policy-configured issuer timeout; the sink-path traces fetch uses ACCOUNT_TRACES_MAX_RESPONSE_BYTES. - Removed the AdminScope::enroll_instance_trace_commons wrapper from the v1 monolith (crate-side entry point is onboard_instance_with_sink; noted in the slice1 plan). - Tests: direct account-traces path covered for 500/404; new regression test pins instance-target status sync (subject + instance device-key dir). - Plan docs: server login-link contract callout, no developer-local paths, resolver errors propagate, 404-only zero-state, scope_dir threading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Pin DNS resolution on the background trace submit/status/revoke lane The background lane (queue flush submission, status sync, revocation) previously relied only on enrollment-time endpoint validation (validate_trace_commons_ingest_url); the per-request client did a fresh unrestricted DNS lookup. Replace trace_remote_http_client with pinned_trace_remote_http_client: per-request host resolution through resolve_trace_upload_claim_issuer_host (private/internal IPs rejected, literal-loopback local-dev exception) pinned via resolve_to_addrs, so an endpoint host that passed validation at enrollment cannot later rebind to an internal address and receive bearer-authenticated requests. Timeout behavior (env/test task-local override) is unchanged. Regression test: pinned_trace_remote_client_rejects_private_endpoint_hosts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address CodeRabbit follow-up: sanitize status log, sync plan snippets - trace commons status dispatcher no longer formats the resolver error into the log (it can embed the policy file's host path); logs the safe fact only, matching the sibling dispatchers. - slice4 plan: AccountTraceItem snippet derives Deserialize (matches shipped code, which parses the response). - slice3 plan: login-link parsing snippet fails loud on missing account_id/url instead of unwrap_or_default (matches shipped code). - slice1 plan: the AdminScope wrapper task is marked SUPERSEDED up front so the plan no longer gives conflicting guidance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address round-2 review: opt-out precedence, salted subjects, UI branch tests - Explicit per-user opt-out (scoped policy present with enabled=false, as written by 'traces opt-out') now blocks the instance-enrollment fallback in resolve_trace_credentials and resolve_effective_flush_target (and thus capture) — only a never-configured scope falls through to the instance policy. Regression test covers all three resolution surfaces. - Instance-enrollment subjects are now salted: a per-instance random salt (persisted 0600 at the instance trace dir, create_new race-safe) feeds sha256(salt:scope), so the server or ledger holders cannot dictionary-match guessable tenant/user ids against an unsalted scope hash. Unsalted local_pseudonymous_contributor_id remains for local state keying/log refs. - contribution.rs carries the architecture-rule file-size justification referencing decomposition tracking issue nearai#4088; state_scope field docs now say which state it does (and does not) locate. - Submitted-traces UI: extracted the pure tracesSectionMode decision (error wins over list; list needs enrolled + non-empty) and covered it plus the row formatters in trace-commons-tab.test.mjs. - Docs: slice4 plan points at crates/ironclaw_webui_v2 (static crate was folded in), slice3 signature snippet matches the typed contract, and the webui_v2 CLAUDE.md route table gains the three trace routes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Update crossbeam-epoch 0.9.18 -> 0.9.20 for RUSTSEC-2026-0204 Lockfile-only patch bump of a transitive dep (via termimad/crossbeam) to clear the new advisory failing cargo-deny; verified locally with cargo deny check advisories. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Route login-link/account-traces claim mint through the caller's sink The sink-based entry points (mint_account_login_link_via_sink, fetch_account_traces_via_sink) used the sink for the final POST/GET but minted the upload-claim bearer via DefaultTraceUploadCredentialProvider, whose issuer request takes the direct reqwest path — so an agent-invoked account_login_link performed a network call outside RuntimeHttpEgress. New trace_upload_bearer_token_via threads Option<sink> into the claim mint (cache behavior unchanged; the default provider passes None), and both sink paths pass Some(sink), matching the profile-token/profile-set flows. Tests now use a RecordingSink to pin the invariant that both the claim mint and the follow-up request route through the sink. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-generated by release CI. Updates SHA256 checksums and version-pinned artifact URLs in registry manifests to match the released WASM artifacts. Only extensions whose version changed since the last release are included.