chore: promote staging to staging-promote/42623ed1-23780941831 (2026-04-01 23:10 UTC) - #1893
Merged
Merged
Conversation
Chore sync staging with main
github-actions
Bot
added
scope: tool/mcp
* Add Docker Hub workflow and optimize Dockerfile for size Switch from Debian bookworm-slim to Alpine for both build and runtime stages, reducing image size from 162MB to 64MB (60% reduction): - Alpine + musl: runtime drops from ~88MB to ~11MB, no libssl needed (project uses rustls throughout) - Profile dist: thin LTO for smaller binary (74MB → 53MB) - panic=abort + codegen-units=1: eliminates unwinding tables and enables better whole-program optimization New docker.yml workflow builds and pushes to nearaidev/ironclaw on Docker Hub, triggered on push to main, version tags, or manual dispatch. Uses GitHub Actions cache for Docker layer caching. Tested locally (macOS) and on gpu07 (Ubuntu): gateway serves HTTP 200, SSE streams, chat API, memory/tools/settings endpoints all functional, ~700ms startup time. * Address review feedback - Fix image size comment (~30MB → remove specific number) - Add actions:write permission for GHA cache - Only push semver tag on version tag events (not on every main push) - Add comment explaining why panic=abort is set via env var * Make docker workflow release-only via workflow_call Replace push trigger with workflow_call so release.yml can invoke it, plus workflow_dispatch for on-demand builds. This prevents every staging promotion to main from publishing a new image. Simplified tag logic — always pushes version + latest + sha. * Pin cargo-chef and wasm-tools versions for reproducible builds
…g, and patch (#1723) * feat(workspace): metadata-driven indexing/hygiene, document versioning, and patch support Foundation for the extensible frontend system. Workspace documents now support metadata flags (skip_indexing, skip_versioning, hygiene config) via folder-level .config documents and per-file overrides, replacing hardcoded hygiene targets and indexing behavior. Key changes: - DocumentMetadata type with resolution chain (doc → folder .config → defaults) - Document versioning: auto-saves previous content on write/append/patch - Workspace patch: search-and-replace editing via memory_write tool - Hygiene rewrite: discovers cleanup targets from .config metadata instead of hardcoded daily/ and conversations/ directories - memory_read gains version/list_versions params - memory_write gains metadata/old_string/new_string/replace_all params - V14 migration adds memory_document_versions table (both PG + libSQL) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback — transaction safety, patch mode, formatting - Wrap libSQL save_version in a transaction to prevent race condition where concurrent writers could allocate the same version number - Make content optional in memory_write when in patch mode (old_string present) — LLM no longer forced to provide unused content param - Improve metadata update error handling with explicit match arms - Run cargo fmt across all files Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings — write-path performance, version pruning, descriptions 1. Resolve metadata once per write: write(), append(), and patch() now call resolve_metadata() once and pass the result to both maybe_save_version() and reindex_document_with_metadata(), cutting redundant DB queries from 3-5 per write down to 1 resolution. 2. Optimize version hash check: replaced get_latest_version_number() + get_version() (2 queries) with list_versions(id, 1) (1 query) for the duplicate-hash check in maybe_save_version(). 3. Wire up version_keep_count: hygiene passes now prune old versions for documents in cleaned directories, enforcing the configured version_keep_count (default: 50). Removes the TODO comment. 4. Fix misleading tool description: patch mode works with any target including 'memory', not just custom paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: wire remaining unwired components — changed_by, layer versioning 1. changed_by now populated: all write paths pass self.user_id as the changed_by field in version records instead of None, so version history shows who made each change. 2. Layer write/append versioned: write_to_layer() and append_to_layer() now auto-version and use metadata-optimized reindexing, matching the standard write()/append() paths. 3. append_memory versioned: MEMORY.md appends now auto-version with metadata-driven skip and shared metadata resolution. 4. Remove unused reindex_document wrapper: all callers now use reindex_document_with_metadata directly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: comprehensive coverage for versioning, metadata, patch, and hygiene 26 new tests covering critical and high-priority gaps: document.rs (7 unit tests): - is_config_path edge cases (foo.config, empty string, .config/bar) - content_sha256 with empty string (known SHA-256 constant) - content_sha256 with unicode (multi-byte UTF-8) - DocumentMetadata merge: null overlay, nested hygiene replaced wholesale, both empty, non-object base memory.rs (2 schema tests): - memory_write schema includes patch/metadata params, content not required - memory_read schema includes version/list_versions params hygiene.rs (5 integration tests): - No .config docs → no cleanup happens - .config with hygiene disabled → directory skipped - Multiple dirs with different retention (fast=0, slow=9999) - Documents newer than retention not deleted - Version pruning during hygiene (keep_count=2, verify pruned) workspace/mod.rs (14 integration tests): - write creates version with correct hash and changed_by - Identical writes deduplicated (hash check) - Append versions pre-append content - Patch: single replacement, replace_all, not-found error, creates version - Patch with unicode characters - Patch with empty replacement string - resolve_metadata: no config (defaults), inherits from folder .config, document overrides .config, nearest ancestor wins - skip_versioning via .config prevents version creation [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address zmanian review — PG transaction safety, identity protection, perf Must-fix: 1. PostgreSQL save_version now uses a transaction with SELECT FOR UPDATE to prevent concurrent writers from allocating the same version number, matching the libSQL implementation. 2. Restore identity document protection in hygiene cleanup_directory(). MEMORY.md, SOUL.md, IDENTITY.md, etc. are now protected from deletion regardless of which directory they appear in, via is_identity_document() case-insensitive check. This restores the safety net that was removed when migrating from hardcoded to metadata-driven hygiene. Should-fix: 3. resolve_metadata() now uses find_config_documents (single query) + in-memory nearest-ancestor lookup, instead of O(depth) serial DB queries walking up the directory tree. 4. memory_write validates that at least one mode is provided (content for write/append, or old_string+new_string for patch) with a clear error message upfront, instead of relying on downstream empty checks. 5. Fixed misleading GIN index comment in V15 migration. 9. Added "Fail-open: versioning failures must not block writes" comments to all `let _ = self.maybe_save_version(...)` call sites. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: cargo fmt * fix: address Copilot review — DoS prevention, no-op skip, duplicate hygiene Security: - Reject empty old_string in both workspace.patch() and memory_write tool to prevent pathological .matches("") behavior (DoS vector) Correctness: - Remove duplicate hygiene spawn in multi-user heartbeat — was running both via untracked tokio::spawn AND inside the JoinSet, causing double work and immediate skip via global AtomicBool guard - Disallow layer param in patch mode — patch always targets the default workspace scope; combining with layer could silently patch the wrong document - Restore trim-based whitespace rejection for non-patch content validation (was broken when refactoring required fields) Performance: - Short-circuit write() when content is identical to current content, skipping versioning, update, and reindex entirely - Normalize path once at start of resolve_metadata instead of only for config lookup (prevents missed document metadata on unnormalized paths) Cleanup: - Remove duplicate tests/workspace_versioning_integration.rs (same tests already exist in workspace/mod.rs versioning_tests module) [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: eliminate flaky hygiene tests caused by global AtomicBool contention All hygiene tests that used run_if_due() were flaky when running concurrently because they competed for the global RUNNING AtomicBool guard. Rewrote them to test the underlying components directly: - metadata_driven_cleanup_discovers_directories: now uses find_config_documents() + cleanup_directory() directly - multiple_directories_with_different_retention: now uses cleanup_directory() per directory directly - cleanup_respects_cadence: rewritten as a sync unit test that validates state file + timestamp logic without touching the global guard Verified stable across 3 consecutive runs (3793 tests, 0 failures). [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review comments — metadata ordering, PG locking, hygiene safety 1. Metadata applied BEFORE write/patch (#10-11,15): metadata param is now set via get_or_create + update_metadata before the write/patch call, so skip_indexing/skip_versioning take effect for the same operation instead of only subsequent ones. 2. Layer write doc ID (#13-14): metadata no longer re-reads after write since it's applied upfront. Removes the stale-scope risk. 3. Version param overflow (#16): validates version is 1..i32::MAX before casting, returns InvalidParameters on out-of-range. 4. Hygiene protection list (#18): added HYGIENE_PROTECTED_PATHS that includes MEMORY.md, HEARTBEAT.md, README.md (missing from IDENTITY_PATHS). cleanup_directory now uses is_protected_document() which checks both lists with case-insensitive matching. 5. PG FOR UPDATE on empty table (#22-24): now locks the parent memory_documents row (SELECT 1 FROM memory_documents WHERE id=$1 FOR UPDATE) before computing MAX(version), which works even when no version rows exist yet. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review comments — metadata merge, retention guard, migration ordering 1. **Metadata merge in memory_write tool**: incoming metadata is now merged with existing document metadata via `DocumentMetadata::merge()` instead of full replacement, so setting `{hygiene: {enabled: true}}` no longer silently drops a previously-set `skip_versioning: true`. 2. **Minimum retention_days**: `HygieneMetadata.retention_days` is now clamped to a minimum of 1 day during deserialization, preventing an LLM from writing `retention_days: 0` and causing mass-deletion on the next hygiene pass. 3. **Migration version ordering**: renumbered document_versions migration to come after staging's already-deployed migrations (PG: V15→V16, libSQL: 15→17). Documented the convention that new migrations must always be numbered after the highest version on staging/main. 4. **Duplicate doc comment**: removed duplicated line on `reindex_document_with_metadata`. 5. **HygieneSettings**: added `version_keep_count` field to persist the setting through the DB-first config resolution chain. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: skip metadata pre-apply when layer is specified, clean up stale comment 1. When a layer is specified, skip the metadata pre-apply via get_or_create — it operates on the primary scope and would create a ghost document there while the actual content write targets the layer's scope. 2. Removed stale "See review comments #10-11,15" reference; the surrounding comment already explains the rationale. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use BEGIN IMMEDIATE for libSQL save_version to serialize writers The default DEFERRED transaction only acquires a write lock at the first write statement (INSERT), not at the SELECT. Two concurrent writers could both read the same MAX(version) before either inserts, causing a UNIQUE violation. BEGIN IMMEDIATE acquires the write lock upfront, matching the existing pattern in conversations.rs. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: document trust boundary on metadata/versioning WorkspaceStore methods These methods accept bare document UUIDs without user_id checks at the DB layer. The Workspace struct (the only caller) always obtains UUIDs through user-scoped queries first. Document this trust boundary explicitly on the trait so future implementors/callers know not to pass unverified UUIDs from external input. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address new Copilot review comments — ghost doc, param validation, overflow 1. Skip metadata pre-apply in patch mode to avoid creating a ghost empty document via get_or_create when the document doesn't exist, which would change a "not found" error into "old_string not found". 2. Validate list_versions and version as mutually exclusive in memory_read to avoid ambiguous behavior (list_versions silently won). 3. Clamp version_keep_count to i32::MAX before casting to prevent overflow on extreme config values. 4. Mark daily_retention_days and conversation_retention_days as deprecated in HygieneSettings — retention is now per-folder via .config metadata. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: apply metadata in patch mode via read(), reorder libSQL migrations 1. Metadata is no longer silently ignored in patch mode — uses workspace.read() (which won't create ghost docs) instead of skipping entirely, so skip_versioning/skip_indexing flags take effect for patches on existing documents. 2. Reorder INCREMENTAL_MIGRATIONS to strictly ascending version order (16 before 17) to match iteration order in run_incremental(). [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove duplicate is_patch_mode, add TODO comments for known limitations - Remove duplicate `is_patch_mode` binding in memory_write (was computed at line 280 and again at line 368). - Document multi-scope hygiene edge case: workspace.list() includes secondary scopes but workspace.delete() is primary-only, causing silent no-ops for cross-scope entries. - Document O(n) reads in version pruning as acceptable for typical directory sizes. - Add TODO on WorkspaceError::SearchFailed catch-all for future cleanup into more specific variants. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: reindex on no-op writes for metadata changes, use read_primary in patch 1. write() no longer fully short-circuits when content is unchanged — it still resolves metadata and reindexes so that metadata-driven flags (e.g. skip_indexing toggled via memory_write's metadata param) take effect immediately even without a content change. 2. Patch-mode metadata pre-apply now uses workspace.read_primary() instead of workspace.read() to ensure we target the same scope that patch() operates on, preventing cross-scope metadata mutation in multi-scope mode. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…compatible coding agent (#1600) * feat: add ACP (Agent Client Protocol) job mode for delegating to any compatible coding agent Add a third container job mode (`JobMode::Acp`) that spawns any ACP-compliant agent (Goose, Codex, Gemini CLI, Cline, Copilot, etc.) as a subprocess inside a Docker container and communicates via the standard ACP protocol (JSON-RPC over stdio). **Bridge runtime** (`src/worker/acp_bridge.rs`): - Spawns agent subprocess, performs ACP handshake (initialize → session → prompt) - Translates ACP SessionNotification events to IronClaw's JobEventPayload stream - Auto-approves permissions (Docker container is the security boundary) - Supports follow-up prompts from the orchestrator - Detects agent process exit via oneshot channel to prevent infinite polling **User configuration** (mirrors MCP server pattern): - `ironclaw acp add/list/remove/toggle/test` CLI commands - DB-backed persistence with `~/.ironclaw/acp-agents.json` disk fallback - Per-agent `enabled` flag + global `ACP_ENABLED` toggle - `ironclaw acp test` spawns agent, verifies ACP handshake, reports capabilities **System integration**: - `ExtensionKind::AcpAgent` in extension manager (12 match arms) - `agent_name` parameter in CreateJobTool resolves agent from AcpAgentsFile - Mode stored as `"acp:<agent_name>"` for restart support - Doctor validation, status display, boot screen, app startup logging - Web UI: extension install mapping, job restart, follow-up prompt support Closes #1506 * test: add comprehensive ACP test coverage (22 new tests) Bridge: ToolCall, ToolCallUpdate, thought-image, max_turn_requests, session_id propagation, text_from_content_block, multibyte truncation. Config: AcpModeConfig defaults, settings resolution, env overrides. Job tool: schema includes "acp" mode + agent_name, mode="acp" requires agent_name parameter, JobMode::Acp as_str/display. Job manager: JobMode::Acp as_str/display, acp_memory_limit_mb default. CLI: parse_env_var valid/invalid/equals-in-value, command variants. * refactor: make IronClawAcpClient reusable for CLI test command Extract AcpEventSink trait so the same Client implementation (permission auto-approval, event translation) is shared between the container bridge (posts to orchestrator HTTP API) and the CLI test command (prints to stdout). Also extracts ironclaw_init_request() to avoid duplicating the ACP handshake parameters between bridge and test command. * fix(sandbox): use host.docker.internal on all platforms for orchestrator URL The orchestrator host was hardcoded to 172.17.0.1 on Linux, which is only correct for the default Docker bridge network. Environments with custom bridge IPs break container-to-host connectivity. Since all containers already set extra_hosts with host-gateway, using host.docker.internal works on all platforms and network configurations. * fix ACP PR review feedback * fix ACP DB error fallback * fix clippy after staging merge --------- Co-authored-by: Rajul Bhatnagar <brajul@amazon.com> Co-authored-by: Firat Sertgoz <f@nuff.tech>
…1471) * fix(routines): add bounded retry for transient lightweight execution failures (#1320) Lightweight routine executions that fail with transient errors (LLM failures, empty responses, truncated responses) are now retried up to 3 times with exponential backoff (1s, 2s, 4s) before reporting failure. Full-job routines are not retried since the scheduler/watcher handles their lifecycle. Hard failures (disabled, not found, auth, DB errors) fail immediately without retry. Adds RoutineError::is_retryable() to classify transient vs hard errors, with comprehensive regression tests covering all error variants. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: run cargo fmt to fix formatting https://claude.ai/code/session_01Nv2TJ3so5WQqpRUcirAhT3 * fix(deps): resolve RUSTSEC-2026-0049 rustls-webpki CRL advisory Update rustls-webpki 0.103.9 -> 0.103.10 and ignore the advisory for 0.102.8 which is pinned by libsql's rustls 0.22.4 dependency chain. https://claude.ai/code/session_01MmxvBgAMn4m45pZguFKBEX * fix(routines): classify permanent LLM failures, accumulate tokens, warn on tool retry (#1320) Address three review comments on the bounded-retry implementation: 1. Permanent LLM failures no longer retried: `is_retryable()` now inspects the `LlmFailed` reason string for known permanent patterns (auth, content policy, context length, model not available, moderation). These map to `LlmError` variants already classified as non-retryable by the LLM retry layer. 2. Token usage accumulated across retries: `RoutineError::LlmFailed` now carries an optional `partial_tokens` field populated from tokens consumed before the failure. The retry loop sums partial tokens from failed attempts with the final successful attempt's tokens. 3. Tool loop retry limitation documented: added a code comment explaining that the retry wraps the entire `execute_lightweight()` call, and a warning log when retrying a tools-enabled routine so operators know side effects may be repeated. [skip-regression-check] Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use structured tracing for transient routine retry logging [skip-regression-check] Replace tracing::warn! with tracing::event! targeting "transient_routine_errors" for better log filtering and structured field capture on retry attempts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(routines): address PR review feedback for bounded retry (#1320) - Skip retry for tools-enabled routines to prevent duplicate side effects - Replace fragile PERMANENT_LLM_PATTERNS substring matching with a retryable bool field on RoutineError::LlmFailed, set at the LlmError conversion site using llm::retry::is_retryable() - Use saturating_add for token accumulation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix rustfmt formatting in routine_engine.rs https://claude.ai/code/session_01CsP5wMZ2evEMHgGghjAfR1 * fix(routines): address review feedback on retry loop - Remove outer retry for LlmFailed errors since RetryProvider already handles transient LLM failures with its own bounded budget, preventing multiplicative retry counts - Preserve Option<i32> semantics for token accumulation: None means "unknown/not tracked" rather than converting to Some(0) via unwrap_or(0), so downstream API/UI correctly distinguishes null from zero - Add partial_tokens field to EmptyResponse and TruncatedResponse variants so token usage from those failed attempts is captured in the retry accumulator - Persist accumulated token total on final failure path so usage from earlier retry attempts is not silently discarded Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(clippy): replace find().is_none() with !any() in user stats test https://claude.ai/code/session_01Nm95eCjdrwxDjwkHTRieZs * fix(routines): address code review on retry budget, token semantics, and accounting - Reduce MAX_RETRIES from 3 to 2 and document multiplicative retry budget (2 outer x 3 inner RetryProvider = 6 worst-case LLM calls per routine) - Preserve None token semantics: extract tokens_to_option() helper that returns None when both input/output are zero (unknown), avoiding the conversion of "not tracked" into "zero" in downstream reporting - Use tokens_to_option consistently in handle_text_response and tool-path error handlers for uniform token accounting - Add comment explaining why partial_tokens is None in no-tools LlmFailed path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… NEAR wallet (#1798) * feat(auth): add direct OAuth/social login with Google and GitHub (#1771) Add optional OAuth authentication so users can sign in directly via Google or GitHub without requiring admin-created tokens or a reverse-proxy SSO setup. On successful OAuth, the system creates or links a user via the existing UserStore, issues an API token, and sets it as an HttpOnly cookie — reusing the existing DbAuthenticator for subsequent requests. Key changes: - user_identities table (PostgreSQL V15 + libSQL migration) for linking external provider accounts to internal users - IdentityStore trait with dual-backend implementations - OAuthProvider trait with Google (OIDC id_token) and GitHub (API-based) provider implementations - In-memory CSRF + PKCE state store with TTL and capacity bounds - Cookie-based session extraction in auth middleware - User resolution: existing identity → email linking → new account creation - All behind OAUTH_ENABLED=true flag; existing auth paths unchanged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add email domain restrictions for OAuth and OIDC login (#1771) Add configurable email domain restrictions so admins can limit OAuth and OIDC login to specific organizations: - OAUTH_ALLOWED_DOMAINS: comma-separated list of allowed email domains, applied to all OAuth providers and OIDC (e.g., company.com,partner.org) - GOOGLE_ALLOWED_HD: restrict Google login to a specific Workspace domain via the `hd` authorization parameter + server-side validation - Domain check enforced in both the OAuth callback handler and the OIDC JWT middleware path (extracts email claim from validated JWT) - Add setup documentation in .env.example with step-by-step instructions for configuring Google and GitHub OAuth credentials Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR review — security hardening and cleanup (#1798) Fixes from Gemini and Copilot review: Security (critical/high): - Validate `aud` claim in Google id_token to prevent token substitution - Sanitize `redirect_after` to relative paths only (prevent open redirects) Correctness (medium): - Remove orphaned token: replace `create_user_with_identity_and_token` with `create_user_with_identity` — single token created in callback handler - Logout now revokes the API token (not just clears cookie) - Session tokens expire after 30 days (matching cookie lifetime) - Store decoded Google claims in raw_profile (not JWT string) - Propagate GitHub email fetch errors instead of swallowing - Fix `list_identities_for_user` to propagate row iteration errors Cleanup (low): - Extract `SESSION_COOKIE_NAME` constant - Add Secure flag to logout cookie clearing - Add single-quote escaping in error_page HTML - Fix garbled unicode in auth.rs comment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add Apple Sign In provider (#1807) Add Apple Sign In as an OAuth provider alongside Google and GitHub. Apple-specific handling: - JWT client_secret generation (ES256-signed, team_id/key_id/private_key) - response_mode=form_post — Apple POSTs the callback instead of GET - POST callback route added alongside existing GET route - User name extracted from Apple's `user` form field (sent only on first authorization) and merged into the profile - id_token decoded with aud + issuer validation - email_verified handles both boolean and string "true"/"false" formats Configuration: - APPLE_CLIENT_ID, APPLE_TEAM_ID, APPLE_KEY_ID - APPLE_PRIVATE_KEY_PATH (file) or APPLE_PRIVATE_KEY_PEM (inline) - Setup instructions added to .env.example Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add NEAR wallet login via NEP-413 signature verification (#1807) Add NEAR wallet authentication as a fourth login method alongside Google, GitHub, and Apple. Unlike OAuth, NEAR uses a challenge-response flow with Ed25519 signature verification. Backend: - GET /auth/near/challenge — generate a random nonce (32 bytes hex) - POST /auth/near/verify — verify Ed25519 signature + NEAR RPC access key check, then issue session token via existing user resolution pipeline - NearNonceStore: in-memory nonce store with 5-min TTL and replay protection - Supports both base58 (NEAR standard) and hex key/signature encoding - New dependency: bs58 0.5 for base58 decoding Frontend: - Login screen discovers enabled providers via GET /auth/providers - Shows social login buttons (Google, GitHub, Apple, NEAR) dynamically - NEAR button loads @hot-labs/near-connect via ESM CDN import - Wallet connection → signMessage → POST to /auth/near/verify → session - OAuth cookie-based sessions auto-detected on page load (existing flow) Configuration: - NEAR_AUTH_ENABLED=true - NEAR_AUTH_NETWORK=mainnet|testnet (defaults to mainnet) - NEAR_AUTH_RPC_URL (auto-detected from network) Closes #1807 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address second round of PR review comments (#1798) - Fix early return in with_oauth() that skipped NEAR setup and OIDC domain restrictions when no OAuth redirect providers were configured - Add active-status check before linking identity by verified email (prevents linking to suspended/deactivated accounts) - Remove inline onclick handlers from login buttons (CSP compliance) - Export SESSION_COOKIE_NAME from auth.rs, reuse in handlers and middleware - NEAR challenge returns structured message ("Sign in to IronClaw\nNonce: {nonce}") that both client and server use for signature verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address human reviewer security findings (#1798) Six fixes from serrrfirat's review: 1. Domain check now requires email_verified=true before trusting the email for domain restriction — prevents unverified emails from bypassing access control (e.g., GitHub unverified fallback) 2. First-user bootstrap race documented — concurrent first logins may both see has_any_users()=false, but the second gets member role. Acceptable tradeoff; unique constraint prevents identity duplication. 3. NEP-413 payload mismatch fixed — server now builds the exact borsh-serialized NEP-413 payload (tag + message + nonce + recipient) that the wallet signs, instead of raw message bytes 4. Token extraction priority fixed — explicit ?token= query param now takes precedence over session cookie, preventing SSE/WS user mismatch when a browser has both a cookie and a query-param token 5. NEAR domain suffix check hardened — requires exact match or dotted subdomain boundary (alice.company.near passes, evilcompany.near does not) 6. NEAR network surfaced to frontend — /auth/providers response includes near_network field, frontend wallet connector uses it instead of hardcoded mainnet Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): improve login page UX when OAuth providers are enabled When OAUTH_ENABLED=true with providers configured, the login screen now shows social login buttons (Google, GitHub, Apple, NEAR) as the primary action. The token input is collapsed behind a clickable "or use a token" divider for API users. Without OAuth: unchanged — token input is the only option. With OAuth: social buttons first, token input expandable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): hide token form until providers are discovered The token input was visible by default, causing it to flash before OAuth buttons appeared. Now: - Token form starts hidden (display:none) - /auth/providers fetch determines what to show - With providers: social buttons shown, token form behind "or use a token" - Without providers (or fetch fails): token form shown as fallback - After OAuth redirect: cookie-based autoAuth skips login screen entirely Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): replace Connected indicator with user avatar + account menu Replace the "Connected" status indicator in the header with a user avatar button that shows connection status via an overlay dot. Clicking the avatar opens a dropdown with: - Display name, email, and role - Connection status (green/red dot + text) with gateway stats - Sign out button (calls POST /auth/logout, clears session, reloads) Avatar source: - OAuth logins: profile photo from Google/GitHub/Apple (avatar_url) - Token logins: initials from display_name (colored circle) Backend: profile_get_handler now queries user_identities for avatar_url from linked OAuth accounts. Frontend: social buttons are primary when OAuth is enabled, token input collapsed behind "or use a token" divider. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): remove Connected section from dropdown, fix avatar loading - Remove the connection status section from the user dropdown (was redundant with the avatar dot) - Add update_identity_profile() to IdentityStore — updates display_name and avatar_url on re-login so avatars load for accounts created before the avatar field was wired - Call update_identity_profile() in resolve_user() when an existing identity is found (re-login path) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): restore gateway stats in dropdown, add avatar debug logging - Bring back gateway stats section in user dropdown (without the word "Connected" — just the server stats like uptime, model, channels) - Add debug tracing to Google provider (logs picture claim from id_token) and profile handler (logs identity count + avatar_url) to diagnose why avatar isn't loading for Google OAuth accounts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix Google avatar not loading, restore gateway stats - Add referrerpolicy="no-referrer" to avatar img — Google's lh3.googleusercontent.com returns 403 when Referer header is sent from a different origin - Add crossorigin="anonymous" for CORS - Use display:block explicitly instead of empty string - Add onerror fallback to initials if image fails to load - Restore gateway stats section in dropdown (without "Connected" text) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix squeezed avatar image in header Add min-width/min-height and flex-shrink:0 to both the avatar button and the img element so they don't get compressed by the tab-bar flex layout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): position avatar img and initials absolutely inside button Both children were competing for flex space, causing 0px width. Now both are position:absolute inside the 32px button, layered on top of each other. The JS toggles display:block/none to show the right one. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): rewrite avatar loading — CSS src selector + onload/onerror Previous approach: inline style display:none toggled by JS. Failed because display:none prevented image fetch in some browsers, and position:absolute elements competed for z-index. New approach: - No inline style on img — CSS hides it via .user-avatar-img (display:none) - CSS .user-avatar-img[src] shows it (display:block, z-index:1) - JS sets src, onload hides initials, onerror removes src as fallback - Initials always rendered first as the base layer - Removed crossorigin="anonymous" which can cause CORS failures Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): prefetch avatar with new Image() before showing Use a throwaway Image() to prefetch the avatar URL. Only when onload fires, set src on the real <img> and unhide it. This avoids all CSS display/src selector issues — the real img element only gets a src after the image is confirmed loadable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): set avatar src directly, set referrerPolicy in JS The new Image() prefetch was failing because the programmatic Image object didn't have referrerPolicy set. Simplify: set referrerPolicy and src directly on the real <img> element, unhide it immediately, and use onload to hide initials. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): explicit display:block on avatar img, removeAttribute hidden - Add display:block to .user-avatar-img CSS (img elements default to inline which can cause rendering issues with position:absolute) - Bump z-index to 2 to ensure img renders above initials - Use removeAttribute('hidden') instead of hidden=false - Use style.display='none' on initials instead of hidden attribute Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): swap img/initials DOM order so img paints on top Put <img> after <span> in DOM order. With both position:absolute, later elements paint on top. Combined with z-index:2 on img vs z-index:0 on initials, the avatar photo should now reliably cover the initials circle when loaded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): add OAuth avatar domains to Content-Security-Policy The CSP had img-src 'self' data: which blocked Google and GitHub avatar images from loading. Added: - img-src: *.googleusercontent.com, avatars.githubusercontent.com - script-src: esm.sh (for near-connect dynamic import) - connect-src: esm.sh, *.near.org (for NEAR RPC) - form-action: Google, GitHub, Apple OAuth endpoints This was the root cause of avatar images not rendering despite correct src URLs — the browser silently blocked them via CSP. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): show welcome card for new OAuth users with no threads New OAuth users have no assistant thread yet, so switchToAssistant() was never called, and loadHistory() never ran to show the welcome card. Now explicitly show the welcome card when there's no current thread and no assistant thread (brand-new user). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): persist bootstrap greeting for new OAuth users on workspace creation New OAuth users saw an empty chat because the bootstrap greeting was only persisted when the agent loop processed the first message (take_bootstrap_pending check in agent_loop.rs:1314). But OAuth users land on the web UI without sending any message. Fix: WorkspacePool now checks take_bootstrap_pending() after seed_if_empty() and persists the GREETING.md content into the assistant conversation immediately. This runs in a background task so it doesn't block the workspace creation. The greeting is in the DB before the frontend loads threads/history. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): persist bootstrap greeting synchronously, not in background Move greeting persistence from tokio::spawn to the same await chain as seed_if_empty() so the greeting is guaranteed to be in the DB before the workspace is returned to the caller. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): seed bootstrap greeting in chat_threads_handler for new users The WorkspacePool approach didn't work because the workspace pool is only accessed by memory handlers — chat_threads_handler runs first when a new user loads the page. Move the greeting seed to chat_threads_handler: after get_or_create_assistant_conversation, check if the conversation has zero messages and inject the GREETING.md content. This guarantees the greeting is in the DB before the thread list is returned to the frontend. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(agent): consolidate bootstrap greeting into chat_threads_handler Remove three redundant greeting insertion paths from agent_loop.rs: 1. Single-user startup (Agent::run bootstrap_thread_id) 2. Single-user SSE broadcast after startup 3. Multi-tenant message handler (take_bootstrap_pending on first msg) Also remove the dead WorkspacePool greeting code in server.rs. The single source of truth is now chat_threads_handler: when the assistant conversation is created with zero messages, GREETING.md is inserted. This works for all auth modes (token, OAuth, OIDC) and both single-user and multi-tenant deployments. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: downgrade workspace seed log from info to debug Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address zmanian's blocking review items 1. Add Google iss validation — set_issuer(&["https://accounts.google.com"]) to match Apple's issuer check. Prevents cross-provider id_token acceptance. 2. Convert oauth_rate_limiter from global RateLimiter to per-IP PerUserRateLimiter(20, 60). Extracts client IP from X-Forwarded-For header. One user retrying no longer locks out all OAuth for everyone. Also: - sanitize_redirect now rejects backslash (/\) open redirect vector - All auth handlers extract headers for per-IP rate limiting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): stop inserting greeting on every page load The previous check used list_conversations_with_preview with limit=1 and defaulted to is_empty=true when the assistant thread wasn't in the result (unwrap_or(true)). This caused the greeting to be inserted on every chat_threads_handler call. Fix: use list_conversation_messages_paginated(assistant_id, None, 1) to directly check if the assistant conversation has any messages. Only insert the greeting when the message list is truly empty. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add integration tests for bootstrap greeting and cookie auth Five tests covering the greeting behavior and OAuth session auth: 1. test_greeting_inserted_once_for_new_user — verifies greeting appears exactly once and is not duplicated on second page load 2. test_greeting_not_duplicated_on_rapid_calls — 5 concurrent /api/chat/threads requests produce exactly 1 greeting 3. test_each_user_gets_own_greeting — multi-user: Alice and Bob each get their own assistant thread with separate greetings 4. test_cookie_auth_works_for_threads — cookie-based auth (ironclaw_session=token) works for protected endpoints 5. test_existing_conversation_no_greeting — pre-populated conversations are not overwritten with the greeting These tests would have caught the unwrap_or(true) bug that caused greeting re-insertion on every page load. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): fix near-connect CDN URL (package is v0.x, not v1) The @hot-labs/near-connect package is version 0.11.1 — there is no v1 release. The @1 version specifier returned 404 from esm.sh. Changed to @0.11 which resolves correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): support base64 encoding for NEAR wallet signatures NEAR wallets (e.g. HOT) may return signatures and public keys in base64 format, not just base58/hex. Added base64 standard and URL-safe decoding to decode_multiformat(), which is used by both decode_near_public_key and decode_near_signature. Also: - Added debug logging to near_verify_handler to trace credential formats - Updated CSP img-src to allow wallet logos (raw.githubusercontent.com, jsdelivr.net, near.org, pages.near.org) - Added CSP frame-src for near-connect wallet sandboxes (iframes) - Added blob: to img-src for inline wallet icons Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): widen CSP connect-src and img-src for NEAR wallet resources near-connect fetches wallet manifests from raw.githubusercontent.com and cdn.jsdelivr.net, and wallet logos from app.hot-labs.org. These were blocked by the restrictive connect-src and img-src policies. - connect-src: added raw.githubusercontent.com, *.jsdelivr.net, *.cloudflare.com - img-src: added *.hot-labs.org Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): try both NEP-413 and raw message for NEAR signature verification Different NEAR wallets may sign the full NEP-413 borsh payload or just the raw message string. Try NEP-413 first, fall back to raw message bytes. This makes verification work with HOT wallet and other wallets that may not implement the full NEP-413 serialization. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): fix NEP-413 field order and try both payload layouts The NEP-413 borsh payload field order was wrong. Our implementation had tag → message → nonce → recipient → callback_url, but the NEAR docs (docs.near.org/web3-apps/backend-login) show tag → message → recipient → nonce. Now tries both field orderings (v1 and v2), plus SHA256 variants, plus raw message bytes — covering all known wallet implementations. Tests updated: verify_near_signature tested with raw message, NEP-413 v2, and wrong-key rejection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): relax CSP for wallet ecosystem — allow all HTTPS for connect/img/frame The NEAR wallet ecosystem spans dozens of domains (intear.tech, hot-labs.org, meteorwallet.app, herewallet.app, etc.) that change as new wallets are added. Whitelisting each one is a losing game. Relax CSP to allow all HTTPS for: - connect-src: wallet manifests, wallet JS modules, RPC endpoints - img-src: wallet logos from various CDNs - frame-src: wallet sandbox iframes script-src remains restricted to specific CDNs (jsdelivr, cloudflare, esm.sh) — this is the security-critical directive. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): allow unsafe-inline scripts for NEAR wallet sandbox iframes NEAR wallet sandboxes (MeteorWallet, etc.) use inline scripts inside their iframe sandboxes. The CSP script-src blocked these, preventing wallets from loading. Added 'unsafe-inline' to script-src. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): relax CSP style-src and font-src for wallet iframes NEAR wallet sandboxes load fonts from rsms.me, cdnfonts.com, and embed data: font URIs. Relaxed style-src and font-src to allow all HTTPS sources and data: URIs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address review round 4 — 14 fixes serrrfirat (high/medium): 1. decode_multiformat ambiguity: replaced with context-aware decoders. NEAR pubkeys enforce ed25519: prefix + base58 (unambiguous). Signatures try base64 first (most wallets), then base58. 2. UTF-8 slicing panic: use safe_truncate() with char_indices() 3. NEAR pubkey → RPC format: re-encode decoded bytes as ed25519:{base58} for the canonical format expected by view_access_key 4. GitHub redirect_uri: now included in token exchange form body Copilot (medium/low): 5. near_network stored explicitly in GatewayState (not inferred from URL) 6. NEAR verify sets HttpOnly session cookie (consistent with OAuth flow) 7. Reuse reqwest::Client via LazyLock (no per-request allocation) 8. OAuth module doc updated to list all 4 providers 9. Rate limiter comment fixed (was stale "10 requests") 10. Profile identity error logged at warn (not silently swallowed) 11. Config doc updated for Apple/NEAR requirements 12. Test file doc comment updated to match actual coverage 13. RPC status check before JSON parse 14. GitHub token exchange includes redirect_uri Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR feedback on cookie auth and NEAR sessions * fix(auth): address code review — tighten CSP, fix races, improve security - Tighten CSP: narrow connect-src/img-src/frame-src to specific origins instead of blanket `https:` (prevents data exfiltration) - Fix greeting race: add atomic add_conversation_message_if_empty using INSERT...WHERE NOT EXISTS (both PostgreSQL and libSQL) - Case-insensitive email matching: use LOWER() in identity lookups and normalize emails to lowercase on storage - Add X-Real-IP fallback for rate limit key when X-Forwarded-For missing - Add OAuthError::SignatureVerification variant (was misusing ProfileFetch) - Fix dead branch in with_oauth (has_near check inside !has_near block) - Fix _user → user in logout_handler (variable is actually used) - Add partial index WHERE email IS NOT NULL to libSQL (match PostgreSQL) - Downgrade noisy tracing::debug to trace in profile handler - Add i18n for "Sign out" button (en + zh-CN) - Remove duplicate test_session_cookie_auth_passes test - Update E2E bootstrap tests to match new DB-based greeting architecture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): remove garbled unicode character in section comment Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address review round 5 — admin race, 303 redirect, OIDC email_verified - Atomic first-user admin: create_user_with_identity now promotes to admin inside the DB transaction with UPDATE...WHERE COUNT(*)=1, eliminating the TOCTOU race where two concurrent first logins both get admin role (both PostgreSQL and libSQL) - Apple callback redirect: use 303 See Other instead of 307 Temporary so POST form_post callbacks are converted to GET on redirect - OIDC domain restriction: now requires email_verified=true before checking domain allowlist, preventing unverified emails from bypassing the restriction - Postgres add_conversation_message_if_empty: call touch_conversation after insert to match libSQL behavior and keep last_activity current - Greeting seeding: log errors instead of silently discarding with let _ Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): advisory lock for admin election, skip empty query tokens - Postgres first-user admin: add pg_advisory_xact_lock before the COUNT(*)=1 promotion to serialize concurrent transactions under READ COMMITTED isolation (prevents two admins on concurrent signup) - Empty ?token= query parameter no longer overrides a valid session cookie — trimmed empty tokens return None from query_token() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address zmanian security review — redirect, sweep, NEAR sigs Blocking issues from security review: 1. redirect_after hardened: strict URL-safe char allowlist in sanitize_redirect (blocks /%09/ and encoded separators), plus re-validation before use in handle_callback (defense in depth) 2. Sweep tasks shutdown-aware: OAuth state store and NEAR nonce store sweep loops now select on a watch channel and exit when the sender is dropped (stored in GatewayState.oauth_sweep_shutdown) 3. NEAR signature verification tightened: removed raw-message-bytes and SHA256-of-raw fallbacks that lacked nonce binding (replay risk). Only NEP-413 structured payloads (v1 + v2) are accepted. Added test_verify_near_signature_rejects_raw_message regression test. Non-blocking: 4. NEAR RPC client timeout: set 10s timeout on the static reqwest client to prevent indefinite hangs on slow RPC endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): percent-decode redirect_after before validation is_safe_redirect now percent-decodes the URL and re-validates against the // and /\ guards, preventing smuggling via %2f%2f or %5c. Added 5 regression tests covering normal paths, protocol-relative, absolute URLs, encoded smuggling, and sanitize_redirect filtering. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): case-insensitive get_user_by_email, require OIDC iss/aud claims - get_user_by_email now uses LOWER() in both PostgreSQL and libSQL, matching the case-insensitive identity lookup. This ensures admin-created users with different email casing are correctly linked during OAuth account resolution. - OIDC validation now adds iss/aud to required_spec_claims when configured, rejecting JWTs that omit these claims entirely (not just mismatches). Updated two tests from assert-passes to assert-rejects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): normalize UserRecord.email to lowercase, add aria-label to avatar - UserRecord.email now lowercased on create (matching identity records), preventing case-mismatched duplicates against the UNIQUE constraint - Avatar button: added aria-label with i18n (en + zh-CN) for screen readers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Firat Sertgoz <f@nuff.tech>
* fix(db): resolve V15 migration numbering conflict between user_identities and conversation_source_channel A merge conflict left two PostgreSQL migrations at V15. This renumbers them (V15=user_identities, V16=conversation_source_channel, V17=document_versions) to match the libSQL incremental ordering. Adds user_identities, document_versions, and source_channel to the libSQL base schema so fresh databases get all tables. Includes a one-time repair for existing databases where V15 was mis-recorded. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * style: fix rustfmt formatting in repair_misnumbered_v15 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(db): address PR review — proper error handling and tighter repair condition - Replace .ok().flatten() with explicit error propagation via .map_err()? so DB errors during V15 repair are surfaced, not silently swallowed - Tighten repair condition from `!= "user_identities"` to `== "document_versions"` to only fix the specific known-bad case from the merge conflict Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…1928) Production PostgreSQL already has V15__conversation_source_channel applied. Renumber to match: V15=conversation_source_channel, V16=user_identities, V17=document_versions. Update libSQL incremental migrations and idempotent list to match. The V15 repair still handles databases that mis-recorded V15 as "document_versions". Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…tch [skip-regression-check] (#1925)
…e user_identities) (#1931) Production PostgreSQL already has V15=conversation_source_channel and V16=document_versions applied. user_identities must be V17. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The musl-linked binary segfaults when reopening an existing libSQL database after container restart. The bundled SQLite C code in libsql-ffi has threading/mmap issues when statically linked against musl. Switching build and runtime to Debian/glibc resolves this. Tested on sysbox with multiple restart cycles — no segfault. [skip-regression-check]
…architecture) (#1557) * v2 architecture phase 1 * feat(engine): Phase 2 — execution loop, capability system, thread runtime Add the core execution engine to ironclaw_engine crate: - CapabilityRegistry: register/get/list capabilities and actions - LeaseManager: async lease lifecycle (grant, check, consume, revoke, expire) - PolicyEngine: deterministic effect-level allow/deny/approve - ThreadTree: parent-child relationship tracking - ThreadSignal/ThreadOutcome: inter-thread messaging via mpsc - ThreadManager: spawn threads as tokio tasks, stop, inject messages, join - ExecutionLoop: core loop replacing run_agentic_loop() with signals, context building, LLM calls, action execution, and event recording - Structured executor (Tier 0): lease lookup → policy check → effect execution - Tool intent nudge detection - MemoryStore + RetrievalEngine stubs for Phase 4 - Full 8-phase architecture plan in docs/plans/ - CLAUDE.md spec for the engine crate 74 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): Phase 3 — Monty Python executor with RLM pattern Add CodeAct execution (Tier 1) using the Monty embedded Python interpreter, following the Recursive Language Model (RLM) pattern from arXiv:2512.24601. Key additions: - executor/scripting.rs: Monty integration with FunctionCall-based tool dispatch, catch_unwind panic safety, resource limits (30s, 64MB, 1M allocs) - LlmResponse::Code variant + ExecutionTier::Scripting - Context-as-variables (RLM 3.4): thread messages, goal, step_number, previous_results injected as Python variables — LLM context stays lean while code accesses data selectively - llm_query(prompt, context) (RLM 3.5): recursive subagent calls from within Python code — results stored as variables, not injected into parent's attention window (symbolic composition) - Compact output metadata between code steps instead of full stdout - MontyObject ↔ serde_json::Value bidirectional conversion - Updated architecture plan with RLM design principles 74 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): RLM best-practices enhancements from cross-reference analysis Cross-referenced our implementation against the official RLM (alexzhang13/rlm), fast-rlm (avbiswas/fast-rlm), and Prime Intellect's verifiers implementation. Key enhancements: - FINAL(answer) / FINAL_VAR(name): explicit termination pattern matching all three reference implementations. Code can signal completion at any point, not just via return value. - llm_query_batched(prompts): parallel recursive sub-calls via tokio::spawn, matching fast-rlm's asyncio.gather pattern and Prime Intellect's llm_batch. - Output truncation increased to 8000 chars (from 120), matching Prime Intellect's 8192 default. Shows [TRUNCATED: last N chars] or [FULL OUTPUT]. - Step 0 orientation preamble: auto-injects context metadata (message count, total chars, goal, last user message preview) before first code step, matching fast-rlm's auto-print pattern. - Error-to-LLM flow: Python parse errors, runtime errors, NameErrors, OS errors, and async errors now flow back as stdout content instead of terminating the step, enabling LLM self-correction on next iteration. Only VM panics (catch_unwind) terminate as EngineError. 74 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): update architecture plan with RLM cross-reference learnings Comprehensive update after cross-referencing against official RLM (alexzhang13/rlm), fast-rlm (avbiswas/fast-rlm), Prime Intellect (verifiers/RLMEnv), rlm-rs (zircote/rlm-rs), and Google ADK RLM. Changes: - Mark Phases 1-3 as DONE with commit refs and test counts - Add "Key Influences" section documenting all reference implementations - Phase 3: full table of implemented RLM features with sources - Phase 3: "Remaining gaps" table with which phase addresses each - Phase 4: expanded with compaction (85% context), rlm_query() (full recursive sub-agent), dual model routing, budget controls (USD, timeout, tokens, consecutive errors), lazy loading, pass-by-reference - Add "RLM Execution Model" cross-cutting section - Add "Implementation Progress" tracking table - Remove stale "TO IMPLEMENT" markers (all Phase 3 work is done) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): Phase 4 — budget controls, compaction, reflection pipeline Budget enforcement in ExecutionLoop: - max_tokens_total: cumulative token limit, checked before each iteration - max_duration: wall-clock timeout for entire thread - max_consecutive_errors: consecutive error steps threshold (resets on success, matching official RLM behavior) - All produce ThreadOutcome::Failed with descriptive messages Context compaction (from RLM paper, 85% threshold): - estimate_tokens(): char-based estimation (chars/4, matching RLM) - should_compact(): triggers when tokens >= threshold_pct * context_limit - compact_messages(): asks LLM to summarize progress, replaces history with [system, summary, continuation_note], preserves intermediate results - Configurable via ThreadConfig: model_context_limit, compaction_threshold Dual model routing: - LlmCallConfig gains depth field (0=root, 1+=sub-call) - Implementations can route to cheaper models for sub-calls - ExecutionLoop passes thread depth to every LLM call Reflection pipeline (reflection/pipeline.rs): - reflect(thread, llm): analyzes completed thread via LLM - Produces Summary doc (always), Lesson doc (if errors), Issue doc (if failed) - Builds transcript from thread messages + error events - Returns ReflectionResult with docs + token usage ThreadConfig extended with: max_tokens_total, max_consecutive_errors, model_context_limit, enable_compaction, compaction_threshold, depth, max_depth. 78 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): Phase 5 — conversation surface separated from execution Conversation is now a UI layer, not an execution boundary. Multiple threads can run concurrently within one conversation; threads can outlive their originating conversation. New types (types/conversation.rs): - ConversationSurface: channel + user + entries + active_threads - ConversationEntry: sender (User/Agent/System) + content + origin_thread_id - ConversationId, EntryId (UUID newtypes) - EntrySender enum (User, Agent{thread_id}, System) ConversationManager (runtime/conversation.rs): - get_or_create_conversation(channel, user) — indexed by (channel, user) - handle_user_message() — injects into active foreground thread or spawns new - record_thread_outcome() — adds agent/system entries, untracks completed threads - get_conversation(), list_conversations() This enables the key architectural insight: a user can ask "what's the weather?" while a deployment thread is still running. Both produce entries in the same conversation. 85 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): simplify execution tiers — Monty-only for CodeAct/RLM Restructure phases 6-8 to clarify execution model: - Monty is the sole Python executor for CodeAct/RLM. No WASM or Docker Python runtimes for LLM-generated code. - WASM sandbox is for third-party tool isolation (existing infra, Phase 8) - Docker containers are for thread-level isolation of high-risk work (Phase 8) - Two-phase commit moves to Phase 6 (integration) at the adapter boundary Phase renumbering: - Old Phase 6 (Tier 2-3) → removed as separate phase - Old Phase 7 (integration) → Phase 6 - Old Phase 8 (cleanup) → Phase 7 - New Phase 8: WASM tools + Docker thread isolation (infra integration) Updated progress table: Phases 1-5 marked DONE with test counts and commits. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): Phase 6 — bridge adapters for main crate integration Strategy C parallel deployment: when ENGINE_V2=true env var is set, user messages route through the engine instead of the existing agentic loop. All existing behavior is unchanged when the flag is off. Bridge module (src/bridge/): - LlmBridgeAdapter: wraps LlmProvider as engine LlmBackend, converts ThreadMessage↔ChatMessage, ActionDef↔ToolDefinition, depth-based model routing (primary vs cheap_llm) - EffectBridgeAdapter: wraps ToolRegistry+SafetyLayer as EffectExecutor, routes tool calls through existing execute_tool_with_safety pipeline - InMemoryStore: HashMap-backed Store impl (no DB tables needed yet) - EngineRouter: is_engine_v2_enabled() + handle_with_engine() that builds engine from Agent deps and processes messages end-to-end Integration touchpoint (4 lines in agent_loop.rs): After hook processing, before session resolution, check ENGINE_V2 flag and route UserInput through the engine path. Accessor visibility widened: llm(), cheap_llm(), safety(), tools() changed from pub(super) to pub(crate) for bridge access. 85 engine tests + main crate clippy clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): add user message and system prompt to thread before execution The ExecutionLoop was sending empty messages to the LLM because the thread was spawned with the user's input as the goal but no messages. Fixes: - ThreadManager.spawn_thread() now adds the goal as an initial user message before starting the execution loop - ExecutionLoop.run() injects a default system prompt if none exists Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): match existing LLM request format to prevent 400 errors The LLM bridge was missing several defaults that the existing Reasoning.respond_with_tools() sets: - tool_choice: "auto" when tools are present (required by some providers) - max_tokens: 4096 (default) - temperature: 0.7 (default) - When no tools (force_text): use plain complete() instead of complete_with_tools() with empty tools array — matches existing no-tools fallback path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): persist conversation context across messages The engine was creating a fresh ThreadManager and InMemoryStore per message, losing all context between turns. A follow-up question like "what are the latest 10 issues?" had no memory of the prior "how many issues" response. Fixes: - EngineState (ThreadManager, ConversationManager, InMemoryStore) now persists across messages via OnceLock, initialized on first use - ConversationManager builds message history from prior conversation entries (user messages + agent responses) and passes it to new threads - ThreadManager.spawn_thread_with_history() accepts initial_messages that are prepended before the current user message - System notifications (thread started/completed) are filtered out of the history (not useful as LLM context) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): enable CodeAct/RLM mode with code block detection The engine now operates in CodeAct/RLM mode: System prompt (executor/prompt.rs): - Instructs LLM to write Python in ```repl fenced blocks - Documents available tools as callable Python functions - Documents llm_query(), llm_query_batched(), FINAL() - Documents context variables (context, goal, step_number, previous_results) - Strategy guidance: examine context, break into steps, use tools, call FINAL() Code block detection (bridge/llm_adapter.rs): - extract_code_block() scans LLM text responses for ```repl or ```python blocks - When detected, returns LlmResponse::Code instead of LlmResponse::Text - The ExecutionLoop routes Code responses through Monty for execution No structured tool definitions sent to LLM: - Tools are described in the system prompt as Python functions - The LLM call sends empty actions array, forcing text-mode responses - This ensures the LLM writes code blocks (CodeAct) instead of structured tool calls (which would bypass the REPL) 85 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(engine): add 8 CodeAct/RLM E2E tests with mock LLM Comprehensive test coverage for the Monty Python execution path: - codeact_simple_final: Python code calls FINAL('answer') → thread completes - codeact_tool_call_then_final: code calls test_tool() → FunctionCall suspends VM → MockEffects returns result → code resumes → FINAL() - codeact_pure_python_computation: sum([1,2,3,4,5]) → FINAL('Sum is 15') with no tool calls — pure Python in Monty - codeact_multi_step: first step prints output (no FINAL), second step sees output metadata and calls FINAL — tests iterative REPL flow - codeact_error_recovery: first step has NameError → error flows to LLM as stdout → second step recovers with FINAL — tests error transparency - codeact_context_variables_available: code accesses `goal` and `context` variables injected by the RLM context builder - codeact_multiple_tool_calls_in_loop: for loop calls test_tool() 3 times → 3 FunctionCall suspensions → all results collected → FINAL - codeact_llm_query_recursive: code calls llm_query('prompt') → VM suspends → MockLlm provides sub-agent response → result returned as Python string variable 93 tests passing (85 prior + 8 new), zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): detect code blocks in plain completion path + multi-block support Two bugs fixed: 1. The no-tools completion path (used by CodeAct since we send empty actions) returned LlmResponse::Text without checking for code blocks. Code blocks were rendered as markdown text instead of being executed. 2. extract_code_block now: - Handles bare ``` fences (skips non-Python languages) - Collects ALL code blocks in the response and concatenates them (models often split code across multiple blocks with explanation) - Tries markers in order: ```repl, ```python, ```py, then bare ``` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(bridge): add 11 regression tests for code block extraction Covers the exact failure modes discovered during live testing: - extract_repl_block: standard ```repl fenced block - extract_python_block: ```python marker - extract_py_block: ```py shorthand - extract_bare_backtick_block: bare ``` with Python content - skip_non_python_language: ```json should NOT be extracted - no_code_blocks_returns_none: plain text, no fences - multiple_code_blocks_concatenated: two ```repl blocks with explanation between them → concatenated with \n\n - mixed_thinking_and_code: model outputs explanation + two ```python blocks (the Hyperliquid case) → both extracted - repl_preferred_over_bare: ```repl takes priority over bare ``` - empty_code_block_skipped: empty fenced block returns None - unclosed_block_returns_none: no closing ``` returns None Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): detect FINAL() in text responses + regression tests Models sometimes write FINAL() outside code blocks — as plain text after an explanation. The Hyperliquid case: model outputs a long analysis then FINAL("""...""") at the end, not inside ```repl fences. Fixes: - extract_final_from_text(): regex-based FINAL detection in text responses, matching the official RLM's find_final_answer() fallback - Handles: double-quoted, single-quoted, triple-quoted, unquoted, nested parens - Checked in LlmResponse::Text handler BEFORE tool intent nudge (FINAL takes priority) 9 new tests: - codeact_final_in_text_response: FINAL("answer") in plain text - codeact_final_triple_quoted_in_text: FINAL("""multi\nline""") in text - final_double_quoted, final_single_quoted, final_triple_quoted, final_unquoted, final_with_nested_parens, final_after_long_text, no_final_returns_none 102 tests passing (93 + 9 new), zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add crate extraction & cleanup roadmap Documents architectural recommendations from the engine v2 design process for future reference: - Root directory consolidation (channels-src + tools-src → extensions/) - Crate extraction tiers: zero-coupling (estimation, observability, tunnel), trivial-coupling (document_extraction, pairing, hooks), medium-coupling (secrets, MCP, db, workspace, llm, skills), heavy-coupling (web gateway, agent, extensions) - src/ module reorganization into logical groups (core, persistence, infra, media, support) - main.rs/app.rs slimming targets (100/500 lines after migration) - WASM module candidates (document_extraction) and non-candidates (REPL, web gateway → separate crates instead) - Priority ordering for extraction work - Tracks completed items (ironclaw_safety, ironclaw_engine, transcription move) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): live progress status updates via event broadcast Engine v2 now shows live progress in the CLI (and any channel): - "Thinking..." when a step starts - Tool name + success/error when actions execute - "Processing results..." when a step completes Implementation: - ThreadManager holds a broadcast::Sender<ThreadEvent> (capacity 256) - ExecutionLoop.emit_event() writes to thread.events AND broadcasts - ThreadManager.subscribe_events() returns a receiver - Router uses tokio::select! to listen for events while waiting for thread completion, forwarding them as StatusUpdate to the channel This replaces the polling approach with zero-latency event streaming. Agent.channels visibility widened to pub(crate) for bridge access. 102 tests passing, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): include tool results in code step output for LLM context The LLM was ignoring tool results and answering from training data because the compact output metadata didn't include what tools returned. Tool results lived only as ActionResult messages (role: Tool) which some providers flatten or the model ignores. Now the code step output includes: - stdout from Python print() statements - [tool_name result] with the actual output (truncated to 4K per tool) - [tool_name error] for failed tools - [return] for the code's return value - Total output truncated to 8K chars to prevent context bloat This ensures the model sees web_search results, API responses, etc. in the next iteration and can reason about them instead of hallucinating. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): add debug/trace logging for CodeAct execution Three verbosity levels for debugging the engine: RUST_LOG=ironclaw_engine=debug: - LLM call: message count, iteration, force_text - LLM response: type (text/code/action_calls), token usage - Code execution: code length, action count, had_error, final_answer - Text response: length, FINAL() detection RUST_LOG=ironclaw_engine=trace: - Full message list sent to LLM (role, length, first 200 chars each) - Full code block being executed - stdout preview (first 500 chars) - Per-tool results (name, success, first 300 chars of output) - Text response preview (first 500 chars) Usage: ENGINE_V2=true RUST_LOG=ironclaw_engine=debug cargo run ENGINE_V2=true RUST_LOG=ironclaw_engine=trace cargo run Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): execution trace recording + retrospective analysis Enable with ENGINE_V2_TRACE=1 to get full execution traces and automatic issue detection after each thread completes. Trace recording (executor/trace.rs): - build_trace(): captures full thread state — messages (with full content), events, step count, token usage, detected issues - write_trace(): writes JSON to engine_trace_{timestamp}.json - log_trace_summary(): logs summary + issues at info/warn level Retrospective analyzer detects 8 issue categories: - thread_failure: thread ended in Failed state - no_response: no assistant message generated - tool_error: specific tool failures with error details - code_error: Python errors (NameError, SyntaxError, etc.) in output - missing_tool_output: tool results exist but not in system messages - excessive_steps: >10 steps (may be stuck in loop) - no_tools_used: single-step answer without tools (hallucination risk) - mixed_mode: text responses without code blocks (prompt not followed) Thread state now saved to store after execution completes (for trace access after join_thread). Usage: ENGINE_V2=true ENGINE_V2_TRACE=1 cargo run # After each message: trace JSON + issue log in terminal Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): wire reflection pipeline + trace analysis into thread lifecycle After every thread completes, ThreadManager now automatically runs: 1. Retrospective trace analysis (non-LLM, always): - Detects 8 issue categories (tool errors, code errors, missing outputs, excessive steps, hallucination risk, etc.) - Logs issues at warn level when found 2. Trace file recording (when ENGINE_V2_TRACE=1): - Writes full JSON trace to engine_trace_{timestamp}.json 3. LLM reflection (when enable_reflection=true): - Calls reflection pipeline to produce Summary, Lesson, Issue docs - Saves docs to store for future context retrieval - Enabled by default in the bridge router All three run inside the spawned tokio task after exec.run() completes, before saving the final thread state. No external wiring needed. Removed duplicate trace recording from the router — it's now handled by ThreadManager automatically. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): convert tool name hyphens to underscores for Python compatibility Root cause from trace analysis: the LLM writes `web_search()` (valid Python identifier) but the tool registry has `web-search` (with hyphen). The EffectBridgeAdapter couldn't find the tool → "Tool not found" error → model fabricated fake data instead. Fixes: - available_actions(): converts tool names from hyphens to underscores (web-search → web_search) so the system prompt lists valid Python names - execute_action(): tries the original name first, then falls back to hyphenated form (web_search → web-search) for tool registry lookup - Same conversion in router's capability registry builder Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): parse JSON tool output to prevent double-serialization From trace analysis: web_search returned a JSON string, which was wrapped as serde_json::json!(string) creating a Value::String containing JSON. When Monty got this as MontyObject::String, the Python code couldn't index it with result['title'] → TypeError. Fix: try parsing the tool output string as JSON first. If valid, use the parsed Value (becomes a Python dict/list). If not valid JSON, keep as string. This means web_search results are directly indexable in Python: results = web_search(query="...") print(results["results"][0]["title"]) # works now Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): persist variables across code steps via `state` dict Monty creates a fresh runtime per code step, so variables are lost between steps. This caused the model to re-paste tool results from system messages, wasting tokens. Fix: maintain a `persisted_state` JSON dict in the ExecutionLoop that accumulates across steps: - Tool results stored by tool name: state["web_search"] = {results...} - Return values stored: state["last_return"], state["step_0_return"] - Injected as a `state` Python variable in each new MontyRun Now the model can do: Step 1: results = web_search(query="...") # tool result saved in state Step 2: data = state["web_search"] # access previous result summary = llm_query("summarize", str(data)) FINAL(summary) System prompt updated to document the `state` variable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): add state hint on code errors + retrieval engine integration When code fails with NameError/UnboundLocalError (model trying to access variables from a previous step), the error output now includes: [HINT] Variables don't persist between code blocks. Use the `state` dict to access data from previous steps. Available keys: ["web_search", "last_return"] This teaches the model to use `state["web_search"]` instead of `result` after a NameError, reducing wasted steps from 3-4 to 1. Also integrates RetrievalEngine into context building and ThreadManager: - build_step_context() now accepts optional RetrievalEngine to inject relevant memory docs (Lessons, Specs, Playbooks) into LLM context - RetrievalEngine uses keyword matching with doc-type priority scoring - Memory docs from reflection (Phase 4) now feed back into future threads Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove trace files and add to .gitignore Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): replace web_fetch example with web_search in CodeAct prompt The system prompt example used web_fetch(url="...") which doesn't exist as a tool. The model learned from the example and tried web_fetch, getting "Tool not found". Changed to web_search(query="...") which is an actual registered tool. Found via trace analysis — reflection pipeline correctly identified this as a "Tool Name Correction" spec doc. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(engine): extract prompt templates to markdown files Prompt templates moved from inline Rust strings to plain markdown files at crates/ironclaw_engine/prompts/ for easy inspection and iteration: - prompts/codeact_preamble.md — main instructions, special functions, context variables, rules - prompts/codeact_postamble.md — strategy section Loaded at compile time via include_str!(), so no runtime file I/O. Edit the .md files and rebuild to iterate on prompts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): replace byte-index slicing with char-safe truncation Panic: 'byte index 80 is not a char boundary; it is inside ''' when tool output contained multi-byte UTF-8 characters (smart quotes from web search results). Fixed 4 unsafe byte-index slices: - thread.rs:281: message preview &content[..80] → chars().take(80) - loop_engine.rs:556: tool output &str[..4000] → chars().take(4000) - loop_engine.rs:579: output tail &str[len-8000..] → chars().skip() - scripting.rs:82: stdout tail &str[len-N..] → chars().skip() All now use .chars().take() or .chars().skip() which respect character boundaries. Follows CLAUDE.md rule: "Never use byte-index slicing on user-supplied or external strings." Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): fix false positive missing_tool_output warning in trace analyzer The check was looking for "[" + "result]" in System-role messages only, but tool output metadata is added with patterns like "[shell result]" and may appear in messages with any role. Changed to scan all messages for " result]" or " error]" patterns regardless of role. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): update architecture plan with Phase 6 status and approval flow design Phase 6 updated to reflect what was actually built: - Bridge adapters (LLM, Effect, InMemoryStore, Router) — all done - Integration touchpoint (4 lines in handle_message) — done - Live progress via broadcast events — done - Conversation persistence across messages — done - Trace recording + retrospective analysis — done - 8 bugs found and fixed via trace analysis — documented Phase 6 remaining work documented: - Approval flow: detailed 5-step design (send to channel, pause thread, route response, resume execution, always handling) with v1 reference - Database persistence (InMemoryStore → real DB tables) - Acceptance testing (TestRig + TraceLlm fixtures) - Two-phase commit for high-stakes effects Progress table updated: Phase 6 marked as DONE (partial), 134 tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add self-improving engine design plan Designs a system where the engine debugs and improves itself, based on the pattern observed in the last session: 5 consecutive bug fixes all followed trace → read → identify → edit → test, using tools the engine already has access to. Three levels of self-improvement: - Level 1 (Prompt): edit prompts/*.md to prevent LLM mistakes. Auto-apply. - Level 2 (Config): adjust defaults/mappings. Branch + test + PR. - Level 3 (Code): Rust patches for engine bugs. Branch + test + clippy + PR. Architecture: Self-improvement Mission spawns a Reflection thread that reads traces, reads source, proposes fixes, validates via cargo test, and either auto-applies (Level 1) or creates a PR (Level 2-3). Includes: fix pattern database (seeded from our 8 debugging session fixes), feedback loop diagram, safety model, implementation phases (A through D), and what exists vs what's new. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add engine v2 security model and audit Comprehensive security analysis of engine v2 covering: Threat model: 4 attacker profiles (malicious input, prompt injection via tools, poisoned memory, supply chain). Current state audit: 9 controls working (Monty sandbox, safety layer, policy engine, leases, provenance, events) and 9 gaps identified. Critical finding: ALL tools granted by default — CodeAct code can call shell, write_file, apply_patch without approval. Proposed fix: 3-tier tool classification (auto/approve-once/always-approve). CodeAct-specific threats: tool call amplification, prompt injection via search results, data exfiltration via tool chains, Monty escape. Self-improvement security: poisoned trace attacks, memory poisoning via reflection. Mitigations: edit validation, frequency caps, audit trail, auto-rollback, reflection output scanning. 6-layer security architecture proposed: input validation, capability gating, output sanitization, execution sandboxing, self-improvement controls, observability. Prioritized implementation plan with severity/effort ratings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(security): cross-reference v1 controls — use, don't reinvent Updated security plan with detailed audit of ALL existing v1 security controls and how they map to engine v2 bridge gaps: Key finding: v1 already has solutions for every security gap identified. The bridge just needs to wire them in: - Tool::requires_approval() exists but bridge doesn't call it - safety.wrap_for_llm() exists but tool results enter context unwrapped - RateLimiter exists but bridge doesn't check rate limits - BeforeToolCall hooks exist but bridge doesn't run them - redact_params() exists but bridge doesn't redact sensitive params - Shell risk classification (Low/Medium/High) is inherited but ignored Revised priority: most fixes are small wiring tasks in EffectBridgeAdapter, not new security infrastructure. The bridge is the security boundary. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): add missions, reliability tracker, reflection executor, and provenance-aware policy - Add Mission type and MissionManager for recurring thread scheduling - Add ReliabilityTracker for per-capability success/failure/latency tracking - Add reflection executor that spawns CodeAct threads for post-completion reflection - Extend PolicyEngine with provenance-aware taint checking (LLM-generated data requires approval for financial/external-write effects) - Extend Store trait with mission CRUD methods - Add conversation surface tracking, compaction token fix, context memory injection - Wire new modules through lib.rs re-exports and bridge adapters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): wire v1 security controls into engine v2 adapter Zero engine crate changes. All security controls enforced at the bridge boundary in EffectBridgeAdapter: 1. Tool approval (v1: Tool::requires_approval): - Checks each tool's approval requirement with actual params - Always → returns EngineError::LeaseDenied (blocks execution) - UnlessAutoApproved → checks auto_approved set, blocks if not approved - Never → proceeds - Per-session auto_approved HashSet (for future "always" handling) 2. Hook interception (v1: BeforeToolCall): - Runs HookEvent::ToolCall before every execution - HookOutcome::Reject → blocks with reason - HookError::Rejected → blocks with reason - Hook errors → fail-open (logged, execution continues) 3. Output sanitization (v1: sanitize_tool_output + wrap_for_llm): - Leak detection: API keys in tool output are redacted - Policy enforcement: content policy rules applied - Length truncation: output capped at 100KB - XML boundary protection: prevents injection via tool output 4. Sensitive param redaction (v1: redact_params): - Tool's sensitive_params() consulted before hooks see parameters - Redacted params sent to hooks, original params used for execution 5. available_actions() now sets requires_approval based on each tool's default approval requirement, so the engine's PolicyEngine can gate tools it hasn't seen before. 6. Actual execution timing measured via Instant::now() (replaces placeholder Duration::from_millis(1)). Accessor visibility: hooks() widened to pub(crate). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): implement tool approval flow for engine v2 Adds a complete approval flow that mirrors v1 behavior, using the existing v1 security controls (Tool::requires_approval, auto-approve sets, StatusUpdate::ApprovalNeeded). ## How it works ### Step 1: Tool blocked at execution When the LLM's code calls a tool (e.g., `shell("ls")`): 1. EffectBridgeAdapter.execute_action() looks up the Tool object 2. Calls tool.requires_approval(¶ms) — returns ApprovalRequirement 3. If Always → EngineError::LeaseDenied (always blocks) 4. If UnlessAutoApproved → checks auto_approved HashSet → if not in set, returns EngineError::LeaseDenied 5. If Never → proceeds to execution ### Step 2: Engine returns NeedApproval The LeaseDenied error propagates through: - CodeAct path: becomes Python RuntimeError, code halts, thread returns NeedApproval with action_name + parameters - Structured path: same via ActionResult.is_error ### Step 3: Router stores pending approval - PendingApproval { action_name, original_content } stored on EngineState - StatusUpdate::ApprovalNeeded sent to channel (shows approval card in CLI/web with tool name, parameters, yes/always/no buttons) - Returns text: "Tool 'shell' requires approval. Reply yes/always/no." ### Step 4: User responds handle_message() intercepts Submission::ApprovalResponse when ENGINE_V2: - 'yes' → auto_approve_tool(name) on EffectBridgeAdapter, re-processes original message (tool now passes the approval check on second run) - 'always' → same + logs for session persistence - 'no' → returns "Denied: tool was not executed." ### Key design choice Instead of pausing/resuming mid-execution (which needs engine changes to freeze/restore the Monty VM state), we auto-approve the tool and re-run the full message. The EffectBridgeAdapter's auto_approved set persists across runs, so the second execution passes immediately. This trades one extra LLM call for zero engine modifications. ## Files changed - src/bridge/router.rs: PendingApproval struct, handle_approval(), NeedApproval → StatusUpdate::ApprovalNeeded conversion - src/bridge/mod.rs: export handle_approval - src/agent/agent_loop.rs: intercept ApprovalResponse for engine v2 - src/bridge/effect_adapter.rs: fmt fixes 151 tests passing, clippy + fmt clean. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): demote trace/reflection logging from info to debug INFO-level log output from background tasks (trace analysis, reflection) corrupts the REPL terminal UI. The trace summary, issue warnings, and reflection doc previews were printing mid-approval-card, breaking the interactive display. Fix: all logging in trace.rs changed from info!/warn! to debug!/warn!. Trace analysis and reflection results now only show when RUST_LOG=ironclaw_engine=debug is set. Also added logging discipline rule to global CLAUDE.md: - info! → user-facing status the REPL intentionally renders - debug! → internal diagnostics (traces, reflection, engine internals) - Background tasks must NEVER use info! — it breaks the TUI Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): demote all router info! logging to debug! "engine v2: initializing" and "engine v2: handling message" were printing at INFO level, corrupting the REPL UI. All router logging now uses debug! — only visible with RUST_LOG=ironclaw=debug. Zero info! calls remain in crates/ironclaw_engine/ or src/bridge/. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(safety): demote leak detector warn-action logs from warn! to debug! The leak detector's Warn-action matches (high_entropy_hex pattern on web search results containing commit SHAs, CSS colors, URL hashes) were logging at warn! level, corrupting the REPL UI with lines like: WARN Potential secret leak detected pattern=high_entropy_hex preview=a96f********cee5 These are informational false positives — real leaks use LeakAction::Redact which silently modifies the content. Warn-action matches only log for debugging purposes and should not appear in production output. Changed to debug! level — visible with RUST_LOG=ironclaw_safety=debug. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): strengthen CodeAct prompt to prevent shallow text answers The model was answering "Suggested 45 improvements" as a brief text summary from training data without actually searching or listing them. The trace showed: no code block, no tool calls, no FINAL(). Prompt changes: - Rule 1: "ALWAYS respond with a ```repl code block. NEVER answer with plain text only." (was: "Always write code... plain text for brief explanations") - Rule 2 (NEW): "NEVER answer from memory or training data alone. Always use tools to get real, current information before answering." - Rule 3: FINAL answer "should be detailed and complete — not just a summary like 'found 45 items'" - Rule 8 (NEW): "Include the actual content in your FINAL() answer, not just a count or summary. Users want to see the details." Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): persist reflection docs to workspace for cross-session learning Replaces InMemoryStore with HybridStore: - Ephemeral data (threads, steps, events, leases) stays in-memory - MemoryDocs (lessons, specs, playbooks from reflection) persist to the workspace at engine/docs/{type}/{id}.json On engine init, load_docs_from_workspace() reads existing docs back into the in-memory cache. This means: - Lessons learned in session 1 are available in session 2 - The RetrievalEngine injects relevant past lessons into new threads - The engine genuinely improves over time as reflection accumulates Workspace paths: engine/docs/lessons/{uuid}.json engine/docs/specs/{uuid}.json engine/docs/playbooks/{uuid}.json engine/docs/summaries/{uuid}.json engine/docs/issues/{uuid}.json No new database tables. Uses existing workspace write/read/list. workspace() accessor widened to pub(crate). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(bridge): adapt to execute_tool_with_safety params-by-value change Staging merge changed execute_tool_with_safety to take params by value instead of by reference (perf optimization from PR #926). Updated bridge adapter to clone params before passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): add web gateway integration plan to Phase 6 Documents three gaps between engine v2 and the web gateway: 1. No SSE streaming (engine emits ThreadEvent, gateway expects SseEvent) 2. No conversation persistence (engine uses HybridStore, gateway reads v1 DB) 3. No cross-channel visibility (REPL ↔ web messages invisible to each other) Implementation plan: bridge ThreadEvent→AppEvent, write messages to v1 conversation tables after thread completion. Prerequisite: AppEvent extraction PR (in progress separately). Also updated DB persistence status: HybridStore with workspace-backed MemoryDocs is now implemented (partial persistence). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(engine): document routine/job gap and SIGKILL crash scenario Routines are entirely v1 — not hooked up to engine v2. When a user asks "create a routine" as natural language, engine v2 tries to call routine_create via CodeAct, but the tool needs RoutineEngine + Database refs that the bridge's minimal JobContext doesn't provide. This caused a SIGKILL crash during testing. Options documented: block routine tools in v2 (short term), pass refs through context (medium), replace with Mission system (long term). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: extract AppEvent to crates/ironclaw_common SseEvent was defined in src/channels/web/types.rs but imported by 12+ modules across agent, orchestrator, worker, tools, and extensions — it had become the application-wide event protocol, not a web transport concern. Create crates/ironclaw_common as a shared workspace crate and move the enum there as AppEvent. Also move the truncate_preview utility which was similarly leaked from the web gateway into agent modules. - New crate: crates/ironclaw_common (AppEvent, truncate_preview) - Rename SseEvent → AppEvent, from_sse_event → from_app_event - web/types.rs re-exports AppEvent for internal gateway use - web/util.rs re-exports truncate_preview - Wire format unchanged (serde renames are on variants, not the enum) Aligned with the event bus direction on refactor/architectural-hardening where DomainEvent (≡ AppEvent) is wrapped in a SystemEvent envelope. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): integrate with web gateway via AppEvent + v1 conversation DB Three changes to make engine v2 visible in the web gateway: 1. SSE event streaming (AppEvent broadcast): - ThreadEvent → AppEvent conversion via thread_event_to_app_event() - Events broadcast to SseManager during the poll loop - Covers: Thinking, ToolCompleted (success/error), Status, Response - Web gateway receives real-time progress without any gateway changes 2. Conversation persistence to v1 database: - After thread completes, writes user message + agent response to v1 ConversationStore via add_conversation_message() - Uses get_or_create_assistant_conversation() for per-user per-channel - Web gateway reads from DB as usual — chat history appears 3. Final response broadcast: - AppEvent::Response with full text + thread_id sent via SSE - Web gateway renders the response in the chat UI New EngineState fields: sse (Option<Arc<SseManager>>), db (Option<Arc<dyn Database>>). Both populated from Agent.deps. Agent.deps visibility widened to pub(crate). Depends on: ironclaw_common crate with AppEvent type (PR #1615). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): complete Phase 6 — v1-only tool blocking, rate limiting, call limits Three security/stability improvements in EffectBridgeAdapter: 1. V1-only tool blocking: - routine_create, create_job, build_software (and hyphenated variants) return helpful error: "use the slash command instead" - Filtered out of available_actions() so system prompt doesn't list them - Prevents crash from tools needing RoutineEngine/Scheduler refs 2. Per-step tool call limit: - Max 50 tool calls per code block (AtomicU32 counter) - Prevents amplification: `for i in range(10000): shell(...)` - Returns "call limit reached, break into multiple steps" 3. Rate limiting: - Per-user per-tool sliding window via RateLimiter - Checks tool.rate_limit_config() before every execution - Returns "rate limited, try again in Ns" Architecture plan updated: - Gateway integration: DONE - Routines: BLOCKED (gracefully, with slash command fallback) - Rate limiting: DONE - Call limit: DONE - Phase 6 status: DONE (remaining: acceptance tests, two-phase commit) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add Mission system design — goal-oriented autonomous threads Missions replace routines with evolving, knowledge-accumulating autonomous agents. Unlike routines (fixed prompt, stateless), Missions: - Generate prompts from accumulated Project knowledge (lessons, playbooks, issues from prior threads) - Adapt approach when something fails repeatedly - Track progress toward a goal with success criteria - Self-manage: pause when stuck, complete when goal achieved Architecture: MissionManager with cron ticker spawns threads via ThreadManager. Meta-prompt built from mission goal + Project MemoryDocs via RetrievalEngine. Reflection feeds back automatically. 6-step implementation plan: cron trigger, meta-prompt builder, bridge wiring, CodeAct tools, progress tracking, persistence. Includes two worked examples: daily tech news briefing (ongoing) and test coverage improvement (goal-driven, self-completing). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): extend Mission types with webhook/event triggers + evolving strategy Mission types updated to support external activation sources: MissionCadence expanded: - Cron { expression, timezone } — timezone-aware scheduling - OnEvent { event_pattern } — channel message pattern matching - OnSystemEvent { source, event_type } — structured events from tools - Webhook { path, secret } — external HTTP triggers (GitHub, email, etc.) - Manual — explicit triggering only The engine defines trigger TYPES. The bridge implements infrastructure (cron ticker, webhook endpoints, event matchers). GitHub issues, PRs, email, Slack events all use the generic Webhook cadence — no special-casing in the engine. Webhook payload injected as state["trigger_payload"] in the thread's Python context. Mission struct extended: - current_focus: what the next thread should work on (evolving) - approach_history: what we've tried (for adaptation) - max_threads_per_day / threads_today: daily budget - last_trigger_payload: webhook/event data for thread context Plan updated with trigger type table and webhook integration design. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): implement MissionManager execution with meta-prompts The MissionManager now builds evolving meta-prompts and processes thread outcomes for continuous learning: fire_mission() upgraded: - Loads Project MemoryDocs via RetrievalEngine for context - Builds meta-prompt from: goal, current_focus, approach_history, project knowledge docs, trigger payload, thread count - Spawns thread with meta-prompt as user message - Background task waits for completion and processes outcome - Daily thread budget enforcement (max_threads_per_day) Meta-prompt structure: # Mission: {name} Goal: {goal} ## Current Focus (evolves between threads) ## Previous Approaches (what we've tried) ## Knowledge from Prior Threads (lessons, playbooks, issues) ## Trigger Payload (webhook/event data if applicable) ## Instructions (accomplish step, report next focus, check goal) Outcome processing: - Extracts "next focus:" from FINAL() response → updates current_focus - Detects "goal achieved: yes" → completes mission - Records accomplishment in approach_history - Failed threads recorded as "FAILED: {error}" Cron ticker: - start_cron_ticker() spawns tokio task, ticks every 60s - Checks active Cron missions, fires those past next_fire_at 151 tests passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): wire MissionManager into engine v2 for CodeAct access Missions are now callable from CodeAct Python code: ```python # Create a daily briefing mission result = mission_create( name="Tech News", goal="Daily AI/crypto/software news briefing", cadence="0 9 * * *" ) # List all missions missions = mission_list() # Manually fire a mission mission_fire(id="...") # Pause/resume mission_pause(id="...") mission_resume(id="...") ``` Implementation: - MissionManager created on engine init, cron ticker started - EffectBridgeAdapter intercepts mission_* function calls before tool lookup and routes to MissionManager - parse_cadence() handles: "manual", cron expressions, "event:pattern", "webhook:path" - Mission functions documented in CodeAct system prompt - MissionManager set on adapter via set_mission_manager() after init (avoids circular dependency) System prompt updated with mission_create, mission_list, mission_fire, mission_pause, mission_resume documentation. 151 tests passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(bridge): map routine_* calls to mission operations in v2 When the model calls routine_create, routine_list, routine_fire, routine_pause, routine_resume, or routine_delete, the bridge now routes them to the MissionManager instead of blocking with an error. Mapping: routine_create → mission_create (with cadence parsing) routine_list → mission_list routine_fire → mission_fire routine_pause → mission_pause routine_resume → mission_resume routine_update → mission_pause/resume (based on params) routine_delete → mission_complete (marks as done) Routine tools removed from v1-only blocklist and restored in available_actions(). The model can use either "routine" or "mission" vocabulary — both work. Still blocked: create_job, cancel_job, build_software (need v1 Scheduler/ContainerJobManager refs). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(engine): add E2E mission flow tests — 7 new tests Comprehensive mission lifecycle tests: - fire_mission_builds_meta_prompt_with_goal: verifies thread spawned with project context and recorded in history - outcome_processing_extracts_next_focus: "Next focus: X" in FINAL() response → mission.current_focus updated - outcome_processing_detects_goal_achieved: "Goal achieved: yes" → mission status transitions to Completed - mission_evolves_via_direct_outcome_processing: 3-step evolution: step 1 sets focus to "db module", step 2 evolves to "tools module", step 3 detects goal achieved → mission completes. Tests the full learning loop without background task timing dependencies. - fire_with_trigger_payload: webhook payload stored on mission and threads_today counter incremented - daily_budget_enforced: max_threads_per_day=1 → first fire succeeds, second returns None 157 tests passing (151 prior + 6 new mission E2E). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): self-improving engine via Mission system Wire the self-improvement loop as a Mission with OnSystemEvent cadence, inspired by karpathy/autoresearch's program.md approach. The mission fires when threads complete with issues, receives trace data as trigger payload, and uses tools directly to diagnose and fix problems. Key changes: Engine self-improvement (Phase A+B from design doc): - Add fire_on_system_event() to MissionManager for OnSystemEvent cadence - Add start_event_listener() that subscribes to thread events and fires matching missions when non-Mission threads complete with trace issues - Add ensure_self_improvement_mission() with autoresearch-style goal prompt (concrete loop steps, not vague instructions) - Add process_self_improvement_output() for structured JSON fallback - Seed fix pattern database with 8 known patterns from debugging - Runtime prompt overlay via MemoryDoc (build_codeact_system_prompt now async + Store-aware, appends learned rules from prompt_overlay docs) - Pass Store to ExecutionLoop for overlay loading Bridge review fixes (P1/P2): - Scope engine v2 SSE events to requesting user (broadcast_for_user) - Per-user pending approvals via HashMap instead of global Option - Reset tool-call limit counter before each thread execution - Only persist auto-approval when user chose "always", not one-off "yes" - Remove dead store/mission_manager fields from EngineState Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add checkpoint-based engine thread recovery * feat(engine): add Python orchestrator module and host functions Add the orchestrator infrastructure for replacing the Rust execution loop with versioned Python code. This commit adds the module and host functions without switching over — the existing Rust loop is unchanged. New files: - orchestrator/default.py: v0 Python orchestrator (run_loop + helpers) - executor/orchestrator.rs: host function dispatch, orchestrator loading from Store with version selection, OrchestratorResult parsing Host functions exposed to orchestrator Python via Monty suspension: __llm_complete__, __execute_code_step__ (nested Monty VM), __execute_action__, __check_signals__, __emit_event__, __add_message__, __save_checkpoint__, __transition_to__, __retrieve_docs__, __check_budget__, __get_actions__ Also makes json_to_monty, monty_to_json, monty_to_string pub(crate) in scripting.rs for cross-module use. Design doc: docs/plans/2026-03-25-python-orchestrator.md Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): switch ExecutionLoop::run() to Python orchestrator Replace the 900-line Rust execution loop with a ~80-line bootstrap that loads and runs the versioned Python orchestrator via Monty VM. The orchestrator Python code (orchestrator/default.py) is the v0 compiled-in version. Runtime versions can override it via MemoryDoc storage (orchestrator:main with tag orchestrator_code). Key fixes during switchover: - Use ExtFunctionResult::NotFound for unknown functions so Monty falls through to Python-defined functions (extract_final, etc.) - Move helper function definitions above run_loop for Monty scoping - Use FINAL result value (not VM return value) in Complete handler - Rename 'final' variable to 'final_answer' to avoid Python keyword Status: 171/177 tests pass. 6 remaining failures are step_count and token tracking bookkeeping — the orchestrator manages these internally but doesn't yet update the thread's counters via host functions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): all 177 tests pass with Python orchestrator - Increment step_count and track tokens in __emit_event__("step_completed") so thread bookkeeping matches the old Rust loop behavior - Remove double-counting of tokens in bootstrap (orchestrator handles it) - Match nudge text to existing TOOL_INTENT_NUDGE constant - Fix FINAL result propagation (use stored final_result, not VM return) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): orchestrator versioning, auto-rollback, and tests Add version lifecycle for the Python orchestrator: - Failure tracking via MemoryDoc (orchestrator:failures) - Auto-rollback: after 3 consecutive failures, skip the latest version and fall back to previous (or compiled-in v0) - Success resets the failure counter - OrchestratorRollback event for observability Update self-improvement Mission goal with Level 1.5 instructions for orchestrator patches — the agent can now modify the execution loop itself via memory_write with versioned orchestrator docs. 12 new tests: version selection (highest wins), rollback after failures, rollback to default, failure counting/resetting, outcome parsing for all 5 ThreadOutcome variants. 189 tests pass, zero clippy warnings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add engine v2 architecture, self-improvement, and dev history Three new docs for contributors: - engine-v2-architecture.md: Two-layer architecture (Rust kernel + Python orchestrator), five primitives, execution model with nested Monty VMs, bridge layer, memory/reflection, missions, capabilities - self-improvement.md: Three improvement levels (prompt/orchestrator/ config/code), autoresearch-inspired Mission loop, versioned orchestrator with auto-rollback, fix pattern database, safety model - development-history.md: Summary of 6 Claude Code sessions that built the system, key design decisions and debugging moments, architecture evolution from 900-line Rust loop to Python orchestrator Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(engine): complete v2 side-by-side integration with gateway API Wire engine v2 into the full submission pipeline and expose threads, projects, and missions through the web gateway REST API. Bridge routing — route ExecApproval, Interrupt, NewThread, and Clear submissions to engine v2 when ENGINE_V2=true. Previously only UserInput and ApprovalResponse were handled; all other control commands fell through to disconnected v1 sessions. Bridge query layer — add 11 read-only query functions and 6 DTO types so gateway handlers can inspect engine state (threads, steps, events, projects, missions) without direct access to the EngineState singleton. Gateway endpoints — new /api/engine/* routes: GET /threads, /threads/{id}, /threads/{id}/steps, /threads/{id}/events GET /projects, /projects/{id} GET /missions, /missions/{id} POST /missions/{id}/fire, /missions/{id}/pause, /missions/{id}/resume SSE events — add ThreadStateChanged, ChildThreadSpawned, and MissionThreadSpawned AppEvent variants. Expand the bridge event mapper to forward StateChanged and ChildSpawned engine events to the browser. Engine crate — add ConversationManager::clear_conversation() for /new and /clear commands. Code quality — replace 10 .expect() calls with proper error returns, remove dead AgentConfig.engine_v2 field, log silent init errors, fix duplicate doc comment, improve fallthrough documentation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): empty call_id on ActionResult and trace analyzer false positives Fix structured executor not stamping call_id onto ActionResult — the EffectExecutor trait doesn't receive call_id, so the structured executor must copy it from the original ActionCall after execution. Empty call_id caused OpenAI-compatible providers to reject the next LLM request with "Invalid 'input[2].call_id': empty string". Fix trace analyzer false positives: - code_error check now only scans User-role code output messages (prefixed with [stdout]/[stderr]/[code ]/Traceback), not System prompt which contains example error text - missing_tool_output check now recognizes ActionResult messages as valid tool output (Tier 0 structured path) - Add NotImplementedError to detected code error patterns New trace checks: - empty_call_id: detect ActionResult messages with missing/empty call_id before they reach the LLM API (severity: Error) - llm_error: extract LLM provider errors from Failed state reason - orchestrator_error: extract orchestrator errors from Failed state Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): add Missions tab to gateway UI Add a full Missions page to the web gateway with list view, detail view, and action buttons (Fire, Pause, Resume). Backend: add /api/engine/missions/summary endpoint returning counts by status (active/paused/completed/failed). Frontend: - New "Missions" tab between Jobs and Routines - Summary cards showing mission counts by status - Table with name, goal, cadence type, thread count, status, actions - Detail view with goal, cadence, current focus, success criteria, approach history, spawned thread list, and action buttons - Fire/Pause/Resume actions with toast notifications - i18n support (English + Chinese) - CSS following the existing routines/jobs patterns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): eagerly initialize engine v2 at startup The gateway API endpoints (/api/engine/missions, etc.) call bridge query functions that return empty results when the engine state hasn't been initialized yet. Previously, initialization only happened lazily on the first chat message via handle_with_engine(). Now when ENGINE_V2=true, the engine is initialized in Agent::run() before channels start, so the self-improvement mission and other engine state is available to gateway API endpoints immediately. Also rename get_or_init_engine → init_engine and make it public so it can be called from agent_loop.rs at startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): improve mission detail with markdown goal and thread table - Goal rendered as full-width markdown block instead of plain-text meta item (uses existing renderMarkdown/marked) - Current focus and success criteria also rendered as markdown - Spawned threads shown as a clickable table with goal, type, state, steps, tokens, and created date instead of a UUID list - Clicking a thread row opens an inline thread detail view showing metadata grid and full message history with markdown rendering - Back button returns to the mission detail view - Backend: mission detail now returns full thread summaries (goal, state, step_count, tokens) instead of just thread IDs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): close SSE connections on page unload to prevent connection starvation The browser limits concurrent HTTP/1.1 connections per origin to 6. Without cleanup, SSE connections from prior page loads linger after refresh/navigation, eating into the pool. After 2-3 refreshes, all 6 slots are consumed by stale SSE streams and new API fetch calls queue indefinitely — the UI shows "connected" (SSE works) but data never loads. Add a beforeunload handler that closes both eventSource (chat events) and logEventSource (log stream) so the browser can reuse connections immediately on page reload. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(web): support multiple gateway tabs by reducing SSE connections Each browser tab opened 2 SSE connections (chat events + log events). With the HTTP/1.1 per-origin limit of 6, the 3rd tab exhausted the pool and couldn't load any data. Three changes: 1. Lazy log SSE — only connect when the logs tab is active, disconnect when switching away. Most users rarely view logs, so this saves a connection slot per tab. 2. Visibility API — close SSE when the browser tab goes to background (user switches to another tab), reconnect when it becomes visible. Background tabs don't need real-time events. 3. Combined with the existing beforeunload cleanup, this means: - Active foreground tab: 1 connection (chat SSE only, +1 if logs tab) - Background tabs: 0 connections - Closed/refreshed tabs: 0 connections (beforeunload cleanup) This allows many gateway tabs to coexist within the 6-connection limit. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(engine): route messages to correct conversation by thread scope Messages sent from a new conversation in the gateway always appeared in the default assistant conversation because handle_with_engine ignored the thread_id from the frontend. Two fixes: 1. Engine conversation scoping — when the message carries a thread_id (from the frontend's conversation picker), use it as part of the engine conversation key: "gateway:<thread_id>" instead of just "gateway". This creates a distinct engine conversation per v1 thread, so messages don't cross-contaminate. 2. V1 dual-write targeting — write user messages and assistant responses to the v1 conversation matching the thread_id (via ensure_conversation), not the hardcoded assistant conversation. Falls back to the assistant conversation when no thread_id is present (e.g., default chat). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(web): richer activity indicators for engine v2 execution The gateway UI showed only generic "Thinking..." during engine v2 execution with no visibility into CodeAct code execution, tool calls, or reflection. Now the event mapping produces detailed status updates: Step lifecycle: - "Calling LLM..." when a step starts (was "…
* Move safety benches into ironclaw_safety crate * Annotate benchmark JSON unwraps for panic check
) The import-feature-gated help snapshots were stale after the --auto-approve flag and ACP subcommand were added, causing CI snapshot test failures. https://claude.ai/code/session_01Vfo7aFbVDpibtH7EsDAUSE Co-authored-by: Claude <noreply@anthropic.com>
The OpenAI Codex provider was missing the sanitize_tool_messages() call that all other providers use to rewrite orphaned tool results as user messages. This caused HTTP 400 errors when conversation history contained tool result messages whose corresponding tool calls were dropped during context compaction or thread resume. Adds the call to both complete() and complete_with_tools(), matching the pattern used by rig_adapter, nearai_chat, and bedrock providers. Fixes #1969 Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a three-state permission model (AlwaysAllow / AskEachTime / Disabled) backed by per-user DB settings so that safe built-in tools run without approval prompts by default, while destructive tools still require explicit user confirmation. Key changes: - PermissionState enum + TOOL_RISK_DEFAULTS tier map (permissions.rs) - Security: tools returning UnlessAutoApproved (http, create_job, event_emit, routine_create, routine_update) default to AskEachTime to prevent SSRF, resource abuse, and privilege escalation - Dispatcher filters Disabled tools and pre-approves AlwaysAllow tools; clears and re-populates session auto-approvals each iteration so permission downgrades take effect immediately; caches permissions at iteration 0 to avoid repeated DB round-trips - Graceful DB failure handling: keeps existing session approvals on transient DB errors instead of clearing them - "Always Approve" persists to DB across sessions (thread_ops.rs); defense-in-depth skips persist for ApprovalRequirement::Always tools - tool_permission_set LLM tool (always requires approval to change); returns error when no settings store configured - tool_list extended with builtin tools + permission state fields; filters by builtin_tool_names() to avoid duplicating WASM/MCP tools - Web UI: Tools settings tab with 3-way toggle + lock icons; JSON error body for locked tool rejection - Dot-separated tool name validation at registration time - Playwright E2E: 6 test scenarios for permissions lifecycle - seed_tool_permissions writes tier defaults to DB at startup with idempotency test ApprovalRequirement::Always is an unbypassable hard floor — even AlwaysAllow cannot bypass it. All DB operations scoped to user_id via TenantScope for multi-tenant correctness. Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* feat(docker): publish ironclaw-worker image alongside ironclaw Build and push nearaidev/ironclaw-worker from Dockerfile.worker in the same workflow. Both images share the same version/sha/tag scheme. This lets ironclaw-dind pull the pre-built worker image for sandbox baking instead of cloning the repo and building from source. [skip-regression-check] * feat(docker): daily scheduled build of :staging from staging branch * perf(docker): worker image copies binary from ironclaw image instead of rebuilding * revert Dockerfile.worker changes, keep it building from source
…B-backed pairing, and OwnershipCache (#1898) * feat(ownership): add OwnerId, Identity, UserRole, can_act_on types Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): private OwnerId field, ResourceScope serde derives, fix doc comment Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * refactor(tenant): replace SystemScope::db() escape hatch with typed workspace_for_user(), fix stale variable names - Add SystemScope::workspace_for_user() that wraps Workspace::new_with_db - Remove SystemScope::db() which exposed the raw Arc<dyn Database> - Update 3 callers (routine_engine.rs x2, heartbeat.rs x1) to use the new method - Fix stale comment: "admin context" -> "system context" in SystemScope - Rename `admin` bindings to `system` in agent_loop.rs for clarity Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(tenant): rename stale admin binding to system_store in heartbeat.rs * refactor(tenant): TenantScope/TenantCtx carry Identity, add with_identity() constructor and bridge new() - TenantScope: replace `user_id: String` field with `identity: Identity`; add `with_identity()` preferred constructor; keep `new(user_id, db)` as Member-role bridge; add `identity()` accessor; all internal method bodies use `identity.owner_id.as_str()` in place of `&self.user_id` - TenantCtx: replace `user_id: String` field with `identity: Identity`; update constructor signature; add `identity()` accessor; `user_id()` delegates to `identity.owner_id.as_str()`; cost/rate methods updated accordingly - agent_loop: split `tenant_ctx(&str)` into bridge + new `tenant_ctx_with_identity(Identity)` which holds the full body; bridge delegates to avoid duplication Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(db): add V16 tool scope, V17 channel_identities, V18 pairing_requests migrations - PostgreSQL: V16__tool_scope.sql adds scope column to wasm_tools/dynamic_tools - PostgreSQL: V17__channel_identities.sql creates channel identity resolution table - PostgreSQL: V18__pairing_requests.sql creates pairing request table replacing file-based store - libSQL SCHEMA: adds scope column to wasm_tools/dynamic_tools, channel_identities, pairing_requests tables - libSQL INCREMENTAL_MIGRATIONS: versions 17-19 for existing databases - IDEMPOTENT_ADD_COLUMN_MIGRATIONS: handles fresh-install/upgrade dual path for scope columns - Runner updated to check ALL idempotent columns per version before skipping SQL - Test: test_ownership_model_tables_created verifies all new tables/columns exist after migrations Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(db): use correct RFC3339 timestamp default in libSQL, document version sequence offset Replace datetime('now') with strftime('%Y-%m-%dT%H:%M:%fZ', 'now') in the channel_identities and pairing_requests table definitions (both in SCHEMA and INCREMENTAL_MIGRATIONS) to match the project-standard RFC 3339 timestamp format with millisecond precision. Also add a comment clarifying that libSQL incremental migration version numbers are independent from PostgreSQL VN migration numbers. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(ownership): bootstrap_ownership(), migrate_default_owner, V19 FK migration, replace hardcoded 'default' user IDs - Add V19__ownership_fk.sql (programmatic-only, not in auto-migration sweep) - Add `migrate_default_owner` to Database trait + both PgBackend and LibSqlBackend - Add `get_or_create_user` default method to UserStore trait - Add `bootstrap_ownership()` to app.rs, called in init_database() after connect_with_handles - Replace hardcoded "default" owner_id in cli/config.rs, cli/mcp.rs, cli/mod.rs, orchestrator/mod.rs - Add TODO(ownership) comments in llm/session.rs and tools/mcp/client.rs for deferred constructors Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): atomic get_or_create_user, transactional migrate_default_owner, V19 FK inline constant, fix remaining 'default' user IDs - Delete migrations/V19__ownership_fk.sql so refinery no longer auto-applies FK constraints before bootstrap_ownership runs; add OWNERSHIP_FK_SQL constant with TODO for future programmatic application - Remove racy SELECT+INSERT default in UserStore::get_or_create_user; both PostgreSQL (ON CONFLICT DO NOTHING) and libSQL (INSERT OR IGNORE) now use atomic upserts - Wrap migrate_default_owner in explicit transactions on both backends for atomicity - Make bootstrap_ownership failure fatal (propagate error instead of warn-and-continue) - Fix mcp auth/test --user: change from default_value="default" to Option<String> resolved from configured owner_id - Replace hardcoded "default" user IDs in channels/wasm/setup.rs with config.owner_id - Replace "default" sentinel in OrchestratorState test helper with "<unset>" to make the test-only nature explicit Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): remove default user_id from create_job(), change sentinel strings to <unset> - Gate ContextManager::create_job() behind #[cfg(test)]; production code must use create_job_for_user() with an explicit user_id to prevent DB rows with user_id = 'default' being silently created on the production write path. - Change the placeholder user_id in McpClient::new(), new_with_name(), and new_with_config() from "default" to "<unset>" so accidental secrets/settings lookups surface immediately rather than silently touching the wrong DB partition. - Same sentinel change for SessionManager::new() and new_async() in session.rs; these are overwritten by attach_store() at startup with the real owner_id. - Update tests that asserted the old "default" sentinel to expect "<unset>", and switch test_list_jobs_tool / test_job_status_tool to create_job_for_user("default") to keep ownership alignment with JobContext::default(). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(db): add ChannelPairingStore sub-trait with resolve_channel_identity, upsert/approve pairing, PostgreSQL + libSQL implementations Adds PairingRequestRecord, ChannelPairingStore trait (5 methods), and generate_pairing_code() to src/db/mod.rs; implements for PgBackend in postgres.rs and LibSqlBackend in libsql/pairing.rs; wires ChannelPairingStore into the Database supertrait bound; all 6 libSQL unit tests pass. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(db): atomic libSQL approve_pairing with BEGIN IMMEDIATE, add case-insensitive/expired/double-approve tests Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(ownership): add OwnershipCache for zero-DB-read identity resolution on warm path Converts src/ownership.rs to src/ownership/ module directory and adds src/ownership/cache.rs with a write-through in-process cache mapping (channel, external_id) -> Identity. Wired as Arc<OwnershipCache> on AppComponents for Task 8 pairing integration. All 7 cache unit tests pass. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * test(e2e): add ownership model E2E tests and extend pairing tests for DB-backed store Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(e2e): remove unused asyncio import, add fallback assertion in test_pairing_response_structure Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * test(tenant): unit tests for TenantScope::with_identity and AdminScope construction Adds 5 focused unit tests verifying TenantScope::with_identity stores the full Identity (owner_id + role), TenantScope::new creates a Member-role identity, and AdminScope::new returns Some for Admin and None for Member. Uses LibSqlBackend::new_memory() as the test DB stub. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): recover from RwLock poison instead of expect() in OwnershipCache Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * test(ownership): integration tests for bootstrap, tenant isolation, and ChannelPairingStore Adds tests/ownership_integration.rs covering migrate_default_owner idempotency, TenantScope per-user setting isolation (including Admin role bypass check), and the full ChannelPairingStore lifecycle (upsert, approve, remove, multi-channel isolation). Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(test): remove duplicate pairing tests and flaky random-code assertion from integration suite Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(pairing): rewrite PairingStore to DB-backed async with OwnershipCache Replaces the file-based pairing store (~/.ironclaw/*-pairing.json, *-allowFrom.json) with a DB-backed async implementation that delegates to ChannelPairingStore and writes through to OwnershipCache on reads. - PairingStore::new(db, cache) uses the DB; new_noop() for test/no-DB - resolve_identity() cache-first lookup via OwnershipCache - approve(code, owner_id) removes channel arg (DB looks up by code) - All WASM host functions updated: pairing_upsert_request uses block_in_place, pairing-is-allowed renamed to pairing-resolve-identity returning Option<String>, pairing-read-allow-from deprecated (returns empty list) - Signal channel receives PairingStore via new(config, db) constructor - Web gateway pairing handlers read from state.store (DB) directly - extensions.rs derive_activation_status drops PairingStore dependency; derives status from extension.active and owner_binding flag instead - All test call sites updated to use new_noop() Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(pairing): add missing pairing_store field to all GatewayState initializers, fix disk-full post-edit compile Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(channels): remove owner_id from IncomingMessage, user_id is the canonical resolved OwnerId `owner_id` on `IncomingMessage` was always a duplicate of `user_id` — both fields held the same value at every call site. Remove the field and `with_owner_id()` builder, update the four WASM-wrapper and HTTP test assertions to use `user_id`, and drop the redundant struct literal field in the routine_engine test helper. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(channels): remove stale owner_id param from make_message test helper Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * test(e2e): add browser/Playwright tests for ownership model — auth screen, chat UI, owner login Adds five Playwright-based browser tests to the ownership model E2E suite verifying the web UI experience: authenticated owner sees chat input, unauthenticated browser sees auth screen, owner can send a message and receive a response, settings tab renders without errors, and basic page structure is correct after login. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * feat(settings): migrate channel credentials from plaintext settings to encrypted secrets store Moves nearai.session_token from the plaintext DB settings table to the AES-256-GCM encrypted secrets store (key: nearai_session_token). - SessionManager gains an `attach_secrets()` method that wires in the secrets store; `save_session` writes to it when available and `load_session_from_secrets` is called preferentially over settings - `migrate_session_credential()` runs idempotently on each startup in `init_secrets()`, reading the JSON session from settings, writing it to secrets, then deleting the plaintext copy - Wizard's `persist_session_to_db` now writes to secrets first, falling back to plaintext settings only when secrets store is unavailable - Plaintext settings path is preserved as fallback for installs without a secrets store (no master key configured) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(settings): settings fallback only when no secrets store, verify decryption before deleting plaintext Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): ROLLBACK in libSQL migrate_default_owner, shared OwnershipCache across channels, add dynamic_tools to migration, fix doc comment - libSQL migrate_default_owner: wrap UPDATE loop in async closure + match to emit ROLLBACK on any mid-transaction failure (mirroring approve_pairing pattern) - Both backends: add dynamic_tools to the migrate_default_owner table list so agent-built tools are migrated on first pairing - setup_wasm_channels: accept Arc<OwnershipCache> parameter instead of allocating a fresh cache, share the AppComponents cache - SignalChannel::new: accept Arc<OwnershipCache> parameter and pass it to PairingStore instead of allocating a new cache - PairingStore: fix module-level and struct-level doc comments to accurately describe lazy cache population after approve() Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(web): use can_act_on for authorization in job/routine handlers instead of raw string comparisons Replace 12 raw `user_id != user.user_id` / `user_id == user.user_id` string comparisons in jobs.rs and 4 in routines.rs with calls through the canonical `can_act_on` function from `crate::ownership`, which is the spec-mandated authorization mechanism. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore: include remaining modified files in ownership model branch * fix: add pairing_store field to test GatewayState initializers, update PairingStore API calls in integration tests Add missing `pairing_store: None` to all GatewayState struct initializers in test files. Migrate old file-based PairingStore API calls (PairingStore::new(), PairingStore::with_base_dir()) to the new DB-backed API (PairingStore::new_noop()). Rewrite pairing_integration.rs to use LibSqlBackend with the new async DB-backed PairingStore API. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * chore: cargo fmt * fix(pairing): truly no-op PairingStore noop mode, ensure owner user in CLI, fix signal safety comments - PairingStore::upsert_request now returns a dummy record in noop mode instead of erroring, and approve silently succeeds (matching the doc promise of "writes are silently discarded"). - PairingStore::approve now accepts a channel parameter, matching the updated DB trait signature and propagated to all call sites (CLI, web server, tests). - CLI run_pairing_command ensures the owner user row exists before approval to satisfy the FK constraint on channel_identities.owner_id. - Signal channel block_in_place safety comments corrected from "WASM channel callbacks" to "Signal channel message processing". Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(pairing): thread channel through approve_pairing, add created flag, retry on code collision, remove redundant indexes Addresses PR review comments: - approve_pairing validates code belongs to the given channel - PairingRequestRecord.created replaces timing heuristic - upsert retries on UNIQUE violation (up to 3 attempts) - redundant indexes removed (UNIQUE creates implicit index) Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(ownership): migrate api_tokens, serialize PG approvals, propagate resolved owner_id Addresses PR review P1/P2 regressions: - api_tokens included in migrate_default_owner (both backends) - PostgreSQL approve_pairing uses FOR UPDATE to prevent concurrent approvals - Signal resolve_sender_identity returns owner_id, set as IncomingMessage.user_id with raw phone number preserved as sender_id for reply routing - Feishu uses resolved owner_id from pairing_resolve_identity in emitted message - PairingStore noop mode logs warning when pairing admission is impossible [skip-regression-check] Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * fix(pr-review): sanitize DB errors in pairing handlers, fix doc comments, add TODO for derive_activation_status - Pairing list/approve handlers no longer leak DB error details to clients - NotFound errors return user-friendly 'Invalid or expired pairing code' message - Module doc in pairing/store.rs corrected (remove -> evict, no insert method) - wit_compat.rs stub comment corrected to match actual Val shape - TODO added for derive_activation_status has_paired approximation * fix(pr-review): propagate libSQL query errors in approve_pairing, round-trip validate session credential migration, fix test doc comment - libSQL approve_pairing: .ok().flatten() replaced with .map_err() to propagate DB errors - migrate_session_credential: round-trip compares decrypted secret against plaintext before deleting - ownership_integration.rs: doc comment corrected to match actual test coverage * fix(pairing): store meta, wrap upserts in transactions, case-insensitive role/channel, log Signal DB errors, use auth role in handlers - Store meta JSONB/TEXT column in pairing_requests (PG migration V18, libSQL schema + incremental migration 19) - Wrap upsert_pairing_request in transactions (PG: client.transaction(), libSQL: BEGIN IMMEDIATE/COMMIT/ROLLBACK) - Case-insensitive role parsing: eq_ignore_ascii_case("admin") in both backends - Case-insensitive channel matching in approve_pairing: LOWER(channel) = LOWER($2) - Log DB errors in Signal resolve_sender_identity instead of silently discarding - Use auth role from UserIdentity in web handlers (jobs.rs, routines.rs) via identity_from_auth helper - Fix variable shadowing: rename `let channel` to `let req_channel` in libsql approve_pairing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(security): add auth to pairing list, cache eviction on deactivate, runtime assert in Signal, remove default fallback, warn on noop pairing codes Addresses zmanian's review: - #1: pairing_list_handler requires AuthenticatedUser - #2: OwnershipCache.evict_user() evicts all entries for a user on suspension - #3: debug_assert! for multi-thread runtime in Signal block_in_place - #9: Noop PairingStore warns when generating unredeemable codes - #10: cli/mcp.rs default fallback replaced with <unset> * fix(pairing): consistent LOWER() channel matching in resolve_channel_identity, fix wizard doc comment, fix E2E test assertion for ActionResponse convention * fix(pairing): apply LOWER() consistently across all ChannelPairingStore queries (upsert, list_pending, remove) All channel matching now uses LOWER() in both PostgreSQL and libSQL backends: - upsert_pairing_request: WHERE LOWER(channel) = LOWER($1) - list_pending_pairings: WHERE LOWER(channel) = LOWER($1) - remove_channel_identity: WHERE LOWER(channel) = LOWER($1) Previously only resolve_channel_identity and approve_pairing used LOWER(), causing inconsistent matching when channel names differed by case. * fix(pairing): unify code challenge flow and harden web pairing * test: harden pairing review follow-ups * fix: guard wasm pairing callbacks by runtime flavor * fix(pairing): normalize channel keys and serialize pg upserts * chore(web): clean up ownership review follow-ups * Preserve WASM pairing allowlist compatibility --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
* Fix turn cost footer and per-turn usage accounting * Avoid panicking on poisoned turn usage mutex * Tighten SSE usage regression coverage * Report usage for interrupted turns
…tags (#1952) * fix(llm): invert reasoning default — unknown models skip <think>/<final> injection When NEAR AI model="auto" resolves server-side to Qwen 3.5, the system prompt injected <think>/<final> tags because "auto" didn't match any known native-thinking pattern. This caused empty responses: 1. Qwen 3.5's native thinking puts reasoning in a `reasoning` field (not `reasoning_content`) — silently dropped due to field name mismatch 2. Content contained only <think> tags or <tool_call> XML, which clean_response() stripped to empty → "I'm not sure how to respond" Three fixes: - Invert the default: new requires_think_final_tags() with empty allowlist means unknown/alias models get the safe direct-answer prompt - Add #[serde(alias = "reasoning")] so vLLM's field name is accepted - Update active_model from API response.model so capability checks use the resolved model name after the first call Confirmed via direct API testing against NEAR AI staging with Qwen/Qwen3.5-122B-A10B. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * remove model alias resolution from nearai_chat auto should stay as the active model name — no reason to overwrite it with the resolved model since requires_think_final_tags() returns false for both "auto" and the resolved name. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix wording: remove native-thinking assumption from direct-answer prompt The direct-answer prompt is now the default for all models, not just native-thinking ones. Remove misleading "handled natively" language. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix bootstrap ownership migration for dynamic tools * Add bootstrap ownership regression coverage
* feat(embeddings): add bedrock provider * refactor(embeddings): address review feedback * fix: CI failures and review feedback for Bedrock embeddings - Replace ENV_MUTEX.lock() with lock_env() to match staging's test pattern - Use db_first_or_default() for non-bedrock model resolution (staging API) - Validate returned embedding dimension matches configured dimension Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Illia Polosukhin <ilblackdragon@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…4226 chore: promote staging to staging-promote/63a48e4e-24182836482 (2026-04-09 10:24 UTC)
…6482 chore: promote staging to staging-promote/288fe49a-24110798843 (2026-04-09 09:26 UTC)
…8843 chore: promote staging to staging-promote/79c1b0fd-24108317021 (2026-04-08 00:16 UTC)
…7021 chore: promote staging to staging-promote/86c15903-24100112892 (2026-04-07 22:53 UTC)
…2892 chore: promote staging to staging-promote/00fd2e88-24092158668 (2026-04-07 19:23 UTC)
…8668 chore: promote staging to staging-promote/f765958f-24078644272 (2026-04-07 16:22 UTC)
…4272 chore: promote staging to staging-promote/13774cc0-24076446124 (2026-04-07 11:19 UTC)
…3070 chore: promote staging to staging-promote/13c458e3-24192916101 (2026-04-09 15:30 UTC)
…6124 chore: promote staging to staging-promote/6fa2d0ec-24071779175 (2026-04-07 10:20 UTC)
…9175 chore: promote staging to staging-promote/0ab1a474-24064762647 (2026-04-07 08:24 UTC)
…2647 chore: promote staging to staging-promote/5d6a247d-24057733730 (2026-04-07 04:46 UTC)
…3730 chore: promote staging to staging-promote/8b629851-24041939370 (2026-04-07 00:16 UTC)
…9370 chore: promote staging to staging-promote/9cf37364-24039632441 (2026-04-06 17:15 UTC)
…2441 chore: promote staging to staging-promote/d0096dfc-24035427523 (2026-04-06 16:13 UTC)
…7523 chore: promote staging to staging-promote/f9ed8152-24023233420 (2026-04-06 14:19 UTC)
…3420 chore: promote staging to staging-promote/13852ff5-24021660555 (2026-04-06 07:31 UTC)
…0555 chore: promote staging to staging-promote/5083aed4-24002418644 (2026-04-06 06:33 UTC)
…8644 chore: promote staging to staging-promote/733678dd-23996777140 (2026-04-05 13:23 UTC)
…7140 chore: promote staging to staging-promote/e1695914-23995878265 (2026-04-05 07:24 UTC)
…8265 chore: promote staging to staging-promote/f3036388-23995135957 (2026-04-05 06:24 UTC)
…8795 chore: promote staging to staging-promote/4c9a985b-23931806540 (2026-04-03 05:32 UTC)
github-actions
Bot
added
scope: secrets
2 tasks
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Collaborator
CI Failures — Follow-up TrackingTwo CI checks are failing. These are not code bugs — the underlying changes are already validated on staging. Plan is to force-merge this PR and fix in a follow-up PR. 1. Version Bump Check (7 extensions)WASM extension source code changed but registry versions weren't bumped. Required bumps:
Files to edit:
2. No Panics in Production Code (9 violations)The script flagged
Likely cause: Action: Force-merge this PR, then open a follow-up PR with version bumps + panics check fix. |
4 tasks
theredspoon
pushed a commit
to theredspoon/ironclaw
that referenced
this pull request
Jun 21, 2026
…3875318792 chore: promote staging to staging-promote/761efac9-23780941831 (2026-04-01 23:10 UTC)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
17 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Auto-promotion from staging CI
Batch range:
420f165a81a5fc65ecb510664bf894028b5d21e9..9c6d8cb3476c91e24d5243d4e84833a6c4280319Promotion branch:
staging-promote/9c6d8cb3-23875318792Base:
staging-promote/42623ed1-23780941831Triggered by: Staging CI batch at 2026-04-01 23:10 UTC
Commits in this batch (6):
Current commits in this promotion (96)
Current base:
mainCurrent head:
staging-promote/9c6d8cb3-23875318792Current range:
origin/main..origin/staging-promote/9c6d8cb3-23875318792Auto-updated by staging promotion metadata workflow
Waiting for gates:
Auto-created by staging-ci workflow