Skip to content

Onboarding: select bundled Telegram channel and auto-install - #7

Closed
serrrfirat wants to merge 3 commits into
nearai:mainfrom
serrrfirat:claude/review-feature-parity-p1s-5keuh
Closed

serrrfirat wants to merge 3 commits into
nearai:mainfrom
serrrfirat:claude/review-feature-parity-p1s-5keuh

Conversation

@serrrfirat

Copy link
Copy Markdown
Collaborator

Summary

  • Bundled WASM channels: Adds bundled.rs module that embeds the Telegram WASM channel binary, allowing it to be installed without downloading externally
  • Onboarding wizard improvements: The setup wizard now shows bundled channels (e.g., Telegram) as selectable options even when not yet installed, and auto-installs them on selection
  • UrlPath credential location: New UrlPath variant for credential injection, supporting URL placeholder replacement (e.g., {TELEGRAM_BOT_TOKEN} in API URLs)

Changes

  • src/channels/wasm/bundled.rs — New module: bundled_channel_names(), install_bundled_channel() with include_bytes! for telegram WASM + capabilities
  • src/setup/wizard.rs — Refactored channel selection to merge discovered + bundled channels, auto-install selected bundled channels, and run migrations before channel setup
  • src/tools/wasm/capabilities_schema.rs — Added UrlPath credential location variant with JSON parsing support
  • src/tools/wasm/credential_injector.rs — Handle UrlPath in injection (deferred to channel/tool wrappers)
  • src/secrets/types.rs — Added UrlPath variant to CredentialLocation enum

Test plan

  • Verify cargo test passes for new tests in bundled.rs, wizard.rs, and capabilities_schema.rs
  • Verify bundled Telegram channel installs correctly during onboarding wizard
  • Verify UrlPath credential location parses from capabilities JSON

🤖 Generated with Claude Code

claude and others added 3 commits February 7, 2026 11:01
Adds bidirectional WebSocket transport to the web gateway alongside
the existing SSE stream. Clients can send messages, approvals, and
pings over a single persistent connection at /api/chat/ws.

- Enable axum `ws` feature for built-in WebSocket support
- Add WsClientMessage/WsServerMessage types with tagged JSON protocol
- Add subscribe_raw() to SseManager for non-SSE consumers
- Create ws.rs with connection handler (split sender/receiver tasks)
- Add WsConnectionTracker for active connection counting
- Add /api/gateway/status control plane endpoint (SSE + WS counts)
- 35 new tests covering message types, broadcast, and handler logic

https://claude.ai/code/session_01KEaLN6Xq2j5EeV3SGHQT6b
- Add tokio-tungstenite dev-dependency for WebSocket client in tests
- Update start_server to return actual bound SocketAddr (enables port 0)
- Add 10 e2e tests covering full HTTP upgrade → WebSocket → message flow:
  ping/pong, message routing to agent, broadcast event delivery,
  connection tracking, invalid message handling, auth rejection,
  gateway status endpoint, and multi-event sequencing

https://claude.ai/code/session_01KEaLN6Xq2j5EeV3SGHQT6b
@serrrfirat serrrfirat closed this Feb 7, 2026
ilblackdragon added a commit that referenced this pull request Feb 19, 2026
- Use manifest.name (not crate_name) for installed filenames so
  discovery, auth, and CLI commands all agree on the stem (#1)
- Add AlreadyInstalled error variant instead of misleading
  ExtensionNotFound (#2)
- Add DownloadFailed error variant with URL context instead of
  stuffing URLs into PathBuf (#3)
- Validate HTTP status with error_for_status() before reading
  response bytes in artifact downloads (#4)
- Switch build_wasm_component to tokio::process::Command with
  status() so build output streams to the terminal (#6)
- Find WASM artifact by crate_name specifically instead of picking
  the first .wasm file in the release directory (#7)
- Add is_file() guard in catalog loader to skip directories (#8)
- Detect ambiguous bare-name lookups when both tools/<name> and
  channels/<name> exist, with get_strict() returning an error (#9)
- Fix wizard step_extensions to check tool.name for installed
  detection, consistent with the new naming (#11, #12)
- Fix redundant closures and map_or clippy warnings in changed files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Feb 20, 2026
- Use manifest.name (not crate_name) for installed filenames so
  discovery, auth, and CLI commands all agree on the stem (#1)
- Add AlreadyInstalled error variant instead of misleading
  ExtensionNotFound (#2)
- Add DownloadFailed error variant with URL context instead of
  stuffing URLs into PathBuf (#3)
- Validate HTTP status with error_for_status() before reading
  response bytes in artifact downloads (#4)
- Switch build_wasm_component to tokio::process::Command with
  status() so build output streams to the terminal (#6)
- Find WASM artifact by crate_name specifically instead of picking
  the first .wasm file in the release directory (#7)
- Add is_file() guard in catalog loader to skip directories (#8)
- Detect ambiguous bare-name lookups when both tools/<name> and
  channels/<name> exist, with get_strict() returning an error (#9)
- Fix wizard step_extensions to check tool.name for installed
  detection, consistent with the new naming (#11, #12)
- Fix redundant closures and map_or clippy warnings in changed files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Feb 20, 2026
…tion (#238)

* feat: add extension registry with metadata catalog, CLI, and onboarding integration

Adds a central registry that catalogs all 14 available extensions (10 tools,
4 channels) with their capabilities, auth requirements, and artifact references.
The onboarding wizard now shows installable channels from the registry and
offers tool installation as a new Step 7.

- registry/ folder with per-extension JSON manifests and bundle definitions
- src/registry/ module: manifest structs, catalog loader, installer
- `ironclaw registry list|info|install|install-defaults` CLI commands
- Setup wizard enhanced: channels from registry, new extensions step (8 steps)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(setup): resolve workspace errors for tool crates and channels-only onboarding

Tool crates in tools-src/ and channels-src/ failed `cargo metadata` during
onboard install because Cargo resolved them as part of the root workspace.
Add `[workspace]` table to each standalone crate and extend the root
`workspace.exclude` list so they build independently.

Channels-only mode (`onboard --channels-only`) failed with "Secrets not
configured" and "No database connection" because it skipped database and
security setup. Add `reconnect_existing_db()` to establish the DB connection
and load saved settings before running channel configuration.

Also improve the tunnel "already configured" display to show full provider
details (domain, mode, command) instead of just the provider name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(registry): address PR review feedback on installer and catalog

- Use manifest.name (not crate_name) for installed filenames so
  discovery, auth, and CLI commands all agree on the stem (#1)
- Add AlreadyInstalled error variant instead of misleading
  ExtensionNotFound (#2)
- Add DownloadFailed error variant with URL context instead of
  stuffing URLs into PathBuf (#3)
- Validate HTTP status with error_for_status() before reading
  response bytes in artifact downloads (#4)
- Switch build_wasm_component to tokio::process::Command with
  status() so build output streams to the terminal (#6)
- Find WASM artifact by crate_name specifically instead of picking
  the first .wasm file in the release directory (#7)
- Add is_file() guard in catalog loader to skip directories (#8)
- Detect ambiguous bare-name lookups when both tools/<name> and
  channels/<name> exist, with get_strict() returning an error (#9)
- Fix wizard step_extensions to check tool.name for installed
  detection, consistent with the new naming (#11, #12)
- Fix redundant closures and map_or clippy warnings in changed files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(setup): restore DB connection fields after settings reload

reconnect_postgres() and reconnect_libsql() called Settings::from_db_map()
which overwrote database_url / libsql_path / libsql_url set from env vars.
Also use get_strict() in cmd_info to surface ambiguous bare-name errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix clippy collapsible_if and print_literal warnings

Collapse nested if-let chains and inline string literals in format
macros to satisfy CI clippy lint checks (deny warnings).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(registry): prefer artifacts for install-defaults and improve dir lookup

- InstallDefaults now defaults to downloading pre-built artifacts
  (matching `registry install` behavior), with --build flag for source builds.
- find_registry_dir() walks up 3 ancestor levels from the exe and adds
  a CARGO_MANIFEST_DIR fallback, matching load_registry_catalog() logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
jaswinder6991 pushed a commit to jaswinder6991/ironclaw that referenced this pull request Feb 26, 2026
…tion (nearai#238)

* feat: add extension registry with metadata catalog, CLI, and onboarding integration

Adds a central registry that catalogs all 14 available extensions (10 tools,
4 channels) with their capabilities, auth requirements, and artifact references.
The onboarding wizard now shows installable channels from the registry and
offers tool installation as a new Step 7.

- registry/ folder with per-extension JSON manifests and bundle definitions
- src/registry/ module: manifest structs, catalog loader, installer
- `ironclaw registry list|info|install|install-defaults` CLI commands
- Setup wizard enhanced: channels from registry, new extensions step (8 steps)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(setup): resolve workspace errors for tool crates and channels-only onboarding

Tool crates in tools-src/ and channels-src/ failed `cargo metadata` during
onboard install because Cargo resolved them as part of the root workspace.
Add `[workspace]` table to each standalone crate and extend the root
`workspace.exclude` list so they build independently.

Channels-only mode (`onboard --channels-only`) failed with "Secrets not
configured" and "No database connection" because it skipped database and
security setup. Add `reconnect_existing_db()` to establish the DB connection
and load saved settings before running channel configuration.

Also improve the tunnel "already configured" display to show full provider
details (domain, mode, command) instead of just the provider name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(registry): address PR review feedback on installer and catalog

- Use manifest.name (not crate_name) for installed filenames so
  discovery, auth, and CLI commands all agree on the stem (nearai#1)
- Add AlreadyInstalled error variant instead of misleading
  ExtensionNotFound (nearai#2)
- Add DownloadFailed error variant with URL context instead of
  stuffing URLs into PathBuf (nearai#3)
- Validate HTTP status with error_for_status() before reading
  response bytes in artifact downloads (nearai#4)
- Switch build_wasm_component to tokio::process::Command with
  status() so build output streams to the terminal (nearai#6)
- Find WASM artifact by crate_name specifically instead of picking
  the first .wasm file in the release directory (nearai#7)
- Add is_file() guard in catalog loader to skip directories (nearai#8)
- Detect ambiguous bare-name lookups when both tools/<name> and
  channels/<name> exist, with get_strict() returning an error (nearai#9)
- Fix wizard step_extensions to check tool.name for installed
  detection, consistent with the new naming (nearai#11, nearai#12)
- Fix redundant closures and map_or clippy warnings in changed files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(setup): restore DB connection fields after settings reload

reconnect_postgres() and reconnect_libsql() called Settings::from_db_map()
which overwrote database_url / libsql_path / libsql_url set from env vars.
Also use get_strict() in cmd_info to surface ambiguous bare-name errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix clippy collapsible_if and print_literal warnings

Collapse nested if-let chains and inline string literals in format
macros to satisfy CI clippy lint checks (deny warnings).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(registry): prefer artifacts for install-defaults and improve dir lookup

- InstallDefaults now defaults to downloading pre-built artifacts
  (matching `registry install` behavior), with --build flag for source builds.
- find_registry_dir() walks up 3 ancestor levels from the exe and adds
  a CARGO_MANIFEST_DIR fallback, matching load_registry_catalog() logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Mar 6, 2026
- Switch build script from python3 to jq for JSON parsing, consistent
  with release.yml and avoids python3 dependency (#1, #7)
- Use dirs::home_dir() instead of HOME env var for portability (#2)
- Filter extensions by manifest "kind" field instead of path (#3)
- Replace .flatten() with explicit error handling in dir iteration (#4, #5)
- Split stub_tool_host_functions into stub_shared_host_functions +
  tool-only tool-invoke stub, since tool-invoke is not in channel WIT (#6)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Mar 6, 2026
* test: add WIT compatibility tests for all WASM tools and channels

Adds CI and integration tests to catch WIT interface breakage across
all 14 WASM extensions (10 tools + 4 channels). Previously, changing
wit/tool.wit or wit/channel.wit could silently break guest-side tools
that weren't rebuilt until release time.

Three new pieces:

1. scripts/build-wasm-extensions.sh — builds all WASM extensions from
   source by reading registry manifests. Used by CI and locally.

2. tests/wit_compat.rs — integration tests that compile and instantiate
   each .wasm binary against the current wasmtime host linker with
   stubbed host functions. Catches added/removed/renamed WIT functions,
   signature mismatches, and missing exports. Skips gracefully when
   artifacts aren't built so `cargo test` still passes standalone.

3. .github/workflows/test.yml — new wasm-wit-compat CI job that builds
   all extensions then runs instantiation tests on every PR. Added to
   the branch protection roll-up.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix rustfmt formatting in wit_compat tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback on WIT compat tests

- Switch build script from python3 to jq for JSON parsing, consistent
  with release.yml and avoids python3 dependency (#1, #7)
- Use dirs::home_dir() instead of HOME env var for portability (#2)
- Filter extensions by manifest "kind" field instead of path (#3)
- Replace .flatten() with explicit error handling in dir iteration (#4, #5)
- Split stub_tool_host_functions into stub_shared_host_functions +
  tool-only tool-invoke stub, since tool-invoke is not in channel WIT (#6)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Mar 7, 2026
…lity

Security fixes:
- Remove SSRF-prone download() from DocumentExtractionMiddleware (#13)
- Sanitize filenames in workspace path to prevent directory traversal (#11)
- Pre-check file size before reading in WASM wrapper to prevent OOM (#2)
- Percent-encode file_id in Telegram source URLs (#7)

Correctness fixes:
- Clear image_content_parts on turn end to prevent memory leak (#1)
- Find first *successful* transcription instead of first overall (#3)
- Enforce data.len() size limit in document extraction (#10)
- Use UTF-8 safe truncation with char_indices() (#12)

Robustness & code quality:
- Add 120s timeout to OpenAI Whisper HTTP client (#5)
- Trim trailing slash from Whisper base_url (#6)
- Allow ~/.ironclaw/ paths in WASM wrapper (#8)
- Return error from on_broadcast in Slack/Discord/WhatsApp (#9)
- Fix doc comment in HTTP tool (#4)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Mar 7, 2026
* feat: add inbound attachment support to WASM channel system

Add attachment record to WIT interface and implement inbound media
parsing across all four channel implementations (Telegram, Slack,
WhatsApp, Discord). Attachments flow from WASM channels through
EmittedMessage to IncomingMessage with validation (size limits,
MIME allowlist, count caps) at the host boundary.

- Add `attachment` record to `emitted-message` in wit/channel.wit
- Add `IncomingAttachment` struct to channel.rs and re-export
- Add host-side validation (20MB total, 10 max, MIME allowlist)
- Telegram: parse photo, document, audio, video, voice, sticker
- Slack: parse file attachments with url_private
- WhatsApp: parse image, audio, video, document with captions
- Discord: backward-compatible empty attachments
- Update FEATURE_PARITY.md section 7
- Add fixture-based tests per channel and host integration tests

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: integrate outbound attachment support and reconcile WIT types (#409)

Reconcile PR #409's outbound attachment work with our inbound attachment
support into a unified design:

WIT type split:
- `inbound-attachment` in channel-host: metadata-only (id, mime_type,
  filename, size_bytes, source_url, storage_key, extracted_text)
- `attachment` in channel: raw bytes (filename, mime_type, data) on
  agent-response for outbound sending

Outbound features (from PR #409):
- `on-broadcast` WIT export for proactive messages without prior inbound
- Telegram: multipart sendPhoto/sendDocument with auto photo→document
  fallback for files >10MB
- wrapper.rs: `call_on_broadcast`, `read_attachments` from disk,
  attachment params threaded through `call_on_respond`
- HTTP tool: `save_to` param for binary downloads to /tmp/ (50MB limit,
  path traversal protection, SSRF-safe redirect following)
- Message tool: allow /tmp/ paths for attachments alongside base_dir
- Credential env var fallback in inject_channel_credentials

Channel updates:
- All 4 channels implement on_broadcast (Telegram full, others stub)
- Telegram: polling_enabled config, adjusted poll timeout
- Inbound attachment types renamed to InboundAttachment in all channels

Tests: 1965 passing (9 new), 0 clippy warnings

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add audio transcription pipeline and extensible WIT attachment design

Add host-side transcription middleware (OpenAI Whisper) that detects audio
attachments with inline data on incoming messages and transcribes them
automatically. Refactor WIT inbound-attachment to use extras-json and a
store-attachment-data host function instead of typed fields, so future
attachment properties (dimensions, codec, etc.) don't require WIT changes
that invalidate all channel plugins.

- Add src/transcription/ module: TranscriptionProvider trait,
  TranscriptionMiddleware, AudioFormat enum, OpenAI Whisper provider
- Add src/config/transcription.rs: TRANSCRIPTION_ENABLED/MODEL/BASE_URL
- Wire middleware into agent message loop via AgentDeps
- WIT: replace data + duration-secs with extras-json + store-attachment-data
- Host: parse extras-json for well-known keys, merge stored binary data
- Telegram: download voice files via store-attachment-data, add duration
  to extras-json, add /file/bot to HTTP allowlist, voice-only placeholder
- Add reqwest multipart feature for Whisper API uploads
- 5 regression tests for transcription middleware

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: wire attachment processing into LLM pipeline with multimodal image support

Attachments on incoming messages are now augmented into user text via XML tags
before entering the turn system, and images with data are passed as multimodal
content parts (base64 data URIs) to LLM providers. This enables audio transcripts,
document text, and image content to reach the LLM without changes to ChatMessage
serialization or provider interfaces.

- Add src/agent/attachments.rs with augment_with_attachments() and 9 unit tests
- Add ContentPart/ImageUrl types to llm::provider with OpenAI-compatible serde
- Carry image_content_parts transiently on Turn (skipped in serialization)
- Update nearai_chat and rig_adapter to serialize multimodal content
- Add 3 e2e tests verifying attachments flow through the full agent loop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, version bumps, and Telegram voice test

- Fix cargo fmt formatting in attachments.rs, nearai_chat.rs, rig_adapter.rs,
  e2e_attachments.rs
- Bump channel registry versions 0.1.0 → 0.2.0 (discord, slack, telegram,
  whatsapp) to satisfy version-bump CI check
- Fix Telegram test_extract_attachments_voice: add missing required `duration`
  field to voice fixture JSON

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: bump WIT channel version to 0.3.0, fix Telegram voice test, add pre-commit hook

- Bump wit/channel.wit package version 0.2.0 → 0.3.0 (interface changed with
  store-attachment-data)
- Update WIT_CHANNEL_VERSION constant and registry wit_version fields to match
- Fix Telegram test_extract_attachments_voice: gate voice download behind
  #[cfg(target_arch = "wasm32")] so host functions aren't called in native tests,
  update assertions for generated filename and extras_json duration
- Add @0.3.0 linker stubs in wit_compat.rs
- Add .githooks/pre-commit hook that runs scripts/check-version-bumps.sh when
  WIT or extension sources are staged
- Symlink commit-msg regression hook into .githooks/

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract voice download from extract_attachments into handle_message

Move download_voice_file + store_attachment_data calls out of
extract_attachments into a separate download_and_store_voice function
called from handle_message. This keeps extract_attachments as a pure
data-mapping function with no host calls, making it fully testable
in native unit tests without #[cfg(target_arch)] gates.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Add path validation to read_attachments (restrict to /tmp/) preventing
  arbitrary file reads from compromised tools
- Escape XML special characters in attachment filenames, MIME types, and
  extracted text to prevent prompt injection via tag spoofing
- Percent-encode file_id in Telegram getFile URL to prevent query injection
- Clone SecretString directly instead of expose_secret().to_string()

Correctness fixes:
- Fix store_attachment_data overwrite accounting: subtract old entry size
  before adding new to prevent inflated totals and false rejections
- Use max(reported, stored_size) for attachment size accounting to prevent
  WASM channels from under-reporting size_bytes to bypass limits
- Add application/octet-stream to MIME allowlist (channels default unknown
  types to this)

Code quality:
- Extract send_response helper in Telegram, deduplicating on_respond and
  on_broadcast
- Rename misleading Discord test to test_parse_slash_command_interaction
- Fix .githooks/commit-msg to use relative symlink (portable across machines)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add tool_upgrade command + fix TOCTOU in save_to path validation

Add `tool_upgrade` — a new extension management tool that automatically
detects and reinstalls WASM extensions with outdated WIT versions.
Preserves authentication secrets during upgrade. Supports upgrading a
single extension by name or all installed WASM tools/channels at once.

Fix TOCTOU in `validate_save_to_path`: validate the path *before*
creating parent directories, so traversal paths like `/tmp/../../etc/`
cannot cause filesystem mutations outside /tmp before being rejected.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: unify WIT package version to 0.3.0 across tool.wit and all capabilities

tool.wit and channel.wit share the `near:agent` package namespace, so they
must declare the same version. Bumps tool.wit from 0.2.0 to 0.3.0 and
updates all capabilities files and registry entries to match.

Fixes `cargo component build` failure: "package identifier near:agent@0.2.0
does not match previous package name of near:agent@0.3.0"

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move WIT file comments after package declaration

WIT treats `//` comments before `package` as doc comments. When both
tool.wit and channel.wit had header comments, the parser rejected them
as "doc comments on multiple 'package' items". Move comments after the
package declaration in both files.

Also bumps tool registry versions to 0.2.0 to match the WIT 0.3.0 bump.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: display extension versions in gateway Extensions tab

Add version field to InstalledExtension and RegistryEntry types, pipe
through the web API (ExtensionInfo, RegistryEntryInfo), and render as
a badge in the gateway UI for both installed and available extensions.

For installed WASM extensions, version is read from the capabilities
file with a fallback to the registry entry when the local file has no
version (old installations). Bump all extension Cargo.toml and registry
JSON versions from 0.1.0 to 0.2.0 to keep them in sync.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add document text extraction middleware for PDF, Office, and text files

Extract text from document attachments (PDF, DOCX, PPTX, XLSX, RTF, plain text,
code files) so the LLM can reason about uploaded documents. Uses pdf-extract for
PDFs, zip+XML parsing for Office XML formats, and UTF-8 decode for text files.
Wired into the agent loop after transcription middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: download document files in Telegram channel for text extraction

The DocumentExtractionMiddleware needs file bytes in the attachment `data`
field, but only voice files were being downloaded. Document attachments
(PDFs, DOCX, etc.) had empty `data` and a source_url with a credential
placeholder that only works inside the WASM host's http_request.

Add `download_and_store_documents()` that downloads non-voice, non-image,
non-audio attachments via the existing two-step getFile→download flow and
stores bytes via `store_attachment_data` for host-side extraction.

Also rename `download_voice_file` → `download_telegram_file` since it's
generic for any file_id.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: allow Office MIME types and increase file download limit for Telegram

Two issues preventing document extraction from Telegram:

1. PPTX/DOCX/XLSX MIME types (application/vnd.*) were dropped by the
   WASM host attachment allowlist — add application/vnd., application/msword,
   and application/rtf prefixes.

2. Telegram file downloads over 10 MB failed with "Response body too large" —
   set max_response_bytes to 20 MB in Telegram capabilities.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: report document extraction errors back to user instead of silently skipping

- Bump max_response_bytes to 50 MB for Telegram file downloads
- When document extraction fails (too large, download error, parse error),
  set extracted_text to a user-friendly error message instead of leaving it
  None. This ensures the LLM tells the user what went wrong.
- On Telegram download failure, set extracted_text with the error so the
  user sees feedback even when the file never reaches the extraction middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: store extracted document text in workspace memory for search/recall

After document extraction succeeds, write the extracted text to workspace
memory at `documents/{date}/{filename}`. This enables:
- Full-text and semantic search over past uploaded documents
- Cross-conversation recall ("what did that PDF say?")
- Automatic chunking and embedding via the workspace pipeline

Documents are stored with metadata header (uploader, channel, date, MIME type).
Error messages (extraction failures) are not stored — only successful extractions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, unused assignment warning

- Run cargo fmt on document_extraction and agent_loop modules
- Suppress unused_assignments warning on trace_llm_ref (used only
  behind #[cfg(feature = "libsql")])

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Remove SSRF-prone download() from DocumentExtractionMiddleware (#13)
- Sanitize filenames in workspace path to prevent directory traversal (#11)
- Pre-check file size before reading in WASM wrapper to prevent OOM (#2)
- Percent-encode file_id in Telegram source URLs (#7)

Correctness fixes:
- Clear image_content_parts on turn end to prevent memory leak (#1)
- Find first *successful* transcription instead of first overall (#3)
- Enforce data.len() size limit in document extraction (#10)
- Use UTF-8 safe truncation with char_indices() (#12)

Robustness & code quality:
- Add 120s timeout to OpenAI Whisper HTTP client (#5)
- Trim trailing slash from Whisper base_url (#6)
- Allow ~/.ironclaw/ paths in WASM wrapper (#8)
- Return error from on_broadcast in Slack/Discord/WhatsApp (#9)
- Fix doc comment in HTTP tool (#4)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: formatting — cargo fmt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address latest PR review — doc comments, error messages, version bumps

- Fix DocumentExtractionMiddleware doc comment (no longer downloads from source_url)
- Fix error message: "no inline data" instead of "no download URL"
- Log error + fallback instead of silent unwrap_or_default on Whisper HTTP client
- Bump all capabilities.json versions from 0.1.0 to 0.2.0 to match Cargo.toml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unsupported profile: minimal from CI workflows [skip-regression-check]

dtolnay/rust-toolchain@stable does not accept the 'profile' input
(it was a parameter for the deprecated actions-rs/toolchain action).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: merge with latest main — resolve compilation errors and PR review nits

- Add version: None to RegistryEntry/InstalledExtension test constructors
- Fix MessageContent type mismatches in nearai_chat tests (String → MessageContent::Text)
- Fix .contains() calls on MessageContent — use .as_text().unwrap()
- Remove redundant trace_llm_ref = None assignment in test_rig
- Check data size before clone in document extraction to avoid unnecessary allocation

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Mar 25, 2026
GATEWAY_USER_TOKENS never went to production — replaced entirely by
DB-backed user management via /api/admin/users and /api/tokens.

Removed:
- UserTokenConfig struct and GATEWAY_USER_TOKENS env var parsing
- user_tokens field from GatewayConfig
- GatewayChannel::new_multi_auth() constructor
- Env-var user migration block in main.rs (~90 lines)
- multi_tenant auto-detection from GATEWAY_USER_TOKENS (now runtime
  via db.has_any_users() in app.rs)

Review fixes (zmanian):
- User ID generation: UUID instead of display-name derivation (#1)
- Invitation accept moved to public router (no auth needed) (#3)
- libSQL get_invitation_by_hash aligned with postgres: filters
  status='pending' AND expires_at > now (#4)
- UUID parse: returns DatabaseError::Serialization instead of
  unwrap_or_default (#7)
- PostgreSQL SELECT * replaced with explicit column lists (#8)
- Sort order aligned (both backends use DESC) (#6)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ilblackdragon added a commit that referenced this pull request Mar 28, 2026
…i-tenant isolation (#1626)

* feat: complete multi-tenant isolation — per-user budgets, model selection, heartbeat cycling

Finishes the remaining isolation work from phases 2–4 of #59:

Phase 2 (DB scoping): Fix /status and /list commands to use _for_user
DB variants instead of global queries that leaked cross-user job data.

Phase 3 (Runtime isolation): Per-user workspace in routine engine's
spawn_fire so lightweight routines run in the correct user context.
Per-user daily cost tracking in CostGuard with configurable budget via
MAX_COST_PER_USER_PER_DAY_CENTS. Multi-user heartbeat that cycles
through all users with routines, auto-detected from GATEWAY_USER_TOKENS.

Phase 4 (Provider/tools): Per-user model selection via preferred_model
setting — looked up from SettingsStore on first iteration, threaded
through ReasoningContext.model_override to CompletionRequest. Works
with providers that support per-request model overrides (NearAI).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use selected_model setting key to match /model command persistence

The dispatcher was reading "preferred_model" but the /model command
(merged from staging) persists to "selected_model". Since set_setting
is already per-user scoped, using the same key makes /model work as
the per-user model override in multi-tenant mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: heartbeat hygiene, /model multi-tenant guard, RigAdapter model override

Three follow-up fixes for multi-tenant isolation:

1. Multi-user heartbeat now runs memory hygiene per user before each
   heartbeat check, matching single-user heartbeat behavior.

2. /model command in multi-tenant mode only persists to per-user
   settings (selected_model) without calling set_model() on the shared
   LlmProvider. The per-request model_override in the dispatcher reads
   from the same setting. Added multi_tenant flag to AgentConfig
   (auto-detected from GATEWAY_USER_TOKENS).

3. RigAdapter now supports per-request model overrides by injecting the
   model name into rig-core's additional_params. OpenAI/Anthropic/Ollama
   API servers use last-key-wins for duplicate JSON keys, so the override
   takes effect via serde's flatten serialization order.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review — cost model attribution, heartbeat concurrency, pruning

Fixes from review comments on #1614:

- Cost tracking now uses the override model name (not active_model_name)
  when a per-user model override is active, for accurate attribution.
- Multi-user heartbeat runs per-user checks concurrently via JoinSet
  instead of sequentially, preventing one slow user from blocking others.
- Per-user failure counts tracked independently; users exceeding
  max_failures are skipped (matching single-user semantics).
- per_user_daily_cost HashMap pruned on day rollover to prevent
  unbounded growth in long-lived deployments.
- Doc comment fixed: says "routines" not "active routines".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: /status ownership, model persistence scoping, heartbeat robustness

Addresses second round of PR review on #1614:

- /status <job_id> DB path now validates job.user_id == requesting user
  before returning data (was missing ownership check, security fix).

- persist_selected_model takes user_id param instead of owner_id, and
  skips .env/TOML writes in multi-tenant mode (these are shared global
  files). handle_system_command now receives user_id from caller.

- JoinSet collection handles Err(JoinError) explicitly instead of
  silently dropping panicked tasks.

- Notification forwarder extracts owner_id from response metadata in
  multi-tenant mode for per-user routing instead of broadcasting to
  the agent owner.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: cost pricing, fire_manual workspace, heartbeat concurrency cap

Round 3 review fixes:

- Cost tracking passes None for cost_per_token when model override is
  active, letting CostGuard look up pricing by model name instead of
  using the default provider's rates (serrrfirat).

- fire_manual() now uses per-user workspace, matching spawn_fire()
  pattern (serrrfirat).

- Removed MULTI_TENANT env var — multi-tenant mode is auto-detected
  solely from GATEWAY_USER_TOKENS presence (serrrfirat + Copilot).

- Multi-user heartbeat capped at 8 concurrent tasks to avoid flooding
  the LLM provider (serrrfirat + Copilot).

- Fixed inject_model_override doc comment accuracy (Copilot).

- Added comment explaining multi-tenant notification routing priority
  (Copilot).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: user-scoped webhook endpoint for multi-tenant isolation

Adds POST /api/webhooks/u/{user_id}/{path} — a user-scoped webhook
endpoint that filters the routine lookup by user_id, preventing
cross-user webhook triggering when paths collide.

The existing /api/webhooks/{path} endpoint remains unchanged for
backward compatibility in single-user deployments.

Changes:
- get_webhook_routine_by_path gains user_id: Option<&str> param
- Both postgres and libsql implementations add AND user_id = ? filter
  when user_id is provided
- New webhook_trigger_user_scoped_handler extracts (user_id, path)
  from URL and passes to shared fire_webhook_inner logic
- Route registered on public router (webhooks are called by external
  services that can't send bearer tokens)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(db): add UserStore trait with users, api_tokens, invitations tables

Foundation for DB-backed user management (#1605):

- UserRecord, ApiTokenRecord, InvitationRecord types in db/mod.rs
- UserStore sub-trait (17 methods) added to Database supertrait
- PostgreSQL migration V14__users.sql (users, api_tokens, invitations)
- libSQL schema + incremental migration V14
- Full implementations for both PgBackend (via Store delegation) and
  LibSqlBackend (direct SQL in libsql/users.rs)
- authenticate_token JOINs api_tokens+users with active/non-revoked
  checks; has_any_users for bootstrap detection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): DB-backed auth, user/token/invitation API handlers

Adds the web gateway layer for DB-backed user management (#1605):

Auth refactor:
- CombinedAuthState wraps env-var tokens (MultiAuthState) + optional
  DbAuthenticator for DB-backed token lookup with LRU cache (60s TTL,
  1024 max entries)
- auth_middleware tries env-var tokens first, then DB fallback
- From<MultiAuthState> impl for backward compatibility
- main.rs wires with_db_auth when database is available

API handlers (12 new endpoints):
- /api/admin/users — CRUD: create, list, detail, update, suspend, activate
- /api/tokens — create (returns plaintext once), list, revoke
- /api/invitations — create, list, accept (creates user + first token)

Token creation: 32 random bytes → hex plaintext, SHA-256 hash stored.
Invitation accept: validates hash + pending + not expired, creates
user record and first API token atomically.

All test files updated for CombinedAuthState type change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: startup env-var user migration + UserStore integration tests

Completes the DB-backed user management feature (#1605):

- Startup migration: when GATEWAY_USER_TOKENS is set and the users
  table is empty, inserts env-var users + hashed tokens into DB.
  Logs deprecation notice when DB already has users.
- hash_token made pub for reuse in migration code.
- 10 integration tests for UserStore (libsql file-backed):
  - has_any_users bootstrap detection
  - create/get/get_by_email/list/update user lifecycle
  - token create → authenticate → revoke → reject cycle
  - suspended user tokens rejected
  - wrong-user token revoke returns false
  - invitation create → accept → user created
  - record_login and record_token_usage timestamps
- libSQL migration: removed FK constraints from V14 (incompatible
  with execute_batch inside transactions). Tables in both base SCHEMA
  and incremental migration for fresh and existing databases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove GATEWAY_USER_TOKENS, fix review feedback

GATEWAY_USER_TOKENS never went to production — replaced entirely by
DB-backed user management via /api/admin/users and /api/tokens.

Removed:
- UserTokenConfig struct and GATEWAY_USER_TOKENS env var parsing
- user_tokens field from GatewayConfig
- GatewayChannel::new_multi_auth() constructor
- Env-var user migration block in main.rs (~90 lines)
- multi_tenant auto-detection from GATEWAY_USER_TOKENS (now runtime
  via db.has_any_users() in app.rs)

Review fixes (zmanian):
- User ID generation: UUID instead of display-name derivation (#1)
- Invitation accept moved to public router (no auth needed) (#3)
- libSQL get_invitation_by_hash aligned with postgres: filters
  status='pending' AND expires_at > now (#4)
- UUID parse: returns DatabaseError::Serialization instead of
  unwrap_or_default (#7)
- PostgreSQL SELECT * replaced with explicit column lists (#8)
- Sort order aligned (both backends use DESC) (#6)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add role-based access control (admin/member)

Adds a `role` field (admin|member) to user management:

Schema:
- `role TEXT NOT NULL DEFAULT 'member'` added to users table in both
  PostgreSQL V14 migration and libSQL schema/incremental migration
- UserRecord gains `role: String` field
- UserIdentity gains `role: String` field, populated from DB in
  DbAuthenticator and defaulting to "admin" for single-user mode

Access control:
- AdminUser extractor: returns 403 Forbidden if role != "admin"
- /api/admin/users/* handlers: require AdminUser (create, list,
  detail, update, suspend, activate)
- POST /api/invitations: requires AdminUser (only admins can invite)
- User creation accepts optional "role" param (defaults to "member")
- Invitation acceptance creates users with "member" role

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): add Users admin tab to web UI

Adds a Users tab to the web gateway UI for managing users, tokens,
and roles without needing direct API calls.

Features:
- User list table with ID, name, email, role, status, created date
- Create user form with display name, email, role selector
- Suspend/activate actions per user
- Create API token for any user (shows plaintext once with copy button)
- Role badges (admin highlighted, member muted)
- Non-admin users see "Admin access required" message
- Keyboard shortcut: Cmd/Ctrl+5 switches to Users tab

CSS:
- Reuses routines-table styles for the user list
- Badge, token-display, btn-small, btn-danger, btn-primary components

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move Users to Settings subtab, bootstrap admin user on first run

- Moved Users from top-level tab to Settings sidebar subtab (under
  Skills, before Theme toggle)
- On first startup with empty users table, automatically creates an
  admin user from GATEWAY_USER_ID config with a corresponding API
  token from GATEWAY_AUTH_TOKEN. This ensures the owner appears in
  the Users panel immediately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: user creation shows token, + Token works, no password save popup

Three UI/UX fixes:

1. Create user now generates an initial API token and shows it in a
   copy-able banner instead of triggering the browser's password save
   dialog. Uses autocomplete="off" and type="text" for email field.

2. "+ Token" button works: exposed createTokenForUser/suspendUser/
   activateUser on window for inline onclick handlers in dynamically
   generated table rows. Token creation uses showTokenBanner helper.

3. Admin token creation: POST /api/tokens now accepts optional
   "user_id" field when the requesting user is admin, allowing
   token creation for other users from the Users panel.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use event delegation for user action buttons (CSP compliance)

Inline onclick handlers are blocked by the Content-Security-Policy
(script-src 'self' without 'unsafe-inline'). Switched to data-action
attributes with a delegated click listener on the users table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add i18n for Users subtab, show login link on user creation

- Added 'settings.users' i18n key for English and Chinese
- Token banner now shows a full login link (domain/?token=xxx)
  with a Copy Link button, plus the raw token below
- Login link works automatically via existing ?token= auto-auth

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: token hash mismatch — hash hex string, not raw bytes

Critical auth bug: token creation hashed the raw 32 bytes
(hasher.update(token_bytes)) but authentication hashed the hex-encoded
string (hash_token(candidate) where candidate is the hex string the
user sends). This meant newly created tokens could never authenticate.

Fixed all 4 token creation sites (users, tokens, invitations create,
invitations accept) to use hash_token(&plaintext_token) which hashes
the hex string consistently with the auth lookup path.

Removed now-unused sha2::Digest imports from handlers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove invitation system

The invitation flow is redundant — admin create user already generates
a token and shows a login link. Invitations add complexity without
value until email integration exists.

Removed:
- InvitationRecord struct and 4 UserStore trait methods
- invitations table from V14 migration (postgres + both libsql schemas)
- PostgreSQL Store methods (create/get/accept/list invitations)
- libSQL UserStore invitation methods + row_to_invitation helper
- invitations.rs handler file (212 lines)
- /api/invitations routes (create, list, accept)
- test_invitation_lifecycle test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: user deletion, self-service profile, per-user job limits, usage API

Four multi-tenancy improvements:

1. User deletion cascade (DELETE /api/admin/users/{id}):
   Deletes user and all data across 11 user-scoped tables (settings,
   secrets, routines, memory, jobs, conversations, etc.). Admin only.

2. Self-service profile (GET/PATCH /api/profile):
   Users can read and update their own display_name and metadata
   without admin privileges.

3. Per-user job concurrency (MAX_JOBS_PER_USER env var):
   Scheduler checks active_jobs_for(user_id) before dispatch.
   Prevents one user from exhausting all job slots.

4. Usage reporting (GET /api/admin/usage?user_id=X&period=day|week|month):
   Aggregates LLM costs from llm_calls via agent_jobs.user_id.
   Returns per-user, per-model breakdown of calls, tokens, and cost.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add TenantCtx for compile-time tenant isolation

Implements zmanian's architectural proposal from #1614 review:
two-tier scoped database access (TenantScope/AdminScope) so handler
code cannot accidentally bypass tenant scoping.

TenantScope (default): wraps user_id + Arc<dyn Database>, auto-binds
user_id on every operation. ID-based lookups return None for cross-
tenant resources. No escape hatch — forgetting to scope is a compile
error.

AdminScope (explicit opt-in): cross-tenant access for system-level
components (heartbeat, routine engine, self-repair, scheduler, worker).

TenantCtx bundles TenantScope + workspace + cost guard + per-user
rate limiting. Constructed once per request in handle_message, threaded
through all command handlers and ChatDelegate.

Key changes:
- New src/tenant.rs (~920 lines): TenantScope, AdminScope, TenantCtx,
  TenantRateState, TenantRateRegistry
- All command handlers: user_id: &str → ctx: &TenantCtx
- ChatDelegate: cost check/record/settings via self.tenant
- System components: store field changed to AdminScope
- Config: TENANT_MAX_LLM_CONCURRENT, TENANT_MAX_JOBS_CONCURRENT env vars
- Fixes bug: /status <job_id> cross-tenant leak (now auto-filtered)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR #1626 review feedback — bounded LRU cache, admin auth, FK cleanup

- Replace HashMap with lru::LruCache in DbAuthenticator so the token
  cache is hard-bounded at 1024 entries (evicts LRU, not just expired)
- Gate admin user endpoints (list/detail/update/suspend/activate) with
  AdminUser extractor so members get 403 instead of full access
- Add api_tokens to libSQL delete_user cleanup list to prevent orphaned
  tokens (libSQL has no FK cascade)
- Add regression tests for all three fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update CA certificates in runtime Docker image

Ensures the root certificate bundle is current so TLS handshakes
to services like Supabase succeed on Railway.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve CI failures — formatting, no-panics check

- Run cargo fmt on test code
- Replace .expect() with const NonZeroUsize in DbAuthenticator
- Add // safety: comments for test-only code in multi_tenant.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: switch PostgreSQL TLS from rustls to native-tls

rustls with rustls-native-certs fails TLS handshake on Railway's
slim container (empty or stale root cert store). native-tls delegates
to OpenSSL on Linux which handles system certs more reliably.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Adding user management api

* feat: admin secrets provisioning API + API documentation

- Add PUT/GET/DELETE /api/admin/users/{id}/secrets/{name} endpoints for
  application backends to provision per-user secrets (AES-256-GCM encrypted)
- Add secrets_store field to GatewayState with builder wiring
- Create docs/USER_MANAGEMENT_API.md with full API spec covering users,
  secrets, tokens, profile, and usage endpoints
- Update web gateway CLAUDE.md route table

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add CatchPanicLayer to capture handler panics

Without this, panics in async handlers silently drop the connection
and the edge proxy returns a generic 503. Now panics are caught,
logged, and returned as 500 with the panic message.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address second-round review — transactional delete, overflow, error logging

- C1: Wrap PostgreSQL delete_user() in a transaction so partial cleanup
  can't leave users in a half-deleted state
- M2: Add job_events to delete cleanup (both backends) — FK to
  agent_jobs without CASCADE would cause FK violation
- H1/M4: Cap expires_in_days to 36500 before i64 cast (tokens + secrets)
- H2: Validate target user exists before creating admin token to prevent
  orphan tokens on libSQL
- H3: Log DB errors in DbAuthenticator::authenticate() instead of
  silently swallowing them as 401

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert to rustls with webpki-roots fallback for PostgreSQL TLS

native-tls/OpenSSL caused silent crashes (segfaults in C code) during
DB writes on Railway containers. Switch back to rustls but add
webpki-roots as a fallback when system certs are missing, which was
the original TLS handshake failure on slim container images.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update Cargo.lock for rustls + webpki-roots

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* debug: add /api/debug/db-write endpoint to diagnose user insert failure

Temporary diagnostic endpoint that tests DB INSERT to users table
with full error logging. No auth required. Will be removed after
debugging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf: use cargo-chef in Dockerfile for dependency caching

Splits the build into planner/deps/builder stages. Dependencies are
only recompiled when Cargo.toml or Cargo.lock change. Source-only
changes skip straight to the final build stage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* debug: add tracing to users_create_handler

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: guard created_by FK in user creation handler

The auth identity user_id (from owner_id scope) may not match any
user row in the DB, causing a FK violation on the created_by column.
Check that the referenced user exists before setting created_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: collapse GATEWAY_USER_ID into IRONCLAW_OWNER_ID

Remove the separate GATEWAY_USER_ID config. The gateway now uses
IRONCLAW_OWNER_ID (config.owner_id) directly for auth identity,
bootstrap user creation, and workspace scoping.

Previously, with_owner_scope() rebinds the auth identity to owner_id
while keeping default_sender_id as the gateway user_id. This caused
a FK constraint violation when creating users because the auth
identity ("default") didn't match any user in the DB ("nearai").

Changes:
- Remove GATEWAY_USER_ID env var and gateway_user_id from settings
- Remove user_id field from GatewayConfig
- Add owner_id parameter to GatewayChannel::new()
- Remove with_owner_scope() method
- Remove default_sender_id from GatewayState
- Remove sender override logic in chat/approval handlers
- Remove debug endpoint and tracing from prior debugging
- Update all tests and E2E fixtures

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: hide Users tab for non-admins, remove auth hint text

- Fetch /api/profile after login and hide the Users settings tab
  when the user's role is not admin
- Remove the "Enter the GATEWAY_AUTH_TOKEN" hint from the login page
  since tokens are now managed via the admin panel, not .env files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback (auth 503, token expiry, CORS PATCH)

- DB auth errors now return 503 instead of 401 so outages are
  distinguishable from invalid tokens (serrrfirat H3)
- Cap expires_in_days to 36500 before i64 cast to prevent negative
  duration from u64 overflow (serrrfirat H1)
- Add PATCH to CORS allowed methods for profile/user update
  endpoints (Copilot)
- Stop leaking panic details in CatchPanicLayer response body

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: harden multi-tenant isolation — review fixes from #1614

- Add conversation ownership checks in TenantScope: add_conversation_message,
  touch_conversation, list_conversation_messages (+ paginated),
  update_conversation_metadata_field, get_conversation_metadata now return
  NotFound for conversations not owned by the tenant (cross-tenant data leak)
- Fix multi-user heartbeat: clear notify_user_id per runner so notifications
  persist to the correct user, not the shared config target
- Move hygiene tasks into bounded JoinSet instead of unbounded tokio::spawn
- Revert send_notification to private visibility (only used within module)
- Use effective_model_name() for cost attribution in dispatcher so providers
  that ignore per-request model overrides report the actual model used
- Fix inject_model_override doc comment; add 3 unit tests
- Fix heartbeat doc comment ("routines" not "active routines")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add Jobs, Cost, Last Active columns to admin Users table

Add UserSummaryStats struct and user_summary_stats() batch query to the
UserStore trait (both PostgreSQL and libSQL backends). The admin users
list endpoint now fetches per-user aggregates (job count, total LLM
spend, most recent activity) in a single query and includes them inline
in the response. The frontend Users table displays three new columns.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review comments and CI formatting failures

CI fixes:
- cargo fmt fixes in cli/mod.rs and db/tls.rs

Security/correctness (from Copilot + serrrfirat + pranavraja99 reviews):
- Token create: reject expires_in_days > 36500 with 400 instead of silent clamp
- Token create: return 404 when admin targets non-existent user
- User create: map duplicate email constraint violations to 409 Conflict
- User create: remove unnecessary DB roundtrip for created_by (use AdminUser directly)
- DB auth: log warn on DB lookup failures instead of silently swallowing errors
- libSQL: add FK constraints on users.created_by and api_tokens.user_id

Config fixes:
- agent.multi_tenant: resolve from AGENT_MULTI_TENANT env var instead of hardcoding false
- heartbeat.multi_tenant: fix doc comment to match actual env-var-based behavior

UI fix:
- showTokenBanner: pass correct title ("Token created!" vs "User created!")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address remaining review comments (round 2)

- Secrets handlers: normalize name to lowercase before store operations,
  validate target user_id exists (returns 404 if not found)
- libSQL: propagate cost parsing errors instead of unwrap_or_default()
  in both user_usage_stats and user_summary_stats
- users_list_handler: propagate user_summary_stats DB errors (was
  silently swallowed with unwrap_or_default)
- loadUsers: distinguish 401/403 (admin required) from other errors
- Docs: fix users.id type (TEXT not UUID), remove "invitation flow"
  from V14 migration comment

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: i18n for Users tab, atomic user+token creation, transactional delete_user

i18n:
- Add 31 translation keys for all Users tab strings (en + zh-CN)
- Wire data-i18n attributes on HTML elements (headings, buttons, inputs,
  table headers, empty state)
- Replace all hard-coded strings in app.js with I18n.t() calls

Atomic user+token creation:
- Add create_user_with_token() to UserStore trait
- PostgreSQL: wraps both INSERTs in conn.transaction() with auto-rollback
- libSQL: wraps in explicit BEGIN/COMMIT with ROLLBACK on error
- Handler uses single atomic call instead of two separate operations

Transactional delete_user for libSQL:
- Wrap multi-table DELETE cascade in BEGIN/COMMIT transaction
- ROLLBACK on any error to prevent partial cleanup / inconsistent state
- Matches the PostgreSQL implementation which already used transactions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert V14 migration to match deployed checksum [skip-regression-check]

Refinery checksums applied migrations — editing V14__users.sql after
it was already applied causes deployment failures. Revert the cosmetic
comment changes (added in df40b22) to restore the original checksum.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: bootstrap onboarding flow for multi-tenant users

The bootstrap greeting and workspace seeding only ran for the owner
workspace at startup, so new users created via the admin API never
received the welcome message or identity files (BOOTSTRAP.md, SOUL.md,
AGENTS.md, USER.md, etc.).

Three fixes:
- tenant_ctx(): seed per-user workspace on first creation via
  seed_if_empty(), which writes identity files and sets
  bootstrap_pending when the workspace is truly fresh
- handle_message(): check take_bootstrap_pending() on the tenant
  workspace (not the owner workspace) and persist the greeting to
  the user's own assistant conversation + broadcast via SSE
- WorkspacePool: seed new per-user workspaces in the web gateway
  so memory tools also see identity files immediately

The existing single-user bootstrap in Agent::run() is preserved for
non-multi-tenant deployments.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address remaining PR review comments (round 3)

- Docs: fix metadata description from "merge patch" to "full replacement"
- Secrets: reject expires_in_days > 36500 with 400 (was silently clamped)
- libSQL: CAST(SUM(cost) AS TEXT) in user_usage_stats and user_summary_stats
  to prevent SQLite numeric coercion from crashing get_text() — this was
  the root cause of the Copilot "SUM returns numeric type" comments
- Add 3 regression tests: user_summary_stats (empty + with data) and
  user_usage_stats (multi-model aggregation)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add role change support for users (admin/member toggle)

- Add update_user_role() to UserStore trait + both backends (PostgreSQL
  and libSQL)
- Extend PATCH /api/admin/users/{id} to accept optional "role" field
  with validation (must be "admin" or "member")
- Add "Make Admin" / "Make Member" toggle button in Users table actions
- Add i18n keys for role change (en + zh-CN)
- Update API docs to document the role field on PATCH
- Fix test helpers to use fmt_ts() for timestamps (was using SQLite
  datetime('now') which produces incompatible format for string comparison)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show live LLM spend in Users table instead of only DB-recorded costs [skip-regression-check]

Chat turns record LLM cost in CostGuard (in-memory) but don't create
agent_jobs/llm_calls DB rows — those are only written for background
jobs. The Users table was querying only from DB, so it showed $0.00
for users who only chatted.

Now supplements DB stats with CostGuard.daily_spend_for_user() —
the same source displayed in the status bar token counter. Shows
whichever is larger (DB historical total vs live daily spend).

Also falls back to last_login_at for "Last Active" when no DB job
activity exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: persist chat LLM calls to DB and fix usage stats query

Two root causes for zero usage stats:

1. ChatDelegate only recorded LLM costs to CostGuard (in-memory) —
   never to the llm_calls DB table. Added DB persistence via
   TenantScope.record_llm_call() after each chat LLM call, with
   job_id=NULL and conversation_id=thread_id.

2. user_summary_stats query only joined agent_jobs→llm_calls, missing
   chat calls (which have job_id=NULL). Redesigned query to start from
   llm_calls and resolve user_id via COALESCE(agent_jobs.user_id,
   conversations.user_id) — covers both job and chat LLM calls.

Both PostgreSQL and libSQL queries updated. TenantScope gets
record_llm_call() method. Tests updated for new query semantics.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review comments — input validation, cost semantics, panic safety [skip-regression-check]

- Validate display_name: trim whitespace, reject empty strings (create + update)
- Validate metadata: must be a JSON object, return 400 if not (admin + profile)
- secrets_list_handler: verify target user_id exists before listing
- Cost display: use DB total directly (chat calls now persist to DB),
  remove confusing max(db,live) CostGuard fallback
- CatchPanicLayer: truncate panic payload to 200 chars in log to limit
  potential sensitive data exposure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Copilot round 5 — docs, secrets consistency, token name, provider field [skip-regression-check]

- Docs: users.id note updated to "typically UUID v4 strings (bootstrap
  admin may use a custom ID)"
- secrets_list_handler: return 503 when DB store is None (was falling
  through to list secrets without user validation)
- tokens_create: trim + reject empty token name (matching display_name
  pattern)
- LlmCallRecord.provider: use llm_backend ("nearai","openai") instead
  of model_name() which returns the model identifier
- user_summary_stats zero-LLM users: acceptable — handler already falls
  back to 0 cost and last_login_at for missing entries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: DB auth returns 503 on outage, scheduler counts only blocking jobs

From serrrfirat review:
- DB auth: return Err(()) on database errors so middleware returns 503
  instead of silently returning Ok(None) → 401 (auth miss)
- Scheduler: add parallel_blocking_count_for() that uses
  is_parallel_blocking() (Pending/InProgress/Stuck) instead of
  is_active() for per-user concurrency — Completed/Submitted jobs
  no longer count against MAX_JOBS_PER_USER

From Copilot:
- CLAUDE.md: fix secrets route paths from {id} to {user_id}
- token_hash: use .as_slice() instead of .to_vec() to avoid
  heap allocation on every token auth/creation call

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: immediate auth cache invalidation on security-critical actions (zmanian review #6)

Add DbAuthenticator::invalidate_user() that evicts all cached entries
for a user. Called after:
- Suspend user (immediate lockout, was 60s delay)
- Activate user (immediate access restoration)
- Role change (admin↔member takes effect immediately)
- Token revocation (revoked token can't be reused from cache)

The DbAuthenticator is shared (via Clone, which Arc-clones the cache)
between the auth middleware and GatewayState, so handlers can evict
entries from the same cache the middleware reads.

Also from zmanian's review:
- Items 1-5, 7-11 were already resolved in prior commits
- Item 12 (String→enum for status/role) is deferred as a broader refactor

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: last-admin protection, usage stats for chat calls, UTF-8 safe panic truncation

Last-admin protection:
- Suspend, delete, and role-demotion of the last active admin now
  return 409 Conflict instead of succeeding and locking out the admin API
- Helper is_last_admin() checks active admin count before destructive ops

Usage stats:
- user_usage_stats() now includes chat LLM calls (job_id=NULL) by
  joining via conversations.user_id, matching user_summary_stats()
- Both PostgreSQL and libSQL queries updated

Panic handler:
- Use floor_char_boundary(200) instead of byte-index [..200] to
  prevent panic on multi-byte UTF-8 characters in panic messages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: workspace seed race, bootstrap atomicity, email trim, secrets upsert response [skip-regression-check]

- WorkspacePool: await seed_if_empty() synchronously after inserting
  into cache (drop lock first to avoid blocking), so callers see
  identity files immediately instead of racing a background task
- Bootstrap admin: use create_user_with_token() for atomic user+token
  creation, matching the admin create endpoint
- Email: trim whitespace, treat empty as None to prevent " " being
  stored and breaking uniqueness
- Secrets PUT: report "updated" vs "created" based on prior existence
- Last token_hash.to_vec() → .as_slice() in authenticate_token

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: disable unscoped webhook endpoint in multi-tenant mode [skip-regression-check]

The original /api/webhooks/{path} endpoint looks up routines across all
users. In multi-tenant mode, anyone who knows the webhook path + secret
could trigger another user's routine. Now returns 410 Gone with a
message pointing to the scoped endpoint /api/webhooks/u/{user_id}/{path}.

Detection uses state.db_auth.is_some() — present only when DB-backed
auth is enabled (multi-tenant). Single-user deployments are unaffected.

From: standardtoaster review comment

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: webhook multi-tenant check, secrets error propagation, stale doc comment [skip-regression-check]

- Webhook: use workspace_pool.is_some() instead of db_auth.is_some()
  for multi-tenant detection — db_auth is set for any DB deployment,
  workspace_pool is only set when has_any_users() was true at startup
- Secrets: propagate exists() errors instead of unwrap_or(false) so
  backend outages surface as 500 rather than incorrect "created" status
- Config: fix stale workspace_read_scopes comment referencing user_id

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bkutasi pushed a commit to bkutasi/ironclaw that referenced this pull request Mar 28, 2026
…tion (nearai#238)

* feat: add extension registry with metadata catalog, CLI, and onboarding integration

Adds a central registry that catalogs all 14 available extensions (10 tools,
4 channels) with their capabilities, auth requirements, and artifact references.
The onboarding wizard now shows installable channels from the registry and
offers tool installation as a new Step 7.

- registry/ folder with per-extension JSON manifests and bundle definitions
- src/registry/ module: manifest structs, catalog loader, installer
- `ironclaw registry list|info|install|install-defaults` CLI commands
- Setup wizard enhanced: channels from registry, new extensions step (8 steps)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(setup): resolve workspace errors for tool crates and channels-only onboarding

Tool crates in tools-src/ and channels-src/ failed `cargo metadata` during
onboard install because Cargo resolved them as part of the root workspace.
Add `[workspace]` table to each standalone crate and extend the root
`workspace.exclude` list so they build independently.

Channels-only mode (`onboard --channels-only`) failed with "Secrets not
configured" and "No database connection" because it skipped database and
security setup. Add `reconnect_existing_db()` to establish the DB connection
and load saved settings before running channel configuration.

Also improve the tunnel "already configured" display to show full provider
details (domain, mode, command) instead of just the provider name.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(registry): address PR review feedback on installer and catalog

- Use manifest.name (not crate_name) for installed filenames so
  discovery, auth, and CLI commands all agree on the stem (nearai#1)
- Add AlreadyInstalled error variant instead of misleading
  ExtensionNotFound (nearai#2)
- Add DownloadFailed error variant with URL context instead of
  stuffing URLs into PathBuf (nearai#3)
- Validate HTTP status with error_for_status() before reading
  response bytes in artifact downloads (nearai#4)
- Switch build_wasm_component to tokio::process::Command with
  status() so build output streams to the terminal (nearai#6)
- Find WASM artifact by crate_name specifically instead of picking
  the first .wasm file in the release directory (nearai#7)
- Add is_file() guard in catalog loader to skip directories (nearai#8)
- Detect ambiguous bare-name lookups when both tools/<name> and
  channels/<name> exist, with get_strict() returning an error (nearai#9)
- Fix wizard step_extensions to check tool.name for installed
  detection, consistent with the new naming (nearai#11, nearai#12)
- Fix redundant closures and map_or clippy warnings in changed files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(setup): restore DB connection fields after settings reload

reconnect_postgres() and reconnect_libsql() called Settings::from_db_map()
which overwrote database_url / libsql_path / libsql_url set from env vars.
Also use get_strict() in cmd_info to surface ambiguous bare-name errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix clippy collapsible_if and print_literal warnings

Collapse nested if-let chains and inline string literals in format
macros to satisfy CI clippy lint checks (deny warnings).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(registry): prefer artifacts for install-defaults and improve dir lookup

- InstallDefaults now defaults to downloading pre-built artifacts
  (matching `registry install` behavior), with --build flag for source builds.
- find_registry_dir() walks up 3 ancestor levels from the exe and adds
  a CARGO_MANIFEST_DIR fallback, matching load_registry_catalog() logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
bkutasi pushed a commit to bkutasi/ironclaw that referenced this pull request Mar 28, 2026
* test: add WIT compatibility tests for all WASM tools and channels

Adds CI and integration tests to catch WIT interface breakage across
all 14 WASM extensions (10 tools + 4 channels). Previously, changing
wit/tool.wit or wit/channel.wit could silently break guest-side tools
that weren't rebuilt until release time.

Three new pieces:

1. scripts/build-wasm-extensions.sh — builds all WASM extensions from
   source by reading registry manifests. Used by CI and locally.

2. tests/wit_compat.rs — integration tests that compile and instantiate
   each .wasm binary against the current wasmtime host linker with
   stubbed host functions. Catches added/removed/renamed WIT functions,
   signature mismatches, and missing exports. Skips gracefully when
   artifacts aren't built so `cargo test` still passes standalone.

3. .github/workflows/test.yml — new wasm-wit-compat CI job that builds
   all extensions then runs instantiation tests on every PR. Added to
   the branch protection roll-up.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: fix rustfmt formatting in wit_compat tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback on WIT compat tests

- Switch build script from python3 to jq for JSON parsing, consistent
  with release.yml and avoids python3 dependency (nearai#1, nearai#7)
- Use dirs::home_dir() instead of HOME env var for portability (nearai#2)
- Filter extensions by manifest "kind" field instead of path (nearai#3)
- Replace .flatten() with explicit error handling in dir iteration (nearai#4, nearai#5)
- Split stub_tool_host_functions into stub_shared_host_functions +
  tool-only tool-invoke stub, since tool-invoke is not in channel WIT (nearai#6)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
bkutasi pushed a commit to bkutasi/ironclaw that referenced this pull request Mar 28, 2026
)

* feat: add inbound attachment support to WASM channel system

Add attachment record to WIT interface and implement inbound media
parsing across all four channel implementations (Telegram, Slack,
WhatsApp, Discord). Attachments flow from WASM channels through
EmittedMessage to IncomingMessage with validation (size limits,
MIME allowlist, count caps) at the host boundary.

- Add `attachment` record to `emitted-message` in wit/channel.wit
- Add `IncomingAttachment` struct to channel.rs and re-export
- Add host-side validation (20MB total, 10 max, MIME allowlist)
- Telegram: parse photo, document, audio, video, voice, sticker
- Slack: parse file attachments with url_private
- WhatsApp: parse image, audio, video, document with captions
- Discord: backward-compatible empty attachments
- Update FEATURE_PARITY.md section 7
- Add fixture-based tests per channel and host integration tests

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: integrate outbound attachment support and reconcile WIT types (nearai#409)

Reconcile PR nearai#409's outbound attachment work with our inbound attachment
support into a unified design:

WIT type split:
- `inbound-attachment` in channel-host: metadata-only (id, mime_type,
  filename, size_bytes, source_url, storage_key, extracted_text)
- `attachment` in channel: raw bytes (filename, mime_type, data) on
  agent-response for outbound sending

Outbound features (from PR nearai#409):
- `on-broadcast` WIT export for proactive messages without prior inbound
- Telegram: multipart sendPhoto/sendDocument with auto photo→document
  fallback for files >10MB
- wrapper.rs: `call_on_broadcast`, `read_attachments` from disk,
  attachment params threaded through `call_on_respond`
- HTTP tool: `save_to` param for binary downloads to /tmp/ (50MB limit,
  path traversal protection, SSRF-safe redirect following)
- Message tool: allow /tmp/ paths for attachments alongside base_dir
- Credential env var fallback in inject_channel_credentials

Channel updates:
- All 4 channels implement on_broadcast (Telegram full, others stub)
- Telegram: polling_enabled config, adjusted poll timeout
- Inbound attachment types renamed to InboundAttachment in all channels

Tests: 1965 passing (9 new), 0 clippy warnings

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add audio transcription pipeline and extensible WIT attachment design

Add host-side transcription middleware (OpenAI Whisper) that detects audio
attachments with inline data on incoming messages and transcribes them
automatically. Refactor WIT inbound-attachment to use extras-json and a
store-attachment-data host function instead of typed fields, so future
attachment properties (dimensions, codec, etc.) don't require WIT changes
that invalidate all channel plugins.

- Add src/transcription/ module: TranscriptionProvider trait,
  TranscriptionMiddleware, AudioFormat enum, OpenAI Whisper provider
- Add src/config/transcription.rs: TRANSCRIPTION_ENABLED/MODEL/BASE_URL
- Wire middleware into agent message loop via AgentDeps
- WIT: replace data + duration-secs with extras-json + store-attachment-data
- Host: parse extras-json for well-known keys, merge stored binary data
- Telegram: download voice files via store-attachment-data, add duration
  to extras-json, add /file/bot to HTTP allowlist, voice-only placeholder
- Add reqwest multipart feature for Whisper API uploads
- 5 regression tests for transcription middleware

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: wire attachment processing into LLM pipeline with multimodal image support

Attachments on incoming messages are now augmented into user text via XML tags
before entering the turn system, and images with data are passed as multimodal
content parts (base64 data URIs) to LLM providers. This enables audio transcripts,
document text, and image content to reach the LLM without changes to ChatMessage
serialization or provider interfaces.

- Add src/agent/attachments.rs with augment_with_attachments() and 9 unit tests
- Add ContentPart/ImageUrl types to llm::provider with OpenAI-compatible serde
- Carry image_content_parts transiently on Turn (skipped in serialization)
- Update nearai_chat and rig_adapter to serialize multimodal content
- Add 3 e2e tests verifying attachments flow through the full agent loop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, version bumps, and Telegram voice test

- Fix cargo fmt formatting in attachments.rs, nearai_chat.rs, rig_adapter.rs,
  e2e_attachments.rs
- Bump channel registry versions 0.1.0 → 0.2.0 (discord, slack, telegram,
  whatsapp) to satisfy version-bump CI check
- Fix Telegram test_extract_attachments_voice: add missing required `duration`
  field to voice fixture JSON

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: bump WIT channel version to 0.3.0, fix Telegram voice test, add pre-commit hook

- Bump wit/channel.wit package version 0.2.0 → 0.3.0 (interface changed with
  store-attachment-data)
- Update WIT_CHANNEL_VERSION constant and registry wit_version fields to match
- Fix Telegram test_extract_attachments_voice: gate voice download behind
  #[cfg(target_arch = "wasm32")] so host functions aren't called in native tests,
  update assertions for generated filename and extras_json duration
- Add @0.3.0 linker stubs in wit_compat.rs
- Add .githooks/pre-commit hook that runs scripts/check-version-bumps.sh when
  WIT or extension sources are staged
- Symlink commit-msg regression hook into .githooks/

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract voice download from extract_attachments into handle_message

Move download_voice_file + store_attachment_data calls out of
extract_attachments into a separate download_and_store_voice function
called from handle_message. This keeps extract_attachments as a pure
data-mapping function with no host calls, making it fully testable
in native unit tests without #[cfg(target_arch)] gates.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Add path validation to read_attachments (restrict to /tmp/) preventing
  arbitrary file reads from compromised tools
- Escape XML special characters in attachment filenames, MIME types, and
  extracted text to prevent prompt injection via tag spoofing
- Percent-encode file_id in Telegram getFile URL to prevent query injection
- Clone SecretString directly instead of expose_secret().to_string()

Correctness fixes:
- Fix store_attachment_data overwrite accounting: subtract old entry size
  before adding new to prevent inflated totals and false rejections
- Use max(reported, stored_size) for attachment size accounting to prevent
  WASM channels from under-reporting size_bytes to bypass limits
- Add application/octet-stream to MIME allowlist (channels default unknown
  types to this)

Code quality:
- Extract send_response helper in Telegram, deduplicating on_respond and
  on_broadcast
- Rename misleading Discord test to test_parse_slash_command_interaction
- Fix .githooks/commit-msg to use relative symlink (portable across machines)

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add tool_upgrade command + fix TOCTOU in save_to path validation

Add `tool_upgrade` — a new extension management tool that automatically
detects and reinstalls WASM extensions with outdated WIT versions.
Preserves authentication secrets during upgrade. Supports upgrading a
single extension by name or all installed WASM tools/channels at once.

Fix TOCTOU in `validate_save_to_path`: validate the path *before*
creating parent directories, so traversal paths like `/tmp/../../etc/`
cannot cause filesystem mutations outside /tmp before being rejected.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: unify WIT package version to 0.3.0 across tool.wit and all capabilities

tool.wit and channel.wit share the `near:agent` package namespace, so they
must declare the same version. Bumps tool.wit from 0.2.0 to 0.3.0 and
updates all capabilities files and registry entries to match.

Fixes `cargo component build` failure: "package identifier near:agent@0.2.0
does not match previous package name of near:agent@0.3.0"

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move WIT file comments after package declaration

WIT treats `//` comments before `package` as doc comments. When both
tool.wit and channel.wit had header comments, the parser rejected them
as "doc comments on multiple 'package' items". Move comments after the
package declaration in both files.

Also bumps tool registry versions to 0.2.0 to match the WIT 0.3.0 bump.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: display extension versions in gateway Extensions tab

Add version field to InstalledExtension and RegistryEntry types, pipe
through the web API (ExtensionInfo, RegistryEntryInfo), and render as
a badge in the gateway UI for both installed and available extensions.

For installed WASM extensions, version is read from the capabilities
file with a fallback to the registry entry when the local file has no
version (old installations). Bump all extension Cargo.toml and registry
JSON versions from 0.1.0 to 0.2.0 to keep them in sync.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add document text extraction middleware for PDF, Office, and text files

Extract text from document attachments (PDF, DOCX, PPTX, XLSX, RTF, plain text,
code files) so the LLM can reason about uploaded documents. Uses pdf-extract for
PDFs, zip+XML parsing for Office XML formats, and UTF-8 decode for text files.
Wired into the agent loop after transcription middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: download document files in Telegram channel for text extraction

The DocumentExtractionMiddleware needs file bytes in the attachment `data`
field, but only voice files were being downloaded. Document attachments
(PDFs, DOCX, etc.) had empty `data` and a source_url with a credential
placeholder that only works inside the WASM host's http_request.

Add `download_and_store_documents()` that downloads non-voice, non-image,
non-audio attachments via the existing two-step getFile→download flow and
stores bytes via `store_attachment_data` for host-side extraction.

Also rename `download_voice_file` → `download_telegram_file` since it's
generic for any file_id.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: allow Office MIME types and increase file download limit for Telegram

Two issues preventing document extraction from Telegram:

1. PPTX/DOCX/XLSX MIME types (application/vnd.*) were dropped by the
   WASM host attachment allowlist — add application/vnd., application/msword,
   and application/rtf prefixes.

2. Telegram file downloads over 10 MB failed with "Response body too large" —
   set max_response_bytes to 20 MB in Telegram capabilities.

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: report document extraction errors back to user instead of silently skipping

- Bump max_response_bytes to 50 MB for Telegram file downloads
- When document extraction fails (too large, download error, parse error),
  set extracted_text to a user-friendly error message instead of leaving it
  None. This ensures the LLM tells the user what went wrong.
- On Telegram download failure, set extracted_text with the error so the
  user sees feedback even when the file never reaches the extraction middleware.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: store extracted document text in workspace memory for search/recall

After document extraction succeeds, write the extracted text to workspace
memory at `documents/{date}/{filename}`. This enables:
- Full-text and semantic search over past uploaded documents
- Cross-conversation recall ("what did that PDF say?")
- Automatic chunking and embedding via the workspace pipeline

Documents are stored with metadata header (uploader, channel, date, MIME type).
Error messages (extraction failures) are not stored — only successful extractions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: CI failures — formatting, unused assignment warning

- Run cargo fmt on document_extraction and agent_loop modules
- Suppress unused_assignments warning on trace_llm_ref (used only
  behind #[cfg(feature = "libsql")])

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review comments — security, correctness, and code quality

Security fixes:
- Remove SSRF-prone download() from DocumentExtractionMiddleware (nearai#13)
- Sanitize filenames in workspace path to prevent directory traversal (nearai#11)
- Pre-check file size before reading in WASM wrapper to prevent OOM (nearai#2)
- Percent-encode file_id in Telegram source URLs (nearai#7)

Correctness fixes:
- Clear image_content_parts on turn end to prevent memory leak (nearai#1)
- Find first *successful* transcription instead of first overall (nearai#3)
- Enforce data.len() size limit in document extraction (nearai#10)
- Use UTF-8 safe truncation with char_indices() (nearai#12)

Robustness & code quality:
- Add 120s timeout to OpenAI Whisper HTTP client (nearai#5)
- Trim trailing slash from Whisper base_url (nearai#6)
- Allow ~/.ironclaw/ paths in WASM wrapper (nearai#8)
- Return error from on_broadcast in Slack/Discord/WhatsApp (nearai#9)
- Fix doc comment in HTTP tool (nearai#4)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: formatting — cargo fmt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address latest PR review — doc comments, error messages, version bumps

- Fix DocumentExtractionMiddleware doc comment (no longer downloads from source_url)
- Fix error message: "no inline data" instead of "no download URL"
- Log error + fallback instead of silent unwrap_or_default on Whisper HTTP client
- Bump all capabilities.json versions from 0.1.0 to 0.2.0 to match Cargo.toml

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unsupported profile: minimal from CI workflows [skip-regression-check]

dtolnay/rust-toolchain@stable does not accept the 'profile' input
(it was a parameter for the deprecated actions-rs/toolchain action).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: merge with latest main — resolve compilation errors and PR review nits

- Add version: None to RegistryEntry/InstalledExtension test constructors
- Fix MessageContent type mismatches in nearai_chat tests (String → MessageContent::Text)
- Fix .contains() calls on MessageContent — use .as_text().unwrap()
- Remove redundant trace_llm_ref = None assignment in test_rig
- Check data size before clone in document extraction to avoid unnecessary allocation

[skip-regression-check]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
serrrfirat pushed a commit that referenced this pull request Mar 29, 2026
GATEWAY_USER_TOKENS never went to production — replaced entirely by
DB-backed user management via /api/admin/users and /api/tokens.

Removed:
- UserTokenConfig struct and GATEWAY_USER_TOKENS env var parsing
- user_tokens field from GatewayConfig
- GatewayChannel::new_multi_auth() constructor
- Env-var user migration block in main.rs (~90 lines)
- multi_tenant auto-detection from GATEWAY_USER_TOKENS (now runtime
  via db.has_any_users() in app.rs)

Review fixes (zmanian):
- User ID generation: UUID instead of display-name derivation (#1)
- Invitation accept moved to public router (no auth needed) (#3)
- libSQL get_invitation_by_hash aligned with postgres: filters
  status='pending' AND expires_at > now (#4)
- UUID parse: returns DatabaseError::Serialization instead of
  unwrap_or_default (#7)
- PostgreSQL SELECT * replaced with explicit column lists (#8)
- Sort order aligned (both backends use DESC) (#6)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DougAnderson444 pushed a commit to DougAnderson444/ironclaw that referenced this pull request Mar 29, 2026
…i-tenant isolation (nearai#1626)

* feat: complete multi-tenant isolation — per-user budgets, model selection, heartbeat cycling

Finishes the remaining isolation work from phases 2–4 of nearai#59:

Phase 2 (DB scoping): Fix /status and /list commands to use _for_user
DB variants instead of global queries that leaked cross-user job data.

Phase 3 (Runtime isolation): Per-user workspace in routine engine's
spawn_fire so lightweight routines run in the correct user context.
Per-user daily cost tracking in CostGuard with configurable budget via
MAX_COST_PER_USER_PER_DAY_CENTS. Multi-user heartbeat that cycles
through all users with routines, auto-detected from GATEWAY_USER_TOKENS.

Phase 4 (Provider/tools): Per-user model selection via preferred_model
setting — looked up from SettingsStore on first iteration, threaded
through ReasoningContext.model_override to CompletionRequest. Works
with providers that support per-request model overrides (NearAI).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use selected_model setting key to match /model command persistence

The dispatcher was reading "preferred_model" but the /model command
(merged from staging) persists to "selected_model". Since set_setting
is already per-user scoped, using the same key makes /model work as
the per-user model override in multi-tenant mode.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: heartbeat hygiene, /model multi-tenant guard, RigAdapter model override

Three follow-up fixes for multi-tenant isolation:

1. Multi-user heartbeat now runs memory hygiene per user before each
   heartbeat check, matching single-user heartbeat behavior.

2. /model command in multi-tenant mode only persists to per-user
   settings (selected_model) without calling set_model() on the shared
   LlmProvider. The per-request model_override in the dispatcher reads
   from the same setting. Added multi_tenant flag to AgentConfig
   (auto-detected from GATEWAY_USER_TOKENS).

3. RigAdapter now supports per-request model overrides by injecting the
   model name into rig-core's additional_params. OpenAI/Anthropic/Ollama
   API servers use last-key-wins for duplicate JSON keys, so the override
   takes effect via serde's flatten serialization order.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review — cost model attribution, heartbeat concurrency, pruning

Fixes from review comments on nearai#1614:

- Cost tracking now uses the override model name (not active_model_name)
  when a per-user model override is active, for accurate attribution.
- Multi-user heartbeat runs per-user checks concurrently via JoinSet
  instead of sequentially, preventing one slow user from blocking others.
- Per-user failure counts tracked independently; users exceeding
  max_failures are skipped (matching single-user semantics).
- per_user_daily_cost HashMap pruned on day rollover to prevent
  unbounded growth in long-lived deployments.
- Doc comment fixed: says "routines" not "active routines".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: /status ownership, model persistence scoping, heartbeat robustness

Addresses second round of PR review on nearai#1614:

- /status <job_id> DB path now validates job.user_id == requesting user
  before returning data (was missing ownership check, security fix).

- persist_selected_model takes user_id param instead of owner_id, and
  skips .env/TOML writes in multi-tenant mode (these are shared global
  files). handle_system_command now receives user_id from caller.

- JoinSet collection handles Err(JoinError) explicitly instead of
  silently dropping panicked tasks.

- Notification forwarder extracts owner_id from response metadata in
  multi-tenant mode for per-user routing instead of broadcasting to
  the agent owner.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: cost pricing, fire_manual workspace, heartbeat concurrency cap

Round 3 review fixes:

- Cost tracking passes None for cost_per_token when model override is
  active, letting CostGuard look up pricing by model name instead of
  using the default provider's rates (serrrfirat).

- fire_manual() now uses per-user workspace, matching spawn_fire()
  pattern (serrrfirat).

- Removed MULTI_TENANT env var — multi-tenant mode is auto-detected
  solely from GATEWAY_USER_TOKENS presence (serrrfirat + Copilot).

- Multi-user heartbeat capped at 8 concurrent tasks to avoid flooding
  the LLM provider (serrrfirat + Copilot).

- Fixed inject_model_override doc comment accuracy (Copilot).

- Added comment explaining multi-tenant notification routing priority
  (Copilot).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: user-scoped webhook endpoint for multi-tenant isolation

Adds POST /api/webhooks/u/{user_id}/{path} — a user-scoped webhook
endpoint that filters the routine lookup by user_id, preventing
cross-user webhook triggering when paths collide.

The existing /api/webhooks/{path} endpoint remains unchanged for
backward compatibility in single-user deployments.

Changes:
- get_webhook_routine_by_path gains user_id: Option<&str> param
- Both postgres and libsql implementations add AND user_id = ? filter
  when user_id is provided
- New webhook_trigger_user_scoped_handler extracts (user_id, path)
  from URL and passes to shared fire_webhook_inner logic
- Route registered on public router (webhooks are called by external
  services that can't send bearer tokens)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(db): add UserStore trait with users, api_tokens, invitations tables

Foundation for DB-backed user management (nearai#1605):

- UserRecord, ApiTokenRecord, InvitationRecord types in db/mod.rs
- UserStore sub-trait (17 methods) added to Database supertrait
- PostgreSQL migration V14__users.sql (users, api_tokens, invitations)
- libSQL schema + incremental migration V14
- Full implementations for both PgBackend (via Store delegation) and
  LibSqlBackend (direct SQL in libsql/users.rs)
- authenticate_token JOINs api_tokens+users with active/non-revoked
  checks; has_any_users for bootstrap detection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): DB-backed auth, user/token/invitation API handlers

Adds the web gateway layer for DB-backed user management (nearai#1605):

Auth refactor:
- CombinedAuthState wraps env-var tokens (MultiAuthState) + optional
  DbAuthenticator for DB-backed token lookup with LRU cache (60s TTL,
  1024 max entries)
- auth_middleware tries env-var tokens first, then DB fallback
- From<MultiAuthState> impl for backward compatibility
- main.rs wires with_db_auth when database is available

API handlers (12 new endpoints):
- /api/admin/users — CRUD: create, list, detail, update, suspend, activate
- /api/tokens — create (returns plaintext once), list, revoke
- /api/invitations — create, list, accept (creates user + first token)

Token creation: 32 random bytes → hex plaintext, SHA-256 hash stored.
Invitation accept: validates hash + pending + not expired, creates
user record and first API token atomically.

All test files updated for CombinedAuthState type change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: startup env-var user migration + UserStore integration tests

Completes the DB-backed user management feature (nearai#1605):

- Startup migration: when GATEWAY_USER_TOKENS is set and the users
  table is empty, inserts env-var users + hashed tokens into DB.
  Logs deprecation notice when DB already has users.
- hash_token made pub for reuse in migration code.
- 10 integration tests for UserStore (libsql file-backed):
  - has_any_users bootstrap detection
  - create/get/get_by_email/list/update user lifecycle
  - token create → authenticate → revoke → reject cycle
  - suspended user tokens rejected
  - wrong-user token revoke returns false
  - invitation create → accept → user created
  - record_login and record_token_usage timestamps
- libSQL migration: removed FK constraints from V14 (incompatible
  with execute_batch inside transactions). Tables in both base SCHEMA
  and incremental migration for fresh and existing databases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove GATEWAY_USER_TOKENS, fix review feedback

GATEWAY_USER_TOKENS never went to production — replaced entirely by
DB-backed user management via /api/admin/users and /api/tokens.

Removed:
- UserTokenConfig struct and GATEWAY_USER_TOKENS env var parsing
- user_tokens field from GatewayConfig
- GatewayChannel::new_multi_auth() constructor
- Env-var user migration block in main.rs (~90 lines)
- multi_tenant auto-detection from GATEWAY_USER_TOKENS (now runtime
  via db.has_any_users() in app.rs)

Review fixes (zmanian):
- User ID generation: UUID instead of display-name derivation (nearai#1)
- Invitation accept moved to public router (no auth needed) (nearai#3)
- libSQL get_invitation_by_hash aligned with postgres: filters
  status='pending' AND expires_at > now (nearai#4)
- UUID parse: returns DatabaseError::Serialization instead of
  unwrap_or_default (nearai#7)
- PostgreSQL SELECT * replaced with explicit column lists (nearai#8)
- Sort order aligned (both backends use DESC) (nearai#6)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add role-based access control (admin/member)

Adds a `role` field (admin|member) to user management:

Schema:
- `role TEXT NOT NULL DEFAULT 'member'` added to users table in both
  PostgreSQL V14 migration and libSQL schema/incremental migration
- UserRecord gains `role: String` field
- UserIdentity gains `role: String` field, populated from DB in
  DbAuthenticator and defaulting to "admin" for single-user mode

Access control:
- AdminUser extractor: returns 403 Forbidden if role != "admin"
- /api/admin/users/* handlers: require AdminUser (create, list,
  detail, update, suspend, activate)
- POST /api/invitations: requires AdminUser (only admins can invite)
- User creation accepts optional "role" param (defaults to "member")
- Invitation acceptance creates users with "member" role

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(web): add Users admin tab to web UI

Adds a Users tab to the web gateway UI for managing users, tokens,
and roles without needing direct API calls.

Features:
- User list table with ID, name, email, role, status, created date
- Create user form with display name, email, role selector
- Suspend/activate actions per user
- Create API token for any user (shows plaintext once with copy button)
- Role badges (admin highlighted, member muted)
- Non-admin users see "Admin access required" message
- Keyboard shortcut: Cmd/Ctrl+5 switches to Users tab

CSS:
- Reuses routines-table styles for the user list
- Badge, token-display, btn-small, btn-danger, btn-primary components

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move Users to Settings subtab, bootstrap admin user on first run

- Moved Users from top-level tab to Settings sidebar subtab (under
  Skills, before Theme toggle)
- On first startup with empty users table, automatically creates an
  admin user from GATEWAY_USER_ID config with a corresponding API
  token from GATEWAY_AUTH_TOKEN. This ensures the owner appears in
  the Users panel immediately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: user creation shows token, + Token works, no password save popup

Three UI/UX fixes:

1. Create user now generates an initial API token and shows it in a
   copy-able banner instead of triggering the browser's password save
   dialog. Uses autocomplete="off" and type="text" for email field.

2. "+ Token" button works: exposed createTokenForUser/suspendUser/
   activateUser on window for inline onclick handlers in dynamically
   generated table rows. Token creation uses showTokenBanner helper.

3. Admin token creation: POST /api/tokens now accepts optional
   "user_id" field when the requesting user is admin, allowing
   token creation for other users from the Users panel.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use event delegation for user action buttons (CSP compliance)

Inline onclick handlers are blocked by the Content-Security-Policy
(script-src 'self' without 'unsafe-inline'). Switched to data-action
attributes with a delegated click listener on the users table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add i18n for Users subtab, show login link on user creation

- Added 'settings.users' i18n key for English and Chinese
- Token banner now shows a full login link (domain/?token=xxx)
  with a Copy Link button, plus the raw token below
- Login link works automatically via existing ?token= auto-auth

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: token hash mismatch — hash hex string, not raw bytes

Critical auth bug: token creation hashed the raw 32 bytes
(hasher.update(token_bytes)) but authentication hashed the hex-encoded
string (hash_token(candidate) where candidate is the hex string the
user sends). This meant newly created tokens could never authenticate.

Fixed all 4 token creation sites (users, tokens, invitations create,
invitations accept) to use hash_token(&plaintext_token) which hashes
the hex string consistently with the auth lookup path.

Removed now-unused sha2::Digest imports from handlers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove invitation system

The invitation flow is redundant — admin create user already generates
a token and shows a login link. Invitations add complexity without
value until email integration exists.

Removed:
- InvitationRecord struct and 4 UserStore trait methods
- invitations table from V14 migration (postgres + both libsql schemas)
- PostgreSQL Store methods (create/get/accept/list invitations)
- libSQL UserStore invitation methods + row_to_invitation helper
- invitations.rs handler file (212 lines)
- /api/invitations routes (create, list, accept)
- test_invitation_lifecycle test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: user deletion, self-service profile, per-user job limits, usage API

Four multi-tenancy improvements:

1. User deletion cascade (DELETE /api/admin/users/{id}):
   Deletes user and all data across 11 user-scoped tables (settings,
   secrets, routines, memory, jobs, conversations, etc.). Admin only.

2. Self-service profile (GET/PATCH /api/profile):
   Users can read and update their own display_name and metadata
   without admin privileges.

3. Per-user job concurrency (MAX_JOBS_PER_USER env var):
   Scheduler checks active_jobs_for(user_id) before dispatch.
   Prevents one user from exhausting all job slots.

4. Usage reporting (GET /api/admin/usage?user_id=X&period=day|week|month):
   Aggregates LLM costs from llm_calls via agent_jobs.user_id.
   Returns per-user, per-model breakdown of calls, tokens, and cost.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add TenantCtx for compile-time tenant isolation

Implements zmanian's architectural proposal from nearai#1614 review:
two-tier scoped database access (TenantScope/AdminScope) so handler
code cannot accidentally bypass tenant scoping.

TenantScope (default): wraps user_id + Arc<dyn Database>, auto-binds
user_id on every operation. ID-based lookups return None for cross-
tenant resources. No escape hatch — forgetting to scope is a compile
error.

AdminScope (explicit opt-in): cross-tenant access for system-level
components (heartbeat, routine engine, self-repair, scheduler, worker).

TenantCtx bundles TenantScope + workspace + cost guard + per-user
rate limiting. Constructed once per request in handle_message, threaded
through all command handlers and ChatDelegate.

Key changes:
- New src/tenant.rs (~920 lines): TenantScope, AdminScope, TenantCtx,
  TenantRateState, TenantRateRegistry
- All command handlers: user_id: &str → ctx: &TenantCtx
- ChatDelegate: cost check/record/settings via self.tenant
- System components: store field changed to AdminScope
- Config: TENANT_MAX_LLM_CONCURRENT, TENANT_MAX_JOBS_CONCURRENT env vars
- Fixes bug: /status <job_id> cross-tenant leak (now auto-filtered)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR nearai#1626 review feedback — bounded LRU cache, admin auth, FK cleanup

- Replace HashMap with lru::LruCache in DbAuthenticator so the token
  cache is hard-bounded at 1024 entries (evicts LRU, not just expired)
- Gate admin user endpoints (list/detail/update/suspend/activate) with
  AdminUser extractor so members get 403 instead of full access
- Add api_tokens to libSQL delete_user cleanup list to prevent orphaned
  tokens (libSQL has no FK cascade)
- Add regression tests for all three fixes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update CA certificates in runtime Docker image

Ensures the root certificate bundle is current so TLS handshakes
to services like Supabase succeed on Railway.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve CI failures — formatting, no-panics check

- Run cargo fmt on test code
- Replace .expect() with const NonZeroUsize in DbAuthenticator
- Add // safety: comments for test-only code in multi_tenant.rs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: switch PostgreSQL TLS from rustls to native-tls

rustls with rustls-native-certs fails TLS handshake on Railway's
slim container (empty or stale root cert store). native-tls delegates
to OpenSSL on Linux which handles system certs more reliably.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Adding user management api

* feat: admin secrets provisioning API + API documentation

- Add PUT/GET/DELETE /api/admin/users/{id}/secrets/{name} endpoints for
  application backends to provision per-user secrets (AES-256-GCM encrypted)
- Add secrets_store field to GatewayState with builder wiring
- Create docs/USER_MANAGEMENT_API.md with full API spec covering users,
  secrets, tokens, profile, and usage endpoints
- Update web gateway CLAUDE.md route table

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add CatchPanicLayer to capture handler panics

Without this, panics in async handlers silently drop the connection
and the edge proxy returns a generic 503. Now panics are caught,
logged, and returned as 500 with the panic message.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address second-round review — transactional delete, overflow, error logging

- C1: Wrap PostgreSQL delete_user() in a transaction so partial cleanup
  can't leave users in a half-deleted state
- M2: Add job_events to delete cleanup (both backends) — FK to
  agent_jobs without CASCADE would cause FK violation
- H1/M4: Cap expires_in_days to 36500 before i64 cast (tokens + secrets)
- H2: Validate target user exists before creating admin token to prevent
  orphan tokens on libSQL
- H3: Log DB errors in DbAuthenticator::authenticate() instead of
  silently swallowing them as 401

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert to rustls with webpki-roots fallback for PostgreSQL TLS

native-tls/OpenSSL caused silent crashes (segfaults in C code) during
DB writes on Railway containers. Switch back to rustls but add
webpki-roots as a fallback when system certs are missing, which was
the original TLS handshake failure on slim container images.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update Cargo.lock for rustls + webpki-roots

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* debug: add /api/debug/db-write endpoint to diagnose user insert failure

Temporary diagnostic endpoint that tests DB INSERT to users table
with full error logging. No auth required. Will be removed after
debugging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf: use cargo-chef in Dockerfile for dependency caching

Splits the build into planner/deps/builder stages. Dependencies are
only recompiled when Cargo.toml or Cargo.lock change. Source-only
changes skip straight to the final build stage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* debug: add tracing to users_create_handler

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: guard created_by FK in user creation handler

The auth identity user_id (from owner_id scope) may not match any
user row in the DB, causing a FK violation on the created_by column.
Check that the referenced user exists before setting created_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: collapse GATEWAY_USER_ID into IRONCLAW_OWNER_ID

Remove the separate GATEWAY_USER_ID config. The gateway now uses
IRONCLAW_OWNER_ID (config.owner_id) directly for auth identity,
bootstrap user creation, and workspace scoping.

Previously, with_owner_scope() rebinds the auth identity to owner_id
while keeping default_sender_id as the gateway user_id. This caused
a FK constraint violation when creating users because the auth
identity ("default") didn't match any user in the DB ("nearai").

Changes:
- Remove GATEWAY_USER_ID env var and gateway_user_id from settings
- Remove user_id field from GatewayConfig
- Add owner_id parameter to GatewayChannel::new()
- Remove with_owner_scope() method
- Remove default_sender_id from GatewayState
- Remove sender override logic in chat/approval handlers
- Remove debug endpoint and tracing from prior debugging
- Update all tests and E2E fixtures

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: hide Users tab for non-admins, remove auth hint text

- Fetch /api/profile after login and hide the Users settings tab
  when the user's role is not admin
- Remove the "Enter the GATEWAY_AUTH_TOKEN" hint from the login page
  since tokens are now managed via the admin panel, not .env files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback (auth 503, token expiry, CORS PATCH)

- DB auth errors now return 503 instead of 401 so outages are
  distinguishable from invalid tokens (serrrfirat H3)
- Cap expires_in_days to 36500 before i64 cast to prevent negative
  duration from u64 overflow (serrrfirat H1)
- Add PATCH to CORS allowed methods for profile/user update
  endpoints (Copilot)
- Stop leaking panic details in CatchPanicLayer response body

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: harden multi-tenant isolation — review fixes from nearai#1614

- Add conversation ownership checks in TenantScope: add_conversation_message,
  touch_conversation, list_conversation_messages (+ paginated),
  update_conversation_metadata_field, get_conversation_metadata now return
  NotFound for conversations not owned by the tenant (cross-tenant data leak)
- Fix multi-user heartbeat: clear notify_user_id per runner so notifications
  persist to the correct user, not the shared config target
- Move hygiene tasks into bounded JoinSet instead of unbounded tokio::spawn
- Revert send_notification to private visibility (only used within module)
- Use effective_model_name() for cost attribution in dispatcher so providers
  that ignore per-request model overrides report the actual model used
- Fix inject_model_override doc comment; add 3 unit tests
- Fix heartbeat doc comment ("routines" not "active routines")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add Jobs, Cost, Last Active columns to admin Users table

Add UserSummaryStats struct and user_summary_stats() batch query to the
UserStore trait (both PostgreSQL and libSQL backends). The admin users
list endpoint now fetches per-user aggregates (job count, total LLM
spend, most recent activity) in a single query and includes them inline
in the response. The frontend Users table displays three new columns.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review comments and CI formatting failures

CI fixes:
- cargo fmt fixes in cli/mod.rs and db/tls.rs

Security/correctness (from Copilot + serrrfirat + pranavraja99 reviews):
- Token create: reject expires_in_days > 36500 with 400 instead of silent clamp
- Token create: return 404 when admin targets non-existent user
- User create: map duplicate email constraint violations to 409 Conflict
- User create: remove unnecessary DB roundtrip for created_by (use AdminUser directly)
- DB auth: log warn on DB lookup failures instead of silently swallowing errors
- libSQL: add FK constraints on users.created_by and api_tokens.user_id

Config fixes:
- agent.multi_tenant: resolve from AGENT_MULTI_TENANT env var instead of hardcoding false
- heartbeat.multi_tenant: fix doc comment to match actual env-var-based behavior

UI fix:
- showTokenBanner: pass correct title ("Token created!" vs "User created!")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address remaining review comments (round 2)

- Secrets handlers: normalize name to lowercase before store operations,
  validate target user_id exists (returns 404 if not found)
- libSQL: propagate cost parsing errors instead of unwrap_or_default()
  in both user_usage_stats and user_summary_stats
- users_list_handler: propagate user_summary_stats DB errors (was
  silently swallowed with unwrap_or_default)
- loadUsers: distinguish 401/403 (admin required) from other errors
- Docs: fix users.id type (TEXT not UUID), remove "invitation flow"
  from V14 migration comment

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: i18n for Users tab, atomic user+token creation, transactional delete_user

i18n:
- Add 31 translation keys for all Users tab strings (en + zh-CN)
- Wire data-i18n attributes on HTML elements (headings, buttons, inputs,
  table headers, empty state)
- Replace all hard-coded strings in app.js with I18n.t() calls

Atomic user+token creation:
- Add create_user_with_token() to UserStore trait
- PostgreSQL: wraps both INSERTs in conn.transaction() with auto-rollback
- libSQL: wraps in explicit BEGIN/COMMIT with ROLLBACK on error
- Handler uses single atomic call instead of two separate operations

Transactional delete_user for libSQL:
- Wrap multi-table DELETE cascade in BEGIN/COMMIT transaction
- ROLLBACK on any error to prevent partial cleanup / inconsistent state
- Matches the PostgreSQL implementation which already used transactions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert V14 migration to match deployed checksum [skip-regression-check]

Refinery checksums applied migrations — editing V14__users.sql after
it was already applied causes deployment failures. Revert the cosmetic
comment changes (added in df40b22) to restore the original checksum.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: bootstrap onboarding flow for multi-tenant users

The bootstrap greeting and workspace seeding only ran for the owner
workspace at startup, so new users created via the admin API never
received the welcome message or identity files (BOOTSTRAP.md, SOUL.md,
AGENTS.md, USER.md, etc.).

Three fixes:
- tenant_ctx(): seed per-user workspace on first creation via
  seed_if_empty(), which writes identity files and sets
  bootstrap_pending when the workspace is truly fresh
- handle_message(): check take_bootstrap_pending() on the tenant
  workspace (not the owner workspace) and persist the greeting to
  the user's own assistant conversation + broadcast via SSE
- WorkspacePool: seed new per-user workspaces in the web gateway
  so memory tools also see identity files immediately

The existing single-user bootstrap in Agent::run() is preserved for
non-multi-tenant deployments.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address remaining PR review comments (round 3)

- Docs: fix metadata description from "merge patch" to "full replacement"
- Secrets: reject expires_in_days > 36500 with 400 (was silently clamped)
- libSQL: CAST(SUM(cost) AS TEXT) in user_usage_stats and user_summary_stats
  to prevent SQLite numeric coercion from crashing get_text() — this was
  the root cause of the Copilot "SUM returns numeric type" comments
- Add 3 regression tests: user_summary_stats (empty + with data) and
  user_usage_stats (multi-model aggregation)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add role change support for users (admin/member toggle)

- Add update_user_role() to UserStore trait + both backends (PostgreSQL
  and libSQL)
- Extend PATCH /api/admin/users/{id} to accept optional "role" field
  with validation (must be "admin" or "member")
- Add "Make Admin" / "Make Member" toggle button in Users table actions
- Add i18n keys for role change (en + zh-CN)
- Update API docs to document the role field on PATCH
- Fix test helpers to use fmt_ts() for timestamps (was using SQLite
  datetime('now') which produces incompatible format for string comparison)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show live LLM spend in Users table instead of only DB-recorded costs [skip-regression-check]

Chat turns record LLM cost in CostGuard (in-memory) but don't create
agent_jobs/llm_calls DB rows — those are only written for background
jobs. The Users table was querying only from DB, so it showed $0.00
for users who only chatted.

Now supplements DB stats with CostGuard.daily_spend_for_user() —
the same source displayed in the status bar token counter. Shows
whichever is larger (DB historical total vs live daily spend).

Also falls back to last_login_at for "Last Active" when no DB job
activity exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: persist chat LLM calls to DB and fix usage stats query

Two root causes for zero usage stats:

1. ChatDelegate only recorded LLM costs to CostGuard (in-memory) —
   never to the llm_calls DB table. Added DB persistence via
   TenantScope.record_llm_call() after each chat LLM call, with
   job_id=NULL and conversation_id=thread_id.

2. user_summary_stats query only joined agent_jobs→llm_calls, missing
   chat calls (which have job_id=NULL). Redesigned query to start from
   llm_calls and resolve user_id via COALESCE(agent_jobs.user_id,
   conversations.user_id) — covers both job and chat LLM calls.

Both PostgreSQL and libSQL queries updated. TenantScope gets
record_llm_call() method. Tests updated for new query semantics.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review comments — input validation, cost semantics, panic safety [skip-regression-check]

- Validate display_name: trim whitespace, reject empty strings (create + update)
- Validate metadata: must be a JSON object, return 400 if not (admin + profile)
- secrets_list_handler: verify target user_id exists before listing
- Cost display: use DB total directly (chat calls now persist to DB),
  remove confusing max(db,live) CostGuard fallback
- CatchPanicLayer: truncate panic payload to 200 chars in log to limit
  potential sensitive data exposure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Copilot round 5 — docs, secrets consistency, token name, provider field [skip-regression-check]

- Docs: users.id note updated to "typically UUID v4 strings (bootstrap
  admin may use a custom ID)"
- secrets_list_handler: return 503 when DB store is None (was falling
  through to list secrets without user validation)
- tokens_create: trim + reject empty token name (matching display_name
  pattern)
- LlmCallRecord.provider: use llm_backend ("nearai","openai") instead
  of model_name() which returns the model identifier
- user_summary_stats zero-LLM users: acceptable — handler already falls
  back to 0 cost and last_login_at for missing entries

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: DB auth returns 503 on outage, scheduler counts only blocking jobs

From serrrfirat review:
- DB auth: return Err(()) on database errors so middleware returns 503
  instead of silently returning Ok(None) → 401 (auth miss)
- Scheduler: add parallel_blocking_count_for() that uses
  is_parallel_blocking() (Pending/InProgress/Stuck) instead of
  is_active() for per-user concurrency — Completed/Submitted jobs
  no longer count against MAX_JOBS_PER_USER

From Copilot:
- CLAUDE.md: fix secrets route paths from {id} to {user_id}
- token_hash: use .as_slice() instead of .to_vec() to avoid
  heap allocation on every token auth/creation call

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: immediate auth cache invalidation on security-critical actions (zmanian review nearai#6)

Add DbAuthenticator::invalidate_user() that evicts all cached entries
for a user. Called after:
- Suspend user (immediate lockout, was 60s delay)
- Activate user (immediate access restoration)
- Role change (admin↔member takes effect immediately)
- Token revocation (revoked token can't be reused from cache)

The DbAuthenticator is shared (via Clone, which Arc-clones the cache)
between the auth middleware and GatewayState, so handlers can evict
entries from the same cache the middleware reads.

Also from zmanian's review:
- Items 1-5, 7-11 were already resolved in prior commits
- Item 12 (String→enum for status/role) is deferred as a broader refactor

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: last-admin protection, usage stats for chat calls, UTF-8 safe panic truncation

Last-admin protection:
- Suspend, delete, and role-demotion of the last active admin now
  return 409 Conflict instead of succeeding and locking out the admin API
- Helper is_last_admin() checks active admin count before destructive ops

Usage stats:
- user_usage_stats() now includes chat LLM calls (job_id=NULL) by
  joining via conversations.user_id, matching user_summary_stats()
- Both PostgreSQL and libSQL queries updated

Panic handler:
- Use floor_char_boundary(200) instead of byte-index [..200] to
  prevent panic on multi-byte UTF-8 characters in panic messages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: workspace seed race, bootstrap atomicity, email trim, secrets upsert response [skip-regression-check]

- WorkspacePool: await seed_if_empty() synchronously after inserting
  into cache (drop lock first to avoid blocking), so callers see
  identity files immediately instead of racing a background task
- Bootstrap admin: use create_user_with_token() for atomic user+token
  creation, matching the admin create endpoint
- Email: trim whitespace, treat empty as None to prevent " " being
  stored and breaking uniqueness
- Secrets PUT: report "updated" vs "created" based on prior existence
- Last token_hash.to_vec() → .as_slice() in authenticate_token

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: disable unscoped webhook endpoint in multi-tenant mode [skip-regression-check]

The original /api/webhooks/{path} endpoint looks up routines across all
users. In multi-tenant mode, anyone who knows the webhook path + secret
could trigger another user's routine. Now returns 410 Gone with a
message pointing to the scoped endpoint /api/webhooks/u/{user_id}/{path}.

Detection uses state.db_auth.is_some() — present only when DB-backed
auth is enabled (multi-tenant). Single-user deployments are unaffected.

From: standardtoaster review comment

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: webhook multi-tenant check, secrets error propagation, stale doc comment [skip-regression-check]

- Webhook: use workspace_pool.is_some() instead of db_auth.is_some()
  for multi-tenant detection — db_auth is set for any DB deployment,
  workspace_pool is only set when has_any_users() was true at startup
- Secrets: propagate exists() errors instead of unwrap_or(false) so
  backend outages surface as 500 rather than incorrect "created" status
- Config: fix stale workspace_read_scopes comment referencing user_id

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
serrrfirat added a commit that referenced this pull request May 29, 2026
- Fix product_workflow test compile errors:
  - Add missing mark_continuation_dispatched to RecordingFlowManager impl
  - Add missing continuation_emitted_at field to AuthFlowRecord literal

- Move consumed_at write to after successful account write in submit_manual_token
  (finding #6: one-shot consume used to burn the interaction on transient failure)

- Add MAX_SECRET_LEN (65536 bytes) to validate_secret
  (finding #7: no max length on submitted secrets)

All 234 lib tests green. Clippy clean on affected packages.
ilblackdragon added a commit that referenced this pull request Jun 1, 2026
* Reborn budgets: address all #3841 follow-ups end-to-end

Implements every open follow-up from PR #3841 (cost-based budgets
foundation), driven by the plan in
`docs/plans/2026-05-22-reborn-budgets-followups.md`:

- **C2 (provider tokens)**: `LoopModelResponse.usage` carries real
  `(input_tokens, output_tokens)` from `CompletionResponse` /
  `ToolCompletionResponse`; `usage_for_response` reconciles to actual
  USD via the cost table instead of the conservative estimate.
- **D1 (cascade warnings)**: `CascadeOutcome` variants carry
  `Vec<BudgetWarning>` so warnings preceding a pause or hard deny
  reach the audit sink. `ResourceError::LimitExceeded` /
  `RequiresApproval` reshaped to struct variants.
- **C1 (cancellation safety)**: new
  `LoopModelBudgetAccountant::release_in_flight` trait hook + RAII
  `ReservationReleaseGuard` in `HostManagedLoopModelPort::stream_model`
  so a cancelled future doesn't orphan its reservation.
- **E1 (dead code)**: removed the never-set `budget_accountant` field
  on `ThreadBackedLoopModelPort`.
- **Real cost table**: new `StaticModelCostTable` +
  `LlmModelProfilePolicy::build_cost_table()` populated from
  `ironclaw_llm::costs::model_cost` with `default_cost` fallback so
  unknown providers never silently reconcile to zero.
- **B1 (filesystem gate store)**: new `FilesystemBudgetGateStore`
  mirroring `FilesystemResourceGovernorStore`; pending gates survive
  process restart.
- **A1 (production wiring)**: composition builds
  `GovernorBackedAccountant` from the cost table + governor and
  threads it through `RebornLoopDriverHostFactory::with_model_budget_accountant`.
- **A2 (audit / SSE projection)**:
  `InMemoryResourceGovernor::with_event_sink` emits `Reserved`,
  `Reconciled`, `Released`, `Warned`, `ApprovalRequested`, `Denied`,
  `LimitChanged`; composition holds an `InMemoryBudgetEventSink` ready
  for downstream SSE projection.
- **F1 (stuck-loop normalization)**: `CapabilityCallSignature::from_call`
  now runs `progress::normalize_for_hash` so the existing repetition
  window collapses request-id / UUID / timestamp noise.

Side fix: `ResourceValue` moved to adjacent serde tagging (the
combination of internal tagging + `Decimal`'s `serde-with-str`
representation breaks JSON serialization — rust-lang/serde#1402).

Regression tests added per item — see the acceptance evidence appendix
in the plan doc.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Reborn budgets: end-to-end test coverage via test-support feature

Adds 13 e2e tests covering the budget pipeline through
`build_reborn_runtime` + `send_user_message`. Required infrastructure:

- **`test-support` feature** on `ironclaw_reborn_composition` exposing
  `BudgetTestGateway` (scripted token usage) and
  `RebornRuntimeInputTestExt`. Existing `model_gateway_override` field
  promoted from `#[cfg(test)]` to `#[cfg(any(test, feature = ...))]`
  with a new public `with_model_gateway_override_for_tests` setter.
- **Cost-table override** on `RebornRuntimeInput` so tests can pair
  the gateway with a deterministic `ModelCostTable`. Without this, an
  override gateway dropped the cost table and the accountant never
  fired.
- **Budget accessors** on `RebornRuntime`: `budget_resource_governor`,
  `budget_event_sink`, `budget_gate_store`, and
  `apply_resolved_budget_gate`. Test-feature gated.
- **`ResourceGovernor::usage_for`** added as a default-impl trait
  method so tests read spend through the trait surface.
- **`BudgetGateStore` wired into the accountant**:
  `GovernorBackedAccountant::with_gate_store(...)` opens a pending
  gate whenever the governor cascade returns `RequiresApproval`. The
  approval-required host error is unchanged; the gate is the
  out-of-band channel a user-facing handler resolves.

Scenarios covered:

| # | Test | What it asserts |
|---|---|---|
| F1 | `f1_happy_path_records_actual_usd_in_ledger` | Ledger depletes by provider tokens × cost table |
| F2 | `f2_crossing_warn_threshold_emits_warned_event` | Warn fires alongside successful Reserved/Reconciled |
| F3 | `f3_approval_with_increased_limit_unblocks_retry` | Approve → set_limit applies → retry succeeds |
| F4 | `f4_cancel_keeps_budget_blocked_on_retry` | Cancel → retry still short-circuits |
| F5 | `f5_expiry_marks_gate_terminal_and_keeps_budget_blocked` | Expiry → gate drops from pending list, retry still blocked |
| F6 | `f6_hard_cap_denied_before_provider_call` | Estimate over cap → zero model calls, Denied event |
| C1 | `c1_provider_tokens_reconcile_to_actual_usd` | Real numbers, not estimate |
| C2 | `c2_unknown_model_in_cost_table_reconciles_to_zero` | Unknown profile → zero spend |
| C3 | `c3_zero_cost_model_records_zero_spend` | Free model → zero USD with non-zero tokens |
| D1 | `d1_agent_deny_preserves_user_warn_event` | Cascade emits both Warned and Denied |
| D3 | `d3_fresh_user_without_limits_runs_without_denial` | No limit → no denial |
| + | `pause_in_distinct_runs_produces_distinct_pending_gates` | Per-run gate identity |
| + | `budget_test_gateway_scripted_replies_drive_per_turn_costs` | Multi-turn scripted accumulation |

F7 (cancellation mid-stream) is unit-covered by
`release_in_flight_drains_orphan_reservation_on_cancellation`.
D2 (period rollover) is unit-covered by
`rolling_24h_snapshot_reports_anchored_window_not_now_window`.
B-series (background ticks) await the BackgroundKind scheduler
call site (no production caller in Reborn yet).

Run via `cargo test -p ironclaw_reborn_composition --features test-support`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Budget review feedback: address all 7 findings from PR #3899 review

Two High and five Medium issues raised by serrrfirat's multi-agent review.

**High #1 — `FilesystemBudgetGateStore` cross-tenant leakage**
The store hardcoded `ResourceScope::system()` for every op, so all
tenants wrote into the same `/tenants/__SYSTEM__/...` snapshot and
`list_pending` would expose gates across tenants. Fix: `new(...)` now
takes a `ResourceScope`; each tenant gets its own store, and the
`ScopedFilesystem` mount view routes the snapshot under that tenant's
path. Added `list_pending_does_not_leak_across_tenants` regression.

**High #2 — accountant wired without default budget limits**
Composition built `GovernorBackedAccountant` without
`with_seeding_policy`, so the local-dev governor started empty and
`reserve_with_outcome_in_state` skipped accounts that had no
configured limit — model calls reconciled spend but never enforced a
cap. Fix: `build_reborn_runtime` now loads
`BudgetDefaults::compiled_defaults().with_env()` and wires
`BudgetSeedingPolicy` + `with_overestimate_factor`. Renamed the D3
test to `d3_seeding_policy_installs_default_cap_on_first_touch` to
prove the wiring fires.

**Medium #3 — RAII guard disarmed before post_model_call await**
`HostManagedLoopModelPort::stream_model` was disarming the
`ReservationReleaseGuard` before awaiting `post_model_call`. A
cancellation during that await dropped the future without cleanup,
orphaning the reservation. Fix: disarm AFTER `post_model_call`
returns. `release_in_flight` is now idempotent (peek-then-release-
then-remove) so a successful post-call + subsequent guard drop is a
no-op.

**Medium #4 — failed release drops the retry handle**
`release_in_flight` removed the in-flight entry before calling
`governor.release`. A transient storage error left the reservation
active in the governor with the id discarded. Fix: peek first,
release, only remove on success. Errors keep the entry retained for
a future retry / cleanup hook.

**Medium #5 — unknown model silently reconciles to zero USD**
Both `estimate_for` and `usage_for_response` fell back to
`ModelCost { 0, 0, 0 }` when the cost table had no entry for the
effective model. Cost-table drift would silently bypass daily caps.
Fix: `GovernorBackedAccountant` carries a `default_cost` (default ~
GPT-4o pricing, ~`$0.0000025 input + $0.00001 output per token`) used
for unknown models. Callers wiring `ZeroCostTable` for free / Ollama
explicitly opt out of the fallback. Updated the C2 e2e test to
assert the new fail-closed shape.

**Medium #6 — paused dimension lost when another hard-denies**
`check_thresholds_all_interventions` stored `Approval` only in the
`approval` slot, so when one dimension paused and another hard-denied,
the `Deny { warnings, denial }` outcome lost the pause signal.
Fix: also push a warning-shaped record for the paused dimension.

**Medium #7 — unbounded terminal-gate retention**
The snapshot kept every gate forever; `open` / `resolve` / `get` /
`list_pending` were O(total historical gates). Fix:
`with_terminal_retention` (default 30 days). Every mutation prunes
terminal gates whose resolution timestamp is older than the window.
Added `terminal_gates_older_than_retention_are_pruned_on_next_write`
regression.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci: replace lock-poisoned expects with PoisonError::into_inner

scripts/check_no_panics.py flagged five .expect("...lock poisoned")
calls in the new test_support.rs. Use the same idiomatic recovery
pattern the rest of the codebase uses (see InMemoryBudgetGateStore,
InMemoryBudgetEventSink): on a poisoned lock, recover the inner data
via PoisonError::into_inner rather than panicking. The test gateway's
state is append-only logs / replies queues, so reading them through a
poisoned lock is safe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Finish A1 / A2 / F1 from plan + honest plan doc update

The plan claimed "all nine items landed" but A1 (production wiring),
A2 (SSE projection), and F1 (full progress strategy) were partials.
This commit finishes the work so the plan matches reality.

**A1 — production-shape accountant builder**

New `ironclaw_reborn_composition::build_default_budget_accountant`
public helper that wires the seeding policy + overestimate factor +
gate store from `BudgetDefaults::compiled_defaults().with_env()` and
returns an `Arc<dyn LoopModelBudgetAccountant>`. Production loop
composers call this with their `PersistentResourceGovernor` +
`FilesystemBudgetGateStore` + LLM-policy-derived cost table; the
local-dev runtime in `build_reborn_runtime` now uses the same helper
instead of duplicating the seeding logic inline. Unit-tier regression
`seeds_compiled_default_user_cap_on_first_touch` proves the helper
installs the compiled-default $5 user cap on first model call.

**A2 — broadcast sink + AppEvent projection**

- `ironclaw_resources::BroadcastBudgetEventSink` wraps
  `tokio::sync::broadcast::Sender<BudgetEvent>` with `subscribe()` /
  `subscriber_count()`. `CompositeBudgetEventSink` fans events to
  multiple sinks.
- Composition fans every `BudgetEvent` to the in-memory sink (for
  tests) AND the broadcast sink (for SSE projection) via
  `CompositeBudgetEventSink`.
- New `AppEvent::BudgetWarn` / `BudgetPause` / `BudgetDenied` /
  `BudgetLimitChanged` wire-stable variants in
  `ironclaw_common::event`.
- `src/bridge/budget_events.rs` carries the projection: a tokio task
  spawned by `spawn_budget_event_projection` drains the broadcast
  receiver and emits the appropriate `AppEvent` via
  `SseManager::broadcast_for_user`. System-scoped events (no user
  identity) are skipped. This is the only producer of these
  `AppEvent` variants per `.claude/rules/gateway-events.md`.
- `RebornRuntime::broadcast_budget_event_sink()` exposes the sink to
  the binary so the startup path subscribes. E2E test
  `broadcast_sink_publishes_events_to_subscribers` drives a real
  `send_user_message` and asserts Reserved + Reconciled lands on the
  broadcast.

**F1 — diminishing-returns stop condition**

The earlier shipped `ParamHash` normalization in
`CapabilityCallSignature` strengthened the existing
`recent_call_signatures`-based repetition detector. This commit adds
the second half of F1: a rolling output-token window that detects
"wedged" loops the repetition detector misses (model keeps
responding but produces no useful output).

- `LoopExecutionState.recent_output_token_counts: BoundedRing<u32, 8>`
  populated by the executor from `LoopModelResponse::usage`.
- `BoundedRing::iter` returns `impl DoubleEndedIterator` so the
  strategy can scan the trailing window.
- `DefaultStopConditionStrategy` gets `min_delta_tokens` (default
  4) + `noprogress_window` (default 4). When the last N turns all
  produce ≤ min_delta_tokens of output, fire
  `StopKind::NoProgressDetected`.
- Regression tests:
  `four_consecutive_low_token_turns_trigger_no_progress` proves the
  detector fires; `occasional_low_token_turn_does_not_trip_no_progress`
  proves a productive turn resets the trailing count.

**Plan doc**

Updated the status header from "all nine items landed" to the
honest per-item shape. Acceptance evidence table expanded with the
new test names. New "Review-feedback fixes layered on top" subsection
documenting all 2 High + 5 Medium findings addressed during review.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Address thermo-nuclear review: collapse filesystem-store duplication, flatten cfg permutations, split budget accountant

Five structural simplifications surfaced by the deep audit on PR #3899, plus
two bug fixes from the earlier review pass:

- ironclaw_resources: extract `cas_snapshot` shared infrastructure
  (`StorageError` + `Snapshot` traits + `CasSnapshotStore<F>` + async-runtime
  worker + per-path lock map) and merge `filesystem_gate_store.rs` into
  `filesystem_store.rs`. Deletes ~350 lines of duplicated read-modify-write +
  worker-thread + CAS machinery; both stores are now thin shims over the
  shared helper.

- ironclaw_reborn_composition: flatten the 4-way cfg permutation in
  `build_reborn_runtime` model-gateway resolution into three flat steps
  (normalize override → build production gateway via cfg-gated helper →
  test override wins). Also drops the `unused_mut` warning.

- ironclaw_reborn_composition: collapse the 3-layer test-only setter dance
  for `model_gateway_override` / `model_cost_table_override` into a single
  setter pair gated on `cfg(any(test, feature = "test-support"))`. Deletes
  the `RebornRuntimeInputTestExt` extension trait — integration tests now
  call the inherent methods directly.

- ironclaw_loop_support: split the 1305-line `budget_accountant.rs` into
  `budget_cost_table.rs` (ModelCost/ModelCostTable trait/ZeroCostTable/
  StaticModelCostTable), `budget_seeding.rs` (BudgetSeedingPolicy), and
  `budget_accountant.rs` (just GovernorBackedAccountant). Each module now
  owns one concern.

- ironclaw_resources: add `impl Display for ResourceAccount` and route the
  hierarchical account-label rendering through it; delete the 60-line
  bespoke `account_label` helper from `src/bridge/budget_events.rs`.

- ironclaw_common + bridge: collapse the four `AppEvent::Budget*` variants
  into a single typed `AppEvent::Budget(AppBudgetEvent)` with the four
  shapes carried inside the enum. Wire-shape stays identical (snake_case
  serde tag).

- ironclaw_resources + ironclaw_loop_support: thread real gate id through
  `BudgetEvent::GateOpened { gate_id, needed, at }` (new variant) and have
  the accountant emit it via the broadcast event sink after store.open
  succeeds. The bridge now projects `BudgetEvent::GateOpened` (not
  `ApprovalRequested`) into `AppEvent::Budget(Pause { gate_id, ... })` so
  SSE consumers receive the persisted gate id rather than a fabricated
  zero uuid.

- ironclaw_agent_loop: in the F1 token-counting path, push to
  `recent_output_token_counts` only when the model response carries
  `Some(usage)` and only on the `AssistantReply` arm (instead of
  `unwrap_or(0)`). Diminishing-returns detection now reflects real spend.

Net delta: -461 lines (+999 / -1460). Workspace `cargo clippy` clean,
`cargo test` clean on ironclaw_resources / ironclaw_loop_support /
ironclaw_reborn_composition; budget_e2e + budget_approval_e2e both green.

Pre-existing CI failures (`cli::tests::test_version` stack overflow,
`facade_factory::production_*` RuntimeProcessPort missing) are unrelated
and reproduce on the pristine branch tip.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(cli): refresh insta snapshots after runtime-policy flag additions

The `import`-feature variants of the help snapshots were left stale when
`--deployment-mode`, `--runtime-profile`, `--yolo-disclosure` were added in
cc04481 (#3243); the `_without_import` variants were updated but these
were not. CI was failing the snapshot assertion under the slim PR matrix
(`--features postgres,libsql,html-to-markdown,bedrock,import`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Address PR #3899 thermo-nuclear review (TN #1, #2, #3)

TN #1 — budget defaults resolved in wrong layer:
  - `build_default_budget_accountant` no longer reads process env; it
    now takes `&BudgetDefaults` as a parameter and the caller owns the
    config-layer precedence (compiled → section → env) plus the
    `validate()` call.
  - `RebornRuntimeInput` gains an optional `budget_defaults` field +
    `with_budget_defaults()` builder so the composition root passes a
    pre-resolved value. `build_reborn_runtime` falls back to
    `compiled_defaults().with_env() + validate()` when none is supplied
    so existing call sites keep working.

TN #2 — gate-store scoping at wrong boundary:
  - `BudgetGateStore` trait methods (`open`, `resolve`,
    `expire_pending_older_than`, `get`, `list_pending`) now take
    `&ResourceScope` as first arg. `GovernorBackedAccountant` passes
    the caller's scope from `resource_scope(context)`.
  - `CasSnapshotStore` gains `update_with_scope` so the same store
    instance can route per-operation. `FilesystemBudgetGateStore` no
    longer takes scope at construction — one shared instance serves
    every tenant via the `ScopedFilesystem` mount view.
  - `InMemoryBudgetGateStore` ignores scope (suitable for single-tenant
    tests / local-dev); production multi-tenant filesystem path is
    correctly partitioned by `ResourceScope`.
  - `RebornRuntime::apply_resolved_budget_gate` now takes scope too.

TN #3 — half-wired projection bridge:
  - Removed `src/bridge/budget_events.rs`, its `spawn_budget_event_projection`
    helper, the `AppEvent::Budget` variant, and the `AppBudgetEvent`
    type. No production caller ever subscribed the broadcast sink
    onto SSE and no frontend consumed the variant, so the
    half-wired bridge is gone pending a real owner that spawns a
    projection task with shutdown cancellation.
  - The runtime's `broadcast_budget_event_sink()` accessor stays so
    a future production composer can still subscribe without
    rebuilding the runtime.

Bonus — to keep budget e2e tests working under the new libsql local-
dev path that origin/reborn-integration introduced, added
`PersistentResourceGovernor::with_event_sink` (parity with the
`InMemoryResourceGovernor` accessor). The libsql variant of
`build_local_dev_store_graph` now wires the composite sink to the
persistent governor so governor-emitted `Warned`/`Denied`/`Reserved`/
`Reconciled` events reach subscribers on both feature paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Wire budget-event projection task into RebornRuntime

Re-implements PR #3899 follow-up A2 / Thermo-Nuclear #3 with a real
production owner instead of leaving the broadcast sink half-wired:

- `crates/ironclaw_reborn_composition/src/budget_events.rs` (new):
  `BudgetEventObserver` trait + `TracingBudgetEventObserver` default
  observer + crate-internal `BudgetEventProjection` task that drains
  the runtime's broadcast `Receiver<BudgetEvent>` and forwards every
  event to the observer. Cancellation via `CancellationToken`; lagged
  subscribers logged and resumed; receiver-closed exits cleanly.

- `RebornRuntimeInput::with_budget_event_observer(...)` lets
  production owners install a custom observer (SSE projection, WS
  fan-out, telemetry export). When unset, the runtime installs the
  tracing observer so events always surface in structured logs.

- `build_reborn_runtime` always spawns the projection task at runtime
  construction; `RebornRuntime::shutdown` cancels it and awaits the
  handle so background state drains before the runtime drops.

- E2E test `projection_delivers_budget_events_to_installed_observer`
  drives `build_reborn_runtime` with a capturing observer and asserts
  the observer sees `Reserved` + `Reconciled` from a real model call,
  testing through the caller per `.claude/rules/testing.md`.

- Existing `broadcast_sink_publishes_events_to_subscribers` updated
  to expect the runtime's own projection task as a baseline
  subscriber.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(reborn): rustfmt the merged loop_support import block

The conflict resolution for the post-merge import list was not run
through rustfmt; CI Formatting flagged the wrapping. No logic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
pranavraja99 pushed a commit that referenced this pull request Jun 12, 2026
Resolves Henry + Firat review comments on #4588:

- Composition-owned RebornTrajectoryObserver trait + adapter to the
  loop-support CapabilityTrajectoryObserver, instead of re-exporting the
  substrate trait directly (CLAUDE.md: facade-shaped handles only). Loop-support
  contract changes no longer break the public Reborn API. (Henry#8)

- Safe-preview by default: with_trajectory_observer now forwards bounded
  (truncated strings / capped arrays) payloads so a logs/UI/telemetry sink stays
  within the model-visible display boundary; a trusted in-process consumer that
  needs verbatim tool I/O opts in via the new with_raw_trajectory_observer.
  (Henry#5)

- catch_unwind around both observer call sites (input hook in capability_port,
  result hook in LocalDevCapabilityIo) so a panicking observer can't unwind the
  capability hot path; trait doc now states the never-block / panic-caught
  contract. (Henry#1/#6)

- e2e test local_dev_runtime_forwards_tool_call_trajectory_to_raw_observer:
  drives a real build_reborn_runtime turn dispatching builtin.echo and asserts
  BOTH input and result callbacks fire on the genuine dispatch path — honest
  coverage that replaces the dropped direct-call result-hook test, and proves
  the observer threads through build_reborn_runtime. (Firat#1, Henry#3/#7)

- Strengthened provider-injection docs: the config-vs-override invariant and why
  the feature-gated seam takes the LlmProvider substrate trait. (Henry#4/#9/#11)

- Fixed the LocalDevCapabilityIo observer field comment to describe its actual
  result-only responsibility. (Henry#10)

Provider-override coverage (Firat#2/Henry#2) already landed in
build_llm_gateway_drives_provider_override_not_config.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zmanian pushed a commit to zmanian/ironclaw that referenced this pull request Jun 15, 2026
…r injection (nearai#4588)

* feat(reborn): expose a trajectory observer hook on RebornRuntimeInput

The reborn runtime is sealed: build_reborn_runtime returns only the final
AssistantReply, and per-step capability (tool) calls + results live in internal
stores. Downstream consumers (benchmark harnesses, UI/debuggers) can't observe
the agent's trajectory.

Add `RebornTrajectoryObserver` (pub trait: on_capability_input(call_id, name,
args) / on_capability_result(call_id, output)) and
`RebornRuntimeInput::with_trajectory_observer`. The local-dev capability IO
(`LocalDevCapabilityIo`) forwards each tool call's name+args (at input staging)
and result (at result write) to the observer when present — reusing the same
data it already records for display previews. No-op when unset; best-effort.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* debug: trace observer hook firing (temporary)

* feat(reborn): trajectory observer — capability_id on result, reliable spine

Provider tool calls are staged by a lower decorator that bypasses the
LocalDevCapabilityIo input path, so on_capability_input does not fire for
them. on_capability_result fires for every completed capability — make it
carry the capability_id so consumers can reconstruct the trajectory (name +
output) from results alone. Input args capture is a follow-up.

* feat(reborn): capture capability input args at the host port chokepoint

Provider tool calls are staged by ProviderToolCallInputResolver, which keeps
args in a private map and bypasses the capability-IO input hook — so inputs
never reached the trajectory observer (only results did). Move the observer
trait down to ironclaw_loop_support (CapabilityTrajectoryObserver, re-exported
from composition as RebornTrajectoryObserver) and hook it in
HostRuntimeLoopCapabilityPort::invoke_capability right after the input
resolves — the one place the model's resolved arguments are visible. Threaded
through HostRuntimeLoopCapabilityPortFactory + the local-dev factory. Result
hook unchanged. Now name + args + output are all captured.

* feat(reborn): host LLM-provider injection seam

ResolvedRebornLlm::with_provider — drive the runtime with a caller-supplied
LlmProvider (e.g. an instrumented wrapper that counts tokens/cost and captures
reasoning) instead of always building one from config; build_llm_gateway honors
the override. The only viable observability path for reborn, whose model calls
run in spawned worker tasks a per-task tracing subscriber can't reach.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(reborn): cover trajectory observer + LLM provider override seams

Addresses Firat's two blocking review findings on nearai#4588 (both
missing-integration-test, per AGENTS.md "test through the caller"):

1. Trajectory observer callbacks — drive the real call sites with a
   recording CapabilityTrajectoryObserver:
   - host port: invoke_capability via HostRuntimeLoopCapabilityPortFactory
     ::with_trajectory_observer asserts on_capability_input fires with the
     resolved capability id + tool-call arguments.
   - local-dev IO: register_provider_tool_call_input + write_capability_result
     assert on_capability_input and on_capability_result fire and correlate by
     input ref.

2. LLM provider override — build_llm_gateway_drives_provider_override_not_config
   injects a counting mock via ResolvedRebornLlm::with_provider, points config
   at a dead endpoint, and asserts the gateway returns the mock's sentinel
   (proving the override is driven, not a config-built chain).

Also fixes pre-existing breakage this surfaced: 5 LocalDevLoopCapabilityPort
Factory test initializers (shell_tests.rs + tests.rs) were missing the
trajectory_observer field added by this PR, so the composition crate's tests
did not compile under --features root-llm-provider.

loop_support: 301 passed; composition (root-llm-provider): 520 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(reborn): make trajectory observer input semantics consistent

Addresses Copilot's follow-up findings on the observer seam:

- Drop the `on_capability_input` callback from `LocalDevCapabilityIo::
  register_provider_tool_call_input`. It forwarded the raw provider tool
  name (`builtin_echo`) as the capability id — conflicting with the
  observer contract (resolved dotted `builtin.echo`) and the authoritative
  port-level hook — and `ProviderToolCallInputResolver` doesn't delegate
  here for provider tool calls, so it never fired in practice anyway.
  `HostRuntimeLoopCapabilityPort::invoke_capability` remains the single
  source of `on_capability_input` (resolved id); `LocalDevCapabilityIo`
  remains the source of `on_capability_result`.

- Clarify the trait doc: `arguments` is the raw model-emitted tool-call
  input resolved from the input ref (the callback fires before schema
  normalization), which is what the trajectory should record.

- Refocus the local-dev test on `on_capability_result` forwarding +
  correlation, and assert input staging does NOT emit `on_capability_input`
  from local-dev IO. Port-level input semantics stay covered by the
  capability_port.rs test.

loop_support: 301 passed; composition (root-llm-provider): 520 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* wire trajectory_observer through RefreshingLocalDevCapabilityPortConfig

Completes the main-merge conflict resolution: local_dev.rs passes
trajectory_observer into the refreshing-port config, so the config struct +
port struct must carry it and build_inner must apply it via
.with_trajectory_observer(). (Missed staging this file in the merge commit.)

* test(reborn): lock down the observability seams against regression

nearai#4588 exposes two seams a downstream harness relies on. Add tests so a
future refactor can't silently break either:

- capability_io_forwards_result_to_trajectory_observer: drives
  write_capability_result and asserts on_capability_result fires with the
  correct (call_id, capability_id, output) — the result half of the
  trajectory observer (tool-call outputs).
- build_llm_gateway_drives_provider_override_not_config: asserts the gateway
  drives a provider injected via ResolvedRebornLlm::with_provider (config
  points at a dead endpoint), proving the provider-injection seam works —
  this is how the bench captures reasoning / tokens / cost / system-prompt /
  tool-definitions. (Restores the test dropped during the main merge.)

The input half (on_capability_input) is already covered by
invoke_capability_forwards_resolved_input_to_trajectory_observer in
ironclaw_loop_support. All three pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(reborn): drop the false-confidence result-hook test

capability_io_forwards_result_to_trajectory_observer called
write_capability_result directly, so it stayed green even though the
result hook is unreachable end-to-end while capability dispatch fails
(the LocalDevYolo InputEncode regression) — i.e. it did not fail when
the feature it claimed to cover was actually broken. Remove it rather
than ship false confidence.

The result hook lives in LocalDevCapabilityIo and is only reached by a
real local-dev runtime turn, so an honest guard must drive the full
runtime and is red until the dispatch regression is fixed; that guard
belongs as an end-to-end test (PR, once green) or a bench pre-flight,
not a direct-call unit test.

Kept: invoke_capability_forwards_resolved_input_to_trajectory_observer
(input hook, real port code path) and
build_llm_gateway_drives_provider_override_not_config (provider seam) —
both genuinely fail if their seam regresses.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(reborn): address review on the trajectory-observer + provider seams

Resolves Henry + Firat review comments on nearai#4588:

- Composition-owned RebornTrajectoryObserver trait + adapter to the
  loop-support CapabilityTrajectoryObserver, instead of re-exporting the
  substrate trait directly (CLAUDE.md: facade-shaped handles only). Loop-support
  contract changes no longer break the public Reborn API. (Henry#8)

- Safe-preview by default: with_trajectory_observer now forwards bounded
  (truncated strings / capped arrays) payloads so a logs/UI/telemetry sink stays
  within the model-visible display boundary; a trusted in-process consumer that
  needs verbatim tool I/O opts in via the new with_raw_trajectory_observer.
  (Henry#5)

- catch_unwind around both observer call sites (input hook in capability_port,
  result hook in LocalDevCapabilityIo) so a panicking observer can't unwind the
  capability hot path; trait doc now states the never-block / panic-caught
  contract. (Henry#1/nearai#6)

- e2e test local_dev_runtime_forwards_tool_call_trajectory_to_raw_observer:
  drives a real build_reborn_runtime turn dispatching builtin.echo and asserts
  BOTH input and result callbacks fire on the genuine dispatch path — honest
  coverage that replaces the dropped direct-call result-hook test, and proves
  the observer threads through build_reborn_runtime. (Firat#1, Henry#3/nearai#7)

- Strengthened provider-injection docs: the config-vs-override invariant and why
  the feature-gated seam takes the LlmProvider substrate trait. (Henry#4/nearai#9/nearai#11)

- Fixed the LocalDevCapabilityIo observer field comment to describe its actual
  result-only responsibility. (Henry#10)

Provider-override coverage (Firat#2/Henry#2) already landed in
build_llm_gateway_drives_provider_override_not_config.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(reborn): second-round review fixes on the trajectory/provider seams

Addresses Henry's review of the first round (nearai#4588):

- safe_preview_value now bounds objects (entry cap), recursion depth, and total
  nodes — not just strings/arrays — so a wide or deeply nested capability result
  can't force unbounded traversal/allocation on the hot path. (3405419089)

- Narrowed the loop-support CapabilityTrajectoryObserver to input-only:
  HostRuntimeLoopCapabilityPort never staged results through the port (results
  go via LoopCapabilityResultWriter), so advertising on_capability_result there
  was a contract a direct user could never see fire. Result observation stays on
  the composition path (LocalDevCapabilityIo). (3405419104)

- Synthetic capabilities (e.g. builtin.skill_activate) bypass the inner port's
  input hook, so the synthetic wrapper now emits on_capability_input itself after
  resolving input — otherwise consumers saw an unpaired result with no args.
  (3405419110)

- Provider injection no longer accepts a wholesale Arc<dyn LlmProvider> through
  the facade: with_provider is replaced by with_provider_factory, a decorator
  Fn(Arc<dyn LlmProvider>) -> Arc<dyn LlmProvider>. The composition always builds
  the provider from config (config stays the single construction source —
  collapses the old config-vs-override invariant too) and hands it to the factory
  to wrap. (3405419100, 3405419146)

- New caller-level test local_dev_runtime_safe_preview_observer_receives_bounded_payload:
  installs the default with_trajectory_observer, drives a real turn with a large
  echo payload, asserts the observer receives a truncated preview. (3405419095)

- Dropped the stale nearai#4588/main-rebase comment for a durable invariant. (3405419113)

cargo test (loop_support + reborn_composition, single-threaded) green; clippy
clean on touched files.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(reborn): rustfmt the trajectory/provider review changes

Formatting-only: import grouping + mod ordering in the two lib.rs re-export
blocks, and wrapping in runtime.rs / local_dev.rs / trajectory_observer.rs.
Fixes the Formatting + Code Style CI checks. No behaviour change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(reborn): drop std-Mutex guard before await in observer e2e tests

clippy::await_holding_lock (-D warnings): the two trajectory-observer e2e
tests held the observer's std::sync::Mutex guard across runtime.shutdown().await.
Shut down before inspecting the recorded callbacks (the data is already captured
during the turn) so no guard is held across an await. Fixes Clippy (all-features).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WIP(bench): http empty-body + multi-tool-call port reuse + final-answer nudge

Local checkpoint so the bench builds against a stable tree (uncommitted
edits were being reverted mid-session). Bundles: http body() empty-field
fix, RefreshingLocalDevCapabilityPort register reuse, the gated
final-answer nudge + interactive_profile gate flip, and the
trajectory-observer safe_preview borrow fix.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* style(reborn): wrap an over-long line for rustfmt 1.9.0

CI installs the latest stable rustfmt (1.9.0 / Rust 1.96), which wraps a
long eprintln! that older rustfmt left inline. Fixes the Formatting check.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(reborn): collapse nested if for clippy 1.96 collapsible_if

clippy 1.96 (CI's stable) flags the nested if-let in the final-answer-nudge
site as collapsible; fold it into a let-chain. No behaviour change. Fixes
Clippy (all-features).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WIP(bench): multi-tool-call port reuse (matches main nearai#4790)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* WIP(bench): nudge isolation - disable gate to measure marginal contribution

* Revert stray bench WIP accidentally committed onto this branch

Removes the http/nudge/multi-tool-call/diagnostic WIP commits
(c4bbb5f, 2c670b4, 6da818a) that were committed onto the
reborn-trajectory-observer branch by mistake during benchmarking and
swept to origin by a main-merge push. Restores the affected files to
origin/main (multi-tool-call is already fixed there by nearai#4790; the http
fix lives in PR nearai#4827). Observer-owned changes in state.rs,
refreshing_capability_port.rs, and local_dev.rs are preserved minus the
stray WIP additions. No history rewrite / force-push.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(reborn): preserve provider factory across reload + reject observer off local-dev

Addresses Firat's review of the trajectory/provider seams (nearai#4588):

- Provider factory now survives a live config reload. build_llm_gateway applied
  the factory to the bare config provider *before* wrapping it in the
  SwappableLlmProvider, so the first WebUI/settings reload (which swaps the
  swappable's inner) silently dropped the instrumentation wrapper. Invert the
  layering: build the config provider, put it behind the swappable + reload
  handle, then apply the factory *over the swappable* for the gateway-facing
  provider. Reloads swap the inner; the wrapper stays in the call path.
  Regression test provider_factory_survives_live_reload reloads and proves the
  wrapper still observes subsequent model calls.

- Reject a trajectory observer on profiles without a local runtime. The observer
  is wired only through the local-dev capability path; Production silently
  dropped it, so a caller got an empty trajectory with no error. Fail fast with
  InvalidArgument and document the seam as local-dev/bench-only. Test
  build_reborn_runtime_rejects_trajectory_observer_for_production.

cargo fmt + clippy (all-features, -D warnings) clean under rustfmt 1.9/clippy 1.96.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(reborn): note trajectory observer is local-dev/bench-only

Document the local-dev-only constraint + fail-fast behavior on the public
with_trajectory_observer / with_raw_trajectory_observer setters (Firat review).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Pranav Raja <pranav.raja@near.ai>
henrypark133 added a commit that referenced this pull request Jun 16, 2026
… (auth + approval)

Review finding #7: the denied-gate replay paths derived idempotency from
current run state (TurnStatus::is_terminal() guard) rather than replaying
through resume_turn. After the first Deny resumed the run, a transport
retry with the same idempotency key arriving after the runner completed
returned StaleGate/StaleAuth instead of the original ResumeTurnResponse —
the observable result depended on runner timing.

resume_turn is idempotent by key (memory.rs:665 returns the cached
Result from resume_idempotency before the precondition check). Both the
approval (replay_denied_gate) and auth (resume_denied_auth replay arm)
paths now replay through resume_turn with the same key, deleting the
terminal-guard branching: a retried key replays the original response
regardless of run state; a genuinely stale request with a fresh key
still errors via the precondition. Auth and approval kept symmetric.

FakeTurnCoordinator now models resume idempotency by key so the replay
tests are meaningful; terminal-guard assertions re-framed around
same-key replay vs fresh-key stale, plus an explicit idempotent-replay
test on both services.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
henrypark133 added a commit that referenced this pull request Jun 16, 2026
… resume

Review round 3 (design + High):

- WebUiGateResolution: the approval card sends `denied`, the auth cards
  send `cancelled`, and both are now treated identically (resume the run
  and surface the decision to the model). Run termination is a separate
  control (the X -> cancelRun route), not a gate resolution. Collapsed
  the two equivalent variants into one `Declined` (serde aliases
  "denied"/"cancelled" keep the wire stable; no JS change). Facade maps
  Declined -> Deny for auth, approval, and the generic fallback.

- #6 WebUI desync (High): useChat.resolveGate kept processing only for
  approved/credential_provided, dropping processing + activeRun on
  denied/cancelled — but those now resume the run. resolveGate now always
  keeps processing/activeRun; the terminal run_status SSE event clears it
  and the X/cancelRun path remains the only stop. Fixes the latent
  auth-cancelled desync from #4944. assets.rs assertion + useChat tests
  updated.

- #7 helper weight: short_circuit_denied_resume no longer returns the
  DeniedResumeOutcome enum / boxes LoopExecutionState / clones the batch.
  It returns ControlFlow<TurnCompletedStep, (state, remaining_calls)>; the
  completed_turn/empty-remaining tail moved to the two call sites. Heavy
  per-denied-call failure synthesis stays shared (one helper).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
henrypark133 added a commit that referenced this pull request Jun 16, 2026
…ing the run (#4954)

* fix(reborn): surface approval-gate denial to model instead of cancelling the run

Approval-gate denial in Reborn cancelled the run (deny_gate /
replay_denied_gate -> cancel_run), so the model never learned the user
declined and the next trigger re-issued the same approval-gated
capability and re-blocked — the same loop class #4944 removed for auth
gates.

Mirror #4944 for approval gates: denial now RESUMES the parked run
carrying a denial disposition; the capability stage converts ONLY the
approval-gated call into a model-visible non-retryable Authorization
failure ("approval gate denied by user", SameCallRetryConstraint::
Forbidden) and the loop continues. Unrelated parallel calls are
unaffected.

Per the maintainability review of the plan, this unifies rather than
duplicates the #4944 plumbing:
- ironclaw_turns: AuthResumeDisposition -> GateResumeDisposition (one
  gate-agnostic enum); ResumeTurnRequest/TurnRunRecord/TurnRunState/
  AgentLoopDriverResumeRequest field auth_resume_disposition ->
  resume_disposition. Serde key pinned to "auth_resume_disposition"
  (rename attr) so persisted run records still deserialize; legacy-key
  round-trip test added.
- ironclaw_agent_loop: PendingApprovalResume gains a disposition field;
  the auth denied short-circuit in CapabilityStage::process is extracted
  into ONE shared short_circuit_denied_resume helper used by both the
  auth and approval paths (no second copy).
- ironclaw_product_workflow: approval deny_gate / replay_denied_gate
  resume instead of cancel; ResolveApprovalInteractionResponse::Denied
  (CancelRunResponse) -> Resumed(ResumeTurnResponse); idempotent replay
  guarded by terminal run status.
- ironclaw_reborn: PlannedDriver::resume stamps the disposition onto the
  pending resume that is set (auth or approval).

Decisions (plan docs/plans/2026-06-15-reborn-approval-deny-continue.md):
both Denied and Cancelled continue (consistent with #4944, no Cancel
variant). The extension_install/extension_search missing-observation gap
is a separate PR; the user-visible extension-install loop is only fully
closed when both land.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(reborn): address PR #4954 review — stamp denial on matching gate slot only

Review round 1 fixes:

- planned_driver: the denial disposition was stamped onto BOTH
  pending_auth_resume and pending_approval_resume on a comment-only "one
  slot at a time" invariant. GateStage deliberately preserves a pending
  auth resume when a non-auth gate blocks mid-re-dispatch, so both slots
  can be set at once; stamping both corrupted an unrelated auth resume.
  Now stamps only the pending slot whose gate_ref matches the blocking
  gate (state.last_gate). Adds a regression test asserting the auth slot
  stays None when the approval gate is denied, plus an end-to-end
  resume() drive.
- approval replay: match GateResumeDisposition::Denied explicitly rather
  than is_some(), keeping the gate-agnostic carrier tied to denial.
- tests: real TurnRunRecord struct-level serde test (legacy
  auth_resume_disposition key → resume_disposition) + snapshot-level
  legacy denied-marker test; new deny-path resume-error test asserting
  the record is denied and the run is never cancelled on resume failure.
- arch-exempt annotation on short_circuit_denied_resume's
  too_many_arguments allow (plan #4954); stale comments/typos fixed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(reborn): route denied-gate replay through resume_turn idempotency (auth + approval)

Review finding #7: the denied-gate replay paths derived idempotency from
current run state (TurnStatus::is_terminal() guard) rather than replaying
through resume_turn. After the first Deny resumed the run, a transport
retry with the same idempotency key arriving after the runner completed
returned StaleGate/StaleAuth instead of the original ResumeTurnResponse —
the observable result depended on runner timing.

resume_turn is idempotent by key (memory.rs:665 returns the cached
Result from resume_idempotency before the precondition check). Both the
approval (replay_denied_gate) and auth (resume_denied_auth replay arm)
paths now replay through resume_turn with the same key, deleting the
terminal-guard branching: a retried key replays the original response
regardless of run state; a genuinely stale request with a fresh key
still errors via the precondition. Auth and approval kept symmetric.

FakeTurnCoordinator now models resume idempotency by key so the replay
tests are meaningful; terminal-guard assertions re-framed around
same-key replay vs fresh-key stale, plus an explicit idempotent-replay
test on both services.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(reborn): fail closed on ambiguous dual-slot stamp; lock deny-before-resume order

Review round 2 (both Major):

- planned_driver stamp_resume_disposition: the if/else-if silently stamped
  the auth slot if both pending slots matched last_gate. At the denial-
  attribution boundary that could misattribute an approval denial. Now an
  explicit 4-way match fails closed on the ambiguous (true, true) case
  (warn + stamp neither). Test added.
- approval_interaction_contract deny-resume-error test: asserted only
  aggregate call counts, which pass even if call order regressed. Added a
  shared ordered trace across the resolver (deny) and coordinator
  (resume_turn) fakes and assert deny is recorded strictly before resume.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(reborn): strengthen replay/checkpoint coverage; downgrade fail-closed log to debug

Review round 3 (straightforward):
- idempotent deny-replay tests (auth + approval) now assert full
  ResumeTurnResponse payload equality, not just run_id.
- stamp_resume_disposition ambiguous-dual-slot diagnostic: warn! -> debug!
  (REPL/TUI logging rule — internal fail-closed diagnostics use debug!).
- executor: assert the first approval BeforeBlock checkpoint carries
  pending_approval_resume.disposition == None before any denial.
- executor: denied-approval short-circuit no-matching-call test (denied X,
  model emits only Y -> X not surfaced, Y dispatches, pending cleared).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(reborn): unify gate Declined resolution; keep WebUI processing on resume

Review round 3 (design + High):

- WebUiGateResolution: the approval card sends `denied`, the auth cards
  send `cancelled`, and both are now treated identically (resume the run
  and surface the decision to the model). Run termination is a separate
  control (the X -> cancelRun route), not a gate resolution. Collapsed
  the two equivalent variants into one `Declined` (serde aliases
  "denied"/"cancelled" keep the wire stable; no JS change). Facade maps
  Declined -> Deny for auth, approval, and the generic fallback.

- #6 WebUI desync (High): useChat.resolveGate kept processing only for
  approved/credential_provided, dropping processing + activeRun on
  denied/cancelled — but those now resume the run. resolveGate now always
  keeps processing/activeRun; the terminal run_status SSE event clears it
  and the X/cancelRun path remains the only stop. Fixes the latent
  auth-cancelled desync from #4944. assets.rs assertion + useChat tests
  updated.

- #7 helper weight: short_circuit_denied_resume no longer returns the
  DeniedResumeOutcome enum / boxes LoopExecutionState / clones the batch.
  It returns ControlFlow<TurnCompletedStep, (state, remaining_calls)>; the
  completed_turn/empty-remaining tail moved to the two call sites. Heavy
  per-denied-call failure synthesis stays shared (one helper).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(reborn): share denied-approval resume between deny_gate and replay

Review (Medium): deny_gate and replay_denied_gate built an identical
ResumeTurnRequest, mapped the same errors, and returned the same Resumed
shape — the only difference was deny_gate's one-off resolver.deny side
effect. Extracted a shared resume_denied(request, run_id) helper; deny_gate
performs the durable denial then delegates to it, and replay_denied_gate
calls it directly. Removes the duplicated request construction / path
handling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zmanian added a commit that referenced this pull request Jun 27, 2026
- account_login_link manifest: declare ReadFilesystem effect (it reads
  local enrollment/policy/device-key state before egress), matching
  profile_token. (CR #2)
- account-traces fetch: always send a bounded, clamped limit
  ([1, 500], default 200) so None never triggers an unbounded server
  history fetch. (CR #3)
- direct fetch path: bound the response body with a hard byte ceiling
  (256 KiB) via a chunked bounded reader, instead of buffering unbounded. (CR #5)
- account-traces fetch (both sink + direct): stop swallowing every
  non-2xx as an empty list — 404 = legitimate empty (no account yet),
  all other non-2xx surface as Err so the WebUI renders a sanitized
  unavailable state. Add regression tests (500 -> err, 404 -> empty). (CR #6)
- trace-commons-tab.js: render missing final_credit as "—" not "0.00";
  surface useAccountTraces() query errors instead of collapsing them to
  "no traces". (CR #7, #8)
- handlers contract test: capture the forwarded caller in the
  trace_account_traces stub and assert the route threads the
  authenticated user id (test-through-the-caller). (CR #9)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
pranavraja99 added a commit that referenced this pull request Jul 1, 2026
Merge origin/main (resolve tool_disclosure vs NonZeroU32 conflicts in
reborn/runtime.rs and reborn_composition/runtime.rs) and address the
structural review + red CI on the context-management PR.

Blockers:
- #1 Shared-type sentinel: VisibleCapabilitySurface.callable_capability_ids
  is now Option<Vec<CapabilityId>> (None = same as advertised) instead of
  an empty-Vec sentinel, so a legitimately-empty callable set is distinct
  from "no narrowing". Updates producer/consumer + all constructors. Also
  fixes the E0063 missing-field compile error that reddened clippy + every
  Reborn test job.
- #2 Four-way trait-default workaround: extract one
  delegate_and_scope_tool_call_capability_ids helper shared by the
  visible/deny/profile filters (folds the profile All-guard into its
  predicate); the SurfaceTracking passthrough stays a one-line delegate.
- #3 Rollout debt: ToolDisclosureMode::from_raw defaults Off (byte-identical
  request path when unset/empty/unrecognized; explicit `bridged` opts in),
  and the disclosure_build / matcher_selftest / filter_build startup markers
  (+ the orphaned CAPABILITY_FILTER_RESOLUTION_BUILD const) are removed.

High-value findings:
- #4 De-dup the describe-first pre-check into should_describe_first().
- #5 Document the validate-returns-Ok-on-bridge quirk as a tracked upstream
  workaround (gateway discards whole response on validate error).
- #6 Typed BridgeKind replaces name-string discrimination in invoke_bridge
  (dispatch is now exhaustive; no catch-all arm).

CI:
- no-panics: bridge_tool_definition uses an explicit match/panic (mirrors
  bridge_capability_id) instead of .expect on a static name.
- fmt: cargo fmt (tool_result_reference.rs).
- merge-drift: SpyPort provider_call_capability_ids inserts use
  provider_name(..) (ProviderToolName key), not String.

Not addressed here — #7 (split the context-dedup
collapse_repeated_failure_observations feature into its own PR) is a scope
decision left for the author.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
zmanian added a commit that referenced this pull request Jul 8, 2026
… inspection (#5280)

* docs: spec for Trace Commons instance enrollment, profiles, and trace inspection

Cross-repo design (ironclaw + trace-commons-server) for three coexisting
capabilities: instance-wide enrollment, per-user contributor accounts via
login-links, and submitted-trace inspection. Introduces a trace-credential
resolver so the existing user-invite model and the new instance-wide model
both function on one instance, with personal-invite enrollment taking
precedence. Server change is additive (optional per-user subject through
claim issuance + login-link + account resolution).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: Slice 0 plan — trace-commons-server per-user subject

TDD plan for the one server change the whole effort depends on: accept an
optional opaque subject in the upload-claim request and derive a per-user,
tenant-namespaced principal at device-key issuance. Submission attribution,
login-link account resolution, and trace readback all become per-user
automatically from the shared bearer principal; absent subject reproduces
today's behavior. Targets trace-commons-server (contributor-account-slice1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: IronClaw plans for Trace Commons slices 1-4

Slice 1: trace-credential resolver (personal-invite wins, instance fallback
  with per-user subject) + admin-gated instance enrollment.
Slice 2: per-user subject plumbing through upload-claim request + submission.
Slice 3: trace_commons.account_login_link first-party capability (profiles).
Slice 4: per-user submitted-trace inspection across reborn_traces →
  product_workflow facade → webui_v2 handler → frontend.

Each plan is bite-sized TDD against verbatim-extracted current code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(traces): trace-credential resolver (personal invite wins, instance fallback w/ subject)

* refactor(traces): single dir-parameterized policy-path site (remove resolver duplication)

Extract `trace_contribution_dir_for_scope_at`, `trace_policy_path_at`,
`read_trace_policy_for_scope_at`, and `write_trace_policy_for_scope_at`
as the canonical base-dir-parameterized path helpers. All public
functions (`trace_contribution_dir_for_scope`, `read_trace_policy_for_scope`,
`write_trace_policy_for_scope`) now delegate to the `_at` variants with
`ironclaw_base_dir()` — signatures unchanged.

The inline `read_policy` closure in `resolve_trace_credentials_at` that
re-implemented path layout is deleted; it now calls
`read_trace_policy_for_scope_at` directly. The test `write_policy_at`
helper's bespoke path construction is replaced with a call to
`write_trace_policy_for_scope_at`. The now-dead `trace_policy_path`
function is removed. Path layout is encoded in exactly one place.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(traces): instance-level enrollment write path (scope None)

* test(traces): make instance-enrollment test hermetic (tempdir, no global base)

Rework `instance_onboard_writes_instance_level_policy` to operate entirely
under a `tempfile::tempdir()`:
- Compute instance_dir as base.path().join("trace_contributions") (scope=None
  layout, no users/<hash> segment) rather than calling the global LazyLock.
- Call `onboard_at_dir_with_sink` directly against the tempdir so the test
  never touches the real ~/.ironclaw tree.
- Assert policy.json by reading and deserializing it from the tempdir.
- Remove all manual std::fs::remove_* cleanup lines; tempdir drops automatically.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(admin): AdminScope::enroll_instance_trace_commons (admin-gated instance enrollment)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(traces): carry optional per-user subject in upload-claim request

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(traces): thread resolver subject into submission claim context

* test(traces): claim request carries per-user subject end-to-end

* feat(traces): mint_account_login_link_via_sink (POST /v1/account/login-links)

Add `mint_account_login_link_via_sink` to ironclaw_reborn_traces:

- `TraceUploadClaimContext::for_account(subject)` constructor for
  account-management call contexts (no trace/submission ids, no
  consent scopes).
- `AccountLoginLink { account_id, url }` return type.
- `account_login_links_url(policy)` helper that derives the login-links
  URL from the upload-claim issuer URL (strip /v1/trace-upload-claim,
  append /v1/account/login-links).
- `mint_account_login_link_inner(base_dir, ...)` private dir-parameterised
  core: resolves credentials, selects correct scope_dir for DeviceKey
  auth (instance enrollment → instance scope dir; personal → user scope
  dir), mints bearer, POSTs subject, parses response.
- `mint_account_login_link_via_sink(tenant_id, user_id, sink)` public
  entry point wrapping the inner function with the real base dir.

Tests (hermetic, tempdir-isolated):
- `mint_account_login_link_posts_subject_and_returns_url`: verifies the
  posted subject equals `local_pseudonymous_contributor_id(trace_scope_key(...))`
  for instance-enrolled users via an axum mock serving both the
  upload-claim issuer and the login-links endpoint.
- `mint_account_login_link_errors_when_not_enrolled`: verifies error path.
- `ReqwestContributionSink` test helper added to the test module.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(traces): error instead of silent misroute in account_login_links_url

Replace the unwrap_or_else fallback (which silently used the full issuer
URL as a base when the /v1/trace-upload-claim suffix was absent) with an
explicit anyhow error. Add two unit tests: one asserting an Err on a
wrong-suffix URL, one asserting the correct .../v1/account/login-links
URL on a valid issuer.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(host_runtime): add consent-gated trace_commons.account_login_link capability

Mints a Trace Commons browser login URL via host network egress, mirroring
dispatch_profile_token. Includes consent gate, enrollment pre-check,
HostEgressContributionSink routing, and two e2e tests.

Also fixes a sanitizer bug: validate_runtime_request was rejecting
authorization headers on all requests, including RuntimeKind::FirstParty.
FirstParty requests are host-internal and trusted to carry bearer tokens;
the sensitive-header and manual-credentials guards now only apply to
untrusted plugin runtimes (WASM/MCP/Script).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(host_runtime): route trace bearer via credential injection; restore FirstParty sensitive-header guard

Commit 9e25d99 blanket-exempted all RuntimeKind::FirstParty requests from the
egress sensitive-header and manual-credentials guards so the host-minted Trace
Commons bearer could pass. builtin.http is also FirstParty but forwards
model-supplied headers, so this let the model smuggle Authorization/Cookie/
x-api-key headers (or user:pass@ URLs) to allowlisted hosts.

Revert the sanitize.rs exemption (guards now apply to ALL runtimes again) and
deliver the trace bearer through the staged credential-injection path instead:
the HostEgressContributionSink stages the minted token one-shot via
RuntimeSecretMaterialStager and declares a StagedObligation Authorization-header
injection, mirroring the SlackProtocolHttpEgress pattern. The stager is now
exposed to first-party handlers via InvocationServices. Covers the profile_token,
profile_set/community-profile, and account_login_link bearer paths.

Regression tests: FirstParty + raw authorization header -> denied; FirstParty +
user:pass@ URL -> denied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(traces): fetch_account_traces_via_sink (GET /v1/account/traces, per-user)

- Add ContributionHttpMethod::Get variant; update all exhaustive match
  sites in ironclaw_reborn_traces and HostEgressContributionSink in
  ironclaw_host_runtime.
- Extract account_api_base_url() shared helper; account_login_links_url
  and new account_traces_url both delegate to it (DRY).
- Add AccountTraceItem (Debug, Clone, Serialize, Deserialize; serde defaults).
- Add fetch_account_traces_via_sink / fetch_account_traces_inner mirroring
  mint_account_login_link pattern: unenrolled -> Ok(vec![]), non-2xx ->
  Ok(vec![]), transport error -> Err.
- Tests: hermetic axum mock (GET /v1/account/traces), unenrolled empty-list,
  URL shape with/without limit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(reborn): trace_account_traces facade method + wire types

Adds RebornAccountTrace / RebornAccountTracesResponse wire types and
a trace_account_traces default method on RebornServicesApi, mirroring
the trace_credits egress pattern (crate-local hardened reqwest, no
host-egress sink). Also adds fetch_account_traces (direct path) to
ironclaw_reborn_traces::contribution so the facade can fetch server
traces without coupling to RuntimeHttpEgress.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(reborn): GET /api/webchat/v2/traces/account handler + contract test

* feat(reborn-ui): render submitted Trace Commons traces in settings

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(traces): document flush-gate limitation, hermetic account-traces contract test, annotate sink scaffold

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style(traces): cargo fmt across Trace Commons slice changes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(traces): resolver-aware flush gate (instance-enrolled users can contribute)

The autonomous trace-flush gate read only the per-scope (personal-invite)
policy and aborted when it was disabled, so instance-enrolled users (whose
enrollment lives at scope None) could never contribute traces — and the
per-user scope_dir would also fail to load the instance device key.

Introduce a single EffectiveFlushTarget resolver (resolve_effective_flush_target,
mirroring resolve_trace_credentials but keyed on the already-composed scope
string) that returns the policy, device-key dir, and per-user subject in one
policy-read/path pass. The flush gate now proceeds for instance-only enrollment,
loads the device key from the instance (None) dir, and attributes uploads via
the per-user pseudonymous subject. The redundant subject_for_scope helper (which
re-read the same policies with silent .ok() error swallowing) is removed and its
logic folded into the new helper with proper error propagation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): include per-user subject in upload-claim cache key

Under instance enrollment every user shares the same instance device-key
dir (scope None), so the upload-claim cache key — which keyed on scope_dir
but not subject — collided across users. A bearer minted for one subject
could be served from cache to another, mis-attributing traces / leaking
across users. Add a hashed subject component to the DeviceKey cache key and
a regression test proving two subjects sharing a scope_dir get distinct keys
(and a no-subject personal-invite context stays distinct from both).

Found by Codex review of PR #5280.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): address CodeRabbit review on PR #5280

- account_login_link manifest: declare ReadFilesystem effect (it reads
  local enrollment/policy/device-key state before egress), matching
  profile_token. (CR #2)
- account-traces fetch: always send a bounded, clamped limit
  ([1, 500], default 200) so None never triggers an unbounded server
  history fetch. (CR #3)
- direct fetch path: bound the response body with a hard byte ceiling
  (256 KiB) via a chunked bounded reader, instead of buffering unbounded. (CR #5)
- account-traces fetch (both sink + direct): stop swallowing every
  non-2xx as an empty list — 404 = legitimate empty (no account yet),
  all other non-2xx surface as Err so the WebUI renders a sanitized
  unavailable state. Add regression tests (500 -> err, 404 -> empty). (CR #6)
- trace-commons-tab.js: render missing final_credit as "—" not "0.00";
  surface useAccountTraces() query errors instead of collapsing them to
  "no traces". (CR #7, #8)
- handlers contract test: capture the forwarded caller in the
  trace_account_traces stub and assert the route threads the
  authenticated user id (test-through-the-caller). (CR #9)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): cover trace_commons.account_login_link + backfill trace i18n keys

PR #5280 added the builtin.trace_commons.account_login_link capability and a
submitted-traces UI section, but left three guardrail/parity tests un-updated,
turning CI red:

- ironclaw_host_runtime builtin_first_party_package_declares_expected_capabilities:
  register account_login_link in the expected id list and its Ask-permission arm.
- reborn_builtin_first_party_capability_e2e_coverage_is_complete: add genuine
  e2e coverage by exercising account_login_link in the existing trace_commons
  parity test (confirmed=true on a not-enrolled scope returns a deterministic
  NotEnrolled, no network), grant it in the harness allow-set, and add it to the
  model-visible surface test and the covered-capability list.
- ironclaw_webui_v2_static all_locales_share_the_en_key_set: backfill the six new
  traceCommons.* submitted-traces keys into all ten non-en locales.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): address CodeRabbit review — withhold login URL, type errors, wire i18n

- Security (host_runtime): dispatch_account_login_link returned the one-time
  login `url` (a code-bearing account-access credential) on the model-visible
  surface, persisting it into the LLM transcript and any downstream logging.
  Follow the profile_token pattern: persist the URL to a 0600 private file
  (atomic temp+rename) and return an opaque `link_delivery` marker instead.
  E2e test now asserts the URL/code never appears in the result and is
  delivered out-of-band to the private file.

- Typed error (product_workflow): account_traces_for_user flattened backend
  errors into String before the WebUI boundary. Introduce AccountTracesError
  (thiserror) that names the failing operation and preserves the full cause
  chain ({:#}); the boundary keeps returning a sanitized, diagnosable 500. Also
  document that fetch_account_traces(None) is already server-bounded (default
  200, clamp 500, 256 KiB response cap) — the "unbounded fetch" concern was
  resolved by prior hardening.

- i18n (webui_v2_static): the traceStatus and traceReceivedAt keys backfilled
  for locale parity were unused by the consumer. Wire traceStatus as the status
  badge's accessible title/aria-label and render traceReceivedAt as the
  timestamp label, so all six submitted-traces keys are now consumed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): address CodeRabbit re-review — async persist + typed identifiers

- Blocking I/O (host_runtime): persist_account_login_link does mkdir/write/
  fsync/rename with std::fs on the async dispatch path. Wrap the persist call in
  tokio::task::spawn_blocking so it never stalls a Tokio worker (coding guideline:
  all I/O async). Atomic temp+rename behavior is unchanged; a join failure maps to
  the same sanitized "could not write" result.
- Typed identifiers (product_workflow): account_traces_for_user took bare &str
  tenant/user; the caller already holds TenantId/UserId newtypes. Take
  &TenantId/&UserId and only cross to &str at the ironclaw_reborn_traces boundary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): instance-aware enrollment across trace_commons dispatch + UI

Instance-only-enrolled users (admin-provisioned instance policy, no personal
invite) were falsely rejected across the Trace Commons surface: the dispatch
gates and the profile mints read only the personal per-scope policy, and the
submitted-traces UI was gated behind the personal-credits branch. Addresses
CodeRabbit re-review (#3, #4, #5) on PR #5280.

reborn_traces:
- Add instance-aware entry points mint_profile_attribution_token_for_user_via_sink
  and set_community_profile_for_user_via_sink that resolve enrollment via
  resolve_trace_credentials (personal OR instance) and build the claim context
  with the instance scope_dir + per-user pseudonymous subject, mirroring
  mint_account_login_link_inner. Refactor the token mint to share a
  context-based core. New tests assert the per-user subject reaches the issuer.

host_runtime (trace_commons dispatch):
- Route the enrollment gates in dispatch_status, dispatch_profile_token,
  dispatch_profile_set, and dispatch_account_login_link through
  resolve_trace_credentials so instance-only contributors pass. status now
  reports the resolved (instance or personal) policy. profile_token/profile_set
  call the new instance-aware mints.
- #4: preserve the stage_secret_material_once failure cause (log it) instead of
  discarding it with map_err(|_|); wire message stays sanitized.

webui_v2_static (#3):
- Lift the submitted-traces section out of the credits/empty-state branch so
  instance-enrolled users with no personal credits still see their traces and
  any tracesQuery errors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(traces): isolated dispatch-layer e2e for instance-only enrollment

Extract the trace_commons dispatch e2e helpers into a shared
tests/support/trace_commons_dispatch.rs module (base-dir setup, mock issuer,
runtime/dispatch helpers, find_persisted_login_link, test_jwt_eddsa) so a second
test binary can reuse them.

Add trace_commons_instance_dispatch_e2e.rs — a SEPARATE binary (fresh process =
private IRONCLAW_BASE_DIR) that provisions the process-global instance policy
(scope None) without bleeding into the personal-invite suite. It pins the
CodeRabbit #5 fix at the layer it manifests: an instance-only-enrolled user
(no personal invite) passes dispatch_status and dispatch_account_login_link and
mints under the shared instance device key with a per-user pseudonymous subject
(asserted via the subject on the login-links POST).

No production changes; trace_commons_dispatch_e2e.rs behavior is unchanged
(5 tests still pass) — only its helpers moved to the shared module.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): sanitize bearer-staging log + typed IDs on mint entry points

Addresses CodeRabbit overnight review on PR #5280.

- Security (#1): the trace-bearer staging error was debug-logged via `?error`,
  which can leak secret-store/backend detail on the credential path. The
  host_runtime logging guideline forbids backend error detail here — log only
  the safe fact of failure; the wire message stays sanitized. (Supersedes the
  earlier "preserve cause" change specifically on this bearer-material path.)

- Typed identities (#2): the three agent-facing Trace Commons mint entry points
  (mint_account_login_link_via_sink, mint_profile_attribution_token_for_user_via_sink,
  set_community_profile_for_user_via_sink) now take &TenantId/&UserId instead of
  adjacent &str, so callers can't transpose tenant/user and misattribute a
  contributor. Identity stays typed to the public boundary and is stringified
  only when handing off to the dir-parameterised `_inner` cores / resolver
  (the storage edge). Adds ironclaw_host_api as a reborn_traces dependency
  (no cycle: host_api does not depend on reborn_traces). Dispatch callers pass
  the typed scope ids directly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): sanitize persist-path logs, preserve handle-validation cause

Two follow-up CodeRabbit findings on PR #5280:

- Security (Major): dispatch_account_login_link's spawn_blocking persist arms
  logged %error / %join_error at debug. Filesystem errors (mkdir/write/fsync/
  rename) can carry raw host paths, which the host_runtime guideline forbids in
  logs. Drop the interpolation; log only the generic fact, keep the message
  sanitized — same treatment as the bearer-staging path.

- Maintainability (Minor): SecretHandle::new(TRACE_COMMONS_BEARER_HANDLE) used
  map_err(|_| ...), discarding the cause (non-exemptible per the guideline). The
  handle name is a compile-time constant, so its validation error carries no
  secret/path — bind and log it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): typed login-link errors, per-request bearer handle, doc accuracy

Addresses the third CodeRabbit review round on PR #5280.

- Security (Major): the trace-bearer staging used a constant SecretHandle
  (TRACE_COMMONS_BEARER_HANDLE). The injection store is a HashMap keyed by
  (scope, capability, handle) with overwrite-on-insert, so two concurrent
  same-scope Trace Commons egresses could race and stage/consume the wrong
  bearer. Suffix the handle with a per-request uuid so every staged bearer key
  is distinct. Localized to the shared HostEgressContributionSink, so all
  trace_commons flows benefit.

- Correctness (Major): account_login_link_error_value classified failures by
  substring-matching upstream error wording, coupling the public error_code
  contract to phrasing. Introduce a typed AccountLoginLinkError (thiserror) in
  reborn_traces; mint_account_login_link_via_sink returns it, producing the
  specific variant at each failure site. The host maps variants -> error_code
  with no substring checks. NotEnrolled (the only tested code) is preserved;
  the two bearer-derived codes collapse into EnrollmentIncomplete (both meant
  "re-run onboarding"), and persist failures get a distinct LocalStateWrite.

- Docs (Minor): the persist_account_login_link comments promised 0600 across
  platforms though only Unix enforces it. Softened to "private local file
  (0600 on Unix)".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(traces): type profile_token/profile_set error mappers (systemic)

Follow-up to the account_login_link typed-error change: convert the remaining
substring-based error mappers so all four trace_commons dispatch flows derive
the public error_code contract from typed variants instead of matching upstream
error wording. (onboard was already typed via OnboardError.)

- reborn_traces: add ProfileAttributionError (shared by the profile_token and
  profile_set token mints) and CommunityProfileError (profile_set wrapper adding
  InvalidProfile). mint_profile_attribution_token_for_user_via_sink and
  set_community_profile_for_user_via_sink now return these; each failure site
  produces the specific variant (NotEnrolled / PolicyRead / EnrollmentIncomplete
  / Backend / LocalStateWrite, plus InvalidProfile for profile_set).

- host_runtime: profile_token_error_value / profile_set_error_value now match on
  the typed variants — no error.contains(...) anywhere in the file. NotEnrolled
  and InvalidProfile (the tested codes) are preserved; the issuer/device/refused
  substrings collapse into EnrollmentIncomplete, consistent with the
  account_login_link mapping. Also sanitized the profile_token persist-failure
  log (host-path leak class), matching the login-link path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): split enrollment precondition from backend in token mints

CodeRabbit re-review: collapsing every mint_profile_attribution_token_with_context
(and bearer_token) failure into EnrollmentIncomplete mislabels transient
transport/status/serde failures as "re-run onboarding".

Split the local precondition (upload-claim issuer URL configured) from
post-resolution failures: a missing issuer URL maps to EnrollmentIncomplete via
an explicit upload_claim_issuer_missing() check (typed, no substring), while the
claim mint / bearer fetch / PUT failures now map to Backend. Applied
consistently across profile_token, profile_set, and account_login_link so the
error_code contract reflects the real failure class. URL-derivation
preconditions (ingest/login-links URL) stay EnrollmentIncomplete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): check login-link URL precondition before minting bearer

Fail-closed ordering: the local account_login_links_url derivation ran after
bearer_token, so a malformed/absent login-links URL would mint a device-key
bearer and hit the issuer before failing. Move that local precondition ahead of
all secret/egress work so incomplete enrollment fails closed with no side
effects. (profile_token/profile_set already order local preconditions first.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): make upload-claim cache key match issuer payload exactly

The subject cache-key component trimmed/collapsed context.subject, but the
DeviceKey issuer request sends it unchanged — so None, Some(""), and
whitespace variants could share a cache key while minting different payloads,
letting one user's claim be served from cache to another (cross-user trace
mis-attribution). This is #5280's per-user-subject cache-key path.

Hash the exact optional bytes the request sends (DeviceKey → subject,
WorkloadTokenEnv → None) with a None/Some discriminator. Extend the cache-key
test with the Some("")-vs-None and whitespace-variant collision cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): check response-size cap before growing the buffer

Both bounded response readers (upload-claim and account-traces) enforced the
hard byte ceiling only after extend_from_slice, so a single oversized chunk
could push the buffer past the advertised limit before the error returned.
Compute bytes.len() + chunk.len() (checked_add) and validate before appending.

Pre-existing pattern (from #4559), fixed here per review.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): fail loud when trace policy cannot be statted

read_trace_policy_for_scope_at used Path::exists(), which maps stat/permission
errors to false — silently treating an unreadable policy as missing and
default-disabled, flipping enrollment/flush behavior. Use try_exists() and
propagate the stat error with context; only a confirmed non-existent path
returns the not-enrolled default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(traces): capture traces for instance-only enrolled users

Codex P1: capture_turn_trace gated on the per-user scope policy
(read_trace_policy_for_scope(Some(scope)) + policy.enabled), so an instance-only
enrolled user — whose per-user policy is absent/disabled — had every turn
dropped before an envelope was queued, leaving the instance-aware flush gate
nothing to submit. The headline instance-enrollment feature never captured for
exactly the users it targets.

Gate capture on the effective enrollment instead, mirroring the flush gate: add
resolve_effective_capture_policy (personal-invite policy if enabled, else the
admin-provisioned instance policy at scope None, else None) and prepare the
envelope under that governing policy. Add a resolver test covering the
personal / instance-only / neither cases.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Remove accidentally committed frontend node_modules, restore .gitignore

The merge commit f34dfa7 dropped crates/ironclaw_webui_v2_static/frontend/.gitignore
and swept 1065 node_modules files into the index. Untrack them and restore the ignore.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address PR review feedback: egress hardening, effect declaration, instance status sync

- fetch_account_traces_direct now uses a pinned-DNS, private-IP-filtered
  HTTP client (shared pinned_trace_commons_http_client) instead of an
  unrestricted reqwest lookup, closing the DNS-rebinding window between
  claim validation and the bearer-authenticated account-traces GET.
- account_login_link capability manifest declares EffectKind::WriteFilesystem
  for the local delivery-file write.
- Queue-flush status sync (and the public sync entry point) now run off the
  resolved effective flush target (policy, device-key dir, per-user subject)
  instead of re-reading the per-scope policy, so instance-enrolled users get
  final credit status after submission; subject is threaded into the
  status-sync claim context.
- Each login-link mint persists to a unique account_login_link.<uuid>.url
  file so concurrent mints cannot clobber each other; stale link files are
  pruned best-effort after one hour.
- resolve_trace_credentials takes typed &TenantId/&UserId at the public
  boundary; call sites drop their .as_str() conversions.
- Login-link/account-traces requests honor the policy-configured issuer
  timeout; the sink-path traces fetch uses ACCOUNT_TRACES_MAX_RESPONSE_BYTES.
- Removed the AdminScope::enroll_instance_trace_commons wrapper from the v1
  monolith (crate-side entry point is onboard_instance_with_sink; noted in
  the slice1 plan).
- Tests: direct account-traces path covered for 500/404; new regression test
  pins instance-target status sync (subject + instance device-key dir).
- Plan docs: server login-link contract callout, no developer-local paths,
  resolver errors propagate, 404-only zero-state, scope_dir threading.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Pin DNS resolution on the background trace submit/status/revoke lane

The background lane (queue flush submission, status sync, revocation)
previously relied only on enrollment-time endpoint validation
(validate_trace_commons_ingest_url); the per-request client did a fresh
unrestricted DNS lookup. Replace trace_remote_http_client with
pinned_trace_remote_http_client: per-request host resolution through
resolve_trace_upload_claim_issuer_host (private/internal IPs rejected,
literal-loopback local-dev exception) pinned via resolve_to_addrs, so an
endpoint host that passed validation at enrollment cannot later rebind to
an internal address and receive bearer-authenticated requests. Timeout
behavior (env/test task-local override) is unchanged.

Regression test: pinned_trace_remote_client_rejects_private_endpoint_hosts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address CodeRabbit follow-up: sanitize status log, sync plan snippets

- trace commons status dispatcher no longer formats the resolver error into
  the log (it can embed the policy file's host path); logs the safe fact only,
  matching the sibling dispatchers.
- slice4 plan: AccountTraceItem snippet derives Deserialize (matches shipped
  code, which parses the response).
- slice3 plan: login-link parsing snippet fails loud on missing account_id/url
  instead of unwrap_or_default (matches shipped code).
- slice1 plan: the AdminScope wrapper task is marked SUPERSEDED up front so the
  plan no longer gives conflicting guidance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address round-2 review: opt-out precedence, salted subjects, UI branch tests

- Explicit per-user opt-out (scoped policy present with enabled=false, as
  written by 'traces opt-out') now blocks the instance-enrollment fallback in
  resolve_trace_credentials and resolve_effective_flush_target (and thus
  capture) — only a never-configured scope falls through to the instance
  policy. Regression test covers all three resolution surfaces.
- Instance-enrollment subjects are now salted: a per-instance random salt
  (persisted 0600 at the instance trace dir, create_new race-safe) feeds
  sha256(salt:scope), so the server or ledger holders cannot dictionary-match
  guessable tenant/user ids against an unsalted scope hash. Unsalted
  local_pseudonymous_contributor_id remains for local state keying/log refs.
- contribution.rs carries the architecture-rule file-size justification
  referencing decomposition tracking issue #4088; state_scope field docs now
  say which state it does (and does not) locate.
- Submitted-traces UI: extracted the pure tracesSectionMode decision (error
  wins over list; list needs enrolled + non-empty) and covered it plus the
  row formatters in trace-commons-tab.test.mjs.
- Docs: slice4 plan points at crates/ironclaw_webui_v2 (static crate was
  folded in), slice3 signature snippet matches the typed contract, and the
  webui_v2 CLAUDE.md route table gains the three trace routes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Update crossbeam-epoch 0.9.18 -> 0.9.20 for RUSTSEC-2026-0204

Lockfile-only patch bump of a transitive dep (via termimad/crossbeam) to
clear the new advisory failing cargo-deny; verified locally with
cargo deny check advisories.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Route login-link/account-traces claim mint through the caller's sink

The sink-based entry points (mint_account_login_link_via_sink,
fetch_account_traces_via_sink) used the sink for the final POST/GET but
minted the upload-claim bearer via DefaultTraceUploadCredentialProvider,
whose issuer request takes the direct reqwest path — so an agent-invoked
account_login_link performed a network call outside RuntimeHttpEgress.
New trace_upload_bearer_token_via threads Option<sink> into the claim
mint (cache behavior unchanged; the default provider passes None), and
both sink paths pass Some(sink), matching the profile-token/profile-set
flows. Tests now use a RecordingSink to pin the invariant that both the
claim mint and the follow-up request route through the sink.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants