feat(reborn-ironhub) port IronHub install flow to Reborn - #4479
Closed
serrrfirat wants to merge 11 commits into
Closed
serrrfirat wants to merge 11 commits into
serrrfirat wants to merge 11 commits into
Conversation
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
github-actions
Bot
added
size: XL
serrrfirat
changed the title
serrrfirat
removed
risk: high
Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com>
Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com>
…9-merge # Conflicts: # crates/ironclaw_reborn_composition/src/lib.rs
This was referenced Jun 6, 2026
Member
|
Closing this because it still updates legacy top-level I opened #6320 to track the IronHub extension install flow as Reborn-native work with implementation notes. |
Member
|
Reopening this one after re-checking the changed files. The only top-level |
railway-app
Bot
temporarily deployed
to
ironclaw-ci-preview / ironclaw-pr-4479
July 22, 2026 07:05
Destroyed
|
🚅 Deployed to the ironclaw-pr-4479 environment in ironclaw-ci-preview
|
Collaborator
Author
|
Superseded by #6754. This branch was 903 commits behind main and conflicted structurally, not just textually: the extension lifecycle it hooked into moved out of Since no IronHub code had landed on main in the meantime, the work was re-implemented natively against the current extension-host model rather than rebased through the conflicts. #6754 carries the same behavior — signed Ed25519 catalog verification, size/digest-enforced artifact downloads, provenance acknowledgement gating, scoped skill installs, v3 WASM extension packages, and the Closing in favor of #6754. Original authorship credit carried forward. |
Collaborator
Author
|
Correction to my earlier note: I said nothing depended on this branch, and that was wrong — #5409 was stacked directly on it. My search was truncated (a 100-PR limit against 240 open PRs), so I stated a partial result as fact. This branch ( @neo-sky's authorship is being carried into the replacement commits via |
This was referenced Jul 28, 2026
gagdiez
pushed a commit
to gagdiez/ironclaw
that referenced
this pull request
Aug 3, 2026
…nifest source (nearai#6780) * feat(reborn): port IronHub install flow * fix(reborn-ironhub): preserve skill install source and scope on rollback - Preserve URL-sourced skill provenance during forced-replacement rollback. - Restore exact extension installation ownership during compensation. - Reject persisted HostBundled provenance before manifest parsing. - Bound IronHub coordination maps and evict idle keyed locks. - Retain serde error causes in debug logs without changing public error kinds. - Add execute-seam coverage for replacement rollback, integrity checks, and replay rejection. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(reborn-ironhub): skip host-bundled stamps per entry instead of aborting the catalog Fixes an availability regression introduced by e97c124: persisted HostBundled stamps remain rejected, but now skip only the affected extension so valid catalog entries still load. Also removes the IronHub test fixture lint exemption and isolates lock-eviction assertions with fixture-unique identities. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * feat(reborn-ironhub): deep-link register/install gateway + private manifest source Re-port of nearai#5409 onto the current extension-host layout. The original branch was stacked on nearai#4479 and predates the extension-host extraction (nearai#6116, nearai#6616), the removal of ironclaw_product_workflow (nearai#6583), and the webui_v2 crate rename, so the integration is reimplemented against today's APIs rather than rebased. - Public POST /api/ironhub/register handshake (HMAC-SHA256, constant-time via verify_slice) mounted outside bearer auth, plus the bearer-authed ironhub_deliver_install route on the v2 webui surface. - The gateway is disabled by default: it mounts only when IRONHUB_AGENT_SHARED_KEY is set to a non-empty value of at least 16 bytes, and the install-delivery route stays fail-closed as unavailable unless that service is attached. - Install delivery is scoped to the authenticated caller's UserId, so egress runs under the caller rather than a runtime owner. - Install nonces are single-use and consumed durably (keyed by SHA-256 digest, bounded length, control characters rejected); timestamps are drift-checked within a 300-second window. - Private manifest source: Ed25519-signed org-scoped manifests install from the configured catalog host behind a Private provenance tier that is rejected unless the install genuinely came from a private manifest. - Manifest replay/downgrade protection keys on (host, signed repo) so it is independent of the rotating per-install access token in the URL. Link logic lives in ironclaw_extension_host::ironhub (agent_link, link_service); the product-layer service moved to ironclaw_product::reborn_services::ironhub_link now that product_workflow is gone; serve wiring is composition-owned. Supersedes nearai#5409. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(turns): trust verified catalog descriptions instead of denying the whole prompt Installing Attio from the signed IronHub catalog exposed an official description containing API key and Bearer authentication vocabulary. Prompt validation treated that trusted text as an unsafe summary and denied every subsequent turn. Carry verified catalog provenance into capability descriptors and route it through a trusted prompt-text surface, following the certified-skill fix from nearai#5169/nearai#5258. Structural checks still apply, while invalid untrusted descriptors are omitted individually with host diagnostics naming the capability and matched pattern. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * test(golden): re-bless capability surface hashes after the description-trust field Adding CapabilityDescriptionTrust to CapabilityDescriptorView changes the capability surface fingerprint, so the golden payload snapshots carry a new surface sha256. Verified the change is hash-only: all 7 changed content lines are byte-identical once the surface hash is normalized, with zero other content deltas. The trust field does not appear in model-visible prompt content — the capability list and every description are unchanged. Only insta's stale assertion_line metadata was additionally dropped. * fix(ironhub): return complete, self-describing search results instead of a silent truncation (nearai#6808) IronHub search returned a result-reference prefix, leading the agent to report attio missing even though it was present in the signed catalog. Return compact catalog projections with explicit completeness metadata and a bounded, unmistakably incomplete fallback. Closes nearai#6788 Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * test(integration): cover the install-then-turn prompt-denial incident at the turn seam Drive registry-verified and local extension descriptions through the real product workflow, scheduler, agent loop, and model request boundary. This closes the Attio incident gap by proving verified Bearer-header wording survives intact while one unsafe local prompt entry degrades without collapsing the remaining capability surface. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(ironhub): measure catalog_total inside the truncation budget Follow-up to b218c05. The bounded-fallback path sized `Self::incomplete(...)` against MAX_SEARCH_RESPONSE_BYTES while that shape still had `catalog_total: None` — omitted from JSON by skip_serializing_if — and then assigned Some(..) to the value actually returned. The emitted payload was therefore ~20 bytes larger than the budget that admitted it, so a truncated result could exceed the cap it exists to enforce. `incomplete` now takes `catalog_total`, so the shape measured is exactly the shape emitted. Extended the existing oversized-catalog test rather than adding a fourth search test (it already owns the truncated path): it now pins catalog_total on both the struct and the serialized payload, alongside its existing size bound. * fix(host_api): redact credentials in model previews instead of dropping them A deployed agent could not return the IronHub catalog. The payload was fine — 13,755 bytes, complete, well under every size bound. It never reached the model. `result_preview_parts` built the preview, then discarded it because `ModelResultPreview::new` refuses any content containing a credential marker ("access token", "api key", "bearer ", "password", "secret", ...). One catalog entry's summary says "no API key" — describing the ABSENCE of one — and that phrase refused the entire catalog. The caller's `else` arm drops the preview AND the continuation metadata that travels with it, so the model received a bare result reference with no preview, no total_bytes and no next_offset: unreadable and unpageable. The logs show it then trying result_read (which returned a reference to a reference), curl, wget, python3, an HTTP fetch that 404'd, and five more identical searches. Masking, not refusal, for model-visible CONTENT: - `credential_redaction::redact_credential_text` masks credential markers (at a word boundary, so "Secretary" survives) and credential-shaped tokens (sk-, ghp_, AKIA...) with [redacted], preserving everything else. - `ModelResultPreview::redacted` falls back to the masked text when the strict contract refuses; `ModelResultPreview::new` is unchanged for callers that can legitimately reject an operation. - The preview path uses it, so content and its continuation metadata survive. The security property is unchanged: credential material still never reaches the model. Only the disposal changed — mask the span rather than discard the payload. The existing resolution test now asserts exactly that: the secret is absent from the preview AND the surrounding content survives. REVISIT: the marker list is credential *vocabulary*, so prose like "no API key required" is masked despite containing no credential. `contains_unredacted_credential_value` already models the sharper "label followed by a value" rule and its own doc notes that vocabulary alone is valid diagnostic context. Narrowing this is a separate decision about a shared credential boundary and is deliberately not made here — masking is strictly better than today's wholesale refusal. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(host_api): share the marker boundary rule so redaction masks every marker 4b85acc added credential masking but reimplemented the marker boundary check instead of reusing the detector's, and got it wrong: it required an alphanumeric boundary on BOTH sides for every marker. Markers that already carry a delimiter ("bearer ", "authorization:") therefore never matched — "presented as a Bearer header" left `bearer ` untouched, `contains_credential_marker` still returned true, validation still failed, and the preview was still dropped. Net effect: the fix did not fix the production case. An `extension_search` result for attio (1088 bytes) still reached the model as a bare reference with no preview, and a `result_read` on it returned another reference whose own read failed with "result reference is unavailable in this thread". `marker_match_at` is now extracted from `contains_marker_at_word_boundary` and used by BOTH the detector and the redactor, so the two cannot drift again: a marker carrying its own delimiter skips the boundary check on that side. The seam regression now uses the real production string ("Authenticated with a workspace API key presented as a Bearer header") rather than a single-marker stand-in. That string is what the earlier test missed: it contained only "api key", which masked correctly, so the bug hid behind a passing test. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(ironhub): carry published credential recipes into the generated manifest An IronHub tool that needs a credential installed "successfully" and could never authenticate. Attio is the reported case: it was installed, activated, and callable as attio.invoke, but nothing in the extension model knew an API key was required, so no auth challenge was raised and the in-chat credential card never rendered. The agent filled the gap by inventing a CLI command. Cause: `generic_tool_manifest` synthesised the v3 manifest from the catalog entry's name/version/description alone and hardcoded `effects = ["network"]`. The tool's own credential recipe travels in the capabilities artifact — already downloaded, digest-verified, and written into the package as `legacy/capabilities.json` — and was never read. Attio publishes: "http": { "credentials": { "attio_api_key": { "location": { "type": "bearer" }, "host_patterns": ["api.attio.com"] } } } `mapped_credentials` now reads that and emits a `[[tools.credentials]]` block plus the `use_secret` effect, matching how bundled first-party extensions (github, slack) declare credentials. Location mapping, from a survey of all nine credentialed catalog tools: - bearer (7 tools) -> header "authorization" with prefix "Bearer " - header (monday) -> that header name, NO prefix; monday.com sends the raw token as the Authorization value, so an invented prefix would break every request - basic (wazuh) -> UNSUPPORTED: v3 injection models header/query/path/pointer, not HTTP Basic. Fails the install with a message naming the tool and the location type, rather than repeating the silent-success failure this change fixes. Policy fields stay host-authored. `trust`, `origin_gate_matrix`, `default_permission` and `visibility` remain hardcoded: a third-party package declares which credential it needs, never what it is allowed to do. That is why generation is kept and the tool's own manifest.toml is still not trusted. Tests cover each location shape, the credential-free path (unchanged output), and that the generated manifest parses as real v3 with the credential block and use_secret effect present — not just that the string contains them. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(ironhub): emit the [auth.<vendor>] recipe credentials require e53ecc0 propagated credential recipes into the generated manifest but omitted the vendor auth recipe, so every credentialed IronHub install failed: credential vendor `attio` has no [auth.attio] recipe; v3 manifests must declare one for every referenced vendor That surfaced to the user as `ironhub_install` -> operation_failed with no diagnostic detail, which is worse than the bug it replaced: before, install "succeeded" and could not authenticate; after, install failed opaquely. The generated manifest now carries `[auth.<vendor>]` with `method = "api_key"` — the variant that maps to RuntimeCredentialAccountSetup::ManualToken, which is the flow that renders the masked in-chat credential card. display_name and the per-field labels come from the tool's own published `auth` and `setup.required_secrets` blocks, so the user sees the vendor's own wording. `validation` is omitted deliberately: it is optional, and a probe the host invents could fail against a service it has never contacted. Why this shipped: the previous test asserted the manifest was valid TOML and contained the right keys. It was, and it did — but `registry_extension_package` runs the production v3 parser and package validation, which the string test never reached. The new test drives `ironhub_tool_package` (the real caller seam) with a real WASI component fixture, so manifest validation is actually exercised. Per .claude/rules/testing.md: test through the caller, not the helper. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(ironhub): derive the auth method from the tool, not a hardcoded api_key dfff895 emitted `method = "api_key"` for every credentialed tool. Four catalog tools (gitlab, clickup, microsoft-365, xero) are genuinely OAuth2 and publish an `auth.oauth` block; forcing api_key on them means the user pastes an access token by hand that then expires with no refresh — gitlab's own description promises "host-managed token refresh". The method now follows what the tool published: - `auth.oauth` present -> `method = "oauth2_code"`, carrying authorization/token endpoints, the scope ceiling, PKCE (S256 default, explicit "none" only on opt-out), and client_id_env/client_secret_env as deployment secret HANDLES so no secret material enters the manifest. - absent -> `method = "api_key"`, which maps to RuntimeCredentialAccountSetup::ManualToken and renders the masked in-chat card. `token_response` is required by the recipe but absent from the capabilities artifact, so it is synthesised as /access_token, /refresh_token, /expires_in. Unlike the `validation` probe (a URL only the vendor can know, still omitted), this shape is defined by RFC 6749 section 5.1 and implemented by every OAuth2 vendor in the catalog; the pointers declare where to look, not that the fields must be present. `identity`, `refresh` and `revoke` stay absent — all optional. Both arms are now proven through `ironhub_tool_package`, the production package validator, with a real WASI component fixture. The api_key arm alone would have kept the four OAuth tools broken in a way string assertions could not catch: the oauth2_code recipe has stricter requirements than api_key, and it was exactly `missing field token_response` that the caller-seam test surfaced. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * docs(ironhub): name the supported credential locations in the failure The fail-closed arm now lists what the host can inject ('bearer', 'header') and records why the rest are absent: QueryParam/PathPlaceholder/BodyJsonPointer are modelled by RuntimeCredentialTarget but published by no catalog tool, so their tool-side spelling is unverified and mapping them now would be speculative; 'basic' is genuinely inexpressible in v3 injection, which has no HTTP Basic variant. * refactor(extension-host): key persisted manifest sources by ExtensionId manifest_sources was BTreeMap<String, ManifestSource>, introduced by aabbc70. The construction site in factory.rs already held a validated ExtensionId and threw the type away (`record.manifest().id.as_str().to_string()`), so an unnormalized key would silently miss every lookup rather than fail — the exact class .claude/rules/types.md exists to prevent. Keyed by ExtensionId end to end: the boundary keeps the typed identity, the catalog lookup drops `.as_str()`, and the two test fixtures construct real ids. Contained to 2 files; no behavior change, and the per-entry host-bundled provenance regression still passes. * feat(ironhub): install from the published manifest and carry its setup steps An IronHub tool did not ship the manifest IronClaw installs from. IronClaw built one at install time by string-concatenating TOML from `capabilities.json`, a schema this repository does not own. That reconstruction lost fields silently three times — the credential blocks, then the `[auth.<vendor>]` recipe those credentials referenced, then the OAuth versus API-key distinction — and each loss reached a user as an install that could not authenticate, because the only machine the translation ran on was theirs. nearai/ironhub#254 publishes the manifest as a signed catalog artifact. Install from it: - `IronHubToolEntry` gains an optional `manifest` artifact, downloaded and digest-verified like the wasm and capabilities. Optional so a catalog predating published manifests still lists every tool; installing one of those is what fails, naming the cause. - `ironhub_tool_package` places the wasm and the two host-owned generic schemas at the paths the manifest declares, so publisher and host never have to agree a filename convention across two repositories. `crate_name` existed only to build those paths and is gone. - The 349-line translator is deleted, along with the catalog field it needed. - A manifest whose id contradicts the catalog entry is refused rather than resolved in the manifest's favour: installing `github` when the user asked for `attio` would shadow an unrelated extension. The second half is what a user actually feels. `capabilities.json` already recorded how to obtain each credential — Attio's says to open Workspace Settings > Developers and create an access token, with the URL beside it — but the auth recipes had nowhere to put it, so an installed tool could say it needed a secret and nothing about where the secret comes from. Users had no way to activate what they had just installed, and models asked for help invented the steps. `ApiKeyRecipe` and `OAuth2CodeRecipe` gain optional `instructions` and `setup_url`, and the shared import seam turns them into the onboarding copy the extensions UI already knows how to render. Deriving it at that seam means uploaded packages get it too, not just registry ones. `setup_url` is an `HttpsEndpoint` because it becomes a link the user is invited to follow; the text is rendered through JSX interpolation and never reaches model-visible prompt text. Verified against the live IronHub catalog: all 17 publishable tools install through the production package validator, 8 of them now carrying their vendor's real setup steps (the other 9 declare no `auth.instructions` upstream). Attio surfaces "open Workspace Settings > Developers ..." and its settings URL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ironhub): stop naming a concrete extension in generic package code The extension-specificity gate flags a first-party extension name appearing in generic code. The identity-pin comment used two real extensions to illustrate the shadowing it prevents, and the regression test built its contradicting manifest under a real extension id. Neither needed to be a real name: the rule is about an id the user did not ask for, whichever id that is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(architecture): extract the IronHub client into ironclaw_ironhub (nearai#6870) crates/AGENTS.md gives ironclaw_extension_host an explicit non-goal: "Host authority (signing secrets, bot tokens, network egress)". The ironhub module inside it did network egress (catalog and artifact downloads), held the pinned Ed25519 catalog trust anchor, and owned the HMAC deep-link shared key — plus skill installs, which are not extension lifecycle at all. It landed there because the seam it drives (registry_extension_package) lives there, not because the crate owns the concern; the boundary tests never saw it because no dependency edge changed. Move the module wholesale to a new crate directly above extension_host. The placement rule it restores: generic registry seams (registry_extension_package, parse_imported_manifest, ManifestSource::RegistryInstalled) stay in extension_host, and the one concrete catalog client is vendor-scoped by charter, the same way each concrete extension crate is scoped to its product. The module already touched its host crate through exactly four public symbols, so extension_host's API is unchanged apart from deleting `pub mod ironhub` and dropping ed25519-dalek, which nothing else in the crate used. Wiring changes, all shape-preserving: - The binary's exact dependency allowlist stays closed: composition re-exports the command vocabulary as `ironclaw_reborn_composition::ironhub` with the house consumer-and-test doc comment, and the CLI imports the facade instead of reaching extension_host. - ironclaw_ironhub gets its dependency BoundaryRule in the same PR (a new crate is unruled by default): no execution runtimes, no secret storage, no serve/assembly layers, no concrete extension crates. - extension_host's `test-support` feature now forwards the filesystem/host_api/processes test-support seams and exposes lifecycle_test_support behind it, so the moved integration-style tests drive the same real lifecycle services from outside the crate. - Fixture include paths shorten by one directory level; the moved files are otherwise verbatim (git tracks them as renames). Verified: full architecture suite (18/18 result sets), ironclaw_ironhub 34/34, workspace clippy clean with -D warnings. Composition lib tests: 638 passed, with two known parallel-execution flakes that pass serially and one pre-existing trace-capture failure unrelated to this move (fails identically with the module in either location). Co-authored-by: serrrfirat <firatsertgoz@alumni.sabanciuniv.edu> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * fix(ironhub): accept hub-prefixed artifact digests * fix(ironhub): address review feedback * fix(ironhub): propagate manifest URL to CLI commands * fix(ironhub): close rereview findings * fix(composition): preserve configured IronHub catalog * fix(ironhub): restore failed skill replacements safely * fix(ironhub): restore complete skill bundles * fix(skills): preserve restore failure context * test(ironhub): cover mediated service entrypoints * fix(ironhub): install verified tool schemas * test(coverage): recapture extension host after IronHub extraction * test(coverage): track IronHub changed-code gaps * test(ironhub): execute reviewed coverage paths * test(ironhub): close changed coverage gap * test(composition): pin IronHub register default-off * Fix changed coverage exemption lines * fix(ironhub): address replay persistence review * test(ironhub): cover install error paths * ci: rebase changed coverage exemptions * fix(ironhub): share durable link state across surfaces * ci(coverage): rebase IronHub runtime exemptions * fix: refresh capabilities after IronHub install * test: remove stale extension host lifecycle fixture * fix: use canonical extension schemas in the loop * fix(ironhub): address coderabbit review — harden shared key validation (nearai#6780) * fix(ci): update IronHub fixture paths after package move * fix(ci): preserve selected integration coverage mode * fix(ci): keep MSRV override for selected integration lanes * fix(extensions): address think-in-universe review — provider schemas and setup copy (nearai#6780) --------- Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> Co-authored-by: serrrfirat <firatsertgoz@alumni.sabanciuniv.edu> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
l3ocifer
pushed a commit
to l3ocifer/frick-ironclaw
that referenced
this pull request
Sep 3, 2026
…nifest source (nearai#6780) * feat(reborn): port IronHub install flow * fix(reborn-ironhub): preserve skill install source and scope on rollback - Preserve URL-sourced skill provenance during forced-replacement rollback. - Restore exact extension installation ownership during compensation. - Reject persisted HostBundled provenance before manifest parsing. - Bound IronHub coordination maps and evict idle keyed locks. - Retain serde error causes in debug logs without changing public error kinds. - Add execute-seam coverage for replacement rollback, integrity checks, and replay rejection. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(reborn-ironhub): skip host-bundled stamps per entry instead of aborting the catalog Fixes an availability regression introduced by e97c124: persisted HostBundled stamps remain rejected, but now skip only the affected extension so valid catalog entries still load. Also removes the IronHub test fixture lint exemption and isolates lock-eviction assertions with fixture-unique identities. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * feat(reborn-ironhub): deep-link register/install gateway + private manifest source Re-port of nearai#5409 onto the current extension-host layout. The original branch was stacked on nearai#4479 and predates the extension-host extraction (nearai#6116, nearai#6616), the removal of ironclaw_product_workflow (nearai#6583), and the webui_v2 crate rename, so the integration is reimplemented against today's APIs rather than rebased. - Public POST /api/ironhub/register handshake (HMAC-SHA256, constant-time via verify_slice) mounted outside bearer auth, plus the bearer-authed ironhub_deliver_install route on the v2 webui surface. - The gateway is disabled by default: it mounts only when IRONHUB_AGENT_SHARED_KEY is set to a non-empty value of at least 16 bytes, and the install-delivery route stays fail-closed as unavailable unless that service is attached. - Install delivery is scoped to the authenticated caller's UserId, so egress runs under the caller rather than a runtime owner. - Install nonces are single-use and consumed durably (keyed by SHA-256 digest, bounded length, control characters rejected); timestamps are drift-checked within a 300-second window. - Private manifest source: Ed25519-signed org-scoped manifests install from the configured catalog host behind a Private provenance tier that is rejected unless the install genuinely came from a private manifest. - Manifest replay/downgrade protection keys on (host, signed repo) so it is independent of the rotating per-install access token in the URL. Link logic lives in ironclaw_extension_host::ironhub (agent_link, link_service); the product-layer service moved to ironclaw_product::reborn_services::ironhub_link now that product_workflow is gone; serve wiring is composition-owned. Supersedes nearai#5409. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(turns): trust verified catalog descriptions instead of denying the whole prompt Installing Attio from the signed IronHub catalog exposed an official description containing API key and Bearer authentication vocabulary. Prompt validation treated that trusted text as an unsafe summary and denied every subsequent turn. Carry verified catalog provenance into capability descriptors and route it through a trusted prompt-text surface, following the certified-skill fix from nearai#5169/nearai#5258. Structural checks still apply, while invalid untrusted descriptors are omitted individually with host diagnostics naming the capability and matched pattern. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * test(golden): re-bless capability surface hashes after the description-trust field Adding CapabilityDescriptionTrust to CapabilityDescriptorView changes the capability surface fingerprint, so the golden payload snapshots carry a new surface sha256. Verified the change is hash-only: all 7 changed content lines are byte-identical once the surface hash is normalized, with zero other content deltas. The trust field does not appear in model-visible prompt content — the capability list and every description are unchanged. Only insta's stale assertion_line metadata was additionally dropped. * fix(ironhub): return complete, self-describing search results instead of a silent truncation (nearai#6808) IronHub search returned a result-reference prefix, leading the agent to report attio missing even though it was present in the signed catalog. Return compact catalog projections with explicit completeness metadata and a bounded, unmistakably incomplete fallback. Closes nearai#6788 Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * test(integration): cover the install-then-turn prompt-denial incident at the turn seam Drive registry-verified and local extension descriptions through the real product workflow, scheduler, agent loop, and model request boundary. This closes the Attio incident gap by proving verified Bearer-header wording survives intact while one unsafe local prompt entry degrades without collapsing the remaining capability surface. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(ironhub): measure catalog_total inside the truncation budget Follow-up to b218c05. The bounded-fallback path sized `Self::incomplete(...)` against MAX_SEARCH_RESPONSE_BYTES while that shape still had `catalog_total: None` — omitted from JSON by skip_serializing_if — and then assigned Some(..) to the value actually returned. The emitted payload was therefore ~20 bytes larger than the budget that admitted it, so a truncated result could exceed the cap it exists to enforce. `incomplete` now takes `catalog_total`, so the shape measured is exactly the shape emitted. Extended the existing oversized-catalog test rather than adding a fourth search test (it already owns the truncated path): it now pins catalog_total on both the struct and the serialized payload, alongside its existing size bound. * fix(host_api): redact credentials in model previews instead of dropping them A deployed agent could not return the IronHub catalog. The payload was fine — 13,755 bytes, complete, well under every size bound. It never reached the model. `result_preview_parts` built the preview, then discarded it because `ModelResultPreview::new` refuses any content containing a credential marker ("access token", "api key", "bearer ", "password", "secret", ...). One catalog entry's summary says "no API key" — describing the ABSENCE of one — and that phrase refused the entire catalog. The caller's `else` arm drops the preview AND the continuation metadata that travels with it, so the model received a bare result reference with no preview, no total_bytes and no next_offset: unreadable and unpageable. The logs show it then trying result_read (which returned a reference to a reference), curl, wget, python3, an HTTP fetch that 404'd, and five more identical searches. Masking, not refusal, for model-visible CONTENT: - `credential_redaction::redact_credential_text` masks credential markers (at a word boundary, so "Secretary" survives) and credential-shaped tokens (sk-, ghp_, AKIA...) with [redacted], preserving everything else. - `ModelResultPreview::redacted` falls back to the masked text when the strict contract refuses; `ModelResultPreview::new` is unchanged for callers that can legitimately reject an operation. - The preview path uses it, so content and its continuation metadata survive. The security property is unchanged: credential material still never reaches the model. Only the disposal changed — mask the span rather than discard the payload. The existing resolution test now asserts exactly that: the secret is absent from the preview AND the surrounding content survives. REVISIT: the marker list is credential *vocabulary*, so prose like "no API key required" is masked despite containing no credential. `contains_unredacted_credential_value` already models the sharper "label followed by a value" rule and its own doc notes that vocabulary alone is valid diagnostic context. Narrowing this is a separate decision about a shared credential boundary and is deliberately not made here — masking is strictly better than today's wholesale refusal. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(host_api): share the marker boundary rule so redaction masks every marker 4b85acc added credential masking but reimplemented the marker boundary check instead of reusing the detector's, and got it wrong: it required an alphanumeric boundary on BOTH sides for every marker. Markers that already carry a delimiter ("bearer ", "authorization:") therefore never matched — "presented as a Bearer header" left `bearer ` untouched, `contains_credential_marker` still returned true, validation still failed, and the preview was still dropped. Net effect: the fix did not fix the production case. An `extension_search` result for attio (1088 bytes) still reached the model as a bare reference with no preview, and a `result_read` on it returned another reference whose own read failed with "result reference is unavailable in this thread". `marker_match_at` is now extracted from `contains_marker_at_word_boundary` and used by BOTH the detector and the redactor, so the two cannot drift again: a marker carrying its own delimiter skips the boundary check on that side. The seam regression now uses the real production string ("Authenticated with a workspace API key presented as a Bearer header") rather than a single-marker stand-in. That string is what the earlier test missed: it contained only "api key", which masked correctly, so the bug hid behind a passing test. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(ironhub): carry published credential recipes into the generated manifest An IronHub tool that needs a credential installed "successfully" and could never authenticate. Attio is the reported case: it was installed, activated, and callable as attio.invoke, but nothing in the extension model knew an API key was required, so no auth challenge was raised and the in-chat credential card never rendered. The agent filled the gap by inventing a CLI command. Cause: `generic_tool_manifest` synthesised the v3 manifest from the catalog entry's name/version/description alone and hardcoded `effects = ["network"]`. The tool's own credential recipe travels in the capabilities artifact — already downloaded, digest-verified, and written into the package as `legacy/capabilities.json` — and was never read. Attio publishes: "http": { "credentials": { "attio_api_key": { "location": { "type": "bearer" }, "host_patterns": ["api.attio.com"] } } } `mapped_credentials` now reads that and emits a `[[tools.credentials]]` block plus the `use_secret` effect, matching how bundled first-party extensions (github, slack) declare credentials. Location mapping, from a survey of all nine credentialed catalog tools: - bearer (7 tools) -> header "authorization" with prefix "Bearer " - header (monday) -> that header name, NO prefix; monday.com sends the raw token as the Authorization value, so an invented prefix would break every request - basic (wazuh) -> UNSUPPORTED: v3 injection models header/query/path/pointer, not HTTP Basic. Fails the install with a message naming the tool and the location type, rather than repeating the silent-success failure this change fixes. Policy fields stay host-authored. `trust`, `origin_gate_matrix`, `default_permission` and `visibility` remain hardcoded: a third-party package declares which credential it needs, never what it is allowed to do. That is why generation is kept and the tool's own manifest.toml is still not trusted. Tests cover each location shape, the credential-free path (unchanged output), and that the generated manifest parses as real v3 with the credential block and use_secret effect present — not just that the string contains them. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(ironhub): emit the [auth.<vendor>] recipe credentials require e53ecc0 propagated credential recipes into the generated manifest but omitted the vendor auth recipe, so every credentialed IronHub install failed: credential vendor `attio` has no [auth.attio] recipe; v3 manifests must declare one for every referenced vendor That surfaced to the user as `ironhub_install` -> operation_failed with no diagnostic detail, which is worse than the bug it replaced: before, install "succeeded" and could not authenticate; after, install failed opaquely. The generated manifest now carries `[auth.<vendor>]` with `method = "api_key"` — the variant that maps to RuntimeCredentialAccountSetup::ManualToken, which is the flow that renders the masked in-chat credential card. display_name and the per-field labels come from the tool's own published `auth` and `setup.required_secrets` blocks, so the user sees the vendor's own wording. `validation` is omitted deliberately: it is optional, and a probe the host invents could fail against a service it has never contacted. Why this shipped: the previous test asserted the manifest was valid TOML and contained the right keys. It was, and it did — but `registry_extension_package` runs the production v3 parser and package validation, which the string test never reached. The new test drives `ironhub_tool_package` (the real caller seam) with a real WASI component fixture, so manifest validation is actually exercised. Per .claude/rules/testing.md: test through the caller, not the helper. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * fix(ironhub): derive the auth method from the tool, not a hardcoded api_key dfff895 emitted `method = "api_key"` for every credentialed tool. Four catalog tools (gitlab, clickup, microsoft-365, xero) are genuinely OAuth2 and publish an `auth.oauth` block; forcing api_key on them means the user pastes an access token by hand that then expires with no refresh — gitlab's own description promises "host-managed token refresh". The method now follows what the tool published: - `auth.oauth` present -> `method = "oauth2_code"`, carrying authorization/token endpoints, the scope ceiling, PKCE (S256 default, explicit "none" only on opt-out), and client_id_env/client_secret_env as deployment secret HANDLES so no secret material enters the manifest. - absent -> `method = "api_key"`, which maps to RuntimeCredentialAccountSetup::ManualToken and renders the masked in-chat card. `token_response` is required by the recipe but absent from the capabilities artifact, so it is synthesised as /access_token, /refresh_token, /expires_in. Unlike the `validation` probe (a URL only the vendor can know, still omitted), this shape is defined by RFC 6749 section 5.1 and implemented by every OAuth2 vendor in the catalog; the pointers declare where to look, not that the fields must be present. `identity`, `refresh` and `revoke` stay absent — all optional. Both arms are now proven through `ironhub_tool_package`, the production package validator, with a real WASI component fixture. The api_key arm alone would have kept the four OAuth tools broken in a way string assertions could not catch: the oauth2_code recipe has stricter requirements than api_key, and it was exactly `missing field token_response` that the caller-seam test surfaced. Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> * docs(ironhub): name the supported credential locations in the failure The fail-closed arm now lists what the host can inject ('bearer', 'header') and records why the rest are absent: QueryParam/PathPlaceholder/BodyJsonPointer are modelled by RuntimeCredentialTarget but published by no catalog tool, so their tool-side spelling is unverified and mapping them now would be speculative; 'basic' is genuinely inexpressible in v3 injection, which has no HTTP Basic variant. * refactor(extension-host): key persisted manifest sources by ExtensionId manifest_sources was BTreeMap<String, ManifestSource>, introduced by aabbc70. The construction site in factory.rs already held a validated ExtensionId and threw the type away (`record.manifest().id.as_str().to_string()`), so an unnormalized key would silently miss every lookup rather than fail — the exact class .claude/rules/types.md exists to prevent. Keyed by ExtensionId end to end: the boundary keeps the typed identity, the catalog lookup drops `.as_str()`, and the two test fixtures construct real ids. Contained to 2 files; no behavior change, and the per-entry host-bundled provenance regression still passes. * feat(ironhub): install from the published manifest and carry its setup steps An IronHub tool did not ship the manifest IronClaw installs from. IronClaw built one at install time by string-concatenating TOML from `capabilities.json`, a schema this repository does not own. That reconstruction lost fields silently three times — the credential blocks, then the `[auth.<vendor>]` recipe those credentials referenced, then the OAuth versus API-key distinction — and each loss reached a user as an install that could not authenticate, because the only machine the translation ran on was theirs. nearai/ironhub#254 publishes the manifest as a signed catalog artifact. Install from it: - `IronHubToolEntry` gains an optional `manifest` artifact, downloaded and digest-verified like the wasm and capabilities. Optional so a catalog predating published manifests still lists every tool; installing one of those is what fails, naming the cause. - `ironhub_tool_package` places the wasm and the two host-owned generic schemas at the paths the manifest declares, so publisher and host never have to agree a filename convention across two repositories. `crate_name` existed only to build those paths and is gone. - The 349-line translator is deleted, along with the catalog field it needed. - A manifest whose id contradicts the catalog entry is refused rather than resolved in the manifest's favour: installing `github` when the user asked for `attio` would shadow an unrelated extension. The second half is what a user actually feels. `capabilities.json` already recorded how to obtain each credential — Attio's says to open Workspace Settings > Developers and create an access token, with the URL beside it — but the auth recipes had nowhere to put it, so an installed tool could say it needed a secret and nothing about where the secret comes from. Users had no way to activate what they had just installed, and models asked for help invented the steps. `ApiKeyRecipe` and `OAuth2CodeRecipe` gain optional `instructions` and `setup_url`, and the shared import seam turns them into the onboarding copy the extensions UI already knows how to render. Deriving it at that seam means uploaded packages get it too, not just registry ones. `setup_url` is an `HttpsEndpoint` because it becomes a link the user is invited to follow; the text is rendered through JSX interpolation and never reaches model-visible prompt text. Verified against the live IronHub catalog: all 17 publishable tools install through the production package validator, 8 of them now carrying their vendor's real setup steps (the other 9 declare no `auth.instructions` upstream). Attio surfaces "open Workspace Settings > Developers ..." and its settings URL. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ironhub): stop naming a concrete extension in generic package code The extension-specificity gate flags a first-party extension name appearing in generic code. The identity-pin comment used two real extensions to illustrate the shadowing it prevents, and the regression test built its contradicting manifest under a real extension id. Neither needed to be a real name: the rule is about an id the user did not ask for, whichever id that is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(architecture): extract the IronHub client into ironclaw_ironhub (nearai#6870) crates/AGENTS.md gives ironclaw_extension_host an explicit non-goal: "Host authority (signing secrets, bot tokens, network egress)". The ironhub module inside it did network egress (catalog and artifact downloads), held the pinned Ed25519 catalog trust anchor, and owned the HMAC deep-link shared key — plus skill installs, which are not extension lifecycle at all. It landed there because the seam it drives (registry_extension_package) lives there, not because the crate owns the concern; the boundary tests never saw it because no dependency edge changed. Move the module wholesale to a new crate directly above extension_host. The placement rule it restores: generic registry seams (registry_extension_package, parse_imported_manifest, ManifestSource::RegistryInstalled) stay in extension_host, and the one concrete catalog client is vendor-scoped by charter, the same way each concrete extension crate is scoped to its product. The module already touched its host crate through exactly four public symbols, so extension_host's API is unchanged apart from deleting `pub mod ironhub` and dropping ed25519-dalek, which nothing else in the crate used. Wiring changes, all shape-preserving: - The binary's exact dependency allowlist stays closed: composition re-exports the command vocabulary as `ironclaw_reborn_composition::ironhub` with the house consumer-and-test doc comment, and the CLI imports the facade instead of reaching extension_host. - ironclaw_ironhub gets its dependency BoundaryRule in the same PR (a new crate is unruled by default): no execution runtimes, no secret storage, no serve/assembly layers, no concrete extension crates. - extension_host's `test-support` feature now forwards the filesystem/host_api/processes test-support seams and exposes lifecycle_test_support behind it, so the moved integration-style tests drive the same real lifecycle services from outside the crate. - Fixture include paths shorten by one directory level; the moved files are otherwise verbatim (git tracks them as renames). Verified: full architecture suite (18/18 result sets), ironclaw_ironhub 34/34, workspace clippy clean with -D warnings. Composition lib tests: 638 passed, with two known parallel-execution flakes that pass serially and one pre-existing trace-capture failure unrelated to this move (fails identically with the module in either location). Co-authored-by: serrrfirat <firatsertgoz@alumni.sabanciuniv.edu> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * fix(ironhub): accept hub-prefixed artifact digests * fix(ironhub): address review feedback * fix(ironhub): propagate manifest URL to CLI commands * fix(ironhub): close rereview findings * fix(composition): preserve configured IronHub catalog * fix(ironhub): restore failed skill replacements safely * fix(ironhub): restore complete skill bundles * fix(skills): preserve restore failure context * test(ironhub): cover mediated service entrypoints * fix(ironhub): install verified tool schemas * test(coverage): recapture extension host after IronHub extraction * test(coverage): track IronHub changed-code gaps * test(ironhub): execute reviewed coverage paths * test(ironhub): close changed coverage gap * test(composition): pin IronHub register default-off * Fix changed coverage exemption lines * fix(ironhub): address replay persistence review * test(ironhub): cover install error paths * ci: rebase changed coverage exemptions * fix(ironhub): share durable link state across surfaces * ci(coverage): rebase IronHub runtime exemptions * fix: refresh capabilities after IronHub install * test: remove stale extension host lifecycle fixture * fix: use canonical extension schemas in the loop * fix(ironhub): address coderabbit review — harden shared key validation (nearai#6780) * fix(ci): update IronHub fixture paths after package move * fix(ci): preserve selected integration coverage mode * fix(ci): keep MSRV override for selected integration lanes * fix(extensions): address think-in-universe review — provider schemas and setup copy (nearai#6780) --------- Co-authored-by: neo-sky <brandon.m.henderson93@gmail.com> Co-authored-by: serrrfirat <firatsertgoz@alumni.sabanciuniv.edu> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ironclaw-reborn ironhub search/list/info/installplus first-party search/info/install capabilitiesValidation
CARGO_INCREMENTAL=0 cargo test -p ironclaw_reborn_composition ironhub --libCARGO_INCREMENTAL=0 cargo test -p ironclaw_reborn_cli help_mentions_reborn_commands --test smokeCARGO_INCREMENTAL=0 cargo test -p ironclaw_reborn_cli ironhub_help_mentions_catalog_commands --test smokeCARGO_INCREMENTAL=0 cargo test -p ironclaw_architecture reborn_cli_binary_crate_stays_separate_from_v1_rootCARGO_INCREMENTAL=0 cargo clippy -p ironclaw_product_workflow -p ironclaw_reborn_composition -p ironclaw_reborn_cli --all-targets -- -D warningsCo-authored-by: neo-sky brandon.m.henderson93@gmail.com