Skip to content

refactor(extensions)!: complete manifest v2 cutover — host_api contracts everywhere (NEA-25 stack 2/7) - #5839

Closed
BenKurrek wants to merge 1 commit into
nea25/01-manifest-surface-modelfrom
nea25/02-manifest-cutover
Closed

BenKurrek wants to merge 1 commit into
nea25/01-manifest-surface-modelfrom
nea25/02-manifest-cutover

Conversation

@BenKurrek

Copy link
Copy Markdown
Collaborator

Summary

Stack PR 2/7 for NEA-25, on top of #5833. Per Ben's directive the stack carries zero residual legacy code: this PR finishes the manifest v2 cutover instead of keeping the legacy form alive for host-bundled packages.

  • All manifests declare [[host_api]] contracts. Top-level [[capabilities]] is rejected for every ManifestSource (host-bundled exactly as installed) with an actionable error. The 10 remaining legacy first-party manifests (gmail, google-calendar/docs/drive/sheets/slides, nearai-mcp, notion-mcp, slack, web-access) are converted to ironclaw.capability_provider/v1 sections.
  • One parse entry point. ExtensionManifestV2::parse(input, source, catalog, contracts); deleted: the 3-arg contract-free parse, parse_with_host_api_contracts, parse_with_optional_host_api_contracts, contract-free ExtensionManifestRecord::from_toml (the _with_contracts variant becomes from_toml), contract-free ExtensionDiscovery::discover, and ManifestV2Error::LegacyTopLevelCapabilitiesForInstalledSource.
  • Typed contract errors. New HostApiSectionError { Manifest(ManifestV2Error), Contract(String) }: in-crate contracts preserve precise typed variants (DuplicateEffect, UnknownHostPort, CapabilityIdNotPrefixed, …) through the section channel — previously only the deleted legacy path reported them typed; the host_api path string-flattened everything. Domain crates (product-adapter registry) keep redacted reasons wrapped as HostApiSectionRejected.
  • Production TOML surgery (NEAR AI endpoint audience rewrite in composition) navigates the new section path; the shared test fixture converters (legacy_capability_fixture_to_v2 in dispatcher + host_runtime test support) emit host_api form, so pre-v2 fixtures keep working through one converter instead of ~100 hand edits.
  • Contract doc: docs/reborn/contracts/extensions.md — "Cutover (complete)" replaces the migration-rules section; examples converted; §11 test list extended.

No persisted-state impact: installed manifests (InstalledLocal/RegistryInstalled) already could not use the legacy form; it was exclusively host-bundled assets, synthesis, and test fixtures.

Testing

  • cargo check --workspace --all-targets --all-features — clean
  • Suites green: ironclaw_extensions (incl. new pins: top-level capabilities rejected for every source; unknown top-level tables → UnreferencedOperationalSection; typed variant passthrough), ironclaw_product_adapter_registry, ironclaw_host_runtime, ironclaw_capabilities, ironclaw_event_projections, ironclaw_mcp, ironclaw_wasm, ironclaw_scripts, ironclaw_dispatcher, ironclaw_reborn_migration, ironclaw_reborn, ironclaw_reborn_composition (1,050 tests incl. NEAR AI renderer path), ironclaw_reborn_cli, ironclaw_architecture
  • cargo clippy --all --benches --tests --examples --all-features — zero warnings
  • Not run here: 3 pre-existing sandbox_process tests that need a live Docker socket (none on this machine; unrelated to this change), and --features integration Postgres tier (no DB-shaped change — parse-layer only)

🤖 Generated with Claude Code

@ironloopai

ironloopai Bot commented Jul 8, 2026 •

Copy link
Copy Markdown
Contributor

🔎 IronLoop Review Status

Head: a635cd9b0a7bcd50bb8a5b84ab866e7af47c7f0d
Result: No reviewer jobs are scheduled yet.
Next: Run @ironloopai review to start reviewers.
Updated: 2026-07-09T17:40:44.190Z

Current reviewers:

Reviewer State Verdict Findings Last update
none Queued N/A No reviewer jobs scheduled yet. N/A
Reviewer summaries
Reviewer Detail
none No reviewer jobs scheduled yet.
Recent activity
Time Reviewer State Detail
N/A N/A Waiting No progress events recorded yet.
Available commands
  • @ironloopai help
  • @ironloopai agents
  • @ironloopai review
  • @ironloopai review --agent <agent>
  • @ironloopai status
Run metadata

Admission: webhook accepted the request and IronLoop persisted reviewer state before this projection.

@coderabbitai

coderabbitai Bot commented Jul 8, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (2)
  • staging
  • reborn-integration

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b0e2d053-5f56-4ab7-84bd-7b3aec356f08

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5839 July 8, 2026 15:16 Destroyed
@github-actions github-actions Bot added scope: docs Documentation size: XL 500+ changed lines risk: low Changes to docs, tests, or low-risk modules contributor: core 20+ merged PRs labels Jul 8, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the extension manifest parsing and validation by completely removing legacy top-level [[capabilities]] declarations and enforcing that all capabilities are declared under the ironclaw.capability_provider/v1 host API contract section. This change applies to both host-bundled and installed manifests, making contract-based parsing mandatory across all manifest sources. The reviewer provided several constructive suggestions to improve the code, including removing an unused manifest_hash parameter, deserializing directly from TOML value references to avoid cloning, implementing standard error traits for HostApiSectionError, and simplifying string projection helpers in test support files using replacen and replace.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@@ -54,31 +54,10 @@ impl ExtensionManifestRecord {
source: ManifestSource,
host_port_catalog: &HostPortCatalog,
manifest_hash: Option<ManifestHash>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The manifest_hash parameter is completely unused in ExtensionManifestRecord::from_toml. Since this PR already refactors and updates the signature of from_toml across the codebase, you should remove this unused parameter to keep the API clean and avoid dead code.

Comment on lines 73 to +76
let parsed: CapabilityProviderToolsSection = section
.clone()
.try_into()
.map_err(|error: toml::de::Error| error.to_string())?;
.map_err(|error: toml::de::Error| HostApiSectionError::from(error.to_string()))?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

You can deserialize directly from the &toml::Value reference using Deserialize::deserialize instead of cloning the entire section and calling try_into(). This avoids unnecessary allocations and cloning of the TOML value.

    let parsed = CapabilityProviderToolsSection::deserialize(section)
        .map_err(|error| HostApiSectionError::from(error.to_string()))?;

Comment on lines +261 to +265
#[derive(Debug)]
pub enum HostApiSectionError {
Manifest(Box<ManifestV2Error>),
Contract(String),
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since HostApiSectionError is a public/shared error type returned by host API contracts, it should implement std::fmt::Display and std::error::Error to be idiomatic and easy to propagate or log. You can derive thiserror::Error directly since thiserror is already a dependency of this crate.

#[derive(Debug, thiserror::Error)]
pub enum HostApiSectionError {
    #[error(transparent)]
    Manifest(Box<ManifestV2Error>),
    #[error("{0}")]
    Contract(String),
}

Comment on lines +28 to 42
fn project_top_level_capabilities_to_host_api(manifest: String) -> String {
if !manifest.contains("[[capabilities]]") || manifest.contains("[[host_api]]") {
return manifest;
}
let host_api_block = "[[host_api]]\nid = \"ironclaw.capability_provider/v1\"\nsection = \"capability_provider.tools\"\n\n[capability_provider.tools]\n\n";
let idx = manifest.find("[[capabilities]]").expect("checked above");
let mut out = String::with_capacity(manifest.len() + host_api_block.len());
out.push_str(&manifest[..idx]);
out.push_str(host_api_block);
out.push_str(&manifest[idx..]);
out.replace(
"[[capabilities]]",
"[[capability_provider.tools.capabilities]]",
)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This helper can be simplified significantly by using replacen to replace only the first occurrence of [[capabilities]] with the host_api block and the new capability header, and then using replace to convert any remaining occurrences. This avoids manual index searching, slicing, and capacity allocation, making the code much more readable and maintainable.

Suggested change
fn project_top_level_capabilities_to_host_api(manifest: String) -> String {
if !manifest.contains("[[capabilities]]") || manifest.contains("[[host_api]]") {
return manifest;
}
let host_api_block = "[[host_api]]\nid = \"ironclaw.capability_provider/v1\"\nsection = \"capability_provider.tools\"\n\n[capability_provider.tools]\n\n";
let idx = manifest.find("[[capabilities]]").expect("checked above");
let mut out = String::with_capacity(manifest.len() + host_api_block.len());
out.push_str(&manifest[..idx]);
out.push_str(host_api_block);
out.push_str(&manifest[idx..]);
out.replace(
"[[capabilities]]",
"[[capability_provider.tools.capabilities]]",
)
}
fn project_top_level_capabilities_to_host_api(manifest: String) -> String {
if !manifest.contains("[[capabilities]]") || manifest.contains("[[host_api]]") {
return manifest;
}
let first_replace = r#"[[host_api]]
id = "ironclaw.capability_provider/v1"
section = "capability_provider.tools"
[capability_provider.tools]
[[capability_provider.tools.capabilities]]"#;
manifest
.replacen("[[capabilities]]", first_replace, 1)
.replace("[[capabilities]]", "[[capability_provider.tools.capabilities]]")
}
References
  1. When canonicalizing a string, perform cheaper validation checks (like length, emptiness, and character set) on a trimmed slice before performing more expensive operations like replace that allocate a new string. This avoids unnecessary allocations for already-invalid inputs.

Comment on lines +55 to 69
fn project_top_level_capabilities_to_host_api(manifest: String) -> String {
if !manifest.contains("[[capabilities]]") || manifest.contains("[[host_api]]") {
return manifest;
}
let host_api_block = "[[host_api]]\nid = \"ironclaw.capability_provider/v1\"\nsection = \"capability_provider.tools\"\n\n[capability_provider.tools]\n\n";
let idx = manifest.find("[[capabilities]]").expect("checked above");
let mut out = String::with_capacity(manifest.len() + host_api_block.len());
out.push_str(&manifest[..idx]);
out.push_str(host_api_block);
out.push_str(&manifest[idx..]);
out.replace(
"[[capabilities]]",
"[[capability_provider.tools.capabilities]]",
)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This helper can be simplified significantly by using replacen to replace only the first occurrence of [[capabilities]] with the host_api block and the new capability header, and then using replace to convert any remaining occurrences. This avoids manual index searching, slicing, and capacity allocation, making the code much more readable and maintainable.

Suggested change
fn project_top_level_capabilities_to_host_api(manifest: String) -> String {
if !manifest.contains("[[capabilities]]") || manifest.contains("[[host_api]]") {
return manifest;
}
let host_api_block = "[[host_api]]\nid = \"ironclaw.capability_provider/v1\"\nsection = \"capability_provider.tools\"\n\n[capability_provider.tools]\n\n";
let idx = manifest.find("[[capabilities]]").expect("checked above");
let mut out = String::with_capacity(manifest.len() + host_api_block.len());
out.push_str(&manifest[..idx]);
out.push_str(host_api_block);
out.push_str(&manifest[idx..]);
out.replace(
"[[capabilities]]",
"[[capability_provider.tools.capabilities]]",
)
}
fn project_top_level_capabilities_to_host_api(manifest: String) -> String {
if !manifest.contains("[[capabilities]]") || manifest.contains("[[host_api]]") {
return manifest;
}
let first_replace = r#"[[host_api]]
id = "ironclaw.capability_provider/v1"
section = "capability_provider.tools"
[capability_provider.tools]
[[capability_provider.tools.capabilities]]"#;
manifest
.replacen("[[capabilities]]", first_replace, 1)
.replace("[[capabilities]]", "[[capability_provider.tools.capabilities]]")
}
References
  1. When canonicalizing a string, perform cheaper validation checks (like length, emptiness, and character set) on a trimmed slice before performing more expensive operations like replace that allocate a new string. This avoids unnecessary allocations for already-invalid inputs.

@ironloopai ironloopai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ IronLoop Review: reviewer

Review at a glance

Verdict Blocking Notes Inline Head
❌ Changes requested 1 0 1 2fdd7303f360

Head: 2fdd7303f360cf518ba4c58ff86f96267fbf4c5b
Next: Fix the blocking findings, push the PR branch, then re-run this reviewer.

Run details

Status: Current
Needs human: no
Needs validation: no

Summary

Found one blocking upgrade compatibility issue in the manifest v2 cutover: persisted host-bundled extension installation state from the previous manifest shape can make Reborn fail to load before the existing bundled-manifest migration can run.

Findings

Blocking: 1 / Notes: 0

Blocking findings

1. ❌ [HIGH] Persisted legacy host-bundled manifests fail store load before migration

Location: crates/ironclaw_reborn_composition/src/extension_host/extension_installation_store.rs:233-239
WireManifestRecord::into_manifest_record now reparses every persisted raw manifest through the new strict ExtensionManifestRecord::from_toml path, which rejects top-level [[capabilities]] for all sources. Existing installation state written by the previous version can contain source = HostBundled records whose raw_toml came from the old bundled assets with top-level capabilities. On upgrade, load_at returns an error while reading state, and build_reborn_services fails before migrate_host_bundled_manifest_hash can replace those records with the new bundled manifests. Add a compatibility path for persisted HostBundled legacy records, or rebuild/migrate those records during state load instead of failing the entire store.

Developer follow-up

After fixing this feedback:

  1. Push the fix to this PR branch.
  2. Re-run this reviewer with @ironloopai review --agent reviewer if you only changed this reviewer's findings.
  3. Re-run all reviewers with @ironloopai review when the fix may affect multiple areas.
  4. Use @ironloopai status to check queued/running/completed/stale/stalled state while reviewers run.

let contracts = ironclaw_host_runtime::default_host_api_contract_registry()
.map_err(invalid_installation_error)?;
ExtensionManifestRecord::from_toml_with_contracts(
ExtensionManifestRecord::from_toml(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This strict reparse makes old persisted HostBundled records unreadable. Previous versions stored bundled manifests with top-level [[capabilities]]; after this cutover ExtensionManifestRecord::from_toml rejects them before the bundled manifest hash migration can run, so FilesystemExtensionInstallationStore::load_at fails and Reborn startup/config load is blocked for users with existing installed extensions. Please add a load-time compatibility/migration path for those HostBundled legacy records.

@railway-app

railway-app Bot commented Jul 8, 2026 •

Copy link
Copy Markdown

🚅 Deployed to the ironclaw-pr-5839 environment in ironclaw-ci-preview

Service Status Web Updated (UTC)
ironclaw ✅ Success (View Logs) Web Jul 13, 2026 at 5:21 pm

@BenKurrek
BenKurrek force-pushed the nea25/01-manifest-surface-model branch from 014e49a to b081db0 Compare July 8, 2026 21:22
@BenKurrek
BenKurrek force-pushed the nea25/02-manifest-cutover branch from 2fdd730 to 6b2eb09 Compare July 8, 2026 21:22
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5839 July 8, 2026 21:22 Destroyed
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5839 July 9, 2026 17:40 Destroyed
@BenKurrek
BenKurrek force-pushed the nea25/01-manifest-surface-model branch from 60e98ab to 0fda67b Compare July 13, 2026 16:15
@BenKurrek
BenKurrek force-pushed the nea25/02-manifest-cutover branch from a635cd9 to 412810e Compare July 13, 2026 17:08
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5839 July 13, 2026 17:08 Destroyed
…cts everywhere

Every manifest now declares its sections through [[host_api]] contracts;
the legacy top-level [[capabilities]] form is rejected for every source,
host-bundled exactly as installed. All 10 remaining legacy first-party
manifests (gmail, google-calendar/docs/drive/sheets/slides, nearai-mcp,
notion-mcp, slack, web-access) move onto the
ironclaw.capability_provider/v1 section form.

One parse entry point remains: ExtensionManifestV2::parse(input, source,
catalog, contracts). The contract-free record constructor, the optional-
contracts variant, and contract-free ExtensionDiscovery::discover are
deleted, along with LegacyTopLevelCapabilitiesForInstalledSource.

Host API contracts now raise a typed HostApiSectionError: in-crate
contracts (capability provider) preserve precise ManifestV2Error
variants (DuplicateEffect, UnknownHostPort, CapabilityIdNotPrefixed, ...)
instead of string-flattening them - previously only the deleted legacy
path reported typed errors. Domain crates keep redacted reason strings
wrapped as HostApiSectionRejected.

Production TOML surgery (NEAR AI endpoint audience rewrite) and the
shared test fixture converters (legacy_capability_fixture_to_v2 in the
dispatcher and host_runtime test support) now emit the host_api form.

Contract: docs/reborn/contracts/extensions.md "Cutover (complete)" +
converted examples. NEA-25 stack PR 2; no persisted-state impact
(installed manifests already could not use the legacy form).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@BenKurrek
BenKurrek force-pushed the nea25/02-manifest-cutover branch from 412810e to 9a6ef20 Compare July 13, 2026 17:12
@railway-app
railway-app Bot temporarily deployed to ironclaw-ci-preview / ironclaw-pr-5839 July 13, 2026 17:12 Destroyed
BenKurrek added a commit that referenced this pull request Jul 13, 2026
…#5850)

Atomic roll-up of the 8-PR NEA-25 taxonomy stack onto current main. Extension is
the only installable product object; tool/channel/auth are derived capability
surfaces; runtime kind controls loading only; manifest projection (v2, host_api
contracts) is the sole surface-discovery source of truth. The connectable-channels
rail and the parallel `kind` taxonomy are removed and pinned by a zero-legacy gate.
slack_bot and slack_personal are retired into one `slack` extension with bounded
forward migrations. Extensions wire carries runtime + surfaces, not a conflated kind.

Supersedes #5833, #5839, #5842, #5845, #5847, #5848, #5849, #5850. Conflicts with
main since the train forked were reconciled preserving main's newer behavior
(#5851 unified slack cleanup, #6054 get_conversation_info DM resolution, #5499
extension import, #6057 TS source conventions). provider_identity domain
duplication removed; the residual is a legitimate up-layer port adapter. See the PR
description for the per-PR crosswalk, resolutions, placement audit, and verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@BenKurrek BenKurrek closed this Jul 20, 2026

This branch was successfully deployed

No deployments
ironclaw-ci-preview / ironclaw-pr-5839 — 9a6ef20b Deployed Jul 13, 2026 by railway-app[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor: core 20+ merged PRs risk: low Changes to docs, tests, or low-risk modules scope: docs Documentation size: XL 500+ changed lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant