Skip to content

Skills M4: Run addon-mcp and @storybook/mcp on the shared core toolsets - #35677

Merged
JReinhold merged 7 commits into
m2b-core-toolsetsfrom
m4-addon-mcp-core-services
Aug 12, 2026
Merged

JReinhold merged 7 commits into
m2b-core-toolsetsfrom
m4-addon-mcp-core-services

Conversation

@kasperpeulen

@kasperpeulen kasperpeulen commented Jul 30, 2026 •

Copy link
Copy Markdown
Member

Closes #35673 · Closes #35725

Stacked: this PR's base is #35726 — the core toolsets layer this PR consumes; review that one first. #35719 (the storybook tools CLI) stacks on this one. Merge #35726 first; GitHub retargets this PR to next automatically.

What I did

#35726 built Storybook's agent-facing capabilities as core toolsets. This PR is the switch: both MCP surfaces — @storybook/addon-mcp (the dev-server MCP) and @storybook/mcp (the hosted/composition MCP) — now render their tools from those shared definitions, and the engines they each carried are deleted. Net: −11,199 lines.

What remains in each package is an adapter. It resolves a toolset method and maps the outcome onto the MCP reply mechanically: schemas and prose come from the definition, markdown becomes the text blocks, data becomes structuredContent, ok becomes isError, and agent-facing errors surface verbatim. Meaning lives in the toolset — the adapter is not allowed to re-derive it from the data.

Concretely, an MCP tool is now a registry row pointing at a toolset method (real code):

function docsRow(method: 'docs.list' | 'docs.show' | 'docs.showStory'): AddonToolDefinition {
  const forContext = (context: AddonToolRegistryContext): ToolsetToolOptions =>
    context.multiSource
      ? { method, resolveToolset: (server) => compositionDocsToolset(server) }
      : { method };

  return {
    name: MCP_TOOL_NAMES[method],
    toolset: 'docs',
    available: ({ availability }) => availability.docsEnabled,
    getMetadata: (context) => getToolsetToolMetadata(forContext(context)),
    register: async (server, context, enabled) => {
      registerToolsetTool(server, forContext(context), enabled);
    },
  };
}

The one review question

Does the way the MCP uses the toolsets make sense — and is there business logic still in the MCP that belongs in a toolset? Everything in commits 1–3 should read as mechanical: naming, gating, unwrapping, transport. If you find an adapter interpreting a result — branching on domain data, rewriting prose, computing anything the future CLI consumer would have to duplicate — that is the review finding to raise.

You do not need to review the deleted engines: commit 4/7 is −12,570 lines of pure deletions, and behavioral equality with them is pinned by the e2e wire snapshots (evidence below), not by reading them.

How to review (~1½ hours)

The branch's seven commits are these seven blocks, in this order. Each block heading links straight to its commit's diff — review one commit at a time, and read the block's intro here before opening any file.

  1. The addon adapter — 30 min
  2. The hosted adapter — 25 min
  3. Addon rewiring — 15 min
  4. Delete the replaced engines — 0 min
  5. Retire the PUSH_REVIEW channel adapter — 5 min
  6. The adapter tests and the e2e proof — 15 min
  7. Docs and bookkeeping — 5 min

Block 1 · The addon adapter — 30 min

Files: toolset-tools.ts, tool-registry.ts, ui-root.ts, tool-names.ts (4 files, +425 −109)

The entire addon-side integration is two ideas. The first is one generic unwrap that turns any toolset outcome into an MCP result — toolset-tools.ts, 187 lines, read it in full:

callToolsetMethod(server, { method }, input):
  toolset = getToolset(id)                       // throws loudly if missing
  outcome = method.handler(input, ctx(server))   // ctx carries origin, uiRoot, telemetry, consumer:'mcp'
  return {
    content:           one text block per outcome.markdown entry
    structuredContent: outputSchema ? narrowed(outcome.data) : undefined
    isError:           outcome.ok ? undefined : true
  }
  on throw: error.agentFacing ? its message, verbatim
                              : a generic message + logger.error on the server

The second is a declarative registry where every MCP tool is a row pointing at a toolset method — the docsRow snippet above is one real row. tool-registry.ts walks the rows; the availability gate × the toolsets config decides what registers.

What to check: nothing in this block may interpret data — no branching on domain fields, no re-deriving prose from the payload. If you find the adapter deciding what a result means, that is the finding to raise.

Block 2 · The hosted adapter — 25 min

Files: register.ts, error-to-mcp-content.ts, multi-source-manifests.ts, index.ts, types.ts, bin.ts, package.json (7 files, +497 −225)

The same unwrap for @storybook/mcp, with one structural difference: the hosted server builds its docs toolset per request, because the provider and sources belong to the request:

handle(request):
  sources = resolveSources(request)          // which Storybooks, which URLs, which auth
  toolset = createDocsToolset({ sources })   // the portable core factory — no second engine here
  register each method → the same mechanical unwrap as block 1

packaging: 'storybook' becomes a bundled devDependency —
           the published dependencies stay free of it (dist-contract.test.ts pins this)

What to check: register.ts (346 lines) in full — the per-request construction, the twin error mapping, and that the package boundary holds.

Block 3 · Addon rewiring — 15 min

Files: preset, mcp-handler, availability, telemetry, instructions, auth (16 files, +131 −386)

Mechanical re-pointing: everything that used to call an engine now consults the registry. Net −255 lines. Skim with one question: did any logic sneak in here, or is it all naming and plumbing?

Block 4 · Delete the replaced engines — 0 min

Files: the old tools, manifest pipeline, formatter and utils of both packages (47 files, −12,570, zero insertions)

Don't review. This is the point of the PR: every deleted line is an engine the core toolsets replaced. Behavioral equality is proven by block 6, not by reading these.

Block 5 · Retire the PUSH_REVIEW channel adapter — 5 min

Files: common-preset.ts, review-channel.ts (+ test), events.ts (4 files, +3 −200)

before: the addon emits 'storybook/review/push-review' on the channel; a core adapter listens
after : the review toolset method is called directly — the channel surface is deleted

#35726 deliberately kept this adapter alive so released addon-mcp versions kept working against the new core; with the addon switched over in this PR, it retires. One glance.

Block 6 · The adapter tests and the e2e proof — 15 min

Files: adapter unit tests, dist-contract.test.ts, test-storybooks/mcp (22 files, +1,569 −366)

Skim as evidence, not as code under review: the contract tests pin the unwrap (outcome → isError, structuredContent narrowing), gating (including the preview app resource), telemetry, and agentFacing error surfacing; the e2e suite exercises the real wire against real servers.

Block 7 · Docs and bookkeeping — 5 min

Files: AGENTS.md, agent-eval templates, yarn.lock (4 files, +89 −57)

The AGENTS.md architecture section documents the end state of the whole layer — worth an actual read: it is also the text future agents working in this repo obey.

Why skipping the deletions is safe:

Claim Evidence
The wire behavior did not change The MCP e2e suite (46/46 — real dev servers and real hosted manifests) passes with byte-identical inline snapshots before and after the swap
The adapters' contracts are pinned Unit tests cover the outcome→isError mapping, structuredContent narrowing, telemetry, agentFacing error surfacing, and toolset gating (including the preview app resource)
Nothing else regressed The full 9,212-test unit suite, TS 7 per-package checks across 47 projects plus workspace TS 6, and manual QA of the preview, test, docs and review flows
@storybook/mcp ships without storybook at runtime dist-contract.test.ts pins the published dependency surface; the docs toolset arrives through the portable storybook/internal/toolsets-docs entry

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

  • stories
  • unit tests
  • integration tests
  • end-to-end tests

Manual testing

  1. yarn nx run-many -t compile
  2. cd test-storybooks/mcp && yarn install && yarn vitest run --project=e2e — 46 tests, inline snapshots byte-identical to the old engines
  3. yarn storybook in test-storybooks/mcp, connect an MCP client to http://localhost:6006/mcp — the tool list and every tool response match the released addon
  4. The review flow end-to-end (display-review → UI) — now served by the review toolset; the PUSH_REVIEW channel event no longer exists
  5. Composition: the hosted @storybook/mcp against the Chromatic-hosted Storybooks — listing groups per source, lookups take storybookId

Documentation

  • Add or update documentation reflecting your changes — the AGENTS.md architecture section in this PR documents the end state of the toolset layer and its consumers
  • If you are deprecating/removing a feature, make sure to update
    MIGRATION.MD

Checklist for Maintainers

  • When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli-storybook/src/sandbox-templates.ts

  • Declare whether manual QA will be needed for this PR during the next release, through qa:needed or qa:skip

  • Make sure this PR contains one of the labels below:

    Available labels
    • bug: Internal changes that fixes incorrect behavior.
    • maintenance: User-facing maintenance tasks.
    • dependencies: Upgrading (sometimes downgrading) dependencies.
    • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
    • cleanup: Minor cleanup style change. Will not show up in release changelog.
    • documentation: Documentation only changes. Will not show up in release changelog.
    • feature request: Introducing a new feature.
    • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
    • other: Changes that don't fit in the above categories.

@kasperpeulen
kasperpeulen requested a review from a team July 30, 2026 18:39
@kasperpeulen kasperpeulen changed the title Skills M4: run addon-mcp and @storybook/mcp on the shared core toolsets Skills M4: Run addon-mcp and @storybook/mcp on the shared core toolsets Jul 30, 2026
@kasperpeulen kasperpeulen added maintenance User-facing maintenance tasks ci:normal Run our default set of CI jobs (choose this for most PRs). qa:needed Pull Requests that will need manual QA prior to release. labels Jul 30, 2026
@storybook-app-bot

storybook-app-bot Bot commented Jul 30, 2026 •

Copy link
Copy Markdown
Contributor

Package Benchmarks

Commit: b0ac819, ran on 28 August 2026 at 08:24:30 UTC

The following packages have significant changes to their size or dependencies:

@storybook/addon-a11y

Before After Difference
Dependency count 0 2 🚨 +2 🚨
Self size 0 B 188 KB 🚨 +188 KB 🚨
Dependency size 0 B 3.12 MB 🚨 +3.12 MB 🚨
Bundle Size Analyzer Link Link

@storybook/addon-docs

Before After Difference
Dependency count 0 18 🚨 +18 🚨
Self size 0 B 1.29 MB 🚨 +1.29 MB 🚨
Dependency size 0 B 9.29 MB 🚨 +9.29 MB 🚨
Bundle Size Analyzer Link Link

@storybook/addon-links

Before After Difference
Dependency count 0 1 🚨 +1 🚨
Self size 0 B 14 KB 🚨 +14 KB 🚨
Dependency size 0 B 5 KB 🚨 +5 KB 🚨
Bundle Size Analyzer Link Link

@storybook/addon-mcp

Before After Difference
Dependency count 0 11 🚨 +11 🚨
Self size 0 B 129 KB 🚨 +129 KB 🚨
Dependency size 0 B 2.74 MB 🚨 +2.74 MB 🚨
Bundle Size Analyzer Link Link

@storybook/addon-onboarding

Before After Difference
Dependency count 0 0 0
Self size 0 B 332 KB 🚨 +332 KB 🚨
Dependency size 0 B 670 B 🚨 +670 B 🚨
Bundle Size Analyzer Link Link

storybook-addon-pseudo-states

Before After Difference
Dependency count 0 0 0
Self size 0 B 21 KB 🚨 +21 KB 🚨
Dependency size 0 B 689 B 🚨 +689 B 🚨
Bundle Size Analyzer Link Link

@storybook/addon-themes

Before After Difference
Dependency count 0 1 🚨 +1 🚨
Self size 0 B 18 KB 🚨 +18 KB 🚨
Dependency size 0 B 28 KB 🚨 +28 KB 🚨
Bundle Size Analyzer Link Link

@storybook/addon-vitest

Before After Difference
Dependency count 0 2 🚨 +2 🚨
Self size 0 B 430 KB 🚨 +430 KB 🚨
Dependency size 0 B 350 KB 🚨 +350 KB 🚨
Bundle Size Analyzer Link Link

@storybook/builder-vite

Before After Difference
Dependency count 0 11 🚨 +11 🚨
Self size 0 B 136 KB 🚨 +136 KB 🚨
Dependency size 0 B 1.33 MB 🚨 +1.33 MB 🚨
Bundle Size Analyzer Link Link

@storybook/builder-webpack5

Before After Difference
Dependency count 0 182 🚨 +182 🚨
Self size 0 B 79 KB 🚨 +79 KB 🚨
Dependency size 0 B 37.43 MB 🚨 +37.43 MB 🚨
Bundle Size Analyzer Link Link

storybook

Before After Difference
Dependency count 0 73 🚨 +73 🚨
Self size 0 B 21.77 MB 🚨 +21.77 MB 🚨
Dependency size 0 B 30.98 MB 🚨 +30.98 MB 🚨
Bundle Size Analyzer Link Link

@storybook/angular

Before After Difference
Dependency count 0 185 🚨 +185 🚨
Self size 0 B 253 KB 🚨 +253 KB 🚨
Dependency size 0 B 30.28 MB 🚨 +30.28 MB 🚨
Bundle Size Analyzer Link Link

@storybook/angular-vite

Before After Difference
Dependency count 0 29 🚨 +29 🚨
Self size 0 B 23.04 MB 🚨 +23.04 MB 🚨
Dependency size 0 B 12.69 MB 🚨 +12.69 MB 🚨
Bundle Size Analyzer Link Link

@storybook/ember

Before After Difference
Dependency count 0 181 🚨 +181 🚨
Self size 0 B 13 KB 🚨 +13 KB 🚨
Dependency size 0 B 32.78 MB 🚨 +32.78 MB 🚨
Bundle Size Analyzer Link Link

@storybook/html-vite

Before After Difference
Dependency count 0 14 🚨 +14 🚨
Self size 0 B 22 KB 🚨 +22 KB 🚨
Dependency size 0 B 1.50 MB 🚨 +1.50 MB 🚨
Bundle Size Analyzer Link Link

@storybook/nextjs

Before After Difference
Dependency count 0 527 🚨 +527 🚨
Self size 0 B 641 KB 🚨 +641 KB 🚨
Dependency size 0 B 64.12 MB 🚨 +64.12 MB 🚨
Bundle Size Analyzer Link Link

@storybook/nextjs-vite

Before After Difference
Dependency count 0 94 🚨 +94 🚨
Self size 0 B 1.42 MB 🚨 +1.42 MB 🚨
Dependency size 0 B 23.88 MB 🚨 +23.88 MB 🚨
Bundle Size Analyzer Link Link

@storybook/preact-vite

Before After Difference
Dependency count 0 14 🚨 +14 🚨
Self size 0 B 12 KB 🚨 +12 KB 🚨
Dependency size 0 B 1.52 MB 🚨 +1.52 MB 🚨
Bundle Size Analyzer Link Link

@storybook/react-native-web-vite

Before After Difference
Dependency count 0 125 🚨 +125 🚨
Self size 0 B 29 KB 🚨 +29 KB 🚨
Dependency size 0 B 25.90 MB 🚨 +25.90 MB 🚨
Bundle Size Analyzer Link Link

@storybook/react-vite

Before After Difference
Dependency count 0 83 🚨 +83 🚨
Self size 0 B 32 KB 🚨 +32 KB 🚨
Dependency size 0 B 21.19 MB 🚨 +21.19 MB 🚨
Bundle Size Analyzer Link Link

@storybook/react-webpack5

Before After Difference
Dependency count 0 268 🚨 +268 🚨
Self size 0 B 23 KB 🚨 +23 KB 🚨
Dependency size 0 B 49.86 MB 🚨 +49.86 MB 🚨
Bundle Size Analyzer Link Link

@storybook/server-webpack5

Before After Difference
Dependency count 0 194 🚨 +194 🚨
Self size 0 B 15 KB 🚨 +15 KB 🚨
Dependency size 0 B 38.69 MB 🚨 +38.69 MB 🚨
Bundle Size Analyzer Link Link

@storybook/svelte-vite

Before After Difference
Dependency count 0 20 🚨 +20 🚨
Self size 0 B 54 KB 🚨 +54 KB 🚨
Dependency size 0 B 26.67 MB 🚨 +26.67 MB 🚨
Bundle Size Analyzer Link Link

@storybook/sveltekit

Before After Difference
Dependency count 0 21 🚨 +21 🚨
Self size 0 B 56 KB 🚨 +56 KB 🚨
Dependency size 0 B 26.73 MB 🚨 +26.73 MB 🚨
Bundle Size Analyzer Link Link

@storybook/tanstack-react

Before After Difference
Dependency count 0 84 🚨 +84 🚨
Self size 0 B 118 KB 🚨 +118 KB 🚨
Dependency size 0 B 21.22 MB 🚨 +21.22 MB 🚨
Bundle Size Analyzer Link Link

@storybook/vue3-vite

Before After Difference
Dependency count 0 103 🚨 +103 🚨
Self size 0 B 31 KB 🚨 +31 KB 🚨
Dependency size 0 B 19.60 MB 🚨 +19.60 MB 🚨
Bundle Size Analyzer Link Link

@storybook/web-components-vite

Before After Difference
Dependency count 0 15 🚨 +15 🚨
Self size 0 B 19 KB 🚨 +19 KB 🚨
Dependency size 0 B 1.56 MB 🚨 +1.56 MB 🚨
Bundle Size Analyzer Link Link

@storybook/cli

Before After Difference
Dependency count 0 205 🚨 +205 🚨
Self size 0 B 833 KB 🚨 +833 KB 🚨
Dependency size 0 B 87.11 MB 🚨 +87.11 MB 🚨
Bundle Size Analyzer Link Link

@storybook/codemod

Before After Difference
Dependency count 0 198 🚨 +198 🚨
Self size 0 B 32 KB 🚨 +32 KB 🚨
Dependency size 0 B 85.59 MB 🚨 +85.59 MB 🚨
Bundle Size Analyzer Link Link

@storybook/core-webpack

Before After Difference
Dependency count 0 1 🚨 +1 🚨
Self size 0 B 11 KB 🚨 +11 KB 🚨
Dependency size 0 B 28 KB 🚨 +28 KB 🚨
Bundle Size Analyzer Link Link

create-storybook

Before After Difference
Dependency count 0 74 🚨 +74 🚨
Self size 0 B 1.09 MB 🚨 +1.09 MB 🚨
Dependency size 0 B 52.75 MB 🚨 +52.75 MB 🚨
Bundle Size Analyzer node node

@storybook/csf-plugin

Before After Difference
Dependency count 0 9 🚨 +9 🚨
Self size 0 B 7 KB 🚨 +7 KB 🚨
Dependency size 0 B 1.29 MB 🚨 +1.29 MB 🚨
Bundle Size Analyzer Link Link

eslint-plugin-storybook

Before After Difference
Dependency count 0 20 🚨 +20 🚨
Self size 0 B 137 KB 🚨 +137 KB 🚨
Dependency size 0 B 3.70 MB 🚨 +3.70 MB 🚨
Bundle Size Analyzer Link Link

@storybook/mcp

Before After Difference
Dependency count 0 11 🚨 +11 🚨
Self size 0 B 144 KB 🚨 +144 KB 🚨
Dependency size 0 B 2.74 MB 🚨 +2.74 MB 🚨
Bundle Size Analyzer Link Link

@storybook/react-dom-shim

Before After Difference
Dependency count 0 0 0
Self size 0 B 19 KB 🚨 +19 KB 🚨
Dependency size 0 B 1 KB 🚨 +1 KB 🚨
Bundle Size Analyzer Link Link

@storybook/preset-create-react-app

Before After Difference
Dependency count 0 68 🚨 +68 🚨
Self size 0 B 32 KB 🚨 +32 KB 🚨
Dependency size 0 B 6.12 MB 🚨 +6.12 MB 🚨
Bundle Size Analyzer Link Link

@storybook/preset-react-webpack

Before After Difference
Dependency count 0 157 🚨 +157 🚨
Self size 0 B 18 KB 🚨 +18 KB 🚨
Dependency size 0 B 34.39 MB 🚨 +34.39 MB 🚨
Bundle Size Analyzer Link Link

@storybook/preset-server-webpack

Before After Difference
Dependency count 0 10 🚨 +10 🚨
Self size 0 B 7 KB 🚨 +7 KB 🚨
Dependency size 0 B 1.20 MB 🚨 +1.20 MB 🚨
Bundle Size Analyzer Link Link

@storybook/html

Before After Difference
Dependency count 0 2 🚨 +2 🚨
Self size 0 B 29 KB 🚨 +29 KB 🚨
Dependency size 0 B 33 KB 🚨 +33 KB 🚨
Bundle Size Analyzer Link Link

@storybook/preact

Before After Difference
Dependency count 0 2 🚨 +2 🚨
Self size 0 B 47 KB 🚨 +47 KB 🚨
Dependency size 0 B 33 KB 🚨 +33 KB 🚨
Bundle Size Analyzer Link Link

@storybook/react

Before After Difference
Dependency count 0 59 🚨 +59 🚨
Self size 0 B 1.46 MB 🚨 +1.46 MB 🚨
Dependency size 0 B 12.28 MB 🚨 +12.28 MB 🚨
Bundle Size Analyzer Link Link

@storybook/server

Before After Difference
Dependency count 0 3 🚨 +3 🚨
Self size 0 B 9 KB 🚨 +9 KB 🚨
Dependency size 0 B 719 KB 🚨 +719 KB 🚨
Bundle Size Analyzer Link Link

@storybook/svelte

Before After Difference
Dependency count 0 3 🚨 +3 🚨
Self size 0 B 49 KB 🚨 +49 KB 🚨
Dependency size 0 B 601 KB 🚨 +601 KB 🚨
Bundle Size Analyzer Link Link

@storybook/vue3

Before After Difference
Dependency count 0 91 🚨 +91 🚨
Self size 0 B 103 KB 🚨 +103 KB 🚨
Dependency size 0 B 18.15 MB 🚨 +18.15 MB 🚨
Bundle Size Analyzer Link Link

@storybook/web-components

Before After Difference
Dependency count 0 3 🚨 +3 🚨
Self size 0 B 80 KB 🚨 +80 KB 🚨
Dependency size 0 B 48 KB 🚨 +48 KB 🚨
Bundle Size Analyzer Link Link

@coderabbitai

coderabbitai Bot commented Jul 30, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The MCP migration centralizes Storybook tools in shared toolsets. It adds hosted documentation adapters, request-scoped manifest access, structured outcomes, feature gating, CLI consumer telemetry, and package contract tests. Legacy MCP implementations and review-channel wiring are removed.

Changes

MCP toolset migration

Layer / File(s) Summary
Core registration and feature gates
AGENTS.md, code/core/src/core-server/presets/common-preset.ts, code/addons/mcp/src/utils/*
Core services register Stories, Review, and Docs toolsets. Vitest availability now checks whether the addon is enabled.
Toolset-backed MCP adapter
code/addons/mcp/src/tools/toolset-tools.ts, code/addons/mcp/src/tools/tool-registry.ts, code/addons/mcp/src/mcp-handler.ts
MCP metadata, request context, structured output, error handling, UI roots, telemetry, and registration use shared toolsets.
Hosted documentation and composition access
code/lib/mcp/src/tools/register.ts, code/lib/mcp/src/utils/*, code/addons/mcp/src/preset.ts, code/addons/mcp/src/auth/*
Documentation tools use request-scoped manifest providers, shared docs types, local docs access, and composition-aware source handling.
Validation and distribution
code/lib/mcp/src/dist-contract.test.ts, test-storybooks/mcp/tests/*, code/addons/mcp/src/tools/*.test.ts
Tests cover tool availability, composition, outcomes, telemetry, package artifacts, Vitest gating, and unusable Git behavior.

Possibly related issues

Possibly related PRs

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🧹 Nitpick comments (9)
code/addons/mcp/src/tools/toolset-tools.test.ts (1)

1-11: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Module mocks don't use spy: true / beforeEach per repo Vitest guidelines.

vi.mock('storybook/internal/core-server', ...) and vi.mock('../telemetry.ts', ...) use plain factory functions instead of the spy: true option, and the mock return values are set at module scope instead of inside a beforeEach block.

As per coding guidelines, "Use vi.mock() with the spy: true option for all package and file mocks in Vitest tests" and "Implement mock behaviors in beforeEach blocks in Vitest tests".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/addons/mcp/src/tools/toolset-tools.test.ts` around lines 1 - 11, Update
the module mocks in the toolset-tools test to use Vitest’s spy mode with spy:
true instead of factory functions, and move mock behavior/configuration into a
beforeEach block. Preserve the existing mocked getService and collectTelemetry
behavior while ensuring each test starts with freshly configured mocks.

Source: Coding guidelines

code/core/src/shared/open-service/toolset-definition.ts (1)

53-60: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

outputSchema is unconstrained relative to TOutput.

handler output and the published outputSchema can drift silently; since the adapter validates structuredContent against that schema, a mismatch only surfaces as an MCP runtime validation failure. Consider tying the schema's inferred input to the handler's data.

♻️ Sketch
-export type ToolsetMethod<TSchema extends AnySchema = AnySchema, TOutput = unknown> = {
+export type ToolsetMethod<TSchema extends AnySchema = AnySchema, TOutput = unknown> = {
   description: ToolsetMethodDescription;
   schema: TSchema;
   /** Published as the MCP tool's `outputSchema`. Declare it only where the JSON is contractual. */
-  outputSchema?: AnySchema;
+  outputSchema?: StandardSchemaV1<Awaited<TOutput>, unknown>;
   handler: (input: StandardSchemaV1.InferOutput<TSchema>, context: ToolsetCtx) => TOutput;
   format: (data: Awaited<TOutput>, context: ToolsetCtx) => string | string[];
 };
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/shared/open-service/toolset-definition.ts` around lines 53 -
60, Update ToolsetMethod so the optional outputSchema is generically tied to
TOutput, constraining its schema type to one whose inferred output matches the
handler’s Awaited<TOutput>. Preserve outputSchema as optional and keep the
existing handler and format signatures unchanged.
code/core/src/shared/open-service/toolsets/docs/definition.test.ts (1)

68-75: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use the exported resolveToolsetDescription here.

This re-implements the helper that ships in toolset-definition.ts (and is used by the stories toolset tests), so the test can drift from production resolution semantics.

♻️ Proposed change
-    const describe_ = toolset.methods.list.description;
-    const resolved = typeof describe_ === 'function' ? describe_(mcpCtx) : describe_;
-    const resolvedCli = typeof describe_ === 'function' ? describe_(cliCtx) : describe_;
+    const resolved = resolveToolsetDescription(toolset.methods.list.description, mcpCtx);
+    const resolvedCli = resolveToolsetDescription(toolset.methods.list.description, cliCtx);

Plus the import:

-import type { ToolsetCtx } from '../../toolset-definition.ts';
+import { resolveToolsetDescription, type ToolsetCtx } from '../../toolset-definition.ts';
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/shared/open-service/toolsets/docs/definition.test.ts` around
lines 68 - 75, Update the test around “cross-references the show tool per
consumer in its description” to import and use the exported
resolveToolsetDescription helper from toolset-definition.ts for both mcpCtx and
cliCtx, removing the inline typeof/function resolution so the test follows
production semantics.
code/core/src/shared/open-service/toolset-definition.test-d.ts (1)

39-60: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assignability checks won't catch an any regression.

If inference degraded to any for handler input or format data, these assignments would still compile and toBeFunction() would still pass. Exact-type assertions on the parameters would actually pin the contract, e.g.:

♻️ Suggested assertion style
-    const greet: (input: { name: string }, context: ToolsetCtx) => Promise<{ greeting: string }> =
-      exampleToolset.methods.greet.handler;
+    expectTypeOf(exampleToolset.methods.greet.handler)
+      .parameter(0)
+      .toEqualTypeOf<{ name: string }>();
+    expectTypeOf(exampleToolset.methods.greet.handler).returns.resolves.toEqualTypeOf<{
+      greeting: string;
+    }>();
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/shared/open-service/toolset-definition.test-d.ts` around lines
39 - 60, Strengthen the type assertions in the handler and format tests around
exampleToolset.methods.greet.handler and reviewToolset.methods.create.handler,
plus their format counterparts, so parameter types are checked exactly rather
than only through assignability. Use exact-type assertions for the input/data
parameters to detect any regression to any, while retaining the existing return
and context contract checks.
code/core/src/shared/open-service/toolsets/docs/access-parity.test.ts (1)

155-173: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Parity assertions can pass vacuously.

If both modes regressed to empty (or identical error) text, every toBe here still passes. Anchoring one side to expected content keeps the comparison meaningful.

💚 Suggested guard
   it.each([false, true])('list with withStoryIds=%s', async (withStoryIds) => {
-    expect(await renderList(serviceToolset(), withStoryIds)).toBe(
-      await renderList(manifestToolset(), withStoryIds)
-    );
+    const serviceText = await renderList(serviceToolset(), withStoryIds);
+    expect(serviceText).toContain('button');
+    expect(serviceText).toBe(await renderList(manifestToolset(), withStoryIds));
   });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/shared/open-service/toolsets/docs/access-parity.test.ts` around
lines 155 - 173, Update the parity tests for renderList, renderShow, and
showStory so each assertion also verifies non-empty, expected output
independently of the other toolset. Retain the
serviceToolset-versus-manifestToolset equality checks, but anchor at least one
side of each comparison to meaningful content to prevent both implementations
returning identical empty or error text from passing vacuously.
code/core/src/shared/open-service/index.ts (1)

24-29: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Keep clearToolsetRegistry out of storybook/open-service This is a test-only reset helper; export it from ./toolset-registry.ts instead of widening the shared API surface.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/shared/open-service/index.ts` around lines 24 - 29, Remove
clearToolsetRegistry from the export list in the open-service index while
retaining the other toolset-registry exports, so the test-only reset helper is
not exposed through the shared storybook/open-service API.

Source: Coding guidelines

code/core/src/shared/open-service/toolsets/docs/access-service.ts (1)

146-160: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Use the batch story-docs query here. storyDocsForAllComponents already exists, so withStoryIds: true can load once and pick the requested ids instead of spawning one extraction per component.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/shared/open-service/toolsets/docs/access-service.ts` around
lines 146 - 160, Replace the per-ID Promise.all loading in the storiesById
construction with the existing storyDocsForAllComponents batch query when
withStoryIds is true, then select only the requested storyBasedIds from the
batch result. Preserve the empty-map behavior when withStoryIds is false and
keep the existing ID-to-payload mapping contract.
code/core/src/shared/open-service/toolsets/test/definition.ts (1)

193-210: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value

Telemetry is awaited while the run queue lock is still held.

reportRunTelemetry runs inside the try block before done(), so a slow/hanging telemetry call delays the next queued run even though the test run itself has finished. Consider reporting after releasing the lock (or not awaiting it).

♻️ Release the queue before reporting
         handler: async (input, ctx): Promise<TestRunData> => {
           const done = await queue.wait();
+          let data: TestRunData;
           try {
             const output = await runStoryTests({
               channel,
               getIndex: storyIndex.getIndex,
               stories: input.stories,
               a11y: input.a11y,
             });
-            const data: TestRunData = { ...output, a11y: input.a11y };
-
-            await reportRunTelemetry(data, input, ctx);
-
-            return data;
+            data = { ...output, a11y: input.a11y };
           } finally {
             done();
           }
+
+          await reportRunTelemetry(data, input, ctx);
+
+          return data;
         },
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/shared/open-service/toolsets/test/definition.ts` around lines
193 - 210, Update the handler around queue.wait and reportRunTelemetry so done()
releases the queue immediately after runStoryTests and TestRunData creation,
before telemetry reporting. Preserve returning the completed data and ensure the
lock is still released when the test run throws.
code/core/src/core-server/utils/manifests/manifests.ts (1)

126-134: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Cache the manifest loader here. list and resolve both call getManifests() again, so the docs path re-runs getManifestEntries() and experimental_manifests on every request. If this stays on the hot path, memoize the watched result instead of rebuilding it each time.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/core-server/utils/manifests/manifests.ts` around lines 126 -
134, Update loadManifests to memoize the watched manifest result per Presets
input, so repeated list and resolve requests reuse the same getManifests output
instead of rerunning getManifestEntries and experimental_manifests. Preserve the
existing watch behavior and ensure distinct preset configurations do not
incorrectly share cached data.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@code/addons/mcp/src/storybook-ai-metadata.test.ts`:
- Around line 44-47: Update the mock for ./utils/addon-vitest.ts to use Vitest’s
spy mode with { spy: true } instead of an async factory. Configure
vi.mocked(getAddonVitestConstants) in the test file’s beforeEach while
preserving the existing mocked behavior.

In `@code/core/src/core-server/presets/common-preset.ts`:
- Around line 398-405: Update the docs toolset registration around
createDocsToolset and createServiceDocsAccess so experimentalDocgenServer uses
service-backed docs only when core/docgen is registered, including ignorePreview
and missing-worker cases. Otherwise provide the established unavailable
docs-access path or retain manifest-backed access, ensuring registered docs
tools cannot invoke getService('core/docgen') when that service is absent.

In `@code/core/src/shared/open-service/toolsets/docs/definition.ts`:
- Around line 123-130: Update the matched-story path in the tool definition
around formatStoryDocumentation so a story whose documentation result is empty
falls back to a clear message indicating that the story has no snippet, instead
of returning blank text. Preserve the existing not-found handling and normal
formatted documentation for stories with content.
- Around line 35-38: Update describeList to remove the storybookId guidance from
its returned description, including the multi-Storybook scoping instruction,
while preserving the existing withStoryIds guidance and all other documentation
workflow text.

In `@code/core/src/shared/open-service/toolsets/review/definition.ts`:
- Around line 64-77: Update reviewCreateOutputSchema to include numeric
collectionCount and storyCount fields alongside reviewUrl, matching the
ReviewCreateOutput type and the handler/format() output so structuredContent
preserves both counters.

In `@code/core/src/shared/open-service/toolsets/stories/definition.ts`:
- Around line 237-239: Update the changed method definition in the stories
toolset to declare outputSchema using the existing changedOutputSchema, matching
preview and findByComponent. Keep the handler’s ChangedStoriesOutput return
behavior unchanged so stories.changed exposes schema-validated structured
content.

In `@code/core/src/shared/open-service/toolsets/stories/story-input.ts`:
- Around line 11-23: Update the storyInputProps.props description to reference a
capability backed by an MCP_TOOL_NAMES entry instead of the hardcoded
get-storybook-story-instructions tool name, preserving the guidance about
locating component props while keeping the description meaningful for non-MCP
consumers.

In `@code/core/src/shared/open-service/toolsets/test/format.ts`:
- Around line 56-92: Harden getA11yViolations by validating each violation’s
nodes value is an array before mapping it, returning an empty node list
otherwise. In countA11yViolations, validate each a11yReports value is an array
before iterating, and apply the same guard to the mirrored counting path, so
malformed payloads degrade without throwing.

In `@code/lib/mcp/src/dist-contract.test.ts`:
- Around line 10-42: Update the filesystem setup in the published package
contract tests to use memfs instead of direct node:fs access. Add the standard
memfs volume setup, reset vol in beforeEach, and seed package.json and
dist/index.js fixtures so existsSync, readFileSync, and statSync operate on
isolated test data while preserving the existing assertions.

---

Nitpick comments:
In `@code/addons/mcp/src/tools/toolset-tools.test.ts`:
- Around line 1-11: Update the module mocks in the toolset-tools test to use
Vitest’s spy mode with spy: true instead of factory functions, and move mock
behavior/configuration into a beforeEach block. Preserve the existing mocked
getService and collectTelemetry behavior while ensuring each test starts with
freshly configured mocks.

In `@code/core/src/core-server/utils/manifests/manifests.ts`:
- Around line 126-134: Update loadManifests to memoize the watched manifest
result per Presets input, so repeated list and resolve requests reuse the same
getManifests output instead of rerunning getManifestEntries and
experimental_manifests. Preserve the existing watch behavior and ensure distinct
preset configurations do not incorrectly share cached data.

In `@code/core/src/shared/open-service/index.ts`:
- Around line 24-29: Remove clearToolsetRegistry from the export list in the
open-service index while retaining the other toolset-registry exports, so the
test-only reset helper is not exposed through the shared storybook/open-service
API.

In `@code/core/src/shared/open-service/toolset-definition.test-d.ts`:
- Around line 39-60: Strengthen the type assertions in the handler and format
tests around exampleToolset.methods.greet.handler and
reviewToolset.methods.create.handler, plus their format counterparts, so
parameter types are checked exactly rather than only through assignability. Use
exact-type assertions for the input/data parameters to detect any regression to
any, while retaining the existing return and context contract checks.

In `@code/core/src/shared/open-service/toolset-definition.ts`:
- Around line 53-60: Update ToolsetMethod so the optional outputSchema is
generically tied to TOutput, constraining its schema type to one whose inferred
output matches the handler’s Awaited<TOutput>. Preserve outputSchema as optional
and keep the existing handler and format signatures unchanged.

In `@code/core/src/shared/open-service/toolsets/docs/access-parity.test.ts`:
- Around line 155-173: Update the parity tests for renderList, renderShow, and
showStory so each assertion also verifies non-empty, expected output
independently of the other toolset. Retain the
serviceToolset-versus-manifestToolset equality checks, but anchor at least one
side of each comparison to meaningful content to prevent both implementations
returning identical empty or error text from passing vacuously.

In `@code/core/src/shared/open-service/toolsets/docs/access-service.ts`:
- Around line 146-160: Replace the per-ID Promise.all loading in the storiesById
construction with the existing storyDocsForAllComponents batch query when
withStoryIds is true, then select only the requested storyBasedIds from the
batch result. Preserve the empty-map behavior when withStoryIds is false and
keep the existing ID-to-payload mapping contract.

In `@code/core/src/shared/open-service/toolsets/docs/definition.test.ts`:
- Around line 68-75: Update the test around “cross-references the show tool per
consumer in its description” to import and use the exported
resolveToolsetDescription helper from toolset-definition.ts for both mcpCtx and
cliCtx, removing the inline typeof/function resolution so the test follows
production semantics.

In `@code/core/src/shared/open-service/toolsets/test/definition.ts`:
- Around line 193-210: Update the handler around queue.wait and
reportRunTelemetry so done() releases the queue immediately after runStoryTests
and TestRunData creation, before telemetry reporting. Preserve returning the
completed data and ensure the lock is still released when the test run throws.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9f070aee-18b2-4346-b8cb-abea88bbc116

📥 Commits

Reviewing files that changed from the base of the PR and between 49205b2 and 25f5030.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (92)
  • AGENTS.md
  • code/addons/mcp/src/constants.ts
  • code/addons/mcp/src/mcp-handler.test.ts
  • code/addons/mcp/src/preset.test.ts
  • code/addons/mcp/src/preset.ts
  • code/addons/mcp/src/storybook-ai-metadata.test.ts
  • code/addons/mcp/src/test-support/register-core-toolsets.ts
  • code/addons/mcp/src/tools/display-review.test.ts
  • code/addons/mcp/src/tools/display-review.ts
  • code/addons/mcp/src/tools/get-changed-stories.test.ts
  • code/addons/mcp/src/tools/get-changed-stories.ts
  • code/addons/mcp/src/tools/get-stories-by-component.test.ts
  • code/addons/mcp/src/tools/get-stories-by-component.ts
  • code/addons/mcp/src/tools/get-storybook-story-instructions.test.ts
  • code/addons/mcp/src/tools/get-storybook-story-instructions.ts
  • code/addons/mcp/src/tools/preview-stories.test.ts
  • code/addons/mcp/src/tools/preview-stories.ts
  • code/addons/mcp/src/tools/preview-stories/preview-stories-app-script.ts
  • code/addons/mcp/src/tools/review-origin.ts
  • code/addons/mcp/src/tools/run-story-tests.test.ts
  • code/addons/mcp/src/tools/run-story-tests.ts
  • code/addons/mcp/src/tools/tool-names.ts
  • code/addons/mcp/src/tools/tool-registry.ts
  • code/addons/mcp/src/tools/toolset-tools.test.ts
  • code/addons/mcp/src/tools/toolset-tools.ts
  • code/addons/mcp/src/utils/addon-vitest.ts
  • code/addons/mcp/src/utils/build-args-param.test.ts
  • code/addons/mcp/src/utils/build-args-param.ts
  • code/addons/mcp/src/utils/detect-unreachable-changes.test.ts
  • code/addons/mcp/src/utils/detect-unreachable-changes.ts
  • code/addons/mcp/src/utils/find-story-ids.test.ts
  • code/addons/mcp/src/utils/find-story-ids.ts
  • code/addons/mcp/src/utils/get-tool-availability.ts
  • code/addons/vitest/src/preset.ts
  • code/core/build-config.ts
  • code/core/package.json
  • code/core/src/core-server/index.ts
  • code/core/src/core-server/presets/common-preset.ts
  • code/core/src/core-server/server-channel/review-channel.test.ts
  • code/core/src/core-server/server-channel/review-channel.ts
  • code/core/src/core-server/utils/manifests/manifests.ts
  • code/core/src/server-errors.ts
  • code/core/src/shared/open-service/index.ts
  • code/core/src/shared/open-service/toolset-definition.test-d.ts
  • code/core/src/shared/open-service/toolset-definition.ts
  • code/core/src/shared/open-service/toolset-names.ts
  • code/core/src/shared/open-service/toolset-registry.test.ts
  • code/core/src/shared/open-service/toolset-registry.ts
  • code/core/src/shared/open-service/toolset-types.ts
  • code/core/src/shared/open-service/toolsets/docs/access-manifest.test.ts
  • code/core/src/shared/open-service/toolsets/docs/access-manifest.ts
  • code/core/src/shared/open-service/toolsets/docs/access-parity.test.ts
  • code/core/src/shared/open-service/toolsets/docs/access-service.test.ts
  • code/core/src/shared/open-service/toolsets/docs/access-service.ts
  • code/core/src/shared/open-service/toolsets/docs/access.ts
  • code/core/src/shared/open-service/toolsets/docs/classify-services.test.ts
  • code/core/src/shared/open-service/toolsets/docs/classify-services.ts
  • code/core/src/shared/open-service/toolsets/docs/definition.test.ts
  • code/core/src/shared/open-service/toolsets/docs/definition.ts
  • code/core/src/shared/open-service/toolsets/docs/map.test.ts
  • code/core/src/shared/open-service/toolsets/docs/map.ts
  • code/core/src/shared/open-service/toolsets/docs/public.ts
  • code/core/src/shared/open-service/toolsets/docs/runtime-agnostic.test.ts
  • code/core/src/shared/open-service/toolsets/review/definition.test.ts
  • code/core/src/shared/open-service/toolsets/review/definition.ts
  • code/core/src/shared/open-service/toolsets/stories/definition.test.ts
  • code/core/src/shared/open-service/toolsets/stories/definition.ts
  • code/core/src/shared/open-service/toolsets/stories/find-by-component.test.ts
  • code/core/src/shared/open-service/toolsets/stories/find-by-component.ts
  • code/core/src/shared/open-service/toolsets/stories/format.ts
  • code/core/src/shared/open-service/toolsets/stories/resolve-component-stories.test.ts
  • code/core/src/shared/open-service/toolsets/stories/resolve-component-stories.ts
  • code/core/src/shared/open-service/toolsets/stories/story-input.ts
  • code/core/src/shared/open-service/toolsets/stories/unreachable-files.ts
  • code/core/src/shared/open-service/toolsets/test/definition.test.ts
  • code/core/src/shared/open-service/toolsets/test/definition.ts
  • code/core/src/shared/open-service/toolsets/test/format.ts
  • code/core/src/shared/open-service/toolsets/test/run.test.ts
  • code/core/src/shared/open-service/toolsets/test/run.ts
  • code/core/src/shared/review/events.ts
  • code/core/src/shared/review/review-state.ts
  • code/lib/mcp/package.json
  • code/lib/mcp/src/dist-contract.test.ts
  • code/lib/mcp/src/tools/get-documentation-for-story.ts
  • code/lib/mcp/src/tools/get-documentation.ts
  • code/lib/mcp/src/utils/adapt-core-manifest.ts
  • code/lib/mcp/src/utils/dedent.ts
  • code/lib/mcp/src/utils/manifest-formatter/extract-docs-summary.test.ts
  • code/lib/mcp/src/utils/manifest-formatter/extract-docs-summary.ts
  • code/lib/mcp/src/utils/manifest-formatter/markdown.test.ts
  • code/lib/mcp/src/utils/manifest-formatter/markdown.ts
  • code/lib/mcp/src/utils/parse-react-docgen.ts
💤 Files with no reviewable changes (24)
  • code/addons/mcp/src/tools/display-review.ts
  • code/core/src/shared/open-service/toolsets/docs/map.test.ts
  • code/core/src/core-server/server-channel/review-channel.ts
  • code/addons/mcp/src/tools/preview-stories.test.ts
  • code/lib/mcp/src/utils/dedent.ts
  • code/core/src/shared/open-service/toolsets/docs/classify-services.test.ts
  • code/addons/mcp/src/tools/get-changed-stories.test.ts
  • code/addons/mcp/src/utils/build-args-param.test.ts
  • code/addons/mcp/src/utils/detect-unreachable-changes.test.ts
  • code/core/src/shared/open-service/toolsets/docs/classify-services.ts
  • code/addons/mcp/src/utils/find-story-ids.ts
  • code/addons/mcp/src/utils/build-args-param.ts
  • code/addons/mcp/src/tools/run-story-tests.test.ts
  • code/addons/mcp/src/utils/find-story-ids.test.ts
  • code/lib/mcp/src/utils/manifest-formatter/extract-docs-summary.ts
  • code/addons/mcp/src/tools/display-review.test.ts
  • code/addons/mcp/src/tools/get-changed-stories.ts
  • code/addons/mcp/src/utils/detect-unreachable-changes.ts
  • code/addons/mcp/src/tools/run-story-tests.ts
  • code/addons/mcp/src/tools/get-stories-by-component.ts
  • code/addons/mcp/src/constants.ts
  • code/core/src/shared/open-service/toolsets/docs/map.ts
  • code/addons/mcp/src/tools/get-stories-by-component.test.ts
  • code/core/src/core-server/server-channel/review-channel.test.ts

Comment thread code/addons/mcp/src/storybook-ai-metadata.test.ts Outdated
Comment thread code/core/src/core-server/presets/common-preset.ts Outdated
Comment thread code/core/src/shared/open-service/toolsets/docs/definition.ts
Comment thread code/core/src/shared/open-service/toolsets/docs/definition.ts Outdated
Comment thread code/core/src/shared/open-service/toolsets/review/definition.ts
Comment thread code/core/src/shared/open-service/toolsets/stories/definition.ts
Comment thread code/core/src/shared/open-service/toolsets/stories/story-input.ts
Comment thread code/core/src/shared/open-service/toolsets/test/format.ts
Comment thread code/lib/mcp/src/dist-contract.test.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
code/core/src/core-server/presets/common-preset.ts (1)

320-325: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Remove the empty review-channel branch.

After removing initReviewChannel(channel), this if contains only stale comments describing a listener and teardown that no longer exist. Delete the conditional so the server-channel code reflects its actual behavior.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/core/src/core-server/presets/common-preset.ts` around lines 320 - 325,
Remove the empty isReviewFeatureEnabled conditional and its stale teardown
comments from the server-channel initialization flow. Keep the surrounding
server-channel setup and other init*Channel calls unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@code/core/src/core-server/presets/common-preset.ts`:
- Around line 320-325: Remove the empty isReviewFeatureEnabled conditional and
its stale teardown comments from the server-channel initialization flow. Keep
the surrounding server-channel setup and other init*Channel calls unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b0ea7bff-8708-4c32-aafe-e441fe9cb02a

📥 Commits

Reviewing files that changed from the base of the PR and between 25f5030 and b1e148f.

📒 Files selected for processing (4)
  • code/addons/mcp/src/storybook-ai-metadata.test.ts
  • code/core/src/core-server/presets/common-preset.ts
  • code/core/src/shared/review/features.test.ts
  • code/core/src/shared/review/features.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@code/core/src/shared/open-service/README.md`:
- Around line 108-110: Complete the outputSchema documentation sentence in the
open-service README by stating that structuredContent is narrowed to the
declared outputSchema. Leave the surrounding handler documentation unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 83221765-b166-4f5f-afc0-62613efa4964

📥 Commits

Reviewing files that changed from the base of the PR and between b1e148f and efbc750.

📒 Files selected for processing (12)
  • AGENTS.md
  • code/addons/mcp/src/constants.ts
  • code/addons/vitest/src/preset.test.ts
  • code/addons/vitest/src/preset.ts
  • code/core/src/core-server/presets/common-preset.ts
  • code/core/src/shared/open-service/README.md
  • code/core/src/shared/open-service/toolsets/docs/classify-services.ts
  • code/core/src/shared/open-service/toolsets/docs/manifest-formatter/adapt-core-manifest.ts
  • code/core/src/shared/open-service/toolsets/docs/manifest-formatter/parse-react-docgen.test.ts
  • code/core/src/shared/open-service/toolsets/review/definition.ts
  • code/core/src/shared/open-service/toolsets/stories/story-input.ts
  • code/lib/mcp/src/utils/parse-react-docgen.ts
💤 Files with no reviewable changes (2)
  • code/addons/mcp/src/constants.ts
  • code/lib/mcp/src/utils/parse-react-docgen.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • code/core/src/shared/open-service/toolsets/stories/story-input.ts
  • AGENTS.md

Comment thread code/core/src/shared/open-service/README.md Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
code/addons/mcp/src/instructions/build-server-instructions.ts (1)

92-100: 📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Drop the CI run-ID from the comment.

Line 96 embeds a run ID (run 28673251562)) inside the maintenance comment. As per coding guidelines, comments should explain rationale, not carry provenance/ticket-style references.

✏️ Proposed fix
   // Validation Workflow from the latest release (plus the run-story-tests-only
-  // rule added after agents substituted `npm run test:stories`, run
-  // 28673251562), and the shared docs-toolset Documentation Workflow. With
+  // rule added after agents substituted `npm run test:stories`), and the
+  // shared docs-toolset Documentation Workflow. With
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/addons/mcp/src/instructions/build-server-instructions.ts` around lines
92 - 100, Remove the CI run ID “28673251562” from the maintenance comment in the
test/docs section, while preserving the surrounding explanation of the legacy
Validation Workflow and run-story-tests-only rule.

Source: Coding guidelines

🧹 Nitpick comments (2)
code/lib/mcp/src/tools/get-documentation-for-story.test.ts (2)

398-398: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Stale test name. getManifests no longer exists in this flow; rename to reflect the manifest provider (e.g. "should pass remote source to the manifest provider"). Same wording appears in code/lib/mcp/src/tools/get-documentation.test.ts.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/lib/mcp/src/tools/get-documentation-for-story.test.ts` at line 398,
Rename the test case around the remote-source assertion in
get-documentation-for-story.test.ts to refer to the manifest provider instead of
getManifests, and apply the same wording update to the corresponding test in
get-documentation.test.ts.

25-45: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extract the duplicated createManifestProvider test helper. The identical ServedManifests type and provider factory are copied into three test files; they will drift as the manifest paths/errors evolve.

  • code/lib/mcp/src/tools/get-documentation-for-story.test.ts#L25-L45: move the type and factory into a shared test util (e.g. code/lib/mcp/src/tools/test-manifest-provider.ts) and import it here.
  • code/lib/mcp/src/tools/get-documentation.test.ts#L26-L46: delete the local copy and import the shared helper.
  • code/lib/mcp/src/tools/list-all-documentation.test.ts#L31-L51: delete the local copy and import the shared helper.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@code/lib/mcp/src/tools/get-documentation-for-story.test.ts` around lines 25 -
45, Extract the shared ServedManifests type and createManifestProvider factory
into a test utility, then import and use it in
code/lib/mcp/src/tools/get-documentation-for-story.test.ts#L25-L45,
code/lib/mcp/src/tools/get-documentation.test.ts#L26-L46, and
code/lib/mcp/src/tools/list-all-documentation.test.ts#L31-L51; remove each
duplicated local definition while preserving the existing manifest paths and
error behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@code/core/src/shared/open-service/toolsets/docs/access-provider.ts`:
- Around line 223-232: Update the JSON-pointer traversal loop around the target
lookup to use Object.hasOwn for key membership instead of the in operator, so
inherited properties such as constructor and __proto__ are rejected as missing
while own properties continue resolving normally.

---

Outside diff comments:
In `@code/addons/mcp/src/instructions/build-server-instructions.ts`:
- Around line 92-100: Remove the CI run ID “28673251562” from the maintenance
comment in the test/docs section, while preserving the surrounding explanation
of the legacy Validation Workflow and run-story-tests-only rule.

---

Nitpick comments:
In `@code/lib/mcp/src/tools/get-documentation-for-story.test.ts`:
- Line 398: Rename the test case around the remote-source assertion in
get-documentation-for-story.test.ts to refer to the manifest provider instead of
getManifests, and apply the same wording update to the corresponding test in
get-documentation.test.ts.
- Around line 25-45: Extract the shared ServedManifests type and
createManifestProvider factory into a test utility, then import and use it in
code/lib/mcp/src/tools/get-documentation-for-story.test.ts#L25-L45,
code/lib/mcp/src/tools/get-documentation.test.ts#L26-L46, and
code/lib/mcp/src/tools/list-all-documentation.test.ts#L31-L51; remove each
duplicated local definition while preserving the existing manifest paths and
error behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4de2b4e3-601f-4c8f-a810-9fa6d936b90b

📥 Commits

Reviewing files that changed from the base of the PR and between 875c3aa and 3f1ff5c.

⛔ Files ignored due to path filters (2)
  • code/lib/mcp/src/utils/manifest-formatter/__snapshots__/markdown.test.ts.snap is excluded by !**/*.snap
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (49)
  • AGENTS.md
  • agent-eval/lib/templates.test.ts
  • agent-eval/lib/templates.ts
  • code/addons/mcp/package.json
  • code/addons/mcp/src/auth/composition-auth.ts
  • code/addons/mcp/src/auth/resolve-composition-sources.ts
  • code/addons/mcp/src/instructions/build-server-instructions.ts
  • code/addons/mcp/src/manifests/in-process-provider.ts
  • code/addons/mcp/src/mcp-handler.ts
  • code/addons/mcp/src/telemetry.test.ts
  • code/addons/mcp/src/telemetry.ts
  • code/addons/mcp/src/tools/tool-registry.ts
  • code/addons/mcp/src/tools/toolset-tools.ts
  • code/addons/mcp/src/types.ts
  • code/core/src/shared/open-service/toolsets/docs/access-manifest.ts
  • code/core/src/shared/open-service/toolsets/docs/access-parity.test.ts
  • code/core/src/shared/open-service/toolsets/docs/access-provider.test.ts
  • code/core/src/shared/open-service/toolsets/docs/access-provider.ts
  • code/core/src/shared/open-service/toolsets/docs/definition.test.ts
  • code/core/src/shared/open-service/toolsets/docs/definition.ts
  • code/core/src/shared/open-service/toolsets/docs/instructions.ts
  • code/core/src/shared/open-service/toolsets/docs/manifest-formatter/adapt-core-manifest.test.ts
  • code/core/src/shared/open-service/toolsets/docs/manifest-formatter/manifest-types.ts
  • code/core/src/shared/open-service/toolsets/docs/manifest-formatter/markdown.test.ts
  • code/core/src/shared/open-service/toolsets/docs/manifest-formatter/markdown.ts
  • code/core/src/shared/open-service/toolsets/docs/multi-source.ts
  • code/core/src/shared/open-service/toolsets/docs/public.ts
  • code/core/src/shared/open-service/toolsets/docs/sources.test.ts
  • code/core/src/shared/open-service/toolsets/docs/sources.ts
  • code/lib/mcp/bin.ts
  • code/lib/mcp/src/dist-contract.test.ts
  • code/lib/mcp/src/index.ts
  • code/lib/mcp/src/instructions.md
  • code/lib/mcp/src/tools/get-documentation-for-story.test.ts
  • code/lib/mcp/src/tools/get-documentation-for-story.ts
  • code/lib/mcp/src/tools/get-documentation.test.ts
  • code/lib/mcp/src/tools/get-documentation.ts
  • code/lib/mcp/src/tools/list-all-documentation.test.ts
  • code/lib/mcp/src/tools/list-all-documentation.ts
  • code/lib/mcp/src/tools/register.ts
  • code/lib/mcp/src/utils/adapt-core-manifest.ts
  • code/lib/mcp/src/utils/error-to-mcp-content.test.ts
  • code/lib/mcp/src/utils/error-to-mcp-content.ts
  • code/lib/mcp/src/utils/get-manifest.ts
  • code/lib/mcp/src/utils/manifest-formatter/extract-docs-summary.test.ts
  • code/lib/mcp/src/utils/manifest-formatter/markdown.test.ts
  • code/lib/mcp/src/utils/manifest-formatter/markdown.ts
  • code/lib/mcp/src/utils/map-with-concurrency.ts
  • code/lib/mcp/src/utils/requires-own-mcp.ts
💤 Files with no reviewable changes (8)
  • code/lib/mcp/src/instructions.md
  • code/addons/mcp/package.json
  • code/lib/mcp/src/tools/list-all-documentation.ts
  • code/lib/mcp/src/tools/get-documentation.ts
  • code/lib/mcp/src/utils/get-manifest.ts
  • code/lib/mcp/src/utils/manifest-formatter/extract-docs-summary.test.ts
  • code/lib/mcp/src/tools/get-documentation-for-story.ts
  • code/lib/mcp/src/utils/adapt-core-manifest.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • AGENTS.md
  • code/core/src/shared/open-service/toolsets/docs/access-manifest.ts

Comment thread code/core/src/shared/open-service/toolsets/docs/access-provider.ts

@kasperpeulen kasperpeulen left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Review Codex:

Reviewed current head cc487c61188b5f20a25907e3fa3a44249e1ebcb2. The inline findings below are the issues I believe still need attention.

Comment thread code/core/src/shared/open-service/toolsets/docs/access-provider.ts Outdated
Comment thread code/core/src/shared/open-service/toolsets/docs/definition.ts Outdated
Comment thread code/core/src/shared/open-service/toolsets/test/format.ts
Comment thread code/lib/mcp/src/utils/parse-react-docgen.ts
Comment thread code/core/src/shared/open-service/toolsets/docs/classify-services.ts Outdated
Comment thread code/lib/mcp/src/index.ts
Comment thread code/addons/mcp/src/tools/tool-registry.ts Outdated
Comment thread code/addons/mcp/src/preset.ts Outdated
Comment thread code/core/src/shared/open-service/toolsets/docs/access-provider.ts Outdated

@kasperpeulen kasperpeulen left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Review Codex:

Second-pass review on head 146a52e1a46a99ab41b1bfc7c79758962e8c1778. The new inline findings below are additional to the earlier review threads.

Comment thread code/addons/mcp/src/tools/tool-registry.ts
Comment thread code/core/src/core-server/presets/common-preset.ts
Comment thread scripts/build/utils/generate-types-rolldown.ts Outdated
Comment thread code/addons/mcp/src/tools/tool-registry.ts Outdated
Comment thread test-storybooks/mcp/.storybook-no-vitest/main.ts
Comment thread code/lib/mcp/src/tools/register.ts Outdated
Comment thread code/core/src/shared/open-service/toolsets/stories/story-input.ts
@kasperpeulen

Copy link
Copy Markdown
Member Author

🤖 Review Codex:

[P1] Refresh downstream canary QA from the final fix-round head

#35688 user story 17 requires a fresh canary containing all fixes, followed by re-running the chromaui/mcp-server#66 suite and production build. npm currently exposes only @storybook/mcp@0.0.0-pr-35677-sha-6395aeab, and chromaui/mcp-server#66 still pins and reports QA against that version. PR head is now 146a52e1, with the substantive c6d33bf98d fix round and 146a52e1a4 cleanup after 6395aeab, so the downstream proof has not exercised the final state. Publish a new canary, update the exact pin and lockfile, rerun the 150-test suite plus yarn build:prod, and update both PR descriptions with that evidence.

@kasperpeulen kasperpeulen left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Review Codex:

Re-review of the latest fix delta at cf401786a4; one new inline correctness finding follows.

Comment thread code/addons/mcp/src/tools/tool-registry.ts Outdated
@kasperpeulen

Copy link
Copy Markdown
Member Author

🤖 Review Codex:

[P1] Refresh downstream proof from the new final head

The previous canary follow-up was satisfied for 146a52e1, but the PR head then moved to cf401786. npm still exposes only the …146a52e1 and …6395aeab PR canaries, while chromaui/mcp-server#66 pins …146a52e1 and its reported 150/150 tests plus production build all cover that older artifact. This is material because cf401786 changes the H1 missing-toolset containment behavior and generate-types-rolldown.ts, the declaration-build path used when compiling @storybook/mcp. #35688 US17 requires a fresh canary after the fixes land green and downstream proof of the final state. Once current-head CI is green, publish a cf401786 canary, update #66’s exact pin and lockfile, rerun install/tests/yarn build:prod, and update both PR descriptions.

[P3] Keep the documented ToolsetMethod contract complete

The PR description still says the type grows “from three fields to five” and lists only description, schema, outputSchema, handler, and format. At this head the public type has a sixth optional field, reportUsage, which runs after formatting and is central to the shared telemetry design. Please add it to the construct and adjust the field count/telemetry explanation so the review artifact matches the public API being introduced.

@kasperpeulen

Copy link
Copy Markdown
Member Author

Re the two [P1] downstream-refresh comments: done from the final head. Canary 0.0.0-pr-35677-sha-5d4368ba (46 packages, run 30636872156) is verified on npm; chromaui/mcp-server#66 is re-pinned to it with yarn test 150/150 and yarn build:prod exit 0, and both PR descriptions now name this version. The intermediate …sha-146a52e1 canary is superseded; the earlier …sha-cf401786 publish run was cancelled before it produced anything.

🤖 Addressed by Claude Code

@kasperpeulen kasperpeulen left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Review Codex:

Re-review of the final follow-up at 5d4368ba; the functional fix and final-head canary verification pass. One new Standards comment follows.

Comment thread code/addons/mcp/src/tools/tool-registry.test.ts Outdated
@kasperpeulen

Copy link
Copy Markdown
Member Author

🤖 Review Codex:

[P3] Correct the final-head SHA in the downstream QA record

The substantive #35688 US17 proof now checks out: npm reports gitHead=5d4368ba589a3d9fc6d28b7b35010ca0cf343ab8, chromaui/mcp-server#66 pins that exact version in both package.json and its lockfile, and its 150 tests plus build:prod pass. However, #66’s historical QA paragraph still identifies …sha-146a52e1 as “the final fix-round head after review follow-ups.” That contradicts the exact …sha-5d4368ba pin immediately above it and leaves the required final-state record internally inconsistent. Please describe 146a52e1 as the intermediate post-fix canary and 5d4368ba as the final follow-up canary.

The earlier [P3] on this PR’s “three fields to five” construct remains unresolved and is already recorded in the prior Codex comment, so I have not duplicated it here.

@kasperpeulen kasperpeulen left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Review Codex: Standards re-review of af29f015. I found two new actionable standards issues; the inline comments contain the concrete fixes.

Comment thread code/core/src/shared/open-service/README.md Outdated
Comment thread code/core/src/shared/open-service/toolsets/docs/definition.ts

@kasperpeulen kasperpeulen left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Review Codex: Spec re-review of af29f015 against #35695, #35673, and #35688. Four new spec findings were confirmed; three are inline and the release-proof blocker is in the PR conversation.

Comment thread code/core/src/shared/open-service/README.md Outdated
Comment thread code/lib/mcp/src/tools/register.ts Outdated
@kasperpeulen

Copy link
Copy Markdown
Member Author

🤖 Review Codex: [P1] Publish and prove a canary from the final head before merging

The final head is af29f015, but npm has no 0.0.0-pr-35677-sha-af29f015 releases for storybook, @storybook/mcp, or @storybook/addon-mcp; the downstream validation PR chromaui/mcp-server#66 still pins 0.0.0-pr-35677-sha-5d4368ba. The PR description itself marks that proof as superseded and says a fresh final-head canary follows. Until an af29f015 canary is published, #66 is repinned, and its 150 tests plus production build are rerun, the release-critical consumer proof required by #35695 does not cover the code being merged.

@kasperpeulen

Copy link
Copy Markdown
Member Author

Re the [P1] final-head canary: done. 0.0.0-pr-35677-sha-480f17b8 is published from the current head 480f17b88b (workflow run 30658250581); storybook, @storybook/mcp, and @storybook/addon-mcp are verified on npm with gitHead = 480f17b88b57c5e1648e065d5118ef38d78dbb50. chromaui/mcp-server#66 is re-pinned to it (exact pin + lockfile, commit 631809f): yarn install clean, yarn test 150/150, yarn build:prod exit 0, and the strict-consumer tsc repro against the published tarball (skipLibCheck: false, no @types/react) reports zero diagnostics from this package's declarations (its shipped d.ts imports only valibot/tmcp). Both PR descriptions now record this canary as current, with the earlier …sha-5d4368ba proof marked superseded.

🤖 Addressed by Claude Code

@kasperpeulen

Copy link
Copy Markdown
Member Author

Real-project QA on canary 0.0.0-pr-35677-sha-480f17b8 — 16/16 wire-level checks passed on a fresh Vite React TS app (npx storybook@<canary> init --yes; every Storybook dep resolved to the exact canary; experimentalReview on; git-initialized with a tracked Button.tsx edit):

  • tools/list serves all 9 tools, each called through its published input schema.
  • Docs: list-all-documentation → get-documentation (found: props + stories; not-found: isError + list-tool nudge) → get-documentation-for-story (found + missing-story with the available-stories list).
  • Stories: preview-stories returns the UI URL in content and structuredContent; get-stories-by-component maps Button.tsx → example-button--*; get-changed-stories reports the tracked edit through the live module graph (with the coverage-gap note for the non-story main.ts edit).
  • Tests: run-story-tests runs a real filtered Vitest run for one story id and passes; an a11y: true run also passes.
  • Review: display-review publishes (reviewUrl in structuredContent; prose names the running UI root exactly: "already running at http://localhost:6017") and refuses fabricated story ids with the recovery coaching as isError.
  • Manager UI serves (HTTP 200, 11 index entries).

One QA-step correction folded into the description: the stock dev server answers MCP only at the root /mcp (a prefixed path 404s — verified empirically), so the sub-path uiRoot fact engages via embedding middleware; it is pinned at the adapter seam by toolset-tools.test.ts and at the prose by the review definition test, and the root-hosted prose is covered live above. The hosted-package side is covered by chromaui/mcp-server#66's real-handler integration suite (150/150 on this canary).

🤖 Addressed by Claude Code

@kasperpeulen

Copy link
Copy Markdown
Member Author

Full local eval matrix on canary 0.0.0-pr-35677-sha-480f17b8 (all six experiments, full 8xx line — the ci:eval + ci:extra-evals + ci:extra-models set, run locally in Vercel sandboxes):

Experiment Result
cc-mcp-opus-high 12/12
cc-plugin-opus-high 12/12 (+4/4 lifecycle)
codex-mcp-gpt-5.5-medium 12/12
cc-mcp-sonnet-medium 11/12
cc-plugin-sonnet-medium 11/12
codex-plugin-gpt-5.5-medium 11/12 (+4/4 lifecycle)

81/84 (96%) overall; every failure classifies as model, zero infra, zero product failures. The three misses are agent-behavior gaps on the extra-model line: Sonnet-medium twice on 811-fix-a11y-violations (partial fix, no verifying re-run) and codex-plugin once on 807-docs-request (answered correctly but read source instead of calling the docs tools — the same behavior class as 807's documented known-failure on the Claude MCP path). The flagship experiments (Opus high, both integrations, and Codex MCP) are 100%.

Harness note, for whoever runs evals on cross-cutting core+addon PRs: with core pinned to npm next, every MCP eval infra-fails by construction — pre-M4 core lacks storybook/internal/toolsets-docs, and even pinning the root to the canary leaves npm nesting a second pre-M4 core inside the file:-linked local addon to satisfy its workspace-derived ^10.6.0-alpha.3 dev/peer ranges (overrides cannot reach file: subtrees). The run above used a local, uncommitted harness tweak (EVAL_STORYBOOK_DIST_TAG=canary, plus rewriting the local packages' storybook specs to the pinned tag) — worth upstreaming separately; the skew disappears on its own once this PR merges and a next prerelease ships.

🤖 Addressed by Claude Code

@JReinhold JReinhold left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One important thing I don't fully get here, is whether or not addon-mcp now includes MCP tools for any toolsets registered, or if it only has a pre-defined list of tools still. We want the former, eventually.

Comment thread code/addons/mcp/src/tools/tool-registry.ts
@JReinhold
JReinhold force-pushed the m4-addon-mcp-core-services branch from 1da8a9f to 0a5ad88 Compare August 11, 2026 11:34
@JReinhold
JReinhold merged commit 7c209cf into next Aug 12, 2026
144 of 149 checks passed
@JReinhold
JReinhold deleted the m4-addon-mcp-core-services branch August 12, 2026 11:16
@JReinhold
JReinhold restored the m4-addon-mcp-core-services branch August 12, 2026 11:18
@JReinhold JReinhold added qa:skip Pull Requests that do not need any QA. (e.g. documentation) and removed qa:needed Pull Requests that will need manual QA prior to release. labels Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:normal Run our default set of CI jobs (choose this for most PRs). maintenance User-facing maintenance tasks qa:skip Pull Requests that do not need any QA. (e.g. documentation)

Projects

None yet

2 participants