Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,19 @@ v2/main/
│ # caller's teardown reaches it on every throw path —
│ # the readiness timeout included, which is what used
│ # to orphan a live server holding its port — #2000;
│ # ensure-test-servers.mjs is the ONE place
│ # `test-servers/build` is produced for a script
│ # consumer, and it builds UNCONDITIONALLY (once per
│ # process per repo root): the five hand-rolled copies
│ # it replaced each returned early when the output was
Comment thread
cliffhall marked this conversation as resolved.
│ # already on disk, so after the first run an edit to
│ # `test-servers/src` was never picked up and the smoke
│ # silently drove the previous fixture — reported not as
│ # staleness but as a product failure in whatever the
│ # smoke was checking. Unconditional emit is still not a
│ # clean: a DELETED src file leaves its stale `.js`
│ # behind, so `rm -rf test-servers/build` after one —
│ # #2111;
│ # resolve-node-bin.mjs resolves a
│ # package's bin through its own package.json, so the
│ # verify/smoke scripts spawn it with `process.execPath`
Expand All @@ -264,7 +277,9 @@ v2/main/
│ # the exception — its children are `npm` itself and the
│ # installed `.bin` shim, neither resolvable that way, so
│ # it keeps a Windows shell and quotes its generated
│ # paths via win-shell-args.mjs — #1939). Prettier-gated via
│ # paths via win-shell-args.mjs — #1939, though its
│ # test-server build now goes through the shared helper
│ # like everyone else's). Prettier-gated via
│ # `format:check:scripts`; its own pure parsers are
│ # unit-tested by `npm run test:scripts` (node --test).
├── specification/ # Build specification
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -398,7 +398,7 @@ Each client self-validates from its own folder; the root scripts chain them. The
| `npm run verify:build-gate` | Runs a real `vite build` with a Node built-in forced into the browser graph and asserts the build **fails** via the #1769 gate (which turns Vite's browser-externalization warning into a hard error). Guards against the warning phrasing drifting in a Vite bump and silently disabling the gate. Part of `npm run ci`. |
| `npm run verify:bundle-externals` | Guards the must-not-bundle invariant (#2067): for each tsup-bundled client it reads the **built** `build/` output and fails if any package that must stay external was inlined anyway. Candidates are the union of the client's own `external` array **and the root manifest's `dependencies`** — the latter because #2067 was a *missing* `external` entry, which a self-referential check would have passed. Detection is via esbuild's `// <path>` module banners, so it covers both shapes — a separate `<pkg>-HASH.js` chunk (what a dynamically `import()`ed CommonJS package produces) and a statically-imported package folded straight into `index.js`, which emits no chunk at all. A build with no banners fails as such rather than passing clean, so enabling `minify` cannot silently retire the check. `undici` was declared only in the root and `clients/cli` manifests, and tsup auto-externalizes only what the _nearest_ manifest declares, so the web and TUI bundles inlined 1.05MB of it — and CommonJS inlined into an ESM bundle throws `Dynamic require of "assert" is not supported` on first use, from a specifier no user-side install can satisfy. Reads the output rather than the config because those two disagreed for four releases. Part of `npm run ci`. |
| `npm run verify:format-coverage` | Parses the `format:check` globs out of every `package.json` (only those reachable from `validate`), enumerates all tracked source files, and **fails** listing any not covered by a glob — the durable guard for the "every first-party source file is format-gated" invariant (#1792). Runs first in `validate`. |
| `npm run test:scripts` | Table-driven unit tests (`node --test`) for the guard's own pure parsers (`scripts/lib/npm-scripts.mjs`, `scripts/lib/tsc-program.mjs` + the exported helpers of `verify-typecheck-coverage.mjs` and `verify-dep-lockstep.mjs`), one case per rule they encode, plus two suites over shared `scripts/lib` helpers that no smoke can check itself: `resolve-node-bin.test.mjs` — the cross-platform bin resolver (#1939), pinned against the real `bin`/`exports` shapes of the packages the scripts actually spawn — and `announced-child.test.mjs` — the spawn/readiness ownership helper (#2000), which drives real `node -e` children to prove a child that never announces is still published to the caller before the timeout throws, and so is reachable by teardown rather than orphaned. A third, `mcp-app-flow.test.mjs`, covers the shared MCP Apps flow (#2003) — the deep link's two CSRF gates and `appArgs` encoding, plus `driveAppFlow`'s failure branches against a stand-in page, all of which are dead code from the happy-path smokes' point of view and would otherwise surface only as opaque timeouts. Runs in `validate` — and `verify:typecheck-coverage` guards *this* gate in turn (reachable from `validate`, non-empty test set, every test file matched by the `test:scripts` glob), since `node --test` silently skips a file its glob misses and still exits 0. |
| `npm run test:scripts` | Table-driven unit tests (`node --test`) for the guard's own pure parsers (`scripts/lib/npm-scripts.mjs`, `scripts/lib/tsc-program.mjs` + the exported helpers of `verify-typecheck-coverage.mjs` and `verify-dep-lockstep.mjs`), one case per rule they encode, plus two suites over shared `scripts/lib` helpers that no smoke can check itself: `resolve-node-bin.test.mjs` — the cross-platform bin resolver (#1939), pinned against the real `bin`/`exports` shapes of the packages the scripts actually spawn — and `announced-child.test.mjs` — the spawn/readiness ownership helper (#2000), which drives real `node -e` children to prove a child that never announces is still published to the caller before the timeout throws, and so is reachable by teardown rather than orphaned. Two more do the same: `mcp-app-flow.test.mjs` covers the shared MCP Apps flow (#2003) — the deep link's two CSRF gates and `appArgs` encoding, plus `driveAppFlow`'s failure branches against a stand-in page, all of which are dead code from the happy-path smokes' point of view and would otherwise surface only as opaque timeouts; and `ensure-test-servers.test.mjs` pins the [#2111](https://github.com/modelcontextprotocol/inspector/issues/2111) invariant — that `test-servers/build` is rebuilt **even when it already exists** — which no smoke can assert about itself, since one driving a stale fixture reports a product failure rather than a staleness one. Runs in `validate` — and `verify:typecheck-coverage` guards *this* gate in turn (reachable from `validate`, non-empty test set, every test file matched by the `test:scripts` glob), since `node --test` silently skips a file its glob misses and still exits 0. |
| `npm run verify:typecheck-coverage` | The typecheck-coverage analog of the above (#1791): for each Node client (auto-discovered from disk — enrolled via its `typecheck` script's projects, or for a `tsc -b` client like `clients/web` via its `tsconfig.json` `references`) it runs those projects with `tsc --listFilesOnly`, unions them, and **fails** listing any tracked `.ts`/`.tsx`/`.mts`/`.cts` under the client that lands in no project (so a new top-level config/helper can't silently go untypechecked). It also requires, deny-by-default, the first-party TS no client owns (`test-servers/src`, the root `vitest.shared.mts`, all of `core/`, and any new top-level location) to land in some client project's tsc pass — so a `core` `*.tsx` web's projects don't reach is caught too. Also asserts the gate is wired (each client's typecheck pass — its `typecheck` script, or web's `tsc -b` — is reachable from its `validate`, and the root chain runs each client's `validate`). Runs in `validate`. |
| `npm run verify:dep-lockstep` | Guards the "one version per install-crossing dependency" invariant (#1896). v2 is not a workspace, so a client's test project compiles the shared first-party TypeScript — `core/`, `test-servers/src`, and the root-owned `vitest.shared.mts`, all of which resolve their dependencies from the **root** install — alongside the client's own sources, putting the same package in one `tsc` program twice. At the same version that's harmless; skewed, TypeScript must relate two structurally-distinct copies of every type, which for a recursive-generic surface is exponential (zod `4.3.6` vs `4.4.3` exhausted the 4GB tsc heap in `clients/web`). Derives its candidate set from **what actually enters each program** (#1965) — every client tsconfig project listed with `tsc --listFilesOnly` via the shared `scripts/lib/tsc-program.mjs`, each resolved `node_modules` file mapped to its owning install, keeping the packages that reach one program from two installs (a package whose declarations arrive only through another package's `.d.ts`, as `@modelcontextprotocol/sdk`'s do, is invisible to a scan of first-party imports). Prices each copy from the lockfile entry for the exact install path the program resolved, compares only the installs that met in one program, and **fails deny-by-default** on any disagreement not in the annotated `TOLERATED_SKEW` allowlist — empty today — with an allowlisted package tolerated only *within a major version*. Runs in `validate`.
| `npm run ci` | **Mandatory pre-push command.** `validate` → `coverage` → `verify:build-gate` → `verify:bundle-externals` → `smoke` → Storybook. A true superset of GitHub CI. |
Expand Down
160 changes: 160 additions & 0 deletions scripts/lib/ensure-test-servers.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
/**
* The one place `test-servers/build` is produced for a script consumer (#2111).
*
* Five scripts used to carry their own `ensureTestServer()`, each returning
* early when the output was already on disk. Presence is not freshness: once
* `test-servers/build` exists — which it does on any machine that has run the
* smokes once — an edit to `test-servers/src` was never picked up again, and
* the smoke kept driving the previously-built fixture.
*
* That failure is silent *and* misattributed. It does not read as "your fixture
* is stale", it reads as a product bug: a `v2/main` merge that touched
* `preset-registry.ts` made `smoke:web:app` report the dedicated app-origin
* path as broken, and two unrelated hypotheses were measured out before the
* real cause surfaced. The **tests** have always built the same fixture
* unconditionally (`pretest`, `test:coverage`, `test:integration` all run
* `test-servers:build`), so the weaker policy sat on exactly the checks that
* spawn real servers and are hardest to debug when wrong.
*
* So: build every time, from here. `tsc -p test-servers --noCheck` costs ~1s
* cold and less when `incremental` finds its `.tsbuildinfo` up to date, against
* smokes that take tens of seconds and a `pack:verify` that does a real
* `npm install` from the registry.
*
* ⚠️ Unconditional emit is not a clean: `--noCheck` emit leaves the stale `.js`
* of a *deleted* `src` file behind, and no path here removes it. That is
* unchanged from before (the old early-return only skipped the build; it never
* cleaned either), and it is a loud failure rather than a silent one — an
* import of a removed module fails at the consumer. Fixing it properly means
* `tsc -b` with `composite`, which forces `declaration` on and drops
* `--noCheck`; not worth it for this. `rm -rf test-servers/build` if you have
* deleted a source file.
*
* The root-installed tsc is run via this Node — `npx` is a `.cmd` shim on
* Windows that a shell-free spawnSync can't start (ENOENT — #1939). That is
* also one of the drifts consolidating fixed: `pack-and-verify.mjs` still
* shelled out to `npx tsc`, so it alone was unrunnable on Windows.
*/

import { spawnSync } from "node:child_process";
import { existsSync } from "node:fs";
import { join } from "node:path";
import { resolveNodeBin } from "./resolve-node-bin.mjs";

/**
* The build entries a consumer can require, by short name. Every one of them is
* emitted by the same single `tsc -p test-servers` pass — the names exist so a
* caller can say which files its own run depends on, and get told precisely
* which is missing when the build silently produces nothing.
*/
export const TEST_SERVER_ENTRIES = {
stdio: "test-server-stdio.js",
http: "test-server-http.js",
composable: "server-composable.js",
fixtures: "test-server-fixtures.js",
};

/** Absolute path of a named `test-servers/build` entry. */
export function testServerEntryPath(repoRoot, name) {
const file = TEST_SERVER_ENTRIES[name];
if (!file) {
throw new Error(
`unknown test-server entry "${name}" (known: ${Object.keys(
TEST_SERVER_ENTRIES,
).join(", ")})`,
);
}
return join(repoRoot, "test-servers", "build", file);
}

/**
* The actionable message a failed build reports. Kept as its own function
* because it is the half a consumer sees when things go wrong, and every
* previous copy worded it differently — including naming the wrong client to
* run `npm run test-servers:build` from.
*/
export function buildFailureMessage(missing) {
const detail = missing.length
? ` (missing ${missing.join(", ")})`
: " (tsc exited non-zero)";
return (
`could not build the test servers${detail}. ` +
"Run `npm run test-servers:build` from clients/web."
);
}

/** Repo roots already built in this process — see `ensureTestServers`. */
const built = new Set();

/**
* Build `test-servers/build`, unconditionally, and assert the entries the
* caller named exist afterwards.
*
* Unconditional per *process*, not per call: `pack-and-verify.mjs` asks twice
* (once for the stdio entry, once for the composable one) and one `tsc` pass
* emits both, so a second call within the same run skips the build. A fresh
* process always rebuilds, which is the whole point — that is what makes an
* edit to `test-servers/src` reach the next smoke. The *existence* assertion is
* not cached, because a later call can name an entry the first one didn't.
*
* `log` receives the bare message; the default prefixes it with `label`, which
* is what a caller owning its own prefixed logger (pack-and-verify's `step`)
* passes instead. `run` and `exists` are injectable so the failure branches —
* the ones no smoke's happy path can reach — are testable without spawning
* tsc.
*
* @returns {string[]} absolute paths of the required entries, in order.
*/
export function ensureTestServers({
repoRoot,
label,
requires = [],
log = (message) => console.log(`${label} — ${message}`),
run = runTsc,
exists = existsSync,
}) {
const paths = requires.map((name) => testServerEntryPath(repoRoot, name));

// A cached call can name an entry the first one didn't, so the existence
// assertion runs on every call. Skipping it on the cache hit would return a
// path to a file tsc never emitted, and the consumer would then fail with
// exactly the opaque ERR_MODULE_NOT_FOUND this helper exists to replace.
if (built.has(repoRoot)) {
assertEmitted(paths, exists, 0);
return paths;
}

log("building test-servers...");
const status = run(repoRoot);
assertEmitted(paths, exists, status);
built.add(repoRoot);
return paths;
}

/** Throw the shared actionable message unless tsc succeeded and emitted each path. */
function assertEmitted(paths, exists, status) {
const missing = paths.filter((p) => !exists(p));
if (status !== 0 || missing.length) {
throw new Error(buildFailureMessage(missing));
}
}

/** Test-only: forget which roots this process has built. */
export function resetTestServerBuildCache() {
built.clear();
}

/** Run the root-installed tsc over `test-servers`; returns its exit status. */
function runTsc(repoRoot) {
const r = spawnSync(
process.execPath,
[
resolveNodeBin("typescript", "tsc", repoRoot),
"-p",
"test-servers",
"--noCheck",
],
{ cwd: repoRoot, stdio: "inherit" },
);
return r.status;
}
Loading