diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 7beba3ef5..661106e9c 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -53,6 +53,12 @@ entry. See `CONTRIBUTING.md` § Releases & changelog. host-global single-flight. New env vars documented in `middleware/.env.example`: `CLI_TOOLS_DIR`, `CODEX_HOME`. +### Fixed — handoff plans now stay inside the package after symlinks and fail closed on declared dry runs (#470 C15) + +- `middleware/src/platform/pluginHandoffPlan.ts` now re-checks `permissions.sql.handoff` containment after resolving real paths for BOTH the package root and the target, closing the two escapes PR #815 left open: a file symlink inside the package pointing outside, and a directory symlink inside the package whose child path points outside. Missing targets still refuse as `unreadable`, not as an escape, so the operator still hears "the package does not ship this file" for the case they can actually fix. +- The same loader now refuses `"dryRun": true` in a kernel-run handoff plan. Preview mode belongs to `middleware/scripts/plugin-ledger-handoff.mjs --dry-run`; if core honoured a plan-level dry run it would write nothing, then immediately let its own migration runner apply every file, silently recreating the exact G7 failure C15 exists to remove. +- Regression locks now cover the two fail-closed properties the feature lives or dies on: a witness that fails at the database aborts activation before the migration runner can run, and a manifest whose SQL grant no longer matches its declared ledger is treated as ungranted so neither the handoff nor the runner can reach the database. + ### Fixed — core-decoupling zero floor no longer hides same-named files (#470 C13 review) - `scripts/check-core-decoupling.mjs` now excludes only the exact detector path `scripts/check-core-decoupling.mjs` instead of any basename match, closing the hole where a same-named file dropped under `middleware/src/` could hide Dev Platform identifiers from the permanent zero floor. A colocated regression test proves the detector stays self-excluded while a probe file at `middleware/src/__probe/check-core-decoupling.mjs` is counted. diff --git a/middleware/packages/plugin-api/CHANGELOG.md b/middleware/packages/plugin-api/CHANGELOG.md index 39515eda2..f94c86538 100644 --- a/middleware/packages/plugin-api/CHANGELOG.md +++ b/middleware/packages/plugin-api/CHANGELOG.md @@ -8,6 +8,52 @@ Versioning is SemVer over the **exported type surface**. Removing or narrowing an exported type, or adding a required member to an interface a plugin implements, is a major. +## 1.6.0 — 2026-08-21 + +Additive. `permissions.sql` gains an optional `handoff` — a path, inside the +package, to the JSON plan the kernel runs BEFORE it runs the plugin's +migrations directory (epic #470, C15). + +### Added + +- **`SqlPermission.handoff?: string`** — names a plan of the same shape + {@link SqlAccessor.seedLedger} accepts: + `{ "entries": [{ "filename", "witnessSql" }], "dryRun"?: false }`. The kernel + reads it at activation, validates it against the package root, and performs + the handoff through the same seeder — read-only witness fence, advisory lock + and entry validation included — before its own migration runner. A shared + file MAY carry `"dryRun": false` for the operator CLI's benefit; the kernel + refuses `"dryRun": true`, because a preview that writes nothing would hand + every file straight to the migration runner below. Use the CLI's + `--dry-run` / `--apply` flags to preview or apply. + +### Why a declaration and not the call C11 already shipped + +`seedLedger` was documented as "call this BEFORE `runMigrations`", and a plugin +cannot honour that. The kernel runs the migrations directory ITSELF, before +`activate()`, so that "the tables exist" is an invariant `activate()` can rely +on rather than a race each plugin re-loses in its own way. The plugin's own +call therefore always arrived second, after every ledger row was already +written. + +The 2026-08-21 acceptance run of the first extracted plugin measured the +consequence on the exact upgrade C11 exists for: `0 seeded, 9 already seeded`, +with `skippedNoWitness` — the one alarm the feature was built to raise — +unreachable. Nothing failed, and that is the problem: the line is +indistinguishable from a healthy re-run. + +The witnesses are knowledge only the plugin has; the ordering is a decision +only the kernel can make. So the plugin declares and the kernel executes. + +### Compatibility + +Every existing consumer keeps compiling: `handoff` is optional, and a plugin +that omits it sees the pre-1.6.0 behaviour exactly. `SqlAccessor.seedLedger` +is unchanged and stays the right call for a plugin that manages its own +ordering or must work against a kernel older than this one — against a kernel +that honours `handoff`, that call simply reports `alreadySeeded`, which is what +it should report once the work is done. + ## 1.5.0 — 2026-08-21 > **Why 1.4.0 and not 1.3.0.** This change was written against a tree where diff --git a/middleware/packages/plugin-api/api-snapshot/plugin-api.d.ts.snap b/middleware/packages/plugin-api/api-snapshot/plugin-api.d.ts.snap index 00540120d..fa9156181 100644 --- a/middleware/packages/plugin-api/api-snapshot/plugin-api.d.ts.snap +++ b/middleware/packages/plugin-api/api-snapshot/plugin-api.d.ts.snap @@ -1792,6 +1792,7 @@ constructor(agentId: string, fromVersion: string, toVersion: string, cause: unkn export interface SqlPermission { readonly migrations?: string; readonly ledger: string; +readonly handoff?: string; } export interface MigrationReport { readonly applied: readonly string[]; diff --git a/middleware/packages/plugin-api/package.json b/middleware/packages/plugin-api/package.json index 2e7193632..b4c8f0c84 100644 --- a/middleware/packages/plugin-api/package.json +++ b/middleware/packages/plugin-api/package.json @@ -1,6 +1,6 @@ { "name": "@omadia/plugin-api", - "version": "1.5.0", + "version": "1.6.0", "private": true, "type": "module", "main": "dist/index.js", diff --git a/middleware/packages/plugin-api/src/pluginContext.ts b/middleware/packages/plugin-api/src/pluginContext.ts index 7ec1f6a25..da6a99491 100644 --- a/middleware/packages/plugin-api/src/pluginContext.ts +++ b/middleware/packages/plugin-api/src/pluginContext.ts @@ -1927,6 +1927,7 @@ export class MigrationHookError extends Error { * sql: * migrations: migrations # optional; directory inside the package * ledger: omadia_verifier_migrations + * handoff: handoff-plan.json # optional; run before `migrations` * ``` */ export interface SqlPermission { @@ -1938,6 +1939,37 @@ export interface SqlPermission { * Must match `^[a-z][a-z0-9_]{2,62}$` AND begin with the plugin's sanitized * id, so a manifest cannot nominate another plugin's ledger. */ readonly ledger: string; + /** + * Path (relative to the package root) to a JSON ledger-handoff plan the + * kernel runs BEFORE {@link SqlPermission.migrations} — epic #470 C15. + * + * ```json + * { + * "entries": [ + * { "filename": "0001_x.js", "witnessSql": "SELECT to_regclass('public.x') IS NOT NULL" } + * ], + * "dryRun": false + * } + * ``` + * + * Same shape {@link SqlAccessor.seedLedger} accepts, and the same shape the + * operator CLI (`middleware/scripts/plugin-ledger-handoff.mjs --plan`) + * reads, so one file serves all three readers. A shared file MAY carry + * `"dryRun": false`; `"dryRun": true` is refused on the kernel-run path. + * Preview mode belongs to the CLI flag, not to plugin data: if core read a + * plan that asked it to "write nothing", then core's own migration runner + * would immediately apply every file underneath it, silently recreating the + * exact "0 seeded, 9 already seeded" failure C15 exists to remove. + * + * DECLARE THIS RATHER THAN CALLING `seedLedger` YOURSELF when the manifest + * also declares `migrations`. The kernel runs that directory before your + * `activate()`, so a `seedLedger` call inside `activate()` arrives after + * every ledger row is already written and can only ever report + * `alreadySeeded` — the `skippedNoWitness` alarm never fires. Keeping the + * in-`activate` call as well is safe and is the right fallback for older + * kernels, where it does the work instead. + */ + readonly handoff?: string; } /** What one `runMigrations` pass did. Returned rather than logged so a plugin diff --git a/middleware/scripts/plugin-ledger-handoff.mjs b/middleware/scripts/plugin-ledger-handoff.mjs index 5212e267d..bd1a77c06 100644 --- a/middleware/scripts/plugin-ledger-handoff.mjs +++ b/middleware/scripts/plugin-ledger-handoff.mjs @@ -48,6 +48,39 @@ * `migrationsDir` is resolved relative to the plan file, so a plan shipped * inside a plugin package works from wherever the operator copied it. * + * ONE PLAN, THREE READERS (epic #470 C15) + * --------------------------------------- + * The same JSON is read by three things: + * + * 1. this CLI, via `--plan`; + * 2. the kernel, when a manifest declares `permissions.sql.handoff` + * (`middleware/src/platform/pluginHandoffPlan.ts`); + * 3. a plugin that manages its own ordering, via `ctx.sql.seedLedger`. + * + * `entries` (and the optional `dryRun`) are what all three consume, and they + * are exactly `SeedLedgerOptions`. `pluginId`, `ledger` and `migrationsDir` + * are for THIS tool alone: it runs with no manifest, so it has to be told + * them. The kernel knows all three authoritatively and deliberately ignores + * the file's copies — a plan that could redirect the write would undo the + * grant matching the manifest — though it does WARN when the plan's `ledger` + * disagrees with the manifest's, because then the table an operator previewed + * here is not the table the kernel is about to write. + * + * So a plan shipped inside a package for the manifest carries only `entries`, + * and this tool reads that same file when the three missing fields are + * supplied as flags: + * + * node middleware/scripts/plugin-ledger-handoff.mjs \ + * --plan node_modules/@vendor/thing/handoff-plan.json \ + * --plugin-id @vendor/thing \ + * --ledger plg_vendor_thing_migrations \ + * --migrations-dir migrations + * + * The kernel's reader is STRICTER than this one: it rejects unknown keys + * (notably `dir`, which `SeedLedgerOptions` accepts and the kernel refuses to + * honour) and caps the file size. A plan that the kernel accepts always works + * here; the reverse is not guaranteed. + * * Exit codes: 0 = plan computed (or applied), 1 = the handoff refused, * 2 = usage / plan-file error. * @@ -71,6 +104,14 @@ Usage: node middleware/scripts/plugin-ledger-handoff.mjs --plan [opt --apply Actually write the ledger rows. Default is a dry run. --database-url Overrides $DATABASE_URL. --json Machine-readable output. + + For a plan shipped inside a package for 'permissions.sql.handoff', which + carries only 'entries', supply the three fields the manifest would have + told the kernel: + + --plugin-id e.g. @vendor/thing + --ledger e.g. plg_vendor_thing_migrations + --migrations-dir resolved relative to the plan file `; function parseArgs(argv) { @@ -82,6 +123,9 @@ function parseArgs(argv) { else if (arg === '--dry-run') args.apply = false; else if (arg === '--plan') args.plan = argv[(i += 1)]; else if (arg === '--database-url') args.databaseUrl = argv[(i += 1)]; + else if (arg === '--plugin-id') args.pluginId = argv[(i += 1)]; + else if (arg === '--ledger') args.ledger = argv[(i += 1)]; + else if (arg === '--migrations-dir') args.migrationsDir = argv[(i += 1)]; else if (arg === '--help' || arg === '-h') args.help = true; else fail(`unknown argument '${arg}'`); } @@ -93,7 +137,7 @@ function fail(msg) { process.exit(2); } -function loadPlan(planPath) { +function loadPlan(planPath, args) { let raw; try { raw = readFileSync(planPath, 'utf8'); @@ -106,9 +150,28 @@ function loadPlan(planPath) { } catch (err) { fail(`plan '${planPath}' is not valid JSON: ${err.message}`); } + // Epic #470 C15 — the same file may also be the one a manifest names in + // `permissions.sql.handoff`, and that reader knows the plugin, the ledger + // and the directory authoritatively, so a package-shipped plan carries only + // `entries` (and optionally `dryRun`). This tool has no manifest, so it + // still needs all three — but they may now come from flags instead of from + // the file. That is what lets ONE plan serve both readers: forcing a plugin + // to ship two files would let the one an operator previews drift from the + // one the kernel runs. + if (typeof args.pluginId === 'string' && args.pluginId.length > 0) { + plan.pluginId = args.pluginId; + } + if (typeof args.ledger === 'string' && args.ledger.length > 0) { + plan.ledger = args.ledger; + } + if (typeof args.migrationsDir === 'string' && args.migrationsDir.length > 0) { + plan.migrationsDir = args.migrationsDir; + } for (const field of ['pluginId', 'ledger', 'migrationsDir']) { if (typeof plan[field] !== 'string' || plan[field].length === 0) { - fail(`plan is missing a non-empty '${field}'`); + fail( + `plan is missing a non-empty '${field}' — add it to the plan file, or pass --${field.replace(/[A-Z]/g, (c) => `-${c.toLowerCase()}`)}`, + ); } } if (!Array.isArray(plan.entries) || plan.entries.length === 0) { @@ -181,7 +244,7 @@ async function main() { fail('set DATABASE_URL or pass --database-url'); } - const plan = loadPlan(args.plan); + const plan = loadPlan(args.plan, args); const pool = new pg.Pool({ connectionString, max: 2 }); try { const result = await seedPluginLedgerFromDonor({ diff --git a/middleware/src/platform/pluginHandoffPlan.ts b/middleware/src/platform/pluginHandoffPlan.ts new file mode 100644 index 000000000..703d358ab --- /dev/null +++ b/middleware/src/platform/pluginHandoffPlan.ts @@ -0,0 +1,302 @@ +import { promises as fs } from 'node:fs'; +import path from 'node:path'; + +import { z } from 'zod'; + +/** + * Epic #470 C15 — the manifest-declared migration handoff plan. + * + * WHY THIS IS DECLARATIVE AND NOT A PLUGIN CALL + * --------------------------------------------- + * C11 gave a plugin `ctx.sql.seedLedger` and documented it as "call this + * before `runMigrations`". A plugin cannot honour that. Core runs the plugin's + * migrations ITSELF, before `activate()` (C7 / G4 — so that "the tables exist" + * is an invariant `activate()` can rely on rather than a race each plugin + * re-loses in its own way), and there is no ordering a plugin can choose that + * puts its own call ahead of a call core makes before handing it control. The + * 2026-08-21 acceptance run measured the consequence: `0 seeded, 9 already + * seeded` on the exact upgrade C11 exists for, with `skippedNoWitness` + * unreachable. + * + * The witnesses are knowledge only the plugin has; the ordering is a decision + * only core can make. So the plugin DECLARES and core EXECUTES: the manifest + * names a JSON file, core reads it and runs the same seeder ahead of its own + * runner. + * + * WHY IT IS VALIDATED THIS HARD + * ----------------------------- + * The file is plugin-supplied data that reaches core's filesystem and then + * core's database, on the boot path, before the plugin has run a line of its + * own code. Everything it can get wrong is caught HERE, at the boundary, + * where the refusal can still name the plugin and the path — rather than + * later, inside a transaction, as a constraint violation nobody can trace back + * to a manifest line. + */ + +/** Largest plan file core will read. A nine-file handoff is about 1 KB; this + * is three orders of magnitude of headroom and still a bound, so a package + * cannot make the boot path read an arbitrarily large file. */ +export const MAX_HANDOFF_PLAN_BYTES = 128 * 1024; + +/** + * Why a plan was refused. + * + * Typed rather than message-only because an activation failure reaches an + * operator through the circuit-breaker, and "the plan is invalid" tells them + * nothing they can act on. These five say which mistake was made. + */ +export type HandoffPlanRefusal = + | 'escapes-package-root' + | 'unreadable' + | 'too-large' + | 'not-json' + | 'malformed' + | 'dry-run-declared'; + +/** Thrown when a manifest declares a handoff plan core will not run. */ +export class PluginHandoffPlanError extends Error { + public readonly pluginId: string; + public readonly declaredPath: string; + public readonly reason: HandoffPlanRefusal; + + constructor( + pluginId: string, + declaredPath: string, + reason: HandoffPlanRefusal, + detail: string, + ) { + super( + `plugin '${pluginId}': permissions.sql.handoff '${declaredPath}' — ${detail}`, + ); + this.name = 'PluginHandoffPlanError'; + this.pluginId = pluginId; + this.declaredPath = declaredPath; + this.reason = reason; + } +} + +export interface HandoffPlanEntry { + readonly filename: string; + readonly witnessSql: string; +} + +export interface HandoffPlan { + readonly entries: readonly HandoffPlanEntry[]; + /** + * The ledger the plan file names, when it names one. + * + * ADVISORY. Core resolves the ledger from the manifest and the operator's + * grant, and never from plugin data — a plan that could redirect the write + * would undo the whole point of the grant matching the manifest. It is + * surfaced only so core can WARN when the two disagree, which is a real + * split-brain: the operator previews one table with the CLI and core writes + * another. + */ + readonly declaredLedger?: string; +} + +/** + * The plan file's shape. + * + * `entries` are core's, and `dryRun` is accepted ONLY so one file can serve + * both readers without drift. The operator CLI owns preview mode and derives + * it from `--apply` / `--dry-run`; the kernel-run path below refuses + * `"dryRun": true` because a preview that writes nothing would hand every + * file straight to the migration runner one line later, silently recreating + * the exact G7 failure C15 exists to remove. + * + * The rest of the shape is deliberately the one `SqlAccessor.seedLedger` + * accepts, so a plugin that manages its own ordering against an older core + * can pass the same parsed object straight to `ctx.sql.seedLedger`. + * + * `pluginId` / `ledger` / `migrationsDir` belong to the operator CLI + * (`middleware/scripts/plugin-ledger-handoff.mjs`), which runs with no + * manifest and therefore has to be told them. They are accepted and ignored so + * that ONE file can serve both readers: forcing a plugin to ship two would let + * the file an operator previews drift from the file core runs, which is + * precisely the class of surprise this feature exists to remove. + * + * Strict, not permissive. The key that makes it matter is `dir`: + * `SeedLedgerOptions` accepts it, so a plugin author could reasonably expect + * it to work here, and silently ignoring it would leave them believing a + * directory override took effect. Core takes the migrations directory from the + * manifest and nowhere else — a second path-containment surface buys nothing. + */ +const planSchema = z.strictObject({ + entries: z + .array( + z.strictObject({ + filename: z.string().min(1), + witnessSql: z + .string() + .refine((s) => s.trim().length > 0, 'must not be blank'), + }), + ) + .min(1, 'entries must list at least one file — a handoff that claims nothing is a mistake, not a no-op'), + dryRun: z.boolean().optional(), + // Operator-CLI fields. Reported or ignored, never obeyed. See above. + pluginId: z.string().optional(), + ledger: z.string().optional(), + migrationsDir: z.string().optional(), +}); + +export interface LoadHandoffPlanOptions { + /** Kernel-known plugin id, for the error messages. */ + readonly pluginId: string; + /** Absolute path to the installed package. */ + readonly packageRoot: string; + /** `permissions.sql.handoff`, verbatim from the manifest. */ + readonly declaredPath: string; +} + +/** + * Read and validate a declared handoff plan. + * + * Throws {@link PluginHandoffPlanError} for every rejection. Throwing rather + * than degrading is the right direction here and the opposite of the manifest + * loader's rule elsewhere: dropping a malformed `public_paths` entry leaves + * one fewer unauthenticated surface, but dropping a malformed handoff leaves + * core's pre-activate runner to write the very ledger rows the plan existed to + * decide on. A silent skip would reproduce G7 exactly. + */ +export async function loadHandoffPlan( + opts: LoadHandoffPlanOptions, +): Promise { + const { pluginId, declaredPath } = opts; + const refuse = ( + reason: HandoffPlanRefusal, + detail: string, + ): PluginHandoffPlanError => + new PluginHandoffPlanError(pluginId, declaredPath, reason, detail); + + // Containment first: nothing else may run against a path that is not inside + // the package. The `+ path.sep` is load-bearing — without it a sibling + // directory named `-evil` passes a bare `startsWith`. + const root = path.resolve(opts.packageRoot); + const resolved = path.resolve(root, declaredPath); + if (!resolved.startsWith(root + path.sep)) { + throw refuse( + 'escapes-package-root', + 'resolves outside the package root — a handoff plan must ship inside the package it describes', + ); + } + + // `path.resolve` closes `../` tricks and sibling-prefix lookalikes, but it + // is still lexical. A package can ship `handoff-plan.json -> /outside/file` + // or `plans -> /outside` and pass the string check while `stat()` follows + // the link. So after lexical containment we resolve BOTH real paths and + // assert containment again. The root must be realpathed too: on macOS `/var` + // is really `/private/var`, and realpathing only the target would make + // every tmpdir-based package look like an escape. + const realRoot = await fs.realpath(root).catch((err: unknown) => { + throw refuse( + 'unreadable', + `package root could not be resolved: ${err instanceof Error ? err.message : String(err)}`, + ); + }); + const realResolved = await fs.realpath(resolved).catch((err: unknown) => { + if (hasErrnoCode(err, 'ENOENT')) { + throw refuse( + 'unreadable', + 'is not a readable file — the manifest declares a handoff the package does not ship', + ); + } + throw refuse( + 'unreadable', + `could not be resolved: ${err instanceof Error ? err.message : String(err)}`, + ); + }); + if (!realResolved.startsWith(realRoot + path.sep)) { + throw refuse( + 'escapes-package-root', + 'resolves outside the package root after following symlinks — the resolution went through a link, so the package is pointing core at a file it does not ship', + ); + } + + const stat = await fs.stat(realResolved).catch(() => undefined); + if (!stat?.isFile()) { + throw refuse( + 'unreadable', + 'is not a readable file — the manifest declares a handoff the package does not ship', + ); + } + if (stat.size > MAX_HANDOFF_PLAN_BYTES) { + throw refuse( + 'too-large', + `is ${String(stat.size)} bytes, over the ${String(MAX_HANDOFF_PLAN_BYTES)}-byte cap`, + ); + } + + const raw = await fs.readFile(realResolved, 'utf8').catch((err: unknown) => { + throw refuse( + 'unreadable', + `could not be read: ${err instanceof Error ? err.message : String(err)}`, + ); + }); + + let parsed: unknown; + try { + parsed = JSON.parse(raw); + } catch (err) { + throw refuse( + 'not-json', + `is not valid JSON: ${err instanceof Error ? err.message : String(err)}`, + ); + } + + const result = planSchema.safeParse(parsed); + if (!result.success) { + throw refuse('malformed', describeIssues(result.error)); + } + + // Duplicates survive the schema — an array of two valid entries is a valid + // array. Two witnesses for one file makes the outcome depend on iteration + // order, and `Object.fromEntries` downstream would silently keep the last. + // `seedLedger` refuses this too; refusing it here means the operator hears + // about it at load time, with the filename in the message. + const seen = new Set(); + for (const entry of result.data.entries) { + if (seen.has(entry.filename)) { + throw refuse( + 'malformed', + `lists '${entry.filename}' twice — two witnesses for one file makes the outcome depend on iteration order`, + ); + } + seen.add(entry.filename); + } + if (result.data.dryRun === true) { + throw refuse( + 'dry-run-declared', + 'asks the kernel to preview and write nothing — that hands every file straight to the migration runner below, silently recreating the exact failure this handoff exists to remove; use middleware/scripts/plugin-ledger-handoff.mjs --dry-run to preview instead', + ); + } + + return { + entries: result.data.entries.map((e) => ({ + filename: e.filename, + witnessSql: e.witnessSql, + })), + ...(result.data.ledger === undefined + ? {} + : { declaredLedger: result.data.ledger }), + }; +} + +/** Flatten zod's issues into one line that names the offending keys. */ +function describeIssues(error: z.ZodError): string { + return error.issues + .map((issue) => { + const where = issue.path.length > 0 ? issue.path.join('.') : '(root)'; + return `${where}: ${issue.message}`; + }) + .join('; '); +} + +function hasErrnoCode(err: unknown, code: string): err is NodeJS.ErrnoException { + return ( + typeof err === 'object' && + err !== null && + 'code' in err && + (err as { code?: unknown }).code === code + ); +} diff --git a/middleware/src/platform/pluginSqlGrants.ts b/middleware/src/platform/pluginSqlGrants.ts index c20274758..0dec88863 100644 --- a/middleware/src/platform/pluginSqlGrants.ts +++ b/middleware/src/platform/pluginSqlGrants.ts @@ -293,7 +293,30 @@ export function parseSqlPermission( `[catalog] plugin '${pluginId}': permissions.sql.migrations must be a non-empty string — falling back to the default '${DEFAULT_MIGRATIONS_DIR}'`, ); } - return migrations === undefined ? { ledger } : { ledger, migrations }; + // Epic #470 C15 — the ledger-handoff plan the kernel runs BEFORE the + // migrations directory. Only the shape of the STRING is decided here; the + // file it names is read and validated at activation + // (`platform/pluginHandoffPlan.ts`), where the package root is known and a + // refusal can still fail the activation. + // + // Dropped-with-a-warning rather than rejected, matching `migrations` and + // this loader's rule everywhere else. The degradation is not silent + // downstream: `handoff` absent means the kernel runs no handoff, so the + // migrations simply apply — the pre-C15 behaviour, which is safe. + const handoffRaw = rec['handoff']; + let handoff: string | undefined; + if (typeof handoffRaw === 'string' && handoffRaw.length > 0) { + handoff = handoffRaw; + } else if (handoffRaw !== undefined) { + warn( + `[catalog] plugin '${pluginId}': permissions.sql.handoff must be a non-empty path inside the package — ignored, so the migration runner will apply every file`, + ); + } + return { + ledger, + ...(migrations === undefined ? {} : { migrations }), + ...(handoff === undefined ? {} : { handoff }), + }; } /** Directory a plugin's migrations live in when the manifest does not say. */ diff --git a/middleware/src/plugins/toolPluginRuntime.ts b/middleware/src/plugins/toolPluginRuntime.ts index fee7d98c4..2602e9861 100644 --- a/middleware/src/plugins/toolPluginRuntime.ts +++ b/middleware/src/plugins/toolPluginRuntime.ts @@ -3,6 +3,10 @@ import { pathToFileURL } from 'node:url'; import { promises as fs } from 'node:fs'; import { createPluginContext } from '../platform/pluginContext.js'; +import { + PluginHandoffPlanError, + loadHandoffPlan, +} from '../platform/pluginHandoffPlan.js'; import { eventEmitIds } from '../platform/eventCatalogRegistry.js'; import type { PluginRouteRegistry } from '../platform/pluginRouteRegistry.js'; import { PublicPathClaimError } from '../platform/publicPathGrants.js'; @@ -356,6 +360,89 @@ export class ToolPluginRuntime { logger: (...args) => console.log(`[${agentId}]`, ...args), }); + // Epic #470 C15 — run the declared ledger handoff BEFORE the migration + // runner below. + // + // C11 gave a plugin `ctx.sql.seedLedger` and documented it as "call this + // before `runMigrations`", which a plugin cannot honour: the runner below + // runs before `activate()`, so the plugin's own call always arrived after + // every ledger row was already written. The 2026-08-21 acceptance run + // measured `0 seeded, 9 already seeded` on the exact upgrade C11 exists + // for, with `skippedNoWitness` — the one alarm it was built to raise — + // unreachable. And nothing failed: that log line is indistinguishable + // from a healthy re-run. + // + // The witnesses are knowledge only the plugin has; the ordering is a + // decision only core can make. So the plugin declares the plan in its + // manifest and core executes it here, through the SAME accessor a plugin + // would have called — read-only witness fence, advisory lock, entry + // validation and all. `ctx.sql.seedLedger` stays for plugins that manage + // their own order; against this core it degrades to `alreadySeeded`, + // which is what it should report once the work is done. + // + // A refusal fails the activation, and deliberately takes the migration + // runner with it: running the files anyway would write exactly the ledger + // rows the unreadable plan existed to decide on. + if (declaredSql?.handoff && ctx.sql) { + const plan = await loadHandoffPlan({ + pluginId: agentId, + packageRoot: packagePath, + declaredPath: declaredSql.handoff, + }); + if (plan.declaredLedger && plan.declaredLedger !== declaredSql.ledger) { + // Not fatal — core's ledger is the granted one and the plan's copy is + // advisory. But an operator who previewed the handoff with + // `plugin-ledger-handoff.mjs` read a different table than the one + // about to be written, and that is worth saying out loud. + log( + `[tool-runtime] WARN ${agentId}: handoff plan names ledger '${plan.declaredLedger}' but the manifest declares '${declaredSql.ledger}' — the manifest wins; an operator dry-run against the plan's table showed a different database`, + ); + } + const seedLedger = ctx.sql.seedLedger; + if (!seedLedger) { + throw new PluginHandoffPlanError( + agentId, + declaredSql.handoff, + 'malformed', + 'this kernel builds a SQL accessor with no ledger seeder — the handoff cannot run, and running the migrations instead would silently do the thing the plan exists to prevent', + ); + } + // Bounded with the same budget as the runner it precedes, and for the + // same reason: `seedPluginLedgerFromDonor` sets its timeouts + // server-side, but neither covers a connection that never answers, and + // this await sits on the boot path. The two steps also share one + // advisory lock, so a tighter bound here would only move where the pair + // gives up. + const report = await withTimeout( + seedLedger.call(ctx.sql, { + entries: plan.entries, + }), + MIGRATION_TIMEOUT_MS, + `seedLedger(${agentId}) timed out after ${String(MIGRATION_TIMEOUT_MS / 1000)}s`, + ); + log( + `[tool-runtime] ${agentId}: ledger handoff — ${String(report.seeded.length)} seeded, ` + + `${String(report.alreadySeeded.length)} already seeded, ` + + `${String(report.applied.length)} left for the migration runner ` + + `(ledger '${report.ledger}', donor '${report.donorLedger}', ${String(report.durationMs)}ms)`, + ); + if (report.skippedNoWitness.length > 0) { + // THE output this feature exists to produce. The donor ledger records + // these, but their witness says the schema object is not there: a + // restore from an older snapshot, a rolled-back deploy, a table + // dropped during an incident. The runner below repairs it, so this is + // a warning rather than a refusal — but the operator has to be told + // the database is not the one they think it is, and told WHICH files, + // because a count alone gives them nowhere to look. + log( + `[tool-runtime] WARN ${agentId}: ledger handoff — the donor ledger records ` + + `${String(report.skippedNoWitness.length)} file(s) whose witness says the schema object is ABSENT; ` + + `the migration runner will apply them, which is the repair — confirm this is the database you think it is: ` + + report.skippedNoWitness.join(', '), + ); + } + } + // Epic #470 C7 / G4 — apply the plugin's schema BEFORE its `activate()` // runs. A plugin whose first act is to query its own tables must not have // to remember to migrate first, and the ordering is not a convenience: it diff --git a/middleware/test/pluginHandoffPlan.test.ts b/middleware/test/pluginHandoffPlan.test.ts new file mode 100644 index 000000000..09fd1a7a6 --- /dev/null +++ b/middleware/test/pluginHandoffPlan.test.ts @@ -0,0 +1,299 @@ +/** + * Epic #470 C15 — the declarative half of the migration handoff. + * + * `permissions.sql.handoff` names a JSON file INSIDE the package, and core + * reads it on the boot path, before the plugin has run a single line of its + * own code. That makes the file plugin-supplied data reaching core's + * filesystem and then core's database, which is exactly the shape that has to + * be validated at the boundary rather than trusted and repaired later. + * + * So this suite is about refusal, not about the happy path: the happy path is + * one assertion, and everything else here is a way the file can be wrong. + * + * The refusals are TYPED (`PluginHandoffPlanError.reason`) because activation + * failures are read by operators through the circuit-breaker's message. "plan + * is invalid" tells an operator nothing they can act on; "escapes the package + * root" and "is not valid JSON" tell them which of two very different mistakes + * they made. + */ + +import { strict as assert } from 'node:assert'; +import { mkdtemp, mkdir, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { basename, join } from 'node:path'; +import { afterEach, beforeEach, describe, it } from 'node:test'; + +import { + MAX_HANDOFF_PLAN_BYTES, + PluginHandoffPlanError, + loadHandoffPlan, +} from '../src/platform/pluginHandoffPlan.js'; + +const PLUGIN_ID = '@test/handoff-plan'; + +describe('#470 C15 loadHandoffPlan', () => { + let root: string; + let outsideRoots: string[]; + + beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'c15-plan-')); + outsideRoots = []; + }); + + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + await Promise.all( + outsideRoots.map((dir) => rm(dir, { recursive: true, force: true })), + ); + }); + + async function writePlan(relative: string, body: string): Promise { + const abs = join(root, relative); + await mkdir(join(abs, '..'), { recursive: true }); + await writeFile(abs, body, 'utf8'); + } + + async function makeOutsideRoot(): Promise { + const dir = await mkdtemp(join(tmpdir(), 'c15-plan-outside-')); + outsideRoots.push(dir); + return dir; + } + + function load(declaredPath: string): Promise { + return loadHandoffPlan({ + pluginId: PLUGIN_ID, + packageRoot: root, + declaredPath, + }); + } + + /** Assert a refusal and its reason in one place, so a test that stops + * throwing cannot pass by throwing something else. */ + async function refusal( + declaredPath: string, + reason: PluginHandoffPlanError['reason'], + ): Promise { + const err = await load(declaredPath).then( + () => undefined, + (e: unknown) => e, + ); + assert.ok( + err instanceof PluginHandoffPlanError, + `expected a PluginHandoffPlanError, got ${String(err)}`, + ); + assert.equal(err.reason, reason); + assert.equal(err.pluginId, PLUGIN_ID); + assert.equal(err.declaredPath, declaredPath); + return err; + } + + it('reads a well-formed plan', async () => { + await writePlan( + 'handoff-plan.json', + JSON.stringify({ + entries: [ + { filename: '0001_a.js', witnessSql: "SELECT to_regclass('public.a') IS NOT NULL" }, + { filename: '0002_b.js', witnessSql: "SELECT to_regclass('public.b') IS NOT NULL" }, + ], + }), + ); + + const plan = await loadHandoffPlan({ + pluginId: PLUGIN_ID, + packageRoot: root, + declaredPath: 'handoff-plan.json', + }); + + assert.deepEqual( + plan.entries.map((e) => e.filename), + ['0001_a.js', '0002_b.js'], + ); + assert.ok( + !('dryRun' in plan), + 'the kernel-run plan is executable data, not a preview request', + ); + }); + + it('refuses dryRun: true because the kernel-run path may not preview and then fall through', async () => { + await writePlan( + 'plans/preview.json', + JSON.stringify({ + dryRun: true, + entries: [{ filename: '0001_a.js', witnessSql: 'SELECT true' }], + }), + ); + const err = await refusal('plans/preview.json', 'dry-run-declared'); + assert.match(err.message, /migration runner/i); + }); + + it('accepts the operator-CLI fields and reports the ledger as advisory only', async () => { + // One file has to serve both `middleware/scripts/plugin-ledger-handoff.mjs` + // (which needs to be told the plugin, the ledger and the directory, + // because it runs with no manifest) and this loader (which knows all + // three authoritatively and must never take them from plugin data). + // Rejecting the CLI's fields would force a plugin to ship two files that + // can drift apart — the one an operator previews and the one core runs. + await writePlan( + 'handoff-plan.json', + JSON.stringify({ + pluginId: '@vendor/thing', + ledger: 'plg_vendor_thing_migrations', + migrationsDir: 'packages/plugin/migrations', + entries: [{ filename: '0001_a.js', witnessSql: 'SELECT true' }], + }), + ); + const plan = await loadHandoffPlan({ + pluginId: PLUGIN_ID, + packageRoot: root, + declaredPath: 'handoff-plan.json', + }); + assert.deepEqual( + plan.entries.map((e) => e.filename), + ['0001_a.js'], + ); + assert.equal( + plan.declaredLedger, + 'plg_vendor_thing_migrations', + 'reported so core can warn on a disagreement — never obeyed', + ); + }); + + it('refuses a plan that escapes the package root', async () => { + await refusal('../outside.json', 'escapes-package-root'); + }); + + it('refuses an absolute path', async () => { + await refusal('/etc/passwd', 'escapes-package-root'); + }); + + it('refuses a sibling directory that merely shares the root prefix', async () => { + // The `+ path.sep` on the containment check is what separates + // `-evil/plan.json` from `/plan.json`. A bare `startsWith` + // passes the first one, and the file it reads is outside the package. + await refusal(`../${basename(root)}-evil/plan.json`, 'escapes-package-root'); + }); + + it('refuses a file symlink that escapes the package root', async () => { + const outside = await makeOutsideRoot(); + await writeFile( + join(outside, 'outside-plan.json'), + JSON.stringify({ + entries: [{ filename: '0001_a.js', witnessSql: 'SELECT true' }], + }), + 'utf8', + ); + await mkdir(join(root, 'plans'), { recursive: true }); + await symlink(join(outside, 'outside-plan.json'), join(root, 'plans', 'current.json')); + + const err = await refusal('plans/current.json', 'escapes-package-root'); + assert.match(err.message, /link/i); + }); + + it('refuses a directory symlink that escapes the package root', async () => { + const outside = await makeOutsideRoot(); + await writeFile( + join(outside, 'secret.json'), + JSON.stringify({ + entries: [{ filename: '0001_a.js', witnessSql: 'SELECT true' }], + }), + 'utf8', + ); + await symlink(outside, join(root, 'plans')); + + const err = await refusal('plans/secret.json', 'escapes-package-root'); + assert.match(err.message, /link/i); + }); + + it('accepts a symlink that stays inside the package root', async () => { + await writePlan( + 'real/plan.json', + JSON.stringify({ + entries: [{ filename: '0001_a.js', witnessSql: 'SELECT true' }], + }), + ); + await mkdir(join(root, 'plans'), { recursive: true }); + await symlink('../real/plan.json', join(root, 'plans', 'current.json')); + + const plan = await loadHandoffPlan({ + pluginId: PLUGIN_ID, + packageRoot: root, + declaredPath: 'plans/current.json', + }); + assert.deepEqual(plan.entries.map((e) => e.filename), ['0001_a.js']); + }); + + it('refuses a missing file', async () => { + await refusal('handoff-plan.json', 'unreadable'); + }); + + it('refuses a file that is not JSON', async () => { + await writePlan('handoff-plan.json', 'entries: [] # yaml, not json\n'); + await refusal('handoff-plan.json', 'not-json'); + }); + + it('refuses a plan larger than the cap', async () => { + const filler = 'x'.repeat(MAX_HANDOFF_PLAN_BYTES + 1); + await writePlan( + 'handoff-plan.json', + JSON.stringify({ + entries: [{ filename: '0001_a.js', witnessSql: `SELECT '${filler}'` }], + }), + ); + await refusal('handoff-plan.json', 'too-large'); + }); + + it('refuses an empty entries list', async () => { + await writePlan('handoff-plan.json', JSON.stringify({ entries: [] })); + const err = await refusal('handoff-plan.json', 'malformed'); + assert.match(err.message, /entries/); + }); + + it('refuses an entry with no witness', async () => { + await writePlan( + 'handoff-plan.json', + JSON.stringify({ entries: [{ filename: '0001_a.js', witnessSql: ' ' }] }), + ); + await refusal('handoff-plan.json', 'malformed'); + }); + + it('refuses an unknown key rather than ignoring it', async () => { + // `dir` is the one that matters: `SeedLedgerOptions` accepts it and it + // would be a SECOND way to point the seeder at a directory, next to the + // manifest's `migrations`. Silently ignoring it would leave a plugin + // author believing a directory override took effect. + await writePlan( + 'handoff-plan.json', + JSON.stringify({ + dir: '../../elsewhere', + entries: [{ filename: '0001_a.js', witnessSql: 'SELECT true' }], + }), + ); + const err = await refusal('handoff-plan.json', 'malformed'); + assert.match(err.message, /dir/); + }); + + it('refuses a duplicated filename', async () => { + await writePlan( + 'handoff-plan.json', + JSON.stringify({ + entries: [ + { filename: '0001_a.js', witnessSql: 'SELECT true' }, + { filename: '0001_a.js', witnessSql: 'SELECT false' }, + ], + }), + ); + // Two witnesses for one file makes the outcome depend on iteration order. + // `seedLedger` refuses this too; refusing it HERE means the operator hears + // about it at load time, naming the file. + const err = await refusal('handoff-plan.json', 'malformed'); + assert.match(err.message, /0001_a\.js/); + }); + + it('refuses a top-level array', async () => { + await writePlan( + 'handoff-plan.json', + JSON.stringify([{ filename: '0001_a.js', witnessSql: 'SELECT true' }]), + ); + await refusal('handoff-plan.json', 'malformed'); + }); +}); diff --git a/middleware/test/toolPluginRuntimeHandoff.pg.test.ts b/middleware/test/toolPluginRuntimeHandoff.pg.test.ts new file mode 100644 index 000000000..d96fd43e0 --- /dev/null +++ b/middleware/test/toolPluginRuntimeHandoff.pg.test.ts @@ -0,0 +1,567 @@ +/** + * Epic #470 C15 — core runs the declared ledger handoff BEFORE its own + * pre-activate migration runner. + * + * WHAT WENT WRONG (gap G7 of the 2026-08-21 acceptance run) + * -------------------------------------------------------- + * C11 gave a plugin `ctx.sql.seedLedger` and documented it as "call this + * before `runMigrations`". But core runs the plugin's migrations ITSELF, + * before `activate()` — deliberately, so "the tables exist" is an invariant + * activate() can rely on (C7). So on the one upgrade C11 was built for the + * plugin's own call always arrived second, every ledger row was already + * written, and the handoff could only ever report `alreadySeeded`. + * + * Nothing failed. The log read `0 seeded, 9 already seeded`, which is + * indistinguishable from a healthy re-run — and `skippedNoWitness`, the one + * alarm C11 exists to raise, could never fire. + * + * WHY THESE TESTS LOOK LIKE THIS + * ------------------------------ + * The defect was ORDER, so asserting the outcome is not enough: a run that + * seeds nothing and applies everything reaches a correct database too. Each + * case therefore records the order the two steps ran in and asserts on it, and + * `no-witness` asserts on the WARN — the output an operator actually sees. + * + * The suite drives the real `ToolPluginRuntime.activate()` against a real + * on-disk package and a real Postgres, because the failure being fixed is a + * WIRING failure: every unit of this was already correct and tested in + * isolation. `pluginMigrationHandoffAccessor.pg.test.ts` was green while the + * feature it covers was unreachable in production. + * + * Fixture names are neutral and suffixed per case: core's decoupling ratchet + * counts the extracted plugin's identifiers in `middleware/test`, and this + * suite writes into the REAL core donor ledger, which it must leave exactly as + * it found it. + */ + +import { strict as assert } from 'node:assert'; +import { randomUUID } from 'node:crypto'; +import { mkdtemp, mkdir, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { after, afterEach, before, beforeEach, describe, it } from 'node:test'; + +import { SqlMigrationError } from '@omadia/plugin-api'; +import { Pool } from 'pg'; + +import { probePgTest } from './_helpers/pgTestDb.js'; + +import { CORE_MIGRATION_DONOR_LEDGER } from '../src/platform/pluginMigrationHandoff.js'; +import { PluginHandoffPlanError } from '../src/platform/pluginHandoffPlan.js'; +import { ServiceRegistry } from '../src/platform/serviceRegistry.js'; +import { + adaptManifestV1, + type PluginCatalog, +} from '../src/plugins/manifestLoader.js'; +import { + ToolPluginRuntime, + type ToolPluginRuntimeDeps, +} from '../src/plugins/toolPluginRuntime.js'; + +const { url: PG_URL, reachable: pgAvailable } = await probePgTest({ + label: 'toolPluginRuntimeHandoff', + vars: ['PLUGIN_SQL_PG_TEST_URL', 'GRAPH_PG_TEST_URL', 'DATABASE_URL'], + timeoutMs: 1_500, +}); + +/** How many migration files the fixture package ships. */ +const FILE_COUNT = 3; + +describe('#470 C15 pre-activate ledger handoff', { skip: !pgAvailable }, () => { + let pool: Pool; + let root: string; + let suffix: string; + let pluginId: string; + let ledger: string; + /** The plugin's own migration filenames, in order. */ + let files: string[]; + /** The tables those files create. */ + let tables: string[]; + /** The names core's migrator recorded for the same migrations. */ + let donorRows: string[]; + /** Every log line the runtime emitted during the case. */ + let logs: string[]; + + before(() => { + pool = new Pool({ connectionString: PG_URL, max: 4 }); + }); + + after(async () => { + await pool.end(); + }); + + beforeEach(async () => { + suffix = randomUUID().replace(/-/g, '').slice(0, 8); + pluginId = `@test/adopt-${suffix}`; + ledger = `plg_test_adopt_${suffix}_mig`; + logs = []; + files = []; + tables = []; + donorRows = []; + + root = await mkdtemp(join(tmpdir(), 'c15-runtime-')); + await mkdir(join(root, 'migrations'), { recursive: true }); + await mkdir(join(root, 'dist'), { recursive: true }); + + for (let i = 1; i <= FILE_COUNT; i++) { + const step = String(i).padStart(4, '0'); + const table = `c15_${suffix}_${step}`; + const file = `${step}_step_${suffix}.js`; + files.push(file); + tables.push(table); + // Core's migrator would have shipped the same migration as `.sql`; the + // extracted plugin re-emits it as `.js`. The handoff matches by stem, + // so the fixture has to reproduce that skew or it proves nothing. + donorRows.push(`${step}_step_${suffix}.sql`); + await writeFile( + join(root, 'migrations', file), + `export default async (client) => { await client.query('CREATE TABLE IF NOT EXISTS ${table} (id int)'); };\n`, + 'utf8', + ); + } + + // A plugin whose activate() does nothing. The handoff under test happens + // BEFORE this runs, which is the entire point. + await writeFile( + join(root, 'dist', 'plugin.js'), + 'export async function activate() { return { close: async () => {} }; }\n', + 'utf8', + ); + }); + + afterEach(async () => { + await rm(root, { recursive: true, force: true }); + for (const table of tables) { + await pool.query(`DROP TABLE IF EXISTS ${table}`).catch(() => undefined); + } + await pool.query(`DROP TABLE IF EXISTS ${ledger}`).catch(() => undefined); + // The donor here is the REAL core ledger. This suite only ever adds its + // own suffixed rows and removes exactly those. It never drops the table. + for (const row of donorRows) { + await pool + .query(`DELETE FROM ${CORE_MIGRATION_DONOR_LEDGER} WHERE id = $1`, [row]) + .catch(() => undefined); + } + }); + + function witnessFor(table: string): string { + return `SELECT to_regclass('public.${table}') IS NOT NULL`; + } + + async function writeHandoffPlan( + body: unknown = { + entries: files.map((file, i) => ({ + filename: file, + witnessSql: witnessFor(tables[i] as string), + })), + }, + ): Promise { + await writeFile( + join(root, 'handoff-plan.json'), + typeof body === 'string' ? body : JSON.stringify(body, null, 2), + 'utf8', + ); + } + + async function ensureDonorLedger(): Promise { + await pool.query( + `CREATE TABLE IF NOT EXISTS ${CORE_MIGRATION_DONOR_LEDGER} (id TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT now())`, + ); + } + + /** The state a core-to-plugin upgrade starts from: core ran these. */ + async function seedDonorRows(): Promise { + await ensureDonorLedger(); + for (const row of donorRows) { + await pool.query( + `INSERT INTO ${CORE_MIGRATION_DONOR_LEDGER} (id) VALUES ($1) ON CONFLICT DO NOTHING`, + [row], + ); + } + } + + /** ...and left these tables behind. */ + async function createDonorTables(): Promise { + for (const table of tables) { + await pool.query(`CREATE TABLE IF NOT EXISTS ${table} (id int)`); + } + } + + function catalogFor(sqlPermission: Record): PluginCatalog { + const manifest = { + schema_version: '1', + identity: { + id: pluginId, + name: pluginId, + version: '1.0.0', + kind: 'extension', + domain: 'test.adopt', + }, + lifecycle: { entry: 'dist/plugin.js' }, + requires: ['graphPool@^1'], + provides: [], + permissions: { sql: sqlPermission }, + }; + const plugin = adaptManifestV1(manifest); + assert.ok(plugin, 'fixture manifest must adapt'); + const entries = new Map([ + [ + plugin.id, + { plugin, manifest, source_path: 'test', source_kind: 'manifest-v1' }, + ], + ]); + return { + get: (id: string) => entries.get(id), + list: () => [...entries.values()], + } as unknown as PluginCatalog; + } + + function runtimeFor( + sqlPermission: Record, + opts: { grantLedger?: string } = {}, + ): ToolPluginRuntime { + const stub = (): (() => void) => (): void => {}; + const serviceRegistry = new ServiceRegistry(); + serviceRegistry.provide('graphPool', pool); + const grantedLedger = opts.grantLedger ?? ledger; + const deps = { + catalog: catalogFor(sqlPermission), + registry: { get: () => ({ status: 'active' }) }, + vault: { + get: async (): Promise => undefined, + listKeys: async (): Promise => [], + }, + uploadedStore: { + get: (id: string) => (id === pluginId ? { id, path: root } : undefined), + list: () => [{ id: pluginId, path: root }], + }, + serviceRegistry, + nativeToolRegistry: { register: stub, registerHandler: stub }, + pluginRouteRegistry: { register: stub, disposeBySource: () => 0 }, + notificationRouter: { dispatch: (): void => {}, registerChannel: stub }, + uiRouteCatalog: { register: stub, registerNav: stub }, + jobScheduler: { register: stub, stopForPlugin: (): void => {} }, + // The operator granted exactly the ledger the manifest declares. + sqlGrantStore: { + get: async (): Promise<{ ledger: string }> => ({ ledger: grantedLedger }), + }, + log: (msg: string): void => { + logs.push(msg); + }, + } as unknown as ToolPluginRuntimeDeps; + return new ToolPluginRuntime(deps); + } + + /** + * Which of the two steps logged first. + * + * This is the assertion the old code could not pass. Reading the database + * afterwards cannot tell the two orderings apart — both end with the tables + * present and the ledger full. + */ + function stepOrder(): string[] { + const order: string[] = []; + for (const line of logs) { + // The WARN also says "ledger handoff" — it is the same step reporting a + // second time, not a second step. Counting it would make the order read + // `handoff, handoff, migrations` and quietly weaken every assertion + // here into "a handoff happened at some point". + if (line.includes('WARN')) continue; + if (line.includes('ledger handoff')) order.push('handoff'); + else if (line.includes('migration(s) to ledger')) order.push('migrations'); + } + return order; + } + + function handoffLine(): string { + const line = logs.find( + (l) => l.includes('ledger handoff') && !l.includes('WARN'), + ); + assert.ok(line, `no handoff line in:\n${logs.join('\n')}`); + return line; + } + + async function ledgerRows(): Promise { + // `filename` is the plugin ledger's primary key — see `pluginLedgerDdl`, + // which is exported precisely so the runner and the seeder cannot drift. + const res = await pool.query<{ filename: string }>( + `SELECT filename FROM ${ledger} ORDER BY filename`, + ); + return res.rows.map((r) => r.filename); + } + + async function tableExists(name: string): Promise { + const res = await pool.query<{ ok: boolean }>( + `SELECT to_regclass($1) IS NOT NULL AS ok`, + [`public.${name}`], + ); + return res.rows[0]?.ok === true; + } + + async function ledgerExists(): Promise { + const res = await pool.query<{ ok: boolean }>( + `SELECT to_regclass($1) IS NOT NULL AS ok`, + [`public.${ledger}`], + ); + return res.rows[0]?.ok === true; + } + + it('seeds from the donor and leaves the runner nothing to do', async () => { + // The upgrade C11 was built for: core's rows AND core's tables are there. + await seedDonorRows(); + await createDonorTables(); + await writeHandoffPlan(); + + await runtimeFor({ ledger, migrations: 'migrations', handoff: 'handoff-plan.json' }).activate( + pluginId, + ); + + assert.deepEqual( + stepOrder(), + ['handoff'], + 'the handoff ran, and the runner then had nothing to log — the runner only logs when it applies', + ); + assert.match(handoffLine(), new RegExp(`${String(FILE_COUNT)} seeded`)); + assert.match(handoffLine(), /0 left for the migration runner/); + assert.deepEqual( + await ledgerRows(), + [...files].sort(), + 'every file is recorded as applied without having been re-applied', + ); + + const donor = await pool.query( + `SELECT id FROM ${CORE_MIGRATION_DONOR_LEDGER} WHERE id = ANY($1::text[])`, + [donorRows], + ); + assert.equal( + donor.rows.length, + FILE_COUNT, + 'nothing in the handoff deletes — those rows are the rollback path', + ); + }); + + it('runs the handoff BEFORE the migration runner', async () => { + // Same donor rows, but the tables are gone. Both steps have work to do, + // so both log — which is the only configuration where the ORDER is + // directly observable. + await seedDonorRows(); + await writeHandoffPlan(); + + await runtimeFor({ ledger, migrations: 'migrations', handoff: 'handoff-plan.json' }).activate( + pluginId, + ); + + assert.deepEqual( + stepOrder(), + ['handoff', 'migrations'], + 'inverted here is the whole bug: the runner writes the ledger rows the handoff was meant to decide on', + ); + }); + + it('raises the skippedNoWitness alarm when the rows are there and the schema is not', async () => { + // A restore from an older snapshot, a rolled-back deploy, a dropped table. + // Donor rows present, schema objects absent. This is the one output C11 + // was built to produce and the one G7 made unreachable. + await seedDonorRows(); + await writeHandoffPlan(); + + await runtimeFor({ ledger, migrations: 'migrations', handoff: 'handoff-plan.json' }).activate( + pluginId, + ); + + assert.match(handoffLine(), /0 seeded/); + assert.match( + handoffLine(), + new RegExp(`${String(FILE_COUNT)} left for the migration runner`), + ); + + const warn = logs.find((l) => l.includes('WARN') && l.includes('witness')); + assert.ok(warn, `no witness WARN in:\n${logs.join('\n')}`); + for (const file of files) { + assert.ok( + warn.includes(file), + `the WARN must name '${file}' — a count alone does not tell an operator where to look`, + ); + } + + // And the runner then repaired it, which is what makes the alarm safe to + // be a warning rather than a refusal. + assert.deepEqual(stepOrder(), ['handoff', 'migrations']); + assert.deepEqual(await ledgerRows(), [...files].sort()); + for (const table of tables) { + const exists = await pool.query<{ ok: boolean }>( + `SELECT to_regclass($1) IS NOT NULL AS ok`, + [`public.${table}`], + ); + assert.equal(exists.rows[0]?.ok, true, `${table} was repaired by the runner`); + } + }); + + it('changes nothing for a plugin that declares no handoff', async () => { + await seedDonorRows(); + await createDonorTables(); + await writeHandoffPlan(); + + // The plan file is present on disk and is deliberately ignored: what + // switches the step on is the MANIFEST, not the presence of a file. + await runtimeFor({ ledger, migrations: 'migrations' }).activate(pluginId); + + assert.deepEqual( + stepOrder(), + ['migrations'], + 'no handoff step, and the runner behaves exactly as it did before C15', + ); + assert.deepEqual(await ledgerRows(), [...files].sort()); + }); + + it('refuses activation when the declared plan is malformed', async () => { + await seedDonorRows(); + await createDonorTables(); + await writeHandoffPlan({ entries: [{ filename: files[0] }] }); + + const err = await runtimeFor({ + ledger, + migrations: 'migrations', + handoff: 'handoff-plan.json', + }) + .activate(pluginId) + .then( + () => undefined, + (e: unknown) => e, + ); + + assert.ok( + err instanceof PluginHandoffPlanError, + `expected a PluginHandoffPlanError, got ${String(err)}`, + ); + assert.equal(err.reason, 'malformed'); + assert.deepEqual( + stepOrder(), + [], + 'a plugin whose handoff cannot be read does not get its migrations run either — ' + + 'core would otherwise write the ledger rows the refused plan was meant to decide on, ' + + 'which is exactly the state the plan existed to avoid', + ); + }); + + it('refuses activation when the declared plan escapes the package root', async () => { + await seedDonorRows(); + const err = await runtimeFor({ + ledger, + migrations: 'migrations', + handoff: '../handoff-plan.json', + }) + .activate(pluginId) + .then( + () => undefined, + (e: unknown) => e, + ); + assert.ok(err instanceof PluginHandoffPlanError, `got ${String(err)}`); + assert.equal(err.reason, 'escapes-package-root'); + }); + + it('refuses activation when the plan asks for a dry run', async () => { + await seedDonorRows(); + await writeHandoffPlan({ + dryRun: true, + entries: files.map((file, i) => ({ + filename: file, + witnessSql: witnessFor(tables[i] as string), + })), + }); + + const err = await runtimeFor({ + ledger, + migrations: 'migrations', + handoff: 'handoff-plan.json', + }) + .activate(pluginId) + .then( + () => undefined, + (e: unknown) => e, + ); + assert.ok(err instanceof PluginHandoffPlanError, `got ${String(err)}`); + assert.equal(err.reason, 'dry-run-declared'); + assert.deepEqual( + stepOrder(), + [], + 'a declared dry run fails closed before either step can run — letting the runner continue would recreate G7 from one stray key', + ); + assert.equal(await ledgerExists(), false, 'the plugin ledger was never created'); + for (const table of tables) { + assert.equal( + await tableExists(table), + false, + `${table} must stay absent because the migration runner never ran`, + ); + } + }); + + it('refuses activation when a witness fails at the database and never reaches the migration runner', async () => { + await writeHandoffPlan({ + entries: [ + { + filename: files[0], + witnessSql: `SELECT * FROM c15_missing_relation_${suffix}`, + }, + ...files.slice(1).map((file, i) => ({ + filename: file, + witnessSql: witnessFor(tables[i + 1] as string), + })), + ], + }); + + const err = await runtimeFor({ + ledger, + migrations: 'migrations', + handoff: 'handoff-plan.json', + }) + .activate(pluginId) + .then( + () => undefined, + (e: unknown) => e, + ); + assert.ok(err instanceof SqlMigrationError, `got ${String(err)}`); + assert.deepEqual( + stepOrder(), + [], + 'the seeder failed before it could log success, and the migration runner below stayed dark', + ); + assert.equal(await ledgerExists(), false, 'the failed handoff rolled its ledger DDL back'); + for (const table of tables) { + assert.equal( + await tableExists(table), + false, + `${table} must stay absent because the migration runner never ran`, + ); + } + }); + + it('treats a mismatched SQL grant as ungranted and skips both pre-activate steps', async () => { + await seedDonorRows(); + await writeHandoffPlan(); + + await runtimeFor( + { ledger, migrations: 'migrations', handoff: 'handoff-plan.json' }, + { grantLedger: `${ledger}_other` }, + ).activate(pluginId); + + assert.deepEqual( + stepOrder(), + [], + 'without a matching grant there is no SQL accessor, so neither the handoff nor the runner can run', + ); + assert.equal(await ledgerExists(), false, 'the plugin ledger was never created'); + for (const table of tables) { + assert.equal( + await tableExists(table), + false, + `${table} must stay absent because the plugin was treated as ungranted`, + ); + } + assert.ok( + logs.some((line) => line.includes('treating as ungranted')), + `missing mismatched-grant log line:\n${logs.join('\n')}`, + ); + }); +});