C11: adopt core's migration ledger on witnesses, not trust (epic byte5ai/omadia#470) - #6
Merged
Merged
Conversation
…0 C11) Slots 0022-0030 are already applied on every installation that ran the Dev Platform inside core, recorded in CORE's ledger. This plugin's ledger starts empty, so `runMigrations()` re-applies all nine. They are idempotent, so that is merely slow on a healthy database — but idempotence is a property of the files, and betting an upgrade on it nine times over is not a plan. `ctx.sql.seedLedger()` (core PR byte5ai/omadia#806, plugin-api 1.3.0) records them as applied instead, and will not take core's word for it: each file needs a WITNESS that the schema object it creates is actually present. The case that makes this necessary is rows present, tables ABSENT — a restore from a snapshot older than the migrations, a version-skewed rollback, an operator who dropped a table during an incident. A handoff that trusted core's rows would activate this plugin green and make every request 500. With witnesses the seed declines, `runMigrations()` applies the files, and that is the repair. - `src/ledgerHandoff.ts` — the nine witnesses. Each proves the LAST object its file creates, because a core migration file ran in one transaction, so the last object exists exactly when the whole file was applied. 0025 is the odd one: it REPLACES a CHECK constraint, so its witness reads the constraint DEFINITION for 'plugin' — presence alone would be true before it ran too. - `activate()` seeds BEFORE migrating, guarded on `ctx.sql.seedLedger` being present: a core older than plugin-api 1.3.0 falls through to the apply loop and says so, rather than refusing to activate on a core that can run it. - A non-empty `skippedNoWitness` is logged as a loud WARNING, not a refusal — the apply loop below is the repair, and the operator needs to know the database is not what they thought. - `handoff-plan.json` + REQUIRED in the ZIP. It is how an operator dry-runs the handoff against production BEFORE installing, with core's `middleware/scripts/plugin-ledger-handoff.mjs`. A ZIP without it installs perfectly and quietly removes the only step that de-risks the upgrade. Tests: 12 new. The entry list must cover exactly the shipped migrations (a missing witness re-applies forever, invisibly); no witness may cast to regclass (it throws on the very case a witness detects); every witness must name an object that appears in its own migration — all nine touch dev_jobs, so proving the wrong one is the easy mistake; the plan file must match the code; and activate() must seed before it migrates, degrade on an old core, and warn on a disagreement. Mutation check: pointing 0029's witness at 0022's table fails exactly the target test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Epic byte5ai/omadia#470 C11, plugin side. Core half: byte5ai/omadia#806 (adds
ctx.sql.seedLedger, plugin-api 1.3.0).The problem
Slots
0022–0030are already applied on every installation that ran the Dev Platform inside core — recorded in core's ledger (_multi_orchestrator_migrations). This plugin's ledger (plg_omadia_dev_platform_migrations) starts empty, sorunMigrations()re-applies all nine on the first activation after the upgrade. They are idempotent, so on a healthy database that is merely slow. But idempotence is a property of the files, and betting an upgrade on it nine times over is not a plan.Why the obvious handoff is wrong
Copy core's rows into this plugin's ledger and skip those files. Correct on a healthy database; silently destroys one specific installation:
Nine rows get written, nothing is applied, the plugin activates green, and every request against those tables 500s.
So core's ledger is corroboration and a witness is the decision: a query against the live catalog, true only when the schema object that file creates is actually there.
What ships
src/ledgerHandoff.ts— the nine witnesses. Each proves the last object its file creates, because each core migration ran inside a single transaction: the last object exists exactly when the whole file was applied. One rule, checkable by a reviewer against the.sqlinmigrations/.0022_dev_platform.jsdev_job_artifacts0023_dev_platform_pipeline.jsdev_github_app_installations0024_dev_platform_w3.jsdev_jobs.conductor_await_id0025_dev_jobs_source_plugin.jsdev_jobs_source_checkadmitting'plugin'0026_dev_job_gate_kind.jsdev_job_gates.gate_kind0027_dev_platform_triggers.jsdev_jobs.usage_estimated0028_dev_jobs_webhook_one_active.jsdev_jobs_webhook_one_active0029_dev_platform_retention.jsdev_jobs_terminal_ended_idx0030_dev_job_events_truncated_marker.jsdev_job_events_truncated_once_idx0025is the odd one out: it replaces a CHECK constraint to admit a newsourcevalue, so its witness reads the constraint'spg_get_constraintdeffor'plugin'. The constraint name is present before the migration too, so presence alone would be true on a database that never ran it.activate()seeds before it migrates, guarded onctx.sql.seedLedgerexisting. A core older than plugin-api 1.3.0 falls through to the apply loop and says so in the log — declaring the method required would make this plugin refuse to activate on a core that can in fact run it.A non-empty
skippedNoWitnessis a loud WARNING, not a refusal. The apply loop below is the repair, so refusing would replace a self-healing upgrade with an outage. But the operator has to be told: core's ledger and the live catalog disagree, and that means this database is not what they think it is.handoff-plan.json, and it is REQUIRED in the ZIP. It is how an operator dry-runs the handoff against production before installing, with core'smiddleware/scripts/plugin-ledger-handoff.mjs --plan …. A ZIP without it installs perfectly and quietly removes the only step that de-risks the upgrade — the same reasoning that mademigrations/required after the first cut ofbuild-zip.mjsshipped a ZIP without it.Two traps, both encoded as tests
'public.dev_jobs'::regclassthrows for a missing table — the exact case a witness exists to detect — so a cast turns the restore scenario into an activation crash. Every witness usesto_regclassor a catalog join. A test rejects the cast syntax outright.SELECT count(*)is not a witness:1for a table that exists,0for one that exists and is empty, a throw for one that does not. The kernel enforces exactly one row, one column, a real boolean; a test forbidscount(here too.Tests
12 new, all offline. What each one catches, since none of this can be proved against a database this repo can see:
regclasscount(; no;dev_jobs, so naming a different migration's object is the easy mistake; this greps the migration's own SQL0025reads the constraint DEFINITION0025ranhandoff-plan.jsonmatches the codeactivate()order is seed → migrateseedLedgerand logs the degradationskippedNoWitnessMutation check: pointing
0029's witness at0022's table (dev_job_artifacts) fails exactly one test —proves an object the migration it belongs to actually creates— and nothing else. 682 pass / 1 fail. The rule is enforced, not just documented.Gates
npm run typecheck(all workspaces)npm run buildnpm testnpm run packagehandoff-plan.jsonstaged, 9 migrations verified, version drift guard passedOrdering
This plugin is safe to merge before core #806:
ctx.sql.seedLedgeris guarded, so against today's core the handoff is skipped, the log says why, and the nine idempotent migrations are applied as they are today. The handoff starts working the moment a core carrying #806 is deployed.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.