Give every migration a version the ledger can record

.github/workflows/ci.yml

@@ -190,11 +190,12 @@ jobs:
         with:
           version: 2.109.1
 
-      # The repo keeps a few historically duplicated migration versions (see
-      # KNOWN_DUPLICATE_MIGRATIONS); the CLI's schema_migrations table rejects
-      # the second file of each pair. Give them unique versions in THIS
-      # disposable checkout only, preserving apply order (verified by the script
-      # + tests/ci-dedupe-migration-versions.test.ts). Never committed.
+      # The 17 historical duplicate versions were renamed in the repo itself, so
+      # this is now a NO-OP that renames nothing. It is kept as a safety net: if
+      # a duplicate is ever reintroduced, this gives the pair unique versions in
+      # THIS disposable checkout, preserving apply order, rather than letting
+      # `supabase start` die on schema_migrations_pkey with no explanation.
+      # (db:audit:migrations above is the real gate and fails first.)
       - name: De-duplicate migration versions for the isolated apply
         run: node scripts/ci-dedupe-migration-versions.mjs
         env:

docs/audit/migration-ledger-state.sql

@@ -0,0 +1,54 @@
+-- Bubaly :: migration ledger pre-flight — READ ONLY, changes nothing
+-- ============================================================================
+-- Run this before attempting the ledger repair described in
+-- docs/runbooks/LB-016-wallet-permissive-policy-finding.md §4. It answers the
+-- three questions that decide whether the repair is safe to start, and it
+-- writes nothing, locks nothing and can be run on a live database any time.
+--
+-- Background: production's supabase_migrations.schema_migrations records only
+-- 0001-0003 while the schema is far ahead of that, so `supabase db push` wants
+-- to replay everything from 0004 and the workflow's --enforce-history guard
+-- stops it. Three migrations (0249, 0254, 0275) were additionally applied BY
+-- HAND and are recorded nowhere.
+
+-- ── 1. What the ledger thinks is applied ───────────────────────────────────
+select 'ledger' as source, version, name
+from   supabase_migrations.schema_migrations
+order  by version;
+
+-- ── 2. How far ahead the schema actually is ────────────────────────────────
+-- A ledger at 0003 with hundreds of tables means the schema was built by some
+-- path other than `db push`. Large numbers here are expected, not alarming.
+select (select count(*) from pg_tables  where schemaname = 'public') as public_tables,
+       (select count(*) from pg_policies where schemaname = 'public') as public_policies,
+       (select max(version) from supabase_migrations.schema_migrations) as ledger_high_water;
+
+-- ── 3. Spot-check the hand-applied three ───────────────────────────────────
+-- Each row says whether that migration's effect is present in the schema and
+-- whether the ledger knows about it. "present / unrecorded" is the state this
+-- pre-flight exists to surface: the work is done, the ledger cannot prove it.
+with probe(version, what, present) as (
+  values
+    ('0249', 'mark_model_dirty() skips a deleted family',
+      exists (select 1 from pg_proc p join pg_namespace n on n.oid = p.pronamespace
+              where n.nspname = 'public' and p.proname = 'mark_model_dirty'
+                and pg_get_functiondef(p.oid) like '%not exists (select 1 from public.families%')),
+    ('0254', 'restrictive manager guards on wallet_transactions',
+      exists (select 1 from pg_policy pol join pg_class c on c.oid = pol.polrelid
+              where c.relname = 'wallet_transactions' and not pol.polpermissive)),
+    ('0275', 'no permissive write policy on bills lacking can_manage_family',
+      not exists (select 1 from pg_policy pol join pg_class c on c.oid = pol.polrelid
+                  where c.relname = 'bills' and pol.polpermissive
+                    and pol.polcmd in ('a','w','d','*')
+                    and coalesce(pg_get_expr(pol.polqual, pol.polrelid), '') ||
+                        coalesce(pg_get_expr(pol.polwithcheck, pol.polrelid), '')
+                        not like '%can_manage_family%'))
+)
+select version,
+       what,
+       case when present then 'present in schema' else 'NOT present' end as schema_state,
+       case when exists (select 1 from supabase_migrations.schema_migrations m
+                         where m.version = probe.version)
+            then 'recorded' else 'unrecorded' end as ledger_state
+from   probe
+order  by version;

docs/runbooks/LB-016-wallet-permissive-policy-finding.md

@@ -123,35 +123,84 @@ stray permissive write policy remains, so this cannot quietly regress.
 
 ## 4. The one thing still open — and it is an operator action
 
-`0275` closes this **when it is applied**. Production cannot currently apply it,
-for a reason unrelated to wallets:
-
 `scripts/audit-production-migration-state.mjs` refuses historical replay when
 production carries the `profiles_insert_self` policy but has **no migration
-`0004` recorded** (`hasUnrecordedBaseline`). The last release attempt hit exactly
-this: it failed the safety check, **rolled back atomically**, and a follow-up
-audit showed only `0001`–`0003` in the ledger. Nothing was applied and no data
-was lost.
+`0004` recorded** (`hasUnrecordedBaseline`). Production's ledger holds only
+`0001`–`0003` while the schema is hundreds of tables ahead, so every release
+that touches `supabase/` stops here.
 
-That check is correct and must not be bypassed. Repairing it means stamping the
-production migration ledger to match the schema that is really there — a
-deliberate, credentialed operator action against the production database.
+That check is correct and must not be bypassed.
 
 **Do not:**
 - disable or weaken `hasUnrecordedBaseline`;
 - pass `--enforce-history` off to "get past" it;
 - hand-apply anything that touches **data or schema** out of band — that is what
   the ledger exists to keep honest.
 
-**Do:** repair the ledger baseline, then let the normal forward release apply
-`0272` onward, `0275` included.
+### 4.1 What the repair actually is
+
+Not a stamping exercise. Every migration in this repository is **additive and
+idempotent** — `tests/migrations-are-additive.test.ts` holds the whole history
+to zero `DROP TABLE` / `DROP COLUMN` / `TRUNCATE` / `DROP TYPE`, and CI replays
+all 291 of them against a real Postgres on every PR (`Database (migration
+replay · RLS boundary probes)`). So the ledger does not need to be *told* what
+is applied; it repairs itself by letting `supabase db push` run from `0004`,
+where the already-applied migrations no-op and the genuinely missing ones land.
+
+Writing rows into `supabase_migrations.schema_migrations` by hand is the option
+NOT to take: it asserts that work was done without doing it, and a migration
+wrongly marked applied is skipped forever.
+
+### 4.2 Prerequisite, now satisfied
+
+`schema_migrations` has a **PRIMARY KEY on `version`**. The repository used to
+carry 17 duplicate version numbers (`0010`, `0026`, `0042`, …), so a replay died
+at the second `0010` with
+
+```
+ERROR: duplicate key value violates unique constraint "schema_migrations_pkey"
+```
 
-`0275` itself is a defensible exception, and §5 walks through running it by hand:
-it changes policies only, touches no row and no column, is idempotent, and
-asserts its own end state. Applying it early closes the finding now rather than
-after the ledger work. It does widen the ledger gap by one more unrecorded
-migration, which is a real cost — see the caveat at the end of §5 — so it is a
-trade, not a free action.
+Every group has since been renamed to a unique version preserving apply order
+(`0010_blog_posts` → `00100_blog_posts`, `0010_support_tickets_admin_users` →
+`00101_support_tickets_admin_users`). `KNOWN_DUPLICATE_MIGRATIONS` is now empty
+and `tests/migration-version-safety.test.ts` fails if a duplicate returns.
+
+### 4.3 The procedure
+
+1. **Pre-flight.** Run `docs/audit/migration-ledger-state.sql` — read-only. It
+   reports what the ledger holds, how far ahead the schema is, and whether the
+   three hand-applied migrations (`0249`, `0254`, `0275`) show as
+   *present / unrecorded*. Expect exactly that for all three.
+2. **Take a restore point.** A PITR checkpoint or a backup, not a mental note.
+   This is the step that makes everything after it reversible.
+3. **Pick a maintenance window** — see the warning in 4.4.
+4. **Replay**, outside the workflow, with the ledger guard intact:
+   `supabase db push` against the production project.
+5. **Verify.** Re-run the pre-flight; the ledger high-water mark should now be
+   the newest migration. Then run `node scripts/audit-production-migration-state.mjs`
+   and confirm `requiresBaselineReview: false` and `moneyWrites.exploitable: false`.
+6. **Re-run the release workflow.** With `0004` recorded, the guard passes on
+   its own — nothing in the code needs changing to make that happen.
+
+### 4.4 The window nobody has flagged before
+
+A replay runs the history **in order**, and the history contains a period where
+the money boundary is open:
+
+- `0006` creates the household finance policies gated on `is_family_member`
+  (membership, not role);
+- `0267` narrows them to `can_manage_family`;
+- `0275` sweeps whatever survived and adds the restrictive guards.
+
+Between `0006` and `0267` executing, production briefly carries the exact
+permissive policies §5 describes. The end state is correct — that is the whole
+point of running them in order — but the intermediate state is not. Replay in a
+maintenance window with the app unavailable, not against live traffic.
+
+That risk is a consequence of repairing the ledger late. It gets no smaller by
+waiting, and every migration hand-applied in the meantime adds one more row the
+replay has to no-op through.
 
 ## 5. Running this by hand in the Supabase SQL editor
 

scripts/audit-migration-versions.mjs

@@ -8,29 +8,18 @@ const DEFAULT_MIGRATIONS_DIR = resolve(ROOT, 'supabase', 'migrations');
 // These collisions are already part of the repository's historical migration
 // lineage. Renaming them without checking applied remote history could create
 // drift in a production database.
-export const KNOWN_DUPLICATE_MIGRATIONS = Object.freeze({
-  '0010': ['0010_blog_posts.sql', '0010_support_tickets_admin_users.sql'],
-  '0026': ['0026_display_layouts.sql', '0026_medication_doses.sql'],
-  '0042': ['0042_family_location.sql', '0042_loyalty.sql'],
-  '0043': ['0043_chore_missions.sql', '0043_wishlists.sql'],
-  '0073': ['0073_behavior_tracking.sql', '0073_habits.sql'],
-  '0080': ['0080_food_household.sql', '0080_health_wellness.sql'],
-  '0089': ['0089_avatars_bucket.sql', '0089_dashboard_layouts.sql'],
-  '0090': ['0090_communications_hub.sql', '0090_stripe_money.sql'],
-  '0095': ['0095_pay_handles.sql', '0095_wallet_transfers.sql'],
-  '0098': ['0098_relationship_helper.sql', '0098_trip_intelligence.sql'],
-  '0105': ['0105_calendar_events_rls_repair.sql', '0105_child_logins.sql'],
-  '0108': ['0108_album_highlight_kind.sql', '0108_messages_enhance.sql'],
-  '0109': [
-    '0109_documents_favorite.sql',
-    '0109_finance_rls_repair.sql',
-    '0109_user_preferences_rls_repair.sql',
-  ],
-  '0110': ['0110_family_profile.sql', '0110_transactions_member.sql'],
-  '0137': ['0137_ai_call_guardian.sql', '0137_child_login_throttle.sql'],
-  '0138': ['0138_demo_sessions.sql', '0138_onboarding_imports.sql'],
-  '0142': ['0142_family_signals.sql', '0142_poll_facilitation.sql'],
-});
+// Historically this repository carried 17 version collisions — two or three
+// files sharing one numeric prefix. They are gone: every colliding group was
+// renamed to a unique version that keeps the same apply order (0010_blog_posts
+// -> 00100_blog_posts, 0010_support_tickets_admin_users -> 00101_…), because
+// supabase_migrations.schema_migrations keys on version and physically cannot
+// record two rows for 0010. That blocked Supabase branching outright and would
+// have blocked the production ledger repair at the same point.
+//
+// This map stays, deliberately empty: the collision check below still runs, so
+// a NEW duplicate fails the audit instead of being quietly absorbed. Do not add
+// entries to make a red audit go green — give the new migration a free number.
+export const KNOWN_DUPLICATE_MIGRATIONS = Object.freeze({})
 
 export function readMigrationInventory(directory = DEFAULT_MIGRATIONS_DIR) {
   return readdirSync(directory)
@@ -63,7 +52,13 @@ export function auditMigrationVersions(directory = DEFAULT_MIGRATIONS_DIR) {
     return !known || known.join('|') !== names.join('|');
   });
 
-  const versions = entries.map(({ version }) => Number(version)).filter(Number.isFinite);
+  // Versions are 4-digit generations. The de-duplicated ones carry a 5th digit
+  // that orders them WITHIN a generation (00100/00101 both sit in 0010), so the
+  // next free number comes from the first four digits — Number('01421') is 1421
+  // and would otherwise push the next migration to 1422.
+  const versions = entries
+    .map(({ version }) => Number(version.slice(0, 4)))
+    .filter(Number.isFinite);
   const nextVersion = String(Math.max(0, ...versions) + 1).padStart(4, '0');
 
   return { entries, duplicates, unexpectedDuplicates, nextVersion };

scripts/ci-dedupe-migration-versions.mjs

@@ -1,16 +1,21 @@
 // scripts/ci-dedupe-migration-versions.mjs — CI-only helper for `supabase start`.
 //
-// The repository's migration lineage contains a handful of files that share a
-// version prefix (see KNOWN_DUPLICATE_MIGRATIONS in audit-migration-versions.mjs).
-// They are historical and must NOT be renamed in the repo (a production database
-// records applied versions by that prefix). The Supabase CLI, however, records
-// each applied migration in supabase_migrations.schema_migrations keyed by
-// version, so an isolated `supabase start` fails on the second file of a pair:
+// The repository USED to carry 17 duplicated version prefixes. They have since
+// been renamed in the repo itself, using exactly the scheme below, so this
+// script now finds nothing to do. It is kept as a safety net for a duplicate
+// reintroduced by a future branch.
+//
+// Why they could not stay: the Supabase CLI records each applied migration in
+// supabase_migrations.schema_migrations keyed by version, and that table has a
+// PRIMARY KEY on version, so the second file of a pair fails outright:
 //   ERROR: duplicate key value violates unique constraint "schema_migrations_pkey"
+// That blocked Supabase branching, and would have blocked the production ledger
+// repair at the same statement.
 //
-// This script runs ONLY in the disposable CI checkout, right before
-// `supabase start`, and renames every duplicated group to unique versions that
-// keep the exact same apply order (string order, as the CLI sorts filenames):
+// When it does have work to do, it runs ONLY in the disposable CI checkout,
+// right before `supabase start`, and renames every duplicated group to unique
+// versions that keep the exact same apply order (string order, as the CLI sorts
+// filenames):
 //   0010_blog_posts.sql                 -> 00100_blog_posts.sql
 //   0010_support_tickets_admin_users.sql -> 00101_support_tickets_admin_users.sql
 // "00100_…" < "00101_…" < "0011_…" and > "0009_…", so nothing moves relative to

tests/a10-food-rls.test.ts

@@ -17,7 +17,7 @@ const mig = (n: string) => readFileSync(`supabase/migrations/${n}`, 'utf8');
 const rls0004 = mig('0004_rls.sql');
 const grocery0025 = mig('0025_fix_grocery_lists_rls.sql');
 const recipes0014 = mig('0014_core_platform.sql');
-const pantry0080 = mig('0080_food_household.sql');
+const pantry0080 = mig('00800_food_household.sql');
 const dining0104 = mig('0104_dining_out.sql');
 const hub0115 = mig('0115_meals_hub.sql');
 

tests/a11-documents-storage-rls.test.ts

@@ -13,7 +13,7 @@ function migration(file: string): string {
 }
 
 describe('A-11 documents DB table is RLS family-scoped', () => {
-  const sql = migration('0109_documents_favorite.sql');
+  const sql = migration('01090_documents_favorite.sql');
   it('enables RLS and scopes every op to is_family_member(family_id)', () => {
     expect(sql).toContain('alter table public.documents enable row level security');
     expect(sql).toContain('create policy documents_select on public.documents');

tests/migration-version-safety.test.ts

@@ -10,19 +10,35 @@ import {
 describe('Supabase migration filename safety', () => {
   const audit = auditMigrationVersions();
 
-  it('keeps the known historical collision set explicit', () => {
+  // The 17 historical collisions are gone — every group was renamed to a
+  // unique version preserving apply order. KNOWN_DUPLICATE_MIGRATIONS is now
+  // empty and must STAY empty: it is the escape hatch that let duplicates
+  // accumulate, and re-populating it to quiet a red audit is how they came
+  // back the first time.
+  it('has no duplicate versions at all, and no escape hatch left open', () => {
     expect(audit.unexpectedDuplicates).toEqual([]);
-    expect(Object.fromEntries(audit.duplicates.map(({ version, names }) => [version, names])))
-      .toEqual(KNOWN_DUPLICATE_MIGRATIONS);
+    expect(audit.duplicates).toEqual([]);
+    expect(KNOWN_DUPLICATE_MIGRATIONS).toEqual({});
   });
 
-  // 0275 is the newest file, so the next free number is 0276. The gap this
-  // leaves is deliberate and is not the collision this suite exists to catch:
-  // 0270 is reserved by the travel-confirmations branch and 0274 by the
-  // finance-receipts branch, both of which are held behind an unproven
-  // production migration ledger. Taking a reserved number to close the gap
-  // would hand whichever of them merges first a real collision; the audit
-  // takes max+1, so a hole costs nothing.
+  // Why this matters beyond tidiness: supabase_migrations.schema_migrations has
+  // a PRIMARY KEY on version, so it cannot record two files numbered 0010. That
+  // is a hard blocker for Supabase branching AND for the production ledger
+  // repair, both of which replay the history into that table.
+  it('gives every migration a version the ledger can actually record', () => {
+    const seen = new Map<string, string>();
+    for (const entry of audit.entries as { name: string; version: string }[]) {
+      expect(
+        seen.has(entry.version),
+        `${entry.version} used by ${seen.get(entry.version)} and ${entry.name}`,
+      ).toBe(false);
+      seen.set(entry.version, entry.name);
+    }
+  });
+
+  // The de-duplicated files carry a 5th digit that orders them within their
+  // generation (00100 and 00101 both live in 0010). nextVersion reads the first
+  // four digits, so those do not drag the next free number up to 1422.
   it('points new migrations at the next unused version', () => {
     expect(audit.nextVersion).toBe('0276');
   });