Skip to content

fix(OMN-15312): wrap vendored delegation reconcile migration 0009a in an explicit transaction - #2524

Merged
jonahgabriel merged 1 commit into
devfrom
jonah/omn-15312-migration-0009a-transaction-block
Jul 28, 2026
Merged

jonahgabriel merged 1 commit into
devfrom
jonah/omn-15312-migration-0009a-transaction-block

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Jul 28, 2026 •

Copy link
Copy Markdown
Collaborator

Problem (proven RED, live)

0009a_delegation_events_legacy_schema_reconcile.sql opens with three statements that are only legal inside a transaction block:

SET LOCAL lock_timeout = '5s';
SET LOCAL statement_timeout = '2min';
LOCK TABLE delegation_events IN ACCESS EXCLUSIVE MODE;

The forward-migration runner (omnibase_infra/scripts/run-forward-migrations.sh, node loop ~L418) applies node migrations with psql -v ON_ERROR_STOP=1 -f "$migration_file" — no -1/--single-transaction, and the file carries no BEGIN;. psql runs each statement in autocommit, so the SET LOCALs degrade to warnings and LOCK TABLE hard-errors. Container omnibase-infra-forward-migration exits 3.

Blast radius

This aborts the entire cold-lane bring-up path (OMN-13414, docs/runbooks/cold-lane-full-bringup.md) — the only documented recovery for the GC-reclaimed .201 dev lane (OMN-15190). While that lane is absent, every repo's occ-autobind/occ-companion-effect publish fails connection-refused against omninode-pc.tail75df5e.ts.net:19092 and cascades into occ-preflight/Receipt Gate org-wide. A migration-source defect was holding the org CI receipt path hostage.

Warm lanes were unaffected only because 0009a is already recorded in schema_migrations there and is skipped — so this was invisible until a cold rebuild.

Fix

Wrap the body in BEGIN; ... COMMIT;. This also supplies the atomicity the file's own header already claims ("retry without leaving a partial conversion"), which did not exist under autocommit.

All 80 node migrations under docker/migrations/forward/nodes/ were checked: zero contain BEGIN;. Statement-level autocommit is the established runner contract and 0009a was the sole violator, so this is fixed in the migration, not the shared runner — no blast radius onto the other 79.

Seam (two-repo, matched byte-for-byte)

The canonical source is omnimarket src/omnimarket/nodes/node_projection_delegation/migrations/; omnibase_infra vendors it via OMN-12559 auto-discovery. Pre-fix both copies hashed ae71fec2aaab6626…. Fixing only one would let the next vendor sync silently revert it, so both land together and both now hash:

b2e66b31070e676933f89b72876f05fdd3e812ad6a87d426c95290c6d0b9a489

The artifact that was executed in the RED/GREEN transcripts below is byte-identical (same sha256) to the file in both PRs — the test ran against the artifact that ships, not a surrogate.

RED / GREEN transcript (executed, .201 dev lane, cold omnidash_analytics)

RED — pristine file, scratch DB seeded with 0007_delegation_events.sql:

psql:/tmp/0009a_RED.sql:11: WARNING:  SET LOCAL can only be used in transaction blocks
psql:/tmp/0009a_RED.sql:12: WARNING:  SET LOCAL can only be used in transaction blocks
psql:/tmp/0009a_RED.sql:13: ERROR:  LOCK TABLE can only be used in transaction blocks
RED_EXIT=3

GREEN — patched file, same DB:

ALTER TABLE / ALTER TABLE / DO / DO / COMMIT
GREEN_EXIT=0

Idempotent re-apply (warm no-op, acceptance test 4):

ALTER TABLE / DO / DO / COMMIT
REAPPLY_EXIT=0

No SET LOCAL warnings remain post-fix.

Acceptance tests (OMN-15312)

# Test State
1 RED reproduced: cold DB apply exits nonzero on LOCK TABLE PASS (exit 3, above)
2 GREEN: patched apply exits 0, no SET LOCAL warnings PASS (exit 0, above)
3 Cold-lane e2e: deploy-runtime.sh --execute --cold --force clears migration preflight PENDING — needs this merged; see below
4 Warm no-op: re-apply is idempotent PASS (exit 0, above)

Acceptance test 3 cannot be satisfied pre-merge and this is not a gap in the evidence — it is the OMN-13415 gate working. deploy-runtime.sh asserts the deployed migration tree byte-matches the canonical clone at its committed SHA, so an uncommitted working-tree patch is correctly rejected:

FAIL: deployed migration tree ... is OUT OF SYNC with the canonical clone @ 416b7b9945b6 — 1 drift finding(s) (OMN-13415):
  - STALE/MODIFIED in deployed tree: nodes/node_projection_delegation/0009a_...sql (bytes differ from clone @ 416b7b99...)

That gate was not bypassed, skip-tokened, or weakened. Test 3 runs as the post-merge verification.

proof_class

replay-proven for tests 1/2/4 (RED and GREEN executed against the shipping artifact, verified by sha256 identity). code-only for test 3 until merge.

Refs: OMN-15190 (dev-lane GC-reclaim), OMN-13414 (cold-lane runbook), OMN-14974 (introduced the migration, #2507/#2509/#2510), OMN-12559 (vendor auto-discovery seam), OMN-15291 (adjacent runner advisory-lock defect — same file, different failure).

Evidence-Source: OCC#5275
Evidence-Ticket: OMN-15312
Omnimarket-Source-Ref: jonah/omn-15312-migration-0009a-transaction-block

… an explicit transaction

Vendored-copy half of the fix; canonical source is omnimarket
src/omnimarket/nodes/node_projection_delegation/migrations/ (OMN-12559
auto-discovery). Both copies are byte-identical (sha256 b2e66b31...) so the
next vendor sync is a no-op rather than a silent revert.

The forward-migration runner applies node migrations with psql -f in
autocommit, so LOCK TABLE errored with

  ERROR:  LOCK TABLE can only be used in transaction blocks

exiting 3 and aborting every cold-lane bring-up (OMN-13414), the only
documented recovery for the GC-reclaimed dev lane (OMN-15190).

Proven RED then GREEN against a cold omnidash_analytics on the .201 dev lane.
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 51 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b7277dfb-ae88-42d5-b205-feeda2cd46f2

📥 Commits

Reviewing files that changed from the base of the PR and between 416b7b9 and 3de9992.

📒 Files selected for processing (1)
  • docker/migrations/forward/nodes/node_projection_delegation/0009a_delegation_events_legacy_schema_reconcile.sql
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jonah/omn-15312-migration-0009a-transaction-block

Comment @coderabbitai help to get the list of available commands.

@jonahgabriel

Copy link
Copy Markdown
Collaborator Author

MERGE ORDER: this PR is blocked on omnimarket#1930 by design, not by a defect.

node-migration-sync compares docker/migrations/forward/nodes/ against omnimarket@dev, so while the canonical-source fix is unmerged this PR reports:

[sync-node-migrations] DRIFT: node_projection_delegation/0009a_delegation_events_legacy_schema_reconcile.sql

That is the vendoring seam working correctly — it is the same gate that would have caught an infra-only fix being silently reverted by the next sync.

Land omnimarket#1930 first; node-migration-sync here then goes green with no further change, because both files are already byte-identical (sha256 b2e66b31070e676933f89b72876f05fdd3e812ad6a87d426c95290c6d0b9a489). No rebase or re-vendor is required.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Hostile Reviewer — DEGRADED (informational)

Blocking findings (critical): 0
Total findings: 0
Models succeeded: none

Note: All reviewer models failed or were unavailable. Degraded results are informational during the pilot phase (OMN-8468/OMN-8524) and do not block merge. Error: all review endpoints [192.168.86.201:8000 192.168.86.201:8001 ] unreachable — preflight short-circuit (no models available)


Gate semantics (pilot phase)

Verdict Meaning Blocks merge?
passed No critical findings No
blocked CRITICAL findings found Yes
degraded All models unavailable (infra) No (pilot)

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8468/OMN-8524)

@jonahgabriel

Copy link
Copy Markdown
Collaborator Author

Temporary close/reopen by codex-a merge sweep to emit a fresh pull_request.reopened event after adding Omnimarket-Source-Ref. No head change; OCC binding remains valid.

@jonahgabriel

Copy link
Copy Markdown
Collaborator Author

Temporarily closing/reopening to refresh a scheduler-stuck CI run (30351396694 remained queued after cancellation request). No head change; OCC binding remains intact.

@jonahgabriel jonahgabriel reopened this Jul 28, 2026
@jonahgabriel
jonahgabriel merged commit c448c67 into dev Jul 28, 2026
319 of 374 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-15312-migration-0009a-transaction-block branch July 28, 2026 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant