Skip to content

fix(OMN-15312): wrap delegation reconcile migration 0009a in an explicit transaction - #1930

Merged
jonahgabriel merged 1 commit into
devfrom
jonah/omn-15312-migration-0009a-transaction-block
Jul 28, 2026
Merged

jonahgabriel merged 1 commit into
devfrom
jonah/omn-15312-migration-0009a-transaction-block

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Jul 28, 2026 •

Copy link
Copy Markdown
Contributor

Problem (proven RED, live)

0009a_delegation_events_legacy_schema_reconcile.sql opens with three statements that are only legal inside a transaction block:

SET LOCAL lock_timeout = '5s';
SET LOCAL statement_timeout = '2min';
LOCK TABLE delegation_events IN ACCESS EXCLUSIVE MODE;

The forward-migration runner (omnibase_infra/scripts/run-forward-migrations.sh, node loop ~L418) applies node migrations with psql -v ON_ERROR_STOP=1 -f "$migration_file" — no -1/--single-transaction, and the file carries no BEGIN;. psql runs each statement in autocommit, so the SET LOCALs degrade to warnings and LOCK TABLE hard-errors. Container omnibase-infra-forward-migration exits 3.

Blast radius

This aborts the entire cold-lane bring-up path (OMN-13414, docs/runbooks/cold-lane-full-bringup.md) — the only documented recovery for the GC-reclaimed .201 dev lane (OMN-15190). While that lane is absent, every repo's occ-autobind/occ-companion-effect publish fails connection-refused against omninode-pc.tail75df5e.ts.net:19092 and cascades into occ-preflight/Receipt Gate org-wide. A migration-source defect was holding the org CI receipt path hostage.

Warm lanes were unaffected only because 0009a is already recorded in schema_migrations there and is skipped — so this was invisible until a cold rebuild.

Fix

Wrap the body in BEGIN; ... COMMIT;. This also supplies the atomicity the file's own header already claims ("retry without leaving a partial conversion"), which did not exist under autocommit.

All 80 node migrations under docker/migrations/forward/nodes/ were checked: zero contain BEGIN;. Statement-level autocommit is the established runner contract and 0009a was the sole violator, so this is fixed in the migration, not the shared runner — no blast radius onto the other 79.

Seam (two-repo, matched byte-for-byte)

The canonical source is omnimarket src/omnimarket/nodes/node_projection_delegation/migrations/; omnibase_infra vendors it via OMN-12559 auto-discovery. Pre-fix both copies hashed ae71fec2aaab6626…. Fixing only one would let the next vendor sync silently revert it, so both land together and both now hash:

b2e66b31070e676933f89b72876f05fdd3e812ad6a87d426c95290c6d0b9a489

The artifact that was executed in the RED/GREEN transcripts below is byte-identical (same sha256) to the file in both PRs — the test ran against the artifact that ships, not a surrogate.

RED / GREEN transcript (executed, .201 dev lane, cold omnidash_analytics)

RED — pristine file, scratch DB seeded with 0007_delegation_events.sql:

psql:/tmp/0009a_RED.sql:11: WARNING:  SET LOCAL can only be used in transaction blocks
psql:/tmp/0009a_RED.sql:12: WARNING:  SET LOCAL can only be used in transaction blocks
psql:/tmp/0009a_RED.sql:13: ERROR:  LOCK TABLE can only be used in transaction blocks
RED_EXIT=3

GREEN — patched file, same DB:

ALTER TABLE / ALTER TABLE / DO / DO / COMMIT
GREEN_EXIT=0

Idempotent re-apply (warm no-op, acceptance test 4):

ALTER TABLE / DO / DO / COMMIT
REAPPLY_EXIT=0

No SET LOCAL warnings remain post-fix.

Acceptance tests (OMN-15312)

# Test State
1 RED reproduced: cold DB apply exits nonzero on LOCK TABLE PASS (exit 3, above)
2 GREEN: patched apply exits 0, no SET LOCAL warnings PASS (exit 0, above)
3 Cold-lane e2e: deploy-runtime.sh --execute --cold --force clears migration preflight PENDING — needs this merged; see below
4 Warm no-op: re-apply is idempotent PASS (exit 0, above)

Acceptance test 3 cannot be satisfied pre-merge and this is not a gap in the evidence — it is the OMN-13415 gate working. deploy-runtime.sh asserts the deployed migration tree byte-matches the canonical clone at its committed SHA, so an uncommitted working-tree patch is correctly rejected:

FAIL: deployed migration tree ... is OUT OF SYNC with the canonical clone @ 416b7b9945b6 — 1 drift finding(s) (OMN-13415):
  - STALE/MODIFIED in deployed tree: nodes/node_projection_delegation/0009a_...sql (bytes differ from clone @ 416b7b99...)

That gate was not bypassed, skip-tokened, or weakened. Test 3 runs as the post-merge verification.

proof_class

replay-proven for tests 1/2/4 (RED and GREEN executed against the shipping artifact, verified by sha256 identity). code-only for test 3 until merge.

Refs: OMN-15190 (dev-lane GC-reclaim), OMN-13414 (cold-lane runbook), OMN-14974 (introduced the migration, #2507/#2509/#2510), OMN-12559 (vendor auto-discovery seam), OMN-15291 (adjacent runner advisory-lock defect — same file, different failure).

Evidence-Source: OCC#5275
Evidence-Ticket: OMN-15312

…cit transaction

The forward-migration runner applies node migrations with psql -f in
autocommit (no --single-transaction), so the SET LOCAL statements degraded
to warnings and LOCK TABLE hard-errored with

  ERROR:  LOCK TABLE can only be used in transaction blocks

exiting 3 and aborting every cold-lane bring-up (OMN-13414 path), which is
the only documented recovery for the GC-reclaimed dev lane (OMN-15190).
While that lane is down, occ-autobind/occ-companion-effect publishes fail
org-wide against omninode-pc:19092 and cascade into the Receipt Gate.

Wrapping the body in BEGIN/COMMIT also supplies the atomicity the file header
already claims (retry without leaving a partial conversion), which did not
exist under autocommit. No other node migration uses BEGIN; this file was the
sole one relying on a transaction context the runner never provided.

Proven RED then GREEN against a cold omnidash_analytics on the .201 dev lane:
RED exit 3 (LOCK TABLE ...), GREEN exit 0, and a second apply exit 0
(idempotent).
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 51 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 03c31dc9-52de-4f4f-931b-7d87426f8e7c

📥 Commits

Reviewing files that changed from the base of the PR and between ffc7736 and 7e415d1.

📒 Files selected for processing (1)
  • src/omnimarket/nodes/node_projection_delegation/migrations/0009a_delegation_events_legacy_schema_reconcile.sql
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jonah/omn-15312-migration-0009a-transaction-block

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

✅ Architectural Review — PASSED

Errors: 0
Warnings: 0

What this checks

Rule Description
ARCH-TOPIC-001 No hardcoded Kafka topic strings in handler code
ARCH-DI-001 No event_bus=None bypass
ARCH-DI-002 No direct Handler instantiation outside workflow_runner/adapters
ARCH-DI-003 No reinvented DI containers
ARCH-TOPIC-002 contract.yaml topics follow `onex.{cmd

No architectural violations found.

Static architectural lint — no model inference (OMN-14176).

@github-actions

Copy link
Copy Markdown

✅ Hostile Reviewer — PASSED

Blocking findings (critical/error): 0
Total findings: 0
Models succeeded: qwen3-review,qwen3-review-b


Gate semantics

Verdict Meaning Blocks merge?
passed >=2 models succeeded, no critical/error findings No
blocked CRITICAL or ERROR findings found Yes
degraded Fewer than 2 models succeeded (infra unavailable/timeout) Yes (OMN-15110)

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8524)

@jonahgabriel
jonahgabriel merged commit 6dfe677 into dev Jul 28, 2026
151 of 164 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-15312-migration-0009a-transaction-block branch July 28, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant