Skip to content

fix(ci): gate cloud-cf-deploy on db migrations before code goes live - #11289

Merged
lalalune merged 1 commit into
developfrom
nubs/cf-deploy-migrate-gate
Jul 2, 2026
Merged

lalalune merged 1 commit into
developfrom
nubs/cf-deploy-migrate-gate

Conversation

@NubsCarson

Copy link
Copy Markdown
Member

Closes #11208.

The gap

Prod refreshes via a workflow_dispatch of cloud-cf-deploy.yml (Worker + both Pages projects), but the migrate-db job lived only in cloud-deploy-backend.yml. The two were fully decoupled: a prod refresh shipped new code against whatever schema prod happened to have — the review_status-outage class, primed for launch day. This implements fix option (a) from the issue: migrations run inside cloud-cf-deploy.yml, before any code goes live, fail-closed.

Job graph

Before (cloud-cf-deploy.yml — three root jobs, zero migration steps):

deploy-api      (no needs)
deploy-console  (no needs)
deploy-app      (no needs)

After:

migrate-db  (skipped on pull_request; environment = production on prod dispatch / main push, else staging)
  ├─> deploy-api      if: !cancelled() && event != PR && needs.migrate-db.result == 'success'
  ├─> deploy-console  if: !cancelled() && (result == 'success' || (PR && result == 'skipped'))
  └─> deploy-app      if: !cancelled() && (result == 'success' || (PR && result == 'skipped'))

Ordering guarantee (traced per trigger)

trigger migrate-db deploy-api deploy-console / deploy-app
dispatch env=production (the prod-refresh path) runs vs prod DB only after migrate success only after migrate success
dispatch env=staging runs vs staging DB gated gated
push develop runs vs staging DB gated gated
push main runs vs prod DB gated gated
migration FAILS ✗ skipped skipped
run cancelled mid-migration cancelled skipped (!cancelled()) skipped
pull_request skipped skipped (unchanged) runs (preview, unchanged behavior)
  • No-op still deploys: db:cloud:migrate (migrate-with-diagnostics.ts) is journal-based — zero pending migrations logs [db:migrate] pending migrations: 0 and exits 0, satisfying the gate.
  • No failure path deploys: every deploy job requires needs.migrate-db.result == 'success' for non-PR events; there is no always()-style bypass to a deploy step.
  • Missing DB secret fails loudly (mirrors the cloud-deploy-backend fail-fast) instead of silently skipping — the pre-2026-05-20 silent-skip regression class.
  • Cross-workflow serialization: a develop push fires both this workflow and cloud-deploy-backend.yml. Both migrate-db jobs now share a repo-wide job-level concurrency group cloud-db-migrate-<env> (cancel-in-progress: false), so the migrator never runs concurrently against the same database. At most 2 jobs can ever occupy a group (each workflow already serializes itself per-ref), so the queued job is never superseded-cancelled — no reintroduction of the G3 perma-cancel pattern.
  • migrate-db runs on GitHub-hosted ubuntu-latest deliberately (mirrors cloud-deploy-backend) so migrations don't inherit the shared self-hosted fleet's failure modes.

Verification

  • actionlint 1.7.12 → exit 0 on both modified workflows.
  • Negative control: actionlint correctly flags a synthetic workflow with a bad needs: ref and bad needs.* expression, proving the linter exercises exactly the constructs this PR touches.
  • YAML parse + programmatic job-graph dump (above) confirms needs: migrate-db on all three deploy jobs.
  • Note for ops: migrate-db uses the production GitHub environment on prod dispatches, so any environment protection rules (required reviewers) now also gate the prod refresh — same behavior cloud-deploy-backend migrations already have.

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9490a34b-62c8-4806-bb18-e6722e35b6f5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch nubs/cf-deploy-migrate-gate

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lalalune
lalalune merged commit 97c1737 into develop Jul 2, 2026
35 of 39 checks passed
@lalalune
lalalune deleted the nubs/cf-deploy-migrate-gate branch July 2, 2026 08:09
@lalalune

lalalune commented Jul 2, 2026

Copy link
Copy Markdown
Member

Maintainer review (deploy pipeline): approved with one pushed hardening commit. Verified the gate table trigger-by-trigger — every deploy job needs migrate-db with fail-closed semantics, the ONE allowed skip (PR previews, migrate-db skipped, staging-API frontend builds) is correctly scoped via !cancelled() + result=='skipped', and the cross-workflow serialization group matches on both sides (inline ternary here == determine-env output in cloud-deploy-backend, both yielding cloud-db-migrate-{production,staging}). The missing-secret fail-fast closes the silent-skip landmine class. Pushed: the new migrate-db job used bun canary — the exact regression class #11235 just pinned the sibling jobs off after it stranded two prod deploys; pinned to latest with the same rationale. This directly fixes the skew window we lived through on the #11150 promote (Worker deploying for 2h while migrate-db starved). Merging.

lalalune added a commit that referenced this pull request Jul 2, 2026
…deploy job (#10839) (#11318)

#11289's new migrate-db job — the schema gate every deploy now depends on —
was the one job left on bun canary after #11235 pinned the rest of this
workflow following the link-phase hang that stranded two prod deploys
(runs 28552075615, 28569068598). A canary regression in the gate would
fail-closed-block every deploy.

Co-authored-by: moon <thefutureisaweirdplace@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the ci label Jul 2, 2026
@claude

claude Bot commented Jul 2, 2026 •

Copy link
Copy Markdown
Contributor

Claude encountered an error —— View job


I'll analyze this and get back to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

deploy/CI: prod-refresh (cloud-cf-deploy dispatch) ships code WITHOUT migrations — decoupled from migrate-db (G2, launch-day landmine)

2 participants