Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 19 additions & 77 deletions .github/workflows/cloud-vm-migrate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,7 @@ on:
required: true
default: staging
type: choice
options:
- staging
- production
options: [staging, production]
cleanup_iroh_challenges:
description: Prune expired, consumed, and duplicate Iroh registration challenges after migration
required: false
Expand All @@ -19,7 +17,6 @@ on:

permissions:
contents: read
id-token: write

concurrency:
group: cloud-vm-migrate-${{ inputs.target }}
Expand All @@ -35,24 +32,15 @@ jobs:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
# Staging must rehearse the dispatched branch before it can merge.
# Production remains pinned to reviewed main for both preflight and
# the prerequisite staging migration.
ref: ${{ inputs.target == 'production' && 'refs/heads/main' || github.sha }}

- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2

- name: Install dependencies
run: bun install --frozen-lockfile

- name: Cloud VM migration preflight
run: bun run cloud-vm:preflight -- --schema-only .

- name: Rehearse Iroh cleanup on isolated Postgres
if: ${{ inputs.cleanup_iroh_challenges }}
# db:test discovers every CMUX_DB_TEST suite, including
# iroh-challenge-cleanup-db.test.ts (audit, apply, and rerun).
run: bun run cloud-vm:preflight -- .

migrate-staging:
Expand All @@ -64,54 +52,31 @@ jobs:
run:
working-directory: web
env:
AWS_REGION: ${{ vars.AWS_REGION || 'us-west-2' }}
AWS_ROLE_ARN: ${{ secrets.AWS_MIGRATION_ROLE_ARN }}
CMUX_CLOUD_VM_ENV_SOURCE: process
CMUX_DB_DRIVER: aws-rds-iam
CMUX_DB_SSL_REJECT_UNAUTHORIZED: ${{ vars.CMUX_DB_SSL_REJECT_UNAUTHORIZED || 'true' }}
PGDATABASE: ${{ vars.PGDATABASE }}
PGHOST: ${{ vars.PGHOST }}
PGPORT: ${{ vars.PGPORT || '5432' }}
PGUSER: ${{ vars.PGUSER }}
CMUX_DB_DRIVER: url
DATABASE_URL: ${{ secrets.DATABASE_URL }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ inputs.target == 'production' && 'refs/heads/main' || github.sha }}

- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2

- name: Install dependencies
run: bun install --frozen-lockfile

- name: Prepare AWS web identity credentials
- name: Validate PlanetScale credentials
run: |
set -euo pipefail
if [ -z "${AWS_ROLE_ARN:-}" ]; then
echo "::error::Missing cloud-vm-staging secret AWS_MIGRATION_ROLE_ARN"
if [ -z "${DATABASE_URL:-}" ]; then
echo "::error::Missing cloud-vm-staging secret DATABASE_URL"
exit 1
fi
for key in PGHOST PGPORT PGUSER PGDATABASE AWS_REGION; do
if [ -z "${!key:-}" ]; then
echo "::error::Missing cloud-vm-staging variable $key"
exit 1
fi
done
token_file="$RUNNER_TEMP/aws-web-identity-token"
curl -fsSL \
-H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=sts.amazonaws.com" \
| jq -e -r '.value' > "$token_file"
echo "AWS_WEB_IDENTITY_TOKEN_FILE=$token_file" >> "$GITHUB_ENV"
echo "AWS_ROLE_ARN=$AWS_ROLE_ARN" >> "$GITHUB_ENV"
echo "AWS_REGION=$AWS_REGION" >> "$GITHUB_ENV"
export AWS_WEB_IDENTITY_TOKEN_FILE="$token_file"
aws sts get-caller-identity --no-cli-pager >/dev/null

case "$DATABASE_URL" in
postgres://*|postgresql://*) ;;
*) echo "::error::DATABASE_URL must use the PostgreSQL URL scheme"; exit 1 ;;
esac
- name: Apply staging migration
run: bun run cloud-vm:migrate -- staging

- name: Clean up staging Iroh challenges
if: ${{ inputs.cleanup_iroh_challenges }}
run: bun run cloud-vm:cleanup-iroh -- staging --apply
Expand All @@ -125,54 +90,31 @@ jobs:
run:
working-directory: web
env:
AWS_REGION: ${{ vars.AWS_REGION || 'us-west-2' }}
AWS_ROLE_ARN: ${{ secrets.AWS_MIGRATION_ROLE_ARN }}
CMUX_CLOUD_VM_ENV_SOURCE: process
CMUX_DB_DRIVER: aws-rds-iam
CMUX_DB_SSL_REJECT_UNAUTHORIZED: ${{ vars.CMUX_DB_SSL_REJECT_UNAUTHORIZED || 'true' }}
PGDATABASE: ${{ vars.PGDATABASE }}
PGHOST: ${{ vars.PGHOST }}
PGPORT: ${{ vars.PGPORT || '5432' }}
PGUSER: ${{ vars.PGUSER }}
CMUX_DB_DRIVER: url
DATABASE_URL: ${{ secrets.DATABASE_URL }}
steps:
- name: Checkout
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: refs/heads/main

- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2

- name: Install dependencies
run: bun install --frozen-lockfile

- name: Prepare AWS web identity credentials
- name: Validate PlanetScale credentials
run: |
set -euo pipefail
if [ -z "${AWS_ROLE_ARN:-}" ]; then
echo "::error::Missing cloud-vm-production secret AWS_MIGRATION_ROLE_ARN"
if [ -z "${DATABASE_URL:-}" ]; then
echo "::error::Missing cloud-vm-production secret DATABASE_URL"
exit 1
fi
for key in PGHOST PGPORT PGUSER PGDATABASE AWS_REGION; do
if [ -z "${!key:-}" ]; then
echo "::error::Missing cloud-vm-production variable $key"
exit 1
fi
done
token_file="$RUNNER_TEMP/aws-web-identity-token"
curl -fsSL \
-H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" \
"${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=sts.amazonaws.com" \
| jq -e -r '.value' > "$token_file"
echo "AWS_WEB_IDENTITY_TOKEN_FILE=$token_file" >> "$GITHUB_ENV"
echo "AWS_ROLE_ARN=$AWS_ROLE_ARN" >> "$GITHUB_ENV"
echo "AWS_REGION=$AWS_REGION" >> "$GITHUB_ENV"
export AWS_WEB_IDENTITY_TOKEN_FILE="$token_file"
aws sts get-caller-identity --no-cli-pager >/dev/null

case "$DATABASE_URL" in
postgres://*|postgresql://*) ;;
*) echo "::error::DATABASE_URL must use the PostgreSQL URL scheme"; exit 1 ;;
esac
- name: Apply production migration
run: bun run cloud-vm:migrate -- production

- name: Clean up production Iroh challenges
if: ${{ inputs.cleanup_iroh_challenges }}
run: bun run cloud-vm:cleanup-iroh -- production --apply
4 changes: 4 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# cmux agent notes

## Database provider

cmux Cloud uses PlanetScale PostgreSQL, organization `cmux`, database `cmux-prod`. Branches are `main` (production), `staging`, and `development`. Vercel uses a PlanetScale `DATABASE_URL`; migration jobs use `DATABASE_URL` and `bun run cloud-vm:migrate -- <target>`. Aurora/RDS IAM and AWS migration-role instructions are retired. AWS KMS access for coderouter encryption is separate from database access. For PlanetScale CLI work, run `pscale auth check --format json` and pass `--org cmux` plus the confirmed branch.

## Setup

`./scripts/setup.sh` initializes submodules, builds GhosttyKit, and installs the pbxproj normalization pre-commit hook.
Expand Down
2 changes: 2 additions & 0 deletions docs/cloud-vm-backend-rollout-todo.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
> **Provider update (2026-09-16):** This rollout checklist is historical. cmux Cloud now uses PlanetScale PostgreSQL (`cmux` / `cmux-prod`, `main` production, `staging` staging, `development` development). Do not execute the old AWS Aurora/RDS steps below. Use `skills/cmux-backend/references/cloud-vm-control-plane.md` and the current `cloud-vm-migrate.yml` workflow.

# Cloud VM Backend Rollout Todo

This is the scoped todo list for making the Cloud VM backend production-ready with application logic running in the existing Vercel `manaflow/cmux` project.
Expand Down
2 changes: 1 addition & 1 deletion docs/iroh-v2/ACCEPTANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Capture continuous video in bounded segments, dense frame splits around launches
| A13 | Every HTTP status and socket error has explicit client behavior. | Table-driven behavior tests cover 101, successful 2xx/204 as applicable, redirects/unexpected content, malformed response, 400 validation, 401 expiry/auth, 403 permission, 404 missing, 409 identity/revision conflict, 413 size, 429 with retry delay, storage full/quota, unsupported schema, 5xx, timeout, cancellation and socket-close codes. Retry only retryable work and reuse mutation request IDs. Never sign out on a transient server error. |
| A14 | Generous user-based limits and finite buffers protect resources without charging terminal packets or internal calls. | Separate operation buckets; registration 300/hour burst 50; two phones/five Macs fit initial enrollment. 16 KiB inbound frame, 64 KiB snapshot chunks; 1,024 messages/2 MiB per connection, 4,096/8 MiB per user, including transport-buffered bytes. Slow subscriber safely resyncs without losing revocation. No delivered-message or local IROH-attempt quota. |
| A15 | Drizzle schema activation gates traffic; immutable migrations; DB usage constraints reject atomically. | Real Miniflare/workerd persistence, populated upgrade, repeated activation, partial failure rollback, unsupported-schema refusal, compatible code rollback, storage guard and concurrent quota tests. No successful migration marker on failure. Offline devices do not expire from inactivity. |
| A16 | Shared PostgreSQL owns global EndpointID reservation and genuinely shared records; team SQLite owns team-local state. PlanetScale is the recommended target after the Aurora migration. | Unique constraint and recovery tests, one authoritative home per record, no periodic global lookup on relay renewal, plus a migration receipt and live uniqueness check before cutover. Per-user product storage allowance and team physical bounds are explicit and tested. |
| A16 | Shared PostgreSQL owns global EndpointID reservation and genuinely shared records; team SQLite owns team-local state. PlanetScale is the current shared PostgreSQL provider. | Unique constraint and recovery tests, one authoritative home per record, no periodic global lookup on relay renewal, plus a migration receipt and live uniqueness check before cutover. Per-user product storage allowance and team physical bounds are explicit and tested. |
| A17 | Dashboard uses verified team directory/settings/revocations and filtered ordered changes. | Two-team/two-user browser tests; permission-filtered list and revisions, changed metadata, revocation, cursor-gap snapshot and team-switch cache isolation. Same operations and limit policy as native clients. |
| A18 | Complete bounded observability works on deployed backends and relays. | Query success/denial/rate-limit/error completion events for HTTP and sockets plus enrollment, renewal, DB, revision, socket and deployment events. Confirm Cloudflare metrics, Axiom ingestion and Sentry test exception. Sink outage never blocks response, memory/time are bounded and loss is visible. Durable authority audit survives export failure. No credentials/bodies/terminal bytes/SQL parameters in records. |
| A19 | Relays validate 30-minute credentials locally using public verification keys and correct audience. | Real relay rejects expired/forged/wrong-audience credentials; key rotation supports still-valid issuances; no backend call per handshake. Logs cover result and aggregate bytes. Load test per-user relay guard separately from exact cross-team backend counters. |
Expand Down
2 changes: 2 additions & 0 deletions docs/iroh-v2/design/IROH-DECISIONS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# IROH v2 decisions

> Provider update (2026-09-16): cmux Cloud now uses PlanetScale Postgres. Aurora references below describe the historical design and do not authorize AWS database operations. Use `skills/cmux-backend/references/cloud-vm-control-plane.md` for the current database workflow.

Updated 15 September 2026, revision 23. Accepted directions are recorded here; unresolved implementation details are listed at the end. Revision 23 clarifies that new Macs must support older iOS apps; new iOS apps only need to support new Macs.

## Backend and scope
Expand Down
8 changes: 4 additions & 4 deletions docs/presence-service.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ GET /v1/presence/subscribe -> forward w/ verified team ------> WS (hibernation)

- **State machine** (`src/core.ts`): pure and synchronous. A team's presence
is a map of app instances keyed by `(deviceId, tag)`, the same identity as
the Aurora registry (`devices.device_uuid` + `device_app_instances.tag`).
the PlanetScale Postgres registry (`devices.device_uuid` + `device_app_instances.tag`).
Online is set by a heartbeat; offline is an explicit event, produced either
by a goodbye heartbeat (`stopping: true`, clean shutdown) or by the DO alarm
when heartbeats stop.
Expand Down Expand Up @@ -79,8 +79,8 @@ GET /v1/presence/subscribe -> forward w/ verified team ------> WS (hibernation)
## Migrations and durability

Presence is deliberately ephemeral. The durable source of device identity is
the Aurora `devices` / `device_app_instances` registry
(https://github.com/manaflow-ai/cmux/pull/5626); this service adds no Aurora
the PlanetScale Postgres `devices` / `device_app_instances` registry
(https://github.com/manaflow-ai/cmux/pull/5626); this service adds no Postgres
columns and therefore ships no Drizzle migration. DO storage keeps the live
instance map plus a 24h offline tail for "last seen", pruned by the same
alarm, and the durable per-device owner pins. Losing the service's storage
Expand All @@ -91,7 +91,7 @@ The service's own schema story is the `[[migrations]]` block in
`wrangler.toml`: Durable Object class migrations are applied by
`wrangler deploy` in the deploy-on-push workflow, atomically with the code, so
storage classes can never lag the deployed code the way the prod Aurora
migrations once lagged the web deploy. If presence ever does need an Aurora
migrations once lagged the web deploy. If presence ever does need a Postgres
column, the Drizzle migration must land in `web/db/migrations` and is applied
by the `web-db-migrations` CI job and the cloud-vm migrate workflow
(`.github/workflows/cloud-vm-migrate.yml`), per the cloud-vm-ops runbook.
Expand Down
2 changes: 2 additions & 0 deletions plans/feat-do-device-list/DESIGN.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Local-first sync for cmux (and the iOS device list as its first consumer)

> Provider update (2026-09-16): cmux Cloud now uses PlanetScale Postgres. Aurora references below describe the historical design and do not authorize AWS database operations. Use `skills/cmux-backend/references/cloud-vm-control-plane.md` for the current database workflow.

Status: proposed. Phase 1 ships the generic sync substrate plus the device-list
consumer behind a flag, with the Aurora registry kept intact as a fallback.

Expand Down
4 changes: 2 additions & 2 deletions skills/cmux-backend/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ description: "Backend TypeScript and Cloud VM development rules for cmux. Use wh
- Plain TypeScript is for trivial data shapes, constants, config files, frontend React, and small glue where Effect would add ceremony without improving failure handling.
- Cloud VM backend logic stays in Vercel route handlers and Effect services backed by Postgres. Do not reintroduce Rivet or a raw actor protocol unless a later architecture doc explicitly changes the control plane.
- Postgres is the source of truth for VM lifecycle, active VM limits, idempotency, and usage events.
- Production and staging Cloud VM Postgres use the Vercel Marketplace AWS Aurora PostgreSQL OIDC/RDS IAM path, with runtime env `CMUX_DB_DRIVER=aws-rds-iam`, `AWS_ROLE_ARN`, `AWS_REGION`, `PGHOST`, `PGPORT`, `PGUSER`, `PGDATABASE`.
- Run production/staging migrations with `bun db:migrate:aws-rds-iam`; never from Vercel build or route startup. Local dev keeps the `CMUX_PORT`-derived Docker Postgres path from `bun dev`.
- Production and staging Cloud VM Postgres use PlanetScale PostgreSQL database `cmux-prod` in organization `cmux`. The runtime reads `DATABASE_URL` with `CMUX_DB_DRIVER=url`; migration jobs use the protected `DATABASE_URL` secret. AWS credentials are not database credentials.
- Run production/staging migrations with `bun run cloud-vm:migrate -- staging` followed by `-- production`; never from Vercel build or route startup. Local dev keeps the `CMUX_PORT`-derived Docker Postgres path from `bun dev`.
- Cloud VM create pricing gates use Stack Auth team payment items when enabled.

## Secrets
Expand Down
6 changes: 3 additions & 3 deletions skills/cmux-backend/references/cloud-vm-control-plane.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ Cloud VM backend logic lives in Vercel route handlers and Effect services. Reque

## Migrations

Production and staging: `bun db:migrate:aws-rds-iam`. Never run Drizzle migrations from Vercel build or route startup; that makes deploy behavior non-deterministic and couples app availability to schema mutation. Local development keeps the `CMUX_PORT`-derived Docker Postgres path from `bun dev`.
Production and staging: `bun run cloud-vm:migrate -- staging` followed by `-- production`. Never run Drizzle migrations from Vercel build or route startup; that makes deploy behavior non-deterministic and couples app availability to schema mutation. Local development keeps the `CMUX_PORT`-derived Docker Postgres path from `bun dev`.

## AWS RDS IAM runtime
## PlanetScale PostgreSQL runtime

Production and staging use the Vercel Marketplace AWS Aurora PostgreSQL OIDC/RDS IAM path with `CMUX_DB_DRIVER=aws-rds-iam`, `AWS_ROLE_ARN`, `AWS_REGION`, `PGHOST`, `PGPORT`, `PGUSER`, `PGDATABASE`. Do not invent parallel env names for the same settings; each new name is another migration and deploy surface.
Production and staging use PlanetScale Postgres database `cmux-prod` in organization `cmux`. Production is branch `main`; staging is `staging`; development is `development`. The application reads `DATABASE_URL` and uses `CMUX_DB_DRIVER=url`. Migration jobs use the protected `DATABASE_URL` secret. AWS credentials are not database credentials.

## Pricing and active limits

Expand Down
4 changes: 2 additions & 2 deletions web/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
"stripe:backfill-subscriptions": "bun scripts/stripe/backfill-subscriptions-from-stripe.ts",
"billing:backfill-founders-lockout": "bun scripts/backfill-founders-lockout.ts",
"cloud-vm:env:audit": "bun scripts/cloud-vm/audit-vercel-env.mjs",
"cloud-vm:migrate": "bun scripts/cloud-vm/migrate-vercel-aurora-iam.mjs",
"cloud-vm:migrate": "bun scripts/cloud-vm/migrate-planetscale.mjs",
"cloud-vm:cleanup-iroh": "bun scripts/cloud-vm/cleanup-iroh-challenges.ts",
"cloud-vm:preflight": "bash scripts/cloud-vm/verify-migration-preflight.sh",
"cloud-vm:smoke": "bun scripts/cloud-vm/smoke-vm-api.mjs",
Expand All @@ -33,7 +33,7 @@
"db:down": "bash scripts/db-local.sh down",
"db:generate": "bunx drizzle-kit generate --config drizzle.config.ts",
"db:migrate": "bash scripts/db-local.sh migrate",
"db:migrate:aws-rds-iam": "bun scripts/migrate-aws-rds-iam.ts",
"db:migrate:planetscale": "bun scripts/cloud-vm/migrate-planetscale.mjs",
"db:ready": "bash scripts/db-local.sh ready",
"db:reset": "bash scripts/db-local.sh reset",
"db:status": "bash scripts/db-local.sh status",
Expand Down
Loading
Loading