docs(juicefs): L6 design — scoped meta role, and why the pooler option was wrong here

[POWERFULMOVES]

L6 cross-node design, plus the documentation review that overturned my own first recommendation. The reasoning is the durable part — this exists so the pooler idea isn't re-proposed on architectural instinct.

Blocker mechanism

supabase-db sits on pmoves_api + pmoves_data, both internal: true. Docker installs no DNAT for internal-only networks, so published ports are recorded but never plumbed — b850:5432 measured unreachable from z890 and nano-1. The in-repo NATS precedent (docker-compose.yml:2906) already solves exactly this by multi-homing onto pmoves_external.

JuiceFS documents the requirement plainly: "ensure that all nodes has access to the Metadata Engine." There's no way around exposing something.

⚠️ Correction — the pooler recommendation was wrong for this deployment

I first recommended exposing supavisor rather than Postgres, on the architectural grounds that it's the purpose-built front door. Reviewing official docs and then the live deployment overturned it:

Two official sources conflict, and the one I'd cited doesn't apply here:

Source	Says
Supabase platform docs	5432=session, 6543=transaction; "Transaction mode does not support prepared statements"
Supavisor project docs	"Configure the mode_type on the user" — mode is per-user, not per-port

The port convention is hosted-platform behavior. Self-hosted resolves mode from the user row, and this stack sets POOLER_POOL_MODE=transaction. So "use 5432 for session mode" does not hold here.

JuiceFS's docs never mention poolers and point to the lib/pq driver, which uses the extended query protocol (prepared statements) — transaction mode would break it.

Decisive against the actual deployment:

_supavisor.tenants : 0 rows      (z890 AND B850)
_supavisor.users   : 0 rows

The pooler is Up (healthy) answering /api/health 204s — but has never been provisioned to pool anything; its health check probes the API, not tenant existence. Exposing it would publish a port that cannot serve a connection. (Same shape as Archon reporting container-healthy while ready:false for three days.)

Decision

Expose supabase-db multi-homed onto pmoves_external, bound to the tailnet interface only — not 0.0.0.0. The NATS precedent defaults to 0.0.0.0 on the grounds that NATS is credential-guarded; that's weaker for a database, and weaker still while the admin credential is un-rotated. Supavisor stays a future lane (provision tenant + session-mode user + verify lib/pq), not a step in this one.

Ordering refinement

Scoped role → rotate → expose (approved). Once JuiceFS authenticates as a single-schema role, the pending admin rotation no longer touches the mount at all — it shrinks the rotation's blast radius rather than widening it, and it's cheap/reversible where rotation is expensive (~27 consumers, one window).

JuiceFS currently authenticates as supabase_admin (rolsuper=t, rolcreaterole=t) — the same credential leaked for ~11 days and still un-rotated. Exposing a superuser surface is materially different from exposing a single-schema one.

Migration status

The SQL is included inline, reviewed and ready — DML on one schema, NOLOGIN until CHIT delivers a password, no CREATE/BYPASSRLS/SUPERUSER/CREATEROLE, idempotent, applied via make -C pmoves supa-migrate (no raw psql).

It is not committed to pmoves/supabase/migrations/ because that path is protected and needs an operator-set KNOWN_ROAD=migrations:handoff:<this file>. An agent must not self-grant a protected-path bypass — that gate is the point.

🤖 Generated with Claude Code

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8a88dc19-be88-4859-8a46-15d5b7e154be

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9da6137f99

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[POWERFULMOVES]

Reviewed both findings against source. Both confirmed.

1. The migration target and the privilege it needs

Verified in pmoves/Makefile:

• supa-migrate applies every file with psql -v ON_ERROR_STOP=1 -U postgres
• supabase-bootstrap uses admin_psql → psql -U supabase_admin, with the reason stated inline: "supabase_admin is the self-hosted superuser; the plain 'postgres' role is NOT superuser and lacks CREATE on public."

So Step 1 as written points at the target that cannot do the work.

Worth noting the implementation PR (#2614) already anticipates this — its DO block tests pg_has_role(current_user,'supabase_admin','USAGE') and skips rather than aborting. That is the right instinct for not taking down a shared migration run, but it means Step 1 does not fail loudly either: it reports success and creates nothing. Combined with the bootstrap ledger recording the file as applied, the skip is permanent. I have written that up in full on #2614.

For this doc the fix is small: name supabase-bootstrap (or an explicit supabase_admin psql invocation) in Step 1, and say what a correct run looks like — otherwise an operator follows the steps, sees no error, and reasonably concludes the role exists.

2. Cut over before rotating — confirmed

pmoves/scripts/juicefs-cross-node-setup.sh:58 builds the metadata URL unconditionally:

META_URL="postgres://supabase_admin@${JUICEFS_HOST}:${DB_PORT}/postgres?search_path=juicefs_meta&sslmode=disable"

No branch, no variable — every cross-node join authenticates as the superuser. And #2614 creates juicefs_meta as NOLOGIN, with the live mount repoint explicitly deferred.

Follow Steps 1–3 in order and you rotate or expose a credential that is still the only one in use. The scoped role exists but nothing consumes it, so the exposure surface is unchanged while the doc reads as though it has been reduced. That is the worst shape for a security handoff — it retires the operator's attention on a problem that is still live.

Sequencing that holds: create the scoped role → grant LOGIN → repoint juicefs-cross-node-setup.sh and the compose defaults → verify the mount runs as juicefs_meta → only then rotate or expose. Rotation is the last step, not the third.

Verdict

The analysis in this doc is good — particularly the section on why the pooler option was wrong here, which is the kind of negative result that saves the next person from re-deriving it. Both findings are about sequencing and target-naming rather than the design.

Holding with #2614 so the pair lands together and in order.

[POWERFULMOVES]

Root-cause + remediation analysis posted on #2614 (the implementation side), since the fix lands there. Two things from it change what this design doc should say:

1. The node-conditional premise does not hold. The doc and the migration both treat "schema juicefs_meta absent" as meaning "not the metadata home", citing z890: ABSENT. But supabase/initdb/00_2_juicefs_meta_schema.sql is an unconditional seed — create schema if not exists juicefs_meta; — applied by apply_dir seed on every node. After any supabase-bootstrap, z890 has an empty juicefs_meta schema too.

That measurement was taken on a node which had not run bootstrap since the seed landed (2026-06-28). So schema presence distinguishes bootstrapped from not-yet-bootstrapped, not metadata-home from gateway. Anything here reasoning from that distinction needs revisiting.

2. The failure window is narrower than "it never applies", and worse for it. On an already-bootstrapped database — B850, where this was developed — the migration works. On a fresh database it is recorded-but-skipped and never applies again.

So the control validates where it was built and is absent on any node rebuilt from scratch. For a handoff whose whole subject is retiring a superuser credential, that is worth stating outright: the scoped role would be missing precisely during a rebuild, which is when someone is most likely to reach for the superuser to get things moving.

Neither finding changes the design's direction. Scoped role, DML only, no CREATE on the schema, NOLOGIN until a password exists — that is all correct, and the section on why the pooler option was wrong here is the kind of negative result worth keeping. It is the placement and the preconditions that need the edit.

Still held with #2614 so the pair lands together and in order.