feat(juicefs): scoped juicefs_meta role — stop authenticating as a superuser

[POWERFULMOVES]

L6 step 1. Applied and verified on B850 (the metadata home). Design rationale is in #2613.

Why

JuiceFS's metadata DSN authenticates as supabase_admin — rolsuper=t, rolcreaterole=t, a full superuser — using the credential that was exposed for ~11 days and is still un-rotated.

Cross-node mounts require every node to reach the metadata engine ("ensure that all nodes has access to the Metadata Engine"). So the moment that port is tailnet-reachable, whatever role the DSN carries becomes a network-exposed auth surface. Least privilege before reachability, not after.

Verified on B850 after apply

Check	Result
Role attributes	super=f createrole=f createdb=f login=f bypassrls=f repl=f
Tables	SELECT/INSERT on 18/18
Sequences	USAGE on 7/7 (JuiceFS allocates inodes from these)
CREATE on schema	f

NOLOGIN until the operator delivers a password via CHIT — a login-capable role with no password is a worse default.

Two guards — both found by running it, not reasoning about it

1. Node-conditional. supa-migrate applies every migration on whatever node it runs, but the metadata engine lives on one host (B850: schema + 18 tables; z890: schema absent). An unconditional GRANT would abort the whole migration run on every non-metadata node. ✅ Verified clean no-op on z890, exit 0.

2. Privilege precondition. supa-migrate connects as -U postgres, but on B850 the hardened Supabase image leaves postgres NON-superuser (rolsuper=f) while every juicefs_meta object is owned by supabase_admin. My first apply failed with permission denied to change default privileges — and since a DO block is one transaction under ON_ERROR_STOP, that aborts the entire supa-migrate run, not just this file. It now detects and skips with an actionable NOTICE.
✅ Verified both ways: as postgres it skips cleanly (exit 0); as supabase_admin it applies and is idempotent on re-run.

Standing operational finding (own follow-up): the sanctioned migration Known Road cannot apply owner-scoped DDL on the metadata home, because it connects as a non-owner, non-superuser. Any future migration touching supabase_admin-owned objects hits this.

Scope

Does not repoint the live mount's DSN — that's a separate reviewable step, so this lands without touching a live filesystem.

Applied via the migrations: Known Road with the handoff on disk; the bypass is recorded in known-roads.jsonl.

Next in L6

rotate supabase_admin (operator — ~27 consumers, one window) → repoint the mount DSN to juicefs_meta → expose supabase-db multi-homed, tailnet-bound only → verify a real file read from 4090/5090/jetson (not just a directory listing — that's the failure mode this lane exists for).

🤖 Generated with Claude Code

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: cab73e4a-c882-4ef2-9465-1bc47075b210

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3a42f0b8f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[POWERFULMOVES]

Reviewed both findings against source. Both confirmed, and the first is broader than reported.

1. Ledger poisoning — confirmed, and it applies to BOTH guards

pmoves/Makefile runs apply_dir migration supabase/migrations; then apply_dir seed supabase/initdb — migrations first, seeds second. On a fresh DB the schema does not exist yet when this file runs.

The part worth adding: the DO block has two early exits, and both have the same consequence.

• schema absent → RAISE NOTICE ... RETURN
• pg_has_role(current_user,'supabase_admin','USAGE') false → skip

Both RETURN cleanly, so psql exits 0, so apply_dir records the file in public.pmoves_bootstrap_history. Every later bootstrap then skips it by filename. The role and grants are never created — not on the metadata home either, once the seed has since created the schema.

So the guard that makes this file safe to land in a shared migrations directory is the same thing that guarantees it never takes effect on the one node it exists for.

The tension is real and it is not your bug alone: the ledger has two states (applied / not applied) and this migration needs a third (not applicable here). A clean RETURN cannot express that.

The narrowest fix that resolves it: move the role DDL into supabase/initdb/00_2_juicefs_meta_schema.sql — the file that creates the schema — so schema, role and grants are one unit that only exists on the metadata home. No cross-node guard needed, no ordering hazard, and re-running stays idempotent. If it must stay a migration, then the skip has to be non-recording, which is a bootstrap change rather than a migration change.

2. PUBLIC grants — confirmed, and the comment is the load-bearing part

public.complete_studio_board_publish is security definer (initdb/18_publisher_publish_state.sql:216-224), granted to service_role at :325. Searched the whole of pmoves/supabase/ for a counterweight:

grep -in "revoke.*from public|revoke all on schema public" pmoves/supabase/   ->  no matches

PostgreSQL grants EXECUTE to PUBLIC by default on functions, and every role inherits PUBLIC. There are 5 more SECURITY DEFINER occurrences under supabase/migrations/.

The role is NOLOGIN today, so this is latent rather than live — but the design doc's Step 3 contemplates enabling LOGIN, and at that moment this block becomes wrong:

--   * No privileges on public / auth / storage / any other schema

That is accurate about what this migration grants and inaccurate about what the role has. The distinction matters because the comment is what a future operator will trust. Either narrow it to "grants no privileges on ..." and note the inherited PUBLIC set, or pair the LOGIN step with REVOKE EXECUTE ON ALL FUNCTIONS IN SCHEMA public FROM PUBLIC — the latter is a database-wide hardening decision, not this PR's to make alone.

Verdict

The direction is right and the migration is careful — the node-conditional guard and the non-superuser detection are both good instincts, and the comment explaining why a DO block aborts the whole run under ON_ERROR_STOP is the kind of thing that saves the next person an hour.

Holding rather than merging: fix 1 is a placement decision on your lane, and fix 2 touches database-wide PUBLIC policy. Both want you or the operator, not me guessing at the JuiceFS security model.

[POWERFULMOVES]

Root cause + remediation, as requested

Read the whole apply path rather than reasoning from the guard. The finding stands, but the shape is more specific — and more alarming — than "the ledger is wrong".

The ledger contract

pmoves/Makefile, apply_dir:

if admin_psql < "$f" >/tmp/sb_apply.log 2>&1; then
    admin_psql -c "INSERT INTO public.pmoves_bootstrap_history(kind, filename) ..."; ok++
else
    fail++; echo "   ⚠️  $akind FAILED (skipped): $name ..."
fi

It records on psql exit 0. That conflates transport succeeded with intent achieved. For an unconditional migration those are the same statement. For a conditional one they are not: a DO block that RETURNs cleanly exits 0 and is recorded as applied.

The table is (kind, filename, applied_at) with PK (kind, filename). It can express applied and not applied. A conditional migration needs a third state — not applicable here — and there is nowhere to put it.

The ordering that makes it fire

apply_dir migration supabase/migrations;
apply_dir seed      supabase/initdb

Migrations run first. And juicefs_meta is created by a seed — supabase/initdb/00_2_juicefs_meta_schema.sql, which is just create schema if not exists juicefs_meta;.

So on a fresh database the schema this migration guards on is guaranteed absent at the moment the migration runs.

The precise failure window

This is the part worth pinning down, because it is not "the migration never works":

starting state	what happens
fresh DB, first bootstrap	migrations pass: schema absent → guard 1 RETURNs → exit 0 → recorded. seeds pass: schema created. Next bootstrap: filename in ledger → skipped. Role never created. Permanent.
already-bootstrapped DB (schema present from an earlier seed run)	migration proceeds; supabase-bootstrap runs as supabase_admin so guard 2 passes; role created, grants applied, recorded correctly. Works.
supa-migrate (-U postgres)	guard 2 skips — but supa-migrate has no ledger at all, no history table, no INSERT. Ineffective, not poisoned.

So it works on B850 — a database bootstrapped before this migration existed — and silently does nothing on any node rebuilt from scratch afterwards.

That is the "works on my machine, absent after DR" shape. For a security control it is the worst one available: it validates in testing and is missing exactly when someone rebuilds.

A second finding: the node-conditional premise is void

The PR guards on schema existence to stay a no-op on gateway nodes, citing z890: schema juicefs_meta ABSENT.

But 00_2_juicefs_meta_schema.sql is an unconditional seed applied by apply_dir seed on every node. After any supabase-bootstrap, z890 has an empty juicefs_meta schema too. The measurement was taken on a node that had not run bootstrap since that seed landed (2026-06-28).

So guard 1 does not distinguish metadata home from gateway. It distinguishes bootstrapped from not-yet-bootstrapped — and on the fresh path it fires on the metadata home as well.

Remediation

Recommended — move the role DDL into a seed. New file supabase/initdb/00_3_juicefs_meta_role.sql, sorting immediately after the schema seed (LC_ALL=C sort: 00_2_… → 00_3_… → 00_pmoves_schema.sql).

Why this is the right shape rather than a workaround:

• Seeds run after migrations, so the schema dependency is satisfied by construction — guard 1 becomes unnecessary and can be deleted rather than fixed.
• It is the documented house pattern for exactly this. From 00_2's own header: "Dedicated new seed filename (not an edit to an existing seed) so make supabase-bootstrap applies it on BOTH fresh and already-bootstrapped databases — see the pmoves_kb seed (00_1) for the same rationale."
• The body is already idempotent (IF NOT EXISTS (SELECT 1 FROM pg_roles …), GRANT is idempotent), so re-running is safe and the ledger recording it is honest.
• supabase-bootstrap applies seeds as supabase_admin, so guard 2 is satisfied on the canonical path. Keep it anyway as a cheap assertion — but it should now be genuinely unreachable.
• Granting ON ALL TABLES on a node where the volume was never formatted touches zero tables, and the ALTER DEFAULT PRIVILEGES FOR ROLE supabase_admin clause covers tables JuiceFS creates later. So a gateway node gets a harmless unused role rather than needing to be excluded.

Not recommended, and why:

• Make the skip exit non-zero. It gives the right ledger behaviour — not recorded, retried next run — but prints FAILED (skipped) on every bootstrap forever on nodes where the skip is correct. Trains operators to ignore that line, which is the line that matters.
• Add a status column / sentinel to the ledger. The correct general fix, and worth doing if conditional migrations become a pattern. Today this is the only one — RETURN; guards across all 60 migrations: zero others. Changing the platform for a population of one is the wrong order.

Scope note

I checked whether this is systemic before proposing anything: no other migration in supabase/migrations/ (60 files) contains an early RETURN guard or a pg_namespace / pg_roles existence check. This is the first conditional migration in the estate, which is why the ledger has never had to answer the question.

Happy to implement the seed move if you want it off your plate — say the word and I'll do it against this branch. Leaving it with you since it is your lane and the placement is a design call.