Skip to content

feat(juicefs): META_ROLE scoped-role param + cross-node cutover checklist (Jellyfin) - #2683

Merged
POWERFULMOVES merged 1 commit into
mainfrom
feat/juicefs-crossnode-scoped-role
Aug 22, 2026
Merged

POWERFULMOVES merged 1 commit into
mainfrom
feat/juicefs-crossnode-scoped-role

Conversation

@POWERFULMOVES

Copy link
Copy Markdown
Owner

What

Stages the cross-node shared pmoves-media lane (back Jellyfin with the mesh FS; also the fleet file-mover) up to the operator gates. Nothing here applies anything — it's the enabler + the ordered runbook.

Changes

  • juicefs-cross-node-setup.sh — add META_ROLE (default supabase_admin; set juicefs_meta at the step-2 cutover) so the metadata DSN can move off the full superuser to DML-on-one-schema, per the 08-18 handoff. Refreshed the stale file:// header: pmoves-media is MinIO-backed now, so the real remaining blocker is metadata reachability (supabase-db on internal:true networks — published :5432 not plumbed).
  • JUICEFS_CROSSNODE_CUTOVER_CHECKLIST.md — one ordered runbook, each step marked [operator] vs [agent]:
    1. [operator] apply scoped juicefs_meta role (KNOWN_ROAD=migrations: + supabase-bootstrap)
    2. [operator] grant LOGIN + cut mount over (META_ROLE=juicefs_meta)
    3. [operator] rotate supabase_admin (after cutover, not before)
    4. [operator] expose supabase-db tailnet-bound (KNOWN_ROAD=compose:)
    5. [agent] mount pmoves-media on 5090
    6. [agent] repoint JELLYFIN_MEDIA_DIR + rebuild-external-svc SVC=jellyfin-ext

Safety

Back-compat default keeps existing behavior; the superuser→scoped-role cutover + tailnet exposure stay behind the operator's Known Roads and the supabase_admin rotation, in the order the 08-18 handoff mandates.

🤖 Generated with Claude Code

…s-node checklist

Stage the cross-node JuiceFS lane for Jellyfin up to the operator gates:
- juicefs-cross-node-setup.sh: add META_ROLE (default supabase_admin; set juicefs_meta
  at step-2 cutover) so the DSN can move off the full superuser to DML-on-one-schema,
  per juicefs-meta-scoped-role-and-tailnet-exposure-2026-08-18. Refresh the stale
  file:// header — pmoves-media is MinIO-backed now; the real blocker is metadata
  reachability (supabase-db on internal:true networks).
- JUICEFS_CROSSNODE_CUTOVER_CHECKLIST.md: one ordered runbook marking each step
  [operator] (Known Roads for migrations/compose, supabase_admin rotation) vs [agent]
  (mount + repoint JELLYFIN_MEDIA_DIR + rebuild-external-svc).

Changes nothing until used. Applying is gated on the operator's Known Roads + rotation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g8jC7dupS2ubafo6zPQY6
@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: aa1ce676-8e58-4469-ac95-db56ee1e6cb2


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the docs Documentation label Aug 22, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8818266849

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +48 to +49
META_ROLE=juicefs_meta DB_PASS=<juicefs_meta pw, from pipeline> \
make -C pmoves juicefs-cross-node-setup JUICEFS_HOST=pmoves-b850-ai-top

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Cut over the in-stack mount before rotating the admin

When this checklist is followed in order, this cross-node command cannot perform step 2: the document itself establishes that ${JUICEFS_HOST}:5432 is unreachable until step 4, while the script uses that endpoint for its preflight. It also does not repoint the active in-stack consumers—pmoves/mk/egress.mk:354-379 and the defaults in pmoves/docker-compose.juicefs.yml:48,87 still authenticate as supabase_admin. Consequently the required read cannot be verified, and proceeding to the instructed rotation would leave those consumers with the retired credential; step 2 needs an in-stack scoped-role cutover and verification before the remote endpoint is exposed.

Useful? React with 👍 / 👎.

### 6. [agent] Point Jellyfin at the mount
Set `JELLYFIN_MEDIA_DIR` to the mounted `pmoves-media` path, then:
```
make -C pmoves rebuild-external-svc SVC=jellyfin-ext

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Replace the nonexistent Jellyfin rebuild target

At the final cutover step, this command exits immediately with No rule to make target 'rebuild-external-svc'; a repo-wide target search finds only the default-stack rebuild-svc, the edge-specific target, and the external-overlay up-external target. Thus following the new runbook never recreates jellyfin-ext with JELLYFIN_MEDIA_DIR, so Jellyfin remains bound to its old local directory. Use or add a canonical target that operates on docker-compose.external.yml.

AGENTS.md reference: AGENTS.md:L28-L34

Useful? React with 👍 / 👎.

@POWERFULMOVES
POWERFULMOVES merged commit f4f8c60 into main Aug 22, 2026
28 of 29 checks passed
@POWERFULMOVES
POWERFULMOVES deleted the feat/juicefs-crossnode-scoped-role branch August 22, 2026 21:22
POWERFULMOVES added a commit that referenced this pull request Aug 24, 2026
…it targets

The META_ROLE forwarding in this PR is the important half and is correct — the
script has supported META_ROLE since #2683 but the target never passed it, so
the canonical make path silently used supabase_admin, which pg_hba now REJECTS
from the tailnet (#2702). Step 5 would have failed with an auth error that reads
like a bad secret rather than a rejected role.

The password fallback, though, could not fire. `$(JUICEFS_META_PASSWORD)` is a
MAKE variable reference, and make populates its variables only from the
environment and from Makefiles — nothing includes the generated tier files. The
funnel writes this key to env.tier-data and .env.generated, not to env.shared,
so it is never exported into an operator's session either. Net effect: the
operator hits "DB_PASS or JUICEFS_META_PASSWORD required" on precisely the node
the funnel just delivered to, and the documented remedy is to read the value out
of the tier file by hand — which the script's own error text tells them not to
do.

Switched to the recipe-time shell read this file already uses ~line 229:

    DB_PASS="$(or $(DB_PASS),$$(grep -m1 '^JUICEFS_META_PASSWORD=' env.tier-data ...))"

`$$(...)` not `$(...)`: shell at recipe time, which can read generated files.
`cut -d= -f2-` not `-f2`, so a value containing '=' is not truncated at the
first one (the neighbouring line has that bug; not inheriting it).

Dropped the $(error): the script already fails with a better message that names
both DB_PASS and the funnel path, and a make-level abort pre-empts it.

Verified all three paths:
  funnel-delivered  key in env.tier-data, no DB_PASS -> script receives
                    META_ROLE=juicefs_meta with a non-empty DB_PASS
  explicit          DB_PASS=... on the command line wins
  neither           script's own "no metadata password" error, no make abort
and separately that -f2- preserves `abc=def==` where -f2 yields `abc`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FkwiW3VY1xWmahTAtVioxz
POWERFULMOVES added a commit that referenced this pull request Aug 24, 2026
… JUICEFS_META_PASSWORD fallback) (#2708)

* fix(juicefs): forward META_ROLE through the make target + JUICEFS_META_PASSWORD fallback

#2683 added META_ROLE to juicefs-cross-node-setup.sh but the make target never
forwarded it, so the canonical path (make juicefs-cross-node-setup META_ROLE=juicefs_meta)
silently defaulted to supabase_admin — the scoped-role cutover was unreachable on remote
nodes (the 5090 step-5 mount). Forward META_ROLE (default supabase_admin, back-compat).

Also: DB_PASS now falls back to the funnel-delivered JUICEFS_META_PASSWORD (registered
in chit_manifest_register.py, #2705), so a node that received the secret via the pipeline
runs with just META_ROLE=juicefs_meta. Both pass as sub-process env (not argv) -> handed
to JuiceFS via META_PASSWORD, never in ps. Same 'wired end-to-end?' class as the node-local
fixes B850 surfaced tonight.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017g8jC7dupS2ubafo6zPQY6

* fix(juicefs): $(JUICEFS_META_PASSWORD) is empty on exactly the nodes it targets

The META_ROLE forwarding in this PR is the important half and is correct — the
script has supported META_ROLE since #2683 but the target never passed it, so
the canonical make path silently used supabase_admin, which pg_hba now REJECTS
from the tailnet (#2702). Step 5 would have failed with an auth error that reads
like a bad secret rather than a rejected role.

The password fallback, though, could not fire. `$(JUICEFS_META_PASSWORD)` is a
MAKE variable reference, and make populates its variables only from the
environment and from Makefiles — nothing includes the generated tier files. The
funnel writes this key to env.tier-data and .env.generated, not to env.shared,
so it is never exported into an operator's session either. Net effect: the
operator hits "DB_PASS or JUICEFS_META_PASSWORD required" on precisely the node
the funnel just delivered to, and the documented remedy is to read the value out
of the tier file by hand — which the script's own error text tells them not to
do.

Switched to the recipe-time shell read this file already uses ~line 229:

    DB_PASS="$(or $(DB_PASS),$$(grep -m1 '^JUICEFS_META_PASSWORD=' env.tier-data ...))"

`$$(...)` not `$(...)`: shell at recipe time, which can read generated files.
`cut -d= -f2-` not `-f2`, so a value containing '=' is not truncated at the
first one (the neighbouring line has that bug; not inheriting it).

Dropped the $(error): the script already fails with a better message that names
both DB_PASS and the funnel path, and a make-level abort pre-empts it.

Verified all three paths:
  funnel-delivered  key in env.tier-data, no DB_PASS -> script receives
                    META_ROLE=juicefs_meta with a non-empty DB_PASS
  explicit          DB_PASS=... on the command line wins
  neither           script's own "no metadata password" error, no make abort
and separately that -f2- preserves `abc=def==` where -f2 yields `abc`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FkwiW3VY1xWmahTAtVioxz

* fix(juicefs): resolve the funnel password through with-env.sh, the canonical loader

The previous commit read env.tier-data with an inline grep. That works, but it
is not the house pattern and it only reads one file.

scripts/with-env.sh is the canonical loader — "env.shared* -> tier env files ->
.env* overlays", mirroring compose layering — so it honours precedence instead
of hard-coding which tier file the value happens to land in today. Same idiom as
mk/yt-cookies.mk:18, and infra.mk:603 already carries a comment recording this
exact lesson as a prior Codex P1 ("uses with-env.sh to load tier files").

Using it also drops the `cut -d= -f2-` parsing entirely, so there is no longer a
quoting or embedded-'=' edge case to get right.

Verified all three paths with a stubbed script:
  tier-delivered   key in env.tier-data, no DB_PASS -> DB_PASS_len=27
  explicit         DB_PASS=explicit                 -> DB_PASS_len=8
  neither          script's own "no metadata password" error, no make abort

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FkwiW3VY1xWmahTAtVioxz

---------

Co-authored-by: Mavis <Mavis@pmoves.local>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant