From bcd35eb8a0fd37adcd5bb69a6e743ba0a68e782b Mon Sep 17 00:00:00 2001 From: POWERFULMOVES <142271328+POWERFULMOVES@users.noreply.github.com> Date: Mon, 24 Aug 2026 17:02:17 -0400 Subject: [PATCH 1/3] =?UTF-8?q?feat(juicefs):=20Step=204=20=E2=80=94=20plu?= =?UTF-8?q?mb=20the=20tailnet-bound=20DB=20port=20the=20cross-node=20lane?= =?UTF-8?q?=20needs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The compose has published this port since March: ports: - ${SUPABASE_DB_BIND:-127.0.0.1}:${SUPABASE_DB_PORT:-54322}:5432 but it has never been reachable, and the reason is not the bind. supabase-db sits on pmoves_data and pmoves_api, both `internal: true`. A published port on a container attached only to internal networks maps and then answers nothing. Multi-homing onto pmoves_external is what actually plumbs it — the pattern the NATS bus already uses, and the one the lane handoff specified. This does not put a database on the open internet: - SUPABASE_DB_BIND is set per node to that node's TAILNET address, never 0.0.0.0, so the listener exists only on the tailnet interface. The default stays 127.0.0.1, so any node that does not set it publishes nothing new and this change is inert there. - pg_hba (supabase/config/pg_hba.conf, #2702) admits ONLY juicefs_meta from 100.64.0.0/10 and REJECTS every other role there. That is the control that keeps this from being a superuser surface. The scoped role alone never was — it changes which credential JuiceFS uses, not which one is accepted. sslmode is a recorded decision rather than an inherited default, which is what JUICEFS_META_CREDENTIAL_RUNBOOK.md §1.3 asked for at this step: the cross-node DSN carries sslmode=disable, JuiceFS's own PostgreSQL best-practices advise against it, and the justification for keeping it is that WireGuard encrypts the tailnet transport and the metadata role is non-superuser and pg_hba-scoped. Revisit if the DB is ever reachable off-tailnet. Operators should publish on 5432, not the 54322 default: PORT_REGISTRY already assigns 5432 to this service, and juicefs-cross-node-setup.sh:28 defaults DB_PORT=5432, so the canonical port means the remote node needs no override. Registry row updated from "internal only" to reflect that. Verified: compose parses; supabase-db resolves to [pmoves_data, pmoves_api, pmoves_external]; split overlays regenerated with the pinned toolchain; both 5432 and 54322 are free on this host. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FkwiW3VY1xWmahTAtVioxz --- pmoves/docker-compose.core.yml | 22 ++++++++++++++++++++++ pmoves/docker-compose.yml | 22 ++++++++++++++++++++++ pmoves/docs/operations/PORT_REGISTRY.md | 2 +- 3 files changed, 45 insertions(+), 1 deletion(-) diff --git a/pmoves/docker-compose.core.yml b/pmoves/docker-compose.core.yml index 7da58e080f..e9f4b2bcc9 100644 --- a/pmoves/docker-compose.core.yml +++ b/pmoves/docker-compose.core.yml @@ -165,6 +165,28 @@ services: networks: - pmoves_data - pmoves_api # Services on pmoves_api need DB access + # Step 4 of the juicefs cross-node lane. pmoves_data and pmoves_api are both + # `internal: true`, and a published port on a container attached only to + # internal networks is not reachable -- the port maps, nothing can dial it. + # Multi-homing onto pmoves_external (internal: false) is what actually plumbs + # it. Same pattern the NATS bus already uses. + # + # This does NOT put the database on the open internet. Reachability is + # governed by two things: + # 1. SUPABASE_DB_BIND -- set to the node's TAILNET address, not 0.0.0.0, so + # the listener exists only on the tailnet interface. Default stays + # 127.0.0.1, so a node that does not set it publishes nothing new. + # 2. pg_hba (supabase/config/pg_hba.conf) -- admits ONLY juicefs_meta from + # 100.64.0.0/10 and REJECTS every other role there. That is the control + # that keeps this from being a superuser surface; the scoped role alone + # never was. + # + # sslmode: the cross-node DSN carries sslmode=disable. JuiceFS's PostgreSQL + # best-practices doc advises against it. Recorded decision rather than an + # inherited default: WireGuard encrypts the tailnet transport, so the session + # is not on the wire in plaintext, and the metadata role is non-superuser and + # pg_hba-scoped. Revisit if the DB is ever reachable off-tailnet. + - pmoves_external deploy: resources: diff --git a/pmoves/docker-compose.yml b/pmoves/docker-compose.yml index c4f7ccec4a..bf1816ee7f 100644 --- a/pmoves/docker-compose.yml +++ b/pmoves/docker-compose.yml @@ -617,6 +617,28 @@ services: networks: - pmoves_data - pmoves_api # Services on pmoves_api need DB access + # Step 4 of the juicefs cross-node lane. pmoves_data and pmoves_api are both + # `internal: true`, and a published port on a container attached only to + # internal networks is not reachable -- the port maps, nothing can dial it. + # Multi-homing onto pmoves_external (internal: false) is what actually plumbs + # it. Same pattern the NATS bus already uses. + # + # This does NOT put the database on the open internet. Reachability is + # governed by two things: + # 1. SUPABASE_DB_BIND -- set to the node's TAILNET address, not 0.0.0.0, so + # the listener exists only on the tailnet interface. Default stays + # 127.0.0.1, so a node that does not set it publishes nothing new. + # 2. pg_hba (supabase/config/pg_hba.conf) -- admits ONLY juicefs_meta from + # 100.64.0.0/10 and REJECTS every other role there. That is the control + # that keeps this from being a superuser surface; the scoped role alone + # never was. + # + # sslmode: the cross-node DSN carries sslmode=disable. JuiceFS's PostgreSQL + # best-practices doc advises against it. Recorded decision rather than an + # inherited default: WireGuard encrypts the tailnet transport, so the session + # is not on the wire in plaintext, and the metadata role is non-superuser and + # pg_hba-scoped. Revisit if the DB is ever reachable off-tailnet. + - pmoves_external deploy: resources: diff --git a/pmoves/docs/operations/PORT_REGISTRY.md b/pmoves/docs/operations/PORT_REGISTRY.md index 999f1f060f..5c6ff5058a 100644 --- a/pmoves/docs/operations/PORT_REGISTRY.md +++ b/pmoves/docs/operations/PORT_REGISTRY.md @@ -92,7 +92,7 @@ Central registry of all service ports to prevent conflicts and ensure consistenc | Port | Service | Description | Network | |------|---------|-------------|---------| -| 5432 | Supabase DB | PostgreSQL 17 (internal only) | pmoves_data | +| 5432 | Supabase DB | PostgreSQL 17 (tailnet-bound on the juicefs meta host; internal elsewhere) | pmoves_data, pmoves_api, pmoves_external | | 3010 | PostgREST | Supabase REST API — **host** port (container port stays 3000) | pmoves_api, pmoves_data | | 9999 | GoTrue | JWT authentication service | pmoves_api, pmoves_data | | 4010 | Realtime | WebSocket for real-time subscriptions (remapped from 4000) | pmoves_api, pmoves_data | From a3e6d759b72b726470be275ebeea7d4a680729fb Mon Sep 17 00:00:00 2001 From: POWERFULMOVES <142271328+POWERFULMOVES@users.noreply.github.com> Date: Tue, 25 Aug 2026 02:38:38 -0400 Subject: [PATCH 2/3] fix(compose): regenerate PMOVES_NETWORKS mirror for the external network MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Step 4 hand-edited the networks: lists but not the generated PMOVES_NETWORKS env that topology.TopologyContext.from_env() reads — the injector drift gate failed. Re-ran the injector and compose-split; one mirrored env line per file, no other changes. --- pmoves/docker-compose.core.yml | 2 +- pmoves/docker-compose.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pmoves/docker-compose.core.yml b/pmoves/docker-compose.core.yml index e9f4b2bcc9..d7c75309d6 100644 --- a/pmoves/docker-compose.core.yml +++ b/pmoves/docker-compose.core.yml @@ -145,7 +145,7 @@ services: - TOPOLOGY_MODE=${TOPOLOGY_MODE:-docked} - PARENT_SYSTEM=${PARENT_SYSTEM:-PMOVES.AI} - PARENT_VERSION=${PARENT_VERSION:-1.0.0-hardened} - - PMOVES_NETWORKS=pmoves_data,pmoves_api + - PMOVES_NETWORKS=pmoves_data,pmoves_api,pmoves_external healthcheck: test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"] interval: 20s diff --git a/pmoves/docker-compose.yml b/pmoves/docker-compose.yml index bf1816ee7f..0ff419bfe4 100644 --- a/pmoves/docker-compose.yml +++ b/pmoves/docker-compose.yml @@ -597,7 +597,7 @@ services: - TOPOLOGY_MODE=${TOPOLOGY_MODE:-docked} - PARENT_SYSTEM=${PARENT_SYSTEM:-PMOVES.AI} - PARENT_VERSION=${PARENT_VERSION:-1.0.0-hardened} - - PMOVES_NETWORKS=pmoves_data,pmoves_api + - PMOVES_NETWORKS=pmoves_data,pmoves_api,pmoves_external healthcheck: test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"] interval: 20s From d287702f87b26965c71b19e01d48fad756e79383 Mon Sep 17 00:00:00 2001 From: POWERFULMOVES <142271328+POWERFULMOVES@users.noreply.github.com> Date: Tue, 25 Aug 2026 11:03:42 -0400 Subject: [PATCH 3/3] fix(juicefs): dedicated DB-egress bridge instead of the shared network Review P1: joining supabase-db to pmoves_external handed every internet-facing container on that bridge a direct route to supabase-db:5432, and pg_hba's 172.16.0.0/12 catch-all accepts every role from any docker bridge -- the tailnet scoping never sees those sources. Replaced with pmoves_db_egress (172.30.8.0/24, internal: false, external network carrying ONLY the database): - pg_hba scopes the new subnet above the catch-all: juicefs_meta only, everything else rejected -- identical treatment to the tailnet rules. Legitimate inbound still DNATs in with its real 100.x source and hits the 100.64.0.0/10 block; only a container that joined this bridge sources from 172.30.8.x, and that bridge is supposed to hold the database alone. - Subnet is .8, not .7: pmoves_public is compose-declared at 172.30.7.0/24 (docker-compose.yml) -- .7 would collide. - Network ensured alongside the others in ensure-networks and ensure-overlay-networks; overlays regenerated via injector + split. The 5090's reachability Monitor is unaffected: SUPABASE_DB_BIND still pins the listener to the node's tailnet address, and the dedicated bridge is what makes the published port answer at all. --- pmoves/Makefile | 7 +++-- pmoves/docker-compose.base.yml | 10 +++++++ pmoves/docker-compose.core.yml | 26 +++++++++++------- pmoves/docker-compose.yml | 36 ++++++++++++++++++------- pmoves/docs/operations/PORT_REGISTRY.md | 2 +- pmoves/supabase/config/pg_hba.conf | 12 +++++++++ 6 files changed, 72 insertions(+), 21 deletions(-) diff --git a/pmoves/Makefile b/pmoves/Makefile index 58eba67672..c73cd1e68c 100644 --- a/pmoves/Makefile +++ b/pmoves/Makefile @@ -354,6 +354,7 @@ ensure-networks: ## Materialize the shared compose-owned networks (idempotent) s @docker network inspect pmoves_bus >/dev/null 2>&1 || docker network create --driver bridge --internal --subnet 172.30.3.0/24 --gateway 172.30.3.1 --label com.docker.compose.network=pmoves_bus --label com.docker.compose.project=$(PROJECT) pmoves_bus >/dev/null 2>&1 || true @docker network inspect pmoves_data >/dev/null 2>&1 || docker network create --driver bridge --internal --subnet 172.30.4.0/24 --gateway 172.30.4.1 --label com.docker.compose.network=pmoves_data --label com.docker.compose.project=$(PROJECT) pmoves_data >/dev/null 2>&1 || true @docker network inspect pmoves_monitoring >/dev/null 2>&1 || docker network create --driver bridge --internal --subnet 172.30.5.0/24 --gateway 172.30.5.1 --label com.docker.compose.network=pmoves_monitoring --label com.docker.compose.project=$(PROJECT) pmoves_monitoring >/dev/null 2>&1 || true + @docker network inspect pmoves_db_egress >/dev/null 2>&1 || docker network create --driver bridge --subnet 172.30.8.0/24 --gateway 172.30.8.1 pmoves_db_egress >/dev/null 2>&1 || true @docker network inspect pmoves-net >/dev/null 2>&1 || docker network create --driver bridge --label com.docker.compose.network=pmoves --label com.docker.compose.project=$(PROJECT) pmoves-net >/dev/null 2>&1 || true @echo "✔ shared networks ensured" @@ -4983,12 +4984,14 @@ overlay-up-full: ## Start ALL services via overlay files (base + all tiers) # files declare networks `external: true` (so each overlay parses standalone), so they # must EXIST at runtime; docker-compose.base.yml owns them but only materializes them # in a full `up`. Subnets here MUST match docker-compose.base.yml. -ensure-overlay-networks: ## Create the bus overlay networks (pmoves_bus, pmoves_external) if missing +ensure-overlay-networks: ## Create the bus overlay networks (pmoves_bus, pmoves_external, pmoves_db_egress) if missing @docker network inspect pmoves_bus >/dev/null 2>&1 || \ docker network create --internal --driver bridge --subnet 172.30.3.0/24 --gateway 172.30.3.1 pmoves_bus >/dev/null @docker network inspect pmoves_external >/dev/null 2>&1 || \ docker network create --driver bridge --subnet 172.30.6.0/24 --gateway 172.30.6.1 pmoves_external >/dev/null - @echo "✅ overlay bus networks present (pmoves_bus, pmoves_external)" + @docker network inspect pmoves_db_egress >/dev/null 2>&1 || \ + docker network create --driver bridge --subnet 172.30.8.0/24 --gateway 172.30.8.1 pmoves_db_egress >/dev/null + @echo "✅ overlay bus networks present (pmoves_bus, pmoves_external, pmoves_db_egress)" # overlay-up-bus brings up ONLY the NATS event bus via the split overlays # (OVERLAY_DC = base+core), NOT the monolith via DC/STACK_FILES. The monolith's diff --git a/pmoves/docker-compose.base.yml b/pmoves/docker-compose.base.yml index ba4e8548c8..013f23c7ac 100644 --- a/pmoves/docker-compose.base.yml +++ b/pmoves/docker-compose.base.yml @@ -581,3 +581,13 @@ networks: pmoves_external: external: true name: pmoves_external + # DEDICATED non-internal bridge for the supabase-db tailnet port publish + # (juicefs cross-node lane, PR #2728). Deliberately NOT pmoves_external: + # that bridge is shared with internet-facing containers, and pg_hba's + # 172.16.0.0/12 catch-all would accept every role from any of them. This + # network carries ONLY the database; pg_hba scopes its subnet + # (172.30.8.0/24) to juicefs_meta alone. External for the same + # cross-stack-adoption reason as pmoves_external. + pmoves_db_egress: + external: true + name: pmoves_db_egress diff --git a/pmoves/docker-compose.core.yml b/pmoves/docker-compose.core.yml index d7c75309d6..e9f9b29f9a 100644 --- a/pmoves/docker-compose.core.yml +++ b/pmoves/docker-compose.core.yml @@ -145,7 +145,7 @@ services: - TOPOLOGY_MODE=${TOPOLOGY_MODE:-docked} - PARENT_SYSTEM=${PARENT_SYSTEM:-PMOVES.AI} - PARENT_VERSION=${PARENT_VERSION:-1.0.0-hardened} - - PMOVES_NETWORKS=pmoves_data,pmoves_api,pmoves_external + - PMOVES_NETWORKS=pmoves_data,pmoves_api,pmoves_db_egress healthcheck: test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"] interval: 20s @@ -168,25 +168,33 @@ services: # Step 4 of the juicefs cross-node lane. pmoves_data and pmoves_api are both # `internal: true`, and a published port on a container attached only to # internal networks is not reachable -- the port maps, nothing can dial it. - # Multi-homing onto pmoves_external (internal: false) is what actually plumbs - # it. Same pattern the NATS bus already uses. + # A non-internal bridge is what actually plumbs it. + # + # DEDICATED bridge, NOT the shared egress network (review P1): joining + # pmoves_external would hand every internet-facing container on that bridge + # a direct route to supabase-db:5432, and the pg_hba catch-all accepts every + # role from 172.16.0.0/12 (docker bridges included) -- the tailnet scoping + # never sees those sources. pmoves_db_egress (172.30.8.0/24, internal: + # false) carries ONLY this database, and its pg_hba ruleset admits nothing + # from that subnet except juicefs_meta. # # This does NOT put the database on the open internet. Reachability is - # governed by two things: + # governed by: # 1. SUPABASE_DB_BIND -- set to the node's TAILNET address, not 0.0.0.0, so # the listener exists only on the tailnet interface. Default stays # 127.0.0.1, so a node that does not set it publishes nothing new. - # 2. pg_hba (supabase/config/pg_hba.conf) -- admits ONLY juicefs_meta from - # 100.64.0.0/10 and REJECTS every other role there. That is the control - # that keeps this from being a superuser surface; the scoped role alone - # never was. + # 2. pg_hba (supabase/config/pg_hba.conf) -- tailnet sources + # (100.64.0.0/10, which DNAT preserves end-to-end) admit ONLY + # juicefs_meta and reject every other role; the dedicated bridge subnet + # carries the same juicefs_meta-only rule so a container that somehow + # joined it gains nothing either. # # sslmode: the cross-node DSN carries sslmode=disable. JuiceFS's PostgreSQL # best-practices doc advises against it. Recorded decision rather than an # inherited default: WireGuard encrypts the tailnet transport, so the session # is not on the wire in plaintext, and the metadata role is non-superuser and # pg_hba-scoped. Revisit if the DB is ever reachable off-tailnet. - - pmoves_external + - pmoves_db_egress deploy: resources: diff --git a/pmoves/docker-compose.yml b/pmoves/docker-compose.yml index 0ff419bfe4..f2ba06b9ec 100644 --- a/pmoves/docker-compose.yml +++ b/pmoves/docker-compose.yml @@ -597,7 +597,7 @@ services: - TOPOLOGY_MODE=${TOPOLOGY_MODE:-docked} - PARENT_SYSTEM=${PARENT_SYSTEM:-PMOVES.AI} - PARENT_VERSION=${PARENT_VERSION:-1.0.0-hardened} - - PMOVES_NETWORKS=pmoves_data,pmoves_api,pmoves_external + - PMOVES_NETWORKS=pmoves_data,pmoves_api,pmoves_db_egress healthcheck: test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER -d $$POSTGRES_DB"] interval: 20s @@ -620,25 +620,33 @@ services: # Step 4 of the juicefs cross-node lane. pmoves_data and pmoves_api are both # `internal: true`, and a published port on a container attached only to # internal networks is not reachable -- the port maps, nothing can dial it. - # Multi-homing onto pmoves_external (internal: false) is what actually plumbs - # it. Same pattern the NATS bus already uses. + # A non-internal bridge is what actually plumbs it. + # + # DEDICATED bridge, NOT the shared egress network (review P1): joining + # pmoves_external would hand every internet-facing container on that bridge + # a direct route to supabase-db:5432, and the pg_hba catch-all accepts every + # role from 172.16.0.0/12 (docker bridges included) -- the tailnet scoping + # never sees those sources. pmoves_db_egress (172.30.8.0/24, internal: + # false) carries ONLY this database, and its pg_hba ruleset admits nothing + # from that subnet except juicefs_meta. # # This does NOT put the database on the open internet. Reachability is - # governed by two things: + # governed by: # 1. SUPABASE_DB_BIND -- set to the node's TAILNET address, not 0.0.0.0, so # the listener exists only on the tailnet interface. Default stays # 127.0.0.1, so a node that does not set it publishes nothing new. - # 2. pg_hba (supabase/config/pg_hba.conf) -- admits ONLY juicefs_meta from - # 100.64.0.0/10 and REJECTS every other role there. That is the control - # that keeps this from being a superuser surface; the scoped role alone - # never was. + # 2. pg_hba (supabase/config/pg_hba.conf) -- tailnet sources + # (100.64.0.0/10, which DNAT preserves end-to-end) admit ONLY + # juicefs_meta and reject every other role; the dedicated bridge subnet + # carries the same juicefs_meta-only rule so a container that somehow + # joined it gains nothing either. # # sslmode: the cross-node DSN carries sslmode=disable. JuiceFS's PostgreSQL # best-practices doc advises against it. Recorded decision rather than an # inherited default: WireGuard encrypts the tailnet transport, so the session # is not on the wire in plaintext, and the metadata role is non-superuser and # pg_hba-scoped. Revisit if the DB is ever reachable off-tailnet. - - pmoves_external + - pmoves_db_egress deploy: resources: @@ -5893,3 +5901,13 @@ networks: pmoves_external: external: true name: pmoves_external + # DEDICATED non-internal bridge for the supabase-db tailnet port publish + # (juicefs cross-node lane, PR #2728). Deliberately NOT pmoves_external: + # that bridge is shared with internet-facing containers, and pg_hba's + # 172.16.0.0/12 catch-all would accept every role from any of them. This + # network carries ONLY the database; pg_hba scopes its subnet + # (172.30.8.0/24) to juicefs_meta alone. External for the same + # cross-stack-adoption reason as pmoves_external. + pmoves_db_egress: + external: true + name: pmoves_db_egress diff --git a/pmoves/docs/operations/PORT_REGISTRY.md b/pmoves/docs/operations/PORT_REGISTRY.md index 5c6ff5058a..6569ff4fa4 100644 --- a/pmoves/docs/operations/PORT_REGISTRY.md +++ b/pmoves/docs/operations/PORT_REGISTRY.md @@ -92,7 +92,7 @@ Central registry of all service ports to prevent conflicts and ensure consistenc | Port | Service | Description | Network | |------|---------|-------------|---------| -| 5432 | Supabase DB | PostgreSQL 17 (tailnet-bound on the juicefs meta host; internal elsewhere) | pmoves_data, pmoves_api, pmoves_external | +| 5432 | Supabase DB | PostgreSQL 17 (tailnet-bound on the juicefs meta host; internal elsewhere) | pmoves_data, pmoves_api, pmoves_db_egress (dedicated, pg_hba-scoped) | | 3010 | PostgREST | Supabase REST API — **host** port (container port stays 3000) | pmoves_api, pmoves_data | | 9999 | GoTrue | JWT authentication service | pmoves_api, pmoves_data | | 4010 | Realtime | WebSocket for real-time subscriptions (remapped from 4000) | pmoves_api, pmoves_data | diff --git a/pmoves/supabase/config/pg_hba.conf b/pmoves/supabase/config/pg_hba.conf index 248b1be09c..41234323a6 100644 --- a/pmoves/supabase/config/pg_hba.conf +++ b/pmoves/supabase/config/pg_hba.conf @@ -105,6 +105,18 @@ host all all 100.64.0.0/10 reject host all juicefs_meta fd7a:115c:a1e0::/48 scram-sha-256 host all all fd7a:115c:a1e0::/48 reject +# Dedicated DB-egress bridge (172.30.8.0/24, docker-compose.base.yml +# pmoves_db_egress). The DB joins this non-internal bridge solely so its +# tailnet-published port is actually plumbed (PR #2728). Legitimate inbound +# traffic DNATs in with its real 100.x source and matches the rules above; +# the only packets bearing a 172.30.7.x source are containers that joined +# this bridge -- which is supposed to carry the database alone. Scope them +# identically to the tailnet: juicefs_meta only, everything else rejected, +# ABOVE the 172.16.0.0/12 catch-all (which otherwise accepts every role from +# any docker bridge). +host all juicefs_meta 172.30.8.0/24 scram-sha-256 +host all all 172.30.8.0/24 reject + # IPv4 external connections host all all 10.0.0.0/8 scram-sha-256 host all all 172.16.0.0/12 scram-sha-256