Skip to content

fix(openroom): 758 restarts because nginx pointed at the P7 definition nobody starts - #2922

Closed
POWERFULMOVES wants to merge 2 commits into
mainfrom
fix/openroom-p7-upstream-alias
Closed

POWERFULMOVES wants to merge 2 commits into
mainfrom
fix/openroom-p7-upstream-alias

Conversation

@POWERFULMOVES

Copy link
Copy Markdown
Owner

758 restarts

pmoves-openroom was crash-looping. nginx aborts at startup:

[emerg] host not found in upstream "p7-room-orchestrator:8120"

Measured on pmoves_app:

p7:8120                    -> 200
p7-room-orchestrator:8120  -> 000

The name isn't invented — P7 is defined twice

docker-compose.yml has two definitions, both building from the same services/p7-room-orchestrator/Dockerfile:

line service container state
~3813 p7 pmoves-p7 running
~4310 p7-room-orchestrator pmoves-p7-room-orchestrator never started

openroom's nginx (PMOVES-OpenRoom apps/webuiapps/nginx/default.conf:24) targets the second. This compose file's own comment twelve lines below says the proxy targets p7:8120. Two definitions of one service, and the consumers disagree about the name.

Why an alias and not the source fix

The nginx conf is baked in at image build. Correcting it needs a submodule PR + image rebuild + gitlink bump — three cross-repo steps while the container keeps looping. The alias stops it now with no rebuild.

This is a stopgap and the comment in the file says so. The real defect is the duplicate definition; deduplicating is a separate change with a much larger blast radius. Do not read this as blessing two names for one service.

Verified

p7:8120  and  p7-room-orchestrator:8120   -> both 200
pmoves-openroom                            -> Up, RestartCount 0 (was 774)
http://localhost:5173/webuiapps/           -> 200  (app serving)
/api/p7/ proxy                             -> reaches p7 (404 FROM p7, not a
                                              dead upstream — nginx starts)
docker compose config                      -> rc=0
inject_pmoves_networks.py --check          -> rc=0

Known Road: compose:handoff:network-planes-and-package-sharing-2026-09-03.md (operator-granted, recorded in known-roads.jsonl).

🤖 Generated with Claude Code

https://claude.ai/code/session_01X9mpK1ZFjexjthZbvcb94f

…n nobody starts

pmoves-openroom had restarted 758 times. nginx aborts at startup:

  [emerg] host not found in upstream "p7-room-orchestrator:8120"

Measured on pmoves_app before this change:

  p7:8120                    -> 200
  p7-room-orchestrator:8120  -> 000

The name is not invented. docker-compose.yml defines P7 TWICE, and both build
from the SAME file, services/p7-room-orchestrator/Dockerfile:

  ~3813  p7                     -> container pmoves-p7                   RUNNING
  ~4310  p7-room-orchestrator   -> container pmoves-p7-room-orchestrator  never started

openroom's nginx (PMOVES-OpenRoom apps/webuiapps/nginx/default.conf:24) targets
the second one. This compose file's own comment twelve lines below says the
proxy targets `p7:8120`. Two definitions of one service, and the consumers
disagree about the name.

Adds `p7-room-orchestrator` as a network alias on the service that actually
runs, so both names resolve. Chosen over the source fix because the nginx conf
is baked in at image build: correcting it needs a submodule PR, an image
rebuild and a gitlink bump, three cross-repo steps while the container keeps
looping. The alias stops it now with no rebuild.

This is a STOPGAP and the comment says so. The real defect is the duplicate
definition; deduplicating it is a separate change with a much larger blast
radius, and this must not be read as blessing two names for one service.

Verified after:
  p7:8120 and p7-room-orchestrator:8120   -> both 200
  pmoves-openroom                          -> Up, RestartCount 0 (was 774)
  http://localhost:5173/webuiapps/         -> 200 (app serving)
  /api/p7/ proxy                           -> reaches p7 (404 FROM p7, not a
                                              dead upstream; nginx starts)
  docker compose config                    -> rc=0
  inject_pmoves_networks.py --check        -> rc=0

Known Road: compose:handoff:network-planes-and-package-sharing-2026-09-03.md
(operator-granted; recorded in known-roads.jsonl).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9mpK1ZFjexjthZbvcb94f
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • coderabbit

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 805dfd32-3503-48ac-8e22-ceb70e73642b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-04T03:47:26.446629Z 96017f8 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Docker Hardening Validation

Hardening Validation Report

Validated: Fri Sep 4 03:44:12 UTC 2026

Services Checked

PMOVES.AI Docker Hardening Validation

[INFO] Checking: pmoves/docker-compose.hardened.yml

[INFO] Validating: hi-rag-gateway-v2
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: extract-worker
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: langextract
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: presign
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: render-webhook
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: retrieval-eval
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: pdf-ingest
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: jellyfin-bridge
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: invidious-companion-proxy
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: ffmpeg-whisper
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: media-video
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: media-audio
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: hi-rag-gateway-v2-gpu
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: hi-rag-gateway-gpu
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: deepresearch
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: supaserch
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: publisher-discord
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: mesh-agent
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: nats-echo-req
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: nats-echo-res
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: comfy-watcher
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: grayjay-plugin-host
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: agent-zero
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: p7-room-orchestrator
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: archon
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: channel-monitor
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: pmoves-yt
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: notebook-sync
[PASS] Non-root user: 65532:65532
[PASS] Read-only filesystem
[PASS] All capabilities dropped
[PASS] No-new-privileges enabled
[WARN] No resource limits

[INFO] Validating: supabase_service_role_key
[WARN] No user directive
[WARN] No read_only directive
[WARN] No cap_drop: ["ALL"]
[WARN] No no-new-privileges
[WARN] No resource limits

[INFO] Validating: supabase_jwt_secret
[WARN] No user directive
[WARN] No read_only directive
[WARN] No cap_drop: ["ALL"]
[WARN] No no-new-privileges
[WARN] No resource limits

[INFO] Validating: p7_control_token
[WARN] No user directive
[WARN] No read_only directive
[WARN] No cap_drop: ["ALL"]
[WARN] No no-new-privileges
[WARN] No resource limits

======================================
Summary: 112 passed, 43 warnings, 0 errors

@github-actions github-actions Bot added compose Compose files / service Dockerfiles governance AGNOTE register / agent definitions / damage-control hooks labels Sep 4, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 96017f842a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pmoves/docker-compose.yml Outdated
Comment thread pmoves/docker-compose.yml Outdated
Reverts the network alias from the previous commit. Review was right that it is
worse than the bug it fixed.

p7-room-orchestrator is not a dead name -- it is a REAL service that the
`agents` and `orchestration` profiles start, on pmoves_app, listening on 8122:

  p7                    8120   ui profile               <- the one that runs here
  p7-room-orchestrator  8122   agents, orchestration    <- started by up-agents

Aliasing p7 to that name on pmoves_app puts TWO containers behind one DNS name
whenever the agents profile is active, which it is on this node. Docker would
round-robin, openroom connects on :8120, and roughly half of those requests
would reach a container that does not listen there. Intermittent 502s are
strictly harder to diagnose than the clean crash loop I was trying to fix.

The fix belongs in the config that names the upstream, not in DNS. Done in
PMOVES-OpenRoom (fix/p7-upstream-deferred-resolution): a variable in proxy_pass
defers resolution to request time, so nginx starts even when the upstream is
absent and degrades to a 502 on one route instead of aborting the process.
Verified with nginx -t on a network with no p7 present -- old config emerg,
new config ok.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X9mpK1ZFjexjthZbvcb94f
@POWERFULMOVES

Copy link
Copy Markdown
Owner Author

Closing: this PR's diff against main is empty (0 files) — the nginx/p7-alias fix it carries already landed on main through another commit, so there is nothing left to merge. Found while driving the room-PR queue to land (operator: 'open room prs need to land so i can view room'). The fork-side half remains open at PMOVES-OpenRoom#5 and is being handled there.

POWERFULMOVES added a commit that referenced this pull request Sep 5, 2026
…nx p7 fixes) (#2949)

PMOVES-OpenRoom 81ca6f58f -> 162d573f4 (merge of fix/docker-build-node-env
into PMOVES.AI-Edition-Hardened). Three fixes the room view needs:

- Dockerfile: pnpm i --prod=false — NODE_ENV=production was stripping
  devDependencies workspace-wide, so vite was absent and the frozen
  install completed green while leaving no build toolchain
- nginx upstream: p7-room-orchestrator:8120 -> :8122 — the session
  endpoint (POST /rooms/{id}/session) lives only on the legacy
  orchestrator, route-verified against both services' openapi.json
- nginx rooms location: drop try_files under regex+alias ($uri keeps the
  original request path, so every manifest 404'd despite files present;
  measured: all 13 rooms 404 pre-fix)

Companion parent-side fix (nginx P7 alias) already on main — PR #2922
closed as its diff was empty. Operator: room PRs need to land so the
room is viewable.

Co-authored-by: Agent Zero <agent.zero@pmoves.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compose Compose files / service Dockerfiles governance AGNOTE register / agent definitions / damage-control hooks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant