fix(security): GPU access + production hardening (health checks, image pins, required password) - #355
Conversation
- Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Caution Review failedThe pull request is closed. 📝 WalkthroughWalkthroughThis PR introduces GPU infrastructure support, environment tier consolidation patterns, and comprehensive service health monitoring. Key changes include adopting NVIDIA CUDA base image for media services, adding extensive healthchecks across docker-compose services, pinning image versions to explicit tags, creating environment tier example configurations, and documenting tier consolidation learnings. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes Possibly related PRs
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
📜 Recent review detailsConfiguration used: defaults Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Critical security and reliability fixes:
1. **POSTGRES_PASSWORD now required** - Changed from insecure default
`:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if
POSTGRES_PASSWORD is not set (lines 59, 78, 937).
2. **Pinned 8 :latest images to specific versions:**
- postgrest/postgrest:latest → v12.2.3
- minio/minio:latest → RELEASE.2024-12-18T13-15-44Z
- ollama/ollama:latest → 0.5.4
- tensorzero/gateway:latest → 2024.12.18
- tensorzero/ui:latest → 2024.12.18
- invidious:latest → 2024.12.09
- invidious-companion:latest → 2024.12.20
- grayjay:latest → 2024.11.01
3. **Added health checks to 37 services** (52 total, up from 15):
- Data tier: qdrant, meilisearch, minio
- API tier: hi-rag-*, retrieval-eval, presign, render-webhook
- Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync,
ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor
- Agent tier: agent-zero, mesh-agent, deepresearch, supaserch
- TensorZero: gateway, ui, ollama
- Others: publisher-discord, messaging-gateway, jellyfin-bridge
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
pmoves/docker-compose.yml (2)
905-905: Redundant GPU configuration in hi-rag-gateway-v2-gpu.The service specifies GPU access using three different methods:
runtime: nvidia(line 905) - legacy Docker runtimegpus: all(line 949) - docker-compose v1.28+ shorthanddeploy.resources.reservations.devices(lines 950-955) - Compose Specification formatDocker Compose only needs ONE method. Mixing methods can cause conflicts or unexpected behavior. The
deploy.resources.reservationsformat is the current standard and is already used by all other GPU services in this file.🔎 Proposed fix to use consistent GPU configuration
runtime: nvidia restart: unless-stopped command: [ "uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8086" ] - runtime: nvidia restart: unless-stopped command: [ "uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8086" ] env_file: [ env.shared.generated, env.shared, .env.generated, .env.local ] environment: # ... environment variables ... - NVIDIA_DRIVER_CAPABILITIES=${NVIDIA_DRIVER_CAPABILITIES:-compute,utility} - NVIDIA_VISIBLE_DEVICES=${NVIDIA_VISIBLE_DEVICES:-all} - gpus: all deploy: resources: reservations: devices: - capabilities: [ gpu ] count: ${GPU_COUNT:-all}Note: Also noticed the
runtime: nvidiaappears twice at line 905 (duplicate line).Also applies to: 949-955
676-703: Add multi-arch build support and Trivy scanning to media-audio Dockerfile.The CUDA base image is correctly set to
nvidia/cuda:12.4.1-runtime-ubuntu22.04, and GPU configuration in docker-compose.yml (lines 689-694) is properly set. However, per multi-arch build requirements, the Dockerfile must support both amd64 and arm64 architectures usingARG BUILDPLATFORMandARG TARGETPLATFORMstatements in the FROM directive. Additionally, the build process should include Trivy scanning to detect HIGH/CRITICAL vulnerabilities. Apply the same multi-arch pattern used in media-video and ensure docker-compose.yml includes Trivy scanning integration.
🧹 Nitpick comments (1)
pmoves/docker-compose.yml (1)
5-49: Document tier-based environment file structure and consider .env.local security implications.The tier-based environment file approach implements least privilege well by separating credentials by service tier. However:
.env.localis included in ALL tier anchors, potentially granting broader access than intended if it contains sensitive credentials.- Missing documentation on which environment variables belong in which tier file (e.g.,
env.tier-datavsenv.tier-apivsenv.tier-llm).Consider:
- Creating example/template files:
env.tier-data.example,env.tier-api.example, etc., documenting which variables belong in each tier.- Documenting that
.env.localshould only contain non-sensitive local overrides, or splitting it into tier-specific local override files like.env.local.data,.env.local.api.Based on learnings and coding guidelines, verify that the tier-based approach is documented in service READMEs and
pmoves/docs/PMOVES.AI PLANS/runbooks.
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
pmoves/docker-compose.yml
🧰 Additional context used
📓 Path-based instructions (1)
pmoves/**/docker-compose.yml
📄 CodeRabbit inference engine (pmoves/AGENTS.md)
Use Compose profiles (
data,workers) to scope what runs locally in docker-compose.yml
Files:
pmoves/docker-compose.yml
🧠 Learnings (6)
📓 Common learnings
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: GEMINI.md:0-0
Timestamp: 2025-12-07T11:03:07.638Z
Learning: Applies to **/pmoves/**/{media,analysis,pipeline}*.py : Implement `media-video` and `media-audio` analysis pipelines with GPU auto-detect for faster-whisper
📚 Learning: 2025-12-15T12:03:17.291Z
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-15T12:03:17.291Z
Learning: Applies to services/**/README.md : Update services/*/README.md and pmoves/docs/PMOVES.AI PLANS/ runbooks when touching service operational code
Applied to files:
pmoves/docker-compose.yml
📚 Learning: 2025-12-15T12:03:17.291Z
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-15T12:03:17.291Z
Learning: Applies to {Dockerfile,docker-compose.yml,docker-compose.*.yml,pmoves/docker-compose.*.override.yml} : Dockerfile and docker-compose.yml must follow multi-arch build practices (amd64+arm64) with Trivy scanning for HIGH/CRITICAL vulnerabilities
Applied to files:
pmoves/docker-compose.yml
📚 Learning: 2025-12-15T12:03:17.291Z
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-15T12:03:17.291Z
Learning: Mandatory context before changes: read pmoves/docs/PMOVES.AI PLANS/ROADMAP.md and pmoves/docs/NEXT_STEPS.md to align with current sprint focus
Applied to files:
pmoves/docker-compose.yml
📚 Learning: 2025-12-07T11:03:53.415Z
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: pmoves/AGENTS.md:0-0
Timestamp: 2025-12-07T11:03:53.415Z
Learning: Applies to pmoves/**/docker-compose.yml : Use Compose profiles (`data`, `workers`) to scope what runs locally in docker-compose.yml
Applied to files:
pmoves/docker-compose.yml
📚 Learning: 2025-12-15T12:03:17.291Z
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-15T12:03:17.291Z
Learning: Agents/Archon: for full-stack validation, follow the 'All Services Up, Then Tests' section in pmoves/docs/SMOKETESTS.md and the Archon service guide
Applied to files:
pmoves/docker-compose.yml
🔇 Additional comments (4)
pmoves/docker-compose.yml (4)
59-59: Excellent security improvement - enforcing required POSTGRES_PASSWORD.The use of
${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required}syntax correctly prevents the stack from starting with default or missing database credentials. This eliminates a common security misconfiguration.Also applies to: 78-78, 937-937
64-64: Excellent use of Compose profiles for service scoping.The profile structure (
data,workers,orchestration,agents,gpu,tensorzero, etc.) follows the coding guidelines and allows fine-grained control over which services run locally. This makes it easy to:
- Run just the data tier:
docker-compose --profile data up- Run worker services:
docker-compose --profile workers up- Run GPU-accelerated services:
docker-compose --profile gpu up- Run specific stacks like TensorZero or Invidious
Services appropriately use multiple profiles where needed (e.g., MinIO in
data,orchestration,workers,agents).As per coding guidelines, Compose profiles (
data,workers) are used to scope what runs locally in docker-compose.yml.Also applies to: 84-84, 171-171, 203-203, 230-230, 254-254, 273-273, 308-308, 334-334, 366-366, 397-397, 428-428, 452-452, 480-480, 508-508, 552-552, 636-636, 667-667, 696-696, 735-735, 749-749, 777-777, 826-826, 880-880, 957-957, 994-994, 1022-1022, 1061-1061, 1246-1246, 1282-1282, 1307-1307, 1376-1376, 1394-1394, 1411-1411, 1437-1437, 1467-1467, 1488-1488, 1516-1516, 1543-1543, 1567-1567, 1610-1610, 1625-1625, 1646-1646, 1677-1677
1024-1029: Verify NATS monitoring port is exposed.The health check attempts to reach
http://localhost:8222/healthz, which is NATS's HTTP monitoring port. The service definition at line 1018 only exposes port 4222 (NATS client port). Verify that:
- NATS image
nats:2.10-alpineexposes port 8222 by default for monitoring, or- Port 8222 needs to be added to the
portssection if external health checks are required.#!/bin/bash # Description: Verify NATS monitoring port configuration # Check if NATS exposes port 8222 for monitoring by default echo "Checking NATS 2.10-alpine image for exposed ports..." docker pull nats:2.10-alpine 2>/dev/null docker inspect nats:2.10-alpine | jq -r '.[0].Config.ExposedPorts | keys[]' # Search for NATS health check patterns in codebase echo -e "\nSearching for NATS monitoring port usage in codebase..." rg -n "8222" --type yaml -g "docker-compose*.yml"
73-73: Good practice: Image version pinning for reproducibility and security.Pinning images to explicit versions (postgrest:v12.2.3, minio:RELEASE.2024-12-18T13-15-44Z, ollama:0.5.4, tensorzero:2024.12.18, invidious:2024.12.09, grayjay:2024.11.01) prevents unexpected changes from
:latesttags and makes deployments reproducible.Also applies to: 104-104, 163-163, 1421-1421, 1471-1471, 1500-1500, 1547-1547, 1571-1571, 1635-1635
| deploy: | ||
| resources: | ||
| reservations: | ||
| devices: | ||
| - driver: nvidia | ||
| count: 1 | ||
| capabilities: [ gpu ] |
There was a problem hiding this comment.
Inconsistent GPU count configuration in gpu-orchestrator.
The gpu-orchestrator service hardcodes count: 1 while other GPU services use count: ${GPU_COUNT:-all} (e.g., ffmpeg-whisper line 634, media-video line 665, media-audio line 694). This inconsistency could cause issues in multi-GPU environments where users want centralized GPU allocation control via the GPU_COUNT variable.
🔎 Proposed fix to use GPU_COUNT variable
deploy:
resources:
reservations:
devices:
- driver: nvidia
- count: 1
+ count: ${GPU_COUNT:-1}
capabilities: [ gpu ]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| deploy: | |
| resources: | |
| reservations: | |
| devices: | |
| - driver: nvidia | |
| count: 1 | |
| capabilities: [ gpu ] | |
| deploy: | |
| resources: | |
| reservations: | |
| devices: | |
| - driver: nvidia | |
| count: ${GPU_COUNT:-1} | |
| capabilities: [ gpu ] |
🤖 Prompt for AI Agents
In pmoves/docker-compose.yml around lines 1000 to 1006, the gpu-orchestrator
service hardcodes GPUs with `count: 1` while other GPU services use the
environment variable `${GPU_COUNT:-all}`; change the hardcoded `count: 1` to
`count: ${GPU_COUNT:-all}` (or the project-standard variable) so
gpu-orchestrator follows the centralized GPU_COUNT setting, maintaining
consistent multi-GPU behavior across services.
Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
pmoves/docker-compose.yml (1)
1002-1008: gpu-orchestrator still uses hardcoded GPU count.This issue was previously flagged:
gpu-orchestratorhardcodescount: 1while other GPU services usecount: ${GPU_COUNT:-all}. This inconsistency prevents centralized GPU allocation control.
🧹 Nitpick comments (1)
pmoves/docker-compose.yml (1)
512-518: Inconsistent GPU count in ultimate-tts-studio.The
ultimate-tts-studioservice hardcodescount: 1while most other GPU services usecount: ${GPU_COUNT:-all}(e.g., media-video line 667, media-audio line 696). Consider using theGPU_COUNTvariable for consistency.🔎 Proposed fix for consistency
deploy: resources: reservations: devices: - driver: nvidia - count: 1 + count: ${GPU_COUNT:-1} capabilities: [ gpu ]
📜 Review details
Configuration used: defaults
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
pmoves/docker-compose.yml
🧰 Additional context used
📓 Path-based instructions (1)
pmoves/**/docker-compose.yml
📄 CodeRabbit inference engine (pmoves/AGENTS.md)
Use Compose profiles (
data,workers) to scope what runs locally in docker-compose.yml
Files:
pmoves/docker-compose.yml
🧠 Learnings (4)
📓 Common learnings
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: GEMINI.md:0-0
Timestamp: 2025-12-07T11:03:07.638Z
Learning: Applies to **/pmoves/**/{media,analysis,pipeline}*.py : Implement `media-video` and `media-audio` analysis pipelines with GPU auto-detect for faster-whisper
📚 Learning: 2025-12-07T11:03:53.415Z
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: pmoves/AGENTS.md:0-0
Timestamp: 2025-12-07T11:03:53.415Z
Learning: Applies to pmoves/services/{agent-zero,archon}/**/*.py : For Agents/Archon full-stack validation, follow the 'All Services Up, Then Tests' section in `pmoves/docs/SMOKETESTS.md` and use `make -C pmoves agents-headless-smoke`, `make -C pmoves smoke-gpu`, and `make -C pmoves verify-all`
Applied to files:
pmoves/docker-compose.yml
📚 Learning: 2025-12-07T11:03:53.415Z
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: pmoves/AGENTS.md:0-0
Timestamp: 2025-12-07T11:03:53.415Z
Learning: Applies to pmoves/**/docker-compose.yml : Use Compose profiles (`data`, `workers`) to scope what runs locally in docker-compose.yml
Applied to files:
pmoves/docker-compose.yml
📚 Learning: 2025-12-15T12:03:17.291Z
Learnt from: CR
Repo: POWERFULMOVES/PMOVES.AI PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-12-15T12:03:17.291Z
Learning: Agents/Archon: for full-stack validation, follow the 'All Services Up, Then Tests' section in pmoves/docs/SMOKETESTS.md and the Archon service guide
Applied to files:
pmoves/docker-compose.yml
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Analyze (python)
🔇 Additional comments (15)
pmoves/docker-compose.yml (15)
59-59: Good security improvement - POSTGRES_PASSWORD now required.The required password syntax prevents deployments with unset credentials. The implementation is consistent across all three usages.
Also applies to: 78-78, 939-939
66-71: LGTM - postgres health check start_period added.The 15s start period gives postgres adequate initialization time before health checks begin.
123-128: LGTM - data tier health checks are appropriate.Health checks for qdrant, meilisearch, and minio use correct endpoints and reasonable timing parameters. The MinIO check correctly uses
curlinstead ofmcwhich is not available in the image.Also applies to: 138-143, 174-179
1020-1031: LGTM - NATS monitoring port enables health checks.The
-m 8222flag enables the HTTP monitoring endpoint, which the health check correctly uses. This is a proper operational improvement.
311-317: Health check improved to verify module imports.The health check now verifies that the watcher module and its dependencies can be imported. While this doesn't test runtime connectivity to NATS/MinIO, it catches missing dependencies and import errors at the Docker health check level.
1251-1257: Health check improved to verify main module and NATS client availability.The health check now imports the main module and verifies the NATS client is available, catching dependency and configuration issues at startup.
340-345: LGTM - worker service health checks are well-configured.All worker services use appropriate
/healthzendpoints with consistent timing. GPU-enabled workers (ffmpeg-whisper, media-video, media-audio) correctly use longer start periods (60s) to account for GPU initialization.Also applies to: 370-375, 401-406, 431-436, 456-461, 484-489, 640-645, 671-676, 700-705
213-218: LGTM - API service health checks are properly configured.All API services use appropriate health endpoints with reasonable timing. The GPU-enabled v2 gateway correctly uses a 90s start period to accommodate model loading time.
Also applies to: 235-240, 257-262, 279-284, 558-563, 739-744, 753-758, 781-786, 836-841, 891-896, 970-975
1065-1072: LGTM - agent service health checks are appropriate.Agent services use proper health endpoints with consistent timing. Agent-zero correctly uses a longer 60s start period and includes host.docker.internal access for external API calls.
Also applies to: 1291-1296, 1313-1318, 1383-1388, 1401-1406, 1418-1423
1448-1453: LGTM - TensorZero stack health checks are well-configured.Ollama, gateway, and UI services all have appropriate health endpoints. Ollama correctly uses a 60s start period for GPU initialization.
Also applies to: 1497-1502, 1523-1530
1543-1548: LGTM - invidious-db health check start_period added.The 15s start period is consistent with the main postgres service and gives adequate initialization time.
1633-1638: LGTM - Grayjay service health checks are appropriate.Both plugin-host and server use suitable health endpoints with reasonable timing parameters.
Also applies to: 1654-1659
1210-1215: LGTM - E2B runner health check start_period added.The 30s start period completes the health check configuration with a reasonable initialization window.
1336-1341: LGTM - consciousness-service health check start_period added.The 30s start period appropriately completes the health check configuration.
74-74: Compose profiles already properly used; verify tensorzero/gateway version.Image pinning is correct—all 8 images use specific versions instead of
:latest. Compose profiles (data,workers, etc.) are already extensively used throughout the file, meeting the guideline requirement. However, thetensorzero/gateway:2024.12.18tag could not be verified; confirm this version exists on Docker Hub or update to a verified release tag.Also applies to: 74-74, 105-105, 164-164, 1426-1426, 1476-1476, 1505-1505, 1553-1553, 1577-1577, 1641-1641
Resolve conflict in env.tier-agent.example - keep main's version which has more comprehensive service list and better formatting.
This commit implements the PR #392 accessibility and silent failure fixes, standardizing healthchecks across 40+ services and adding production safeguards. ## Healthcheck Standardization (Golden Rule Compliance) **Replaced all `python3 -c` healthchecks with `curl -sf`** (38 services) - Python3 healthchecks fail in minimal images (ollama, alpine) - Pattern from PR #355: "NEVER use python3 -c in healthchecks" - Now uses: `test: ["CMD", "curl", "-sf", "http://localhost:PORT/healthz"]` Services fixed: - hi-rag-gateway-v2, hi-rag-gateway, presign, render-webhook - extract-worker, pdf-ingest, langextract, notebook-sync - session-context-worker, chat-relay, flute-gateway, ffmpeg-whisper - media-video, media-audio, pmoves-yt, gpu-orchestrator - agent-zero, archon, botz-gateway, gateway-agent, consciousness-service - tensorzero-ui, retrieval-eval, publisher-discord, messaging-gateway - tokenism-simulator, jellyfin-bridge, deepresearch, supaserch - github-runner-ctl, a2ui-nats-bridge, and more... ## Missing Healthchecks Added - **postgrest** (port 3010): Added healthcheck using root endpoint - **postgrest-cli** (port 3011): Added healthcheck using root endpoint - **invidious-db**: Already had proper `pg_isready` healthcheck ✅ ## Non-Standard Endpoints Fixed - **archon-agent-work-orders**: Added `/healthz` endpoint (server.py) - **pmz-e2b-runner**: Added `/healthz` endpoint (app_e2b.py) Both now return simplified status matching PMOVES standard pattern. ## Dev Server Production Safeguards **Makefile (ui-dev-start)**: Prevents dev server in production ```makefile @if [ "$${NODE_ENV:-}" = "production" ]; then \ echo "❌ ERROR: Development server cannot start in production"; \ exit 1; \ fi ``` **package.json**: Explicitly sets NODE_ENV=development ```json "dev:3001": "NODE_ENV=development PORT=3001 node scripts/with-env.mjs next dev -p 3001" ``` ## Files Modified - `pmoves/docker-compose.yml`: 38+ healthcheck replacements + 2 new - `pmoves/Makefile`: Added NODE_ENV production check - `pmoves/ui/package.json`: Set NODE_ENV=development in dev:3001 - `pmoves/integrations/archon`: Added /healthz endpoint - `PMOVES-BoTZ/features/e2b`: Added /healthz endpoint 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix(security): GPU access + production hardening (health checks, image pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(hirag): use container DNS for Supabase realtime instead of host.docker.internal (#335) The host.docker.internal hostname resolves to Docker Desktop's gateway IP (192.168.65.254) in WSL2, which doesn't properly route to host-bound ports. This caused Hi-RAG to fail connecting to Supabase realtime after Docker restarts. Changed default SUPABASE_REALTIME_URL from: ws://host.docker.internal:65421/realtime/v1 to: ws://supabase_kong_PMOVES.AI:8000/realtime/v1 Since both containers are on pmoves-net, direct container-to-container DNS resolution works reliably. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(tts): use nvidia/cuda base and fix Higgs Audio pydantic error - Switch runtime stage to nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04 (includes nvrtc and full CUDA toolkit for ONNX GPU provider) - Add LD_LIBRARY_PATH for CUDA libraries - Install cuda-nvrtc-12-4 package - Update submodule with Higgs Audio dataclass fix (default_factory) - Follow Pinokio-matched installation order for dependencies - Add MCP server support (GRADIO_MCP_SERVER=true) - Add WSL2/CUDA compatibility settings Fixes: - libnvrtc.so.12 missing error - HiggsAudioResponse mutable default ValueError - All 7 TTS engines now load correctly 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…tion (#443) * refactor(env): Phase 2 - Migrate API tier to env-tier-api anchor (#350) Phase 2 of env consolidation: API tier migration Changes: - Fix tier anchor syntax (block-style env_file with required: false) - Migrate postgrest, presign, retrieval-eval to <<: *env-tier-api - Create env.tier-api.example with secure defaults API tier services receive data tier URLs and internal credentials only, no external API keys (OPENAI_API_KEY, etc.) Services migrated: - postgrest: PostgREST database gateway - presign: MinIO URL presigner - retrieval-eval: Hi-RAG evaluation service 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * refactor(env): Phase 3 - Migrate LLM tier to env-tier-llm anchor (#351) Phase 3 of env consolidation: LLM tier migration Changes: - Fix tier anchor syntax (block-style env_file with required: false) - Migrate tensorzero-ui to <<: *env-tier-llm - Create env.tier-llm.example with all LLM provider API keys CRITICAL: LLM tier is the ONLY tier with access to external API keys. All other services call TensorZero internally, not providers directly. Services in LLM tier: - tensorzero-gateway: Already using <<: *env-tier-llm - tensorzero-ui: Migrated from legacy env_file - tensorzero-clickhouse: Uses inline env (no API keys needed) - pmoves-ollama: Uses inline env (no API keys needed) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * refactor(env): Phase 6 - Migrate agent tier + API additions (#354) Completes the 6-tier environment consolidation by migrating: **Agent Tier (10 services):** - mesh-agent, deepresearch, supaserch, consciousness-service - archon-agent-work-orders, botz-gateway, publisher-discord - messaging-gateway, jellyfin-bridge, chat-relay **API Tier Additions (4 services):** - postgrest-cli, hi-rag-gateway, hi-rag-gateway-gpu, hi-rag-gateway-v2-gpu **Worker Tier (6 services) - included in this commit:** - render-webhook, comfy-watcher, pdf-ingest, langextract - notebook-sync, session-context-worker **Media Tier (10 services) - included in this commit:** - ultimate-tts-studio, flute-gateway, ffmpeg-whisper - media-video, media-audio, channel-monitor - invidious, invidious-companion, grayjay-plugin-host, grayjay-server Security improvement: 30 services now use tier-based env_file anchors instead of legacy x-env-legacy pattern. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): GPU access + production hardening (health checks, image pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * security(dockerfile): Harden notebook-sync and vibevoice-realtime Increase Dockerfile hardening from 33/36 (91.7%) to 35/36 (97.2%): - notebook-sync: Add USER pmoves (user creation already existed) - vibevoice-realtime: Add full hardening with UID/GID 65532 Accepted exception: agent-zero uses root for initialization then drops to pmoves via 'su' for the service process. This pattern is required by upstream Agent Zero's prepare.py and /ins/copy_A0.sh. Pattern reference: flute-gateway/Dockerfile 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(infra): Address PR #345 CodeRabbit review comments - Normalize Supabase Kong hostnames to lowercase (DNS convention) - Remove duplicate ARCHON_SUPABASE_BASE_URL env var definition - Fix notebook-sync Dockerfile: remove USER directive so entrypoint can run chown/su as root before dropping privileges - Improve deepresearch _get_or_create_counter: use module-level cache to avoid private prometheus_client API dependency 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(infra): Archon Supabase connectivity and UI dashboard fixes Changes: - Archon health check: Add Kong gateway hostname support for internal Docker networking (supabase_kong_pmoves.ai:8000) - UI dashboard: Add /dashboard redirect page to /dashboard/services - Grafana: Fix env var placeholders and job name mismatches in queries - Docs: Add placeholder for Jellyfin service documentation The Archon health check now properly handles both: - Supabase CLI endpoint (host.docker.internal:65421) - Internal Kong gateway (supabase_kong_pmoves.ai:8000) This resolves the 404 errors in Archon health checks when using internal Docker DNS names instead of host.docker.internal. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore(gitignore): Exclude agent-zero runtime data directory Add pmoves/data/agent-zero/ to gitignore. This directory contains runtime settings (settings.json) that persist via Docker volume mount. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deepresearch): Prometheus counter double-registration Use registry lookup pattern instead of try/except for counter registration. Prevents 'Duplicated timeseries' error when container restarts with existing registry state. Before: try/except around Counter() creation After: REGISTRY._names_to_collectors.get() for existing lookup 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(flute-gateway): add logging to silent exception handlers (#323) - Add warning log to Supabase health check exception handler - Improve persona fetch error logging with status code and truncated body - Add metrics tracking for non-200 persona fetch responses Closes #322 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * docs: add comprehensive service documentation (#338) Merged after rebase to resolve Jellyfin README conflict * fix(compose): Agent Zero settings persistence and Archon hostname - Add volume mount ./data/agent-zero/tmp:/a0/tmp for settings.json Root cause: PMOVES-Agent-Zero/python/helpers/settings.py:162 stores settings at /a0/tmp/settings.json but path was not mounted - Normalize Archon SUPABASE_URL to lowercase supabase_kong_pmoves.ai Root cause: Docker DNS is case-sensitive on some configurations - Fix archon-agent-work-orders health check: /healthz → /health Root cause: Service exposes /health, not /healthz 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat(infra): implement 6-tier environment architecture with 5-tier network segmentation This commit implements the comprehensive tier architecture for improved security and organization of PMOVES.AI infrastructure. ## Changes ### 6-Tier Environment Architecture (Security-Based) - tier-data: Infrastructure credentials (postgres, qdrant, neo4j, meilisearch, minio, nats) - tier-api: Data access APIs (postgrest, presign, hi-rag-gateway) - tier-llm: ALL external LLM API keys (tensorzero-gateway, tensorzero-ui, ollama) - tier-worker: Background workers (extract-worker, langextract, pdf-ingest, notebook-sync) - tier-media: Media processing (pmoves-yt, ffmpeg-whisper, media-video, media-audio) - tier-agent: Agent orchestration (agent-zero, archon, supaserch, deepresearch) ### 5-Tier Docker Network Architecture - pmoves_api (172.30.1.0/24): Public-facing services - pmoves_app (172.30.2.0/24): Application services - pmoves_bus (172.30.3.0/24): NATS message bus - pmoves_data (172.30.4.0/24): Data stores - pmoves_monitoring (172.30.5.0/24): Observability ### Security Improvements - Compose-time secret expansion removed (now runtime from tier env files) - YAML anchors for reusable tier env configurations - Network segmentation for inter-tier communication control - tier-llm is the ONLY tier with external LLM API keys ### Services Updated (45 core services) All core services now use `<<: *env-tier-*` anchors and assigned to correct network tiers. External/integration services (Invidious, Grayjay, Cloudflare) retain existing env_file pattern. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): GPU access + production hardening (health checks, image pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * feat(security): Add gateway-agent hardening, Z.ai provider, Prometheus fix (#374) * feat(infra): Add Prometheus metrics to Tier 4 services (#361) Add prometheus-client instrumentation to: - session-context-worker (port 8100): Convert JSON /metrics to Prometheus format * Counter: messages_received_total (by subject) * Counter: messages_processed_total (by context_type) * Counter: messages_failed_total (by error_type) * Counter: kb_upserts_published_total (by namespace) * Histogram: processing_duration_seconds (by context_type) - messaging-gateway (port 8101): Add comprehensive metrics * Counter: messages_sent_total (by platform, status) * Counter: nats_messages_received_total (by subject) * Counter: api_requests_total (by endpoint, status) * Histogram: request_duration_seconds (by endpoint) Cherry-picked from: 769a938 * fix(infra): Add missing Prometheus metric definitions (#362) Fix missing Prometheus metric definitions for Tier 4 services follow-up. Cherry-picked from: 6532c34 * feat(infra): Restore GPU Orchestrator to docker-compose.yml (#P3) - Added gpu-orchestrator service entry - Uses tier-llm env configuration - Requires NVIDIA GPU with nvidia-docker runtime - Networks: pmoves_api, pmoves_bus - Port: 8100 - Config: ./config/gpu-models.yaml - Docker socket mount for container management * feat(infra): Add missing feature services to docker-compose.yml (#P4) Added services: - evo-controller (port 8113): CHIT geometry parameter optimization via EvoSwarm * Profile: orchestration * Network: pmoves_app, pmoves_bus * Tier: tier-agent (SUPABASE access) - flute-gateway (ports 8055, 8056): Multimodal voice communication layer (TTS/STT) * Profiles: orchestration, media * Network: pmoves_app, pmoves_bus * Tier: tier-media - session-context-worker (host port 8102): Claude Code session to Hi-RAG KB transformer * Profile: workers * Network: pmoves_app, pmoves_bus * Tier: tier-worker * Depends on: hi-rag-gateway-v2 Skipped: - ultimate-tts-studio: Has own docker-compose.ultimate-tts-studio.yml - tensorzero-config-api: Directory is empty, not yet implemented --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix(security): GPU access + production hardening (health checks, image pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(hirag): use container DNS for Supabase realtime instead of host.docker.internal (#335) The host.docker.internal hostname resolves to Docker Desktop's gateway IP (192.168.65.254) in WSL2, which doesn't properly route to host-bound ports. This caused Hi-RAG to fail connecting to Supabase realtime after Docker restarts. Changed default SUPABASE_REALTIME_URL from: ws://host.docker.internal:65421/realtime/v1 to: ws://supabase_kong_PMOVES.AI:8000/realtime/v1 Since both containers are on pmoves-net, direct container-to-container DNS resolution works reliably. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(tts): use nvidia/cuda base and fix Higgs Audio pydantic error - Switch runtime stage to nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04 (includes nvrtc and full CUDA toolkit for ONNX GPU provider) - Add LD_LIBRARY_PATH for CUDA libraries - Install cuda-nvrtc-12-4 package - Update submodule with Higgs Audio dataclass fix (default_factory) - Follow Pinokio-matched installation order for dependencies - Add MCP server support (GRADIO_MCP_SERVER=true) - Add WSL2/CUDA compatibility settings Fixes: - libnvrtc.so.12 missing error - HiggsAudioResponse mutable default ValueError - All 7 TTS engines now load correctly 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…tion (#443) * refactor(env): Phase 2 - Migrate API tier to env-tier-api anchor (#350) Phase 2 of env consolidation: API tier migration Changes: - Fix tier anchor syntax (block-style env_file with required: false) - Migrate postgrest, presign, retrieval-eval to <<: *env-tier-api - Create env.tier-api.example with secure defaults API tier services receive data tier URLs and internal credentials only, no external API keys (OPENAI_API_KEY, etc.) Services migrated: - postgrest: PostgREST database gateway - presign: MinIO URL presigner - retrieval-eval: Hi-RAG evaluation service 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * refactor(env): Phase 3 - Migrate LLM tier to env-tier-llm anchor (#351) Phase 3 of env consolidation: LLM tier migration Changes: - Fix tier anchor syntax (block-style env_file with required: false) - Migrate tensorzero-ui to <<: *env-tier-llm - Create env.tier-llm.example with all LLM provider API keys CRITICAL: LLM tier is the ONLY tier with access to external API keys. All other services call TensorZero internally, not providers directly. Services in LLM tier: - tensorzero-gateway: Already using <<: *env-tier-llm - tensorzero-ui: Migrated from legacy env_file - tensorzero-clickhouse: Uses inline env (no API keys needed) - pmoves-ollama: Uses inline env (no API keys needed) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * refactor(env): Phase 6 - Migrate agent tier + API additions (#354) Completes the 6-tier environment consolidation by migrating: **Agent Tier (10 services):** - mesh-agent, deepresearch, supaserch, consciousness-service - archon-agent-work-orders, botz-gateway, publisher-discord - messaging-gateway, jellyfin-bridge, chat-relay **API Tier Additions (4 services):** - postgrest-cli, hi-rag-gateway, hi-rag-gateway-gpu, hi-rag-gateway-v2-gpu **Worker Tier (6 services) - included in this commit:** - render-webhook, comfy-watcher, pdf-ingest, langextract - notebook-sync, session-context-worker **Media Tier (10 services) - included in this commit:** - ultimate-tts-studio, flute-gateway, ffmpeg-whisper - media-video, media-audio, channel-monitor - invidious, invidious-companion, grayjay-plugin-host, grayjay-server Security improvement: 30 services now use tier-based env_file anchors instead of legacy x-env-legacy pattern. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): GPU access + production hardening (health checks, image pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * security(dockerfile): Harden notebook-sync and vibevoice-realtime Increase Dockerfile hardening from 33/36 (91.7%) to 35/36 (97.2%): - notebook-sync: Add USER pmoves (user creation already existed) - vibevoice-realtime: Add full hardening with UID/GID 65532 Accepted exception: agent-zero uses root for initialization then drops to pmoves via 'su' for the service process. This pattern is required by upstream Agent Zero's prepare.py and /ins/copy_A0.sh. Pattern reference: flute-gateway/Dockerfile 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(infra): Address PR #345 CodeRabbit review comments - Normalize Supabase Kong hostnames to lowercase (DNS convention) - Remove duplicate ARCHON_SUPABASE_BASE_URL env var definition - Fix notebook-sync Dockerfile: remove USER directive so entrypoint can run chown/su as root before dropping privileges - Improve deepresearch _get_or_create_counter: use module-level cache to avoid private prometheus_client API dependency 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(infra): Archon Supabase connectivity and UI dashboard fixes Changes: - Archon health check: Add Kong gateway hostname support for internal Docker networking (supabase_kong_pmoves.ai:8000) - UI dashboard: Add /dashboard redirect page to /dashboard/services - Grafana: Fix env var placeholders and job name mismatches in queries - Docs: Add placeholder for Jellyfin service documentation The Archon health check now properly handles both: - Supabase CLI endpoint (host.docker.internal:65421) - Internal Kong gateway (supabase_kong_pmoves.ai:8000) This resolves the 404 errors in Archon health checks when using internal Docker DNS names instead of host.docker.internal. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore(gitignore): Exclude agent-zero runtime data directory Add pmoves/data/agent-zero/ to gitignore. This directory contains runtime settings (settings.json) that persist via Docker volume mount. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(deepresearch): Prometheus counter double-registration Use registry lookup pattern instead of try/except for counter registration. Prevents 'Duplicated timeseries' error when container restarts with existing registry state. Before: try/except around Counter() creation After: REGISTRY._names_to_collectors.get() for existing lookup 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(flute-gateway): add logging to silent exception handlers (#323) - Add warning log to Supabase health check exception handler - Improve persona fetch error logging with status code and truncated body - Add metrics tracking for non-200 persona fetch responses Closes #322 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * docs: add comprehensive service documentation (#338) Merged after rebase to resolve Jellyfin README conflict * fix(compose): Agent Zero settings persistence and Archon hostname - Add volume mount ./data/agent-zero/tmp:/a0/tmp for settings.json Root cause: PMOVES-Agent-Zero/python/helpers/settings.py:162 stores settings at /a0/tmp/settings.json but path was not mounted - Normalize Archon SUPABASE_URL to lowercase supabase_kong_pmoves.ai Root cause: Docker DNS is case-sensitive on some configurations - Fix archon-agent-work-orders health check: /healthz → /health Root cause: Service exposes /health, not /healthz 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat(infra): implement 6-tier environment architecture with 5-tier network segmentation This commit implements the comprehensive tier architecture for improved security and organization of PMOVES.AI infrastructure. ## Changes ### 6-Tier Environment Architecture (Security-Based) - tier-data: Infrastructure credentials (postgres, qdrant, neo4j, meilisearch, minio, nats) - tier-api: Data access APIs (postgrest, presign, hi-rag-gateway) - tier-llm: ALL external LLM API keys (tensorzero-gateway, tensorzero-ui, ollama) - tier-worker: Background workers (extract-worker, langextract, pdf-ingest, notebook-sync) - tier-media: Media processing (pmoves-yt, ffmpeg-whisper, media-video, media-audio) - tier-agent: Agent orchestration (agent-zero, archon, supaserch, deepresearch) ### 5-Tier Docker Network Architecture - pmoves_api (172.30.1.0/24): Public-facing services - pmoves_app (172.30.2.0/24): Application services - pmoves_bus (172.30.3.0/24): NATS message bus - pmoves_data (172.30.4.0/24): Data stores - pmoves_monitoring (172.30.5.0/24): Observability ### Security Improvements - Compose-time secret expansion removed (now runtime from tier env files) - YAML anchors for reusable tier env configurations - Network segmentation for inter-tier communication control - tier-llm is the ONLY tier with external LLM API keys ### Services Updated (45 core services) All core services now use `<<: *env-tier-*` anchors and assigned to correct network tiers. External/integration services (Invidious, Grayjay, Cloudflare) retain existing env_file pattern. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): GPU access + production hardening (health checks, image pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * feat(security): Add gateway-agent hardening, Z.ai provider, Prometheus fix (#374) * feat(infra): Add Prometheus metrics to Tier 4 services (#361) Add prometheus-client instrumentation to: - session-context-worker (port 8100): Convert JSON /metrics to Prometheus format * Counter: messages_received_total (by subject) * Counter: messages_processed_total (by context_type) * Counter: messages_failed_total (by error_type) * Counter: kb_upserts_published_total (by namespace) * Histogram: processing_duration_seconds (by context_type) - messaging-gateway (port 8101): Add comprehensive metrics * Counter: messages_sent_total (by platform, status) * Counter: nats_messages_received_total (by subject) * Counter: api_requests_total (by endpoint, status) * Histogram: request_duration_seconds (by endpoint) Cherry-picked from: 769a938 * fix(infra): Add missing Prometheus metric definitions (#362) Fix missing Prometheus metric definitions for Tier 4 services follow-up. Cherry-picked from: 6532c34 * feat(infra): Restore GPU Orchestrator to docker-compose.yml (#P3) - Added gpu-orchestrator service entry - Uses tier-llm env configuration - Requires NVIDIA GPU with nvidia-docker runtime - Networks: pmoves_api, pmoves_bus - Port: 8100 - Config: ./config/gpu-models.yaml - Docker socket mount for container management * feat(infra): Add missing feature services to docker-compose.yml (#P4) Added services: - evo-controller (port 8113): CHIT geometry parameter optimization via EvoSwarm * Profile: orchestration * Network: pmoves_app, pmoves_bus * Tier: tier-agent (SUPABASE access) - flute-gateway (ports 8055, 8056): Multimodal voice communication layer (TTS/STT) * Profiles: orchestration, media * Network: pmoves_app, pmoves_bus * Tier: tier-media - session-context-worker (host port 8102): Claude Code session to Hi-RAG KB transformer * Profile: workers * Network: pmoves_app, pmoves_bus * Tier: tier-worker * Depends on: hi-rag-gateway-v2 Skipped: - ultimate-tts-studio: Has own docker-compose.ultimate-tts-studio.yml - tensorzero-config-api: Directory is empty, not yet implemented --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355) * docs: address PR review comments for env tier consolidation - Add GPU Orchestrator and E2B Runner to services-catalog.md - Add gpu-orchestrator to env.tier-api.example service list - Create learnings file documenting 6-tier env architecture - Add env.tier-*.example files for worker, media, agent tiers Addresses nitpick comments from PRs #349-354. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(gpu): ensure GPU access for all CUDA-enabled services - gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES - ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var - hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var - media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04 (was python:3.11-slim which caused silent CPU fallback) This fixes silent CPU fallbacks where PyTorch CUDA packages were installed but the CUDA runtime was not available in the container. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(security): production hardening - health checks and image pinning Critical security and reliability fixes: 1. **POSTGRES_PASSWORD now required** - Changed from insecure default `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if POSTGRES_PASSWORD is not set (lines 59, 78, 937). 2. **Pinned 8 :latest images to specific versions:** - postgrest/postgrest:latest → v12.2.3 - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z - ollama/ollama:latest → 0.5.4 - tensorzero/gateway:latest → 2024.12.18 - tensorzero/ui:latest → 2024.12.18 - invidious:latest → 2024.12.09 - invidious-companion:latest → 2024.12.20 - grayjay:latest → 2024.11.01 3. **Added health checks to 37 services** (52 total, up from 15): - Data tier: qdrant, meilisearch, minio - API tier: hi-rag-*, retrieval-eval, presign, render-webhook - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync, ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch - TensorZero: gateway, ui, ollama - Others: publisher-discord, messaging-gateway, jellyfin-bridge 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(health): address PR review critical issues Fixes critical issues identified by PR review: MinIO: - Change health check from `mc ready local` to curl-based (minio/minio image doesn't include mc binary) NATS: - Add `-m 8222` flag to enable HTTP monitoring port (health check was targeting port that wasn't enabled) Health checks: - comfy-watcher: verify module imports instead of just `import sys` - mesh-agent: verify main module and NATS client availability Missing start_period: - postgres: add 15s start_period - chat-relay: add 30s start_period - n8n-agent: add 30s start_period - invidious-postgres: add 15s start_period All 52 health checks now have start_period defined. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Codex Agent <codex-agent@example.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Summary
This PR combines GPU configuration fixes with comprehensive production hardening:
1. GPU Access Fixes ✅
2. POSTGRES_PASSWORD Now Required ✅ (Security Critical)
:-pmovesdefault to required syntax${VAR:?error}3. Pinned :latest Images ✅ (8 images)
postgrest/postgrest:latestv12.2.3minio/minio:latestRELEASE.2024-12-18T13-15-44Zollama/ollama:latest0.5.4tensorzero/gateway:latest2024.12.18tensorzero/ui:latest2024.12.18invidious:latest2024.12.09invidious-companion:latest2024.12.20grayjay:latest2024.11.014. Health Checks ✅ (37 new, 52 total)
Added health checks to services that were missing them:
Test Plan
docker compose configvalidates successfully with required vars:latesttags only remain in comments (Docker MCP TODO)Breaking Changes
.env.localor env tier file.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation
Chores
✏️ Tip: You can customize this high-level summary in your review settings.