Skip to content

[FEATURE] Security Phase 2 - PyTorch 2.6, Docker hardening, env migration - #436

Closed
POWERFULMOVES wants to merge 15 commits into
PMOVES.AI-Edition-Hardenedfrom
feat/hardened-security-phase2
Closed

POWERFULMOVES wants to merge 15 commits into
PMOVES.AI-Edition-Hardenedfrom
feat/hardened-security-phase2

Conversation

@POWERFULMOVES

Copy link
Copy Markdown
Owner

Summary

This PR contains 13 commits from `feat/hardened-security-phase2` not yet in hardened.

Changes

Security Hardening

Environment Migration (6-Tier Architecture)

Infrastructure Fixes

Documentation

Testing

```bash
docker compose -p pmoves config --quiet
curl -sf http://localhost:8080/healthz # Agent Zero
curl -sf http://localhost:8091/healthz # Archon
```

Related

Follows env migration pattern from PR #435 (submodule-sync)

POWERFULMOVES and others added 15 commits January 1, 2026 20:59
Phase 2 of env consolidation: API tier migration

Changes:
- Fix tier anchor syntax (block-style env_file with required: false)
- Migrate postgrest, presign, retrieval-eval to <<: *env-tier-api
- Create env.tier-api.example with secure defaults

API tier services receive data tier URLs and internal credentials only,
no external API keys (OPENAI_API_KEY, etc.)

Services migrated:
- postgrest: PostgREST database gateway
- presign: MinIO URL presigner
- retrieval-eval: Hi-RAG evaluation service

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Codex Agent <codex-agent@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Phase 3 of env consolidation: LLM tier migration

Changes:
- Fix tier anchor syntax (block-style env_file with required: false)
- Migrate tensorzero-ui to <<: *env-tier-llm
- Create env.tier-llm.example with all LLM provider API keys

CRITICAL: LLM tier is the ONLY tier with access to external API keys.
All other services call TensorZero internally, not providers directly.

Services in LLM tier:
- tensorzero-gateway: Already using <<: *env-tier-llm
- tensorzero-ui: Migrated from legacy env_file
- tensorzero-clickhouse: Uses inline env (no API keys needed)
- pmoves-ollama: Uses inline env (no API keys needed)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Codex Agent <codex-agent@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Completes the 6-tier environment consolidation by migrating:

**Agent Tier (10 services):**
- mesh-agent, deepresearch, supaserch, consciousness-service
- archon-agent-work-orders, botz-gateway, publisher-discord
- messaging-gateway, jellyfin-bridge, chat-relay

**API Tier Additions (4 services):**
- postgrest-cli, hi-rag-gateway, hi-rag-gateway-gpu, hi-rag-gateway-v2-gpu

**Worker Tier (6 services) - included in this commit:**
- render-webhook, comfy-watcher, pdf-ingest, langextract
- notebook-sync, session-context-worker

**Media Tier (10 services) - included in this commit:**
- ultimate-tts-studio, flute-gateway, ffmpeg-whisper
- media-video, media-audio, channel-monitor
- invidious, invidious-companion, grayjay-plugin-host, grayjay-server

Security improvement: 30 services now use tier-based env_file anchors
instead of legacy x-env-legacy pattern.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Codex Agent <codex-agent@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Phase 2 hardening: Docker security, Prometheus patterns, DNS fixes
Addresses critical CVE in torch.load with weights_only=True.

Changes:
- torch: 2.3.1 → 2.6.0
- torchvision: 0.18.1 → 0.21.0
- torchaudio: 2.3.1 → 2.6.0
- CUDA wheels: cu121 → cu124

⚠️ Breaking: Requires CUDA 12.4 runtime (was 12.1)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Codex Agent <codex-agent@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
…e pins, required password) (#355)

* docs: address PR review comments for env tier consolidation

- Add GPU Orchestrator and E2B Runner to services-catalog.md
- Add gpu-orchestrator to env.tier-api.example service list
- Create learnings file documenting 6-tier env architecture
- Add env.tier-*.example files for worker, media, agent tiers

Addresses nitpick comments from PRs #349-354.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(gpu): ensure GPU access for all CUDA-enabled services

- gpu-orchestrator: Add GPU deploy section + NVIDIA_VISIBLE_DEVICES
- ultimate-tts-studio: Add NVIDIA_VISIBLE_DEVICES env var
- hi-rag-gateway-gpu: Add NVIDIA_VISIBLE_DEVICES env var
- hi-rag-gateway-v2-gpu: Add NVIDIA_VISIBLE_DEVICES env var
- media-audio: Change base image to nvidia/cuda:12.4.1-runtime-ubuntu22.04
  (was python:3.11-slim which caused silent CPU fallback)

This fixes silent CPU fallbacks where PyTorch CUDA packages were installed
but the CUDA runtime was not available in the container.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(security): production hardening - health checks and image pinning

Critical security and reliability fixes:

1. **POSTGRES_PASSWORD now required** - Changed from insecure default
   `:-pmoves` to required syntax `${VAR:?error}`. Compose will fail if
   POSTGRES_PASSWORD is not set (lines 59, 78, 937).

2. **Pinned 8 :latest images to specific versions:**
   - postgrest/postgrest:latest → v12.2.3
   - minio/minio:latest → RELEASE.2024-12-18T13-15-44Z
   - ollama/ollama:latest → 0.5.4
   - tensorzero/gateway:latest → 2024.12.18
   - tensorzero/ui:latest → 2024.12.18
   - invidious:latest → 2024.12.09
   - invidious-companion:latest → 2024.12.20
   - grayjay:latest → 2024.11.01

3. **Added health checks to 37 services** (52 total, up from 15):
   - Data tier: qdrant, meilisearch, minio
   - API tier: hi-rag-*, retrieval-eval, presign, render-webhook
   - Worker tier: extract-worker, pdf-ingest, langextract, notebook-sync,
     ffmpeg-whisper, media-video, media-audio, pmoves-yt, channel-monitor
   - Agent tier: agent-zero, mesh-agent, deepresearch, supaserch
   - TensorZero: gateway, ui, ollama
   - Others: publisher-discord, messaging-gateway, jellyfin-bridge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(health): address PR review critical issues

Fixes critical issues identified by PR review:

MinIO:
- Change health check from `mc ready local` to curl-based
  (minio/minio image doesn't include mc binary)

NATS:
- Add `-m 8222` flag to enable HTTP monitoring port
  (health check was targeting port that wasn't enabled)

Health checks:
- comfy-watcher: verify module imports instead of just `import sys`
- mesh-agent: verify main module and NATS client availability

Missing start_period:
- postgres: add 15s start_period
- chat-relay: add 30s start_period
- n8n-agent: add 30s start_period
- invidious-postgres: add 15s start_period

All 52 health checks now have start_period defined.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Codex Agent <codex-agent@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Increase Dockerfile hardening from 33/36 (91.7%) to 35/36 (97.2%):

- notebook-sync: Add USER pmoves (user creation already existed)
- vibevoice-realtime: Add full hardening with UID/GID 65532

Accepted exception: agent-zero uses root for initialization then
drops to pmoves via 'su' for the service process. This pattern is
required by upstream Agent Zero's prepare.py and /ins/copy_A0.sh.

Pattern reference: flute-gateway/Dockerfile

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Normalize Supabase Kong hostnames to lowercase (DNS convention)
- Remove duplicate ARCHON_SUPABASE_BASE_URL env var definition
- Fix notebook-sync Dockerfile: remove USER directive so entrypoint
  can run chown/su as root before dropping privileges
- Improve deepresearch _get_or_create_counter: use module-level cache
  to avoid private prometheus_client API dependency

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changes:
- Archon health check: Add Kong gateway hostname support for internal
  Docker networking (supabase_kong_pmoves.ai:8000)
- UI dashboard: Add /dashboard redirect page to /dashboard/services
- Grafana: Fix env var placeholders and job name mismatches in queries
- Docs: Add placeholder for Jellyfin service documentation

The Archon health check now properly handles both:
- Supabase CLI endpoint (host.docker.internal:65421)
- Internal Kong gateway (supabase_kong_pmoves.ai:8000)

This resolves the 404 errors in Archon health checks when using
internal Docker DNS names instead of host.docker.internal.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add pmoves/data/agent-zero/ to gitignore. This directory contains
runtime settings (settings.json) that persist via Docker volume mount.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use registry lookup pattern instead of try/except for counter
registration. Prevents 'Duplicated timeseries' error when container
restarts with existing registry state.

Before: try/except around Counter() creation
After: REGISTRY._names_to_collectors.get() for existing lookup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add warning log to Supabase health check exception handler
- Improve persona fetch error logging with status code and truncated body
- Add metrics tracking for non-200 persona fetch responses

Closes #322

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Codex Agent <codex-agent@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix(voice): harden WSL2 playback and Makefile voice-say target

- Add OSError handling in _play_wav_via_windows() for environments where
  Windows binary interop is disabled (hardened WSL, nested shells)
- Fix voice-say Makefile target: proper bash -lc wrapping, jq quoting
- Update doc reference: PMOVES.AI-Edition-Hardened.md → Hardened-Full.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(flute): add Ultimate-TTS-Studio provider

- Add UltimateTTSProvider for Gradio API integration
- Support multiple engines: kitten_tts, f5_tts, kokoro, indextts2
- Update synthesize endpoints to support ultimate_tts provider
- Add engine parameter to SynthesizeRequest model
- Register provider in health check and config endpoints

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(flute): correct Ultimate-TTS API parameter handling

- Build full 92-parameter list for generate_unified_tts endpoint
- Add ENGINE_NAMES mapping (internal -> Gradio API names)
- Set required indextts2_emotion_description parameter
- Configure engine-specific voice parameters at correct indices

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: notebook-sync API path and Ultimate-TTS config

- Fix notebook-sync to use correct /api/{resource} endpoint path
  Open Notebook API requires /api/ prefix for all endpoints
- Add ULTIMATE_TTS_URL to flute-gateway docker-compose config
  Points to Pinokio instance on port 7863

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore(submodule): update PMOVES-BoTZ with n8n-agent

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore(n8n): upgrade to v2.1.0 from 1.123.4

- n8n 2.x uses minimal BusyBox base (no apk)
- Removed sqlite-cli customization
- Updated runners to 2.1.0

Note: API key must be regenerated in n8n UI after upgrade

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(n8n): add 9 new workflow exports for PMOVES automation

Add new n8n workflow definitions for:
- pmoves_audio_analysis: Audio processing pipeline
- pmoves_video_analysis: Video processing pipeline
- pmoves_channel_monitor: YouTube channel watching
- pmoves_comfy_hub: ComfyUI orchestration
- pmoves_deepresearch_orchestrator: Research automation
- pmoves_ingestion_hub: Content ingestion hub
- pmoves_jellyfin_watcher: Jellyfin event handling
- pmoves_notebook_content_feed: Open Notebook sync
- pmoves_social_publisher: Social media publishing

Also updates contracts/topics.json with new event subjects.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat(mcp): add n8n-agent MCP config with TensorZero integration

- Add pmoves/config/mcp/n8n-agent.yaml for Claude Code/Codex integration
- Configure stdio transport for docker-based MCP server
- Include TensorZero environment variables for local LLM inference
- Update PMOVES-BoTZ submodule with TensorZero and monitor agent

The n8n-agent is now registered with Claude Code CLI and can be used
for workflow automation with intelligent suggestions powered by local models.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Codex Agent <codex-agent@example.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Merged after rebase to resolve Jellyfin README conflict
- Add volume mount ./data/agent-zero/tmp:/a0/tmp for settings.json
  Root cause: PMOVES-Agent-Zero/python/helpers/settings.py:162 stores
  settings at /a0/tmp/settings.json but path was not mounted

- Normalize Archon SUPABASE_URL to lowercase supabase_kong_pmoves.ai
  Root cause: Docker DNS is case-sensitive on some configurations

- Fix archon-agent-work-orders health check: /healthz → /health
  Root cause: Service exposes /health, not /healthz

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jan 3, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3983831714

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 38 to +40
RUN groupadd -r pmoves --gid=65532 && \
useradd -r -g pmoves --uid=65532 --home-dir=/app --shell=/sbin/nologin pmoves && \
mkdir -p /models && \
chown -R pmoves:pmoves /app /app/vibevoice /models
chown -R pmoves:pmoves /app /app/vibevoice

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Ensure /models is writable for VibeVoice at runtime

The container now runs as the non-root pmoves user, but the Dockerfile no longer creates or chowns /models. In the default compose (docker-compose.voice.yml), /models is a named volume, which is root-owned on first mount. The entrypoint writes model files under /models (and /models/.hf), so fresh installs will hit “permission denied” and the service won’t start. Consider creating/chowning /models in the image or dropping privileges after a root-owned chown in the entrypoint.

Useful? React with 👍 / 👎.

Comment thread pmoves/Makefile
Comment on lines +385 to +387
@$(LOAD_ENV_SHARED) \
if [ -z "$$CLOUDFLARE_TUNNEL_TOKEN" ] && [ -z "$$CLOUDFLARE_TUNNEL_NAME" ]; then \
echo "↷ Set CLOUDFLARE_TUNNEL_TOKEN (Zero Trust token) or CLOUDFLARE_TUNNEL_NAME + credentials in $(ENV_SHARED_FILE) before running make up-cloudflare."; \
exit 1; \
fi; \
exec docker compose -p $(PROJECT) --profile cloudflare up -d cloudflared'
echo "↷ Set CLOUDFLARE_TUNNEL_TOKEN (Zero Trust token) or CLOUDFLARE_TUNNEL_NAME + credentials in $(ENV_SHARED_FILE) before running make up-cloudflare."; \

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Guard optional env checks against set -u

LOAD_ENV_SHARED now sources scripts/with-env.sh, which enables set -u. That makes the if [ -z "$$CLOUDFLARE_TUNNEL_TOKEN" ] guard fail with “unbound variable” when the token is unset, so the target exits before showing the intended help message. This is a regression for users who haven’t configured Cloudflare yet. Use ${VAR:-} or disable set -u after loading env.

Useful? React with 👍 / 👎.

@POWERFULMOVES

Copy link
Copy Markdown
Owner Author

Closing for rebase - will create new PR with resolved conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant