From dbdd95b3ea2ae3c6d90457a9fb12ec5c700d3066 Mon Sep 17 00:00:00 2001 From: POWERFULMOVES Date: Sat, 14 Mar 2026 21:13:11 -0400 Subject: [PATCH 1/3] feat(topology): master topology, runner map, agent teams (61 agents) Master TOPOLOGY.md documenting 6 nodes, all services, DNS subdomains. 19 GitHub Actions workflows mapped to runners with failover strategy. 11 agent teams with 61 agents defined in agent-teams.yaml. Agent registry expanded with full team assignments. Updated CLAUDE.md with topology and runner strategy sections. Cloudflare wrangler config with KV namespace for runner routing. Includes submodule pointer updates from git submodule init/update. Co-Authored-By: Claude Opus 4.6 (1M context) --- .claude/CLAUDE.md | 18 ++ .claude/context/runner-topology.md | 66 ++++ PMOVES-BotZ-gateway | 2 +- PMOVES-Danger-infra | 2 +- PMOVES-E2B-Danger-Room | 2 +- PMOVES-E2B-Danger-Room-Desktop | 2 +- PMOVES-E2b-Spells | 2 +- PMOVES-Headscale | 2 +- PMOVES-MAI-UI | 2 +- PMOVES-Pinokio-Ultimate-TTS-Studio | 2 +- PMOVES-Pipecat | 2 +- PMOVES-Tailscale | 2 +- PMOVES-Ultimate-TTS-Studio | 2 +- PMOVES-llama-throughput-lab | 2 +- PMOVES-supabase | 2 +- PMOVES-surf | 2 +- PMOVES-tensorzero | 2 +- PMOVES-transcribe-and-fetch | 2 +- Pmoves-AgentGym-RL | 2 +- Pmoves-Health-wger | 2 +- Pmoves-cipher | 2 +- Pmoves-hyperdimensions | 2 +- deploy/cloudflare/wrangler.toml | 22 +- pmoves-e2b-mcp-server | 2 +- pmoves/config/agent_registry.yaml | 305 ++++++++++++++++++ pmoves/configs/agent-teams.yaml | 176 ++++++++++ pmoves/docs/operations/TOPOLOGY.md | 235 ++++++++++++++ pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md | 106 ++++++ pmoves/integrations/archon | 2 +- 29 files changed, 944 insertions(+), 28 deletions(-) create mode 100644 .claude/context/runner-topology.md create mode 100644 pmoves/configs/agent-teams.yaml create mode 100644 pmoves/docs/operations/TOPOLOGY.md create mode 100644 pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 78ec2771c4..d18a57371f 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -504,9 +504,27 @@ Based on CodeRabbit learnings (see `.claude/learnings/ui-error-handling-review-2 - [ ] Shared utilities extracted (no duplicate functions like `ownerFromJwt`) - [ ] Unused imports removed +## Topology & Runner Strategy + +**Master topology:** `pmoves/docs/operations/TOPOLOGY.md` — single source of truth for all nodes, services, routes, and DNS. + +**Nodes:** Z890 (dev/GPU), 5090 (primary GPU, pending), KVM4-1 (API gateway), KVM4-2 (data/storage), KVM2 (exit proxy), Cloudflare Edge (DNS/Worker). + +**Agent Teams (11 teams, 61 agents):** `pmoves/configs/agent-teams.yaml` — orchestration, research, media, data, ui, automation, evolution, infra, sandbox, life, external. + +**CI Runners:** AI Lab (GPU), cloudstartup (staging), kvm4 (production), kvm2 (backup), ubuntu-latest (lightweight). Routing via Cloudflare Worker (`deploy/cloudflare/worker.js`). + +**DNS:** `pmoves.ai` zone (pending Cloudflare migration). Subdomains: api, agent, rag, tts, n8n, grafana, search, nats, minio, headscale, ci. + +**Quick references:** +- `.claude/context/runner-topology.md` — condensed topology for agent context +- `pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md` — all 19 GitHub Actions workflows mapped to runners +- `deploy/HYBRID_RUNNER_STRATEGY.md` — runner fleet documentation + ## Additional References See `.claude/context/` for detailed documentation: +- `runner-topology.md` - Condensed node/runner/team topology for agent context loading - `credentials-workflow.md` - Credential bootstrap, env-setup, secrets-funnel, JWT-from-Supabase flow - `services-catalog.md` - Complete service listing with all details - `submodules.md` - Complete submodules catalog (20 submodules) diff --git a/.claude/context/runner-topology.md b/.claude/context/runner-topology.md new file mode 100644 index 0000000000..fe02d32340 --- /dev/null +++ b/.claude/context/runner-topology.md @@ -0,0 +1,66 @@ +# Runner & Topology Context (Condensed) + +> Quick-reference for agents. Full details: `pmoves/docs/operations/TOPOLOGY.md` + +## Nodes + +| Node | Role | Key Services | Runner | +|------|------|-------------|--------| +| **Z890** | Dev + GPU (3090Ti) | All (local Docker Compose) | `ai-lab, gpu, cuda` | +| **5090** | Primary GPU (pending) | Future inference | (pending) | +| **KVM4-1** | API Gateway | TensorZero, Agent Zero, Hi-RAG, Archon, Gateway Agent | `kvm4, production` | +| **KVM4-2** | Data/Storage | Supabase, NATS, Qdrant, Neo4j, Meilisearch, MinIO, monitoring | `kvm4, production` | +| **KVM2** | Exit Node | nginx (SSL termination) | `kvm2, backup` | +| **Cloudflare** | Edge | DNS, CI Worker | — | + +## Route: Public → Services + +``` +Internet → Cloudflare DNS → KVM2 (nginx/SSL) → KVM4-1 (API) or KVM4-2 (data) +``` + +## Route: CI/CD + +``` +GitHub event → CF Worker (analyzes files) → ai-lab (GPU) / vps (Docker) / ubuntu-latest (light) +``` + +## Agent Teams (11 teams, 61 agents) + +| Team | Node Affinity | CI Runner | Count | +|------|--------------|-----------|-------| +| orchestration | kvm4-1, z890, 5090 | ai-lab | 6 | +| research | kvm4-1, kvm4-2, z890 | vps | 9 | +| media | 5090, z890 | ai-lab | 11 | +| data | kvm4-2, z890 | vps | 9 | +| ui | z890, 5090 | ubuntu-latest | 6 | +| automation | kvm4-1, z890 | vps | 4 | +| evolution | 5090, z890 | ai-lab | 4 | +| infra | kvm4-1, kvm2 | vps | 3 | +| sandbox | cloud | ubuntu-latest | 7 | +| life | z890 | ubuntu-latest | 2 | +| external | N/A (CLI) | N/A | 7 (contributors) | + +## DNS Subdomains (pmoves.ai) + +| Subdomain | Target Node | Proxy | +|-----------|------------|-------| +| `api.pmoves.ai` | KVM4-1 | Yes | +| `agent.pmoves.ai` | KVM4-1 | Yes | +| `rag.pmoves.ai` | KVM4-1 | Yes | +| `tts.pmoves.ai` | KVM4-1 | Yes | +| `n8n.pmoves.ai` | KVM4-1 | Yes | +| `grafana.pmoves.ai` | KVM4-2 | Yes | +| `search.pmoves.ai` | KVM4-2 | Yes | +| `nats.pmoves.ai` | KVM4-2 | DNS only | +| `minio.pmoves.ai` | KVM4-2 | DNS only | +| `headscale.pmoves.ai` | KVM2 | DNS only | +| `ci.pmoves.ai` | CF Worker | Yes | + +## Key Files + +- `pmoves/configs/agent-teams.yaml` — Team definitions +- `pmoves/config/agent_registry.yaml` — Full 61-agent registry +- `deploy/cloudflare/worker.js` — CI routing logic +- `deploy/scripts/deploy-vps.sh` — VPS deployment +- `pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md` — All 19 workflows mapped diff --git a/PMOVES-BotZ-gateway b/PMOVES-BotZ-gateway index 0dc9aaafb4..25650220db 160000 --- a/PMOVES-BotZ-gateway +++ b/PMOVES-BotZ-gateway @@ -1 +1 @@ -Subproject commit 0dc9aaafb4deafabe851a266b904ec089a17d948 +Subproject commit 25650220dbe94535d66f49a2189cb633a0220666 diff --git a/PMOVES-Danger-infra b/PMOVES-Danger-infra index eeb0443657..44d80c70b0 160000 --- a/PMOVES-Danger-infra +++ b/PMOVES-Danger-infra @@ -1 +1 @@ -Subproject commit eeb04436572d1f8797702dbd369a0455a9f46ace +Subproject commit 44d80c70b001165c470a14be743be41bab55effb diff --git a/PMOVES-E2B-Danger-Room b/PMOVES-E2B-Danger-Room index bc8eafebc4..7a38b33bec 160000 --- a/PMOVES-E2B-Danger-Room +++ b/PMOVES-E2B-Danger-Room @@ -1 +1 @@ -Subproject commit bc8eafebc4e432357dcb3bdff4cb10c2e5244a54 +Subproject commit 7a38b33bec74f07a7abfbebeeae1593d7c081415 diff --git a/PMOVES-E2B-Danger-Room-Desktop b/PMOVES-E2B-Danger-Room-Desktop index a6b79c2735..fcb2834148 160000 --- a/PMOVES-E2B-Danger-Room-Desktop +++ b/PMOVES-E2B-Danger-Room-Desktop @@ -1 +1 @@ -Subproject commit a6b79c27358f4632599ce87453b332cacec346c8 +Subproject commit fcb283414827d647c70bc7ca07c4fa1b785a0798 diff --git a/PMOVES-E2b-Spells b/PMOVES-E2b-Spells index 43f4f8b8bf..75a3ec6324 160000 --- a/PMOVES-E2b-Spells +++ b/PMOVES-E2b-Spells @@ -1 +1 @@ -Subproject commit 43f4f8b8bf375c4c2de3df1a76d011943bbd727e +Subproject commit 75a3ec6324959ad1d30afb47e224848b666b8e14 diff --git a/PMOVES-Headscale b/PMOVES-Headscale index 7dc70cb01e..a3ef4d7966 160000 --- a/PMOVES-Headscale +++ b/PMOVES-Headscale @@ -1 +1 @@ -Subproject commit 7dc70cb01ebf0a0e4e17fe03f9d9d06485b66704 +Subproject commit a3ef4d7966bc51b0fa08cff145b88921e4250dbd diff --git a/PMOVES-MAI-UI b/PMOVES-MAI-UI index 2872e3a053..182bcf4db0 160000 --- a/PMOVES-MAI-UI +++ b/PMOVES-MAI-UI @@ -1 +1 @@ -Subproject commit 2872e3a053cab2a55a0663632e574cf48e23f706 +Subproject commit 182bcf4db0889529116cbced1d0896e8bb3bcbba diff --git a/PMOVES-Pinokio-Ultimate-TTS-Studio b/PMOVES-Pinokio-Ultimate-TTS-Studio index 7a91ca90cb..ef5d4b3728 160000 --- a/PMOVES-Pinokio-Ultimate-TTS-Studio +++ b/PMOVES-Pinokio-Ultimate-TTS-Studio @@ -1 +1 @@ -Subproject commit 7a91ca90cbee2eb69babbcf4590d840915f7ed6a +Subproject commit ef5d4b37286ac208b08b104f69536ff59126ddfc diff --git a/PMOVES-Pipecat b/PMOVES-Pipecat index fafeaa85fd..a74aa0cc9e 160000 --- a/PMOVES-Pipecat +++ b/PMOVES-Pipecat @@ -1 +1 @@ -Subproject commit fafeaa85fdc663702b72034c19b1ebdf23b4766a +Subproject commit a74aa0cc9e99001f86739a7dd1545af737139371 diff --git a/PMOVES-Tailscale b/PMOVES-Tailscale index 43a3bc7bd3..2ad2d4d409 160000 --- a/PMOVES-Tailscale +++ b/PMOVES-Tailscale @@ -1 +1 @@ -Subproject commit 43a3bc7bd3ef3886806953e0a9075a70aa44cb41 +Subproject commit 2ad2d4d409e6b5eac5dbecb59ce307eb3297587c diff --git a/PMOVES-Ultimate-TTS-Studio b/PMOVES-Ultimate-TTS-Studio index 7ee6b9df1d..26ca5a2acd 160000 --- a/PMOVES-Ultimate-TTS-Studio +++ b/PMOVES-Ultimate-TTS-Studio @@ -1 +1 @@ -Subproject commit 7ee6b9df1db3d6632f99e3a4d3c48cd008a5cc5f +Subproject commit 26ca5a2acd238a21d5242a72fba3dbd24c5996d5 diff --git a/PMOVES-llama-throughput-lab b/PMOVES-llama-throughput-lab index 5c96bcfcf6..823922faf0 160000 --- a/PMOVES-llama-throughput-lab +++ b/PMOVES-llama-throughput-lab @@ -1 +1 @@ -Subproject commit 5c96bcfcf6f4c7649e65007ffe4be93883ef951f +Subproject commit 823922faf0c938c484398ca6c52942c18d1e1082 diff --git a/PMOVES-supabase b/PMOVES-supabase index c84e9d5b97..28443251ff 160000 --- a/PMOVES-supabase +++ b/PMOVES-supabase @@ -1 +1 @@ -Subproject commit c84e9d5b9778a0f6bb90bc0d73f3fba29cb93fb7 +Subproject commit 28443251ffcec2f4346013a3a2fa339ab631e09b diff --git a/PMOVES-surf b/PMOVES-surf index a27fbb28ea..135748a4fd 160000 --- a/PMOVES-surf +++ b/PMOVES-surf @@ -1 +1 @@ -Subproject commit a27fbb28ea8b69b76798a7fd3d293a1a26dcdace +Subproject commit 135748a4fd428d3c1a23b345776e13e2c7ec6f31 diff --git a/PMOVES-tensorzero b/PMOVES-tensorzero index d71f4d14f1..6b1bc23f54 160000 --- a/PMOVES-tensorzero +++ b/PMOVES-tensorzero @@ -1 +1 @@ -Subproject commit d71f4d14f1026e165243dfa281520ae7a84b0cc5 +Subproject commit 6b1bc23f540899764a9fcbc32ad85b4e24dbae47 diff --git a/PMOVES-transcribe-and-fetch b/PMOVES-transcribe-and-fetch index 5a57ca0018..aef3a86e81 160000 --- a/PMOVES-transcribe-and-fetch +++ b/PMOVES-transcribe-and-fetch @@ -1 +1 @@ -Subproject commit 5a57ca00185c6e04527cd2962feb2ac90491a0ad +Subproject commit aef3a86e817bc2d266b8b0845b6b118062e8dc7a diff --git a/Pmoves-AgentGym-RL b/Pmoves-AgentGym-RL index b30b84a372..b208734fc9 160000 --- a/Pmoves-AgentGym-RL +++ b/Pmoves-AgentGym-RL @@ -1 +1 @@ -Subproject commit b30b84a37223b01a68f843be8a8b2af7f80fcb07 +Subproject commit b208734fc94dbdbc6a6e585664407b7f07367a3b diff --git a/Pmoves-Health-wger b/Pmoves-Health-wger index 8e4939418b..341c7e065d 160000 --- a/Pmoves-Health-wger +++ b/Pmoves-Health-wger @@ -1 +1 @@ -Subproject commit 8e4939418b1ab2ccc918c9f4807fc09afc3fc7e8 +Subproject commit 341c7e065d7de801ea42ffeb8d0421f310cef0a9 diff --git a/Pmoves-cipher b/Pmoves-cipher index 2bd33b9d2d..873abb1bc1 160000 --- a/Pmoves-cipher +++ b/Pmoves-cipher @@ -1 +1 @@ -Subproject commit 2bd33b9d2dc8f7974c5bf995375658f291bab7c2 +Subproject commit 873abb1bc17d9a78c0c9875926740ea760e2d70a diff --git a/Pmoves-hyperdimensions b/Pmoves-hyperdimensions index eca5305560..1a6cb72547 160000 --- a/Pmoves-hyperdimensions +++ b/Pmoves-hyperdimensions @@ -1 +1 @@ -Subproject commit eca5305560a934ed729e4dd9ef2eae6c3acb656c +Subproject commit 1a6cb7254732785b5b429ff2fe5a534a7c845d89 diff --git a/deploy/cloudflare/wrangler.toml b/deploy/cloudflare/wrangler.toml index 13f05fd783..7c8e0f5aab 100644 --- a/deploy/cloudflare/wrangler.toml +++ b/deploy/cloudflare/wrangler.toml @@ -6,14 +6,14 @@ main = "worker.js" compatibility_date = "2025-12-08" workers_dev = true -# Account ID (override with CLOUDFLARE_ACCOUNT_ID env var) -# account_id = "your_account_id_here" +# Account ID +account_id = "fe5ef93f1ffc7070b2218b96f8b256f5" # KV Namespace for build state tracking (create via: wrangler kv:namespace create "CI_STATE") [[kv_namespaces]] binding = "CI_STATE" -id = "your_kv_namespace_id" -preview_id = "your_preview_kv_namespace_id" +id = "9f6cdc5ecd6540698c9e2ca7d6376fd4" +# preview_id = "your_preview_kv_namespace_id" # Create via: wrangler kv:namespace create "CI_STATE" --preview # R2 Bucket for build artifacts (optional) # [[r2_buckets]] @@ -42,10 +42,20 @@ vars = { RUNNER_DISPATCH_MODE = "cloudflare-only" } # - CLOUDFLARE_API_TOKEN: For deploying via Workers API # - DISCORD_WEBHOOK_URL: Discord notification endpoint -# Routes (configure after deployment) +# Routes — uncomment after pmoves.ai zone is active in Cloudflare # routes = [ -# { pattern = "ci.pmoves.ai/webhook", zone_name = "pmoves.ai" } +# { pattern = "ci.pmoves.ai/webhook/*", zone_name = "pmoves.ai" }, +# { pattern = "ci.pmoves.ai/health", zone_name = "pmoves.ai" }, +# { pattern = "ci.pmoves.ai/status", zone_name = "pmoves.ai" }, +# { pattern = "ci.pmoves.ai/metrics", zone_name = "pmoves.ai" } # ] +# +# Secrets to set before deploy: +# wrangler secret put WEBHOOK_SECRET # GitHub webhook HMAC secret +# wrangler secret put GITHUB_TOKEN # GitHub PAT for PR file fetching +# wrangler secret put DISCORD_WEBHOOK_URL # Discord notification endpoint +# +# Deploy: cd deploy/cloudflare && npm install && npx wrangler deploy # Limits limits = { cpu_ms = 50 } diff --git a/pmoves-e2b-mcp-server b/pmoves-e2b-mcp-server index 1dcc5234e5..d01ec6315a 160000 --- a/pmoves-e2b-mcp-server +++ b/pmoves-e2b-mcp-server @@ -1 +1 @@ -Subproject commit 1dcc5234e567439f527457eb34e7539444ee1087 +Subproject commit d01ec6315a6539fcd425cdc63945503c45016dae diff --git a/pmoves/config/agent_registry.yaml b/pmoves/config/agent_registry.yaml index 67d8c73b91..e14d0a1752 100644 --- a/pmoves/config/agent_registry.yaml +++ b/pmoves/config/agent_registry.yaml @@ -85,6 +85,11 @@ agents: recovery_strategy: cipher_resumable cipher_categories: [agent_plan, agent_checkpoint, agent_completion] submodule: "PMOVES-Agent-Zero" + topology: + node_affinity: [kvm4-1, z890, 5090] + team: orchestration + ci_runner: ai-lab + compose_profile: agents description: "Primary L1 orchestrator with embedded agent runtime and MCP API" archon: @@ -111,6 +116,11 @@ agents: recovery_strategy: cipher_resumable cipher_categories: [agent_plan, agent_checkpoint] submodule: "PMOVES-Archon" + topology: + node_affinity: [kvm4-1, z890, 5090] + team: orchestration + ci_runner: ai-lab + compose_profile: agents description: "Supabase-driven planning/execution copilot" supaserch: @@ -136,6 +146,11 @@ agents: checkpoint_frequency: per_submodule recovery_strategy: manual_handoff cipher_categories: [agent_plan, agent_checkpoint] + topology: + node_affinity: [kvm4-1, z890, 5090] + team: orchestration + ci_runner: ai-lab + compose_profile: orchestration description: "Multimodal holographic deep research orchestrator" botz_gateway: @@ -162,6 +177,11 @@ agents: recovery_strategy: cipher_resumable cipher_categories: [agent_plan, agent_checkpoint] submodule: "PMOVES-BotZ-gateway" + topology: + node_affinity: [kvm4-1, z890, 5090] + team: orchestration + ci_runner: ai-lab + compose_profile: agents description: "Work item distribution across BoTZ CLI instances" gateway_agent: @@ -183,6 +203,11 @@ agents: swarm_participant: false attribution_gated: false submodule: null + topology: + node_affinity: [kvm4-1, z890, 5090] + team: orchestration + ci_runner: ai-lab + compose_profile: tier-agent description: "MCP tool orchestration service — discovers and executes 100+ tools via Agent Zero" mesh_agent: @@ -203,6 +228,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, kvm2] + team: infra + ci_runner: vps + compose_profile: agents description: "Distributed node announcer for multi-host orchestration" hirag_v2: @@ -230,6 +260,11 @@ agents: recovery_strategy: idempotent_replay cipher_categories: [agent_plan] submodule: "PMOVES-HiRAG" + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: workers description: "Hybrid RAG gateway combining Qdrant + Neo4j + Meilisearch" deep_research: @@ -256,6 +291,11 @@ agents: recovery_strategy: manual_handoff cipher_categories: [agent_plan, agent_checkpoint] submodule: "PMOVES-Deep-Serch" + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: orchestration description: "LLM-based research planner (Alibaba Tongyi)" tensorzero: @@ -276,6 +316,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [5090, z890] + team: evolution + ci_runner: ai-lab + compose_profile: null description: "Centralized LLM gateway for all model providers" flute_gateway: @@ -300,6 +345,11 @@ agents: bpm_capable: true geometry_bus_subjects: ["tokenism.prosodic.bpm.v1"] tac_tree: "pmoves/docs/TAC/TAC_FLUTE.md" + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: agents description: "Multimodal voice communication layer with Pipecat" pmoves_yt: @@ -321,6 +371,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES.YT" + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: yt description: "YouTube ingestion service" ffmpeg_whisper: @@ -341,6 +396,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: gpu description: "Media transcription with OpenAI Whisper (GPU)" media_video: @@ -361,6 +421,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: workers description: "Object/frame analysis with YOLOv8" media_audio: @@ -381,6 +446,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: workers description: "Audio analysis (emotion/speaker detection)" ultimate_tts: @@ -401,6 +471,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: gpu description: "Multi-engine TTS with 7 engines (GPU)" extract_worker: @@ -426,6 +501,11 @@ agents: checkpoint_frequency: per_file recovery_strategy: idempotent_replay cipher_categories: [] + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: workers description: "Text embedding and indexing to Qdrant + Meilisearch" channel_monitor: @@ -446,6 +526,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: workers description: "External content watcher (YouTube channels)" notebook_sync: @@ -466,6 +551,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: workers description: "SurrealDB Open Notebook synchronizer" pdf_ingest: @@ -486,6 +576,11 @@ agents: hz_sensitive: true swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: workers description: "Document ingestion orchestrator for PDFs" publisher_discord: @@ -506,6 +601,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, z890] + team: automation + ci_runner: vps + compose_profile: workers description: "Discord notification bot" langextract: @@ -526,6 +626,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: workers description: "Language detection and NLP preprocessing" presign: @@ -546,6 +651,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, z890] + team: automation + ci_runner: vps + compose_profile: null description: "MinIO URL presigner for short-lived download URLs" render_webhook: @@ -566,6 +676,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, z890] + team: automation + ci_runner: vps + compose_profile: workers description: "ComfyUI render callback handler" jellyfin_bridge: @@ -587,6 +702,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-Jellyfin" + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: null description: "Jellyfin metadata webhook and helper" mai_ui: @@ -608,6 +728,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-MAI-UI" + topology: + node_affinity: [z890, 5090] + team: ui + ci_runner: ubuntu-latest + compose_profile: null description: "Main user interface" crush: @@ -635,6 +760,11 @@ agents: recovery_strategy: cipher_resumable cipher_categories: [agent_plan, agent_checkpoint] submodule: "PMOVES-crush" + topology: + node_affinity: [z890, 5090] + team: ui + ci_runner: ubuntu-latest + compose_profile: null description: "Terminal AI coding assistant — the gateway where model and user begin their journey" cipher_memory: @@ -661,6 +791,11 @@ agents: recovery_strategy: idempotent_replay cipher_categories: [] submodule: "Pmoves-cipher" + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: null description: "Knowledge-graph memory for agents (Neo4j backend). Also serves as the resilience backbone — stores agent_plan, agent_checkpoint, and agent_completion snapshots for all agents." hyperdimensions: @@ -682,6 +817,11 @@ agents: swarm_participant: true attribution_gated: true submodule: "Pmoves-hyperdimensions" + topology: + node_affinity: [z890, 5090] + team: ui + ci_runner: ubuntu-latest + compose_profile: null description: "Geometry visualization surface and control plane (L2.5)" vps_fleet_manager: @@ -706,6 +846,11 @@ agents: context_budget: small checkpoint_frequency: per_wave recovery_strategy: manual_handoff + topology: + node_affinity: [kvm4-1, kvm2] + team: infra + ci_runner: vps + compose_profile: null description: "Hostinger KVM fleet orchestration — deploy, status, restart via MCP tools" # --- Data infrastructure (Utility class) --- @@ -728,6 +873,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: null description: "JetStream-enabled event broker — the nervous system" supabase: @@ -748,6 +898,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: supabase-local description: "Postgres with pgvector — metadata and state" qdrant: @@ -768,6 +923,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: null description: "Vector embeddings for semantic search" neo4j: @@ -788,6 +948,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: null description: "Knowledge graph storage" meilisearch: @@ -808,6 +973,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: null description: "Full-text keyword search" minio: @@ -828,6 +998,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: null description: "S3-compatible object storage" prometheus: @@ -848,6 +1023,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: monitoring description: "Metrics scraping from all services" grafana: @@ -868,6 +1048,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: monitoring description: "Dashboard visualization" loki: @@ -888,6 +1073,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-2, z890] + team: data + ci_runner: vps + compose_profile: monitoring description: "Centralized log aggregation" # --- Missing agents added in Phase E (source dual enforcement) --- @@ -911,6 +1101,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-DoX" + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: workers description: "Document intelligence — path-safe processing with fail-closed JWT" open_notebook: @@ -932,6 +1127,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-Open-Notebook" + topology: + node_affinity: [kvm4-1, kvm4-2, z890] + team: research + ci_runner: vps + compose_profile: null description: "Knowledge base / note-taking integration (SurrealDB)" consciousness_service: @@ -952,6 +1152,11 @@ agents: hz_sensitive: true swarm_participant: false attribution_gated: true + topology: + node_affinity: [kvm4-1, z890, 5090] + team: orchestration + ci_runner: ai-lab + compose_profile: null description: "CGP consciousness mapper — geometric cognitive processing" n8n: @@ -972,6 +1177,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, z890] + team: automation + ci_runner: vps + compose_profile: null description: "Workflow automation — glue for multi-step processes" headscale: @@ -992,6 +1202,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [kvm4-1, kvm2] + team: infra + ci_runner: vps + compose_profile: null description: "Network coordination — mesh VPN overlay" rustdesk: @@ -1012,6 +1227,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [z890, 5090] + team: ui + ci_runner: ubuntu-latest + compose_profile: null description: "Remote desktop — self-hosted remote access" invidious: @@ -1032,6 +1252,11 @@ agents: hz_sensitive: false swarm_participant: false attribution_gated: false + topology: + node_affinity: [z890, 5090] + team: ui + ci_runner: ubuntu-latest + compose_profile: null description: "Privacy-focused video frontend" wealth: @@ -1060,6 +1285,11 @@ agents: geometry_bus_subjects: [] tac_tree: "pmoves/docs/TAC/TAC_WEALTH.md" submodule: "PMOVES-Wealth" + topology: + node_affinity: [z890] + team: life + ci_runner: ubuntu-latest + compose_profile: null description: "Finance management — tongue-in-cheek wealth tracking" health: @@ -1088,6 +1318,11 @@ agents: geometry_bus_subjects: [] tac_tree: "pmoves/docs/TAC/TAC_HEALTH.md" submodule: "Pmoves-Health-wger" + topology: + node_affinity: [z890] + team: life + ci_runner: ubuntu-latest + compose_profile: null description: "Fitness tracking — the body is a system too" evoswarm_controller: @@ -1123,6 +1358,11 @@ agents: fitness_function: "reconstruction_fidelity * cost_efficiency" population_size: 16 generations_per_epoch: 5 + topology: + node_affinity: [5090, z890] + team: evolution + ci_runner: ai-lab + compose_profile: gpu description: "Evolutionary optimizer for CGP geometry parameters — evolves CG Builder, Decoder, Energy, and Training genomes" swarm_attribution: @@ -1143,6 +1383,11 @@ agents: hz_sensitive: true swarm_participant: true attribution_gated: true + topology: + node_affinity: [5090, z890] + team: evolution + ci_runner: ai-lab + compose_profile: null description: "Shape-attribution engine for swarm consensus (CHIT L3)" # --- Agents added in v1.4.0 (12 new agents) --- @@ -1166,6 +1411,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-A2UI" + topology: + node_affinity: [z890, 5090] + team: ui + ci_runner: ubuntu-latest + compose_profile: null description: "Research UI surface — artifact-to-UI prototyping" agentgym: @@ -1187,6 +1437,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-AgentGym" + topology: + node_affinity: [cloud] + team: sandbox + ci_runner: ubuntu-latest + compose_profile: null description: "Agent training gymnasium — environment suite for RL evaluation" agentgym_rl: @@ -1208,6 +1463,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "Pmoves-AgentGym-RL" + topology: + node_affinity: [cloud] + team: sandbox + ci_runner: ubuntu-latest + compose_profile: null description: "Reinforcement learning extension for AgentGym" creator: @@ -1229,6 +1489,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-Creator" + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: null description: "Media creation pipeline — ComfyUI + render orchestration" llama_lab: @@ -1250,6 +1515,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-llama-throughput-lab" + topology: + node_affinity: [5090, z890] + team: evolution + ci_runner: ai-lab + compose_profile: null description: "LLM benchmarking and throughput testing laboratory" surf: @@ -1271,6 +1541,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-surf" + topology: + node_affinity: [cloud] + team: sandbox + ci_runner: ubuntu-latest + compose_profile: null description: "E2B browser automation agent — web interaction sandbox" e2b_danger_room: @@ -1292,6 +1567,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-E2B-Danger-Room" + topology: + node_affinity: [cloud] + team: sandbox + ci_runner: ubuntu-latest + compose_profile: null description: "Sandboxed code execution environment for untrusted workloads" e2b_desktop: @@ -1313,6 +1593,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-E2B-Danger-Room-Desktop" + topology: + node_affinity: [cloud] + team: sandbox + ci_runner: ubuntu-latest + compose_profile: null description: "Desktop sandbox environment for GUI-based agent tasks" danger_infra: @@ -1334,6 +1619,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-Danger-infra" + topology: + node_affinity: [cloud] + team: sandbox + ci_runner: ubuntu-latest + compose_profile: null description: "E2B infrastructure provisioning and sandbox orchestration" e2b_spells: @@ -1355,6 +1645,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-E2b-Spells" + topology: + node_affinity: [cloud] + team: sandbox + ci_runner: ubuntu-latest + compose_profile: null description: "Pre-built sandbox templates and execution recipes" transcribe_and_fetch: @@ -1376,6 +1671,11 @@ agents: swarm_participant: false attribution_gated: false submodule: "PMOVES-transcribe-and-fetch" + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: null description: "Standalone transcription and content fetching utility" jellyfin_ai: @@ -1397,4 +1697,9 @@ agents: swarm_participant: false attribution_gated: false submodule: "Pmoves-Jellyfin-AI-Media-Stack" + topology: + node_affinity: [5090, z890] + team: media + ci_runner: ai-lab + compose_profile: null description: "AI-enhanced media management with Jellyfin integration" diff --git a/pmoves/configs/agent-teams.yaml b/pmoves/configs/agent-teams.yaml new file mode 100644 index 0000000000..136bd10c07 --- /dev/null +++ b/pmoves/configs/agent-teams.yaml @@ -0,0 +1,176 @@ +# PMOVES.AI Agent Team Definitions +# Groups 61 agents into functional teams with node affinity and runner assignments. +# +# Query: python -c "import yaml; t=yaml.safe_load(open('pmoves/configs/agent-teams.yaml')); [print(f'{k}: {len(v[\"agents\"])} agents') for k,v in t['teams'].items()]" +# Last updated: 2026-03-14 + +version: "1.0.0" + +teams: + orchestration: + name: "Orchestration" + description: "Core agent coordination, MCP routing, task delegation" + node_affinity: [kvm4-1, z890, 5090] + ci_runner: ai-lab + compose_profiles: [agents, tier-agent] + agents: + - agent_zero # Port 8080 — Primary orchestrator + - archon # Port 8091 — Planning/execution copilot + - supaserch # Port 8099 — Holographic deep research + - botz_gateway # Port 8054 — Work item distribution + - gateway_agent # Port 8100 — MCP tool orchestration + - consciousness_service # CGP consciousness mapper + + research: + name: "Research & Knowledge" + description: "RAG, search, embedding, document processing" + node_affinity: [kvm4-1, kvm4-2, z890] + ci_runner: vps + compose_profiles: [workers] + agents: + - hirag_v2 # Port 8086/8087 — Hybrid RAG + - deep_research # Port 8098 — Research planner + - extract_worker # Port 8083 — Text embedding + indexing + - langextract # Port 8084 — Language detection/NLP + - pdf_ingest # Port 8092 — Document ingestion + - notebook_sync # Port 8095 — Open Notebook sync + - dox # Document intelligence + - open_notebook # Knowledge base (SurrealDB) + - cipher_memory # Port 8096 — Knowledge graph memory + + media: + name: "Media & Voice" + description: "Audio/video processing, TTS, transcription, content creation" + node_affinity: [5090, z890] + ci_runner: ai-lab + compose_profiles: [gpu, yt] + agents: + - pmoves_yt # Port 8077 — YouTube ingestion + - ffmpeg_whisper # Port 8078 — Whisper transcription (GPU) + - media_video # Port 8079 — YOLOv8 video analysis + - media_audio # Port 8082 — Emotion/speaker detection + - ultimate_tts # Port 7861 — Multi-engine TTS (GPU) + - flute_gateway # Port 8055 — Prosodic voice synthesis + - channel_monitor # Port 8097 — Content watcher + - jellyfin_bridge # Port 8093 — Jellyfin metadata + - jellyfin_ai # AI-enhanced media management + - creator # ComfyUI render pipeline + - transcribe_and_fetch # Standalone transcription + + data: + name: "Data & Storage" + description: "Databases, message bus, object storage, monitoring" + node_affinity: [kvm4-2, z890] + ci_runner: vps + compose_profiles: [monitoring, supabase-local] + agents: + - nats # Port 4222 — Event broker + - supabase # Port 3010 — Postgres + pgvector + - qdrant # Port 6333 — Vector embeddings + - neo4j # Port 7474 — Knowledge graph + - meilisearch # Port 7700 — Full-text search + - minio # Port 9000 — Object storage + - prometheus # Port 9090 — Metrics + - grafana # Port 3000 — Dashboards + - loki # Port 3100 — Log aggregation + + ui: + name: "User Interfaces" + description: "Frontend surfaces, visualization, remote access" + node_affinity: [z890, 5090] + ci_runner: ubuntu-latest + compose_profiles: [] + agents: + - mai_ui # Main user interface + - crush # Terminal AI coding assistant + - a2ui # Research UI / artifact-to-UI + - hyperdimensions # CHIT geometry visualization + - invidious # Port 3333 — Privacy video frontend + - rustdesk # Port 21115 — Remote desktop + + automation: + name: "Automation & Notifications" + description: "Workflow automation, notifications, webhook handling" + node_affinity: [kvm4-1, z890] + ci_runner: vps + compose_profiles: [workers] + agents: + - n8n # Port 5678 — Workflow automation + - publisher_discord # Port 8094 — Discord notifications + - render_webhook # Port 8085 — ComfyUI callback handler + - presign # Port 8088 — MinIO URL presigner + + evolution: + name: "Evolution & CHIT" + description: "Evolutionary optimization, CHIT encoding, swarm attribution" + node_affinity: [5090, z890] + ci_runner: ai-lab + compose_profiles: [gpu] + agents: + - evoswarm_controller # Port 8113 — Evolutionary optimizer + - swarm_attribution # Shape-attribution engine + - tensorzero # Port 3030 — LLM gateway (all model calls) + - llama_lab # LLM benchmarking + + infra: + name: "Infrastructure & Networking" + description: "VPN, fleet management, node coordination" + node_affinity: [kvm4-1, kvm2] + ci_runner: vps + compose_profiles: [] + agents: + - mesh_agent # Node announcer + - headscale # Port 8181 — Mesh VPN + - vps_fleet_manager # Hostinger KVM orchestration + + sandbox: + name: "Sandbox & Execution" + description: "Isolated code execution environments (E2B)" + node_affinity: [cloud] + ci_runner: ubuntu-latest + compose_profiles: [] + agents: + - e2b_danger_room # Sandboxed code execution + - e2b_desktop # Desktop sandbox (GUI) + - e2b_spells # Sandbox templates/recipes + - danger_infra # E2B infrastructure provisioning + - surf # Browser automation agent + - agentgym # Agent training gymnasium + - agentgym_rl # RL extension for AgentGym + + external: + name: "External Contributors" + description: "AI agents and humans who contribute via git (no node affinity)" + node_affinity: [] + ci_runner: null + compose_profiles: [] + agents: [] + # External contributors are tracked in agent_registry.yaml under external_contributors: + # claude-opus, kilocode, codex, gemini, cline, powerfulmoves, crush + + life: + name: "Life Integration" + description: "Health, wealth, and personal data integrations" + node_affinity: [z890] + ci_runner: ubuntu-latest + compose_profiles: [] + agents: + - wealth # Firefly III finance tracking + - health # wger fitness tracking + +# Validation: all 61 agents from agent_registry.yaml must appear in exactly one team +# Run: python -c " +# import yaml +# reg = yaml.safe_load(open('pmoves/config/agent_registry.yaml')) +# teams = yaml.safe_load(open('pmoves/configs/agent-teams.yaml')) +# reg_agents = set(reg['agents'].keys()) +# team_agents = set() +# for t in teams['teams'].values(): +# team_agents.update(t.get('agents', [])) +# missing = reg_agents - team_agents +# extra = team_agents - reg_agents +# print(f'Registry: {len(reg_agents)}, Teams: {len(team_agents)}') +# if missing: print(f'Missing from teams: {missing}') +# if extra: print(f'Extra in teams: {extra}') +# if not missing and not extra: print('All agents assigned!') +# " diff --git a/pmoves/docs/operations/TOPOLOGY.md b/pmoves/docs/operations/TOPOLOGY.md new file mode 100644 index 0000000000..06aa53cfa6 --- /dev/null +++ b/pmoves/docs/operations/TOPOLOGY.md @@ -0,0 +1,235 @@ +# PMOVES.AI Master Topology + +> Single source of truth for all physical/virtual nodes, service assignments, agent teams, route flows, and runner strategy. +> +> Last updated: 2026-03-14 + +--- + +## Node Inventory + +| Node | LAN IP | Tailscale Hostname | Public IP | Role | Runner Labels | vCPU / RAM | Cost | +|------|--------|--------------------|-----------|------|---------------|------------|------| +| Z890 (Windows 11) | 192.168.1.92 / .94 | 100.113.38.37 | — | Dev, GPU (RTX 3090 Ti) | `ai-lab, gpu, cuda` | 32C / 128GB | electricity | +| 5090 PC | 192.168.1.65 / .66 | (pending onboarding) | — | Primary GPU (RTX 5090) | (future: `ai-lab`) | TBD | electricity | +| KVM4-1 | — | pmoves-kvm4-1 | Hostinger (TBD) | API Gateway | `self-hosted, vps, kvm4, production` | 8C / 16GB | $10/mo | +| KVM4-2 | — | pmoves-kvm4-2 | Hostinger (TBD) | Data / Storage | `self-hosted, vps, kvm4, production` | 8C / 16GB | $10/mo | +| KVM2 | — | pmoves-kvm2 | Hostinger (TBD) | Exit Node / Proxy | `self-hosted, vps, kvm2, backup` | 4C / 8GB | $10/mo | +| Cloudflare Edge | — | — | Anycast | DNS, Worker routing | — | Edge | Free plan | +| GitHub Cloud | — | — | — | Lightweight CI | `ubuntu-latest` | 2C / 7GB | $0.008/min | + +**Total VPS cost:** $30/mo + electricity for local nodes. + +--- + +## Service-to-Node Assignments + +### KVM4-1 — API Gateway Node + +Services deployed via `docker-compose.vps.override.yml`: + +| Service | Port | Health | Compose Profile | +|---------|------|--------|-----------------| +| TensorZero Gateway | 3030 | `/healthz` | — | +| Agent Zero | 8080 | `/healthz` | agents | +| Hi-RAG v2 (CPU) | 8086 | `/healthz` | — | +| Archon | 8091 | `/healthz` | agents | +| Mesh Agent | — | — | agents | +| Gateway Agent | 8100 | `/healthz` | tier-agent | +| Extract Worker | 8083 | `/healthz` | workers | + +### KVM4-2 — Data / Storage Node + +| Service | Port | Health | Compose Profile | +|---------|------|--------|-----------------| +| Supabase DB (Postgres) | 5432 (int) / 54322 (ext) | — | supabase-local | +| Supabase PostgREST | 3000 (int) | — | supabase-local | +| Kong Gateway | 8000 / 65421 | — | supabase-local | +| Qdrant | 6333 | `/healthz` | — | +| Neo4j | 7474 (HTTP) / 7687 (Bolt) | `/db/neo4j/health` | — | +| Meilisearch | 7700 | `/health` | — | +| NATS | 4222 / 9222 (WS) | `http://:8222/varz` | — | +| Prometheus | 9090 | `/-/healthy` | monitoring | +| Grafana | 3000 | `/api/health` | monitoring | +| Loki | 3100 | `/ready` | monitoring | +| MinIO | 9000 (API) / 9001 (Console) | `/minio/health/live` | — | + +### KVM2 — Exit Node / Reverse Proxy + +| Service | Port | Health | Notes | +|---------|------|--------|-------| +| nginx | 80 / 443 | `nginx -t` | SSL termination via Let's Encrypt | + +### Z890 — Development (Local) + +All services can run locally via Docker Compose profiles. Primary use: development, GPU inference (3090 Ti), self-hosted GitHub Actions runner (`ai-lab`). + +### 5090 PC — Primary GPU (Pending) + +Future primary inference node. Blocked on: Tailscale onboarding, OpenSSH setup. + +--- + +## Route Flows + +### Public Internet → Production Services + +``` +User (HTTPS) + → Cloudflare DNS (pmoves.ai zone) + → KVM2 nginx (SSL termination, reverse proxy) + → KVM4-1 (API services: Agent Zero, TensorZero, Hi-RAG, etc.) + → KVM4-2 (Grafana dashboard, search) +``` + +### GitHub CI/CD → Runner Routing + +``` +GitHub Push/PR Event + → Cloudflare Worker (pmoves-ci-orchestrator) + → Analyze changed files + ├── GPU required? → AI Lab runner (Z890/5090) + ├── Docker build? → VPS runner (kvm4) + └── Lightweight? → GitHub hosted (ubuntu-latest) +``` + +### LAN Development + +``` +Developer (Z890) + → Docker Compose (all services locally) + → Tailscale mesh → KVM nodes (for production testing) +``` + +### NATS Mesh (All Nodes) + +``` +All nodes interconnected via Tailscale overlay network. +NATS URL: nats://nats:pmoves@nats:4222 + +Z890 ←→ KVM4-1 ←→ KVM4-2 + ↕ ↕ +5090 KVM2 +``` + +--- + +## DNS Subdomain Map (Cloudflare) + +**Zone:** `pmoves.ai` (registered at Hostinger, NS delegated to Cloudflare) + +| Subdomain | Record | Target | Proxy | Node | Protocol | +|-----------|--------|--------|-------|------|----------| +| `headscale.pmoves.ai` | A | KVM2 IP | DNS only | KVM2 | HTTPS (Tailscale) | +| `api.pmoves.ai` | A | KVM4-1 IP | Proxied | KVM4-1 | HTTPS | +| `ci.pmoves.ai` | CNAME | CF Worker | Proxied | Edge | HTTPS | +| `grafana.pmoves.ai` | A | KVM4-2 IP | Proxied | KVM4-2 | HTTPS | +| `n8n.pmoves.ai` | A | KVM4-1 IP | Proxied | KVM4-1 | HTTPS | +| `nats.pmoves.ai` | A | KVM4-2 IP | DNS only | KVM4-2 | TCP (4222) | +| `minio.pmoves.ai` | A | KVM4-2 IP | DNS only | KVM4-2 | HTTPS (9000) | +| `rag.pmoves.ai` | A | KVM4-1 IP | Proxied | KVM4-1 | HTTPS | +| `agent.pmoves.ai` | A | KVM4-1 IP | Proxied | KVM4-1 | HTTPS | +| `search.pmoves.ai` | A | KVM4-2 IP | Proxied | KVM4-2 | HTTPS | +| `tts.pmoves.ai` | A | KVM4-1 IP | Proxied | KVM4-1 | HTTPS | + +**Note:** NATS, MinIO, and Headscale use DNS-only (no Cloudflare proxy) because they use non-HTTP protocols or need direct TCP connections. + +--- + +## Runner Fleet + +| Runner | Labels | Node | Hardware | Role | Cost | +|--------|--------|------|----------|------|------| +| AI Lab | `self-hosted, ai-lab, gpu, cuda` | Z890 (→ 5090) | RTX 3090Ti/5090, 128GB | GPU builds, CUDA, inference | $0 (electricity) | +| cloudstartup | `self-hosted, vps, cloudstartup, staging` | KVM4-1 | 8C / 16GB | Staging deploys, CPU builds | $10/mo | +| kvm4 | `self-hosted, vps, kvm4, production` | KVM4-1 | 8C / 16GB | Production deploys | (shared) | +| kvm2 | `self-hosted, vps, kvm2, backup` | KVM2 | 4C / 8GB | Overflow, backup | $10/mo | +| ubuntu-latest | GitHub-hosted | Cloud | 2C / 7GB | Lightweight (lint, tests, docs) | $0.008/min | + +**Routing logic:** See `deploy/cloudflare/worker.js` for the full decision tree. + +**Estimated savings:** 88% vs GitHub-hosted only (per `deploy/HYBRID_RUNNER_STRATEGY.md`). + +--- + +## Docker Compose Profiles + +| Profile | Services | Typical Node | +|---------|----------|-------------| +| `agents` | Agent Zero, Archon, Mesh Agent | KVM4-1, Z890 | +| `workers` | Extract, LangExtract, media analyzers | KVM4-1, Z890 | +| `orchestration` | SupaSerch, DeepResearch | KVM4-1, Z890 | +| `yt` | PMOVES.YT ingestion | Z890 | +| `gpu` | Hi-RAG GPU, Ollama, Ultimate-TTS | 5090, Z890 | +| `monitoring` | Prometheus, Grafana, Loki, Promtail | KVM4-2, Z890 | +| `supabase-local` | Full Supabase 13-service stack | KVM4-2, Z890 | +| `tier-agent` | Gateway Agent | KVM4-1 | + +**Start services:** +```bash +docker compose --profile agents --profile workers up -d +``` + +--- + +## Docker Networks (Segmentation) + +| Network | Purpose | Services | +|---------|---------|----------| +| `pmoves-net` | Default cross-service mesh | All services | +| `pmoves_api` | API tier isolation | Kong, PostgREST, Agent Zero, Gateway Agent | +| `pmoves_data` | Data tier isolation | PostgreSQL, internal Supabase stack | + +--- + +## Deployment Order + +1. **KVM4-2** (Data first — NATS, Supabase, Qdrant, Neo4j, Meilisearch, MinIO) +2. **KVM4-1** (API second — TensorZero, Agent Zero, Hi-RAG, Archon, Gateway Agent) +3. **KVM2** (Proxy last — nginx with upstream configs pointing to KVM4-1/KVM4-2) + +**Script:** `deploy/scripts/deploy-vps.sh [kvm4-1|kvm4-2|kvm2|all|status]` + +--- + +## Credential Flow + +``` +GitHub Secrets (source of truth) + → sync-secrets-local.yml (workflow_dispatch) → .env.local + → deploy-vps.sh → SSH → /opt/pmoves/.env.vps + → Cloudflare Worker → wrangler secret put (manual) +``` + +Key secrets: +- `HOSTINGER_KVM4_1_IP`, `HOSTINGER_KVM4_2_IP`, `HOSTINGER_KVM2_IP` — VPS public IPs +- `HOSTINGER_API_KEY` — Hostinger API access +- `TAILSCALE_AUTHKEY` — Tailscale node join +- `WEBHOOK_SECRET` — GitHub → Cloudflare Worker verification +- `GITHUB_TOKEN` — Worker → GitHub API calls + +--- + +## Blocked Items (Requires User Action) + +| Item | Blocker | Depends On | +|------|---------|------------| +| 5090 Tailscale join | OpenSSH not installed on 5090 | User installs OpenSSH | +| DNS records creation | pmoves.ai zone not in Cloudflare | User adds zone + updates NS at Hostinger | +| KVM public IPs | Not retrieved yet | Hostinger MCP or dashboard | +| Cloudflare Worker deploy | Config ready, needs manual `wrangler deploy` | Zone active | +| Hostinger MCP | `HOSTINGER_API_KEY` not local | Run `sync-secrets-local.yml` or copy manually | + +--- + +## References + +- `deploy/HYBRID_RUNNER_STRATEGY.md` — Full runner fleet documentation +- `deploy/cloudflare/worker.js` — CI orchestrator Worker code +- `deploy/cloudflare/wrangler.toml` — Worker configuration +- `deploy/scripts/deploy-vps.sh` — VPS deployment script +- `pmoves/docker-compose.vps.override.yml` — VPS compose overrides +- `pmoves/docs/operations/PORT_REGISTRY.md` — Port allocations +- `pmoves/config/agent_registry.yaml` — 61-agent registry +- `pmoves/configs/agent-teams.yaml` — Agent team assignments +- `.claude/context/runner-topology.md` — Condensed topology for agent context diff --git a/pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md b/pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md new file mode 100644 index 0000000000..6b7b7c6618 --- /dev/null +++ b/pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md @@ -0,0 +1,106 @@ +# GitHub Actions Workflow → Runner Map + +> Maps all workflows to their runner assignments, triggers, and target nodes. +> +> Last updated: 2026-03-14 + +--- + +## Workflow Inventory (19 workflows) + +### CI / Testing Workflows + +| Workflow | File | Trigger | Runner(s) | Purpose | +|----------|------|---------|-----------|---------| +| **Python Tests** | `python-tests.yml` | push, PR, schedule | `ubuntu-latest` | pytest matrix (Python 3.11) | +| **UI Tests** | `ui-tests.yml` | push, PR, schedule | `ubuntu-latest` (4 jobs) | Node 20 frontend tests | +| **SQL Policy Lint** | `sql-policy-lint.yml` | push, PR (sql/, migrations/) | `ubuntu-latest` | Migration validation | +| **CHIT Contract** | `chit-contract.yml` | push, PR | `ubuntu-latest` | CHIT schema validation | +| **Integration Gate** | `integration-gate.yml` | push, PR | `ubuntu-latest` | Integration contract checks | +| **Integration Contract** | `integration-contract.yml` | push, PR | `ubuntu-latest` | Python 3.11 contract validation | +| **Codex Parity Advisory** | `codex-parity-advisory.yml` | push, PR | `ubuntu-latest` | Codex compatibility check | + +### Security & Hardening + +| Workflow | File | Trigger | Runner(s) | Purpose | +|----------|------|---------|-----------|---------| +| **CodeQL** | `codeql.yml` | push (main), PR, schedule (Fri 20:16 UTC) | PR: `ubuntu-latest`; protected: `self-hosted, Linux, X64` | Security scanning | +| **Hardening Validation** | `hardening-validation.yml` | push/PR (Dockerfiles, compose) | `ubuntu-latest` (4 jobs) + `self-hosted, ai-lab` (docker-bench) | Container hardening | + +### Build & Deploy + +| Workflow | File | Trigger | Runner(s) | Purpose | +|----------|------|---------|-----------|---------| +| **Self-Hosted Builds** | `self-hosted-builds.yml` | push (main/develop), PR, dispatch | GPU: `ai-lab, gpu` (disabled); CPU: `self-hosted, Linux, X64`; Staging: `cloudstartup, staging`; Prod: `kvm4, production` | Main build pipeline | +| **Self-Hosted Builds (Hardened)** | `self-hosted-builds-hardened.yml` | push (main/develop), dispatch | Same as above (hardened variant) | Hardened build pipeline | +| **Build Images** | `build-images.yml` | dispatch | Setup: `ubuntu-latest`; Build: `self-hosted, Linux, X64` | Multi-arch image builds | +| **Deploy Gateway Agent** | `deploy-gateway-agent.yml` | push (main), dispatch | Validate: `ubuntu-latest`; Build+Test: `ai-lab`; VPS: `kvm4` | Gateway Agent deployment | +| **Integrations GHCR** | `integrations-ghcr.yml` | push, dispatch | Setup: `ubuntu-latest`; Build: `self-hosted, Linux, X64` | Integration image builds | + +### Maintenance & Tooling + +| Workflow | File | Trigger | Runner(s) | Purpose | +|----------|------|---------|-----------|---------| +| **Sync Secrets Local** | `sync-secrets-local.yml` | dispatch | `self-hosted, ai-lab` | Pull GitHub Secrets → local .env | +| **Env Preflight** | `env-preflight.yml` | push, dispatch | `windows-latest` | Windows environment validation | +| **Webhook Smoke** | `webhook-smoke.yml` | dispatch | `ubuntu-latest` | Render-webhook smoke test | +| **yt-dlp Bump** | `yt-dlp-bump.yml` | schedule (Mon 08:00 UTC) | `ubuntu-latest` | Auto-update yt-dlp dependency | +| **Python Images Canary** | `python-images-toolchain-canary.yml` | schedule (Mon 09:00 UTC), dispatch | `self-hosted, Linux, X64, vps` | Python base image freshness | + +--- + +## Runner Assignment Summary + +| Runner | Workflow Count | Typical Use | +|--------|---------------|-------------| +| `ubuntu-latest` | 12 (sole or partial) | Tests, linting, validation, lightweight CI | +| `self-hosted, Linux, X64` | 5 | CPU builds, image publishing, contract validation | +| `self-hosted, ai-lab, gpu` | 3 | GPU builds (currently disabled in main pipeline) | +| `self-hosted, cloudstartup, staging` | 2 | Staging deployments | +| `self-hosted, kvm4, production` | 2 | Production deployments | +| `self-hosted, vps` | 1 | Python images canary | +| `windows-latest` | 1 | Windows env preflight | + +--- + +## Deployment Pipeline Flow + +``` +Push to main + → self-hosted-builds.yml + ├── setup-matrix (ubuntu-latest) + ├── build-gpu (ai-lab, gpu) [DISABLED] + ├── build-cpu (self-hosted, X64) — 9 service matrix + ├── validate-contracts (self-hosted, X64) + ├── deploy-staging (cloudstartup) — needs: build-cpu + ├── functional-tests (self-hosted, X64) — needs: deploy-staging + └── deploy-production (kvm4) — needs: build-cpu, deploy-staging +``` + +--- + +## Key Credentials for Workflows + +| Secret | Used By | Purpose | +|--------|---------|---------| +| `GHCR_TOKEN` | build-images, self-hosted-builds, integrations-ghcr | Push to GitHub Container Registry | +| `GH_PAT_PUBLISH` | build-images | Cross-repo publishing | +| `DOCKERHUB_TOKEN` | build-images | Push to Docker Hub | +| `GH_APP_ID` / `GH_APP_SEC` | deploy-gateway-agent | GitHub App token generation | +| `HOSTINGER_API_KEY` | deploy-gateway-agent | Hostinger API access | +| `TAILSCALE_AUTHKEY` | deploy-gateway-agent | Tailscale node join | +| `DISCORD_WEBHOOK_URL` | self-hosted-builds | Deployment notifications | + +--- + +## Concurrency Strategy + +All workflows use: +```yaml +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: ${{ github.ref != 'refs/heads/main' }} +``` + +- **PRs:** Cancel stale runs (saves runner minutes) +- **main branch:** Never cancel (all runs complete) diff --git a/pmoves/integrations/archon b/pmoves/integrations/archon index e965b4f50d..f4bd252c0e 160000 --- a/pmoves/integrations/archon +++ b/pmoves/integrations/archon @@ -1 +1 @@ -Subproject commit e965b4f50d1c1362a0442cd1ab99b2d790fa3f82 +Subproject commit f4bd252c0ecf9ff86d31ed42b5da55034c7afe9f From 97caf8c85c65a646c7331633686fcf22cb1893f2 Mon Sep 17 00:00:00 2001 From: POWERFULMOVES Date: Sat, 14 Mar 2026 22:34:59 -0400 Subject: [PATCH 2/3] fix(topology): revert hardcoded CF account ID, fix port conflicts, add E2B MCP agent MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SECURITY: Revert hardcoded Cloudflare account_id and KV namespace ID to placeholder values — these should come from env vars or wrangler login. Also fixes: - PostgREST port 3000 annotated as container-internal (via Kong at 8000) - Grafana port corrected to 3002 (actual host port) - TensorZero node_affinity fixed to [kvm4-1, z890], team to infra - tts.pmoves.ai DNS route corrected to Z890 (GPU-bound service) - Added pmoves_e2b_mcp_server to agent registry - Clarified external team count (0 agents, 7 human contributors) Co-Authored-By: Claude Opus 4.6 (1M context) --- .claude/context/runner-topology.md | 4 ++-- deploy/cloudflare/wrangler.toml | 8 ++++---- pmoves/config/agent_registry.yaml | 32 +++++++++++++++++++++++++++--- pmoves/docs/operations/TOPOLOGY.md | 6 +++--- 4 files changed, 38 insertions(+), 12 deletions(-) diff --git a/.claude/context/runner-topology.md b/.claude/context/runner-topology.md index fe02d32340..57662c7638 100644 --- a/.claude/context/runner-topology.md +++ b/.claude/context/runner-topology.md @@ -39,7 +39,7 @@ GitHub event → CF Worker (analyzes files) → ai-lab (GPU) / vps (Docker) / ub | infra | kvm4-1, kvm2 | vps | 3 | | sandbox | cloud | ubuntu-latest | 7 | | life | z890 | ubuntu-latest | 2 | -| external | N/A (CLI) | N/A | 7 (contributors) | +| external | N/A (CLI) | N/A | 0 agents (7 human contributors) | ## DNS Subdomains (pmoves.ai) @@ -48,7 +48,7 @@ GitHub event → CF Worker (analyzes files) → ai-lab (GPU) / vps (Docker) / ub | `api.pmoves.ai` | KVM4-1 | Yes | | `agent.pmoves.ai` | KVM4-1 | Yes | | `rag.pmoves.ai` | KVM4-1 | Yes | -| `tts.pmoves.ai` | KVM4-1 | Yes | +| `tts.pmoves.ai` | Z890 (via Tailscale relay) | Yes | | `n8n.pmoves.ai` | KVM4-1 | Yes | | `grafana.pmoves.ai` | KVM4-2 | Yes | | `search.pmoves.ai` | KVM4-2 | Yes | diff --git a/deploy/cloudflare/wrangler.toml b/deploy/cloudflare/wrangler.toml index 7c8e0f5aab..c78e8c7ee2 100644 --- a/deploy/cloudflare/wrangler.toml +++ b/deploy/cloudflare/wrangler.toml @@ -6,14 +6,14 @@ main = "worker.js" compatibility_date = "2025-12-08" workers_dev = true -# Account ID -account_id = "fe5ef93f1ffc7070b2218b96f8b256f5" +# Account ID — set via CLOUDFLARE_ACCOUNT_ID env var or wrangler login +# account_id = "your_account_id_here" # KV Namespace for build state tracking (create via: wrangler kv:namespace create "CI_STATE") [[kv_namespaces]] binding = "CI_STATE" -id = "9f6cdc5ecd6540698c9e2ca7d6376fd4" -# preview_id = "your_preview_kv_namespace_id" # Create via: wrangler kv:namespace create "CI_STATE" --preview +id = "your_kv_namespace_id" +preview_id = "your_preview_kv_namespace_id" # R2 Bucket for build artifacts (optional) # [[r2_buckets]] diff --git a/pmoves/config/agent_registry.yaml b/pmoves/config/agent_registry.yaml index e14d0a1752..a867952937 100644 --- a/pmoves/config/agent_registry.yaml +++ b/pmoves/config/agent_registry.yaml @@ -317,9 +317,9 @@ agents: swarm_participant: false attribution_gated: false topology: - node_affinity: [5090, z890] - team: evolution - ci_runner: ai-lab + node_affinity: [kvm4-1, z890] + team: infra + ci_runner: self-hosted compose_profile: null description: "Centralized LLM gateway for all model providers" @@ -1703,3 +1703,29 @@ agents: ci_runner: ai-lab compose_profile: null description: "AI-enhanced media management with Jellyfin integration" + + pmoves_e2b_mcp_server: + name: "E2B MCP Server" + class: utility + primary_type: agent + secondary_type: worker + port: null # stdio transport, no HTTP port + health: null + layers: [L0] + evolution_stage: base + nats: + publishes: [] + subscribes: [] + chit_toggles: + delta_sensitive: false + kappa_sensitive: false + hz_sensitive: false + swarm_participant: false + attribution_gated: false + submodule: "pmoves-e2b-mcp-server" + topology: + node_affinity: [z890] + team: sandbox + ci_runner: self-hosted + compose_profile: null + description: "MCP bridge for E2B sandboxed code execution" diff --git a/pmoves/docs/operations/TOPOLOGY.md b/pmoves/docs/operations/TOPOLOGY.md index 06aa53cfa6..a97f43fd5d 100644 --- a/pmoves/docs/operations/TOPOLOGY.md +++ b/pmoves/docs/operations/TOPOLOGY.md @@ -43,14 +43,14 @@ Services deployed via `docker-compose.vps.override.yml`: | Service | Port | Health | Compose Profile | |---------|------|--------|-----------------| | Supabase DB (Postgres) | 5432 (int) / 54322 (ext) | — | supabase-local | -| Supabase PostgREST | 3000 (int) | — | supabase-local | +| Supabase PostgREST | 3000 (container-internal, via Kong at 8000) | — | supabase-local | | Kong Gateway | 8000 / 65421 | — | supabase-local | | Qdrant | 6333 | `/healthz` | — | | Neo4j | 7474 (HTTP) / 7687 (Bolt) | `/db/neo4j/health` | — | | Meilisearch | 7700 | `/health` | — | | NATS | 4222 / 9222 (WS) | `http://:8222/varz` | — | | Prometheus | 9090 | `/-/healthy` | monitoring | -| Grafana | 3000 | `/api/health` | monitoring | +| Grafana | 3002 | `/api/health` | monitoring | | Loki | 3100 | `/ready` | monitoring | | MinIO | 9000 (API) / 9001 (Console) | `/minio/health/live` | — | @@ -130,7 +130,7 @@ Z890 ←→ KVM4-1 ←→ KVM4-2 | `rag.pmoves.ai` | A | KVM4-1 IP | Proxied | KVM4-1 | HTTPS | | `agent.pmoves.ai` | A | KVM4-1 IP | Proxied | KVM4-1 | HTTPS | | `search.pmoves.ai` | A | KVM4-2 IP | Proxied | KVM4-2 | HTTPS | -| `tts.pmoves.ai` | A | KVM4-1 IP | Proxied | KVM4-1 | HTTPS | +| `tts.pmoves.ai` | A | Z890 IP | Proxied | Z890 (via Tailscale relay from KVM4-1) | HTTPS | **Note:** NATS, MinIO, and Headscale use DNS-only (no Cloudflare proxy) because they use non-HTTP protocols or need direct TCP connections. From 69919ccedccf1d5d21d437663cef955e8f05aa07 Mon Sep 17 00:00:00 2001 From: Shaela Bello Date: Sun, 15 Mar 2026 02:48:38 -0400 Subject: [PATCH 3/3] =?UTF-8?q?fix(topology):=20address=20CodeRabbit=20rev?= =?UTF-8?q?iew=20=E2=80=94=20NATS=20URL,=20markdown=20lint,=20runner=20lab?= =?UTF-8?q?els,=20count-neutral=20comments?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 4.6 (1M context) --- .claude/CLAUDE.md | 4 ++-- .claude/context/runner-topology.md | 16 ++++++++-------- pmoves/config/agent_registry.yaml | 8 ++++---- pmoves/configs/agent-teams.yaml | 7 ++++--- pmoves/docs/operations/TOPOLOGY.md | 6 +++--- pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md | 8 ++++---- 6 files changed, 25 insertions(+), 24 deletions(-) diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md index 480dc6e73c..c7b5acc513 100644 --- a/.claude/CLAUDE.md +++ b/.claude/CLAUDE.md @@ -510,9 +510,9 @@ Based on CodeRabbit learnings (see `.claude/learnings/ui-error-handling-review-2 **Nodes:** Z890 (dev/GPU), 5090 (primary GPU, pending), KVM4-1 (API gateway), KVM4-2 (data/storage), KVM2 (exit proxy), Cloudflare Edge (DNS/Worker). -**Agent Teams (11 teams, 61 agents):** `pmoves/configs/agent-teams.yaml` — orchestration, research, media, data, ui, automation, evolution, infra, sandbox, life, external. +**Agent Teams (11 teams, 62 agents):** `pmoves/configs/agent-teams.yaml` — orchestration, research, media, data, ui, automation, evolution, infra, sandbox, life, external. -**CI Runners:** AI Lab (GPU), cloudstartup (staging), kvm4 (production), kvm2 (backup), ubuntu-latest (lightweight). Routing via Cloudflare Worker (`deploy/cloudflare/worker.js`). +**CI Runners:** `self-hosted, ai-lab` (GPU), `self-hosted, cloudstartup` (staging), `self-hosted, kvm4` (production), `self-hosted, kvm2` (backup), `ubuntu-latest` (lightweight). Routing via Cloudflare Worker (`deploy/cloudflare/worker.js`). **DNS:** `pmoves.ai` zone (pending Cloudflare migration). Subdomains: api, agent, rag, tts, n8n, grafana, search, nats, minio, headscale, ci. diff --git a/.claude/context/runner-topology.md b/.claude/context/runner-topology.md index 57662c7638..6100772b53 100644 --- a/.claude/context/runner-topology.md +++ b/.claude/context/runner-topology.md @@ -6,26 +6,26 @@ | Node | Role | Key Services | Runner | |------|------|-------------|--------| -| **Z890** | Dev + GPU (3090Ti) | All (local Docker Compose) | `ai-lab, gpu, cuda` | +| **Z890** | Dev + GPU (3090Ti) | All (local Docker Compose) | `self-hosted, ai-lab, gpu, cuda` | | **5090** | Primary GPU (pending) | Future inference | (pending) | -| **KVM4-1** | API Gateway | TensorZero, Agent Zero, Hi-RAG, Archon, Gateway Agent | `kvm4, production` | -| **KVM4-2** | Data/Storage | Supabase, NATS, Qdrant, Neo4j, Meilisearch, MinIO, monitoring | `kvm4, production` | -| **KVM2** | Exit Node | nginx (SSL termination) | `kvm2, backup` | +| **KVM4-1** | API Gateway | TensorZero, Agent Zero, Hi-RAG, Archon, Gateway Agent | `self-hosted, vps, kvm4, production` | +| **KVM4-2** | Data/Storage | Supabase, NATS, Qdrant, Neo4j, Meilisearch, MinIO, monitoring | `self-hosted, vps, kvm4, production` | +| **KVM2** | Exit Node | nginx (SSL termination) | `self-hosted, vps, kvm2, backup` | | **Cloudflare** | Edge | DNS, CI Worker | — | ## Route: Public → Services -``` +```text Internet → Cloudflare DNS → KVM2 (nginx/SSL) → KVM4-1 (API) or KVM4-2 (data) ``` ## Route: CI/CD -``` +```text GitHub event → CF Worker (analyzes files) → ai-lab (GPU) / vps (Docker) / ubuntu-latest (light) ``` -## Agent Teams (11 teams, 61 agents) +## Agent Teams (11 teams, 62 agents) | Team | Node Affinity | CI Runner | Count | |------|--------------|-----------|-------| @@ -60,7 +60,7 @@ GitHub event → CF Worker (analyzes files) → ai-lab (GPU) / vps (Docker) / ub ## Key Files - `pmoves/configs/agent-teams.yaml` — Team definitions -- `pmoves/config/agent_registry.yaml` — Full 61-agent registry +- `pmoves/config/agent_registry.yaml` — Full agent registry - `deploy/cloudflare/worker.js` — CI routing logic - `deploy/scripts/deploy-vps.sh` — VPS deployment - `pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md` — All 19 workflows mapped diff --git a/pmoves/config/agent_registry.yaml b/pmoves/config/agent_registry.yaml index a867952937..13e7ca2227 100644 --- a/pmoves/config/agent_registry.yaml +++ b/pmoves/config/agent_registry.yaml @@ -318,8 +318,8 @@ agents: attribution_gated: false topology: node_affinity: [kvm4-1, z890] - team: infra - ci_runner: self-hosted + team: evolution + ci_runner: ai-lab compose_profile: null description: "Centralized LLM gateway for all model providers" @@ -1035,7 +1035,7 @@ agents: class: utility primary_type: ui secondary_type: data - port: 3000 + port: 3002 health: "/api/health" layers: [L0] evolution_stage: base @@ -1726,6 +1726,6 @@ agents: topology: node_affinity: [z890] team: sandbox - ci_runner: self-hosted + ci_runner: ubuntu-latest compose_profile: null description: "MCP bridge for E2B sandboxed code execution" diff --git a/pmoves/configs/agent-teams.yaml b/pmoves/configs/agent-teams.yaml index 136bd10c07..0776a34fd7 100644 --- a/pmoves/configs/agent-teams.yaml +++ b/pmoves/configs/agent-teams.yaml @@ -1,5 +1,5 @@ # PMOVES.AI Agent Team Definitions -# Groups 61 agents into functional teams with node affinity and runner assignments. +# Full agent roster grouped into functional teams with node affinity and runner assignments. # # Query: python -c "import yaml; t=yaml.safe_load(open('pmoves/configs/agent-teams.yaml')); [print(f'{k}: {len(v[\"agents\"])} agents') for k,v in t['teams'].items()]" # Last updated: 2026-03-14 @@ -71,7 +71,7 @@ teams: - meilisearch # Port 7700 — Full-text search - minio # Port 9000 — Object storage - prometheus # Port 9090 — Metrics - - grafana # Port 3000 — Dashboards + - grafana # Port 3002 — Dashboards - loki # Port 3100 — Log aggregation ui: @@ -137,6 +137,7 @@ teams: - surf # Browser automation agent - agentgym # Agent training gymnasium - agentgym_rl # RL extension for AgentGym + - pmoves_e2b_mcp_server # MCP bridge for E2B sandboxed execution external: name: "External Contributors" @@ -158,7 +159,7 @@ teams: - wealth # Firefly III finance tracking - health # wger fitness tracking -# Validation: all 61 agents from agent_registry.yaml must appear in exactly one team +# Validation: all agents from agent_registry.yaml must appear in exactly one team # Run: python -c " # import yaml # reg = yaml.safe_load(open('pmoves/config/agent_registry.yaml')) diff --git a/pmoves/docs/operations/TOPOLOGY.md b/pmoves/docs/operations/TOPOLOGY.md index a97f43fd5d..b26beafb5c 100644 --- a/pmoves/docs/operations/TOPOLOGY.md +++ b/pmoves/docs/operations/TOPOLOGY.md @@ -10,7 +10,7 @@ | Node | LAN IP | Tailscale Hostname | Public IP | Role | Runner Labels | vCPU / RAM | Cost | |------|--------|--------------------|-----------|------|---------------|------------|------| -| Z890 (Windows 11) | 192.168.1.92 / .94 | 100.113.38.37 | — | Dev, GPU (RTX 3090 Ti) | `ai-lab, gpu, cuda` | 32C / 128GB | electricity | +| Z890 (Windows 11) | 192.168.1.92 / .94 | 100.113.38.37 | — | Dev, GPU (RTX 3090 Ti) | `self-hosted, ai-lab, gpu, cuda` | 32C / 128GB | electricity | | 5090 PC | 192.168.1.65 / .66 | (pending onboarding) | — | Primary GPU (RTX 5090) | (future: `ai-lab`) | TBD | electricity | | KVM4-1 | — | pmoves-kvm4-1 | Hostinger (TBD) | API Gateway | `self-hosted, vps, kvm4, production` | 8C / 16GB | $10/mo | | KVM4-2 | — | pmoves-kvm4-2 | Hostinger (TBD) | Data / Storage | `self-hosted, vps, kvm4, production` | 8C / 16GB | $10/mo | @@ -48,7 +48,7 @@ Services deployed via `docker-compose.vps.override.yml`: | Qdrant | 6333 | `/healthz` | — | | Neo4j | 7474 (HTTP) / 7687 (Bolt) | `/db/neo4j/health` | — | | Meilisearch | 7700 | `/health` | — | -| NATS | 4222 / 9222 (WS) | `http://:8222/varz` | — | +| NATS | 4222 / 9222 (WS) | `http://nats:8222/varz` | — | | Prometheus | 9090 | `/-/healthy` | monitoring | | Grafana | 3002 | `/api/health` | monitoring | | Loki | 3100 | `/ready` | monitoring | @@ -230,6 +230,6 @@ Key secrets: - `deploy/scripts/deploy-vps.sh` — VPS deployment script - `pmoves/docker-compose.vps.override.yml` — VPS compose overrides - `pmoves/docs/operations/PORT_REGISTRY.md` — Port allocations -- `pmoves/config/agent_registry.yaml` — 61-agent registry +- `pmoves/config/agent_registry.yaml` — Full agent registry - `pmoves/configs/agent-teams.yaml` — Agent team assignments - `.claude/context/runner-topology.md` — Condensed topology for agent context diff --git a/pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md b/pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md index 6b7b7c6618..ef7704d54d 100644 --- a/pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md +++ b/pmoves/docs/operations/WORKFLOW_RUNNER_MAP.md @@ -65,16 +65,16 @@ ## Deployment Pipeline Flow -``` +```text Push to main → self-hosted-builds.yml ├── setup-matrix (ubuntu-latest) - ├── build-gpu (ai-lab, gpu) [DISABLED] + ├── build-gpu (self-hosted, ai-lab, gpu) [DISABLED] ├── build-cpu (self-hosted, X64) — 9 service matrix ├── validate-contracts (self-hosted, X64) - ├── deploy-staging (cloudstartup) — needs: build-cpu + ├── deploy-staging (self-hosted, cloudstartup) — needs: build-cpu ├── functional-tests (self-hosted, X64) — needs: deploy-staging - └── deploy-production (kvm4) — needs: build-cpu, deploy-staging + └── deploy-production (self-hosted, kvm4) — needs: build-cpu, deploy-staging ``` ---