Skip to content

Chore/deepresearch health - #234

Merged
POWERFULMOVES merged 6 commits into
mainfrom
chore/deepresearch-health
Nov 6, 2025
Merged

POWERFULMOVES merged 6 commits into
mainfrom
chore/deepresearch-health

Conversation

@POWERFULMOVES

@POWERFULMOVES POWERFULMOVES commented Nov 6, 2025 •

Copy link
Copy Markdown
Owner

Branded images + SUPASERCH scaffolding are in, and I updated CI, monitoring, and docs. Summary and what’s next:

What I implemented

  • SUPASERCH service (pmoves-supaserch)
    • Service scaffold under pmoves/services/supaserch:
      • Dockerfile (python:3.11-slim, uvicorn entry), requirements.txt, app.py with /healthz and stub /v1/search.
    • Compose entry with port 8099, envs, and profiles agents/workers.
    • Monitoring: Prometheus blackbox job supaserch; Grafana “Up: SupaSerch” panel.
    • Make targets:
      • supaserch-health (checks 200 on /healthz)
      • build-push-supaserch (local multi-arch buildx to docker registry)
    • Docs: pmoves/docs/services/supaserch/README.md (goals, env, monitoring, build/publish, integration points).
    • Roadmap/NEXT_STEPS: Added PMOVES‑SUPASERCH milestones (M3 kickoff section 3b).
  • Branded multi-arch builds (local)
    • buildx-setup, docker-login, build-push-archon-ui, build-push-deepresearch, build-push-supaserch targets in pmoves/Makefile.
    • I used your creds from pmoves/env.shared to login and kicked off Archon UI build and DeepResearch build; Archon UI tag exists (OK_ARCHON). DeepResearch build initially mispointed context; fixed Makefile
      and CI to use context pmoves and Dockerfile services/deepresearch/Dockerfile.
  • CI GHCR matrix
    • .github/workflows/integrations-ghcr.yml: added deepresearch (fixed paths) and supaserch (pmoves/services/supaserch/Dockerfile) multi-arch builds with reproducible tags.
    • This keeps nightly/on-demand reproducible images for consistent deployments.
  • Agents bring-up
    • up-agents-ui now includes supaserch and deepresearch; health targets added for both.
  • DeepResearch polish (earlier)
    • Worker /healthz on :8098; Prometheus probe; Grafana tile; deepresearch-health and deepresearch-smoke added.

How to build branded images right now (local)

  • Ensure env creds are loaded from pmoves/env.shared:

    • export DOCKER_USERNAME=…; export DOCKER_PASS=…
      Pin in env.shared after push
  • Set:

    • ARCHON_UI_IMAGE=docker.io/$IMAGE_NAMESPACE/pmoves-archon-ui:$IMAGE_TAG
    • DEEPRESEARCH_IMAGE=docker.io/$IMAGE_NAMESPACE/pmoves-deepresearch:$IMAGE_TAG
    • SUPASERCH_IMAGE=docker.io/$IMAGE_NAMESPACE/pmoves-supaserch:$IMAGE_TAG

Bring up + smokes

  • make -C pmoves up-agents-ui
  • make -C pmoves archon-ui-smoke
  • make -C pmoves deepresearch-health
  • make -C pmoves deepresearch-smoke
  • make -C pmoves supaserch-health
  • MON_INCLUDE_CADVISOR=true make -C pmoves up-monitoring (Grafana http://localhost:3002)

Summary by CodeRabbit

Release Notes

  • New Features

    • Added DeepResearch and SupaSerch integration services to the platform.
    • Enabled multi-architecture container builds for improved deployment flexibility.
    • Introduced health monitoring endpoints for DeepResearch and SupaSerch services.
  • Chores

    • Updated CI/CD pipeline to build and publish new integrations.
    • Enhanced Grafana dashboards with monitoring panels for new services.
    • Configured Prometheus to track health metrics for DeepResearch and SupaSerch.
    • Added smoke test validation for DeepResearch service.

@POWERFULMOVES
POWERFULMOVES merged commit 7a55249 into main Nov 6, 2025
1 check failed
@coderabbitai

coderabbitai Bot commented Nov 6, 2025 •

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

Walkthrough

The PR introduces DeepResearch and SupaSerch services to the PMOVES architecture, adding multi-architecture Docker build support, health check endpoints, Docker Compose configurations, monitoring dashboards, Prometheus scrape jobs, CI/CD pipeline entries, and supporting documentation and smoke tests.

Changes

Cohort / File(s) Summary
CI/CD & Build Infrastructure
.github/workflows/integrations-ghcr.yml, pmoves/Makefile
Added deepresearch and supaserch entries to GHCR integration matrix; introduced multi-arch build variables (REGISTRY, TARGET_PLATFORMS, IMAGE_NAMESPACE, IMAGE_TAG), new build-push targets for archon-ui, deepresearch, and supaserch, plus health/smoke test targets. Updated up-agents-ui to include deepresearch and supaserch prerequisites.
Orchestration Configuration
pmoves/docker-compose.yml
Added DEEPRESEARCH_HEALTH_PORT environment variable and port mapping to deepresearch service; introduced new supaserch service with build context, environment variables (SUPASERCH_PORT, NATS_URL, HIRAG_URL, SUPA_REST_URL), port mapping, and agent/worker profiles.
DeepResearch Service
pmoves/services/deepresearch/requirements.txt, pmoves/services/deepresearch/worker.py
Added fastapi and uvicorn dependencies; implemented health check server in worker with /healthz endpoint reporting NATS connectivity, running concurrently on configurable port (default 8098).
SupaSerch Service
pmoves/services/supaserch/Dockerfile, pmoves/services/supaserch/app.py, pmoves/services/supaserch/requirements.txt
Created new SupaSerch service with FastAPI-based app.py exposing /healthz and /v1/search endpoints; Dockerfile for building Python 3.11 image; requirements listing fastapi, uvicorn, httpx, and nats-py.
Monitoring & Observability
pmoves/monitoring/grafana/dashboards/services-overview.json, pmoves/monitoring/prometheus/prometheus.yml
Added Grafana stat panels for DeepResearch and SupaSerch health status; added Prometheus scrape jobs for deepresearch (port 8098) and supaserch (port 8099) health endpoints via blackbox exporter.
Documentation
pmoves/docs/NEXT_STEPS.md, pmoves/docs/SMOKETESTS.md, pmoves/docs/services/monitoring/README.md, pmoves/docs/services/supaserch/README.md
Added SupaSerch subsection to NEXT_STEPS describing broker wiring and supporting tasks; documented new smoke test targets in SMOKETESTS; added DeepResearch health endpoint to monitoring docs; introduced comprehensive SupaSerch README with orchestration details, environment variables, health behavior, and build/publish steps.
Testing Tools
pmoves/tools/deepresearch_smoke.py
Implemented NATS-based smoke test script for DeepResearch with envelope building, request/response correlation, timeout handling, and formatted result output.

Sequence Diagram

sequenceDiagram
    participant ext as External Monitor
    participant worker as DeepResearch Worker
    participant nats as NATS
    participant app as SupaSerch App
    participant prom as Prometheus

    rect rgb(200, 220, 255)
    Note over worker: Worker Startup
    worker->>nats: connect()
    nats-->>worker: connected
    worker->>worker: create FastAPI health app
    worker->>worker: asyncio.create_task(_serve_health)
    worker->>worker: run uvicorn on :8098
    end

    rect rgb(220, 255, 220)
    Note over ext,prom: Health Check Loop
    prom->>ext: GET /healthz (via blackbox)
    ext->>worker: GET :8098/healthz
    worker-->>ext: {status, nats_connected, ...}
    ext-->>prom: 200 OK / healthz response
    prom->>app: GET :8099/healthz (via blackbox)
    app-->>prom: {status, service: supaserch, ...}
    end

    rect rgb(255, 240, 220)
    Note over nats,app: SupaSerch Search Request
    app->>nats: subscribe(RESULT_SUBJECT)
    nats-->>app: subscribed
    app->>nats: publish(REQUEST_SUBJECT, envelope)
    nats->>worker: message received
    worker->>nats: publish(RESULT_SUBJECT, result)
    nats-->>app: result with correlation_id
    app-->>app: match & return
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

  • Multiple interconnected services: DeepResearch adds async health server integration to existing worker; SupaSerch introduces new FastAPI service scaffold
  • Infrastructure across multiple layers: CI/CD matrix, Makefile build targets, Docker Compose orchestration, Prometheus/Grafana monitoring all need alignment verification
  • Async/concurrency concerns: DeepResearch worker.py uses asyncio.create_task for background health server—verify no race conditions or blocking operations
  • New public APIs: SupaSerch app.py exposes HTTP endpoints; verify request/response contracts and error handling
  • Documentation scope: Four doc files updated with varying detail levels; cross-reference consistency needed

Areas requiring extra attention:

  • pmoves/services/deepresearch/worker.py: Asyncio task lifecycle, NATS connection establishment timing, health server startup race conditions
  • pmoves/services/supaserch/app.py: Stub implementation intent and future expansion points; verify endpoint signatures match downstream consumers
  • pmoves/Makefile: New build-push targets with registry/platform variables; test multi-arch build execution and variable precedence
  • pmoves/monitoring/prometheus/prometheus.yml: Environment variable interpolation (SUPASERCH_HOST_PORT) in YAML context
  • pmoves/tools/deepresearch_smoke.py: NATS envelope schema and correlation_id matching logic

Possibly related PRs

Poem

🐰 Hoppy Code Hopping!

Two new services spring to life,
DeepResearch and SupaSerch shine bright—
Health checks probe through the NATS night,
Multi-arch builds hop without strife,
Monitoring dashboards show them right! ✨


📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 64c317a and ff1e659.

📒 Files selected for processing (15)
  • .github/workflows/integrations-ghcr.yml (1 hunks)
  • pmoves/Makefile (3 hunks)
  • pmoves/docker-compose.yml (1 hunks)
  • pmoves/docs/NEXT_STEPS.md (1 hunks)
  • pmoves/docs/SMOKETESTS.md (1 hunks)
  • pmoves/docs/services/monitoring/README.md (2 hunks)
  • pmoves/docs/services/supaserch/README.md (1 hunks)
  • pmoves/monitoring/grafana/dashboards/services-overview.json (1 hunks)
  • pmoves/monitoring/prometheus/prometheus.yml (1 hunks)
  • pmoves/services/deepresearch/requirements.txt (1 hunks)
  • pmoves/services/deepresearch/worker.py (2 hunks)
  • pmoves/services/supaserch/Dockerfile (1 hunks)
  • pmoves/services/supaserch/app.py (1 hunks)
  • pmoves/services/supaserch/requirements.txt (1 hunks)
  • pmoves/tools/deepresearch_smoke.py (1 hunks)

Note

Free review on us!

CodeRabbit is offering free reviews until Fri Nov 07 2025 to showcase some of the refinements we've made.

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant