Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,12 +145,13 @@ Each SDLC phase can have configurable automated checks that run before completio
- `merge-conflict`: Detects merge conflicts with base branch
- `draft-validation`: Validates refine phase draft documents
- `plan-yaml`: Validates plan phase YAML appendix
- `deployment`: Validates changes against locally running devserver (DinD)
- `fixer`: Auto-fixes certain check failures when possible

**Phase defaults:**
- Refine: draft validation
- Plan: YAML validation
- Implement: merge conflict, lint (auto-retry), tests, auto-fixer
- Implement: merge conflict, lint (auto-retry), tests, auto-fixer, deployment (optional, auto-retry)
- PR: none

Contracts can override phase defaults via the `phase_configs` field, allowing per-issue check customization.
Expand Down
83 changes: 83 additions & 0 deletions docs/architecture/orchestrator.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,9 @@ Fixed IPs:
- `GET /pipelines/stream` - Unified SSE stream for all active pipelines (supports `?ascii=true`, `?active_only=false`, `?full_dag=true`)
- `POST /pipelines/{id}/signal` - Sandbox signals (complete, progress, error)
- `GET /pipelines/{id}/decisions` - HITL decision queue
- `POST /pipelines/{id}/deployment-check/start` - Start devserver for deployment validation
- `GET /pipelines/{id}/deployment-check/status` - Poll devserver status
- `POST /pipelines/{id}/deployment-check/teardown` - Tear down devserver

**CLI Access:**
The `egg-orch` CLI (`sandbox/bin/egg-orch`) provides command-line access to all orchestrator API endpoints. Available in sandbox containers for agent use, or can be run from the host with appropriate environment variables. See the [README CLI Reference](../../README.md#egg-orch-cli) for command details.
Expand All @@ -220,6 +223,86 @@ The `egg-orch` CLI (`sandbox/bin/egg-orch`) provides command-line access to all
3. **Gateway → Orchestrator**: Health check (optional)
4. **Orchestrator → GitHub**: Webhook responses, PR updates

## Devserver Management (Deployment Validation)

The orchestrator manages Docker-in-Docker (DinD) devserver stacks during deployment validation checks. This enables testing agent-modified code against locally running services before merge.

### Architecture

**Orchestrator responsibilities:**
- Extract `docker-compose.yml` from committed state (before agent changes)
- Generate override mounts for agent-modified services
- Create isolated Docker network (`egg-check-{pipeline_id}`)
- Start devserver stack with resource limits
- Provide status polling endpoints for sandbox check runner
- Tear down stack after validation completes

**Sandbox check runner responsibilities:**
- Signal orchestrator to start devserver via REST API
- Poll status until healthy or timeout
- Run health checks against service endpoints
- Run validation tests from `.egg/deployment.yml`
- Signal teardown

### Security Properties

**Network isolation:**
- Devserver containers run in dedicated `egg-check-{pipeline_id}` bridge network
- No internet access (internal-only, no gateway, no DNS)
- Services can only communicate within the isolated network
- Sandbox checker makes HTTP requests from outside the devserver network

**Resource limits (per container):**
- CPU: 1.0 core
- Memory: 512 MB
- PIDs: 256 (prevents fork bombs)
- Hard timeout: 5 minutes for entire lifecycle

**Credential safety:**
- No cloud credentials or production secrets injected
- Environment variables scanned for suspicious patterns (AWS_*, GCP_*, AZURE_*, GOOGLE_CLOUD_*, *_SECRET_KEY, *_API_KEY, *_ACCESS_KEY, *_TOKEN, *_PASSWORD, *_CREDENTIALS)
- Only target repo code is mounted (no access to egg internals)

### Configuration

Target repositories opt in by providing `.egg/deployment.yml`:

```yaml
compose_file: "docker-compose.yml"
services:
- source_dir: "services/api"
service_name: "api"
container_mount_path: "/app"
health_endpoints:
api: "/health"
validation_tests:
- service: "api"
path: "/users"
method: "GET"
expected_status: 200
description: "API smoke test"
```

See `shared/egg_contracts/deployment.py` for full schema.

### API Flow

1. **Start**: Sandbox calls `POST /api/v1/pipelines/{id}/deployment-check/start`
- Orchestrator extracts compose config, generates overrides, starts stack
- Returns immediately with `{"status": "starting"}`

2. **Poll**: Sandbox polls `GET /api/v1/pipelines/{id}/deployment-check/status`
- Returns `{"status": "starting" | "healthy" | "unhealthy" | "error"}`
- Includes service IPs and ports when healthy

3. **Validate**: Sandbox runs health checks and tests against service endpoints

4. **Teardown**: Sandbox calls `POST /api/v1/pipelines/{id}/deployment-check/teardown`
- Orchestrator stops containers, removes network
- Returns `{"status": "stopped"}`

See `orchestrator/devserver.py` and `orchestrator/routes/checks.py` for implementation.

## Sandbox Lifecycle

### Orchestrator Mode Detection
Expand Down
13 changes: 11 additions & 2 deletions docs/development/STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ orchestrator/
├── dag_visualizer.py # ASCII DAG visualization for pipeline status
├── decision_queue.py # HITL decision queue
├── decision_timeout.py # Decision timeout handling
├── devserver.py # Devserver lifecycle manager for deployment validation (DinD)
├── dispatch.py # Agent dispatch logic
├── docker_client.py # Docker API client
├── events.py # Event bus for pipeline events
Expand All @@ -99,6 +100,7 @@ orchestrator/
├── unified_sse.py # Unified SSE stream for all pipelines
├── webhooks.py # GitHub webhook handlers
├── routes/ # API route handlers
│ ├── checks.py # Deployment validation check endpoints
│ ├── containers.py # Container management endpoints
│ ├── decisions.py # HITL decision endpoints
│ ├── health.py # Health check endpoints
Expand Down Expand Up @@ -149,13 +151,14 @@ sandbox/
```
shared/
├── egg_config/ # Configuration utilities
│ ├── constants.py # Centralized constants (ports, networks, container names)
│ ├── constants.py # Centralized constants (ports, networks, container names, devserver resource limits)
│ └── validators.py # Validation functions (URLs, emails, tokens, check commands)
├── egg_container/ # Shared container-launch config builder
│ └── __init__.py # build_sandbox_config(), build_sandbox_docker_cmd(), git_shadow_mounts(), to_dockerpy_kwargs()
├── egg_contracts/ # SDLC contract models, plan parser, role-based validation, HITL, feedback, phase checks, multi-agent orchestration, checkpoints
│ ├── models.py # Pydantic models including CheckDefinition, CheckResult, PhaseConfig, AgentExecutionModel
│ ├── phase_defaults.py # Default check configurations per SDLC phase
│ ├── deployment.py # Deployment validation configuration models (.egg/deployment.yml)
│ ├── agent_roles.py # Multi-agent role definitions (Coder, Tester, Documenter, Integrator)
│ ├── orchestrator.py # Multi-agent orchestration dispatch logic
│ ├── orchestration.py # Agent execution state management
Expand Down Expand Up @@ -197,6 +200,9 @@ integration_tests/
├── test_policy_enforcement.py # Policy enforcement tests
├── test_rate_limiting.py # Rate limiting tests
├── test_stack_lifecycle.py # Container lifecycle tests
├── deployment_validation/ # Deployment validation integration tests
│ ├── __init__.py
│ └── test_deployment_check_e2e.py # End-to-end devserver lifecycle tests
├── local_pipeline/ # Local orchestrator integration tests
│ ├── conftest.py # Local pipeline test fixtures
│ ├── docker-compose.yml # Orchestrator test environment
Expand Down Expand Up @@ -227,11 +233,13 @@ tests/
│ ├── test_contract_cli.py # Contract CLI tests
│ └── ...
├── scripts/
│ └── test_checks.py # Check script framework tests
│ ├── test_checks.py # Check script framework tests
│ └── test_deployment_check.py # Deployment check unit tests
├── shared/
│ └── egg_contracts/
│ ├── test_models.py # Contract model tests including check models
│ ├── test_phase_defaults.py # Phase default configuration tests
│ ├── test_deployment_config.py # Deployment configuration tests
│ ├── test_agent_recovery.py # Agent recovery and circuit breaker tests
│ ├── test_checkpoints.py # Checkpoint model tests
│ ├── test_redactor.py # Redactor tests for sensitive data masking
Expand Down Expand Up @@ -295,6 +303,7 @@ Key workflows for the SDLC pipeline (see `.github/workflows/` for complete list)
│ ├── base.py # CheckRunner base class
│ ├── run_check.py # Check execution entry point
│ ├── check_fixer.py # Auto-fix check runner
│ ├── deployment_check.py # Deployment validation (DinD devserver)
│ ├── draft_validation_check.py # Draft document validation
│ ├── lint_check.py # Lint check runner
│ ├── merge_conflict_check.py # Merge conflict detection
Expand Down
54 changes: 54 additions & 0 deletions docs/guides/sdlc-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -797,6 +797,7 @@ If not configured, the checker falls back to auto-discovery (scanning for Makefi
| **Merge Conflict** | `check-merge-conflict` | Detects conflicts with base branch | No |
| **Lint** | `check-lint` | Runs `make lint` if available | Yes |
| **Test** | `check-test` | Runs `make test` or pytest | No |
| **Deployment Validation** | `check-deployment` | Validates changes against locally running devserver (opt-in via `.egg/deployment.yml`) | No |
| **Auto-Fixer** | `check-fixer` | Attempts to auto-fix failed checks | N/A |

### Phase Default Configurations
Expand All @@ -814,10 +815,63 @@ Default checks for each phase are defined in `shared/egg_contracts/phase_default
- Lint check (required, 1 retry)
- Test check (required)
- Auto-fixer (optional)
- Deployment validation (optional, 1 retry, requires `.egg/deployment.yml`)

**PR phase:**
- No checks

### Deployment Validation

The deployment validation check (`check-deployment`) runs agent-modified code against a locally running devserver to catch integration issues before merge. This check is **opt-in** and requires target repositories to provide a `.egg/deployment.yml` configuration file.

**How it works:**

1. The orchestrator extracts the `docker-compose.yml` from the committed state (before agent changes)
2. Generates override mounts for agent-modified services based on service-to-source mappings
3. Starts the devserver stack in an isolated Docker network with resource limits
4. The sandbox check runner polls health endpoints and runs validation tests
5. The orchestrator tears down the stack after validation completes

**Configuration (`.egg/deployment.yml`):**

```yaml
compose_file: "docker-compose.yml" # Path relative to repo root
services:
- source_dir: "services/api" # Source directory (agent changes)
service_name: "api" # docker-compose service name
container_mount_path: "/app" # Mount path inside container
health_endpoints:
api: "/health" # Service name → health check path
validation_tests:
- service: "api" # Target service name
path: "/users" # Request path
method: "GET"
expected_status: 200
description: "API smoke test"
```

**Security guarantees:**

- Devserver containers run in an isolated Docker network (no internet, no access to other containers)
- Resource limits prevent exhaustion attacks (CPU, memory, PIDs)
- Hard timeout of 5 minutes for the entire devserver lifecycle
- No cloud credentials or production secrets are injected
- Suspicious environment variables (AWS_*, GCP_*, AZURE_*, GOOGLE_CLOUD_*, *_SECRET_KEY, *_API_KEY, *_ACCESS_KEY, *_TOKEN, *_PASSWORD, *_CREDENTIALS) are rejected

**When to use:**

- Microservices with docker-compose devserver setups
- Integration testing that requires multiple services running
- Validating API contracts between services

**When not to use:**

- Projects without docker-compose devserver infrastructure
- Simple single-service applications (use `make test` instead)
- Projects where devserver setup is complex or requires external dependencies

The check is optional by default and will skip if `.egg/deployment.yml` is not present. When enabled, it runs with 1 retry on failure.

### Customizing Phase Checks

Contracts can override phase defaults via the `phase_configs` field:
Expand Down