diff --git a/README.md b/README.md index 2c59f0a5d1..8c7b351cd6 100644 --- a/README.md +++ b/README.md @@ -88,7 +88,7 @@ Tier 3 decomposes large features into independent plan phases that run as parall ### Concurrent Execution Mode -An optional execution mode where all implement-phase agents (coder, tester, documenter, checker, reviewers) start simultaneously, each in their own worktree branch. Agents communicate via the orchestrator message bus and signal readiness via a consensus protocol. Phase completion requires all agents to reach `READY` state. Enable with `concurrent_execution: true` in pipeline config. +An optional execution mode where all implement-phase agents (coder, tester, documenter, checker, reviewers) start simultaneously, all sharing the pipeline branch. Agents communicate via the orchestrator message bus and signal readiness via a consensus protocol. Phase completion requires all agents to reach `READY` state. Enable with `concurrent_execution: true` in pipeline config. See [Concurrent Execution Guide](docs/guides/concurrent-execution.md) for the full protocol. diff --git a/docs/guides/concurrent-execution.md b/docs/guides/concurrent-execution.md index bd126e8220..b686fce6b6 100644 --- a/docs/guides/concurrent-execution.md +++ b/docs/guides/concurrent-execution.md @@ -1,6 +1,6 @@ # Concurrent Execution Mode -Concurrent execution mode runs all implement-phase agents simultaneously — each in its own worktree branch — rather than sequentially in dependency-ordered waves. Agents communicate via the orchestrator message bus and signal readiness for phase completion via a consensus protocol. +Concurrent execution mode runs all implement-phase agents simultaneously — all sharing the pipeline branch — rather than sequentially in dependency-ordered waves. Agents communicate via the orchestrator message bus and signal readiness for phase completion via a consensus protocol. This is distinct from the standard wave-based parallel execution (Tier 2), where agents run in dependency order but multiple independent agents execute in parallel within each wave. @@ -34,7 +34,7 @@ When concurrent execution starts for the implement phase, the `ConcurrentPhaseEx - `reviewer_code` - `reviewer_contract` -**Worktree isolation**: Each agent is assigned a dedicated branch `egg/issue-{N}/{role}` (e.g., `egg/issue-123/coder`). This gives each agent its own isolated working directory while all agents share visibility into the same remote branch history. +**Shared branch**: All agents operate on the pipeline's shared branch (e.g., `egg/issue-123`). Agents coordinate commits via the message bus to sequence their work and avoid conflicts. **Environment injection**: Each concurrent agent receives: @@ -206,15 +206,9 @@ The 60-second window is tracked via the `_failure_times` list, filtered to recen | Consensus timeout | Advance anyway, Wait longer, Abort | | Agent objection | Resolve then advance, Override, Abort | -## Per-Agent Worktree Isolation +## Shared Pipeline Branch -Each concurrent agent operates on its own branch `egg/issue-{N}/{role}`, created as a git worktree by the gateway. This provides: - -- **File system isolation**: Each agent's workspace is separate; one agent cannot accidentally overwrite another's in-progress changes -- **Concurrent commits**: Agents can commit independently without coordination -- **Conflict visibility**: Merge conflicts between agents' branches become explicit at integration time (when the integrator merges results) - -After concurrent execution completes, the integrator role is responsible for merging the per-role branches and resolving any conflicts. +All concurrent agents operate on the pipeline's shared branch (e.g., `egg/issue-{N}`). Rather than each agent having an isolated worktree branch, all agents commit directly to a single shared history. Agents coordinate via the message bus to sequence commits and avoid conflicts — for example, the coder signals `HANDOFF` when its changes are committed so downstream agents (tester, documenter) know it is safe to pull and build on top. ## Orchestrator API Reference diff --git a/docs/guides/coordinator.md b/docs/guides/coordinator.md index f552dc560e..7f985b4f00 100644 --- a/docs/guides/coordinator.md +++ b/docs/guides/coordinator.md @@ -140,7 +140,10 @@ Common error codes: - **400** — Invalid request (missing fields, invalid role/phase) - **403** — Coordinator mode not enabled on the pipeline - **404** — Pipeline or agent not found -- **409** — Phase advancement blocked (e.g., no contract exists before implement/pr) +- **409** — Conflict. Possible causes: + - Phase advancement blocked because no contract exists before implement/pr phase + - Agent spawn rejected because the pipeline has no contract in implement/pr phase + - Role's dependencies have not yet completed (e.g., spawning `tester` before `coder` is done) - **429** — Guardrail limit exceeded (max agents or max retries per role) - **500** — Internal error (container spawn failure, etc.) @@ -164,6 +167,10 @@ The `role` must be a valid `AgentRole` **and** must be appropriate for the curre | `implement` | `coder`, `tester`, `documenter`, `integrator`, `reviewer_code`, `reviewer_contract` | | `pr`, `coordinator` | Any role (no phase-role restriction) | +**Dependency enforcement**: The orchestrator checks that the role's declared dependencies are complete before spawning. For example, `tester` depends on `coder` — spawning `tester` before `coder` has a `"complete"` status returns HTTP 409 with a `missing_dependencies` list. Dependencies are defined per-role in `shared/egg_contracts/agent_roles.py`. Spawn dependencies across the coordinator's full agent history are checked (including agents from prior phases). + +**Contract enforcement**: Spawning any agent in the `implement` or `pr` phase when the pipeline has no contract (`contract_synced: false`) returns HTTP 409. Contracts are auto-created at pipeline startup; a 409 here indicates that creation failed — check orchestrator logs. + Returns 429 if guardrail limits are exceeded. The response includes the `spawn_record` with the assigned `retry_number` (0 for the first spawn of a given role, incremented for each subsequent spawn of the same role). ### Get Coordinator State @@ -390,6 +397,10 @@ The coordinator runs with `phase="coordinator"` — a special phase value distin **Agent spawn rejected (HTTP 400 — invalid phase-role)**: The role is not valid for the current pipeline phase. Check the current phase via `egg-orch coordinator state ` and spawn a role that matches. For example, `coder` is only valid in the `implement` phase; `refiner` is only valid in the `refine` phase. Phases `pr` and `coordinator` have no restriction. +**Agent spawn rejected (HTTP 409 — missing dependencies)**: The role's declared dependencies have not yet completed. Check `egg-orch coordinator state ` and look at `completed_agents` to see which roles have finished. Spawn the dependency roles first (e.g., spawn `coder` before `tester`). The error response includes a `missing_dependencies` field listing which roles still need to complete. + +**Agent spawn rejected (HTTP 409 — no contract)**: The pipeline is in the implement or pr phase but has no contract. Run `egg-orch pipeline get ` and check `contract_synced`. If false, contract creation at startup failed — see the troubleshooting entry for "Phase advance blocked (HTTP 409 — no contract)" below. + **Agent spawn rejected (HTTP 429)**: Check guardrail limits via `egg-orch coordinator state `. The `guardrail_counters` section shows current counts vs. configured limits. **Escalation not resolved**: Check pending decisions via `egg-orch decision list `. Escalations create standard HITL decisions that need human resolution. diff --git a/docs/guides/sdlc-pipeline.md b/docs/guides/sdlc-pipeline.md index 13454e6ab3..eb1d819b5f 100644 --- a/docs/guides/sdlc-pipeline.md +++ b/docs/guides/sdlc-pipeline.md @@ -1414,22 +1414,18 @@ all changes. **Reviewer (code/contract)**: Reviews committed code or contract artifacts. Polls for `PROGRESS` from coder. Signals `READY` after review is complete. -### Per-Agent Worktrees +### Shared Pipeline Branch -Each concurrent agent operates on its own worktree branch to avoid git conflicts: - -``` -egg/issue-999/coder ← coder's work -egg/issue-999/tester ← tester's work -egg/issue-999/documenter ← documenter's work -egg/issue-999/checker ← checker's work -egg/issue-999/reviewer_code ← code reviewer's work -egg/issue-999/reviewer_contract ← contract reviewer's work -``` +All concurrent agents operate on the pipeline's shared branch (e.g., `egg/issue-999`). +Rather than each agent having an isolated worktree branch, all agents commit directly +to a single shared history. Agents coordinate via the message bus to sequence commits +and avoid conflicts — for example, the coder signals `HANDOFF` when its changes are +committed so downstream agents (tester, documenter) know it is safe to pull and build +on top. After all concurrent agents reach consensus, the integrator runs in a separate -(non-concurrent) step to merge all per-agent worktree branches, resolve conflicts, -run the full test suite, and validate integration before the phase completes. +(non-concurrent) step to validate integration, run the full test suite, and confirm +the shared branch is ready for merge. ### Failure Handling @@ -1502,10 +1498,11 @@ identify blocked or stuck agents. **Message bus empty**: Verify the pipeline has `concurrent_execution: true` in its config. The message bus is only active for concurrent pipelines. -**Merge conflicts at integration**: After all concurrent agents reach consensus, the -integrator runs in a separate step to merge per-agent worktree branches. If conflicts -are complex, the integrator signals `BLOCKED` and a HITL decision is created. Consider -adding role-based file restrictions to minimize overlap. +**Commit conflicts**: Since all concurrent agents share a single branch, agents +coordinate commits via the message bus to avoid conflicts. If an agent encounters a +conflict when pushing, it should pull, rebase, and retry. If conflicts persist, the +agent signals `BLOCKED` and a HITL decision is created. Consider adding role-based +file restrictions to minimize overlap. --- diff --git a/orchestrator/README.md b/orchestrator/README.md index 1646102478..eacef04176 100644 --- a/orchestrator/README.md +++ b/orchestrator/README.md @@ -84,7 +84,7 @@ When `concurrent_execution: true` is set in the pipeline configuration, agents w - **Message bus** — Agents exchange typed messages (PROGRESS, QUESTION, RESPONSE, STATUS, AGENT_FAILED) via the orchestrator's message API. Messages can target a specific role or broadcast to all agents. - **Readiness consensus** — Each agent signals its readiness state (WORKING, READY, BLOCKED, OBJECTING). The phase advances only when all agents reach READY. Any OBJECTING agent blocks phase completion. -- **Per-agent worktrees** — Each concurrent agent gets an isolated worktree branch (e.g., `egg/issue-999/coder`, `egg/issue-999/tester`). The integrator merges these at the end. +- **Shared pipeline branch** — All concurrent agents operate on the pipeline's shared branch (e.g., `egg/issue-999`). Agents coordinate commits via the message bus. The `GET /pipelines/{id}/status` endpoint includes a `concurrent` section when this mode is active, showing message counts, consensus state, and agent lifecycle info. See [SDLC Pipeline Guide — Concurrent Execution](../docs/guides/sdlc-pipeline.md#concurrent-execution-mode) for full details.