Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 0 additions & 30 deletions .egg-state/agent-outputs/reviewer_code/brc-memory.md

This file was deleted.

6 changes: 3 additions & 3 deletions docs/architecture/brc-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ as ephemeral, regardless of any "durable" framing in older design notes.

## What problem this solves

A consensus reviewer's lifecycle today is a long-running session that holds
a blocking `egg-orch message wait-loop` between events. Issue #2908 is
removing that seam: a wrapper will hold the wait, and an agent will be
A consensus reviewer's lifecycle before #2908 was a long-running session
that held a blocking `egg-orch message wait-loop` between events. Issue
#2908 removed that seam: the wrapper holds the wait, and the agent is
invoked one-shot per actionable event. A stateless one-shot has no working
memory across invocations, so something else has to carry the reviewer's
distilled understanding of the codebase, the change under review, and the
Expand Down
23 changes: 9 additions & 14 deletions docs/guides/concurrent-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,15 @@ Agents communicate with each other during concurrent execution via the orchestra

### How to Wait

Agents wait for BRC messages with a single canonical command — `egg-orch message wait-loop` — which long-polls the bus server-side and exits only on a terminal match or a permanent error. The full contract (the one-liner for producers and reviewers, the five anti-patterns to avoid, the `egg-orch message wait` exit codes, the `HEARTBEAT` schema, and the `EGG_MESSAGE_POLL_MAX_WAIT` ↔ gateway-Squid coupling) is in [Agent Wait Patterns](../reference/agent-wait-patterns.md) — read it before writing an outer `for`-loop, a `sleep`, or a multi-call poll sequence.
**Agents don't wait — the consensus wrapper does.** Since [#2908](https://github.com/jwbron/egg/issues/2908) every blocking read on the message bus belongs to the wrapper tier: the wrapper blocks in `egg-orch message wait-loop` between actionable events (see [Consensus Wrapper](#consensus-wrapper) above) and invokes the agent one-shot per event. The agent handles the single event in its prompt and exits naturally; it must never run `egg-orch message wait` / `wait-loop`, an outer poll loop, or a `sleep` — the wrapper owns the wait and the heartbeat, and an agent-tier wait double-waits against it while holding the SDK turn open. The agent-facing contract is stated in the per-event prompt itself (`orchestrator/routes/event_prompt.py`); the wait-side mechanics — the wrapper's `wait-loop` idiom and per-state filters, the five anti-patterns the pump retired, the `egg-orch message wait` exit codes, the `HEARTBEAT` schema, and the `EGG_MESSAGE_POLL_MAX_WAIT` ↔ gateway-Squid coupling — are in [Agent Wait Patterns](../reference/agent-wait-patterns.md).

The wrapper constructs the `--for` filter set conditionally per BRC state ([wait-filter construction](../architecture/orchestrator.md#wait-filter-construction-pre-confirm-vs-post-confirm)):

```bash
# Producer RESPOND TO REVIEWS (pre-confirm, step 4) — must NOT include CONSENSUS_CONFIRMED
# Pre-confirm (wrapper-issued) — must NOT include CONSENSUS_CONFIRMED
# (orchestrator rejects it HTTP 400 — own confirm generates the signal; #2064, #2482)
egg-orch message wait-loop \
--for CONSENSUS_PROPOSE \
--for CONSENSUS_ACK \
--for CONSENSUS_NACK \
--for CONSENSUS_RE_REVIEW \
Expand All @@ -118,20 +121,12 @@ egg-orch message wait-loop \
# `--for STATUS` is required (#2531): once every reviewer has ACKed the current version,
# no further CONSENSUS_ACK/CONSENSUS_NACK arrive; the orchestrator emits a directed
# STATUS (metadata.ready_to_confirm == true) as the only signal that confirms are clear.
# On wake: if metadata.ready_to_confirm is true, go to CONFIRM; otherwise re-enter the wait.

# Producer STAY ALIVE (post-confirm, step 6) — exits on consensus, re-review, or overseer alert
# Post-confirm (wrapper-issued) — exits on consensus, re-review, or overseer alert
egg-orch message wait-loop \
--for CONSENSUS_CONFIRMED \
--for CONSENSUS_RE_REVIEW \
--for OVERSEER_ALERT

# Reviewer STAY ALIVE — also wakes on new proposals
egg-orch message wait-loop \
--for CONSENSUS_PROPOSE \
--for CONSENSUS_RE_REVIEW \
--for CONSENSUS_CONFIRMED \
--for OVERSEER_ALERT
```

### Sending Messages
Expand Down Expand Up @@ -364,13 +359,13 @@ Each agent tracks two state machines (producer and reviewer) independently:
3. **Converge**: When all critical reviewers ACK, the producer sends `CONSENSUS_CONFIRMED`. When all agents are confirmed, the phase advances. Reviewers also call `CONSENSUS_CONFIRMED` after completing all reviews; the protocol enforces multiple guards in both the producer and reviewer confirmation paths (see [Action Guards](#action-guards) and [Deadlock Prevention Guards](#deadlock-prevention-guards) below).
4. **Re-propose**: If a NACK is received, the producer addresses the feedback and re-proposes (with `changed_artifacts` to scope re-evaluation). Flip-flop cycles are capped at `max_flip_flops` (default: 3). If any reviewer had already confirmed on a prior proposal version, they automatically receive a `CONSENSUS_RE_REVIEW` message and are un-confirmed so they re-enter the review loop — preventing a deadlock where a stale-confirmed reviewer can never see the new proposal.

> **Note — `CONSENSUS_RE_REVIEW` handling:** Agents that receive a `CONSENSUS_RE_REVIEW` while staying alive **must** act on it immediately: reviewers of the re-proposing producer must re-review and ACK/NACK; all other agents must re-confirm via `egg-orch consensus confirmed`. Ignoring this message stalls the pipeline.
> **Note — `CONSENSUS_RE_REVIEW` handling:** A `CONSENSUS_RE_REVIEW` wakes the wrapper's wait and the agent is re-invoked with it as the actionable event; the agent **must** act on it in that invocation: reviewers of the re-proposing producer must re-review and ACK/NACK; all other agents must re-confirm via `egg-orch consensus confirmed`. Ignoring this message stalls the pipeline.

> **Multi-reviewer NACK aggregation barrier ([#2142](https://github.com/jwbron/egg/issues/2142)):** When **two or more distinct reviewers** have NACKed the current proposal version, the orchestrator rejects the producer's first re-propose attempt with HTTP 409 and a structured `open_nacks_blocked` envelope. The response inlines every unresolved NACK (`reviewer`, `reason`, `artifact_refs`, `timestamp`) so the producer can address all blocking findings in one re-propose without a separate fetch. After this single round-trip the producer has been informed of the full NACK set; the retry advances the version. Single-reviewer NACKs do not trigger the barrier — the producer received that NACK via wait-loop and is already acting on it. The barrier exists to prevent the race where the producer's wait-loop returns one NACK, the producer fixes it and re-proposes, and other in-flight NACKs against the same version are silently superseded — wasting a full review cycle when those reviewers re-NACK the new version verbatim.
> **Multi-reviewer NACK aggregation barrier ([#2142](https://github.com/jwbron/egg/issues/2142)):** When **two or more distinct reviewers** have NACKed the current proposal version, the orchestrator rejects the producer's first re-propose attempt with HTTP 409 and a structured `open_nacks_blocked` envelope. The response inlines every unresolved NACK (`reviewer`, `reason`, `artifact_refs`, `timestamp`) so the producer can address all blocking findings in one re-propose without a separate fetch. After this single round-trip the producer has been informed of the full NACK set; the retry advances the version. Single-reviewer NACKs do not trigger the barrier — the wrapper already re-invoked the producer with that NACK as its actionable event. The barrier exists to prevent the race where the producer's wait-loop returns one NACK, the producer fixes it and re-proposes, and other in-flight NACKs against the same version are silently superseded — wasting a full review cycle when those reviewers re-NACK the new version verbatim.

> **Stale-version verdict rejection ([#2142](https://github.com/jwbron/egg/issues/2142)):** A reviewer whose ACK or NACK lands after the producer has re-proposed (i.e. the verdict targets a superseded version) is rejected with HTTP 409 and a structured `stale_version` envelope. The response inlines the producer's current proposal snapshot (`current_proposal.version`, `artifacts`, `commit_sha`) so the reviewer can re-fetch, re-review the diff, and re-submit without a separate status query. ACK guard already enforced version-match in `check_ack_guard`; the NACK guard now mirrors it via `check_nack_guard(..., nack_version=...)`.

> **Note — `pending_acks` (exit code 2):** After a re-proposal, previously-confirmed reviewers are un-confirmed and must re-ACK. If the producer calls `confirmed` before those re-ACKs arrive, the command returns exit code **2** (`pending_acks`) — this is transient, not an error. The orchestrator re-arms the "ready to confirm" `STATUS` nudge on every producer `pending_acks` rejection ([#2100](https://github.com/jwbron/egg/issues/2100)), so the producer should wait for `STATUS` for any `pending_acks` branch — it fires automatically when the blocking condition (missing proposal, missing ACK, or stale ACK) clears. Disambiguate via `metadata.ready_to_confirm == True` (the same `STATUS` type is also used for unrelated notifications such as "Producer X excused from consensus"). Via `mcp__brc__confirm`, the equivalent is `ok=False` with `status="pending_acks"`; wait for the STATUS nudge and retry. **Do not** enter the STAY ALIVE `wait_loop --for CONSENSUS_CONFIRMED` as a recovery path — a producer whose own confirm hasn't succeeded yet deadlocks the pipeline (see [Anti-pattern 5](../reference/agent-wait-patterns.md#anti-pattern-5--producer-waits-on-consensus_confirmed-before-its-own-confirm-has-succeeded-2064)).
> **Note — `pending_acks` (exit code 2):** After a re-proposal, previously-confirmed reviewers are un-confirmed and must re-ACK. If the producer calls `confirmed` before those re-ACKs arrive, the command returns exit code **2** (`pending_acks`) — this is transient, not an error. The orchestrator re-arms the "ready to confirm" `STATUS` nudge on every producer `pending_acks` rejection ([#2100](https://github.com/jwbron/egg/issues/2100)), and `STATUS` is in the wrapper's pre-confirm wait filter — the nudge fires automatically when the blocking condition (missing proposal, missing ACK, or stale ACK) clears and the wrapper re-invokes the producer. Disambiguate via `metadata.ready_to_confirm == True` (the same `STATUS` type is also used for unrelated notifications such as "Producer X excused from consensus"). Via `mcp__brc__confirm`, the equivalent is `ok=False` with `status="pending_acks"`; the agent exits and the wrapper re-invokes it when the STATUS nudge arrives. The wrapper's pre-confirm filter deliberately **omits** `CONSENSUS_CONFIRMED` — waiting on it before the producer's own confirm has succeeded deadlocks the pipeline (see [Anti-pattern 5](../reference/agent-wait-patterns.md#anti-pattern-5--producer-waits-on-consensus_confirmed-before-its-own-confirm-has-succeeded-2064)).
>
> **Note — Reviewer `pending_acks`:** Reviewers can also receive exit code 2 from `confirmed` when they have stale ACKs (e.g., an ACK recorded before the producer proposed) **or unresolved NACKs** (a NACK issued against a producer that has not yet re-proposed). In the stale-ACK case, the reviewer must re-ACK the listed producers at their current proposal version before confirming. In the unresolved-NACK case, the reviewer must wait for the NACKed producer to re-propose, then re-review and ACK/NACK the new version before confirming.

Expand Down
4 changes: 2 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ This index helps both humans and LLMs navigate the documentation efficiently.
| [SDLC Contract](reference/sdlc-contract.md) | Full `egg-contract` command reference for tracking tasks, commits, decisions |
| [MCP Deployment Tools](reference/mcp-deployment-tools.md) | Six k8s-facing MCP tools: `get_deployment_context`, `validate_deployment_manifests`, `prune_stale_worktrees`, `validate_network_isolation`, `rebuild_and_rollout`, `get_service_logs` |
| [Agent MCP Tools](reference/agent-tools.md) | In-process SDK MCP tools sandbox agents call on the `tool_use` stream (45 verbs across 7 namespaces: `mcp__sdlc__*`, `mcp__brc__*`, `mcp__phase__*`, `mcp__progress__*`, `mcp__task__*`, `mcp__confluence__*`, `mcp__jira__*`); on by default — set `EGG_MCP_TOOLS=false` to opt out |
| [Agent Wait Patterns](reference/agent-wait-patterns.md) | Canonical `egg-orch message wait-loop` idiom for BRC STAY ALIVE, the five anti-patterns to avoid, the `egg-orch message wait` exit-code contract, the `HEARTBEAT` metadata schema, the `EGG_MESSAGE_POLL_MAX_WAIT` / `EGG_ORCH_WAITRESS_THREADS` env-var couplings, §7 host-side `egg-orch pipeline wait-status` for event-driven pipeline monitoring, §10 BRC event-pump wrapper wait surface (conditional filter construction, heartbeat ownership migration, `slice_id` propagation invariant, and idle-budget escalation path), and §10.9 per-event prompt composer + preamble collapse (prompt shape, full git-log delta, `EGG_BRC_MEMORY` matrix) |
| [Agent Wait Patterns](reference/agent-wait-patterns.md) | §0 the agent contract (agents never wait on the bus — the consensus wrapper owns all blocking waits and invokes agents one-shot per event; #2908/#3157), the wrapper's `egg-orch message wait-loop` idiom and per-state filters, the five anti-patterns the event pump retired, the `egg-orch message wait` exit-code contract, the `HEARTBEAT` metadata schema, the `EGG_MESSAGE_POLL_MAX_WAIT` / `EGG_ORCH_WAITRESS_THREADS` env-var couplings, §7 host-side `egg-orch pipeline wait-status` for event-driven pipeline monitoring, §10 BRC event-pump wrapper wait surface (conditional filter construction, heartbeat ownership migration, `slice_id` propagation invariant, and idle-budget escalation path), and §10.9 per-event prompt composer + preamble collapse (prompt shape, full git-log delta, `EGG_BRC_MEMORY` matrix) |
| [Jira Wrapper](reference/jira-wrapper.md) | `/api/v1/jira/*` gateway endpoints — read verbs (ticket read, JQL search with static project-scope extraction, ticket comments, GET-only execute passthrough) plus the bounded write extension (`ticket/create`, `ticket/edit`, `ticket/comment/add`, `issue-link/create`); private-mode only; project allowlist via `config/context-filters.yaml`; per-verb body schema with size caps; ADF wrapping for plain-text `description` / `comment` bodies (`gateway/jira_adf.py`); caller-supplied idempotency key with 5-minute in-process cache (`gateway/jira_idempotency.py`); operator-configurable `jira.link_types` / `jira.epic_link_field` knobs; audit redaction never logs body content; `not_found` envelope on read 404s |
| [Confluence Wrapper](reference/confluence-wrapper.md) | `/api/v1/confluence/*` read-only gateway endpoints (page read, descendants, footer/inline comments with v1 fallback, space list/pages, CQL search with static space-scope extraction, GET-only execute passthrough); private-mode only; space allowlist via `config/context-filters.yaml`; `not_found` envelope; response redaction (`accountId` / `emailAddress` / user-profile `_links.webui` / user-profile `_links.self`); future write-verb extension points |
| [Conditional ACK](reference/conditional-ack.md) | Reviewer verdict variant: ACK + `--pre-merge-condition "..."` attaches a merge-time human obligation (e.g. `git mv`) that surfaces in `egg-orch consensus status` and in a "Pre-merge Obligations" section on the auto-created PR body |
Expand Down Expand Up @@ -142,7 +142,7 @@ Each major component has detailed documentation:
| **Kubernetes / k3s migration** | [Kubernetes Migration](architecture/kubernetes-migration.md) | [Deployment Guide](guides/deployment.md), [Network Isolation](architecture/network-isolation.md), [Orchestrator Architecture](architecture/orchestrator.md) |
| **Concurrent execution mode** | [Concurrent Execution Guide](guides/concurrent-execution.md) | [SDLC Pipeline Guide](guides/sdlc-pipeline.md), [Orchestrator Architecture](architecture/orchestrator.md) |
| **Directed agent coordination** | [Concurrent Execution: Directed Coordination](guides/concurrent-execution.md#directed-coordination) | [Orchestrator CLI](reference/orchestrator-cli.md), [SDLC Pipeline Guide](guides/sdlc-pipeline.md) |
| **Agent STAY ALIVE / bus waits** | [Agent Wait Patterns](reference/agent-wait-patterns.md) | [Concurrent Execution: Message Bus](guides/concurrent-execution.md#how-to-wait), [Orchestrator CLI](reference/orchestrator-cli.md) |
| **Bus waits (wrapper-owned — agents never wait)** | [Agent Wait Patterns](reference/agent-wait-patterns.md) | [Concurrent Execution: Message Bus](guides/concurrent-execution.md#how-to-wait), [Orchestrator CLI](reference/orchestrator-cli.md) |
| **Agent roles and file permissions** | [Agent Roles Reference](reference/agent-roles.md) | [SDLC Pipeline Guide](guides/sdlc-pipeline.md), [Architecture Overview](architecture/README.md) |
| **Agent failure recovery** | [Agent Recovery Reference](reference/agent-recovery.md) | [Concurrent Execution Guide](guides/concurrent-execution.md), [Orchestrator Architecture](architecture/orchestrator.md) |
| **Restarting stuck agents/phases** | [Agent Recovery Reference](reference/agent-recovery.md#agent-level-restart) | [Pipeline Health Monitoring](guides/pipeline-health-monitoring.md), [Orchestrator CLI](reference/orchestrator-cli.md), [Phase Management MCP Tools](reference/orchestrator-cli.md#phase-management-mcp-tools) |
Expand Down
Loading
Loading