fix(simplifier): de-role the producer + move companion gate into the verdict rubric (#3381) - #3382
Conversation
…rubric (#3381) The simplifier's *-analysis-human.md / *-plan-human.md companion kept coming out as a review/verification memo (verdict framing + file:line detail) despite #3376/#3377. Two independent failures, fixed separately. Problem A — production side (wrong format from the start). The simplifier was still structurally a reviewer: both prompt paths required it to emit an ADVISORY ACK/NACK verdict on the upstream draft in the same pass, so the agent entered every invocation wearing a reviewer hat and its output came out review-shaped. #3376 only fenced the companion section ("not a review HERE") while leaving the role identity intact. - Strip ALL reviewer/verdict semantics from the three simplifier prompt blocks (dual-role banner, dual-role-agents note, _build_producer_ orientation, and the _build_agent_prompt "## Reviewer role" section). The simplifier is now framed as a producer only: the upstream's PROPOSE is purely its cue to summarize; it issues no ACK/NACK. - The advisory review edge stays in review_graph.py as the pure event-pump wake-wire (removing it reintroduces the "v|" first-propose dedupe deadlock). No consensus-engine change is needed: is_fully_acked only counts CRITICAL reviewers, and the #3043 confirm-guard exclusion already drops a never-voted advisory edge once the upstream confirms, so the simplifier confirms normally without ever voting. Problem B — gate side (reviewer ACKed it anyway). The companion criteria #3376 added lived only in _build_reviewer_preparation (the "while waiting" prep text), not in the rubric the reviewer actually runs at verdict time. _get_refine_review_criteria / _get_plan_review_criteria were entirely about the refiner/planner draft with zero companion criteria, and criteria are keyed by reviewer TYPE, not by which producer is under review — so the gate had no grounds to NACK a review-shaped companion and ACKed by default. - Add a forcing _human_companion_review_criteria checklist into both verdict rubrics: the reviewer must walk it (summary-not-review, no file:line/symbol detail, materially lighter, non-engineer readable, faithful) before ACKing the simplifier, and NACK the simplifier (not the producer it summarises) on any defect. Tests: de-roled banner/orientation/agent-prompt regression guards (no verdict, no "## Reviewer role"), and verdict-rubric companion-checklist guards for both refine and plan. Targeted suite: 535 pass; ruff clean.
There was a problem hiding this comment.
Review — request changes
I focused on the one part of this PR that changes runtime BRC behavior (Problem A: de-roling the simplifier so it casts no ACK/NACK). The prompt edits themselves are clean and Problem B is well-scoped. But the "No consensus-engine change needed" claim is incomplete: it analyzes is_fully_acked and the #3043 confirm-guard, but not the event-loop's next-action derivation, which still treats the advisory edge as requiring a vote. The result is a behavioral regression.
Blocking — the advisory ACK was load-bearing for terminating the wake-arm; removing it turns a quiet wait into repeated ack spawns
The PR keeps the simplifier → refiner advisory edge "as the pure event-pump wake-wire" but removes the only thing that ever acknowledged it. Pre-PR the wake-wire was self-terminating; post-PR it is self-perpetuating. Trace:
_has_pending_peer_proposals(reviewer)(routes/consensus.py:158) returns a producer as "pending review" whenever the producer isPROPOSEDand the reviewer has no current-version verdict entry (entry is None or entry.version < current_version, line 193). It does not special-case advisory edges —producers_for(review_graph.py:69) includes them.- For the de-roled simplifier, the
(simplifier, refiner)matrix entry now never gets created (it casts no verdict — confirmed against the wrapper: theackarm runs the agent, which under the new prompt issues nothing; the wrapper does not auto-ACK,consensus_wrapper.py:484-489). So_has_pending_peer_proposals(simplifier)returnsrefinerfor the entire window refiner isPROPOSED. - In
_derive_next_action(routes/consensus.py:296), the dual-role simplifier inPROPOSEDfailscheck_confirm_guard(reviewer Guard 1must_have_reviewed,action_guards.py:456-463, becausehas_reviewed(simplifier, refiner)is now permanentlyFalse), falls through to the reviewer block, and returns"ack"(line 404-410). "ack"is a SPAWN action (event_loop.py:94). Terminal (EXITED) Jobs do not suppress re-spawn (kubernetes_spawner/_events.py:22-66and theoutcome_fordesign note: "an event whose pod failed without advancing the tracker would be silently swallowed … instead of respawned"). The driver re-derives every role on a fixed 5s poll (event_loop.py:1116-1125).
Net effect: for the whole span [simplifier proposes companion → refiner confirms] (which includes reviewer_agent_design and reviewer_refine reviewing refiner), the simplifier's derived action is ack and it gets re-spawned — invoking a full agent each time that can no longer satisfy the event. Pre-PR this same span was a quiet wait (line 414): the agent's advisory ACK had advanced the matrix entry so _has_pending_peer_proposals returned False.
Two concrete harms:
- Resource/robustness regression — repeated agent (pod) spawns for the duration the upstream is under review, where before there were none.
- Correctness risk — the re-spawned
ackinvocation still hands the agent the producer prompt ("write and PROPOSE the companion"). If the agent re-proposes the companion, it bumps the simplifier's own proposal version and invalidatesreviewer_refine's ACK on it (stale-version), churning the companion's own consensus.
This is not a permanent deadlock — confirm is agent-free (AGENT_FREE_ACTIONS, event_loop.py:95) and once refiner confirms, the #3043 filter (action_guards.py:434-454) drops the never-voted edge and the loop auto-confirms the simplifier. But "converges eventually after churn" is not the same as "no consensus-engine change needed."
What's needed:
- Make the wake-wire self-terminating again without re-introducing a verdict. Options: (a) skip advisory edges for a producer-only-configured reviewer in
_has_pending_peer_proposals/_derive_next_actionso a de-roled advisory reviewer is never assignedack; or (b) auto-record the advisory edge orchestrator-side when the simplifier is woken, sohas_reviewed/pending clears exactly as the agent's ACK used to. Either keeps the proven wake-up while stopping the re-spawn. - Add a behavioral test (not just prompt-text assertions) over
_derive_next_action/ the event loop: refinerPROPOSED+ simplifierPROPOSED(no verdict) must not keep yielding a spawn-ableack, and the phase must converge. The current tests (test_pipeline_prompts.py) only assert the prompt strings — they would not catch this.
Non-blocking observations
- Problem B scoping is correct. The companion checklist is added only to
_get_refine_review_criteria/_get_plan_review_criteria, which dispatch from reviewer typesrefine/plan(_get_review_criteria_for_type,pipelines.py:5789).reviewer_agent_design(typeagent-design) does not get it — good, since it doesn't review the simplifier. The explicit "NACK the simplifier, never the {producer}" guard inside the shared verdict is the right call given the reviewer renders separate per-edge verdicts. _human_companion_review_criteriais module-level and forward-referenced by_get_refine_review_criteria; resolved at call time, so the definition-after-use ordering is fine.- The prompt de-roling text itself reads cleanly and consistently across all four blocks.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Address the blocking review on #3382: removing the simplifier's advisory ACK left the simplifier->upstream edge as a wake-wire that was no longer acknowledged, so _has_pending_peer_proposals returned the upstream as pending for the whole window it was PROPOSED and _derive_next_action kept yielding a spawn-able 'ack' — re-invoking an agent that can no longer satisfy the event (and risking a companion re-PROPOSE that invalidates the companion reviewer's ACK). Add an explicit wake_only flag on ReviewEdge and mark both simplifier advisory edges (refine + plan). The flag carries no review obligation: - _has_pending_peer_proposals skips wake_only edges, so the de-roled simplifier is never derived 'ack' (quiet 'wait' instead). - check_confirm_guard excludes wake_only producers from the reviewer guards, so the simplifier confirms its companion without a verdict it will never cast — restoring pre-PR convergence timing. Behavioral regression tests over the next-action derivation: the simplifier never yields a spawn-able 'ack' while the upstream is PROPOSED, and the phase converges to global consensus without a simplifier verdict.
|
Thanks for the trace — the blocking finding was exactly right, and the fix is now pushed. Disposition1. Blocking — advisory ACK was load-bearing; removing it turns a quiet You were correct: the wake-wire became self-perpetuating because nothing acknowledged the I took option (a) — make the wake-wire self-terminating without reintroducing a verdict — via an explicit, principled marker rather than a role-name special-case:
Behavioral test (as requested) —
2. Non-blocking — Problem B scoping / forward-ref / prompt de-roling — Agreed on all three observations; no change needed. The companion checklist stays scoped to
— Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Re-review — approve (prior blocking issue resolved); non-blocking concerns on the rationale
The blocking finding from my last review is fixed and verified. The de-roled simplifier no longer churns a spawn-able ack for the window the upstream is PROPOSED: _has_pending_peer_proposals (routes/consensus.py:172,180-191) and check_confirm_guard (action_guards.py:459-463) both now exclude wake_only producers, so the simplifier derives a quiet wait post-propose and confirms its companion once reviewer_refine ACKs it. The two new derivation tests (test_consensus_next_action.py) exercise the real get_default_refine_graph() tracker and assert the post-propose wait/confirm/converge path — which is exactly the segment that was regressing. Good.
I also confirmed the fix is complete across the graph: every other reviewers_for/producers_for consumer safely no-ops on the wake-only edge, because the de-roled simplifier never creates a matrix entry over the refiner —
is_fully_acked(refiner)is critical-only (approval_matrix.py:294), so the refiner's convergence is unaffected by the never-cast advisory edge;_un_confirm_stale_reviewers/_invalidate_pre_proposal_acks(_state.py:206,260) and the_confirm.pyinvalidation loops all guard onentry is None/invalidate_ackno-op;- the
check_invariantsreviewer loop (action_guards.py:743-755)continues onentry is None, so a confirmed simplifier with no refiner verdict trips no invariant.
So I'm approving. The remaining items below are non-blocking — the feature works end-to-end — but they matter for this being critical, long-lived infra.
Non-blocking 1 — the justifying comments (and commit message) describe the wrong wake mechanism, and contradict themselves
The new comments assert the ack arm is the simplifier's wake-up:
"the BRC
ackarm re-invokes it when the refiner proposes (the proven wake-up the spawn-dedupe key relies on; a pure producer's first-propose key is constant and would never re-spawn)" —review_graph.pyrefine/plan graphs
This is self-contradictory: the same wake_only flag that the comment introduces is precisely what removes the edge from pending-review derivation, so the ack arm can never fire for it. More importantly, the underlying claim is false in the post-#3164 one-shot model:
- The simplifier is a producer; in WORKING it derives
proposefirst (R11a,routes/consensus.py:357-372) — notack. Theackarm only ever applied after it proposed (PROPOSED → reviewer fall-through). So theack/advisory edge was never the wake-to-propose; it was the post-propose advisory re-invocation (the thing de-roling removed, and the churn this PR fixes). - A WORKING producer whose derived action stays
proposedoes re-spawn every poll: a clean orient-and-exit is classifiedsuccess/legitimate, which frees the dedupe key, and the next poll re-derives the sameproposekey and spawns again. This is exactly what your owntest_event_loop.py::test_stale_exit_is_a_non_trigger_through_loopasserts ("Legitimate outcomes free the key, so it keeps getting respawned"). So the propose arm is the simplifier's wake-to-propose, and has been all along.
Net: the behavior is correct, but the documented reason for the wake_only edge's existence is wrong. That's a maintenance trap in consensus code — it tells the next person the ack arm is the load-bearing wake-wire when it is inert. Please correct the comments to state that (a) the simplifier is woken to propose by the propose-arm re-spawn, and (b) wake_only exists only to neutralize the residual advisory edge (no ack, no confirm-block), not to "drive" a wake.
Non-blocking 2 — given (1), the simplifier → refiner edge looks unnecessary; deleting it is simpler and removes a prompt contradiction
If the propose arm is the real wake, the advisory edge buys nothing functional — dropping ReviewEdge("simplifier", "refiner", …) / ("simplifier", "task_planner", …) entirely yields identical behavior (simplifier woken via propose arm; not a reviewer, so no ack churn and nothing to skip). It would also fix a concrete inconsistency this PR's de-roling leaves behind: because the edge keeps graph.is_reviewer("simplifier") == True, the BRC preamble still renders Your role type: **PRODUCER and REVIEWER (dual role)** and Your assigned producers: refiner (pipelines.py:13238-13241) — directly above the new "You are a producer only" banner. That mixed signal is the kind of thing #3381 was trying to kill. If there's a real reason to retain the edge (e.g. the health-monitor idle-suppression in _queries.py:179), call it out in the comment instead of the false "ack wake-wire" story.
Non-blocking 3 — the new tests don't exercise the actual wake path
Both new tests call _propose(t, "simplifier"), which drives the tracker FSM straight to PROPOSED, bypassing the event loop's spawn/dedupe entirely. They validate "given the simplifier has proposed, derivation converges without a verdict" — valuable for the churn fix — but they do not cover "the WORKING simplifier actually gets woken to PROPOSED." A regression that prevented the simplifier from ever proposing would not break them. The real wake is covered generically by the test_event_loop.py re-spawn tests, so this is acceptable; but since the PR's stated mechanism is both wrong and untested, a small loop-level test (or just fixing the comments to point at the mechanism that is tested) would close the gap.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Address review non-blocking feedback on #3382: - Fix the misleading 'ack-arm wake-wire' comments in review_graph.py and pipelines.py. The de-roled simplifier is woken to PROPOSE by the ordinary producer propose-arm (a WORKING producer re-derives 'propose' each poll; a legitimate orient-exit frees the spawn-dedupe key, so it is re-spawned until it proposes). wake_only only NEUTRALIZES the residual advisory edge; it does not drive the wake. - Stop the BRC preamble from rendering the simplifier as 'PRODUCER and REVIEWER (dual role)' with 'Your assigned producers: refiner'. A role whose only reviewed producers are reached via wake_only edges casts no verdict, so it now renders PRODUCER-only, consistent with its producer-only execution banner. is_dual_role is kept intact for the banner dispatch. - Add a derivation test asserting a WORKING simplifier derives 'propose' (the real wake-to-propose), covering the path the prior two tests skipped.
|
Thanks for the approval and the careful follow-up. The blocking finding was fixed in 99a3405; this commit addresses all three non-blocking items. DispositionNon-blocking 1 — justifying comments describe the wrong wake mechanism / contradict themselves — You're right: the Non-blocking 2 — edge looks unnecessary; preamble contradiction —
Non-blocking 3 — tests don't exercise the actual wake path — Added
— Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Re-review — request changes: Problem A's de-roling is incomplete; the de-roled simplifier still receives the full reviewer playbook
The delta since my last approval (commit 1712801) correctly addresses the three non-blocking items I raised:
- Wake-mechanism comments (
review_graph.py,_build_brc_preamble) now accurately describe the propose-arm as the wake andwake_onlyas a pure neutralizer. Good. - Role-type label now renders
Your role type: **PRODUCER**and drops theYour assigned producers: refinerline via the newreal_producers/casts_real_verdictscomputation (pipelines.py:13241-13242). Verified by rendering. - New behavioral tests in
test_consensus_next_action.pyexercise the realget_default_refine_graph()tracker and cover the post-proposewait/confirm/converge path and the propose-arm wake. Sound, and they close the gap I flagged.
The Problem B (gate-side) work is also correct: _human_companion_review_criteria is now in both verdict rubrics (_get_refine_review_criteria / _get_plan_review_criteria), scoped to NACK the simplifier, and the forward reference resolves at call time. Verified both render.
But there is a blocking gap that this PR's own thesis makes blocking, and that my prior non-blocking #2 pointed straight at. I fixed the symptom I named (the role-type label) but the root cause I named (graph.is_reviewer("simplifier") == True) drives two far larger reviewer-instruction blocks in the same preamble, and neither was de-roled.
Blocking — the simplifier preamble still hands the de-roled agent a complete ACK/NACK reviewer playbook
Both of these blocks in _build_brc_preamble are gated on raw is_reviewer, which remains True for the simplifier because the wake_only edge keeps it in _reviewer_roles (review_graph.py:95-97):
### Reviewer Lifecycle(pipelines.py:13499) — emitted in full to the simplifier:4. **REVIEW**: ... form independent judgment from the referenced code artifacts5. **ACK/NACK**: Your --reason IS your review ...with the literalegg-orch consensus nack <role> --files-reviewed ... --reasoncommand template6. **CONFIRM**: When all assigned producers reviewed ...7. **HANDLE RE-REVIEW** ... This is adversarial re-review ... NACK without hesitance
- The
**As a reviewer**directed-coordination block (pipelines.py:13698) —when you need clarification before ACK/NACKing, put the question in your NACK --reason block.
Rendered output for the simplifier (refine and plan, confirmed by running _build_brc_preamble) is self-contradictory: the new banner says
You do not review, critique, score, or vote on that draft — you never issue an ACK or a NACK.
and roughly five lines later the same preamble says
5. **ACK/NACK**:…egg-orch consensus nack <role> …
plus a full "adversarial re-review … NACK without hesitance" mandate.
This is exactly the failure mode the PR's Problem A is built to eliminate. Your own diagnosis is that the agent "entered every invocation wearing a reviewer hat and its output came out review-shaped," and the fix is to "strip all reviewer/verdict semantics from the … simplifier prompt blocks." You stripped four blocks (the two banners, _build_producer_orientation, the _build_agent_prompt reviewer section) but left the most operationally detailed reviewer instruction of all — a complete, actionable ACK/NACK + adversarial-re-review lifecycle — rendered verbatim to the de-roled simplifier. A single inserted producer-only sentence in step 2 (pipelines.py:13517-13525) does not neutralize the five surrounding steps that instruct it to REVIEW, ACK/NACK, CONFIRM, and adversarially re-review.
The codebase already encodes the correct invariant: test_producer_only_no_sync_step (test_pipeline_prompts.py:5320-5324) asserts that a producer-only role (coder) gets no ### Reviewer Lifecycle at all. This PR explicitly redefines the simplifier as "a producer only" — so by the project's own standard it must satisfy that same invariant, and it does not.
Per the review rules this is blocking: the feature's core purpose (de-role the simplifier so it stops receiving reviewer framing) does not work end-to-end in the very function the PR modifies to achieve it — a single-module break in _build_brc_preamble.
Suggested fix. You already compute casts_real_verdicts (pipelines.py:13242). Gate both reviewer blocks on it instead of raw is_reviewer:
if is_reviewer and casts_real_verdicts: # 13499 — Reviewer Lifecycle
...
if is_reviewer and casts_real_verdicts: # 13698 — "As a reviewer" coordination
...A pure reviewer (reviewer_refine) and the tester both have casts_real_verdicts == True (their reviewed producers are non-wake-only), so they keep the Reviewer Lifecycle; the simplifier (real_producers == []) drops it. Once gated out, the simplifier-specific producer-only note inside step 2 becomes dead code and should be removed with it.
Why the new guards didn't catch this
test_simplifier_banner_issues_no_verdict asserts only the absence of the specific phrases you removed ("issue your advisory verdict", "advisory ack/nack") and the presence of "you never issue an ack or a nack". None of those match the leaked ### Reviewer Lifecycle / **As a reviewer** text, so the guard passes while the contradiction stands. After the fix, add a positive regression guard mirroring the existing producer-only invariant, e.g.:
assert "### Reviewer Lifecycle" not in preamble
assert "egg-orch consensus nack <role>" not in preamble
assert "**As a reviewer**" not in preamble— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Gate both reviewer-instruction blocks in _build_brc_preamble on casts_real_verdicts instead of raw is_reviewer. The de-roled simplifier casts no verdict (its only reviewed producer is reached via a wake_only edge), so it must not receive the ACK/NACK + adversarial re-review playbook, which contradicted its producer-only execution banner. Removes the now-dead simplifier branch in Reviewer Lifecycle step 2 and adds a regression guard mirroring test_producer_only_no_sync_step.
|
Thanks — the blocking finding was exactly right: gating those two blocks on raw DispositionBlocking — the de-roled simplifier still receives the full ACK/NACK + adversarial-re-review reviewer playbook — You diagnosed the root cause precisely:
This brings the simplifier into compliance with the project's own producer-only invariant ( Regression guard (as requested) — assert "### Reviewer Lifecycle" not in preamble
assert "egg-orch consensus nack <role>" not in preamble
assert "**As a reviewer**" not in preamble
assert "NACK without hesitance" not in preambleVerified by rendering: the simplifier preamble (refine + plan) now reads
— Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Re-review — request changes: the de-roling is still incomplete; producer-lifecycle step 6 leaks dual-role reviewer framing (and a now-dangling cross-reference) to the simplifier
The delta since my last review (1712801..3a198b0) correctly applies the fix I suggested for the two blocks I named:
### Reviewer Lifecycle(pipelines.py:13509) and the**As a reviewer**coordination block (pipelines.py:13702) are now gated oncasts_real_verdicts(pipelines.py:13242) instead of rawis_reviewer. Verified by rendering: the simplifier (refine + plan) readsYour role type: **PRODUCER**, gets no Reviewer Lifecycle, noAs a reviewerblock, no NACK template;tester(dual-role) andreviewer_refine(pure reviewer) retain both blocks unchanged.- The dead
if role_value == "simplifier"branch inside Reviewer-Lifecycle step 2 is correctly removed — it was unreachable once the block is gated oncasts_real_verdicts(the simplifier never satisfies it). - The new test
test_simplifier_preamble_has_no_reviewer_lifecycleexercises the real_build_brc_preamblefor both phases. Good. Andtest_preparation_in_reviewer_lifecycle(reviewer_code) guards against over-aggressive gating that would strip a genuine reviewer.
casts_real_verdicts = bool([p for p in producers if p not in wake_only_producers]) is the right discriminator: the two wake_only=True edges (review_graph.py:253,304) are the only wake-only edges, both with the simplifier as reviewer, so the simplifier is the only role for which casts_real_verdicts == False. The gate is precise.
But the same root cause I flagged last round — graph.is_reviewer("simplifier") == True, and now also is_dual_role == True — drives a third reviewer-framing site that this commit did not de-role. The fix is incomplete in the very same way, in the same function.
Blocking — Producer-Lifecycle step 6 still hands the de-roled simplifier dual-role re-review/ACK-NACK framing, and points it at a Reviewer Lifecycle section this PR deleted from its preamble
### Producer Lifecycle is gated on is_producer (correct — the simplifier is a producer and its companion is reviewed). But step 6 HANDLE RE-REVIEW (pipelines.py:13456) contains an is_dual_role-gated parenthetical (pipelines.py:13458-13466):
"(or a
CONSENSUS_PROPOSEfor a re-propose — version > 1, after you NACKed a prior version; dual-role agents handle both — see Reviewer Lifecycle step 8 for the adversarial re-review framing)"
and the following sentence (pipelines.py:13468-13472):
"If you are a reviewer of the re-proposing producer, re-review and ACK/NACK the new proposal (dual-role agents: see Reviewer Lifecycle step 8 below for the adversarial re-review framing that applies to this case)."
is_dual_role is still True for the simplifier (the PR deliberately keeps the graph-level flag for banner dispatch, pipelines.py:13237), so the parenthetical renders. I confirmed by rendering _build_brc_preamble("simplifier", "refine"/"plan"): the simplifier preamble contains dual-role agents handle both, re-review and ACK/NACK the new proposal, and Reviewer Lifecycle step 8 — all True.
Two concrete harms, both newly introduced/left by this PR:
-
Self-contradiction — the exact failure mode Problem A exists to kill. The producer-only banner this PR adds says (
pipelines.py:13300-13304):"You do not review, critique, score, or vote on that draft — you never issue an ACK or a NACK."
Step 6, a few lines down in the same preamble, tells the same agent "dual-role agents handle both [re-reviews]" and "re-review and ACK/NACK the new proposal." This is the identical self-contradiction my prior review (and the round before it) called blocking — just relocated from the two big blocks into producer step 6. Your own diagnosis was to "strip all reviewer/verdict semantics from the simplifier prompt blocks"; this is residual reviewer/verdict semantics.
-
Dangling cross-reference introduced by this PR. Both clauses say "see Reviewer Lifecycle step 8 below." This PR removes the entire Reviewer Lifecycle from the simplifier's preamble — so for the simplifier the reference now points at a section that is not there at all. Pre-PR the simplifier got the full Reviewer Lifecycle, so the reference resolved; this PR breaks it. The next reader of the simplifier prompt is told to consult a section the same change deleted.
This is a single-module break in _build_brc_preamble — the feature's stated purpose (de-role the simplifier so it receives no reviewer framing) does not fully hold end-to-end in the function the PR modifies to achieve it.
Why the new guard misses it: test_simplifier_preamble_has_no_reviewer_lifecycle only asserts the absence of "### Reviewer Lifecycle", "egg-orch consensus nack <role>", "**As a reviewer**", and "NACK without hesitance" — none of which match the leaked step-6 strings. The guard passes while the contradiction stands, exactly as the previous guard did before this round.
Suggested fix. Gate the dual-role reviewer references in producer step 6 on casts_real_verdicts, mirroring the two blocks you already fixed:
# pipelines.py:13458 — the parenthetical
+ (
" (or a `CONSENSUS_PROPOSE` for a re-propose — ... dual-role "
"agents handle both — see Reviewer Lifecycle step 7 for the "
"adversarial re-review framing)"
if is_dual_role and casts_real_verdicts # was: if is_dual_role
else ""
),and make the "(dual-role agents: see Reviewer Lifecycle step 8 below ...)" aside in 13468-13472 conditional on the same casts_real_verdicts (today it is ungated, so it also renders for pure producers — see Non-blocking below). The tester keeps it (casts_real_verdicts == True); the simplifier drops it. Then extend the regression guard to assert the simplifier preamble contains none of "re-review and ACK/NACK the new proposal", "dual-role agents handle both", or "Reviewer Lifecycle step".
Non-blocking 1 — "Reviewer Lifecycle step 8" is itself a stale/off-by-one reference (pre-existing, affects the tester too)
The Reviewer Lifecycle renders only 7 steps (PREPARE … HANDLE RE-REVIEW); the adversarial re-review framing lives in step 7 (pipelines.py:13643), not step 8. So even for the tester, "see Reviewer Lifecycle step 8" points one step past the end. This predates the PR, but since you'll be editing these two references to fix the blocking leak anyway, correct step 8 → step 7 while you're there.
Non-blocking 2 — the ungated dual-role aside renders for pure producers with no Reviewer Lifecycle
"(dual-role agents: see Reviewer Lifecycle step 8 below ...)" (13468-13472) is ungated, so it renders in the coder/refiner preambles too — neither of which has a Reviewer Lifecycle. It is addressed to "dual-role agents," so a pure producer reasonably skips it, but it's a reference to a non-existent "step 8 below" for every pure producer. Gating it on casts_real_verdicts per the suggested fix above resolves this at the same time.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Gate the dual-role reviewer framing in Producer Lifecycle step 6 on casts_real_verdicts instead of raw is_dual_role / leaving it ungated, so the de-roled simplifier no longer receives 're-review and ACK/NACK the new proposal' / 'dual-role agents handle both' framing that contradicts its producer-only banner, and pure producers (coder) no longer get the ungated reviewer aside. Correct the stale 'Reviewer Lifecycle step 8' cross-reference to step 7 (the adversarial re-review framing lives in step 7). Extend the regression guard to assert the simplifier preamble contains none of the leaked step-6 strings.
|
Thanks — the blocking finding was exactly right, and the two non-blocking items rode along in the same edit. Fixed and pushed in DispositionBlocking — Producer-Lifecycle step 6 still leaks dual-role reviewer framing (and a dangling cross-reference) to the de-roled simplifier — You diagnosed it precisely: the same root cause (
Verified by rendering Regression guard (as requested) — Non-blocking 1 — "Reviewer Lifecycle step 8" is a stale/off-by-one reference — Correct — the Reviewer Lifecycle renders 7 steps and the adversarial re-review framing is step 7 ( Non-blocking 2 — the ungated dual-role aside renders for pure producers with no Reviewer Lifecycle — Resolved by the same
— Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Re-review — approve (prior blocking issue resolved); one non-blocking note on the degraded fallback path
The delta since my last review (3a198b0..b44a5ed) correctly resolves the blocking finding I raised and both non-blocking items that rode along.
Prior blocking issue — Producer-Lifecycle step 6 leaked dual-role reviewer framing to the de-roled simplifier — fixed and verified
I confirmed by rendering _build_brc_preamble("simplifier", "refine"/"plan"):
- Role label is
Your role type: **PRODUCER**with noYour assigned producers:line. - No
### Reviewer Lifecycle, no**As a reviewer**coordination block, noegg-orch consensus nack <role>template, noNACK without hesitance. - Step 6 (
HANDLE RE-REVIEW) now renders onlyRe-confirm via \egg-orch consensus confirmed`.— thedual-role agents handle bothparenthetical and there-review and ACK/NACK the new proposalsentence are gone. Both are gated oncasts_real_verdicts(pipelines.py:13509parenthetical viais_dual_role and casts_real_verdicts; the following sentence viacasts_real_verdictswith a producer-onlyelse`). - The only remaining
ACK/NACKstrings in the simplifier preamble are legitimate: the agent roster's descriptions of other reviewers, the producer-only banner that negates it ("you never issue an ACK or a NACK"), and the producer's own RESPOND-TO-REVIEWS / CONFIRM steps for its companion (whichreviewer_refinedoes gate CRITICAL).
Non-blocking 1 (step 8 → 7 off-by-one) — fixed. The two step-6 cross-references now say "Reviewer Lifecycle step 7", and I verified the rendered Reviewer Lifecycle has exactly 7 steps with step 7 = HANDLE RE-REVIEW (where the adversarial re-review framing lives). No step 8 string remains anywhere.
Non-blocking 2 (ungated aside rendering for pure producers) — fixed. coder/refiner/task_planner step 6 now renders the producer-only Re-confirm via … form (no dangling step reference).
Gate precision — verified
I rendered the full role matrix to confirm the casts_real_verdicts gate neither over- nor under-strips:
| role | phase | role label | Reviewer Lifecycle |
|---|---|---|---|
| simplifier | refine/plan | PRODUCER | absent ✓ |
| tester | implement | PRODUCER and REVIEWER | present ✓ |
| reviewer_refine | refine | REVIEWER | present ✓ |
| reviewer_plan | plan | REVIEWER | present ✓ |
| reviewer_code | implement | REVIEWER | present ✓ |
| risk_analyst | plan | REVIEWER | present ✓ |
| coder / refiner / task_planner | — | PRODUCER | absent ✓ |
casts_real_verdicts = bool([p for p in producers if p not in wake_only_producers]) is the right discriminator: the two wake_only=True edges (review_graph.py:253,304) are the only wake-only edges in the graph, both with the simplifier as reviewer, so the simplifier is the only role for which it is False. to_dict/from_dict round-trips the flag (verified). Any is imported (review_graph.py:12).
Tests — sound, exercise the production path
The new test_simplifier_preamble_has_no_reviewer_lifecycle / test_simplifier_banner_issues_no_verdict / test_simplifier_agent_prompt_has_no_reviewer_section and the two verdict-rubric tests drive the real _build_brc_preamble / _build_agent_prompt / _get_*_review_criteria helpers (no golden fixtures, no hand-built bypass), and the names match their assertions. test_preparation_in_reviewer_lifecycle (reviewer_code) guards against over-aggressive gating. The three test_consensus_next_action.py wake-only tests drive a real get_default_refine_graph() tracker through _derive_next_action. I ran the targeted selection locally: test_consensus_next_action.py -k wake_only (3) + TestSimplifierHumanCompanionPrompt (19) all pass.
Non-blocking — the casts_real_verdicts gate degrades all reviewers to PARTICIPANT in the graph-load fallback path
Changing the gate from raw is_reviewer to is_reviewer and casts_real_verdicts has an interaction with the except Exception: fallback in _build_brc_preamble (pipelines.py:13193) that this PR does not account for. In that branch producers = [] is set unconditionally (and wake_only_producers = set()), so real_producers == [] and casts_real_verdicts == False for every role — not just the simplifier.
Consequence in the fallback path:
- A genuine pure reviewer (
reviewer_code,reviewer_refine,reviewer_plan) now rendersYour role type: **PARTICIPANT**(it hits the finalelseatpipelines.py:13251, since it is neitheris_producernorcasts_real_verdicts) and gets no Reviewer Lifecycle and noAs a reviewerblock. - Pre-PR, the same fallback gated on raw
is_reviewer, so those roles renderedREVIEWERwith the full reviewer playbook.
So this PR silently strips the reviewer playbook from real reviewers whenever the graph load fails. The realistic impact is low: get_review_graph_for_phase is a pure dict lookup with no I/O (review_graph.py:388), so the except only fires on catastrophic review_graph import failure — a state where BRC is broken regardless. That's why I'm classifying this non-blocking, not as a feature break. But it is a latent inconsistency introduced in code the PR modifies, and it's cheap to close: in the except block, either set casts_real_verdicts = is_reviewer explicitly (the fallback can't distinguish wake-only edges, and the only wake-only role — the simplifier — is independently rendered producer-only by the is_dual_role and role_value == "simplifier" banner dispatch, which still fires in the fallback), or seed a non-empty producers sentinel for the known reviewer roles. A one-line assertion that reviewer_code's fallback preamble still contains ### Reviewer Lifecycle would lock it.
Net: Problem A (de-role the simplifier) and Problem B (companion checklist in the verdict rubric) both work end-to-end in the normal path, the three rounds of blocking findings are all resolved, and the fix is precise. Approving.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
The casts_real_verdicts gate (#3381) was computed unconditionally from producers - wake_only_producers. In the _build_brc_preamble graph-load fallback (except branch), producers == [] for every role, so casts_real_verdicts was False everywhere — silently degrading genuine pure reviewers (reviewer_code/refine/plan) to PARTICIPANT with no Reviewer Lifecycle whenever the review_graph load fails. Pre-#3381 that path gated on raw is_reviewer. Track whether the graph loaded and, in the degraded path, revert casts_real_verdicts to raw is_reviewer (excluding the simplifier, the only wake_only role, which stays producer-only via the existing banner dispatch). Extend the existing fallback test to lock in that genuine reviewers keep their Reviewer Lifecycle and the simplifier does not.
|
Thanks for the approval and the careful note on the degraded fallback path. I took it in-PR rather than leaving the latent inconsistency in code this PR modifies. DispositionNon-blocking — the You diagnosed it precisely: I took your first suggested option, refined to avoid even the simplifier label mismatch:
I kept Regression guard (as requested) —
— Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Re-review — approve. The one outstanding non-blocking item is now fixed, precisely and with a real regression guard.
Delta since my last review (b44a5ed..e145e62): a single commit, two files (orchestrator/routes/pipelines.py +16, orchestrator/tests/test_pipeline_prompts.py +37). It closes exactly the latent inconsistency I flagged in my prior approval — the casts_real_verdicts gate degrading every reviewer to PARTICIPANT in the except Exception graph-load fallback.
The fix — verified correct
real_producers = [p for p in producers if p not in wake_only_producers]
if graph_available:
casts_real_verdicts = bool(real_producers) # normal path, unchanged
else:
casts_real_verdicts = is_reviewer and role_value != "simplifier"This is the first option I suggested, refined to also keep the role-type label right (a bare is_reviewer would have rendered the simplifier PRODUCER and REVIEWER). I rendered the full role matrix through the real fallback path (get_review_graph_for_phase patched to raise) and confirmed:
| role | phase | role type | Reviewer Lifecycle |
|---|---|---|---|
| simplifier | refine/plan | PRODUCER | absent ✓ |
| tester | implement | PRODUCER and REVIEWER | present ✓ |
| reviewer_code | implement | REVIEWER | present ✓ |
| reviewer_refine | refine | REVIEWER | present ✓ |
| reviewer_plan | plan | REVIEWER | present ✓ |
| coder | implement | PRODUCER | absent ✓ |
- Genuine reviewers regain the pre-#3381 raw-
is_reviewerbehavior (REVIEWER + full Reviewer Lifecycle) in the degraded path. - The simplifier stays producer-only — excluded here and independently rendered producer-only by the
is_dual_role and role_value == "simplifier"banner dispatch, which still fires in the fallback. real_producersstays[]in the fallback, so the "Your assigned producers:" line correctly doesn't render — only the gating discriminator reverts. Matches pre-PR fallback output.
The hard-coded role_value != "simplifier" is sound: the simplifier is the only wake_only role — the only two wake_only=True edges are simplifier→refiner and simplifier→task_planner (review_graph.py:253,304), both with the simplifier as reviewer. The fallback is_reviewer set also lists the simplifier, so the two definitions are coupled in the same function.
Test — exercises the production path, not a golden
test_genuine_reviewer_keeps_lifecycle_in_review_graph_fallback monkeypatches review_graph.get_review_graph_for_phase to raise, forcing the real except branch (the from review_graph import … inside the function re-binds to the patched attribute at call time — I verified the patch actually takes effect). It then asserts reviewer_code/reviewer_refine/reviewer_plan each keep ### Reviewer Lifecycle + Your role type: **REVIEWER**, and the simplifier (refine + plan) has neither ### Reviewer Lifecycle nor the egg-orch consensus nack <role> template. The negative assertion is meaningful — that string only renders inside the gated Reviewer Lifecycle (pipelines.py:13583). No self-seeding golden, no hand-built bypass; name matches assertions. It's placed beside the existing test_dual_role_banner_present_in_review_graph_fallback and follows the same pattern. Both pass locally (2 passed).
Minor non-blocking observation (pre-existing, out of scope, zero net change)
risk_analyst — a genuine dual-role reviewer in the plan graph — is not in the fallback is_reviewer set, so in the degraded path it renders PRODUCER with no Reviewer Lifecycle. This predates the PR (origin/main's fallback set omits it too) and this delta doesn't touch the membership list, so the net change for risk_analyst is zero — I confirmed it. Worth a one-line addition to the fallback set someday for symmetry with the live plan graph, but it only matters under a catastrophic review_graph import failure (where BRC is already broken), so it does not block.
Net: the fix is correct and minimal, the regression guard is real, and no new issues. Approving.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
|
Automated feedback loop has reached the maximum of 5 rounds. Human review is needed to make further progress on this PR. |
risk_analyst is a genuine dual-role reviewer in the plan graph (CRITICAL reviewer of architect + task_planner, #2809) plus producer of the risk register, but it was missing from the degraded-path is_reviewer fallback set. So under a review_graph import failure it rendered PRODUCER-only with no Reviewer Lifecycle — diverging from the live plan graph. Add it to the fallback set (it already appears in the is_producer set), so the degraded path keeps its dual-role rendering. Unlike the simplifier its edges are real verdicts, so casts_real_verdicts correctly stays True. Extends the fallback regression guard to cover risk_analyst. Addresses PR #3382 review observation (pullrequestreview-4597187864).
There was a problem hiding this comment.
Re-review — approve. The delta closes exactly the non-blocking item from my prior approval, correctly and with a real regression guard.
Delta since my last review (e145e62..d6b1cdc): one commit, two files (orchestrator/routes/pipelines.py +9, orchestrator/tests/test_pipeline_prompts.py +14). It adds risk_analyst to the except Exception graph-load fallback is_reviewer set — the symmetry gap I flagged as out-of-scope/non-blocking last time.
The fix — verified correct
risk_analyst is a genuine dual-role node in the live plan graph: review_graph.py:291–294 gives it two CRITICAL review edges (over architect and task_planner) plus an advisory edge where reviewer_plan reviews it. Before this commit the degraded fallback rendered it PRODUCER-only (missing from the is_reviewer set), diverging from the live graph. The commit lists it so the fallback keeps its dual-role rendering.
I confirmed the rendering is right rather than over-claiming:
- The only
wake_only=Trueedges in the entire graph are the simplifier's (simplifier→refiner,simplifier→task_planner,review_graph.py:253,304).risk_analysthas none, so in the fallbackcasts_real_verdicts = is_reviewer and role_value != "simplifier"→True. Role type rendersPRODUCER and REVIEWER (dual role)(pipelines.py:13268–13269) with the Reviewer Lifecycle present (gated at13538). This matches the live plan graph. risk_analystwas already in theis_producerfallback set, so the producer/reviewer fallback definitions are now symmetric — no half-state.
Test — exercises the production path, not a golden
The new assertions sit inside test_genuine_reviewer_keeps_lifecycle_in_review_graph_fallback, which monkeypatches get_review_graph_for_phase to raise, forcing the real except branch. It asserts ### Reviewer Lifecycle is present and the role-type label byte-matches Your role type: **PRODUCER and REVIEWER (dual role)**. No self-seeding golden, no hand-built bypass; the name matches the behaviour asserted. Placed beside the existing fallback regression tests, same pattern.
One pre-existing, non-blocking note (out of scope, zero net change)
The fallback is_reviewer set is phase-agnostic, so risk_analyst now renders dual-role even for a refine/implement fallback, where the live graph would yield PARTICIPANT (it only exists in the plan graph). This is identical to how reviewer_refine/reviewer_plan — phase-scoped in the live graph — are already flattened into the same fallback set, and risk_analyst only spawns in the plan phase in practice. So it's moot and consistent with existing design; it only matters under a catastrophic review_graph import failure (where BRC is already degraded). Not blocking.
Net: correct, minimal, well-guarded, and faithfully addresses the prior observation. No new issues.
— Authored by egg
|
egg review completed. View run logs 16 previous review(s) hidden. |
Fixes #3381.
The simplifier's
*-analysis-human.md/*-plan-human.mdcompanion kept coming out as a review/verification memo (verdict framing,file:linedetail, plan-phase directives) even after #3376/#3377. Per the issue there are two independent failures; this PR fixes each at its root. No automated keyword guard — the reviewer remains the content gate, as requested.Problem A — production side: the doc was the wrong format from the start
Even post-#3376 the simplifier was still structurally a reviewer: both prompt paths (
_build_producer_orientationand the_build_agent_prompt"## Reviewer role" section) required it to emit an ADVISORY ACK/NACK verdict on the upstream draft in the same pass. So the agent entered every invocation wearing a reviewer hat and its output came out review-shaped. #3376 fenced the companion section ("not a review here") but left the role identity intact — fencing can't beat "you are the reviewer of this draft."_build_producer_orientation, and the_build_agent_promptreviewer-role section). The simplifier is now a producer only: the upstream'sCONSENSUS_PROPOSEis purely its cue to read-and-summarize; it issues no ACK/NACK.review_graph.pyas the pure event-pump wake-wire (it's what re-invokes the simplifier on the upstream's propose; removing it reintroduces the"v|"first-propose dedupe deadlock).is_fully_ackedcounts only CRITICAL reviewers, and the Post-BRC event-pump deadlock: confirmed slice never closes (containers won't exit); no self-heal; advance_phase(force=true) trips #2806 + re-entry thrash #3043 confirm-guard exclusion already drops a never-voted advisory edge once the upstream producer confirms — so the simplifier confirms normally without ever casting a verdict.Problem B — gate side: reviewer_refine approved it anyway
The companion criteria #3376 added lived only in
_build_reviewer_preparation— the "while waiting, prepare by…" prep text — not in the rubric the reviewer runs at verdict time._get_refine_review_criteria/_get_plan_review_criteriawere entirely about the refiner/planner draft (sections 1–N) with zero companion criteria, and_get_review_criteria_for_typeis keyed by reviewer type, not by which producer is under review. So whenreviewer_refinerendered its verdict on the simplifier's companion it applied the refiner-analysis rubric, which gave it no grounds to NACK — and it ACKed by default._human_companion_review_criteriachecklist into both verdict rubrics. The reviewer must walk it before ACKing the simplifier — summary-not-review, nofile:line/symbol detail, materially lighter than the parent, readable by a non-engineer, faithful — and NACK the simplifier (not the producer it summarises) on any defect.The two are complementary: A removes the source of the bad format; B is the defense-in-depth gate that should have caught it and didn't.
Tests
## Reviewer rolesection.test_pipeline_prompts.py+test_concurrent_executor.py): 535 pass;ruffclean. Fullmake test/make test-allnot run.