From dd477281f7779d52fa6682eb9833475bf4e3c106 Mon Sep 17 00:00:00 2001 From: James Wiesebron Date: Thu, 2 Jul 2026 21:47:39 -0700 Subject: [PATCH 1/2] Surface explicit-none decision attestations as confirmable HITL decisions (#3462) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An agent could bypass the entire register -> bridge -> resolve chain by attesting no_decisions_rationale and documenting decisions as draft prose — the attestation rode along as a sentence in the phase_gate question, so the operator never confirmed it as a decision. Gate side: when a refine/plan phase reaches its gate with an explicit-none attestation standing in for a ledger, queue a dedicated confirmable choice decision quoting the role and rationale. Confirm (bare keyword or full option label; anything else conservatively rejects) proceeds to the normal gate with "Operator confirmed the attestation" recorded on the ledger note; reject re-runs the phase with a directive to register each decision — including believed-resolved ones, registered with the recommended answer as the first option. Idempotent across converge rounds (same question -> reuse prior confirmation or pending decision). Autonomous pipelines emit a phase.decision_ledger_explicit_none event instead of blocking, mirroring the #3392 gate-skip posture. Prompt side: task-named decisions are non-optional — a "surface as HITL" directive or operator-owned decision named by the task must be registered even when believed resolved; belief about resolution is a recommended disposition (recommended option citing the resolving context), not a reason to skip. The #2481 skip-already-resolved rule is narrowed to answers from THIS pipeline's pre-refine HITL round. Reviewer side: reviewer_refine (§7) and reviewer_plan (§14) NACK an explicit-none ledger on a task that names decisions to surface. --- docs/hitl-decisions.md | 38 +++ orchestrator/routes/pipelines.py | 317 +++++++++++++++++- .../tests/test_decision_ledger_gate.py | 99 +++++- orchestrator/tests/test_pipeline_prompts.py | 62 ++++ 4 files changed, 493 insertions(+), 23 deletions(-) diff --git a/docs/hitl-decisions.md b/docs/hitl-decisions.md index cd6351f4cc..753671d654 100644 --- a/docs/hitl-decisions.md +++ b/docs/hitl-decisions.md @@ -435,6 +435,44 @@ normal gate. On autonomous pipelines (`hitl_gates: false`) the missing ledger is surfaced as a loud `phase.decision_ledger_missing` event but never blocks, mirroring the autonomous gate-skip posture. +### Explicit-none attestations are confirmed, not trusted (#3462) + +An explicit-none attestation is itself a judgment call about what *is* a +judgment call — exactly the class of decision the HITL contract assigns to +the operator, and the one escape hatch through which an agent under +convergence pressure can bypass the whole register → bridge → resolve chain. +So the gate does not fold it into the `phase_gate` question as prose: when a +refine/plan phase reaches its gate with an explicit-none attestation standing +in for a ledger, the orchestrator first surfaces a dedicated **confirmable +`choice` decision** quoting the role and rationale ("the <role> attests +this phase deliberately raises no operator decisions — confirm?"). + +- **Confirm** (the bare keyword or the full option label — anything else is + conservatively treated as a rejection, mirroring the phase_gate's + "bare approve advances" posture) proceeds to the normal phase gate, and the + gate's ledger note records "Operator confirmed the attestation". +- **Re-run** (or any free-text reply, which rides along as an operator note) + kicks the phase back via the converge loop's standard re-run, with a + directive telling producers to register each decision — including ones they + believe prior context already resolves, registered with the recommended + answer as the first option rather than attested away. + +The confirmation is idempotent across converge rounds: a re-entered gate with +the *same* attestation reuses the operator's prior confirmation (or a pending +confirmation decision) instead of re-asking; a changed rationale is a new +claim and is asked again. On autonomous pipelines the unconfirmed attestation +is surfaced as a `phase.decision_ledger_explicit_none` event but never +blocks. + +Prompt-side, the same issue closes the loophole at the source: decisions the +task description names as operator-owned (or covered by any +"surface as HITL" directive) **must be registered** even when the producer +believes they are already resolved, non-blocking, or deferred — belief about +resolution is a *recommended disposition* (recommended option citing the +resolving context), never a reason to skip registration — and +`reviewer_refine` (§7) / `reviewer_plan` (§14) NACK an explicit-none ledger +on a task that names decisions to surface. + ### Judgment: reviewer obligation against un-surfaced decisions A draft that quietly **commits** to an operator-grade choice ("we will drop diff --git a/orchestrator/routes/pipelines.py b/orchestrator/routes/pipelines.py index 93fc058b16..88b1453044 100644 --- a/orchestrator/routes/pipelines.py +++ b/orchestrator/routes/pipelines.py @@ -5856,7 +5856,16 @@ def _get_refine_review_criteria() -> str: "- If the ledger is deliberately empty (the producer attested " "`no_decisions_rationale`), verify the rationale holds: requirements " "genuinely unambiguous, no assumptions made silently. NACK if you " - "find a hidden operator-grade choice.\n\n" + "find a hidden operator-grade choice.\n" + "- **Task-named decisions — NACK an explicit-none ledger (#3462).** " + "If the task description names decisions as the operator's to make " + "(or directs that decisions be surfaced as HITL questions), each " + "must have a registered `cq-N` — even when the draft argues prior " + 'context already resolves it. "Already resolved" is a recommended ' + "disposition to register (recommended option citing the resolving " + "context), not a reason to skip; a `no_decisions_rationale` " + "attestation on such a task is a **NACK** regardless of how " + "defensible the rationale reads.\n\n" + _human_companion_review_criteria( companion="`*-analysis-human.md`", parent="the refine analysis", @@ -6291,7 +6300,16 @@ def _get_plan_review_criteria() -> str: "boundaries, external commitments, user-visible behavior).\n" "- A deliberately empty ledger arrives as a producer's " "`no_decisions_rationale` attestation — verify it holds; NACK if " - "the plan hides an operator-grade choice.\n\n" + "the plan hides an operator-grade choice.\n" + "- **Task-named decisions — NACK an explicit-none ledger (#3462).** " + "If the task description or refine analysis names decisions as the " + "operator's to make (or directs that decisions be surfaced as HITL " + "questions), each must have a registered `cq-N` — even when the " + 'plan argues prior context already resolves it. "Already ' + 'resolved" is a recommended disposition to register (recommended ' + "option citing the resolving context), not a reason to skip; a " + "`no_decisions_rationale` attestation on such a task is a **NACK** " + "regardless of how defensible the rationale reads.\n\n" + _human_companion_review_criteria( companion="`*-plan-human.md`", parent="the implementation plan", @@ -14141,7 +14159,26 @@ def _build_phase_prompt( "anything, list those items in a `### Resolved in Pre-Refine` " "subsection at the top of `## Open Questions` (one bullet per resolved " "item, citing the answer). Only register questions that go beyond what " - "`## Additional Context` covers.\n", + "`## Additional Context` covers. This skip rule is NARROW: it covers " + "only answers THIS pipeline's operator recorded in " + "`## Additional Context`. It never covers decisions the task " + "description names as operator-owned, and never answers inherited " + "from a prior or cancelled run's seeded context — register those " + "(see the next rule).\n", + "**Task-named decisions are non-optional (#3462).** If the task " + "description or contract names specific decisions as the operator's " + "to make — or contains any directive to surface decisions as HITL " + "questions — you MUST register each one via " + "`egg-contract add-decision`, even when you believe prior context " + "already resolves it, or that it is non-blocking or deferred. " + "Belief about resolution is a *recommended disposition*, not a " + "reason to skip registration: make your recommended answer the " + "first option (suffix its label with `(recommended)`) and cite the " + "resolving context in that option's description, so the operator " + "can confirm in one click while retaining the authority to choose " + "differently. Documenting a decision in draft prose is a " + "supplement to registration, never a substitute — unregistered " + "decisions never reach the operator's decision surface.\n", "Surface uncertainties, ambiguities, and assumptions **that genuinely " "need a human to answer**. Filter ruthlessly: a good open question is " "one the operator must answer because the answer changes what we're " @@ -14210,6 +14247,13 @@ def _build_phase_prompt( "`` instead of the contract CLI", "- Skip registration because you think the questions are minor — " "register every question", + "- Skip registration because you believe a decision is already " + "resolved, non-blocking, or deferred — register it with your " + "recommended disposition instead (#3462)", + "- Attest `no_decisions_rationale` when the task names decisions " + "to surface — the attestation is presented to the operator as its " + "own confirmable decision, and a rejected 'none' sends the phase " + "back for a re-run (#3462)", "- Transcribe this `## How to Populate Open Questions` section " "into your analysis document — it is meta-guidance, not template " "content\n", @@ -14226,7 +14270,13 @@ def _build_phase_prompt( "request (no `cq-N` decisions), attest the rationale form and " "name the feedback request in it. This is what lets the " "operator trust that an empty gate means *deliberately no " - "decisions*, not *forgot to register*.\n", + "decisions*, not *forgot to register*. The explicit-none form " + "is not a shortcut (#3462): the orchestrator surfaces it to " + "the operator as its own confirmable decision before the " + "phase gate, and it is only valid when the phase genuinely " + "raises no meaningful decision — never when the task names " + "decisions to surface, and never as a substitute for " + "registering a decision you believe is already resolved.\n", "", ] ) @@ -14972,7 +15022,13 @@ def _build_brc_preamble( "`cq-N` (copying the `--format markdown` output into the " "draft satisfies this). A decision your draft commits to " "without a registered `cq-N` is a reviewer NACK — register " - "it or remove the unilateral commitment." + "it or remove the unilateral commitment. The rationale form " + "is not a shortcut (#3462): the operator is asked to confirm " + "it as its own decision before the phase gate, and a rejected " + "'none' re-runs the phase. If the task names decisions to " + "surface — or you believe a decision is already resolved by " + "prior context — register it with your recommended answer as " + "the first option instead of attesting none." ) if phase == "implement": propose_line += ( @@ -24750,6 +24806,64 @@ def _sync_pipeline_decisions_to_contract( _LEDGER_BACKSTOP_RERUN_OPTION = "Re-run phase to register decisions" _LEDGER_BACKSTOP_PROCEED_OPTION = "Proceed without a decision ledger" +# Explicit-none attestation confirmation option (#3462). Paired with +# ``_LEDGER_BACKSTOP_RERUN_OPTION`` on the confirmation decision; only a +# resolution that IS a confirmation (the bare keyword or the full label) +# proceeds — any other text is treated as a re-run directive, mirroring +# the phase_gate's "bare approve advances, notes request changes" posture. +_LEDGER_ATTESTATION_CONFIRM_OPTION = "Confirm — no open decisions this phase" + + +def _ledger_attestation_question(role: str, rationale: str, phase_value: str) -> str: + """Compose the explicit-none confirmation question (#3462). + + A producer's claim that a phase raises no operator decisions is itself + a judgment call about what *is* a judgment call — exactly the class of + decision the HITL contract assigns to the operator. It therefore + surfaces as its own confirmable decision, not a sentence embedded in + the phase_gate question. + """ + return ( + f"The {role} attests the {phase_value} phase deliberately raises " + f"no operator decisions (#3462):\n\n" + f"> {rationale}\n\n" + f"Confirm to proceed to the phase gate, or choose " + f"“{_LEDGER_BACKSTOP_RERUN_OPTION}” to send the phase back so its " + f"agents register the decisions as first-class contract entries " + f"(cq-N). Any free-text reply is treated as a re-run directive and " + f"forwarded to the agents." + ) + + +def _unwrap_choice_resolution(resolution: str) -> str: + """Unwrap the ``{"action":"select","selected":