Skip to content
Merged
13 changes: 7 additions & 6 deletions docs/architecture/slice-dag.md
Original file line number Diff line number Diff line change
Expand Up @@ -551,9 +551,10 @@ shape:
after `record_failure`) and the per-slice green
gate (`slice_green_gate.run_slice_green_gate`, #3398, which
spawns a sandboxed one-shot check-runner Job to execute the
repo's configured checks at the integration-branch tip and
blocks PR-open on a red verdict; staged rollout via
`EGG_SLICE_GREEN_GATE`, fail-open on infra errors, including
repo's configured checks at the integration-branch tip; staged
rollout via `EGG_SLICE_GREEN_GATE` — default `on` (a red verdict
withholds the slice PR), `log` runs the checks and logs the
verdict without blocking; fail-open on infra errors, including
infra-signature-tagged reds inside check execution, #3417) — calls
`GatewayClient.create_slice_pr` with `base` resolved from the
slice's DAG parent (root → latest completed chain tip, else the
Expand Down Expand Up @@ -1058,10 +1059,10 @@ on parse failure. The green-gate knobs below are read directly via
| `EGG_ORCH_STACKED_PR_RECONCILER_INTERVAL_SECONDS` | float | 30.0 | Reconciler polling cadence for orphaned child PRs. |
| `EGG_ORCH_CROSS_REPO_MERGE_GATE_MAX_ATTEMPTS` | int | 240 | Poll-attempt budget for the cross-repo merge-sequencing gate (#3393) before a never-merging upstream escalates to a HITL hold; ~2h at the default reconciler cadence. See [Cross-repo merge-sequencing hold](#cross-repo-merge-sequencing-hold-two-tier). |
| `EGG_SLICE_BASE_ANCESTRY_GATE` | str | `on` | Operator kill switch for the admission-time base-ancestry gate (#3541 — see [Root linearization & the base-ancestry gate](#root-linearization--the-base-ancestry-gate-3541)): any of `off`/`0`/`false`/`no` (case-insensitive, whitespace-tolerant) disables the gate; any other value (including unset) leaves it enabled. |
| `EGG_SLICE_GREEN_GATE` | str | `off` | Per-slice green gate rollout switch (#3398): `off` skips the gate entirely; `log` runs the repo's configured checks at the slice tip and logs a red verdict without blocking; `on` blocks slice PR-open on a red verdict. Case-insensitive, with aliases — `on` also accepts `1`/`true`/`yes`, and `log` also accepts `log-only`/`log_only`. Unknown values resolve to `off`. |
| `EGG_SLICE_GREEN_GATE` | str | `on` | Per-slice green gate rollout switch (#3398): `off` skips the gate entirely; `log` runs the repo's configured checks at the slice tip and logs a red verdict without blocking; `on` (the default) withholds the slice PR on a red verdict. Case-insensitive, with aliases — `on` also accepts `1`/`true`/`yes`, `log` also accepts `log-only`/`log_only`, and `off` also accepts `0`/`false`/`no`. Weakening the gate takes an explicit, correctly-spelled `off` or `log`; unset or unrecognised values resolve to `on`, so a typo cannot silently drop a deployment below the product default. Expect the first reds to be gate wiring rather than slice code — a stale contract snapshot reddening contract-hygiene tests (#3301), or `make test`'s changeset narrowing resolving its baseline against `git merge-base` in a fresh worktree — and note either of those reds *every* slice close until fixed (a missing prebuilt-deps snapshot is not in this list: the runner exits non-zero and the gate fails open, costing coverage rather than throughput); the failure message names the branch to fix and quotes `EGG_SLICE_GREEN_GATE=off` as the bypass, and the slice's commits stay on the integration branch (a red gate withholds the PR, it does not discard work). Latency is identical under `log` and `on` — both run the checks and wait for the runner pod, so slice-close latency grows by the check duration (bounded by `EGG_SLICE_GREEN_GATE_TIMEOUT_SECONDS`). The worst case is not a slow suite but a runner pod that never schedules — the wait is `timeout` plus a 120s scheduling grace (~32 min at the defaults) before failing open, so a capacity-starved cluster pays that per slice close. A *partially* delayed pod is quieter and more common: the runner's deadline is the **Job's** `activeDeadlineSeconds`, counted from the Job's `startTime` (before any pod is bound), so time spent Pending or pulling is subtracted from the in-pod check budget rather than added to the wait — the scheduling grace widens only the orchestrator's wait. A pod delayed N seconds gets N fewer seconds to run checks, and a `DeadlineExceeded` kill emits no verdict line, so the gate fails open with no verdict at all. Capacity starvation therefore raises the rate of spurious no-verdict fail-opens as well as dead time ([#3622](https://github.com/jwbron/egg/issues/3622)) — under the `on` default that is a slice close you believed was gated and wasn't, though the direction is always *under*-blocking, never a false red. Grep for *both* fail-open log lines when diagnosing one: the Job controller deletes the active pod on `DeadlineExceeded` rather than leaving it terminal, so the wait normally times out ("runner pod did not reach a terminal state"); a poll that catches the pod reporting `Failed` mid-termination reads partial output and lands on "no parseable verdict from runner" instead. |
| `EGG_SLICE_GREEN_GATE_SKIP_CHECKS` | str (comma-separated) | `security` | Configured check *names* (from `repositories.yaml` `checks`) the gate skips. |
| `EGG_SLICE_GREEN_GATE_TIMEOUT_SECONDS` | int | 1800 | Wall-clock budget for the check-runner pod (spawn-to-terminal); a hung suite degrades to fail-open rather than wedging the slice close. |
| `EGG_SLICE_GREEN_GATE_INFRA_FAIL_OPEN` | str | `on` | Infra-red fail-open (#3417): the runner tags red checks whose full output matches an exact infra signature (the sandbox git wrapper's gateway-down / missing-env / session-auth errors, the kernel's ENOSPC message) or whose process died by SIGKILL; a verdict where *every* red check is infra-tagged fails open instead of blocking, and mixed verdicts block on the genuine reds only. `off`/`0`/`false`/`no` restores strict every-red-blocks behavior; any other value resolves to `on`. |
| `EGG_SLICE_GREEN_GATE_TIMEOUT_SECONDS` | int | 1800 | Wall-clock budget for the check-runner pod (spawn-to-terminal); a hung suite degrades to fail-open rather than wedging the slice close. Enforced as a Job-level `activeDeadlineSeconds` (`timeout + 60`) counted from Job start, so it is a ceiling on scheduling *plus* checks, not on checks alone — see the `EGG_SLICE_GREEN_GATE` row and [#3622](https://github.com/jwbron/egg/issues/3622). |
| `EGG_SLICE_GREEN_GATE_INFRA_FAIL_OPEN` | str | `on` | Infra-red fail-open (#3417): the runner tags red checks whose full output matches an exact infra signature (the sandbox git wrapper's gateway-down / missing-env / session-auth errors, the kernel's ENOSPC message) or whose process died by SIGKILL; a verdict where *every* red check is infra-tagged fails open instead of blocking, and mixed verdicts block on the genuine reds only. `off`/`0`/`false`/`no` restores strict every-red-blocks behavior; `on`/`1`/`true`/`yes` (and unset, and an empty or whitespace-only value) enable it silently; any other value resolves to `on` and logs a warning, since the typo direction here is strict → lenient. |

### Per-pipeline vs. global slice caps

Expand Down
2 changes: 1 addition & 1 deletion docs/development/STRUCTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ orchestrator/
├── slice_scheduler.py # Wave-based scheduler for the implement-phase slice DAG: computes execution waves, caps concurrency, two-tier max_cycles accounting, failure-cascade detection (#2137)
├── stacked_pr_reconciler.py # Stacked-PR rebase reconciler: detects child slice PRs whose base branch was deleted after a parent merge and retargets them via gateway rebase_onto (#2137)
├── cross_repo_merge_gate.py # Cross-repo merge-sequencing gate for multi-repo pipelines: auto-readies (or HITL-holds) a dependent slice's draft PR once its cross-repo upstream PR merges; rides the stacked-PR reconciler cadence (#3393 slice-5)
├── slice_green_gate.py # Per-slice green gate: sandboxed one-shot Job runs the repo's configured checks at the integration-branch tip and blocks PR-open on red; staged rollout via EGG_SLICE_GREEN_GATE (off/log/on), fail-open on infra errors (#3398) and on infra-signature-tagged reds inside check execution (#3417)
├── slice_green_gate.py # Per-slice green gate: sandboxed one-shot Job runs the repo's configured checks at the integration-branch tip; staged rollout via EGG_SLICE_GREEN_GATE (off/log/on, default on — on withholds the slice PR on a red verdict, log runs the checks and logs the verdict without blocking), fail-open on infra errors (#3398) and on infra-signature-tagged reds inside check execution (#3417)
├── action_guards.py # Formal BRC state machine action guards (preconditions for propose/ack/nack/confirm/withdraw)
├── approval_matrix.py # Per-reviewer ACK/NACK matrix for BRC consensus
├── attestation_schemas.py # Attestation payload validation for BRC proposals
Expand Down
15 changes: 8 additions & 7 deletions orchestrator/evidence_gatherer.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,14 @@
# ---------------------------------------------------------------------------
#
# The shared-evidence prefix ships behind the same ``off -> log -> on`` staged
# flag every #3523 behaviour-shift rides, resolved EXACTLY like
# ``slice_green_gate.green_gate_mode()``: an operator typo must degrade to
# "reviewer prompts unchanged", never to "one gatherer silently anchors every
# lens". The resolver lives HERE, with the feature's core, and is imported by
# the S7 wiring (``_criteria.py`` assembly seam, ``consensus_wrapper.py`` log
# recording) — mirroring how ``risk_router`` owns ``ReviewStance`` for the S6
# wiring to consume.
# flag every #3523 behaviour-shift rides, using that shared pattern but keeping
# an ``off``-default (``slice_green_gate.green_gate_mode()`` now defaults to
# ``on`` and degrades unknown to ``on`` — this resolver deliberately does
# not): an operator typo must degrade to "reviewer prompts unchanged", never
# to "one gatherer silently anchors every lens". The resolver lives HERE, with
# the feature's core, and is imported by the S7 wiring (``_criteria.py``
# assembly seam, ``consensus_wrapper.py`` log recording) — mirroring how
# ``risk_router`` owns ``ReviewStance`` for the S6 wiring to consume.

EVIDENCE_PREFIX_ENV_VAR = "EGG_REVIEW_EVIDENCE_PREFIX"
_ENABLED_VALUES = frozenset({"on", "1", "true", "yes"})
Expand Down
31 changes: 19 additions & 12 deletions orchestrator/review_findings_verdict.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,17 @@
on escalation) is the point.

**Staged rollout.** The whole path is gated behind ``EGG_REVIEW_FINDINGS_MODE``,
resolved EXACTLY like ``slice_green_gate.green_gate_mode()`` (``off`` default,
unknown => ``off``, ``log`` records the computed-vs-legacy verdict into the BRC
artifacts without acting, ``on`` uses the computed verdict). Everything in this
module is a pure function of its inputs; it never reads/writes matrix state or
the environment except through :func:`review_findings_mode`. The caller (a
later wiring slice) decides — based on the mode — whether to *act* on the
computed verdict or merely *log* it, which is what keeps ``off``/``log``
outcomes byte-identical to the legacy prose-NACK path.
resolved with the shared staged ``off``/``log``/``on`` pattern but keeping an
``off``-default (unknown => ``off``; note ``slice_green_gate.green_gate_mode()``
now defaults to ``on`` and degrades unknown to ``on`` — this resolver
deliberately does not, so a typo leaves the legacy path authoritative). ``log``
records the computed-vs-legacy verdict into the BRC artifacts without acting;
``on`` uses the computed verdict. Everything in this module is a pure function
of its inputs; it never reads/writes matrix state or the environment except
through :func:`review_findings_mode`. The caller (a later wiring slice) decides
— based on the mode — whether to *act* on the computed verdict or merely *log*
it, which is what keeps ``off``/``log`` outcomes byte-identical to the legacy
prose-NACK path.
"""

from __future__ import annotations
Expand All @@ -60,7 +63,7 @@
VERDICT_NACK = "NACK"


# --- staged-flag resolution (mirrors slice_green_gate.green_gate_mode) --------
# --- staged-flag resolution (shared off/log/on pattern, off-default) ---------

# Operator switch for the findings-computed verdict path. Three-state,
# default off during rollout (#3523 S3): "off"/unset => the legacy
Expand All @@ -77,9 +80,13 @@
def review_findings_mode() -> Literal["off", "log", "on"]:
"""Resolve the operator switch to one of ``off`` / ``log`` / ``on``.

Resolved EXACTLY like ``slice_green_gate.green_gate_mode()``: unknown
values resolve to ``off`` so an operator typo degrades to "legacy path
unchanged", never to "computed verdict silently drives consensus".
Shares the staged ``off``/``log``/``on`` shape but keeps an ``off``-default:
unknown values resolve to ``off`` so an operator typo degrades to "legacy
path unchanged", never to "computed verdict silently drives consensus".
(Unlike ``slice_green_gate.green_gate_mode()``, which now defaults to
``on`` and degrades unknown values to ``on`` + a warning: a wrong
resolution there blocks a slice visibly and recoverably, whereas a wrong
resolution here would silently drive consensus off a computed verdict.)
"""
raw = os.environ.get(FINDINGS_MODE_ENV_VAR, "off").strip().lower()
if raw in _ENABLED_VALUES:
Expand Down
15 changes: 10 additions & 5 deletions orchestrator/review_graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -475,8 +475,10 @@ def register_phase_graph(phase: str, graph: ReviewGraph) -> None:
# (``agent_model_resolution.resolve_agent_model`` imports :func:`risk_router_mode`
# and :func:`resolve_risk_decision` from here).
#
# Everything rides ONE staged flag, ``EGG_RISK_ROUTER``, resolved EXACTLY like
# ``slice_green_gate.green_gate_mode()`` (``off`` default, unknown => ``off``):
# Everything rides ONE staged flag, ``EGG_RISK_ROUTER``, resolved with the
# shared staged ``off``/``log``/``on`` pattern but keeping an ``off``-default
# (unknown => ``off``; ``slice_green_gate.green_gate_mode()`` now defaults to
# ``on`` and degrades unknown to ``on`` — this resolver deliberately does not):
# * ``off`` — inert. The live graph + efforts are byte-identical to legacy.
# * ``log`` — compute the would-be gated graph / tier / effort and record it
# (:func:`risk_route_log_record` + a structured log line), but
Expand All @@ -497,9 +499,12 @@ def register_phase_graph(phase: str, graph: ReviewGraph) -> None:
def risk_router_mode() -> Literal["off", "log", "on"]:
"""Resolve the ``EGG_RISK_ROUTER`` switch to ``off`` / ``log`` / ``on``.

Resolved EXACTLY like ``slice_green_gate.green_gate_mode()``: an unknown
value resolves to ``off`` so an operator typo degrades to "router does
nothing" (full graph, legacy effort), never to "silently review less".
Shares the staged ``off``/``log``/``on`` shape but keeps an ``off``-default:
an unknown value resolves to ``off`` so an operator typo degrades to "router
does nothing" (full graph, legacy effort), never to "silently review less".
(Unlike ``slice_green_gate.green_gate_mode()``, which now defaults to
``on`` and degrades unknown values to ``on`` + a warning: over-verifying
is that switch's safe direction, under-reviewing is never this one's.)
"""
raw = os.environ.get(RISK_ROUTER_ENV_VAR, "off").strip().lower()
if raw in _RISK_ROUTER_ENABLED_VALUES:
Expand Down
12 changes: 8 additions & 4 deletions orchestrator/routes/pipelines/_run_implement.py
Original file line number Diff line number Diff line change
Expand Up @@ -910,14 +910,18 @@ def _run_one_slice_inner(
# #3398 — per-slice green gate: execute the repo's
# configured checks (repositories.yaml, via
# get_repo_checks) against the integration-branch tip
# in a sandboxed one-shot runner, and refuse to open
# the slice PR while any check is red. Closes the
# trust-vs-verify gap in the propose-time
# in a sandboxed one-shot runner and, in "on" mode,
# refuse to open the slice PR while any check is red.
# Closes the trust-vs-verify gap in the propose-time
# checks_passed self-report. Same posture as the
# evidence gate above: fail-open on infra errors,
# fail-closed only on a definitive red verdict;
# EGG_SLICE_GREEN_GATE is the operator switch
# (off during rollout / log / on).
# (off / log / on), defaulting to "on": the checks
# run on every slice close and a definitive red
# withholds the PR. "log" runs them without
# blocking; "off" is the escape hatch, and is
# quoted in the failure message itself.
if pipeline.repo:
try:
import slice_green_gate as _green_gate
Expand Down
Loading
Loading