Default the slice green gate to on - #3609
Conversation
The #3398 green gate shipped with EGG_SLICE_GREEN_GATE defaulting to off, and the variable was never set in k8s/ or config/. Three weeks after the gate landed, its check-runner path had not executed once: a gate nobody runs verifies nothing. Default to log instead. Every slice close now spawns the check-runner and produces a real verdict, which is the soak evidence that on mode needs, while the blocking decision stays opt-in. Disabling now takes an explicit off/0/false/no; unset or unrecognised values resolve to log, so an operator typo can neither start blocking slices nor silently drop a deployment below the product default. The cost is deliberate and documented: log mode still runs the checks and waits for the runner pod, so slice-close latency grows by the check duration (bounded by EGG_SLICE_GREEN_GATE_TIMEOUT_SECONDS, after which the gate fails open). test_kill_switch_off_skips_everything relied on the implicit off default and now sets the switch explicitly; without that it would have silently started exercising the runner.
There was a problem hiding this comment.
No agent-mode design concerns.
This is a configuration-default change in orchestrator infrastructure (green_gate_mode() resolving EGG_SLICE_GREEN_GATE to off/log/on), with matching doc and test updates. It touches none of the agent-mode design surface: no prompt assembly or pre-fetching, no structured-output requirement for human-facing text, no post-processing pipeline, no direct Anthropic API calls or Agent-SDK bypass, and no hardcoded model identifiers.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Approve. This is a clean, well-reasoned default flip. I traced the full path and verified the core claims.
What I verified
green_gate_mode()resolution is exhaustive and correct (orchestrator/slice_green_gate.py:259): unset/whitespace →log, enabled aliases →on, log aliases →log, explicitoff/0/false/no→off, any other value →log+ alogger.warning. No gap.logmode is functional end-to-end, not just structurally present. Atslice_green_gate.py:831a red verdict logslogger.error(... mode=mode), thenif mode == "log": return None(:840) — sologruns the runner pod, emits the soak verdict, and does not block. Onlyonreturns the blocking failure string. Blocking behaviour is unchanged, as stated.- Safety direction improved. The pre-existing hazard was
onn→off(an operator reaching foronsilently gets a gate that does nothing). The new unrecognised→log+warning path is strictly safer and satisfies "operator misconfiguration must produce a signal" better than the old silent-offfallback. - Tests preserve the production path.
test_kill_switch_off_skips_everything(test_slice_green_gate.py:513) now setsoffexplicitly — necessary becausegate_envdeletes the var and the unset case now runs the gate. Every otherTestRunSliceGreenGatecase sets the switch viaenabled_gate(:487,on), so none silently changed behaviour. New parametrized cases cover the realistic typo shapes (tru,onn). - Deployment impact is the intended rollout.
EGG_SLICE_GREEN_GATEis set nowhere ink8s/orconfig/, so all deployments now resolve tolog. The added slice-close latency (bounded byEGG_SLICE_GREEN_GATE_TIMEOUT_SECONDS, fail-open) is documented as deliberate cost.
Non-blocking
Stale cross-references to green_gate_mode() semantics. Three sibling staged-flag resolvers document themselves as resolving "EXACTLY like slice_green_gate.green_gate_mode()" with the parenthetical "(off default, unknown => off)":
orchestrator/review_findings_verdict.py:33and:80orchestrator/review_graph.py:478and:500orchestrator/evidence_gatherer.py:65
After this PR that cross-reference is no longer accurate: green_gate_mode() now defaults to log and degrades unrecognised values to log+warning, whereas these three still (correctly, for their own intent) default to off and silently fall to off. The functions themselves are fine — only the "EXACTLY like green_gate_mode" wording is now misleading to a future reader who follows the reference. Worth a one-line reword in each (e.g. "resolved like the staged off/log/on pattern, but off-default") so the docs don't drift. Not blocking — no behaviour depends on these comments.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
The three sibling staged-flag resolvers (review_findings_mode, risk_router_mode, evidence_prefix_mode) documented themselves as resolving EXACTLY like green_gate_mode() with an (off default, unknown => off) parenthetical. After this PR green_gate_mode() defaults to log and degrades unknown values to log + a warning, so that cross-reference no longer holds. Reword each to describe the shared staged off/log/on pattern while noting these three keep an off-default (unlike green_gate_mode now).
Review feedback addressedBoth reviews approved. The one actionable item was the non-blocking suggestion from the Stale cross-references to The three sibling staged-flag resolvers documented themselves as resolving "EXACTLY like
The functions' own behaviour is unchanged — this was a docs-only reword, as the reviewer noted.
— Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
No agent-mode design concerns.
Delta re-reviewed: 1f149a8a → 3a04103b (one commit, Reword stale green_gate_mode cross-references).
The delta is docs-only — three comment/docstring rewords in orchestrator/evidence_gatherer.py, orchestrator/review_findings_verdict.py, and orchestrator/review_graph.py, addressing the base review bot's non-blocking note that those files claimed to resolve their switches "EXACTLY like slice_green_gate.green_gate_mode()" after this PR changed that function's default. No executable line is touched: review_findings_mode() (review_findings_verdict.py:80) and risk_router_mode() (review_graph.py:499) both still read os.environ.get(..., "off") and fall through to return "off", matching what the new prose now says about them.
Nothing in the delta touches the agent-mode design surface:
- No prompt assembly or pre-fetching.
evidence_gatherer.py's reworded block is the module's staged-flag section header; the shared-evidence prefix assembly it gates is unchanged. - No structured output for human-facing text, no post-processing pipeline, no rigid-procedure prompts.
- No direct Anthropic API calls or Agent-SDK bypass, and no hardcoded model identifiers — the
review_graph.pycomment referencesagent_model_resolution.resolve_agent_modelfor tier/effort routing, but the delta adds no model literal of any kind (EGG200/EGG201 surface untouched).
The previous cycle's only actionable item came from the general review bot and is addressed here; my prior agent-mode review raised nothing to re-check.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Approve (re-review of 1f149a8..3a04103). The delta is documentation-only — I confirmed via gh api repos/jwbron/egg/compare/1f149a8...3a04103 that all 23 changed lines across the three files are inside comment blocks or docstrings. Zero executable lines changed, so the base commit's behaviour (which I verified last round) is untouched.
My prior non-blocking suggestion is addressed for the five sites I named. Four residual items below, all non-blocking, one of them introduced by this delta.
Verified
- The reworded claims are factually accurate. Each sibling resolver still reads
os.environ.get(VAR, "off")and falls through toreturn "off"on unrecognised input, so "keeps anoff-default, unknown =>off" holds:review_findings_mode()(orchestrator/review_findings_verdict.py:80, env read at:89),risk_router_mode()(orchestrator/review_graph.py:499,:508),evidence_prefix_mode()(orchestrator/evidence_gatherer.py:80,:87). green_gate_mode()is unchanged from my last review (orchestrator/slice_green_gate.py:259): unset →_DEFAULT_MODE(log), enabled aliases →on, log aliases →log, explicitoff/0/false/no→off, anything else →logger.warning+log. The "now defaults tologand degrades unknown tolog+ a warning" phrasing in all three reworded sites matches the implementation exactly.ruff checkandruff format --checkare clean on all three changed files.EGG_SLICE_GREEN_GATEis still set nowhere ink8s/orconfig/— the new default is the deployed behaviour, which the PR's premise requires.
Non-blocking
1. The delta introduced an orphaned closing paren — orchestrator/review_findings_verdict.py:32-38
The reword opened a new parenthetical at :34 ((unknown => ...) and closed it at :36 (authoritative).), but left the original sentence's closing paren in place at :38:
deliberately does not, so a typo leaves the legacy path authoritative). ``log``
records the computed-vs-legacy verdict into the BRC
artifacts without acting, ``on`` uses the computed verdict). Everything in this
The paragraph has two ( and three ). The trailing verdict) is orphaned. Line :37 is also left ragged at 51 chars — the edit inserted text without re-wrapping the tail of the paragraph. Suggested:
deliberately does not, so a typo leaves the legacy path authoritative). ``log``
records the computed-vs-legacy verdict into the BRC artifacts without acting;
``on`` uses the computed verdict. Everything in this
Same ragged-wrap artifact, without the paren bug, at orchestrator/evidence_gatherer.py:68 (# not): an operator typo must degrade to, 40 chars).
2. The most load-bearing stale comment was missed — orchestrator/routes/pipelines/_run_implement.py:920
This is the call site of run_slice_green_gate and it still describes the rollout posture this PR is flipping:
# EGG_SLICE_GREEN_GATE is the operator switch
# (off during rollout / log / on).A reader tracing why a slice close suddenly waits on a runner pod lands here first. "off during rollout" is now wrong. Neither commit touched it.
3. Two test-module docstrings carry the same stale claim the delta set out to fix
orchestrator/tests/test_review_findings_verdict.py:19— "review_findings_moderesolves a flag typo tooff(never silently toon), exactly likeslice_green_gate.green_gate_mode". This one is now inverted:green_gate_moderesolves a typo tolog, which is precisely the distinction the delta added to the module docstring 60 lines away.orchestrator/tests/test_risk_router_wiring.py:7— "EGG_RISK_ROUTERstaged flag (offdefault /log/on, resolved exactly likeslice_green_gate.green_gate_mode())". Theoff-default half no longer holds.
4. Optional — orchestrator/review_findings_verdict.py:66
The section header # --- staged-flag resolution (mirrors slice_green_gate.green_gate_mode) --- keeps the "mirrors" framing. Weaker than the "EXACTLY like" wording that was fixed, and the docstring 14 lines below now spells out the difference, so this is defensible as-is. Flagging only for completeness.
None of the above changes behaviour or blocks the rollout. CI is green on everything reported so far (Unit Tests and Integration Tests still pending at review time).
— Authored by egg
This comment has been minimized.
This comment has been minimized.
…in docs/architecture/slice-dag.md, docs/development/STRUCTURE.md
Conflict Resolution SummaryMerged
Verification
Please review: the two doc sentences now assert both defaults-and-fail-open behaviours in one breath — worth a read to confirm the combined phrasing still reads correctly, particularly — Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Approve (re-review of 3a04103b..c45d3cfb). No agent-mode design concerns.
The delta since my last review is a single commit — merge c45d3cfb, bringing origin/main through 026196c7 (#3518 / #3417) into the PR branch. There is no new PR-authored work to review beyond the conflict resolution.
What I checked
- Both conflict resolutions are additive doc-line merges, no design surface.
docs/development/STRUCTURE.md:113anddocs/architecture/slice-dag.md:553-559each had this PR's "defaultlog" clause and main's #3417 infra-red fail-open clause landing in the same sentence; both resolutions keep both facts. Theslice-dag.mdenv-var table row forEGG_SLICE_GREEN_GATEcorrectly flips the documented defaultoff→logand now spells out the latency cost. Documentation only — nothing that shapes agent context or behaviour. - The auto-merged Python is clean of the anti-patterns in scope.
orchestrator/slice_green_gate.pycombines this PR'sgreen_gate_mode()(:374, unset →_DEFAULT_MODE) with main's_infra_fail_open_enabled()with no interference. Grepping the merged module and the full PR diff vs the main-side parent for hardcoded model IDs (claude-<tier>-<date>),anthropic/httpx/requests.postAPI calls, andmodel=literals returns zero hits — so items 6, 7, and 8 of the criteria are REFUTED, not merely unobserved. - The gate's execution model still delegates correctly. The green gate runs the repo's configured checks by spawning a sandboxed one-shot check-runner Job at the integration-branch tip — orchestrator-side code spawning a sandbox container, which is the pattern the guidelines ask for, not orchestrator-side work that should have been delegated. The merge did not change this.
- No prompt-assembly, structured-output, or post-processing surface is touched anywhere in the delta. The switch resolution is operator configuration read from the environment, not agent-facing context.
Note
The general review bot's four non-blocking items from the previous cycle (the orphaned closing paren at orchestrator/review_findings_verdict.py:32-38, the stale call-site comment at orchestrator/routes/pipelines/_run_implement.py:920, and the two stale test-module docstrings) are documentation-accuracy items outside this lens — I am deliberately not re-raising them here, and they remain non-blocking regardless. Flagging only so their absence from this review isn't read as them having been fixed by the merge; they were not touched by it.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Approve (re-review of 3a04103..c45d3cf). The delta is the conflict-resolution merge of origin/main (through 026196c, bringing #3518/#3417) into the PR branch. I reviewed the merge rather than trusting the summary: both hand-resolved doc lines and the slice_green_gate.py / test_slice_green_gate.py auto-merges are correct, and the semantic interaction between #3417's infra fail-open and this PR's new default is sound. No blocking issues.
The author asked for a specific read on the combined doc phrasing — answered under Answering the merge question below.
What I verified in the merge
The auto-merge of slice_green_gate.py is semantically correct, not just textually clean. The two sides edit the same function's neighbourhood, so I traced the merged control flow in run_slice_green_gate:
mode = green_gate_mode()(:781) →offshort-circuits (:782).- #3417's infra classification runs before the mode branch:
genuine_failedis computed at:968-970, the all-infra fail-open returnsNoneat:988-995, and only then does:996reachlogger.error("Green gate red…", mode=mode)and:1002if mode == "log": return None.
That ordering is what you want. In the new default log mode, an all-infra-red verdict is logged as the #3417 warning and never counted as a red in the soak signal — an infra fault does not pollute the very evidence this PR exists to collect. Blocking behaviour is unchanged: only on reaches the return at :1003.
No test was lost by the merge. I diffed the test symbol sets across all three sides. Relative to 026196c, exactly two names are gone (test_default_is_off, test_everything_else_is_off) and three are new (test_default_is_log, TestGreenGateMode.test_disabled_values, test_unrecognised_values_degrade_to_default). Every PR-side test survives; every #3417 test survives. Coverage strictly widened — the old parametrization mixed off/0/false with ""/banana/enabled in one all-resolve-to-off case; the split now pins the two classes separately and adds the realistic typo shapes (tru, onn) plus whitespace (" off ", " ").
Main's #3417 tests run under the changed default without silently changing meaning. Every TestRunSliceGreenGate case main added (test_all_infra_reds_fail_open, test_mixed_reds_block_on_genuine_only, test_infra_fail_open_switch_off_blocks_on_infra_reds, test_pre_3417_verdict_without_infra_field_blocks) takes the enabled_gate fixture, which sets the switch to on explicitly — none of them rode the implicit off default, so none of them changed behaviour under the merge. This was the real hazard of merging #3417 into a default flip and it is clean.
Tests pass on merged HEAD (targeted, not the full suite):
orchestrator/tests/test_slice_green_gate.py— 92 passedorchestrator/tests/test_run_implement_slice_closed.py+test_slice_run_loop_integration.py— 59 passed
The PR's premise still holds post-merge. grep -rn EGG_SLICE_GREEN_GATE k8s/ config/ bin/ scripts/ returns nothing — the new default is the deployed behaviour. Main's docs/guides/deployment.md addition (bin/egg-init pointer) does not touch the green gate, so it introduced no competing default claim.
green_gate_mode() itself is byte-identical to what I approved last round (:374-398). Unset/whitespace → _DEFAULT_MODE, enabled aliases → on, log aliases → log, explicit off/0/false/no → off, anything else → logger.warning + log.
The two green_gate_mode cross-references main added inside this module are still accurate. slice_green_gate.py:153 ("Any other value degrades to the default, matching green_gate_mode's typo posture") and :405 ("Mirrors green_gate_mode's posture: an operator typo resolves to the documented default behavior") were written against the old resolver. Both survive the flip — the claim is "degrades to the documented default", which holds before and after. This is the same stale-cross-reference class 3a04103 set out to fix, so I checked it specifically; no reword needed. (One caveat under item 5 below.)
Answering the merge question
the two doc sentences now assert both defaults-and-fail-open behaviours in one breath — worth a read to confirm the combined phrasing still reads correctly, particularly
slice-dag.md
Parens balance and every clause is factually true. slice-dag.md:552-558 opens ( at "gate (", nests (observe, never block), and closes at #3417). #3417 correctly scopes to the fail-open clause it trails, and #3398 to the gate itself.
The residual problem is not the parens, it is the adjacency:
blocks PR-open on a red verdict; staged rollout via
`EGG_SLICE_GREEN_GATE`, default `log` (observe, never block),
"blocks PR-open on a red verdict" and "never block" sit one clause apart and read as a contradiction on first pass. It resolves if you know log is a mode name, but this is the module-index line — the entry point for a reader who does not yet know that. STRUCTURE.md:113 has the same shape (blocks PR-open on red; … (off/log/on, default log)), milder because it's a single index line. Suggested for slice-dag.md:
gates PR-open on a red verdict at the integration-branch tip;
staged rollout via `EGG_SLICE_GREEN_GATE` — default `log`
(runs the checks, logs the verdict, never blocks), `on` blocks;
fail-open on infra errors, including infra-signature-tagged
reds inside check execution, #3417) — calls
Also cosmetic: :557 is left ragged at 40 chars — the conflict resolution inserted text without re-wrapping the paragraph tail.
Non-blocking
Items 1–4 are carried forward unchanged from my 3a04103 review. The only new commit is the conflict-resolution merge, so none were addressed — expected, not a criticism. Restating so they are not lost, plus one new item from the merge.
1. Orphaned closing paren — orchestrator/review_findings_verdict.py:32-38
Still present. The 3a04103 reword opened a parenthetical at :34 and closed it at :36, but left the original sentence's closer at :38. Two (, three ):
deliberately does not, so a typo leaves the legacy path authoritative). ``log``
records the computed-vs-legacy verdict into the BRC
artifacts without acting, ``on`` uses the computed verdict). Everything in this
:37 is also ragged at 51 chars. Same ragged-wrap artifact without the paren bug at orchestrator/evidence_gatherer.py:68 (# not): an operator typo must degrade to, 40 chars).
2. The most load-bearing stale comment is still missed — orchestrator/routes/pipelines/_run_implement.py:919-920
# EGG_SLICE_GREEN_GATE is the operator switch
# (off during rollout / log / on).This is the call site of run_slice_green_gate (:927). An operator asking "why does slice close suddenly wait on a runner pod" lands here first, and it still describes the posture this PR is flipping. Neither the original commit nor the merge touched it. Of everything in this list, this is the one I would actually fix before merge.
3. Two test-module docstrings carry the stale claim 3a04103 set out to fix
orchestrator/tests/test_review_findings_verdict.py:18-19— "review_findings_moderesolves a flag typo tooff(never silently toon), exactly likeslice_green_gate.green_gate_mode". Now inverted:green_gate_moderesolves a typo tolog. This is precisely the distinction3a04103spelled out in the module docstring 60 lines away, so the two files now disagree with each other.orchestrator/tests/test_risk_router_wiring.py:6-7— "(offdefault /log/on, resolved exactly likeslice_green_gate.green_gate_mode())". Theoff-default half no longer holds.
4. Optional — orchestrator/review_findings_verdict.py:66
# --- staged-flag resolution (mirrors slice_green_gate.green_gate_mode) --- keeps the "mirrors" framing. Weaker than the "EXACTLY like" wording that was fixed, and the docstring 14 lines below spells out the difference. Flagging for completeness only.
5. New from the merge — _infra_fail_open_enabled() has no warning on an unrecognised value
orchestrator/slice_green_gate.py:401-410:
raw = os.environ.get(GREEN_GATE_INFRA_FAIL_OPEN_ENV_VAR, "on").strip().lower()
return raw not in _INFRA_FAIL_OPEN_DISABLED_VALUESAn operator who sets EGG_SLICE_GREEN_GATE_INFRA_FAIL_OPEN=offf reaching for strict every-red-blocks silently gets on (lenient) with no log line. green_gate_mode() now emits logger.warning on exactly this input class; this sibling does not, which makes the :153 comment's "matching green_gate_mode's typo posture" true on the resolution but incomplete on the signalling.
Why I am raising code this PR does not modify: before this PR the gate never ran, so _infra_fail_open_enabled() was never called in any deployment. This PR makes it execute on every slice close. The consequence is nil in log mode (nothing blocks either way), which is why this is advisory rather than blocking — but it should be fixed before the on flip, where the typo direction is strict→lenient. A three-line logger.warning mirroring :392-397 closes it.
6. Operational note for the soak — worst case is not "check duration"
The docstring says slice-close latency "grows by the check duration (bounded by EGG_SLICE_GREEN_GATE_TIMEOUT_SECONDS, default 1800s)". Accurate, and the bound is stated. Worth being explicit with whoever watches the soak: the worst case is not a slow suite but an unschedulable runner pod — _wait_for_runner_pod waits timeout + _POD_SCHEDULING_GRACE_SECONDS before failing open, so a capacity-starved cluster costs ~30 min of dead time per slice close, in every deployment, starting the moment this merges. Nothing to change in the code; the sequencing plan already has off as the escape hatch. Just make sure someone is watching the first wave rather than discovering it from slice throughput.
On sequencing
The PR body lists #3518 (#3417) as "effectively a prerequisite for on". That prerequisite is now satisfied in-branch by this merge, not merely pending — _infra_fail_open_enabled and _INFRA_LINE_SIGNATURES are on the branch and their tests pass under the new default. The soak this PR starts will therefore already carry infra-vs-genuine classification, which is the right ordering and makes the verdicts collected genuinely usable as on-flip evidence.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Fixes the six non-blocking items from the c45d3cf review: - review_findings_verdict.py: drop the orphaned closing paren the earlier reword left behind and re-wrap the paragraph tail; reword the section header away from the "mirrors green_gate_mode" framing. - evidence_gatherer.py: re-wrap the ragged tail from the earlier reword. - routes/pipelines/_run_implement.py: the run_slice_green_gate call site no longer says "off during rollout" — it is the first place an operator lands when asking why slice close waits on a runner pod. - test_review_findings_verdict.py / test_risk_router_wiring.py: drop the now-inverted "exactly like green_gate_mode" claims from the module docstrings; both resolvers keep an off-default, green_gate_mode no longer does. - slice_green_gate.py: _infra_fail_open_enabled() now warns on an unrecognised value instead of silently resolving to the lenient default. This PR makes that resolver execute on every slice close for the first time, and its typo direction is strict -> lenient. - slice_green_gate.py / slice-dag.md: record that the worst-case latency is an unschedulable runner pod (timeout + scheduling grace, ~32 min), not a slow suite. Also resolves the merge question on the combined doc phrasing: the slice-dag.md and STRUCTURE.md lines no longer read "blocks PR-open ... never block" one clause apart.
Review feedback addressedBoth reviews approved. All six non-blocking items from the 1. Orphaned closing paren — You were right that the earlier reword opened a parenthetical and closed it without removing the original sentence's closer. Took the suggested fix ( 2. Stale call-site comment — Agreed this was the one that actually mattered — it's the first place an operator lands when asking why slice close suddenly waits on a runner pod, and "off during rollout" was exactly backwards. Now reads 3. Two stale test-module docstrings — Both "exactly like 4. You flagged it as defensible as-is, but it's a one-line change and leaving a weaker version of the wording you just had fixed invites the same drift back. Now 5. Fixed rather than deferred, and your reasoning for raising code this PR doesn't modify is what convinced me: this PR is what makes that resolver execute in any deployment for the first time. Added Tests split to match the 6. Operational note for the soak — worst case is not "check duration" — Documented rather than left as a verbal handoff, since "somebody should be told" tends not to survive. Both the module docstring and the Merge question: the combined doc phrasing — Your read was right — "blocks PR-open on a red verdict" and "never block" sitting one clause apart only resolves if you already know Also updated the Verification — — Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Approve (re-review of c45d3cf..910f6ed). No agent-mode design concerns.
Delta re-reviewed: a single commit, 910f6ed "Address review: green-gate doc accuracy + infra-fail-open typo warning" (9 files, +85/-29). The local clone in this sandbox is depth-1, so I pulled the patch via gh api repos/jwbron/egg/commits/910f6ed... rather than git log -p.
The delta is docs, comments, one operator-log line, and tests:
docs/architecture/slice-dag.md,docs/development/STRUCTURE.md— reword the module-index and env-table rows sologmode's "runs the checks, never blocks" no longer sits one clause from "blocks PR-open"; add the unschedulable-runner-pod worst case.review_findings_verdict.py,evidence_gatherer.py,_run_implement.py, and two test-module docstrings — comment/docstring accuracy after the default flip. Every changed line inevidence_gatherer.pyis a#comment (verified by filtering the patch for non-comment+/-lines: empty).slice_green_gate.py—_infra_fail_open_enabled()gains an explicit three-way resolution pluslogger.warningon an unrecognised value, with_INFRA_FAIL_OPEN_ENABLED_VALUESadded.test_slice_green_gate.py— four resolver tests covering enabled/disabled/typo-warns/recognised-doesn't-warn.
Against the anti-pattern list: nothing here touches prompt assembly, agent context, or output handling. No pre-fetched context is added to any prompt; no structured output is introduced for a human-facing surface; no post-processing script parses agent output; the new logger.warning is operator telemetry, not an agent instruction. I grepped the four changed Python modules for claude-*-<date> literals, anthropic, httpx, and requests. — no hits, so no hardcoded model ID (EGG201) and no direct-API-call (EGG200) surface enters via this delta.
The one design-shaped fact in the delta is the _infra_fail_open_enabled() change, and it moves in the right direction: it replaces a silent lenient fallback with an explicit resolution that logs. That is a sandbox/config-enforced constraint carrying an operator signal, not a prompt-level one — squarely on the right side of the "prompt-level security" guideline.
On the previous round. All six non-blocking items from the c45d3cf general review were addressed in-PR, including the two I would have expected to be deferred (item 5, the infra-fail-open warning in code this PR does not otherwise modify, and item 6, documenting the soak's worst case rather than leaving it as a verbal handoff). My own prior agent-design reviews at 1f149a8, 3a04103, and c45d3cf each found no concerns; nothing in this delta changes that.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Request changes (re-review of c45d3cf..910f6ed). The delta is a single commit, 910f6ed (9 files, +85/−29): docs, comments, one added operator log line, and four tests. Every one of the six non-blocking items from my c45d3cf review is genuinely fixed — I verified each against the current file rather than the commit message, details below. 194 passed on the three touched test files, ruff check + ruff format --check clean on all five touched Python files, and the PR's premise still holds (grep -rn EGG_SLICE_GREEN_GATE k8s/ config/ bin/ scripts/ → nothing, so the new default is the deployed behaviour).
One blocking item, and it is new-in-this-delta as much as pre-existing: the operator guidance this commit adds about the gate's timeout clock describes a mechanism Kubernetes does not implement, and the constant it rests on does not do what the module claims.
Blocking
The deadline clock is misdescribed, and _POD_SCHEDULING_GRACE_SECONDS does not protect the check budget
orchestrator/slice_green_gate.py:221-229 (pre-existing) and the new paragraph at :91-97 / the new env-table sentence at docs/architecture/slice-dag.md:1062.
The comment at :226-229 asserts:
# The pod's own ``activeDeadlineSeconds`` (which the
# kubelet counts from pod *start*, i.e. after scheduling) still caps the
# actual check duration, so a genuinely hung check is killed by the pod
# deadline rather than lingering for the full grace-padded wait.That describes PodSpec.activeDeadlineSeconds. The code does not set that field. :574 puts the deadline at manifest["spec"]["activeDeadlineSeconds"] — a sibling of ttlSecondsAfterFinished / backoffLimit / template, i.e. the Job spec — and :631 passes it to V1JobSpec(active_deadline_seconds=...), not to the pod template. The two fields have different clocks:
JobSpec.activeDeadlineSeconds— "Specifies the duration in seconds relative to the startTime that the job may be continuously active before the system tries to terminate it" — the Job's.status.startTime, set when the controller starts the Job, before any pod is bound.PodSpec.activeDeadlineSeconds— "Optional duration in seconds the pod may be active on the node relative to StartTime" — the field the comment describes, and the one the code never sets.
So scheduling and image-pull latency do count against the deadline, which is precisely what :221-229 says the grace exists to prevent ("on a cold node a long image pull would otherwise eat into the check budget and trip a spurious fail-open timeout even when the checks would have passed"). The grace widens the orchestrator's wait (:946, timeout + _POD_SCHEDULING_GRACE_SECONDS); it does nothing to the in-pod budget, which the Job deadline still cuts by the scheduling delay.
Failure scenario — the capacity-starved cluster the new :91-97 paragraph names as the worst case. Defaults: timeout=1800, Job activeDeadlineSeconds=1860. Job created at t=0; pod Pending 300s for capacity; checks begin t≈300s and are killed by DeadlineExceeded at t=1860s — 1560s of the intended 1800s. restartPolicy: Never + backoffLimit: 0 mean no retry. The terminated pod never prints EGG_GREEN_GATE_VERDICT:, so parse_verdict returns None and :968 logs "Green gate skipped: no parseable verdict from runner" → fail open. The slice close produces no verdict — the artifact this PR exists to create — and the only signal is a warning indistinguishable from a runner-harness crash.
Why this is in scope, on two counts.
- Amplified pre-existing defect. Before this PR
green_gate_mode()defaulted tooffand the variable was set in no deployment, so this path never executed anywhere. This PR puts it on every slice close in every deployment — the defect moves from unreachable to primary-path. - The new text is itself incomplete in the same direction.
:91-97andslice-dag.md:1062are new operator guidance about this exact clock, added so "whoever watches the rollout" knows the cost. They tell the operator an unschedulable pod costs ~32 min of dead time (accurate — I checked: 1800 + 120 = 1920s, and the Job goingFailedat 1860s deletes the Pending pod, so_wait_for_runner_podpolls out the remaining 60s and returnsNone). They do not tell the operator that a partially delayed pod silently shrinks the check budget by the delay and can convert a would-be verdict into a skipped gate. On a capacity-starved cluster that is the more common outcome and the one that degrades the soak signal, which is the whole point of the default flip.
Minimum to unblock — text only, in the file this commit is already rewriting for accuracy:
- Correct
:221-229to name the Job's deadline, counted from JobstartTime, and state that scheduling + pull time counts against it. Drop the "kubelet counts from pod start" parenthetical and the claim that the check duration is protected. - Add to
:91-97andslice-dag.md:1062: a delayed pod reduces the in-pod check budget by the delay, so a capacity-starved cluster raises the rate of spurious no-verdict fail-opens as well as dead time.
Not asking for in this PR: actually moving activeDeadlineSeconds onto spec.template.spec, or deriving the in-pod budget from observed scheduling time. That is a behaviour change and deserves its own issue — but it should land before the on flip, since under on a spurious no-verdict is a gate that silently does not gate. Filing it is fine; shipping the wrong description of it is not.
Verified fixed from the c45d3cf round
Checked against the current files, not the commit message.
- Orphaned paren,
review_findings_verdict.py:32-43— fixed. The parenthetical now opens at:34((unknown => off; …) and closes once at:36(authoritative)); 1(/ 1)in that sentence, and the, ``on`` … )closer is gone in favour of;. Paragraph re-wrapped, no ragged line remains. _run_implement.py:919-922— fixed. Now(off / log / on), defaulting to log: … only "on" blocks PR-open. (One residual, item 1 under non-blocking.)- Two stale test-module docstrings — fixed, and the replacement claims are true:
review_findings_mode(review_findings_verdict.py:89,os.environ.get(…, "off"), unknown →off) andrisk_router_mode(review_graph.py:508, same shape) both keep anoff-default and neither warns, so "unlikegreen_gate_mode" is accurate in both files.evidence_prefix_mode(evidence_gatherer.py:87) likewise. review_findings_verdict.py:66header — fixed, now carries the distinguishing fact instead of pointing at a sibling._infra_fail_open_enabled()warning — fixed, and behaviour-preserving. Old:return raw not in _INFRA_FAIL_OPEN_DISABLED_VALUES. New (:425-435): enabled →True, disabled →False, else warn +True. Every input maps to the same boolean as before; only the log line is new. Coverage strictly widened — the retiredtest_everything_else_is_onvalues (on/1/true/""/banana) are all still asserted acrosstest_enabled_valuesandtest_unrecognised_values_degrade_to_default_with_a_warning, plusON/" 1 "/yes/" "/offf/fals/disabled.gate_env(:67-75) delenvs the var sotest_default_is_onis meaningful, andenabled_gate(:679) never sets it, so noTestRunSliceGreenGatecase picks up an unexpected warning. Ran it: 42 passed on-k "InfraFailOpen or GreenGateMode".- Soak worst-case documented — done, but see blocking above: what is documented is arithmetically right and mechanically incomplete.
Merge question (combined doc phrasing) — resolved. slice-dag.md:551-558 now attributes blocking to the mode that does it; parens balance 2/2 (gate ( … (runs the … blocks) … #3417)), #3417 still scopes to the fail-open clause, and the paragraph is re-wrapped so the ragged :557 is gone. STRUCTURE.md:113 got the same disambiguation.
Non-blocking
_run_implement.py:908-910still carries the pre-flip claim, six lines above the corrected one.# in a sandboxed one-shot runner, and refuse to open / # the slice PR while any check is red.then:921-922# but only "on" blocks PR-open.This is the same adjacency problem you just fixed inslice-dag.md— an unconditional "refuse to open the PR" one clause from "onlyonblocks" — inside a single comment block. Reword:908-910to "…and, inonmode, refuse to open the slice PR while any check is red."evidence_gatherer.py:70— the ragged line moved rather than went away.# the feature's core, and is imported byis 40 chars mid-paragraph; the old offender at:68is now full at 76. Same artifact, one line down. Cosmetic._INFRA_FAIL_OPEN_ENABLED_VALUES(:165) is byte-identical to_ENABLED_VALUES(:138) in the same module. Defensible if you want the two switches free to diverge; worth a one-line comment saying so, otherwise a reader will "fix" the duplication.- The normalization assertion in
test_unrecognised_values_degrade_to_default_with_a_warningdoesn't bite.assert warn.call_args.kwargs["value"] == valuepasses only because every parametrized value is already lowercase and unpadded. The resolver logsvalue=raw(post-.strip().lower()), so adding" Offf "to the parametrize list would pin what is actually logged; today a change to log the raw env value would not fail this test. - The new warning omits a structured resolved-value field.
green_gate_mode's warning carriesmode=_DEFAULT_MODE(:407); the infra one puts the resolution in prose only ("falling back to the default (fail open on all-infra reds)"). Aresolved=True/fail_open=Truekwarg would make the two greppable the same way. slice-dag.md:1065says "on/1/true/yes(and unset) enable it". An explicitly-empty or whitespace-only value also enables it without a warning (:425,if not raw or …). Trivially incomplete.
Sequencing
Unchanged from last round and still right: #3417 is on the branch, so the verdicts this default starts collecting already carry infra-vs-genuine classification. The blocking item above is the one thing I'd want closed before the soak starts, because it determines whether a missing verdict on a busy cluster reads as "runner crashed" or "we under-budgeted the checks" — and the operator reading the docstring today would reach the wrong conclusion.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Three security fields (automountServiceAccountToken, allowPrivilegeEscalation, capabilities.drop) were hardcoded in _submit_runner_job while the manifest dict also declared them. No live bug — the values agreed — but the assertions on them passed against a constant and would have held with the manifest saying the opposite. Add test_every_manifest_field_reaches_the_body, a reflection walk over the manifest and the submitted body via each model's attribute_map. It closes the field-drop class generally rather than one field, with a negative control for #3622's dict-only fix shape. Skip TestSubmitRunnerJob when the real kubernetes SDK is absent — the conftest stub lacks V1PodSecurityContext and attribute_map, so the class errored rather than skipped outside CI.
Review feedback addressedCommit 1.
|
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
No agent-mode design concerns.
Delta re-reviewed: 23068071 → 16e5e0fa (one commit, "Make the green-gate submitter read every field from the manifest"). Two files, +163/−15: orchestrator/slice_green_gate.py (+20/−4) and orchestrator/tests/test_slice_green_gate.py (+143/−11).
Note: 23068071 is not reachable in this worktree (grafted clone — 16e5e0fa has no local parents), so I took the delta from gh api repos/jwbron/egg/compare/23068071...16e5e0fa rather than a local git log range.
What the delta does
_submit_runner_job stops restating three security fields as Python constants and reads them from the manifest dict instead — automountServiceAccountToken (slice_green_gate.py:704), allowPrivilegeEscalation (:721-723), and capabilities.drop (:724-726). The rest is test work: a reflection walk (_assert_manifest_value_reaches) that translates camelCase manifest keys through each SDK model's attribute_map, a skipif guard so TestSubmitRunnerJob skips rather than errors when only the conftest k8s stub is present, and a tightened green_gate_mode parametrization that now asserts the resolved mode alongside the warn count.
Agent-mode surface: nothing touched
- No prompt assembly or pre-fetching. The green gate builds a k8s Job manifest, not agent context. The runner container's command is
["python3", "-c", _RUNNER_PROGRAM](:653), and_RUNNER_PROGRAM(:312) is a deterministic check executor — it readsEGG_GREEN_GATE_CHECKS, shells each check viasubprocess.run(["bash", "-c", ...]), and prints oneEGG_GREEN_GATE_VERDICT:JSON line. No LLM in the loop, so items 1–4 have no surface here: the verdict JSON is a program-to-program protocol between a scripted pod and the orchestrator, not structured output imposed on an agent or a pipeline parsing agent prose. - Item 5 (prompt-level security) — CONFIRMED aligned, and marginally strengthened. This is the one criterion the delta genuinely touches, and it moves the right way. The three constraints stay k8s manifest fields enforced by the API server and kubelet, never instructions:
automountServiceAccountToken: False(:639),allowPrivilegeEscalation: False(:656),capabilities: {"drop": ["ALL"]}(:657) are still hardcoded literals in_build_runner_job_manifest, not env-read or operator-tunable, so nothing about the sandbox posture became configurable. What changed is fidelity: previously the manifest and the submitted body could disagree and no test could tell, since the assertions passed against a constant. Now the dict is the single source of truth andtest_every_manifest_field_reaches_the_bodygoes red on any manifest key the submitter fails to copy — a silently-dropped security field is exactly the failure that guard now catches. - Items 6, 7, 8 — REFUTED, not merely unobserved. Grepping both changed files for
anthropic,httpx,requests.post|get,claude-<tier>-<date>, andmodel=returns zero hits. There is no agent invocation in this path at all, so there is no Agent-SDK bypass to assess: the orchestrator spawns a sandboxed one-shot Job to run the repo's checks, which is the delegation pattern the guidelines ask for rather than orchestrator-side work that should have been delegated.
Prior feedback
My six earlier agent-mode reviews on this PR (1f149a8 through 2306807) all approved with nothing to re-check, so there is no outstanding item from this lens carried into the delta. The general review bot's non-blocking documentation-accuracy items from earlier cycles are outside this lens and I am deliberately not re-raising them — flagging only so their absence here isn't read as this commit having addressed them.
One observation I am explicitly not filing as an agent-mode finding: the new class-level skipif means the security-posture assertions in TestSubmitRunnerJob no longer execute where only the conftest stub is installed, which narrows local coverage of the values discussed under item 5 (CI installs the dev extra, so they still run there). That is test-coverage reasoning and belongs to the base review bot.
— Authored by egg
|
egg agent-mode-design completed. View run logs 24 previous review(s) hidden. |
There was a problem hiding this comment.
Re-reviewed 16e5e0f against 2306807. Delta is one commit, two files: 12 production lines in orchestrator/slice_green_gate.py (the three hardcodes → manifest reads, plus a docstring) and 154 lines of test.
All three items from the 2306807 round are fixed, and I verified the fixes by mutation rather than by reading. No blocking issues. Three non-blocking findings below, all CONFIRMED by running code.
Verification of the previous round
I installed the real kubernetes SDK into this sandbox so TestSubmitRunnerJob would actually run, then re-derived each claim.
1. The enforcing test exists and bites. I built a mutated in-memory copy of slice_green_gate.py with the three hardcodes restored and ran the new parametrize against it:
mutant killed: body.spec.template.spec.automountServiceAccountToken is False, manifest says True
mutant killed: body...containers[0].securityContext.allowPrivilegeEscalation is False, manifest says True
mutant killed: body...containers[0].securityContext.capabilities.drop length 1 != 0
3/3. The assertions discriminate where the old ones passed against a constant.
2. The reflection walk catches dropped keys, including the falsy cases I was worried about. assert got is not None (test_slice_green_gate.py:737) looked like it could miss a dropped field whose manifest value is falsy. It cannot, and the reason is worth writing down: the generated SDK models do self._x = None then if x is not None: self.x = x, so an unset attribute is None regardless of what the manifest said. Seeded three drops against the real submitter:
spec.suspend = True -> body.spec.suspend dropped by _submit_runner_job
spec.suspend = False -> body.spec.suspend dropped by _submit_runner_job
pod.tolerations = [] -> body.spec.template.spec.tolerations dropped by _submit_runner_job
3. _real_k8s_sdk_available is not shadowed by the stub. conftest.py:74-76 guards the stub behind try: import kubernetes / except ImportError, so sys.modules.setdefault never fires when the SDK is present — the setdefault shadowing hazard I looked for isn't there. pyproject.toml:36 puts kubernetes>=31,<33 in the dev extra and test.yml:33 / :54 run uv sync --extra dev, so the class runs in CI rather than skipping. Locally: 117 passed with the SDK, and the 9-test class accounts exactly for the claimed 108/9 split without it.
4. Remaining items. test_recognised_values_do_not_warn asserts the resolved mode again (:184). container.image and volume.host_path.type are covered by the walk. ruff check + ruff format --check clean on both files.
On the production change itself. It is behaviour-preserving: _build_runner_job_manifest declares all three as literals (:634, :651-652), there is exactly one caller of _submit_runner_job (:989) and it always passes builder output, and a KeyError from the new subscripts would land in the fail-open except Exception at :990-999 rather than crashing slice close. Reading the posture from the dict rather than restating it is the right of the two options offered — the manifest builder is where an auditor looks.
Non-blocking
1. The hardened pod-security posture is now pinned only inside a class that can skip — CONFIRMED
grep -n "automount_service_account_token\|allow_privilege_escalation\|capabilities" over the test file returns exactly three assertions on the safe values — :799, :811, :812 — and all three are inside TestSubmitRunnerJob, which now carries @pytest.mark.skipif(not _real_k8s_sdk_available()). TestBuildRunnerJobManifest, which never skips, asserts labels, env, mounts, job shape and runAsUser/runAsGroup/fsGroup — nothing about automount, privilege escalation, or dropped capabilities.
Before this commit those three values were enforced by code: the hardcodes in _submit_runner_job meant no test could lose them. Now they are enforced by a manifest literal plus tests that evaporate if the SDK is ever absent from a runner. CI installs it today, so this is not a live gap — but the failure mode is silent (9 quiet skips), and it is the security posture of a pod that mounts host worktrees.
Four SDK-independent lines restore the property:
def test_hardened_pod_security_posture(self) -> None:
"""The runner pod's privilege floor, pinned without the SDK."""
pod = _manifest()["spec"]["template"]["spec"]
assert pod["automountServiceAccountToken"] is False
ctx = pod["containers"][0]["securityContext"]
assert ctx["allowPrivilegeEscalation"] is False
assert ctx["capabilities"]["drop"] == ["ALL"]in TestBuildRunnerJobManifest. The submitter-side assertions then prove translation; this proves the value.
2. test_a_dict_only_pod_deadline_is_caught fails the moment #3622 lands — CONFIRMED
Simulated the fix (added active_deadline_seconds=pod_spec.get("activeDeadlineSeconds") to the pod spec in a mutated module) and re-ran the control:
post-#3622: walk PASSES -> test_a_dict_only_pod_deadline_is_caught fails with DID NOT RAISE
That is inherent to seeding the control with the very field #3622 will make flow. Its sibling test_pod_level_deadline_is_not_set_today carries an explicit "When #3622 lands, flip this to == <budget>" (:831); this one carries no such note, and it is the harder of the two to read when it goes red — a DID NOT RAISE on a test named ..._is_caught looks like the safety net broke, not like the bug got fixed.
Two ways out; I'd take the second:
- Add the same "expected to fail when #3622 lands — delete it then" line to the docstring.
- Seed the control with a field
_submit_runner_jobwill never copy. I checkedspec.suspendagainst the #3622-fixed submitter and it still bites (body.spec.suspend dropped by _submit_runner_job), so the control stays durable andtest_pod_level_deadline_is_not_set_today's flip note remains the single place #3622 has to touch.
3. The walk cannot see a new hardcode that agrees — worth one sentence — CONFIRMED
_submit_runner_job's new docstring says test_every_manifest_field_reaches_the_body "fails on any manifest key this function does not copy". That is accurate as written. But the class it lives in is easy to over-trust, because the defect this commit fixes — a field restated as a literal that happens to match the dict — is exactly the one the walk is blind to. Replacing restart_policy=pod_spec["restartPolicy"] with restart_policy="Never":
hardcode-that-agrees: UNDETECTED (walk passes)
test_security_fields_follow_the_manifest_not_a_constant closes it for the three known fields by mutating the manifest; a fourth would slip in silently. Not generically fixable short of mutation testing, so this is a docs ask: one sentence in the TestSubmitRunnerJob docstring saying the walk catches dropped keys but not restated ones, and that a new field which must follow the dict needs a row in the security parametrize (or a sibling of it). Otherwise the next author reads "closes that trap for the whole class of fields" and stops there.
Status
CI at review time on 16e5e0f: Python, Docker, Security Scan, Shell, YAML, Custom Checks, Aggregate Lint, Actions all green; Unit Tests and Integration Tests in progress. Per review policy I did not run make test; the runs above were the changed file (117 passed), test_run_implement_slice_closed.py (2 passed), and read-only mutation harnesses in /tmp that exec a modified copy of the module in memory — the working tree was not touched.
The delta does what the last round asked and does it with evidence rather than assertion: the reflection walk genuinely bites, the negative control genuinely distinguishes the dict-only fix, and the three security assertions genuinely discriminate. Item 1 is the only one I'd like to see before merge and it is four lines; items 2 and 3 are comments. Ship it.
— Authored by egg
|
egg review completed. View run logs 24 previous review(s) hidden. |
|
Automated feedback loop has reached the maximum of 5 rounds. Human review is needed to make further progress on this PR. |
The staged rollout in this PR stopped at log on the reasoning that on needed soak evidence first. That reasoning assumed a fleet. egg has one deployment, and on it the argument inverts. Log mode's evidence is passive: a verdict is a structured log line with no metric, audit event, or PR comment behind it (now tracked in #3623). It informs an operator who goes looking, and this switch's own history is that nobody does -- the #3398 default sat at off, unset in every deployment, for three weeks. Under on a wrong verdict announces itself on the next slice close, to the operator who can act on it, which makes on the better instrument for measuring the false-red rate rather than merely the stricter one. Log also would not have prevented #3602, the incident that motivated the flip: it would have logged a line and opened the PR anyway. The blast radius is bounded and self-documenting. A red verdict records a slice failure and withholds the PR; the commits stay on the integration branch, the failure message names the branch to fix, and it quotes EGG_SLICE_GREEN_GATE=off as the bypass. Latency is unchanged -- log and on both run the checks and wait for the runner pod, so the cost was already being paid to collect evidence nobody was reading. Unrecognised values still resolve to the default, now on, so weakening the gate takes an explicit correctly-spelled off or log. The "a typo must not start blocking slices" half of that rule retires with the log default: under an on default, setting nothing at all blocks, so a mistyped value is no stricter than doing nothing. This also aligns the module's two resolvers -- both now degrade unrecognised input to their default, loudly. Docs record what to expect from the first wave: the earliest reds are likelier to be gate wiring (stale contract snapshot per #3301, missing prebuilt-deps snapshot, make test's merge-base narrowing) than slice code, and each of those reds every slice close until fixed. #3622 is re-framed from an on prerequisite to the top follow-up -- its no-verdict fail-open can only under-block, never produce a false red, so it is a coverage gap rather than a correctness risk. test_unset_switch_blocks_on_a_red_verdict is the new seam: every other blocking test sets the switch explicitly, so without it the default could regress to log with the suite still green.
The new operator guidance listed a declared-but-missing prebuilt-deps snapshot alongside #3301 and the make test merge-base baseline as a source of reds that would block every slice close. It is not one: the module documents that case two paragraphs down as an infrastructure failure that fails open, never a false red from missing tools. It costs gate coverage, not slice throughput. Correct the docstring and the slice-dag env table, and name the distinction rather than just removing the item, since "runner exits non-zero" reads like a red until you know which branch it takes.
There was a problem hiding this comment.
No agent-mode design concerns.
Delta re-reviewed: 16e5e0fa → 4d0c6404, two commits ("Default the slice green gate to on, not log", "Drop the prebuilt-deps case from the expected-first-reds list"), 10 files. Note 16e5e0fa is not reachable in this worktree (grafted clone — git log 16e5e0fa..HEAD errors with Invalid revision range), so I took the delta from gh api repos/jwbron/egg/compare/16e5e0fa...4d0c6404 rather than a local range.
What the delta does
_DEFAULT_MODE in orchestrator/slice_green_gate.py:232 flips from "log" to "on", so an unset EGG_SLICE_GREEN_GATE now withholds the slice PR on a definitive red instead of only logging the verdict. Everything else is that flip's blast radius: green_gate_mode's docstring (:459), the module preamble's rationale, the call-site comment in _run_implement.py:917-924, three cross-reference comments in evidence_gatherer.py / review_findings_verdict.py / review_graph.py that cite this resolver's default as the contrast case for their own off-defaults, the EGG_SLICE_GREEN_GATE row in slice-dag.md, and the tests. Commit 2 removes the missing-prebuilt-deps case from the "expect these reds first" list on the grounds that it fails open rather than red — a docs correction, no code change beyond the prose.
Agent-mode surface: nothing touched, and item 5 moves the right way
-
Items 1–4 have no surface here. The gate builds a k8s Job manifest, not agent context. The runner's command is still
["python3", "-c", _RUNNER_PROGRAM]— a deterministic executor that readsEGG_GREEN_GATE_CHECKS, shells each check, and prints oneEGG_GREEN_GATE_VERDICT:JSON line. No LLM in that loop, so the verdict JSON is a program-to-program protocol between a scripted pod and the orchestrator, not structured output imposed on an agent or a pipeline re-parsing agent prose. The delta adds no prompt assembly. -
Item 5 (prompt-level security) — CONFIRMED aligned, and this is the delta's whole point. The comment the flip rewrites names the property directly: the gate "[c]loses the trust-vs-verify gap in the propose-time
checks_passedself-report" (_run_implement.py:915-916). That self-report is an agent attestation — the producer asserting its checks are green. Before this commit the default deployment took that attestation at face value and merely logged the contradicting verdict; after it, the constraint is enforced by orchestrator code (if green_gate_failure is not None: return 1, green_gate_failure,:939-941) against a verdict produced in a sandboxed pod. Replacing an agent's word with infrastructure verification is the direction the guideline asks for, so the default flip strengthens item 5 rather than straining it. #3602 — cited intest_default_is_on's docstring as a task markedcompleteover five failing tests — is exactly the failure an attestation-only posture permits. -
The failure string stays operator-facing and small. Worth checking since
onmakes the blocking path the default:_format_failed_checks(:871-878) truncates each check's output to_FAILURE_MESSAGE_TAIL_CHARS = 1500(:312), and the result is returned as the slice's failure reason to the scheduler — logged and surfaced to the operator, not baked into a re-spawn prompt. Even if it were, 1500 chars per failing check is orienting, not the 10KB-plus pre-fetching item 1 targets. -
Items 6, 7, 8 — REFUTED, not merely unobserved.
grep -rn "claude-[a-z]*-[0-9]\|anthropic\|api\.anthropic"across all six changed Python files returns zero hits. There is no agent invocation anywhere in this path: the orchestrator delegates check execution to a sandboxed one-shot Job, which is the delegation pattern the guidelines want rather than orchestrator-side work that should have been delegated.
Prior feedback
My seven earlier agent-mode reviews on this PR all approved with nothing outstanding, so nothing from this lens carries into the delta. The general review bot's 16e5e0fa round raised three non-blocking items (SDK-independent pinning of the pod-security posture, the #3622 negative-control note, the walk's blindness to a restated hardcode); those are test-design items outside this lens and I am deliberately not re-raising them — flagging only so their absence here isn't read as this delta having addressed them.
One thing I considered and am explicitly not filing: the default flip changes the fleet's failure posture — a false red now stalls a pipeline whose owner may not be watching the rollout. The module preamble argues that trade-off on its own terms (passive log evidence vs. on announcing itself to someone who can act), and the direction is always under-blocking rather than false-red on the fail-open paths. That is an operational rollout judgment for the human reviewer, not an agent-mode design question.
— Authored by egg
|
egg agent-mode-design completed. View run logs 24 previous review(s) hidden. |
There was a problem hiding this comment.
Re-review of 4d0c640 (delta from 16e5e0f)
Two commits since the last round, both about the log → on flip: 6e306d0 moves _DEFAULT_MODE, and 4d0c640 corrects the expected-first-reds list. I traced the on path end-to-end this time, because it is the first revision where run_slice_green_gate can actually return a failure string in a default deployment.
Verification of the delta
4d0c640's claim checks out. A declared-but-missing prebuilt snapshot really is not a false red: slice_green_gate.py:422-428 (sys.exit(1)) fires before the EGG_GREEN_GATE_VERDICT: print at :432, so parse_verdict returns None and the gate takes the "no parseable verdict" fail-open at :1105-1109. Removing it from the throughput list and naming the distinction rather than deleting the line silently is the right call.
The resolver is correct and genuinely pinned. gate_env (test_slice_green_gate.py:73-82) delenvs the switch, so test_default_is_on and test_unset_switch_blocks_on_a_red_verdict exercise the real unset path — not a fixture artifact. test_unset_switch_blocks_on_a_red_verdict is the right seam: the verdict it feeds carries no infra key, so it survives _infra_fail_open_enabled's filter at :1104-1116 and lands on the genuine-red branch. Ran TestGreenGateMode + the new test: 29 passed. Repo Unit Tests are SUCCESS on this head.
The offf parametrize row is the right addition. It pins the resolved value to on rather than merely "not off", which is what makes the direction-of-typo argument load-bearing rather than prose.
Blocking
1. Under on, a red verdict fails a consensus-complete slice with no HITL Decision — the exact gap #3572 closed for the sibling gate 35 lines earlier — CONFIRMED
_run_implement.py:939-941:
if green_gate_failure is not None:
scheduler.record_failure(slice_id)
return 1, green_gate_failureNow compare the evidence gate at _run_implement.py:906-908, three statements above it:
if evidence_failure is not None:
scheduler.record_failure(slice_id)
return 1, evidence_failureMechanically identical — except _slice_close_evidence_gate calls _escalate_evidence_gate_to_hitl before returning (_run_implement_support.py:483-490). Its docstring at _slice_state.py:972-977 states the rationale in terms that transfer verbatim to the green gate:
A consensus-complete slice whose close fails the #3125 gate previously parked silently:
record_failureonly arms the descendant cascade, nothing in the running phase re-drives the close, and recovery required an operator to notice the failed phase and re-run the entire confirmed wave viarestart_phase. Landing an unresolved Decision on the contract makes the block an explicit operator question instead.
The green gate has no such escalation. Failure scenario, on egg's own deployment with nothing set (the PR body confirms grep -rn EGG_SLICE_GREEN_GATE k8s/ config/ is empty):
- Slice-2 reaches consensus. Contract-hygiene tests are red at the integration tip from a stale contract snapshot — the #3301 red this PR's own docs tell operators to expect on the first wave, on every slice close until fixed.
:940-941→record_failure+return 1._run_phase.py:296-305setsphase_execution.status = FAILEDandpipeline.status = FAILED;_run_pipeline.py:601emitspipeline.failedand breaks.- After the 60 s grace,
poll_cascades(slice_scheduler.py:443-470) marks slice-2's whole descendant subtreeBLOCKED_ON_FAILED_DEPENDENCY. - Nothing lands on
contract.decisions. There is nopending_decisionsentry, so the block is not resolvable through/sdlc/provide_input.
Credit where due: the cascade does emit an OVERSEER_ALERT (_run_implement.py:1376-1420), so this is not "no signal at all" — that is more than I initially expected and it is worth saying. But #3572 already litigated whether an alert plus a FAILED phase is sufficient for this exact shape and concluded it is not. The green gate is the same shape, one gate later, and this PR is what makes its branch live.
Two consequences for the PR's own argument:
_run_implement.py:918— inside the comment block this delta rewrites — says "Same posture as the evidence gate above." That is now a false analogy on the axis that matters. The evidence gate's posture includes the Decision; the green gate's does not.slice_green_gate.py:101-103: "Recovery from a wrong red is bounded and self-documenting: the failure message names the branch to fix, the slice restarts, andEGG_SLICE_GREEN_GATE=offis quoted inline as the bypass." Nothing restarts the slice. Per #3572's own account, recovery is an operator noticing a FAILED phase and re-running the entire confirmed wave viarestart_phase— re-driving every agent in that wave, not re-running the checks. On a gate-wiring red (which by construction recurs), the re-run hits the same red.
That last point is what pushes this past advisory. The PR's case for on over log is that log's evidence is passive — "a structured log line, with no metric, audit event, or PR comment behind it… only informs an operator who goes looking." The failure surface on actually ships is a logger.error line plus a generic Container exited with code 1 in pipeline.error. That is the same passive channel the PR rejects as inadequate, at the moment it matters most.
Either fix is fine by me:
- Preferred: mirror
_escalate_evidence_gate_to_hitlwith a[#3398 green-gate]marker. The failure string is already deterministic per incident (integration branch + failed check names), so_escalate_layer_c_hitl's #3427 dedupe/carry-forward guard holds across close retries and phase restarts for free — same property #3572 relied on. - If you'd rather defer the escalation to a follow-up: then the docs have to say what recovery actually is. Replace "the slice restarts" with the
restart_phase-re-runs-the-wave cost, and drop or qualify "Same posture as the evidence gate above" at_run_implement.py:918. I'd still rather have the Decision before the default flips, given the first wave is predicted to be false reds — but a truthful doc is a defensible trade and I won't hold the PR for the stronger option.
Non-blocking
1. _infra_fail_open_enabled's new docstring contradicts itself — CONFIRMED
slice_green_gate.py:490-499:
Only the exact disabled values turn it off; anything else degrades to the default and logs a warning… an operator typo resolves to the documented default behavior, and it never does so silently. […] an operator reaching for
offand typingofffgets the lenient posture — silently, without the warning.
The last clause reads as an assertion and flatly contradicts the first two. The code does warn (:509-517), and test_unrecognised_values_degrade_to_default_with_a_warning pins it. The pre-delta wording carried the counterfactual in one word — "would otherwise get the lenient posture with no signal" — and the rewrite dropped it. A reader landing here to check whether a typo pages them gets the wrong answer.
Suggest: …gets the *lenient* posture — which, without the warning, would be silent.
2. Prior round's three items are still open
None of them are in this delta's blast radius (all test-file, and the automated feedback loop capped at 5 rounds), so this is a restate rather than a re-litigation:
- The hardened pod-security posture (
automountServiceAccountToken,allowPrivilegeEscalation,capabilities.drop) is still asserted only at:808,:820-821— all insideTestSubmitRunnerJob, which carries theskipifat:759.TestBuildRunnerJobManifest(:640) still has no SDK-independent equivalent. Four lines fix it. test_a_dict_only_pod_deadline_is_caught(:859-870) still carries no "expected to fail when #3622 lands" note, unlike its sibling at:831.TestSubmitRunnerJob's docstring still doesn't say the reflection walk catches dropped keys but not restated ones.
Status
CI on this head: Python, Docker, Security Scan, Shell, YAML, Custom Checks, Aggregate Lint, Actions, Unit Tests all SUCCESS; Integration Tests pending. Per policy I did not run make test; the run above was the two mode-resolver classes (29 passed). The remaining reads were git show / grep over _run_implement.py, _run_implement_support.py, _slice_state.py, _run_phase.py, _run_pipeline.py, and slice_scheduler.py — working tree untouched.
The flip's reasoning is sound and I'm not arguing against on as the destination. log's evidence really is passive, the latency argument really is a wash, and #3622 really is a coverage gap rather than a correctness risk. What's missing is that the codebase already decided, in #3572, that this precise failure shape needs a contract-tracked operator question — and the green gate is inheriting the pre-#3572 posture at exactly the moment it stops being dead code.
— Authored by egg
|
egg review completed. View run logs 24 previous review(s) hidden. |
|
Automated feedback loop has reached the maximum of 5 rounds. Human review is needed to make further progress on this PR. |
…ze cap Neither parent violated the 1500-line hard cap — main sat at 1165 lines and the Stage-A autofix branch at 1421 — but their union is 1594. Most of the growth is module-docstring prose added independently on both sides (the rollout rationale for the 'on' default, and the autofix self-heal contract). Decomposition is tracked in #3627; allowlisting keeps that refactor out of a merge commit.
PR #3609 flipped EGG_SLICE_GREEN_GATE's default to `on`, which made the green gate's blocking branch live for the first time. That branch inherited the pre-#3572 posture the sibling evidence gate had already left behind three statements earlier in the close path: a red verdict on a consensus-complete slice calls record_failure and returns, the phase goes FAILED and the descendant subtree cascades, but nothing lands on contract.decisions. The block is not resolvable through /sdlc or provide_input, so recovery means an operator noticing a failed phase and re-running the entire confirmed wave via restart_phase; on a gate-wiring red (a stale contract snapshot reddening contract-hygiene tests, say) that recurs on every close, so the re-run hits the same red. Adds _escalate_green_gate_to_hitl with a [#3398 green-gate] marker, called from a new _slice_close_green_gate helper that mirrors _slice_close_evidence_gate, so the call site in _run_implement.py is now symmetric with the evidence gate on every axis including escalation. The escalation embeds slice_green_gate.failure_headline(failure), not the full failure string. The #3427 dedupe/carry-forward guard matches on question text, and only the failure string's leading block is deterministic per incident (slice id, integration branch, red check names); the blocks after it carry per-check output tails that vary between closes on timings and temp paths. Embedding the whole string would mint a fresh cq-N per close retry and re-ask the operator a question they had already answered. failure_headline names that split explicitly and a negative-control test pins that the tails really do defeat the guard. Also from the review: - _infra_fail_open_enabled's docstring asserted the typo path is silent two sentences after stating it warns. The code warns; restore the counterfactual the rewrite dropped. - The module docstring claimed "the slice restarts" as recovery from a wrong red. Nothing restarted it. Replaced with what actually happens now, in the module docstring and both slice-dag.md sites. - TestBuildRunnerJobManifest gains an SDK-independent assertion on the hardened pod-security posture; those three fields were pinned only inside TestSubmitRunnerJob, which carries a skipif on the real kubernetes SDK. - TestSubmitRunnerJob's docstring now says the reflection walk catches dropped keys but not restated ones, and names the sibling test that covers the other direction. - test_a_dict_only_pod_deadline_is_caught carries the "expected to fail when #3622 lands" note its sibling already had. Testing: test_slice_green_gate.py 131 passed (was 119), test_slice_run_loop_integration.py 58 passed, plus test_evidence_reachability_gate / test_review_findings_verdict / test_risk_router_wiring / test_run_implement_slice_closed: 323 passed across the set. make lint clean. Removing the escalation call fails the two new wiring assertions, so they discriminate.
Why
The per-slice green gate (#3398) is the only thing in egg that executes the repo's configured checks against a slice tip independently of any agent's self-report. It shipped with
EGG_SLICE_GREEN_GATEdefaulting tooff, and the variable is set nowhere ink8s/orconfig/. Three weeks after the gate landed, its check-runner path had not executed once.A gate nobody runs verifies nothing. The default is the rollout.
This came out of triaging #3602 (a contract task marked
completewhile its acceptance condition was unmet). In that incident the slice tip carried five failing tests that proved the task undone; the gate would have gone red on it and refused the slice PR. It never ran.What
green_gate_mode()defaults tooninstead ofoff:on(wasoff)off/0/false/noofflog/log-only/log_onlylogon/1/true/yesonon+ a warning log line (wasoff)Weakening the gate now takes an explicit, correctly-spelled
offorlog. Previously every unrecognised value silently resolved tooff, so an operator reaching foronand typingonngot a gate that did nothing.Why
onand notlogAn earlier revision of this PR stopped at
log, on the reasoning thatonneeded soak evidence first. That reasoning assumed a fleet. egg has one deployment, and on it the argument inverts:log's evidence is passive. A verdict is a structured log line — no metric, no audit event, no PR comment (now tracked in Green gate: verdicts are log lines only — no metric, audit event, or queryable record #3623). It informs an operator who goes looking, and this switch's own history is that nobody does. Underona wrong verdict announces itself on the next slice close, to the operator who can act on it. That makesonthe better instrument for measuring the false-red rate, not merely the stricter one.logwould not have prevented Task completion is self-asserted; acompleterow whose acceptance test fails is undetectable #3602. It would have logged a line and opened the PR anyway. The motivating incident is a shape only a blocking gate prevents.EGG_SLICE_GREEN_GATE=offinline as the bypass. Recovery is a slice restart or one env var, by the operator who is already watching.logstays available and is the right posture for a fleet, where a false red stalls a pipeline whose owner is not the person watching the rollout.What to expect from the first wave
The earliest reds are likelier to be the gate's own wiring than slice code, and each of these reds every slice close until fixed:
workto kill add/add slice-PR conflicts #3301);make test's changeset narrowing resolving its baseline againstgit merge-basein a fresh worktree, so it sees the cumulative slice diff.A declared-but-missing prebuilt-deps snapshot is deliberately not in that list: the runner exits non-zero and the gate fails open, so it costs coverage rather than slice throughput.
Documented in the module docstring and the
slice-dag.mdenv table rather than left to be rediscovered.Cost
Deliberate and documented: the gate runs the checks and waits for the runner pod, so slice-close latency grows by the check duration, bounded by
EGG_SLICE_GREEN_GATE_TIMEOUT_SECONDS(default 1800s) after which it fails open. The worst case is not a slow suite but a runner pod that never schedules (~32 min at the defaults).offremains available for deployments that cannot absorb that.Sequencing
activeDeadlineSeconds) — re-framed from anonprerequisite to the top follow-up. Its no-verdict fail-open can only under-block, never produce a false red, so it is a coverage gap rather than a correctness risk: "the gate is on" and "this close was gated" become different claims on a capacity-starved cluster.Note on the test changes
test_kill_switch_off_skips_everythingasserted the gate short-circuits before any spawn side effect but relied on the implicit off default. It now setsoffexplicitly, which is what it meant all along.test_unset_switch_blocks_on_a_red_verdictis new and is the seam that makes the default load-bearing: every other blocking test sets the switch toonexplicitly, so without it the resolver could regress tologwith the whole suite still green.offf— the disable-side typo, which is the direction that matters now that the default blocks.Testing
orchestrator/tests/test_slice_green_gate.py: 119 passed. Sibling staged-flag suites (test_review_findings_verdict,test_risk_router_wiring,test_run_implement_slice_closed): 95 passed. Repo-wideruff check+ruff format --checkclean.