Repository navigation
fix(poll): report a cancelled gating check as unverified, not green - #1323
Conversation
A check run that is COMPLETED with conclusion CANCELLED or STALE landed in neither the pending nor the failed bucket, so `_settle` saw nothing pending and `poll` printed the green verdict. `reduce_rollup` now returns a third `unverified` bucket for those conclusions: `poll` names them and exits 2, and `approval` keeps approving over them but says which check never ran. Closes #1322
tend-agent
left a comment
There was a problem hiding this comment.
reduce_rollup still derives green by subtraction: unverified is an allowlist of CANCELLED/STALE, so a COMPLETED check whose conclusion is outside both sets — one GitHub adds to CheckConclusionState later, or a null conclusion that reaches the reducer as "" — lands in neither bucket, and poll prints green: every gating check … settled green over it. That is the same false green this PR fixes, one conclusion value away. test_terminal_non_success_conclusions_count_red's own docstring already names the shape: "left out of both buckets it would read as green".
Naming the passing set instead is the same amount of code and closes the default:
GREEN_CONCLUSIONS = {"SUCCESS", "NEUTRAL", "SKIPPED"}
...
and context["conclusion"] not in (RED_CONCLUSIONS | GREEN_CONCLUSIONS)Status contexts route exactly as before under it — SUCCESS green, ERROR/FAILURE red, PENDING/EXPECTED never COMPLETED — so only the unrecognized case moves.
I'm pushing that to this branch, along with a test for the description's claim that SUCCESS, NEUTRAL and SKIPPED stay non-blocking (nothing covered NEUTRAL or SKIPPED), and a widening of "cancelled before concluding" to "settled without a result" now that the bucket holds more than cancellations.
Nothing else in the change needs one: supersession, the red set, and the approval policy of approving over a check that never ran all read correctly, and the four new tests fail on main.
…don't Bucketing CANCELLED and STALE explicitly left green as the remainder, so a COMPLETED check carrying a conclusion outside both sets — one GitHub adds to CheckConclusionState later, or none at all, which reaches the reducer as "" — still read as having passed. Green is now the allowlist and the remainder is unverified, which keeps the unrecognized case fail-closed. Status contexts route unchanged: SUCCESS green, ERROR/FAILURE red, PENDING/EXPECTED never COMPLETED.
tend-agent
left a comment
There was a problem hiding this comment.
Two statements of the premise this change removes survive elsewhere in the files it touches, both outside the diff hunks.
references/approving.md — the After the approval bullet beginning A check was cancelled still says "The replacement run will cover the cancelled checks." Under the new reducer that is false for exactly the checks the poll now surfaces: a cancellation with a rerun at the same SHA is superseded in reduce_rollup and never reaches unverified (which test_cancelled_replaced_at_the_same_sha_is_green pins), so a cancelled check that makes the poll exit 2 is one nothing replaced. The bullet tells that session to do nothing, on grounds the Before the APPROVE bullet twenty lines above now contradicts — and it is the only instruction the post-approval poll's reader has for this outcome. The do-not-rerun rule in it still holds and should stay; what has to go is the coverage claim, replaced by naming the check as unverified.
test_terminal_non_success_conclusions_count_red's docstring — "left out of both buckets it would read as green" — now describes the opposite of what the file asserts two tests down, where test_a_conclusion_in_neither_set_is_not_green requires exit 2 for a conclusion in neither set. The reason STARTUP_FAILURE and ACTION_REQUIRED must stay in RED_CONCLUSIONS is now that otherwise they read as unverified rather than red — a different wrong verdict, not a green one.
Neither line falls inside a diff hunk, so no inline suggestion; I'm pushing both to this branch.
approving.md's post-approval bullet told the session a replacement run covers a cancelled check. A cancellation a rerun replaced at the same SHA is superseded in reduce_rollup and never reaches the unverified bucket, so a cancelled check the poll now names is one nothing covered; the bullet now says to report it as unverified and keeps the do-not-rerun rule. test_terminal_non_success_conclusions_count_red's docstring said a conclusion outside both buckets would read as green, which the same file's test_a_conclusion_in_neither_set_is_not_green now contradicts.
Version bump, lockfile, and the `## 0.2.12` section the release workflow publishes verbatim as the GitHub Release notes. 17 commits since 0.2.11. The headline is the sandbox layout: the agent runs in the job's own checkout and home through a copy-on-write view, so what a consumer's `setup:` steps prepare reaches the agent without being rebuilt in `sandbox_setup:`, and the job environment crosses into the sandbox in a `0600` file rather than on `sudo`'s command line ([#1333](#1333), [#1334](#1334)). Alongside it: `/tmp` is writable inside the sandbox as scratch private to the run ([#1324](#1324)), CI sessions ignore the bot's claude.ai skills, plugins and connectors ([#1328](#1328)), the SRT launch retries a transient bridge-socket failure ([#1327](#1327)), and a cancelled gating check polls as unverified rather than green ([#1323](#1323)). Each bullet was written from the commit's diff rather than its subject. No real agent session has run through the new view yet (CI's `test-sandbox` lifecycle runs stub `claude`/`codex` binaries), so the release's live check after tagging is the first one that does. > _This was written by Claude Code on behalf of max-sixty_ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Regenerates the nine `tend-*` workflows against tend 0.2.13 (from 0.2.11). The committed diff is only the pinned action ref and the `uv` setup version (`0.12.13` → `0.12.17`); everything below rides in the action and the plugin it loads. ## What changes for this repo **A cancelled gating check no longer reads as green.** `poll_pr_checks.py` now treats only `SUCCESS`, `NEUTRAL`, and `SKIPPED` as green and exits 2 — unverified — on anything else ([max-sixty/tend#1323](max-sixty/tend#1323)). Cancelled runs are routine here: main serializes the complete suite, so a newer commit replaces the one waiting for its turn, and `running-tend` already carries a rule for reading that shape. Under 0.2.11 a session polling its own push could report such a check as verified; it now has to say it could not verify. **CI sessions ignore the bot's claude.ai account.** Skills, plugins, and MCP connectors enabled on the `leaf-agent` account no longer load into sessions that push and post as it ([max-sixty/tend#1328](max-sixty/tend#1328)). Sessions run with this repo's skills and tend's own, and nothing else. **`/tmp` is writable inside the sandbox**, as scratch private to the run ([max-sixty/tend#1324](max-sixty/tend#1324)). A tool that hard-codes a `/tmp` path no longer needs an override; `$TMPDIR` stays the documented place to write. **The sandbox launch retries a transient bridge-socket failure**, up to three attempts ([max-sixty/tend#1327](max-sixty/tend#1327)). SRT gives socat 600 ms to bind, and a slow runner previously lost the whole job before the agent started. Three plugin-side fixes also land: `review-runs` can now cancel a run parked in GitHub's `waiting` state that holds its concurrency group without ever concluding ([#1320](max-sixty/tend#1320)); `ground-claims` reads a past run's code at that run's commit rather than at the current default branch ([#1319](max-sixty/tend#1319)); and `run-tend` corrects its list of instruction paths restored to the base version on a PR ([#1311](max-sixty/tend#1311)). ## No configuration change needed 0.2.12 moved the agent into the job's own checkout through a copy-on-write view, which would have changed where dependency setup belongs. 0.2.13 reverts it ([#1339](max-sixty/tend#1339)): `setup:` and `sandbox_setup:` behave as they did at 0.2.11, so `.config/tend.yaml` is unchanged here. This repo pins no `model:`, so the harness default applies as before. Full comparison: [0.2.11...0.2.13](max-sixty/tend@0.2.11...0.2.13). Co-authored-by: leaf-agent <318509791+leaf-agent@users.noreply.github.com>
Regenerates the `tend-*` workflows with tend 0.2.13 (from 0.2.11). Mechanically the diff is the action pin `max-sixty/tend/claude@0.2.11` → `@0.2.13` in the eight workflows that run the agent, the `astral-sh/setup-uv` pin `0.12.13` → `0.12.17` in the two that install uv directly (`tend-mention`, `tend-notifications`), and a regenerated header comment in all nine. The upgrade that matters for this repo is in CI polling. `poll_pr_checks.py` previously treated any gating check that settled without a red conclusion as green — so a required check belonging to a **cancelled** or **stale** run, or a `COMPLETED` check carrying no conclusion at all, let the bot report a PR as green when nothing had actually verified it. 0.2.13 splits those out as `unverified` and gives `poll` a distinct exit code for them (max-sixty/tend#1323). Practically: fewer "checks went green" claims on PRs whose required jobs were cancelled, at the cost of some polls ending unverified rather than green. The rest, in rough order of how visible it is here: - **Sandbox reliability** — the sandbox runtime now retries past a transient bridge-socket failure at launch (max-sixty/tend#1327). That failure is one of the shapes that produces a `tend-outage` issue, so it should show up as fewer of those. - **Sandbox environment** — root `/tmp` is writable inside the sandbox (max-sixty/tend#1324); the `run-tend` skill's note that it was read-only is gone. This repo's `running-tend` skill still restates that note, so it goes stale when this merges — I'll follow up separately rather than mix a skill edit into the generated-workflow diff. - **Instruction-path pinning** — the `run-tend` skill's description of which paths get restored from the PR base, and on which events, was wrong in 0.2.11; `.agents/` is now named alongside `.claude/`, and the event list is corrected (max-sixty/tend#1311). The restore behavior itself is unchanged. - **Auth hygiene** — the bot's claude.ai account is kept out of CI sessions (max-sixty/tend#1328). - **Harness pins** — claude-code 2.1.278 (max-sixty/tend#1330), uv 0.12.17 (max-sixty/tend#1331). `.config/tend.yaml` pins no model, so the harness-default check found nothing to adjust. Full comparison: max-sixty/tend@0.2.11...0.2.13 <details><summary>Copy-on-write sandbox view: landed and reverted within this range</summary> max-sixty/tend#1333 and max-sixty/tend#1334 moved the agent into the job's own checkout and home through a copy-on-write view; max-sixty/tend#1339 reverted both before 0.2.13. Nothing from that work is in this diff — noted so a later reader doesn't chase it out of the commit list. </details> Co-authored-by: worktrunk-bot <254187624+worktrunk-bot@users.noreply.github.com>
A check run that is
COMPLETEDwithout a usable conclusion matched neither bucketreduce_rollupderived —pendingisstatus != "COMPLETED",failedisCOMPLETEDplus a conclusion inRED_CONCLUSIONS— so_settlesaw an emptypending, andpollprintedgreen: every gating check … settled greenwith exit 0 over a check that never produced a result. The ordinary trigger is concurrency: a second push to a branch cancels the first head's run, and a session accountable for that first commit is told its long job passed.approvalreachedapprove: no failing checkby the same route.reduce_rollupnow returns a thirdunverifiedbucket, and green is the allowlist rather than the remainder:GREEN_CONCLUSIONS = {"SUCCESS", "NEUTRAL", "SKIPPED"}, and any other conclusion on aCOMPLETEDcheck that isn't red is unverified. That coversCANCELLEDandSTALE, aCOMPLETEDcheck carrying no conclusion, and any conclusion GitHub adds toCheckConclusionStatelater — deriving green by subtraction is what let the original false green through, so the unrecognized case is fail-closed by construction rather than by keeping a second list current.pollnames those checks and exits 2 — which/tend-ci-runner:monitor-cialready reads as "unverified, not green" — and names them beside a red or cap-hit verdict too, so the reader knows the picture is partial.approvalkeeps its existing policy of approving over a check that never ran, since a check that reached no verdict cannot withhold on its merits, but now prints it as unverified instead of silently;references/approving.mddescribes the widened meaning.Supersession is unchanged and still decides first: where a rerun concluded at the same SHA, the latest run of each (name, workflow) wins and nothing is left unverified. Status contexts route exactly as before —
SUCCESSgreen,ERROR/FAILUREred,PENDING/EXPECTEDneverCOMPLETED.Verification
Six test cases in
generator/tests/test_poll_scripts.pyfail onmainand pass here:test_cancelled_is_not_a_verdict(parametrized overCANCELLEDandSTALE, previously asserting exit 0 for a cancelled check, now exit 2 with the check named),test_cancelled_beside_a_real_failure_still_reads_red,test_approval_names_a_cancelled_check_it_approves_over, andtest_a_conclusion_in_neither_set_is_not_green(parametrized over an absent conclusion and one GitHub does not emit today).Two more pass on
mainand guard against this change over-reaching:test_cancelled_replaced_at_the_same_sha_is_greenpins that a cancellation superseded by a green rerun at the same SHA still reads green, andtest_non_blocking_conclusions_still_read_greenpinsNEUTRALandSKIPPED, which nothing covered before.The file's 44 tests pass, as does the full Python suite apart from the two
test_refresh_consumers.pycases that need PyPI DNS.pre-commit run --all-filesis clean.The mechanism and the live reproduction are in #1322, against
poll_pr_checks.py.Closes #1322 — automated triage