Skip to content

fix(poll): report a cancelled gating check as unverified, not green - #1323

Merged
max-sixty merged 3 commits into
mainfrom
fix/issue-1322
Sep 19, 2026
Merged

max-sixty merged 3 commits into
mainfrom
fix/issue-1322

Conversation

@tend-agent

@tend-agent tend-agent commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

A check run that is COMPLETED without a usable conclusion matched neither bucket reduce_rollup derived — pending is status != "COMPLETED", failed is COMPLETED plus a conclusion in RED_CONCLUSIONS — so _settle saw an empty pending, and poll printed green: every gating check … settled green with exit 0 over a check that never produced a result. The ordinary trigger is concurrency: a second push to a branch cancels the first head's run, and a session accountable for that first commit is told its long job passed. approval reached approve: no failing check by the same route.

reduce_rollup now returns a third unverified bucket, and green is the allowlist rather than the remainder: GREEN_CONCLUSIONS = {"SUCCESS", "NEUTRAL", "SKIPPED"}, and any other conclusion on a COMPLETED check that isn't red is unverified. That covers CANCELLED and STALE, a COMPLETED check carrying no conclusion, and any conclusion GitHub adds to CheckConclusionState later — deriving green by subtraction is what let the original false green through, so the unrecognized case is fail-closed by construction rather than by keeping a second list current.

poll names those checks and exits 2 — which /tend-ci-runner:monitor-ci already reads as "unverified, not green" — and names them beside a red or cap-hit verdict too, so the reader knows the picture is partial. approval keeps its existing policy of approving over a check that never ran, since a check that reached no verdict cannot withhold on its merits, but now prints it as unverified instead of silently; references/approving.md describes the widened meaning.

Supersession is unchanged and still decides first: where a rerun concluded at the same SHA, the latest run of each (name, workflow) wins and nothing is left unverified. Status contexts route exactly as before — SUCCESS green, ERROR/FAILURE red, PENDING/EXPECTED never COMPLETED.

Verification

Six test cases in generator/tests/test_poll_scripts.py fail on main and pass here: test_cancelled_is_not_a_verdict (parametrized over CANCELLED and STALE, previously asserting exit 0 for a cancelled check, now exit 2 with the check named), test_cancelled_beside_a_real_failure_still_reads_red, test_approval_names_a_cancelled_check_it_approves_over, and test_a_conclusion_in_neither_set_is_not_green (parametrized over an absent conclusion and one GitHub does not emit today).

Two more pass on main and guard against this change over-reaching: test_cancelled_replaced_at_the_same_sha_is_green pins that a cancellation superseded by a green rerun at the same SHA still reads green, and test_non_blocking_conclusions_still_read_green pins NEUTRAL and SKIPPED, which nothing covered before.

The file's 44 tests pass, as does the full Python suite apart from the two test_refresh_consumers.py cases that need PyPI DNS. pre-commit run --all-files is clean.

The mechanism and the live reproduction are in #1322, against poll_pr_checks.py.

Closes #1322 — automated triage

A check run that is COMPLETED with conclusion CANCELLED or STALE landed in
neither the pending nor the failed bucket, so `_settle` saw nothing pending
and `poll` printed the green verdict. `reduce_rollup` now returns a third
`unverified` bucket for those conclusions: `poll` names them and exits 2,
and `approval` keeps approving over them but says which check never ran.

Closes #1322

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reduce_rollup still derives green by subtraction: unverified is an allowlist of CANCELLED/STALE, so a COMPLETED check whose conclusion is outside both sets — one GitHub adds to CheckConclusionState later, or a null conclusion that reaches the reducer as "" — lands in neither bucket, and poll prints green: every gating check … settled green over it. That is the same false green this PR fixes, one conclusion value away. test_terminal_non_success_conclusions_count_red's own docstring already names the shape: "left out of both buckets it would read as green".

Naming the passing set instead is the same amount of code and closes the default:

GREEN_CONCLUSIONS = {"SUCCESS", "NEUTRAL", "SKIPPED"}
...
and context["conclusion"] not in (RED_CONCLUSIONS | GREEN_CONCLUSIONS)

Status contexts route exactly as before under it — SUCCESS green, ERROR/FAILURE red, PENDING/EXPECTED never COMPLETED — so only the unrecognized case moves.

I'm pushing that to this branch, along with a test for the description's claim that SUCCESS, NEUTRAL and SKIPPED stay non-blocking (nothing covered NEUTRAL or SKIPPED), and a widening of "cancelled before concluding" to "settled without a result" now that the bucket holds more than cancellations.

Nothing else in the change needs one: supersession, the red set, and the approval policy of approving over a check that never ran all read correctly, and the four new tests fail on main.

…don't

Bucketing CANCELLED and STALE explicitly left green as the remainder, so a
COMPLETED check carrying a conclusion outside both sets — one GitHub adds to
CheckConclusionState later, or none at all, which reaches the reducer as "" —
still read as having passed. Green is now the allowlist and the remainder is
unverified, which keeps the unrecognized case fail-closed.

Status contexts route unchanged: SUCCESS green, ERROR/FAILURE red,
PENDING/EXPECTED never COMPLETED.

@tend-agent tend-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two statements of the premise this change removes survive elsewhere in the files it touches, both outside the diff hunks.

references/approving.md — the After the approval bullet beginning A check was cancelled still says "The replacement run will cover the cancelled checks." Under the new reducer that is false for exactly the checks the poll now surfaces: a cancellation with a rerun at the same SHA is superseded in reduce_rollup and never reaches unverified (which test_cancelled_replaced_at_the_same_sha_is_green pins), so a cancelled check that makes the poll exit 2 is one nothing replaced. The bullet tells that session to do nothing, on grounds the Before the APPROVE bullet twenty lines above now contradicts — and it is the only instruction the post-approval poll's reader has for this outcome. The do-not-rerun rule in it still holds and should stay; what has to go is the coverage claim, replaced by naming the check as unverified.

test_terminal_non_success_conclusions_count_red's docstring — "left out of both buckets it would read as green" — now describes the opposite of what the file asserts two tests down, where test_a_conclusion_in_neither_set_is_not_green requires exit 2 for a conclusion in neither set. The reason STARTUP_FAILURE and ACTION_REQUIRED must stay in RED_CONCLUSIONS is now that otherwise they read as unverified rather than red — a different wrong verdict, not a green one.

Neither line falls inside a diff hunk, so no inline suggestion; I'm pushing both to this branch.

approving.md's post-approval bullet told the session a replacement run
covers a cancelled check. A cancellation a rerun replaced at the same SHA
is superseded in reduce_rollup and never reaches the unverified bucket, so
a cancelled check the poll now names is one nothing covered; the bullet now
says to report it as unverified and keeps the do-not-rerun rule.

test_terminal_non_success_conclusions_count_red's docstring said a
conclusion outside both buckets would read as green, which the same file's
test_a_conclusion_in_neither_set_is_not_green now contradicts.
@max-sixty
max-sixty merged commit 19d1534 into main Sep 19, 2026
11 checks passed
@max-sixty
max-sixty deleted the fix/issue-1322 branch September 19, 2026 16:30
@max-sixty max-sixty mentioned this pull request Sep 21, 2026
max-sixty added a commit that referenced this pull request Sep 21, 2026
Version bump, lockfile, and the `## 0.2.12` section the release workflow
publishes verbatim as the GitHub Release notes.

17 commits since 0.2.11. The headline is the sandbox layout: the agent
runs in the job's own checkout and home through a copy-on-write view, so
what a consumer's `setup:` steps prepare reaches the agent without being
rebuilt in `sandbox_setup:`, and the job environment crosses into the
sandbox in a `0600` file rather than on `sudo`'s command line
([#1333](#1333),
[#1334](#1334)). Alongside it:
`/tmp` is writable inside the sandbox as scratch private to the run
([#1324](#1324)), CI sessions
ignore the bot's claude.ai skills, plugins and connectors
([#1328](#1328)), the SRT launch
retries a transient bridge-socket failure
([#1327](#1327)), and a cancelled
gating check polls as unverified rather than green
([#1323](#1323)).

Each bullet was written from the commit's diff rather than its subject.
No real agent session has run through the new view yet (CI's
`test-sandbox` lifecycle runs stub `claude`/`codex` binaries), so the
release's live check after tagging is the first one that does.

> _This was written by Claude Code on behalf of max-sixty_

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
max-sixty pushed a commit to max-sixty/leaf that referenced this pull request Sep 21, 2026
Regenerates the nine `tend-*` workflows against tend 0.2.13 (from
0.2.11). The committed diff is only the pinned action ref and the `uv`
setup version (`0.12.13` → `0.12.17`); everything below rides in the
action and the plugin it loads.

## What changes for this repo

**A cancelled gating check no longer reads as green.**
`poll_pr_checks.py` now treats only `SUCCESS`, `NEUTRAL`, and `SKIPPED`
as green and exits 2 — unverified — on anything else
([max-sixty/tend#1323](max-sixty/tend#1323)).
Cancelled runs are routine here: main serializes the complete suite, so
a newer commit replaces the one waiting for its turn, and `running-tend`
already carries a rule for reading that shape. Under 0.2.11 a session
polling its own push could report such a check as verified; it now has
to say it could not verify.

**CI sessions ignore the bot's claude.ai account.** Skills, plugins, and
MCP connectors enabled on the `leaf-agent` account no longer load into
sessions that push and post as it
([max-sixty/tend#1328](max-sixty/tend#1328)).
Sessions run with this repo's skills and tend's own, and nothing else.

**`/tmp` is writable inside the sandbox**, as scratch private to the run
([max-sixty/tend#1324](max-sixty/tend#1324)). A
tool that hard-codes a `/tmp` path no longer needs an override;
`$TMPDIR` stays the documented place to write.

**The sandbox launch retries a transient bridge-socket failure**, up to
three attempts
([max-sixty/tend#1327](max-sixty/tend#1327)).
SRT gives socat 600 ms to bind, and a slow runner previously lost the
whole job before the agent started.

Three plugin-side fixes also land: `review-runs` can now cancel a run
parked in GitHub's `waiting` state that holds its concurrency group
without ever concluding
([#1320](max-sixty/tend#1320)); `ground-claims`
reads a past run's code at that run's commit rather than at the current
default branch ([#1319](max-sixty/tend#1319));
and `run-tend` corrects its list of instruction paths restored to the
base version on a PR
([#1311](max-sixty/tend#1311)).

## No configuration change needed

0.2.12 moved the agent into the job's own checkout through a
copy-on-write view, which would have changed where dependency setup
belongs. 0.2.13 reverts it
([#1339](max-sixty/tend#1339)): `setup:` and
`sandbox_setup:` behave as they did at 0.2.11, so `.config/tend.yaml` is
unchanged here. This repo pins no `model:`, so the harness default
applies as before.

Full comparison:
[0.2.11...0.2.13](max-sixty/tend@0.2.11...0.2.13).

Co-authored-by: leaf-agent <318509791+leaf-agent@users.noreply.github.com>
max-sixty pushed a commit to max-sixty/worktrunk that referenced this pull request Sep 21, 2026
Regenerates the `tend-*` workflows with tend 0.2.13 (from 0.2.11).
Mechanically the diff is the action pin `max-sixty/tend/claude@0.2.11` →
`@0.2.13` in the eight workflows that run the agent, the
`astral-sh/setup-uv` pin `0.12.13` → `0.12.17` in the two that install
uv directly (`tend-mention`, `tend-notifications`), and a regenerated
header comment in all nine.

The upgrade that matters for this repo is in CI polling.
`poll_pr_checks.py` previously treated any gating check that settled
without a red conclusion as green — so a required check belonging to a
**cancelled** or **stale** run, or a `COMPLETED` check carrying no
conclusion at all, let the bot report a PR as green when nothing had
actually verified it. 0.2.13 splits those out as `unverified` and gives
`poll` a distinct exit code for them (max-sixty/tend#1323). Practically:
fewer "checks went green" claims on PRs whose required jobs were
cancelled, at the cost of some polls ending unverified rather than
green.

The rest, in rough order of how visible it is here:

- **Sandbox reliability** — the sandbox runtime now retries past a
transient bridge-socket failure at launch (max-sixty/tend#1327). That
failure is one of the shapes that produces a `tend-outage` issue, so it
should show up as fewer of those.
- **Sandbox environment** — root `/tmp` is writable inside the sandbox
(max-sixty/tend#1324); the `run-tend` skill's note that it was read-only
is gone. This repo's `running-tend` skill still restates that note, so
it goes stale when this merges — I'll follow up separately rather than
mix a skill edit into the generated-workflow diff.
- **Instruction-path pinning** — the `run-tend` skill's description of
which paths get restored from the PR base, and on which events, was
wrong in 0.2.11; `.agents/` is now named alongside `.claude/`, and the
event list is corrected (max-sixty/tend#1311). The restore behavior
itself is unchanged.
- **Auth hygiene** — the bot's claude.ai account is kept out of CI
sessions (max-sixty/tend#1328).
- **Harness pins** — claude-code 2.1.278 (max-sixty/tend#1330), uv
0.12.17 (max-sixty/tend#1331).

`.config/tend.yaml` pins no model, so the harness-default check found
nothing to adjust.

Full comparison:
max-sixty/tend@0.2.11...0.2.13

<details><summary>Copy-on-write sandbox view: landed and reverted within
this range</summary>

max-sixty/tend#1333 and max-sixty/tend#1334 moved the agent into the
job's own checkout and home through a copy-on-write view;
max-sixty/tend#1339 reverted both before 0.2.13. Nothing from that work
is in this diff — noted so a later reader doesn't chase it out of the
commit list.

</details>

Co-authored-by: worktrunk-bot <254187624+worktrunk-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

poll_pr_checks.py reports green for a cancelled gating check, because CANCELLED is missing from RED_CONCLUSIONS

2 participants