-
Notifications
You must be signed in to change notification settings - Fork 3k
fix(review): stop the fallback comment from firing on superseded runs #9716
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
26d2484
49e9207
36b9a5b
46cec25
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -3069,9 +3069,28 @@ describe('fallback comment resilience (PR #8894 incident class)', () => { | |||||||||||||||||||||||
| expect(job.if).toContain("needs.review-pr.result == 'failure'"); | ||||||||||||||||||||||||
| // A review-pr that dies to its own job-level timeout is auto-cancelled | ||||||||||||||||||||||||
| // (result 'cancelled', failure() false), which would open neither this | ||||||||||||||||||||||||
| // gate nor the in-job step; a run-level cancel cancels this queued job | ||||||||||||||||||||||||
| // too, so a live evaluation seeing 'cancelled' is the timeout case. | ||||||||||||||||||||||||
| expect(job.if).toContain("needs.review-pr.result == 'cancelled'"); | ||||||||||||||||||||||||
| // gate nor the in-job step — but `always()` keeps this job alive through | ||||||||||||||||||||||||
| // a RUN-level cancel, so a bare 'cancelled' clause posts a false "did | ||||||||||||||||||||||||
| // not complete" from a concurrency-superseded run while its same-head | ||||||||||||||||||||||||
| // twin is still reviewing (PR #9131, run 32558544379 — same head, so | ||||||||||||||||||||||||
| // the in-step head-moved guard cannot catch it). The two cancels differ | ||||||||||||||||||||||||
| // in `needs`: a timeout cancels review-pr ALONE, a run-level cancel | ||||||||||||||||||||||||
| // sweeps the upstream chain too. Pin the full compound clause — the | ||||||||||||||||||||||||
| // grouping included — so reverting either upstream conjunct fails here. | ||||||||||||||||||||||||
| expect(job.if).toContain( | ||||||||||||||||||||||||
| "(needs.review-pr.result == 'cancelled' &&\n" + | ||||||||||||||||||||||||
| " needs.authorize.result != 'cancelled' &&\n" + | ||||||||||||||||||||||||
| " needs.delay-automatic-review.result != 'cancelled') ||", | ||||||||||||||||||||||||
| ); | ||||||||||||||||||||||||
|
Comment on lines
+3080
to
+3084
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [Suggestion] The pin proves the compound clause is present in
Suggested change
中文说明这个固定断言只能证明复合子句存在于 — qwen3.8-max via Qwen Code /review (v0.21.15)
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Landed in Also took this round's deferred note in the same commit: the job comment now states that a run-level cancel landing after the upstream chain finished still opens the gate, that the push-supersede flavor is then suppressed by the in-step head-moved guard, and why a same-head twin cannot land that late (its cancel fires at run creation). Re-verified by mutation: with the bare disjunct re-added beside the intact compound clause the gate test fails on the new negation; the pristine workflow passes the suite at parity with base. |
||||||||||||||||||||||||
| // ...and that clause must be the ONLY place the gate tests review-pr | ||||||||||||||||||||||||
| // for 'cancelled': a merge-conflict resolution keeping both sides | ||||||||||||||||||||||||
| // re-adds a bare `== 'cancelled'` disjunct beside the intact compound | ||||||||||||||||||||||||
| // clause — in any rendering (bare, parenthesized, respaced) — and the | ||||||||||||||||||||||||
| // gate opens on every cancelled review-pr again while the pin above | ||||||||||||||||||||||||
| // stays green. Counting occurrences catches every rendering. | ||||||||||||||||||||||||
| expect( | ||||||||||||||||||||||||
| job.if.match(/needs\.review-pr\.result == 'cancelled'/g), | ||||||||||||||||||||||||
| ).toHaveLength(1); | ||||||||||||||||||||||||
| expect(job.if).toContain("needs.authorize.result == 'failure'"); | ||||||||||||||||||||||||
| expect(job.if).toContain("needs.review-config.result == 'failure'"); | ||||||||||||||||||||||||
| expect(job.if).toContain( | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Suggestion] The new upstream-cancel guard on this gate is pinned by no test. The resilience suite in
scripts/tests/qwen-pr-review-workflow.test.js(block'fallback comment resilience (PR #8894 incident class)') promises "reverting any hunk must fail here", but its only cancelled-related assertion —toContain("needs.review-pr.result == 'cancelled'")— survives verbatim inside this new compound clause, so it passes identically for the old and the new gate. If a future change reverts the two!= 'cancelled'conjuncts, the exact #9131 regression this PR fixes returns — a concurrency-superseded run posting "review did not complete" while its same-head twin is still reviewing — and the suite stays green. Verified by mutation:Tests 174 passed (174)on the PR gate andTests 174 passed (174)on a reverted-gate mutant, while a comparator probe pinning the two conjuncts fails on the mutant and passes on the PR gate. The test's rationale comment (~lines 3070–3073) also still asserts the premise this PR falsifies ("a run-level cancel cancels this queued job with it"). Consider pinning the new shape (e.g.toContain("needs.authorize.result != 'cancelled'")andtoContain("needs.delay-automatic-review.result != 'cancelled'")) and refreshing that comment.中文说明
新增的上游取消守卫没有任何测试固定。
scripts/tests/qwen-pr-review-workflow.test.js中的'fallback comment resilience (PR #8894 incident class)'测试块声明 "reverting any hunk must fail here",但其唯一与 cancelled 相关的断言 ——toContain("needs.review-pr.result == 'cancelled'")—— 在新的复合子句中依然逐字存在,因此新旧 gate 都能通过同一条断言。如果未来还原这两个!= 'cancelled'条件,本 PR 修复的 #9131 误报回归(被并发组顶掉的 run 在同 head 的幸存 run 仍在评审时发出 "review did not complete")将悄然重现,而测试套件仍是绿色。已通过变异验证:PR gate 下Tests 174 passed (174);还原 gate 的变异体同样Tests 174 passed (174);而固定这两个条件的对比探针在变异体上失败、在 PR gate 上通过。另外,该测试约第 3070–3073 行的注释仍在断言本 PR 已推翻的前提("run 级取消会连带取消排队中的 fallback job")。建议固定新的子句形状(例如toContain("needs.authorize.result != 'cancelled'")和toContain("needs.delay-automatic-review.result != 'cancelled'")),并更新该注释。— qwen3.8-max via Qwen Code /review (v0.21.15)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Landed in
49e92078fb.Pinned the full compound clause — grouping included — rather than the two conjuncts separately, so dropping either
!= 'cancelled'conjunct or the parenthesization fails the gate test, and rewrote the rationale comment that still asserted the falsified premise (a run-level cancel taking the queued fallback job down with it).Mutation-verified locally the same way as the review's probe: the gate test fails on a reverted-gate mutant and passes on the PR gate. The two
runHealthProbefailures a root shell shows locally are environmental (chmod-based unwritability is void for root) and green in CI.