Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 25 additions & 7 deletions .github/workflows/qwen-code-pr-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1903,12 +1903,28 @@ jobs:
# re-runs, which keep the same run id. A review-pr that dies to its own
# job-level timeout is auto-CANCELLED by GitHub — result 'cancelled' and
# failure() false — which opens neither a failure-only gate nor the in-job
# step, so the gate admits 'cancelled' too; a run-level cancel cancels this
# queued job with it, so a live gate evaluation seeing 'cancelled' is
# overwhelmingly the timeout case, and the residual manual single-job
# cancel just gets a benign retry-guidance comment. The PR number comes
# from the event payload, not the dead job's outputs, which do not survive
# a crash.
# step, so the gate admits 'cancelled' — but only when the upstream chain
# finished. `always()` keeps this job running through a RUN-level cancel
# (it does not die with the run), so a concurrency supersede used to post
# a false "did not complete" while the surviving run was still reviewing:
# on PR #9131 a same-head pull_request_target pair started 1s apart, the
# newer run cancelled the older inside authorize, and the older run's gate
# saw review-pr 'cancelled' (run 32558544379) — same-head, so the in-step
# head-moved guard could not catch it. The two cancels are separable in
# `needs`: a job-level timeout cancels review-pr ALONE — authorize and
# delay-automatic-review completed long before — while a run-level cancel
# sweeps the whole chain, so 'cancelled' opens the gate only when neither
# upstream job was itself cancelled. A run-level cancel landing AFTER the
# chain finished (mid-review) still opens the gate: the push-supersede
# flavor is then suppressed by the in-step head-moved guard, the close
# flavor (a `closed`-action run joining the PR-scoped group hours in, the
# head unchanged) by the in-step PR-state check, and a same-head twin
# cannot land that late — its cancel fires at run creation, seconds in.
# A manual run-cancel during the delay window goes
# silent under this rule (the person who cancelled does not need retry
# guidance); a manual cancel of review-pr alone mid-review still posts,
# benign as before. The PR number comes from the event payload, not the
# dead job's outputs, which do not survive a crash.
fallback-comment:
needs:
[
Expand All @@ -1921,7 +1937,9 @@ jobs:
if: |-
always() &&
(needs.review-pr.result == 'failure' ||
needs.review-pr.result == 'cancelled' ||
(needs.review-pr.result == 'cancelled' &&
needs.authorize.result != 'cancelled' &&
needs.delay-automatic-review.result != 'cancelled') ||
Comment on lines +1940 to +1942

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The new upstream-cancel guard on this gate is pinned by no test. The resilience suite in scripts/tests/qwen-pr-review-workflow.test.js (block 'fallback comment resilience (PR #8894 incident class)') promises "reverting any hunk must fail here", but its only cancelled-related assertion — toContain("needs.review-pr.result == 'cancelled'") — survives verbatim inside this new compound clause, so it passes identically for the old and the new gate. If a future change reverts the two != 'cancelled' conjuncts, the exact #9131 regression this PR fixes returns — a concurrency-superseded run posting "review did not complete" while its same-head twin is still reviewing — and the suite stays green. Verified by mutation: Tests 174 passed (174) on the PR gate and Tests 174 passed (174) on a reverted-gate mutant, while a comparator probe pinning the two conjuncts fails on the mutant and passes on the PR gate. The test's rationale comment (~lines 3070–3073) also still asserts the premise this PR falsifies ("a run-level cancel cancels this queued job with it"). Consider pinning the new shape (e.g. toContain("needs.authorize.result != 'cancelled'") and toContain("needs.delay-automatic-review.result != 'cancelled'")) and refreshing that comment.

中文说明

新增的上游取消守卫没有任何测试固定。scripts/tests/qwen-pr-review-workflow.test.js 中的 'fallback comment resilience (PR #8894 incident class)' 测试块声明 "reverting any hunk must fail here",但其唯一与 cancelled 相关的断言 —— toContain("needs.review-pr.result == 'cancelled'") —— 在新的复合子句中依然逐字存在,因此新旧 gate 都能通过同一条断言。如果未来还原这两个 != 'cancelled' 条件,本 PR 修复的 #9131 误报回归(被并发组顶掉的 run 在同 head 的幸存 run 仍在评审时发出 "review did not complete")将悄然重现,而测试套件仍是绿色。已通过变异验证:PR gate 下 Tests 174 passed (174);还原 gate 的变异体同样 Tests 174 passed (174);而固定这两个条件的对比探针在变异体上失败、在 PR gate 上通过。另外,该测试约第 3070–3073 行的注释仍在断言本 PR 已推翻的前提("run 级取消会连带取消排队中的 fallback job")。建议固定新的子句形状(例如 toContain("needs.authorize.result != 'cancelled'")toContain("needs.delay-automatic-review.result != 'cancelled'")),并更新该注释。

— qwen3.8-max via Qwen Code /review (v0.21.15)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Landed in 49e92078fb.

Pinned the full compound clause — grouping included — rather than the two conjuncts separately, so dropping either != 'cancelled' conjunct or the parenthesization fails the gate test, and rewrote the rationale comment that still asserted the falsified premise (a run-level cancel taking the queued fallback job down with it).

Mutation-verified locally the same way as the review's probe: the gate test fails on a reverted-gate mutant and passes on the PR gate. The two runHealthProbe failures a root shell shows locally are environmental (chmod-based unwritability is void for root) and green in CI.

needs.authorize.result == 'failure' ||
needs.review-config.result == 'failure' ||
needs.delay-automatic-review.result == 'failure' ||
Expand Down
25 changes: 22 additions & 3 deletions scripts/tests/qwen-pr-review-workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -3069,9 +3069,28 @@ describe('fallback comment resilience (PR #8894 incident class)', () => {
expect(job.if).toContain("needs.review-pr.result == 'failure'");
// A review-pr that dies to its own job-level timeout is auto-cancelled
// (result 'cancelled', failure() false), which would open neither this
// gate nor the in-job step; a run-level cancel cancels this queued job
// too, so a live evaluation seeing 'cancelled' is the timeout case.
expect(job.if).toContain("needs.review-pr.result == 'cancelled'");
// gate nor the in-job step — but `always()` keeps this job alive through
// a RUN-level cancel, so a bare 'cancelled' clause posts a false "did
// not complete" from a concurrency-superseded run while its same-head
// twin is still reviewing (PR #9131, run 32558544379 — same head, so
// the in-step head-moved guard cannot catch it). The two cancels differ
// in `needs`: a timeout cancels review-pr ALONE, a run-level cancel
// sweeps the upstream chain too. Pin the full compound clause — the
// grouping included — so reverting either upstream conjunct fails here.
expect(job.if).toContain(
"(needs.review-pr.result == 'cancelled' &&\n" +
" needs.authorize.result != 'cancelled' &&\n" +
" needs.delay-automatic-review.result != 'cancelled') ||",
);
Comment on lines +3080 to +3084

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The pin proves the compound clause is present in fallback-comment.if, but nothing proves the old bare needs.review-pr.result == 'cancelled' || disjunct is absent — the replaced assertion would still pass on the new workflow, because that substring survives inside the compound clause. A merge-conflict resolution that keeps both sides of this hunk re-adds the bare disjunct beside the intact compound clause; the gate then opens on every cancelled review-pr again and the false "review did not complete" fallback comment returns on concurrency-superseded runs (the PR #9131 incident class), while this pin and every other assertion stay green. A scratch-tree probe confirmed the mutant: with the bare disjunct re-added the full suite still reports Tests 174 passed (174); adding the negative assertion below makes that mutant fail (expected … not to contain 'needs.review-pr.result == 'cancelled' ||') while the pristine workflow still passes 174/174.

Suggested change
expect(job.if).toContain(
"(needs.review-pr.result == 'cancelled' &&\n" +
" needs.authorize.result != 'cancelled' &&\n" +
" needs.delay-automatic-review.result != 'cancelled') ||",
);
expect(job.if).toContain(
"(needs.review-pr.result == 'cancelled' &&\n" +
" needs.authorize.result != 'cancelled' &&\n" +
" needs.delay-automatic-review.result != 'cancelled') ||",
);
expect(job.if).not.toContain("needs.review-pr.result == 'cancelled' ||");
中文说明

这个固定断言只能证明复合子句存在fallback-comment.if,但无法证明旧的裸 needs.review-pr.result == 'cancelled' || 析取项不存在——被替换的断言在新工作流上依然会通过,因为该子串仍包含在复合子句内部。如果某次 merge 冲突解决同时保留了本 hunk 的两侧内容,裸析取项就会与完整的复合子句并存;门控将再次在所有被取消的 review-pr 上放行,并发顶替场景下错误的 "review did not complete" 兜底评论会复发(即 PR #9131 事故类别),而本断言和所有其他断言仍全部为绿。临时工作树中的探针确认了该变异体:重新加回裸析取项后完整测试套件仍报告 Tests 174 passed (174);加入上面的负向断言后该变异体失败(expected … not to contain 'needs.review-pr.result == 'cancelled' ||'),而原始工作流仍 174/174 通过。

— qwen3.8-max via Qwen Code /review (v0.21.15)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Landed in 36b9a5b699 — applied the suggested negative assertion verbatim, with a comment noting why the negation holds on the intended gate (inside the compound clause the substring is followed by &&).

Also took this round's deferred note in the same commit: the job comment now states that a run-level cancel landing after the upstream chain finished still opens the gate, that the push-supersede flavor is then suppressed by the in-step head-moved guard, and why a same-head twin cannot land that late (its cancel fires at run creation).

Re-verified by mutation: with the bare disjunct re-added beside the intact compound clause the gate test fails on the new negation; the pristine workflow passes the suite at parity with base.

// ...and that clause must be the ONLY place the gate tests review-pr
// for 'cancelled': a merge-conflict resolution keeping both sides
// re-adds a bare `== 'cancelled'` disjunct beside the intact compound
// clause — in any rendering (bare, parenthesized, respaced) — and the
// gate opens on every cancelled review-pr again while the pin above
// stays green. Counting occurrences catches every rendering.
expect(
job.if.match(/needs\.review-pr\.result == 'cancelled'/g),
).toHaveLength(1);
expect(job.if).toContain("needs.authorize.result == 'failure'");
expect(job.if).toContain("needs.review-config.result == 'failure'");
expect(job.if).toContain(
Expand Down
Loading