Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 83 additions & 7 deletions .github/workflows/qwen-triage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -578,13 +578,65 @@ jobs:
id: 'ensure_qwen'
run: |-
set -euo pipefail
# Resolve latest and upgrade when stale — the previous shape
# short-circuited on any pre-installed qwen, so a stale image kept
# its version forever and pinned the action's reinstall to it (the
# same bug the review runner fixed in c9f0d0657). The install runs
# from RUNNER_TEMP because the persistent workspace still holds the
# previous run's checked-out tree, and npm reads a cwd .npmrc into a
# global install — same hazard the verify lane documents. Retries
# cover the transient npm crashes the shared runners produce (a
# triage run died on exit 217 mid-install); the fallback covers the
# registry being the thing that is broken.
want=''
for attempt in 1 2 3; do
want="$(npm view --registry=https://registry.npmjs.org '@qwen-code/qwen-code@latest' version | tail -n 1 || true)"
if [ -n "$want" ]; then break; fi
if [ "$attempt" -lt 3 ]; then sleep $((attempt * 5)); fi
done
have=''
if command -v qwen >/dev/null 2>&1; then
echo "qwen already installed:"
else
npm install -g --loglevel=error --no-audit @qwen-code/qwen-code@latest
have="$(qwen --version 2>/dev/null || true)"
fi
qwen --version
echo "version=$(qwen --version)" >> "${GITHUB_OUTPUT}"
if [ -n "$have" ] && [ "$have" = "$want" ]; then
echo "qwen $have (latest)"
echo "version=$have" >> "${GITHUB_OUTPUT}"
exit 0
fi
if [ -z "$want" ]; then
if [ -n "$have" ]; then
echo "::warning::npm registry query failed; triaging with installed qwen $have"
echo "version=$have" >> "${GITHUB_OUTPUT}"
exit 0
fi
echo '::error::qwen is not installed and the npm registry query failed'
exit 1
fi
installed=''
for attempt in 1 2 3; do
if (cd "${RUNNER_TEMP:?}" && npm install -g --loglevel=error --no-audit --registry=https://registry.npmjs.org "@qwen-code/qwen-code@$want"); then
installed=1
break
fi
echo "::warning::qwen install attempt $attempt failed (wanted $want)"
if [ "$attempt" -lt 3 ]; then sleep $((attempt * 10)); fi
done
if [ -z "$installed" ]; then
if [ -n "$have" ]; then
echo "::warning::upgrade to qwen $want failed after 3 attempts; triaging with installed qwen $have"
echo "version=$have" >> "${GITHUB_OUTPUT}"
exit 0
fi
echo "::error::qwen is not installed and installing $want failed after 3 attempts"
exit 1
fi
hash -r
actual="$(qwen --version)"
echo "qwen $actual"
if [ "$actual" != "$want" ]; then
echo "::warning::wanted qwen $want but PATH resolves $actual — a system-wide install shadows the per-run upgrade"
fi
echo "version=$actual" >> "${GITHUB_OUTPUT}"

- name: 'Run Qwen Triage'
id: 'triage'
Expand Down Expand Up @@ -1114,7 +1166,19 @@ jobs:
# reads a cwd .npmrc — whose settings (script-shell, hooks) a
# --registry flag does not override — into a root-privileged
# install. Same fix as the verify lane.
(cd "${RUNNER_TEMP:?}" && npm install -g --registry=https://registry.npmjs.org '@qwen-code/qwen-code@latest')
# Retried: the shared runners produce transient npm crashes (a
# triage install died on exit 217), and one flake must not kill the
# whole job when the next attempt succeeds.
installed=''
for attempt in 1 2 3; do
if (cd "${RUNNER_TEMP:?}" && npm install -g --registry=https://registry.npmjs.org '@qwen-code/qwen-code@latest'); then
installed=1
break
fi
echo "::warning::qwen install attempt $attempt failed"
if [ "$attempt" -lt 3 ]; then sleep $((attempt * 10)); fi
done
[ -n "$installed" ] || { echo '::error::installing @qwen-code/qwen-code@latest failed after 3 attempts'; exit 1; }
qwen --version
tmux -V
Comment on lines +1181 to 1183

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The 10-line retry-install block is duplicated verbatim across the workflow (lines 615–623, 1172–1181, 2456–2465), and the copies have already drifted — the Ensure qwen CLI step carries --loglevel=error --no-audit and pins @$want, while these two siblings do not. — Failure scenario: when the retry policy next needs a change (bumping to 5 attempts, adding --no-audit), the fix must be applied to three locations in lockstep; missing one silently diverges the lanes. The drift is already present.

Note: the tmux/verify installs run before checkout, so a repo-local .github/scripts/install-qwen.sh would not be on disk there — extraction needs a pre-checkout vehicle (composite action or script fetched at a pinned ref).

Suggested change
[ -n "$installed" ] || { echo '::error::installing @qwen-code/qwen-code@latest failed after 3 attempts'; exit 1; }
qwen --version
tmux -V
# TODO: extract the retry-install into a composite action
# (e.g. .github/actions/install-qwen/action.yml) callable
# from all three sites — the triage lane (post-checkout)
# and the tmux/verify lanes (pre-checkout).
[ -n "$installed" ] || { echo '::error::installing @qwen-code/qwen-code@latest failed after 3 attempts'; exit 1; }
qwen --version
tmux -V
中文说明

[建议] 10 行重试安装块在 workflow 中逐字复制了三份(615–623、1172–1181、2456–2465 行),且副本已经漂移——Ensure qwen CLI 步骤带有 --loglevel=error --no-audit 并钉住 @$want,而这两个兄弟步骤没有。— 故障场景:当下次需要更改重试策略(增加到 5 次、添加 --no-audit)时,必须同步修改三处;漏掉一处就会悄悄分化各 lane。漂移已经存在。

注意:tmux/verify 安装在 checkout 之前运行,因此仓库内的 .github/scripts/install-qwen.sh 在那里不存在——提取需要 checkout 前的载体(composite action 或按固定 ref 获取的脚本)。

— qwen3.8-max-preview via Qwen Code /review


Expand Down Expand Up @@ -2386,7 +2450,19 @@ jobs:
# npm reads a cwd .npmrc — whose settings (script-shell, hooks) a
# --registry flag does not override — into a root-privileged
# install.
(cd "${RUNNER_TEMP:?}" && npm install -g --registry=https://registry.npmjs.org '@qwen-code/qwen-code@latest')
# Retried: the shared runners produce transient npm crashes (a
# triage install died on exit 217), and one flake must not kill the
# whole job when the next attempt succeeds.
installed=''
for attempt in 1 2 3; do
if (cd "${RUNNER_TEMP:?}" && npm install -g --registry=https://registry.npmjs.org '@qwen-code/qwen-code@latest'); then
installed=1
break
fi
echo "::warning::qwen install attempt $attempt failed"
if [ "$attempt" -lt 3 ]; then sleep $((attempt * 10)); fi
done
[ -n "$installed" ] || { echo '::error::installing @qwen-code/qwen-code@latest failed after 3 attempts'; exit 1; }
qwen --version

# Chromium system dependencies (apt packages) for evidence
Expand Down
43 changes: 40 additions & 3 deletions scripts/tests/qwen-triage-workflow.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -220,15 +220,48 @@ describe('qwen-triage tmux workflow', () => {
});

it('pins the action reinstall to the version the job already runs', () => {
const ensure = step('Ensure qwen CLI');
expect(workflow).toContain("id: 'ensure_qwen'");
expect(workflow).toContain(
'echo "version=$(qwen --version)" >> "${GITHUB_OUTPUT}"',
);
// Every exit-0 path of the step must emit the version output — the action
// pin reads it, and a path that forgets it re-pins the reinstall to an
// empty string (which the action treats as `latest`, resurrecting the
// stale-dist-tag bug the pin exists to prevent). Three success paths,
// four emission sites (fallback-to-installed is reached twice: registry
// down, install failed), plus fresh-install.
const emissions = (
ensure.match(/echo "version=\$\{?\w+\}?" >> "\$\{GITHUB_OUTPUT\}"/g) ?? []
).length;
expect(emissions).toBeGreaterThanOrEqual(4);
expect(workflow).toContain(
"qwen_cli_version: '${{ steps.ensure_qwen.outputs.version }}'",
);
});

it('resolves latest, retries the install, and never trusts a cwd .npmrc', () => {
// The step used to short-circuit on any pre-installed qwen — the same bug
// the review runner fixed in c9f0d0657 — and its stale version output then
// pinned the action reinstall to a stale release. Pin the repaired shape:
// resolve-then-compare, a bounded retry loop, the RUNNER_TEMP cwd (npm

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The comment below claims this test pins "resolve-then-compare", but no assertion covers the staleness comparison [ "$have" = "$want" ] or the qwen $have (latest) early-exit (qwen-triage.yml ~601) — only the resolve half (the npm view line, the retry loop) and the fallback paths are asserted. — Failure scenario: a one-line mutation =!= inverts the PR's headline behaviour: on a stale image (have != want, the exact case this PR fixes) the step enters the early-exit block, emits version=$have and exits 0, so triage runs the stale CLI and the action reinstall is pinned to the stale version — resurrecting the c9f0d06 bug the comment cites. Every test stays green (the emission-count regex still finds 4 textual sites).

Add an assertion for the comparison, e.g.:

expect(ensure).toContain('[ "$have" = "$want" ]');
中文说明

下方注释声称本测试钉住了 “resolve-then-compare”,但没有任何断言覆盖版本比较 [ "$have" = "$want" ]qwen $have (latest) 提前退出分支(qwen-triage.yml ~601)——只断言了 resolve 部分(npm view 行、重试循环)和回退路径。 — 故障场景:单行变异 =!= 会反转本 PR 的核心行为:在过时镜像上(have != want,正是本 PR 要修的场景)步骤会进入提前退出分支,输出 version=$have 并以 0 退出,于是 triage 用过时 CLI 运行、action 重装被钉到过时版本——复活注释所引用的 c9f0d06 bug。所有测试仍为绿(emission 计数正则仍能匹配到 4 处文本)。

— qwen3.8-max-preview via Qwen Code /review (v0.21.3)

// reads a cwd .npmrc from the persistent workspace into a global install),
// and the registry-down fallback that still triages on the installed CLI.
const ensure = step('Ensure qwen CLI');
expect(ensure).toContain("@qwen-code/qwen-code@latest' version");

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] No assertion covers the --registry=https://registry.npmjs.org flag on the npm view resolution command (qwen-triage.yml ~593). That line runs from the persistent workspace (not RUNNER_TEMP), so its --registry flag is the sole defense against a cwd .npmrc registry override. The assertion below matches both the flagged and the flagless form (verified), so it does not pin the flag — a gap between the test's name ("never trusts a cwd .npmrc") and its actual coverage. — Failure scenario: if the --registry flag is removed from the npm view line, a .npmrc left in the persistent workspace redirects the dist-tag resolution; want resolves to a stale version, and if the runner already has it, have == want triggers the early exit and the action reinstalls the stale release — the bug this PR fixes. The existing assertion still matches the flagless line, so every test stays green.

Suggested change
expect(ensure).toContain("@qwen-code/qwen-code@latest' version");
expect(ensure).toContain("@qwen-code/qwen-code@latest' version");
expect(ensure).toContain("npm view --registry=https://registry.npmjs.org '@qwen-code/qwen-code@latest' version");
中文说明

没有断言覆盖 npm view 解析命令上的 --registry=https://registry.npmjs.org 标志(qwen-triage.yml ~593)。该行从持久工作区运行(而非 RUNNER_TEMP),因此它的 --registry 标志是抵御 cwd .npmrc registry 覆盖的唯一防线。下方断言对带标志和不带标志两种形态都匹配(已验证),所以并未钉住该标志——这是测试名(“never trusts a cwd .npmrc”)与其实际覆盖之间的缺口。 — 故障场景:若从 npm view 行移除 --registry 标志,持久工作区里残留的 .npmrc 会重定向 dist-tag 解析;want 解析到过时版本,若 runner 已装有该版本,have == want 触发提前退出、action 重装过时版本——正是本 PR 修的 bug。现有断言仍匹配无标志行,所有测试仍为绿。

— qwen3.8-max-preview via Qwen Code /review (v0.21.3)

expect(ensure).toContain('for attempt in 1 2 3; do');

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The retry loops added to Install tmux runner tools (~line 1172) and Install verify runner tools (~line 2456) have no test coverage; only the Ensure qwen CLI step's retry is pinned here. — Failure scenario: a future edit removes or restructures the retry loop in either sibling step. The existing tests for those steps check for the npm install command and RUNNER_TEMP cwd but not for the retry loop or the failed after 3 attempts error guard. The next transient npm crash (the exit-217 class) kills the whole job instead of retrying, and the test suite passes green (verified: 121/121 pass with the sibling loops unasserted).

Suggested change
expect(ensure).toContain('for attempt in 1 2 3; do');
expect(ensure).toContain('for attempt in 1 2 3; do');
// Also pin the sibling retry loops (added in the same diff):
const tmuxInstall = stepIn('tmux-testing', 'Install tmux runner tools');
expect(tmuxInstall).toContain('for attempt in 1 2 3; do');
expect(tmuxInstall).toContain('failed after 3 attempts');
中文说明

[建议] 添加到 Install tmux runner tools(约 1172 行)和 Install verify runner tools(约 2456 行)的重试循环没有测试覆盖;此处仅钉住了 Ensure qwen CLI 步骤的重试。— 故障场景:未来编辑删除或重构了任一兄弟步骤中的重试循环。这些步骤的现有测试检查了 npm install 命令和 RUNNER_TEMP 工作目录,但没有检查重试循环或 failed after 3 attempts 错误守卫。下一次瞬态 npm 崩溃(exit-217 类)将杀死整个 job 而不是重试,且测试套件全绿通过(已验证:121/121 通过,兄弟循环未被断言)。

— qwen3.8-max-preview via Qwen Code /review

expect(ensure).toContain('cd "${RUNNER_TEMP:?}"');

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] No assertion covers the install command's target "@qwen-code/qwen-code@$want" (qwen-triage.yml ~617) — the line that installs the resolved version rather than re-resolving @latest. Probe-confirmed: applying the mutation below leaves both relevant tests green, and adding the assertion flips the result. — Failure scenario: change the install line from "@qwen-code/qwen-code@$want" back to '@qwen-code/qwen-code@latest' (the old shape). On a runner whose npm cache holds a stale @latest dist-tag resolution, the install resolves a different (stale) release than npm view just resolved; the post-install shadow warning fires but the step still outputs the stale version= to GITHUB_OUTPUT, pinning the action reinstall to the stale release. Every test stays green.

Suggested change
expect(ensure).toContain('cd "${RUNNER_TEMP:?}"');
expect(ensure).toContain('cd "${RUNNER_TEMP:?}"');
expect(ensure).toContain('"@qwen-code/qwen-code@$want"');
中文说明

没有断言覆盖安装命令的目标 "@qwen-code/qwen-code@$want"qwen-triage.yml ~617)——正是这一行安装已解析的版本、而非重新解析 @latest。已用探针确认:施加下面的变异后两个相关测试仍为绿,加上该断言后结果翻转。 — 故障场景:把安装行从 "@qwen-code/qwen-code@$want" 改回 '@qwen-code/qwen-code@latest'(旧形态)。在 npm 缓存持有过时 @latest dist-tag 解析的 runner 上,安装会解析出与刚才 npm view 不同的(过时)版本;安装后的遮蔽警告会触发,但步骤仍把过时 version= 写入 GITHUB_OUTPUT,把 action 重装钉到过时版本。所有测试仍为绿。

— qwen3.8-max-preview via Qwen Code /review (v0.21.3)

expect(ensure).toContain('triaging with installed qwen');
Comment on lines +250 to +251

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The rewritten Ensure qwen CLI step has two hard-fail paths that no test pins: echo '::error::qwen is not installed and the npm registry query failed'; exit 1 (qwen-triage.yml:612-613) and echo "::error::qwen is not installed and installing $want failed after 3 attempts"; exit 1 (qwen-triage.yml:630-631). The new/changed tests assert only the success and fallback-to-installed paths; the emissions >= 4 regex counts only the four success-path version= writes. — Failure scenario: probe-confirmed — softening both exit 1exit 0 keeps all 121 tests green (the emissions test counts only success paths), so the step would pass with no qwen and no version output, re-pinning the action reinstall to an empty string (the action treats it as latest, resurrecting the stale-dist-tag bug) and losing the fail-fast ::error:: diagnostic. Note: asserting on the message text alone does not catch an exit-code softening — assert on exit 1 / the ::error:: prefix.

Suggested change
expect(ensure).toContain('cd "${RUNNER_TEMP:?}"');
expect(ensure).toContain('triaging with installed qwen');
expect(ensure).toContain('cd "${RUNNER_TEMP:?}"');
expect(ensure).toContain('triaging with installed qwen');
// The two hard-fail paths must keep failing the job — pin the exit code,
// not just the message: a softened `exit 0` would otherwise ship undetected.
expect(ensure).toContain('::error::');
expect(ensure).toContain('exit 1');
中文说明

重写后的 Ensure qwen CLI 步骤有两条必须让 job 失败的路径,但没有任何测试钉住它们:echo '::error::qwen is not installed and the npm registry query failed'; exit 1(qwen-triage.yml:612-613)和 echo "::error::qwen is not installed and installing $want failed after 3 attempts"; exit 1(qwen-triage.yml:630-631)。新增/修改的测试只断言成功路径与回退到已装版本的路径;emissions >= 4 正则只统计四条成功路径的 version= 输出。— 失败场景:经探针实测确认——把两处 exit 1 软化为 exit 0,全部 121 个测试仍全绿(emissions 测试只数成功路径),于是该步骤会在没有 qwen、也没有 version 输出的情况下通过,把 action 的重装版本钉成空字符串(action 会当作 latest,复活 stale-dist-tag bug),并丢掉预期的 fail-fast ::error:: 诊断。注意:只断言错误信息文本抓不到退出码被软化的情况——要断言 exit 1 / ::error:: 前缀。

— qwen3.8-max-preview via Qwen Code /review (v0.21.3)

// The two hard-fail paths must keep failing the job — pin the exit code,
// not just the message: a softened `exit 0` would otherwise ship undetected.
// Anchor each message to its own `exit 1`; the step has two hard-fail
// sites, so an unanchored `toContain('exit 1')` stays green when either
// one is softened while the other still fails the job.
expect(ensure).toMatch(
/::error::qwen is not installed and the npm registry query failed'\n\s*exit 1/,
);
expect(ensure).toMatch(
/::error::qwen is not installed and installing \$want failed after 3 attempts"\n\s*exit 1/,
);
});

it('passes triage output through env before bash reads it', () => {
const checkStep = step('Check triage response');

Expand Down Expand Up @@ -565,6 +598,8 @@ describe('qwen-triage tmux workflow', () => {
);
expect(installStep).toContain('qwen --version');
expect(installStep).toContain('tmux -V');
expect(installStep).toContain('for attempt in 1 2 3; do');
expect(installStep).toContain('failed after 3 attempts');

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] This lane's new retry assertion pins the error message but not the guard's exit 1 (qwen-triage.yml ~1181). The Ensure qwen CLI test in this same PR deliberately anchors each hard-fail message to its own exit 1, with a comment explaining why ("a softened exit 0 would otherwise ship undetected"); the two retry lanes (here and the verify lane at ~4017) omit that anchoring. — Failure scenario: a future edit softens the guard [ -n "$installed" ] || { …; exit 1; } to exit 0. On a stale-image runner (the PR's stated premise) all three installs fail, the step continues, and the immediately-following qwen --version succeeds against the stale pre-installed binary — so the lane silently runs against the stale qwen this PR exists to eliminate. Both toContain assertions stay green because the message string is still present.

Suggested change
expect(installStep).toContain('failed after 3 attempts');
expect(installStep).toMatch(/failed after 3 attempts'; exit 1/);
中文说明

本 lane 新增的重试断言只钉住了错误消息,没钉住守卫的 exit 1qwen-triage.yml ~1181)。同一 PR 里 Ensure qwen CLI 的测试刻意把每个硬失败消息锚定到它自己的 exit 1,并有注释解释原因(“否则被软化的 exit 0 会悄然上线”);两个重试 lane(此处与 ~4017 的 verify lane)漏掉了这一锚定。 — 故障场景:未来某次编辑把守卫 [ -n "$installed" ] || { …; exit 1; } 软化为 exit 0。在过时镜像 runner 上(本 PR 的前提)三次安装全失败、步骤继续执行,紧随其后的 qwen --version against 过时预装二进制成功——于是该 lane 悄然 against 本 PR 要消除的过时 qwen 运行。两个 toContain 断言仍为绿,因为消息字符串仍在。

— qwen3.8-max-preview via Qwen Code /review (v0.21.3)

expect(resolverStep).not.toContain('tmux');
expect(resolverStep).not.toContain('npm install');
expect(resolverStep).not.toContain('qwen --version');
Expand Down Expand Up @@ -3979,6 +4014,8 @@ describe('qwen-triage verify round-3 hardening', () => {
// No hardcoded Playwright pin here either: the apt list must track
// current Playwright so it covers the lockfile-matched binary below.
expect(tools).not.toMatch(/playwright@[\d.]/);
expect(tools).toContain('for attempt in 1 2 3; do');
expect(tools).toContain('failed after 3 attempts');

// Browser binary: downloaded after npm ci, and by the CLI of the
// package the capture harness actually imports — never a hardcoded pin
Expand Down
Loading