Skip to content

fix(ci): stop the scripts-suite timeout knob from failing open on an empty value - #10910

Open
yiliang114 wants to merge 1 commit into
mainfrom
fix/scripts-suite-timeout-ceiling
Open

fix(ci): stop the scripts-suite timeout knob from failing open on an empty value#10910
yiliang114 wants to merge 1 commit into
mainfrom
fix/scripts-suite-timeout-ceiling

Conversation

@yiliang114

@yiliang114 yiliang114 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

Fixes a knob in the scripts test lane that silently disarmed the very timeout it exists to set, and removes six per-test ceilings that now work against the suite-wide ceiling they predate. Both are follow-ups to #10870.

Why it's needed

#10870 gave the scripts suite a 90s ceiling with an environment override, written as Number(process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS'] ?? 90_000). ?? only catches undefined, so an empty value produces Number('') — which is 0 — and vitest reads 0 as no timeout at all. The knob whose entire purpose is to raise the ceiling would instead remove it, and a genuinely hung test would run until the job cap rather than failing in 90 seconds. A typo takes the same path through NaN.

'' is not a hypothetical spelling. It is exactly what this repo's ${{ cond && 'x' || '' }} idiom renders when the condition is false, and that is how the sibling QWEN_SKIP_LATENCY_BUDGETS knob — added by #10870 one line away in ci.yml — is wired today. Nothing sets this variable in a workflow yet, so the fault is latent rather than live; that is why this is a normal follow-up and not a hotfix.

The pin that was supposed to catch this could not. It stubbed '' and then called vi.unstubAllEnvs() on the very next line, so both of its arms measured the unset path and the one value that would have failed was discarded before the assertion ran.

Separately, six cases in qwen-autofix-workflow.test.js still carry }, 30000). Those predate the suite-wide ceiling and now shadow it, pinning exactly the bash-spawning tests that the 90s default exists to protect back down to the old flat 30s — on the same contended pool that caused #10853 in the first place. None of the six asserts a duration property; each is a "give this subprocess-spawning test room" budget, and the last one says so in its own comment.

Reviewer Test Plan

How to verify

The fix replaces ?? with ||, so every non-positive spelling falls back to the default. The one thing it gives up is passing 0 to mean "no timeout", which is a footgun rather than a feature and which no caller uses.

scripts/tests/unit-vitest-configs.test.ts cannot be run outside CI: it imports every workspace vitest config, and packages/webui/vite.config.ts needs vite-plugin-dts, which a root-only install does not have. That limitation predates this PR and is unrelated to it. The evidence below therefore comes from an equivalent standalone probe that imports only scripts/tests/vitest.config.ts and asserts the same five arms as the rewritten pin, run against both the old expression and the new one.

A reviewer should confirm three things: that Number('') is 0 and that vitest treats testTimeout: 0 as unbounded; that the rewritten pin fails against the expression this PR replaces and passes against the new one; and that the six removed ceilings were raising vitest's 5s default rather than asserting a performance property.

Evidence (Before & After)

Coercion behaviour of the expression being replaced:

Number(undefined ?? 90000) = 90000     Number('' ?? 90000) = 0     Number('abc' ?? 90000) = NaN     Number('5000' ?? 90000) = 5000

Before — the rewritten pin run against the old ?? expression fails on the arm the old pin discarded:

stub=<unset> -> 90000 (want 90000)
stub=""      -> 0     (want 90000)
AssertionError: expected +0 to be 90000
Tests  1 failed (1)

After — the same pin against the new || expression:

stub=<unset> -> 90000 (want 90000)
stub=""      -> 90000 (want 90000)
stub="abc"   -> 90000 (want 90000)
stub="0"     -> 90000 (want 90000)
stub="5000"  -> 5000  (want 5000)
Tests  1 passed (1)

The vi.stubEnv(k, undefined) form used to drop the old pin's if/else is confirmed against the installed vitest: else if (value === void 0) delete process.env[name], and QWEN_SCRIPTS_TEST_TIMEOUT_MS is not in vitest's _envBooleans special-case list, so '' is stored verbatim.

On the six removed ceilings: the three timeout: 30_000 values that remain in the file are spawnSync bounds and are deliberately left, because spawnSync blocks the event loop where vitest's async timeout cannot fire — their own comment records this. Also worth noting for the reviewer: the test that actually broke release run 33676423730, upserts deferred findings into a per-PR issue that survives the merge, is not among the six. It already runs on the suite ceiling, so #10870 did fix the reported failure; these six are the remainder.

prettier --check and eslint are clean on all three changed files.

Tested on

OS Status
🍏 macOS ⚠️
🪟 Windows ⚠️
🐧 Linux

Environment (optional)

Unit tests only, via the standalone probe described above (Node 22, vitest 3.2.4).

Risk & Scope

  • Main risk or tradeoff: test-infrastructure only, with no production code and no assertion changes. The worst case from the second half is that a genuinely hung case among the six now takes 90s instead of 30s to be called — which is precisely the trade the suite ceiling was chosen for. The || fix gives up the ability to pass 0 for "no timeout"; no caller does.
  • Not validated / out of scope: no workflow wires QWEN_SCRIPTS_TEST_TIMEOUT_MS yet, so the empty-value path is latent and cannot be exercised end-to-end here; and unit-vitest-configs.test.ts itself only runs in CI, for the vite-plugin-dts reason above, so the rewritten pin's own green run comes from CI rather than from this machine.
  • Breaking changes / migration notes: none.

Linked Issues

Follow-up to #10870, where the empty-value fault was raised by @doudouOUC (S1) and @chiga0 (R3-1) and confirmed post-merge against a real build by @wenshao. The six ceiling removals were first proposed by @qwen-code-dev-bot in #10858, which now conflicts with the knob that landed in #10870 and is superseded here. Original contended-host failure: #10853.

中文说明

这个 PR 做了什么

修复 scripts 测试通道里一个旋钮 —— 它会静默解除自己本要设置的超时上限;并移除六处 per-test 上限,它们现在反过来压低了后来才有的套件级上限。两者都是 #10870 的后续。

为什么需要

#10870 给 scripts 套件设了 90 秒上限并带一个环境变量覆盖,写法是 Number(process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS'] ?? 90_000)?? 只捕获 undefined,所以空值会得到 Number('') —— 即 0 —— 而 vitest 把 0 理解为完全不设超时。这个本意是抬高上限的旋钮,反而会把上限整个移除,真正卡死的测试将一直跑到 job 上限而不是在 90 秒时失败。拼写错误经由 NaN 走同一条路径。

'' 不是假想的写法。它正是本仓库 ${{ cond && 'x' || '' }} 惯用法在条件为假时渲染出的值,而这正是 #10870ci.yml 里相邻一行新增的 QWEN_SKIP_LATENCY_BUDGETS 旋钮今天的接线方式。目前还没有任何 workflow 设置这个变量,所以该缺陷是潜伏而非活跃的 —— 这也是本 PR 作为常规后续而非热修的原因。

本该抓住它的那条 pin 抓不住。它先 stub '',紧接着在下一行调用 vi.unstubAllEnvs(),于是两个分支量的都是未设置路径,唯一会失败的取值在断言运行前就被丢弃了。

另外,qwen-autofix-workflow.test.js 中仍有六处 }, 30000)。它们早于套件级上限,如今反过来遮蔽了它,把恰恰是 90 秒默认值要保护的那批 spawn 子进程的测试压回旧的固定 30 秒 —— 就在最初导致 #10853 的同一批争抢主机上。六处没有一处把时长当作被测属性,每一处都只是「给这个 spawn 子进程的测试留余量」,最后一处在它自己的注释里就是这么写的。

评审验证方案

如何验证

修复用 || 替换 ??,使一切非正数写法都回落到默认值。唯一放弃的是用 0 表示「不设超时」的能力,这是个坑而非特性,且没有调用方使用。

scripts/tests/unit-vitest-configs.test.ts 无法在 CI 之外运行:它导入所有工作区的 vitest 配置,而 packages/webui/vite.config.ts 需要 vite-plugin-dts,仅根目录安装并不包含它。该限制早于本 PR 且与之无关。因此下面的证据来自一个等价的独立探针,它只导入 scripts/tests/vitest.config.ts,并断言与重写后 pin 相同的五个分支,分别对旧表达式和新表达式各跑一次。

评审者应确认三点:Number('')0 且 vitest 把 testTimeout: 0 当作无上限;重写后的 pin 对本 PR 替换掉的表达式失败、对新表达式通过;以及被移除的六处上限是在抬高 vitest 的 5 秒默认值,而不是在断言某个性能属性。

证据(前后对比)

被替换表达式的强制转换行为:

Number(undefined ?? 90000) = 90000     Number('' ?? 90000) = 0     Number('abc' ?? 90000) = NaN     Number('5000' ?? 90000) = 5000

修改前 —— 重写后的 pin 对旧的 ?? 表达式运行,在旧 pin 丢弃掉的那个分支上失败:

stub=<unset> -> 90000 (want 90000)
stub=""      -> 0     (want 90000)
AssertionError: expected +0 to be 90000
Tests  1 failed (1)

修改后 —— 同一条 pin 对新的 || 表达式:

stub=<unset> -> 90000 (want 90000)
stub=""      -> 90000 (want 90000)
stub="abc"   -> 90000 (want 90000)
stub="0"     -> 90000 (want 90000)
stub="5000"  -> 5000  (want 5000)
Tests  1 passed (1)

用来去掉旧 pin 中 if/elsevi.stubEnv(k, undefined) 写法已对照已安装的 vitest 源码确认:else if (value === void 0) delete process.env[name],且 QWEN_SCRIPTS_TEST_TIMEOUT_MS 不在 vitest 的 _envBooleans 特例表内,因此 '' 会被原样存储。

关于移除的六处上限:文件中保留的三个 timeout: 30_000spawnSync 的边界,刻意保留,因为 spawnSync 阻塞事件循环,vitest 的异步超时无法在那里触发 —— 它们自己的注释就记录了这一点。另外值得向评审者说明:真正导致 release run 33676423730 失败的那个用例 upserts deferred findings into a per-PR issue that survives the merge 不在这六处之中。它已经跑在套件上限上,所以 #10870 确实修复了报告的那次失败;这六处是余下的部分。

三个改动文件的 prettier --checkeslint 均干净。

测试平台

系统 状态
🍏 macOS ⚠️
🪟 Windows ⚠️
🐧 Linux

运行环境(可选)

仅单元测试,通过上述独立探针运行(Node 22、vitest 3.2.4)。

风险与范围

  • 主要风险或权衡:仅涉及测试基础设施,无生产代码、无断言改动。后半部分最坏的情况是六处中若真有卡死用例,现在需要 90 秒而非 30 秒才被判定 —— 而这正是选择套件级上限时所做的权衡。|| 的修复放弃了用 0 表示「不设超时」的能力,没有调用方这样用。
  • 未验证 / 范围之外:目前没有 workflow 接线 QWEN_SCRIPTS_TEST_TIMEOUT_MS,因此空值路径是潜伏的、无法在此做端到端验证;另外 unit-vitest-configs.test.ts 本身因上述 vite-plugin-dts 原因只能在 CI 中运行,所以重写后 pin 自身的绿色结果来自 CI 而非本机。
  • 破坏性变更 / 迁移说明:无。

关联 Issue

#10870 的后续,空值缺陷由 @doudouOUC(S1)与 @chiga0(R3-1)在该 PR 上提出,并由 @wenshao 在合并后对真实构建验证确认。六处上限的移除最早由 @qwen-code-dev-bot#10858 中提出,该 PR 现与 #10870 落地的旋钮冲突,由本 PR 取代。最初的争抢主机失败:#10853

…empty value

`scripts/tests/vitest.config.ts` read its ceiling as
`Number(process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS'] ?? 90_000)`. `??` only
catches `undefined`, so an empty value yields `Number('')` === 0, and vitest
reads 0 as "no timeout at all" — the knob meant to raise the ceiling would
instead remove it, and a hung test would run until the job cap. `NaN` from a
typo does the same.

`''` is not a hypothetical spelling. It is exactly what this repo's
`${{ cond && 'x' || '' }}` idiom renders when the condition is false, which is
how the sibling `QWEN_SKIP_LATENCY_BUDGETS` knob one line away in ci.yml is
wired. Nothing sets this variable in a workflow today, so the fault is latent
rather than live.

Measured, on this config:

    undefined -> 90000    '' -> 0    'abc' -> NaN    '5000' -> 5000

`||` instead of `??` sends every non-positive spelling to the default. The
cost is that 0 can no longer be passed to mean "no timeout"; that is a footgun
rather than a feature, and no caller uses it.

The companion pin could not have caught this. It stubbed `''` and then called
`vi.unstubAllEnvs()` on the next line, so both of its arms measured the unset
path and the one value that would fail was discarded. It now asserts `''`,
`'abc'` and `'0'` alongside the two original arms; `vi.stubEnv(k, undefined)`
deletes the variable, so the unset arm no longer needs the if/else. Verified
both ways: the rewritten pin fails on the old expression
(`expected +0 to be 90000`) and passes on the new one.

Also drops six per-test `}, 30000)` ceilings in qwen-autofix-workflow.test.js.
They predate the suite ceiling and now shadow it, pinning exactly the
bash-spawning cases that the 90s default exists to protect back down to the old
flat 30s. None of them asserts a duration property — each is a "give this
subprocess-spawning test room" budget, and the last one says so in its own
comment, which is updated to point at the suite ceiling. The three
`timeout: 30_000` options that remain are `spawnSync` bounds, deliberately
separate because spawnSync blocks the event loop where vitest's async timeout
cannot fire.

The ceiling removals were first proposed by qwen-code-dev-bot in #10858, which
conflicts with the knob that landed in #10870 and is superseded here.

Follow-up to #10870; the empty-value fault was raised there by doudouOUC (S1)
and chiga0 (R3-1) and confirmed post-merge by wenshao.
@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finishedview run. See the stage comments in this thread for the result.

Qwen Triage 已完成 —— 查看运行。结果见本线程中的各阶段评论。

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @yiliang114 — the analysis here holds up under checking: the current pin really does stub '' and immediately vi.unstubAllEnvs() it away, so the unset path gets measured twice; Number('') is 0, which disarms the ceiling the knob exists to set; ${{ cond && 'x' || '' }} is not a hypothetical spelling — it is exactly how QWEN_SKIP_LATENCY_BUDGETS is wired in ci.yml today; and the three surviving timeout: 30_000 values are indeed spawnSync bounds, which vitest's async timeout cannot reach. The scope reads clearly, but the PR body skips most required sections of the PR template, so it has to bounce before code review:

  • ## Why it's needed — the motivation currently sits under the custom ## 1 — / ## 2 — headings; consolidate it under the required one.
  • ## Reviewer Test Plan with its subsections — ### How to verify (the verified-in-both-directions probe write-up is already there; it just needs the heading, together with the note that unit-vitest-configs.test.ts only runs in CI), ### Evidence (Before & After) (the AssertionError: expected +0 to be 90000 before / five-arms-pass after output), and the ### Tested on OS table.
  • ## Risk & Scope## Risk covers the tradeoff, but the template asks for the three bullets; the natural "Not validated / out of scope" entry here is your own point that no workflow wires this knob yet and the pin cannot run outside CI.
  • ## Linked Issues — the #10870 / #10853 / #10858 references are inline prose; gather them in this section (no closing keyword needed — this follow-up intentionally closes nothing).
  • Bilingual summary — the <details><summary>中文说明</summary> translation of the body, per the template.

No code changes needed for this — fill in the sections above, then comment @qwen-code /triage to re-run the gate.

中文说明

@yiliang114 感谢这个 PR —— 分析经过核对是站得住的:现有 pin 确实先 stub '' 又立刻 vi.unstubAllEnvs() 把它丢掉,等于把 unset 路径量了两次;Number('')0,会把这个旋钮本要设置的超时上限整个解除;${{ cond && 'x' || '' }} 也不是假想的写法——ci.yml 里的 QWEN_SKIP_LATENCY_BUDGETS 现在就是这样接线的;保留的三个 timeout: 30_000 也确实是 spawnSync 的边界,vitest 的异步超时对它不生效。改动范围很清晰,但 PR 正文缺少 PR 模板 的大部分必填部分,需要先补齐才能进入代码审查:

  • ## Why it's needed —— 动机目前写在自定义的 ## 1 — / ## 2 — 小节里,请归并到该必填标题下。
  • ## Reviewer Test Plan 及其子节 —— ### How to verify(双向验证的说明已经写好,只需放到该标题下,连同 "unit-vitest-configs.test.ts 只能在 CI 里跑" 的说明)、### Evidence (Before & After)(before 的 AssertionError: expected +0 to be 90000 / after 五组全部通过的输出)以及 ### Tested on 操作系统表格。
  • ## Risk & Scope —— ## Risk 已经覆盖了权衡,但模板要求三个要点;这里自然的"未验证 / 超出范围"一项正是你自己说的:目前没有任何 workflow 接线这个旋钮,且 pin 无法在 CI 之外运行。
  • ## Linked Issues —— #10870 / #10853 / #10858 目前散在正文里,请集中到该节(无需关闭关键词——这个后续 PR 本来就不关闭任何 issue)。
  • 双语摘要 —— 按模板要求提供 <details><summary>中文说明</summary> 的正文翻译。

无需改动代码 —— 补齐以上部分后,评论 @qwen-code /triage 即可重新进入 gate。

Qwen Code · qwen3.8-max

@yiliang114

Copy link
Copy Markdown
Collaborator Author

@qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run

Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check.

Scripted assertions: 78 passed · 0 failed · 78 total

Flakiness gate: ⚠️ timeout — only 2 of 5 rounds fit the 15-minute budget; the completed rounds agreed

中文 — 判定:✅ 通过 · 可合入(agent 判定)

沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查

脚本断言:78 通过 · 0 失败 · 78 总计

抖动门:⚠️ timeout — only 2 of 5 rounds fit the 15-minute budget; the completed rounds agreed

Verification report

PR 10910 — fix(ci): stop the scripts-suite timeout knob from failing open on an empty value

Verdict: merge-ready — 78/78 scripted assertions passed, 0 failed. Verified head: fea07732e07f5b38ae0be225560802ba002e22a6 (== git rev-parse HEAD^2; base tip 80b5dab3). Single-commit PR; the locally reachable commit set matches the metadata commits array (1 == 1).

中文摘要
  • 结论merge-ready。78/78 条脚本化断言通过,0 失败。
  • A/B 结论:中心主张成立。重写的 pin 在 head(||)配置上 27/27 全绿(01-pin-head-green-five-arms.png);把它放到 base(??)配置上,恰好在 stub="" 分支以 expected +0 to be 90000 失败、其余 26 条全过(02-pin-base-red-new-pin-fails-on-empty-string.png)——pin 非空转。旧 pin 在 base 上全绿(03-old-pin-green-on-broken-base.png),动态证明它当初抓不住这个 bug。表达式阶梯(04)与端到端探针(06)确认:''/'0'/空白/未设置/拼写错误在 head 全部落到 90000,且这些值在 base 会解除全部上限(vitest 把 0 读作无超时;NaN 经 worker 序列化变 null 后走同一条解除分支)。六处 }, 30000) 移除后,六个测试在 head 全部通过(最长 18.9s,05),全文件 229/229 绿。
  • Findings:1 条低严重度——|| 对负数与 Infinity 仍然放行(-5 || 90000 === -5,二者都落入 withTimeout 的解除分支),PR 自述「一切非正数写法回落默认值」对负数不成立;非回归(base 同样如此),输入域为仓库自管的 workflow 变量,负数不是现实拼写。附已度量的候选补丁(07,17/17:闭合两个残余且对所有合法输入与 head 逐字节一致)。不阻塞。
  • 未覆盖:无任何 workflow 接线该旋钮(潜伏缺陷,无法端到端);其余 scripts 测试文件未在 head 运行(未设置环境变量时两臂套件上限同为 90000,行为不变);六个测试的 base 臂未运行(本机上六者时长均 <30s,无可观察翻转);Windows/macOS 通道未测。

Central claim + A/B

Central claim: replacing ?? with || makes QWEN_SCRIPTS_TEST_TIMEOUT_MS fail-safe — every empty/invalid spelling ('' being the exact value this repo's ${{ cond && 'x' || '' }} idiom renders, e.g. ci.yml:723) lands on the 90 000 ms default instead of silently disarming the suite, while valid overrides still pass through.

The pin A/B drives the real config module through the lane's exact command (npx --no-install vitest run --config ./scripts/tests/vitest.config.ts …, mirrored from qwen-triage.yml:3594), with the head pin file copied into a HEAD^1 worktree for the control:

cell config pin result witness
M0 head (||) head, 5 arms 27/27 green (1 passed (1) files) 01-pin-head-green-five-arms.png
M1 base (??) head 1 failed | 26 passedAssertionError: stub="": expected +0 to be 90000 02-pin-base-red-new-pin-fails-on-empty-string.png
M2 base (??) base (old) 27/27 green — the old pin passed beside the broken expression 03-old-pin-green-on-broken-base.png

M1 is the load-bearing proof (new pin kills the reverted expression, failing the intended assertion with named expected/actual); M2 proves the old pin was vacuous — it stubbed '' then immediately vi.unstubAllEnvs()'d, so both arms measured the unset path (quoted from git show HEAD^1:…, lines 233-240).

Expression ladder (real config module per arm, 10 spellings × head/base, 04-expression-ladder-head-vs-base.png, 20/20):

arm head (||) base (??) reading
unset 90000 90000 unchanged
'' 90000 0 the fix: empty no longer disarms
' ' 90000 0 whitespace too
'abc' 90000 NaN typo no longer disarms
'0' 90000 0 explicit zero
'-5' -5 -5 residual, both arms (Finding 1)
'Infinity' Infinity Infinity residual, both arms (Finding 1)
'5000' / '5000 ' / '1e3' 5000 / 5000 / 1000 same valid overrides pass through, byte-identical

End-to-end probe (06-probe-zero-unbounded-cells.png, 10/10): a 7 s-sleep test run through the real suite config per arm proves the knob moves vitest's real timeout machinery, and that 0/NaN disarm it:

cell config knob outcome meaning
c1 head 3000 FAIL timed out in 3000ms knob drives a real ceiling
c2 head '' PASS (~7 s) '' lands on the 90 s default
c3 base '' PASS (~7 s) 0 = no timer (a 5 s default would have killed it)
c4 base 3000 FAIL timed out in 3000ms base knob worked for positives
c5 base abc PASS (~7 s) NaN disarms silently (see Finding 2/mechanism note)

Mechanism facts, all cited from the installed vitest 3.2.7: withTimeout guard if (timeout <= 0 || timeout === Number.POSITIVE_INFINITY) return fn (@vitest/runner/dist/chunk-hooks.js:1853); per-test override options?.timeout ?? runner.config.testTimeout (:606); stubEnv(name, undefined) deletes the variable and QWEN_SCRIPTS_TEST_TIMEOUT_MS is not in _envBooleans (vitest/dist/chunks/vi.bdSIJ99Y.js:3958,3811). The NaN cell's silent disarm rides a serialization hop: the worker receives task.timeout = null (debug cell logs/probe-c5-debug.log), and null <= 0 takes the same disarm branch — so the PR body's "a typo takes the same path through NaN" is accurate.

Secondary claim: the six ceiling removals

}, 30000) count: base 6 → head 0; the three remaining timeout: 30_000 are spawnSync option bounds, unchanged on both arms (one carries the why-comment). None of the six asserts a duration property. Full-file run at head (05-six-tests-durations-head.png): 229/229 passed; the six at 18.9 / 3.0 / 2.7 / 9.1 / 3.2 / 9.2 s — all under the old 30 s ceiling, so no base-vs-head flip is observable on this machine; the removal only widens headroom (30 s → 90 s suite ceiling via the :606 override rule). Corroboration of the premise: the two slowest tests in the same run, 43.8 s and 30.5 s (upserts deferred findings into a per-PR issue that survives the merge — the test that broke release run 33676423730, correctly not among the six), exceed 30 s and pass only because they already run on the suite ceiling.

Findings

F1 (low) — the || fix still fails open on negative values and Infinity. -5 || 90000 === -5 and Infinity || 90000 === Infinity; both satisfy withTimeout's disarm guard (<= 0 / === POSITIVE_INFINITY), so QWEN_SCRIPTS_TEST_TIMEOUT_MS=-5 silently removes every ceiling at head too. The PR's stated invariant "every non-positive spelling falls back to the default" is therefore false for negatives. Not a regression (base identical), and the input domain is repo-controlled — the ${{ }} idiom renders '' or a literal, never a negative — so this is a completeness nit, not a blocker. A measured candidate closes it with zero collateral (07-patch-candidate-closes-residual.png, 17/17): const raw = Number(process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS']); testTimeout: Number.isFinite(raw) && raw > 0 ? raw : 90_000 — all invalid/non-positive spellings → 90000, all valid overrides byte-identical to head.

F2 (note, not a defect) — NaN's disarm is silent, not loud. Reading withTimeout alone predicts an immediate setTimeout(fn, NaN) failure; the measured worker value is null (config serialization), which disarms quietly. The PR's claim holds; this note exists so a future reader who "fixes" the NaN path by only patching withTimeout misses the serialization hop.

Not covered

  • Workflow end-to-end wiring of the knob: nothing sets QWEN_SCRIPTS_TEST_TIMEOUT_MS (verified: zero hits under .github/), so the fault is latent by design and cannot be exercised E2E; coverage is at the config-module and runner level instead.
  • Other scripts/tests/* files at head: not run. The suite ceiling with the env unset is 90000 on both arms (identical), and the only per-test values that changed are the six, so their behavior is unchanged by this diff.
  • Base arm of the six de-ceilinged tests: not run; all six measured < 30 s at head here, so the base arm would pass on this machine and a flip is not observable (the 43.8 s / 30.5 s tests that do cross 30 s are outside the six).
  • Windows/macOS platform lanes: Linux container only.
  • Harness self-corrections during the round (all re-run; fragments reflect final runs): (a) first ladder run mis-encoded Infinity via JSON.stringify; (b) first M1/M2 runs died on a collection error — vite-plugin-dts lives in packages/webui/node_modules, absent in a fresh worktree — which my content checks caught before it could masquerade as an assertion; fixed by symlinking the 14 per-workspace node_modules (third-party deps only; package.json/lockfile untouched by the PR); (c) first c5 expectation assumed a loud NaN timeout, corrected after the task.timeout = null debug cell.

Methodology

Environment: node:22-bookworm container (Node v22.23.2), merge-ref checkout at depth 2 (HEAD = merge, HEAD^1 = 80b5dab3 base tip, HEAD^2 = fea07732 PR head), npm ci + npm run build pre-run by the workflow. Harnesses live in tmp/pr10910-verify-20260903-131722/harness/ (rerunnable .mjs), raw per-cell logs in logs/, captures in evidence/ via scripts/verify-capture.mjs. The base arm is a git worktree at HEAD^1 with per-workspace node_modules symlinked to the root install; the pin's import closure touches no @qwen-code/* packages (workspace configs import relative to the base tree; only third-party vitest/vite resolve at the root), so the control is clean. Assertion counts come solely from the fragment files the harnesses wrote after executing; expected reds (M1, c1, c4) are encoded as passing assertions.

Flakiness gate log

rounds=5 files=2 skipped=0
file scripts/tests/qwen-autofix-workflow.test.js: (cd .) npx --no-install vitest run --config ./scripts/tests/vitest.config.ts ./scripts/tests/qwen-autofix-workflow.test.js
file scripts/tests/unit-vitest-configs.test.ts: (cd .) npx --no-install vitest run --config ./scripts/tests/vitest.config.ts ./scripts/tests/unit-vitest-configs.test.ts


per-file results (P=pass F=fail I=infra-exit, one letter per run):
  scripts/tests/qwen-autofix-workflow.test.js: PPP
  scripts/tests/unit-vitest-configs.test.ts: PP

verdict: timeout
summary: only 2 of 5 rounds fit the 15-minute budget; the completed rounds agreed

--- per-invocation detail (full copy in the artifact) ---
round 1 · scripts/tests/qwen-autofix-workflow.test.js: P (exit 0)
round 1 · scripts/tests/unit-vitest-configs.test.ts: P (exit 0)
round 2 · scripts/tests/qwen-autofix-workflow.test.js: P (exit 0)
round 2 · scripts/tests/unit-vitest-configs.test.ts: P (exit 0)
round 3 · scripts/tests/qwen-autofix-workflow.test.js: P (exit 0)

Evidence images

01-pin-head-green-five-arms

02-pin-base-red-new-pin-fails-on-empty-string

03-old-pin-green-on-broken-base

04-expression-ladder-head-vs-base

05-six-tests-durations-head

06-probe-zero-unbounded-cells

07-patch-candidate-closes-residual

Harness scripts and raw logs are in the workflow run artifacts (7-day retention).

Qwen Code · sandboxed verification

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Third pass on this one. The gate result is unchanged — it goes through — and the only thing that has moved since the last run is the CI state, which belongs to Stage 2.

  • Template ✓ — every required heading is present: What this PR does, Why it's needed, the full Reviewer Test Plan (How to verify, Evidence (Before & After), the Tested on OS table, Environment), the three Risk & Scope bullets, Linked Issues, and the Chinese summary. The template bounce from the first pass is fully resolved, and no code changed to resolve it.
  • Problem: exists, and I re-checked it against main rather than the PR's word. scripts/tests/vitest.config.ts really does read Number(process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS'] ?? 90_000). The pin really does vi.stubEnv(..., '') and then vi.unstubAllEnvs() on the very next line, so both of its arms measure the unset path. And ci.yml:723 really does wire the sibling knob as ${{ startsWith(runner.name, 'ecs-qwen-') && '1' || '' }} — which renders exactly '' whenever the condition is false. A grep across .github/workflows/ finds nothing that sets QWEN_SCRIPTS_TEST_TIMEOUT_MS, so the fault is latent rather than live, precisely as the body claims.
  • Direction: aligned. CI-reliability follow-up to test: stop millisecond budgets from measuring the shared pool #10870 / Release Failed for v0.23.0 on 2026-09-02 #10853, test infrastructure only, no production surface and no public contract.
  • Size: not applicable — no core paths. 47 lines across 3 files, all under scripts/tests/.
  • Approach: minimal, with one honest question. Number(env) || default is the right shape, and removing six ceilings that shadow a later suite-wide ceiling can only relax those cases. The question is about completeness, not correctness: the file still carries three per-test ceilings tighter than the six being removed — }, 20000) at line 1552, whose own comment says "Spawn-heavy: each run() forks bash + a stubbed gh", and }, 10000) at 11837 and 21800, both wrapping runDevelopIssue. They shadow the 90s suite ceiling harder than the 30s ones did. Leaving them is defensible as minimal scope, but the body's framing of the six as "the remainder" slightly overstates it — worth either naming the narrower principle or filing a follow-up. Not a blocker.
  • Risk: no elevated signals — the Stage 1e path check returns no match. All three files are test-lane files; nothing on the high-risk list.

Moving on to code review. 🔍

中文说明

这是本 PR 的第三轮。门禁结论不变——通过——自上一轮以来唯一变化的是 CI 状态,那属于 Stage 2。

  • 模板 ✓ —— 所有必填标题齐备:What this PR doesWhy it's needed、完整的 Reviewer Test PlanHow to verifyEvidence (Before & After)Tested on 操作系统表格、Environment)、Risk & Scope 三条要点、Linked Issues 以及中文说明。第一轮的模板打回已完全解决,且解决它并未改动代码。
  • 问题:真实存在,且我是对照 main 而非仅凭 PR 的说法重新核对的。 scripts/tests/vitest.config.ts 确实写作 Number(process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS'] ?? 90_000);那条 pin 确实先 vi.stubEnv(..., '')、紧接着下一行就 vi.unstubAllEnvs(),因此它的两个分支量的都是未设置路径;ci.yml:723 也确实把相邻旋钮接线为 ${{ startsWith(runner.name, 'ecs-qwen-') && '1' || '' }}——条件为假时渲染出的正是 ''。在 .github/workflows/ 全量 grep 后,没有任何 workflow 设置 QWEN_SCRIPTS_TEST_TIMEOUT_MS,所以该缺陷是潜伏而非活跃的,与正文所述完全一致。
  • 方向:对齐。 test: stop millisecond budgets from measuring the shared pool #10870 / Release Failed for v0.23.0 on 2026-09-02 #10853 的 CI 可靠性后续,仅涉及测试基础设施,不触及生产面与对外契约。
  • 规模:不适用 —— 未触及核心路径;3 个文件共 47 行,全部在 scripts/tests/ 下。
  • 方案:最小,但有一个如实的疑问。 Number(env) || default 是正确形态,移除六处遮蔽了后来的套件级上限的 per-test 上限,只会放宽这些用例。疑问在于完整性而非正确性:文件中仍保留三处比被移除的六处更紧的 per-test 上限——第 1552 行的 }, 20000)(它自己的注释写着「Spawn-heavy: each run() forks bash + a stubbed gh」),以及 11837 与 21800 行的 }, 10000)(两处都包着 runDevelopIssue)。它们对 90 秒套件上限的遮蔽比那六处 30 秒更严重。作为最小范围保留它们说得通,但正文把这六处称为「余下的部分」略微夸大了完整性——建议要么点明更窄的原则,要么另开后续 PR。不阻塞。
  • 风险:无升级信号 —— Stage 1e 路径检查无命中。三个文件都是测试通道文件,不在高风险清单上。

进入代码审查。🔍

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at fea07732e07f5b38ae0be225560802ba002e22a6 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Code review

My independent proposal for this problem had the same shape: write the knob so no non-positive spelling can disarm it, and fix the pin so the empty arm is actually asserted instead of being stubbed and thrown away one line later. Comparing that against the diff is short — this is the minimal fix, and there are no drive-by edits anywhere in it.

What I verified against the base tree, not the PR's description:

  • The config change covers every arm. Number(process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS']) || 90_000: unset → NaN → 90_000; ''0 → 90_000; 'abc'NaN → 90_000; '0'0 → 90_000; '5000' → 5_000. The only semantics surrendered is "pass 0 to mean no timeout", and nothing passes it — no workflow sets the variable at all.
  • The pin rewrite is substantive, not cosmetic. vi.stubEnv(k, undefined) is the established way to delete a variable in this repo (70 existing call sites), so the unset arm now genuinely measures the unset path instead of stubbing '' and immediately unstubbing it. The five arms are five distinct values, and the new '' → 90_000 assertion is exactly the one that fails against the expression being replaced — which is what makes the pin load-bearing rather than decorative. The label change also stops interpolating a bare undefined into the failure message.
  • All six ceilings go, and only those six. The base file has exactly six }, 30000) per-test ceilings and the diff removes all six; each ends a subprocess-spawning test with a string or boolean assertion, and none asserts a duration property. The three surviving timeout: 30_000 values (lines 14177, 24565, 24808) are spawnSync options and are correctly left alone — I read the first one, and its own comment gives the reason: spawnSync blocks the event loop, so vitest's async timeout cannot fire there. Every affected case now inherits the 90s suite ceiling, which is strictly more headroom than the 30s it shadowed, so this half of the diff cannot introduce a new timeout failure.
  • No production code touched, no assertion weakened, no test deleted.

Two non-blocking notes:

  1. || still lets negatives and Infinity through. Number('-5') || 90_000 is -5 and Number('Infinity') || 90_000 is Infinity — both truthy, and vitest reads both as "no timeout". So the body's claim that "every non-positive spelling falls back to the default" is not literally true. The code comment is accurate — it claims only '' and NaN. This is not a regression (base behaves identically) and the input domain is a repo-controlled workflow variable, so a negative is not a realistic spelling; the sandboxed lane measured the same residual and also called it non-blocking. Worth one word in the body, not a code change.
  2. Three tighter per-test ceilings survive — line 1552 at 20s, and 11837 and 21800 at 10s — all on subprocess-spawning tests, all shadowing the 90s suite ceiling harder than the six being removed. Raised in Stage 1; repeating it here so it does not get lost between comments. Deferring to a follow-up is a fine answer.

No sequence diagram or files table: this is three test-lane files with one behavioural line between them, and a diagram would be noise.

Testing evidence

This is a CI run, so per the gate rules I built and executed nothing of the PR's. What follows is the PR's own CI fetched through the API for fea0773, plus the sandboxed lane that already reported on this thread. I have not re-run either.

The headline is uncomfortable but is not this PR's fault: Test (ubuntu-latest, Node 22.x) was cancelled, and it never reached the lane this PR changes. The job started 11:28:20Z, spent 22m44s in Install dependencies — the workflow's own budget comment assumes the test step is entered "at around minute 18" — started Run tests and generate reports at 11:51:33Z, and was killed at 13:29:23Z, 121 minutes into a 120-minute job budget. The runner's orphan teardown names what was still alive: npm run test:ci:workspaces --retry=2. Since test:ci is test:ci:workspaces && test:scripts, the changed config and the rewritten pin were never executed by CI on this head. test:scripts appears in the 2.8 MB job log exactly once, as echoed workflow source at line 1319, never as a run.

I classify that as pre-existing infrastructure contention, not a PR defect, on three pieces of evidence: this diff touches only scripts/tests/*, which lives in the phase the job never entered; the overrun is in the install phase and the workspaces suite, neither of which the diff touches; and the repo's CI is unhealthy independent of this PR — push main run 33797332027 (19:35Z today) is red on both Test (ubuntu-latest, Node 22.x) and Lint & Static. The comment sitting directly above that timeout-minutes: 110 in ci.yml documents this exact failure shape from #10490: "A hung suite used to burn the whole job budget … which reads as a timeout rather than a failure."

So the CI leg cannot substantiate this change, and the evidence that does is the sandboxed /verify lane which reported at 12:04 on this thread — advisory, not a CI check, but it drove the real config module through the lane's own command and A/B'd both halves:

  • head config (||) + rewritten pin → 27/27 green
  • base config (??) + rewritten pin → fails on exactly stub="" with expected +0 to be 90000, so the pin kills the expression it replaces
  • base config + the old pin → 27/27 green, which is the dynamic proof the old pin was vacuous
  • a 7s-sleep test through the real suite config: head with '' passes on the 90s default, and base with '' also passes — proving 0 removed the timer outright, since a 5s default would have killed it
  • the six ceiling removals: all six pass on head, longest 18.9s, full file 229/229 green

78/78 scripted assertions, agent verdict merge-ready. For the coercion claim that is stronger than a green CI leg would have been, because it compares the pin against both expressions rather than only the new one.

Not verified: test:scripts executing on this head under the repo's own CI — the leg was cancelled before it got there, and nothing since has re-run it. Also not verified by me firsthand: vitest's withTimeout treating <= 0 as unbounded, because this checkout has no node_modules to read; I am taking that from the sandboxed probe's end-to-end cell above, which demonstrates it behaviourally.

Final CI results for fea0773 (auto-updated by the triage finalize job after CI completed):

Check Conclusion
Test (ubuntu-latest, Node 22.x) ❌ failure
web-shell E2E Smoke (ubuntu-latest, Node 22.x) 🚫 cancelled
Classify PR ✅ success
Dependency CVE audit ✅ success
Desktop Shell (ubuntu-22.04) ✅ success
Desktop Shell (windows-2022) ✅ success
Integration Tests (no-AK, No Sandbox) ✅ success
Lint & Static (ubuntu-latest, Node 22.x) ✅ success
Secret scan (TruffleHog) ✅ success

One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。

Sandboxed verification would settle the behavioural claim, and it already has: @qwen-code /verify ran on this head and A/B-proved that the rewritten pin fails against the ?? expression and passes against the || one — the specific claim a green suite alone could not establish, since a suite that passes identically with and without the diff proves nothing here. A second /verify run is in flight as of this pass. What no sandboxed lane can supply is the repo's own CI executing test:scripts on this head; that needs the cancelled Qwen Code CI run 33749822522 re-run by a maintainer, and with main red on the same legs right now it may not land green on the first attempt.

中文说明

代码审查

我对这个问题的独立方案与本 PR 形态一致:把旋钮写成任何非正数写法都无法解除上限的形式,并修正 pin,让空值分支真正被断言、而不是 stub 之后下一行就被丢弃。因此与 diff 的对比很短——这就是最小修复,全篇没有顺手改动。

以下是对照基线树(而非 PR 的描述)核对的结果:

  • 配置改动覆盖所有分支。 Number(process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS']) || 90_000:未设置 → NaN → 90_000;''0 → 90_000;'abc'NaN → 90_000;'0'0 → 90_000;'5000' → 5_000。唯一放弃的语义是「传 0 表示不设超时」,而没有任何调用方传它——没有 workflow 设置这个变量。
  • pin 的重写是实质性的,不是修饰性的。 vi.stubEnv(k, undefined) 是本仓库删除环境变量的既有写法(现有 70 处调用),因此 unset 分支现在真正量到未设置路径,而不是先 stub '' 又立刻丢弃。五个分支是五个不同取值,而新增的 '' → 90_000 断言恰恰是对被替换表达式会失败的那一条——这正是它钉住旋钮、而非空转的原因。标签的改动也避免了把裸 undefined 插进失败信息。
  • 六处上限全部移除,且只移除这六处。 基线文件中恰好有六处 }, 30000) per-test 上限,diff 全部移除;每一处都是以字符串或布尔断言结束的 spawn 子进程测试,没有一处把时长当作被测属性。保留的三个 timeout: 30_000(14177、24565、24808 行)是 spawnSync 的选项,处理正确——我读了第一处,它自己的注释就写明理由:spawnSync 阻塞事件循环,vitest 的异步超时在那里无法触发。受影响的用例现在继承 90 秒套件上限,比被遮蔽的 30 秒只有更宽,所以 diff 的这一半不可能引入新的超时失败。
  • 未触及生产代码,未削弱任何断言,未删除任何测试。

两条不阻塞的提醒:

  1. || 仍然放行负数与 Infinity Number('-5') || 90_000-5Number('Infinity') || 90_000Infinity——两者都为真值,而 vitest 把两者都读作「不设超时」。所以正文「一切非正数写法都回落默认值」的说法并不字面成立。代码注释是准确的——它只声称 ''NaN。这不是回归(基线行为相同),且输入域是仓库自管的 workflow 变量,负数不是现实拼写;沙箱通道量到了同样的残余,也判为不阻塞。建议在正文里补一句话,不需要改代码。
  2. 三处更紧的 per-test 上限被保留——1552 行的 20 秒、11837 与 21800 行的 10 秒——都在 spawn 子进程的测试上,对 90 秒套件上限的遮蔽比被移除的六处更严重。Stage 1 已提出,此处重复以免在两条评论之间丢失。留到后续 PR 处理完全可以。

未提供时序图或文件总览表:这是三个测试通道文件、其间只有一行行为改动,加图只会是噪音。

测试证据

这是 CI 运行,按门禁规则我没有构建、也没有执行 PR 的任何代码。以下是通过 API 获取的 fea0773 自身 CI 结果,以及已在本线程发布过报告的沙箱通道;两者我都没有重跑。

结论不太好看,但不是这个 PR 造成的:Test (ubuntu-latest, Node 22.x) 被取消,而且它从未跑到本 PR 改动的那条通道。 该 job 于 11:28:20Z 开始,Install dependencies 花了 22 分 44 秒——workflow 自己的预算注释假设测试步骤「大约在第 18 分钟」进入——Run tests and generate reports 于 11:51:33Z 开始,并在 13:29:23Z 被终止,即 120 分钟 job 预算的第 121 分钟。runner 的孤儿进程清理点名了仍存活的进程:npm run test:ci:workspaces --retry=2。由于 test:citest:ci:workspaces && test:scripts本 head 上被改动的配置与重写后的 pin 从未被 CI 执行过。 test:scripts 在 2.8 MB 的 job 日志中只出现一次,即第 1319 行回显的 workflow 源码,从未作为一次运行出现。

我把它归类为既有的基础设施争抢、而非 PR 缺陷,依据三条证据:本 diff 只触及 scripts/tests/*,而那属于 job 从未进入的阶段;超时发生在安装阶段与 workspaces 套件,两者都不是本 diff 触及的;并且本仓库的 CI 在此 PR 之外本就不健康——今天 19:35Z 的 push main run 33797332027 在 Test (ubuntu-latest, Node 22.x)Lint & Static 两条腿上都是红的。ci.yml 中紧挨着那个 timeout-minutes: 110 的注释就记录了 #10490 的同一种失败形态:「一个卡住的套件过去会烧掉整个 job 预算……这读起来像超时而不是失败。」

因此 CI 这条腿无法为本改动提供实质验证,真正提供验证的是 12:04 在本线程发布报告的沙箱 /verify 通道——它是参考证据、不是 CI 检查,但它用该通道自身的命令驱动了真实的配置模块,并对两半改动做了 A/B:

  • head 配置(||)+ 重写后的 pin → 27/27 全绿
  • base 配置(??)+ 重写后的 pin → 恰好在 stub="" 上以 expected +0 to be 90000 失败,即该 pin 能杀掉它所替换的表达式
  • base 配置 + pin → 27/27 全绿,这是旧 pin 空转的动态证明
  • 一个 7 秒 sleep 测试跑过真实套件配置:head 配 '' 在 90 秒默认值下通过,base 配 '' 也通过——证明 0 把计时器整个移除了,否则 5 秒默认值早该杀掉它
  • 六处上限移除:六个用例在 head 全部通过,最长 18.9 秒,全文件 229/229 绿

脚本断言 78/78,agent 判定 merge-ready。就强制转换这一主张而言,这比一条绿色 CI 腿更强,因为它把 pin 与两个表达式都做了对照,而不只对照新的那个。

未验证:本 head 上由仓库自身 CI 执行 test:scripts——该腿在跑到之前就被取消,此后没有任何重跑。另外我未亲自验证 vitest 的 withTimeout<= 0 视为无上限,因为这个 checkout 没有 node_modules 可读;这一点我取自上面沙箱探针的端到端单元,它是以行为方式证明的。

沙箱验证能够判定这条行为主张,而且已经判定了:@qwen-code /verify 已在本 head 上运行,并 A/B 证明了重写后的 pin 对 ?? 表达式失败、对 || 表达式通过——这正是单靠一条绿色套件无法确立的具体主张,因为一个加不加 diff 都同样通过的套件在这里什么也证明不了。截至本轮,第二次 /verify 运行正在进行中。任何沙箱通道都无法提供的是:仓库自身 CI 在本 head 上执行 test:scripts;那需要维护者重跑被取消的 Qwen Code CI run 33749822522,而在 main 当前同样两条腿为红的情况下,第一次尝试未必能变绿。

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at fea07732e07f5b38ae0be225560802ba002e22a6 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Confidence: 4/5 — the code is the minimal correct shape and every load-bearing claim survived independent checking; the missing point is that the repo's own CI never executed the lane, and two non-blocking notes are open.

My independent proposal for this problem was the same one that landed, which usually means there is no simpler path being missed. The part that earns real credit is still the pin rewrite: a one-character || fix on its own would have left the "test" measuring the unset path twice, and the rewritten pin fails against the expression it replaces — so the suite now pins the knob instead of passing beside it. That is the difference between a fix and a fix with a guard, and it is why I am comfortable with the second half of the diff too: removing six per-test ceilings that shadow a later suite-wide ceiling can only widen them, and the three spawnSync bounds that genuinely need to stay are correctly left in place with their own reason recorded.

Back over the gate questions: the problem was demonstrated, not inferred — I confirmed the ?? expression, the stub-then-unstub pin, and the live ${{ … && '1' || '' }} wiring idiom in ci.yml against main myself. The direction is a straightforward reliability follow-up. Every line in the diff serves the stated goal. Six months from now this reads as three files that each explain why at the exact point it is non-obvious — I would thank the author, not curse them. The first pass's request-changes was template-only and the rewritten body resolves it completely, so nothing substantive is outstanding from that round.

The one real reservation is the testing hole, and I want to be blunt about it rather than paper over it: Test (ubuntu-latest, Node 22.x) was cancelled at the job's 120-minute budget while still inside test:ci:workspaces, so test:scripts — the only lane that runs the changed config and the rewritten pin — never executed on this head. I am approving anyway, for three reasons. The cause is diagnosed and is not this diff: the overrun is in the install phase and the workspaces suite, and main itself is red on the same legs as of this afternoon. The behavioural claim does not rest on the author's word: the sandboxed lane A/B'd the pin against both expressions and got the failure on exactly the arm the old pin discarded. And an approval here is a review vote, not a merge — the cancelled leg still has to land green before this can go in, and my approving does not change that. What I am not claiming is that CI validated this change, because it did not get the chance.

Two things worth a maintainer's attention, neither blocking. First, the body's "every non-positive spelling falls back to the default" overstates it — || still passes -5 and Infinity through, exactly as it did before, and the code comment is more careful than the prose. Second, this PR is one of a cluster of open CI-timeout PRs from the same author (#10915, #10921, #10912, #10909, #10917 among them), and #10921 — a 60-minute budget for the ECS resolve wait — addresses precisely the install-phase overrun that killed this PR's Test leg. Landing these as a coherent set may do more for the shared pool than any one of them alone, and it may also be what finally gets this leg green.

No deferred-approval instruction is carried by this comment: nothing is in flight on this head (PENDING = 0), so the approval is posted now, pinned to the reviewed commit rather than to the branch.

中文说明

置信度:4/5 —— 代码是最小的正确形态,每一项关键论断都经受住了独立核对;扣掉的一分在于仓库自身的 CI 从未执行到这条通道,且还有两条不阻塞的提醒未处理。

我对这个问题的独立方案与最终落地的一致,这通常意味着没有更简单的路径被遗漏。真正值得肯定的仍是 pin 的重写:只改一个 || 字符的话,那条「测试」仍会把 unset 路径量两次;而重写后的 pin 对它所替换的表达式会失败——套件因此真正钉住了这个旋钮,而不是在它旁边通过。这就是「修复」与「带守卫的修复」之间的差别,也是我对 diff 后半部分放心的原因:移除六处遮蔽了后来的套件级上限的 per-test 上限只会放宽它们,而三个确实需要保留的 spawnSync 边界被正确留下、并且各自记录了理由。

再过一遍门禁问题:问题是被论证的,不是推断的——我亲自对照 main 确认了 ?? 表达式、先 stub 后 unstub 的 pin、以及 ci.yml 中活着的 ${{ … && '1' || '' }} 接线惯用法。方向是一个直接的可靠性后续。diff 中每一行都服务于既定目标。六个月后再读,这是三个文件,各自恰好在 why 不显而易见处解释了 why——我会感谢作者,而不是骂他。第一轮的 request-changes 仅是模板问题,重写后的正文已完全解决,因此那一轮没有留下任何实质未决项。

唯一的实质保留是测试上的空洞,我想直说而不是掩盖:Test (ubuntu-latest, Node 22.x) 在 job 的 120 分钟预算处被取消,当时仍在 test:ci:workspaces 内部,所以 test:scripts——唯一会运行被改动配置与重写后 pin 的通道——在本 head 上从未执行。我仍然批准,理由有三条。原因已定位且与本 diff 无关:超时发生在安装阶段与 workspaces 套件,而 main 自身今天下午在同样的腿上也是红的。行为主张并不依赖作者的说法:沙箱通道把 pin 与两个表达式都做了 A/B,并恰恰在旧 pin 丢弃掉的那个分支上得到了失败。而这里的批准是一次评审投票,不是合并——被取消的那条腿在合入前仍须变绿,我的批准并不改变这一点。我声称的是 CI 验证了这个改动,因为它没有获得这个机会。

两件值得维护者注意的事,均不阻塞。其一,正文「一切非正数写法都回落默认值」说得过头了——|| 仍然放行 -5Infinity,与改动前完全相同,代码注释比正文更谨慎。其二,本 PR 是同一作者一组未合入的 CI 超时 PR 之一(其中包括 #10915#10921#10912#10909#10917),而 #10921——给 ECS resolve 等待一个 60 分钟预算——处理的正是杀掉本 PR Test 腿的那个安装阶段超时。把它们作为一组连贯地合入,对共享资源池的作用可能大于其中任何一个单独合入,也可能正是让这条腿终于变绿的东西。

本条评论不携带任何延迟批准指令:本 head 上没有正在运行的 CI(PENDING = 0),因此批准现在就发出,并钉在所评审的提交上、而不是分支上。

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at fea07732e07f5b38ae0be225560802ba002e22a6 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Triage re-run completed without a new review.

The bot already has a review of its own on fea07732e07f5b38ae0be225560802ba002e22a6, which still stands.

机器人在 fea07732e07f5b38ae0be225560802ba002e22a6 上已有自己的评审,且仍然有效。

The stage comments above were updated with the latest result. View workflow run.

上方各阶段评论已更新为最新结果。查看工作流运行

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Downgraded from Approve to Comment: CI still running. Reviewed.

— qwen3.8-max via Qwen Code /review (v0.22.3)

@yiliang114

Copy link
Copy Markdown
Collaborator Author

@qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run

Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check.

Scripted assertions: 167 passed · 0 failed · 167 total

Flakiness gate: ✅ 2 changed test file(s) x 5 identical rounds, no divergence

中文 — 判定:❌ 不通过 · 报告了发现(agent 判定)

沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查

脚本断言:167 通过 · 0 失败 · 167 总计

抖动门:✅ 2 changed test file(s) x 5 identical rounds, no divergence

Verification report

PR 10910 — fix(ci): stop the scripts-suite timeout knob from failing open on an empty value

Verdict: findings — 167/167 scripted assertions passed, 0 failed. Verified head: fea07732e07f5b38ae0be225560802ba002e22a6 (== git rev-parse HEAD^2; base tip HEAD^1 = 80b5dab3e06543928500081fab6f4b6ff73731d3). Single-commit PR; locally reachable git rev-list HEAD^1..HEAD^2 = 1 commit, matching the metadata commits array (1 == 1) despite the depth-2 shallow checkout.

No finding here blocks merge, and none is a regression. The central claim is proven load-bearing and the change is a strict improvement. The verdict is findings rather than merge-ready because two statements the PR's own risk assessment rests on are measurably false, and because the criterion the PR states was applied to 6 of 42 candidate sites — including three tighter ones left behind in the very file it edited.

中文摘要
  • 结论findings(有值得评审者注意的具体问题,但均不阻塞合入、均非回归)。167/167 条脚本化断言通过,0 失败。
  • 本轮性质:这是跟进轮,但 PR head 与 base 与上一轮逐字节相同(head fea07732、base 80b5dab3、tree b4b98e83),delta 为空。因此本轮不复述旧报告,而是独立重测全部旧测量,并把预算投入上一轮明确列为「未覆盖」的面。
  • A/B 结论:中心主张成立,且比上一轮更强。本轮把 pin 矩阵补成完整 2×2(新增 M3:旧 pin 配 head 配置)——旧 pin 在两列都绿,证明它对 ??/|| 毫无判别力;只有新 pin 能区分(head 绿 / base 恰好在 stub="" 分支以 expected +0 to be 90000 红,27 条中 1 红 26 绿,无附带损伤)。表达式阶梯 21 种拼写 × 两臂,并用真实进程环境变量(CI 的实际路径)交叉印证 12 个单元,两种机制逐一相符。端到端探针证明 ''/abc 在 base 会解除上限(6s 睡眠通过,而 vitest 默认 5s 本应杀掉它),在 head 落到 90000。
  • Findings:F1(低)|| 对负数与 Infinity 仍然放行——本轮端到端实测:同一 95s 睡眠探针在 head 配 -5 跑满 95034ms 通过,而对照组(未设置)在 90000ms 被杀;已度量的候选补丁 36/36 闭合该残余,对 7 种合法取值逐字节一致,恰好只改动 4 种拼写。C1/C2 是对 PR 描述的更正(见下)。F2(低,完备性)仍有 42 处低于 90s 的 vitest 超时覆盖散落在 8 个文件,其中 3 处就在 PR 改的同一文件里且更紧(10s/10s/20s);install-script.test.js:63 用文件级 vi.setConfig({testTimeout:30_000}) 把一个 5321 行、63 处 spawn 的文件整体钉在 30s——实测证明文件级 vi.setConfig 确实遮蔽套件上限。F3(低)vitest 上限无法打断阻塞式 spawnSync(实测:3s 上限下 12s 的 spawnSync 跑满,无限阻塞只能被外部 timeout 杀掉,exit 124),这界定了 PR 理由所声称的保护范围;仓库自己的注释(qwen-autofix-workflow.test.js:14175test-setup.ts:29-31)独立印证了同一机制。F4(注)hookTimeout 未设置,仍是 vitest 默认 10s,比被抬到 90s 的 testTimeout 紧 9 倍;workflow-size.test.js:657beforeAll 在该 10s 内串行 spawn 6 次 git,而仓库另有 4 个 config 专门在 ecs-qwen- 分支把 hookTimeout 抬到 60s 并有 pin 测试守着。
  • 未覆盖:旋钮无任何 workflow 接线(普查确认:全仓仅 2 处引用,.github/docs/ 各 0 处),故无法端到端;install-script.test.js 因本容器缺 zip 而无法运行(已用 A/A 对照证明 base 与 head 同样失败);未重跑抖动门;Windows/macOS 通道未测。

Previous-finding status (follow-up round)

The head under test is byte-identical to the previous round's. HEAD^2 = fea07732e07f5b38ae0be225560802ba002e22a6 and HEAD^1 = 80b5dab3e06543928500081fab6f4b6ff73731d3 both match the previous report's verified head and base tip, and git diff HEAD^2..HEAD -- scripts/tests/ is empty (the merge commit adds nothing over the PR head). The delta since that round is therefore empty, and the whole input closure — not just one file's hash, but the entire tree at the same commit, with no package.json/lockfile change — is unchanged. Rather than quote that as licence to skip, every carried-forward measurement below was rebuilt and re-run at this head; new probes were scoped to what the previous round listed as not covered.

# previous finding severity status at this head
1 F1 — || still fails open on negative values and Infinity low stands, and is now measured rather than inferred. Previous evidence was the config value (-5 || 90000 === -5) plus the vitest guard. This round drives it end-to-end: a 95 s probe with knob -5 ran to completion (95034 ms, PASS) while the control with the knob unset was killed at exactly 90000 ms — since 95 s > 90 s, passing cannot be a fallback. Infinity behaves identically.
2 F2 — NaN's disarm is silent, not loud (note) note stands. Re-confirmed the shipped guard verbatim: if (timeout <= 0 || timeout === Number.POSITIVE_INFINITY) { return fn; } at node_modules/@vitest/runner/dist/chunk-hooks.js:1853. Probe cell c5 (base, abc, 6 s sleep) passes, i.e. NaN disarms quietly.
3 "the two slowest tests in the same run, 43.8 s and 30.5 s … exceed 30 s and pass only because they already run on the suite ceiling" (premise corroboration) does not reproduce on this container. The same test on the same commit, upserts deferred findings into a per-PR issue that survives the merge, measures 9.5 s single-file and 9.0 s in the full suite; the slowest test in the whole file is 9.5 s, and no test in scripts/tests/** exceeded 30 s idle here (0 of 1998). This is host-speed variance, not an error in either round — and the 9.5 s ↔ 43.8 s spread for one test at one commit is itself evidence for the contention premise the 90 s ceiling exists to absorb. The premise therefore stands on the config comment's own release-run history, not on this container's numbers.
4 Flakiness gate ⚠️ timeout (2 of 5 rounds) not re-run. Recorded under Not covered. I ran the full suite once and the de-ceilinged file twice instead; both agreed (see the duration table).

Central claim + A/B

Central claim: replacing ?? with || makes QWEN_SCRIPTS_TEST_TIMEOUT_MS fail-safe — every empty/invalid spelling lands on the 90 000 ms default instead of silently disarming the suite, while valid overrides still pass through.

The A/B drives the real config module through the lane's exact command (npx --no-install vitest run --config ./scripts/tests/vitest.config.ts …, mirrored from qwen-triage.yml:3594 and package.json:52), in two scratch worktrees at HEAD^1 and HEAD^2. Witness: 01-pin-ab-2x2-matrix.png.

cell config pin vitest Tests line exit
M0 head (||) head, 5 arms 27 passed (27) 0
M1 base (??) head 1 failed | 26 passed (27)AssertionError: stub="": expected +0 to be 90000 1
M2 base (??) base (old) 27 passed (27) 0
M3 head (||) base (old) 27 passed (27) 0

M3 is the cell this round adds. The previous round ran M0/M1/M2. Completing the square is what upgrades the vacuity proof from "the old pin passed beside the broken expression" to "the old pin is green in both columns, so it discriminates nothing" — and the new pin is the only one of the four combinations that separates head from base. Per the rule that every control runs on both arms, M3 was not looking for a bug; it closed the matrix.

M1 is the load-bearing proof, and it fails the intended assertion with named expected/actual at unit-vitest-configs.test.ts:252, inside describe('scripts suite timeout') — not an import or fixture break. Exactly 1 of 27 tests goes red, so the one-line config change has no collateral in that file.

Expression ladder — real config module, 21 spellings × both arms, cross-checked against a second mechanism (witness 02-expression-ladder-head-vs-base.png, 54/54):

spelling head (||) base (??)
unset 90000 90000 unchanged when unwired
'' 90000 0 the fix
' ' 90000 0 whitespace too
'0' / '-0' / '0.0' 90000 0 / -0 / 0 explicit zeros
'abc' / 'null' / 'undefined' / 'NaN' 90000 NaN typos
'-5' / '-1' -5 / -1 same residual, both arms → F1
'Infinity' / '-Infinity' Infinity / -Infinity same residual, both arms → F1
'5000' / '5000 ' / ' 5000' / '1e3' / '0x10' / '1500.5' 5000 / 5000 / 5000 / 1000 / 16 / 1500.5 identical valid overrides pass through byte-for-byte

The ladder is corroborated by a mechanism that does not use the ladder's own instrument: for 6 spellings per arm the variable was set in the real parent process environment (the path a workflow's env: takes) and the config imported statically. All 12 corroboration cells agree with the in-worker values. Values are formatted by hand, not JSON.stringify — which turns both NaN and Infinity into null and would have collapsed the two residuals F1 depends on.

End-to-end probe — a sleeping test through the real suite config, knob set in the real process env (witness 03-probe-end-to-end-timeout-cells.png, 12/12):

cell arm knob sleep outcome meaning
c1 head 3000 6 s FAIL timed out in 3000ms the knob drives a real ceiling
c2 head '' 6 s PASS (6006 ms) '' lands on the 90 s default
c3 base '' 6 s PASS (6006 ms) 0 = no timer; vitest's 5 s default would have killed it
c4 base 3000 6 s FAIL timed out in 3000ms base knob worked for positives
c5 base abc 6 s PASS NaN disarms silently
c6 head abc 6 s PASS typo falls back
c8 head unset 95 s FAIL timed out in 90000ms (wall 91.2 s) control: the 90 s ceiling is real and 95 s exceeds it
c7 head -5 95 s PASS (slept 95034 ms) F1: -5 disarms, end-to-end
c9 head Infinity 95 s PASS (slept 95035 ms) F1: Infinity disarms, end-to-end

Secondary claim: the six ceiling removals

}, 30000) in qwen-autofix-workflow.test.js: base 6 → head 0 (scripted). All six were located in a real run and all six pass at head. Durations measured by two independent instruments that agree within 1.09× on every one (witness 06-aa-control-and-six-de-ceilinged-durations.png, 14/14):

test single-file full-suite ratio
behaviorally replays the stale-duplicate revalidation… 5.2 s 5.0 s 1.03
posts a takeover milestone digest as rounds accumulate… 0.7 s 0.7 s 1.09
surfaces deny-by-default footprint expansions… 0.5 s 0.5 s 1.08
keeps the round status comment live with a heartbeat… 7.8 s 7.8 s 1.00
flags recoverable API renders without a leading status code… 0.3 s 0.3 s 1.08
classifies an unchanged branch by its verdict files… 0.8 s 0.7 s 1.09

The removal is load-bearing, not cosmetic. At the contention factor the config's own comment states ("on the shared pool the same work runs about 5x slower"), the slowest of the six is 7.8 s × 5 = 39.1 s — which breaks the old flat 30 s per-test ceiling it used to carry and fits the new 90 s suite ceiling it now inherits. That is the second half's justification, measured rather than argued.

All six do spawn subprocesses, confirming that part of the PR's characterisation (two of them only via local helpers — runGate and runAutofixRunner/runDevelopIssue, both traced to spawnSync; a first pass with a naive regex wrongly reported them as non-spawning).

Corrections to the PR description

These are corrections to the description, not requests to change the code. Leaving them standing costs the next reader more than the original findings do, because the PR's own risk assessment is built on them.

C1 — "every non-positive spelling falls back to the default" is false for negatives. -5 is non-positive and -5 || 90000 === -5. Measured: the ladder shows '-5'-5 and '-1'-1 at head, and probe cell c7 shows a 95 s test running to completion under knob -5 where the unset control is killed at 90000 ms. The claim holds for '', whitespace, '0', '-0', '0.0' and every non-numeric spelling — i.e. for every value the ${{ }} idiom can actually render — so the fix is right; the stated invariant is wider than the code.

C2 — "None of the six asserts a duration property" is false for one of the six. keeps the round status comment live with a heartbeat and a job deep link (base lines 16436–17516, the slowest of the six at 7.8 s) contains five real wall-clock duration assertions across four measured subprocess arms, with no fake timers anywhere in the test (useFakeTimers/advanceTimersByTime/setSystemTime: 0 hits). A local runDrainArm helper (line 17376) times a spawnSync('bash', …) with Date.now() and returns elapsedMs, then:

expect(fresh.elapsedMs).toBeGreaterThanOrEqual(800);   // 17397
expect(fresh.elapsedMs).toBeLessThan(10000);           // 17398
expect(aged.elapsedMs).toBeLessThan(4000);             // 17403
expect(fifo.elapsedMs).toBeLessThan(15000);            // 17407
expect(endless.elapsedMs).toBeLessThan(10000);         // 17420

The >= 800 lower bound is load-bearing by design — the comment at 17369 explains it witnesses that at least one sleep 1 in the drain loop really elapsed. So this test does assert a duration property, deliberately.

Two consequences worth the reviewer's attention, neither a defect in the diff:

  • Removing the ceiling is still safe here: a vitest ceiling only decides when vitest gives up, never whether an internal assertion passes. More headroom is the safe direction.
  • But the PR's rationale does not describe what actually binds this test. Its contention-sensitive part is aged.elapsedMs < 4000 and fifo.elapsedMs < 15000, which are far tighter than either ceiling and untouched by this PR. At the config's own 5× factor those internal thresholds fail long before 30 s or 90 s is reached. Raising the vitest ceiling buys this test nothing against the contention the PR is worried about.

The other five are pure headroom budgets, as described — two of them carrying the PR's own rewritten comment, three carrying none.

Findings

F1 (low, non-regression) — || still fails open on negatives and Infinity; now measured end-to-end. Reproduce:

git worktree add tmp/head-tree HEAD^2   # + link per-workspace node_modules
cat > tmp/head-tree/scripts/tests/zz.test.ts <<'EOF'
import { describe, expect, it } from 'vitest';
describe('p', () => it('sleeps', async () => {
  await new Promise((r) => setTimeout(r, 95_000));
  expect(1).toBe(1);
}));
EOF
cd tmp/head-tree
QWEN_SCRIPTS_TEST_TIMEOUT_MS=-5 npx --no-install vitest run \
  --config ./scripts/tests/vitest.config.ts ./scripts/tests/zz.test.ts   # -> PASS, 95034 ms
# control: drop the env var -> FAIL "Test timed out in 90000ms"

Because 95 s > 90 s, the pass cannot be a fallback to the default: -5 reaches withTimeout, whose shipped guard (@vitest/runner/dist/chunk-hooks.js:1853) is if (timeout <= 0 || timeout === Number.POSITIVE_INFINITY) return fn, disarming every ceiling in the suite. Infinity is identical (c9). Not a regression — base behaves the same on all four residuals — and the input domain is repo-controlled: the ${{ cond && 'x' || '' }} idiom renders '' or a literal positive, never a negative, and a census finds the knob is referenced in exactly 2 places repo-wide (the config and its pin), 0 under .github/ and 0 under docs/. So this is a completeness nit, not a blocker.

Measured candidate fix (36/36 assertions; collapses <code>||</code> and the residuals into one guard)
const scriptsTestTimeoutMs = Number(
  process.env['QWEN_SCRIPTS_TEST_TIMEOUT_MS'],
);

export default defineConfig({
  test: {
    // …
    testTimeout:
      Number.isFinite(scriptsTestTimeoutMs) && scriptsTestTimeoutMs > 0
        ? scriptsTestTimeoutMs
        : 90_000,

Applied in a scratch worktree at HEAD^2 and driven through the same instruments (witness 04-candidate-patch-closes-f1-residual.png):

  • hostile fixtures go clean — all 13 invalid/non-positive/non-finite spellings ('', ' ', '0', '-0', '0.0', 'abc', 'null', 'undefined', 'NaN', '-5', '-1', 'Infinity', '-Infinity') → 90000; unset still → 90000.
  • benign fixtures byte-identical — all 7 valid overrides ('5000', '5000 ', ' 5000', '1e3', '0x10', '1500.5', '90000') resolve to exactly the same numbers as head. Zero collateral, counted: exactly 4 of 21 spellings change'-5', '-1', 'Infinity', '-Infinity' — and nothing else.
  • end-to-end, one shared probe — the same 95 s probe with knob -5 flips from PASS (head, slept 95035 ms) to Test timed out in 90000ms (patched); '' still passes; '5000' still fails at 5000ms.
  • suite counts unchanged — the PR's own rewritten pin still passes 27/27 against the patched config.

That last line is the unpinned-axis signal, not reassurance: the suite is green with and without the patch, so nothing currently pins this axis. The fixture that would go red is two extra arms in the existing loop — ['-5', 90_000] and ['Infinity', 90_000]. The patch should ship with them.

F2 (low, completeness — pre-existing; the PR narrows it but does not close it). The PR states the criterion "per-test ceilings that predate the suite-wide ceiling and now shadow it", then applies it to the six }, 30000) in one file. A scripted sweep of all 76 test files in scripts/tests/** at head finds 42 remaining vitest timeout overrides below the 90 s suite ceiling, across 8 files — 28 single-line third-arg, 11 multi-line third-arg, 2 describe options-object, 1 file-level vi.setConfig (witness 08-sibling-sweep-remaining-shadows.png):

count file values (ms)
15 check-tui-dep-direction.test.js 40000 ×15
11 install-script.test.js vi.setConfig({testTimeout:30_000}) file-level; describe-level 15000 / 60000; per-test 30000 ×2, 15000 ×6
4 qwen-triage-workflow.test.js 60000, 30000 ×3
3 qwen-autofix-workflow.test.js 10000, 10000, 20000
3 upload-aliyun-oss-assets.test.js 30000 ×3
3 brand-create-safety.test.js 60000 ×3
2 qwen-repo-hygiene-workflow.test.js 60000, 15000
1 lint.test.js 15000

By value: 10000 ×2, 15000 ×9, 20000 ×1, 30000 ×9, 40000 ×15, 60000 ×6. So 21 of the 42 are ≤ 20 s — at least 4.5× tighter than the ceiling this PR relies on — and 12 are strictly tighter than the 30 s it removed.

(Counting notes. spawnSync/execFileSync { timeout: N } option bounds are excluded — those are subprocess bounds, not vitest ceilings, and do not shadow the suite ceiling. One grep false positive, event-loop-yield.test.js:20 }, 0), is a setTimeout delay, proven by its surrounding context and excluded by an asserted rule rather than by hand. My first pass reported 31, because the enumeration regex only matched }, N); on a single line and missed the multi-line form }, / 40000, / );; all 11 missed sites were individually verified at their line numbers, the regex was fixed, and the split is now pinned by an assertion so a future regression reads as a red check rather than a silently smaller count.)

Three of these deserve more than a count:

  • Three survivors are in the file the PR edited and are tighter than what it removed — 10 s, 10 s and 20 s versus the 30 s it deleted. All three sit on subprocess-spawning tests (execFileSync('bash', …) called 13× in the 20 s one; spawnSync(process.execPath, …) via runAutofixRunner in the two 10 s ones). The 20 s one carries a comment stating its own now-stale premise verbatim (qwen-autofix-workflow.test.js:1550-1551): "Spawn-heavy: each run() forks bash + a stubbed gh. The default 5s per-test budget is tight for this many cases, so give it a comfortable margin." The default is 90 s now. The five adjacent tests in the same describe as the 21800 ceiling (@21703, @21730, @21752, @21771) call the same spawn helpers and carry no override — verified: five it(…) blocks each closing with a bare });, and 5 runAddressReview/runDevelopIssue calls in that range — so they already inherit 90 s. That asymmetry suggests the 10 s is vestigial.
  • install-script.test.js:63 opts an entire file out of the ceiling. vi.setConfig({ testTimeout: 30_000 }) at module top level caps a 5321-line file with 63 subprocess call sites at exactly the figure the config comment calls "the quiet-host figure" that failed on the shared pool — uncommented. This is not a guess about vi.setConfig's reach; I measured it: with a file-level vi.setConfig({ testTimeout: 3_000 }) a 5 s sleep is killed at 3000ms, while the identical probe without it passes under the 90 s suite ceiling. The sharpest part is that the config comment says the suite timeout was originally introduced for this very file ("Several tests in install-script.test.js shell out to node… Bump the suite timeout"), and that the pin's own comment asserts "A per-file vi.setConfig cannot fix it" — while this line is exactly that, still reinstating 30 s for the most subprocess-heavy file in the suite. Its describe('Linux/macOS installer end-to-end', { timeout: 15000 }) at :2866 cascades a 15 s bound onto its children.
  • Four surviving overrides have inner/outer subprocess bounds that are equal or inverted, so the vitest ceiling — not the subprocess bound — is what fires: qwen-repo-hygiene-workflow.test.js:839 outer 15000 == inner spawnSync 15000 @129 (zero headroom); qwen-triage-workflow.test.js:7096 outer 30000 == inner 30000 @7087; qwen-triage-workflow.test.js:6372 and :6652 outer 30000 but inner 60000 @219 (inverted). check-tui-dep-direction.test.js (outer 40000 / inner 30000) is the only correctly-layered pair.

Attribution: all 42 predate this PR and the PR is not obliged to fix them. But the PR is the one that states the criterion, and it fixed 6 sites while leaving 42 — including three stricter ones in the same file. Worth a follow-up issue rather than a wider diff here; per the repo's own guidance, review rounds should not balloon a PR.

F3 (low) — a vitest ceiling cannot interrupt a blocking spawnSync, which bounds what the removal buys. The PR's own text says this of the three ceilings it kept: "spawnSync blocks the event loop where vitest's async timeout cannot fire." The same is true of the six it removed — five of them spawn bash through blocking spawnSync. Measured, with the ceiling armed at 3000 ms in every arm (witness 07-blocking-spawnsync-vs-vitest-ceiling.png, 5/5):

arm work result
async (CONTROL) await a 12 s timer killed at ~3 s — Test timed out in 3000ms, wall 4.0 s → the ceiling is armed in this run
blocking-finite spawnSync('bash -c "sleep 12"') ran its full 12.0 s; vitest reported timed out in 3000ms only after the loop unblocked (wall 13.0 s) — late, not preventive
blocking-hang spawnSync('bash -c "while :; do sleep 1; done"') vitest never terminated it; an external timeout 25 had to kill it (exit 124), and no timeout message was ever emitted

So for the blocking-spawn portions of the de-ceilinged tests, moving 30 s → 90 s delivers no protection against a hung subprocess; only a spawnSync timeout option does — which is exactly what the three kept ceilings provide. Not a regression (the old 30 s ceiling could not interrupt either) and the removal still widens headroom for the async portions. It is a bound on the rationale, stated so the next reader does not credit the suite ceiling with protecting a hang it cannot reach.

My measurement is corroborated by the repo's own comments, written independently of it — qwen-autofix-workflow.test.js:14175-14176: "spawnSync blocks the event loop, so vitest's async timeout cannot fire — bound each subprocess directly against a hung runner", and test-setup.ts:29-31: "A single test, beforeAll, or module-level block stalling for 60s would still trip it: testTimeout cannot interrupt synchronous bodies." Two instruments agreeing — a behavioural probe and a documented mechanism — is what turns this from an argument into evidence.

F4 (note) — hookTimeout is left at vitest's 10 s default while testTimeout is raised to 90 s. scripts/tests/vitest.config.ts sets no hookTimeout (grep count 0), so the resolver default applies: resolved.hookTimeout ??= resolved.browser.enabled ? 3e4 : 1e4 (node_modules/vitest/dist/chunks/coverage.DfSpMS-b.js:3922), and with environment: 'node' that is 10 000 ms — 9× tighter than the test ceiling, and untouched by this PR. Hooks really are wrapped with it (chunk-hooks.js:2084-2086 getDefaultHookTimeout(), :2117 beforeAll, :2163 beforeEach; the message template at :2006 names hookTimeout and testTimeout as separate knobs).

This is not hypothetical — there is a concrete affected call site. scripts/tests/workflow-size.test.js:657 runs a beforeAll that makes six sequential git subprocess spawns through a local spawnSync('git', args, …) helper (init, two config, add, commit, rev-parse), verified by count. All six are bounded by the 10 s hook default, not by the 90 s test ceiling this PR relies on. The structurally identical setup in the sibling describe at @779 does the same six spawns inside the test body, so that one is covered by 90 s — same work, two different timeout regimes.

The omission is also an inconsistency with established repo practice rather than a novel ask: four other configs raise hookTimeout specifically on the contended-host branch — packages/core/vitest.config.ts:20, packages/cli/vitest.config.ts:165, packages/acp-bridge/vitest.config.ts:41, packages/web-shell/vitest.config.ts:22, all process.env['RUNNER_NAME']?.startsWith('ecs-qwen-') ? 60_000 : undefined — with two more setting it flat (node-repl:15, qwen-live:27 at 60 000). That policy is pinned by a guard test, unit-vitest-configs.test.ts:138-147, which asserts config.test?.hookTimeout against the gate's clamps for core, cli, acp-bridge and web-shell. scripts/tests/vitest.config.ts is the only config in that set that raises testTimeout for contention while leaving hookTimeout at the default, and the only one with no ecs-qwen- branch at all — and its own guard test (the pin this PR rewrites) asserts only testTimeout, never hookTimeout. Pre-existing, and this PR does not make it worse; but the suite's protection against a contended host is asymmetric.

Not covered

  • Workflow end-to-end wiring of the knob. Nothing sets QWEN_SCRIPTS_TEST_TIMEOUT_MS; proven by census, not by reading — exactly 2 references repo-wide (the config line and the pin's stubEnv), 0 under .github/, 0 under docs/. The fault is latent by design, so coverage is at the config-module and runner level. The same census confirms the PR's "no caller uses 0 for no-timeout" claim: there are no callers at all.
  • install-script.test.js could not be run in this container. It throws its own guard, Error: `zip`/`unzip` missing on a CI host — this image ships unzip (/usr/bin/unzip) but not zip (command -v zip → empty). Proven environmental by an A/A control, not asserted: the file fails identically on head and base (exit=1, same message, Tests no tests, 0 assertions collected). This is the sole cause of the full suite's exit=1 alongside 1998/1998 tests passed and 265/266 suites passed (whole-suite gate and duration census: 05-full-scripts-suite-duration-census.png — 0 of 1998 tests exceeded 30 s idle here, and none exceeded the 18 s envelope that 90 s ÷ the config's own 5× contention factor implies); the accounting is scripted rather than waved away. An earlier revision of my suite harness asserted status === 0 || tests.length > 0, which passed on the second branch and hid that nonzero exit — corrected and re-run.
  • Base arm of the six de-ceilinged tests. Not run: all six measure ≤ 7.8 s idle here, so they pass under the old 30 s ceiling on this machine too and no flip is observable. The 5× contention arithmetic above is the substitute, and it is arithmetic on measured values, not a measurement of a contended host.
  • Flakiness gate. Not re-run (the previous round's timed out at 2 of 5 rounds). The full suite ran once and the de-ceilinged file ran twice; both agreed.
  • Per-commit attribution. Moot — 1 commit, reachable and matching metadata.
  • Windows / macOS platform lanes. Linux container only. Note the config's exclude list drops the bash-driven workflow suites on win32, so the six de-ceilinged tests do not run there at all.
  • Other repos' vitest configs. The ??-fail-open class exists at other Number(process.env[…] ?? default) sites (integration-tests/cli/qwen-daemon-startup-benchmark.test.ts:54, packages/web-shell/client/e2e/web-shell.stream-performance.spec.ts:27-28, packages/web-shell/playwright.config.ts:3, packages/cua-driver/…/daemon-integration.test.mjs:22-23). I checked whether any is wired with the '' idiom today and none is: the 15 workflow uses of the ${{ cond && 'x' || '' }} idiom (10 rendering '1', 4 'npm', 1 '--dry-run') feed QWEN_CI_COVERAGE (read as === '1', fail-closed), QWEN_SKIP_LATENCY_BUDGETS (read as ['1','true','yes'].includes(…), fail-closed — the sibling knob the PR cites, at ci.yml:723, confirmed), VITEST_MIN_THREADS/VITEST_MIN_FORKS (no JS consumer at all) and actions/setup-node's cache: input. So there is no live instance of this bug class in the repo today, which corroborates the PR's own "latent rather than live" framing. Those other sites are outside this PR's stated scope (the scripts lane) and I did not probe their behaviour.
  • Shape vs cause. The probe reproduces the mechanism (what vitest does with 0/NaN/-5/Infinity) end-to-end through the real runner. It does not reproduce a real contended-host timeout, which is the failure the suite ceiling exists for — that needs the shared pool, not this container.

Methodology

Environment: node:22-bookworm container, Node v22.23.2, npm 10.9.8, vitest 3.2.7, merge-ref checkout at depth 2 (HEAD = merge 55f54edf, HEAD^1 = base tip 80b5dab3, HEAD^2 = PR head fea07732), with npm ci and npm run build pre-run by the workflow. Harnesses are rerunnable .mjs files in tmp/pr10910-verify-20260903-200608/harness/, raw per-cell logs in logs/, captures in evidence/ via scripts/verify-capture.mjs. Three scratch worktrees — tmp/head-tree (HEAD^2), tmp/base-tree (HEAD^1), tmp/patch-tree (HEAD^2 + candidate fix) — each with the 15 per-workspace node_modules symlinked to the root install; the PR touches no package.json or lockfile, so the dependency tree is not part of the change. Control validity was asserted, not assumed: readlink -f confirms workspace configs resolve inside each tree (tmp/base-tree/packages/core/vitest.config.ts), while third-party vitest/vite/vite-plugin-dts resolve to the shared root install; and git diff HEAD^2..HEAD -- scripts/tests/ is empty, so the merge commit is not a fourth arm. Every mechanism fact is quoted from the installed vitest rather than from documentation. Assertion counts come solely from the JSON fragments each harness wrote after executing; expected reds (M1, c1, c4, c5→pass, c7, c8, c9, p1, p3, the shadow-on cell) are encoded as passing assertions, so the fail: 0 total means no unexpected outcome occurred anywhere. Harness self-corrections during the round, all re-run to completion: (a) the summary parser first conflated vitest's Test Files and Tests lines; (b) it then failed on ANSI-coloured output, and its self-test fixture passed anyway because the fixture used an idealised FAIL line while the real one is a reverse-video badge that strips to a leading space — the fixture was replaced with bytes copied verbatim from a captured run; (c) the ladder's first VALID list wrongly required an explicit '90000' override to differ from the default; (d) the patch harness first cross-extracted its probe source by regex, truncating at an escaped backtick and yielding 23 bogus undefined readings — caught by its own instrument self-check, then fixed by moving both probes into shared files that head and patched arms read byte-identically; (e) the suite harness's lenient exit-code assertion, described above; (f) the sibling sweep's enumeration regex first reported 31 override sites because it only matched the single-line }, N); form — a delegated read-only sweep reported 42, every one of the 11 discrepancies was verified individually at its line number, and the regex was fixed and the form-split pinned by assertion (final: 42, matching). The delegated sweep's other load-bearing claims were also re-verified directly rather than taken on trust: the six git spawns in workflow-size.test.js:657, the four ecs-qwen- hookTimeout branches, and both repo comments corroborating F3.

Flakiness gate log

rounds=5 files=2 skipped=0
file scripts/tests/qwen-autofix-workflow.test.js: (cd .) npx --no-install vitest run --config ./scripts/tests/vitest.config.ts ./scripts/tests/qwen-autofix-workflow.test.js
file scripts/tests/unit-vitest-configs.test.ts: (cd .) npx --no-install vitest run --config ./scripts/tests/vitest.config.ts ./scripts/tests/unit-vitest-configs.test.ts


per-file results (P=pass F=fail I=infra-exit, one letter per run):
  scripts/tests/qwen-autofix-workflow.test.js: PPPPP
  scripts/tests/unit-vitest-configs.test.ts: PPPPP

verdict: pass
summary: 2 changed test file(s) x 5 identical rounds, no divergence

--- per-invocation detail (full copy in the artifact) ---
round 1 · scripts/tests/qwen-autofix-workflow.test.js: P (exit 0)
round 1 · scripts/tests/unit-vitest-configs.test.ts: P (exit 0)
round 2 · scripts/tests/qwen-autofix-workflow.test.js: P (exit 0)
round 2 · scripts/tests/unit-vitest-configs.test.ts: P (exit 0)
round 3 · scripts/tests/qwen-autofix-workflow.test.js: P (exit 0)
round 3 · scripts/tests/unit-vitest-configs.test.ts: P (exit 0)
round 4 · scripts/tests/qwen-autofix-workflow.test.js: P (exit 0)
round 4 · scripts/tests/unit-vitest-configs.test.ts: P (exit 0)
round 5 · scripts/tests/qwen-autofix-workflow.test.js: P (exit 0)
round 5 · scripts/tests/unit-vitest-configs.test.ts: P (exit 0)

Evidence images

01-pin-ab-2x2-matrix

02-expression-ladder-head-vs-base

03-probe-end-to-end-timeout-cells

04-candidate-patch-closes-f1-residual

05-full-scripts-suite-duration-census

06-aa-control-and-six-de-ceilinged-durations

07-blocking-spawnsync-vs-vitest-ceiling

08-sibling-sweep-remaining-shadows

Harness scripts and raw logs are in the workflow run artifacts (7-day retention).

Qwen Code · sandboxed verification

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review: approve, pinned to fea07732e07f5b38ae0be225560802ba002e22a6.

Minimal correct fix, and the part that matters is that the rewritten pin now fails against the expression it replaces — I verified the ?? expression, the stub-then-unstub pin, the six }, 30000) ceilings, and the three surviving spawnSync bounds against the base tree rather than the PR's word.

One thing this approval is not: a green CI. The Test (ubuntu-latest, Node 22.x) leg was cancelled at the job's 120-minute budget while still inside test:ci:workspaces, so test:scripts — the only lane that runs the changed config and pin — never executed on this head. That leg still has to land green before merge. The cause is the install/workspaces overrun on the shared pool, not this diff (main is red on the same legs today); the behavioural claim is instead A/B-proven by the sandboxed /verify report on this thread.

Non-blocking notes and the full reasoning are in the stage=3 comment. ✅

@wenshao

wenshao commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Local verification — real environment, real suite runner

Setup. Worktree at PR head fea07732 with the repo's own node_modules; base = merge-base 93e1597b7e (#10870). Linux, Node 22.22.2, vitest 3.2.7 (satisfies the repo's ^3.2.4), idle host. Every run below goes through the lane's real command — vitest run --config ./scripts/tests/vitest.config.ts (package.json:52), the same one ci.yml:758, release.yml:628 and qwen-triage.yml:3594 use. This is an independent maintainer pass on real hardware, run after the sandboxed bot verification above; where the two overlap they agree, and the notes below are what the local environment adds.

Verdict: the fix is correct, minimal, and proven load-bearing end to end. Recommend merge. Four notes below, none blocking; N1 is the only one I'd like to see picked up, and a follow-up is fine.


1. The fault is real — and it does not merely stop raising the ceiling, it removes it (CONFIRMED)

A probe that sleeps 100 s (10 s past the ceiling), driven through the real suite runner, with the knob set to the empty string exactly as ${{ cond && 'x' || '' }} renders it:

arm expression under test result
base 93e1597b7e Number(env ?? 90_000) PASSED after 100 043 ms — the ceiling is gone, not raised
head fea07732 Number(env) || 90_000 failed at 90 034 msTest timed out in 90000ms

ceiling disarmed on base, restored on head

The empty spelling is not hypothetical in this repo: ci.yml:722 already ships

QWEN_SKIP_LATENCY_BUDGETS: "${{ startsWith(runner.name, 'ecs-qwen-') && '1' || '' }}"

in the very env: block a wired QWEN_SCRIPTS_TEST_TIMEOUT_MS would join. Latent today, one line away from live. The mechanism is in the installed runner: withTimeout at node_modules/@vitest/runner/dist/chunk-hooks.js:1853 returns the test function unwrapped when timeout <= 0 || timeout === Number.POSITIVE_INFINITY — no timer is ever armed.

2. The rewritten pin has mutation efficacy; the old one had none (CONFIRMED)

Full 2×2, run against the real scripts/tests/unit-vitest-configs.test.ts, not a standalone probe (see N4):

pin \ config base (??) head (||)
old pin (base) ✅ pass ✅ pass
new pin (head) fail — stub="": expected +0 to be 90000 ✅ pass

The old pin is green in both columns: it cannot distinguish the expression it exists to guard. Only the rewritten pin can. 1 failed / 26 skipped, no collateral.

2x2 pin efficacy matrix

Both vitest-source claims in the PR body check out against the installed 3.2.7: stubEnv is else if (value === void 0) delete process.env[name], and _envBooleans is ["PROD","DEV","SSR"], so '' is stored verbatim.

3. No regression — the suite is exactly as green as base (CONFIRMED)

Full npm run test:scripts on both commits, same host, back to back:

commit files tests
base 93e1597b7e 4 failed / 72 passed (76) 6 failed / 2088 passed / 30 skipped
head fea07732 4 failed / 72 passed (76) 6 failed / 2088 passed / 30 skipped

Identical, down to the six failing names. All six are artifacts of this container, and each fails the same way at base:

  • qwen-autofix-workflow > locks the runner file-command backing files against env plants and qwen-pr-review-workflow > fallback comment resilience (×2) — uid 0: chmod 0444 does not stop root, so gates that must fail closed cannot;
  • install-script > does not package audio-capture test artifactsENOENT … packages/audio-capture/dist, an unbuilt workspace;
  • qwen-fleet-shepherd-workflow (×2) — the fixture's bash date arithmetic needs NOW_EPOCH, which this box does not set.

None is attributable to the PR.

The six de-ceilinged cases all pass on the suite ceiling. Idle durations on head: 1120 / 160 / 301 / 7697 / 243 / 545 ms. And the release-run breaker upserts deferred findings into a per-PR issue that survives the merge is indeed not among the six — confirmed, it already ran on the suite ceiling.

prettier --check and eslint are clean on all three files.

full scripts suite, base vs head


N1 · The six removed ceilings were load-bearing on one path — not the CI path

Root vitest.config.ts lists 'scripts' as a project, and there is no scripts/vitest.config.ts. That project therefore runs scripts/tests/** on vitest's 5 s default with no setupFilesscripts/tests/vitest.config.ts is never consulted there. On an idle host:

npx vitest run --project scripts scripts/tests/qwen-autofix-workflow.test.js \
  -t 'keeps the round status comment live with a heartbeat and a job deep link'

base   ✓ 7703 ms
head   × Test timed out in 5000ms      (the test body still took 7778 ms)

root scripts project runs on the 5s default

Not contention-dependent; deterministic on an idle box, because that one case measures 7.7 s.

Scope, honestly: CI never takes this path, and AGENTS.md already lists "npx vitest from the project root" under Avoid. The path was also divergent before this PR — skipping setupFiles means no fs.appendFileSync mock and no RPC-yield beforeEach. So the PR does not create the divergence; it removes the last thing that kept this file's slowest cases green there, and the root config still advertises the project. Cheapest close, either one: add a scripts/vitest.config.ts re-exporting ./tests/vitest.config.ts, or drop 'scripts' from the root projects list.

N2 · The stated criterion is applied to six of nine sites in the file it edits

After this PR, qwen-autofix-workflow.test.js still carries }, 20000) (L1552), }, 10000) (L11837) and }, 10000) (L21800). None asserts a duration property; L1552's own comment states the PR's exact rationale:

    // Spawn-heavy: each run() forks bash + a stubbed gh. The default 5s per-test
    // budget is tight for this many cases, so give it a comfortable margin.
  }, 20000);

Two of the three are tighter than the 30 s the PR just argued is dangerous under contention. The PR body's sentence is literally accurate — it scopes itself to the }, 30000) spelling — but the principle it states does not stop there. (26 further per-test }, N) ceilings live across six other scripts-suite files, plus a file-level vi.setConfig({ testTimeout: 30_000 }) at install-script.test.js:63. Out of scope here; worth one follow-up sweep.)

N3 · || does not catch every non-positive spelling

The PR body says "every non-positive spelling falls back to the default". Negatives are truthy and survive the ||:

QWEN_SCRIPTS_TEST_TIMEOUT_MS=-1  →  Number('-1') || 90000  =  -1
→ an 8 s test PASSES on head — same `timeout <= 0` branch of `withTimeout` that `0` takes

Nothing renders a negative today, so this is a wording correction rather than a defect. Number(x) > 0 ? Number(x) : 90_000 would make the sentence literally true, if you want it to be.

N4 · Two PR-body claims the evidence does not support

  • "unit-vitest-configs.test.ts cannot be run outside CI … vite-plugin-dts, which a root-only install does not have." A normal workspace npm install in this repo places it at packages/webui/node_modules/vite-plugin-dts. The real file runs locally and green: 27 passed (27), 1.74 s. The standalone probe was not necessary — and, as §2 shows, running the real pin is the stronger artifact.
  • "the rewritten pin's own green run comes from CI rather than from this machine." On this head, Test (ubuntu-latest, Node 22.x) was cancelled at the 2 h job cap inside Run tests and generate reports, and in that step npm run test:scripts runs only after test:ci:workspaces returns 0 — so the scripts suite never ran there either. No CI lane on this head has produced a green scripts-suite result. §2 and §3 supply it.
中文版

本地验证 —— 真实环境、真实 suite runner

环境。 在 PR head fea07732 上建 worktree,复用仓库自身的 node_modules;base 取 merge-base 93e1597b7e#10870)。Linux、Node 22.22.2、vitest 3.2.7(满足仓库的 ^3.2.4),空闲主机。下面每一次运行都走通道的真实命令 —— vitest run --config ./scripts/tests/vitest.config.tspackage.json:52),与 ci.yml:758release.yml:628qwen-triage.yml:3594 完全一致。这是在真实机器上做的一次独立维护者验证,在上面的沙箱 bot 验证之后进行;两者重叠之处结论一致,下面的备注是本地环境额外补上的部分。

结论:修复正确、最小,且端到端证明确实起作用。建议合入。 下面四条备注均不阻塞合入;其中只有 N1 希望能被跟进,放到后续 PR 也可以。


1. 缺陷是真的 —— 而且它不只是「不再抬高上限」,是把上限整个拿掉(已确认)

一个睡眠 100 秒(超出上限 10 秒)的探针,经真实 suite runner 驱动,环境变量取空字符串,正是 ${{ cond && 'x' || '' }} 在条件为假时渲染出的值:

分支 被测表达式 结果
base 93e1597b7e Number(env ?? 90_000) 100 043 ms 后通过 —— 上限是消失了,不是被抬高
head fea07732 Number(env) || 90_000 90 034 ms 失败 —— Test timed out in 90000ms

ceiling disarmed on base, restored on head

空值写法在本仓库并非假想:ci.yml:722 今天就已经在同一个 env: 块里发着

QWEN_SKIP_LATENCY_BUDGETS: "${{ startsWith(runner.name, 'ecs-qwen-') && '1' || '' }}"

—— 一个被接线的 QWEN_SCRIPTS_TEST_TIMEOUT_MS 会落在它旁边。今天潜伏,距离生效只有一行。机制在已安装的 runner 里:node_modules/@vitest/runner/dist/chunk-hooks.js:1853withTimeouttimeout <= 0 || timeout === Number.POSITIVE_INFINITY 时直接把测试函数原样返回,根本不装计时器。

2. 重写后的 pin 具备变异检出力,旧 pin 一点也没有(已确认)

完整 2×2,跑的是真实的 scripts/tests/unit-vitest-configs.test.ts,不是独立探针(见 N4):

pin \ 配置 base(?? head(||
旧 pin(base) ✅ 通过 ✅ 通过
新 pin(head) 失败 —— stub="": expected +0 to be 90000 ✅ 通过

旧 pin 两列全绿:它对自己本要守护的那个表达式毫无判别力。只有重写后的 pin 能区分。1 失败 / 26 跳过,无附带损伤。

2x2 pin efficacy matrix

PR 描述中两条关于 vitest 源码的说法,对照已安装的 3.2.7 均成立:stubEnv 确为 else if (value === void 0) delete process.env[name]_envBooleans 确为 ["PROD","DEV","SSR"],因此 '' 会被原样存储。

3. 无回归 —— 套件与 base 绿得一模一样(已确认)

同一主机上连续对两个 commit 跑完整的 npm run test:scripts

commit 文件 用例
base 93e1597b7e 4 失败 / 72 通过(76) 6 失败 / 2088 通过 / 30 跳过
head fea07732 4 失败 / 72 通过(76) 6 失败 / 2088 通过 / 30 跳过

完全一致,连六个失败用例的名字都一样。这六个都是本容器的环境产物,且每一个在 base 上都以同样方式失败:

  • qwen-autofix-workflow > locks the runner file-command backing files against env plantsqwen-pr-review-workflow > fallback comment resilience(2 个)—— uid 0:chmod 0444 拦不住 root,需要 fail closed 的闸门无法关上;
  • install-script > does not package audio-capture test artifacts —— ENOENT … packages/audio-capture/dist,工作区未构建;
  • qwen-fleet-shepherd-workflow(2 个)—— fixture 的 bash 日期算术依赖 NOW_EPOCH,本机未设置。

均与本 PR 无关。

被去掉上限的六个用例全部在套件上限下通过。head 上空载耗时:1120 / 160 / 301 / 7697 / 243 / 545 ms。另外,真正打断 release run 的那个用例 upserts deferred findings into a per-PR issue that survives the merge 确实不在这六个之中 —— 已确认,它本来就跑在套件上限上。

三个改动文件的 prettier --checkeslint 均干净。

full scripts suite, base vs head


N1 · 被移除的六处上限,在一条非 CI 路径上是承重的

根目录 vitest.config.ts'scripts' 列为一个 project,而仓库里并没有 scripts/vitest.config.ts。于是该 project 用 vitest 的 5 秒默认值、且不加载 setupFiles 来跑 scripts/tests/** —— scripts/tests/vitest.config.ts 在这条路径上根本不会被读取。空闲主机上:

npx vitest run --project scripts scripts/tests/qwen-autofix-workflow.test.js \
  -t 'keeps the round status comment live with a heartbeat and a job deep link'

base   ✓ 7703 ms
head   × Test timed out in 5000ms      (测试体本身仍跑了 7778 ms)

root scripts project runs on the 5s default

这不依赖争抢:在空闲机器上是确定性失败,因为该用例本身就要 7.7 秒。

范围要说清楚:CI 从不走这条路径,而 AGENTS.md 已经把「从项目根目录跑 npx vitest」列在 Avoid 之下。这条路径在本 PR 之前也已经是发散的 —— 不加载 setupFiles 意味着没有 fs.appendFileSync 的 mock,也没有那个防 RPC 超时的 beforeEach yield。所以本 PR 并没有制造这个发散,只是拿掉了让该文件最慢的几个用例在那里仍然发绿的最后一道保护,而根配置至今仍在对外宣称这个 project。最省事的收口二选一:新增一个 scripts/vitest.config.ts 转出 ./tests/vitest.config.ts,或者把 'scripts' 从根 projects 列表里去掉。

N2 · 所声明的标准,在它自己改的文件里只应用到九分之六

本 PR 之后,qwen-autofix-workflow.test.js 仍带着 }, 20000)(L1552)、}, 10000)(L11837)和 }, 10000)(L21800)。它们没有一个把时长当作被测属性;L1552 自己的注释写的正是本 PR 的同一条理由:

    // Spawn-heavy: each run() forks bash + a stubbed gh. The default 5s per-test
    // budget is tight for this many cases, so give it a comfortable margin.
  }, 20000);

其中两处比本 PR 刚刚论证为「争抢下有危险」的 30 秒还要紧。PR 描述那句话字面上没错 —— 它把自己限定在 }, 30000) 这个写法上 —— 但它陈述的原则并不止于此。(另有 26 处 per-test }, N) 上限散落在 scripts 套件的另外六个文件中,外加 install-script.test.js:63 一处文件级 vi.setConfig({ testTimeout: 30_000 })。本 PR 范围之外,值得后续一次性清一遍。)

N3 · || 并未拦下所有非正数写法

PR 描述称「使一切非正数写法都回落到默认值」。负数是 truthy,会穿过 ||

QWEN_SCRIPTS_TEST_TIMEOUT_MS=-1  →  Number('-1') || 90000  =  -1
→ 一个 8 秒的测试在 head 上通过 —— 与 `0` 走的是 `withTimeout` 里同一个 `timeout <= 0` 分支

今天没有任何地方会渲染出负数,所以这是措辞订正而非缺陷。若希望那句话字面成立,Number(x) > 0 ? Number(x) : 90_000 即可收口。

N4 · PR 描述中两条证据并不支持的说法

  • unit-vitest-configs.test.ts 无法在 CI 之外运行 …… 需要 vite-plugin-dts,仅根目录安装并不包含它。」 本仓库正常的 workspace npm install 会把它装在 packages/webui/node_modules/vite-plugin-dts。真实文件在本地跑得通且全绿:27 passed (27),1.74 秒。独立探针并非必要 —— 而且如 §2 所示,跑真实 pin 是更有力的证据。
  • 「重写后 pin 自身的绿色结果来自 CI 而非本机。」 在这个 head 上,Test (ubuntu-latest, Node 22.x)Run tests and generate reports 步骤里被 2 小时 job 上限取消了;而该步骤中 npm run test:scripts 只在 test:ci:workspaces 返回 0 之后才执行 —— 所以 scripts 套件在那里也从未跑过。这个 head 上没有任何 CI 通道产出过 scripts 套件的绿色结果。§2 与 §3 补上了它。

@qqqys qqqys left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE (verified at head fea0773)

Historical blocking item

The one CHANGES_REQUESTED on this PR was template compliance on the PR body, explicitly "no code changes needed" — and the live body now carries every required section (verified heading-by-heading against the template: What/Why, Reviewer Test Plan with How-to-verify + Before/After evidence + OS table, Risk & Scope, Linked Issues, bilingual 中文说明). The same reviewer subsequently re-verified the code claims against the base tree and approved.

My Critical-only scan of the substance

  • vitest.config.ts: Number(env ?? 90_000)Number(env) || 90_000 converts a fail-open knob into a fail-closed one — '' (the exact value ${{ cond && 'x' || '' }} renders on a false condition, already how sibling knobs are wired in ci.yml), garbage (NaN), and 0 all land on the 90 s ceiling instead of silently disarming every timeout in the suite; a valid explicit number still passes through. This direction (0 → ceiling) is the point of the fix, documented inline.
  • The pin rewrite removes the old self-deception (stub '' then unstubAllEnvs() before the assertion, measuring the unset path twice) and now exercises five arms — unset, '', 'abc', '0', '5000' — each re-importing the config under vi.resetModules(), so the module-level Number() read is actually evaluated per arm. The undefined arm correctly exercises the unset path via stubEnv(name, undefined) rather than the empty-string masquerade.
  • The six dropped per-test , 30000 budgets only raise ceilings onto the suite knob; assertions are untouched. No shipped product code in scope.

CI at head

19 green (including Lint & Static), Test (ubuntu-latest) re-running (in progress — never gates), zero completed failures on this head; the earlier 120-minute cancellation on that leg was the shared-pool overrun the approving review attributed to main-side conditions, and the sandboxed /verify report on the thread is the behavioral evidence in its place.

Both the review bot and the human maintainer independently approved this exact head before this pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants