test(core): pin Config.initialize() join ordering so the await-drop mutant fails - #11113
Conversation
…utant fails The 'makes a concurrent caller join the in-flight initialization' test released the gate before asserting anything and never checked that the joining caller was still pending mid-flight. A join branch that drops the await (returning immediately) therefore passed this test while silently reproducing QwenLM#11002: the joiner proceeds before initialization completes and dies on 'Chat not initialized'. It was killed only incidentally by the failure-sharing test, for an unrelated reason. Assert nothing has settled while the first flight is still gated, so the named test actually pins the ordering property it describes. Verified: full config suite 621 passed; removing the await now fails this test with ['second'] vs []. Follow-up to QwenLM#11037 (review finding 3, not covered by QwenLM#11075).
|
|
|
Thanks for the PR! Template looks good ✓ — every required heading is there, the Tested-on table is filled in, and the Chinese section mirrors the English. Problem: demonstrated rather than theoretical, which is the part I checked hardest. The claim is that a join branch with the Direction: aligned. This sits at the end of a real chain — #11002 was a main-branch OpenTUI interactive E2E failure where a submitted prompt was lost to Size: the file is under Approach: the scope feels exactly right, and I arrived at the same shape independently — track when each caller settles, flush the microtask queue with a macrotask, assert nothing has settled, and only then release the gate. It is also the pattern the neighbouring Risk: no elevated risk signals. The revert-history path check matches nothing here (the only changed file is a test). Moving on to code review. 🔍 中文说明感谢贡献! 模板完整 ✓ —— 所有必需标题都在,Tested on 表格已填写,中文部分与英文逐段对应。 问题: 这是已被证实的问题,而不是理论性加固,也是我核查最用力的一点。PR 的主张是:去掉 方向: 对齐。这处在一条真实链路的末端——#11002 是 main 分支上一次 OpenTUI 交互式 E2E 失败,提交的 prompt 死于 规模: 该文件位于 方案: 范围恰当,我独立推导出的形状也一致——记录每个调用方何时落定,用一个宏任务冲刷微任务队列,断言无人落定,然后才释放门闩。这也正是相邻的 风险: 无升级风险信号。回退历史路径检查在此没有命中(唯一改动的文件是测试)。 进入代码审查 🔍 — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
Code reviewI reviewed this statically against the merge-base code — the diff plus The determinism claim holds. The fake-timer hazard is absent. This was the one way the change could have quietly introduced a hang rather than a failure: if fake timers were active in this file, The unhandled-rejection hazard is absent too, and it was worth checking. The two Nothing was weakened. The change is purely additive (+15/−0) — The comment earns its 9 lines. AGENTS.md defaults comments to none, and nine comment lines against six code lines would normally make me look twice. Here the why is genuinely non-obvious — which mutant it kills, and why the old ordering let that mutant through — and the two neighbouring tests in this block already carry the same kind of explanatory comment for the same reason. Consistent with the file, not a departure from it. Test evidenceThis is an unattended CI run, so per the triage rules I did not build, run, or execute anything from this PR — the evidence below is the PR's own CI, read off the check-runs API for the reviewed commit. Nothing was re-run locally. There are no failures on this commit. The unit suite, lint, and the no-AK integration run were still in flight at review time, so they are listed as pending rather than guessed at; the table is refreshed in place once CI settles. Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。 A green suite cannot substantiate this PR's actual claim, and that is worth naming. The point of the change is that the test now fails when the Sandboxed verification would settle it: Real-scenario testing: N/A — unattended CI run, and nothing user-visible changes. No tmux capture was driven, per the CI-path rules. 中文说明代码审查我以静态方式对照 merge-base 代码做了审查——diff 加上 确定性主张成立。 不存在 fake timer 隐患。 这是本改动可能悄悄引入「挂起」而非「失败」的唯一路径:若本文件启用了 fake timers, 同样不存在未处理 rejection 隐患,而这一点值得查。 两个 没有任何断言被削弱。 改动是纯增量的(+15/−0)—— 那段注释对得起它的 9 行。 AGENTS.md 默认不写注释,9 行注释对 6 行代码通常会让我多看两眼。这里的 why 确实不显然——它杀掉的是哪个变异体、以及旧的顺序为何会放它过去——而且本块中相邻的两个用例已经因为同样的理由带着同类解释性注释。与本文件一致,并非偏离。 测试证据这是一次无人值守的 CI 运行,因此按 triage 规则,我没有构建、运行或执行本 PR 的任何代码——下面的证据是 PR 自身的 CI,通过 check-runs API 针对被审查的提交读取。本地未重跑任何内容。 该提交上没有失败项。审查时单元套件、lint 与 no-AK 集成运行仍在进行中,因此列为 pending 而非猜测结果;CI 落定后表格会就地刷新。 绿色套件无法证实本 PR 真正的主张,这一点值得点名。 本改动的意义在于:当 join 分支的 沙箱化验证可以定这件事: 真实场景测试:N/A —— 无人值守 CI 运行,且无任何用户可见变化。按 CI 路径规则未驱动 tmux 抓取。 — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
|
Confidence: 5/5 — I went looking for the two ways a Stepping back: this is the third PR in the #11002 → #11037 → #11075 chain, and chain fatigue is a real way to wave something through, so I tried to judge it on its own. It survives that. The two merged PRs fixed the bug and hardened the join path, but neither left behind a test that fails when the fix is removed — I confirmed that by walking the await-drop mutant through the existing test, where it passes every assertion including the trailing The implementation matches what I would have written independently, down to flushing the microtask queue with a macrotask and asserting before releasing the gate — which is also what the neighbouring aborted-signal test already does, so this reads as the file's own idiom rather than an imported one. Six lines of test code and nine lines explaining which mutant they exist to kill. If I come back to this in six months and feel tempted to tidy the assertion into something shorter, that comment is what stops me, which is the whole argument for it. I checked the two failure modes that would not show up in a passing run. Fake timers would turn Nothing is deferred, nothing is weakened, and there is no production surface to regress. The one thing I could not establish by execution is the mutation witness itself, and I would rather be plain about that than let a green suite stand in for it: CI runs this test against unmutated code, so it passes with or without these 15 lines. I verified the witness statically and named Approval is deferred until CI lands green on 中文说明信心度:5/5 —— 我专门去找「用 退一步看:这是 #11002 → #11037 → #11075 这条链上的第三个 PR,而链路疲劳确实会让人顺手放行,所以我试着单独评判它。它经得起。前两个已合并的 PR 修了 bug、加固了 join 路径,但都没有留下一个「把修复删掉就会失败」的用例——我把去掉 await 的变异体沿现有用例走了一遍确认了这点:它通过每一条断言,包括结尾的 实现与我独立会写出的方案一致,细到用宏任务冲刷微任务队列、并在释放门闩之前断言——这也正是相邻的 aborted-signal 用例已经在做的,所以读起来是本文件自己的写法,而不是外来的一套。6 行测试代码,9 行说明它们是为了杀掉哪个变异体而存在。半年后我若回来,想把这条断言「收拾」得更短,拦住我的正是那段注释——这也正是它存在的理由。 我查了两种在绿色运行里不会暴露的失败模式。fake timers 会把 没有任何东西被推迟处理、被削弱,也没有可回退的生产面。唯一我无法通过执行确立的是变异见证本身,我宁可把这点讲明白,也不愿让一套绿色套件替它作证:CI 是针对未变异的代码运行这个用例的,所以有无这 15 行它都通过。我以静态方式验证了该见证,并在上面点名 批准推迟至 CI 在 — Qwen Code · qwen3.8-max-2026-09-02 Reviewed at |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Not reviewed: reverse audit — stopped before round 1 by the review time budget.
Test Plan (not a blocker): src/config/config.test.ts — no such file or directory; 621 passed — this review observed 23448, 1946, 28642, 298, 1816, 504, 6013 passed.
中文说明
未审查:反向审计——评审时间预算不足,未能开始第 1 轮。
Test Plan(非阻断):src/config/config.test.ts — no such file or directory; 621 passed — this review observed 23448, 1946, 28642, 298, 1816, 504, 6013 passed。
— qwen3.8-max via Qwen Code /review (v0.23.0)
Maintainer verification — built and run locally on LinuxI rebuilt the change in a clean worktree and drove every claim in the PR body end to end, plus the two things a test-only diff has to prove on its own: that the new assertion kills the mutant it names, and that the pre-PR test did not. Both hold. Recommend merge. Environment — Debian 13 (trixie), kernel 6.12.63, 16 vCPU, Node v22.22.2, npm 10.9.7, vitest 3.2.7. Worktree at PR head 1. The PR body's numbers reproduce exactly
2. The mutation witness holds, and the counterfactual proves it is newRemoving
That right-hand column is the load-bearing half: the mutant survives the case that names the behaviour on Running the whole file under M3 confirms §3's other claim too. On the pre-PR file exactly one test goes red — 3. The assertion generalises past the one mutant it was written forI ran four more "joins the flight but doesn't wait for it" mutants through the same command. All four are newly killed — the assertion pins ordering, not one particular token:
The macrotask row matters: the mutant's own 4. The determinism claim survives CPU starvation
5. What the pinned property actually buys, on the real built coreTo confirm the consequence is what the PR says it is, I drove the caller shape from Shipped code: the joiner waits 180 ms and gets a started chat. With the Two non-blocking notesNeither affects the green path; both are one edit in the same block, so I mention them while the code is open. N1 — N2 — the two floating Both are covered by the two-argument const record = (who: string) => () => {
settled.push(who);
};
first.then(record('first'), record('first'));
second.then(record('second'), record('second'));
Status note
Verdict: approve / merge. The claim is demonstrated, the counterfactual is demonstrated, and the assertion is stable under load. 中文说明维护者验证 —— 在本地 Linux 上真实构建并运行我在干净的 worktree 中重建了这个改动,把 PR 描述里的每一条断言都端到端跑了一遍;另外补上了纯测试改动必须自证的两点:新断言确实杀掉它所指名的变异体,而改动前的用例并没有。两点都成立。建议合并。 环境 —— Debian 13 (trixie)、内核 6.12.63、16 vCPU、Node v22.22.2、npm 10.9.7、vitest 3.2.7。worktree 位于 PR head 1. PR 描述里的数字精确复现在 PR head 上执行 2. 变异见证成立,且反事实对照证明它是新增覆盖从
右边那一列才是承重的一半:在 在 M3 下跑整个文件也印证了 §3 的另一条判断:改动前的文件里只有一条用例变红 —— 3. 这条断言的覆盖能力不止于它所针对的那一个变异体我用同一条命令又跑了四个「加入了这次 flight 但不等待它」的变异体,四个全部是新增击杀 —— 断言钉住的是顺序,而不是某个具体 token:
「macrotask」那一行尤其说明问题:该变异体自己的 4. 确定性结论在 CPU 饥饿下依然成立
5. 这条被钉住的性质究竟买到了什么 —— 在真实构建产物上为了确认后果确实如 PR 所述,我按 线上代码:加入方等待 180 ms,拿到已启动的 chat。去掉 两点非阻塞说明两点都不影响绿色路径,且落在同一段代码里,所以顺带提一下。 N1 —— N2 —— 那两个游离的 两点都能用双参数的 const record = (who: string) => () => {
settled.push(who);
};
first.then(record('first'), record('first'));
second.then(record('second'), record('second'));
状态说明我写这份报告时 结论:批准 / 可合并。 论断得到了演示,反事实对照得到了演示,且该断言在负载下稳定。 |
Follow-up: the red
|
| workspace | result | why the run still went red |
|---|---|---|
@qwen-code/qwen-code-core |
643 passed | 1 skipped (644) files, 23523 passed | 10 skipped tests |
Errors 1 error |
@qwen-code/qwen-code (cli) |
1014 passed (1014) files, 28638 passed | 90 skipped tests |
Errors 3 errors |
All four errors are the same one:
Error: [vitest-worker]: Timeout calling "onTaskUpdate"
❯ Object.onTimeoutError ../../node_modules/vitest/dist/chunks/rpc.-pEldfrD.js:53:10
That is worker↔main RPC starvation, and the runner was saturated when it happened — the job's own DFSAMPLE lines show host load ranging 166 → 302 with 46–177 concurrent test jobs on the same box. This is exactly #10490 ("Test (ubuntu) fails non-deterministically on shared runners"), which names this signature and this mechanism: the ubuntu lane is the one lane that keeps the unhandled-error signal (packages/core/vitest.config.ts:44, dangerouslyIgnoreUnhandledErrors: process.platform !== 'linux'), so an infra hiccup with zero failing assertions still reddens the whole job.
The file this PR touches passed in that very job:
✓ src/config/config.test.ts (621 tests) 13409ms
621 — the same count as the PR body and the same count as my local Linux run. Nothing in the failure touches packages/core/src/config.
Attempt 2 is already running (started 18:00 UTC on ecs-qwen-hk4-22). No action needed from the author; my recommend merge stands unchanged.
中文说明
补充:变红的 Test (ubuntu-latest, Node 22.x) 是已知的共享 runner flake,与本 PR 无关
第 1 次尝试(job 101315879912,runner ecs-qwen-hk3-24)失败了,我把日志拉下来看了。整个 job 里没有任何一条用例失败。 全部 21 个 workspace 的 vitest 汇总行都是 passed;job 退出码 1 完全来自未处理错误:
| workspace | 结果 | 为何仍然变红 |
|---|---|---|
@qwen-code/qwen-code-core |
文件 643 passed | 1 skipped (644),用例 23523 passed | 10 skipped |
Errors 1 error |
@qwen-code/qwen-code(cli) |
文件 1014 passed (1014),用例 28638 passed | 90 skipped |
Errors 3 errors |
四个错误是同一个:
Error: [vitest-worker]: Timeout calling "onTaskUpdate"
❯ Object.onTimeoutError ../../node_modules/vitest/dist/chunks/rpc.-pEldfrD.js:53:10
这是 worker 与主进程之间的 RPC 饥饿,而事发时 runner 已经饱和 —— job 自己打的 DFSAMPLE 行显示宿主 load 在 166 → 302 之间,同一台机器上并发着 46–177 个测试任务。这正是 #10490(「Test (ubuntu) 在共享 runner 上非确定性失败」)所描述的签名与机制:ubuntu 这条腿是唯一保留 unhandled-error 信号的腿(packages/core/vitest.config.ts:44,dangerouslyIgnoreUnhandledErrors: process.platform !== 'linux'),因此一次零断言失败的基础设施抖动也会让整个 job 变红。
本 PR 所触及的文件在那次 job 里是通过的:
✓ src/config/config.test.ts (621 tests) 13409ms
621 —— 与 PR 描述、以及我本地 Linux 运行的数字完全一致。这次失败与 packages/core/src/config 毫无关系。
第 2 次尝试已在运行(18:00 UTC 起,runner ecs-qwen-hk4-22)。作者无需做任何处理;我此前的建议合并结论不变。
qqqys
left a comment
There was a problem hiding this comment.
APPROVE (verified at head 2292189)
What the 15 lines do and why they hold
The addition pins the property the surrounding test is named for: while the first initialize() flight sits behind the gate, the joining second call must remain UNSETTLED. I traced both arms: with the join branch's await dropped (the mutant that reproduces #11002 — a joiner proceeding before initialization completes and dying on "Chat not initialized"), second resolves immediately, its .then runs as a microtask strictly before the setTimeout(0) macrotask flush, settled is non-empty, and the new assertion goes red; with correct code both promises are gated and the flush proves nothing settled — deterministic, no timing flake window. The pre-existing assertions (once-called, both-await) demonstrably survive that mutant, so this is exactly the missing pin, not a redundant one.
Gates
Test-only diff in packages/core; the review round at this head posted zero findings, the maintainer approved it, and CI shows 17 green with zero failures (the lone completed non-green is the fleet-side web-shell E2E Smoke cancellation the whole pool has been showing, which never touches this lane's code).



What this PR does
Hardens the
makes a concurrent caller join the in-flight initializationtest inpackages/core/src/config/config.test.tsso it actually pins the ordering property it is named for. The test previously released the gate before asserting anything and never checked that the joining caller was still pending while the first flight was in progress. This adds an ordering assertion: track settlement of both callers and assert nothing has settled before the gate is released.Test-only change; no production code touched.
Why it's needed
This closes the one remaining review finding from #11037 that #11075 did not cover (the post-merge review's §3, a test-quality gap).
A join branch that drops the
await—if (!this.initializationSettled) { return; }instead of awaitingthis.initializationPromise— passed the old test, because the test released the gate before asserting, so by the time it checked, the joiner had already (wrongly) resolved andinitializeInternalhad run exactly once. That mutant silently reproduces #11002: the joiner proceeds before initialization completes and dies onChat not initialized. It was killed only incidentally byshares a failed in-flight initialization, for an unrelated reason (error-identity), so the test that names the join behavior never actually guarded it.Reviewer Test Plan
How to verify
cd packages/core && npx vitest run src/config/config.test.ts— full suite green (621 passed locally).await this.initializationPromise;line from the join branch inconfig.ts. Themakes a concurrent caller join the in-flight initializationtest now fails — the joiner settled while the gate was still held. Restore the line and it is green again.Evidence (Before & After)
Before (join branch with the
awaitdropped, i.e. the #11002-restoring mutant):After (await intact):
621 passed (621).Tested on
Risk & Scope
setTimeout(0)flush drains the microtask queue before the ordering check, so the assertion is deterministic rather than timing-dependent.Linked Issues
Follow-up to #11037 (review finding §3). Complements #11075, which resolved R1-1 through R1-4.
中文说明
这个 PR 做了什么
加固
packages/core/src/config/config.test.ts中的makes a concurrent caller join the in-flight initialization用例,让它真正钉住它所命名的顺序性质。此前该用例在断言任何内容之前就释放了门闩,也从未检查「加入方在第一次初始化仍在进行时是否仍未落定」。本改动新增一条顺序断言:跟踪两个调用方的落定情况,并在释放门闩之前断言没有任何一方落定。仅测试改动,未触碰任何生产代码。
为什么需要它
这闭合了 #11037 中 #11075 未覆盖的那一条遗留评审发现(合并后评审的 §3,一处测试质量缺口)。
一个去掉
await的 join 分支——if (!this.initializationSettled) { return; }而非 awaitthis.initializationPromise——在旧用例下会通过:因为用例在断言前就释放了门闩,等它检查时,加入方已经(错误地)resolve,且initializeInternal恰好运行一次。该变异体会悄悄重现 #11002:加入方在初始化完成前就继续执行,死于Chat not initialized。它只是被shares a failed in-flight initialization因另一个无关理由(错误对象同一性)顺带杀掉,因此那个为 join 行为命名的用例从未真正守住它。审阅者测试计划
如何验证
cd packages/core && npx vitest run src/config/config.test.ts—— 全套绿(本地 621 通过)。config.ts的 join 分支移除await this.initializationPromise;一行。makes a concurrent caller join the in-flight initialization用例随即失败——加入方在门闩仍被挂起时就落定了。还原该行则重新变绿。风险与范围
setTimeout(0)冲刷会在顺序检查前排空微任务队列,因此该断言是确定性的,而非依赖时序。关联 Issue
#11037 的后续(评审发现 §3)。与已解决 R1-1 ~ R1-4 的 #11075 互补。