Skip to content

fix(cli): stop review test-efficacy tests depending on ambient tmpdir vitest - #8537

Merged
wenshao merged 1 commit into
QwenLM:mainfrom
wenshao:fix/cli-test-efficacy-env-flaky
Aug 4, 2026
Merged

fix(cli): stop review test-efficacy tests depending on ambient tmpdir vitest#8537
wenshao merged 1 commit into
QwenLM:mainfrom
wenshao:fix/cli-test-efficacy-env-flaky

Conversation

@wenshao

@wenshao wenshao commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

Stabilizes two environment-dependent tests in packages/cli/src/commands/review/test-efficacy.test.ts that fail on hosts where vitest resolves up-tree from os.tmpdir(). Both tests keep every assertion verbatim; only the way the failure condition is produced becomes host-deterministic. The findVitestBin helper gains an optional resolver parameter whose default is exactly the previous inline behavior, and one test plants a local shadow package instead of relying on a bare tmpdir having no vitest anywhere above it.

Why it's needed

CI's ubuntu Test job fails with AssertionError: expected [Function] to throw an error in two cases: findVitestBin > names the search root when vitest cannot be resolved and runControlMutant > leaves the probe file byte-identical when it cannot run (PR run 30909672420; the same signature appears on the latest main run 30893612230). Both tests assumed nothing up-tree of os.tmpdir() provides a vitest/package.json — true on macOS dev machines, false on the project's self-hosted runner where a node_modules above TMPDIR supplies one. In that environment the first test never throws and the second actually executes the probe suite instead of throwing.

Reviewer Test Plan

How to verify

  1. Reproduce the CI condition locally: mkdir -p /tmp/qwen-fake-tmp/node_modules && ln -s <repo>/node_modules/vitest /tmp/qwen-fake-tmp/node_modules/vitest, then run cd packages/cli && TMPDIR=/tmp/qwen-fake-tmp npx vitest run src/commands/review/test-efficacy.test.ts. With the old code both tests fail exactly as on CI; with this PR the full file passes (128/128).
  2. Confirm the production call path is untouched: findVitestBin is called without the new parameter at its only production call site, so the default resolver (anchored createRequire) is identical to before.
  3. Confirm no assertion changed: both tests still assert the same toThrow(...) messages and the same byte-identical probe file restore.
  4. cd packages/cli && npm run typecheck plus ESLint/Prettier on the two touched files — all clean locally.

Evidence (Before & After)

Before: both tests fail on the self-hosted ubuntu runner (and locally under the fake TMPDIR harness above). After: full file passes 128/128 in the repro environment and in 4 consecutive runs in the normal environment. Full evidence is in the e2e report comment below.

Tested on

OS Status
🍏 macOS
🪟 Windows ⚠️
🐧 Linux

Environment (optional)

macOS local runs plus a fake TMPDIR harness that simulates the Linux self-hosted runner's ambient vitest; the real ubuntu runner validates via this PR's own CI run.

Risk & Scope

  • Main risk or tradeoff: findVitestBin gains an optional parameter used only by tests; its default is the previous inline createRequire anchor, so production behavior is unchanged.
  • Not validated / out of scope: other tests in the same file that plant their own packages were already shadow-deterministic and are untouched; the runner's ambient node_modules itself is left as-is.
  • Breaking changes / migration notes: none.

Linked Issues

None.

中文说明

本 PR 做了什么

稳定 packages/cli/src/commands/review/test-efficacy.test.ts 中两个依赖环境的测试——它们在能从 os.tmpdir() 向上解析到 vitest 的主机上会失败。两个测试的断言全部逐字保留,只把失败条件的构造方式改为在任何宿主上都确定。findVitestBin 增加一个可选解析器参数,其默认值与之前的内联行为完全一致;另一个测试改为种植本地 shadow 包,不再依赖裸 tmpdir 上方恰好没有 vitest。

为什么需要

CI 的 ubuntu Test 任务以 AssertionError: expected [Function] to throw an error 失败于两个用例:findVitestBin > names the search root when vitest cannot be resolvedrunControlMutant > leaves the probe file byte-identical when it cannot run(PR run 30909672420;最新 main run 30893612230 同签名)。两个测试都假设 os.tmpdir() 向上没有任何 vitest/package.json——在 macOS 开发机上成立,在项目自托管 runner 上不成立(TMPDIR 之上的 node_modules 提供了 vitest)。该环境下第一个测试从不抛错,第二个则真实执行了探针套件而非抛错。

Reviewer Test Plan

如何验证

  1. 本地复现 CI 条件:mkdir -p /tmp/qwen-fake-tmp/node_modules && ln -s <repo>/node_modules/vitest /tmp/qwen-fake-tmp/node_modules/vitest,然后 cd packages/cli && TMPDIR=/tmp/qwen-fake-tmp npx vitest run src/commands/review/test-efficacy.test.ts。旧代码下两个测试与 CI 完全一致地失败;本 PR 下整文件通过(128/128)。
  2. 确认生产调用路径未动:findVitestBin 在唯一生产调用点不传新参数,默认解析器(锚定 createRequire)与之前完全相同。
  3. 确认断言未变:两个测试仍断言相同的 toThrow(...) 消息与相同的探针文件字节恢复。
  4. cd packages/cli && npm run typecheck 加两个改动文件的 ESLint/Prettier——本地全部通过。

Evidence (Before & After)

改动前:两个测试在自托管 ubuntu runner 上失败(本地用上述 fake TMPDIR 装置同样复现)。改动后:整文件在复现环境与常规环境 4 轮连续通过(128/128)。完整证据见下方 e2e report 评论。

Tested on

OS Status
🍏 macOS
🪟 Windows ⚠️
🐧 Linux

Environment (optional)

macOS 本地运行,外加模拟 Linux 自托管 runner 环境 vitest 的 fake TMPDIR 装置;真实 ubuntu runner 由本 PR 自身的 CI 运行验证。

Risk & Scope

  • 主要风险或权衡:findVitestBin 新增仅测试使用的可选参数;默认值即原先内联的 createRequire 锚定,生产行为不变。
  • 未验证 / 不在范围内:同文件中其他自种包的测试本就具备 shadow 确定性,未改动;runner 上的环境 node_modules 维持原样。
  • 破坏性变更 / 迁移说明:无。

Linked Issues

无。

… vitest

Two tests failed on hosts where vitest resolves up-tree from os.tmpdir()
(observed on self-hosted CI where a node_modules above TMPDIR provides
one): findVitestBin's "cannot be resolved" case never threw, and
runControlMutant's "cannot run" case executed the probe for real instead
of throwing.

Make the failure conditions host-deterministic while keeping every
assertion: findVitestBin accepts an injected resolver (default
unchanged) so the MODULE_NOT_FOUND case is forced directly, and the
runControlMutant test plants a shadow vitest whose exports hide
package.json, which wins resolution from any ancestor install and makes
the run fail deterministically.
@wenshao

wenshao commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

E2E Report — deflake review test-efficacy tests

Flaky mechanism

packages/cli/src/commands/review/test-efficacy.test.ts had two environment-dependent tests, both failing on CI with AssertionError: expected [Function] to throw an error (run 30909672420, ubuntu-latest job; same signature as latest main run 30893612230):

  • findVitestBin > names the search root when vitest cannot be resolved — expects findVitestBin(<bare tmpdir>) to throw MODULE_NOT_FOUND.
  • runControlMutant > leaves the probe file byte-identical when it cannot run — expects the vitest run to never start, so the function throws out of the suite helper.

Both assumed vitest/package.json is not resolvable up-tree from os.tmpdir(). On the project's self-hosted runner, a node_modules above TMPDIR provides vitest, so resolution succeeds: test 1 never throws, and test 2 runs the probe for real (no throw). On macOS dev machines (/var/folders/... has nothing above it) both pass — the classic environment-dependent flake.

Which fix was applied and why

Allowed-fix class 4 (isolate/serialize interference): remove the dependence on ambient filesystem state, preserving every assertion verbatim.

  1. findVitestBin now accepts an optional injected resolver (default is the unchanged createRequire(join(worktree, 'noop.js')).resolve). The test injects a resolver that throws MODULE_NOT_FOUND directly. A vi.doMock('node:module') alternative was prototyped and empirically rejected — builtin imports inside user modules are externalized by this repo's vite-node setup and bypass the mock registry (verified with a throwaway probe test).
  2. The runControlMutant test plants a shadow node_modules/vitest inside the probe dir whose exports map hides package.json. The innermost node_modules wins Node resolution on every host, so findVitestBin surfaces ERR_PACKAGE_PATH_NOT_EXPORTED, runControlMutant throws through the same finally restore path, and the probe file stays byte-identical — on machines with and without an ambient vitest alike.

No assertion was weakened, skipped, or deleted; the production call path is byte-for-byte the default.

Repeated-run evidence

A/B reproduction harness on macOS: TMPDIR=/tmp/qwen-fake-tmp with node_modules/vitest symlinked to the repo's real vitest — reproduces the CI condition locally.

  • Old code + repro env: both tests FAIL (matches CI signature exactly).
  • New code + repro env: full file 128/128 pass.
  • New code + normal env: full file 128/128 pass, 4 consecutive runs (3 normal + 1 repro).
  • npm run typecheck (packages/cli), ESLint, Prettier: all clean.

中文说明

失败机制

packages/cli/src/commands/review/test-efficacy.test.ts 有两个依赖环境的测试,在 CI 上以 AssertionError: expected [Function] to throw an error 失败(run 30909672420,ubuntu-latest;与最新 main run 30893612230 同签名):两个用例都假设从 os.tmpdir() 向上解析不到 vitest/package.json。项目自托管 runner 上 TMPDIR 之上的 node_modules 提供了 vitest,解析成功 → 用例 1 不抛错,用例 2 真实执行了探针(也不抛错)。macOS 开发机上 /var/folders/... 上方无 vitest,所以通过——典型的环境依赖型 flaky。

修复方式

断言逐字保留,消除对环境文件系统状态的依赖:

  1. findVitestBin 增加可选注入解析器(默认行为不变),测试直接注入抛 MODULE_NOT_FOUND 的解析器。vi.doMock('node:module') 备选方案经探针实验证明在此仓库 vite-node 配置下无效(builtin 导入被 externalize,绕过 mock 注册表)。
  2. runControlMutant 测试在探针目录内种植一个 exports 不含 package.json 的 shadow vitest 包;最内层 node_modules 在任何宿主上都赢得解析,findVitestBin 抛出 ERR_PACKAGE_PATH_NOT_EXPORTEDrunControlMutant 经同一 finally 恢复路径抛错,探针文件保持字节一致。

未弱化、跳过或删除任何断言;生产调用路径与默认值完全一致。

重复运行证据

macOS 本地 A/B 复现:以 TMPDIR=/tmp/qwen-fake-tmp 并在其中 symlink 仓库真实 vitest,复现 CI 条件。旧代码 + 复现环境:两个测试都失败(与 CI 签名完全一致);新代码 + 复现环境:整文件 128/128 通过;新代码 + 常规环境:整文件 128/128 通过,连续 4 轮。packages/cli typecheck、ESLint、Prettier 全部通过。

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finished — CI landed green on d8bb32c and the deferred approval was posted. finalize run

Qwen Triage 已完成 —— d8bb32c 的 CI 全绿,延迟审批已提交。查看 finalize 运行

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

Template looks good ✓

Problem: observed, not theoretical. Verified from the CI logs: run 30909672420's Test (ubuntu-latest, Node 22.x) job fails with AssertionError: expected [Function] to throw an error on exactly the two cited cases — findVitestBin > names the search root when vitest cannot be resolved and runControlMutant > leaves the probe file byte-identical when it cannot run — and the same job is red on main run 30893612230. Both tests assumed nothing up-tree of os.tmpdir() provides a vitest/package.json, an ambient-filesystem dependency that holds on macOS dev machines but not on the self-hosted runner.

Direction: aligned — this is CI stability for the review command's own test-efficacy harness, exactly the class of environment-dependent flake this repo actively works to eliminate.

Size: not core (packages/cli/src/commands/review/** sits outside the core-module paths). 10 production lines (+7/−3 in test-efficacy.ts), 35 test lines (+31/−4). Well under every threshold.

Approach: the scope feels right. Every assertion is preserved verbatim; only the way each failure condition is produced changes, from "ambient filesystem happens to cooperate" to host-deterministic constructions — an injected resolver for the not-found path, and a planted shadow package whose exports map hides package.json for the control-mutant path. The latter reuses the exact construction an existing test in the same file already relies on, and the new resolver parameter defaults to the previous inline behavior, so the single production call site is untouched. No materially simpler path jumps out: the alternative would be mocking node:module, which the author reports prototyping and rejecting because this repo's vite-node setup externalizes builtin imports (author's claim from the E2E report, not independently re-run).

Risk: no elevated risk signals — no high-risk paths touched.

Moving on to code review. 🔍

中文说明

感谢贡献!

模板完整 ✓

问题:已观测到,不是理论问题。已从 CI 日志核实:run 30909672420 的 Test (ubuntu-latest, Node 22.x)AssertionError: expected [Function] to throw an error 失败于两个被点名的用例——findVitestBin > names the search root when vitest cannot be resolvedrunControlMutant > leaves the probe file byte-identical when it cannot run——main run 30893612230 同一任务同样失败。两个用例都假设 os.tmpdir() 向上解析不到 vitest/package.json,这一对环境文件系统的依赖在 macOS 开发机上成立、在自托管 runner 上不成立。

方向:对齐——这是 review 命令自身 test-efficacy 测试框架的 CI 稳定性修复,正是本仓库一直在消除的那类环境依赖型 flaky。

规模:非核心路径(packages/cli/src/commands/review/** 不在核心模块路径内)。生产代码 10 行(test-efficacy.ts +7/−3),测试代码 35 行(+31/−4),远低于所有阈值。

方案:范围合理。断言全部逐字保留,只把失败条件的构造方式从"环境文件系统恰好配合"改为主机确定的构造——not-found 路径注入解析器,control-mutant 路径种植 exports 隐藏 package.json 的 shadow 包。后者复用同文件既有测试已在使用的同款构造;新增解析器参数默认值即原先内联行为,唯一生产调用点不受影响。没有更简路径:备选方案是 mock node:module,作者在 E2E 报告中称已做原型并否决——本仓库 vite-node 配置会 externalize builtin 导入(此为作者陈述,未独立复核)。

风险:无升级风险信号——未触及高风险路径。

进入代码审查 🔍

Qwen Code · qwen3.8-max

Reviewed at d8bb32c757ddc6e2e7d5d4691b41e2b37553b2e7 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Code review

I formed my independent expectation before reading the diff — for two tests that assume nothing up-tree of os.tmpdir() provides vitest, the minimal fix is to produce each failure condition deterministically: inject the resolution failure for the not-found test, and plant a decoy package the resolver must hit first for the control-mutant test. The diff does exactly that, and I couldn't find a materially simpler path.

What I verified against the surrounding code:

  • The new resolveModule parameter defaults to byte-for-byte the previous inline behavior (createRequire(join(worktree, 'noop.js')).resolve), and the single production call site in runProbeSuite passes no second argument — production resolution is unchanged.
  • Test 1's injected resolver throws with code: 'MODULE_NOT_FOUND', which is precisely the branch in findVitestBin's catch that produces the asserted vitest not found searching up from … message — the test now exercises that branch on every host.
  • Test 2's shadow package survives exposeDependencies (which early-returns when probe tree and dependency root are the same directory, as they are here), and as the innermost node_modules it wins Node resolution regardless of any ambient vitest up-tree. Its exports map hides ./package.json, so resolution throws ERR_PACKAGE_PATH_NOT_EXPORTED, which findVitestBin re-throws verbatim (the non-MODULE_NOT_FOUND branch) and which propagates through runControlMutant's finally restore — exactly the path under test. The construction is the same one the neighboring keeps the real error when vitest is present but hides its package.json test already uses.
  • Both assertions are verbatim; no imports were added (mkdirSync/writeFileSync were already there); no unrelated edits anywhere in the diff.

No blockers, no convention violations.

Test evidence — this PR's own CI

The problem itself is proven from CI logs, not the author's word: run 30909672420's Test (ubuntu-latest, Node 22.x) job fails both cited tests with AssertionError: expected [Function] to throw an error (verified from the job log). This commit's own CI runs that same job on the same runner class — a green result there is a before/after proof on the host that flakes. macOS/Windows Test jobs show as skipped here by design: this repo's CI runs them only in the merge queue, not per-PR.

Bot orchestration checks (triage, review-pr) excluded from the table.

Final CI results for d8bb32c (auto-updated by the triage finalize job after CI completed):

Check Conclusion
Classify PR ✅ success
Desktop Shell (ubuntu-22.04) ✅ success
route ✅ success
Test (ubuntu-latest, Node 22.x) ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success

One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。

The decisive signal is still running: Test (ubuntu-latest, Node 22.x). Everything else that ran is green. No sandboxed-verification lane needed here — the claim is test determinism on the previously failing host, which this job settles directly, and production behavior is byte-for-byte the default.

中文说明

代码审查:先独立推演再对照 diff——两个假设 os.tmpdir() 向上没有 vitest 的测试,最小修法就是把失败条件改为主机确定:not-found 测试注入解析失败,control-mutant 测试种植一个解析器必然先命中的影子包。diff 正是这么做的,没有更简路径。已逐一核实:新 resolveModule 参数默认值与原先内联行为逐字节一致,runProbeSuite 中唯一生产调用点未传第二参数;测试 1 注入的解析器抛 code: 'MODULE_NOT_FOUND',恰好命中 findVitestBin catch 中产出断言消息的分支;测试 2 的影子包不会被 exposeDependencies 动到(两者同目录时提前返回),作为最内层 node_modules 必然先于环境 vitest 命中,其 exports 隐藏 package.json 触发 ERR_PACKAGE_PATH_NOT_EXPORTED,由 findVitestBin 原样重抛并穿过 runControlMutantfinally 恢复——正是被测路径,构造与相邻既有用例相同。断言逐字保留,无新增导入,无无关改动。无阻塞项,无规范违规。

测试证据:问题本身由 CI 日志证实(非作者自述)——run 30909672420 的 Test (ubuntu-latest, Node 22.x) 两个用例均以 AssertionError: expected [Function] to throw an error 失败(已从任务日志核实)。本提交的 CI 在同款 runner 上运行同一任务,变绿即为主机级 before/after 证明。macOS/Windows 任务按本仓库 CI 设计仅在合并队列运行,此处 skipped 属预期。决定性信号 Test (ubuntu-latest, Node 22.x) 仍在运行,其余已运行任务全绿。无需沙箱验证通道——本 PR 的主张是此前失败主机上的测试确定性,该任务即可直接定论,且生产行为与默认值逐字节一致。

Qwen Code · qwen3.8-max

Reviewed at d8bb32c757ddc6e2e7d5d4691b41e2b37553b2e7 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Confidence: 5/5 — verified problem, minimal fix, production path untouched; the only thing deferred is waiting for the decisive CI job to land green.

Reflection: I went back to my independent proposal — inject the failure for the not-found test, plant a deterministic decoy for the control-mutant test — and the diff is exactly that, with nothing extra. The failure is proven from real CI logs (run 30909672420), not from the author's word; both assertions survive verbatim; the single production call site is byte-for-byte unchanged; and every edit in the diff earns its place. The author has a high volume of open PRs, but this one stands entirely on its own evidence, and I'm approving it because it is genuinely good — an observed flake killed at the root, not papered over. Approval is deferred only because the Test (ubuntu-latest, Node 22.x) job that proves the fix on the previously failing runner class is still running on this commit.

Approval will land once CI is green on d8bb32c757ddc6e2e7d5d4691b41e2b37553b2e7.

中文说明

置信度:5/5 —— 问题已证实、修法最小、生产路径未动;唯一推迟的事项是等待决定性 CI 任务变绿。

复盘:回到我的独立方案——给 not-found 测试注入失败、给 control-mutant 测试种植确定性影子包——diff 正是如此,没有多余内容。失败由真实 CI 日志证实(run 30909672420),而非作者自述;两个断言逐字保留;唯一生产调用点逐字节未变;diff 中每一处改动都必要。作者有较多开放 PR,但本 PR 完全凭自身证据成立,我批准它是因为它确实好——从根上消除了观测到的 flaky,而不是掩盖它。推迟批准仅因为用于在此前失败 runner 类别上证明修复的 Test (ubuntu-latest, Node 22.x) 任务仍在该提交上运行。

d8bb32c757ddc6e2e7d5d4691b41e2b37553b2e7 的 CI 变绿后即批准。

Qwen Code · qwen3.8-max

Reviewed at d8bb32c757ddc6e2e7d5d4691b41e2b37553b2e7 · re-run with @qwen-code /triage

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship — CI landed green after the review. ✅

@wenshao

wenshao commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

Code Review — #8537

Verdict: LGTM, approve with two minor suggestions. I reproduced the CI condition and verified every claim in the PR body independently.

What it does

Two tests in test-efficacy.test.ts produced their failure condition by relying on os.tmpdir() having no vitest/package.json anywhere up-tree. That is true on macOS dev machines and false on the self-hosted ubuntu runner, where a node_modules above TMPDIR resolves one. The PR replaces the ambient dependency with two host-deterministic constructions: a MODULE_NOT_FOUND-throwing resolver injected via a new optional findVitestBin parameter, and a planted shadow vitest whose exports hides package.json.

Verification I ran

Built the fake-TMPDIR harness from the test plan (node_modules/vitest symlink above TMPDIR), then ran the file in a clean worktree of the PR head:

Case Result
Pre-PR code, fake TMPDIR ✗ 2 failed / 126 passed — AssertionError: expected [Function] to throw an error on exactly the two named tests. Matches the CI signature verbatim.
PR code, fake TMPDIR ✓ 128/128
PR code, clean TMPDIR ✓ 128/128
tsc --noEmit + eslint on both touched files clean (no diagnostics mentioning test-efficacy)

Mutation check — the reworked test still has teeth. I broke the production invariant it guards, moving the restore in runControlMutant out of finally onto the success path only:

const __v = perFile.some((r) => r.verdict === 'gated');
writeFileSync(abs, original, 'utf8');
return __v;
} finally { /* no longer survives the throw path */ }

× leaves the probe file byte-identical when it cannot run. The shadow-package rewrite did not weaken the assertion's ability to catch a restore that doesn't survive the throw. This was my main concern with the change and it is settled.

Production path is untouched. findVitestBin has exactly one production call site (test-efficacy.ts:1477), called with one argument, so the default resolver runs. createRequire(...).resolve extracted unbound is safe — Node's resolve closes over the module rather than using this; confirmed it still throws MODULE_NOT_FOUND correctly when detached. The default is also still exercised for real by the two sibling findVitestBin tests (declares no bin, hides its package.json), so the anchor itself remains covered.

Convention fit. The optional-parameter seam matches what this file already does — runControlMutant(…, now = Date.now), runOneMutant(…, dependencyRoot = probeTree). Consistent, not a new pattern.

Side benefit worth noting: the reworked test 2 now throws before any spawn, dropping from ~604ms (it was actually executing a real vitest run under the ambient-vitest condition) to ~10ms.


Suggestions

1. Pin test 2's assertion to the error the comment promises (test-efficacy.test.ts:349)

The comment states the point precisely — "findVitestBin surfaces the ERR_PACKAGE_PATH_NOT_EXPORTED" — but the assertion is a bare toThrow(). Since the entire change is about which throw the shadow package produces, a bare match doesn't hold that. Anything that throws earlier in a later refactor (probe-tree validation, a spawn failure, even a typo in the planted JSON) keeps this green while silently no longer testing the described path.

-      expect(() => runControlMutant(dir, 'a.test.ts')).toThrow();
+      expect(() => runControlMutant(dir, 'a.test.ts')).toThrow(
+        /not defined by "exports"/,
+      );

I applied this and it passes — the throw genuinely is that one. It also matches the sibling keeps the real error when vitest is present but hides its package.json test, which already asserts that exact regex.

2. mkdtempSync in test 1 is now dead weight (test-efficacy.test.ts:185)

With the resolver injected, worktree is only interpolated into the expected message — the directory is never touched. I confirmed the test passes with a path that does not exist at all. The real mkdtempSync now just leaks an unremoved no-vitest-* dir per run (verified against a scratch TMPDIR). Leaking is a pre-existing pattern in this file, but here it's newly pointless:

-    const worktree = mkdtempSync(join(tmpdir(), 'no-vitest-'));
+    // Never touched on disk — the injected resolver decides the outcome and
+    // the path only has to appear in the message.
+    const worktree = join(tmpdir(), 'no-vitest-unused');

(Alternatively keep mkdtempSync and wrap in try/finally { rmSync } like the runControlMutant tests do — but the dir buys nothing here.)

Note, not a defect

After this change nothing verifies the link "a genuine require.resolve failure carries code === 'MODULE_NOT_FOUND', therefore this branch fires." The branch body is covered via injection; its real-world trigger is now assumed. That is unavoidable — you cannot force a genuine not-found on a host with ambient vitest, and MODULE_NOT_FOUND is a stable Node contract. Worth one sentence in the findVitestBin JSDoc (which still describes only the createRequire anchor) so the next reader knows the second parameter is a test-only seam and why it exists.

Risk / security / performance

No production behavior change, no new dependency, no I/O outside per-test temp dirs, and the planted shadow package lives inside a mkdtemp tree removed in finally. Net performance is a small improvement. Nothing to flag.

@wenshao

wenshao commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

@qwen-code /takeover

@doudouOUC doudouOUC left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The default findVitestBin behavior remains unchanged, and the tests now exercise the intended error paths without relying on ambient TMPDIR node_modules layout.

@qwen-code-dev-bot qwen-code-dev-bot added the autofix/takeover Summon the autofix loop to manage this PR (remove to release; needs triage+) label Aug 4, 2026
@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. This is a fork PR, so the first round comes from the next scheduled scan (usually within minutes). Remove the autofix/takeover label (or comment @qwen-code /takeover stop) to release.

中文说明

🤝 已接管:autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突,直到移除标签或达到轮次上限。本 PR 来自 fork,首轮处理将由下一次定时扫描执行(通常几分钟内)。移除 autofix/takeover 标签(或评论 @qwen-code /takeover stop)即可释放。

@wenshao
wenshao added this pull request to the merge queue Aug 4, 2026
Merged via the queue into QwenLM:main with commit eacc85e Aug 4, 2026
59 of 60 checks passed
@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Released in v0.21.6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autofix/takeover Summon the autofix loop to manage this PR (remove to release; needs triage+)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants