fix(integration): scale interactive read-then-write waits with env timeout (#7942) - #7943
Merged
Conversation
Collaborator
Author
E2E Report — issue #7942Failure under investigationThe issue tracks a main-branch By inspecting the run via the public GitHub API, the failure was isolated to the The raw job log could not be retrieved: the GitHub Actions log endpoint returns Diagnosis (evidence from code inspection)This interactive test runs against a real model (it configures
Those two 15s waits are the only waits in the test that don't scale, and they are FixScale the two Environment-specific check not run hereThe failing path is exercised only by the bundled CLI inside the Docker sandbox Verification
中文说明E2E 报告 — issue #7942正在排查的失败该 issue 跟踪的是 main 分支 通过公开的 GitHub API 查看该 run，失败被定位到 无法获取原始 job 日志：GitHub Actions 的日志接口返回 诊断（来自代码检查的证据）这个交互式测试针对的是 真实模型（它配置了
这两处 15s 等待是该测试中唯一不会伸缩的等待，并且它们是对模型延迟最敏感的步骤 修复将两处 此处未运行的环境相关检查失败的路径只有在 Docker 沙箱内、针对真实模型、通过打包后的 CLI 才能被触发。 验证
🧠 Handled by Qwen Code · model/模型 |
github-actions
Bot
added
the
review/self-reported
Collaborator
|
✅ Qwen Triage finished — CI landed green on ✅ Qwen Triage 已完成 —— |
Collaborator
|
Thanks for the PR! Template looks good ✓ Problem: observed bug with evidence — linked issue #7942 documents a real main-branch CI failure (run 30366660016) in the Direction: aligned — test reliability is core infrastructure work. The fix makes two hard-coded 15s waits use the same environment-aware Size: not applicable — no core paths touched. Single integration test file, 8 additions / 2 deletions. Approach: the scope is exactly right. Two lines change, each replacing Risk: no elevated risk signals — no high-risk paths matched. Moving on to code review. 🔍 中文说明感谢贡献！ 模板完整 ✓ 问题：已观测到的 bug，有证据——关联 issue #7942 记录了 main 分支 CI 的真实失败（run 30366660016），仅在 方向：对齐——测试可靠性是核心基础设施工作。修复让两处硬编码的 15s 等待使用与测试其余部分相同的环境感知 规模：不适用——未触及核心路径。单个集成测试文件，8 行新增 / 2 行删除。 方案：范围恰好。两行改动，各将 风险：无升级风险信号——未匹配高风险路径。 进入代码审查 🔍 — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
Code ReviewIndependent proposal: given a Docker-sandbox timing flake where two Comparison with the diff: the PR does exactly this. Two No critical blockers. No AGENTS.md violations. The change reuses the existing CI Test EvidenceCI is still running on Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行（取最新一次运行），省略 skipped，失败项排在最前。 The unit test suite and integration tests have not completed yet. The Real-scenario testing: N/A — test-infrastructure timing change with no user-visible product behavior. This is an unattended CI run; tmux testing does not apply. 中文说明代码审查独立方案： 鉴于 Docker 沙箱时序偶发失败——两处 与 diff 对比： PR 正是这样做的。两处 无关键阻塞项。无 AGENTS.md 违规。改动复用已有的 CI 测试证据CI 仍在 真实场景测试： N/A——测试基础设施时序改动，无用户可见的产品行为变更。本次为无人值守 CI 运行；tmux 测试不适用。 — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
|
Confidence: 5/5 — clean across every stage; would merge without hesitation. This is exactly what a flaky-test fix should look like. The problem is real — a main-branch CI run failed on the Docker sandbox shard while the same test passed everywhere else, and the root cause is two hard-coded 15s waits that don't scale with the environment while every neighboring wait already does. The fix is two lines, each replacing My independent proposal before reading the diff was identical to what the PR does. There is no simpler path. If I had to maintain this in six months, I'd thank the author — it's the kind of minimal, well-justified fix that makes the test suite more reliable without adding complexity. Approval deferred until CI lands green on 中文说明置信度：5/5 —— 每个阶段都干净；毫不犹豫可以合并。 这正是不稳定测试修复应有的样子。问题是真实的——main 分支 CI 运行仅在 Docker 沙箱分片失败，同一测试在其他所有环境均通过，根因是两处硬编码的 15s 等待不随环境伸缩，而相邻的所有等待已经如此。修复是两行，各将 我在阅读 diff 之前的独立提案与 PR 的做法完全一致。不存在更简路径。如果六个月后需要维护这段代码，我会感谢作者——这是那种最小化、有充分理由的修复，让测试套件更可靠而不增加复杂度。 审批延迟至 CI 在 — Qwen Code · qwen3.8-max-preview Reviewed at |
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
previously approved these changes
Jul 28, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
中文说明
已审查——无阻断问题。 建议见行内评论。
— qwen3.7-max via Qwen Code /review
Collaborator
Author
Collaborator
Author
|
🤖 Addressed the latest review feedback (round 1/10). What changed, and what I pushed back on: · 已处理最新评审反馈（第 1/10 轮）。改动内容与我反驳保留之处如下： Review feedback addressed[rc:3666937227] Hardcoded
|
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
中文说明
已审查——无阻断问题。 建议见行内评论。
— qwen3.7-max via Qwen Code /review
| await type(ptyProcess, '\r'); | ||
|
|
||
| const readCall = await rig.waitForToolCall('read_file', 30000); | ||
| const readCall = await rig.waitForToolCall('read_file'); |
Collaborator
There was a problem hiding this comment.
[Suggestion] Removing the explicit 30000 timeout makes this call fall through to getDefaultTimeout(), which returns 15 000 ms locally — halving the previous 30 s budget for a model-dependent tool-call wait. The same applies to waitForAnyToolCall at line 70. — Failure scenario: a developer running the test locally against a slow or rate-limited model endpoint sees a false timeout at 15 s where the test previously allowed 30 s.
Note: every other waitForToolCall/waitForAnyToolCall call site in the suite already uses the default (15 s locally), so the old 30 s was an outlier. If 15 s is intentionally sufficient, this is fine; if not, consider Math.max(30000, rig.getDefaultTimeout()) to preserve the floor.
中文说明
[Suggestion] 移除显式的 30000 超时后，此调用会回退到 getDefaultTimeout()，在本地环境下返回 15 000 ms——将之前模型相关的工具调用等待的 30 秒预算减半。第 70 行的 waitForAnyToolCall 也存在同样的情况。—— 失败场景：开发者在本地对慢速或限速的模型端点运行测试时，会在 15 秒时看到误报超时，而此前测试允许 30 秒。
注意：套件中所有其他 waitForToolCall/waitForAnyToolCall 调用点已在使用默认值（本地 15 秒），因此原来的 30 秒是一个异常值。如果 15 秒已足够，这没有问题；如果不够，可以考虑使用 Math.max(30000, rig.getDefaultTimeout()) 来保留下限。
— qwen3.7-max via Qwen Code /review
Collaborator
Author
There was a problem hiding this comment.
Declined — the PR's purpose is to normalize hardcoded timeouts to the env-aware default. The old 30 s was an outlier: every other waitForToolCall/waitForAnyToolCall call site in the suite already uses the default (15 s locally, 60 s in CI). Adding Math.max(30000, …) would re-introduce the hardcoded floor this PR removes and make this file inconsistent with the rest of the suite again.
中文说明
已拒绝——本 PR 的目的是将硬编码超时统一为环境感知的默认值。原来的 30 秒是一个异常值：套件中所有其他 waitForToolCall/waitForAnyToolCall 调用点已在使用默认值（本地 15 秒、CI 60 秒）。添加 Math.max(30000, …) 会重新引入本 PR 要移除的硬编码下限，并使此文件再次与套件其余部分不一致。
Collaborator
Author
|
🤖 Addressed the latest review feedback (round 2/10). What changed, and what I pushed back on: · 已处理最新评审反馈（第 2/10 轮）。改动内容与我反驳保留之处如下： Review feedback — round summaryrc:3667406441 —
|
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
中文说明
— qwen3.7-max via Qwen Code /review
Collaborator
Author
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： Autofix review round — no action neededNo changes were made this round. The only feedback newer than the last evaluation is an automated The PR (#7943, issue #7942) remains a small, scoped integration-test change 中文说明Autofix 审查轮次 —— 无需操作本轮未做任何更改。 相对上次评估，唯一更新的反馈是一条自动化的 PR（#7943，issue #7942）仍然是一个小而范围明确的集成测试改动 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
doudouOUC
approved these changes
Jul 28, 2026
doudouOUC
left a comment
doudouOUC
left a comment
Collaborator
There was a problem hiding this comment.
No issues found. LGTM! ✅
中文说明
未发现问题。LGTM！✅
— qwen3.7-max via Qwen Code /review
Collaborator
|
@qwen-code /verify |
Collaborator
|
Sandboxed verification: merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. 沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 Scripted assertions: 19 passed · 0 failed · 19 total 脚本断言：19 通过 · 0 失败 · 19 总计 Verification report (report.md)Harness scripts and raw logs are in the workflow run artifacts (7-day retention). — Qwen Code · sandboxed verification |
Collaborator
Author
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： No action takenThe only new feedback is an automated sandboxed verification report (
No inline comments, no reviews requesting changes, and no failed checks are present. No code change is warranted. 中文说明未采取任何操作唯一的新反馈来自自动化沙箱验证报告（
没有行内评论、没有要求更改的评审、没有失败的检查。无需进行代码更改。 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
yiliang114
approved these changes
Jul 29, 2026
Collaborator
|
Released in v0.21.2. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
This PR hardens a flaky interactive end-to-end test by making two of its waits scale with the environment instead of using a fixed 15-second timeout. The interactive read-then-write test waits for several asynchronous steps: the CLI startup banner, a
read_filetool call, the model's summary of the file contents, awrite_file/edittool call, and finally the file reflecting the new version. Every one of those waits already scales with the environment (30 seconds for tool calls, and the environment-aware default timeout for the final file poll) except the two text-output waits, which were pinned to 15 seconds. This change brings those two in line with the rest of the test by using the same environment-aware default timeout (15s locally, 30s in a sandbox, 60s in CI).Why it's needed
A main-branch
E2E Testsrun failed on this test only in theE2E Test (Linux) - sandbox:dockershard; the same test passed in everysandbox:noneLinux shard and on macOS in the same run, which points to a Docker-sandbox-specific timing flake rather than a product regression. The test drives a real model, and the integration suite retries each test twice, so all three attempts timed out in the Docker shard. The two 15-second text waits are the most model-latency-sensitive steps in the test (each waits for a full real-model round-trip to render) and, under Docker container startup plus slower CI model responses, 15 seconds is routinely exceeded while the neighboring waits tolerate 30–60 seconds. Scaling them with the environment removes that inconsistency. Because the wait helper polls and returns as soon as the text appears, the happy path is unaffected; only the slow path that previously flaked gets more headroom.Reviewer Test Plan
How to verify
This is a test-only timing change with no product-code impact. Confirm the waits now scale by reading the diff: the two
waitForText(...)calls passrig.getDefaultTimeout()instead of15000, matching the finalpoll(...)in the same test. The worst-case total wait budget is 60+30+60+30+60 = 240s, under the 5-minute vitesttestTimeout, so a fully-failing run still fails cleanly rather than being killed by the global timeout. The definitive signal is the Docker E2E shard: this test should stop timing out at the startup-banner and post-read summary steps insandbox:docker.Evidence (Before & After)
N/A — non-UI, test-infrastructure change.
Tested on
Environment (optional)
Linux runner:
npm run build,npm run typecheck,npm run lint, and Prettier/ESLint on the changed file. The Docker-sandbox interactive E2E run itself was not executed locally (no Docker E2E lane or model credentials on this runner); the workflow's CI reproduces that shard and is the final gate.Risk & Scope
Linked Issues
Fixes #7942
中文说明
本 PR 做了什么
本 PR 通过让其中两处等待随环境伸缩、而不是使用固定的 15 秒超时，来加固一个偶发失败的交互式端到端测试。这个交互式“先读后写”测试会等待若干个异步步骤：CLI 启动横幅、一次
read_file工具调用、模型对文件内容的总结、一次write_file/edit工具调用，以及最终文件反映出新的版本。这些等待中，除了两处文本输出等待被固定为 15 秒之外，其余的都已经随环境伸缩（工具调用 30 秒，最终的文件轮询使用环境感知的默认超时）。本改动通过使用同一个环境感知的默认超时（本地 15s、沙箱 30s、CI 60s），让这两处与测试的其余部分保持一致。为什么需要它
一次 main 分支的
E2E Tests运行只在E2E Test (Linux) - sandbox:docker分片上让这个测试失败了；在同一次运行中，同一个测试在所有sandbox:none的 Linux 分片以及 macOS 上都通过了，这指向一个 Docker 沙箱特有的时序偶发失败，而非产品回归。该测试驱动的是真实模型，并且 integration 套件会对每个测试重试两次，因此在 Docker 分片中是三次尝试全部超时。这两处 15 秒的文本等待是该测试中对模型延迟最敏感的步骤（每一处都在等待一次完整的真实模型往返渲染出来），而在 Docker 容器启动加上较慢的 CI 模型响应之下，15 秒经常被超过，而相邻的等待可以容忍 30–60 秒。让它们随环境伸缩消除了这一不一致。由于该等待 helper 采用轮询、一旦文本出现就立即返回，正常路径不受影响；只有此前会偶发失败的慢路径获得了更多余量。评审者测试计划
如何验证
这是一个仅涉及测试的时序改动，对产品代码没有影响。通过阅读 diff 来确认这些等待现在会伸缩：两处
waitForText(...)调用传入的是rig.getDefaultTimeout()而不是15000，与同一个测试中最后的poll(...)保持一致。最坏情况下的总等待预算为 60+30+60+30+60 = 240s，低于 5 分钟的 vitesttestTimeout，因此一次完全失败的运行仍会干净地失败，而不是被全局超时杀掉。最明确的信号是 Docker E2E 分片：这个测试应当不再在sandbox:docker中的启动横幅和读取后总结这两个步骤上超时。证据（Before & After）
N/A — 非 UI、测试基础设施改动。
测试环境
环境（可选）
Linux runner：
npm run build、npm run typecheck、npm run lint，以及对改动文件的 Prettier/ESLint。Docker 沙箱交互式 E2E 运行本身未在本地执行（本 runner 没有 Docker E2E 通道或模型凭据）；工作流的 CI 会复现该分片，并作为最终关口。风险与范围
关联 Issue
Fixes #7942