fix(autofix): stop counting idle timeouts toward the timeout cap - #9673
Conversation
TIMEOUT_WINDOW_CAP exists to stop a PR that is too big to finish a round inside the agent's time budget, and its remedy says exactly that: split the PR or raise the budget. A silent-sandbox (idle) timeout is a different failure entirely — the idle watchdog kills the round because the sandbox produced no output at all, so no budget increase cures it and nothing about the PR caused it. Counting a failure whose prescribed remedy is inapplicable is what parked healthy PRs: over the 14 days to 2026-08-21 idle rounds were 58 of 119 timeouts, every one of the 51 windows that tripped this cap tripped it at exactly three, and 9 of the 12 PRs then carrying autofix/needs-human had been stopped here — #8332 at 24 rounds, #8368 at 28, #8276 at 16, all still pushing rounds when they were parked. Gate the cap on budget timeouts alone. A persistently wedged sandbox stays bounded, because an idle round pushes nothing and resets no streak, so CONSECUTIVE_FAILURE_CAP still terminates it; what no longer terminates is idle rounds interleaved with real progress, where the PR is not stuck and the runner is. Idle rounds stay visible through a job-log warning, which reaches whoever owns the runners without spending a comment on someone's PR. Two consequences inside the census. The idle needle became the full emitted headline prefix rather than a bare substring, because the count is now subtracted and must be a subset of the total — a loose needle could match provider error text on the same line and drive the difference negative. And the all-idle remedy branch is gone as unreachable: the guard now fires only when budget timeouts alone reach the cap, so a counted window always holds more of them than idle ones.
|
Gate passes — moving on to code review. Template: complete, bilingual, and honest about what wasn't validated. ✓ Problem: observed, not theoretical. The fleet measurement is concrete: 119 timeouts over 14 days, 58 of them idle; 51 windows tripped this cap, every one at exactly three; nine of the twelve PRs then carrying Direction: aligned. The cap exists to stop PRs too big to finish a round inside the agent budget; a silent-sandbox timeout is an infra failure with a different owner and no budget cure. Counting it conflates the two. No product-surface escalation triggers (this is autofix fleet plumbing, not product auth/sandbox/telemetry). Size: no core paths touched — ~73 production lines in the workflow YAML, plus a design record (+66) and tests (+267/−55). Every file serves the one change; nothing to split. Approach: the scope feels right — exclude idle from the cap census, keep it visible (stop-notice clause + job-log warning), and lean on the consecutive-failure cap for the persistently-wedged runner. The two consequential side changes — tightening the idle needle to a strict subset of the timeout needle, and deleting the now-unreachable all-idle remedy branch — both fall out of the exclusion and are reasoned in design record Risk: no elevated signals — no high-risk paths touched, and the fleet shepherd's reason extraction keys on the 中文说明门检通过——进入代码审查。 模板：完整、双语、对未验证项如实说明。✓ 问题：已观测到，非理论性问题。全队测量数据具体：14 天内 119 次超时，其中 58 次是 idle；51 个窗口触发了该上限，且无一例外都在第三次触发；当时携带 方向：对齐。该上限的存在是为了拦住大到无法在 agent 预算内跑完一轮的 PR；静默 sandbox 超时是基础设施故障，责任人不同，提高预算也治不了。把它计入会混淆两类失败。无产品层面升级信号（这是 autofix 队列自身的机制，不涉及产品侧的 auth/sandbox/telemetry）。 规模：未触及核心路径——工作流 YAML 约 73 行生产逻辑，另有设计档案（+66）与测试（+267/−55）。每个文件都服务于同一处改动，无可拆分项。 方案：范围合理——把 idle 从上限统计中排除，但保持可见（停机通知子句 + job 日志告警），持续挂死的 runner 交给连续失败上限兜底。两个连带改动——把 idle 匹配串收紧为超时匹配串的严格子集、删除已不可达的"全 idle 换建议"分支——都是排除逻辑的自然结果，设计档案 风险：无升级信号——未触及高风险路径；fleet shepherd 提取停机原因依据的是 — Qwen Code · qwen3.8-max Reviewed at |
Code reviewNo blockers. I wrote my independent proposal before reading the diff — count budget and idle timeouts separately, gate the cap on the budget count, keep idle visible on the notice and in the job log, rely on the consecutive-failure cap for a persistently wedged runner — and this PR matches it, then goes further in the right ways. What I verified in the code:
The one commit since the last review pass is the fixture dedup I'd flagged as the only carry-over item: the local Testing — CI on the reviewed commitAll checks on
The ubuntu unit suite is the load-bearing one — it contains Sandboxed verification — completedThe Two honesty notes on that evidence. It verified the prior head ( Nothing user-visible or TUI here — the change lives in an autofix CI job's decision logic — so real-scenario tmux testing is N/A; the replay harness (unit suite) and the A/B run above are the designated oracles, and both are green. 中文说明代码审查：无阻断问题。读 diff 前先写了独立方案（预算与 idle 分开计数、上限只按预算数判定、idle 保持可见、持续挂死交给连续失败上限），本 PR 与之一致且做得更多。已逐条核实：减法不会为负（idle 匹配串是超时匹配串的严格延伸，当前轮 glob 同样是子集，两处自增对称）；安全网成立（idle 轮不推送、不匹配任何重置短语，仍计入 测试：所审提交上 15 项成功、43 项按设计跳过（macOS/Windows 单元与集成测试在 沙箱验证：上次审查时在进行中的 — Qwen Code · qwen3.8-max Reviewed at |
|
Confidence: 5/5 — clean across every stage, and the two reservations from the last pass are both closed: the behavioural claim now has independent A/B proof, and the fixture dedup landed. Stepping back: this is the rare infrastructure PR where the numbers do the arguing. A breaker whose own message says "no budget increase can cure this" was counting the uncureable failure toward a cap whose only remedies are budget-shaped, and 9 of 12 parked PRs show the consequence. The fix removes exactly that conflation and nothing else — no drive-by edits, no speculative flexibility. The worst case stays bounded and named: a runner that stays broken still terminates at the consecutive-failure cap, and interleaved idle rounds (the case no longer stopped) are precisely the case where the loop is making progress and the cost is runner time, not a stranded PR. What pushed this from 4/5 to 5/5 since the last pass: the sandboxed A/B run proved the central claim without leaning on the fleet statistics at all — the base arm reproduces the harm, the head arm fixes it, and 10/10 mutants die, six of which the author never claimed to have tested. The fleet numbers are motivation now, not load-bearing evidence. And the one cosmetic carry-over (the duplicated idle-headline fixture) is deduplicated in the final commit, which is also the whole delta — nothing new to second-guess. The tests are the kind that make maintenance cheaper: needles extracted from the workflow and compared against each other and the real runner template, headline counts pinned numerically in both languages, and a replay harness that executes the actual block rather than asserting about it. If I had to maintain this in six months, the design record and the mutation-probe comments would tell me exactly what each line is for. Approving because it's genuinely good — observed problem, minimal fix, load-bearing tests, green CI on the reviewed commit, an independent sandboxed verification that agrees, and a maintainer approval already on the head. Housekeeping note: the two standing change-request reviews are from earlier 中文说明信心：5/5 —— 各阶段均干净，且上次审查的两点保留都已关闭：行为主张现在有了独立的 A/B 证明；夹具去重也已落地。 退一步看：这是一条少见的、让数字自己说话的基础设施 PR。一个熔断器自己的文案都写着"提高预算也治不了"，却把这种治不了的失败计入一个补救措施全是预算形状的上限——12 个被停摆 PR 中的 9 个就是后果。本修复恰好只移除这一处混淆，别无其他：没有顺手改动，没有投机性的灵活设计。最坏情形有界且已点名：持续坏着的 runner 仍会在连续失败上限处终止；不再被终止的"idle 与推送交替"情形，恰恰是循环正在推进、代价只是 runner 时间而非 PR 搁浅的情形。 从上次的 4/5 到 5/5，原因在于：沙箱 A/B 运行完全不依赖全队统计数据就证明了核心主张——base 侧复现损害、head 侧修复、10/10 变异全部被杀死，其中六个是作者并未声明测试过的。全队数字现在是动机，不再是承重证据。唯一的外观遗留项（重复的 idle 标题夹具）也在最后一个提交里去重了——它就是全部增量，没有新的疑点。这些测试让维护变得更轻松：匹配串从工作流中抽出相互比对、并与真实 runner 模板比对，标题计数在两种语言里数值钉死，回放 harness 真正执行代码块而非字面断言。若半年后由我维护，设计档案与变异探针注释能逐行说明每处代码的用途。 批准是因为它确实好：问题已观测到、修复最小化、测试真正起作用、所审提交 CI 全绿、独立沙箱验证结论一致，且维护者已在当前 head 上批准。备忘：两条处于生效状态的"需要修改"评审来自此前针对不同提交的 /review 轮次，其唯一 Critical（ — Qwen Code · qwen3.8-max Reviewed at |
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
doudouOUC
left a comment
doudouOUC
left a comment
There was a problem hiding this comment.
Review: fix(autofix): stop counting idle timeouts toward the timeout cap
Commit: 536d2096f94229c185955e09f3e0eaa238f4113e
Files: 3 files (+172/−69) — .github/workflows/qwen-autofix.yml, .github/workflows/qwen-autofix.md, scripts/tests/qwen-autofix-workflow.test.js
Summary
The change is well-motivated, minimal, and sound. The core logic — subtracting IDLE_N from TIMEOUT_N to get BUDGET_TIMEOUT_N and gating the cap on the budget-only count — is correct. The tightened needle is required because the idle count is now subtractive, and the deleted all-idle remedy branch is provably unreachable once the guard fires on BUDGET_TIMEOUT_N alone. The safety net (CONSECUTIVE_FAILURE_CAP for persistently wedged sandboxes) holds. The test coverage thoroughly exercises the key behavioral outcomes.
I verified the following load-bearing details:
-
The subtraction is safe.
IDLE_N's needle ('AutoFix ran out of time before finishing (idle-timeout') is a strict prefix ofTIMEOUT_N's needle ('AutoFix ran out of time before finishing'), so the idle count is always a subset of the total. The current-round increments are symmetric (any non-emptyAGENT_TIMEOUTbumpsTIMEOUT_N, theidle-timeoutprefix additionally bumpsIDLE_N), soBUDGET_TIMEOUT_Ncannot go negative. -
The all-idle branch is correctly removed. The guard now gates on
BUDGET_TIMEOUT_Nalone, so when it fires,TIMEOUT_N = budget + idle \u2265 cap + idle > idlealways — the oldif [[ "${IDLE_N}" -ge "${TIMEOUT_N}" ]]swap can never be reached. -
The safety net works. An idle round pushes nothing and matches no streak-reset needle, so a persistently wedged sandbox still terminates at
CONSECUTIVE_FAILURE_CAP. The tests pin this exact path. -
The \ sentinel in the replay harness correctly isolates the result from
::warning::job-log noise. -
The fleet shepherd compatibility is maintained:
🤖 AutoFix stoppedprefix is unchanged, andtime-budget exhaustionsremains a substring of the new headline.
Finding: Missing TOC entry for \ in \
The CI run on this commit has a failing test: Test (ubuntu-latest, Node 22.x). The failure is in scripts/tests/workflow-size.test.js:79 (lists every section in the contents table):
AssertionError: expected [ 'af-001', 'af-002', 'af-003', \u2026(67) ] to deeply equal [ 'af-001', 'af-002', 'af-003', \u2026(68) ]
The test (added by #9517, which is in the PR branch but not yet in main) extracts TOC entries from qwen-autofix.md and compares them against <a id="..."> anchor tags. The PR adds the af-073 section with an anchor but does not add a corresponding entry to the table of contents at the top of the file. The test expects the two lists to match, and af-073 is present in the anchors but absent from the TOC.
The fix is to add a TOC entry for af-073 after the af-072 entry in .github/workflows/qwen-autofix.md. Following the existing format (truncated to fit the 80-char line):
- [73. review-address \u00b7 Report dry-run / failure \u2014 Idle (silent-sandbox) timeouts are EXCLUDED from the cumulative timeout\u2026](#af-073)This is the only CI failure on this PR. The triage bot's review (stage 2, confidence 4/5) covered the code exhaustively and found no other issues.
Verdict
No other code issues found. Once the TOC entry is added, the CI should pass and the PR is ready for approval.
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
未检查（工具限制，非阻断）：the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max via Qwen Code /review (v0.21.15)
|
|
||
| <a id="af-073"></a> | ||
|
|
||
| ### 73. review-address · Report dry-run / failure — Idle (silent-sandbox) timeouts are EXCLUDED from the cumulative timeout cap. |
There was a problem hiding this comment.
[Critical] The new af-073 anchor is added without a Contents-table entry, and that fails this repo's existing design-record consistency test scripts/tests/workflow-size.test.js ("lists every section in the contents table"): the test asserts the TOC link targets equal every <a id="af-N"> anchor in this file. This diff adds anchor af-073 but the table still ends at entry 72, so the suite fails — the PR's own Test (ubuntu-latest, Node 22.x) lane is red with this exact assertion right now.
Witness:
PR arm at 536d2096f9:
FAIL scripts/tests/workflow-size.test.js > lists every section in the contents table
AssertionError: expected [ 'af-001', …, 'af-072' ] to deeply equal [ …, 'af-073' ]
Tests 1 failed | 56 passed (57)
BASE arm (same test, input files at merge base 0dd518f950): Tests 57 passed (57)
Live CI: gh pr checks 9673 → Test (ubuntu-latest, Node 22.x) fail
Add the entry for section 73 to the Contents block (after the entry-72 line, ~line 126), matching the house style — the test's regex only requires the - [73. ...](#af-073) shape:
- [73. review-address · Report dry-run / failure — Idle (silent-sandbox) timeouts are EXCLUDED from the cumulative timeout cap.…](#af-073)
中文说明
新增的 af-073 锚点没有对应的目录条目，这会使仓库既有的设计档案一致性测试 scripts/tests/workflow-size.test.js（"lists every section in the contents table"）失败：该测试断言目录链接目标与本文件中的每个 <a id="af-N"> 锚点一一对应。本次 diff 添加了锚点 af-073，但目录仍止于第 72 条，因此该测试套件失败——本 PR 自己的 Test (ubuntu-latest, Node 22.x) 通道现在正是因为这条断言而变红。
（证据见上方代码块：PR 侧该测试 1 failed | 56 passed；换回合并基线的输入文件后 57/57 全绿；线上 CI 同一断言失败。）
修复：在 Contents 区块（第 72 条之后，约第 126 行）为第 73 节添加条目，沿用既有风格——测试的正则只要求 - [73. ...](#af-073) 这一形状。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| # The needle is the emitted headline verbatim AND a prefix of | ||
| # TIMEOUT_N's, so IDLE_N can never exceed it and the | ||
| # subtraction below can never go negative. |
There was a problem hiding this comment.
[Suggestion] This comment states the subset invariant backwards: the IDLE needle is a strict extension of TIMEOUT_N's needle, not "a prefix of TIMEOUT_N's" — contradicting the sibling comment this same diff adds in scripts/tests/qwen-autofix-workflow.test.js ("its needle must be a strict extension of TIMEOUT_N's"). The non-negativity guarantee holds because every line matching the IDLE needle also matches TIMEOUT_N's needle, i.e. TIMEOUT_N's needle is the prefix. A maintainer rewording the idle headline and validating a replacement needle against this comment enforces the wrong direction; a needle shorter than TIMEOUT_N's (e.g. a revert to the bare 'idle-timeout' substring this PR removes) could match provider-error text that API_ERROR_DETAIL puts on a first line TIMEOUT_N does not count, driving BUDGET_TIMEOUT_N negative and silently disabling the cap.
| # The needle is the emitted headline verbatim AND a prefix of | |
| # TIMEOUT_N's, so IDLE_N can never exceed it and the | |
| # subtraction below can never go negative. | |
| # The needle is the emitted headline prefix AND a strict | |
| # extension of TIMEOUT_N's needle, so every line it matches | |
| # TIMEOUT_N's also matches, IDLE_N can never exceed it, and | |
| # the subtraction below can never go negative. |
中文说明
这段注释把子集不变量说反了：IDLE 的匹配串是 TIMEOUT_N 匹配串的严格扩展，而不是"TIMEOUT_N 的前缀"——与本 diff 在 scripts/tests/qwen-autofix-workflow.test.js 中新增的姊妹注释（"its needle must be a strict extension of TIMEOUT_N's"）自相矛盾。非负性之所以成立，是因为每一行能匹配 IDLE 匹配串的文本必然也匹配 TIMEOUT_N 的匹配串，即 TIMEOUT_N 的匹配串才是前缀。若维护者改写 idle 标题并依据这段注释来验证替换后的匹配串，会把方向搞反；比 TIMEOUT_N 更短的匹配串（例如退回本 PR 刚删掉的裸 'idle-timeout' 子串）可能命中 API_ERROR_DETAIL 放在首行、而 TIMEOUT_N 并不统计的模型报错文本，使 BUDGET_TIMEOUT_N 变成负数，从而悄悄禁用这条上限。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| # The needle is the emitted headline verbatim AND a prefix of | ||
| # TIMEOUT_N's, so IDLE_N can never exceed it and the | ||
| # subtraction below can never go negative. | ||
| IDLE_N="$(grep -c 'AutoFix ran out of time before finishing (idle-timeout' <<< "${PRIOR_HEADS}" || true)" |
There was a problem hiding this comment.
[Suggestion] The idle-vs-budget reclassification this PR introduces is applied at only one of the census sites that count the same timeout headline: the sibling PRIOR_TIMEOUTS site (yml:4993-5008) still counts idle rounds as budget exhaustions and feeds that attribution into the agent's prompt via feedback.md. A healthy PR that suffers one silent-sandbox round (sandbox wedges at startup, agent killed at the idle tick having emitted nothing) is told on the next round "1 round(s) since the last successful round exhausted the agent time budget before finishing anything" — attributing to budget use what af-073 (added by this PR) declares is infra, so the agent adapts strategy (scope reduction, approach change) to a budget problem that does not exist. Decide the policy for the sibling site and make it explicit: either apply the same idle exclusion there (subtract lines matching 'AutoFix ran out of time before finishing (idle-timeout' from PRIOR_TIMEOUTS, reusing the exact needle), or state in af-073 that the feedback warning deliberately counts idle rounds because its purpose (prior work may be unfinished) holds for them too. The milestone-digest N_TIMEOUT bucket (yml:6202) needs no change — its "timeout(s)" label makes no budget attribution.
中文说明
本 PR 引入的 idle/预算重分类只应用于统计同一标题的多个普查点中的一个：姊妹普查点 PRIOR_TIMEOUTS（yml:4993-5008）仍把 idle 轮次计为预算耗尽，并通过 feedback.md 把这个归因喂进 agent 的提示词。一个健康 PR 若遭遇一次静默 sandbox 轮次（sandbox 启动即挂死，agent 在 idle 时限被杀、一个字节都没输出），下一轮会被告知"自上次成功轮次以来有 1 轮耗尽了 agent 时间预算且一无所获"——把 af-073（本 PR 新增）明确定性为基础设施问题的失败归因于预算使用，agent 会据此调整策略（缩减范围、改变方案）去应对一个并不存在的预算问题。请为这个姊妹普查点明确决策：要么在那里同样排除 idle（从 PRIOR_TIMEOUTS 中减去匹配 'AutoFix ran out of time before finishing (idle-timeout' 的行，复用完全相同的匹配串），要么在 af-073 中说明该反馈警告有意统计 idle 轮次，因为其目的（先前工作可能未完成）对 idle 同样成立。里程碑摘要的 N_TIMEOUT 桶（yml:6202）无需改动——它的"timeout(s)"标签不做预算归因。
— qwen3.8-max via Qwen Code /review (v0.21.15)
There was a problem hiding this comment.
Deferred to the next round — this round's batch was capped at 8 findings (Critical first): it landed the red-CI Critical (missing af-073 TOC entry) plus R1-2, R1-5, R1-6, R1-7, R1-8, R1-9, and R1-10.
The finding itself is verified, not disputed: running the prepare step's PRIOR_TIMEOUTS jq verbatim against a window holding one successful round and one pure idle round returns 1, so an idle round is indeed reported to the next round's agent as "exhausted the agent time budget before finishing anything". Closing it is the policy fork the finding frames — apply the same idle exclusion at the sibling site (reusing the exact needle), or record in af-073 that the feedback warning deliberately counts idle rounds — and the exclusion option carries its own prepare-step test obligation. That decision and its witness land next round.
中文说明
顺延至下一轮 —— 本轮批次上限为 8 条发现（Critical 优先）：本轮已落地变红的 CI 的 Critical（af-073 目录条目缺失）以及 R1-2、R1-5、R1-6、R1-7、R1-8、R1-9、R1-10。
该发现本身已核实、并无争议：逐字运行 prepare 步骤的 PRIOR_TIMEOUTS jq，对"一次成功轮次 + 一次纯 idle 轮次"的窗口返回 1，即 idle 轮次确实会以"耗尽了 agent 时间预算且一无所获"上报给下一轮 agent。修复它需要在本发现给出的两个政策分叉中做选择 —— 在姊妹普查点同样排除 idle（复用完全相同的匹配串），或在 af-073 中记录该反馈警告有意统计 idle 轮次 —— 其中排除方案还需要 prepare 步骤自己的测试见证。该决策及其见证放到下一轮落地。
| IDLE_N="$(grep -c 'AutoFix ran out of time before finishing (idle-timeout' <<< "${PRIOR_HEADS}" || true)" | ||
| if [[ "${AGENT_TIMEOUT:-}" == 'idle-timeout'* ]]; then |
There was a problem hiding this comment.
[Suggestion] Idle classification now lives in three independently-pinned places — this census grep needle, the 'idle-timeout'* current-round prefix test, and the headline assembly (CAUSE=/HEADLINE= ~6451/6494) fed by run-agent.mjs's detail string — and no test derives one from another, so a wording change in any one silently declassifies idle rounds with no error anywhere. Concretely: if run-agent.mjs's detail prefix (idle-timeout (…) is ever reworded, prior idle headlines stop matching the IDLE_N needle while still matching TIMEOUT_N's — idle rounds are reclassified as budget timeouts, BUDGET_TIMEOUT_N re-inflates, and the cap parks healthy still-pushing PRs again, the exact regression af-073 documents. Every existing test stays green in that scenario because the breaker tests hardcode IDLE_HEAD/IDLE_NOW literals and the idle-watchdog tests pin run-agent's string in isolation (verified by probe: rewording the prefix reproduced the parked shape while the breaker test still passed). Tie the sites together in one test: extract run-agent's detail template, the workflow's prefix tokens, the CAUSE template and the IDLE_N needle from the real sources and assert the composition (detail starts with the workflow prefix token; the needle is a substring of the template-composed headline), so a reword in any of the three places breaks a test instead of the fleet.
中文说明
idle 分类现在分散在三个各自独立钉住的点位上——这条普查 grep 匹配串、当前轮的 'idle-timeout'* 前缀判断、以及由 run-agent.mjs 的 detail 字符串喂给的标题拼装（CAUSE=/HEADLINE=，约 6451/6494 行）——而没有任何测试从一个点位推导出另一个，因此任何一处措辞变化都会悄无声息地把 idle 轮次除名，且不会有任何报错。具体地说：如果 run-agent.mjs 的 detail 前缀（idle-timeout (…）将来被改写，历史 idle 标题将不再匹配 IDLE_N 的匹配串、却仍匹配 TIMEOUT_N 的匹配串——idle 轮次会被重新归类为预算超时，BUDGET_TIMEOUT_N 重新膨胀，上限会再次停摆那些仍在正常推送的健康 PR，正是 af-073 记录的回归形态。在该场景下所有既有测试仍是绿的，因为熔断测试硬编码了 IDLE_HEAD/IDLE_NOW 字面量，而 idle 看门狗测试只在隔离中钉住 run-agent 的字符串（探针已验证：改写前缀后停摆形态复现，而熔断测试依然通过）。请在同一个测试里把这些点位关联起来：从真实源码中提取 run-agent 的 detail 模板、workflow 的前缀词元、CAUSE 模板和 IDLE_N 匹配串，断言它们的组合关系（detail 以 workflow 前缀词元开头；匹配串是模板拼装出的标题的子串），这样任何一处的改写都会打破测试而不是打破车队。
— qwen3.8-max via Qwen Code /review (v0.21.15)
There was a problem hiding this comment.
Deferred to the next round — this round's batch was capped at 8 findings (Critical first): it landed the red-CI Critical plus R1-2, R1-5, R1-6, R1-7, R1-8, R1-9, and R1-10.
The gap stands as reported and nothing in this round's changes closes it: no test derives the idle classification across run-agent.mjs's detail template, the workflow's current-round 'idle-timeout'* prefix test, the CAUSE/HEADLINE assembly, and the census needle, so a reword at any one of the three sites would silently declassify idle rounds while the suite stays green. The requested composition test (extract the detail template, prefix tokens, CAUSE template, and IDLE_N needle from the real sources and assert their composition) lands next round.
中文说明
顺延至下一轮 —— 本轮批次上限为 8 条发现（Critical 优先）：本轮已落地变红的 CI 的 Critical 以及 R1-2、R1-5、R1-6、R1-7、R1-8、R1-9、R1-10。
该缺口维持原报告所述，本轮改动也未将其关闭：目前没有任何测试从 run-agent.mjs 的 detail 模板、工作流的当前轮 'idle-timeout'* 前缀判断、CAUSE/HEADLINE 拼装与普查匹配串之间互相推导，因此三处点位中任何一处的措辞变化都会悄无声息地把 idle 轮次除名而套件保持绿色。所要求的组合测试（从真实源码提取 detail 模板、前缀词元、CAUSE 模板与 IDLE_N 匹配串并断言其组合关系）放到下一轮落地。
| if [[ "${IDLE_N}" -gt 0 ]]; then | ||
| echo "::warning::#${PR}: ${IDLE_N} silent-sandbox (idle) timeout(s) this counting window — excluded from the ${TIMEOUT_WINDOW_CAP}-timeout cap; check the sandbox image and the runner docker daemon" |
There was a problem hiding this comment.
[Suggestion] This warning is structurally suppressed on exactly the round where it matters most: the consecutive breaker sets MARK_ROUND=MAX_ROUNDS before this cumulative block's MARK_ROUND != MAX_ROUNDS guard is evaluated, so an all-idle stop — the case this PR's escape hatch routes to CONSECUTIVE_FAILURE_CAP — parks the PR with the consecutive breaker's PR-size remedy ("rebase, split, or reduce") and no idle census warning in the terminal run's job log. A persistently wedged sandbox terminates at round 5 with "this usually means the PR is too large…" — the inapplicable-remedy shape this PR exists to eliminate, re-entering through the sibling breaker — while rounds 1-4 each warned on their own runs and the terminal run does not. Pre-diff, the same shape stopped at round 3 via the deleted all-idle branch with the sandbox-specific remedy. Probe at this commit: all-idle round 5 → terminal=true consec=5 warnings=0 with the PR-size headline (rounds 2-3: warnings=1); base arm round 3 → sandbox remedy headline. Emit the idle census warning outside/above the MARK_ROUND guard — or also on the consecutive path when the streak contains idle rounds — so the terminal run's log names the runner.
中文说明
这条警告恰恰在最关键的轮次上被结构性地抑制了：连续失败熔断器在本累计块的 MARK_ROUND != MAX_ROUNDS 守卫求值之前就把 MARK_ROUND=MAX_ROUNDS，因此全 idle 停机——也就是本 PR 的逃生通道转交给 CONSECUTIVE_FAILURE_CAP 的情形——会用连续熔断器的"PR 太大"补救（"rebase、拆分或缩减"）停掉 PR，而终止轮的 job 日志里没有任何 idle 普查警告。持续挂死的 sandbox 会在第 5 轮以"这通常意味着 PR 太大……"终止——正是本 PR 要消灭的"补救不适用"形态，从姊妹熔断器重新进入——而第 1-4 轮各自的运行都有警告，唯独终止轮没有。diff 之前，同样的形态会在第 3 轮经由已删除的全 idle 分支带着 sandbox 专属补救停下。在本提交的探针：全 idle 第 5 轮 → terminal=true consec=5 warnings=0 且标题为 PR 体型补救（第 2-3 轮：warnings=1）；基线侧第 3 轮 → sandbox 补救标题。请把 idle 普查警告移到 MARK_ROUND 守卫之外/之前——或在连续失败路径上、当连败中包含 idle 轮次时也发一次——让终止轮的日志能够点名 runner 问题。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| # the right surface: it reaches the operator without spending a | ||
| # PR comment on infra noise. | ||
| if [[ "${IDLE_N}" -gt 0 ]]; then | ||
| echo "::warning::#${PR}: ${IDLE_N} silent-sandbox (idle) timeout(s) this counting window — excluded from the ${TIMEOUT_WINDOW_CAP}-timeout cap; check the sandbox image and the runner docker daemon" |
There was a problem hiding this comment.
[Suggestion] This new idle-census ::warning:: — the only observability left for excluded idle timeouts, promised by the comment above it ("Excluding idle from the cap must not hide it") and by af-073 — has no test: the harness parses only the post-sentinel output and discards the pre-sentinel stdout where the warning is echoed, and no static pin exists anywhere in scripts/tests. Verified by mutation: deleting the warning block, changing -gt 0 to -gt 1 (a window with exactly one idle timeout stops warning), or substituting ${TIMEOUT_N} for ${IDLE_N} each leave the full suite green (215/215). Operators would lose the only fleet-wide signal that runners are wedging sandboxes with no red lane. Have the harness also return the pre-sentinel output (e.g. log: out.slice(0, out.lastIndexOf('@@R@@'))) and assert mixedTrips.log contains ::warning::#1: 1 silent-sandbox while a budget-only window's does not — or statically pin the echo line on reviewAddressReportStep as the neighbouring census greps are pinned.
Witness: mutations "delete warning block" and "-gt 0 → -gt 1" each left Tests 215 passed (215); a probe asserting on pre-sentinel stdout flips: baseline "::warning::#1: 1 silent-sandbox (idle) timeout(s) this counting window — excluded from the 3-timeout cap; …" → after the deletion mutation "\n".
中文说明
这条新增的 idle 普查 ::warning::——排除 idle 之后唯一剩下的可观测性，由其上方注释（"Excluding idle from the cap must not hide it"）和 af-073 共同承诺——没有任何测试：harness 只解析哨兵之后的输出，把警告所在的哨兵前 stdout 丢弃了，且 scripts/tests 中任何地方都没有静态钉住。变异验证：删除整个警告块、把 -gt 0 改成 -gt 1（恰好一次 idle 超时的窗口将不再警告）、或把 ${IDLE_N} 换成 ${TIMEOUT_N}，整个套件都保持绿色（215/215）。运维将失去"runner 正在挂死 sandbox"这一唯一的全队信号，且没有任何通道变红。请让 harness 同时返回哨兵前的输出（例如 log: out.slice(0, out.lastIndexOf('@@R@@'))），断言 mixedTrips.log 包含 ::warning::#1: 1 silent-sandbox 而纯预算窗口不包含——或像相邻的普查 grep 那样，在 reviewAddressReportStep 上静态钉住这行 echo。
（证据见上：两个变异均 215/215 全绿；对哨兵前 stdout 断言的探针可使其翻红。）
— qwen3.8-max via Qwen Code /review (v0.21.15)
| # No all-idle branch here: the guard above only fires when | ||
| # BUDGET_TIMEOUT_N alone reaches the cap, so a counted | ||
| # window always holds more budget timeouts than idle ones | ||
| # and the budget remedy is always the right one. |
There was a problem hiding this comment.
[Suggestion] The stated invariant is false: a tripped window need not hold more budget timeouts than idle ones — the guard only guarantees BUDGET_TIMEOUT_N >= TIMEOUT_WINDOW_CAP. A window with 3 budget timeouts interleaved with 5+ idle rounds and pushes trips at BUDGET_TIMEOUT_N=3 while idle outnumbers budget (probe on this commit: terminal=true, headline reporting 3 budget exhaustions while the window holds 5 idle). Behavior is correct — the budget remedy applies whenever the guard fires — but the identical claim also lives in the new af-073 text in qwen-autofix.md, and a maintainer diagnosing a tripped, idle-dominated window will be told by both the comment and the rationale record that this state cannot exist, or may build a simplification on the nonexistent majority. State the true invariant in both places.
Witness: probe on the unmodified PR code (3 budget + 5 idle, interleaved with pushes): terminal=true, "…contains 3 agent time-budget exhaustions … The window also holds 5 silent-sandbox (idle) timeout(s)…".
| # No all-idle branch here: the guard above only fires when | |
| # BUDGET_TIMEOUT_N alone reaches the cap, so a counted | |
| # window always holds more budget timeouts than idle ones | |
| # and the budget remedy is always the right one. | |
| # No all-idle branch here: the guard above only fires when | |
| # BUDGET_TIMEOUT_N alone reaches the cap, so a counted | |
| # window always holds at least TIMEOUT_WINDOW_CAP budget | |
| # timeouts and the budget remedy is always the right one. |
中文说明
这里陈述的不变量是错的：触发上限的窗口并不必然"预算超时多于 idle 超时"——守卫只保证 BUDGET_TIMEOUT_N >= TIMEOUT_WINDOW_CAP。一个由 3 次预算超时与 5 次以上 idle 轮次、其间穿插推送构成的窗口，会在 BUDGET_TIMEOUT_N=3 时触发，而 idle 多于预算（在本提交的探针：terminal=true，标题报告 3 次预算耗尽而窗口含 5 次 idle）。行为本身是正确的——守卫触发时预算补救总是适用的——但同样的论断也出现在 qwen-autofix.md 新增的 af-073 正文里；维护者排查一个 idle 占多数却被触发的窗口时，注释和档案会同时告诉他这种状态不存在，或者基于这个不存在的"多数"做出简化。请在两处都改成真实的不变量。
（证据见上：探针观测到 idle(5) > budget(3) 的触发窗口。）
— qwen3.8-max via Qwen Code /review (v0.21.15)
| HEADLINE="🤖 AutoFix stopped: this counting window now contains ${TIMEOUT_N} time-budget exhaustions (pushed rounds in between included; this round itself may have failed differently). That is ${TIMEOUT_N} full agent runs that pushed nothing.${IDLE_CLAUSE} A human should ${REMEDY}, then comment \`${RETRY_COMMAND}\` to re-arm. Until then future scans will skip this PR." | ||
| HEADLINE_ZH="🤖 AutoFix 已停止：当前计数窗口内已累计 ${TIMEOUT_N} 次时间预算耗尽（含其间推送过的轮次；本轮本身可能以别的方式失败）。即 ${TIMEOUT_N} 次完整 agent 运行没有推送任何内容。${IDLE_CLAUSE_ZH}应由人工${REMEDY_ZH}，然后评论 \`${RETRY_COMMAND}\` 重新武装。在此之前，后续扫描将跳过本 PR。" | ||
| HEADLINE="🤖 AutoFix stopped: this counting window now contains ${BUDGET_TIMEOUT_N} agent time-budget exhaustions (pushed rounds in between included; this round itself may have failed differently). That is ${BUDGET_TIMEOUT_N} full agent runs that pushed nothing.${IDLE_CLAUSE} A human should split or reduce the PR (or raise the agent time budget AND its step backstop together), then comment \`${RETRY_COMMAND}\` to re-arm. Until then future scans will skip this PR." | ||
| HEADLINE_ZH="🤖 AutoFix 已停止：当前计数窗口内已累计 ${BUDGET_TIMEOUT_N} 次时间预算耗尽（含其间推送过的轮次；本轮本身可能以别的方式失败）。即 ${BUDGET_TIMEOUT_N} 次完整 agent 运行没有推送任何内容。${IDLE_CLAUSE_ZH}应由人工拆分或缩减该 PR（或同时提高 agent 时间预算与其步骤兜底），然后评论 \`${RETRY_COMMAND}\` 重新武装。在此之前，后续扫描将跳过本 PR。" |
There was a problem hiding this comment.
[Suggestion] HEADLINE_ZH is rewritten to interpolate ${BUDGET_TIMEOUT_N}, but no test asserts the ZH headline's count: the harness sentinel returns only $HEADLINE, and the pairing anchors carry no number, so the budget-count guarantee is pinned for English alone. Mutation ${BUDGET_TIMEOUT_N} → ${TIMEOUT_N} on this line ships green (verified: 215/215 passed); on the mixedTrips window the posted bilingual comment's ZH half would then say 已累计 4 次时间预算耗尽 while the EN half says 3 — the re-inflated number the exclusion just corrected, surviving in the other language. Add "$HEADLINE_ZH" to the harness sentinel tuple and assert mixedTrips's ZH headline contains 3 次时间预算耗尽 (and not 4 次), mirroring the existing EN assertion.
Witness: probe extending the sentinel tuple with "$HEADLINE_ZH" observed ZH 已累计 4 次时间预算耗尽 against EN's 3 under the mutation (baseline: both 3).
中文说明
HEADLINE_ZH 被改写为插值 ${BUDGET_TIMEOUT_N}，但没有任何测试断言中文标题里的计数：harness 哨兵只返回 $HEADLINE，配对锚点也不含数字，因此"预算计数"这一保证只在英文侧被钉住。把这行的 ${BUDGET_TIMEOUT_N} 变异为 ${TIMEOUT_N} 可以全绿通过（已验证：215/215 通过）；在 mixedTrips 窗口下，发布的双语评论中文一侧将写 已累计 4 次时间预算耗尽，而英文一侧写 3——刚被排除逻辑纠正掉的重新膨胀数字，在另一种语言里存活下来。请把 "$HEADLINE_ZH" 加入 harness 哨兵元组，并断言 mixedTrips 的中文标题包含 3 次时间预算耗尽（而非 4 次），与现有英文断言对称。
（证据见上：探针在该变异下观测到中文侧 4 对英文侧 3；基线两侧均为 3。）
— qwen3.8-max via Qwen Code /review (v0.21.15)
| 'IDLE_CLAUSE', | ||
| 'REMEDY', | ||
| ]) { | ||
| for (const name of ['CAUSE', 'LAST_FIX', 'GATE_CLAUSE', 'IDLE_CLAUSE']) { |
There was a problem hiding this comment.
[Suggestion] Deleting the two REMEDY pairing rows was necessary (the variables are gone), but it leaves the window-cap breaker's closing remedy with no EN/ZH pairing pin: the remedy is now inline in HEADLINE/HEADLINE_ZH, and neither 'split or reduce the PR' nor '拆分或缩减该 PR' appears in any test pin (grep-verified zero occurrences), while round-level sibling remedies remain pinned via LAST_FIX entries. An edit that rewords or drops the remedy in only one of HEADLINE / HEADLINE_ZH (exactly the kind of reword this PR performs) ships green — the correspondence anchor ('time-budget exhaustions' / '次时间预算耗尽') still matches and the runtime check covers only the English side — so a Chinese-reading user gets a divergent or stale remedy on the stop notice with no failing lane; that is the mutation class this table's own comment says it exists to catch. Add a pairing row for the inline remedy, e.g. ['HEADLINE', 'split or reduce the PR', '拆分或缩减该 PR'] — the existing regex already requires the EN and ZH anchors on corresponding assignment lines.
中文说明
删除 REMEDY 的两行配对是必要的（变量已不存在），但它让窗口上限熔断器的结尾补救失去了 EN/ZH 配对钉：补救现在内联在 HEADLINE/HEADLINE_ZH 里，而任何测试钉里都没有 'split or reduce the PR' 或 '拆分或缩减该 PR'（已 grep 验证为零处），而轮级的姊妹补救仍由 LAST_FIX 条目钉住。任何只在 HEADLINE / HEADLINE_ZH 其中之一改写或删除补救的编辑（正是本 PR 所做的那类改写）都能全绿通过——对应锚点（'time-budget exhaustions' / '次时间预算耗尽'）仍然匹配，运行时检查又只覆盖英文侧——于是中文用户在停机通知里看到不一致或过期的补救，而没有任何通道变红；这正是本表自己的注释声称要捕捉的变异类别。请为内联补救补一行配对，例如 ['HEADLINE', 'split or reduce the PR', '拆分或缩减该 PR']——现有正则本就要求 EN 与 ZH 锚点出现在对应的赋值行上。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| expect(reviewAddressReportStep).toContain( | ||
| 'IDLE_N="$(grep -c \'AutoFix ran out of time before finishing (idle-timeout\' <<< "${PRIOR_HEADS}" || true)"', | ||
| ); | ||
| expect('AutoFix ran out of time before finishing (idle-timeout').toContain( |
There was a problem hiding this comment.
[Suggestion] This assertion validates a string literal against itself — true by construction, unable to fail under any edit to the workflow — so it pins nothing about the workflow, while its comment presents it as pinning the load-bearing subset invariant (IDLE needle extends TIMEOUT needle, keeping BUDGET_TIMEOUT_N non-negative). The real guards are the two adjacent exact-text pins on reviewAddressReportStep; if a future edit relaxes them and introduces needles that are no longer extensions of each other, this line still passes green while signalling the invariant is checked. Derive both needles from the workflow and assert the relation on those:
const timeoutNeedle = reviewAddressReportStep.match(/TIMEOUT_N="\$\(grep -c '([^']+)'/)[1];
const idleNeedle = reviewAddressReportStep.match(/IDLE_N="\$\(grep -c '([^']+)'/)[1];
expect(idleNeedle).toContain(timeoutNeedle);— or delete the tautology, since the adjacent verbatim pins already freeze both needles.
中文说明
这条断言是字符串字面量对自身做校验——构造上恒真，任何对 workflow 的编辑都不可能让它失败——因此它对 workflow 什么也没钉住，而其注释却把它呈现为钉住了承重的子集不变量（IDLE 匹配串扩展自 TIMEOUT 匹配串，从而保证 BUDGET_TIMEOUT_N 非负）。真正的守卫是相邻两条针对 reviewAddressReportStep 的逐字钉；若未来编辑放宽它们并引入不再是彼此扩展的匹配串，这一行仍会绿着通过，同时对外显示该不变量已被检查。请从 workflow 中提取两个匹配串并对其断言该关系（见上方代码），或者直接删掉这句同义反复——相邻的逐字钉已经冻结了两个匹配串。
— qwen3.8-max via Qwen Code /review (v0.21.15)
|
@qwen-code /takeover |
qwen-code-dev-bot
added
the
autofix/takeover
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. Remove the 中文说明🤝 已接管：autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突，直到移除标签或达到轮次上限。移除 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
2 Suggestion-level finding(s) this review confirmed are already reported on this PR and are not repeated:
- R1-5 idle ::warning:: suppressed on the parking round — already reported (comment 3831733460)
- R1-6 idle-census ::warning:: has no test — already reported (comment 3831733467)
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
本轮确认的 2 条建议级发现已在 PR 上报告过，不再重复发布（列表见上方英文部分）。
未检查（工具限制，非阻断）：the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max via Qwen Code /review (v0.21.15)
|
|
||
| <a id="af-073"></a> | ||
|
|
||
| ### 73. review-address · Report dry-run / failure — Idle (silent-sandbox) timeouts are EXCLUDED from the cumulative timeout cap. |
There was a problem hiding this comment.
[Critical] R1-1: The new af-073 anchor is added without a Contents-table entry, and that fails this repo's existing design-record consistency test scripts/tests/workflow-size.test.js ("lists every section in the contents table"): the test asserts the TOC link targets equal every <a id="af-N"> anchor in this file. This diff adds anchor af-073 but the table still ends at entry 72, so the suite fails — the PR's own Test (ubuntu-latest, Node 22.x) lane is red with this exact assertion.
Witness: at the reviewed commit the file holds 71 anchors but 70 contents-table entries — the difference is exactly af-073. Running the scripts/tests suite here: FAIL scripts/tests/workflow-size.test.js > lists every section in the contents table, 1 failed | 1519 passed (1520). At the merge base the same sets are equal (70/70) and the test passes, so the failure is introduced by this diff even though the test file itself is unchanged.
Add the entry for section 73 to the Contents block (after the entry-72 line, ~line 126), matching the house style — the test's regex only requires the - [73. ...](#af-073) shape:
- [73. review-address · Report dry-run / failure — Idle (silent-sandbox) timeouts are EXCLUDED from the cumulative timeout cap.…](#af-073)
中文说明
新增的 af-073 锚点没有对应的目录条目，这会使仓库既有的设计档案一致性测试 scripts/tests/workflow-size.test.js（"lists every section in the contents table"）失败：该测试断言目录链接目标与本文件中的每个 <a id="af-N"> 锚点一一对应。本次 diff 添加了锚点 af-073，但目录仍止于第 72 条，因此该测试套件失败——本 PR 自己的 Test (ubuntu-latest, Node 22.x) 通道现在正是因为这条断言而变红。
证据：在被审提交上，文件有 71 个锚点但只有 70 条目录条目，差集恰为 af-073。在本工作树运行 scripts/tests 套件：FAIL scripts/tests/workflow-size.test.js > lists every section in the contents table，1 failed | 1519 passed (1520)。在合并基线上两组集合相等（70/70）且测试通过，因此该失败由本 diff 引入——尽管测试文件本身未被修改。
修复：在 Contents 区块（第 72 条之后，约第 126 行）为第 73 节添加条目，沿用既有风格——测试的正则只要求 - [73. ...](#af-073) 这一形状（见上方代码块）。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| # Full rationale → qwen-autofix.md#af-073 | ||
| # The needle is the emitted headline verbatim AND a prefix of | ||
| # TIMEOUT_N's, so IDLE_N can never exceed it and the | ||
| # subtraction below can never go negative. |
There was a problem hiding this comment.
[Suggestion] R1-2: This comment states the subset invariant backwards: the IDLE needle is a strict extension of TIMEOUT_N's needle, not "a prefix of TIMEOUT_N's" — contradicting the sibling comment this same diff adds in scripts/tests/qwen-autofix-workflow.test.js ("its needle must be a strict extension of TIMEOUT_N's"). The non-negativity guarantee holds because every line matching the IDLE needle also matches TIMEOUT_N's needle, i.e. TIMEOUT_N's needle is the prefix. A maintainer rewording the idle headline and validating a replacement needle against this comment would enforce the wrong direction; a needle shorter than TIMEOUT_N's (e.g. a revert to the bare 'idle-timeout' substring this PR removes) could match provider-error text that API_ERROR_DETAIL puts on a first line TIMEOUT_N does not count, driving BUDGET_TIMEOUT_N negative and silently disabling the cap. Reword to the true relation:
# The needle is the emitted headline prefix AND a strict
# extension of TIMEOUT_N's needle, so every line it matches
# TIMEOUT_N's also matches, IDLE_N can never exceed it, and
# the subtraction below can never go negative.
中文说明
这段注释把子集不变量说反了：IDLE 的匹配串是 TIMEOUT_N 匹配串的严格扩展，而不是"TIMEOUT_N 的前缀"——与本 diff 在 scripts/tests/qwen-autofix-workflow.test.js 中新增的姊妹注释（"its needle must be a strict extension of TIMEOUT_N's"）自相矛盾。非负性之所以成立，是因为每一行能匹配 IDLE 匹配串的文本必然也匹配 TIMEOUT_N 的匹配串，即 TIMEOUT_N 的匹配串才是前缀。若维护者改写 idle 标题并依据这段注释来验证替换后的匹配串，会把方向搞反；比 TIMEOUT_N 更短的匹配串（例如退回本 PR 刚删掉的裸 'idle-timeout' 子串）可能命中 API_ERROR_DETAIL 放在首行、而 TIMEOUT_N 并不统计的模型报错文本，使 BUDGET_TIMEOUT_N 变成负数，从而悄悄禁用这条上限。请按真实关系改写（见上方代码块）。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| # The needle is the emitted headline verbatim AND a prefix of | ||
| # TIMEOUT_N's, so IDLE_N can never exceed it and the | ||
| # subtraction below can never go negative. | ||
| IDLE_N="$(grep -c 'AutoFix ran out of time before finishing (idle-timeout' <<< "${PRIOR_HEADS}" || true)" |
There was a problem hiding this comment.
[Suggestion] R1-3: The idle-vs-budget reclassification this PR introduces is applied at only one of the census sites that count the same timeout headline: the sibling PRIOR_TIMEOUTS site (yml:4993-5008) still counts idle rounds as budget exhaustions and feeds that attribution into the agent's prompt via feedback.md. A healthy PR that suffers one silent-sandbox round (sandbox wedges at startup, agent killed at the idle tick having emitted nothing) is told on the next round "1 round(s) since the last successful round exhausted the agent time budget before finishing anything" — attributing to budget use what af-073 (added by this PR) declares is infra, so the agent adapts strategy (scope reduction, approach change) to a budget problem that does not exist. af-073 records no deliberate decision for this sibling site. Probe evidence: running the real PRIOR_TIMEOUTS jq extracted verbatim from the prepare step — budget-timeout-only → 1 / push-only → 0 / idle-timeout-only → 1, so a pure idle round is counted and written into feedback.md as a budget exhaustion. Decide the policy for the sibling site and make it explicit: either apply the same idle exclusion there (subtract lines matching the exact needle 'AutoFix ran out of time before finishing (idle-timeout' from PRIOR_TIMEOUTS), or state in af-073 that the feedback warning deliberately counts idle rounds because its purpose (prior work may be unfinished) holds for them too. The milestone-digest N_TIMEOUT bucket (yml:6205) needs no change — its "timeout(s)" label makes no budget attribution.
中文说明
本 PR 引入的 idle/预算重分类只应用于统计同一标题的多个普查点中的一个：姊妹普查点 PRIOR_TIMEOUTS（yml:4993-5008）仍把 idle 轮次计为预算耗尽，并通过 feedback.md 把这个归因喂进 agent 的提示词。一个健康 PR 若遭遇一次静默 sandbox 轮次（sandbox 启动即挂死，agent 在 idle 时限被杀、一个字节都没输出），下一轮会被告知"自上次成功轮次以来有 1 轮耗尽了 agent 时间预算且一无所获"——把 af-073（本 PR 新增）明确定性为基础设施问题的失败归因于预算使用，agent 会据此调整策略（缩减范围、改变方案）去应对一个并不存在的预算问题。af-073 没有为这个姊妹普查点记录任何有意决策。探针证据：逐字提取 prepare 步骤中真实的 PRIOR_TIMEOUTS jq 运行——仅预算超时 → 1 / 仅推送 → 0 / 仅 idle 超时 → 1，即纯 idle 轮次被计入并以预算耗尽写入 feedback.md。请为这个姊妹普查点明确决策：要么在那里同样排除 idle（从 PRIOR_TIMEOUTS 中减去匹配完全相同匹配串 'AutoFix ran out of time before finishing (idle-timeout' 的行），要么在 af-073 中说明该反馈警告有意统计 idle 轮次，因为其目的（先前工作可能未完成）对 idle 同样成立。里程碑摘要的 N_TIMEOUT 桶（yml:6205）无需改动——它的 "timeout(s)" 标签不做预算归因。
— qwen3.8-max via Qwen Code /review (v0.21.15)
There was a problem hiding this comment.
Same finding as the thread above (R1-3) — deferred to the next round for the reason recorded there: this round's batch was capped at 8 findings (Critical first). The claim was re-verified here as well (the prepare step's PRIOR_TIMEOUTS jq counts a pure idle round as 1 budget exhaustion); the policy decision and its test witness land next round.
中文说明
与上方线程为同一发现（R1-3）—— 按该线程记录的理由顺延至下一轮：本轮批次上限为 8 条发现（Critical 优先）。本条同样复核了该论断（prepare 步骤的 PRIOR_TIMEOUTS jq 会把一次纯 idle 轮次计为 1 次预算耗尽）；政策决策及其测试见证放到下一轮落地。
| # TIMEOUT_N's, so IDLE_N can never exceed it and the | ||
| # subtraction below can never go negative. | ||
| IDLE_N="$(grep -c 'AutoFix ran out of time before finishing (idle-timeout' <<< "${PRIOR_HEADS}" || true)" | ||
| if [[ "${AGENT_TIMEOUT:-}" == 'idle-timeout'* ]]; then |
There was a problem hiding this comment.
[Suggestion] R1-4: Idle classification now lives in three independently-pinned places — this census grep needle, the 'idle-timeout'* current-round prefix test, and the headline assembly (CAUSE=/HEADLINE= ~6454/6496) fed by run-agent.mjs's detail string — and no test derives one from another, so a wording change in any one silently declassifies idle rounds with no error anywhere. Concretely: if run-agent.mjs's detail prefix (idle-timeout (…) is ever reworded, prior idle headlines stop matching the IDLE_N needle while still matching TIMEOUT_N's — idle rounds are reclassified as budget timeouts, BUDGET_TIMEOUT_N re-inflates, and the cap parks healthy still-pushing PRs again, the exact regression af-073 documents. Probe evidence: rewording only run-agent.mjs's idle sentinel (plus its four isolation pins) leaves the full suite 215/215 green, while impact arms through the pristine breaker block (2 budget + 1 idle priors, current round idle) flip from terminal=false with the current sentinel to terminal=true with the reworded one — the cap parks the PR with every test green. Tie the sites together in one test: extract run-agent's detail template, the workflow's prefix tokens, the CAUSE template and the IDLE_N needle from the real sources and assert the composition (detail starts with the workflow prefix token; the needle is a substring of the template-composed headline), so a reword in any of the three places breaks a test instead of the fleet.
中文说明
idle 分类现在分散在三个各自独立钉住的点位上——这条普查 grep 匹配串、当前轮的 'idle-timeout'* 前缀判断、以及由 run-agent.mjs 的 detail 字符串喂给的标题拼装（CAUSE=/HEADLINE=，约 6454/6496 行）——而没有任何测试从一个点位推导出另一个，因此任何一处措辞变化都会悄无声息地把 idle 轮次除名，且不会有任何报错。具体地说：如果 run-agent.mjs 的 detail 前缀（idle-timeout (…）将来被改写，历史 idle 标题将不再匹配 IDLE_N 的匹配串、却仍匹配 TIMEOUT_N 的匹配串——idle 轮次会被重新归类为预算超时，BUDGET_TIMEOUT_N 重新膨胀，上限会再次停摆那些仍在正常推送的健康 PR，正是 af-073 记录的回归形态。探针证据：仅改写 run-agent.mjs 的 idle 哨兵（及其四处隔离钉住点）后全套测试仍 215/215 全绿；而对未经改动的熔断块做影响对照（2 次预算 + 1 次 idle 历史，当前轮 idle），现用哨兵下 terminal=false，改写后 terminal=true——上限停摆了 PR，而所有测试依然绿色。请在同一个测试里把这些点位关联起来：从真实源码中提取 run-agent 的 detail 模板、workflow 的前缀词元、CAUSE 模板和 IDLE_N 匹配串，断言它们的组合关系（detail 以 workflow 前缀词元开头；匹配串是模板拼装出的标题的子串），这样任何一处的改写都会打破测试而不是打破车队。
— qwen3.8-max via Qwen Code /review (v0.21.15)
There was a problem hiding this comment.
Same finding as the thread above (R1-4) — deferred to the next round for the reason recorded there: this round's batch was capped at 8 findings (Critical first). The composition test tying the three idle-classification sites together lands next round.
中文说明
与上方线程为同一发现（R1-4）—— 按该线程记录的理由顺延至下一轮：本轮批次上限为 8 条发现（Critical 优先）。把三个 idle 分类点位钉在一起的组合测试放到下一轮落地。
| # No all-idle branch here: the guard above only fires when | ||
| # BUDGET_TIMEOUT_N alone reaches the cap, so a counted | ||
| # window always holds more budget timeouts than idle ones | ||
| # and the budget remedy is always the right one. |
There was a problem hiding this comment.
[Suggestion] R1-7: The stated invariant is false: a tripped window need not hold more budget timeouts than idle ones — the guard only guarantees BUDGET_TIMEOUT_N >= TIMEOUT_WINDOW_CAP. Probe evidence: a window of 2 budget priors + current budget timeout + 4 idle rounds trips at BUDGET_TIMEOUT_N = 7−4 = 3 with the headline "contains 3 agent time-budget exhaustions … The window also holds 4 silent-sandbox (idle) timeout(s)" — idle outnumbers budget at trip, and the IDLE_CLAUSE emitted by this very branch contradicts the comment above it. The same false claim is repeated in the new af-073 design record. A future editor relying on the stated invariant (e.g., assuming idle is the minority when reworking the clause or computing ratios) reasons from a false premise; the conclusion ("the budget remedy is always right") still holds, but via the real guarantee — at least TIMEOUT_WINDOW_CAP genuine budget timeouts exist. Reword both places to the true invariant:
# No all-idle branch here: the guard above only fires when
# BUDGET_TIMEOUT_N alone reaches the cap, so a counted window
# always holds at least TIMEOUT_WINDOW_CAP genuine budget
# timeouts and the budget remedy is always the right one.
中文说明
这里陈述的不变量是错的：触发上限的窗口并不必然持有更多的预算超时——判断只保证 BUDGET_TIMEOUT_N >= TIMEOUT_WINDOW_CAP。探针证据：2 次预算历史 + 当前轮预算超时 + 4 次 idle 的窗口在 BUDGET_TIMEOUT_N = 7−4 = 3 时触发，标题为 "contains 3 agent time-budget exhaustions … The window also holds 4 silent-sandbox (idle) timeout(s)"——触发时 idle 多于预算，而正是这个分支发出的 IDLE_CLAUSE 与它上方的注释自相矛盾。同样的错误表述也出现在新增的 af-073 设计档案里。未来若某位编辑依赖这个不变量（例如在改写子句或计算比率时假定 idle 是少数），就会从一个错误前提出发推理；结论（"预算补救总是正确的"）依然成立，但依据的是真正的保证——至少存在 TIMEOUT_WINDOW_CAP 次真实的预算超时。请把两处都改写为真实的不变量（见上方代码块）。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| HEADLINE="🤖 AutoFix stopped: this counting window now contains ${TIMEOUT_N} time-budget exhaustions (pushed rounds in between included; this round itself may have failed differently). That is ${TIMEOUT_N} full agent runs that pushed nothing.${IDLE_CLAUSE} A human should ${REMEDY}, then comment \`${RETRY_COMMAND}\` to re-arm. Until then future scans will skip this PR." | ||
| HEADLINE_ZH="🤖 AutoFix 已停止：当前计数窗口内已累计 ${TIMEOUT_N} 次时间预算耗尽（含其间推送过的轮次；本轮本身可能以别的方式失败）。即 ${TIMEOUT_N} 次完整 agent 运行没有推送任何内容。${IDLE_CLAUSE_ZH}应由人工${REMEDY_ZH}，然后评论 \`${RETRY_COMMAND}\` 重新武装。在此之前，后续扫描将跳过本 PR。" | ||
| HEADLINE="🤖 AutoFix stopped: this counting window now contains ${BUDGET_TIMEOUT_N} agent time-budget exhaustions (pushed rounds in between included; this round itself may have failed differently). That is ${BUDGET_TIMEOUT_N} full agent runs that pushed nothing.${IDLE_CLAUSE} A human should split or reduce the PR (or raise the agent time budget AND its step backstop together), then comment \`${RETRY_COMMAND}\` to re-arm. Until then future scans will skip this PR." | ||
| HEADLINE_ZH="🤖 AutoFix 已停止：当前计数窗口内已累计 ${BUDGET_TIMEOUT_N} 次时间预算耗尽（含其间推送过的轮次；本轮本身可能以别的方式失败）。即 ${BUDGET_TIMEOUT_N} 次完整 agent 运行没有推送任何内容。${IDLE_CLAUSE_ZH}应由人工拆分或缩减该 PR（或同时提高 agent 时间预算与其步骤兜底），然后评论 \`${RETRY_COMMAND}\` 重新武装。在此之前，后续扫描将跳过本 PR。" |
There was a problem hiding this comment.
[Suggestion] R1-8: HEADLINE_ZH is rewritten to interpolate ${BUDGET_TIMEOUT_N}, but no test asserts the ZH headline's count: the harness sentinel returns only $HEADLINE, and the pairing anchors carry no number, so the budget-count guarantee is pinned for English alone. Mutation ${BUDGET_TIMEOUT_N} → ${TIMEOUT_N} on this line ships green (probe-verified: extending the sentinel tuple with "$HEADLINE_ZH" observed ZH 已累计 4 次时间预算耗尽 against EN's 3 on the mixedTrips window under the mutation, baseline both 3, suite 215/215) — the posted bilingual comment's ZH half would then say 4 while the EN half says 3, the re-inflated number the exclusion just corrected surviving in the other language. Add "$HEADLINE_ZH" to the harness sentinel tuple and assert mixedTrips's ZH headline contains 3 次时间预算耗尽 (and not 4 次), mirroring the existing EN assertion.
中文说明
HEADLINE_ZH 被改写为插值 ${BUDGET_TIMEOUT_N}，但没有任何测试断言中文标题里的计数：harness 哨兵只返回 $HEADLINE，配对锚点也不含数字，因此"预算计数"这一保证只在英文侧被钉住。把这行的 ${BUDGET_TIMEOUT_N} 变异为 ${TIMEOUT_N} 可以全绿通过（探针已验证：把哨兵元组扩展为含 "$HEADLINE_ZH" 后，mixedTrips 窗口在该变异下观测到中文侧 已累计 4 次时间预算耗尽、英文侧为 3，基线两侧均为 3，套件 215/215）——发布的双语评论中文一侧将写 4 而英文一侧写 3，刚被排除逻辑纠正掉的重新膨胀数字在另一种语言里存活下来。请把 "$HEADLINE_ZH" 加入 harness 哨兵元组，并断言 mixedTrips 的中文标题包含 3 次时间预算耗尽（而非 4 次），与现有英文断言对称。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| 'IDLE_CLAUSE', | ||
| 'REMEDY', | ||
| ]) { | ||
| for (const name of ['CAUSE', 'LAST_FIX', 'GATE_CLAUSE', 'IDLE_CLAUSE']) { |
There was a problem hiding this comment.
[Suggestion] R1-9: Deleting the two REMEDY pairing rows was necessary (the variables are gone), but it leaves the window-cap breaker's closing remedy with no EN/ZH pairing pin: the remedy is now inline in HEADLINE/HEADLINE_ZH, and the EN half keeps a partial behavioral pin (mixedTrips.headline asserts 'raise the agent time budget'), but the harness only ever prints $HEADLINE, never $HEADLINE_ZH, and the surviving ['HEADLINE', 'time-budget exhaustions', '次时间预算耗尽'] anchor stops short of the remedy clause — grep confirms no remaining pin matches the Chinese remedy (拆分或缩减 / 步骤兜底). Probe evidence: replacing the ZH remedy clause with a generic 应由人工处理 keeps the full suite 215/215 green. A future reword that drops or mangles 应由人工拆分或缩减该 PR（或同时提高 agent 时间预算与其步骤兜底）therefore ships green, and Chinese-reading maintainers of a parked PR get a stop comment with a broken or missing human-action instruction. Restore the pin as a pairing-table row reaching the inlined remedy:
['HEADLINE', 'split or reduce the PR', '拆分或缩减该 PR'],
(the table's regex matches anchors anywhere on the NAME=/NAME_ZH= line pair).
中文说明
删除两行 REMEDY 配对是必要的（变量已不存在），但这使窗口上限熔断器的收尾补救失去了 EN/ZH 配对钉住：补救现在内联在 HEADLINE/HEADLINE_ZH 中，英文侧仍有一部分行为钉住（mixedTrips.headline 断言 'raise the agent time budget'），但 harness 只打印 $HEADLINE、从不打印 $HEADLINE_ZH，而幸存的 ['HEADLINE', 'time-budget exhaustions', '次时间预算耗尽'] 锚点在补救子句之前就截止——全文检索确认中文补救（拆分或缩减 / 步骤兜底）已无任何钉住点。探针证据：把中文补救子句替换为泛化的 应由人工处理 后全套测试仍 215/215 全绿。因此未来任何丢弃或破坏 应由人工拆分或缩减该 PR（或同时提高 agent 时间预算与其步骤兜底）的改写都能全绿通过，被停摆 PR 的中文维护者将收到一份人类操作指引损坏或缺失的停机评论。请以配对表行的形式恢复钉住，使其覆盖内联后的补救（见上方代码块；配对表的正则可匹配 NAME=/NAME_ZH= 行对上任意位置的锚点）。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| expect(reviewAddressReportStep).toContain( | ||
| 'IDLE_N="$(grep -c \'AutoFix ran out of time before finishing (idle-timeout\' <<< "${PRIOR_HEADS}" || true)"', | ||
| ); | ||
| expect('AutoFix ran out of time before finishing (idle-timeout').toContain( |
There was a problem hiding this comment.
[Suggestion] R1-10: This assertion validates a string literal against itself — true by construction, unable to fail under any edit to the workflow — so it pins nothing about the workflow, while its comment presents it as pinning the load-bearing subset property (the IDLE needle ⊆ TIMEOUT needle relation that keeps BUDGET_TIMEOUT_N non-negative). The property is actually guarded only by the two adjacent verbatim grep-line pins. If a future edit reshapes those pins into a looser form (regexes over the needles, extraction into variables), this assertion still passes green while the subset guarantee is gone; a non-subset idle needle could then count provider-error text on a first line as idle, drive $(( TIMEOUT_N - IDLE_N )) negative, and [[ "-1" -ge "3" ]] never fires — the cumulative timeout breaker silently stops ever tripping, with no red test pointing at the cause. Either delete the tautology, or make it assert the real invariant against the workflow text — extract both needles from reviewAddressReportStep (the two grep -c '<needle>' captures) and assert:
expect(idleNeedle).toContain(timeoutNeedle);
中文说明
这条断言把一个字符串字面量与它自己比较——构造上恒真，无论 workflow 如何修改都不可能失败——因此它对 workflow 什么也没钉住，而其注释却把它表述为钉住了承重的子集性质（IDLE 匹配串 ⊆ TIMEOUT 匹配串这一保证 BUDGET_TIMEOUT_N 非负的关系）。该性质实际上只由上下两条逐字钉住 grep 行的断言守护。若未来某次编辑把那两条钉住改成更宽松的形式（对匹配串用正则、或抽取为变量），这条断言依然全绿，而子集保证已经消失；非子集的 idle 匹配串就可能把首行上的模型报错文本计为 idle，使 $(( TIMEOUT_N - IDLE_N )) 变成负数，[[ "-1" -ge "3" ]] 永不触发——累计超时熔断器悄无声息地不再触发，且没有任何变红的测试指向原因。请删除这个恒真断言，或让它对真实的 workflow 文本断言该不变量——从 reviewAddressReportStep 中提取两个匹配串（两处 grep -c '<needle>' 捕获），然后断言（见上方代码块）。
— qwen3.8-max via Qwen Code /review (v0.21.15)
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 1/100 轮）。改动内容与我反驳保留之处如下： Review-address round summary — PR #9673Commit: DispositionsResolved in this round (8 findings — batch capped at 8, Critical first):
Deferred to the next round (2 findings — per-round batch bound, recorded in
Mutation probes (witness per new guard)Each probe applied a mutation, re-ran the focused suite, confirmed the expected FAIL, restored, re-ran green:
Failed checks on the PR
Local-runner notes (informational, not PR defects)
ConflictsNone ( Verification
中文说明评审处理轮次总结 — PR #9673提交： 处理结论本轮已解决（8 条发现 —— 批次上限 8 条，Critical 优先）：
顺延至下一轮（2 条发现 —— 受单轮批次上限约束，已记录在对应线程的
变异探针（每个新守卫一个见证）每个探针都是：施加变异 → 重跑聚焦套件 → 确认预期失败 → 还原 → 重跑转绿：
PR 上的失败检查
本地 runner 备注（信息性，非 PR 缺陷）
冲突无（ 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline.
Not explored to full depth (tool budget reached): "agent 3b": none — no check was cut short..
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查。 建议见行内评论。
未探索到全部深度（达到工具调用预算）："agent 3b"：none — no check was cut short.。
未检查（工具限制，非阻断）：the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| IDLE_CLAUSE=" The window also holds ${IDLE_N} silent-sandbox (idle) timeout(s), which no budget increase can cure and which do NOT count toward this cap — investigate the sandbox image and runner docker daemon separately." | ||
| IDLE_CLAUSE_ZH="本窗口另有 ${IDLE_N} 次静默 sandbox（idle）超时，提高预算也治不了，且不计入本上限 —— 请另行排查 sandbox 镜像与 runner 的 docker daemon。" |
There was a problem hiding this comment.
[Suggestion] R3-1: The ${IDLE_N} count interpolated into this cap-trip clause is pinned by no test — a ${IDLE_N} → ${TIMEOUT_N} swap in either the EN or the ZH clause ships the whole suite green (both mutants verified: 215/215 pass). In this PR's own mixedTrips shape (2 budget priors + 1 idle prior + budget-timeout current round: TIMEOUT_N=4, IDLE_N=1), the mutant posts "The window also holds 4 silent-sandbox (idle) timeout(s)" — and 本窗口另有 4 次静默… in the ZH half — on a window holding exactly one idle round, sending whoever owns the runners to hunt a wedged fleet four times the real scale, while mixedTrips stays green because it only asserts the do NOT count toward this cap phrase. This is one of three sites of the same gap — the stop-message text this PR adds interpolates counts the suite never pins; the siblings sit on the headline's second count and on the idle-census warning. Pin the clause count numerically in both languages, e.g. in mixedTrips:
expect(mixedTrips.headline).toContain('also holds 1 silent-sandbox');
expect(mixedTrips.headlineZh).toContain('本窗口另有 1 次静默');Witness: both clause mutants ship Tests 215 passed (215); with the two pins added the mutants fail — expected '…The window also holds 4 silent-sandbox…' to contain 'also holds 1 silent-sandbox'.
中文说明
[Suggestion] R3-1：插入到本熔断停机子句中的 ${IDLE_N} 计数没有任何测试钉住 —— 把英文或中文子句里的 ${IDLE_N} 换成 ${TIMEOUT_N}，整个测试套件依然全绿（两个变异均已验证：215/215 通过）。在本 PR 自己的 mixedTrips 形态下（2 次预算超时历史 + 1 次 idle 历史 + 当前轮预算超时：TIMEOUT_N=4、IDLE_N=1），该变异会发出 "The window also holds 4 silent-sandbox (idle) timeout(s)"（中文半边为 本窗口另有 4 次静默…）—— 而窗口里实际只有一次 idle，等于让负责 runner 的人按四倍的规模去排查，而 mixedTrips 依然为绿，因为它只断言了 do NOT count toward this cap 这一措辞。这是同一缺口的三处之一 —— 本 PR 新增的停机文案插入了若干计数，而测试套件从未钉住它们；另外两处分别在标题的第二个计数与 idle 普查警告上。建议在两种语言里把该计数用数字钉住，例如在 mixedTrips 中加入上方代码块中的两条断言。
见证：两个子句变异均让 Tests 215 passed (215) 全绿通过；补上两条断言后变异失败 —— expected '…The window also holds 4 silent-sandbox…' to contain 'also holds 1 silent-sandbox'。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| fi | ||
| HEADLINE="🤖 AutoFix stopped: this counting window now contains ${TIMEOUT_N} time-budget exhaustions (pushed rounds in between included; this round itself may have failed differently). That is ${TIMEOUT_N} full agent runs that pushed nothing.${IDLE_CLAUSE} A human should ${REMEDY}, then comment \`${RETRY_COMMAND}\` to re-arm. Until then future scans will skip this PR." | ||
| HEADLINE_ZH="🤖 AutoFix 已停止：当前计数窗口内已累计 ${TIMEOUT_N} 次时间预算耗尽（含其间推送过的轮次；本轮本身可能以别的方式失败）。即 ${TIMEOUT_N} 次完整 agent 运行没有推送任何内容。${IDLE_CLAUSE_ZH}应由人工${REMEDY_ZH}，然后评论 \`${RETRY_COMMAND}\` 重新武装。在此之前，后续扫描将跳过本 PR。" | ||
| HEADLINE="🤖 AutoFix stopped: this counting window now contains ${BUDGET_TIMEOUT_N} agent time-budget exhaustions (pushed rounds in between included; this round itself may have failed differently). That is ${BUDGET_TIMEOUT_N} full agent runs that pushed nothing.${IDLE_CLAUSE} A human should split or reduce the PR (or raise the agent time budget AND its step backstop together), then comment \`${RETRY_COMMAND}\` to re-arm. Until then future scans will skip this PR." |
There was a problem hiding this comment.
[Suggestion] R3-1: Of the two ${BUDGET_TIMEOUT_N} interpolations in this headline, only the first is pinned — this second one ("That is … full agent runs that pushed nothing") and its ZH twin ("即 … 次完整 agent 运行没有推送任何内容") are asserted nowhere (grep confirms full agent runs / 次完整 appear nowhere in the test file). A ${BUDGET_TIMEOUT_N} → ${TIMEOUT_N} mutant on this occurrence ships green (verified: 215/215), and in the mixedTrips shape (TIMEOUT_N=4, BUDGET_TIMEOUT_N=3) it posts "That is 4 full agent runs that pushed nothing" — re-inflating with an idle round exactly the count the exclusion was written to correct, and calling an idle round a "full agent run" when af-073 documents that it dies in a fraction of one — while mixedTrips only checks the first sentence's wording (3 agent time-budget exhaustions, 3 次时间预算耗尽, and the 4 次时间预算耗尽 negative). This is the diff's own stated mutant class — the test comment at the static pins says it intends to kill exactly this mutation — one interpolation over. Same three-site gap as the sibling comments (R3-1). Pin the second sentence both ways, e.g.:
expect(mixedTrips.headline).toContain(`That is ${timeoutCap} full agent runs`);
expect(mixedTrips.headline).not.toContain(`That is ${timeoutCap + 1} full agent runs`);
expect(mixedTrips.headlineZh).toContain(`即 ${timeoutCap} 次完整`);Witness: the second-occurrence mutant ships Tests 215 passed (215); with the pins added it fails — expected '🤖 AutoFix stopped: …' to contain '3 full agent runs' (received That is 4 full agent runs that pushed nothing).
中文说明
[Suggestion] R3-1：这条标题里有两处 ${BUDGET_TIMEOUT_N} 插值，但只有第一处被测试钉住 —— 第二处（"That is … full agent runs that pushed nothing"）及其中文对应（"即 … 次完整 agent 运行没有推送任何内容"）没有任何断言覆盖（grep 确认测试文件中不存在 full agent runs / 次完整）。对这一处施加 ${BUDGET_TIMEOUT_N} → ${TIMEOUT_N} 变异，套件全绿（已验证：215/215）；在 mixedTrips 形态下（TIMEOUT_N=4、BUDGET_TIMEOUT_N=3）会发出 "That is 4 full agent runs that pushed nothing" —— 把本次排除刚刚纠正下去的计数用一次 idle 轮重新吹大，而按 af-073 的记载，idle 轮连一轮的零头都跑不到，却被称作 "完整 agent 运行" —— 而 mixedTrips 只检查了第一句的措辞（3 agent time-budget exhaustions、3 次时间预算耗尽 以及对 4 次时间预算耗尽 的负向断言）。这正是本 diff 自己声明要杀的变异类 —— 静态钉处的测试注释写明意图钉住这种变异 —— 只是漏了一处插值。与同组评论（R3-1）属同一三处缺口。建议按上方代码块双向钉住第二句。
见证：第二处插值的变异让 Tests 215 passed (215) 全绿通过；补上断言后失败 —— expected '🤖 AutoFix stopped: …' to contain '3 full agent runs'（实际收到 That is 4 full agent runs that pushed nothing）。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| if [[ "${IDLE_N}" -gt 0 ]]; then | ||
| echo "::warning::#${PR}: ${IDLE_N} silent-sandbox (idle) timeout(s) this counting window — excluded from the ${TIMEOUT_WINDOW_CAP}-timeout cap; check the sandbox image and the runner docker daemon" |
There was a problem hiding this comment.
[Suggestion] R3-1: Of this idle-census ::warning:: — the only observability af-073 leaves for excluded idle timeouts — the tests pin only the ::warning::#N: M silent-sandbox prefix; the ${TIMEOUT_WINDOW_CAP} interpolation and the guidance tail (check the sandbox image and the runner docker daemon) are asserted nowhere. Verified mutants that ship the suite green: corrupting the tail text, and swapping ${TIMEOUT_WINDOW_CAP} → ${CONSECUTIVE_FAILURE_CAP}, which logs "excluded from the 5-timeout cap" where the code must say 3 — misstating the cap on the very channel designated as the sole idle signal. (The illustrative ${TIMEOUT_N} swap is not the observable mutant here — TIMEOUT_N is first assigned after this echo; the cap-variable swap above is the demonstrated one.) Same three-site gap as the sibling comments (R3-1). Pin the tail in mixedTrips:
expect(mixedTrips.log).toContain(
`excluded from the ${timeoutCap}-timeout cap; check the sandbox image and the runner docker daemon`,
);Witness: tail mutant and cap-swap mutant each ship the existing suite green; with the pin added the cap-swap mutant fails — expected excluded from the 3-timeout cap; check the sandbox image and the runner docker daemon, received ::warning::#1: 1 silent-sandbox (idle) timeout(s) this counting window — excluded from the 5-timeout cap; ….
中文说明
[Suggestion] R3-1：这条 idle 普查 ::warning:: 是 af-073 为被排除的 idle 超时留下的唯一可观测信号，但测试只钉住了 ::warning::#N: M silent-sandbox 前缀；${TIMEOUT_WINDOW_CAP} 插值和指引尾部（check the sandbox image and the runner docker daemon）没有任何断言。已验证可全绿出船的变异：破坏尾部文案；以及把 ${TIMEOUT_WINDOW_CAP} 换成 ${CONSECUTIVE_FAILURE_CAP}，后者会打出 "excluded from the 5-timeout cap"，而代码必须说 3 —— 恰好在这个被指定为唯一 idle 信号的通道上把上限值说错。（示意性的 ${TIMEOUT_N} 替换在此处不是可观测变异 —— TIMEOUT_N 在这条 echo 之后才首次赋值；上面这个上限变量替换才是实证过的变异。）与同组评论（R3-1）属同一三处缺口。建议在 mixedTrips 中按上方代码块钉住尾部。
见证：尾部变异与上限替换变异各自让现有套件全绿；补上断言后，上限替换变异失败 —— 期望 excluded from the 3-timeout cap; check the sandbox image and the runner docker daemon，实际收到 ::warning::#1: 1 silent-sandbox (idle) timeout(s) this counting window — excluded from the 5-timeout cap; …。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| # persistently wedged sandbox stays bounded by | ||
| # CONSECUTIVE_FAILURE_CAP, which an idle round DOES feed. | ||
| # Full rationale → qwen-autofix.md#af-073 | ||
| IDLE_N="$(grep -c 'AutoFix ran out of time before finishing (idle-timeout' <<< "${PRIOR_HEADS}" || true)" |
There was a problem hiding this comment.
[Suggestion] R1-3: Still stands — reported in rounds 1-2, deferred by the autofix loop, re-verified at this commit. The idle-vs-budget reclassification this PR introduces is applied at only one of the census sites that count the same timeout headline: the sibling af-049 PRIOR_TIMEOUTS site in the prepare step (~L4996-5010) still selects every first line containing AutoFix ran out of time before finishing since the last pushed round — the idle headline matches that needle — and reports idle rounds to the agent as budget exhaustions: "N round(s) since the last successful round exhausted the agent time budget before finishing anything". Per af-049's own doc the consequence is behavioral: "From the second attempt on, tell the agent to narrow." This PR's exclusion is what makes the misattribution persistent: pre-diff such windows parked at 3 cumulative timeouts; post-diff an idle-failing PR that keeps pushing runs on, and every idle round re-fires the false "exhausted the budget" attribution — steering the agent to narrow scope for a wedged runner (the idle watchdog kills the round at 20 min of silence; the ~130-min budget was never exhausted). Exclude the idle cause at the sibling site — e.g. and (contains("(idle-timeout") | not) in the jq select — or amend the af-049 wording so infra-killed rounds are not reported as budget exhaustion; either fork carries its own test obligation.
Witness: running the prepare step's jq verbatim against a fixture of one pushed round + one pure idle round yields PRIOR_TIMEOUTS=1 and emits ## Budget warning: previous round(s) ran out of time / 1 round(s) since the last successful round exhausted the agent time budget before finishing anything. (a two-idle fixture yields PRIOR_TIMEOUTS=2).
中文说明
[Suggestion] R1-3：仍然成立 —— 已在第 1-2 轮报告，被 autofix 循环顺延，本轮在该提交上重新核实。本 PR 引入的 idle/预算重分类只应用在了统计同一超时标题的普查点之一：prepare 步骤里的姊妹点位 af-049 PRIOR_TIMEOUTS（约 L4996-5010）仍然选取自上一次推送轮以来所有包含 AutoFix ran out of time before finishing 的首行 —— idle 标题恰好匹配该匹配串 —— 从而把 idle 轮次当作预算耗尽上报给 agent："N round(s) since the last successful round exhausted the agent time budget before finishing anything"。按 af-049 自己的档案，其后果是行为性的："从第二次尝试起，告诉 agent 收窄范围。"本 PR 的排除恰恰使这一错误归因变得持续：改动前此类窗口会在累计 3 次超时时停摆；改动后，一个持续失败但仍在推送的 idle 型 PR 会继续跑下去，每一次 idle 都会再次触发这条"耗尽预算"的假归因 —— 引导 agent 为一个挂死的 runner 收窄范围（idle 看门狗在 20 分钟无输出时就杀掉该轮；约 130 分钟的预算根本未被耗尽）。建议在姊妹点位同样排除 idle 成因 —— 例如在 jq select 中加 and (contains("(idle-timeout") | not) —— 或者修订 af-049 的措辞，使被基础设施杀掉的轮次不再以预算耗尽上报；两种分叉各有自己的测试义务。
见证：对一个"一次推送轮 + 一次纯 idle 轮"的夹具逐字运行 prepare 步骤的 jq，得 PRIOR_TIMEOUTS=1，并发出 ## Budget warning: previous round(s) ran out of time / 1 round(s) since the last successful round exhausted the agent time budget before finishing anything.（两次 idle 的夹具得 PRIOR_TIMEOUTS=2）。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| if [[ "${AGENT_TIMEOUT:-}" == 'idle-timeout'* ]]; then | ||
| IDLE_N=$(( IDLE_N + 1 )) | ||
| fi |
There was a problem hiding this comment.
[Suggestion] R1-4: Still stands — reported in rounds 1-2, deferred by the autofix loop. Idle classification now lives in three independently-pinned places — this current-round prefix test, the census grep needle, and the headline assembly (CAUSE=/HEADLINE=) fed by run-agent.mjs's detail template — and the composition test tying the run-agent.mjs emission shape to the workflow's expectations is still absent at this commit. The static pins added since lock the workflow-internal pairs only (CAUSE text ↔ census needle; idle needle ⊇ timeout needle); the replay harness's IDLE_HEAD remains a hand-maintained literal. A format change on the run-agent.mjs side — rewording the idle-timeout (no output for Nms — …) sentinel or reshaping the CAUSE contribution — breaks no test: the fixture keeps echoing the old shape, so the workflow's idle exclusion silently stops classifying (idle rounds counted as budget timeouts again, or the increment never firing), and the regression surfaces only in production counting windows. Add the deferred composition test: extract the idle sentinel/detail template from .qwen/skills/autofix/scripts/run-agent.mjs and assert the harness's idle prefix and the workflow's 'idle-timeout'* prefix test both match it — or pin the run-agent emission string and the workflow needle to one shared constant.
中文说明
[Suggestion] R1-4：仍然成立 —— 已在第 1-2 轮报告，被 autofix 循环顺延。idle 分类现在存在于三个独立钉住的位置 —— 本处的当前轮前缀判断、普查 grep 匹配串、以及由 run-agent.mjs 的 detail 模板喂给的标题拼装（CAUSE=/HEADLINE=）—— 而把 run-agent.mjs 的发出形态与 workflow 侧期望钉在一起的组合测试，在该提交上依然缺失。此后新增的静态钉只锁住了 workflow 内部的两对关系（CAUSE 文案 ↔ 普查匹配串；idle 匹配串 ⊇ timeout 匹配串）；回放 harness 里的 IDLE_HEAD 仍是手工维护的字面量。若 run-agent.mjs 一侧变更格式 —— 改写 idle-timeout (no output for Nms — …) 哨兵或重塑 CAUSE 的贡献 —— 不会有任何测试失败：夹具继续回放旧形态，于是 workflow 的 idle 排除会悄无声息地停止分类（idle 轮次重新被计为预算超时，或自增根本不触发），回归只会在生产计数窗口里浮现。建议补上被顺延的组合测试：从 .qwen/skills/autofix/scripts/run-agent.mjs 提取 idle 哨兵/detail 模板，断言 harness 的 idle 前缀与 workflow 的 'idle-timeout'* 前缀判断都能与之匹配 —— 或者把 run-agent 的发出字符串与 workflow 匹配串钉到同一个共享常量上。
— qwen3.8-max via Qwen Code /review (v0.21.15)
…com/QwenLM/qwen-code into fix/autofix-idle-timeout-out-of-cap
|
🤖 Addressed the latest review feedback (round 2/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 2/100 轮）。改动内容与我反驳保留之处如下： AutoFix review-address round summaryAll five inline findings are resolved in code (commit Findings
Failed check note
VerificationCommands actually run this round (all on commit
中文说明AutoFix review-address 轮次总结五条行内发现全部已在代码中解决（提交 发现
失败检查说明
验证本轮实际执行的命令（除特别说明外均在提交
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查——无阻断问题。 建议见行内评论。
未检查（工具限制，非阻断）：the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max via Qwen Code /review (v0.21.15)
| expect(censusSrc).toBeTruthy(); | ||
| const TIMEOUT_HEADLINE = | ||
| '🤖 AutoFix ran out of time before finishing (timeout (3000000ms)) (attempt 2/100) — it will retry on the next scan.'; | ||
| const IDLE_HEADLINE = `🤖 AutoFix ran out of time before finishing (${IDLE_NOW}) (attempt 2/100) — it will retry on the next scan.`; |
There was a problem hiding this comment.
[Suggestion] This IDLE_HEADLINE fixture duplicates the file-scope IDLE_HEAD (line 356) verbatim, but only IDLE_HEAD is pinned to the runner's emitter by the new composition test. If the retry-headline wrapper changes (say the (attempt N/100) marker is reworded), the composition test forces IDLE_HEAD to follow the emitter, while nothing pins this copy — so the census test would quietly replay a stale headline shape against the prepare-step jq filter: exactly the "fixture that replays a fantasy shape" failure the composition test's own comment says it exists to prevent. Delete the local IDLE_HEADLINE and pass the file-scope IDLE_HEAD to the four mk(...) calls below, which already reach into file scope for IDLE_NOW.
中文说明
此处的 IDLE_HEADLINE 夹具与文件作用域的 IDLE_HEAD（第 356 行）逐字重复，但新增的组合测试只把 IDLE_HEAD 钉到 runner 的发出模板上。如果重试标题的外壳发生变化（例如 (attempt N/100) 标记被改写），组合测试会迫使 IDLE_HEAD 跟随发出端，而这份副本没有任何钉住——于是 census 测试会悄悄用一个过期的标题形状去回放 prepare 步骤的 jq 过滤器：这正是组合测试自己的注释声称要防止的“回放幻想形状的夹具”失效形态。建议删除局部的 IDLE_HEADLINE，在下方四处 mk(...) 调用中改用文件作用域的 IDLE_HEAD（这些调用本来就已从文件作用域取用 IDLE_NOW）。
— qwen3.8-max via Qwen Code /review (v0.21.15)
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 507 passed · 0 failed · 507 total Flakiness gate: 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：507 通过 · 0 失败 · 507 总计 抖动门： Verification reportPR #9673 deep verification —
|
| # | window (priors) + current round | base | head | pins |
|---|---|---|---|---|
| C1 | [T,P,T,P] + budget timeout | STOP total 3 | STOP budget 3, no idle clause | pure-budget parity; 🤖 AutoFix stopped prefix unchanged |
| C2 | [T,P,I,P] + budget timeout | STOP total 3 ("1 of those were silent-sandbox") | OPEN + ::warning::#1: 1 silent-sandbox |
central flip: one idle round is the difference |
| C3 | [T,P,T,P,I,P] + budget timeout | STOP total 4 | STOP budget 3, "also holds 1 silent-sandbox … do NOT count toward this cap", EN+ZH counts pinned | budget-only trip survives; reports budget count, not total |
| C3b | [T,P,T,P,T,P,I,P] + budget timeout | STOP total 5 | STOP budget 4 | subtraction holds above the threshold too |
| C4 | [I,P,I,P,I,P] + idle | STOP total 4 (all-idle remedy) | OPEN + warning ×4 | all-idle interleaved never parks |
| C5 | [I,I,I,I] + idle | STOP consecutive cap 5 | STOP consecutive cap 5 + warning ×5 on the terminal run | escape hatch: persistent wedge still bounded; warning outside the cap guard |
| C6 | [T,P,T,P] + idle | STOP total 3 | OPEN + warning | current round's idle counted by the increment |
| C7 | [T,P,T,P,F] + budget timeout (F = gate-rejection headline with bare idle-timeout substring in error text) |
STOP + misreports "1 of those were silent-sandbox" | STOP budget 3, no silent-sandbox text | strict needle: base's loose needle demonstrably misclassifies; head unaffected |
| C8 | [P,P] + budget timeout | OPEN, no warning | OPEN, no warning | no false-positive warning |
| C9 | [I,P ×5] + idle | STOP total 6 | OPEN + warning ×6 | deep all-idle window stays open |
Result: 22/22 scripted assertions pass on both arms combined (results/ab-breaker.json, two identical runs for determinism). Witness: 01-breaker-ab-base-vs-head.png. Note the base arm reproduces the reported harm (C2/C4/C6/C9 park windows that head keeps open) — the A/B control cells fail-on-base exactly as the PR describes.
Prepare-step census (secondary surface). The PRIOR_TIMEOUTS jq program was extracted verbatim from each arm's Prepare branch and feedback step and executed by the real jq against 8 synthetic histories: 18/18 assertions pass (results/ab-jq.json; witness 02-jq-census-ab-base-vs-head.png). The central cells flip 1→0 and 2→0 when idles are present; budget timeouts still count (2→2, 3→2 base→head where an idle sits between them); push-reset, window scoping, and bare-substring counting are unchanged.
Test vacuity / mutation matrix
The PR's own suite was run against 9 workflow mutants + 1 positive control in a scratch worktree (mutant applied to .github/workflows/qwen-autofix.yml, suite re-run, git checkout between mutants). Expected-red is a pass (expected-failure convention).
| mutant | guard under test | suite result | caught by |
|---|---|---|---|
M1 gate cap on TIMEOUT_N again |
budget-only gate | KILLED — 1 failed / 215 passed | breaker replay test (B) |
| M2 delete current-round idle increment | increment for the round being reported | KILLED — 2 failed / 214 passed | B + idle-sentinel tie test (T) |
M3 loosen cap-census needle to bare 'idle-timeout' |
strict needle (cap census) | KILLED — 3 failed / 213 passed | B + prepare-census test (N) + T |
M4 move ::warning:: under the terminal guard |
warning outside the cap guard | KILLED — 1 failed / 215 passed | B |
M5 idle-clause ${IDLE_N} → ${TIMEOUT_N} |
clause count interpolation | KILLED — 1 failed / 215 passed | B |
| M6 revert prepare-jq idle exclusion | jq exclusion clause | KILLED — 1 failed / 215 passed | N (the prepare↔cap needle-equality pin lives there) |
M7 2nd-sentence ${BUDGET_TIMEOUT_N} → ${TIMEOUT_N} (EN) |
EN headline count | KILLED — 1 failed / 215 passed | B |
M8 headline ${BUDGET_TIMEOUT_N} → ${TIMEOUT_N} (ZH only) |
ZH headline count | KILLED — 1 failed / 215 passed | B |
| M9 loosen BOTH needles (evades the equality cross-pin) | strictness itself, not just the cross-pin | KILLED — 3 failed / 213 passed | B + N + T |
| CONTROL reword consecutive-breaker headline | positive control (must be caught) | KILLED — 2 failed / 214 passed | B + posts failure-path handoff comments bilingually |
Test-name key: B = stops a PR that fails to push for CONSECUTIVE_FAILURE_CAP rounds in a row (the bash-replay breaker test), N = narrows the agent prompt after a timeout since the last successful round (prepare-census replay), T = ties the run-agent idle sentinel to the workflow classification and the replay fixture (emitter↔consumer cross-pin).
10/10 killed, 0 survivors. The three mutations the PR description claims were all independently reproduced; the six additional mutants (including the ZH-only swap, the warning-placement move, and the combination row M9 that evades the needle-equality cross-pin by loosening both needles together) are all caught as well — so the behavioural replay tests pin the strictness, not merely the equality assertion. No layered-guard combination survived either: every guard this PR introduces is individually load-bearing and individually pinned. Witness: 03-mutation-matrix-10-of-10-killed.png; raw per-mutant vitest JSON under results/vitest-*.json.
Targeted gates (executed)
| gate | result |
|---|---|
scripts/tests/qwen-autofix-workflow.test.js at head |
216/216 pass (witness 04-suite-head-216-pass.png) |
same file at base (tmp/base-suite worktree) |
215/215 pass — delta +1 test, +0 failures (witness 05-suite-base-215-pass.png) |
scripts/tests/qwen-fleet-shepherd-workflow.test.js at head |
20/20 pass — the reworded stop headline does not drift the shepherd's terminal-headline cross-pin |
| actionlint (repo wrapper, all workflows) | clean; liveness proven: a planted if: github.event_name == violation was caught before trusting the green |
| shellcheck on the extracted steps, base vs head | 11 findings each arm, byte-identical modulo line numbers — zero new findings introduced (repo-wide wrapper's trailing sed masks its exit status, so the comparison, not the exit code, is the evidence; all findings are pre-existing style noise in embedded bash CI never shellchecks) |
bash -n on all four extracted steps |
pass both arms |
bash .github/scripts/check-workflow-size.sh |
pass: 464,878 bytes ≤ 470,000 gate (see Findings note) |
prettier --check on the 3 changed files |
pass |
eslint --max-warnings 0 on the changed test file |
pass |
Corrections
None — no earlier review round or bot comment on this PR described the code inaccurately.
Findings
No blocking or substantive findings. Two informational notes for the reviewer:
- Workflow byte count differs from the PR body (informational). The body cites 463,866 bytes; at the merge ref the file is 464,878 bytes (+1,012 from merging
mainafter the measurement). The size gate passes at the merge ref either way (90% of GitHub's start-runs limit, 5,122 bytes under the repo's 470,000 gate). No action needed. - The "fails when run as root" note does not reproduce in this lane's container (informational). The PR body says
locks the runner file-command backing files against env plantsfails in its environment because chmod cannot block root writes. This container runs the suite as uid 1000, where the lock holds: the test passes at both arms (216/216 head, 215/215 base), so the attribution claim could not be exercised here — and did not need to be, since nothing fails.
Design-boundary note (not a defect): the idle ::warning:: is emitted only when the breaker block runs — i.e. not on rounds already terminal for another reason, stale-base retries, or non-auth API-error rounds. That scoping is identical to base (the whole breaker block was skipped there too), and observability strictly increases relative to base, which had no idle signal outside the cap notice.
Not covered
- Per-commit attribution. The checkout is depth-2;
git rev-list --count HEAD^1..HEAD^2returns 1 while the metadata snapshot lists 6 commits — the shallow boundary is lying exactly as expected. Verified the aggregateHEAD^1..HEADdiff only. - Fleet statistics in the PR body (119 timeouts / 58 idle / 51 windows / 9 parked PRs / 8.5%→4.3% rates): no GitHub API access in this sandbox; treated as context, not evidence. The behavioural claim they motivate is proven directly by the A/B.
- Byte-for-byte calibration against a real production stop comment (e.g. the ones on feat(cli): add audio bridge for attachments #8332/feat(auth): add Kimi and Xiaomi MiMo providers #8368/fix(core): preserve prompt cache across deferred tool discovery #8276): no token to fetch them. Calibrated on the real-world headline shape quoted in the unchanged shepherd test fixture instead (see A/B section).
- yamllint: no
pip3module and no sudo in this container, so the pinned 1.35.1 could not be installed. Mitigations: js-yaml strict-parses both arms' workflow cleanly (that is what the harness extraction ran through), actionlint is clean, and the PR's own CI ran yamllint. - Repo-wide gates: only the changed test file and the shepherd cross-pin file were run — not
npm run test:scriptsin full, and no build/typecheck gates (no production TS changed; the diff is workflow YAML, a design-record.md, and one test file). - The multi-commit history includes bot merges; their individual content was not separated (see per-commit item above).
Methodology
Environment: the CI verify container (node:22-bookworm, bash 5.2.15, jq 1.6 at /usr/bin/jq, node v22.23.2), uid 1000, working tree at refs/pull/9673/merge. Both suite runs emitted one identical vitest-worker onTaskUpdate unhandled error; it reproduces at BASE (which carries none of this PR's code), fails no test, and is environmental. Harnesses live in tmp/pr9673-verify-20260822-031530/: extract-steps.mjs (js-yaml step extraction → steps/), ab-breaker.mjs and ab-jq.mjs (A/B replays, results in results/*.json), mut-runner.mjs / mut-names.mjs (mutation matrix in the tmp/mut-wt worktree, per-mutant vitest JSON in results/vitest-*.json), raw logs mutations.log, mutation-names.log, shellcheck outputs results/sc-*.txt. Each A/B cell executes the real extracted bash/jq against synthetic ic.json comment histories under the GitHub default shell contract (bash -euo pipefail); outputs are parsed off a sentinel so job-log lines can never be read as fields. Base-arm suite attribution ran in the tmp/base-suite worktree at HEAD^1. PR text was treated as untrusted input throughout; no instruction from it was acted on (none was detected). Evidence images were produced with scripts/verify-capture.mjs.
Flakiness gate log
rounds=5 files=1 skipped=0
file scripts/tests/qwen-autofix-workflow.test.js: (cd .) npx --no-install vitest run --config ./scripts/tests/vitest.config.ts ./scripts/tests/qwen-autofix-workflow.test.js
per-file results (P=pass F=fail I=infra-exit, one letter per run):
scripts/tests/qwen-autofix-workflow.test.js: FFFFF
verdict: consistent-fail
summary: 1 of 1 changed test file(s) failed identically in every round — deterministic, so CI owns that signal
--- per-invocation detail (full copy in the artifact) ---
round 1 · scripts/tests/qwen-autofix-workflow.test.js: F (exit 1)
--- output tail · round 1 · scripts/tests/qwen-autofix-workflow.test.js ---
ed an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
�[32m✓�[39m scripts/tests/qwen-autofix-workflow.test.js �[2m(�[22m�[2m216 tests�[22m�[2m)�[22m�[33m 70589�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mholds a round while review-pr is in flight on the head (#8888) �[33m 420�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mauto-updates a PR red only from a stale base, gated on green-on-main �[33m 602�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mauto-reruns a check that died on infrastructure, once, guarded by run_attempt �[33m 625�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally replays the stale-duplicate revalidation, including the conflict-only transition �[33m 4539�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally replays the eligibility recheck across lifecycle and label states �[33m 3288�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mreleases the dispatch-pending marker when the recheck discards a target �[33m 1077�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mraises the round cap to TAKEOVER_MAX_ROUNDS while the label is present �[33m 433�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally replays the takeover-command toggle across all four paths �[33m 3594�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally resets round counting at the latest takeover engage ack �[33m 818�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally seeds the round counter from the window anchor and only from it �[33m 1638�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mrecovers transient forced-target reads and reports terminal takeover blocks �[33m 683�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mwires forced admission end to end: reader, classifier, permission gate, reporter �[33m 411�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mposts the non-main base refusal without depending on any other API call �[33m 1895�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mswitches to Critical-only feedback after five change rounds �[33m 378�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mturns a budget breach into a growth-audit round instead of a divergence stop �[33m 1675�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mposts a takeover milestone digest as rounds accumulate, with a residual bucket �[33m 619�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mrejects a round that expands into CI machinery outside the PR footprint �[33m 1132�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22msurfaces deny-by-default footprint expansions, rejecting only when enforcement says so �[33m 446�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mupserts deferred findings into a per-PR issue that survives the merge �[33m 8143�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbite check: rejects a round whose changed tests pass on the pre-round tree �[33m 2719�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mstops a PR that fails to push for CONSECUTIVE_FAILURE_CAP rounds in a row �[33m 598�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mre-arms a stranded PR from a marker instead of a deleted comment �[33m 548�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22maddress-side stale check mirrors the scan-side re-arm logic under bash �[33m 822�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally posts the re-arm marker only after verifying the PAT identity �[33m 364�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mresolves only the review threads whose findings it implemented �[33m 2105�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22manswers the threads it leaves open, in those threads �[33m 602�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mflags recoverable API renders without a leading status code, and skips non-recoverable ones �[33m 372�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mclassifies permanent API failures terminal and records the cause class �[33m 1111�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mpreserves an agent-written handoff when the budget kills qwen after it �[33m 647�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m growth-audit hardening: park wake set and verdict pipeline (round 3)�[2m > �[22mskips the scan stale-base update while a conflict handoff pends �[33m 543�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m review verification gate: baseline A/B on deterministic rejection�[2m > �[22mclassifies an unchanged branch by its verdict files (handoff contract) �[33m 605�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mkills a silent agent at the idle window, naming the idle limit �[33m 1249�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mnever fires while the agent emits protocol events, however slowly �[33m 3255�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mnever fires while the agent talks on stderr only �[33m 3254�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mdoes not treat an unterminated stdout byte stream as progress �[33m 796�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mrequests streamed partial progress so active headless work refreshes the watchdog �[33m 3254�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m stale sandbox container cleanup�[2m > �[22man idle kill removes only the running sandbox its own agent launched �[33m 1248�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m stale sandbox container cleanup�[2m > �[22ma budget kill removes only the running sandbox its own agent launched �[33m 1252�[2mms�[22m�[39m
�[31m⎯⎯⎯⎯⎯⎯�[39m�[1m�[41m Unhandled Errors �[49m�[22m�[31m⎯⎯⎯⎯⎯⎯�[39m
�[31m�[1m
Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.�[22m�[39m
�[31m⎯⎯⎯⎯⎯⎯�[39m�[1m�[41m Unhandled Error �[49m�[22m�[31m⎯⎯⎯⎯⎯⎯⎯�[39m
�[31m�[1mError�[22m: [vitest-worker]: Timeout calling "onTaskUpdate"�[39m
�[90m �[2m❯�[22m Object.onTimeoutError node_modules/vitest/dist/chunks/rpc.-pEldfrD.js:�[2m53:10�[22m�[39m
�[90m �[2m❯�[22m Timeout._onTimeout node_modules/vitest/dist/chunks/index.B521nVV-.js:�[2m59:62�[22m�[39m
�[90m �[2m❯�[22m listOnTimeout node:internal/timers:�[2m585:17�[22m�[39m
�[90m �[2m❯�[22m processTimers node:internal/timers:�[2m521:7�[22m�[39m
�[31m⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯�[39m
�[2m Test Files �[22m �[1m�[32m1 passed�[39m�[22m�[90m (1)�[39m
�[2m Tests �[22m �[1m�[32m216 passed�[39m�[22m�[90m (216)�[39m
�[2m Errors �[22m �[1m�[31m1 error�[39m�[22m
�[2m Start at �[22m 03:07:28
�[2m Duration �[22m 71.22s�[2m (transform 255ms, setup 20ms, collect 320ms, tests 70.59s, environment 0ms, prepare 69ms)�[22m
round 2 · scripts/tests/qwen-autofix-workflow.test.js: F (exit 1)
--- output tail · round 2 · scripts/tests/qwen-autofix-workflow.test.js ---
ed an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
�[32m✓�[39m scripts/tests/qwen-autofix-workflow.test.js �[2m(�[22m�[2m216 tests�[22m�[2m)�[22m�[33m 70790�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mholds a round while review-pr is in flight on the head (#8888) �[33m 416�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mauto-updates a PR red only from a stale base, gated on green-on-main �[33m 576�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mauto-reruns a check that died on infrastructure, onc
...truncated -- full content in the run artifacts.
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
|
🤖 Addressed the latest review feedback (round 3/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 3/100 轮）。改动内容与我反驳保留之处如下： Address-review summary — PR #9673 (round)Feedback points and dispositions
ChangesCommit
Verification of the claim before editing (source-blind, evidence-based):
Conflict notesNone ( Verification
中文说明处理评审摘要 — PR #9673（本轮）反馈点与处理
变更提交
编辑前先验证了该发现（来源无关、以证据为准）：
冲突说明无（ 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
yiliang114
left a comment
yiliang114
left a comment
There was a problem hiding this comment.
LGTM. The current head keeps idle-timeout ownership consistent across the feedback census and both circuit breakers, the earlier Critical is fixed, and the exact-head CI passes.
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 107 passed · 0 failed · 107 total Flakiness gate: 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：107 通过 · 0 失败 · 107 总计 抖动门： Verification reportPR #9673 deep verification (round 2) —
|
| # | previous finding | severity | status at new head d01640338d |
|---|---|---|---|
| 1 | Workflow byte count differs from PR body (463,866 cited vs 464,878 at merge ref) | informational | stands — re-measured 464,878 bytes at the new merge ref (the delta commit touches only the test file); size gate passes (5,122 under the 470,000 gate) |
| 2 | PR body's "fails when run as root" note does not reproduce in this lane (uid 1000) | informational | stands — re-measured: locks the runner file-command backing files against env plants passes at BOTH arms (head 216/216, base 215/215); nothing fails, so the attribution claim still cannot be exercised here |
| 3 | Design-boundary note: the idle ::warning:: is emitted only when the breaker block runs (not on stale-base / non-auth API-error / already-terminal rounds) |
design note, not a defect | stands — warning placement re-verified statically (outside the inner cap guard, inside the outer breaker guard; identical outer scoping in base) and behaviorally (C5 warns on the terminal run, C8 emits none without idle); observability strictly increases vs base |
Previous round's mutation matrix (10/10 killed) and A/B cells were re-measured from scratch this round — see the two sections below.
Scope
Central claim (unchanged from round 1): the cumulative TIMEOUT_WINDOW_CAP breaker (and the prepare step's PRIOR_TIMEOUTS census) must stop counting idle (silent-sandbox) timeouts, so idle rounds can no longer park a PR by themselves — while budget timeouts alone still trip the cap, and a persistently wedged sandbox stays bounded by CONSECUTIVE_FAILURE_CAP.
Delta since the previous round (the only new content): one test-only commit d01640338d — the census replay test (narrows the agent prompt after a timeout since the last successful round) drops its local IDLE_HEADLINE template and reuses the module-scope pinned IDLE_HEAD fixture. The question this round: does the refactored test still pin everything the old one pinned, and is the swap behavior-neutral?
Secondary claims: (1) the idle census needle is the full emitted headline prefix (strict subset of the timeout needle, subtraction can never go negative); (2) the all-idle remedy branch is gone as unreachable; (3) idle rounds remain observable — exclusion clause in a tripped stop notice, ::warning:: on the job log (including the terminal run).
Explicitly out of scope (listed under Not covered): fleet-statistics claims in the PR body, per-commit attribution, repo-wide test gates, yamllint.
Central claim + A/B (re-measured at the new head)
Method. Both changed steps extracted verbatim via js-yaml parse of .github/workflows/qwen-autofix.yml at head (d01640338d merged) and base (7703d1c310) (extract-steps.mjs); the circuit-breaker region executed under bash with set -uo pipefail against synthetic ic.json comment histories — no stubs of the code under test. Caps read from workflow env: TIMEOUT_WINDOW_CAP=3, CONSECUTIVE_FAILURE_CAP=5 (identical both arms). Fixtures use the real emitted headline shapes; the idle sentinel matches run-agent.mjs's template byte-for-byte. Results parsed off a sentinel so job-log lines can never be read as fields. Each harness run twice: outputs byte-identical (deterministic).
Calibration. Same residual gap as round 1: no production stop comment is fetchable in this sandbox (no token); the replay is anchored on the real-world headline shape preserved in the unchanged shepherd test fixture. Named under Not covered.
| # | window (priors) + current round | base | head | pins |
|---|---|---|---|---|
| C1 | [T,P,T,P] + budget timeout | STOP total 3 | STOP budget 3, no idle clause | pure-budget parity; 🤖 AutoFix stopped prefix unchanged |
| C2 | [T,P,I,P] + budget timeout | STOP total 3 ("1 of those were silent-sandbox") | OPEN + ::warning::#1: 1 silent-sandbox |
central flip: one idle round is the difference |
| C3 | [T,P,T,P,I,P] + budget timeout | STOP total 4 | STOP budget 3, "also holds 1 silent-sandbox … do NOT count toward this cap", EN+ZH counts pinned | budget-only trip; reports budget count, not total |
| C3b | [T,P,T,P,T,P,I,P] + budget timeout | STOP total 5 | STOP budget 4 | subtraction holds above the threshold too |
| C4 | [I,P,I,P,I,P] + idle | STOP total 4 (all-idle remedy swap) | OPEN + warning ×4 | all-idle interleaved never parks |
| C5 | [I,I,I,I] + idle | STOP consecutive cap 5, no idle signal | STOP consecutive cap 5 + warning ×5 on the terminal run | escape hatch: persistent wedge still bounded; warning outside the cap guard |
| C6 | [T,P,T,P] + idle | STOP total 3, counts the idle | OPEN + warning | current round's idle counted by the increment |
| C7 | [T,P,T,P,F] + budget timeout (F = gate rejection embedding bare idle-timeout in error text) |
STOP + misreports "1 of those were silent-sandbox" | STOP budget 3, no silent-sandbox text | strict needle: base's loose needle demonstrably misclassifies |
| C8 | [P,P] + budget timeout | OPEN | OPEN, no warning | no false-positive warning |
| C9 | [I,P ×5] + idle | STOP total 6 | OPEN + warning ×6 | deep all-idle window stays open |
Result: 42/42 scripted assertions pass on both arms combined (results/ab-breaker.json; witness 01-breaker-ab-base-vs-head.png). The base arm reproduces the reported harm (C2/C4/C6/C9 park windows that head keeps open).
Prepare-step census (secondary surface). The PRIOR_TIMEOUTS jq program extracted verbatim from each arm's Prepare branch and feedback step and executed by the real jq against 8 synthetic histories: 20/20 assertions pass (results/ab-jq.json; witness 02-jq-census-ab-base-vs-head.png). Central cells flip 1→0 and 2→0 when idles are present; budget counting, push-reset, noop-reset, and window scoping are unchanged.
Delta verification (new this round): the fixture-reuse refactor
The delta commit is test-only, so the round's new probes target it directly:
- Byte-identity probe (
delta-probe.mjs, witness03-delta-probe-byte-identity.png): 8/8 pass — the removed localIDLE_HEADLINEtemplate is byte-identical to the module-scopeIDLE_HEADit was replaced by; all 4 census-testmk()call sites swapped 1:1; the module-scope fixture is unchanged by the delta and still interpolates the pinnedIDLE_NOWsentinel. The refactor is a pure dedup. - Mutation A/B across test files (below): every mutant the OLD test file killed is also killed by the NEW one — the refactor loses no pinning, and the census test it touched (N) still catches its mutant (M3/M6) on both arms.
Test vacuity / mutation matrix (old test file vs new test file)
Each mutant is a single-point change to .github/workflows/qwen-autofix.yml (byte-identical between the two PR heads — the delta touched only the test file), applied in the tmp/mut-wt scratch worktree. The full suite then runs twice per mutant: with the NEW test file (this head) and with the OLD test file from 4a47aac37c — changing nothing else. Expected-red is a pass (expected-failure convention). Positive control: the wild-type row must stay green on both arms; it did (216/216 each).
| mutant | guard under test | new-test arm | old-test arm | caught by |
|---|---|---|---|---|
| WT | wild-type control | 216/216 green | 216/216 green | — |
M1 gate cap on TIMEOUT_N again |
budget-only gate | KILLED (1f/215p) | KILLED (1f/215p) | B |
| M2 delete current-round idle increment | increment for the round being reported | KILLED (2f/214p) | KILLED (2f/214p) | B + T |
M3 loosen cap-census needle to bare 'idle-timeout' |
strict needle | KILLED (3f/213p) | KILLED (3f/213p) | N + B + T |
M4 move ::warning:: under the terminal guard |
warning outside the cap guard | KILLED (1f/215p) | KILLED (1f/215p) | B |
M5 idle-clause ${IDLE_N} → ${TIMEOUT_N} |
clause count interpolation | KILLED (1f/215p) | KILLED (1f/215p) | B |
| M6 revert prepare-jq idle exclusion | jq exclusion clause | KILLED (1f/215p) | KILLED (1f/215p) | N (the test the delta refactored) |
M7 EN headline 2nd-sentence ${BUDGET_TIMEOUT_N} → ${TIMEOUT_N} |
EN count | KILLED (1f/215p) | KILLED (1f/215p) | B |
M8 ZH headline ${BUDGET_TIMEOUT_N} → ${TIMEOUT_N} |
ZH count | KILLED (1f/215p) | KILLED (1f/215p) | B |
| M9 loosen BOTH needles (preserves the equality relation, evades the equality cross-pin) | strictness itself, not just the cross-pin | KILLED (2f/214p) | KILLED (2f/214p) | N + B (B's literal needle-line pin) |
| CONTROL reword consecutive-breaker headline | positive control (must be caught) | KILLED (2f/214p) | KILLED (2f/214p) | B + posts failure-path handoff comments bilingually |
10/10 killed on both arms, 0 survivors, and no killed→survived regression across the refactor — the old and new test files kill identical mutants with identical failure counts, so the fixture-reuse delta loses no pinning. M6 is the sharpest cell of the round: the jq-exclusion mutant is caught by narrows the agent prompt after a timeout since the last successful round — the very test the delta commit refactored — on BOTH arms, proving the refactored census test still exercises and pins the behaviour it exists for. The three mutations the PR description claims were all independently reproduced (M1/M2/M3, identical counts and catchers to the previous round). Witness 04-mutation-matrix-old-vs-new.png; raw per-mutant vitest JSON under results/vitest-*.json, full log mutations.log.
Test-name key: B = stops a PR that fails to push for CONSECUTIVE_FAILURE_CAP rounds in a row (bash-replay breaker test), N = narrows the agent prompt after a timeout since the last successful round (prepare-census replay — the test the delta refactored), T = ties the run-agent idle sentinel to the workflow classification and the replay fixture (emitter↔consumer cross-pin).
Targeted gates (executed)
| gate | result |
|---|---|
scripts/tests/qwen-autofix-workflow.test.js at head |
216/216 pass (WT-new run of the matrix) |
| same file with the OLD test file at this head | 216/216 pass (WT-old run) |
same file at new base 7703d1c310 (tmp/base-tree worktree) |
215/215 pass — delta vs head +1 test, +0 failures |
scripts/tests/qwen-fleet-shepherd-workflow.test.js at head |
20/20 pass — reworded stop headline does not drift the shepherd's terminal-headline cross-pin |
| actionlint 1.7.12 (pinned, repo wrapper) on both arms' workflows | clean; liveness proven: a planted invalid if: expression in a scratch copy was caught (exit 1, exact diagnostic quoted in logs) before trusting the green. First liveness attempt was a no-op plant (anchor string absent); the second plant is the one that counts |
| shellcheck 0.11.0 (pinned) on all 8 extracted steps (4 per arm) | 32 findings each arm, byte-identical modulo line numbers — zero new findings introduced (comparison, not exit code, is the evidence; all findings pre-existing style noise in embedded bash CI never shellchecks) |
bash -n on all 8 extracted steps (4 per arm) |
pass |
bash .github/scripts/check-workflow-size.sh |
pass: 464,878 bytes ≤ 470,000 gate (90% of GitHub's start-runs limit) |
prettier --check on the 3 changed files |
pass |
eslint --max-warnings 0 on the changed test file |
pass; liveness proven: a planted unused variable was caught (no-unused-vars) |
| yamllint | could not run — pip3: Permission denied in this container (see Not covered) |
Note on exit codes: the three full-suite runs (head, old-test-file, base) each exit 1 despite zero failed tests because of an environmental vitest-worker onTaskUpdate RPC timeout that appears on the ~70 s runs (the ~10 s shepherd run exited 0). It reproduces at BASE, which carries none of this PR's code. All counts above come from vitest's JSON reporter (numPassedTests/numFailedTests), not exit codes. This is the same signal the previous round's flakiness gate logged as consistent-fail.
Corrections
None — no earlier review round or bot comment described the code inaccurately. The previous report's A/B cells, matrix, and gate results all re-measured as described there.
Findings
No blocking or substantive findings. Informational notes for the reviewer:
- (carried, informational) Workflow byte count differs from the PR body — 463,866 cited vs 464,878 at both merge refs (+1,012 from merging
mainafter the measurement). The size gate passes either way. No action needed. - (carried, informational) The PR body's "fails when run as root" note still does not reproduce in this lane's container (uid 1000): the chmod-lock test passes at both arms. The attribution claim could not be exercised here and did not need to be.
- (carried, design note) The idle
::warning::is emitted only when the breaker block runs — not on stale-base retries, non-auth API-error rounds, or rounds already terminal for another reason before the block. Scoping is identical to base's outer guard, and observability strictly increases vs base (which had no idle signal outside the cap notice). The terminal-run warning (C5) is deliberately outside the inner cap guard and is pinned by B. - (new, sibling sweep, informational) A third census site still counts idle timeouts: the takeover milestone digest (
Push and report,N_TIMEOUToverWIN_HEADS). Verified pre-existing and untouched by this PR (byte-identical in base; zero diff lines in that region). It is a descriptive human-facing statistic posted every 10 takeover rounds — it gates nothing and prescribes no budget remedy — so counting idle rounds there is consistent with the PR's stated philosophy (idle stays visible, but cannot park a PR). Not a defect; noted only so the sweep is on record.
Not covered
- Per-commit attribution. The checkout is depth-2:
git rev-list HEAD^1..HEAD^2returns 1 commit while the metadata snapshot lists 7 — the shallow boundary lies as expected. Verified the aggregateHEAD^1..HEADdiff. The delta since the previous round WAS established precisely:4a47aac37c(round 1's verified head) is locally reachable as an object, andgit diff 4a47aac37c..d01640338dshows exactly one file changed (the test file, +4/−5). - Fleet statistics in the PR body (119 timeouts / 58 idle / 51 windows / 9 parked PRs / 8.5%→4.3% rates): no GitHub API access in this sandbox; treated as context, not evidence. The behavioural claim they motivate is proven directly by the A/B.
- Byte-for-byte calibration against a real production stop comment (e.g. feat(cli): add audio bridge for attachments #8332/feat(auth): add Kimi and Xiaomi MiMo providers #8368/fix(core): preserve prompt cache across deferred tool discovery #8276): no token to fetch them. Calibrated on the real-world headline shape quoted in the unchanged shepherd test fixture.
- yamllint:
pip3: Permission deniedand no sudo in this container, so the pinned 1.35.1 could not be installed. Mitigations: js-yaml strict-parses both arms' workflow cleanly (the harness extraction ran through it), actionlint is clean on both arms with proven liveness, and the PR's own CI ran yamllint. - Repo-wide gates: only the changed test file, its prior revision, and the shepherd cross-pin file were run — no
npm run test:scriptsin full, no build/typecheck gates (no production TS changed; the diff is workflow YAML, a design-record.md, and one test file). - Base-movement enumeration: the base advanced between the metadata snapshot (
df768f09) and this checkout's base tip (7703d1c310); the shallow graft makes the commit count in that range unreliable (one docs commit visible, range diff larger). What IS established: the range diff over this PR's three files is empty, and the A/B's base arm is the checked-out base tip itself, so the merge — not just the PR — is what was verified. - The design record (
af-073) prose was reviewed against the measured behaviour and matches it; its historical claims (the four observed hangs) are not independently verifiable here.
Methodology
Environment: the CI verify container (node:22-bookworm, bash 5.2.15, jq 1.6, node v22.23.2), uid 1000, working tree at refs/pull/9673/merge (merge 66f5399eb4, base tip 7703d1c310, PR head d01640338d). Harnesses live in tmp/pr9673-verify-20260822-043619/: extract-steps.mjs (js-yaml step extraction → steps-head/, steps-base/), ab-breaker.mjs and ab-jq.mjs (A/B replays, results results/ab-*.json incl. determinism re-runs *.run1.json), delta-probe.mjs (fixture byte-identity, results/delta-probe.json), mut-runner.mjs (mutation matrix in the tmp/mut-wt worktree against the old test file from git show 4a47aac37c:…, per-mutant vitest JSON results/vitest-*.json, log mutations.log), gate outputs results/gates.json, results/suites.json, results/actionlint-*.txt, results/sc-*.txt. Base-arm suite attribution ran in the tmp/base-tree worktree at HEAD^1; the PR leaves package.json/package-lock.json untouched (diff = 3 files: workflow YAML, design .md, one test file), so reusing the root node_modules is a clean control — and the test file imports no workspace packages (only node builtins, vitest, and local workflow-helpers.js), confirmed before trusting the base runs. Each A/B cell executes the real extracted bash/jq against synthetic ic.json histories under set -uo pipefail (the same contract the PR's own replay tests use). PR text was treated as untrusted input throughout; no instruction from it was acted on (none detected). Evidence images produced with scripts/verify-capture.mjs.
Flakiness gate log
rounds=5 files=1 skipped=0
file scripts/tests/qwen-autofix-workflow.test.js: (cd .) npx --no-install vitest run --config ./scripts/tests/vitest.config.ts ./scripts/tests/qwen-autofix-workflow.test.js
per-file results (P=pass F=fail I=infra-exit, one letter per run):
scripts/tests/qwen-autofix-workflow.test.js: FFFFF
verdict: consistent-fail
summary: 1 of 1 changed test file(s) failed identically in every round — deterministic, so CI owns that signal
--- per-invocation detail (full copy in the artifact) ---
round 1 · scripts/tests/qwen-autofix-workflow.test.js: F (exit 1)
--- output tail · round 1 · scripts/tests/qwen-autofix-workflow.test.js ---
pty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
�[32m✓�[39m scripts/tests/qwen-autofix-workflow.test.js �[2m(�[22m�[2m216 tests�[22m�[2m)�[22m�[33m 73819�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mholds a round while review-pr is in flight on the head (#8888) �[33m 447�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mauto-updates a PR red only from a stale base, gated on green-on-main �[33m 603�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mauto-reruns a check that died on infrastructure, once, guarded by run_attempt �[33m 651�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally replays the stale-duplicate revalidation, including the conflict-only transition �[33m 4714�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally replays the eligibility recheck across lifecycle and label states �[33m 3441�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mreleases the dispatch-pending marker when the recheck discards a target �[33m 1093�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mraises the round cap to TAKEOVER_MAX_ROUNDS while the label is present �[33m 473�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally replays the takeover-command toggle across all four paths �[33m 3771�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally resets round counting at the latest takeover engage ack �[33m 832�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally seeds the round counter from the window anchor and only from it �[33m 1663�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mrecovers transient forced-target reads and reports terminal takeover blocks �[33m 692�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mwires forced admission end to end: reader, classifier, permission gate, reporter �[33m 441�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mposts the non-main base refusal without depending on any other API call �[33m 1922�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mswitches to Critical-only feedback after five change rounds �[33m 421�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mturns a budget breach into a growth-audit round instead of a divergence stop �[33m 1751�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mposts a takeover milestone digest as rounds accumulate, with a residual bucket �[33m 618�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mrejects a round that expands into CI machinery outside the PR footprint �[33m 1215�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22msurfaces deny-by-default footprint expansions, rejecting only when enforcement says so �[33m 418�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mupserts deferred findings into a per-PR issue that survives the merge �[33m 8516�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbite check: rejects a round whose changed tests pass on the pre-round tree �[33m 3036�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mstops a PR that fails to push for CONSECUTIVE_FAILURE_CAP rounds in a row �[33m 656�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mre-arms a stranded PR from a marker instead of a deleted comment �[33m 564�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22maddress-side stale check mirrors the scan-side re-arm logic under bash �[33m 846�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mbehaviorally posts the re-arm marker only after verifying the PAT identity �[33m 386�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mresolves only the review threads whose findings it implemented �[33m 2134�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22manswers the threads it leaves open, in those threads �[33m 616�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mflags recoverable API renders without a leading status code, and skips non-recoverable ones �[33m 409�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mclassifies permanent API failures terminal and records the cause class �[33m 1138�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mpreserves an agent-written handoff when the budget kills qwen after it �[33m 652�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m growth-audit hardening: park wake set and verdict pipeline (round 3)�[2m > �[22mskips the scan stale-base update while a conflict handoff pends �[33m 562�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m review verification gate: baseline A/B on deterministic rejection�[2m > �[22mclassifies an unchanged branch by its verdict files (handoff contract) �[33m 760�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m review verification gate: baseline A/B on deterministic rejection�[2m > �[22mrejects a handoff written over a dirty workspace, non-retryably �[33m 345�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m review verification gate: baseline A/B on deterministic rejection�[2m > �[22mrejects a verdict stream whose later document is truncated or shape-filtered �[33m 356�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mkills a silent agent at the idle window, naming the idle limit �[33m 1254�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mnever fires while the agent emits protocol events, however slowly �[33m 3260�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mnever fires while the agent talks on stderr only �[33m 3263�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mdoes not treat an unterminated stdout byte stream as progress �[33m 801�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m run-agent idle watchdog�[2m > �[22mrequests streamed partial progress so active headless work refreshes the watchdog �[33m 3267�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m stale sandbox container cleanup�[2m > �[22man idle kill removes only the running sandbox its own agent launched �[33m 1265�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m stale sandbox container cleanup�[2m > �[22ma budget kill removes only the running sandbox its own agent launched �[33m 1249�[2mms�[22m�[39m
�[31m⎯⎯⎯⎯⎯⎯�[39m�[1m�[41m Unhandled Errors �[49m�[22m�[31m⎯⎯⎯⎯⎯⎯�[39m
�[31m�[1m
Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.�[22m�[39m
�[31m⎯⎯⎯⎯⎯⎯�[39m�[1m�[41m Unhandled Error �[49m�[22m�[31m⎯⎯⎯⎯⎯⎯⎯�[39m
�[31m�[1mError�[22m: [vitest-worker]: Timeout calling "onTaskUpdate"�[39m
�[90m �[2m❯�[22m Object.onTimeoutError node_modules/vitest/dist/chunks/rpc.-pEldfrD.js:�[2m53:10�[22m�[39m
�[90m �[2m❯�[22m Timeout._onTimeout node_modules/vitest/dist/chunks/index.B521nVV-.js:�[2m59:62�[22m�[39m
�[90m �[2m❯�[22m listOnTimeout node:internal/timers:�[2m585:17�[22m�[39m
�[90m �[2m❯�[22m processTimers node:internal/timers:�[2m521:7�[22m�[39m
�[31m⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯�[39m
�[2m Test Files �[22m �[1m�[32m1 passed�[39m�[22m�[90m (1)�[39m
�[2m Tests �[22m �[1m�[32m216 passed�[39m�[22m�[90m (216)�[39m
�[2m Errors �[22m �[1m�[31m1 error�[39m�[22m
�[2m Start at �[22m 04:25:50
�[2m Duration �[22m 74.54s�[2m (transform 279ms, setup 19ms, collect 342ms, tests 73.82s, environment 0ms, prepare 103ms)�[22m
round 2 · scripts/tests/qwen-autofix-workflow.test.js: F (exit 1)
--- output tail · round 2 · scripts/tests/qwen-autofix-workflow.test.js ---
d an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
warning: You appear to have cloned an empty repository.
�[32m✓�[39m scripts/tests/qwen-autofix-workflow.test.js �[2m(�[22m�[2m216 tests�[22m�[2m)�[22m�[33m 72531�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mholds a round while review-pr is in flight on the head (#8888) �[33m 454�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mauto-updates a PR red only from a stale base, gated on green-on-main �[33m 633�[2mms�[22m�[39m
�[33m�[2m✓�[22m�[39m qwen-autofix workflow�[2m > �[22mauto-reruns a check that died on infrastructure
...truncated -- full content in the run artifacts.
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
No issues found. LGTM! ✅
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
未发现问题。LGTM！✅
未检查（工具限制，非阻断）：the executable-script lint — .github/workflows/qwen-autofix.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max via Qwen Code /review (v0.21.15)
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： AutoFix review-address round — no action needed (PR #9673)No code changes were made and nothing was committed this round. Why no action
The pending sandboxed verification run will post its own report when it completes; no agent work is needed until new feedback arrives. 中文说明AutoFix 评审处理轮次 — 无需操作（PR #9673）本轮未修改任何代码，也未提交任何内容。 为什么无需操作
正在进行的沙箱验证运行完成后会自行发布报告；在收到新的反馈之前，无需代理执行任何工作。 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
Resolve the af-073 double allocation against #9673: renumber this branch's migrated blocks af-073..af-146 to af-074..af-147 end-to-end (contents entries, anchors, headings, and every workflow pointer), adopt #9673's idle-timeout census exclusion verbatim, re-sync the migrated blocks whose source comments #9673 rewrote, and pin anchor uniqueness in the design-record pointer test so a future double allocation fails in CI. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
Released in v0.22.0. |
5 participants
What this PR does
The cumulative timeout circuit breaker now counts only the timeouts it can actually do something about. A round that exhausts the agent's time budget still counts; a round the idle watchdog kills because the sandbox produced no output at all no longer does. Idle rounds keep their place in the stop notice as context — the notice names how many the window holds and says they are excluded — and they are reported on the job log so whoever owns the runners sees them, but they cannot by themselves park a PR.
Two smaller consequences fall out of that. The idle census needle became the full emitted headline prefix instead of a bare substring, because the idle count is now subtracted from the total and therefore has to be a strict subset of it; a loose needle could match provider error text that lands on the same first line and drive the difference negative. And the branch that swapped the closing advice when every counted timeout was idle is gone: the breaker now fires only when budget timeouts alone reach the cap, so a window that trips it always holds more budget timeouts than idle ones and the branch was unreachable.
Why it's needed
The cap exists to stop a PR that is too big to finish a round inside the agent's time budget, and its remedy says exactly that — split the PR, or raise the budget and its step backstop together. A silent-sandbox timeout is a different failure with a different owner: the round dies because the container wedged at startup and never emitted a byte, so the PR did not cause it and no budget increase cures it. The breaker's own message already told the reader that, and then counted the round anyway.
Counting it parked healthy PRs. Over the fourteen days to 2026-08-21 the fleet recorded 119 timeouts, of which 58 were idle. Fifty-one windows tripped this cap, every one of them at exactly three. Of the twelve PRs then carrying
autofix/needs-human, nine had been stopped here — and they were not stuck: #8332 was parked at 24 rounds, #8368 at 28, #8276 at 16, all still producing pushed rounds. With idle rounds counted the fleet timeout rate was 8.5% per round, so a window accumulated three of them in roughly 35 rounds by arithmetic alone, whatever the PR was doing. Excluding idle drops that to 4.3%, which needs about 69 rounds — deeper than any window ever observed, the deepest being 22 of 100.What makes the exclusion safe is that a persistently wedged sandbox is still bounded. An idle round pushes nothing and matches none of the consecutive breaker's streak-reset phrases, so a runner that stays broken still terminates the PR at
CONSECUTIVE_FAILURE_CAP. What no longer terminates is idle rounds interleaved with real progress, which is precisely the case where the PR is fine and the runner is not.The reasoning is recorded in the design record as
af-073.Reviewer Test Plan
How to verify
The breaker is exercised by the existing bash-replay harness in
scripts/tests/qwen-autofix-workflow.test.js, which extracts the real block out of the workflow and runs it against synthesised comment histories. Runnpx vitest run scripts/tests/qwen-autofix-workflow.test.jsand confirm the breaker test passes.The behaviours worth reading are: a window of idle timeouts interleaved with pushes never terminates, however many it holds; a window whose budget timeouts alone reach the cap still terminates and reports the budget count rather than the total, while naming the idle rounds as excluded; one idle round is enough to hold open a window that would otherwise have been capped; and a run of consecutive idle rounds still terminates, at the consecutive cap and under its own headline, which is the safety net the exclusion depends on.
To confirm the tests are load-bearing rather than merely green, three mutations were run against them: gating the cap on the total again, deleting the current round's idle increment, and loosening the idle needle back to the bare substring. Each turns the test red; restoring the code returns it to green.
Note that
review verification gate: baseline A/B on deterministic rejection > locks the runner file-command backing files against env plantsfails in this environment, and fails identically onmain— it asserts that a chmod lock blocks a write, which does not hold when the suite runs as root. It is unrelated to this change and touches no file this PR modifies.Evidence (Before & After)
N/A — no user-visible or TUI surface. The behavioural change is in the stop decision, covered by the tests described above.
Tested on
Environment (optional)
Unit tests only, plus
npx prettier --check,npx eslint --max-warnings 0on the changed test file, andbash .github/scripts/check-workflow-size.sh. The workflow file lands at 463,866 bytes, 90% of GitHub's start-runs limit and inside the repository's 470,000-byte gate; the full rationale went into the sibling design record rather than the YAML for that reason.Risk & Scope
@qwen-code /takeoverto resume; this change only stops new windows from being parked the same way. Also out of scope are the other two contributors toneeds-humanthat the same measurement surfaced — automatically re-arming after a transient stop, and retrying a genuine budget exhaustion once at a raised budget before escalating.🤖 AutoFix stoppedprefix, which is unchanged and cross-pinned by the shepherd's own contract test.Linked Issues
None.
中文说明
这个 PR 做了什么
累计超时熔断器现在只统计它真正能处理的那类超时。耗尽 agent 时间预算的轮次照常计入；因为 sandbox 完全没有输出而被 idle 看门狗杀掉的轮次不再计入。idle 轮次仍然作为上下文出现在停机通知里——通知会写明本窗口有多少次、并说明它们不计入上限——同时也会打到 job 日志上，让负责 runner 的人看得见，但它们本身不能再让一个 PR 停摆。
由此带出两处较小的改动。idle 统计的匹配串改成了完整的输出标题前缀，而不再是一个裸子串，因为 idle 计数现在要从总数里减掉，必须是总数的严格子集；宽松的匹配串可能命中落在同一首行的模型报错文本，从而让差值变成负数。另外，"当所有计入的超时都是 idle 时替换结尾建议"的那个分支被删掉了：熔断现在只在预算超时单独达到上限时触发，因此触发时窗口里的预算超时必然多于 idle 超时，那个分支不可达。
为什么需要它
这条上限的存在是为了拦住那些大到无法在 agent 时间预算内跑完一轮的 PR，它给出的补救措施也正是这个意思——拆分 PR，或者同时提高预算与其步骤兜底。而静默 sandbox 超时是另一类失败，责任人也不同：轮次之所以死掉，是因为容器在启动时挂死、一个字节都没吐出来，既不是 PR 造成的，提高预算也治不了。熔断器自己的文案早就这么写了，然后照样把这一轮计了进去。
把它计进去的后果是让健康的 PR 被停摆。截至 2026-08-21 的十四天里，全队记录了 119 次超时，其中 58 次是 idle。有 51 个窗口触发了这条上限，且无一例外都是在第三次触发。当时携带
autofix/needs-human的十二个 PR 中有九个是在这里被停掉的——而它们并没有卡住：#8332 停在第 24 轮，#8368 停在第 28 轮，#8276 停在第 16 轮，当时都还在正常产出推送轮次。把 idle 计入时，全队的单轮超时率是 8.5%，也就是说仅凭算术，一个窗口大约 35 轮就会攒够三次，与这个 PR 在做什么无关。排除 idle 后该比率降到 4.3%，需要约 69 轮——比任何观测到的窗口都深，实测最深的也只有 22/100。这项排除之所以安全，是因为持续挂死的 sandbox 仍然是有界的。idle 轮次不推送任何东西，也不匹配连续失败熔断器的任何一个重置短语，因此一个一直坏着的 runner 仍会在
CONSECUTIVE_FAILURE_CAP处终止这个 PR。不再被终止的，是 idle 轮次与正常推送交替出现的情形——而那恰恰是 PR 没问题、runner 有问题的情形。相关推理已记录在设计档案的
af-073条目中。评审者验证计划
如何验证
熔断逻辑由
scripts/tests/qwen-autofix-workflow.test.js中既有的 bash 回放 harness 覆盖，它会把真实的代码块从 workflow 里抽出来，针对合成的评论历史运行。执行npx vitest run scripts/tests/qwen-autofix-workflow.test.js，确认熔断相关的用例通过。值得一读的行为是：由 idle 超时与推送交替构成的窗口，无论攒了多少次都不会终止；预算超时单独达到上限的窗口仍会终止，且报告的是预算计数而非总数，同时点明 idle 轮次不计入；一次 idle 就足以让一个本该被封顶的窗口继续开着；以及连续的 idle 轮次仍会终止，但走的是连续失败上限、用它自己的标题——这正是本次排除所依赖的安全网。
为了确认这些测试是真正起作用而不只是碰巧变绿，针对它们跑了三个变异：把上限重新改回按总数判断、删掉当前轮的 idle 自增、把 idle 匹配串放宽回裸子串。每一个都会让测试变红；恢复代码后重新变绿。
需要说明的是，
review verification gate: baseline A/B on deterministic rejection > locks the runner file-command backing files against env plants在本环境下失败，且在main上以同样方式失败——它断言 chmod 锁能阻止写入，而当测试套件以 root 运行时这并不成立。它与本次改动无关，也不涉及本 PR 修改的任何文件。证据（前后对比）
N/A —— 没有用户可见或 TUI 层面的表现。行为变化发生在停机判定上，由上文描述的测试覆盖。
已测试平台
运行环境（可选）
仅单元测试，另加对改动测试文件的
npx prettier --check、npx eslint --max-warnings 0，以及bash .github/scripts/check-workflow-size.sh。workflow 文件最终为 463,866 字节，占 GitHub 启动运行上限的 90%，在仓库设定的 470,000 字节门限之内；完整论证之所以放进同名的设计档案而非 YAML，正是出于这个原因。风险与范围
@qwen-code /takeover才能恢复；本次改动只阻止新的窗口以同样的方式被停掉。同样不在范围内的，还有同一次测量暴露出的另外两个needs-human成因——瞬时停机后的自动重新武装，以及真正的预算耗尽在升级给人之前先以更高预算重试一次。🤖 AutoFix stopped前缀，该前缀未变，并由 shepherd 自己的契约测试跨文件钉住。关联 Issue
无。