fix(triage): render the verify report as sanitized markdown, not an escaped pre dump - #8147
Conversation
…scaped pre dump The sandboxed-verification comment embedded report.md inside <details><pre><code> with full HTML escaping. Safe, but unreadable: the report is a curated bilingual document — tables, headings, nested <details> folds — and it displayed as a wall of raw markdown source (#8140's verify comment was the exhibit: literal asterisks, table pipes, and <details> tags shown as text). report.md now renders as markdown through emit_report, which holds the same security floor with four line-independent guarantees: every & < > is escaped and only the structural tags the report legitimately uses (details/summary/pre/code/br) are un-escaped back to live tags, so no other tag can form; the comment-open token is broken (the autofix-proven neutralizer), so no forged qwen-triage:* marker can appear in the raw body the upsert logic greps; @ becomes @, which renders identically but can never fire a mention; and unbalanced <details> opens are counted and closed, so a malformed report cannot swallow the footer. An oversized report falls back to the escaped-pre embedding wholesale (truncated markdown dangles fences and folds), as does any sanitizer failure. The tmux lane's raw-log embedding is untouched — escaped pre remains right for logs. The zero-match grep in the fold balancer carries || true: under the step's pipefail, a report with no folds would otherwise kill the whole composer. Tests: a behavioral replay drives the real emit_report — structure survives (tables, folds, no pre/code), the security floor holds (no live marker/mention/tag, entities escaped, folds balanced), and the oversize fallback produces the escaped shape; the full-render ordering pin follows the new heading.
|
Thanks for the PR! (Re-run on the rewritten sanitizer at Template: the body uses Problem: observed and well-evidenced. #8140's verify comment is a concrete exhibit — literal Direction: aligned. The verify report is a curated bilingual markdown document; embedding it as an escaped Size: not applicable — no core paths touched ( Approach: the scope feels right — one new function ( Risk: no elevated risk signals (no high-risk paths matched). Moving on to code review. 🔍 中文说明感谢贡献!(本次为针对 模板:PR body 使用了 问题:已观测且有实证。#8140 的验证评论就是具体例子——裸露的 方向:对齐。验证报告本身是精排的双语 markdown 文档,用转义 规模:不适用——未触及核心路径(仅 方案:范围合理——一个新函数( 风险:无升级风险信号(未命中高风险路径)。 进入代码审查 🔍 — Qwen Code · qwen3.8-max-preview Reviewed at |
Code reviewIndependent proposal: given the problem (a curated markdown report displayed as raw source inside Comparison with the diff: the PR matches this proposal. The one place it goes further than my baseline is the right call: a flat Security floor — I traced all four guarantees against the current code:
|
|
Confidence: 5/5 — clean across every stage; the security floor holds by construction (fail-closed default), the tests pin every reproduced hole, and CI is green. Would merge without hesitation. Stepping back: this started as "make the verify comment readable" and the review loop turned it into a genuinely hard problem — render markdown without trusting the renderer's prose/code split to match GitHub's. My independent proposal was the escape-all-then-allowlist shape, and the PR matches it; where it goes further (the code-region-aware scanner) is the correct response to CommonMark not decoding entities in code, not complexity for its own sake. The thing that earns the 5/5 rather than a wary 4 is the default direction. A flat scanner will always have some unenumerated divergence from GitHub's container-aware parser, and this design makes every such divergence fail toward the escaped If I had to maintain this in six months, I'd thank the author — the comments document each CommonMark divergence and why its guard exists, and the test names the guarantee each fixture targets. The growth from the earlier The one thing to keep an eye on is the residual the author already named: links and images now render live, so a phishing link under the bot identity is the remaining surface (bounded — it needs a compromised verify-agent output, and images are camo-proxied). The committed follow-up to defuse link targets is the right next step; it doesn't block this. Approving, pinned to the reviewed commit. ✅ 中文说明置信度:5/5 —— 每个阶段都干净;安全底线由构造保证(fail-closed 默认),测试钉住每个复现出的漏洞,CI 全绿。毫不犹豫地合并。 退一步看:这起初只是"让验证评论可读",经 review 循环变成一个真正困难的问题——在不信任渲染器的 prose/code 切分与 GitHub 一致的前提下渲染 markdown。我的独立方案就是"全量转义后白名单回解"形态,PR 与之吻合;它走得更远的地方(代码区域感知扫描器)是对 CommonMark 不在代码中解码实体的正确回应,而非为复杂而复杂。 之所以给 5/5 而非谨慎的 4,关键在于默认方向。扁平扫描器对 GitHub 容器感知解析器总会有某些未枚举的分歧,而本设计让每一个这样的分歧都 fail 向转义 如果六个月后我要维护这段代码,我会感谢作者——注释记录了每个 CommonMark 分歧及其守卫存在之因,测试点名每个 fixture 所针对的保证。从早期 唯一需要留意的是作者已点名的残留面:链接和图片现在会渲染为活的,因此 bot 身份下的钓鱼链接是剩余表面(有界——需要被攻陷的 verify-agent 输出,且图片经 camo 代理)。已承诺的"中和链接目标"后续是正确的下一步;不阻塞本 PR。 审批通过,钉在所审提交。✅ — Qwen Code · qwen3.8-max-preview Reviewed at |
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Not reviewed: build-and-test — Test (macos-latest, Node 22.x) was skipped in CI and its BSD-sed suite did not run locally.
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
未审查:build-and-test — Test (macos-latest, Node 22.x) was skipped in CI and its BSD-sed suite did not run locally。
未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
|
@qwen-code /takeover |
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. Remove the 中文说明🤝 已接管:autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突,直到移除标签或达到轮次上限。移除 |
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 1/100 轮)。改动内容与我反驳保留之处如下: Review feedback addressed[Critical] Post-sanitization size cap (rc:3684460746)Decision: Fixed.
Change: Added a post-sanitization size check immediately after the sed subshell succeeds. If the sanitized file exceeds the budget, it falls back to Test: Added an inflation regression case — a 44,000-byte file of [Suggestion] BRE
|
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline. Not reviewed: build-and-test — Test (macos-latest, Node 22.x) was skipped in CI and its BSD-sed suite did not run locally. Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查。 建议见行内评论。 未审查:build-and-test — Test (macos-latest, Node 22.x) was skipped in CI and its BSD-sed suite did not run locally。 未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
|
🤖 Addressed the latest review feedback (round 2/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 2/100 轮)。改动内容与我反驳保留之处如下: Review feedback addressedBoth inline suggestions from the automated reviewer are implemented. Each named a genuine defect worth fixing in this early round; neither required a maintainer decision. 1. Fold-closer loop can exceed the size cap (rc:3687628086) — fixedThe fold-balancing loop appends Fix: after computing 2.
|
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline. Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查——无阻断问题。 建议见行内评论。 未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
| // tests do not cover it. Execute it: hostile content must stay literal, | ||
| // and the escaped body must land under GitHub's 65,536-char comment cap | ||
| // (the cap is applied AFTER escaping for exactly this reason). | ||
| it('renders report.md as sanitized markdown with an escaped-pre fallback', () => { |
There was a problem hiding this comment.
[Suggestion] The new test covering emit_report() sanitization is unreachable by any CI job. The file lives outside every npm workspace; the per-workspace vitest commands that CI runs never collect it. A root test:scripts script exists (vitest run --config ./scripts/tests/vitest.config.ts) but no workflow in .github/workflows/ invokes it. — Concrete cost: if a future edit to the emit_report() sed pipeline breaks sanitization (e.g. a regex typo re-allows <img> or <!--), no CI gate catches it — the regression reaches production comments silently. — Suggested fix: add a CI job (or step in an existing test job) that runs npm run test:scripts, so this file and its siblings gate PRs.
中文说明
覆盖 emit_report() 净化的新测试无法被任何 CI 作业触达。该文件位于所有 npm 工作区之外;CI 运行的各工作区 vitest 命令不会收集它。根级 test:scripts 脚本存在(vitest run --config ./scripts/tests/vitest.config.ts),但 .github/workflows/ 中无任何工作流调用它。— 具体代价:若未来对 emit_report() sed 管道的编辑破坏了净化(如正则笔误重新允许 <img> 或 <!--),无 CI 闸门捕获——回归将静默进入生产评论。— 建议修复:添加一个 CI 作业(或现有测试作业中的步骤)运行 npm run test:scripts,使此文件及其同级文件成为 PR 闸门。
— qwen3.8-max-preview via Qwen Code /review
There was a problem hiding this comment.
Declined — the premise does not hold. This test is reachable in CI: ci.yml's "Run tests and generate reports" step runs the root npm run test:ci, which chains && npm run test:scripts (package.json), and that runs vitest run --config ./scripts/tests/vitest.config.ts whose include glob covers scripts/tests/**/*.test.{js,ts}. So this file gates every full PR. Verified by running it through that exact config — 108/108 passed. Adding a separate CI job would duplicate an existing gate, so no change was made.
中文说明
拒绝——前提不成立。该测试在 CI 中可达:ci.yml 的 "Run tests and generate reports" 步骤运行根级 npm run test:ci,其后链式调用 && npm run test:scripts(package.json),该脚本运行 vitest run --config ./scripts/tests/vitest.config.ts,其 include glob 覆盖 scripts/tests/**/*.test.{js,ts}。因此本文件对每个 full PR 构成闸门。已通过该配置实际运行验证——108/108 通过。新增独立 CI 作业会重复已有闸门,故不做更改。
…8147) Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
🤖 Addressed the latest review feedback (round 3/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 3/100 轮)。改动内容与我反驳保留之处如下: Autofix review round — PR #8147One commit ( Feedback points1. Added a distinct
On the secondary "(truncated)" title observation: deliberately left unchanged. The sanitize-fail branch falls back because sanitization failed, not because of a size cap — the raw file has already passed the first size gate, so it is not guaranteed to be truncated, and labeling it "(truncated)" would be inaccurate. The stated problem ("at 3 AM the four fallback branches are indistinguishable in the Actions log") is solved by the warning annotations, which is where attribution belongs; the rendered Also added regression assertions in 2. New test unreachable by any CI job (rc:3687883125) — Declined (premise incorrect) The finding states that no workflow runs Verification
中文说明Autofix 审查轮次 — PR #8147一个提交( 反馈点1. 在
关于次要的 "(truncated)" 标题观察:刻意保持不变。净化失败分支回退是因为净化失败,而非大小上限——原始文件已通过第一道大小闸门,因此不保证被截断,标注 "(truncated)" 会不准确。所述问题("凌晨 3 点时四个回退分支在 Actions 日志中无法区分")已由告警注解解决,归因信息正应放在日志中;渲染出的 同时在 2. 新测试无法被任何 CI 作业触达(rc:3687883125)— 拒绝(前提不成立) 该发现称无任何工作流运行 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Suggestions are inline. Not reviewed: build-and-test — Test (macos-latest, Node 22.x) was skipped in CI and its BSD-sed suite did not run locally. Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查。 建议见行内评论。 未审查:build-and-test — Test (macos-latest, Node 22.x) was skipped in CI and its BSD-sed suite did not run locally。 未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
OverviewReplaces the escaped The motivation is right, and several details are well judged: every degradation announces itself in the Actions log rather than silently downgrading; the Two of the four claimed guarantees don't survive contact, though, and one of them is the readability claim the PR exists for. 1. The escaping still produces 乱码 — it just moved into code spans and fencesCommonMark/GFM does not decode entity references inside code spans or code blocks — they are rendered literally. The sanitizer at I drove the PR's real
Rendered HTML — the entities are re-escaped, i.e. they are displayed: <code class="language-bash">npm run build &amp;&amp; node scripts/probe.mjs --pkg &#64;qwen-code/qwen-code-core</code>
<code>Map&lt;string, Array&lt;T&gt;&gt;</code>What the reader sees:
This is not an edge case for this report format specifically. Two fixes, cheap and full. Cheap (stays in sed). sed -E -e 's/&(#[0-9]+|#[xX][0-9a-fA-F]+|[a-zA-Z][a-zA-Z0-9]*);/\&\1;/g' \
-e 's/</\</g' \
-e 's/<(\/{0,1})(details|summary|pre|code|br)>/<\1\2>/g' \
-e 's/<!--/\<!\\-\\-/g'I ran this against a mixed fixture: Full (node). This step already shells out to Whatever the fix, the fixture at 2. Guarantee #4 (fold balancing) is defeated by a
|
…apse it (#8147) Replace the unconditional sed escaping in emit_report with a node sanitizer that tells code regions apart from prose. CommonMark does not decode entities in code spans/fences, so escaping & < > @ there showed && / <T> / @pkg in the commands, types, and paths a report is read to copy. Prose is still escaped (< only; & and > are not security controls and mangling them killed && and blockquotes), code is left inert, the <!-- break stays global so no forged marker survives in the raw body the upsert greps, and folds are balanced over prose only so a fenced </details> can no longer defeat guarantee 4 (surplus closers dropped, unclosed opens closed). Wrap the rendered report in a collapsed <details> so it costs one line again instead of expanding up to 45 KB inline, narrow the tag allowlist to details/summary, bound the whole wrapped section against the size cap, and make every fallback label say "truncated".
|
🤖 Addressed the latest review feedback (round 4/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 4/100 轮)。改动内容与我反驳保留之处如下: Review feedback addressed on PR #8147This round replaces the unconditional Inline suggestions
@wenshao's review
Nits
Also restored, caught in self-audit: the node sanitizer strips NUL bytes Conflict notesNone. Verification
中文说明已在 PR #8147 上处理的评审反馈本轮将 行内建议
@wenshao 的评审
细节
另外在自审中发现并恢复:node 净化器会剥离 NUL 字节( 冲突说明无。 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline. Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查——无阻断问题。 建议见行内评论。 未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
…failure fallback (#8147)
|
🤖 Addressed the latest review feedback (round 5/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 5/100 轮)。改动内容与我反驳保留之处如下: Address Review Summary — PR #8147Feedback points1. [rc:3689048201] Unclosed code fence at EOF swallows the footerDecision: Implemented. Changes: Added Rationale: The fold-closer already handled unclosed 2. [rc:3689048224] Sanitize-failure fallback is only structurally assertedDecision: Implemented. Changes: Added an execution test that overrides Rationale: The other two fallbacks (raw oversize, sanitized-output oversize) were already execution-tested; this branch was the only one without a test driving it. The Verification
中文说明审查反馈处理总结 — PR #8147反馈要点1. [rc:3689048201] EOF 处未闭合的代码围栏吞掉页脚决定: 已实现。 变更: 在 node 净化器的折叠闭合逻辑之前添加了 理由: 折叠闭合符已经处理了 EOF 处未闭合的 2. [rc:3689048224] 净化失败回退分支仅有结构性断言决定: 已实现。 变更: 添加了一个执行级测试,将 理由: 另外两条回退(原始超限、净化后超限)已有执行级测试;此分支是唯一没有被测试驱动到的。 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline. Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查——无阻断问题。 建议见行内评论。 未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
| const cm = line.match(/^ {0,3}(`{3,}|~{3,})[ \t]*$/); | ||
| if (cm && cm[1][0] === fc && cm[1].length >= fl) inFence = false; |
There was a problem hiding this comment.
[Suggestion] The sanitizer's code-region parser is exercised only through backtick fences and single-backtick code spans; its tilde-fence branch and its multi-backtick span run-length matching have no test coverage, so a regression in either branch ships green. — Failure scenario: every fixture uses fences and single-backtick spans, so two mutations survive the suite (verified by probe): deleting `cm[1][0] === fc &&` here (any fence then closes any fence — a `~~~` fence containing a lone line followed by @everyone is misclassified, the mention treated as prose @everyone instead of inert code), and changing m - k === run to === 1 at line 3428 (a two-backtick span wrapping <img src=x> is escaped to <img src=x> instead of being left literal/inert). The sanitizer code itself is correct; this is purely a coverage gap on two parser branches. Fix: add two fixtures to scripts/tests/qwen-triage-workflow.test.js — a ~~~ tilde fence containing a ``` line plus an @mention/`` (assert the payload stays inert code, i.e. a literal `@`/`
` survives in the raw output), and a prose line with a two-backtick span wrapping `
`/`@` (assert it is left literal like the other code spans).
中文说明
净化器的代码区域解析器只通过反引号围栏和单反引号代码跨度被测试;其波浪号围栏分支与多反引号跨度的 run 长度匹配没有测试覆盖,因此这两个分支的回归会绿灯通过。触发场景:所有 fixture 都用 围栏和单反引号跨度,因此两处突变能在测试套件中存活(已用 probe 验证):删除此处的 `cm[1][0] === fc &&`(任意围栏都能闭合任意围栏——一个包含单独 行、后跟 @everyone 的 ~~~ 围栏会被误判,提及被当作散文 @everyone 而非惰性代码);将第 3428 行的 m - k === run 改为 === 1(一个包裹 <img src=x> 的双反引号跨度会被转义成 <img src=x> 而非保持字面/惰性)。净化器代码本身正确;这纯粹是两个解析器分支的覆盖缺口。修复:向 scripts/tests/qwen-triage-workflow.test.js 增加两个 fixture——一个包含 ``` 行加 @提及/`` 的 `~~~` 波浪号围栏(断言载荷保持惰性代码,即原始输出中存活字面 `@`/`
`),以及一行包含包裹 `
`/`@` 的双反引号跨度的散文(断言其像其他代码跨度一样保持字面)。
— qwen3.8-max-preview via Qwen Code /review
Local runtime verification — real bash, real rendererI verified this PR by executing the real workflow bash (not a re-implementation) and rendering its output through GitHub's own renderer, so every claim below is about what a reader actually sees rather than what a string assertion says. Harness. Fixture. The primary fixture is not synthetic: Result: 26 / 32 assertions pass. The fix itself is decisively proven. Three defects survive, one of which I'd fix before merging.
✅ The fix works — on the real report, through GitHub's renderer
The collapsed- The security floor also holds where it is claimed: hostile prose produced no dangerous element (rendered tag set was ❌ Finding 1 — guarantee 3 does not hold (regression; I'd fix before merge)
Entity-escaping does not defuse mentions on GitHub. CommonMark decodes The raw body is exactly what the PR intends ( Measured against the real renderer:
This is a regression relative to Suggested fix: in Scope of what I proved: the mention renders live (anchor, hovercard, profile href) in GitHub's own pipeline. I did not test notification delivery, since that would mean pinging a real person. ❌ Finding 2 — guarantee 4 is still defeatable
The code-region model covers fenced blocks and single-line code spans. It does not cover two other CommonMark code regions, and in both the balancer credits a closer that GitHub renders as inert text — so a genuinely open fold gets no closer, eats the wrapper's
Minimal repros (both drive the real Impact is presentational, not a security hole — the footer and artifacts line vanish from the collapsed comment. Worth noting the asymmetry I measured: surplus (A multi-line code span whose
|
| Shape | type X = Map<string, T> @pkg renders as |
|---|---|
| fenced block (LF) | type X = Map<string, T> @pkg ✅ |
| 4-space indented block | type X = Map<string, T> @pkg ❌ |
| fence indented ≥4 spaces (nested list item) | cat Map<string> @qwen-code/core ❌ |
| fence with CRLF line endings | cat Map<string> @pkg ❌ |
CRLF is the widest of the three: . does not match \r in JS, so the opening-fence regex never matches and the entire fenced block falls back to prose escaping. Low likelihood on a Linux runner, but it is the same 乱码 class the PR set out to eliminate.
Test suite
108/108 pass, as claimed. To check the new suite has teeth I mutated one guarantee at a time in the real workflow YAML and re-ran the new test:
| Mutations caught | 12 / 14 |
| Caught | prose @ neutralizer · <!-- break in code · allowlist un-escape · prose < escaping · closing unclosed folds · dropping surplus closers · dangling-fence close · NUL strip · code-span inertness · sanitized-size gate · sanitize-failure fallback · <details> wrapper |
| Survived (both benign) | the <!-- break in prose (redundant — prose < is already <, so a literal <!-- can never form) and the raw-oversize gate (subsumed by the sanitized-output gate; only the emitted warning differs, and the test greps warning strings statically rather than checking which one fired) |
Recommendation
The core change is sound and the improvement is large and proven — I'd like this merged. Before that, Finding 1 is worth the one-line fix, because it is a security property the PR explicitly claims and a regression against the escaped-<pre> path it replaces. Findings 2 and 3 are narrower and reasonable follow-ups if you'd rather land this now; both are presentational, and Finding 2's failure direction is at least fail-visible rather than silent.
Reviewed at 289df62 vs. base c3620bc. Advisory local verification by the maintainer — not a CI check.
中文版本
本地运行时验证 —— 真实 bash + 真实渲染器
本次验证执行的是工作流的真实 bash(而非复刻实现),并把产物交给 GitHub 自己的渲染器渲染,因此下面每条结论说的都是读者实际看到的东西,而不是字符串断言说了什么。
验证装置。 用 PyYAML 从 .github/workflows/qwen-triage.yml 中逐字提取 jobs.publish-verify.steps["Post verification report comment"].run(head 289df62,PR 基线 c3620bc),把 html_escape / emit_block / emit_report 片段 source 进 bash --noprofile --norc,并施加该 step 真实的 set -euo pipefail;产出的评论正文再经 POST /markdown(mode=gfm,带仓库上下文)渲染——与 GitHub 渲染评论的是同一条管线。随后在解析出的 DOM 上测量组装器页脚所处的 <details> 嵌套深度。
语料。 主语料不是构造的:report.md(12,589 B)由 #8140 验证评论中 <pre><code> 载荷反转义还原而来——正是本 PR 所针对的那份实例。
结论:32 条断言通过 26 条。 修复本身得到决定性证明;有三个缺陷存活,其中一个建议合入前修掉。
| 判定 | 范围 |
|---|---|
| ✅ | #8140 症状确实被修复——在真实报告上得证,且无体积代价 |
| ✅ | 标签白名单、标记伪造打断、各回退分支、围栏与单行代码跨度的折叠配平 |
| ❌ | 保证 3(@ → @「永不触发提及」)不成立 —— 相对被替换路径是回归 |
| ❌ | 保证 4 仍可被击穿 —— 4 空格缩进代码块、跨行代码跨度 |
| 实体乱码(#8140 症状)在缩进代码、深缩进围栏、CRLF 文件中依然出现 |
✅ 修复有效
真实 #8140 报告经 GitHub 渲染器:表格 0 → 2、标题 0 → 6、加粗 0 → 33、残留字面 ** 66 → 0、<pre> 墙 1 → 0,评论体积 12,728 B → 12,661 B(无膨胀),页脚仍位于所有折叠之外(深度 0)。折叠包裹意味着它在会话中仍只占一行。
所有回退分支均已实测,在该 step 真实的 pipefail 下全部 exit 0,且各自打出独立的 ::warning:::原始超限、净化后膨胀超限、折叠补齐开销顶到 2,000 B 上限、净化器崩溃(把 node 打桩成失败)、空报告、报告缺失。
安全底线在其声称之处也成立:敌意 prose 未渲染出任何危险元素(渲染出的标签集合为 a, br, details, em, h1, p, summary——无 img、无 script);无论来自 prose 还是围栏内的引用,原始 body 中都不存在活的 <!--,因此 upsert 的 startswith / contains 匹配无法被伪造。
❌ 发现 1 —— 保证 3 不成立(回归,建议合入前修复)
实体转义在 GitHub 上并不能熄火提及:CommonMark 先把 @ 解码为普通文本 @,GitHub 的 mention 过滤器在此之后才在文档上运行,于是提及复活。
原始 body 完全符合 PR 意图(Ping @everyone and @wenshao now.),测试里的 expect(prose).not.toContain('@everyone') 也通过——但 GitHub 渲染出两个指向真实主页的 <a class="user-mention" data-hovercard-type=…>。
对真实渲染器的实测:@user(基线)、@(本 PR)、@、@、\@(CommonMark 转义)均产生活提及;只有 @​user(零宽空格)与 `@user`(代码祖先)不产生,且显示文本完全一致。
这是相对 main 的回归而非既有缺口:main 上报告位于 <pre><code> 内,而 GitHub 的 mention 过滤器会跳过带 code/pre 祖先的内容(实测 0 提及)。渲染路径移除了该祖先,而替代控制手段是失效的。report.md 是在沙箱中运行 PR 代码的 agent 的不可信输出,因此构造过的 PR 可让 bot 提及任意用户或团队。
建议改法: 在 escProse 中把 .replace(/@/g, "@") 改为插入零宽空格 .replace(/@/g, "@​")——我已验证其显示效果一致且不产生提及。相应地,测试应断言渲染后的结果(至少断言 @​),因为原始字符串断言看不见这一类缺陷。
已证明的范围: 提及在 GitHub 自己的管线中被渲染为活链接(锚点、hovercard、主页 href)。我没有测试通知投递,因为那意味着真的去 ping 一个真人。
❌ 发现 2 —— 保证 4 仍可被击穿
代码区域模型覆盖了围栏块与单行代码跨度,但没有覆盖另外两种 CommonMark 代码区域;在这两种形态下,配平器会把 GitHub 渲染为惰性文本的闭合标签记为有效闭合,于是真正打开的折叠得不到补齐,转而吃掉包裹层的 </details>,把页脚吞进折叠。
</details> 所在位置 |
配平器 | GitHub 渲染为 | 页脚深度 |
|---|---|---|---|
| prose | 计数 | 活闭合 | 0 ✅ |
| 围栏块 | 忽略 | 惰性代码 | 0 ✅ |
| 单行代码跨度 | 忽略 | 惰性代码 | 0 ✅ |
| 4 空格缩进代码块 | 计数 | 惰性代码 | 1 ❌ |
| 跨行代码跨度 | 计数 | 惰性代码 | 1 ❌ |
影响是呈现层面而非安全问题:页脚与产物说明会从折叠后的评论中消失。另外实测到一个有用的不对称性:多余的 </details> 闭合是惰性的(多 1 个或 3 个闭合,页脚仍在深度 0),而未闭合的打开则不是。因此安全的偏向是——只有在确定闭合为活标签时才计数,不确定时宁可多补一个闭合。
(若跨行代码跨度中的 </details> 位于行首第 0 列则无碍:HTML 块会先打断段落,该闭合确实是活的,配平器计数是正确的。出问题的形态是闭合出现在行中间。)
⚠️ 发现 3 —— #8140 症状在三种代码形态中残留
让转义具备代码区域感知的那次提交修好了围栏与单行跨度,但以下三种形态仍走 prose 转义、字面显示实体(CommonMark 不在代码中解码实体,这正是本 PR 的前提):围栏块(LF)✅;4 空格缩进代码块 ❌;缩进 ≥4 空格的围栏(嵌套列表项下)❌;CRLF 行尾的围栏 ❌。
其中 CRLF 影响面最大:JS 中 . 不匹配 \r,导致开围栏正则永不匹配,整个围栏块都退回 prose 转义。在 Linux runner 上概率很低,但属于本 PR 意在消灭的同一类乱码。
测试套件
108/108 通过,与描述一致。为检验新用例是否有"牙齿",我在真实工作流 YAML 上逐条破坏保证并重跑新测试:14 个变异杀死 12 个。
存活的两个均属良性冗余:prose 中的 <!-- 打断(冗余——prose 的 < 已被转义为 <,字面 <!-- 根本无法成形);原始超限门(被净化后超限门覆盖,只是打出的 warning 文案不同,而测试是静态 grep warning 字符串、并不校验实际触发的是哪一个)。
建议
核心改动是扎实的,改善幅度大且已得证,我希望它被合入。合入之前,发现 1 值得那一行修复——它是 PR 明确声称的安全属性,且相对它所替换的转义 <pre> 路径构成回归。发现 2 与 3 影响面更窄,若希望先合入,作为后续跟进也是合理的:两者都属呈现层面,且发现 2 的失效方向至少是可见的而非静默的。
From round 6 of the live sanitizer verification (#8147): the fix closed the fence-shaped entrance into a raw-HTML block, and the code-span entrance beside it — same divergence, adjacent syntax — stayed open. A re-check that tests only the reported input rules 'fixed' over a hole one backtick away. Both fixed-verdict sites in SKILL Step 6 (the open-Criticals re-check and the round ledger) now require enumerating a divergence-class defect's sibling entrances before ruling fixed; a still-open sibling is a NEW finding, never a reason to withhold the original's fixed — the two rulings stay separate so the second hole cannot ship unreviewed.
Review at
|
report.md shape |
user-mention |
live <img> |
live <a href> |
footer fold depth |
|---|---|---|---|---|
| list-nested unclosed fence, then unindented prose | 1 → github.com/everyone |
1 | 1 (anchor text click) |
0 — wrapper closed early instead |
list-nested unclosed fence, then a - next item |
1 | 1 | — | 1 — footer swallowed |
| blockquote-nested fence (control) | 0 | 0 | — | 0 ✅ |
| all-LF top-level fence (control) | 0 | 0 | — | 0 ✅ |
What breaks, precisely. Guarantee 3 ("@ … can never ping") is false — a live user-mention anchor with a hovercard. Guarantee 1 ("no other tag can form; <img> … never render") is false — <img> and a raw <a href> with attacker-chosen anchor text both form; onerror is stripped, but by GitHub's allowlist, not this sanitizer. That anchor is a disguised link injected as raw HTML, which is a step past the link-rendering tradeoff the PR names in its comment block. Guarantee 4 fails in the second row. Guarantee 2 holds.
Reachability: the PR's own threat model is untrusted agent output from a sandbox that ran PR code, so this is authored, not stumbled into. It is also the more likely accidental shape of the two — an unclosed fence under a bullet is an ordinary authoring slip, whereas the CRLF shape needs mixed line endings.
Fix — one line, measured. Do not guess when the two parsers disagree; use the escape hatch this PR already built:
- if (inFence) out.push(fc.repeat(fl));
+ if (inFence) process.exit(3);A fence still open at EOF is a reliable signal that the flat scanner has diverged, so emit_report degrades to emit_block through its existing non-zero-exit branch. Measured:
| measurement | 7175d35d |
with the one-line change |
|---|---|---|
list-nested (both rows above): mentions / live <img> / footer depth |
1 / 1 / 0–1 | 0 / 0 / 0 |
| LF-open + CRLF-close fence (the shape I reported at 16:07) | 1 / 1 / 1 | 0 / 0 / 0 |
real #8140 report.md, and 6 well-formed/hostile fixtures |
— | byte-identical output |
scripts/tests/qwen-triage-workflow.test.js |
108/108 | 107/108 |
The single flip is the PR's own "A report ending inside an open code fence" case, which now takes the fallback instead of being closed at EOF — an intentional behaviour change (degrade rather than guess) that the test should be updated to assert. Worth giving it its own ::warning:: too; today it reuses the sanitize failed label, which would misattribute the cause in the Actions log.
🟡 Re-confirmed still open from round 5 (presentational, deferred)
Both re-measured at this head, both swallow the footer (fold depth 1):
- 4-space indented code block containing
</details>— the balancer credits a closer GitHub renders as inert code. It also silently deletes that line from the report when the surplus branch drops it. - Multi-line code span with a mid-line
</details>— same miscount.
Same root cause as above: indented code blocks and multi-line spans are code regions the scanner does not model.
Nits
- Half-escaped tag text is promoted to a live tag.
escProseun-escapes/<(\/?)(details|summary)>/, which also matches text that was already<details>in the source — a report trying to show<details>as text gets a real fold instead.<details>(fully escaped) is unaffected. Folds stay balanced, so this is content corruption, not a security issue. - Predictable temp paths.
${TMPDIR:-/tmp}/verify-report-$$and-out-$$are guessable;mktempcosts one word and removes the question entirely. - Performance is a non-issue — I probed
proseLine's backtick scan with a 40 k-backtick line and it runs in 0.02 s.
Test coverage
The new suite has real teeth (my earlier mutation run at b0ae340b killed 17/19, both survivors benign). The gap is that no fixture pins the container axis — the suite is green both before and after the fix above. The fixture that would go red is exactly the repro block in this comment: an unclosed fence under a bullet, followed by an unindented line carrying @ and <.
Recommendation
The change is worth landing and the readability improvement is proven on the real artefact. I'd take the one-line fix first: it is the third instance of a single root cause, it closes the previously-reported CRLF shape at the same time, and it costs nothing on well-formed reports. The two round-5 items are reasonable follow-ups — both are presentational and fail visibly.
Verified at 7175d35d. Advisory local verification by the maintainer — not a CI check.
中文版本
在 7175d35d 上的评审 —— 同族新增一个洞,经 GitHub 渲染器实测
我在当前 head 上独立重审,而非拿上一条评论做差。第 7 轮的两行修复已落地且正确,可读性收益属实。本轮发现一个此前未探过的形态,且无需任何特殊行尾即可触发:列表项内的围栏代码块。它同时击穿保证 1、3、4。好消息是一行改动即可关闭它,并同时了结我在 16:07 报告的 CRLF 形态,而在所有良构输入(含真实 report.md)上输出字节一致。
验证装置。 用 PyYAML 从 7175d35d 逐字提取 jobs.publish-verify.steps["Post verification report comment"].run,把 html_escape/emit_block/emit_report 片段 source 进 bash --noprofile --norc,并施加该 step 自身的 set -euo pipefail。所有渲染结论均出自 GitHub 自己的渲染器(POST /markdown,mode=gfm,context=QwenLM/qwen-code)作用于组装器的真实输出;折叠深度在解析后的 DOM 上、以页脚哨兵为基准测量。
概览
emit_report 以「代码区域感知的 node 净化器 + 折叠包裹」替换了 report.md 的转义 <pre> 嵌入。设计是扎实的,四条保证也选得对。第 5→8 轮反复出现的失效模式始终是同一个:净化器是扁平的逐行扫描器,而 GitHub 是具备容器感知的 CommonMark 解析器,两个模型之间的每一处发散都是一个洞。第 6 轮关闭了「HTML 块内的围栏」发散,第 7 轮关闭了「HTML 块内的代码跨度」与「列表内嵌折叠」。下面这个容器发散是我能触达的最后一个。
✅ 本 head 已确认良好
- 第 7 轮的
inHtml ? balance(escProse(line)) : proseLine(line)与^\s*放宽确实名副其实——我在b0ae340b报告的代码跨度形态与列表内嵌形态均已关闭。 - 来自 refactor(core): extract workflow run ownership #8140 的真实
report.md(11,677 B,由已发布评论的<pre><code>载荷还原)往返后表格、标题、加粗、嵌套折叠均完好,无实体乱码,无体积膨胀。 - 保证 2(标记伪造)在我施加的所有形态下均成立,含本轮新形态:
escCode无视上下文一律打断<!--,因此 upsert 所 grep 的原始 body 无法被伪造。 - 引用块内嵌的围栏处理是安全的(见下方对照组),套件在本 head 为 108/108。
🔴 净新 —— 列表项内未显式闭合的围栏
CommonMark 会在其所属容器块结束处闭合未闭合的围栏代码块,而不是在文件末尾。净化器只在 EOF 处闭合(if (inFence) out.push(fc.repeat(fl))),因此一旦列表内嵌的围栏未闭合,inFence 会在文档剩余部分一直为真,后续每一行都走 escCode——而 escCode 基于「存在代码祖先」的前提,刻意不动 <、@、&。彼时 GitHub 早已离开该代码块,把这些行按 prose 解析。
最小复现(驱动真实 emit_report):
- step one:
```bash
npm test
Back at top level: @everyone <img src=x onerror=alert(1)> <a href="https://evil.example/phish">click</a>
在 7175d35d 上端到端实测,经 GitHub 渲染器:
report.md 形态 |
user-mention |
活 <img> |
活 <a href> |
页脚折叠深度 |
|---|---|---|---|---|
| 列表内嵌未闭合围栏 + 顶格 prose | 1 → github.com/everyone |
1 | 1(锚文本 click) |
0 —— 但包裹层被提前闭合 |
列表内嵌未闭合围栏 + - 下一项 |
1 | 1 | — | 1 —— 页脚被吞 |
| 引用块内嵌围栏(对照) | 0 | 0 | — | 0 ✅ |
| 全 LF 顶层围栏(对照) | 0 | 0 | — | 0 ✅ |
具体击穿了什么。 保证 3("@ …… 永不触发提及")为假——渲染出带 hovercard 的活 user-mention 锚点。保证 1("其余标签永远无法成形;<img> …… 永不渲染")为假——<img> 与由攻击者指定锚文本的裸 <a href> 均成形;onerror 确实被剥离,但那是 GitHub 的白名单干的,不是本净化器。该锚点是以裸 HTML 注入的伪装链接,已越过 PR 注释块中所声明的「链接会渲染」这一让步。保证 4 在第二行失效。保证 2 成立。
可达性:PR 自己声明的威胁模型就是「在沙箱中运行过 PR 代码的 agent 的不可信输出」,因此这是可被构造的,而非偶遇。它同时也是两者中更容易被无意写出的形态——项目符号下漏闭合围栏是很普通的书写疏忽,而 CRLF 形态需要混合行尾。
修复 —— 一行,已实测。 两个解析器不一致时不要去猜,直接用本 PR 已经建好的降级出口:
- if (inFence) out.push(fc.repeat(fl));
+ if (inFence) process.exit(3);EOF 时围栏仍打开,是「扁平扫描器已发散」的可靠信号,于是 emit_report 经其既有的非零退出分支降级到 emit_block。实测:
| 测量项 | 7175d35d |
打上一行改动后 |
|---|---|---|
列表内嵌(上表两行):提及 / 活 <img> / 页脚深度 |
1 / 1 / 0–1 | 0 / 0 / 0 |
| LF 开、CRLF 闭的围栏(我 16:07 报告的形态) | 1 / 1 / 1 | 0 / 0 / 0 |
真实 #8140 report.md 及 6 个良构/敌意 fixture |
— | 输出字节一致 |
scripts/tests/qwen-triage-workflow.test.js |
108/108 | 107/108 |
唯一翻转的是 PR 自带的*"报告结束于未闭合围栏"*用例——它现在走降级而非在 EOF 补齐围栏。这是有意的行为变更(宁可降级,不去猜),相应地该用例应改为断言降级形态。另建议为它配一条独立的 ::warning:::目前它复用 sanitize failed 文案,会在 Actions 日志中误报归因。
🟡 第 5 轮报告、当前仍开(呈现层,已延后)
两者均在本 head 重新测量,且均吞掉页脚(折叠深度 1):
- 4 空格缩进代码块中的
</details>——配平器把 GitHub 渲染为惰性代码的闭合记为有效;且走多余闭合分支时会静默删除报告中的该行。 - 跨行代码跨度中位于行中间的
</details>——同样误计。
根因与上文相同:缩进代码块与跨行代码跨度都是扫描器未建模的代码区域。
细节问题
- 半转义的标签文本被提升为活标签。
escProse回解/<(\/?)(details|summary)>/,这同样匹配源文件中本来就是<details>的文本——一份想把<details>作为文本展示的报告,会得到一个真实折叠。<details>(完全转义)不受影响。折叠仍配平,因此这是内容损坏而非安全问题。 - 可预测的临时文件路径。
${TMPDIR:-/tmp}/verify-report-$$与-out-$$可被猜到;改用mktemp只需一个词,可彻底消除该问题。 - 性能不是问题——我用 4 万个反引号的单行探测
proseLine的反引号扫描,耗时 0.02 s。
测试覆盖
新套件是有"牙齿"的(我在 b0ae340b 上的变异测试杀死 17/19,两个存活者均属良性)。缺口在于没有任何 fixture 钉住容器这一维度——上述修复打与不打,套件都是绿的。能让它变红的 fixture 恰是本评论中的复现块:项目符号下未闭合的围栏,其后跟一行带 @ 与 < 的顶格文本。
建议
这个改动值得合入,可读性提升已在真实产物上得证。我建议先取那一行修复:它是同一根因的第三个实例,同时关闭此前已报的 CRLF 形态,且在良构报告上零代价。第 5 轮的两项作为后续跟进是合理的——两者都属呈现层且失效可见。
|
🤖 Addressed the latest review feedback (round 8/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 8/100 轮)。改动内容与我反驳保留之处如下: Autofix review round — PR #8147Addressed the round-8 review from @wenshao. The net-new container-axis hole is closed by degrading to the existing escaped-pre fallback whenever a code fence is still open at EOF, exactly as recommended. Feedback triage🔴 Critical — fence inside a list item, never explicitly closed (fixed). CommonMark closes an unclosed fenced block at the end of its containing block, not at EOF; the flat sanitizer only closed at EOF, so a list-nested unclosed fence left Distinct Test coverage — container axis (added). Rewrote the "report ending inside an open code fence" case to assert the new degrade (escaped Nit — predictable temp paths (fixed). Replaced Nit — half-escaped tag text promoted to a live tag (declined). Round-5 presentational items — indented code block / multi-line code span (deferred). Both re-confirmed by the review as presentational only, failing visibly (footer swallowed), with the same root cause (code regions the flat scanner does not model). Deferred to a follow-up, per the review's own recommendation to land this one-line fix first. Conflict notes
Verification
中文说明Autofix 评审轮次 — PR #8147处理了 @wenshao 的第 8 轮评审。新增的容器维度(container-axis)漏洞已关闭:只要文件末尾仍有未闭合的围栏代码块,就降级到既有的转义 反馈分类🔴 严重 —— 列表项内未显式闭合的围栏(已修复)。 CommonMark 会在未闭合围栏代码块所属容器块结束处闭合它,而非在文件末尾;扁平净化器只在 EOF 处闭合,因此列表内嵌的未闭合围栏会让 独立 测试覆盖 —— 容器维度(已新增)。 重写了「报告结束于未闭合围栏」用例,改为断言新的降级形态(转义 细节 —— 可预测的临时文件路径(已修复)。 将 细节 —— 半转义标签文本被提升为活标签(已拒绝)。 第 5 轮呈现层项 —— 缩进代码块 / 跨行代码跨度(已延后)。 两者均被评审重新确认为仅属呈现层、失效可见(吞掉页脚),根因相同(扁平扫描器未建模的代码区域)。按评审本人「先合入这一行修复」的建议,延后到后续跟进。 冲突说明
验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
Review —
|
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
| out.push(escCode(line)); | ||
| } | ||
| } | ||
| if (inFence) process.exit(3); |
There was a problem hiding this comment.
[Critical] The EOF-open-fence guard (process.exit(3)) does not catch container-axis divergence when a closing fence appears after the divergent content. A fence opened inside a list item (indented), followed by column-0 prose content and then a closing fence, causes the flat scanner to stay inFence and apply escCode (no @ ZWSP, no < escaping) to what CommonMark/GitHub renders as live prose. At EOF inFence is false, so exit(3) does not fire and the unescaped content ships.
Failure scenario: a report containing - step one:\n\n ```bash\n npm test\n\nBack at top level: @everyone <img src=x onerror=alert(1)>\n\n```\nafter — GitHub closes the fence at the container boundary; the flat scanner stays inFence until the closing ``` and applies escCode. The `@everyone` fires a mention from the bot identity; the `` tag renders live in the comment. Probe-verified: the sanitizer outputs `@everyone` without ZWSP and `
` unescaped on this input.
Track the fence opener's indent and treat a non-blank line at a lower indent as a divergence signal (exit non-zero to trigger the escaped fallback):
| if (inFence) process.exit(3); | |
| if (inFence) { | |
| if (/\S/.test(line) && line.match(/^ */)[0].length < fi) process.exit(3); | |
| } |
(where fi is the fence opener's indent, captured at const fi = ... when the fence opens.)
中文说明
EOF 处开放围栏的守卫(process.exit(3))无法捕获容器轴分歧:当闭合围栏出现在分歧内容之后时,在列表项内(缩进)打开的围栏,后跟列 0 的散文内容和闭合围栏,会使平面扫描器保持 inFence 并对 CommonMark/GitHub 渲染为活跃散文的内容应用 escCode(无 @ ZWSP、无 < 转义)。EOF 时 inFence 为 false,因此 exit(3) 不触发,未转义内容被发出。
触发场景:报告包含 - step one:\n\n ```bash\n npm test\n\nBack at top level: @everyone <img src=x onerror=alert(1)>\n\n```\nafter — GitHub 在容器边界关闭围栏;平面扫描器保持 inFence 直到闭合 ```,并应用 escCode。`@everyone` 以 bot 身份触发提及;`` 标签在评论中活跃渲染。已经过探针验证:净化器对此输入输出无 ZWSP 的 `@everyone` 和未转义的 `
`。
修复:跟踪围栏打开者的缩进,将缩进更低的非空行视为分歧信号(以非零退出触发转义回退)。
— qwen3.8-max-preview via Qwen Code /review
|
🤖 Addressed the latest review feedback (round 9/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 9/100 轮)。改动内容与我反驳保留之处如下: Autofix review round — PR #8147Both 🔴 security findings from the maintainer review (and the inline Critical, Feedback points and dispositionsReview — CHANGES_REQUESTED: actionlint embedded-shell source mapping not [rc:3694176709] Critical — container-axis divergence when a closing fence 🔴 #1 — paragraph-spanning code spans bypass guarantees 1 and 3. RESOLVED. 🔴 #2 — EOF guard only catches odd parity; guarantee 4 also broken. 🟡 #3 — tests validate the implementation against a mirror of itself. 🟡 #4 — multi-line code spans reproduce the #8140 symptom. ADDRESSED 🟢 What holds. No action — confirmation that guarantee 2 (marker forgery), Minor — fallback heading hardcoded to "truncated". RESOLVED. The Minor — Minor — link-rendering tradeoff weight. ADDRESSED (documentation). The Minor — PR description is stale relative to the diff. DEFERRED (not Conflict: VerificationCommands actually run this round (the deterministic gate re-runs the same set):
Commit: 中文说明Autofix 审查轮次 — PR #8147维护者审查中的两个 🔴 安全发现(以及内联 Critical,它与 🔴 #2 同根同源)均已修复,方式是让净化器在审查复现的两条解析器分歧轴上fail closed(失败即兜底)。新的回归测试对原始输出断言,而不是对净化器的镜像断言。 反馈点与处置Review — CHANGES_REQUESTED:actionlint 内嵌 shell 源码映射不支持(工具限制,非阻断)。 无需处理。审查者将其标为工具限制并明确非阻断;内嵌的 shell/JS 由 Vitest 测试套件直接执行——它从 workflow 中抽出 [rc:3694176709] Critical — 当闭合围栏出现在分歧内容之后时的容器轴分歧。已解决。 EOF 未闭合围栏的兜底只在平面扫描器恰好停在围栏内时才触发;补一个列 0 的配平围栏标记即可让它到 EOF 时 🔴 #1 — 跨段落代码 span 绕过保证 1 和 3。已解决。 🔴 #2 — EOF 兜底只覆盖奇数奇偶性;保证 4 也被击穿。已解决,通过与 rc:3694176709 相同的缩进兜底。因为该兜底在第一个缩进更浅的行就触发,并把整份报告降级为转义回退,审查中那个吞掉 🟡 #3 — 测试拿实现的镜像验证实现。已处理。 两个新回归测试对原始输出做解析器无关的 grep 断言( 🟡 #4 — 跨行代码 span 重现 #8140 症状。已处理(文档化)。 在注释块中点名为已知的"安全但难看"的取舍;🔴 #1 的修复已将其涵盖(这种 span 现在 fail closed 到正文转义——惰性,实体字面显示,而非活跃)。 🟢 成立的部分。 无需处理——确认保证 2(标记伪造)、多余 次要 — 回退标题硬编码为 "truncated"。已解决。 净化失败/围栏兜底路径进入 次要 — 次要 — 链接渲染 tradeoff 的权重。已处理(文档)。 tradeoff 注释现在写明:残留的钓鱼链接表面只在四条保证成立时才存在,因此解决 #1 和 #2 正是让"推迟链接去活"站得住脚的依据。链接去活本身仍是已点名的刻意后续工作(此处不在范围内;属产品决策)。 次要 — PR 描述与 diff 已不同步。推迟(此模式下不可操作)。 本 workflow 没有 GitHub 凭据或网络访问,无法编辑 PR 描述;合并提交信息由 workflow/维护者拥有。在此标出,以便维护者刷新描述(它仍写着旧的 冲突: 验证本轮实际运行的命令(确定性 gate 会重跑同一组):
提交: Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
Review —
|
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline. Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查——无阻断问题。 建议见行内评论。 未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
| unmatched = false; | ||
| rendered = proseLine(line); | ||
| if (unmatched) { spanUnknown = true; rendered = balance(escProse(line)); } |
There was a problem hiding this comment.
[Suggestion] depth is double-counted when a prose line contains both a <details> tag and an unmatched backtick run, causing a surplus </details> closer at EOF. — Failure scenario: a report line like <details> + unclosed backtick is first processed by proseLine, whose flush() calls balance(escProse(buf)) and increments depth for the <details>. Because the run is unmatched, the line is re-rendered via balance(escProse(line)), counting the same <details> again. At EOF, repeat(depth) appends one extra closer, closing the wrapping fold one line early — trailing content lands outside the collapsed section (visual glitch, no security impact).
| unmatched = false; | |
| rendered = proseLine(line); | |
| if (unmatched) { spanUnknown = true; rendered = balance(escProse(line)); } | |
| const savedDepth = depth; | |
| unmatched = false; | |
| rendered = proseLine(line); | |
| if (unmatched) { depth = savedDepth; spanUnknown = true; rendered = balance(escProse(line)); } |
中文说明
当一行散文同时包含 <details> 标签和未匹配的反引号运行时,depth 会被重复计数,导致 EOF 处多出 </details> 闭合标签。触发场景:报告行如 <details> + 未闭合反引号,先经 proseLine 处理,其 flush() 调用 balance(escProse(buf)) 并为 <details> 递增 depth。由于运行未匹配,该行经 balance(escProse(line)) 重新渲染,再次计数同一 <details>。EOF 处 repeat(depth) 多追加一个闭合标签,使包装折叠提前一行关闭——尾部内容落在折叠区外(视觉问题,无安全影响)。
— qwen3.8-max-preview via Qwen Code /review
| if [ "$node_status" -eq 3 ]; then | ||
| echo "::warning::emit_report fell back to escaped embedding (report ended inside an open code fence) for $file" >&2 |
There was a problem hiding this comment.
[Suggestion] Exit code 3 conflates two distinct sanitizer bailouts (EOF-open-fence and dedent-below-fence-indent) under a warning that describes only the EOF case. — Failure scenario: a list-nested fence whose content dedents below the opener indent triggers the dedent guard, but the log says "report ended inside an open code fence" — an oncall engineer would look for an unclosed fence at EOF, not find one, and waste time or dismiss the warning.
Use a distinct exit code (e.g. process.exit(4)) for the dedent guard with its own warning message: "emit_report fell back to escaped embedding (fence/container divergence: content dedented below fence indent)".
中文说明
退出码 3 将两种不同的净化器退出(EOF 处开放围栏和缩进低于围栏打开者)合并为一条仅描述 EOF 情况的警告。触发场景:列表嵌套围栏的内容缩进低于打开者时触发缩进守卫,但日志显示"报告在开放代码围栏内结束"——值班工程师会寻找 EOF 处未闭合的围栏,找不到后浪费时间或忽略该警告。建议对缩进守卫使用独立退出码(如 process.exit(4))并配以专属警告消息。
— qwen3.8-max-preview via Qwen Code /review
|
🤖 Addressed the latest review feedback (round 10/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 10/100 轮)。改动内容与我反驳保留之处如下: SummaryAddressed the two security findings from the review — the 🔴 Critical and the 🟡 Medium — with minimal root-cause fixes in the Findings🔴 Critical — backslash-escaped opening backtick classified prose as code — FIXED
Fix: before treating a backtick run as a delimiter, count the immediately preceding backslashes; an odd count sets Regression test added using the reviewer's exact fixture (an escaped backtick, then 🟡 Medium —
|
|
📊 Takeover milestone — round 10/100, in the current window. Census: 10 pushed fix(es), 3 no-change review(s), 0 timeout(s), 0 rejected attempt(s), 0 other round(s) (crash / model error / gate error / infra), 0 base update(s). This many rounds deserves a human look. Options: keep going (fine — nothing changes), split or reduce the PR if rounds keep accumulating, or release takeover (remove the 中文说明📊 接管里程碑 —— 第 10/100 轮(当前窗口)。统计:推送修复 10 次、审阅无需改动 3 次、超时 0 次、验证拒绝 0 次、其他轮次(崩溃/模型错误/门错误/infra)0 次、base 更新 0 次。 轮次到这个量值得人工看一眼。可选:继续(无需操作);若轮次持续累积,考虑拆分或缩减 PR;或释放接管(移除 |
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下: Autofix Review — No Action Required (PR #8147)This round has no actionable feedback:
Critical-only mode is active after 10 change-producing rounds. The deferred non-Critical items (two inline suggestions from the automated reviewer on No code changes were made. 中文说明Autofix 审查 — 无需操作(PR #8147)本轮没有可执行的反馈:
经过 10 个产生改动的轮次后,已进入仅处理 Critical 的模式。被延后的非 Critical 条目(自动审查器在 未做任何代码改动。 Deferred non-Critical feedbackCritical-only mode is active after 10 change-producing rounds. The workflow excluded the non-Critical feedback below from this round's actionable sections; the items remain open for human follow-up. Maintainer feedback is deferred only after its author has used 2 regular feedback batches in this window's Critical-only tail; authors at that budget, if any, are named below. (
中文说明完成 10 个产生改动的轮次后进入仅处理 Critical 的模式。本轮可执行区域已排除下方非 Critical 反馈;这些条目保持开放,留待人工跟进。维护者反馈仅在其本人于本窗口 Critical-only 阶段已使用 2 批常规反馈预算后才会延后;达到预算的作者(如有)在下方点名。(评论 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed. Not reviewed: build-and-test — Test (macos-latest, Node 22.x) and Test (windows-latest, Node 22.x) were skipped in CI; the BSD-sed/Windows fallback path (emit_block sed -E un-escape) did not run locally, which used Linux GNU sed only. Not linted (tool limitation, not a blocker): the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted.
中文说明
已审查。 未审查:build-and-test — Test (macos-latest, Node 22.x) and Test (windows-latest, Node 22.x) were skipped in CI; the BSD-sed/Windows fallback path (emit_block sed -E un-escape) did not run locally, which used Linux GNU sed only。 未检查(工具限制,非阻断):the executable-script lint — .github/workflows/qwen-triage.yml: actionlint embedded-shell source mapping is not yet supported — not linted。
— qwen3.8-max-preview via Qwen Code /review
|
@qwen-code /triage |
|
Sandboxed verification: The verification job did not complete (checkout, runner, or setup error) and produced no report. See the workflow run for details. 中文 — 判定:
|
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下: No action takenThis round has no actionable feedback:
The only feedback present is in the Deferred non-Critical feedback section (automated reviewer suggestions deferred under Critical-only mode after 10 change-producing rounds). Per the review policy, that section is an audit record for human follow-up, not work for this round. No code changes, thread resolutions, or comment replies were made. 中文说明未采取任何操作本轮没有可执行的反馈:
唯一存在的反馈位于延后的非 Critical 反馈区域(在 10 个产生改动的轮次后,仅 Critical 模式下被延后的自动审查建议)。根据评审策略,该区域是供人工跟进的审计记录,不属于本轮的工作范围。未进行任何代码修改、主题解决或评论回复。 Deferred non-Critical feedbackCritical-only mode is active after 10 change-producing rounds. The workflow excluded the non-Critical feedback below from this round's actionable sections; the items remain open for human follow-up. Maintainer feedback is deferred only after its author has used 2 regular feedback batches in this window's Critical-only tail; authors at that budget, if any, are named below. (
中文说明完成 10 个产生改动的轮次后进入仅处理 Critical 的模式。本轮可执行区域已排除下方非 Critical 反馈;这些条目保持开放,留待人工跟进。维护者反馈仅在其本人于本窗口 Critical-only 阶段已使用 2 批常规反馈预算后才会延后;达到预算的作者(如有)在下方点名。(评论 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
Local verification of
|
| Gate | Result |
|---|---|
| PR's own suite (merge worktree) | 117 / 117 pass |
qwen-triage.yml YAML parse |
OK (7 jobs) |
| ESLint on the changed test file | clean |
| Base-arm fidelity vs posted #8140 comment | byte-identical |
| Readability goal | achieved — see below |
| Security floor, 31 hostile bodies through GitHub's renderer | holds on every axis but one |
| Regression vs base | 1 found — entity-encoded @ fires a live mention |
1. The readability goal is met
Same real #8140 report, both arms, rendered by GitHub:
BASE (emit_block) |
HEAD (emit_report) |
|
|---|---|---|
<table> |
0 | 2 |
<code> spans |
1 | 123 |
<h2> headings |
0 | 5 |
<strong> |
3 | 36 |
<pre> |
1 (the entire report) | 0 |
2. The security floor holds
30 adversarial fixtures plus one combined hostile report, each run through the real step and rendered by GitHub. Across all 31 bodies:
- 0 live
<img>/<script>/<iframe>from injected raw HTML; - 0 live event-handler attributes —
<details open ontoggle="…">is not promoted, because the allowlist matches the exact tag only;<DETAILS>,<details/>and<summary />likewise stay escaped; - exactly 2
qwen-triagemarkers in the raw body — the composer's own. Forgery fails in prose, in code spans, in fences, and in entity form (<!-- … -->); - fold balance at the footer is 0 and the footer survives in every case — surplus
</details>are dropped, unclosed opens are closed, and a</details>quoted in code is correctly ignored; - feeding the sanitizer's own stash sentinels (raw
U+0000/U+0001) back in produces no tag — the input strip is complete; - entity-encoded markup is inert:
<img src=x onerror=1>renders as literal text, and[x](javascript:…)does not become a link.
Fallbacks fire exactly where they should — fence still open at EOF, a non-blank line dedenting below the fence opener, an oversized raw report, and a report that only exceeds the cap after sanitizing. Worst case measured body is 46,139 bytes, comfortably inside GitHub's 65,536 limit.
One behaviour worth recording because it is not obvious from the code: a CRLF report opens no fence at all — JavaScript's . excludes \r, so the opener regex never matches — and the whole file is prose-escaped end to end. That is the safe direction; the cost is that a CRLF report shows entities literally inside code.
3. One finding: an entity-encoded @ in prose still fires a real mention
Guarantee 3 defuses a literal @ with a ZWSP, which works. But escProse deliberately leaves & alone, so an @ that arrives already entity-encoded passes through untouched — and GitHub decodes character references before the mention filter runs. All four spellings fire:
| source in report prose | HEAD renders |
|---|---|
@name |
live user-mention anchor |
@name |
live user-mention anchor |
@name |
live user-mention anchor |
@name |
live user-mention anchor |
This is not just an anchor — it reaches the notification path. Two fresh issues, one comment each, self-mention only:
- base shape (
&escaped) → timeline[commented] - head shape (
@name) → timeline[commented, mentioned, subscribed]
Base was immune because html_escape escapes & unconditionally, so this is a regression against main on a guarantee the PR itself claims. Reachability is the ordinary threat model already stated in the code comment: report.md is agent output from a sandbox that ran PR code, and a report about entity handling — a verification report for this very PR, say — is exactly where an entity-encoded @ shows up in prose.
Severity is proportionate, not dramatic: no code execution, and it needs the report to contain an entity-@ immediately followed by a real login. It sits alongside the phishing-link surface the PR already names as accepted. But it is one line to close, so I would rather not ship it.
Validated fix — re-escape the & of the entity forms before the ZWSP pass, inside escProse:
.replace(/<!--/g, "<!\\-\\-")
.replace(/&(#0*(?:64|[xX]0*40);|commat;)/g, "&$1") // <-- add
.replace(/@/g, "@​");Measured with the patch applied: live mentions 4 → 0 (matching base), 117 / 117 tests still pass, YAML still parses, all 30 / 30 adversarial fixtures still clean, and the composed body on the real #8140 report is byte-identical to the unpatched head — so it costs nothing in readability.
4. The description has drifted from the code
The repo squash-merges, so the PR body becomes the commit message. Three items are stale as of cea325f7:
- guarantee 3 still reads "
@becomes@: renders identically, can never ping" — the code uses a ZWSP now, and@is measurably not inert (this is the finding above); - "108/108 pass" — the suite is at 117;
- the
grep/|| truenote in Tests describes the old shell fold-balancer; balancing happens in the node sanitizer now.
Recommendation
The change does what it sets out to do, and the hardening across eleven rounds holds up under a real renderer. My call: apply the one-liner and refresh the description, then merge.
Not covered
Replay is offline — I did not run a real Actions job; the evidence-image hosting path is untouched by this PR and was stubbed; report generation by the verify agent itself is out of scope. Fixtures are mine, not sampled from production reports.
中文说明
对 cea325f7 的本地验证 —— 真实步骤 + GitHub 真实渲染器
我在本地构建了这个 PR,并把工作流的真实步骤离线跑起来,用 GitHub 自己的渲染器做判据,而不是本地的 markdown 近似实现。以下证据供合并参考。
测试装置。 从两个分支的 .github/workflows/qwen-triage.yml 中抽出 Post verification report comment 步骤 —— base 2bbd82f1 与 refs/pull/8147/merge 的合并提交(head cea325f7)—— 去掉 run: 块缩进后,配合打桩的 gh、构造的 verify-results/ 目录树和真实的 report.md 直接执行。生成的 $BODY_FILE 再发到一个临时仓库并以 body_html 读回,这条链路就是真实的评论管线,包含 GitHub 的净化器。
保真锚点(后续结论可信的前提)。 我从 #8140 的验证评论(也就是本 PR 命名所指的那个实例)中还原出真实的 report.md 喂进去:base 臂逐字节复现了当时实际发布的评论 —— 12,616 / 12,616 字符完全一致,唯一差异是我的回放没有证据图片因而缺少图片区块。这说明装置跑的是真实组装器,而非它的模型。
结论
| 门禁 | 结果 |
|---|---|
| PR 自带测试套件(merge worktree) | 117 / 117 通过 |
qwen-triage.yml YAML 解析 |
正常(7 个 job) |
| 改动测试文件的 ESLint | 干净 |
| base 臂与已发布 #8140 评论的保真度 | 逐字节一致 |
| 可读性目标 | 达成 —— 见下 |
| 31 个敌意 body 经 GitHub 渲染器的安全底线 | 除一条外全部成立 |
| 相对 base 的回归 | 发现 1 处 —— 实体编码的 @ 会触发真实提及 |
1. 可读性目标已达成
同一份真实的 #8140 报告,两臂对照,由 GitHub 渲染:
BASE(emit_block) |
HEAD(emit_report) |
|
|---|---|---|
<table> |
0 | 2 |
<code> 代码片段 |
1 | 123 |
<h2> 标题 |
0 | 5 |
<strong> |
3 | 36 |
<pre> |
1(整份报告) | 0 |
2. 安全底线成立
30 个敌意 fixture 外加一份综合敌意报告,每个都经真实步骤组装并由 GitHub 渲染。在全部 31 个 body 上:
- 注入的原始 HTML 产生 0 个活的
<img>/<script>/<iframe>; - 0 个活的事件处理属性 ——
<details open ontoggle="…">不会被回解,因为白名单只匹配精确标签;<DETAILS>、<details/>、<summary />同样保持转义; - 原始 body 中恰好 2 个
qwen-triage标记,即组装器自身的那两个。伪造在正文、代码片段、围栏内以及实体形态(<!-- … -->)下全部失败; - 页脚处折叠配平为 0,且页脚在每个用例中都存活 —— 多余的
</details>被丢弃,未闭合的开标签被补齐,代码中引用的</details>被正确忽略; - 把净化器自己的暂存哨兵(原始
U+0000/U+0001)回灌进去也构造不出标签,说明入口处的剥离是完整的; - 实体编码的标记语言是惰性的:
<img src=x onerror=1>渲染为纯文本,[x](javascript:…)也不会变成链接。
回退分支的触发位置准确 —— EOF 时围栏仍开启、非空行缩进退到围栏开启者之下、原始报告超限、以及净化之后才超限的报告。实测最大 body 为 46,139 字节,远在 GitHub 的 65,536 限制之内。
有一个从代码上不易看出、值得记录的行为:CRLF 报告根本不会开启围栏 —— JavaScript 的 . 不匹配 \r,开启正则永远匹配不上 —— 于是整份文件从头到尾走正文转义。这是安全的方向;代价是 CRLF 报告会在代码中原样显示实体。
3. 一处发现:正文中实体编码的 @ 仍会触发真实提及
保证 3 用 ZWSP 熄火字面量 @,这一点是有效的。但 escProse 有意不动 &,因此本来就以实体编码到达的 @ 会原样穿过 —— 而 GitHub 在提及过滤器运行之前就解码了字符引用。四种写法全部触发:
| 报告正文中的源写法 | HEAD 的渲染结果 |
|---|---|
@name |
活的 user-mention 锚点 |
@name |
活的 user-mention 锚点 |
@name |
活的 user-mention 锚点 |
@name |
活的 user-mention 锚点 |
而且这不只是一个锚点,它会走到通知链路。两个全新 issue、各一条评论、仅自我提及:
- base 形态(
&已转义)→ timeline[commented] - head 形态(
@name)→ timeline[commented, mentioned, subscribed]
base 因为 html_escape 无条件转义 & 而免疫,所以这是相对 main 的回归,且恰好落在 PR 自己声称的那条保证上。可达性就是代码注释里已经写明的常规威胁模型:report.md 是在运行过 PR 代码的沙箱中产生的 agent 输出,而一份讨论实体处理的报告(比如针对本 PR 自身的验证报告)正是实体编码 @ 出现在正文里的典型场合。
严重度需要按比例看待,不必夸大:没有代码执行,且需要报告中的实体 @ 紧跟一个真实登录名。它与 PR 已明确接受的钓鱼链接面属于同一量级。但它只需一行即可关闭,所以我倾向于不要就这样合入。
已验证的修复 —— 在 escProse 中,于 ZWSP 处理之前把实体形态的 & 重新转义:
.replace(/<!--/g, "<!\\-\\-")
.replace(/&(#0*(?:64|[xX]0*40);|commat;)/g, "&$1") // <-- 新增
.replace(/@/g, "@​");打上补丁后的实测:活的提及 4 → 0(与 base 一致),117 / 117 测试仍通过,YAML 仍可解析,全部 30 / 30 敌意 fixture 仍然干净,且在真实 #8140 报告上组装出的 body 与未打补丁的 head 逐字节一致 —— 也就是说它不带来任何可读性成本。
4. 描述已与代码脱节
本仓库使用 squash 合并,PR 正文会成为提交信息。截至 cea325f7 有三处已经过时:
- 保证 3 仍写作 "
@becomes@: renders identically, can never ping" —— 代码现在用的是 ZWSP,而@经实测并非惰性(即上面那处发现); - "108/108 pass" —— 套件现在是 117;
- Tests 一节中关于
grep/|| true的说明描述的是旧的 shell 折叠配平器,配平现在发生在 node 净化器里。
建议
这个改动达成了它设定的目标,十一轮的加固在真实渲染器下也站得住。我的判断:打上那一行补丁并刷新描述,然后合入。
未覆盖范围
回放是离线的 —— 我没有跑真实的 Actions 作业;证据图片托管路径不受本 PR 影响且被打桩;验证 agent 自身的报告生成不在范围内。fixture 由我构造,并非从生产报告中采样。
Verified locally with Claude Code (model: Claude Opus 5, 1M context).
yiliang114
left a comment
There was a problem hiding this comment.
LGTM. Security model is solid: escape-all + allowlist un-escape, global marker breaking, ZWSP mention neutralization, prose-only fold balancing. Fail-closed fallbacks for oversized/fence-unclosed/sanitizer-error. Thorough test coverage. ✅
…ted fixes (QwenLM#8242) * feat(verify): sweep sibling shapes, calibrate replays, measure suggested fixes The verify-pr skill produced a `merge-ready` report on a head that leaks live HTML. Measured on QwenLM#8147 at b0ae340 with two control arms and one treatment arm (real qwen3.8-max-preview, isolated QWEN_HOME each, arms differing only by this file): both control arms concluded `merge-ready`, the treatment arm concluded `findings` and shipped a measured two-line fix. The decisive arm is the control replicate. It spent the same budget as the treatment arm and built a stronger instrument — a micromark+GFM rendering oracle, 23 hostile fixtures, 254 assertions. It built the HTML-block-times-fence fixture and separately probed code spans, and never crossed the two axes. Re-running that arm's own oracle on the fixture it never built renders a live <img> and a live <a href>, so it would have failed its own guarantee-1 assertion. The miss was which fixtures the agent thought to construct, which is what these rules supply. Added to the A/B method: sweep the sibling shapes a parser-class fix does not pin; treat untrusted text reaching a parser as a scaling question and probe it with a timed ladder; audit a stated cost against the repo's own accounting of the same mechanism; enumerate the unnamed siblings of an accepted-tradeoff list. Added to shared-state probing: an instruction in a prompt is not an invariant, and interleavings that fabricate a result outrank ones that crash. Added to vacuity: run the mutation in reverse (a suite green with and without a candidate fix pins nothing along that axis), and read a test's name as a claim about its fixture. Workflow replays now calibrate against a real production artifact before their cells are believed, and suggested fixes ship with collateral measured. The follow-up status enum gains `worsened`, because a fix can move an accepted tradeoff. Rules whose effect was not measured are marked as such in the PR description rather than presented as validated. * feat(verify): read reports as coverage specs, prove negatives by census Six more rules, each taken from a hands-on verification round that found something the skill could not have produced. From QwenLM#8037, a fallback parser for XML tool calls: a bug report enumerates its own cases, and those names are coverage claims the fix inherits — holding the issue's reported 1,898-character preamble fixed and varying only the tool showed the guard declining run_shell_command, which the issue names explicitly, so the fix covered half of what it was filed against. The same round found one defect with a loud variant (schema rejects a dropped argument) and a silent one (a truncated file), which fixes the ordering: rank variants by observability, not blast radius — the same ordering the concurrency rule already uses. And its harness replayed the reported wire bytes without reproducing the model-side degradation that produces them, so Not covered now asks which of the two a reproduction actually has. From QwenLM#8005, a Goal runtime driven through a real TUI: walking the PR's own Reviewer Test Plan step by step turned an unrunnable step into the round's sharpest finding, because no code path could insert real user input into an active turn and the feature's completion criterion needed exactly that. Proving it took a census rather than a reading — 30 captured verifier payloads carried one evidence kind and never the required one. And its mutation matrix is the reason survivors are now gated on a positive control: two mutations survived 429/429 and 326/326, believable only because a third, expected-caught mutation turned one test red. None of these six were measured with an A/B on this file; they are drawn from rounds that found real defects. The PR description says so. * fix(verify-pr): address review — calibration fallback, timeout cap, pinning, section placement (QwenLM#8242) * fix(scripts): pin bare words in skill assertions, not markdown emphasis markers (QwenLM#8242) * fix(verify-pr): address round-3 review — pin payload words, bind timeout, relocate misplaced bullets (QwenLM#8242) --------- Co-authored-by: verify <verify@local> Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> Co-authored-by: Qwen Code Bot <qwen-code-bot@users.noreply.github.com>
…rification lenses (QwenLM#8225) * feat(review): borrowed-verification trio — test-plan check, base-tree A/B, per-hunk probes - qwen review test-plan: rule on the PR Test Plan's checkable claims (paths, npm scripts, test counts) against the reviewed tree; contradictions and differing counts are disclosed via compose-review, never capping. - qwen review base-tree: build the merge base in a sibling worktree so the verifier can A/B a comparative claim instead of reading it; swept by cleanup. - test-efficacy: third probe kind — reverse-apply one hunk at a time and re-run the affected tests, attributing a still-green suite to the specific change nothing gates; shares the mutants' budget window, runs last. * fix(review): survive real runner output — ANSI-laced and trimmed-away summaries Both measured on a live /review of QwenLM#8176 with the built CLI: - test-plan's observedTestCounts strips SGR sequences before matching; a color-enabled pipe interleaves them BETWEEN tokens, and the count claim fell to 'unchecked' with the summary right there in the report. - build-test's trimOutput rescues runner summary lines from the omitted middle (like module-resolution errors): a failing suite's tail is all failure details and npm epilogue, which pushed the one-line summary out of the kept text entirely. * fix(review): address the eight findings from live review of this branch All measured in the review (QwenLM#8215 review comment): - test-plan: linear-time bold-heading scan (the old pattern backtracked catastrophically on an unclosed ** line an untrusted PR body controls); a flag preceding the npm script yields no claim instead of a false 'no package defines this script'. - test-efficacy: a hunk probe's restore recreates the parent directory a reverse-applied 'new file' hunk removed (the ENOENT from finally lost the verdict and marked every remaining hunk inconclusive); hunks get their own skippedForBaseline instead of mislabeling a red baseline as a budget skip; splitDiffIntoHunks re-captures the file header at every diff --git boundary; a hunk-survived finding notes when it restates an inert file-level revert at hunk granularity. - base-tree: idempotent fast path keyed on a build marker + HEAD check — concurrent verifier shards reuse one built tree instead of sweeping it out from under each other mid-A/B (a fabricated base-side difference with a deterministic source tag was the worst case); cost wording is now 'an install and a build' everywhere it was 'one extra build'. * fix(cli): never score a hunk survived when its own test left the baseline (QwenLM#8215) A per-hunk probe reported `survived` whenever the green baseline probes still passed with the hunk reverted. When the hunk's own collocated test dropped out of the baseline (a probe-tree import error collects nothing), the remaining green probes prove only that THEY do not cover the hunk, so the verdict is now `inconclusive` — the same dropped-test asymmetry the mutants already hold. Also scope the hunk-survived cross-reference note to the hunk's own collocated test, and let test-plan match a workspace-scoped run of the plan's bare command instead of falling through to the manifest on an exact-string miss. * fix(review): silence-bias hardening from four live review rounds of this branch The two blocking findings, reproduced on this PR's own Test Plan: - test-plan files no false contradicted notes: npm rulings move from a four-verb denylist to an allowlist (the run form + npm's script aliases — the ~fifty other builtins each used to become 'no package defines this script'); a slash token is claimed as a repo path only with evidence (an extension or ./ prefix), never when it is a flag's value (--repo owner/repo) or under the review's own temp root; HEADING_LINE_RE drops the same quadratic shape its bold sibling was rewritten to remove. - base-tree gets a real mutual-exclusion lock around sweep+add+build (mkdirSync test-and-set; the loser returns busy instead of deleting the tree the winner is mid-install in), and a failed build writes a settled marker so later shards stop re-paying the install to relearn 'unavailable'. Also: Agent 7's brief now names hunk-survived and the hunks.* counters (it is the report's only consumer, and the finding class was invisible); hunk findings anchor at the first ADDED line instead of up to three context lines above the change. * feat(review): four round-2 borrowings — measured failure attribution, round ledger, richer mutants, doc parity Shaped by a live round-2 maintainer re-verification (QwenLM#7818): - qwen review test-delta: rerun the PR side's failed test commands on the built merge base and diff the failing FILE SETS — netNew is the PR's own failure by measurement (whatever files the diff touches), shared is pre-existing by measurement. Counts are never compared: a flaky suite fails different test names between runs of the same tree. An unfinished base rerun attributes nothing. - Round ledger: the incremental review cache persists confirmed findings under round-scoped ids (R1-2); a high-effort re-review rules on each (fixed / still stands / cannot tell) and opens its report with the table, the way a human round-2 comment opens with 'M1 is fixed'. - Three replacement mutation operators in test-efficacy: drop '?? fallback', force a comparison-bearing guard condition to 'true', drop a '+ CONST' term. Each survivor maps to one crisp untested-protection sentence. A line whose raw text and literal-blanked code view disagree yields no candidate — an edit index computed across the two views once spliced 'iftrue 0)' into a guard, and a mangled mutant reads as inconclusive while quietly spending a cap slot. - Quality brief: documentation-parity lens — a new user-facing surface whose siblings are documented is a Suggestion naming the sibling precedent; no documented sibling, no finding. * feat(review): render adjudication, workflow step extraction, three verification lenses Shaped by a live sanitizer-PR verification (QwenLM#8147): - Render-adjudication capability (opt-in): with QWEN_REVIEW_SCRATCH_REPO set, the verifier may post a minimal payload to that user-designated repo and rule on GitHub's own rendered HTML — the measured case being an @ -> @ defusal every local reading called sound while GitHub's real renderer registered the mention and fired the notification. Absent the setting, rendering claims honestly cap at low confidence / cannot tell. Step 7's write ban names the carve-out explicitly. - qwen review extract-step: lift one workflow step's run: script verbatim into an executable, with env (as comments, never half-substituted exports), every ${{ }} site listed unevaluated, and a heuristic invoked- command list as the stubbing starting point. With base-tree, both arms of a by-hand workflow A/B become two invocations. yaml declared as a cli dependency (previously resolved only via hoisting). - Three brief lenses: a borrowed protection idiom missing what made it work at home (the code ancestor did the protecting; only the entity was copied); a second parser for an authoritatively-parsed format is a divergence hunt; tests that pin the mechanism instead of the effect, and oracles that mirror the implementation's own model. * feat(review): sibling-entrance discipline for the fixed verdict From round 6 of the live sanitizer verification (QwenLM#8147): the fix closed the fence-shaped entrance into a raw-HTML block, and the code-span entrance beside it — same divergence, adjacent syntax — stayed open. A re-check that tests only the reported input rules 'fixed' over a hole one backtick away. Both fixed-verdict sites in SKILL Step 6 (the open-Criticals re-check and the round ledger) now require enumerating a divergence-class defect's sibling entrances before ruling fixed; a still-open sibling is a NEW finding, never a reason to withhold the original's fixed — the two rulings stay separate so the second hole cannot ship unreviewed. * feat(review): three measured-verification lenses from live rounds 8037/8005 - Threshold-boundary scan: when a fix is a ratio/length guard, hold the issue's own variables fixed and binary-search the boundary where the behaviour flips; put the number next to what the issue reports. Live case: a prose-ratio guard covered the edit/write_file half of its issue and silently declined the run_shell_command half (~473-char boundary). - Delimiter self-injection named as the first parser-differential probe: a no-escaping extractor fed its own close tag truncates silently. - Shared-gate state enumeration: a deliberate-design defence extends only to the states it argues — an input-hold argued for 'active' silently froze three idle states sharing the same gate. The sibling-entrance rule, applied to a state machine instead of a syntax. * fix(review): address review feedback — false-positive hardening, binary diff guard, error convention (QwenLM#8215) * fix(review): address review feedback — base-tree availability gate, test-plan false positives, hunk-probe ranges (QwenLM#8215) - base-tree: only stamp a base tree available when runBuildTest actually compiled something (ok AND npm toolchain AND a non-empty build). An `unsupported` handoff or an empty npm scope returns ok:true having built nothing; marking that tree available let an A/B read the absence of a build as a behavioural difference. - cleanup: sweep the stale base-tree build lock a killed builder leaves behind. - test-plan: read the root manifest's scripts directly so a root-only script survives when the root defines no build/test; bail on the inline --root=./dir rebasing form; stop treating a positional after an inline --flag=value as the flag's value; prefer a failed scoped run when ruling a bare command; anchor the npm script alias to a full token so `yarn test:unit` is not truncated. - test-efficacy: exclude `\ No newline at end of file` from the startLine offset count; compute the mutant-overlap range from the header's new-side span so it no longer overshoots into a closely following hunk. * fix(review): address review feedback — diff-header false positives, stale prompt enumeration, added-file hunk probes (QwenLM#8215) * fix(review): address review feedback — cd-base exclusion, Test Files count guard, base-tree error handling, probe delegation (QwenLM#8215) * fix(review): port the collocated-dropout test to the post-QwenLM#8050 runner seam Merging main brought QwenLM#8050's Windows-portability refactor, which resolves the probe runner through vitest/package.json's bin — a node_modules/.bin fake is dead weight it never reads. The 8215-only collocated-dropout test still installed the old .bin fake, so the REAL vitest ran its fixtures, price.test.ts genuinely passed, and the hunk scored survived. The test now overrides the fake package's vitest.mjs like every post-refactor test. * fix(review): bound the summary rescue, apply the ATX heading rule, sweep stale build locks The three 8215-layer findings from the latest review, fixed at this layer (they were first patched further up the stack, where the reviewer of THIS PR cannot see them): - trimOutput's summary rescue is capped at 40 lines — uncapped, 40k lines of 'Test <n>: …' prose voided the trim entirely (measured 1.6MB in, 1.6MB out) and the bounded-output contract is the whole point. - A '#' with no following whitespace is prose, not a heading (the ATX rule GitHub applies): 'QwenLM#8176', '#tag', an unfenced '#!/bin/bash' no longer end the Test Plan section mid-body; the bare-#-run crash on the closing scan is guarded. - A base-tree build lock older than 30 minutes is a corpse left by a killed builder — swept and rebuilt instead of reporting busy for the rest of the review. * fix(review): EEXIST-only lock busy, bun test alias, chained cd bail, fence backreference Four live findings from the latest inline review round (the rest of the round was already fixed upstream by the takeover bot - verified by probing head behavior rather than re-reading the threads): - base-tree's lock catch distinguishes EEXIST (a concurrent builder, busy) from EPERM/EROFS/ENOSPC (this run's own failure, reported as such, not as a busy that will never clear). - "bun test" is bun's built-in runner, not a package-script alias: it runs whether or not any manifest defines test, so ruling it against the scripts table filed a false contradicted. - A chained cd matches the leading-cd shape but the single-hop resolver joined file tokens against the FIRST directory; it now bails like the exotic-cd case. - codeSpans' fence regex closes on its own marker via backreference; a tilde fence line inside a backtick block ended the span early and lines after it were lost to extraction. * fix(review): close the ten open findings on this PR - guard-true tested for a comparison anywhere after `if (`, including the then-body, so `if (ready) emit(a !== b);` admitted a mutant on the comparison-less condition the gate exists to exclude. It now tests the condition span only. - The `survived` detail said "when it changes" for legacy DELETION mutants too; it now matches the operator. - test-delta's `unparsed` required both sides to parse zero files, so a PR-side failure whose FAIL lines the trim scattered was silently dropped whenever the base rerun happened to parse. netNew/shared come from the PR side, so the PR side alone decides. - failingFilesOf now matches Windows path shapes (backslashes, C:) - a missed parse is an unattributed failure, not a loud error. - The replacement branch of runOneMutant (write-file -> run-probe -> classify) had no end-to-end test; one now drives a coalesce operator through the real handler and asserts the mutated line, the verdict, the operator-specific wording, and that the shared tree is untouched. - Two tests were vacuous with respect to what they promised: the baseline-dir test never asserted the cwd (its helper swallowed the argument - fixed at the helper), and the one-candidate-per-line test used an input that never triggered the replacement path, so the `continue` under test was not load-bearing. - Reattached the orphaned selectMutants JSDoc; reworded the SKILL line. * fix(review): the test helper's cwd parameter is required, not optional CI's `tsc --build` failed on test-delta.test.ts: the exec seam always passes a cwd, but the helper's signature marked it optional, so pushing it into a string[] was `string | undefined`. Missed locally because vitest runs through esbuild, which strips types without checking them - the suite was green while the build was red. The gate to run before pushing a type-level change is `npx tsc --build`, not the test suite. * fix(review): a base rerun that could not RUN attributes nothing Two Criticals from the latest review, both reachable on the brief's own happy path: - baseUnusable covered only timeouts. Every other way the base side can fail to run - an unbuilt base tree, a missing install, a workspace the PR ADDED (npm test --workspace cannot resolve on base), an ENOBUFS truncation - exits non-zero with zero FAIL lines, which this code read as a green base. Every PR-side failure then became netNew: the strongest evidence the command emits, manufactured from a base that never ran a test. It now attributes nothing and says why. - Timeout detection was the weaker substring form the sibling explicitly rejects; an external SIGTERM (container stop, cancelled job) set neither an ETIMEDOUT message nor an exit code and fed straight into the above. build-test now exports spawnTimedOut and test-delta asks the same question rather than re-deriving it. Also: the base output is trimmed (it precedes the verdict fields in the report the agent reads, so an untrimmed megabyte truncates exactly what the command produces); the guard-true gate no longer reads an arrow function's => as a comparison (every predicate guard was a candidate - the if (ready) noise the gate exists to exclude); the term-drop message no longer calls a string concatenation a reserve term; the unparsed note describes its own PR-side-only condition; and the ledger's Step 6/Step 8 now agree that a still-standing finding keeps its id. * fix(review): extract-step resolves all three env/defaults levels, and comments every env line Two silent-wrongness defects in a command whose whole value is fidelity. `env:`, `shell:` and `working-directory:` are three-level settings on GitHub — workflow, job, step, nearest wins — and only the step level appears in the step's own text. Reading step-level alone reproduced by machine the exact transcription error this command exists to remove: measured, a step under a job-level `NODE_ENV: production` and a workflow-level `GLOBAL_FLAG` extracted with `env: { LOCAL: '1' }` and `workingDirectory: undefined`, so the emitted script ran with both unset and nothing said so. Not a contrived shape: this repo carries workflow-level `env:` in 7 workflows, job-level `env:` in 10, and job-level `defaults.run` in qwen-triage.yml — the workflow the command's own test plan names. The three levels now merge with the runner's precedence, and `envSources` records which level each key came from, so an inherited value is visible rather than indistinguishable from the step's own. The env block was commented per ENTRY, not per LINE. A YAML block scalar (qwen-autofix.yml's `SETTINGS_JSON: |-`) reaches the header as a multi-line string, so its continuation lines landed in command position — and under the `set -e` the header itself emits, the extracted step died in its own preamble before its `run:` body ran. Every line is commented now. Tests pin the effect, not the mechanism: `executableLines()` asserts nothing but the `run:` body ever reaches command position, plus a `bash -n` parse check. Verified to flip — all five new assertions fail against the pre-fix implementation (`{ LOCAL: '1' }`, `undefined` working directory, three executable lines instead of one, and a real `bash -n` syntax error). * fix(review): compare failing files by a normalised, project-keyed identity Critical: the two sides run in DIFFERENT roots (the PR worktree and the base tree), and netNew/shared compared the parsed paths verbatim - so an absolute-path runner turned every pre-existing failure into a fabricated Critical, with the authority of a measurement behind it. Paths are now normalised against each run's own root (and backslashes to /, so a Windows path compares with its POSIX-printed twin), which is why test-delta gained --pr-worktree. The identity also keeps the vitest project token: dropping it collapsed same-named test files across workspaces, so a PR-caused failure in one package could read as pre-existing because another package has a file by the same name - the worse failure direction. Also from the same review, all of them reachable on the brief's own path: - The base rerun now inherits build-test's stdio: ['ignore','pipe','pipe'] ("a build that asks a question is a build that hangs until the deadline") and its trimOutput, which matters because entries[].base precedes the verdict fields in the report the agent reads. - The brief gates on base-tree's `available`, not just its `path`: a tree that was created but did not build populates path too, and measuring against it turns an infrastructure failure into Criticals. - A programmatic caller omitting `timeout` no longer sends NaN into spawnSync. - MutantCandidate is a discriminated union, so an operator without its replacement line - which would delete a line while reporting "with its ?? fallback dropped" - is unrepresentable. - The comparison class no longer requires a trailing space (if (a<b) is the same guard, just unformatted) and matches a brace-less else if. - DeltaEntry.unparsed's doc now describes the PR-side-only condition it actually implements. * fix(review): restore the whole-command budget, keep generics out of guard-true Round-1 findings from a fresh review of this PR: - test-delta had no aggregate deadline: --timeout is PER command and defaults to 300s, so three failed commands is 900s against Agent 7's 600s ceiling - killed with NO report at all, discarding the base-tree install and build just paid for. TOTAL_BUDGET_MS mirrors the one test-efficacy reserves; commands it cannot fit are disclosed. - guard-true matched generic calls: `if (isRecord<string>(v))` produced a mutant, and a type-guard predicate is exactly the `if (ready)` shape whose survivors the gate calls noise. The trailing \s is required, not an accidental asymmetry with [!=]== - telling `a<b` from `fn<T>(x)` needs a parser, and the gate is silence-biased by design. - --pr-worktree had no contract test, and its failure mode is the worst here: arriving undefined, root stripping silently stops and every pre-existing failure becomes a fabricated netNew. The new test feeds parseSync's output straight into runTestDelta and asserts an attribution only reachable when both roots were stripped (verified red against the snake_case field shape that shipped once already). - Merged the two consecutive doc comments on prWorktree. * fix(review): a budget-shortened deadline is not the same fact as a slow rerun Round-2 finding on the budget just restored: `Math.min(perCommandMs, remaining)` can hand a rerun far less than --timeout, and if it dies there the note said only "timed out - infrastructure, not evidence". True, but it sends the reader hunting a hang that is really an exhausted budget - and unlike a real timeout, a rerun with budget to spare would still measure it. The note now names those commands separately and says so. Verified red against removing the tracking line. * fix(review): brace-tolerant stub list, pipefail fidelity, and extract-step in the briefs Round-3 findings on this PR, fixed. `expressionsOf` matched `[^}]*`, so any expression containing a brace — `format('refs/pull/{0}/head', …)`, `fromJSON('{"a":1}')` — was not mis-listed but DROPPED. A stub list reads as "these are all the values to supply", so a silent omission is a value that never gets stubbed. It now scans forward to the closing `}}`, and reports nothing for an unterminated site rather than swallowing the rest of the text. Declaring `shell: bash` is not the runner's default `bash`. The default is `bash -e {0}`; a declared `bash` (at any level) is `bash --noprofile --norc -eo pipefail {0}`, and a pipeline whose middle stage fails aborts under one and not the other. The header now carries `set -eo pipefail` or `set -e` accordingly — 163 of this repo's 434 `run:` steps are under a declared bash and were getting the weaker one. A `shell:` value is also a command template (`perl {0}`), so only its first word goes in the shebang and the whole template is recorded beside it. `extract-step` was registered on the CLI and mentioned in DESIGN.md, and nowhere in SKILL.md or the agent briefs — the runtime prompts. The capability was unreachable by the agents it was written for. The verifier's brief now carries it next to the A/B paragraph it composes with, and Step 4 summarises it. Also: env ordered nearest-first (measured on qwen-autofix.yml:route:0, merge order put 20 inherited entries ahead of the step's own 26 in a 49-line header); a valueless `FOO:` renders as the empty string, not `"null"`, and a non-scalar as JSON rather than `[object Object]`; a missing file no longer reports as a parse failure; DESIGN.md's lens count matches its list. The test oracle is rebuilt around the property instead of a filter: the file is the header plus the body verbatim, and every line before the body is a comment or a directive the test names. The old helper dropped `set -e` unconditionally, so it could not tell the header's from one the body legitimately contains — and would have gone green on a header that leaked exactly that line. 434 real `run:` steps swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): restore the replacement sub-cap, stranded on a downstream branch Round-3 finding, and the third instance of one class: an 8218-layer fix committed on the 8261 branch, four PRs above the code it belongs to. Measured over 40 real commits, the replacement operators produce ~24x the deletion pool (215 vs 9 candidates; guard-true drives it). Every mutant run drains the same window hunk probes draw from LAST, so uncapped, most diffs with any replacement candidates leave hunk probing zero runs - the hunk-survived finding class silently stops firing and nothing says so. Three slots, and what the sub-cap drops is counted in skippedForCap rather than lost. Also swept the other direction: diffed every review file against its 8261 copy to confirm nothing else 8218-layer is stranded up there. The remaining divergence is 8261's own (the positive control, its lenses). * docs(review): complete the "delta cannot rule" enumeration in both places Round-4 finding. The brief and SKILL.md each listed three cases where test-delta attributes nothing - unparsed, timed-out base, no merge base - but the code has five: the later rounds of this PR added "a base rerun that failed without naming any failing file" (it did not measure the base) and "a command the whole-command budget could not fit". Two enumerations of the same set with different membership, in the two places an agent reads. That is the sibling-enumeration lesson this skill teaches, applied to its own prose for the second time: the fix is not just adding the missing members but saying that the report names each case with its own reason rather than folding them into one. * fix(review): the invokes list was mostly prose, not commands Round-4 finding on this PR. `invokes` is documented as a heuristic starting point, and imprecision is fine — but measured over this repo's 434 real `run:` steps it was reporting 435 distinct "commands", 267 of them appearing exactly once, with a worst case of 63 entries made up of words like `CI`, `Evidence`, `PR` and `and`. A list that size, mostly prose, is not a starting point. Three causes, each measured: - A `${{ … }}` expression is not shell, and it routinely contains `||`. Splitting on that as a pipeline separator reported both operands as commands (`matrix.arch`, `github.event.inputs.version`). Expressions are now masked to an opaque token before the split; one sitting in command position contributes nothing, which is honest — what it expands to is unknown here by design. - A heredoc body is input to a command, not a list of them. Its lines were scanned as commands, terminator included. 12 steps in this repo carry one. - The `name=value` skip stepped over the prefix and took the NEXT word as the command — but for a quoted value with spaces that word is inside the value: `EVIDENCE_SECTION=$'### Evidence images'` reported `Evidence`. Quoted spans are now blanked out, with the quote carried across lines so a multi-line string's continuation lines are data too. Command substitutions are read first, so `body="$(sanitize < "$REPORT")"` still reports `sanitize`. A `#` preceded by whitespace ends the live part of a line, so an apostrophe in a trailing comment cannot open a span and eat the rest of the script. Measured after: 435 distinct commands to 187, singletons 267 to 104, worst case 63 to 27 — and the worst case is now real commands (`awk cat chmod curl git jq mktemp pkill tar tee timeout`) plus the script's own shell functions. Also added the builtins a stub could not intercept anyway (`eval`, `exec`, `source`, `unset`, `command`, …) to the keyword set. 434 steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * docs(review): the rationale named only one of the ways base goes unmeasured Third and last copy of the enumeration the previous commit fixed. The bullet's headline already generalised - "base attributes nothing it did not finish" - but its body named only the timeout, so a reader learning the contract from the rationale would conclude timeouts are the only unusable case. Name the set, and say why the report keeps the reasons apart: "we could not measure" and "we measured nothing" are different facts to the author, and only one of them is about their PR. * fix(review): a quoted `<<EOF` is not a heredoc, and a continued line is one command Round-5 findings, both in the previous commit's own scanner. A heredoc opener was matched over the whole line, so one inside a string started heredoc mode: `echo "write <<EOF for a heredoc"` made every later line wait for a terminator that never arrives. The failure is not a missing entry but a missing REST — measured, a three-line script returned `[]` instead of `[curl, jq]`, empty and entirely plausible. Opener detection moved inside the quote walk, where it only fires outside quotes; the quoted forms (`<<'EOF'`) are consumed by the match, so their quotes never open a span either. A backslash-continued command was scanned as several lines, which puts the next ARGUMENT in command position — this is where `apt-get install -y \` / ` libx11-dev` reported the package as an invoked command. Continuations are now joined into one logical line before scanning. Measured after: 187 distinct commands to 185, and the singleton tail is now dominated by real ones — PowerShell cmdlets, macOS tooling, and the scripts' own shell functions. 434 real steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): case labels, a second heredoc, and an expression in command position Round-6 findings, from running the scanner against adversarial shapes rather than reading it again. Two are UNDER-reports, which is the worse direction: a command missing from the list is a stub the verifier never writes, so the extraction reaches the real network. - A `case` pattern label stopped the scan on its own line: `blocked) gh api x` reported nothing, losing `gh`. The label is now stepped over like a `name=` prefix. - Only the first heredoc opener on a line was tracked, so `cat <<A <<B` left the second body and its terminator read as commands (`B`, `y`). Openers are queued and consumed in order. - Masking an expression to a QUOTED token let the quote-stripper delete it entirely, so `${{ steps.x.outputs.cmd }} arg` reported `arg` as the command. The token now survives stripping and cannot match a command word, so an expression in command position contributes nothing. Ten further adversarial shapes were already correct and are pinned as regression guards: nested `$( )` in quotes, subshells, function definitions, indented heredoc terminators, backticks, bare redirects, adjacent and empty `${{ }}` sites, and a JSON literal inside an expression. Verified to flip — exactly the three above fail against the previous commit, the other ten pass. 434 real steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): annotate the continuation accumulator so tsc can type it `invokedCommandsOf`'s backslash-continuation loop failed to compile with TS7022: the narrowed type of `pending` at the join line is the union of the loop-entry value and the back edge, and the back edge is computed from the join itself. The declaration's own annotation does not break that cycle - control-flow narrowing runs after it - so the checker gives up and calls the result `any`. Caught only on a forced rebuild. `tsc --build` had been reporting this tree clean off a stale .tsbuildinfo, which is the same shape of gap that put a type error into CI last week: the test runner strips types, so the suite stayed green either way. Gate with --force. * fix(review): a file-count label stops counting at the end of its line `Test Files 45 passed` filing its 45 as a differing TEST count was fixed once, with a lookbehind on the bare-count pattern. That only ever rejected the all-green shape. The moment any file fails, the runner prints `Test Files 1 failed | 44 passed (45)`, the label is no longer adjacent to the number, and 44 comes through as a test count - so the note reads "claimed 44, observed 1323" on exactly the runs whose summary someone would paste. Adjacency was the wrong invariant; the line is. Masking from the label to end-of-line is distance-independent and picks up jest's `Test Suites: 1 failed, 44 passed, 45 total` at no cost. The label keeps its `Test` word on purpose: a first cut matched a bare `files` and blanked the line in "expect all four files and 471 tests to pass", silencing a real claim. An existing test caught that, which is the argument for the rule being as narrow as it is - anything that suppresses claims is worth exactly its narrowness. Also drops the now-dead lookbehind, which a reviewer had separately (and wrongly) called a JS syntax error; variable-length lookbehind is legal in V8, and the module parsed fine. It goes because the line rule subsumes it, not because it was broken. * fix(review): parse the base rerun before its output is trimmed `trimOutput` rescues module errors and runner summaries out of the omitted middle, not the per-file FAIL lines this command reads. A base suite whose failure section overruns the tail budget therefore lost failing files into the gap - and a SHORT base set is the dangerous direction, because netNew is the PR side minus the base side. Every file the trim hid came back as a Critical attributed to this PR by "measurement". Parse the raw text, report the bounded one. The PR side cannot be fixed here: it is read out of build-test's stored output, trimmed before this command existed. That loss runs the other way - it understates `shared`, never invents a netNew - so the entry carries `prTruncated` and the note says the list may be partial. A silence-biased gap is still a gap the author should hear about. Also names both selection caps in the mutant-skip diagnostic. The count accumulates replacement sub-cap drops, and with 2 deletions and 6 replacements the total is exactly MAX_MUTANTS: the main cap never fires, yet 3 are dropped, and the message sent the reader looking for a pool of 11 candidates that never existed. * test(review): pin the `sh` set-line to exactly what the runner uses A reviewer asked for `expect(script).not.toContain('set -e')` on a `shell: sh` step. The opposite is correct - GitHub runs that step as `sh -e {0}` - but the thread was right that nothing pinned it either way. Assert both halves: `set -e` is present, `pipefail` is not. Dropping the line makes an extracted `sh` step run past a failure the runner stops on; adding pipefail claims a bash feature `sh` does not have. * fix(review): rerun only the command shapes build-test emits This command reads a report off disk and then executes the strings in it with `shell: true`, in the base worktree. Nothing else in the pipeline re-executes a value it read back from a file, so nothing else has to care where that value came from - this does, and the provenance is worse than it looks: the command is `npm test --workspace="<dir>"`, the workspace token is a directory, and a directory is a name a pull request chooses. `packages/x";curl …|sh;"` is a legal path in git and on Linux and it round-trips through the report into a shell. Restricting to the emitter's own grammar costs nothing real, since that grammar is the two shapes build-test produces. A command outside it is skipped and disclosed, the same treatment everything else this command cannot do already gets, so a future shape degrades to "judge it by the diff" rather than to arbitrary execution. * fix(review): the working directory is a setting the extraction was losing Two findings, both of them this file failing its own stated argument. The stub list dropped it. `expressions` covered the script and the env and nothing else, so a `working-directory: ${{ github.workspace }}/x` produced an empty list and a summary line reading "0 ${{ }} site(s) to stub" - and `expressionsOf`'s own comment says why that is the failure this list cannot afford: the caller reads it as "these are all the values to supply". Widened to every setting the command carries, the `shell:` template included. The emitted script never mentioned it. The env block is commented into the header precisely so a reader of the script alone can see it; the working directory changes what the script does just as much and was in the metadata only. The argument for reading all three levels, written in this file, is that a step run "in the wrong directory, and nothing says so" is the transcription error the command exists to remove - which is exactly what the header did. It is a comment rather than a `cd` for the same reason env is comments, not exports: the value may hold `${{ … }}` and this command substitutes nothing. Both pinned, and both checked by deleting the fix: each mutation fails exactly one of the two new tests. * fix(review): $(( )) is arithmetic, and a heredoc's form decides where it ends Three fixes and one deliberate non-fix, all measured against this repo's own 434 `run:` steps rather than argued. `$(( ))` was read as a command substitution, so `N=$((N + 1))` reported `N` as a command to stub. It was the single largest source of junk in the list: 196 distinct "commands" across the corpus, 165 without it. A plain `<<WORD` heredoc ends only on a line that is exactly WORD; the loose match ended it on an indented `EOF` inside the body and then read the body as commands, which is how `rm` got reported for a script that never runs it. `<<-` stays looser than bash (any leading whitespace, not just tabs) because looser can only end a body early, and this file's priority is that an under-report is the worse direction. `[^()]*` matched only the innermost `$( )`, so `X=$(gh api $(u))` lost `gh` - a missed stub, and the extraction reaches the network. Depth counted now, and the assignment-prefix skip no longer steps over an unclosed `X=$(gh` into reading `api` as the command. The non-fix is recorded where the next reader will hit it: the quote walk is flat while shell quoting nests, and over ~300 lines the drift reports fragments of jq source as commands. Inserting a separator where a blanked span was removes nine of those, but it splits `a"X"b`, which is one word to the shell, and the minimal reproducer for the difference is 296 lines - nothing short enough to pin it. A scanner nobody can pin costs more than the junk it removes. * fix(review): refuse an ambiguous step name, and report errors like the siblings Two findings, both about a message the caller never gets. A job may legally hold two steps with the same name. The selector took the first and said nothing, which is the failure this file's own header names - "picks the same-named step from the wrong job" - and it is worst in the use the command exists for: A/B extraction runs it once per tree, so a PR that adds or reorders a duplicate leaves the two sides comparing different steps while reporting on one. Refused out loud now, naming the indices; the index is always available and never ambiguous. The handler also let every throw propagate, so five carefully separated messages - cannot read vs cannot parse vs no job vs no step vs no `run:` - all arrived as "An unexpected critical error occurred" under a stack trace. `base-tree` and `test-plan` in the same directory already catch, write the message, and set exit 1. Matched. The separation between "the path is wrong" and "the YAML is wrong" only pays if the caller sees it. * docs(review): the brief's list of limits was one short "Two limits worth knowing before you spend the step" became three when the selector started refusing an ambiguous step name, and the count went stale in the same commit that made it wrong. This is the enumeration drift the skill teaches, in the text that teaches it. The added entry says what to do rather than only what happens: pass the index, which is what an A/B wants regardless — the two trees have to select the same step, and a name that moved between them is exactly how they stop doing that. * chore(review): drop a scratch probe file that reached the branch `packages/cli/inert.mjs` was a throwaway harness for sweeping the repo's own workflows; its cleanup ran with a relative path from the wrong directory and it got committed by the next `git add -A`. It failed CI and not the local hook because the two lint different sets: lint-staged filters to `*.{js,jsx,ts,tsx}`, which does not include `.mjs`, while CI's flat config picks the file up regardless of `--ext`. Verified with CI's own command this time, not the hook's. --------- Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> Co-authored-by: Qwen Code <qwen-code@users.noreply.github.com> Co-authored-by: Qwen Code Bot <qwen-code-bot@users.noreply.github.com> Co-authored-by: Qwen Code Autofix <qwen-code-autofix@users.noreply.github.com>
…M#8255) * feat(review): borrowed-verification trio — test-plan check, base-tree A/B, per-hunk probes - qwen review test-plan: rule on the PR Test Plan's checkable claims (paths, npm scripts, test counts) against the reviewed tree; contradictions and differing counts are disclosed via compose-review, never capping. - qwen review base-tree: build the merge base in a sibling worktree so the verifier can A/B a comparative claim instead of reading it; swept by cleanup. - test-efficacy: third probe kind — reverse-apply one hunk at a time and re-run the affected tests, attributing a still-green suite to the specific change nothing gates; shares the mutants' budget window, runs last. * fix(review): survive real runner output — ANSI-laced and trimmed-away summaries Both measured on a live /review of QwenLM#8176 with the built CLI: - test-plan's observedTestCounts strips SGR sequences before matching; a color-enabled pipe interleaves them BETWEEN tokens, and the count claim fell to 'unchecked' with the summary right there in the report. - build-test's trimOutput rescues runner summary lines from the omitted middle (like module-resolution errors): a failing suite's tail is all failure details and npm epilogue, which pushed the one-line summary out of the kept text entirely. * fix(review): address the eight findings from live review of this branch All measured in the review (QwenLM#8215 review comment): - test-plan: linear-time bold-heading scan (the old pattern backtracked catastrophically on an unclosed ** line an untrusted PR body controls); a flag preceding the npm script yields no claim instead of a false 'no package defines this script'. - test-efficacy: a hunk probe's restore recreates the parent directory a reverse-applied 'new file' hunk removed (the ENOENT from finally lost the verdict and marked every remaining hunk inconclusive); hunks get their own skippedForBaseline instead of mislabeling a red baseline as a budget skip; splitDiffIntoHunks re-captures the file header at every diff --git boundary; a hunk-survived finding notes when it restates an inert file-level revert at hunk granularity. - base-tree: idempotent fast path keyed on a build marker + HEAD check — concurrent verifier shards reuse one built tree instead of sweeping it out from under each other mid-A/B (a fabricated base-side difference with a deterministic source tag was the worst case); cost wording is now 'an install and a build' everywhere it was 'one extra build'. * fix(cli): never score a hunk survived when its own test left the baseline (QwenLM#8215) A per-hunk probe reported `survived` whenever the green baseline probes still passed with the hunk reverted. When the hunk's own collocated test dropped out of the baseline (a probe-tree import error collects nothing), the remaining green probes prove only that THEY do not cover the hunk, so the verdict is now `inconclusive` — the same dropped-test asymmetry the mutants already hold. Also scope the hunk-survived cross-reference note to the hunk's own collocated test, and let test-plan match a workspace-scoped run of the plan's bare command instead of falling through to the manifest on an exact-string miss. * fix(review): silence-bias hardening from four live review rounds of this branch The two blocking findings, reproduced on this PR's own Test Plan: - test-plan files no false contradicted notes: npm rulings move from a four-verb denylist to an allowlist (the run form + npm's script aliases — the ~fifty other builtins each used to become 'no package defines this script'); a slash token is claimed as a repo path only with evidence (an extension or ./ prefix), never when it is a flag's value (--repo owner/repo) or under the review's own temp root; HEADING_LINE_RE drops the same quadratic shape its bold sibling was rewritten to remove. - base-tree gets a real mutual-exclusion lock around sweep+add+build (mkdirSync test-and-set; the loser returns busy instead of deleting the tree the winner is mid-install in), and a failed build writes a settled marker so later shards stop re-paying the install to relearn 'unavailable'. Also: Agent 7's brief now names hunk-survived and the hunks.* counters (it is the report's only consumer, and the finding class was invisible); hunk findings anchor at the first ADDED line instead of up to three context lines above the change. * feat(review): four round-2 borrowings — measured failure attribution, round ledger, richer mutants, doc parity Shaped by a live round-2 maintainer re-verification (QwenLM#7818): - qwen review test-delta: rerun the PR side's failed test commands on the built merge base and diff the failing FILE SETS — netNew is the PR's own failure by measurement (whatever files the diff touches), shared is pre-existing by measurement. Counts are never compared: a flaky suite fails different test names between runs of the same tree. An unfinished base rerun attributes nothing. - Round ledger: the incremental review cache persists confirmed findings under round-scoped ids (R1-2); a high-effort re-review rules on each (fixed / still stands / cannot tell) and opens its report with the table, the way a human round-2 comment opens with 'M1 is fixed'. - Three replacement mutation operators in test-efficacy: drop '?? fallback', force a comparison-bearing guard condition to 'true', drop a '+ CONST' term. Each survivor maps to one crisp untested-protection sentence. A line whose raw text and literal-blanked code view disagree yields no candidate — an edit index computed across the two views once spliced 'iftrue 0)' into a guard, and a mangled mutant reads as inconclusive while quietly spending a cap slot. - Quality brief: documentation-parity lens — a new user-facing surface whose siblings are documented is a Suggestion naming the sibling precedent; no documented sibling, no finding. * feat(review): render adjudication, workflow step extraction, three verification lenses Shaped by a live sanitizer-PR verification (QwenLM#8147): - Render-adjudication capability (opt-in): with QWEN_REVIEW_SCRATCH_REPO set, the verifier may post a minimal payload to that user-designated repo and rule on GitHub's own rendered HTML — the measured case being an @ -> &QwenLM#64; defusal every local reading called sound while GitHub's real renderer registered the mention and fired the notification. Absent the setting, rendering claims honestly cap at low confidence / cannot tell. Step 7's write ban names the carve-out explicitly. - qwen review extract-step: lift one workflow step's run: script verbatim into an executable, with env (as comments, never half-substituted exports), every ${{ }} site listed unevaluated, and a heuristic invoked- command list as the stubbing starting point. With base-tree, both arms of a by-hand workflow A/B become two invocations. yaml declared as a cli dependency (previously resolved only via hoisting). - Three brief lenses: a borrowed protection idiom missing what made it work at home (the code ancestor did the protecting; only the entity was copied); a second parser for an authoritatively-parsed format is a divergence hunt; tests that pin the mechanism instead of the effect, and oracles that mirror the implementation's own model. * feat(review): sibling-entrance discipline for the fixed verdict From round 6 of the live sanitizer verification (QwenLM#8147): the fix closed the fence-shaped entrance into a raw-HTML block, and the code-span entrance beside it — same divergence, adjacent syntax — stayed open. A re-check that tests only the reported input rules 'fixed' over a hole one backtick away. Both fixed-verdict sites in SKILL Step 6 (the open-Criticals re-check and the round ledger) now require enumerating a divergence-class defect's sibling entrances before ruling fixed; a still-open sibling is a NEW finding, never a reason to withhold the original's fixed — the two rulings stay separate so the second hole cannot ship unreviewed. * feat(review): three measured-verification lenses from live rounds 8037/8005 - Threshold-boundary scan: when a fix is a ratio/length guard, hold the issue's own variables fixed and binary-search the boundary where the behaviour flips; put the number next to what the issue reports. Live case: a prose-ratio guard covered the edit/write_file half of its issue and silently declined the run_shell_command half (~473-char boundary). - Delimiter self-injection named as the first parser-differential probe: a no-escaping extractor fed its own close tag truncates silently. - Shared-gate state enumeration: a deliberate-design defence extends only to the states it argues — an input-hold argued for 'active' silently froze three idle states sharing the same gate. The sibling-entrance rule, applied to a state machine instead of a syntax. * feat(review): carry the round ledger in the posted review body The ledger shipped as a local cache file and its first multi-round live use exposed the flaw: four model-comparison rounds re-reviewed the same two PRs and every round opened from scratch — medium never reads the cache, and from CI or another clone there is no cache at all. The one artifact every environment can see, the posted review, carried nothing machine-readable. - compose-review embeds an HTML-comment marker (invisible on GitHub) carrying this round's findings and round number; the round comes from the side file pr-context recovered, +1 — never from the model. - pr-context recovers the reviewing account's latest marker, renders it as a 'Previous /review round (machine ledger)' work-list section, and writes it beside the context file for compose-review's round counter. Own-account only, latest round wins, fail-quiet on any malformation — the marker is data the next round rules on, never authority it obeys. - Step 6's previous-round ruling now runs at medium too (recovery is free; medium still writes no cache and posts nothing); the local cache is demoted to fallback for rounds that never posted. * fix(review): address review feedback — false-positive hardening, binary diff guard, error convention (QwenLM#8215) * fix(review): address review feedback — base-tree availability gate, test-plan false positives, hunk-probe ranges (QwenLM#8215) - base-tree: only stamp a base tree available when runBuildTest actually compiled something (ok AND npm toolchain AND a non-empty build). An `unsupported` handoff or an empty npm scope returns ok:true having built nothing; marking that tree available let an A/B read the absence of a build as a behavioural difference. - cleanup: sweep the stale base-tree build lock a killed builder leaves behind. - test-plan: read the root manifest's scripts directly so a root-only script survives when the root defines no build/test; bail on the inline --root=./dir rebasing form; stop treating a positional after an inline --flag=value as the flag's value; prefer a failed scoped run when ruling a bare command; anchor the npm script alias to a full token so `yarn test:unit` is not truncated. - test-efficacy: exclude `\ No newline at end of file` from the startLine offset count; compute the mutant-overlap range from the header's new-side span so it no longer overshoots into a closely following hunk. * fix(review): address review feedback — diff-header false positives, stale prompt enumeration, added-file hunk probes (QwenLM#8215) * fix(review): address review feedback — cd-base exclusion, Test Files count guard, base-tree error handling, probe delegation (QwenLM#8215) * fix(review): port the collocated-dropout test to the post-QwenLM#8050 runner seam Merging main brought QwenLM#8050's Windows-portability refactor, which resolves the probe runner through vitest/package.json's bin — a node_modules/.bin fake is dead weight it never reads. The 8215-only collocated-dropout test still installed the old .bin fake, so the REAL vitest ran its fixtures, price.test.ts genuinely passed, and the hunk scored survived. The test now overrides the fake package's vitest.mjs like every post-refactor test. * fix(review): bound the summary rescue, apply the ATX heading rule, sweep stale build locks The three 8215-layer findings from the latest review, fixed at this layer (they were first patched further up the stack, where the reviewer of THIS PR cannot see them): - trimOutput's summary rescue is capped at 40 lines — uncapped, 40k lines of 'Test <n>: …' prose voided the trim entirely (measured 1.6MB in, 1.6MB out) and the bounded-output contract is the whole point. - A '#' with no following whitespace is prose, not a heading (the ATX rule GitHub applies): 'QwenLM#8176', '#tag', an unfenced '#!/bin/bash' no longer end the Test Plan section mid-body; the bare-#-run crash on the closing scan is guarded. - A base-tree build lock older than 30 minutes is a corpse left by a killed builder — swept and rebuilt instead of reporting busy for the rest of the review. * fix(review): EEXIST-only lock busy, bun test alias, chained cd bail, fence backreference Four live findings from the latest inline review round (the rest of the round was already fixed upstream by the takeover bot - verified by probing head behavior rather than re-reading the threads): - base-tree's lock catch distinguishes EEXIST (a concurrent builder, busy) from EPERM/EROFS/ENOSPC (this run's own failure, reported as such, not as a busy that will never clear). - "bun test" is bun's built-in runner, not a package-script alias: it runs whether or not any manifest defines test, so ruling it against the scripts table filed a false contradicted. - A chained cd matches the leading-cd shape but the single-hop resolver joined file tokens against the FIRST directory; it now bails like the exotic-cd case. - codeSpans' fence regex closes on its own marker via backreference; a tilde fence line inside a backtick block ended the span early and lines after it were lost to extraction. * fix(review): make the ledger marker actually reach the posted body Three Criticals from the QwenLM#8255 review, all real: - The marker was appended in the CLI handler, AFTER composeReview() returned, so it only ever reached the composed JSON on disk. submit posts what the PURE function returns, so no posted review carried a marker, latestOwnLedger always returned null, and the whole feature was inert end to end. The append now lives inside composeReview, fed by a draftedComments seam that both CLI boundaries fill from the same array they count (and both strip from a model-written state JSON). Three end-to-end tests now assert on the body composeReview returns. - pr-context's side-file write preceded the mkdir that creates its directory and was unguarded: an ENOENT failed the whole command over a best-effort carry-forward. mkdir first, write guarded. - CANONICAL_LGTM_RE is anchored, so a trailing marker made every no-op round "worth showing" and prior rounds started rendering in full. The marker is now stripped before the filter and the count walk, not only before rendering. Plus the three thread findings: ledger cells are escaped before landing in the context table (a title with a pipe could forge rows), parseLedger takes the LAST marker and type-checks/normalises what it recovers, and the stray verdict doc comment is reattached to verdictLine. * fix(review): close the ten open findings on this PR - guard-true tested for a comparison anywhere after `if (`, including the then-body, so `if (ready) emit(a !== b);` admitted a mutant on the comparison-less condition the gate exists to exclude. It now tests the condition span only. - The `survived` detail said "when it changes" for legacy DELETION mutants too; it now matches the operator. - test-delta's `unparsed` required both sides to parse zero files, so a PR-side failure whose FAIL lines the trim scattered was silently dropped whenever the base rerun happened to parse. netNew/shared come from the PR side, so the PR side alone decides. - failingFilesOf now matches Windows path shapes (backslashes, C:) - a missed parse is an unattributed failure, not a loud error. - The replacement branch of runOneMutant (write-file -> run-probe -> classify) had no end-to-end test; one now drives a coalesce operator through the real handler and asserts the mutated line, the verdict, the operator-specific wording, and that the shared tree is untouched. - Two tests were vacuous with respect to what they promised: the baseline-dir test never asserted the cwd (its helper swallowed the argument - fixed at the helper), and the one-candidate-per-line test used an input that never triggered the replacement path, so the `continue` under test was not load-bearing. - Reattached the orphaned selectMutants JSDoc; reworded the SKILL line. * fix(review): the test helper's cwd parameter is required, not optional CI's `tsc --build` failed on test-delta.test.ts: the exec seam always passes a cwd, but the helper's signature marked it optional, so pushing it into a string[] was `string | undefined`. Missed locally because vitest runs through esbuild, which strips types without checking them - the suite was green while the build was red. The gate to run before pushing a type-level change is `npx tsc --build`, not the test suite. * fix(review): a base rerun that could not RUN attributes nothing Two Criticals from the latest review, both reachable on the brief's own happy path: - baseUnusable covered only timeouts. Every other way the base side can fail to run - an unbuilt base tree, a missing install, a workspace the PR ADDED (npm test --workspace cannot resolve on base), an ENOBUFS truncation - exits non-zero with zero FAIL lines, which this code read as a green base. Every PR-side failure then became netNew: the strongest evidence the command emits, manufactured from a base that never ran a test. It now attributes nothing and says why. - Timeout detection was the weaker substring form the sibling explicitly rejects; an external SIGTERM (container stop, cancelled job) set neither an ETIMEDOUT message nor an exit code and fed straight into the above. build-test now exports spawnTimedOut and test-delta asks the same question rather than re-deriving it. Also: the base output is trimmed (it precedes the verdict fields in the report the agent reads, so an untrimmed megabyte truncates exactly what the command produces); the guard-true gate no longer reads an arrow function's => as a comparison (every predicate guard was a candidate - the if (ready) noise the gate exists to exclude); the term-drop message no longer calls a string concatenation a reserve term; the unparsed note describes its own PR-side-only condition; and the ledger's Step 6/Step 8 now agree that a still-standing finding keeps its id. * fix(review): extract-step resolves all three env/defaults levels, and comments every env line Two silent-wrongness defects in a command whose whole value is fidelity. `env:`, `shell:` and `working-directory:` are three-level settings on GitHub — workflow, job, step, nearest wins — and only the step level appears in the step's own text. Reading step-level alone reproduced by machine the exact transcription error this command exists to remove: measured, a step under a job-level `NODE_ENV: production` and a workflow-level `GLOBAL_FLAG` extracted with `env: { LOCAL: '1' }` and `workingDirectory: undefined`, so the emitted script ran with both unset and nothing said so. Not a contrived shape: this repo carries workflow-level `env:` in 7 workflows, job-level `env:` in 10, and job-level `defaults.run` in qwen-triage.yml — the workflow the command's own test plan names. The three levels now merge with the runner's precedence, and `envSources` records which level each key came from, so an inherited value is visible rather than indistinguishable from the step's own. The env block was commented per ENTRY, not per LINE. A YAML block scalar (qwen-autofix.yml's `SETTINGS_JSON: |-`) reaches the header as a multi-line string, so its continuation lines landed in command position — and under the `set -e` the header itself emits, the extracted step died in its own preamble before its `run:` body ran. Every line is commented now. Tests pin the effect, not the mechanism: `executableLines()` asserts nothing but the `run:` body ever reaches command position, plus a `bash -n` parse check. Verified to flip — all five new assertions fail against the pre-fix implementation (`{ LOCAL: '1' }`, `undefined` working directory, three executable lines instead of one, and a real `bash -n` syntax error). * fix(review): compare failing files by a normalised, project-keyed identity Critical: the two sides run in DIFFERENT roots (the PR worktree and the base tree), and netNew/shared compared the parsed paths verbatim - so an absolute-path runner turned every pre-existing failure into a fabricated Critical, with the authority of a measurement behind it. Paths are now normalised against each run's own root (and backslashes to /, so a Windows path compares with its POSIX-printed twin), which is why test-delta gained --pr-worktree. The identity also keeps the vitest project token: dropping it collapsed same-named test files across workspaces, so a PR-caused failure in one package could read as pre-existing because another package has a file by the same name - the worse failure direction. Also from the same review, all of them reachable on the brief's own path: - The base rerun now inherits build-test's stdio: ['ignore','pipe','pipe'] ("a build that asks a question is a build that hangs until the deadline") and its trimOutput, which matters because entries[].base precedes the verdict fields in the report the agent reads. - The brief gates on base-tree's `available`, not just its `path`: a tree that was created but did not build populates path too, and measuring against it turns an infrastructure failure into Criticals. - A programmatic caller omitting `timeout` no longer sends NaN into spawnSync. - MutantCandidate is a discriminated union, so an operator without its replacement line - which would delete a line while reporting "with its ?? fallback dropped" - is unrepresentable. - The comparison class no longer requires a trailing space (if (a<b) is the same guard, just unformatted) and matches a brace-less else if. - DeltaEntry.unparsed's doc now describes the PR-side-only condition it actually implements. * fix(review): restore the whole-command budget, keep generics out of guard-true Round-1 findings from a fresh review of this PR: - test-delta had no aggregate deadline: --timeout is PER command and defaults to 300s, so three failed commands is 900s against Agent 7's 600s ceiling - killed with NO report at all, discarding the base-tree install and build just paid for. TOTAL_BUDGET_MS mirrors the one test-efficacy reserves; commands it cannot fit are disclosed. - guard-true matched generic calls: `if (isRecord<string>(v))` produced a mutant, and a type-guard predicate is exactly the `if (ready)` shape whose survivors the gate calls noise. The trailing \s is required, not an accidental asymmetry with [!=]== - telling `a<b` from `fn<T>(x)` needs a parser, and the gate is silence-biased by design. - --pr-worktree had no contract test, and its failure mode is the worst here: arriving undefined, root stripping silently stops and every pre-existing failure becomes a fabricated netNew. The new test feeds parseSync's output straight into runTestDelta and asserts an attribution only reachable when both roots were stripped (verified red against the snake_case field shape that shipped once already). - Merged the two consecutive doc comments on prWorktree. * fix(review): a budget-shortened deadline is not the same fact as a slow rerun Round-2 finding on the budget just restored: `Math.min(perCommandMs, remaining)` can hand a rerun far less than --timeout, and if it dies there the note said only "timed out - infrastructure, not evidence". True, but it sends the reader hunting a hang that is really an exhausted budget - and unlike a real timeout, a rerun with budget to spare would still measure it. The note now names those commands separately and says so. Verified red against removing the tracking line. * fix(review): brace-tolerant stub list, pipefail fidelity, and extract-step in the briefs Round-3 findings on this PR, fixed. `expressionsOf` matched `[^}]*`, so any expression containing a brace — `format('refs/pull/{0}/head', …)`, `fromJSON('{"a":1}')` — was not mis-listed but DROPPED. A stub list reads as "these are all the values to supply", so a silent omission is a value that never gets stubbed. It now scans forward to the closing `}}`, and reports nothing for an unterminated site rather than swallowing the rest of the text. Declaring `shell: bash` is not the runner's default `bash`. The default is `bash -e {0}`; a declared `bash` (at any level) is `bash --noprofile --norc -eo pipefail {0}`, and a pipeline whose middle stage fails aborts under one and not the other. The header now carries `set -eo pipefail` or `set -e` accordingly — 163 of this repo's 434 `run:` steps are under a declared bash and were getting the weaker one. A `shell:` value is also a command template (`perl {0}`), so only its first word goes in the shebang and the whole template is recorded beside it. `extract-step` was registered on the CLI and mentioned in DESIGN.md, and nowhere in SKILL.md or the agent briefs — the runtime prompts. The capability was unreachable by the agents it was written for. The verifier's brief now carries it next to the A/B paragraph it composes with, and Step 4 summarises it. Also: env ordered nearest-first (measured on qwen-autofix.yml:route:0, merge order put 20 inherited entries ahead of the step's own 26 in a 49-line header); a valueless `FOO:` renders as the empty string, not `"null"`, and a non-scalar as JSON rather than `[object Object]`; a missing file no longer reports as a parse failure; DESIGN.md's lens count matches its list. The test oracle is rebuilt around the property instead of a filter: the file is the header plus the body verbatim, and every line before the body is a comment or a directive the test names. The old helper dropped `set -e` unconditionally, so it could not tell the header's from one the body legitimately contains — and would have gone green on a header that leaked exactly that line. 434 real `run:` steps swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): restore the replacement sub-cap, stranded on a downstream branch Round-3 finding, and the third instance of one class: an 8218-layer fix committed on the 8261 branch, four PRs above the code it belongs to. Measured over 40 real commits, the replacement operators produce ~24x the deletion pool (215 vs 9 candidates; guard-true drives it). Every mutant run drains the same window hunk probes draw from LAST, so uncapped, most diffs with any replacement candidates leave hunk probing zero runs - the hunk-survived finding class silently stops firing and nothing says so. Three slots, and what the sub-cap drops is counted in skippedForCap rather than lost. Also swept the other direction: diffed every review file against its 8261 copy to confirm nothing else 8218-layer is stranded up there. The remaining divergence is 8261's own (the positive control, its lenses). * fix(review): one severity predicate, carried ids, and a lossless marker Round-1 review of the ledger marker, six findings. Two contradicted contracts this package already states elsewhere: - `buildLedger` restated the severity predicate as a bare `startsWith` while `severityOf` — what `countInlineFindings` and the unmarked-comment gate both use — trims first. A Critical whose body opened with a newline was counted, was posted, blocked the merge, and was silently absent from the ledger, shifting the id of every finding after it. It classifies through `severityOf` now, and strips the marker with the shared prefixes instead of a second copy of the literals. - Ids were assigned by array position every round, so the report saying `R1-2 still stands` shipped beside a marker recording that claim as `R3-4` — and `LedgerFinding.id` documented itself "stable across re-reports". Step 6 now mandates writing the original id into the re-report (`**[Critical]** R1-2: …`), `buildLedger` reads it back, and a duplicate or absent id falls back to the next free id of this round. Four smaller ones: - `stripLedgerMarker` removed only the FIRST marker while `parseLedger` reads the LAST, so a body holding two leaked the trusted marker into model-facing prose and un-filtered the canonical LGTM it was appended to. - The `--` guard rewrote data to an em dash: a finding about `--comment` reached the next round as `—comment`. It escapes at the JSON layer now, which is comment-safe and lossless. - `file` was capped on read (200) but not on write; the cap is one constant binding both halves. - `submit`'s path — the one GitHub's payload is built from — had no test that the marker reaches the posted body. Plus: tie-break `latestOwnLedger` on review id when `submitted_at` ties, neutralise a backtick in the location code span, skip the `currentUser()` round-trip when the PR has no reviews, and say why a stale side file is deliberately left in place. * docs(review): complete the "delta cannot rule" enumeration in both places Round-4 finding. The brief and SKILL.md each listed three cases where test-delta attributes nothing - unparsed, timed-out base, no merge base - but the code has five: the later rounds of this PR added "a base rerun that failed without naming any failing file" (it did not measure the base) and "a command the whole-command budget could not fit". Two enumerations of the same set with different membership, in the two places an agent reads. That is the sibling-enumeration lesson this skill teaches, applied to its own prose for the second time: the fix is not just adding the missing members but saying that the report names each case with its own reason rather than folding them into one. * fix(review): the invokes list was mostly prose, not commands Round-4 finding on this PR. `invokes` is documented as a heuristic starting point, and imprecision is fine — but measured over this repo's 434 real `run:` steps it was reporting 435 distinct "commands", 267 of them appearing exactly once, with a worst case of 63 entries made up of words like `CI`, `Evidence`, `PR` and `and`. A list that size, mostly prose, is not a starting point. Three causes, each measured: - A `${{ … }}` expression is not shell, and it routinely contains `||`. Splitting on that as a pipeline separator reported both operands as commands (`matrix.arch`, `github.event.inputs.version`). Expressions are now masked to an opaque token before the split; one sitting in command position contributes nothing, which is honest — what it expands to is unknown here by design. - A heredoc body is input to a command, not a list of them. Its lines were scanned as commands, terminator included. 12 steps in this repo carry one. - The `name=value` skip stepped over the prefix and took the NEXT word as the command — but for a quoted value with spaces that word is inside the value: `EVIDENCE_SECTION=$'### Evidence images'` reported `Evidence`. Quoted spans are now blanked out, with the quote carried across lines so a multi-line string's continuation lines are data too. Command substitutions are read first, so `body="$(sanitize < "$REPORT")"` still reports `sanitize`. A `#` preceded by whitespace ends the live part of a line, so an apostrophe in a trailing comment cannot open a span and eat the rest of the script. Measured after: 435 distinct commands to 187, singletons 267 to 104, worst case 63 to 27 — and the worst case is now real commands (`awk cat chmod curl git jq mktemp pkill tar tee timeout`) plus the script's own shell functions. Also added the builtins a stub could not intercept anyway (`eval`, `exec`, `source`, `unset`, `command`, …) to the keyword set. 434 steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * docs(review): the rationale named only one of the ways base goes unmeasured Third and last copy of the enumeration the previous commit fixed. The bullet's headline already generalised - "base attributes nothing it did not finish" - but its body named only the timeout, so a reader learning the contract from the rationale would conclude timeouts are the only unusable case. Name the set, and say why the report keeps the reasons apart: "we could not measure" and "we measured nothing" are different facts to the author, and only one of them is about their PR. * fix(review): a quoted `<<EOF` is not a heredoc, and a continued line is one command Round-5 findings, both in the previous commit's own scanner. A heredoc opener was matched over the whole line, so one inside a string started heredoc mode: `echo "write <<EOF for a heredoc"` made every later line wait for a terminator that never arrives. The failure is not a missing entry but a missing REST — measured, a three-line script returned `[]` instead of `[curl, jq]`, empty and entirely plausible. Opener detection moved inside the quote walk, where it only fires outside quotes; the quoted forms (`<<'EOF'`) are consumed by the match, so their quotes never open a span either. A backslash-continued command was scanned as several lines, which puts the next ARGUMENT in command position — this is where `apt-get install -y \` / ` libx11-dev` reported the package as an invoked command. Continuations are now joined into one logical line before scanning. Measured after: 187 distinct commands to 185, and the singleton tail is now dominated by real ones — PowerShell cmdlets, macOS tooling, and the scripts' own shell functions. 434 real steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): case labels, a second heredoc, and an expression in command position Round-6 findings, from running the scanner against adversarial shapes rather than reading it again. Two are UNDER-reports, which is the worse direction: a command missing from the list is a stub the verifier never writes, so the extraction reaches the real network. - A `case` pattern label stopped the scan on its own line: `blocked) gh api x` reported nothing, losing `gh`. The label is now stepped over like a `name=` prefix. - Only the first heredoc opener on a line was tracked, so `cat <<A <<B` left the second body and its terminator read as commands (`B`, `y`). Openers are queued and consumed in order. - Masking an expression to a QUOTED token let the quote-stripper delete it entirely, so `${{ steps.x.outputs.cmd }} arg` reported `arg` as the command. The token now survives stripping and cannot match a command word, so an expression in command position contributes nothing. Ten further adversarial shapes were already correct and are pinned as regression guards: nested `$( )` in quotes, subshells, function definitions, indented heredoc terminators, backticks, bare redirects, adjacent and empty `${{ }}` sites, and a JSON literal inside an expression. Verified to flip — exactly the three above fail against the previous commit, the other ten pass. 434 real steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): annotate the continuation accumulator so tsc can type it `invokedCommandsOf`'s backslash-continuation loop failed to compile with TS7022: the narrowed type of `pending` at the join line is the union of the loop-entry value and the back edge, and the back edge is computed from the join itself. The declaration's own annotation does not break that cycle - control-flow narrowing runs after it - so the checker gives up and calls the result `any`. Caught only on a forced rebuild. `tsc --build` had been reporting this tree clean off a stale .tsbuildinfo, which is the same shape of gap that put a type error into CI last week: the test runner strips types, so the suite stayed green either way. Gate with --force. * fix(review): a file-count label stops counting at the end of its line `Test Files 45 passed` filing its 45 as a differing TEST count was fixed once, with a lookbehind on the bare-count pattern. That only ever rejected the all-green shape. The moment any file fails, the runner prints `Test Files 1 failed | 44 passed (45)`, the label is no longer adjacent to the number, and 44 comes through as a test count - so the note reads "claimed 44, observed 1323" on exactly the runs whose summary someone would paste. Adjacency was the wrong invariant; the line is. Masking from the label to end-of-line is distance-independent and picks up jest's `Test Suites: 1 failed, 44 passed, 45 total` at no cost. The label keeps its `Test` word on purpose: a first cut matched a bare `files` and blanked the line in "expect all four files and 471 tests to pass", silencing a real claim. An existing test caught that, which is the argument for the rule being as narrow as it is - anything that suppresses claims is worth exactly its narrowness. Also drops the now-dead lookbehind, which a reviewer had separately (and wrongly) called a JS syntax error; variable-length lookbehind is legal in V8, and the module parsed fine. It goes because the line rule subsumes it, not because it was broken. * fix(review): parse the base rerun before its output is trimmed `trimOutput` rescues module errors and runner summaries out of the omitted middle, not the per-file FAIL lines this command reads. A base suite whose failure section overruns the tail budget therefore lost failing files into the gap - and a SHORT base set is the dangerous direction, because netNew is the PR side minus the base side. Every file the trim hid came back as a Critical attributed to this PR by "measurement". Parse the raw text, report the bounded one. The PR side cannot be fixed here: it is read out of build-test's stored output, trimmed before this command existed. That loss runs the other way - it understates `shared`, never invents a netNew - so the entry carries `prTruncated` and the note says the list may be partial. A silence-biased gap is still a gap the author should hear about. Also names both selection caps in the mutant-skip diagnostic. The count accumulates replacement sub-cap drops, and with 2 deletions and 6 replacements the total is exactly MAX_MUTANTS: the main cap never fires, yet 3 are dropped, and the message sent the reader looking for a pool of 11 candidates that never existed. * test(review): pin the `sh` set-line to exactly what the runner uses A reviewer asked for `expect(script).not.toContain('set -e')` on a `shell: sh` step. The opposite is correct - GitHub runs that step as `sh -e {0}` - but the thread was right that nothing pinned it either way. Assert both halves: `set -e` is present, `pipefail` is not. Dropping the line makes an extracted `sh` step run past a failure the runner stops on; adding pipefail claims a bash feature `sh` does not have. * fix(review): rerun only the command shapes build-test emits This command reads a report off disk and then executes the strings in it with `shell: true`, in the base worktree. Nothing else in the pipeline re-executes a value it read back from a file, so nothing else has to care where that value came from - this does, and the provenance is worse than it looks: the command is `npm test --workspace="<dir>"`, the workspace token is a directory, and a directory is a name a pull request chooses. `packages/x";curl …|sh;"` is a legal path in git and on Linux and it round-trips through the report into a shell. Restricting to the emitter's own grammar costs nothing real, since that grammar is the two shapes build-test produces. A command outside it is skipped and disclosed, the same treatment everything else this command cannot do already gets, so a future shape degrades to "judge it by the diff" rather than to arbitrary execution. * fix(review): the working directory is a setting the extraction was losing Two findings, both of them this file failing its own stated argument. The stub list dropped it. `expressions` covered the script and the env and nothing else, so a `working-directory: ${{ github.workspace }}/x` produced an empty list and a summary line reading "0 ${{ }} site(s) to stub" - and `expressionsOf`'s own comment says why that is the failure this list cannot afford: the caller reads it as "these are all the values to supply". Widened to every setting the command carries, the `shell:` template included. The emitted script never mentioned it. The env block is commented into the header precisely so a reader of the script alone can see it; the working directory changes what the script does just as much and was in the metadata only. The argument for reading all three levels, written in this file, is that a step run "in the wrong directory, and nothing says so" is the transcription error the command exists to remove - which is exactly what the header did. It is a comment rather than a `cd` for the same reason env is comments, not exports: the value may hold `${{ … }}` and this command substitutes nothing. Both pinned, and both checked by deleting the fix: each mutation fails exactly one of the two new tests. * fix(review): $(( )) is arithmetic, and a heredoc's form decides where it ends Three fixes and one deliberate non-fix, all measured against this repo's own 434 `run:` steps rather than argued. `$(( ))` was read as a command substitution, so `N=$((N + 1))` reported `N` as a command to stub. It was the single largest source of junk in the list: 196 distinct "commands" across the corpus, 165 without it. A plain `<<WORD` heredoc ends only on a line that is exactly WORD; the loose match ended it on an indented `EOF` inside the body and then read the body as commands, which is how `rm` got reported for a script that never runs it. `<<-` stays looser than bash (any leading whitespace, not just tabs) because looser can only end a body early, and this file's priority is that an under-report is the worse direction. `[^()]*` matched only the innermost `$( )`, so `X=$(gh api $(u))` lost `gh` - a missed stub, and the extraction reaches the network. Depth counted now, and the assignment-prefix skip no longer steps over an unclosed `X=$(gh` into reading `api` as the command. The non-fix is recorded where the next reader will hit it: the quote walk is flat while shell quoting nests, and over ~300 lines the drift reports fragments of jq source as commands. Inserting a separator where a blanked span was removes nine of those, but it splits `a"X"b`, which is one word to the shell, and the minimal reproducer for the difference is 296 lines - nothing short enough to pin it. A scanner nobody can pin costs more than the junk it removes. * fix(review): refuse an ambiguous step name, and report errors like the siblings Two findings, both about a message the caller never gets. A job may legally hold two steps with the same name. The selector took the first and said nothing, which is the failure this file's own header names - "picks the same-named step from the wrong job" - and it is worst in the use the command exists for: A/B extraction runs it once per tree, so a PR that adds or reorders a duplicate leaves the two sides comparing different steps while reporting on one. Refused out loud now, naming the indices; the index is always available and never ambiguous. The handler also let every throw propagate, so five carefully separated messages - cannot read vs cannot parse vs no job vs no step vs no `run:` - all arrived as "An unexpected critical error occurred" under a stack trace. `base-tree` and `test-plan` in the same directory already catch, write the message, and set exit 1. Matched. The separation between "the path is wrong" and "the YAML is wrong" only pays if the caller sees it. * docs(review): the brief's list of limits was one short "Two limits worth knowing before you spend the step" became three when the selector started refusing an ambiguous step name, and the count went stale in the same commit that made it wrong. This is the enumeration drift the skill teaches, in the text that teaches it. The added entry says what to do rather than only what happens: pass the index, which is what an A/B wants regardless — the two trees have to select the same step, and a name that moved between them is exactly how they stop doing that. * chore(review): drop a scratch probe file that reached the branch `packages/cli/inert.mjs` was a throwaway harness for sweeping the repo's own workflows; its cleanup ran with a relative path from the wrong directory and it got committed by the next `git add -A`. It failed CI and not the local hook because the two lint different sets: lint-staged filters to `*.{js,jsx,ts,tsx}`, which does not include `.mjs`, while CI's flat config picks the file up regardless of `--ext`. Verified with CI's own command this time, not the hook's. * fix(review): bound the whole ledger marker, not just each of its fields The per-field caps leave the total unbounded: fifty findings at full width serialize to just under 17,000 characters. The module's own paragraph calls the marker "a footnote, never a payload" and reasons about GitHub's 65,536-character body limit, so the total is the number that claim is about and nothing was computing it. Set the budget from measurement rather than from the limit. Across every review this pipeline has posted on its own stack (n=66) the body runs a median of 721 characters, p90 2,178, max 3,925 - the limit has ~61 KiB of headroom, and an over-long marker was never going to fail a post. The real cost is proportion: 17,000 characters of invisible payload on a 3,925-character review is four times more marker than review. 8 KiB holds fifty findings at realistic widths without dropping any, and the truncation path is reached only by a ledger no round has produced. What it drops travels with it. A `dropped` count is the difference between a list the next round reads as complete and one that says it is not - the same distinction this module already draws everywhere else. * fix(review): a partial ledger must not render as a complete one The size cap added in the previous commit can drop entries, and the rendered section still opened with "**Every entry below is owed a this-round ruling**" and said nothing about what was missing. The `dropped` count existed and reached the only place a reader sees the list as nothing at all — which is the failure it was added to prevent, committed one step after adding the field. It now says the list is partial, how many are missing, and that absence is not evidence a finding was fixed. Second, the table escaping escaped `|` but not `\`. `\|` in a title became `\\|`, which markdown reads as an escaped backslash followed by a LIVE separator: the forged row the escaping exists to prevent, produced by the escaping. Backslash first now, and the test counts separators the way markdown does rather than by pattern, because a lookbehind counter is fooled by exactly the input under test. * fix(review): an empty ledger title is a cap the next round cannot lift A comment that is nothing but its severity marker produced a ledger entry with an empty title. That does not merely degrade the entry, it jams the review: the next round is told every ledger entry is owed a this-round ruling, has no claim text to rule on, answers `cannot tell`, and `cannot-tell-existing-critical` is one of the cap reasons. Nothing between rounds changes, so the cap comes back every round. Dropping the entry would hide a Critical that really was posted, so it keeps its place and gets the one handle there is - the location it was filed at, or "the review body" for a body-level one. The field's stated job is "enough for the next round to re-locate the claim", and that is what a text-less finding still has. Found by probing `buildLedger` with ten malformed comment shapes; the other nine - carried ids, id collisions, two comments carrying the same id, `R2D2` not being an id, a leading newline before the marker - all behaved correctly and needed no change. * fix(review): count both caps as dropped, not just the byte one Found by running the three previous fixes together on one round: 51 findings in, 24 kept, and the marker said 26 were missing. `LEDGER_MAX_FINDINGS` truncates before the byte cap ever runs, and measuring `dropped` against the already-sliced list left the count cap's share silent - so the field added two commits ago to stop a truncated list reading as complete was itself under-reporting the truncation. Measured against what came IN now, and pinned across all four combinations: count cap alone, byte cap alone, both, and neither. Kept plus dropped equals given, in every one. --------- Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> Co-authored-by: Qwen Code <qwen-code@users.noreply.github.com> Co-authored-by: Qwen Code Bot <qwen-code-bot@users.noreply.github.com> Co-authored-by: Qwen Code Autofix <qwen-code-autofix@users.noreply.github.com>
…rol, seven lenses (QwenLM#8261) * feat(review): borrowed-verification trio — test-plan check, base-tree A/B, per-hunk probes - qwen review test-plan: rule on the PR Test Plan's checkable claims (paths, npm scripts, test counts) against the reviewed tree; contradictions and differing counts are disclosed via compose-review, never capping. - qwen review base-tree: build the merge base in a sibling worktree so the verifier can A/B a comparative claim instead of reading it; swept by cleanup. - test-efficacy: third probe kind — reverse-apply one hunk at a time and re-run the affected tests, attributing a still-green suite to the specific change nothing gates; shares the mutants' budget window, runs last. * fix(review): survive real runner output — ANSI-laced and trimmed-away summaries Both measured on a live /review of QwenLM/qwen-code#8176 with the built CLI: - test-plan's observedTestCounts strips SGR sequences before matching; a color-enabled pipe interleaves them BETWEEN tokens, and the count claim fell to 'unchecked' with the summary right there in the report. - build-test's trimOutput rescues runner summary lines from the omitted middle (like module-resolution errors): a failing suite's tail is all failure details and npm epilogue, which pushed the one-line summary out of the kept text entirely. * fix(review): address the eight findings from live review of this branch All measured in the review (QwenLM/qwen-code#8215 review comment): - test-plan: linear-time bold-heading scan (the old pattern backtracked catastrophically on an unclosed ** line an untrusted PR body controls); a flag preceding the npm script yields no claim instead of a false 'no package defines this script'. - test-efficacy: a hunk probe's restore recreates the parent directory a reverse-applied 'new file' hunk removed (the ENOENT from finally lost the verdict and marked every remaining hunk inconclusive); hunks get their own skippedForBaseline instead of mislabeling a red baseline as a budget skip; splitDiffIntoHunks re-captures the file header at every diff --git boundary; a hunk-survived finding notes when it restates an inert file-level revert at hunk granularity. - base-tree: idempotent fast path keyed on a build marker + HEAD check — concurrent verifier shards reuse one built tree instead of sweeping it out from under each other mid-A/B (a fabricated base-side difference with a deterministic source tag was the worst case); cost wording is now 'an install and a build' everywhere it was 'one extra build'. * fix(cli): never score a hunk survived when its own test left the baseline (#8215) A per-hunk probe reported `survived` whenever the green baseline probes still passed with the hunk reverted. When the hunk's own collocated test dropped out of the baseline (a probe-tree import error collects nothing), the remaining green probes prove only that THEY do not cover the hunk, so the verdict is now `inconclusive` — the same dropped-test asymmetry the mutants already hold. Also scope the hunk-survived cross-reference note to the hunk's own collocated test, and let test-plan match a workspace-scoped run of the plan's bare command instead of falling through to the manifest on an exact-string miss. * fix(review): silence-bias hardening from four live review rounds of this branch The two blocking findings, reproduced on this PR's own Test Plan: - test-plan files no false contradicted notes: npm rulings move from a four-verb denylist to an allowlist (the run form + npm's script aliases — the ~fifty other builtins each used to become 'no package defines this script'); a slash token is claimed as a repo path only with evidence (an extension or ./ prefix), never when it is a flag's value (--repo owner/repo) or under the review's own temp root; HEADING_LINE_RE drops the same quadratic shape its bold sibling was rewritten to remove. - base-tree gets a real mutual-exclusion lock around sweep+add+build (mkdirSync test-and-set; the loser returns busy instead of deleting the tree the winner is mid-install in), and a failed build writes a settled marker so later shards stop re-paying the install to relearn 'unavailable'. Also: Agent 7's brief now names hunk-survived and the hunks.* counters (it is the report's only consumer, and the finding class was invisible); hunk findings anchor at the first ADDED line instead of up to three context lines above the change. * feat(review): four round-2 borrowings — measured failure attribution, round ledger, richer mutants, doc parity Shaped by a live round-2 maintainer re-verification (QwenLM/qwen-code#7818): - qwen review test-delta: rerun the PR side's failed test commands on the built merge base and diff the failing FILE SETS — netNew is the PR's own failure by measurement (whatever files the diff touches), shared is pre-existing by measurement. Counts are never compared: a flaky suite fails different test names between runs of the same tree. An unfinished base rerun attributes nothing. - Round ledger: the incremental review cache persists confirmed findings under round-scoped ids (R1-2); a high-effort re-review rules on each (fixed / still stands / cannot tell) and opens its report with the table, the way a human round-2 comment opens with 'M1 is fixed'. - Three replacement mutation operators in test-efficacy: drop '?? fallback', force a comparison-bearing guard condition to 'true', drop a '+ CONST' term. Each survivor maps to one crisp untested-protection sentence. A line whose raw text and literal-blanked code view disagree yields no candidate — an edit index computed across the two views once spliced 'iftrue 0)' into a guard, and a mangled mutant reads as inconclusive while quietly spending a cap slot. - Quality brief: documentation-parity lens — a new user-facing surface whose siblings are documented is a Suggestion naming the sibling precedent; no documented sibling, no finding. * feat(review): render adjudication, workflow step extraction, three verification lenses Shaped by a live sanitizer-PR verification (QwenLM/qwen-code#8147): - Render-adjudication capability (opt-in): with QWEN_REVIEW_SCRATCH_REPO set, the verifier may post a minimal payload to that user-designated repo and rule on GitHub's own rendered HTML — the measured case being an @ -> @ defusal every local reading called sound while GitHub's real renderer registered the mention and fired the notification. Absent the setting, rendering claims honestly cap at low confidence / cannot tell. Step 7's write ban names the carve-out explicitly. - qwen review extract-step: lift one workflow step's run: script verbatim into an executable, with env (as comments, never half-substituted exports), every ${{ }} site listed unevaluated, and a heuristic invoked- command list as the stubbing starting point. With base-tree, both arms of a by-hand workflow A/B become two invocations. yaml declared as a cli dependency (previously resolved only via hoisting). - Three brief lenses: a borrowed protection idiom missing what made it work at home (the code ancestor did the protecting; only the entity was copied); a second parser for an authoritatively-parsed format is a divergence hunt; tests that pin the mechanism instead of the effect, and oracles that mirror the implementation's own model. * feat(review): sibling-entrance discipline for the fixed verdict From round 6 of the live sanitizer verification (QwenLM/qwen-code#8147): the fix closed the fence-shaped entrance into a raw-HTML block, and the code-span entrance beside it — same divergence, adjacent syntax — stayed open. A re-check that tests only the reported input rules 'fixed' over a hole one backtick away. Both fixed-verdict sites in SKILL Step 6 (the open-Criticals re-check and the round ledger) now require enumerating a divergence-class defect's sibling entrances before ruling fixed; a still-open sibling is a NEW finding, never a reason to withhold the original's fixed — the two rulings stay separate so the second hole cannot ship unreviewed. * feat(review): three measured-verification lenses from live rounds 8037/8005 - Threshold-boundary scan: when a fix is a ratio/length guard, hold the issue's own variables fixed and binary-search the boundary where the behaviour flips; put the number next to what the issue reports. Live case: a prose-ratio guard covered the edit/write_file half of its issue and silently declined the run_shell_command half (~473-char boundary). - Delimiter self-injection named as the first parser-differential probe: a no-escaping extractor fed its own close tag truncates silently. - Shared-gate state enumeration: a deliberate-design defence extends only to the states it argues — an input-hold argued for 'active' silently froze three idle states sharing the same gate. The sibling-entrance rule, applied to a state machine instead of a syntax. * feat(review): carry the round ledger in the posted review body The ledger shipped as a local cache file and its first multi-round live use exposed the flaw: four model-comparison rounds re-reviewed the same two PRs and every round opened from scratch — medium never reads the cache, and from CI or another clone there is no cache at all. The one artifact every environment can see, the posted review, carried nothing machine-readable. - compose-review embeds an HTML-comment marker (invisible on GitHub) carrying this round's findings and round number; the round comes from the side file pr-context recovered, +1 — never from the model. - pr-context recovers the reviewing account's latest marker, renders it as a 'Previous /review round (machine ledger)' work-list section, and writes it beside the context file for compose-review's round counter. Own-account only, latest round wins, fail-quiet on any malformation — the marker is data the next round rules on, never authority it obeys. - Step 6's previous-round ruling now runs at medium too (recovery is free; medium still writes no cache and posts nothing); the local cache is demoted to fallback for rounds that never posted. * fix(review): address review feedback — false-positive hardening, binary diff guard, error convention (#8215) * fix(review): address review feedback — base-tree availability gate, test-plan false positives, hunk-probe ranges (#8215) - base-tree: only stamp a base tree available when runBuildTest actually compiled something (ok AND npm toolchain AND a non-empty build). An `unsupported` handoff or an empty npm scope returns ok:true having built nothing; marking that tree available let an A/B read the absence of a build as a behavioural difference. - cleanup: sweep the stale base-tree build lock a killed builder leaves behind. - test-plan: read the root manifest's scripts directly so a root-only script survives when the root defines no build/test; bail on the inline --root=./dir rebasing form; stop treating a positional after an inline --flag=value as the flag's value; prefer a failed scoped run when ruling a bare command; anchor the npm script alias to a full token so `yarn test:unit` is not truncated. - test-efficacy: exclude `\ No newline at end of file` from the startLine offset count; compute the mutant-overlap range from the header's new-side span so it no longer overshoots into a closely following hunk. * feat(review): mined disciplines — effective-diff guard, positive control, seven lenses From a full sweep of 108 maintainer verification comments (2026-07-31): - fetch-pr detects an EMPTY merge-base diff (work already landed — the skill stops and recommends close-as-superseded) and a diff collapsed to a residual by overlapping merged PRs (disclosed; body claims about the rest are description-of-history). - test-efficacy runs a POSITIVE CONTROL before spending the mutant window: an injected always-failing test must turn the runner red, or every would-be survivor is re-classed inconclusive — a dead runner cannot kill, and its survivors were the false gap-reports measured live. - Brief lenses: tested-fix + contamination statement; race verdicts as rates with load amplification (lower idle rate without structural change is luck); triangulate-and-label for unreachable authorities; arm-identity and comparator negative-control for A/B; deleted code's incidental effects; self-defeating fallbacks (trigger set within cannot-succeed set); reachability adjudication (fix vs scaffolding); flipped-assertion audit; shape-vs-provenance authorization probing. * fix(review): four fixes from the stack's own round-2 reviews - test-delta (Critical, #8218 review): a base rerun that fails with ZERO parseable failing files (install/toolchain failure) no longer reads as 'base green' — it attributes nothing, like a timeout, instead of promoting every PR-side failure to net-new. - extract-step (Critical, #8225 review): every line of a multi-line env value is comment-prefixed — an unprefixed block-scalar second line sat in the emitted script as an executable line. - extract-step: expressionsOf no longer misses expressions containing their own brace (format('{0}')). - cleanup (#8215 review): sweeps an orphaned base-tree build lock, which otherwise wedges the A/B for that PR permanently. * fix(review): close the remaining round-2 findings across the stack - selectMutants: replacement operators get a 3-slot sub-cap inside the shared cap — measured 24x pool inflation (215 vs 9 candidates over 40 real commits) was draining the time window hunk probes draw from last, silently un-shipping the hunk-survived finding class; sub-cap drops are counted in skippedForCap. - test-delta: whole-command budget (540s, mirroring test-efficacy) so three failed commands cannot blow the 600s tool ceiling and lose the report; run() regains build-test's three deliberate properties (stdin ignored, error.code/SIGTERM timeout detection, trimmed output). - test-plan: pasted unified diffs inside Evidence blocks shed no path claims (+++ b/<path> once ruled contradicted on a correct body); gitignored paths rule unchecked — absent at the reviewed commit by construction, the .qwen/ reasoning generalized. - agent-prompt: the Agent 7 efficacy block now enumerates all FOUR finding kinds including hunk-survived, the hunks.* skip counters, and harnessValidated — the brief and the block in the same prompt no longer present two different complete sets (the sibling-enumeration lesson, applied to our own prompt). * fix(review): address review feedback — diff-header false positives, stale prompt enumeration, added-file hunk probes (#8215) * fix(review): close the round-3 findings on the stack #8261 review: - the dead-harness re-class now runs BEFORE findings are built (a failed positive control could still file survivor findings — the Critical); it also covers the file-level revert probe's inert verdicts, the control pays for its run out of the shared budget (no more silently eaten skippedForBudget slot), a failed control stops the window instead of manufacturing survivors to re-class, and its note names all three causes that share the green-control shape. - emptyDiff no longer fires on a FAILED diff capture (close-as-superseded off an infrastructure error would close a live PR); countDiffChangedLines counts --x/++x body lines. #8215 review: - the cd target itself is no longer claimed as a path (a bare dir carries no evidence; only the file tokens it resolves are); - 'Test Files N passed' is a FILE count — variable-length lookbehind keeps it out of the test-count claims; - base-tree's handler gains the pipeline's catch-and-exit-1 convention; - probeCreateFailureDetail folded into the shared worktreeCreateFailureDetail. * fix(review): address review feedback — cd-base exclusion, Test Files count guard, base-tree error handling, probe delegation (#8215) * fix(review): port the collocated-dropout test to the post-#8050 runner seam Merging main brought #8050's Windows-portability refactor, which resolves the probe runner through vitest/package.json's bin — a node_modules/.bin fake is dead weight it never reads. The 8215-only collocated-dropout test still installed the old .bin fake, so the REAL vitest ran its fixtures, price.test.ts genuinely passed, and the hunk scored survived. The test now overrides the fake package's vitest.mjs like every post-refactor test. * fix(review): close the round-4 findings from both stack reviews From the #8215 review: trimOutput's summary rescue is capped at 40 lines (uncapped, 40k lines of 'Test <n>: …' prose made the whole trim a no-op — 1.6MB in, 1.6MB out); a spaceless # line is prose per the ATX rule GitHub applies (a pasted #!/bin/bash outside a fence once ended the Test Plan mid-body), with the bare-#-run crash on the closing scan guarded; a base-tree build lock older than 30 minutes is a corpse and is swept rather than reporting busy for the rest of the review. From the #8261 review: a pasted diff's BODY lines shed no path claims (-packages/old/gone.ts matched PATH_RE and ruled a false contradicted — the syntax-line filter alone was vacuous against the real failure mode); a gitignored file that nonetheless EXISTS rules reproduces (the ignore guard only ever downgrades a would-be contradiction); test-delta's note now explains each unattributed command (timeout vs toolchain failure) instead of leaving the reader to infer why it is in neither list. * fix(review): bound the summary rescue, apply the ATX heading rule, sweep stale build locks The three 8215-layer findings from the latest review, fixed at this layer (they were first patched further up the stack, where the reviewer of THIS PR cannot see them): - trimOutput's summary rescue is capped at 40 lines — uncapped, 40k lines of 'Test <n>: …' prose voided the trim entirely (measured 1.6MB in, 1.6MB out) and the bounded-output contract is the whole point. - A '#' with no following whitespace is prose, not a heading (the ATX rule GitHub applies): '#8176', '#tag', an unfenced '#!/bin/bash' no longer end the Test Plan section mid-body; the bare-#-run crash on the closing scan is guarded. - A base-tree build lock older than 30 minutes is a corpse left by a killed builder — swept and rebuilt instead of reporting busy for the rest of the review. * fix(review): EEXIST-only lock busy, bun test alias, chained cd bail, fence backreference Four live findings from the latest inline review round (the rest of the round was already fixed upstream by the takeover bot - verified by probing head behavior rather than re-reading the threads): - base-tree's lock catch distinguishes EEXIST (a concurrent builder, busy) from EPERM/EROFS/ENOSPC (this run's own failure, reported as such, not as a busy that will never clear). - "bun test" is bun's built-in runner, not a package-script alias: it runs whether or not any manifest defines test, so ruling it against the scripts table filed a false contradicted. - A chained cd matches the leading-cd shape but the single-hop resolver joined file tokens against the FIRST directory; it now bails like the exotic-cd case. - codeSpans' fence regex closes on its own marker via backreference; a tilde fence line inside a backtick block ended the span early and lines after it were lost to extraction. * fix(review): make the ledger marker actually reach the posted body Three Criticals from the #8255 review, all real: - The marker was appended in the CLI handler, AFTER composeReview() returned, so it only ever reached the composed JSON on disk. submit posts what the PURE function returns, so no posted review carried a marker, latestOwnLedger always returned null, and the whole feature was inert end to end. The append now lives inside composeReview, fed by a draftedComments seam that both CLI boundaries fill from the same array they count (and both strip from a model-written state JSON). Three end-to-end tests now assert on the body composeReview returns. - pr-context's side-file write preceded the mkdir that creates its directory and was unguarded: an ENOENT failed the whole command over a best-effort carry-forward. mkdir first, write guarded. - CANONICAL_LGTM_RE is anchored, so a trailing marker made every no-op round "worth showing" and prior rounds started rendering in full. The marker is now stripped before the filter and the count walk, not only before rendering. Plus the three thread findings: ledger cells are escaped before landing in the context table (a title with a pipe could forge rows), parseLedger takes the LAST marker and type-checks/normalises what it recovers, and the stray verdict doc comment is reattached to verdictLine. * fix(review): close the ten open findings on this PR - guard-true tested for a comparison anywhere after `if (`, including the then-body, so `if (ready) emit(a !== b);` admitted a mutant on the comparison-less condition the gate exists to exclude. It now tests the condition span only. - The `survived` detail said "when it changes" for legacy DELETION mutants too; it now matches the operator. - test-delta's `unparsed` required both sides to parse zero files, so a PR-side failure whose FAIL lines the trim scattered was silently dropped whenever the base rerun happened to parse. netNew/shared come from the PR side, so the PR side alone decides. - failingFilesOf now matches Windows path shapes (backslashes, C:) - a missed parse is an unattributed failure, not a loud error. - The replacement branch of runOneMutant (write-file -> run-probe -> classify) had no end-to-end test; one now drives a coalesce operator through the real handler and asserts the mutated line, the verdict, the operator-specific wording, and that the shared tree is untouched. - Two tests were vacuous with respect to what they promised: the baseline-dir test never asserted the cwd (its helper swallowed the argument - fixed at the helper), and the one-candidate-per-line test used an input that never triggered the replacement path, so the `continue` under test was not load-bearing. - Reattached the orphaned selectMutants JSDoc; reworded the SKILL line. * fix(review): the test helper's cwd parameter is required, not optional CI's `tsc --build` failed on test-delta.test.ts: the exec seam always passes a cwd, but the helper's signature marked it optional, so pushing it into a string[] was `string | undefined`. Missed locally because vitest runs through esbuild, which strips types without checking them - the suite was green while the build was red. The gate to run before pushing a type-level change is `npx tsc --build`, not the test suite. * fix(review): a base rerun that could not RUN attributes nothing Two Criticals from the latest review, both reachable on the brief's own happy path: - baseUnusable covered only timeouts. Every other way the base side can fail to run - an unbuilt base tree, a missing install, a workspace the PR ADDED (npm test --workspace cannot resolve on base), an ENOBUFS truncation - exits non-zero with zero FAIL lines, which this code read as a green base. Every PR-side failure then became netNew: the strongest evidence the command emits, manufactured from a base that never ran a test. It now attributes nothing and says why. - Timeout detection was the weaker substring form the sibling explicitly rejects; an external SIGTERM (container stop, cancelled job) set neither an ETIMEDOUT message nor an exit code and fed straight into the above. build-test now exports spawnTimedOut and test-delta asks the same question rather than re-deriving it. Also: the base output is trimmed (it precedes the verdict fields in the report the agent reads, so an untrimmed megabyte truncates exactly what the command produces); the guard-true gate no longer reads an arrow function's => as a comparison (every predicate guard was a candidate - the if (ready) noise the gate exists to exclude); the term-drop message no longer calls a string concatenation a reserve term; the unparsed note describes its own PR-side-only condition; and the ledger's Step 6/Step 8 now agree that a still-standing finding keeps its id. * fix(review): extract-step resolves all three env/defaults levels, and comments every env line Two silent-wrongness defects in a command whose whole value is fidelity. `env:`, `shell:` and `working-directory:` are three-level settings on GitHub — workflow, job, step, nearest wins — and only the step level appears in the step's own text. Reading step-level alone reproduced by machine the exact transcription error this command exists to remove: measured, a step under a job-level `NODE_ENV: production` and a workflow-level `GLOBAL_FLAG` extracted with `env: { LOCAL: '1' }` and `workingDirectory: undefined`, so the emitted script ran with both unset and nothing said so. Not a contrived shape: this repo carries workflow-level `env:` in 7 workflows, job-level `env:` in 10, and job-level `defaults.run` in qwen-triage.yml — the workflow the command's own test plan names. The three levels now merge with the runner's precedence, and `envSources` records which level each key came from, so an inherited value is visible rather than indistinguishable from the step's own. The env block was commented per ENTRY, not per LINE. A YAML block scalar (qwen-autofix.yml's `SETTINGS_JSON: |-`) reaches the header as a multi-line string, so its continuation lines landed in command position — and under the `set -e` the header itself emits, the extracted step died in its own preamble before its `run:` body ran. Every line is commented now. Tests pin the effect, not the mechanism: `executableLines()` asserts nothing but the `run:` body ever reaches command position, plus a `bash -n` parse check. Verified to flip — all five new assertions fail against the pre-fix implementation (`{ LOCAL: '1' }`, `undefined` working directory, three executable lines instead of one, and a real `bash -n` syntax error). * fix(review): compare failing files by a normalised, project-keyed identity Critical: the two sides run in DIFFERENT roots (the PR worktree and the base tree), and netNew/shared compared the parsed paths verbatim - so an absolute-path runner turned every pre-existing failure into a fabricated Critical, with the authority of a measurement behind it. Paths are now normalised against each run's own root (and backslashes to /, so a Windows path compares with its POSIX-printed twin), which is why test-delta gained --pr-worktree. The identity also keeps the vitest project token: dropping it collapsed same-named test files across workspaces, so a PR-caused failure in one package could read as pre-existing because another package has a file by the same name - the worse failure direction. Also from the same review, all of them reachable on the brief's own path: - The base rerun now inherits build-test's stdio: ['ignore','pipe','pipe'] ("a build that asks a question is a build that hangs until the deadline") and its trimOutput, which matters because entries[].base precedes the verdict fields in the report the agent reads. - The brief gates on base-tree's `available`, not just its `path`: a tree that was created but did not build populates path too, and measuring against it turns an infrastructure failure into Criticals. - A programmatic caller omitting `timeout` no longer sends NaN into spawnSync. - MutantCandidate is a discriminated union, so an operator without its replacement line - which would delete a line while reporting "with its ?? fallback dropped" - is unrepresentable. - The comparison class no longer requires a trailing space (if (a<b) is the same guard, just unformatted) and matches a brace-less else if. - DeltaEntry.unparsed's doc now describes the PR-side-only condition it actually implements. * fix(review): restore the whole-command budget, keep generics out of guard-true Round-1 findings from a fresh review of this PR: - test-delta had no aggregate deadline: --timeout is PER command and defaults to 300s, so three failed commands is 900s against Agent 7's 600s ceiling - killed with NO report at all, discarding the base-tree install and build just paid for. TOTAL_BUDGET_MS mirrors the one test-efficacy reserves; commands it cannot fit are disclosed. - guard-true matched generic calls: `if (isRecord<string>(v))` produced a mutant, and a type-guard predicate is exactly the `if (ready)` shape whose survivors the gate calls noise. The trailing \s is required, not an accidental asymmetry with [!=]== - telling `a<b` from `fn<T>(x)` needs a parser, and the gate is silence-biased by design. - --pr-worktree had no contract test, and its failure mode is the worst here: arriving undefined, root stripping silently stops and every pre-existing failure becomes a fabricated netNew. The new test feeds parseSync's output straight into runTestDelta and asserts an attribution only reachable when both roots were stripped (verified red against the snake_case field shape that shipped once already). - Merged the two consecutive doc comments on prWorktree. * fix(review): a budget-shortened deadline is not the same fact as a slow rerun Round-2 finding on the budget just restored: `Math.min(perCommandMs, remaining)` can hand a rerun far less than --timeout, and if it dies there the note said only "timed out - infrastructure, not evidence". True, but it sends the reader hunting a hang that is really an exhausted budget - and unlike a real timeout, a rerun with budget to spare would still measure it. The note now names those commands separately and says so. Verified red against removing the tracking line. * fix(review): brace-tolerant stub list, pipefail fidelity, and extract-step in the briefs Round-3 findings on this PR, fixed. `expressionsOf` matched `[^}]*`, so any expression containing a brace — `format('refs/pull/{0}/head', …)`, `fromJSON('{"a":1}')` — was not mis-listed but DROPPED. A stub list reads as "these are all the values to supply", so a silent omission is a value that never gets stubbed. It now scans forward to the closing `}}`, and reports nothing for an unterminated site rather than swallowing the rest of the text. Declaring `shell: bash` is not the runner's default `bash`. The default is `bash -e {0}`; a declared `bash` (at any level) is `bash --noprofile --norc -eo pipefail {0}`, and a pipeline whose middle stage fails aborts under one and not the other. The header now carries `set -eo pipefail` or `set -e` accordingly — 163 of this repo's 434 `run:` steps are under a declared bash and were getting the weaker one. A `shell:` value is also a command template (`perl {0}`), so only its first word goes in the shebang and the whole template is recorded beside it. `extract-step` was registered on the CLI and mentioned in DESIGN.md, and nowhere in SKILL.md or the agent briefs — the runtime prompts. The capability was unreachable by the agents it was written for. The verifier's brief now carries it next to the A/B paragraph it composes with, and Step 4 summarises it. Also: env ordered nearest-first (measured on qwen-autofix.yml:route:0, merge order put 20 inherited entries ahead of the step's own 26 in a 49-line header); a valueless `FOO:` renders as the empty string, not `"null"`, and a non-scalar as JSON rather than `[object Object]`; a missing file no longer reports as a parse failure; DESIGN.md's lens count matches its list. The test oracle is rebuilt around the property instead of a filter: the file is the header plus the body verbatim, and every line before the body is a comment or a directive the test names. The old helper dropped `set -e` unconditionally, so it could not tell the header's from one the body legitimately contains — and would have gone green on a header that leaked exactly that line. 434 real `run:` steps swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): restore the replacement sub-cap, stranded on a downstream branch Round-3 finding, and the third instance of one class: an 8218-layer fix committed on the 8261 branch, four PRs above the code it belongs to. Measured over 40 real commits, the replacement operators produce ~24x the deletion pool (215 vs 9 candidates; guard-true drives it). Every mutant run drains the same window hunk probes draw from LAST, so uncapped, most diffs with any replacement candidates leave hunk probing zero runs - the hunk-survived finding class silently stops firing and nothing says so. Three slots, and what the sub-cap drops is counted in skippedForCap rather than lost. Also swept the other direction: diffed every review file against its 8261 copy to confirm nothing else 8218-layer is stranded up there. The remaining divergence is 8261's own (the positive control, its lenses). * fix(review): one severity predicate, carried ids, and a lossless marker Round-1 review of the ledger marker, six findings. Two contradicted contracts this package already states elsewhere: - `buildLedger` restated the severity predicate as a bare `startsWith` while `severityOf` — what `countInlineFindings` and the unmarked-comment gate both use — trims first. A Critical whose body opened with a newline was counted, was posted, blocked the merge, and was silently absent from the ledger, shifting the id of every finding after it. It classifies through `severityOf` now, and strips the marker with the shared prefixes instead of a second copy of the literals. - Ids were assigned by array position every round, so the report saying `R1-2 still stands` shipped beside a marker recording that claim as `R3-4` — and `LedgerFinding.id` documented itself "stable across re-reports". Step 6 now mandates writing the original id into the re-report (`**[Critical]** R1-2: …`), `buildLedger` reads it back, and a duplicate or absent id falls back to the next free id of this round. Four smaller ones: - `stripLedgerMarker` removed only the FIRST marker while `parseLedger` reads the LAST, so a body holding two leaked the trusted marker into model-facing prose and un-filtered the canonical LGTM it was appended to. - The `--` guard rewrote data to an em dash: a finding about `--comment` reached the next round as `—comment`. It escapes at the JSON layer now, which is comment-safe and lossless. - `file` was capped on read (200) but not on write; the cap is one constant binding both halves. - `submit`'s path — the one GitHub's payload is built from — had no test that the marker reaches the posted body. Plus: tie-break `latestOwnLedger` on review id when `submitted_at` ties, neutralise a backtick in the location code span, skip the `currentUser()` round-trip when the PR has no reviews, and say why a stale side file is deliberately left in place. * docs(review): complete the "delta cannot rule" enumeration in both places Round-4 finding. The brief and SKILL.md each listed three cases where test-delta attributes nothing - unparsed, timed-out base, no merge base - but the code has five: the later rounds of this PR added "a base rerun that failed without naming any failing file" (it did not measure the base) and "a command the whole-command budget could not fit". Two enumerations of the same set with different membership, in the two places an agent reads. That is the sibling-enumeration lesson this skill teaches, applied to its own prose for the second time: the fix is not just adding the missing members but saying that the report names each case with its own reason rather than folding them into one. * fix(review): the invokes list was mostly prose, not commands Round-4 finding on this PR. `invokes` is documented as a heuristic starting point, and imprecision is fine — but measured over this repo's 434 real `run:` steps it was reporting 435 distinct "commands", 267 of them appearing exactly once, with a worst case of 63 entries made up of words like `CI`, `Evidence`, `PR` and `and`. A list that size, mostly prose, is not a starting point. Three causes, each measured: - A `${{ … }}` expression is not shell, and it routinely contains `||`. Splitting on that as a pipeline separator reported both operands as commands (`matrix.arch`, `github.event.inputs.version`). Expressions are now masked to an opaque token before the split; one sitting in command position contributes nothing, which is honest — what it expands to is unknown here by design. - A heredoc body is input to a command, not a list of them. Its lines were scanned as commands, terminator included. 12 steps in this repo carry one. - The `name=value` skip stepped over the prefix and took the NEXT word as the command — but for a quoted value with spaces that word is inside the value: `EVIDENCE_SECTION=$'### Evidence images'` reported `Evidence`. Quoted spans are now blanked out, with the quote carried across lines so a multi-line string's continuation lines are data too. Command substitutions are read first, so `body="$(sanitize < "$REPORT")"` still reports `sanitize`. A `#` preceded by whitespace ends the live part of a line, so an apostrophe in a trailing comment cannot open a span and eat the rest of the script. Measured after: 435 distinct commands to 187, singletons 267 to 104, worst case 63 to 27 — and the worst case is now real commands (`awk cat chmod curl git jq mktemp pkill tar tee timeout`) plus the script's own shell functions. Also added the builtins a stub could not intercept anyway (`eval`, `exec`, `source`, `unset`, `command`, …) to the keyword set. 434 steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * docs(review): the rationale named only one of the ways base goes unmeasured Third and last copy of the enumeration the previous commit fixed. The bullet's headline already generalised - "base attributes nothing it did not finish" - but its body named only the timeout, so a reader learning the contract from the rationale would conclude timeouts are the only unusable case. Name the set, and say why the report keeps the reasons apart: "we could not measure" and "we measured nothing" are different facts to the author, and only one of them is about their PR. * fix(review): a quoted `<<EOF` is not a heredoc, and a continued line is one command Round-5 findings, both in the previous commit's own scanner. A heredoc opener was matched over the whole line, so one inside a string started heredoc mode: `echo "write <<EOF for a heredoc"` made every later line wait for a terminator that never arrives. The failure is not a missing entry but a missing REST — measured, a three-line script returned `[]` instead of `[curl, jq]`, empty and entirely plausible. Opener detection moved inside the quote walk, where it only fires outside quotes; the quoted forms (`<<'EOF'`) are consumed by the match, so their quotes never open a span either. A backslash-continued command was scanned as several lines, which puts the next ARGUMENT in command position — this is where `apt-get install -y \` / ` libx11-dev` reported the package as an invoked command. Continuations are now joined into one logical line before scanning. Measured after: 187 distinct commands to 185, and the singleton tail is now dominated by real ones — PowerShell cmdlets, macOS tooling, and the scripts' own shell functions. 434 real steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): case labels, a second heredoc, and an expression in command position Round-6 findings, from running the scanner against adversarial shapes rather than reading it again. Two are UNDER-reports, which is the worse direction: a command missing from the list is a stub the verifier never writes, so the extraction reaches the real network. - A `case` pattern label stopped the scan on its own line: `blocked) gh api x` reported nothing, losing `gh`. The label is now stepped over like a `name=` prefix. - Only the first heredoc opener on a line was tracked, so `cat <<A <<B` left the second body and its terminator read as commands (`B`, `y`). Openers are queued and consumed in order. - Masking an expression to a QUOTED token let the quote-stripper delete it entirely, so `${{ steps.x.outputs.cmd }} arg` reported `arg` as the command. The token now survives stripping and cannot match a command word, so an expression in command position contributes nothing. Ten further adversarial shapes were already correct and are pinned as regression guards: nested `$( )` in quotes, subshells, function definitions, indented heredoc terminators, backticks, bare redirects, adjacent and empty `${{ }}` sites, and a JSON literal inside an expression. Verified to flip — exactly the three above fail against the previous commit, the other ten pass. 434 real steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): annotate the continuation accumulator so tsc can type it `invokedCommandsOf`'s backslash-continuation loop failed to compile with TS7022: the narrowed type of `pending` at the join line is the union of the loop-entry value and the back edge, and the back edge is computed from the join itself. The declaration's own annotation does not break that cycle - control-flow narrowing runs after it - so the checker gives up and calls the result `any`. Caught only on a forced rebuild. `tsc --build` had been reporting this tree clean off a stale .tsbuildinfo, which is the same shape of gap that put a type error into CI last week: the test runner strips types, so the suite stayed green either way. Gate with --force. * fix(review): a file-count label stops counting at the end of its line `Test Files 45 passed` filing its 45 as a differing TEST count was fixed once, with a lookbehind on the bare-count pattern. That only ever rejected the all-green shape. The moment any file fails, the runner prints `Test Files 1 failed | 44 passed (45)`, the label is no longer adjacent to the number, and 44 comes through as a test count - so the note reads "claimed 44, observed 1323" on exactly the runs whose summary someone would paste. Adjacency was the wrong invariant; the line is. Masking from the label to end-of-line is distance-independent and picks up jest's `Test Suites: 1 failed, 44 passed, 45 total` at no cost. The label keeps its `Test` word on purpose: a first cut matched a bare `files` and blanked the line in "expect all four files and 471 tests to pass", silencing a real claim. An existing test caught that, which is the argument for the rule being as narrow as it is - anything that suppresses claims is worth exactly its narrowness. Also drops the now-dead lookbehind, which a reviewer had separately (and wrongly) called a JS syntax error; variable-length lookbehind is legal in V8, and the module parsed fine. It goes because the line rule subsumes it, not because it was broken. * fix(review): address round-1 review of the mined-disciplines batch The positive control could state a verdict it never measured, and its own failure path shipped untested. Ten findings from the review of #8261: - `runControlMutant` returns `boolean | null`. An unreadable probe file injected no test and ran nothing, but returned `false` — which the caller reads as "the injected always-failing test stayed green", re-classes every survivor with that sentence, and discards the whole mutant/hunk window over an I/O error. `null` is the file's own third-outcome rule; the window stays spendable and the note says the harness was not validated. - `mutants.skippedForControl` / `hunks.skippedForControl`. A control that came back red stopped the run with candidates unprobed while every `skipped*` counter read 0 — the silent cap the brief teaching agents to read those counters rules out. Their own reason, not the budget's. - Cover the control's failure half: a dead-runner integration case (survivors re-classed, revert probe's `inert` re-classed, findings empty, nothing spent) and a could-not-set-up case pinning that `null` does NOT discard the window. Both branches, and the re-class block, were previously unreachable by any test. - `rulePath`: one existence check, not two. The second was unreachable behind the first and its note — "exists at the reviewed commit (the diff does not change it)" — silently retired, collapsing tracked state and this run's build output onto one sentence. The ignore status now picks the wording. - `expressionsOf`: forbid the body from crossing another `${{`. A malformed site above a real one swallowed it into the blob, so `${{ github.event.comment.body }}` stopped being enumerated as an injection site — the one direction this helper must not fail in. - `countDiffChangedLines`: count by hunk position, not prefix shape. A deleted line whose content starts `--` arrives as `--- …` (markdown rules, YAML markers, SQL comments), and every dropped line pushes the ratio toward a false `collapsedFromUpstream`. - `emptyDiff` is gated on `baseFetchFailed` too. A stale local base ref that already contains the head commits diffs to empty, and the skill acts on that by recommending close-as-superseded. - `test-delta`: price the budget slot against the command's own measured duration instead of a flat 5s floor, which admitted commands that could only time out and then disclosed them as "infrastructure, not evidence". `skippedForBudget` becomes a structured field; `now` becomes a seam so the test stops reassigning the global `Date.now`. - Drop `probeCreateFailureDetail` (no production caller after the swap to `worktreeCreateFailureDetail`) and the duplicated base-lock sweep in `cleanup.ts`, ten lines above the identical pre-existing one. - `--` before the path in `check-ignore`, and the comment says plainly that no `-`-leading claim survives extraction today. * fix(review): parse the base rerun before its output is trimmed `trimOutput` rescues module errors and runner summaries out of the omitted middle, not the per-file FAIL lines this command reads. A base suite whose failure section overruns the tail budget therefore lost failing files into the gap - and a SHORT base set is the dangerous direction, because netNew is the PR side minus the base side. Every file the trim hid came back as a Critical attributed to this PR by "measurement". Parse the raw text, report the bounded one. The PR side cannot be fixed here: it is read out of build-test's stored output, trimmed before this command existed. That loss runs the other way - it understates `shared`, never invents a netNew - so the entry carries `prTruncated` and the note says the list may be partial. A silence-biased gap is still a gap the author should hear about. Also names both selection caps in the mutant-skip diagnostic. The count accumulates replacement sub-cap drops, and with 2 deletions and 6 replacements the total is exactly MAX_MUTANTS: the main cap never fires, yet 3 are dropped, and the message sent the reader looking for a pool of 11 candidates that never existed. * test(review): pin the `sh` set-line to exactly what the runner uses A reviewer asked for `expect(script).not.toContain('set -e')` on a `shell: sh` step. The opposite is correct - GitHub runs that step as `sh -e {0}` - but the thread was right that nothing pinned it either way. Assert both halves: `set -e` is present, `pipefail` is not. Dropping the line makes an extracted `sh` step run past a failure the runner stops on; adding pipefail claims a bash feature `sh` does not have. * fix(review): rerun only the command shapes build-test emits This command reads a report off disk and then executes the strings in it with `shell: true`, in the base worktree. Nothing else in the pipeline re-executes a value it read back from a file, so nothing else has to care where that value came from - this does, and the provenance is worse than it looks: the command is `npm test --workspace="<dir>"`, the workspace token is a directory, and a directory is a name a pull request chooses. `packages/x";curl …|sh;"` is a legal path in git and on Linux and it round-trips through the report into a shell. Restricting to the emitter's own grammar costs nothing real, since that grammar is the two shapes build-test produces. A command outside it is skipped and disclosed, the same treatment everything else this command cannot do already gets, so a future shape degrades to "judge it by the diff" rather than to arbitrary execution. * test(review): pin the empty-diff and collapse detectors Both lived inline in a large object literal with no test, and one of them is the most expensive disclosure this command emits: the SKILL acts on `emptyDiff` by recommending the PR be closed as superseded. Dropping the 200-line floor, flipping the 4x comparison, or losing either empty-diff guard turned nothing red. Extracted as two pure predicates and pinned at the boundaries, including the two guards that are not about the diff at all - a capture that threw leaves diffText empty, and a merge base resolved from a stale local ref can already contain the head commits. Either would have closed a live PR on an infrastructure error. Checked by mutation rather than by reading: all four edits above now fail exactly one assertion each. * test(review): pin the 30s budget floor, and stop keeping its arithmetic twice Two findings from the same review round, both about budget math nobody was checking. The 30s floor separates "skipped, judge it by the diff" from "started, timed out, disclosed as infrastructure". Every existing budget test left it non-binding - two sit hundreds of seconds from the boundary and the third is capped by the per-command deadline - so regressing it to the old 5s turned nothing red. Verified by doing exactly that: 26 green before the new test, one red after. A one-second command admitted into a twenty-second window comes back labelled `budgetClamped`, which is the mislabelling the priced floor exists to prevent. The other is the same duplication lesson this skill keeps arriving at, in its own tests: one test carried the per-run budget arithmetic in a block comment AND at the `now:` argument, and when the per-run figure changed only one copy moved. The block also predated the positive control, so it omitted a whole suite run. Deleted the copy rather than correcting it - two agreeing copies today are two disagreeing copies later. * fix(review): the working directory is a setting the extraction was losing Two findings, both of them this file failing its own stated argument. The stub list dropped it. `expressions` covered the script and the env and nothing else, so a `working-directory: ${{ github.workspace }}/x` produced an empty list and a summary line reading "0 ${{ }} site(s) to stub" - and `expressionsOf`'s own comment says why that is the failure this list cannot afford: the caller reads it as "these are all the values to supply". Widened to every setting the command carries, the `shell:` template included. The emitted script never mentioned it. The env block is commented into the header precisely so a reader of the script alone can see it; the working directory changes what the script does just as much and was in the metadata only. The argument for reading all three levels, written in this file, is that a step run "in the wrong directory, and nothing says so" is the transcription error the command exists to remove - which is exactly what the header did. It is a comment rather than a `cd` for the same reason env is comments, not exports: the value may hold `${{ … }}` and this command substitutes nothing. Both pinned, and both checked by deleting the fix: each mutation fails exactly one of the two new tests. * fix(review): $(( )) is arithmetic, and a heredoc's form decides where it ends Three fixes and one deliberate non-fix, all measured against this repo's own 434 `run:` steps rather than argued. `$(( ))` was read as a command substitution, so `N=$((N + 1))` reported `N` as a command to stub. It was the single largest source of junk in the list: 196 distinct "commands" across the corpus, 165 without it. A plain `<<WORD` heredoc ends only on a line that is exactly WORD; the loose match ended it on an indented `EOF` inside the body and then read the body as commands, which is how `rm` got reported for a script that never runs it. `<<-` stays looser than bash (any leading whitespace, not just tabs) because looser can only end a body early, and this file's priority is that an under-report is the worse direction. `[^()]*` matched only the innermost `$( )`, so `X=$(gh api $(u))` lost `gh` - a missed stub, and the extraction reaches the network. Depth counted now, and the assignment-prefix skip no longer steps over an unclosed `X=$(gh` into reading `api` as the command. The non-fix is recorded where the next reader will hit it: the quote walk is flat while shell quoting nests, and over ~300 lines the drift reports fragments of jq source as commands. Inserting a separator where a blanked span was removes nine of those, but it splits `a"X"b`, which is one word to the shell, and the minimal reproducer for the difference is 296 lines - nothing short enough to pin it. A scanner nobody can pin costs more than the junk it removes. * fix(review): refuse an ambiguous step name, and report errors like the siblings Two findings, both about a message the caller never gets. A job may legally hold two steps with the same name. The selector took the first and said nothing, which is the failure this file's own header names - "picks the same-named step from the wrong job" - and it is worst in the use the command exists for: A/B extraction runs it once per tree, so a PR that adds or reorders a duplicate leaves the two sides comparing different steps while reporting on one. Refused out loud now, naming the indices; the index is always available and never ambiguous. The handler also let every throw propagate, so five carefully separated messages - cannot read vs cannot parse vs no job vs no step vs no `run:` - all arrived as "An unexpected critical error occurred" under a stack trace. `base-tree` and `test-plan` in the same directory already catch, write the message, and set exit 1. Matched. The separation between "the path is wrong" and "the YAML is wrong" only pays if the caller sees it. * docs(review): the brief's list of limits was one short "Two limits worth knowing before you spend the step" became three when the selector started refusing an ambiguous step name, and the count went stale in the same commit that made it wrong. This is the enumeration drift the skill teaches, in the text that teaches it. The added entry says what to do rather than only what happens: pass the index, which is what an A/B wants regardless — the two trees have to select the same step, and a name that moved between them is exactly how they stop doing that. * chore(review): drop a scratch probe file that reached the branch `packages/cli/inert.mjs` was a throwaway harness for sweeping the repo's own workflows; its cleanup ran with a relative path from the wrong directory and it got committed by the next `git add -A`. It failed CI and not the local hook because the two lint different sets: lint-staged filters to `*.{js,jsx,ts,tsx}`, which does not include `.mjs`, while CI's flat config picks the file up regardless of `--ext`. Verified with CI's own command this time, not the hook's. * fix(review): the collapse disclosure needs the guard its sibling already has Three findings from the round on this branch, all accepted. `isEmptyDiff` refuses to rule when the merge base came from a possibly stale local ref, because such a base can already contain the head commits and diff to empty. The PARTIAL form of that cause lands on `isCollapsedFromUpstream` instead - a stale ref holding most of the head commits shrinks the recomputed diff past the 4x ratio - and the flag then tells Agent 0 to read the body as description-of-history when the body may be perfectly current and the real cause is a fetch failure. Two predicates over the same evidence, one guarded and one not. Guarded now, and pinned: removing the guard fails exactly one test. The pre-set of `mutantsSkippedForBudget`/`hunksSkippedForBudget` in the no-budget-for-control branch was dead in every path and worse than dead in one: both loops run with `harnessValidated` still null, re-check the same budget, and set their own counters - and the hunk loop's figure excludes the collocated-probe inconclusives it pushes first, which the pre-set did not. Dropped, with the reason where the assignment was. `runControlMutant`'s bound is now stated in its doc: it validates ONE file, so a collector that silently drops a different probe file passes the control while that file's survivors stand. The per-file baseline gate bounds what is left. A `true` there is read as covering the run, so what it does not cover belongs next to it. * fix(review): bound the whole ledger marker, not just each of its fields The per-field caps leave the total unbounded: fifty findings at full width serialize to just under 17,000 characters. The module's own paragraph calls the marker "a footnote, never a payload" and reasons about GitHub's 65,536-character body limit, so the total is the number that claim is about and nothing was computing it. Set the budget from measurement rather than from the limit. Across every review this pipeline has posted on its own stack (n=66) the body runs a median of 721 characters, p90 2,178, max 3,925 - the limit has ~61 KiB of headroom, and an over-long marker was never going to fail a post. The real cost is proportion: 17,000 characters of invisible payload on a 3,925-character review is four times more marker than review. 8 KiB holds fifty findings at realistic widths without dropping any, and the truncation path is reached only by a ledger no round has produced. What it drops travels with it. A `dropped` count is the difference between a list the next round reads as complete and one that says it is not - the same distinction this module already draws everywhere else. * fix(review): a partial ledger must not render as a complete one The size cap added in the previous commit can drop entries, and the rendered section still opened with "**Every entry below is owed a this-round ruling**" and said nothing about what was missing. The `dropped` count existed and reached the only place a reader sees the list as nothing at all — which is the failure it was added to prevent, committed one step after adding the field. It now says the list is partial, how many are missing, and that absence is not evidence a finding was fixed. Second, the table escaping escaped `|` but not `\`. `\|` in a title became `\\|`, which markdown reads as an escaped backslash followed by a LIVE separator: the forged row the escaping exists to prevent, produced by the escaping. Backslash first now, and the test counts separators the way markdown does rather than by pattern, because a lookbehind counter is fooled by exactly the input under test. * fix(review): an empty ledger title is a cap the next round cannot lift A comment that is nothing but its severity marker produced a ledger entry with an empty title. That does not merely degrade the entry, it jams the review: the next round is told every ledger entry is owed a this-round ruling, has no claim text to rule on, answers `cannot tell`, and `cannot-tell-existing-critical` is one of the cap reasons. Nothing between rounds changes, so the cap comes back every round. Dropping the entry would hide a Critical that really was posted, so it keeps its place and gets the one handle there is - the location it was filed at, or "the review body" for a body-level one. The field's stated job is "enough for the next round to re-locate the claim", and that is what a text-less finding still has. Found by probing `buildLedger` with ten malformed comment shapes; the other nine - carried ids, id collisions, two comments carrying the same id, `R2D2` not being an id, a leading newline before the marker - all behaved correctly and needed no change. * fix(review): count both caps as dropped, not just the byte one Found by running the three previous fixes together on one round: 51 findings in, 24 kept, and the marker said 26 were missing. `LEDGER_MAX_FINDINGS` truncates before the byte cap ever runs, and measuring `dropped` against the already-sliced list left the count cap's share silent - so the field added two commits ago to stop a truncated list reading as complete was itself under-reporting the truncation. Measured against what came IN now, and pinned across all four combinations: count cap alone, byte cap alone, both, and neither. Kept plus dropped equals given, in every one. * fix(review): the ignore check was the one git spawn without a deadline `isGitIgnored` reached for a bare `execFileSync` while every other git invocation in these commands goes through the package helper, which carries `GIT_TIMEOUT_MS` and `GIT_TERMINAL_PROMPT=0`. That constant's own comment is the argument: "a hang must still end". This one ran against a worktree the review does not control and could not. No behavioural delta for any input that does not hang - all 80 test-plan tests pass unchanged - so it ships without a new test rather than with a mocked-spawn assertion that would pin the call shape instead of the behaviour. * docs(review): a control killed mid-run did not "never run at all" SKILL's `harnessValidated` paragraph enumerated four ways the value lands on `null` - no green baseline, no candidates, no budget, an unreadable probe file - and all four are "the control never ran". There is a fifth: `runControlMutant` throws when its deadline kills the run or the runner cannot be spawned, the outer catch swallows it, and the value stays `null` rather than becoming a fabricated `false`. That is the path a slow machine takes. The ruling is unchanged - neither validated nor refuted, a survivor stands but unconfirmed - so this is about what the terminal says, not what it decides. An agent quoting "the control never ran at all" for a control that ran and was killed states something untrue about the run, and the enumeration is what it quotes from. --------- Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> Co-authored-by: Qwen Code <qwen-code@users.noreply.github.com> Co-authored-by: Qwen Code Bot <qwen-code-bot@users.noreply.github.com> Co-authored-by: Qwen Code Autofix <qwen-code-autofix@users.noreply.github.com>
|
Released in v0.21.3. |






Problem
The sandboxed-verification comment embeds
report.mdinside<details><pre><code>with full HTML escaping. Secure, but unreadable: the report is a curated bilingual markdown document — tables, headings, nested<details>folds — and the pre/code embedding displays it as a wall of raw source. #8140's verify comment is the exhibit: literal**asterisks, table pipes,<details>tag text, and&&remnants — what a reader perceives as 乱码.Change
report.mdnow renders as markdown via a newemit_report, holding the same security floor with four line-independent guarantees:& < >is escaped, then only the structural tags the report legitimately uses (details/summary/pre/code/br) are un-escaped back to live tags. No other tag can form:<img>,<script>,onerrornever render.<!--breaker), so no forgedqwen-triage:*marker can appear in the raw body that the upsert/snapshot logic greps.@becomes@: renders identically, can never ping.<details>opens are counted and closed at the end, so a malformed report cannot swallow the comment footer.Fallbacks: an oversized report (>45 KB) goes through the old escaped-
preembedding wholesale — truncated markdown dangles fences and folds — and so does any sanitizer pipeline failure. The tmux lane's raw-log embedding is untouched (escaped pre remains the right shape for logs).One robustness fix surfaced by the test suite: the fold-balancer's zero-match
grepcarries|| true, since under the step'spipefaila report with no folds would otherwise kill the whole composer.Tests
A behavioral replay drives the real
emit_reportagainst a structural + hostile fixture:<pre><code>on the normal path;<!-- qwen-triagein the raw output, no@everyone, no<img>, entities escaped, only allowlisted tags present, folds balanced;emit_reportwhile the tmux lane keepsemit_block.Full-render ordering pin updated to the new
### Verification reportheading. 108/108 pass; workflow YAML parses.中文说明
问题
沙箱验证评论把
report.md整体 HTML 转义后塞进<details><pre><code>。安全,但不可读:报告本身是精排的双语 markdown 文档(表格、标题、嵌套折叠),pre/code 嵌入让它以纯源码形态显示——#8140 的验证评论即实例:裸露的**星号、表格管道符、<details>标签文本、&&实体残留,读者看到的就是乱码。改动
report.md改经新的emit_report以 markdown 渲染,通过四条与行无关的保证维持同等安全底线:标签白名单(全量转义后仅回解details/summary/pre/code/br,其余标签永远无法成形);标记伪造打断(沿用 autofix 已验证的<!--中和手法,upsert 逻辑 grep 的原始 body 中不可能出现伪造的qwen-triage:*标记);提及熄火(@→@,显示相同、永不触发);折叠配平(统计并补齐未闭合的<details>,畸形报告吞不掉页脚)。回退:超限(>45 KB)报告整体走旧的转义 pre 嵌入(截断的 markdown 会悬挂围栏与折叠),净化管道失败同样回退。tmux 原始日志嵌入保持不动(对日志而言转义 pre 本就是正确形态)。
测试套件揪出一个健壮性问题:折叠配平的零匹配
grep需要|| true——在步骤的pipefail下,无折叠的报告会导致整个组装器退出。测试
行为回放驱动真实
emit_report(结构+敌意混合 fixture):结构存活(表格/加粗/嵌套折叠渲染、常规路径无 pre/code);安全底线成立(原始输出无活标记/提及/标签、实体转义、折叠配平);超限回退到转义形态;报告调用点用emit_report而 tmux 通道保持emit_block。全渲染顺序 pin 更新为新标题。108/108 通过;YAML 解析正常。