feat(review): carry the round ledger in the posted review body - #8255
Merged
Conversation
… A/B, per-hunk probes - qwen review test-plan: rule on the PR Test Plan's checkable claims (paths, npm scripts, test counts) against the reviewed tree; contradictions and differing counts are disclosed via compose-review, never capping. - qwen review base-tree: build the merge base in a sibling worktree so the verifier can A/B a comparative claim instead of reading it; swept by cleanup. - test-efficacy: third probe kind — reverse-apply one hunk at a time and re-run the affected tests, attributing a still-green suite to the specific change nothing gates; shares the mutants' budget window, runs last.
… summaries Both measured on a live /review of #8176 with the built CLI: - test-plan's observedTestCounts strips SGR sequences before matching; a color-enabled pipe interleaves them BETWEEN tokens, and the count claim fell to 'unchecked' with the summary right there in the report. - build-test's trimOutput rescues runner summary lines from the omitted middle (like module-resolution errors): a failing suite's tail is all failure details and npm epilogue, which pushed the one-line summary out of the kept text entirely.
All measured in the review (#8215 review comment): - test-plan: linear-time bold-heading scan (the old pattern backtracked catastrophically on an unclosed ** line an untrusted PR body controls); a flag preceding the npm script yields no claim instead of a false 'no package defines this script'. - test-efficacy: a hunk probe's restore recreates the parent directory a reverse-applied 'new file' hunk removed (the ENOENT from finally lost the verdict and marked every remaining hunk inconclusive); hunks get their own skippedForBaseline instead of mislabeling a red baseline as a budget skip; splitDiffIntoHunks re-captures the file header at every diff --git boundary; a hunk-survived finding notes when it restates an inert file-level revert at hunk granularity. - base-tree: idempotent fast path keyed on a build marker + HEAD check — concurrent verifier shards reuse one built tree instead of sweeping it out from under each other mid-A/B (a fabricated base-side difference with a deterministic source tag was the worst case); cost wording is now 'an install and a build' everywhere it was 'one extra build'.
…line (#8215) A per-hunk probe reported `survived` whenever the green baseline probes still passed with the hunk reverted. When the hunk's own collocated test dropped out of the baseline (a probe-tree import error collects nothing), the remaining green probes prove only that THEY do not cover the hunk, so the verdict is now `inconclusive` — the same dropped-test asymmetry the mutants already hold. Also scope the hunk-survived cross-reference note to the hunk's own collocated test, and let test-plan match a workspace-scoped run of the plan's bare command instead of falling through to the manifest on an exact-string miss.
…his branch The two blocking findings, reproduced on this PR's own Test Plan: - test-plan files no false contradicted notes: npm rulings move from a four-verb denylist to an allowlist (the run form + npm's script aliases — the ~fifty other builtins each used to become 'no package defines this script'); a slash token is claimed as a repo path only with evidence (an extension or ./ prefix), never when it is a flag's value (--repo owner/repo) or under the review's own temp root; HEADING_LINE_RE drops the same quadratic shape its bold sibling was rewritten to remove. - base-tree gets a real mutual-exclusion lock around sweep+add+build (mkdirSync test-and-set; the loser returns busy instead of deleting the tree the winner is mid-install in), and a failed build writes a settled marker so later shards stop re-paying the install to relearn 'unavailable'. Also: Agent 7's brief now names hunk-survived and the hunks.* counters (it is the report's only consumer, and the finding class was invisible); hunk findings anchor at the first ADDED line instead of up to three context lines above the change.
… round ledger, richer mutants, doc parity Shaped by a live round-2 maintainer re-verification (#7818): - qwen review test-delta: rerun the PR side's failed test commands on the built merge base and diff the failing FILE SETS — netNew is the PR's own failure by measurement (whatever files the diff touches), shared is pre-existing by measurement. Counts are never compared: a flaky suite fails different test names between runs of the same tree. An unfinished base rerun attributes nothing. - Round ledger: the incremental review cache persists confirmed findings under round-scoped ids (R1-2); a high-effort re-review rules on each (fixed / still stands / cannot tell) and opens its report with the table, the way a human round-2 comment opens with 'M1 is fixed'. - Three replacement mutation operators in test-efficacy: drop '?? fallback', force a comparison-bearing guard condition to 'true', drop a '+ CONST' term. Each survivor maps to one crisp untested-protection sentence. A line whose raw text and literal-blanked code view disagree yields no candidate — an edit index computed across the two views once spliced 'iftrue 0)' into a guard, and a mangled mutant reads as inconclusive while quietly spending a cap slot. - Quality brief: documentation-parity lens — a new user-facing surface whose siblings are documented is a Suggestion naming the sibling precedent; no documented sibling, no finding.
…rification lenses Shaped by a live sanitizer-PR verification (#8147): - Render-adjudication capability (opt-in): with QWEN_REVIEW_SCRATCH_REPO set, the verifier may post a minimal payload to that user-designated repo and rule on GitHub's own rendered HTML — the measured case being an @ -> @ defusal every local reading called sound while GitHub's real renderer registered the mention and fired the notification. Absent the setting, rendering claims honestly cap at low confidence / cannot tell. Step 7's write ban names the carve-out explicitly. - qwen review extract-step: lift one workflow step's run: script verbatim into an executable, with env (as comments, never half-substituted exports), every ${{ }} site listed unevaluated, and a heuristic invoked- command list as the stubbing starting point. With base-tree, both arms of a by-hand workflow A/B become two invocations. yaml declared as a cli dependency (previously resolved only via hoisting). - Three brief lenses: a borrowed protection idiom missing what made it work at home (the code ancestor did the protecting; only the entity was copied); a second parser for an authoritatively-parsed format is a divergence hunt; tests that pin the mechanism instead of the effect, and oracles that mirror the implementation's own model.
From round 6 of the live sanitizer verification (#8147): the fix closed the fence-shaped entrance into a raw-HTML block, and the code-span entrance beside it — same divergence, adjacent syntax — stayed open. A re-check that tests only the reported input rules 'fixed' over a hole one backtick away. Both fixed-verdict sites in SKILL Step 6 (the open-Criticals re-check and the round ledger) now require enumerating a divergence-class defect's sibling entrances before ruling fixed; a still-open sibling is a NEW finding, never a reason to withhold the original's fixed — the two rulings stay separate so the second hole cannot ship unreviewed.
…7/8005 - Threshold-boundary scan: when a fix is a ratio/length guard, hold the issue's own variables fixed and binary-search the boundary where the behaviour flips; put the number next to what the issue reports. Live case: a prose-ratio guard covered the edit/write_file half of its issue and silently declined the run_shell_command half (~473-char boundary). - Delimiter self-injection named as the first parser-differential probe: a no-escaping extractor fed its own close tag truncates silently. - Shared-gate state enumeration: a deliberate-design defence extends only to the states it argues — an input-hold argued for 'active' silently froze three idle states sharing the same gate. The sibling-entrance rule, applied to a state machine instead of a syntax.
The ledger shipped as a local cache file and its first multi-round live use exposed the flaw: four model-comparison rounds re-reviewed the same two PRs and every round opened from scratch — medium never reads the cache, and from CI or another clone there is no cache at all. The one artifact every environment can see, the posted review, carried nothing machine-readable. - compose-review embeds an HTML-comment marker (invisible on GitHub) carrying this round's findings and round number; the round comes from the side file pr-context recovered, +1 — never from the model. - pr-context recovers the reviewing account's latest marker, renders it as a 'Previous /review round (machine ledger)' work-list section, and writes it beside the context file for compose-review's round counter. Own-account only, latest round wins, fail-quiet on any malformation — the marker is data the next round rules on, never authority it obeys. - Step 6's previous-round ruling now runs at medium too (recovery is free; medium still writes no cache and posts nothing); the local cache is demoted to fallback for rounds that never posted.
wenshao
requested review from
LaZzyMan,
tanzhenxin and
yiliang114
as code owners
July 31, 2026 17:52
Collaborator
Collaborator
|
Re-running the gate on the current head.
Moving on to code review. 🔍 中文说明在当前 head 上重跑 gate。
进入代码审查 🔍 — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
Code reviewNo blockers, no convention violations. This is unusually well-built. My independent proposal for "make the round ledger survive across environments" was exactly what landed — embed a machine-readable copy in the posted body, recover it own-account-only next round — so I spent the review trying to break the safety model rather than rethink the approach. What I verified:
The capping machinery (per-field caps + an 8 KiB whole-marker cap + a The cross-round round-trip the feature introduces: sequenceDiagram
participant P1 as compose-review
participant P2 as GitHub PR reviews
participant P3 as pr-context
participant P4 as prev-ledger side file
P1->>P4: read previous round number
P1->>P1: buildLedger from posted comments
P1->>P2: post review body with ledger marker
P3->>P2: fetch reviews next round
P3->>P3: latestOwnLedger own-account only
P3->>P4: write recovered ledger
P3->>P3: render previous-round work list
Test evidence (this commit's own CI)The load-bearing unit suite is green on the reviewed commit; windows/macos/integration jobs are skipped (matrix-conditional), not failed. The remaining in-flight check-runs (
The round-trip behaviour itself is pinned by unit tests that drive the real 中文说明代码审查无阻塞项，无规范违反。这个 PR 构建得异常扎实。我对"让轮次台账跨环境存续"的独立方案与最终落地完全一致——在发布正文嵌入机器可读副本、下一轮仅本账号恢复——所以我把审查花在尝试击穿安全模型，而非重新审视方案。已验证：
上限机制（字段级上限 + 8 KiB 整体 marker 上限 + 测试证据（本 commit 自身 CI）承载性的单元测试套件在被审 commit 上为绿；windows/macos/integration 作业为跳过（矩阵条件），非失败。其余在跑 check-run（ — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
|
Confidence: 5/5 — clean across every stage; would merge without hesitation. Stepping back: my independent proposal for this problem was the same primitive the PR ships — carry the authoritative ledger in the posted review body, recover it own-account-only next round, demote the cache to fallback — so there was no simpler path I felt the author missed. The security model is the part that could have been hand-waved and wasn't: the marker is treated as data to re-rule, never authority to obey; parsing is fail-quiet against a writable surface; the round number and the ledger contents both come from CLI-controlled inputs, never the model or the caller's state JSON; and the HTML-comment and markdown-table injection paths are escaped and tested. Every change in the diff is load-bearing — I looked for scope creep or a drive-by refactor and didn't find one. The capping machinery is the one thing that could read as over-built, and it isn't: it's justified against a measured body-size distribution and The unit suite is green on the reviewed commit and, importantly, it pins the change — the assertions run through the real If I'm maintaining this in six months I'd thank the author — the tests each document the exact failure they prevent, which is how this code stays maintainable. Approving, pinned to the reviewed commit. ✅ 中文说明置信度：5/5 —— 每个阶段都干净，毫不犹豫地合并。 退一步看：我对这个问题的独立方案与 PR 交付的原语相同——把权威台账放进发布的 review 正文、下一轮仅本账号恢复、缓存降级为回退——因此没有感到作者遗漏更简路径。安全模型是本可敷衍却没有敷衍的部分：marker 被当作待重新裁定的数据，绝非待服从的权威；对可写表面的解析静默失败；轮次编号与台账内容都来自 CLI 控制的输入，绝不来自模型或调用方 state JSON；HTML 注释与 markdown 表格注入路径都被转义且有测试。diff 中每处改动都承重——我找过 scope creep 或顺手重构，没有发现。上限机制是唯一可能显得过度构建的部分，而它并非如此：有针对实测正文尺寸分布的依据，且 单元测试套件在被审 commit 上为绿，且关键的是它钉住了改动——断言走真实的 若六个月后由我维护，我会感谢作者——每个测试都记录了它所阻止的确切失败，这正是这段代码保持可维护的方式。批准，钉在被审 commit 上。✅ — Qwen Code · qwen3.8-max-preview Reviewed at |
qwen-code-ci-bot
approved these changes
Jul 31, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
added 2 commits
July 31, 2026 19:18
…ry diff guard, error convention (#8215)
…est-plan false positives, hunk-probe ranges (#8215) - base-tree: only stamp a base tree available when runBuildTest actually compiled something (ok AND npm toolchain AND a non-empty build). An `unsupported` handoff or an empty npm scope returns ok:true having built nothing; marking that tree available let an A/B read the absence of a build as a behavioural difference. - cleanup: sweep the stale base-tree build lock a killed builder leaves behind. - test-plan: read the root manifest's scripts directly so a root-only script survives when the root defines no build/test; bail on the inline --root=./dir rebasing form; stop treating a positional after an inline --flag=value as the flag's value; prefer a failed scoped run when ruling a bare command; anchor the npm script alias to a full token so `yarn test:unit` is not truncated. - test-efficacy: exclude `\ No newline at end of file` from the startLine offset count; compute the mutant-overlap range from the header's new-side span so it no longer overshoots into a closely following hunk.
Collaborator
Author
OverviewMoves the round ledger's authoritative copy from a local cache file into the posted review body, as an invisible HTML-comment marker:
The design is well argued and the parse half is genuinely well built — treated as an untrusted-input boundary, fail-quiet on every malformation, own-account-and-latest-only, size-capped, stripped before any body reaches a model as prose. The But the write half does not reach the PR. Details below. 🔴 Critical — the marker never reaches the posted body; the feature is inert end to end
// compose-review.ts:1637 (inside composeReviewCommand, which starts at :1555)
const result = composeReview({ ...parsed, ...countInlineFindings(drafted) });
...
result.body = `${result.body}\n\n${serializeLedger(ledger)}`; // :1678But // submit.ts:299 — unchanged by this PR
const r = composeReview({ ...rest, criticalsInline, suggestionsInline });
return { event: r.event, body: r.body, cappedBy: r.cappedBy }; // no markerAnd // submit.ts:316
if (payload.event !== undefined || payload.body !== undefined) {
problems.push('the payload carries `event`/`body`. Those are computed here…');
}
This is exactly where the PR body's "no live posted round has round-tripped the marker end to end yet" is hiding. Posting once with There is a second, independent consequence worth naming: after this change the body in Suggested fix — put the embed where both callers reach it. // lib/ledger.ts
export function withLedgerMarker(
body: string,
planPath: string | undefined,
drafted: DraftedComment[],
bodyCriticals: string[],
): string { /* the block currently at compose-review.ts:1640-1684 */ }then call it from both 🟡 Suggestions1.
Worth noting the 2. Stripping after the worth-showing filter defeats the canonical-LGTM filter for every review this pipeline posts. // pr-context.ts — filter sees the raw body…
const meaningfulReviews = reviews.filter((r) => isReviewWorthShowing(r.body))
// …but rendering strips
parts.push(quoteBlock(fullBody(stripLedgerMarker(r.body ?? ''), r.id, ctx)));
(Same call site, minor: 3. `| ${f.id} | … | \`${f.file}${…}\` | ${f.title} |`A perfectly ordinary finding title — 4. if (!best || at > best.at) best = { at, ledger };GitHub returns reviews in ascending 5. Round counter can silently reset to 1, reusing ids.
Things that are right
VerdictCritical #1 blocks: as written, nothing this PR posts is recoverable, so neither half of the round trip runs in production. It's a small fix (share the embed between the two callers) plus one test across the compose→submit seam — and one live 中文说明概述把轮次台账的权威副本从本地缓存文件搬到发布的 review 正文里，以不可见的 HTML 注释 marker 承载。解析侧做得相当扎实：按不可信输入边界处理、任何畸形都静默失败、仅本账号+仅最新、尺寸受限、喂给模型前先剥离。 但写入侧到不了 PR。 🔴 阻断问题：marker 永远进不了发布的正文
// submit.ts:299
const r = composeReview({ ...rest, criticalsInline, suggestionsInline });
return { event: r.event, body: r.body, cappedBy: r.cappedBy }; // 没有 marker而且 结果：所有发布的 review 都不带 marker → PR 描述里那句“尚无已发布轮次端到端跑通 marker 往返”正是它藏身之处——只要带 还有一个独立的后果：改动之后 建议修法：把嵌入逻辑抽成 🟡 建议
结论Critical #1 阻断：照现状，本 PR 发布出去的东西没有任何可恢复内容，往返的两半在生产路径上都不会运行。修起来不大——把嵌入逻辑在两个调用方之间共享，加一个跨 compose→submit 接缝的测试，再带 |
…verification # Conflicts: # packages/cli/src/commands/review/test-efficacy.integration.test.ts
…r seam Merging main brought #8050's Windows-portability refactor, which resolves the probe runner through vitest/package.json's bin — a node_modules/.bin fake is dead weight it never reads. The 8215-only collocated-dropout test still installed the old .bin fake, so the REAL vitest ran its fixtures, price.test.ts genuinely passed, and the hunk scored survived. The test now overrides the fake package's vitest.mjs like every post-refactor test.
The per-field caps leave the total unbounded: fifty findings at full width serialize to just under 17,000 characters. The module's own paragraph calls the marker "a footnote, never a payload" and reasons about GitHub's 65,536-character body limit, so the total is the number that claim is about and nothing was computing it. Set the budget from measurement rather than from the limit. Across every review this pipeline has posted on its own stack (n=66) the body runs a median of 721 characters, p90 2,178, max 3,925 - the limit has ~61 KiB of headroom, and an over-long marker was never going to fail a post. The real cost is proportion: 17,000 characters of invisible payload on a 3,925-character review is four times more marker than review. 8 KiB holds fifty findings at realistic widths without dropping any, and the truncation path is reached only by a ledger no round has produced. What it drops travels with it. A `dropped` count is the difference between a list the next round reads as complete and one that says it is not - the same distinction this module already draws everywhere else.
yiliang114
previously approved these changes
Aug 1, 2026
yiliang114
left a comment
yiliang114
left a comment
Collaborator
There was a problem hiding this comment.
LGTM. The ledger-as-HTML-comment design is well-bounded and the fail-quiet parse is the right contract for an untrusted-input surface.
Two non-blocking notes:
-
buildLedgerpasses through negativelinevalues (typeof c.line === 'number'includes-1). Harmless since it's re-location metadata only, but ac.line > 0guard would keep the ledger's shape honest — a negative line number is never a real location. -
ledgerMarkerForre-reads and re-parses the plan file thatcomposeReviewBodyalready consumed. Not a correctness issue (the function is called once per review), but threading theprNumberthrough from the already-parsed plan would remove the redundant I/O and the secondtry/catchsurface.
Neither blocks — the serialization round-trip is lossless, the size caps bind both halves, and the carried-id logic is thoroughly tested.
The size cap added in the previous commit can drop entries, and the rendered section still opened with "**Every entry below is owed a this-round ruling**" and said nothing about what was missing. The `dropped` count existed and reached the only place a reader sees the list as nothing at all — which is the failure it was added to prevent, committed one step after adding the field. It now says the list is partial, how many are missing, and that absence is not evidence a finding was fixed. Second, the table escaping escaped `|` but not `\`. `\|` in a title became `\\|`, which markdown reads as an escaped backslash followed by a LIVE separator: the forged row the escaping exists to prevent, produced by the escaping. Backslash first now, and the test counts separators the way markdown does rather than by pattern, because a lookbehind counter is fooled by exactly the input under test.
A comment that is nothing but its severity marker produced a ledger entry with an empty title. That does not merely degrade the entry, it jams the review: the next round is told every ledger entry is owed a this-round ruling, has no claim text to rule on, answers `cannot tell`, and `cannot-tell-existing-critical` is one of the cap reasons. Nothing between rounds changes, so the cap comes back every round. Dropping the entry would hide a Critical that really was posted, so it keeps its place and gets the one handle there is - the location it was filed at, or "the review body" for a body-level one. The field's stated job is "enough for the next round to re-locate the claim", and that is what a text-less finding still has. Found by probing `buildLedger` with ten malformed comment shapes; the other nine - carried ids, id collisions, two comments carrying the same id, `R2D2` not being an id, a leading newline before the marker - all behaved correctly and needed no change.
Found by running the three previous fixes together on one round: 51 findings in, 24 kept, and the marker said 26 were missing. `LEDGER_MAX_FINDINGS` truncates before the byte cap ever runs, and measuring `dropped` against the already-sliced list left the count cap's share silent - so the field added two commits ago to stop a truncated list reading as complete was itself under-reporting the truncation. Measured against what came IN now, and pinned across all four combinations: count cap alone, byte cap alone, both, and neither. Kept plus dropped equals given, in every one.
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
Collaborator
Author
|
@qwen-code /triage |
yiliang114
approved these changes
Aug 1, 2026
yiliang114
left a comment
yiliang114
left a comment
Collaborator
There was a problem hiding this comment.
LGTM on re-review. The four Copilot findings are all fixed correctly:
- Table cell escaping: backslash-first ordering is right (pipe-first would produce
\\|→ escaped backslash + live separator). TheliveSeparatorstest is a good oracle. - Side-file write:
mkdirSync+ try/catch preserves best-effort semantics. parseLedgerlast-marker:lastIndexOfis the correct choice for edited/quote-carrying bodies.- Doc comment reattached.
Also verified: stripLedgerMarker is applied at every model-facing render site (worth-showing filter, fullBody quoteBlock, meaningful count) — no path leaks the JSON blob as prose.
Collaborator
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 68 passed · 0 failed · 68 total 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：68 通过 · 0 失败 · 68 总计 Verification report/verify-pr 8255 — deep verificationVerdict:
中文摘要结论：
Central claim + A/BCentral claim: the The A/B drives the real compiled
Witness: Changed-surface harnesses (mock-free, compiled dist)
Vacuity / gate liveness
Both load-bearing hunks are pinned by behavioural assertions (expected-vs-actual, not import/compile errors); 2/2 mutants killed. Targeted gate ( Corrections (to the PR description, not code-change requests)
FindingsNo blocking findings. Two informational notes:
Not covered
MethodologyRan in the CI verify container ( Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention). — Qwen Code · sandboxed verification |
doudouOUC
approved these changes
Aug 1, 2026
doudouOUC
left a comment
doudouOUC
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed at 24c6d3e. Confirmed first that #8225/#8218/#8215 are all merged and are ancestors of this head, so 1be199ae..24c6d3e is exactly this PR's delta and the stack does not inflate the range.
The --> containment I verified by execution, not by reading. serializeLedger neutralises with JSON.stringify(payload).replace(/--/g, '-\\u002d'), which is subtle enough to be worth probing: I imported the real module and drove 48 adversarial cases — --, ---, ----, -->, --->, a-->b, " -->", a pre-escaped \u002d\u002d, a--b--c--d, a nested fake marker, and a 50-char prefix ending in -->, each injected into title, file, and id in turn. Result: no --> and no bare -- survived into any payload, every case round-tripped losslessly through parseLedger, and every case was removed by stripLedgerMarker. The odd-dash case is the one that would break a naive implementation (--- → -\u002d-, no adjacent dashes) and it holds. So the marker cannot terminate its own comment and spill JSON onto the PR page.
Own-account enforcement is anchored to the authenticated identity, not a spoofable field. latestOwnLedger skips on r.user?.login !== login, and the login it is given comes from currentUser() → gh('api', 'user', '--jq', '.login') — the token's own account, not a display name and not anything from the PR payload. Recovery also scans reviews only, so a crafted marker in an inline comment or reply carries no authority: it cannot inflate the round counter, cannot reach the side file, and cannot forge a work-list. Latest-marker selection is submitted_at with the review id as tiebreak, so array order and a same-second re-submit both resolve deterministically.
Round-counter integrity holds. The number is prevRound + 1 where prevRound comes only from qwen-review-pr-<n>-prev-ledger.json, and the model cannot reach it — submit's compose explicitly destructures draftedComments out of the caller-supplied state and refills it from payload.comments, so the ledger describes what is actually being posted. A forged state.draftedComments is dropped, and there is a test for exactly that. The missing/corrupt side-file fallback restarts at 1, and pr-context deliberately does not clear a stale side file on a recovery failure, which keeps the drift in the safe direction (the count can only advance, never collide with an already-used id space).
One factual note rather than a finding, since I checked it and it does not bite: two sites render bodies into the context file without stripLedgerMarker — pr-context.ts:607 and :881, both snippetWithRef(...) over review-thread roots and replies. That is correct as written, because the marker is appended only to the review body (compose-review.ts:262, { ...result, body: \${result.body}\n\n${marker}` }`) and never to an inline comment, so our own tooling cannot leak there. The only way a marker appears at those two sites is a third party typing one into an inline comment, where it renders as visible JSON inside an already-untrusted quoted snippet and carries no authority. Worth knowing if inline comments ever start carrying markers; nothing to change now.
The rendered work-list is also escape-hardened where it matters: cell() escapes backslash before pipe (the ordering that makes \\| an actual escape rather than a literal backslash plus a live separator) and flattens newlines, code() additionally replaces backticks so a crafted path cannot close its code span — with tests asserting the live-separator count stays at 5 across a | b, back\| slash, and a trailing backslash.
Ran the four touched test files locally at this HEAD: 270/270. (Note for reproduction: the default coverage provider hits EPERM on the sandboxed temp dir here — --coverage.enabled=false is needed locally; unrelated to this PR.) CI green apart from review-pr still running. No findings.
qwen-code-ci-bot
approved these changes
Aug 1, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
pchaganti
pushed a commit
to pchaganti/qx-qwen-code
that referenced
this pull request
Aug 1, 2026
…rol, seven lenses (QwenLM#8261) * feat(review): borrowed-verification trio — test-plan check, base-tree A/B, per-hunk probes - qwen review test-plan: rule on the PR Test Plan's checkable claims (paths, npm scripts, test counts) against the reviewed tree; contradictions and differing counts are disclosed via compose-review, never capping. - qwen review base-tree: build the merge base in a sibling worktree so the verifier can A/B a comparative claim instead of reading it; swept by cleanup. - test-efficacy: third probe kind — reverse-apply one hunk at a time and re-run the affected tests, attributing a still-green suite to the specific change nothing gates; shares the mutants' budget window, runs last. * fix(review): survive real runner output — ANSI-laced and trimmed-away summaries Both measured on a live /review of QwenLM/qwen-code#8176 with the built CLI: - test-plan's observedTestCounts strips SGR sequences before matching; a color-enabled pipe interleaves them BETWEEN tokens, and the count claim fell to 'unchecked' with the summary right there in the report. - build-test's trimOutput rescues runner summary lines from the omitted middle (like module-resolution errors): a failing suite's tail is all failure details and npm epilogue, which pushed the one-line summary out of the kept text entirely. * fix(review): address the eight findings from live review of this branch All measured in the review (QwenLM/qwen-code#8215 review comment): - test-plan: linear-time bold-heading scan (the old pattern backtracked catastrophically on an unclosed ** line an untrusted PR body controls); a flag preceding the npm script yields no claim instead of a false 'no package defines this script'. - test-efficacy: a hunk probe's restore recreates the parent directory a reverse-applied 'new file' hunk removed (the ENOENT from finally lost the verdict and marked every remaining hunk inconclusive); hunks get their own skippedForBaseline instead of mislabeling a red baseline as a budget skip; splitDiffIntoHunks re-captures the file header at every diff --git boundary; a hunk-survived finding notes when it restates an inert file-level revert at hunk granularity. - base-tree: idempotent fast path keyed on a build marker + HEAD check — concurrent verifier shards reuse one built tree instead of sweeping it out from under each other mid-A/B (a fabricated base-side difference with a deterministic source tag was the worst case); cost wording is now 'an install and a build' everywhere it was 'one extra build'. * fix(cli): never score a hunk survived when its own test left the baseline (#8215) A per-hunk probe reported `survived` whenever the green baseline probes still passed with the hunk reverted. When the hunk's own collocated test dropped out of the baseline (a probe-tree import error collects nothing), the remaining green probes prove only that THEY do not cover the hunk, so the verdict is now `inconclusive` — the same dropped-test asymmetry the mutants already hold. Also scope the hunk-survived cross-reference note to the hunk's own collocated test, and let test-plan match a workspace-scoped run of the plan's bare command instead of falling through to the manifest on an exact-string miss. * fix(review): silence-bias hardening from four live review rounds of this branch The two blocking findings, reproduced on this PR's own Test Plan: - test-plan files no false contradicted notes: npm rulings move from a four-verb denylist to an allowlist (the run form + npm's script aliases — the ~fifty other builtins each used to become 'no package defines this script'); a slash token is claimed as a repo path only with evidence (an extension or ./ prefix), never when it is a flag's value (--repo owner/repo) or under the review's own temp root; HEADING_LINE_RE drops the same quadratic shape its bold sibling was rewritten to remove. - base-tree gets a real mutual-exclusion lock around sweep+add+build (mkdirSync test-and-set; the loser returns busy instead of deleting the tree the winner is mid-install in), and a failed build writes a settled marker so later shards stop re-paying the install to relearn 'unavailable'. Also: Agent 7's brief now names hunk-survived and the hunks.* counters (it is the report's only consumer, and the finding class was invisible); hunk findings anchor at the first ADDED line instead of up to three context lines above the change. * feat(review): four round-2 borrowings — measured failure attribution, round ledger, richer mutants, doc parity Shaped by a live round-2 maintainer re-verification (QwenLM/qwen-code#7818): - qwen review test-delta: rerun the PR side's failed test commands on the built merge base and diff the failing FILE SETS — netNew is the PR's own failure by measurement (whatever files the diff touches), shared is pre-existing by measurement. Counts are never compared: a flaky suite fails different test names between runs of the same tree. An unfinished base rerun attributes nothing. - Round ledger: the incremental review cache persists confirmed findings under round-scoped ids (R1-2); a high-effort re-review rules on each (fixed / still stands / cannot tell) and opens its report with the table, the way a human round-2 comment opens with 'M1 is fixed'. - Three replacement mutation operators in test-efficacy: drop '?? fallback', force a comparison-bearing guard condition to 'true', drop a '+ CONST' term. Each survivor maps to one crisp untested-protection sentence. A line whose raw text and literal-blanked code view disagree yields no candidate — an edit index computed across the two views once spliced 'iftrue 0)' into a guard, and a mangled mutant reads as inconclusive while quietly spending a cap slot. - Quality brief: documentation-parity lens — a new user-facing surface whose siblings are documented is a Suggestion naming the sibling precedent; no documented sibling, no finding. * feat(review): render adjudication, workflow step extraction, three verification lenses Shaped by a live sanitizer-PR verification (QwenLM/qwen-code#8147): - Render-adjudication capability (opt-in): with QWEN_REVIEW_SCRATCH_REPO set, the verifier may post a minimal payload to that user-designated repo and rule on GitHub's own rendered HTML — the measured case being an @ -> @ defusal every local reading called sound while GitHub's real renderer registered the mention and fired the notification. Absent the setting, rendering claims honestly cap at low confidence / cannot tell. Step 7's write ban names the carve-out explicitly. - qwen review extract-step: lift one workflow step's run: script verbatim into an executable, with env (as comments, never half-substituted exports), every ${{ }} site listed unevaluated, and a heuristic invoked- command list as the stubbing starting point. With base-tree, both arms of a by-hand workflow A/B become two invocations. yaml declared as a cli dependency (previously resolved only via hoisting). - Three brief lenses: a borrowed protection idiom missing what made it work at home (the code ancestor did the protecting; only the entity was copied); a second parser for an authoritatively-parsed format is a divergence hunt; tests that pin the mechanism instead of the effect, and oracles that mirror the implementation's own model. * feat(review): sibling-entrance discipline for the fixed verdict From round 6 of the live sanitizer verification (QwenLM/qwen-code#8147): the fix closed the fence-shaped entrance into a raw-HTML block, and the code-span entrance beside it — same divergence, adjacent syntax — stayed open. A re-check that tests only the reported input rules 'fixed' over a hole one backtick away. Both fixed-verdict sites in SKILL Step 6 (the open-Criticals re-check and the round ledger) now require enumerating a divergence-class defect's sibling entrances before ruling fixed; a still-open sibling is a NEW finding, never a reason to withhold the original's fixed — the two rulings stay separate so the second hole cannot ship unreviewed. * feat(review): three measured-verification lenses from live rounds 8037/8005 - Threshold-boundary scan: when a fix is a ratio/length guard, hold the issue's own variables fixed and binary-search the boundary where the behaviour flips; put the number next to what the issue reports. Live case: a prose-ratio guard covered the edit/write_file half of its issue and silently declined the run_shell_command half (~473-char boundary). - Delimiter self-injection named as the first parser-differential probe: a no-escaping extractor fed its own close tag truncates silently. - Shared-gate state enumeration: a deliberate-design defence extends only to the states it argues — an input-hold argued for 'active' silently froze three idle states sharing the same gate. The sibling-entrance rule, applied to a state machine instead of a syntax. * feat(review): carry the round ledger in the posted review body The ledger shipped as a local cache file and its first multi-round live use exposed the flaw: four model-comparison rounds re-reviewed the same two PRs and every round opened from scratch — medium never reads the cache, and from CI or another clone there is no cache at all. The one artifact every environment can see, the posted review, carried nothing machine-readable. - compose-review embeds an HTML-comment marker (invisible on GitHub) carrying this round's findings and round number; the round comes from the side file pr-context recovered, +1 — never from the model. - pr-context recovers the reviewing account's latest marker, renders it as a 'Previous /review round (machine ledger)' work-list section, and writes it beside the context file for compose-review's round counter. Own-account only, latest round wins, fail-quiet on any malformation — the marker is data the next round rules on, never authority it obeys. - Step 6's previous-round ruling now runs at medium too (recovery is free; medium still writes no cache and posts nothing); the local cache is demoted to fallback for rounds that never posted. * fix(review): address review feedback — false-positive hardening, binary diff guard, error convention (#8215) * fix(review): address review feedback — base-tree availability gate, test-plan false positives, hunk-probe ranges (#8215) - base-tree: only stamp a base tree available when runBuildTest actually compiled something (ok AND npm toolchain AND a non-empty build). An `unsupported` handoff or an empty npm scope returns ok:true having built nothing; marking that tree available let an A/B read the absence of a build as a behavioural difference. - cleanup: sweep the stale base-tree build lock a killed builder leaves behind. - test-plan: read the root manifest's scripts directly so a root-only script survives when the root defines no build/test; bail on the inline --root=./dir rebasing form; stop treating a positional after an inline --flag=value as the flag's value; prefer a failed scoped run when ruling a bare command; anchor the npm script alias to a full token so `yarn test:unit` is not truncated. - test-efficacy: exclude `\ No newline at end of file` from the startLine offset count; compute the mutant-overlap range from the header's new-side span so it no longer overshoots into a closely following hunk. * feat(review): mined disciplines — effective-diff guard, positive control, seven lenses From a full sweep of 108 maintainer verification comments (2026-07-31): - fetch-pr detects an EMPTY merge-base diff (work already landed — the skill stops and recommends close-as-superseded) and a diff collapsed to a residual by overlapping merged PRs (disclosed; body claims about the rest are description-of-history). - test-efficacy runs a POSITIVE CONTROL before spending the mutant window: an injected always-failing test must turn the runner red, or every would-be survivor is re-classed inconclusive — a dead runner cannot kill, and its survivors were the false gap-reports measured live. - Brief lenses: tested-fix + contamination statement; race verdicts as rates with load amplification (lower idle rate without structural change is luck); triangulate-and-label for unreachable authorities; arm-identity and comparator negative-control for A/B; deleted code's incidental effects; self-defeating fallbacks (trigger set within cannot-succeed set); reachability adjudication (fix vs scaffolding); flipped-assertion audit; shape-vs-provenance authorization probing. * fix(review): four fixes from the stack's own round-2 reviews - test-delta (Critical, #8218 review): a base rerun that fails with ZERO parseable failing files (install/toolchain failure) no longer reads as 'base green' — it attributes nothing, like a timeout, instead of promoting every PR-side failure to net-new. - extract-step (Critical, #8225 review): every line of a multi-line env value is comment-prefixed — an unprefixed block-scalar second line sat in the emitted script as an executable line. - extract-step: expressionsOf no longer misses expressions containing their own brace (format('{0}')). - cleanup (#8215 review): sweeps an orphaned base-tree build lock, which otherwise wedges the A/B for that PR permanently. * fix(review): close the remaining round-2 findings across the stack - selectMutants: replacement operators get a 3-slot sub-cap inside the shared cap — measured 24x pool inflation (215 vs 9 candidates over 40 real commits) was draining the time window hunk probes draw from last, silently un-shipping the hunk-survived finding class; sub-cap drops are counted in skippedForCap. - test-delta: whole-command budget (540s, mirroring test-efficacy) so three failed commands cannot blow the 600s tool ceiling and lose the report; run() regains build-test's three deliberate properties (stdin ignored, error.code/SIGTERM timeout detection, trimmed output). - test-plan: pasted unified diffs inside Evidence blocks shed no path claims (+++ b/<path> once ruled contradicted on a correct body); gitignored paths rule unchecked — absent at the reviewed commit by construction, the .qwen/ reasoning generalized. - agent-prompt: the Agent 7 efficacy block now enumerates all FOUR finding kinds including hunk-survived, the hunks.* skip counters, and harnessValidated — the brief and the block in the same prompt no longer present two different complete sets (the sibling-enumeration lesson, applied to our own prompt). * fix(review): address review feedback — diff-header false positives, stale prompt enumeration, added-file hunk probes (#8215) * fix(review): close the round-3 findings on the stack #8261 review: - the dead-harness re-class now runs BEFORE findings are built (a failed positive control could still file survivor findings — the Critical); it also covers the file-level revert probe's inert verdicts, the control pays for its run out of the shared budget (no more silently eaten skippedForBudget slot), a failed control stops the window instead of manufacturing survivors to re-class, and its note names all three causes that share the green-control shape. - emptyDiff no longer fires on a FAILED diff capture (close-as-superseded off an infrastructure error would close a live PR); countDiffChangedLines counts --x/++x body lines. #8215 review: - the cd target itself is no longer claimed as a path (a bare dir carries no evidence; only the file tokens it resolves are); - 'Test Files N passed' is a FILE count — variable-length lookbehind keeps it out of the test-count claims; - base-tree's handler gains the pipeline's catch-and-exit-1 convention; - probeCreateFailureDetail folded into the shared worktreeCreateFailureDetail. * fix(review): address review feedback — cd-base exclusion, Test Files count guard, base-tree error handling, probe delegation (#8215) * fix(review): port the collocated-dropout test to the post-#8050 runner seam Merging main brought #8050's Windows-portability refactor, which resolves the probe runner through vitest/package.json's bin — a node_modules/.bin fake is dead weight it never reads. The 8215-only collocated-dropout test still installed the old .bin fake, so the REAL vitest ran its fixtures, price.test.ts genuinely passed, and the hunk scored survived. The test now overrides the fake package's vitest.mjs like every post-refactor test. * fix(review): close the round-4 findings from both stack reviews From the #8215 review: trimOutput's summary rescue is capped at 40 lines (uncapped, 40k lines of 'Test <n>: …' prose made the whole trim a no-op — 1.6MB in, 1.6MB out); a spaceless # line is prose per the ATX rule GitHub applies (a pasted #!/bin/bash outside a fence once ended the Test Plan mid-body), with the bare-#-run crash on the closing scan guarded; a base-tree build lock older than 30 minutes is a corpse and is swept rather than reporting busy for the rest of the review. From the #8261 review: a pasted diff's BODY lines shed no path claims (-packages/old/gone.ts matched PATH_RE and ruled a false contradicted — the syntax-line filter alone was vacuous against the real failure mode); a gitignored file that nonetheless EXISTS rules reproduces (the ignore guard only ever downgrades a would-be contradiction); test-delta's note now explains each unattributed command (timeout vs toolchain failure) instead of leaving the reader to infer why it is in neither list. * fix(review): bound the summary rescue, apply the ATX heading rule, sweep stale build locks The three 8215-layer findings from the latest review, fixed at this layer (they were first patched further up the stack, where the reviewer of THIS PR cannot see them): - trimOutput's summary rescue is capped at 40 lines — uncapped, 40k lines of 'Test <n>: …' prose voided the trim entirely (measured 1.6MB in, 1.6MB out) and the bounded-output contract is the whole point. - A '#' with no following whitespace is prose, not a heading (the ATX rule GitHub applies): '#8176', '#tag', an unfenced '#!/bin/bash' no longer end the Test Plan section mid-body; the bare-#-run crash on the closing scan is guarded. - A base-tree build lock older than 30 minutes is a corpse left by a killed builder — swept and rebuilt instead of reporting busy for the rest of the review. * fix(review): EEXIST-only lock busy, bun test alias, chained cd bail, fence backreference Four live findings from the latest inline review round (the rest of the round was already fixed upstream by the takeover bot - verified by probing head behavior rather than re-reading the threads): - base-tree's lock catch distinguishes EEXIST (a concurrent builder, busy) from EPERM/EROFS/ENOSPC (this run's own failure, reported as such, not as a busy that will never clear). - "bun test" is bun's built-in runner, not a package-script alias: it runs whether or not any manifest defines test, so ruling it against the scripts table filed a false contradicted. - A chained cd matches the leading-cd shape but the single-hop resolver joined file tokens against the FIRST directory; it now bails like the exotic-cd case. - codeSpans' fence regex closes on its own marker via backreference; a tilde fence line inside a backtick block ended the span early and lines after it were lost to extraction. * fix(review): make the ledger marker actually reach the posted body Three Criticals from the #8255 review, all real: - The marker was appended in the CLI handler, AFTER composeReview() returned, so it only ever reached the composed JSON on disk. submit posts what the PURE function returns, so no posted review carried a marker, latestOwnLedger always returned null, and the whole feature was inert end to end. The append now lives inside composeReview, fed by a draftedComments seam that both CLI boundaries fill from the same array they count (and both strip from a model-written state JSON). Three end-to-end tests now assert on the body composeReview returns. - pr-context's side-file write preceded the mkdir that creates its directory and was unguarded: an ENOENT failed the whole command over a best-effort carry-forward. mkdir first, write guarded. - CANONICAL_LGTM_RE is anchored, so a trailing marker made every no-op round "worth showing" and prior rounds started rendering in full. The marker is now stripped before the filter and the count walk, not only before rendering. Plus the three thread findings: ledger cells are escaped before landing in the context table (a title with a pipe could forge rows), parseLedger takes the LAST marker and type-checks/normalises what it recovers, and the stray verdict doc comment is reattached to verdictLine. * fix(review): close the ten open findings on this PR - guard-true tested for a comparison anywhere after `if (`, including the then-body, so `if (ready) emit(a !== b);` admitted a mutant on the comparison-less condition the gate exists to exclude. It now tests the condition span only. - The `survived` detail said "when it changes" for legacy DELETION mutants too; it now matches the operator. - test-delta's `unparsed` required both sides to parse zero files, so a PR-side failure whose FAIL lines the trim scattered was silently dropped whenever the base rerun happened to parse. netNew/shared come from the PR side, so the PR side alone decides. - failingFilesOf now matches Windows path shapes (backslashes, C:) - a missed parse is an unattributed failure, not a loud error. - The replacement branch of runOneMutant (write-file -> run-probe -> classify) had no end-to-end test; one now drives a coalesce operator through the real handler and asserts the mutated line, the verdict, the operator-specific wording, and that the shared tree is untouched. - Two tests were vacuous with respect to what they promised: the baseline-dir test never asserted the cwd (its helper swallowed the argument - fixed at the helper), and the one-candidate-per-line test used an input that never triggered the replacement path, so the `continue` under test was not load-bearing. - Reattached the orphaned selectMutants JSDoc; reworded the SKILL line. * fix(review): the test helper's cwd parameter is required, not optional CI's `tsc --build` failed on test-delta.test.ts: the exec seam always passes a cwd, but the helper's signature marked it optional, so pushing it into a string[] was `string | undefined`. Missed locally because vitest runs through esbuild, which strips types without checking them - the suite was green while the build was red. The gate to run before pushing a type-level change is `npx tsc --build`, not the test suite. * fix(review): a base rerun that could not RUN attributes nothing Two Criticals from the latest review, both reachable on the brief's own happy path: - baseUnusable covered only timeouts. Every other way the base side can fail to run - an unbuilt base tree, a missing install, a workspace the PR ADDED (npm test --workspace cannot resolve on base), an ENOBUFS truncation - exits non-zero with zero FAIL lines, which this code read as a green base. Every PR-side failure then became netNew: the strongest evidence the command emits, manufactured from a base that never ran a test. It now attributes nothing and says why. - Timeout detection was the weaker substring form the sibling explicitly rejects; an external SIGTERM (container stop, cancelled job) set neither an ETIMEDOUT message nor an exit code and fed straight into the above. build-test now exports spawnTimedOut and test-delta asks the same question rather than re-deriving it. Also: the base output is trimmed (it precedes the verdict fields in the report the agent reads, so an untrimmed megabyte truncates exactly what the command produces); the guard-true gate no longer reads an arrow function's => as a comparison (every predicate guard was a candidate - the if (ready) noise the gate exists to exclude); the term-drop message no longer calls a string concatenation a reserve term; the unparsed note describes its own PR-side-only condition; and the ledger's Step 6/Step 8 now agree that a still-standing finding keeps its id. * fix(review): extract-step resolves all three env/defaults levels, and comments every env line Two silent-wrongness defects in a command whose whole value is fidelity. `env:`, `shell:` and `working-directory:` are three-level settings on GitHub — workflow, job, step, nearest wins — and only the step level appears in the step's own text. Reading step-level alone reproduced by machine the exact transcription error this command exists to remove: measured, a step under a job-level `NODE_ENV: production` and a workflow-level `GLOBAL_FLAG` extracted with `env: { LOCAL: '1' }` and `workingDirectory: undefined`, so the emitted script ran with both unset and nothing said so. Not a contrived shape: this repo carries workflow-level `env:` in 7 workflows, job-level `env:` in 10, and job-level `defaults.run` in qwen-triage.yml — the workflow the command's own test plan names. The three levels now merge with the runner's precedence, and `envSources` records which level each key came from, so an inherited value is visible rather than indistinguishable from the step's own. The env block was commented per ENTRY, not per LINE. A YAML block scalar (qwen-autofix.yml's `SETTINGS_JSON: |-`) reaches the header as a multi-line string, so its continuation lines landed in command position — and under the `set -e` the header itself emits, the extracted step died in its own preamble before its `run:` body ran. Every line is commented now. Tests pin the effect, not the mechanism: `executableLines()` asserts nothing but the `run:` body ever reaches command position, plus a `bash -n` parse check. Verified to flip — all five new assertions fail against the pre-fix implementation (`{ LOCAL: '1' }`, `undefined` working directory, three executable lines instead of one, and a real `bash -n` syntax error). * fix(review): compare failing files by a normalised, project-keyed identity Critical: the two sides run in DIFFERENT roots (the PR worktree and the base tree), and netNew/shared compared the parsed paths verbatim - so an absolute-path runner turned every pre-existing failure into a fabricated Critical, with the authority of a measurement behind it. Paths are now normalised against each run's own root (and backslashes to /, so a Windows path compares with its POSIX-printed twin), which is why test-delta gained --pr-worktree. The identity also keeps the vitest project token: dropping it collapsed same-named test files across workspaces, so a PR-caused failure in one package could read as pre-existing because another package has a file by the same name - the worse failure direction. Also from the same review, all of them reachable on the brief's own path: - The base rerun now inherits build-test's stdio: ['ignore','pipe','pipe'] ("a build that asks a question is a build that hangs until the deadline") and its trimOutput, which matters because entries[].base precedes the verdict fields in the report the agent reads. - The brief gates on base-tree's `available`, not just its `path`: a tree that was created but did not build populates path too, and measuring against it turns an infrastructure failure into Criticals. - A programmatic caller omitting `timeout` no longer sends NaN into spawnSync. - MutantCandidate is a discriminated union, so an operator without its replacement line - which would delete a line while reporting "with its ?? fallback dropped" - is unrepresentable. - The comparison class no longer requires a trailing space (if (a<b) is the same guard, just unformatted) and matches a brace-less else if. - DeltaEntry.unparsed's doc now describes the PR-side-only condition it actually implements. * fix(review): restore the whole-command budget, keep generics out of guard-true Round-1 findings from a fresh review of this PR: - test-delta had no aggregate deadline: --timeout is PER command and defaults to 300s, so three failed commands is 900s against Agent 7's 600s ceiling - killed with NO report at all, discarding the base-tree install and build just paid for. TOTAL_BUDGET_MS mirrors the one test-efficacy reserves; commands it cannot fit are disclosed. - guard-true matched generic calls: `if (isRecord<string>(v))` produced a mutant, and a type-guard predicate is exactly the `if (ready)` shape whose survivors the gate calls noise. The trailing \s is required, not an accidental asymmetry with [!=]== - telling `a<b` from `fn<T>(x)` needs a parser, and the gate is silence-biased by design. - --pr-worktree had no contract test, and its failure mode is the worst here: arriving undefined, root stripping silently stops and every pre-existing failure becomes a fabricated netNew. The new test feeds parseSync's output straight into runTestDelta and asserts an attribution only reachable when both roots were stripped (verified red against the snake_case field shape that shipped once already). - Merged the two consecutive doc comments on prWorktree. * fix(review): a budget-shortened deadline is not the same fact as a slow rerun Round-2 finding on the budget just restored: `Math.min(perCommandMs, remaining)` can hand a rerun far less than --timeout, and if it dies there the note said only "timed out - infrastructure, not evidence". True, but it sends the reader hunting a hang that is really an exhausted budget - and unlike a real timeout, a rerun with budget to spare would still measure it. The note now names those commands separately and says so. Verified red against removing the tracking line. * fix(review): brace-tolerant stub list, pipefail fidelity, and extract-step in the briefs Round-3 findings on this PR, fixed. `expressionsOf` matched `[^}]*`, so any expression containing a brace — `format('refs/pull/{0}/head', …)`, `fromJSON('{"a":1}')` — was not mis-listed but DROPPED. A stub list reads as "these are all the values to supply", so a silent omission is a value that never gets stubbed. It now scans forward to the closing `}}`, and reports nothing for an unterminated site rather than swallowing the rest of the text. Declaring `shell: bash` is not the runner's default `bash`. The default is `bash -e {0}`; a declared `bash` (at any level) is `bash --noprofile --norc -eo pipefail {0}`, and a pipeline whose middle stage fails aborts under one and not the other. The header now carries `set -eo pipefail` or `set -e` accordingly — 163 of this repo's 434 `run:` steps are under a declared bash and were getting the weaker one. A `shell:` value is also a command template (`perl {0}`), so only its first word goes in the shebang and the whole template is recorded beside it. `extract-step` was registered on the CLI and mentioned in DESIGN.md, and nowhere in SKILL.md or the agent briefs — the runtime prompts. The capability was unreachable by the agents it was written for. The verifier's brief now carries it next to the A/B paragraph it composes with, and Step 4 summarises it. Also: env ordered nearest-first (measured on qwen-autofix.yml:route:0, merge order put 20 inherited entries ahead of the step's own 26 in a 49-line header); a valueless `FOO:` renders as the empty string, not `"null"`, and a non-scalar as JSON rather than `[object Object]`; a missing file no longer reports as a parse failure; DESIGN.md's lens count matches its list. The test oracle is rebuilt around the property instead of a filter: the file is the header plus the body verbatim, and every line before the body is a comment or a directive the test names. The old helper dropped `set -e` unconditionally, so it could not tell the header's from one the body legitimately contains — and would have gone green on a header that leaked exactly that line. 434 real `run:` steps swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): restore the replacement sub-cap, stranded on a downstream branch Round-3 finding, and the third instance of one class: an 8218-layer fix committed on the 8261 branch, four PRs above the code it belongs to. Measured over 40 real commits, the replacement operators produce ~24x the deletion pool (215 vs 9 candidates; guard-true drives it). Every mutant run drains the same window hunk probes draw from LAST, so uncapped, most diffs with any replacement candidates leave hunk probing zero runs - the hunk-survived finding class silently stops firing and nothing says so. Three slots, and what the sub-cap drops is counted in skippedForCap rather than lost. Also swept the other direction: diffed every review file against its 8261 copy to confirm nothing else 8218-layer is stranded up there. The remaining divergence is 8261's own (the positive control, its lenses). * fix(review): one severity predicate, carried ids, and a lossless marker Round-1 review of the ledger marker, six findings. Two contradicted contracts this package already states elsewhere: - `buildLedger` restated the severity predicate as a bare `startsWith` while `severityOf` — what `countInlineFindings` and the unmarked-comment gate both use — trims first. A Critical whose body opened with a newline was counted, was posted, blocked the merge, and was silently absent from the ledger, shifting the id of every finding after it. It classifies through `severityOf` now, and strips the marker with the shared prefixes instead of a second copy of the literals. - Ids were assigned by array position every round, so the report saying `R1-2 still stands` shipped beside a marker recording that claim as `R3-4` — and `LedgerFinding.id` documented itself "stable across re-reports". Step 6 now mandates writing the original id into the re-report (`**[Critical]** R1-2: …`), `buildLedger` reads it back, and a duplicate or absent id falls back to the next free id of this round. Four smaller ones: - `stripLedgerMarker` removed only the FIRST marker while `parseLedger` reads the LAST, so a body holding two leaked the trusted marker into model-facing prose and un-filtered the canonical LGTM it was appended to. - The `--` guard rewrote data to an em dash: a finding about `--comment` reached the next round as `—comment`. It escapes at the JSON layer now, which is comment-safe and lossless. - `file` was capped on read (200) but not on write; the cap is one constant binding both halves. - `submit`'s path — the one GitHub's payload is built from — had no test that the marker reaches the posted body. Plus: tie-break `latestOwnLedger` on review id when `submitted_at` ties, neutralise a backtick in the location code span, skip the `currentUser()` round-trip when the PR has no reviews, and say why a stale side file is deliberately left in place. * docs(review): complete the "delta cannot rule" enumeration in both places Round-4 finding. The brief and SKILL.md each listed three cases where test-delta attributes nothing - unparsed, timed-out base, no merge base - but the code has five: the later rounds of this PR added "a base rerun that failed without naming any failing file" (it did not measure the base) and "a command the whole-command budget could not fit". Two enumerations of the same set with different membership, in the two places an agent reads. That is the sibling-enumeration lesson this skill teaches, applied to its own prose for the second time: the fix is not just adding the missing members but saying that the report names each case with its own reason rather than folding them into one. * fix(review): the invokes list was mostly prose, not commands Round-4 finding on this PR. `invokes` is documented as a heuristic starting point, and imprecision is fine — but measured over this repo's 434 real `run:` steps it was reporting 435 distinct "commands", 267 of them appearing exactly once, with a worst case of 63 entries made up of words like `CI`, `Evidence`, `PR` and `and`. A list that size, mostly prose, is not a starting point. Three causes, each measured: - A `${{ … }}` expression is not shell, and it routinely contains `||`. Splitting on that as a pipeline separator reported both operands as commands (`matrix.arch`, `github.event.inputs.version`). Expressions are now masked to an opaque token before the split; one sitting in command position contributes nothing, which is honest — what it expands to is unknown here by design. - A heredoc body is input to a command, not a list of them. Its lines were scanned as commands, terminator included. 12 steps in this repo carry one. - The `name=value` skip stepped over the prefix and took the NEXT word as the command — but for a quoted value with spaces that word is inside the value: `EVIDENCE_SECTION=$'### Evidence images'` reported `Evidence`. Quoted spans are now blanked out, with the quote carried across lines so a multi-line string's continuation lines are data too. Command substitutions are read first, so `body="$(sanitize < "$REPORT")"` still reports `sanitize`. A `#` preceded by whitespace ends the live part of a line, so an apostrophe in a trailing comment cannot open a span and eat the rest of the script. Measured after: 435 distinct commands to 187, singletons 267 to 104, worst case 63 to 27 — and the worst case is now real commands (`awk cat chmod curl git jq mktemp pkill tar tee timeout`) plus the script's own shell functions. Also added the builtins a stub could not intercept anyway (`eval`, `exec`, `source`, `unset`, `command`, …) to the keyword set. 434 steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * docs(review): the rationale named only one of the ways base goes unmeasured Third and last copy of the enumeration the previous commit fixed. The bullet's headline already generalised - "base attributes nothing it did not finish" - but its body named only the timeout, so a reader learning the contract from the rationale would conclude timeouts are the only unusable case. Name the set, and say why the report keeps the reasons apart: "we could not measure" and "we measured nothing" are different facts to the author, and only one of them is about their PR. * fix(review): a quoted `<<EOF` is not a heredoc, and a continued line is one command Round-5 findings, both in the previous commit's own scanner. A heredoc opener was matched over the whole line, so one inside a string started heredoc mode: `echo "write <<EOF for a heredoc"` made every later line wait for a terminator that never arrives. The failure is not a missing entry but a missing REST — measured, a three-line script returned `[]` instead of `[curl, jq]`, empty and entirely plausible. Opener detection moved inside the quote walk, where it only fires outside quotes; the quoted forms (`<<'EOF'`) are consumed by the match, so their quotes never open a span either. A backslash-continued command was scanned as several lines, which puts the next ARGUMENT in command position — this is where `apt-get install -y \` / ` libx11-dev` reported the package as an invoked command. Continuations are now joined into one logical line before scanning. Measured after: 187 distinct commands to 185, and the singleton tail is now dominated by real ones — PowerShell cmdlets, macOS tooling, and the scripts' own shell functions. 434 real steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): case labels, a second heredoc, and an expression in command position Round-6 findings, from running the scanner against adversarial shapes rather than reading it again. Two are UNDER-reports, which is the worse direction: a command missing from the list is a stub the verifier never writes, so the extraction reaches the real network. - A `case` pattern label stopped the scan on its own line: `blocked) gh api x` reported nothing, losing `gh`. The label is now stepped over like a `name=` prefix. - Only the first heredoc opener on a line was tracked, so `cat <<A <<B` left the second body and its terminator read as commands (`B`, `y`). Openers are queued and consumed in order. - Masking an expression to a QUOTED token let the quote-stripper delete it entirely, so `${{ steps.x.outputs.cmd }} arg` reported `arg` as the command. The token now survives stripping and cannot match a command word, so an expression in command position contributes nothing. Ten further adversarial shapes were already correct and are pinned as regression guards: nested `$( )` in quotes, subshells, function definitions, indented heredoc terminators, backticks, bare redirects, adjacent and empty `${{ }}` sites, and a JSON literal inside an expression. Verified to flip — exactly the three above fail against the previous commit, the other ten pass. 434 real steps re-swept: 0 non-verbatim bodies, 0 live header lines, 0 missed expression sites, 0 out-of-order env, 0 `bash -n` failures. * fix(review): annotate the continuation accumulator so tsc can type it `invokedCommandsOf`'s backslash-continuation loop failed to compile with TS7022: the narrowed type of `pending` at the join line is the union of the loop-entry value and the back edge, and the back edge is computed from the join itself. The declaration's own annotation does not break that cycle - control-flow narrowing runs after it - so the checker gives up and calls the result `any`. Caught only on a forced rebuild. `tsc --build` had been reporting this tree clean off a stale .tsbuildinfo, which is the same shape of gap that put a type error into CI last week: the test runner strips types, so the suite stayed green either way. Gate with --force. * fix(review): a file-count label stops counting at the end of its line `Test Files 45 passed` filing its 45 as a differing TEST count was fixed once, with a lookbehind on the bare-count pattern. That only ever rejected the all-green shape. The moment any file fails, the runner prints `Test Files 1 failed | 44 passed (45)`, the label is no longer adjacent to the number, and 44 comes through as a test count - so the note reads "claimed 44, observed 1323" on exactly the runs whose summary someone would paste. Adjacency was the wrong invariant; the line is. Masking from the label to end-of-line is distance-independent and picks up jest's `Test Suites: 1 failed, 44 passed, 45 total` at no cost. The label keeps its `Test` word on purpose: a first cut matched a bare `files` and blanked the line in "expect all four files and 471 tests to pass", silencing a real claim. An existing test caught that, which is the argument for the rule being as narrow as it is - anything that suppresses claims is worth exactly its narrowness. Also drops the now-dead lookbehind, which a reviewer had separately (and wrongly) called a JS syntax error; variable-length lookbehind is legal in V8, and the module parsed fine. It goes because the line rule subsumes it, not because it was broken. * fix(review): address round-1 review of the mined-disciplines batch The positive control could state a verdict it never measured, and its own failure path shipped untested. Ten findings from the review of #8261: - `runControlMutant` returns `boolean | null`. An unreadable probe file injected no test and ran nothing, but returned `false` — which the caller reads as "the injected always-failing test stayed green", re-classes every survivor with that sentence, and discards the whole mutant/hunk window over an I/O error. `null` is the file's own third-outcome rule; the window stays spendable and the note says the harness was not validated. - `mutants.skippedForControl` / `hunks.skippedForControl`. A control that came back red stopped the run with candidates unprobed while every `skipped*` counter read 0 — the silent cap the brief teaching agents to read those counters rules out. Their own reason, not the budget's. - Cover the control's failure half: a dead-runner integration case (survivors re-classed, revert probe's `inert` re-classed, findings empty, nothing spent) and a could-not-set-up case pinning that `null` does NOT discard the window. Both branches, and the re-class block, were previously unreachable by any test. - `rulePath`: one existence check, not two. The second was unreachable behind the first and its note — "exists at the reviewed commit (the diff does not change it)" — silently retired, collapsing tracked state and this run's build output onto one sentence. The ignore status now picks the wording. - `expressionsOf`: forbid the body from crossing another `${{`. A malformed site above a real one swallowed it into the blob, so `${{ github.event.comment.body }}` stopped being enumerated as an injection site — the one direction this helper must not fail in. - `countDiffChangedLines`: count by hunk position, not prefix shape. A deleted line whose content starts `--` arrives as `--- …` (markdown rules, YAML markers, SQL comments), and every dropped line pushes the ratio toward a false `collapsedFromUpstream`. - `emptyDiff` is gated on `baseFetchFailed` too. A stale local base ref that already contains the head commits diffs to empty, and the skill acts on that by recommending close-as-superseded. - `test-delta`: price the budget slot against the command's own measured duration instead of a flat 5s floor, which admitted commands that could only time out and then disclosed them as "infrastructure, not evidence". `skippedForBudget` becomes a structured field; `now` becomes a seam so the test stops reassigning the global `Date.now`. - Drop `probeCreateFailureDetail` (no production caller after the swap to `worktreeCreateFailureDetail`) and the duplicated base-lock sweep in `cleanup.ts`, ten lines above the identical pre-existing one. - `--` before the path in `check-ignore`, and the comment says plainly that no `-`-leading claim survives extraction today. * fix(review): parse the base rerun before its output is trimmed `trimOutput` rescues module errors and runner summaries out of the omitted middle, not the per-file FAIL lines this command reads. A base suite whose failure section overruns the tail budget therefore lost failing files into the gap - and a SHORT base set is the dangerous direction, because netNew is the PR side minus the base side. Every file the trim hid came back as a Critical attributed to this PR by "measurement". Parse the raw text, report the bounded one. The PR side cannot be fixed here: it is read out of build-test's stored output, trimmed before this command existed. That loss runs the other way - it understates `shared`, never invents a netNew - so the entry carries `prTruncated` and the note says the list may be partial. A silence-biased gap is still a gap the author should hear about. Also names both selection caps in the mutant-skip diagnostic. The count accumulates replacement sub-cap drops, and with 2 deletions and 6 replacements the total is exactly MAX_MUTANTS: the main cap never fires, yet 3 are dropped, and the message sent the reader looking for a pool of 11 candidates that never existed. * test(review): pin the `sh` set-line to exactly what the runner uses A reviewer asked for `expect(script).not.toContain('set -e')` on a `shell: sh` step. The opposite is correct - GitHub runs that step as `sh -e {0}` - but the thread was right that nothing pinned it either way. Assert both halves: `set -e` is present, `pipefail` is not. Dropping the line makes an extracted `sh` step run past a failure the runner stops on; adding pipefail claims a bash feature `sh` does not have. * fix(review): rerun only the command shapes build-test emits This command reads a report off disk and then executes the strings in it with `shell: true`, in the base worktree. Nothing else in the pipeline re-executes a value it read back from a file, so nothing else has to care where that value came from - this does, and the provenance is worse than it looks: the command is `npm test --workspace="<dir>"`, the workspace token is a directory, and a directory is a name a pull request chooses. `packages/x";curl …|sh;"` is a legal path in git and on Linux and it round-trips through the report into a shell. Restricting to the emitter's own grammar costs nothing real, since that grammar is the two shapes build-test produces. A command outside it is skipped and disclosed, the same treatment everything else this command cannot do already gets, so a future shape degrades to "judge it by the diff" rather than to arbitrary execution. * test(review): pin the empty-diff and collapse detectors Both lived inline in a large object literal with no test, and one of them is the most expensive disclosure this command emits: the SKILL acts on `emptyDiff` by recommending the PR be closed as superseded. Dropping the 200-line floor, flipping the 4x comparison, or losing either empty-diff guard turned nothing red. Extracted as two pure predicates and pinned at the boundaries, including the two guards that are not about the diff at all - a capture that threw leaves diffText empty, and a merge base resolved from a stale local ref can already contain the head commits. Either would have closed a live PR on an infrastructure error. Checked by mutation rather than by reading: all four edits above now fail exactly one assertion each. * test(review): pin the 30s budget floor, and stop keeping its arithmetic twice Two findings from the same review round, both about budget math nobody was checking. The 30s floor separates "skipped, judge it by the diff" from "started, timed out, disclosed as infrastructure". Every existing budget test left it non-binding - two sit hundreds of seconds from the boundary and the third is capped by the per-command deadline - so regressing it to the old 5s turned nothing red. Verified by doing exactly that: 26 green before the new test, one red after. A one-second command admitted into a twenty-second window comes back labelled `budgetClamped`, which is the mislabelling the priced floor exists to prevent. The other is the same duplication lesson this skill keeps arriving at, in its own tests: one test carried the per-run budget arithmetic in a block comment AND at the `now:` argument, and when the per-run figure changed only one copy moved. The block also predated the positive control, so it omitted a whole suite run. Deleted the copy rather than correcting it - two agreeing copies today are two disagreeing copies later. * fix(review): the working directory is a setting the extraction was losing Two findings, both of them this file failing its own stated argument. The stub list dropped it. `expressions` covered the script and the env and nothing else, so a `working-directory: ${{ github.workspace }}/x` produced an empty list and a summary line reading "0 ${{ }} site(s) to stub" - and `expressionsOf`'s own comment says why that is the failure this list cannot afford: the caller reads it as "these are all the values to supply". Widened to every setting the command carries, the `shell:` template included. The emitted script never mentioned it. The env block is commented into the header precisely so a reader of the script alone can see it; the working directory changes what the script does just as much and was in the metadata only. The argument for reading all three levels, written in this file, is that a step run "in the wrong directory, and nothing says so" is the transcription error the command exists to remove - which is exactly what the header did. It is a comment rather than a `cd` for the same reason env is comments, not exports: the value may hold `${{ … }}` and this command substitutes nothing. Both pinned, and both checked by deleting the fix: each mutation fails exactly one of the two new tests. * fix(review): $(( )) is arithmetic, and a heredoc's form decides where it ends Three fixes and one deliberate non-fix, all measured against this repo's own 434 `run:` steps rather than argued. `$(( ))` was read as a command substitution, so `N=$((N + 1))` reported `N` as a command to stub. It was the single largest source of junk in the list: 196 distinct "commands" across the corpus, 165 without it. A plain `<<WORD` heredoc ends only on a line that is exactly WORD; the loose match ended it on an indented `EOF` inside the body and then read the body as commands, which is how `rm` got reported for a script that never runs it. `<<-` stays looser than bash (any leading whitespace, not just tabs) because looser can only end a body early, and this file's priority is that an under-report is the worse direction. `[^()]*` matched only the innermost `$( )`, so `X=$(gh api $(u))` lost `gh` - a missed stub, and the extraction reaches the network. Depth counted now, and the assignment-prefix skip no longer steps over an unclosed `X=$(gh` into reading `api` as the command. The non-fix is recorded where the next reader will hit it: the quote walk is flat while shell quoting nests, and over ~300 lines the drift reports fragments of jq source as commands. Inserting a separator where a blanked span was removes nine of those, but it splits `a"X"b`, which is one word to the shell, and the minimal reproducer for the difference is 296 lines - nothing short enough to pin it. A scanner nobody can pin costs more than the junk it removes. * fix(review): refuse an ambiguous step name, and report errors like the siblings Two findings, both about a message the caller never gets. A job may legally hold two steps with the same name. The selector took the first and said nothing, which is the failure this file's own header names - "picks the same-named step from the wrong job" - and it is worst in the use the command exists for: A/B extraction runs it once per tree, so a PR that adds or reorders a duplicate leaves the two sides comparing different steps while reporting on one. Refused out loud now, naming the indices; the index is always available and never ambiguous. The handler also let every throw propagate, so five carefully separated messages - cannot read vs cannot parse vs no job vs no step vs no `run:` - all arrived as "An unexpected critical error occurred" under a stack trace. `base-tree` and `test-plan` in the same directory already catch, write the message, and set exit 1. Matched. The separation between "the path is wrong" and "the YAML is wrong" only pays if the caller sees it. * docs(review): the brief's list of limits was one short "Two limits worth knowing before you spend the step" became three when the selector started refusing an ambiguous step name, and the count went stale in the same commit that made it wrong. This is the enumeration drift the skill teaches, in the text that teaches it. The added entry says what to do rather than only what happens: pass the index, which is what an A/B wants regardless — the two trees have to select the same step, and a name that moved between them is exactly how they stop doing that. * chore(review): drop a scratch probe file that reached the branch `packages/cli/inert.mjs` was a throwaway harness for sweeping the repo's own workflows; its cleanup ran with a relative path from the wrong directory and it got committed by the next `git add -A`. It failed CI and not the local hook because the two lint different sets: lint-staged filters to `*.{js,jsx,ts,tsx}`, which does not include `.mjs`, while CI's flat config picks the file up regardless of `--ext`. Verified with CI's own command this time, not the hook's. * fix(review): the collapse disclosure needs the guard its sibling already has Three findings from the round on this branch, all accepted. `isEmptyDiff` refuses to rule when the merge base came from a possibly stale local ref, because such a base can already contain the head commits and diff to empty. The PARTIAL form of that cause lands on `isCollapsedFromUpstream` instead - a stale ref holding most of the head commits shrinks the recomputed diff past the 4x ratio - and the flag then tells Agent 0 to read the body as description-of-history when the body may be perfectly current and the real cause is a fetch failure. Two predicates over the same evidence, one guarded and one not. Guarded now, and pinned: removing the guard fails exactly one test. The pre-set of `mutantsSkippedForBudget`/`hunksSkippedForBudget` in the no-budget-for-control branch was dead in every path and worse than dead in one: both loops run with `harnessValidated` still null, re-check the same budget, and set their own counters - and the hunk loop's figure excludes the collocated-probe inconclusives it pushes first, which the pre-set did not. Dropped, with the reason where the assignment was. `runControlMutant`'s bound is now stated in its doc: it validates ONE file, so a collector that silently drops a different probe file passes the control while that file's survivors stand. The per-file baseline gate bounds what is left. A `true` there is read as covering the run, so what it does not cover belongs next to it. * fix(review): bound the whole ledger marker, not just each of its fields The per-field caps leave the total unbounded: fifty findings at full width serialize to just under 17,000 characters. The module's own paragraph calls the marker "a footnote, never a payload" and reasons about GitHub's 65,536-character body limit, so the total is the number that claim is about and nothing was computing it. Set the budget from measurement rather than from the limit. Across every review this pipeline has posted on its own stack (n=66) the body runs a median of 721 characters, p90 2,178, max 3,925 - the limit has ~61 KiB of headroom, and an over-long marker was never going to fail a post. The real cost is proportion: 17,000 characters of invisible payload on a 3,925-character review is four times more marker than review. 8 KiB holds fifty findings at realistic widths without dropping any, and the truncation path is reached only by a ledger no round has produced. What it drops travels with it. A `dropped` count is the difference between a list the next round reads as complete and one that says it is not - the same distinction this module already draws everywhere else. * fix(review): a partial ledger must not render as a complete one The size cap added in the previous commit can drop entries, and the rendered section still opened with "**Every entry below is owed a this-round ruling**" and said nothing about what was missing. The `dropped` count existed and reached the only place a reader sees the list as nothing at all — which is the failure it was added to prevent, committed one step after adding the field. It now says the list is partial, how many are missing, and that absence is not evidence a finding was fixed. Second, the table escaping escaped `|` but not `\`. `\|` in a title became `\\|`, which markdown reads as an escaped backslash followed by a LIVE separator: the forged row the escaping exists to prevent, produced by the escaping. Backslash first now, and the test counts separators the way markdown does rather than by pattern, because a lookbehind counter is fooled by exactly the input under test. * fix(review): an empty ledger title is a cap the next round cannot lift A comment that is nothing but its severity marker produced a ledger entry with an empty title. That does not merely degrade the entry, it jams the review: the next round is told every ledger entry is owed a this-round ruling, has no claim text to rule on, answers `cannot tell`, and `cannot-tell-existing-critical` is one of the cap reasons. Nothing between rounds changes, so the cap comes back every round. Dropping the entry would hide a Critical that really was posted, so it keeps its place and gets the one handle there is - the location it was filed at, or "the review body" for a body-level one. The field's stated job is "enough for the next round to re-locate the claim", and that is what a text-less finding still has. Found by probing `buildLedger` with ten malformed comment shapes; the other nine - carried ids, id collisions, two comments carrying the same id, `R2D2` not being an id, a leading newline before the marker - all behaved correctly and needed no change. * fix(review): count both caps as dropped, not just the byte one Found by running the three previous fixes together on one round: 51 findings in, 24 kept, and the marker said 26 were missing. `LEDGER_MAX_FINDINGS` truncates before the byte cap ever runs, and measuring `dropped` against the already-sliced list left the count cap's share silent - so the field added two commits ago to stop a truncated list reading as complete was itself under-reporting the truncation. Measured against what came IN now, and pinned across all four combinations: count cap alone, byte cap alone, both, and neither. Kept plus dropped equals given, in every one. * fix(review): the ignore check was the one git spawn without a deadline `isGitIgnored` reached for a bare `execFileSync` while every other git invocation in these commands goes through the package helper, which carries `GIT_TIMEOUT_MS` and `GIT_TERMINAL_PROMPT=0`. That constant's own comment is the argument: "a hang must still end". This one ran against a worktree the review does not control and could not. No behavioural delta for any input that does not hang - all 80 test-plan tests pass unchanged - so it ships without a new test rather than with a mocked-spawn assertion that would pin the call shape instead of the behaviour. * docs(review): a control killed mid-run did not "never run at all" SKILL's `harnessValidated` paragraph enumerated four ways the value lands on `null` - no green baseline, no candidates, no budget, an unreadable probe file - and all four are "the control never ran". There is a fifth: `runControlMutant` throws when its deadline kills the run or the runner cannot be spawned, the outer catch swallows it, and the value stays `null` rather than becoming a fabricated `false`. That is the path a slow machine takes. The ruling is unchanged - neither validated nor refuted, a survivor stands but unconfirmed - so this is about what the terminal says, not what it decides. An agent quoting "the control never ran at all" for a control that ran and was killed states something untrue about the run, and the enumeration is what it quotes from. --------- Co-authored-by: qwen-code-dev-bot <qwen-code-dev@service.alibaba.com> Co-authored-by: Qwen Code <qwen-code@users.noreply.github.com> Co-authored-by: Qwen Code Bot <qwen-code-bot@users.noreply.github.com> Co-authored-by: Qwen Code Autofix <qwen-code-autofix@users.noreply.github.com>
Collaborator
|
Released in v0.21.3. |
doudouOUC
reviewed
Aug 1, 2026
doudouOUC
left a comment
doudouOUC
left a comment
Collaborator
There was a problem hiding this comment.
— qwen3.7-max via Qwen Code /review
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Stacked on #8225 (→ #8218 → #8215). Makes the round ledger survive across environments by carrying its authoritative copy in the posted review body, and lets the default (medium) effort read it:
compose-reviewembeds a machine-readable marker — an HTML comment, invisible on the PR page — carrying this round's findings (round-scoped ids, severity, location, one-line title) into the body it composes. The round number comes from a CLI-written side file (+1 over the previous recovered round), never from the model.pr-contextrecovers the reviewing account's latest marker from the PR's reviews, renders it as a "Previous /review round (machine ledger)" work-list section in the context file, and writes it beside it asqwen-review-pr-<n>-prev-ledger.jsonforcompose-review's round counter. Own-account only; fail-quiet on any malformation; the marker is stripped from the full-body rendering so the JSON never reaches a model as prose.Why it's needed
The ledger shipped in #8218 as a local cache file, and its first multi-round live use exposed the flaw: four model-comparison rounds re-reviewed the same two PRs and every round opened from scratch — medium never reads the cache, and from CI or another machine there is no cache at all. The human workflow this imitates opens round 2 with "M1 is fixed" precisely because the previous report lives on the PR. Now the machine copy lives there too: the boundaries are own-account/latest-only (another user's marker is data about their tooling), data-not-authority (every recovered entry is owed a Step 6 ruling against the code — a tampered marker costs wasted rulings, never a wrong verdict), and caps on size (50 findings × 80-char titles,
--never survives into the comment payload).Reviewer Test Plan
How to verify
cd packages/cli && npx vitest run src/commands/review src/commands/review.test.ts; expect all 45 files and 1217 tests to pass, including the newlib/ledger.test.tsand thelatestOwnLedger/renderLedgerSection/buildLedgersuites.serializeLedger→ paste into a body →parseLedgerreturns the same ledger; any malformation (bad JSON, wrong version, non-integer round, misshapen entries) contributes nothing and never throws.--inside titles/paths still forms a valid HTML comment (the serializer rewrites--to—).Evidence (Before & After)
The motivating measurement is this branch's own history: four live model-comparison rounds on PRs #8219/#8224 (all medium effort) each opened from scratch despite three earlier rounds' findings being known — the cache was never read at medium and would not have existed off-machine. Non-UI change: N/A.
Tested on
Environment (optional)
Unit suites only.
Risk & Scope
--comment, which these dogfood rounds deliberately withheld); the next posted review is the natural canary.Linked Issues
Stacked on #8225 → #8218 → #8215 (retarget down the stack as they land).
中文说明
本 PR 做了什么
叠在 #8225（→ #8218 → #8215）之上。让轮次台账跨环境存续：权威副本改为随发布的 review 正文携带，并让缺省档（medium）读取它：
compose-review在其组装的正文中嵌入机器可读 marker——HTML 注释，PR 页面上不可见——携带本轮 findings（轮次作用域 id、严重度、位置、单行标题）。轮次编号来自 CLI 写出的 side 文件（上一轮恢复值 +1），绝不来自模型。pr-context从 PR 的 reviews 中恢复本账号最新的 marker，在 context 文件中渲染为 "Previous /review round (machine ledger)" 工作清单章节，并在旁边写出qwen-review-pr-<n>-prev-ledger.json供compose-review的轮次计数使用。仅限本账号；任何畸形输入静默失败；全文渲染时剥离 marker，JSON 不会作为散文喂给模型。为什么需要
台账在 #8218 中以本地缓存文件面世，首次多轮实测立刻暴露缺陷：四轮模型对比复审同样两个 PR，每轮都从零开始——medium 从不读缓存，而在 CI 或另一台机器上根本没有缓存。它模仿的人工流程之所以能以 "M1 已修复" 开场，正因为上一轮报告就在 PR 上。现在机器副本也在那里：边界为仅本账号/仅最新（他人的 marker 是关于他们工具的数据）、数据而非权威（每条恢复条目都欠 Step 6 一次对照代码的裁定——被篡改的 marker 代价是几次白费的裁定，绝不是错误结论）、以及尺寸上限（50 条 × 80 字符标题，
--不会进入注释载荷）。Reviewer 测试计划
验证方式
cd packages/cli && npx vitest run src/commands/review src/commands/review.test.ts；预期 45 个文件、1217 项测试全部通过，含新增的lib/ledger.test.ts及latestOwnLedger/renderLedgerSection/buildLedger套件。serializeLedger→ 粘入正文 →parseLedger返回同一台账；任何畸形（坏 JSON、版本不符、非整数轮次、条目形状错误）都不产出且不抛异常。--时正文仍是合法 HTML 注释（序列化器把--改写为—）。前后对比证据
动机测量即本分支自身的历史：对 #8219/#8224 的四轮真实模型对比（均为 medium 档），每轮都在前几轮 findings 已知的情况下从零开场——medium 不读缓存，且离开本机缓存根本不存在。非 UI 变更：N/A。
测试平台
环境（可选）
仅单元测试。
风险与范围
--comment，而这几轮 dogfood 刻意未发布）；下一次发布的 review 是天然的金丝雀。关联 Issue
叠在 #8225 → #8218 → #8215 之上（随下层合并依次 retarget）。