docs(triage): scale PR verification to the change, add real-run depth - #7648
Merged
Conversation
/triage already drives a real tmux before/after (installed vs dev build). But where it matters most that floor is not enough: a UI change's highlight/caret is invisible in capture-pane text, build/test numbers from a shared or symlinked tree can be environmental TS errors not the PR's, and a perf claim went unmeasured. Strengthen Stage 2b to scale the evidence to the change, pointing at harnesses the repo already ships: the terminal-capture skill (node-pty -> xterm.js -> pixel-accurate PNG) for UI/interaction changes with a named on-screen oracle; a clean build for the numbers cited as evidence (a symlinked node_modules surfaces spurious cross-package TS errors); instrumenting the real built code for a measured before/after on perf changes; and separating merge-blockers from standing follow-ups with a reproducible methodology note. Docs/refactor PRs stay N/A.
Collaborator
Author
|
@qwen-code /takeover |
qwen-code-dev-bot
added
the
autofix/takeover
Collaborator
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. Remove the 中文说明🤝 已接管：autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突，直到移除标签或达到轮次上限。移除 |
Collaborator
|
Thanks for the PR! Template looks good ✓ Problem: observed gap, not theoretical. The PR compares the triage bot's Stage 2b output against a thorough manual review (#7302 round-2 verification) and names specific shortfalls — text-only capture can't show styling, no oracle naming, no perf measurement, no clean-state discipline. This is a concrete quality gap identified from practice. Direction: aligned — improving the triage skill's verification guidance is directly within the project's tooling. The change points at infrastructure that already ships (terminal-capture skill, existing deps) rather than adding new machinery. Size: not applicable — no core paths touched. Approach: scope feels right. Two files, +41/−11, all edits serve the stated goal. The "scale the evidence to the change" framing is the right abstraction — it raises the floor for UI/perf PRs without burdening docs/refactor PRs. No unrelated changes or drive-by edits. Moving on to code review. 🔍 中文说明感谢贡献！ 模板完整 ✓ 问题：已观测到的差距，非理论性问题。PR 将 triage bot 的 Stage 2b 输出与一次扎实的人工评审（#7302 第二轮验证）对比，指出了具体不足——纯文本 capture 无法展示样式、未点名 oracle、无性能实测、无干净状态纪律。这是从实践中发现的具体质量差距。 方向：对齐——改进 triage skill 的验证指引直接属于项目工具链。改动指向已有基建（terminal-capture skill、现有依赖），而非新增机制。 规模：不适用——未触及核心路径。 方案：范围合理。两个文件，+41/−11，所有编辑服务于既定目标。"按改动缩放证据"的框架是正确的抽象——抬高 UI/性能 PR 的下限，同时不给 docs/refactor PR 增加负担。无无关改动。 进入代码审查 🔍 — Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
Code ReviewIndependent proposal: given the goal of scaling verification depth, I would (1) scope the mandatory tmux before/after to user-visible changes with an N/A escape hatch for docs/refactor PRs, (2) point UI reviews at the existing terminal-capture skill for pixel-accurate screenshots, (3) require perf claims to carry measurements, and (4) require build/test numbers from a clean state. That's essentially what this PR does. Comparison with the diff: the PR matches this approach closely. The "Scale the evidence to the change" block in pr-workflow.md is well-structured — three clear categories (UI/styling, build/test numbers, performance) plus the N/A escape and the blockers-vs-follow-ups separation. The SKILL.md edit is the minimal mirror of the same scoping. No correctness issues, no security concerns, no convention violations. Two specific checks:
Nothing to flag. TestingN/A — this is a docs-only change to skill reference files ( CI evidence for
No failures. Skipped checks are expected for a docs-only change. 中文说明代码审查独立方案： 给定"按改动缩放验证深度"的目标，我会 (1) 将强制 tmux before/after 限定于用户可见改动，为 docs/refactor PR 提供 N/A 出口；(2) 将 UI 评审指向现有的 terminal-capture skill 获取像素级截图；(3) 要求性能声明附带实测数据；(4) 要求构建/测试数字取自干净状态。这正是本 PR 所做的。 与 diff 对比： PR 与上述方案高度一致。pr-workflow.md 中的"按改动缩放证据"块结构清晰——三个明确类别（UI/样式、构建/测试数字、性能）加上 N/A 出口和阻塞项/遗留项分离。SKILL.md 的编辑是同一限定的最小镜像。 无正确性问题、无安全隐患、无规范违反。两项具体检查：
无需标记的问题。 测试不适用——纯文档改动，仅涉及 skill 参考文件（
— Qwen Code · qwen3.8-max-preview Reviewed at |
Collaborator
|
Confidence: 5/5 — clean, well-motivated docs improvement; would merge without hesitation. This is exactly the kind of PR that's easy to evaluate: small scope, clear motivation from observed practice (#7302's manual review vs the bot's output), and it points at infrastructure that already exists rather than inventing new machinery. The "scale the evidence to the change" framing is the right call — it raises the floor where verification matters (UI, perf) without adding ceremony where it doesn't (docs, refactors). The diff is minimal and focused. Every edit serves the stated goal. The N/A escape hatch is consistently applied across both files. The terminal-capture reference checks out (skill exists, deps are installed). CI is green. If I had to maintain this in six months, I'd thank the author — the guidance is concrete enough to act on ("name the oracle", "measure it, don't claim it") without being so prescriptive that it becomes a checklist to game. 中文说明置信度：5/5 —— 干净、动机充分的文档改进；毫不犹豫地合并。 这是一个极易评估的 PR：范围小、动机来自观测实践（#7302 的人工评审 vs bot 输出）、指向已有基建而非发明新机制。"按改动缩放证据"的框架是正确的——在验证重要的地方（UI、性能）抬高下限，在不重要的地方（docs、refactor）不增加仪式。 diff 最小且聚焦。每处编辑服务于既定目标。N/A 出口在两个文件中一致应用。terminal-capture 引用经验证有效（skill 存在、依赖已安装）。CI 全绿。 如果六个月后维护这些内容，我会感谢作者——指引足够具体可执行（"点名 oracle"、"实测而非声称"），又不会过于规定性而沦为应付清单。 — Qwen Code · qwen3.8-max-preview Reviewed at |
qwen-code-ci-bot
previously approved these changes
Jul 24, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed. Suggestions are inline. Not reviewed: coverage — no plan was given, so this run cannot show that any of the diff was read.
— qwen3.7-max via Qwen Code /review
Collaborator
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 1/100 轮）。改动内容与我反驳保留之处如下： Review feedback addressed[rc:3643020506] N/A escape hatch tension with unconditional requirements — ImplementedFinding: The N/A escape hatch for docs/types/refactor PRs (added in the original commit) conflicted with two unconditional requirements in the same section:
Decision: Implement. The contradiction is real — a triage agent on a docs-only PR would either waste a tmux session on meaningless output or produce an inconsistent Stage 2 comment. Changes (2 lines):
Both carve-outs are consistent with the existing N/A paragraph and with each other. Verification
No TypeScript or test files were touched; no Vitest or integration runs needed. 中文说明已处理的审查反馈[rc:3643020506] N/A 豁免与无条件要求之间的冲突 — 已实现发现： 针对 docs/types/refactor PR 的 N/A 豁免（在原始提交中添加）与同一节中的两个无条件要求存在冲突：
决定： 实现。矛盾确实存在——在纯文档 PR 上，triage agent 要么浪费一个 tmux 会话产出无意义的输出，要么产出不一致的 Stage 2 评论。 变更（2 行）：
两个豁免与现有的 N/A 段落以及彼此之间保持一致。 验证
未涉及 TypeScript 或测试文件；无需运行 Vitest 或集成测试。 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline. 3 Suggestion-level finding(s) could not be anchored to a changed line and were dropped; nothing further to act on here.
— qwen3.7-max via Qwen Code /review
…cation # Conflicts: # .qwen/skills/triage/references/pr-workflow.md
Collaborator
|
🤖 Addressed the latest review feedback (round 2/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 2/100 轮）。改动内容与我反驳保留之处如下： Review feedback addressed[rc:3643571558] — Suggestion (automated reviewer): the ⛔ paragraph "The tmux output IS the review" unconditionally mandated tmux output ("…the review is worthless"), directly contradicting the Decision: Implemented. The suggestion is correct, valuable, and in scope — it completes the alignment this PR's own title describes ("align Mandatory and BEFORE POSTING checks with N/A escape hatch"). The Mandatory line and the BEFORE POSTING check already carried the escape hatch; this middle ⛔ paragraph was the one place still missing it, so a triage agent reading Stage 2b sequentially would hit the contradiction. Change (
Conflict resolution (
|
Collaborator
Author
Review:
|
Collaborator
|
🤖 Addressed the latest review feedback (round 3/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 3/100 轮）。改动内容与我反驳保留之处如下： Review feedback addressedBoth non-blocking suggestions from @wenshao's LGTM review have been implemented. 1. Restate the CI boundary inside the scale-evidence blockDecision: Implemented. The "Scale the evidence to the change" block now carries 2. Fork-sandboxing back-reference from the terminal-capture bulletDecision: Implemented. The Verification
中文说明已处理的审查反馈@wenshao 的 LGTM 审查中两条非阻塞建议均已实现。 1. 在「按改动缩放证据」块内重申 CI 边界决定： 已实现。 「Scale the evidence to the change」块的标题行现在带有 2. 从 terminal-capture 条目回指 fork 沙箱规则决定： 已实现。
验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline. 1 Suggestion-level finding(s) could not be anchored to a changed line and were dropped; nothing further to act on here.
— qwen3.7-max via Qwen Code /review
Collaborator
|
🤖 Addressed the latest review feedback (round 4/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 4/100 轮）。改动内容与我反驳保留之处如下： Addressed review feedback1. Stage summary table inconsistency (rc:3644377645)Finding: The Stage summary table stated "(+ tmux capture on local runs)" unconditionally, while the body text now limits tmux to PRs with user-visible behavioral changes. Decision: Implemented. Updated the table row to "(+ tmux capture on local runs when user-visible)" so the at-a-glance summary matches the detailed mandate below. Change: 2. SKILL.md cross-file inconsistency (rc:3644377659)Finding: SKILL.md unconditionally stated "Without inlined terminal output, the review is incomplete and useless", contradicting the new N/A escape hatch in pr-workflow.md for docs/types/refactor PRs. Decision: Implemented. Updated SKILL.md's local-invocation paragraph to mirror pr-workflow.md's conditional mandate: tmux is required "for PRs with user-visible behavioral changes", docs/types/refactor PRs state Change: Verification
中文说明已处理的审查反馈1. Stage 摘要表格不一致 (rc:3644377645)发现： Stage 摘要表格中无条件地写了"(+ tmux capture on local runs)"，但正文已将 tmux 限定为"有用户可见行为变更的 PR"。 决定： 已实现。将表格行更新为"(+ tmux capture on local runs when user-visible)"，使概览表格与下方的详细说明保持一致。 变更： 2. SKILL.md 跨文件不一致 (rc:3644377659)发现： SKILL.md 无条件地声明"Without inlined terminal output, the review is incomplete and useless"，与 pr-workflow.md 中新增的 N/A 豁免（针对 docs/types/refactor PR）相矛盾。 决定： 已实现。更新 SKILL.md 的本地调用段落，使其与 pr-workflow.md 的条件性要求一致：tmux 仅对"有用户可见行为变更的 PR"强制要求，docs/types/refactor PR 标注 变更： 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
doudouOUC
reviewed
Jul 24, 2026
doudouOUC
left a comment
doudouOUC
left a comment
Collaborator
There was a problem hiding this comment.
— qwen3.7-max via Qwen Code /review
Collaborator
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： No action required this roundThe only feedback newer than the last evaluation is a single review-status note
This is informational, not a change request. It records that the reviewer There is therefore nothing to address:
No code or documentation change is warranted. The PR (a docs-only update to the 中文说明本轮无需任何改动本次评估之后新增的反馈只有一条来自自动审查者的审查状态说明：
这是一条信息性说明，并非改动请求。它记录的是：审查者在审查时 CI 仍在运行，因此把自己的批准状态从 Approve 降级为 Comment。它没有指出任何缺陷，没有提出任何建议，也没有要求任何改动。 因此没有任何需要处理的内容：
无需任何代码或文档改动。本 PR（仅更新 triage skill 文档：为 tmux 输出强制要求增加 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline. 2 Suggestion-level finding(s) could not be anchored to a changed line and were dropped; nothing further to act on here.
— qwen3.7-max via Qwen Code /review
Collaborator
|
🤖 Addressed the latest review feedback (round 5/100). What changed, and what I pushed back on: · 已处理最新评审反馈（第 5/100 轮）。改动内容与我反驳保留之处如下： Review feedback addressed[rc:3644956010] BEFORE POSTING gate incomplete for new evidence types — ImplementedFinding: The "Scale the evidence" block introduces three evidence types (screenshots for UI changes, measurements for performance claims, clean-state numbers for build/test claims), but the BEFORE POSTING enforcement gate only checked for a fenced terminal capture or Change: Extended the BEFORE POSTING gate's local-run clause with one sentence: File: Conflict notesNo conflicts ( Verification
中文说明已处理的审查反馈[rc:3644956010] BEFORE POSTING 门控对新证据类型不完整 — 已实现发现： "Scale the evidence" 块引入了三种证据类型（UI 变更的截图、性能声明的测量数据、构建/测试声明的干净状态数据），但 BEFORE POSTING 执行门控仅检查是否有围栏式终端捕获或 变更： 在 BEFORE POSTING 门控的本地运行子句中扩展了一句话： 文件： 冲突说明无冲突（ 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审；第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed — no blockers. 2 Suggestion-level finding(s) could not be anchored to a changed line and were dropped; nothing further to act on here.
— qwen3.7-max via Qwen Code /review
Collaborator
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下： Autofix review round — no action neededThe latest review feedback on this PR contains no actionable items, so no
The branch is left unchanged. 中文说明Autofix 审查轮次 —— 无需处理本次 PR 的最新审查反馈不包含任何可处理的事项，因此本轮未做任何改动。
分支保持原样，未做改动。 Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
Collaborator
Author
|
@qwen-code /triage |
Collaborator
qwen-code-ci-bot
approved these changes
Jul 24, 2026
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
gwinthis
self-requested a review
July 24, 2026 13:17
gwinthis
approved these changes
Jul 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Strengthens the
/triageskill's Stage 2b (real-scenario testing) so the bot's verification scales to the change and reaches the depth of a thorough manual review — using harnesses this repo already ships, no new infrastructure.Why
/triagealready does a lot right: an isolated worktree, a mandatory tmux before/after (installed build vsnpm run dev), inlinecapture-paneevidence, head-SHA anchoring. But for the changes where verification matters most, that floor falls short — compare it to a strong manual pass like #7302's round-2 verification:capture-panetext only — can't show what's highlighted or where the caret isTS2307)npm run devin a shared tree — results can be contaminatedHow
Two things made this a small, safe change rather than new infrastructure:
.qwen/skills/terminal-capture(node-pty → xterm.js → Playwright PNG, "used for visual verification during PR reviews"), and@lydell/node-pty+@xterm/headlessare already dependencies.So Stage 2b gains one "scale the evidence to the change" block that points at what already ships:
terminal-captureskill for pixel-accurate styled screenshots (it needsnpx playwright install chromium, installed on demand), and name the on-screen oracle, showing before AND after so the change is visible, not asserted.N/A; and when verification was non-trivial, separate merge-blockers from standing follow-ups with a reproducibility note.Reviewer Test Plan
npx vitest run scripts/tests/qwen-triage-workflow.test.js— 12/12 (the tests pin the workflow; this changes only the skill reference, so nothing breaks).terminal-captureskill it points at exists at.qwen/skills/terminal-capture/SKILL.md, and its deps (@lydell/node-pty,@xterm/headless) are inpackage.json.Risk & Scope
playwright install chromiumstep toqwen-triage.ymlis a small separate follow-up — deliberately not bundled here.Linked Issues
Motivated by #7302's thorough manual verification — the goal is for
/triageto reach that depth automatically, scaled to the change.中文说明
本 PR 做了什么
强化
/triageskill 的 Stage 2b(真机测试),让 bot 的验证按改动缩放深度、达到一次扎实人工评审的水准 —— 全部指向仓库现有 harness,不加新基建。为什么
/triage已经做对很多:隔离 worktree、强制 tmux before/after(installed vsnpm run dev)、内联capture-pane证据、head-SHA 锚定。但在最需要验证的改动上这个地板不够 —— 对比一次扎实的人工验证(如 #7302 第二轮):像素级带样式截图(高亮 pill、光标块)、干净安装下的构建/测试数字(软链树会出跨包TS2307假错)、点名屏幕上的 oracle、对性能改动实测、区分阻塞项与遗留项 + 方法论 —— 这些 Stage 2b 之前都没要求。怎么做
两点让它成为小改动而非新基建:pty+xterm harness 已存在(
.qwen/skills/terminal-capture,node-pty → xterm.js → Playwright PNG,"用于 PR review 视觉验证"),@lydell/node-pty+@xterm/headless已是依赖;Stage 2b 已有 tmux before/after 地基。于是 Stage 2b 加一个"按改动缩放证据"块,指向现有资源:UI/交互改动 →
terminal-capture(像素级样式截图;需npx playwright install chromium,按需装)并点名 oracle、展示前后;构建/测试数字 → 取自干净状态,软链树的跨包 TS 错报为环境问题而非 PR 缺陷;性能改动 → instrument 真实构建产物、老 vs 新、给数字;docs/refactor 保持N/A;验证非平凡时分离阻塞项与遗留项 + 方法论。评审验证
npx vitest run scripts/tests/qwen-triage-workflow.test.js—— 12/12(测试锁的是 workflow;本 PR 只改 skill 引用,无破坏)。terminal-captureskill 存在于.qwen/skills/terminal-capture/SKILL.md,其依赖在package.json。风险与范围
qwen-triage.yml加一步playwright install chromium是另一个小跟进 —— 特意没打包进来。关联 Issue
源于 #7302 的扎实人工验证 —— 目标是让
/triage自动达到那种深度,并按改动缩放。