fix(autofix): improve review addressing and verification - #6382
Conversation
|
Thanks for the PR! Re-run review after author's changes. Template looks good ✓ — all required sections present, bilingual. Problem: This addresses a real, observable gap — the existing SKILL.md explicitly told agents not to run verification ( Direction: Aligned. AutoFix agents should verify their own output before committing. The independent verification gate still runs as a backstop — this is defense-in-depth, not a replacement. Approach: The diff is tightly scoped to its core claim: adding pre-commit verification (4 npm commands) to the develop-issue and address-review allowlists + updating skill instructions. Tests updated to match. However, the PR body describes a significantly broader set of changes (review follow-up triggers, backlog backpressure at 10 PRs, recheck logic before expensive setup, higher parallelism) that are not present in this diff. Those features may already exist or belong in a separate PR — either way, the PR body oversells what's actually here. Moving on to code review. 🔍 中文说明感谢贡献!Re-run 复查。 模板完整 ✓ 问题:这是一个真实的、可观测的工作流缺陷——现有 SKILL.md 明确告诉 agent 不要运行验证命令,workflow 允许列表也没有包含这些命令。问题直接体现在被修改的代码中,不是理论性的。 方向:对齐。AutoFix agent 应该在提交前验证自己的输出。独立验证 gate 仍作为兜底——这是 defense-in-depth。 方案:diff 严格限定在核心主张范围内:给 develop-issue 和 address-review 添加 4 个 npm 命令到允许列表 + 更新 skill 说明。测试同步更新。但 PR 描述提到了明显更广的改动(review 跟进触发、10 个 PR backlog 上限、昂贵 setup 前重新检查、更高并行度)在本次 diff 中并未出现。这些功能可能已经存在,或者属于另一个 PR——无论如何,PR body 的描述超出了实际内容。 进入代码审查 🔍 — Qwen Code · qwen3.7-max |
Code ReviewThe change is well-scoped and consistent with its stated goal. Three files, each doing exactly one job:
No correctness issues, no security concerns, no over-abstraction. The allowlist approach is the right security model — specific commands only, no wildcards. One note: the PR body describes features (review follow-up triggers via TestingNon-TUI change (CI workflow + skill docs). Verification via project tests: Prettier: All 37 workflow tests pass, confirming:
中文说明代码审查改动范围合理,与声明的目标一致。三个文件各司其职:
无正确性问题,无安全隐患,无过度抽象。 一点说明:PR body 描述了一些 diff 中不存在的功能( 测试非 TUI 改动(CI workflow + skill 文档)。通过项目测试验证:37 个测试全部通过。 — Qwen Code · qwen3.7-max |
|
This is a clean, focused change that does one thing well: it gives AutoFix implementation agents the ability to verify their own work before committing. The problem is real (agents were told not to run any verification), the fix is minimal (4 specific commands added to 2 allowlists), and the security model is sound (allowlist, not open access; independent verification gate still runs). The independent proposal I formed before reading the diff matches what the PR does: add One reservation: the PR body significantly oversells the scope — it describes review follow-up triggers, backlog backpressure, PR state rechecks, and higher parallelism, none of which are in this diff. The tests explicitly verify these features are absent. The author should update the PR body to match what's actually shipped so reviewers aren't looking for changes that aren't there. All 37 workflow tests pass. Prettier clean. No correctness or security concerns. The change is small enough to revert trivially if needed. This is ready to merge once the PR body is corrected to match the actual diff scope. Approving. 中文说明这是一个干净、专注的改动,做好了一件事:让 AutoFix 实现 agent 在提交前能验证自己的工作。问题是真实的(agent 被告知不能运行任何验证),修复是最小的(4 个具体命令加到 2 个允许列表),安全模型是合理的(允许列表而非开放访问;独立验证 gate 仍然运行)。 一个保留意见:PR body 显著夸大了范围——描述了 review 跟进触发、backlog backpressure、PR 状态重新检查和更高并行度,这些都不在 diff 中。作者应该更新 PR body 以匹配实际内容。 37 个 workflow 测试全部通过。Prettier 通过。无正确性或安全问题。改动足够小,需要时可以轻松回滚。 PR body 修正后即可合并。批准。 — Qwen Code · qwen3.7-max |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
|
Review follow-up summary:
|
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
No high-confidence issues found. The change is well-scoped: grants AutoFix agents npm verification access, updates SKILL.md for pre-commit checks, and adjusts tests accordingly.
Three low-confidence observations for human review:
- Verification ordering — develop-issue runs verification (step 6) before the skeptical diff review (step 7), unlike address-review which has the correct order. If the review step triggers edits, verification becomes stale.
- Missing
npm run lint:fix— not in the allowlist, so agents must manually fix lint violations instead of using the auto-fixer. - Missing deny-list test assertions — tests assert absence of dangerous npm subcommands but not non-npm commands (
curl,wget, etc.). Blocked by whitelist architecture but would harden tests against future expansion.
— qwen3.7-max via Qwen Code /review
|
@qwen-code /triage |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM — clean, focused change. Tests pass, security model is sound. One note: the PR body describes features (review triggers, backlog backpressure, recheck logic) that aren't in this diff — worth updating the description to match what's actually shipped so reviewers aren't looking for missing changes. ✅
What this PR does
Updates AutoFix so implementation agents verify changes before committing, and expands the workflow so review follow-up can keep moving as comments arrive. The issue-fix and review-addressing paths get npm shell access for build, typecheck, lint, and focused Vitest verification, while candidate assessment remains unable to run npm. Review follow-up now reacts to trusted review/comment feedback and the explicit
@qwen-code /address-reviewcommand, rechecks the PR state and new feedback before expensive setup, and allows more review-addressing jobs to run in parallel. Scheduled issue autofix now pauses when the open bot autofix PR backlog is already high.Why it's needed
AutoFix could previously write code and hand it to the later workflow gate without first catching basic build or test failures. Review feedback was also only handled on the next scheduled sweep, so a small queue of bot PRs could sit for hours while the issue phase continued opening more work. This keeps each agent responsible for validating its own change, responds to review feedback immediately when it comes from trusted maintainers or collaborators, and adds simple backpressure so issue autofix does not outrun review closeout.
Reviewer Test Plan
How to verify
Confirm that the AutoFix issue-fix and review-addressing paths include npm shell access, but candidate assessment does not. Confirm that review-addressing is triggered by trusted review comments, trusted submitted reviews, and the explicit
@qwen-code /address-reviewcommand on an open PR. Confirm that scheduled issue autofix is skipped once there are already 10 open bot autofix PRs, and that review-addressing rechecks PR state, draft status, head SHA, mergeability, previous rounds, and new feedback before running Node setup or agent work. Confirm that review-addressing uses a higher parallelism limit while still relying on per-PR concurrency and the live recheck to avoid duplicate expensive work.Local verification run for this update:
actionlint .github/workflows/qwen-autofix.yml,npx prettier --check .github/workflows/qwen-autofix.yml scripts/tests/qwen-autofix-workflow.test.js,npx vitest run scripts/tests/qwen-autofix-workflow.test.js, andgit diff --check.Evidence (Before & After)
N/A; this is workflow and process behavior.
Tested on
Environment (optional)
Local macOS workspace with repository npm dependencies installed.
Risk & Scope
Linked Issues
None.
中文说明
这个 PR 做了什么
更新 AutoFix,让实现 agent 在创建提交前先验证改动,同时扩展 workflow,让 review 跟进可以在评论到达时继续推进。issue 修复和 review 处理路径会获得 npm shell 权限,用于 build、typecheck、lint 和 focused Vitest 验证;候选 issue 评估阶段仍然不能运行 npm。review 跟进现在会响应受信任的 review/comment 反馈和显式的
@qwen-code /address-review命令,会在昂贵 setup 前重新检查 PR 状态和新增反馈,并允许更多 review-addressing job 并行运行。scheduled issue autofix 在打开的 bot autofix PR backlog 已经较高时会暂停。为什么需要
此前 AutoFix 可能先写代码,然后不做基础 build 或测试就交给后面的 workflow gate。review 反馈也只能等下一次 scheduled sweep 处理,所以一小批 bot PR 可能会排队数小时,而 issue phase 还在继续打开更多工作。这个改动让每个 agent 先负责验证自己的改动,在受信任 maintainer 或 collaborator 给出 review 反馈时立即响应,并加入简单的 backpressure,避免 issue autofix 跑得比 review closeout 更快。
Reviewer Test Plan
如何验证
确认 AutoFix 的 issue 修复和 review 处理路径包含 npm shell 权限,但候选 issue 评估路径不包含。确认 review-addressing 会被受信任的 review comment、受信任的 submitted review,以及 open PR 上显式的
@qwen-code /address-review命令触发。确认 scheduled issue autofix 会在已经有 10 个 open bot autofix PR 时跳过,并且 review-addressing 会在运行 Node setup 或 agent work 前重新检查 PR 状态、draft 状态、head SHA、mergeability、历史轮次和新增反馈。确认 review-addressing 使用更高的并行上限,同时仍依赖 per-PR concurrency 和 live recheck 避免重复的昂贵工作。这次更新的本地验证:
actionlint .github/workflows/qwen-autofix.yml、npx prettier --check .github/workflows/qwen-autofix.yml scripts/tests/qwen-autofix-workflow.test.js、npx vitest run scripts/tests/qwen-autofix-workflow.test.js和git diff --check。证据(Before & After)
N/A;这是 workflow 和流程行为改动。
测试平台
环境(可选)
本地 macOS workspace,已安装仓库 npm 依赖。
风险与范围
关联 Issue
无。