feat(core): add a bundled goal-draft skill that writes verifier-judgeable Goals - #10002
Conversation
…able Goals `/goal` accepts any string, but the Goal verifier only accepts transcript evidence, so objectives like "all tests pass and the lint is clean" or "clean up the auth module" never accumulate proof and loop until a limit. `/goal-draft <intent>` is a read-only bundled skill that checks whether a Goal is warranted, reads the workspace for the real check commands, asks at most one round of multiple-choice questions, drafts the objective in a fixed Outcome / Done when / Must not / Budget / On block / Context contract, runs a self-check against the verifier rules, and prints a `/goal set …` line the user can run as-is. It never starts the work and never sets the Goal. Also adds docs/users/features/goals.md (how a Goal is judged, how to write one), rows in commands.md, a pointer from headless.md, a design note under docs/design/, and a web-shell Goals dialog placeholder that shows a check, a guardrail, and a budget. Claude-Session: https://claude.ai/code/session_01FV7i3w7egJ2kMw4AhQC38Z
|
@qwen-code /takeover |
|
🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. This is a fork PR, so the first round comes from the next scheduled scan (usually within minutes). Remove the 中文说明🤝 已接管:autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突,直到移除标签或达到轮次上限。本 PR 来自 fork,首轮处理将由下一次定时扫描执行(通常几分钟内)。移除 |
|
Thanks for the PR — this one is unusually well grounded. Template looks good ✓ Problem: this is a real, verifiable gap, not a theoretical one. I checked the tree: Direction: aligned. Goals are an established surface here (goal loop, verifier, web-shell dialog, headless docs), and Claude Code's CHANGELOG shows sustained Size: core path touched ( Approach: the scope feels right for what it claims. Everything is additive content — one SKILL.md, one colocated test, docs, and a placeholder swap — with no runtime code changes, and the heavier follow-ups ( Risk: no high-risk paths matched (no streaming/parser/shell/MCP/sandbox/relaunch surfaces). Note for later stages: this is a fork PR touching Moving on to code review. 🔍 中文说明感谢贡献——这个 PR 的依据非常扎实。 模板完整 ✓ 问题:这是一个真实、可验证的产品缺口,不是理论性问题。我在代码树里核实过: 方向:对齐。Goals 在这里是成熟的功能面(goal 循环、verifier、web-shell 对话框、headless 文档),Claude Code 的 CHANGELOG 也显示该领域在持续投入(check-in 退避、evaluator 修复、恢复处理),帮助用户写出可判定的 objective 符合产品方向。不涉及需要升级的敏感面(auth、sandbox、telemetry、公共契约)。 规模:触及核心路径( 方案:范围与其声明的目标匹配。全部是增量内容——一个 SKILL.md、一个同目录测试、文档、一处占位文案替换——没有运行时代码改动,更重的后续项( 风险:未命中任何高风险路径(不涉及 streaming/parser/shell/MCP/sandbox/relaunch 等)。供后续阶段注意:这是触及 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Code reviewI proposed my baseline before reading the diff: for "users write Goals the verifier can't judge", the minimal fix is (a) document the verifier's evidence rules, (b) a content-only drafting aid, and defer any deterministic lint on Then I checked every load-bearing claim against the tree, and they hold up:
No critical issues. One cosmetic nit, non-blocking: the new web-shell placeholder carries Markdown backticks ( I also read SKILL.md as prompt content, since a fork-authored prompt is untrusted input: it contains no tool-widening instructions, no exfiltration patterns, no injection-style directives — it spends its effort teaching the model to refuse starting work, which is safety-positive if anything. TestingUnattended CI run — I did not build or execute any PR code; the evidence below is the PR's own CI read through the API, fetched once (no polling). Final CI results for
One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。 The macOS/Windows/integration skips are the repo's Everything user-visible that the author reports (the headless 中文说明代码审查我在看 diff 之前先给出自己的基线方案:针对"用户写出 verifier 无法判定的 Goal",最小修法是 (a) 把 verifier 的证据规则写成文档,(b) 提供一个纯内容的起草辅助,把 随后我逐条核实了 PR 的关键论断,全部成立:
无阻塞问题。一个非阻塞的小瑕疵:web-shell 新占位文案里带了 Markdown 反引号( 我也把 SKILL.md 当作 prompt 内容审了一遍——fork 提供的 prompt 是不可信输入:其中没有扩权指令、没有外泄模式、没有注入式指令;它花大量篇幅教模型拒绝开工,反而是安全正向的。 测试无人值守 CI 运行——我没有构建或执行任何 PR 代码;以下证据是通过 API 读取的 PR 自身 CI,一次性抓取(不轮询)。 macOS/Windows/集成测试的跳过是仓库 作者报告的所有用户可见效果(headless — Qwen Code · qwen3.8-max Reviewed at |
|
Confidence: 4/5 — clean across every stage; the only open item is CI still in flight, plus one cosmetic nit. Stepping back: this PR does one thing — teach users to write Goals the verifier can actually judge — and does it with zero runtime code. My independent proposal before reading the diff was exactly this shape (docs + a content-only drafting aid, lint deferred), and the PR matches it; I found no simpler path that still engages at the moment of intent. The grounding is unusually honest: every load-bearing claim about Reservations, named plainly: the new web-shell placeholder carries literal backticks (cosmetic), and the "the model drafts a good objective" part is prompt-dependent and rests on the author's Linux headless runs until someone exercises it — the One context note for the maintainer: this author has a batch of goal-related PRs open right now. I evaluated this one on its own merits — it stands on them — but it may be worth reviewing it alongside its siblings (e.g. the goal-summary and stall-detection ones) so the Goals surface evolves as one decision rather than several. CI is still running on this commit (Linux unit suite + web-shell visuals at fetch time), so approval is deferred until CI lands green on 中文说明置信度:4/5 —— 各阶段都很干净;唯一未定的是 CI 仍在运行,外加一个小瑕疵。 退一步看:这个 PR 只做一件事——教用户写出 verifier 真正能判定的 Goal——而且零运行时代码。我在读 diff 之前的独立提案正是这个形态(文档 + 纯内容起草辅助,lint 推迟),PR 与之吻合;我想不到更简单、又能在意图表达当下介入的方案。依据异常扎实:关于 坦率地说出两点保留:web-shell 新占位文案带字面反引号(纯外观问题);"模型能起草出好的 objective"这一点依赖 prompt,在有人实际运行之前只以作者的 Linux headless 运行结果为依据——审查评论里的 给维护者的一个背景提示:该作者目前有一批 goal 相关的 PR 在开。我按各自的价值独立评估了这个——它站得住——但值得和它的兄弟 PR(比如 goal 摘要和停滞检测)放在一起审,让 Goals 这个功能面作为一个整体决策演进,而不是若干个分散的决定。 CI 在该提交上仍在运行(抓取时是 Linux 单元测试套件 + web-shell visuals),因此批准推迟到 CI 在 — Qwen Code · qwen3.8-max Reviewed at |
🖼️ web-shell visual previewRendered against a mock daemon (no real backend): the PR base vs this PR head Screenshots · before / afterFull-resolution recordings (.webm) are attached to the workflow run. — Qwen Code · web-shell visuals |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Partially reviewed — gaps disclosed. Suggestions are inline.
Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.
Test Plan (not a blocker): src/auth/login.js — no such file or directory; test/login.test.js — no such file or directory.
中文说明
仅完成部分审查,审查缺口已披露。 建议见行内评论。
未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。
Test Plan(非阻断):src/auth/login.js — no such file or directory; test/login.test.js — no such file or directory。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 1/100 轮)。改动内容与我反驳保留之处如下: Autofix review round for PR #10002Round commit: Feedback points and dispositions[rc:3852018989] [rc:3852019015] [rc:3852019024] [rc:3852019034]
[rv:5017787224] review body — no code change needed. [ic:5408367318] web-shell visual preview: one or more scenarios failed to render on this head — investigated, no code-level link to this PR. Files changed
Verification
中文说明PR #10002 的 Autofix 审查轮次轮次提交: 反馈点与处理结果[rc:3852018989] [rc:3852019015] [rc:3852019024] [rc:3852019034]
[rv:5017787224] 审查正文 — 无需代码改动。 [ic:5408367318] web-shell 视觉预览:本 head 上有一个或多个场景渲染失败 — 已调查,未发现与本 PR 的代码级关联。 变更文件
验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Not explored to full depth (tool budget reached): "agent 1a": actually executing SKILL.test.ts and localCommands.test.ts via vitest — this worktree has no node_modules (vitest fails to resolve), and a full monorepo ….
Test Plan (not a blocker): src/auth/login.js — no such file or directory; test/login.test.js — no such file or directory; npm run build — exit 1.
Deferred under the convergence posture (round 2, not a blocker) — recorded, not requested in this round:
packages/core/src/skills/bundled/goal-draft/SKILL.test.ts:140 — [probe] ordering assertion is the only pin on the up-front 'not doing the work' sentence; deleting it passes silently (indexOf returns -1)packages/core/src/skills/bundled/goal-draft/SKILL.test.ts:106 — [probe] contract-label assertions use body-wide toContain; 5 of 6 labels also occur in SKILL.md's example table, so a template-label rename survivespackages/core/src/skills/bundled/goal-draft/SKILL.test.ts:128 — [probe] test 'ends with the self-check list and an explicit stop' has no positional pin; relocating the stop line mid-file passes silently
中文说明
未探索到全部深度(达到工具调用预算):"agent 1a":actually executing SKILL.test.ts and localCommands.test.ts via vitest — this worktree has no node_modules (vitest fails to resolve), and a full monorepo …。
Test Plan(非阻断):src/auth/login.js — no such file or directory; test/login.test.js — no such file or directory; npm run build — exit 1。
收敛姿态下延后(第 2 轮,非阻断)——已记录,本轮不要求修改:共 3 条(原文未翻译,列表见上方英文部分)。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
🤖 Addressed the latest review feedback (round 2/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 2/100 轮)。改动内容与我反驳保留之处如下: Autofix review round — PR #10002Commit: Feedback points and dispositions[rc:3853636272] [Critical]
|
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.
Test Plan (not a blocker): src/auth/login.js — no such file or directory; test/login.test.js — no such file or directory; npm test — exit 1; 22 tests pass — this review observed 21490, 1702, 23961, 1659, 601, 4226, 627 passed.
中文说明
未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。
Test Plan(非阻断):src/auth/login.js — no such file or directory; test/login.test.js — no such file or directory; npm test — exit 1; 22 tests pass — this review observed 21490, 1702, 23961, 1659, 601, 4226, 627 passed。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下: 🤖 Addressed the latest review feedback (round 3/100). No code changes this round — the only feedback item was traced outside this PR's footprint and deferred to the follow-up queue. · 已处理最新评审反馈(第 3/100 轮)。本轮无代码改动 —— 唯一的反馈项被追溯到本 PR 范围之外,已移交后续处理队列。 Autofix review round — PR #10002Commit: none (no-action round). No base merge ( Feedback points and dispositions[rv:5021258031] "Downgraded from Approve to Comment: CI failing: Test (windows-latest, Node 22.x)" — root-caused outside this PR; failing tests deferred to the follow-up queueThe review carries zero findings (its ledger shows
Why this is a lane problem, not a PR problem:
All three failures are verified real (exact assertion diffs from the lane) but their fixes live in Verification
中文说明🤖 已处理最新评审反馈(第 3/100 轮)。本轮无代码改动 —— 唯一的反馈项被追溯到本 PR 范围之外,已移交后续处理队列。 Autofix 评审轮次 — PR #10002提交:无(无改动轮次)。未合并 base( 反馈项与处置[rv:5021258031] "已从批准降级为评论:CI failing: Test (windows-latest, Node 22.x)" —— 根因定位在本 PR 之外;失败测试已移交后续处理队列该评审本身没有任何 finding(其台账显示
为什么这是平台通道(lane)的问题,而不是本 PR 的问题:
三个失败均已核实为真实存在(来自该通道的精确断言 diff),但修复分别位于 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Partially reviewed — gaps disclosed.
Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.
Test Plan (not a blocker): src/auth/login.js — no such file or directory; test/login.test.js — no such file or directory.
中文说明
仅完成部分审查,审查缺口已披露。
未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。
Test Plan(非阻断):src/auth/login.js — no such file or directory; test/login.test.js — no such file or directory。
— qwen3.8-max via Qwen Code /review (v0.22.0)
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 61 passed · 0 failed · 61 total Flakiness gate: ✅ 2 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定:✅ 通过 · 可合入(agent 判定)沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查。 脚本断言:61 通过 · 0 失败 · 61 总计 抖动门:✅ 2 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #10002 verification —
|
| # | Cell / oracle | base 0756be0 (control) |
head d15ba05 |
|---|---|---|---|
| 1 | Bundled discovery (SkillManager.refreshCache + listSkills) |
11 skills, no goal-draft, 0 parse errors |
12 skills incl. goal-draft; allowedTools = [get_goal, read_file, glob, grep_search]; model-invocable; user-invocable; hint parsed |
| 2 | Model-visible <available_skills> block (collectAvailableSkillEntries + renderAvailableSkillsBlock) |
no goal-draft entry |
entry present with XML-escaped description, (bundled) |
| 3 | SkillTool.validateToolParams({skill:'goal-draft'}) |
Skill "goal-draft" not found. Available skills: batch, … |
null (accepted) |
| 4 | loadSkill/loadSkillForRuntime |
null |
config returned; body's last line is the stop instruction |
| 5 | web-shell skillDescriptionKey + getTranslator EN/ZH |
undefined (menu falls back to authored English in zh UI) |
skilldesc.goalDraft, EN + ZH descriptions resolve |
| 6 | goals.conditionPlaceholder EN/ZH (GoalsDialog textarea) |
e.g. all tests pass and the lint is clean / 旧中文 |
new strings with check + guardrail + budget, both locales |
| – | Positive controls (both arms identical) | bundled set = head minus goal-draft; /review localization identical; /goal builtin present |
same |
The base arm is the load-bearing proof: the identical harness fails to find the skill at every layer on base and succeeds at every layer on head, with no other delta in the bundled set.
Vacuity / mutation matrix (12/12 as expected)
Witnesses: 02-core-skill-mutation-matrix.png, 03-webshell-hunk-mutation-matrix.png. Controls green on both suites before mutating.
| Mutation | Suite | Outcome |
|---|---|---|
| M0 unmutated control | SKILL.test.ts | 8/8 green |
M1 grant ask_user_question |
SKILL.test.ts | caught — both grant tests fail (list assertion + permission stays ask) |
M2 disable-model-invocation: true |
SKILL.test.ts | caught — invocability test |
| M3 swap Step 2/3 headings | SKILL.test.ts | caught — step-order test |
| M4 delete final stop line | SKILL.test.ts | caught — expected/received mismatch on the last line |
M5 delete Budget: from the ```text template |
SKILL.test.ts | caught — contract-label test |
| M6 reword unpinned table prose | SKILL.test.ts | survives (expected — prose boundary, not a defect) |
M7 delete description: frontmatter |
bundled-skills.integration.test.ts | caught — goal-draft/SKILL.md parses with required fields fails |
| W0 unmutated control | localCommands.test.ts | 10/10 green |
W1 drop 'goal-draft' menu key |
localCommands.test.ts | caught — expected undefined to be 'skilldesc.goalDraft' |
W2 drop EN skilldesc.goalDraft |
localCommands.test.ts | caught — key falls through to itself |
| W3 revert EN placeholder | localCommands.test.ts | survives — no test pins the placeholder (see Findings) |
All reverted files verified clean afterwards (git status --porcelain empty). Note the PR's own tests parse via the extension parser (parseSkillContent), while my A/B exercised the runtime bundled parser (SkillManager.parseSkillContent) — both accept the shipped frontmatter identically.
Targeted gates (9/9) and pre-existing failures
Witness: 04-targeted-gates.png.
| Gate | Result |
|---|---|
| Reviewer Test Plan step 1 (the two named core files) | 23/23 pass |
core src/skills/ domain |
401 passed, 4 failed — pre-existing: the same 4 test names fail byte-identically on the base arm (A/A); they hardcode TEST_HOME=/home/user while this container's HOME differs; PR touches neither file |
| web-shell full suite | 198 files / 4235 tests, all pass |
| core typecheck / web-shell typecheck | exit 0 / exit 0 |
| eslint on the four changed code files | clean; liveness probe (planted unused var) reported as expected |
dist/bundled/goal-draft/SKILL.md |
byte-identical to src; SKILL.test.ts correctly not shipped; built web-shell SPA assets carry the new EN description + placeholder |
Reviewer Test Plan walkthrough
- Step 1 (two core test files): performed, 23/23.
- Step 2 (headless
qwen -p "/goal-draft …"with a real model): not performable here — the sandbox has no model credentials and no network by contract; the oracle would be model-generated prose. The integration mechanics it exercises (discovery → validation → body injection) are covered by A/B cells 1–4 instead. - Step 3 (interactive
/goal-dcompletion): data source verified — the CLI'sBundledSkillLoaderbuilds slash commands dynamically fromskillManager.listSkills({level:'bundled'})(no static list to forget), and cell 1 proves that list gainsgoal-draft. TUI keystrokes themselves not driven. - Step 4 (web-shell Goals dialog placeholder): verified at data level (cells 5–6),
GoalsDialog.tsxconsumes the key on the condition textarea, and the built SPA assets contain both new strings. Browser not rendered.
Corrections to the PR description
- "22 tests pass" — the two named files run 23 tests at this head (15 integration + 8): the integration
it.eachgained a 12th bundled directory. Cosmetic; the count in the body predates the final skill set. - "No runtime code changes here" — the web-shell client runtime is changed (
localCommands.ts,i18n.tsx); what is untouched is the Goal/verifier runtime. The intent reads correctly, but the sentence as written is inaccurate.
Findings
S1 (Suggestion, non-blocking) — the Goals-dialog placeholder is unpinned. W3 survives: no test asserts goals.conditionPlaceholder in either locale, so the PR's most user-visible web-shell string can silently regress (the old placeholder was equally unpinned, so this is an inherited gap, not a regression). Suggested follow-up: extend localCommands.test.ts (or a small i18n test) to assert the placeholder carries a check, a guardrail, and a budget in both locales. Not a merge condition — the string itself is correct in source and in the built SPA.
Completeness notes (explicitly not merge conditions): M6 documents that the Weak→strong table prose is intentionally unpinned; computer-use/coordinate are absent from SKILL_DESCRIPTION_KEYS — pre-existing and by design (unlisted skills fall back to their authored description).
Not covered
- Live model runs of
/goal-draft(headless or interactive) — no credentials/network in this sandbox; the[ASSUMPTION]headless behavior and prose quality are model behavior, unverifiable offline. - TUI completion keystrokes and browser rendering of the Goals dialog (data-level + built-asset checks instead).
- Per-commit attribution — checkout is depth 2 (merge + 2 parents reachable; metadata lists 5 commits), so only the aggregate
HEAD^1..HEADdiff was verified. - Core suites outside
src/skills/; repo-wide lint/format; macOS/Windows. - The 4
skill-manager.test.tsfailures are environmental on this container (identical on base); they would need a normal runner to confirm green there.
Methodology
Environment: CI merge-ref checkout (HEAD = merge, HEAD^1 = base tip, HEAD^2 = PR head), npm ci + npm run build pre-run at head. Base arm: scratch worktree at HEAD^1 with packages/core rebuilt in-place (root node_modules reused; the PR leaves package.json/lockfile untouched, and two symlinks supplied the workspace's unhoisted nested deps — packages/core/node_modules and the worktree root — after realpath controls confirmed every loaded module lived in its own tree). Harnesses (01–10 in this directory) drove the compiled SkillManager/SkillTool and the real web-shell client sources with no mocks; raw logs (ab-run-1.log, mutation-*.log, webmut-*.log, web-shell-full-suite.log, base-core-build.log, docs-check.log, tally.log, matrix-assert.log) sit beside them. Counts: 30 (A/B) + 12 (matrix) + 10 (docs) + 9 (gates) = 61.
Flakiness gate log
rounds=5 files=2 skipped=0
file packages/core/src/skills/bundled/goal-draft/SKILL.test.ts: (cd packages/core) npx --no-install vitest run ./src/skills/bundled/goal-draft/SKILL.test.ts
file packages/web-shell/client/constants/localCommands.test.ts: (cd packages/web-shell) npx --no-install vitest run ./client/constants/localCommands.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/core/src/skills/bundled/goal-draft/SKILL.test.ts: PPPPP
packages/web-shell/client/constants/localCommands.test.ts: PPPPP
verdict: pass
summary: 2 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/core/src/skills/bundled/goal-draft/SKILL.test.ts: P (exit 0)
round 1 · packages/web-shell/client/constants/localCommands.test.ts: P (exit 0)
round 2 · packages/core/src/skills/bundled/goal-draft/SKILL.test.ts: P (exit 0)
round 2 · packages/web-shell/client/constants/localCommands.test.ts: P (exit 0)
round 3 · packages/core/src/skills/bundled/goal-draft/SKILL.test.ts: P (exit 0)
round 3 · packages/web-shell/client/constants/localCommands.test.ts: P (exit 0)
round 4 · packages/core/src/skills/bundled/goal-draft/SKILL.test.ts: P (exit 0)
round 4 · packages/web-shell/client/constants/localCommands.test.ts: P (exit 0)
round 5 · packages/core/src/skills/bundled/goal-draft/SKILL.test.ts: P (exit 0)
round 5 · packages/web-shell/client/constants/localCommands.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
|
🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下: Autofix review round — PR #10002No code changes this round — the new feedback carries zero findings, and both disclosed review gaps were closed with evidence instead: the integration suite the automated reviewer could not see was run locally (192 tests passed; the single failure proven to be a runner filesystem condition, not a code defect), and the test-plan paths were shown to belong to a demo project quoted in the PR body, not to this repository. Commit: none (no-action round). No base merge ( Feedback points and dispositions[rv:5027700986] COMMENTED, zero findings — both disclosed gaps closed with evidenceThe round-4 review of Gap 1 — "build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally."
Gap 2 — "Test Plan (not a blocker):
[ic:5422095679] "Sandboxed verification is running" — informational only, no actionA status line posted by the triage workflow's verify stage in response to the maintainer's Verification
中文说明Autofix 评审轮次 — PR #10002本轮无代码改动 —— 新反馈没有任何 finding,两处已披露的审查缺口改为用证据闭环:自动评审看不到的集成测试套件已在本地实际运行(192 个测试通过;唯一失败已证明是运行器文件系统状态问题,而非代码缺陷);测试计划中的路径已查明属于 PR 正文引用的演示项目,而非本仓库。 提交:无(无改动轮次)。未合并 base( 反馈项与处置[rv:5027700986] COMMENTED,零 finding —— 两处已披露缺口均以证据闭环针对 缺口 1 —— "build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。"
缺口 2 —— "Test Plan(非阻断):
[ic:5422095679] "Sandboxed verification is running" —— 纯状态信息,无需处理这是 triage 工作流的验证阶段针对维护者 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 🧠 Handled by Qwen Code · model/模型 |
|
@qwen-code /review |
|
Qwen Code review request accepted. Review is queued in workflow run. |
Local verification — real build, real model, real TUI + Web ShellI built this branch on my own machine and drove it end to end against a live model. All four steps of the Reviewer Test Plan reproduce, including the numbers. Three non-blocking findings are below; none of them should block merge. Environment
What reproduces1 — Tests and static gates. 2 — It actually ships in the bundle. After 3 — Discovery A/B. Typing 4 — Headless run, grounded and read-only. No 5 — The active-Goal branch of Step 1 works. With a Goal already on the session, 6 — Step 0 gate fires. 7 — 8 — Web Shell placeholder. Base vs. this branch, both locales. (Evidence 4 / 5 / 6) Findings — non-blockingA. Headless The slash path has already injected the SKILL body (confirmed on the wire), and the model then calls B. The hand-off line renders with backslashes. The model wraps the C. The model-invocation risk, measured. The PR flags it; here is the shape of it, first tool call only, default approval:
So the narrow description does what it says: it does not hijack ordinary work requests, only the "keep going until X" phrasing it explicitly claims. Whether that phrasing should be intercepted into a drafting detour is a product call — the documented Documentation spot-checks
EvidenceImages are hosted on 1. Base ( 2. This branch: 3. One round of questions, shown as a real dialog (Ask-permissions mode) 4. Web Shell Goals dialog — base 5. Web Shell Goals dialog — this branch (en) 6. Web Shell Goals dialog — this branch (zh-CN) 7. End-to-end: 中文说明本地验证 —— 真实构建、真实模型、真实 TUI 与 Web Shell我在本机把这个分支完整构建并对着真实模型端到端跑通。Reviewer 测试计划的四步全部复现,包括其中的数字。 下面三条是非阻塞发现,都不影响合并。 环境
复现结果1 —— 测试与静态门。 2 —— 确实进了产物。 3 —— 发现路径 A/B。 TUI 里输入 4 —— headless 运行:有 grounding、且只读。 在夹具里跑 没有 5 —— Step 1 的"已有 Goal"分支有效。 会话里已存在 Goal 时, 6 —— Step 0 的闸门会触发。 7 —— 8 —— Web Shell placeholder。 base 与本分支对照,中英双语。(证据 4 / 5 / 6) 发现 —— 均非阻塞A. headless 不加 slash 路径已经注入了 SKILL 正文(抓包可证),模型却又调了一次 B. 交接行渲染出反斜杠。 模型把 C. 模型可调用带来的风险,实测。 PR 自己标了这条风险,这是它的实际形状——只看第一个动作,默认审批模式:
也就是说,收窄后的 description 确实做到了它承诺的事:不会劫持普通的干活请求,只会命中它明写的"keep going until X"句式。这种句式该不该被拦成一次起草绕行,是产品判断;真嫌烦的话,文档里那条 文档抽查
证据图片托管在 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Partially reviewed — gaps disclosed.
Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.
Test Plan (not a blocker): npm test — exit 1; npm run coverage — no package defines this script; 23 tests pass — this review observed 21616, 4235, 1702, 24738, 1659, 601, 630 passed.
中文说明
仅完成部分审查,审查缺口已披露。
未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。
Test Plan(非阻断):npm test — exit 1; npm run coverage — no package defines this script; 23 tests pass — this review observed 21616, 4235, 1702, 24738, 1659, 601, 630 passed。
— qwen3.8-max via Qwen Code /review (v0.22.0)
…wenLM#10002) The `/goal-draft` slash path injects the skill body, but the model could then call the `skill` tool a second time; in headless runs that call needs an approval nobody can give, so the session stopped without drafting anything. The body now states it is already loaded and must not be re-invoked. The hand-off line could also be wrapped in inline code with escaped backticks, so copying it verbatim pasted backslashes into the objective. Step 5 now requires the line as plain text. Both wording changes are pinned by SKILL.test.ts assertions.
|
🤖 Addressed the latest review feedback (round 3/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 3/100 轮)。改动内容与我反驳保留之处如下: Autofix review round — PR #10002Feedback triage and dispositionsOne feedback item this round: Finding A — headless Finding B — the hand-off line renders with backslashes → addressed. The reviewer's Evidence 7 shows the Finding C — model-invocation risk, measured → declined (with evidence). The reviewer's own measurement shows the narrow description behaves exactly as designed and documented: Changes
Both new guards were mutation-probed before committing: removing either SKILL.md sentence makes exactly its witness test fail (1 of 8, the rest green); restoring returns the suite to green. No conflicts ( Verification
中文说明Autofix 审查轮次 — PR #10002反馈分类与处置本轮只有一条反馈: 发现 A —— headless 下不加 发现 B —— 交接行渲染出反斜杠 → 已处理。 审阅者的证据 7 显示 发现 C —— 模型可调用的风险,实测 → 拒绝修改(附证据)。 审阅者自己的测量表明,收窄后的 description 行为与设计和文档完全一致: 变更内容
两个新守卫在提交前都做了变异探针验证:分别删掉 SKILL.md 中的任一句,恰好使其对应的见证测试失败(8 个中失败 1 个,其余为绿);恢复后整套测试恢复全绿。 无冲突( 验证
Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。 Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。 🧠 Handled by Qwen Code · model/模型 |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
No issues found. LGTM! ✅
Not explored to full depth (tool budget reached): "agent 3b": executing SKILL.test.ts (worktree has no node_modules; npm ci + workspace build exceeds this review's budget — interaction verified by reading the production ….
Test Plan (not a blocker): npm test — exit 1; npm run coverage — no package defines this script; 23 tests pass — this review observed 21616, 1702, 24738, 1659, 601, 4235, 630 passed.
中文说明
未发现问题。LGTM!✅
未探索到全部深度(达到工具调用预算):"agent 3b":executing SKILL.test.ts (worktree has no node_modules; npm ci + workspace build exceeds this review's budget — interaction verified by reading the production …。
Test Plan(非阻断):npm test — exit 1; npm run coverage — no package defines this script; 23 tests pass — this review observed 21616, 1702, 24738, 1659, 601, 4235, 630 passed。
— qwen3.8-max via Qwen Code /review (v0.22.0)











What this PR does
Adds a bundled, read-only skill
/goal-draft <intent>that turns a fuzzy intention into a/goalobjective the Goal verifier can actually judge. The skill first decides whether the request is a Goal at all (one-shot tasks and judgement calls are not), reads the active Goal withget_goal, grounds the draft in the workspace (real test/lint/build commands frompackage.json,Makefile, CI config; never invented paths), asks at most one round of 1–3 multiple-choice questions and only when the answer changes the check, scope, or budget, then drafts the objective in a fixed contract —Outcome:/Done when:(numbered, binary, at least one tool-observable check with "paste that line") /Must not:/Budget:/On block:/Context:— runs a nine-item self-check, and prints the objective plus a one-line/goal set …the user can run as-is. It never starts the work and never sets the Goal itself (built-in commands are not model-invocable by design).It also adds a user doc
docs/users/features/goals.md(commands, how a Goal is judged, how to write an objective,/goal-draft), a/goal-draftrow in the built-in skills table and aGoalslink on the/goalrow incommands.md, a pointer fromheadless.md, a design note underdocs/design/, and replaces the web-shell Goals dialog placeholder (all tests pass and the lint is clean) with an example that carries a check, a guardrail, and a budget in both locales.Why it's needed
/goalaccepts any non-empty string, but the objective is judged by an independent verifier that only sees transcript evidence (goalJudge.ts,goal-verifier.ts): printed text cannot prove that tests passed or files changed, a claim about a user action needs a real user message, and an objective nobody can evidence keeps the loop running until a limit stops it. Nothing in the product told users this; the only guidance was the web-shell placeholder, which itself has no check attached. The skill encodes the verifier's rules as a writing procedure, following the shape the closest existing tools converge on (OpenAI Codex's curateddefine-goalskill, the communityagent-goal-skill, Claude Code's/goalguidance of "one measurable end state, a stated check, constraints that matter" plus a turn/time clause) while adding the qwen-code-specific parts: evidence must land in the transcript as tool results, "user confirms" is a blocker not a completion condition, and the objective is handed over on one line becauseparseGoalCommandre-joins whitespace anyway.Reviewer Test Plan
How to verify
cd packages/core && npx vitest run src/skills/bundled/goal-draft/SKILL.test.ts src/skills/bundled-skills.integration.test.ts— 2 files, 23 tests pass, re-measured on headd15ba05(the new SKILL.md parses with the real loader; the skill test pins the allowed tools, step order, contract labels, question rules, single-line hand-off, and the explicit "do not run /goal, do not begin the task" stop).npm run build, then in any small project with atestscript runqwen -p "/goal-draft make the auth tests pass and raise coverage". Expected: a fenced objective with the six labels citing the project's real scripts,[ASSUMPTION]notes for what headless could not ask, a/goal set …line, and no file changes in the project (git statusclean apart from.qwen/)./goal-d—/goal-draftappears next to/goalin completion;/goal-draftwith no argument asks what you want done instead of drafting.Evidence (Before & After)
Before:
/goal-draftis an unknown command; the Goals dialog placeholder readse.g. all tests pass and the lint is clean.After — built CLI, headless, in a throwaway project whose
package.jsonhastest: node --test test/andcoverage: c8 --reporter=text node --test test/:The run below happened in a throwaway project at
/tmp/goal-draft-demo, not in this repository — every path inside it (login.js,login.test.js, thesrc/authandtest/directories) belongs to that fixture. Indented rather than fenced for the same reason: they are not claims about this tree.The project tree was untouched afterwards. A second run,
/goal-draft clean up the auth module, did not accept "clean" as a condition: it rewrote it into an explicit, assumption-tagged observable definition (npm testexits 0, everysrc/authexport is referenced fromtest/,git diff --statconfined tosrc/auth), noted that no lint config exists so lint was dropped, and ended by inviting the user to redefine the target instead of starting work.Tested on
Environment (optional)
Linux, Node 22,
npm ci+npm run buildfrom this branch, headless run against DashScopeqwen3.8-max.Risk & Scope
disable-model-invocation: trueis the one-line fallback if it fires too eagerly. In headless mode the skill cannot ask, so it picks the recommended default and tags it[ASSUMPTION]rather than refusing.propose_goalcore tool with an approval dialog (mirroring Claude Code'sProposeGoal/modelProposedGoals), keeping newlines inparseGoalCommand, a deterministic lint on/goal set, and a "refine" entry in the web-shell Goals dialog. No runtime code changes here.goal-draftshadows the bundled one, andskills.disabledcan hide it, like every bundled skill.Linked Issues
None.
中文说明
这个 PR 做了什么
新增一个内置、只读的 skill
/goal-draft <意图>,把模糊的意图改写成 Goal verifier 真正能判定的/goalobjective。skill 先判断这个请求该不该设 Goal(一次性任务和需要设计判断的事不该设),用get_goal读当前 Goal,在工作区里做 grounding(从package.json、Makefile、CI 配置里找真实的 test/lint/build 命令,绝不编造路径),最多问一轮 1–3 个选择题、且只在答案会改变检查方式/范围/预算时才问,然后按固定契约起草——Outcome:/Done when:(编号、二值、至少一条工具可观测且要求"贴出那一行")/Must not:/Budget:/On block:/Context:——过一遍九条自检,最后打印 objective 和一行可直接回车的/goal set …。它不会开始干活,也不会自己设 Goal(内置命令按设计不可被模型调用)。同时新增用户文档
docs/users/features/goals.md(命令、Goal 如何被判定、如何写 objective、/goal-draft),在commands.md的内置 skill 表加/goal-draft一行并在/goal行链到 Goals 文档,headless.md加一句指引,docs/design/下加设计说明,并把 web-shell Goals 对话框的 placeholder(all tests pass and the lint is clean)换成带检查、护栏和预算的中英文示例。为什么需要
/goal接受任意非空字符串,但 objective 是由一个只看 transcript 证据的独立 verifier 判定的(goalJudge.ts、goal-verifier.ts):打印出来的文字不能证明测试通过或文件变了,关于用户动作的主张需要真实的用户消息,而一个没人能给出证据的 objective 会让循环一直跑到撞上限。产品里没有任何地方告诉用户这一点;唯一的指引是 web-shell 的 placeholder,而它本身就没有检查。这个 skill 把 verifier 的规则编码成写作流程,沿用现有工具收敛出来的形态(OpenAI Codex 官方的define-goalskill、社区的agent-goal-skill、Claude Code/goal文档的"一个可测量终态、一个明确的检查、必须不变的约束"加轮数/时间条款),并补上 qwen-code 特有的部分:证据必须以 tool result 的形式落在 transcript 里,"用户确认"是阻塞而不是完成条件,objective 单行交接是因为parseGoalCommand本来就会把空白重新拼接。Reviewer 测试计划
如何验证
cd packages/core && npx vitest run src/skills/bundled/goal-draft/SKILL.test.ts src/skills/bundled-skills.integration.test.ts——2 个文件、23 个测试通过,已在 headd15ba05上重新实测(新 SKILL.md 能被真实 loader 解析;skill 测试锁定了允许的工具、步骤顺序、契约标签、提问规则、单行交接以及明确的"不要运行 /goal、不要开始任务"停止语句)。npm run build,然后在任意带test脚本的小项目里运行qwen -p "/goal-draft make the auth tests pass and raise coverage"。预期:一个带六个标签、引用项目真实脚本的 objective 代码块,对 headless 无法提问的部分标注[ASSUMPTION],一行/goal set …,并且项目里没有文件改动(除.qwen/外git status干净)。/goal-d——/goal-draft出现在/goal旁边的补全里;不带参数的/goal-draft会先问你想做什么而不是直接起草。证据(Before & After)
Before:
/goal-draft是未知命令;Goals 对话框 placeholder 是e.g. all tests pass and the lint is clean。After:见上方英文部分的 headless 运行输出——在一个
package.json里有test: node --test test/和coverage: c8 --reporter=text node --test test/的一次性项目里,skill 产出了引用真实脚本的六段式 objective、两处[ASSUMPTION]和/goal set …行,项目文件未被改动。第二次运行/goal-draft clean up the auth module没有把"clean"当成条件接受:它把它改写成带假设标注的可观测定义(npm test退出码 0、src/auth的每个导出都被test/引用、git diff --stat只在src/auth内),指出仓库没有 lint 配置因此放弃 lint 检查,并以邀请用户重新定义目标而不是开始干活结束。测试平台
Linux ✅;macOS、Windows⚠️ 未测试。
环境(可选)
Linux,Node 22,本分支
npm ci+npm run build,headless 运行使用 DashScopeqwen3.8-max。风险与范围
disable-model-invocation: true是一行就能改的兜底。headless 模式下 skill 无法提问,会采用推荐默认并标注[ASSUMPTION],而不是拒绝。propose_goal核心工具(对齐 Claude Code 的ProposeGoal/modelProposedGoals)、让parseGoalCommand保留换行、/goal set上的确定性 lint、web-shell Goals 对话框的"改写"入口。本 PR 不改任何运行时代码。goal-draftskill 会覆盖内置的,skills.disabled可以隐藏它。关联 Issue
无。
https://claude.ai/code/session_01FV7i3w7egJ2kMw4AhQC38Z