fix(core): show edit/exec diffs when a PreToolUse hook returns ask - #9441
fix(core): show edit/exec diffs when a PreToolUse hook returns ask#9441yiliang114 wants to merge 16 commits into
Conversation
When a PreToolUse hook returns an 'ask' decision, the tool was bounced back to awaiting_approval with a synthetic plain-text prompt carrying only the hook reason, so Edit/WriteFile confirmations lost their diff view. Reuse the tool's own confirmation view (edit diff / exec command) when it provides one, attaching the hook reason via a new hookAskReason field rendered above the body. Tools without a structured view keep the plain reason prompt. Fixes #9434
|
Thanks for the PR! (Re-run 2026-08-27 01:31 UTC, triggered by @yiliang114's Template looks good ✓ — all required sections present, with real before/after evidence. Problem: observed, not theoretical. #9434 is an open user report (labeled Direction: aligned. Hook authors use PreToolUse Size: core paths are touched ( Approach: the scope reads as justified rather than bloated. The naive fix (reuse the tool's confirmation view on the bounce) is a few dozen lines; the rest of the production diff closes holes the bounce itself introduces — the re-execution skips the PreToolUse hook, so every modify channel (inline edit payload, Risk: two files match the high-risk-path signal from the repo's revert history — Moving on to code review. 🔍 中文说明感谢贡献！ （Re-run 2026-08-27 01:31 UTC，由 @yiliang114 的 模板完整 ✓ —— 各必填小节齐全，并附有真实的 before/after 证据。 问题：已观测到的真实问题，不是理论性问题。#9434 是一个 open 的用户报告（标签 方向：对齐。Hook 作者用 PreToolUse 规模：触及核心路径（ 方案：范围合理而非膨胀。朴素修法（bounce 时复用工具自己的确认视图）只需几十行；其余生产代码都在堵 bounce 自身引入的洞——重新执行会跳过 PreToolUse hook，所以每个修改通道（行内编辑 payload、 风险：两个文件命中仓库 revert 历史的高风险路径信号—— 进入代码审查 🔍 — Qwen Code · qwen3.8-max Reviewed at |
Code review (re-review at head
|
| File | What changed |
|---|---|
packages/core/src/tools/tools.ts |
Adds optional hookAskReason to ToolCallConfirmationDetails |
packages/core/src/core/coreToolScheduler.ts |
Bounce reuses the structured view; args snapshot, payload drop, epoch guard, plan-shell carry-over, abort checkpoints |
packages/core/src/tools/shell.ts |
Keeps a confirmed sed edit across a re-entrant preview; fail-closed on preview failure |
packages/core/src/agents/workflow-run-registry.ts |
Preserves hookAskReason through confirmation restriction for bubbled approvals |
packages/cli/src/ui/components/messages/ToolConfirmationMessage.tsx |
Renders the hook reason as a warning line capped at 5 lines, reserves its height |
packages/cli/src/nonInteractive/permission-suggestions.ts |
Prepends the hook reason on stream-json permission suggestions |
packages/cli/src/acp-integration/session/permissionUtils.ts |
Adds the hook reason text block to ACP permission requests |
packages/core/src/core/coreToolScheduler.test.ts |
Scheduler-level bounce tests: structured/info paths, decline, aborts, races, IDE epoch, args restore, plan-shell |
packages/core/src/tools/shell.test.ts |
Sed retention across a re-entrant preview, including the failing-preview case |
packages/core/src/agents/workflow-run-registry.test.ts |
hookAskReason survives restriction for exec and info |
packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx |
Reason renders with the diff, absent ordinarily, capped on short terminals |
packages/cli/src/nonInteractive/permission-suggestions.test.ts |
Reason precedence over warnings on exec, edit, and info suggestions |
packages/cli/src/acp-integration/session/permissionUtils.test.ts |
Reason on edit, exec, and info permission requests |
Testing — the PR's own CI on the reviewed commit
| Check | Conclusion |
|---|---|
| Qwen Code CI (workflow) | ✅ success |
| Test (ubuntu-latest, Node 22.x) | ✅ success |
| Test (macos-latest, Node 22.x) | ⏭️ skipped (merge-queue/schedule/dispatch lane by design) |
| Test (windows-latest, Node 22.x) | ⏭️ skipped (merge-queue/schedule/dispatch lane by design) |
| Integration Tests (CLI, No Sandbox) | ⏭️ skipped (merge-queue lane by design) |
| Desktop Shell (ubuntu-22.04) | ✅ success |
| Desktop Shell (windows-2022) | ✅ success |
| web-shell E2E Smoke (ubuntu-latest, Node 22.x) | ✅ success |
| SDK Java (workflow, all OS/Java matrix jobs + Real daemon E2E) | ✅ success |
| Security Checks (TruffleHog + Dependency CVE audit) | ✅ success |
All 156 check-runs on this head are completed with zero failures, cancellations, or timeouts (re-fetched this pass), and all three pull_request-event workflows are green. The skips are by design, not PR-caused — I read the workflow conditions in the tree this pass: test_macos/test_windows run only on merge_group/schedule/workflow_dispatch, and integration_cli only on merge_group, so PR builds never execute them. Known coverage gap (also noted by the maintainer): the interactive path has no PR-lane CI — the sandboxed rounds and the maintainer's E2E below fill that.
Behavioural evidence carried by this comment (unattended CI run — the bot does not drive tmux here; each source named and attributed):
- Sandboxed
/verify— eleven posted green rounds on this head (rounds 5–15): isolated token-free containers, A/B against a rebuilt base, mock-free harnesses. Round 15 (posted since the last pass) re-verified against the advancedmaintip: 185/185 scripted assertions, flakiness gate green (6 changed test files × 5 identical rounds), trial merge conflict-free, agent verdict merge-ready. One more round has been in flight since 00:35 UTC — not polled; its report will post itself. - Maintainer @wenshao's two real-environment verification rounds (08-25, 08-26 comments): both arms built from source, a real PreToolUse command hook, the real TUI driven through node-pty → xterm.js, scripted model on loopback. The 08-26 pass added plan mode (round 2 keeps the UNKNOWN warning instead of degrading to a bare reason prompt), the stream-json/SDK host surface (escalated call arrives as a real edit with the reason attached), a live negative control proving the hook-reviewed-args restore is load-bearing, per-file test-efficacy probes (reverting any single touched production file breaks its tests), and a conflict-free merge into current
mainwith 869 tests green on the merged tree. His verdict: merge-ready; he approved this exact head 08-26 07:27 UTC.
The PR's manual E2E ran on Linux (per its Tested-on table), which matches this repo's PR-lane posture since the macOS/Windows unit lanes run in the merge queue by design; the behavioural claim at this head is substantiated beyond it by the rounds above. Nothing remains unproven that a lane would still need to settle — a maintainer who wants one more fresh bot-driven TUI capture before merge can still trigger @qwen-code /tmux, but that is belt-and-braces, not a gap.
中文说明
代码审查（在 head 2b43f0cf 上复审 —— 2026-08-27 01:31 UTC re-run）
（Re-run 2026-08-27 01:31 UTC —— head OID 未变，同一 OID 即同一棵树。本轮对生产 diff 做了全新完整阅读（不复用旧笔记），并在评审 worktree 中对 head 上的文件本体重新推导了每个关键点：plan-shell 策略的导出签名、invocation-context 助手函数、IDE diff 的唯一调用点、语言包文件、CI workflow 条件。上一轮之后的证据台账：/verify 第 15 轮发布为绿（185/185），自 00:35 UTC 起还有一轮在途。）
不看 diff 时我的独立思路：hook 返回 ask 时，重新调用工具自己的 getConfirmationDetails()，把调用带回 awaiting_approval 并附上 hook 原因，准备视图失败时回退到纯原因提示。PR 的核心正是如此——并且正确地堵上了朴素版本打开的洞：bounce 后的重新执行会跳过 PreToolUse hook，hook 与重新执行之间任何可能改变执行内容的通道都必须关闭。
本轮在 diff 中核实的内容：修改通道全部关闭（payload 丢弃、hideModify、编辑器修改被拒、bouncedHookReviewedArgs 快照恢复），均有专门测试钉住；plan-shell 策略按 callId 跨阶段携带、bounce 视图经 decoratePlanModeShellConfirmation 装饰、批准时在同步 claim 保护下跑 validatePlanModeShellApproval（两个分支都有镜像）——本轮把策略模块的导出签名与调用点逐一比对，一致；IDE diff 竞态用 epoch 保护，且确认了 bounce 本身不会开 IDE diff（openIdeDiffIfEnabled 只在 _schedule 确认路径被调用）；高风险路径 shell.ts 的 sed 保留逻辑失败即收敛（三个测试）；abort 检查点在视图准备周围，取消不会被误记为 hook 拒绝；stream-json 与 ACP 等非 TUI 界面也带上 hook 原因。评审线程 API 复核：零未决线程（59 条中 0 条）。@doudouOUC 08-19 的两个 blocker 确认已在本 diff 中修复（sed 预览可重入保留、plan-shell 跨阶段携带+批准时校验）；其评审仍显示 CHANGES_REQUESTED 只是因为修复落地后从未被 dismiss 或复审。
该 head 无 Critical 发现。两项建议级问题仍开放，仍是非阻断后续项（本轮均对 head 上的文件本体静态复核）：
- F1 —— info 回退路径上 hook 原因渲染两次：
⚠ Hook requested confirmation: ...包装对所有确认类型生效，而合成info回退的正文本身就是这句话，于是显示两次。一行修法：包装与高度预留排除type === 'info'。 - F2 —— 新增用户可见字符串在所有语言包中均未翻译：该 key 在
packages/cli/src/i18n/locales/全部 9 个.js文件中 0 命中（本轮重新 grep），而其邻居（Apply this change?）均有翻译。i18next 回退到 key 文本，功能不受影响——但zh用户会看到整屏中文里夹一行英文。
@wenshao 已重申两者都不是正确性或安全缺陷，且其批准仍在该 head 上——保持为后续项，不构成合并前提。
测试 —— 评审 commit 上 PR 自己的 CI
三个 pull_request 事件工作流全绿；该 head 上 156 个检查全部完成、零失败/取消/超时（本轮重新拉取）。skipped 为设计使然，与本 PR 无关——本轮直接读了树上 workflow 的条件：test_macos/test_windows 仅在 merge_group/schedule/workflow_dispatch 运行，integration_cli 仅在 merge_group，PR 构建本来就不会执行它们。已知覆盖缺口（维护者亦指出）：交互路径没有 PR 级 CI——由下方沙箱轮次与维护者实测补位。
本评论携带的行为证据（无人值守 CI 调用——机器人不在此驱动 tmux；逐条注明来源）：
- 沙箱
/verify—— 该 head 上已发布十一轮全绿（第 5–15 轮）：隔离无凭证容器、与重建 base A/B 对照、无 mock harness。第 15 轮（上一轮之后发布）在前进后的main尖端上复验：185/185 脚本断言、抖动门绿色（6 个改动测试文件 × 5 轮一致）、试合并无冲突、agent 判定可合入。自 00:35 UTC 起另有一轮在途——不轮询，报告会自行发布。 - 维护者 @wenshao 的两轮真实环境验证（08-25、08-26 评论）：两侧均从源码构建，真实 PreToolUse command hook，node-pty → xterm.js 驱动真实 TUI，回环地址上的脚本化模型。08-26 一轮新增：plan 模式（第二轮确认保留 UNKNOWN 警告而非退化为纯原因提示）、stream-json/SDK 宿主面（升级调用以真正的 edit 语义带上原因送达）、证明 hook 已审参数还原逻辑确实起作用的真实负控制、逐文件测试有效性探针（单独回退任一改动生产文件都会使测试失败）、与当前
main的无冲突合并及合并树上 869 个测试绿色。结论：可合入；并已于 08-26 07:27 UTC 批准该 head。
PR 的手工 E2E 在 Linux 上执行（见其 Tested-on 表）——这与本仓库 PR 级的姿态一致（macOS/Windows 单元 lane 按设计只在合并队列运行）；该 head 上的行为主张已由上述轮次在其之外充分证实。没有留下任何仍需某个 lane 去关闭的未证实项——若维护者合并前仍想要一份新鲜的机器人驱动 TUI 截图，可再触发 @qwen-code /tmux，但那是双保险，不是缺口。
— Qwen Code · qwen3.8-max
Reviewed at 2b43f0cf541b45e8e767f3857ec3d95254224004 · re-run with @qwen-code /triage
|
Confidence: 3/5 — clean review at this head and every evidence source green; the withhold is the standing Stage 0 policy cap (637 production-logic lines across core paths keeps the bot's final approval in maintainer hands), not doubt about the change. Re-run note (2026-08-27 01:31 UTC, triggered by @yiliang114's Stepping back: the approach matches my independent proposal at its core and then earns its size — every extra mechanism closes a hole the bounce itself opens (a re-execution that skips the hook must not be reachable by any modify channel), each one carries the review-round finding it answers, and the test suite discriminates every touched production file. Both of @doudouOUC's 08-19 Criticals (sed-preview idempotency, plan-shell carry-over) are verifiably fixed in this diff. If I had to maintain this in six months, the density of why-comments is exactly what I'd want at this boundary. What will not change: this run will not — and no future run at this same head will — cast the bot's own approval. The PR crossed the Stage 0 maintainer-awareness threshold at 500+ production-logic lines in core, and under that cap the final approval stays in maintainer hands no matter how clean the stages look; the cap is pure policy here, and saying so is the point of the 3/5. Further The remaining merge mechanics, in order:
⏸️ Deferring to @wenshao — deliberately, and only on policy. The review is clean, the evidence is complete (eleven green sandboxed 中文说明置信度：3/5 —— 该 head 上评审干净、各证据源全绿；不给批准仅因既有的 Stage 0 政策上限（637 行生产逻辑触及核心路径，最终批准保留在维护者手中），而非对改动有疑虑。 Re-run 说明（2026-08-27 01:31 UTC，由 @yiliang114 的 退一步看：方案核心与我的独立思路一致，其规模也是挣来的——每个额外机制都在堵 bounce 自身打开的洞（跳过 hook 的重新执行不能被任何修改通道触达），每个都标注了它回应的 review 轮次发现，测试套件能区分每一个改动的生产文件。@doudouOUC 08-19 的两个 Critical（sed 预览可重入、plan-shell 跨阶段携带）均已在本 diff 中可验证地修复。若六个月后由我维护，这一边界上如此密度的 why 注释正是我想要的。 不会改变的事：本轮不会、同一 head 上今后任何一轮也不会投出机器人自己的批准。该 PR 以 500+ 行核心生产逻辑越过 Stage 0 维护者关注阈值，按该上限，无论各阶段多干净，最终批准都保留在维护者手中；此处的上限纯属政策，3/5 正是为了如实表达。继续 re-run 余下的合并机制，按顺序：
⏸️ 有意 defer 给 @wenshao——且仅因政策。评审干净、证据齐备（该 head 上沙箱 — Qwen Code · qwen3.8-max Reviewed at |
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship — CI landed green after the review. ✅
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. Suggestions are inline.
中文说明
已审查——无阻断问题。 建议见行内评论。
— qwen3.8-max via Qwen Code /review (v0.21.13)
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Partially reviewed — gaps disclosed.
Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.
Deferred under the convergence posture (round 3, not a blocker) — recorded, not requested in this round:
packages/core/src/core/coreToolScheduler.ts:4400 — [review] Bounced edit confirmation keeps modify affordances but post-approval re-execution skips the PreToolUse hookpackages/core/src/core/coreToolScheduler.ts:4439 — [review] Bounce hand-mirrors the confirmation-phase entry sequence; copies already drifted (invocation context)packages/core/src/core/coreToolScheduler.ts:4443 — [review] IDE diff resolution surface has no channel for the hook reasonpackages/cli/src/ui/components/messages/ToolConfirmationMessage.tsx:728 — [review] New i18n key 'Hook requested confirmation: {{reason}}' missing from all 9 locale files
中文说明
仅完成部分审查，审查缺口已披露。
未审查：build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。
收敛姿态下延后（第 3 轮，非阻断）——已记录，本轮不要求修改：共 4 条（原文未翻译，列表见上方英文部分）。
— qwen3.8-max via Qwen Code /review (v0.21.14)
|
@qwen-code /triage |
|
Sandboxed verification: The verification job did not complete (checkout, runner, or setup error) and produced no report. See the workflow run for details. 中文 — 判定：
|
|
@qwen-code /triage |
|
Sandboxed verification: The verification job did not complete (checkout, runner, or setup error) and produced no report. See the workflow run for details. 中文 — 判定：
|
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
doudouOUC
left a comment
doudouOUC
left a comment
There was a problem hiding this comment.
Re-check of remaining blockers
Blocker 1: restrictWorkflowConfirmationDetails drops hookAskReason (R1-1)
- Ruling: fixed by this diff. The code at
workflow-run-registry.ts:1059now copieshookAskReason: details.hookAskReasonin both theeditandexecbranches. The test atworkflow-run-registry.test.ts:435verifies the sentinel value survives through the restriction. ✅
Blocker 2: ShellToolInvocation.getConfirmationDetails not idempotent for sed-edit previews (R2-2)
- Ruling: still stands. The bounce now calls
getConfirmationDetails()a second time for shell tools. ForShellToolInvocation, this re-runsprepareSedEditwhich resetsthis.confirmedSedNewContent = undefined(shell.ts:2084), silently discarding content the user already edited and approved in the IDE diff. Post-ask re-execution then writes the simulated sed result instead of the user's edits. No reply from author on this thread. - Suggested fix: in
ShellToolInvocation.getConfirmationDetails, guard the reset: only clearconfirmedSedNewContentwhen the freshly prepared edit differs from the stored one.
Blocker 3: Plan-mode shell confirmations lose decorations on bounce (R2-3)
- Ruling: still stands. The bounced confirmation is built from the tool's raw details —
decoratePlanModeShellConfirmationis never re-applied andonConfirmis not wrapped withvalidatePlanModeShellApproval. This meansskipIdeDiff,hideModify, and the UNKNOWN_WARNING + Exact Shell Command warnings are absent on the bounced view. No reply from author. - Suggested fix: carry or recompute the call's
planShellDecisionat bounce time, pass the fetched details throughdecoratePlanModeShellConfirmation, and wrap the bounceonConfirmwithvalidatePlanModeShellApproval.
Verified
- All unit tests pass:
coreToolScheduler(372 tests),workflow-run-registry(77 tests),ToolConfirmationMessage(42 tests) - The core fix (reusing structured confirmation views with
hookAskReason) is correct and well-tested - The abort-signal handling improvement (cancelling rather than denying when the signal aborts during view preparation) is correct
Summary
The core fix is sound and the PR design is good. The two remaining blockers are edge cases in the bounce path that affect sed-edit previews and plan-mode shell confirmations — they should be addressed before merge. The existing open suggestions (non-TUI permission surfaces, docstring drift, cancel-before-execution dedup, warnings vs hookAskReason overlap, missing test coverage for edge branches) are non-blocking and can be addressed in follow-up PRs.
doudouOUC
left a comment
doudouOUC
left a comment
There was a problem hiding this comment.
Review: Changes needed — 2 verified Criticals (please address before merge)
I independently verified the current head 4fbc8b3. The feature — reuse the tool's structured confirmation view for a PreToolUse ask bounce — is well-built for the common case, and I confirmed the round-2 fixes (abort re-check, structured default reason, hookAskReason passthrough). But two Critical correctness paths still stand at head. They share one root cause and are reachable.
Reachability (confirmed): the PreToolUse hook fires in _executeToolCallBody (coreToolScheduler.ts:4576) during the execution phase — i.e. after the normal confirmation phase — and an ask bounces at :4630. So a call can be confirmed once (settling invocation state / applying scheduler-side plan-mode decoration) and then bounced, which re-invokes getConfirmationDetails.
Critical 1 — sed-edit confirmed content silently discarded
bounceToAwaitingApprovalForAsk re-calls invocation.getConfirmationDetails(signal) (coreToolScheduler.ts:4373). For ShellToolInvocation sed edits, getConfirmationDetails unconditionally resets this.confirmedSedNewContent = undefined (shell.ts:2084) on every call. Path: IDE mode on + a sed -i whose preview the user edits & approves (sets confirmedSedNewContent, shell.ts:2101) → a PreToolUse ask bounce re-fetches details → the confirmed edit is wiped → the post-ask re-execution finds confirmedSedNewContent === undefined, so executeSedEdit (shell.ts:1895-1902) keeps the simulated edit.newContent and writes that instead — diverging from what the user approved, with no indication. Pre-PR the bounce never called getConfirmationDetails, so the confirmed content survived.
→ Fix: only reset confirmedSedNewContent when the freshly-prepared edit differs from the stored one, or reuse the already-prepared confirmation details on the bounce instead of re-requesting them.
Critical 2 — plan-mode shell policy bypassed on bounce
The normal confirmation phase applies decoratePlanModeShellConfirmation (coreToolScheduler.ts:3119) and wraps onConfirm with validatePlanModeShellApproval (:3267 / :3537) — both scheduler-side, outside the tool's getConfirmationDetails. The bounce builds the confirmation from the raw details and does neither. For a plan-mode wrapped shell edit (e.g. bash -c "sed -i …" — classifies unknown, unwraps to an edit preview, so it is not hard-blocked): after confirm+approve, a PreToolUse ask bounce fetches undecorated details — decoratePlanModeShellConfirmation sets hideModify/skipIdeDiff: true (plan-mode-shell-policy.ts:271-272) which are now absent, so openIdeDiffIfEnabled (bounce, ~:4453) opens the IDE diff the policy forbids, and the missing validatePlanModeShellApproval lets a payload.newContent approval through that the policy converts to Cancel + STALE_APPROVAL (plan-mode-shell-policy.ts:312-322).
→ Fix: carry/recompute the call's planShellDecision at bounce time, run the fetched details through decoratePlanModeShellConfirmation, and wrap the bounce onConfirm with validatePlanModeShellApproval, exactly as the confirmation phase does.
Both require IDE mode + a PreToolUse ask on a shell-sed call (Critical 2 also plan mode) — narrow, but real data-integrity / policy-bypass paths that the ordinary confirmation phase guards and this bounce does not.
Non-blocking Suggestions (fine as follow-ups)
:4405off-TUI surfaces (buildPermissionSuggestions, ACPbuildPermissionRequestContent) never readhookAskReason, so the feature is TUI-only.:4351the docstring says afalsereturn "falls through to the deny path", but the caller's abort branch always cancels — update the doc so the user-cancel-vs-hook-denial attribution isn't "simplified" away.:4642/ToolConfirmationMessage.tsx:725DRY — three near-identical cancel sites; the hook-reason render forks the existingwarningsmechanism.:4380/ToolConfirmationMessage.tsx:72/workflow-run-registry.ts:1059test-coverage gaps — bounce abort re-check & default-reason fallback; the 5-line reason cap +HOOK_ASK_REASON_HEIGHT; exec-branchhookAskReasonpassthrough (currently only the edit branch is pinned).
Happy to re-review once the two Criticals are addressed.
The PreToolUse ask bounce re-fetches getConfirmationDetails, which re-ran prepareSedEdit and unconditionally cleared confirmedSedNewContent, discarding the content the user already edited and approved. Only clear it when the freshly prepared edit differs from the previously prepared one (a genuinely new sed target).
The bounce built its confirmation from the tool RAW details, so hideModify/skipIdeDiff/warnings and validatePlanModeShellApproval were dropped. Decorate the bounced details with the plan-shell decision and wrap onConfirm to validate at approval time, mirroring the ordinary confirmation phase.
|
@qwen-code /triage |
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 212 passed · 0 failed · 212 total Flakiness gate: ✅ 6 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：212 通过 · 0 失败 · 212 总计 抖动门：✅ 6 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #9441 Deep Verification (round 12) —
|
| previous item (round 11) | status at this head |
|---|---|
| edit/exec/write bounce A/B head vs base | re-measured: head 83/83 across 7 cells, base 57/57 across 6 (01-ab-head-cells.png, 02-ab-base-cells.png) |
| exec-branch live cell | re-measured: head exec view with command preview + hookAskReason + hideAlwaysAllow (10/10); base info (9/9) |
| WriteFile-with-diff case named by #9434 | re-measured: head edit-class diff view with +hello from verify r12 and hookAskReason (13/13); base info-only (10/10) |
| Trial merge into current main | re-run against the SAME main a82a11a0a4 (re-fetched anonymously this round; tip unchanged since round 11 — the container's local origin/main ref was stale at 83da7233a8, the fetch corrected it): conflict-free merge 686a9548a1 with the identical 13-file/+3271 diff, and git diff between the trial-merge tree and the CI merge tree over the 13 PR files is 0 bytes; merged core scheduler suite 404/404; the three sharpest live cells 36/36 on the merged build |
| R6-1c base audit divergence (PostToolUse told smuggled args) | re-measured: base args-mutation control cell 10/10 — the smuggled key leaks into the audit as the expected base failure (encoded as a passing control assertion); head 13/13 with the reviewed snapshot in both execution and audit |
| R6-1b ModifyWithEditor rejection while bounced | re-measured (editor-modify 15/15: ModifyWithEditor answered, zero execution/PostToolUse/file change for 3 s, PreToolUse still ×1; ProceedOnce then wrote exactly once) |
| R7-1 epoch guard (M6) | carries — head OID identical to rounds 8–11, so the guard hunks are byte-identical by construction; mutant not re-run (Not covered) |
| R8-2 fire-and-forget invalidation (M8) | carries — same basis as M6 |
| R5-5 hideModify guard (M0) | re-run: kills the same 3 tests; quoted failures include the hideModify object-match assertion (03-mutation-witness-matrix.png) |
| info-fallback hookAskReason (C1 positive control) | re-run: kills the same 2 tests, both in the mutated file, failing on the behavioral assertion expected undefined to be defined (waiting for hookAskReason) |
| sed-retention S1–S5 | suite-green (shell.test.ts 306, re-run this round inside the 811-test core gate) |
| M5/M9 survivors (coverage gaps) | stands — same head OID; round-4/8/9 adjudication carries; completeness reporting, not merge conditions |
| Observation 1: FORCE_COLOR=1 reds two pre-existing URL tests | re-measured: same two tests (should display urls if prompt and url are different, preserves urls when the prompt is rendered as plain text), 2 failed | 41 passed under FORCE_COLOR=1, 43/43 without it |
| Observation 2: base-side PostToolUse audit divergence | re-measured (args-mutation rows in the A/B table) |
Central claim + A/B
Central claim: when a PreToolUse hook returns ask, a tool call with a structured confirmation view is bounced back to awaiting_approval reusing that view — hook reason attached via hookAskReason — instead of the synthetic reason-only info prompt; approval executes exactly once; tools without a preparable view fall back to the reason prompt (keeping it on head).
Harness (harness/cell.mjs driven by harness/run-cells.mjs, per-cell logs in runs/<arm>-<cell>/): mock-free. Each cell boots the real compiled Config + HookSystem + MessageBus + CoreToolScheduler of the tree under test (absolute-path dist/ import of packages/core/dist/index.js; config.initialize() builds the real registry — ≥30 tools asserted per cell — plus the real HookSystem and MessageBus), with real PreToolUse + PostToolUse command hooks (spawned node child processes via bash, JSON on stdin/stdout; every stdin payload appended to JSONL; PreToolUse answers {"decision":"ask","reason":…} for the cell's tool, {} otherwise), project hooks injected as ConfigParameters.projectHooks exactly where the CLI injects them, approval mode YOLO so only the hook gates the call, interactive: true, trustedFolder: true (required for getProjectHooks()), and a per-cell HOME/workspace so no ambient settings leak in. Edit/write cells seed the FileReadCache through a real read_file call scheduled through the same scheduler (both edit and write_file enforce prior read — without it the bounce falls back and execution errors, which is itself a covered cell). Answers go through confirmationDetails.onConfirm(...) — the seam the TUI/stream-json responders use; the args-mutation cell reproduces the stream-json responder channel by rewriting waitingCall.request.args before answering.
Control-arm validity: base worktree at HEAD^1 (3a46420d59) rebuilt with tsc --build (exit 0, logs/build-base-core.txt), wired to the shared root node_modules plus the nested packages/core/node_modules symlink (tsconfig paths mapping + hoisted/nested dep split). readlink -f of the base dist scheduler resolves inside the base tree (…/tmp/base-tree/packages/core/dist/src/core/coreToolScheduler.js); package.json/package-lock.json diff between HEAD^1 and HEAD is 0 lines, so reusing the root node_modules is a faithful dependency control. Marker greps separate the arms: head dist hookAskReason=5 / bouncedHookReviewedArgs+confirmationEpochs=11; base dist 0/0. Hook timeout is 30 000 ms (CommandHookConfig.timeout is milliseconds — round 11's harness note about the 30 ms budget applies equally here and was honored).
| cell | arm | key observables | result |
|---|---|---|---|
| edit-approve | head | edit view, real unified fileDiff (-beta version / +BETA-CONFIRMED), hookAskReason, hideModify:true, hideAlwaysAllow:true; success; file = confirmed; PreToolUse ×1, PostToolUse ×1, same tool_use_id |
13/13 |
| edit-approve | base | info view, reason-only prompt, no hookAskReason field; same execution observables |
11/11 |
| write-approve | head | prior-read write_file bounces to the edit-class diff view (+hello from verify r12) with hookAskReason; approved write lands byte-exactly once |
13/13 |
| write-approve | base | info view, no hookAskReason; executes once |
10/10 |
| edit-decline | head / base | Cancel → call cancelled, file unchanged, PostToolUse ×0 | 12/12 · 10/10 |
| write-fallback | head / base | unread overwrite: view prep fails prior-read enforcement → info fallback on both arms (head keeps hookAskReason, base has no field); approval does not write, call not success |
7/7 · 7/7 |
| exec-approve | head | exec view with the command preview (echo VERIFY-EXEC-R12), hookAskReason, hideAlwaysAllow; success; PreToolUse ×1, PostToolUse ×1; audited command reviewed |
10/10 |
| exec-approve | base | info view, reason-only, no hookAskReason field; executes once |
9/9 |
| args-mutation | head | responder rewrote request.args while waiting → file = confirmed content AND PostToolUse audit received the reviewed snapshot (new_string: BETA-CONFIRMED, no smuggled key) |
13/13 |
| args-mutation | base | file = confirmed but audit received new_string: "SMUGGLED-CONTENT" / SMUGGLED_ARGS_CHANNEL: "responder" — the pre-existing divergence, encoded as the expected base failure (control cell passes) |
10/10 |
| editor-modify | head | ModifyWithEditor rejected while bounced (call still awaiting_approval 3 s later; zero execution/PostToolUse/file change; PreToolUse still ×1); ProceedOnce recovers and writes exactly once |
15/15 |
Head 83/83, base 57/57 — 140/140 cell assertions (runs/assertions-cells-head.json / runs/assertions-cells-base.json; witness re-runs captured as 01-ab-head-cells.png and 02-ab-base-cells.png).
Serialized bounced views (runs/*/details.json) confirm the table's shape per cell: head-edit-approve {type:"edit", title:"Confirm Edit: config.txt", fileDiff:<unified diff>, hookAskReason:"protected path: …", hideModify:true, hideAlwaysAllow:true} vs base-edit-approve {type:"info", title:"Hook requested confirmation to run edit", prompt:<reason>, hideAlwaysAllow:true} (no hookAskReason key); head-write-fallback {type:"info", hookAskReason:<reason>} vs base-write-fallback {type:"info"} (no key) — the fallback keeps the reason on head exactly as the description claims; head-exec-approve {type:"exec", command:"echo VERIFY-EXEC-R12", rootCommand:"echo", hookAskReason:<reason>, hideAlwaysAllow:true} vs base info.
Mutation witnesses
Unmutated control first: the scheduler suite is 404/404 at this head (same as round 11 — the base 3a46420d59 already carried #10082's added scheduler test). Each mutant is an interface-preserving source edit with an anchor-uniqueness check (driver aborts unless the anchor occurs exactly once), the suite, then git checkout restore (harness/mutate.mjs, orchestrated for the capture by harness/matrix-witness.mjs; logs in logs/mutant-*.txt).
| mutant | guard reverted | suite at mutant | killed tests |
|---|---|---|---|
| C1 (positive control) | info-fallback view loses hookAskReason |
2 failed | 402 passed | approves a bounced plan-shell call through the info fallback, cancels a stale plan-shell approval when the bounce lands in the info fallback — both in the SAME file, failing on the behavioral assertion expected undefined to be defined (waiting for hookAskReason) |
| M0 | hideModify dropped from bounced edit views (R5-5) |
3 failed | 401 passed | keeps the IDE diff closed when a PreToolUse ask bounces an edit call (fails on expected { type: 'edit', …(9) } to match object { type: 'edit', hideModify: true }), rejects ModifyWithEditor while a PreToolUse ask bounce is pending, closes the modify surface and drops modify payloads on a bounced edit confirmation |
| CLICAP (cli-side vacuity witness) | HOOK_ASK_REASON_MAX_LINES 5 → 8 |
1 failed | 42 passed | caps a verbose hook reason at five lines and reserves its height on a short terminal, failing on expected '…' not to contain 'hook reason line 6' |
C1 proves the harness can make the suite fail, and lands its reds in the mutated file. M0 and CLICAP kill exactly the test sets rounds 9–11 reported. After every mutant the source sha256 returned byte-identical to the HEAD blob (core scheduler 51be5a2d6c8a…, ToolConfirmationMessage bbd278ed…), git status --porcelain was clean, and the restored suites were green again (core 404/404, cli 43/43). Witness capture: 03-mutation-witness-matrix.png.
Targeted gates (all re-run this round)
- Changed test files — core:
coreToolScheduler.test.ts404,shell.test.ts306,workflow-run-registry.test.ts101 → 811/811, exit 0 (logs/gate-core.txt); cli:ToolConfirmationMessage.test.tsx43 +permissionUtils.test.ts13 +permission-suggestions.test.ts4 → 60/60, exit 0 (logs/gate-cli.txt). Same counts as round 11. - Merged tree (trial merge
686a9548a1of2b43f0cf54intoorigin/maina82a11a0a4, conflict-free, identical 13-file/+3271 diff; the 13 PR files are byte-identical to the CI merge tree — 0-bytegit diff): core scheduler suite 404/404 (logs/gate-merged-suite.txt); the three sharpest live cells (edit/write/exec approve) 36/36 on the merged build (runs/merged-*). - tsc:
tsc --noEmitexit 0 forpackages/coreandpackages/cli(logs/gate-typecheck.txt). Liveness proven: a plantedhookAskReason: details.hookAskReason as unknown as numberinworkflow-run-registry.tsproduced exactlyerror TS2322: Type 'number' is not assignable to type 'string'at the carry site (line 1460), then removed and clean again (logs/gate-typecheck-live.txt). - eslint: all 13 changed files clean, exit 0 (
logs/gate-eslint.txt). Liveness proven: a plantedanyintools.tsproduced exactly one@typescript-eslint/no-explicit-anyerror (exit 1), then removed and clean again (logs/gate-eslint-live.txt). - FORCE_COLOR re-measurement (carried observation): under
FORCE_COLOR=1,ToolConfirmationMessage.test.tsxfails the same two pre-existing URL tests (2 failed | 41 passed); 43/43 without it (logs/gate-force-color.txt).
Corrections
None this round.
Findings
None new, none blocking. Observations, for the record:
- Pre-existing color-forcing sensitivity in two URL tests (carried, re-measured). Under
FORCE_COLOR=1,ToolConfirmationMessage.test.tsxfails exactly the same two tests as rounds 5–11; without it, 43/43. The PR's hook-reason tests pass under FORCE_COLOR (they are among the 41). Not a PR defect. - Base-side PostToolUse audit divergence (carried, pre-existing). Re-measured in the args-mutation control cell: base's audit receives the responder-mutated args (
new_string: SMUGGLED-CONTENT,SMUGGLED_ARGS_CHANNEL: responder) while executing the reviewed content; head restores the hook-reviewed snapshot for both execution and audit. The PR closes this on the bounced path. - Container's local
origin/mainref was stale (informational, resolved). The checkout carriedorigin/mainat83da7233a8(an older commit); a fresh anonymous fetch proved the actual tip is stilla82a11a0a4— exactly round 11's trial-merge target. Main has not moved under the PR since round 11; the landing story is re-verified against the same main, not assumed.
Not covered
- TUI visual rendering. The hook-reason line and views are covered by the 43 green
ToolConfirmationMessagetests (plus the cli cap mutant witness); no rendered terminal frame was captured (verify-capture.mjscovers flat command output only). - Live IDE round-trip. R7-1/R8-2 guards carry rounds 8/9's M6/M8 kills — justified by the identical head OID (same commit ⇒ byte-identical guard hunks); no live IDE client this round.
- stream-json / ACP surfaces end-to-end. Covered by the 17 green cli tests (13 permissionUtils + 4 permission-suggestions) and by the args-mutation cell, which reproduces the responder's
request.argsmutation channel; no live client round-trip. - Repo-wide suite. Only the affected workspaces' changed test files ran (plus the scheduler suite and the three live cells on the merged tree).
- Plan-shell policy cells were exercised through the suite (404 green incl. the Plan shell routing section), not as live cells — the live cells ran in YOLO mode where the policy is not applicable.
- M6/M8/M5/M9 mutants were not re-run: the head OID is identical to rounds 8–11, so those rounds' kill/adjudication carries by construction; completeness reporting, not merge conditions.
- Per-commit behavioral attribution. All 16 PR commits from the snapshot are individually reachable (deepen fetch connected the grafted history — verified 16/16, and
rev-listmerge-base..head counts exactly 16), but per-commit attribution carries from rounds 5–9 on the identical head OID; this round exercised the aggregateHEAD^1..HEADdiff. - Flakiness gate. Owned by the workflow's own lane.
- Merged-tree cli suite. Main is exactly one commit past the CI base (fix(core): report broadcast delivery failures in send_message #10081,
send_messagebroadcast failures — a core-only change touching no file this PR touches), so the merged-tree gate ran the core scheduler suite + the three sharpest live cells; the cli changed test files ran on the head tree, whose cli files are byte-identical to the merged tree's (0-byte diff shown above).
Methodology
Verified on the CI merge-ref checkout (HEAD = merge 837f9825b4, HEAD^1 = base tip 3a46420d59, HEAD^2 = verified PR head 2b43f0cf54 — all three OIDs identical to round 11's recorded values, so the tree under test is byte-identical and every measurement was nonetheless re-executed live); npm ci + npm run build had completed at HEAD. The base control worktree at HEAD^1 and the trial-merge worktree at origin/main+2b43f0cf54 were wired to the shared root node_modules (plus the nested packages/core/node_modules symlink, needed by the tsconfig paths mapping and the hoisted/nested dep split) and rebuilt with tsc --build. Harnesses drove each tree's compiled dist/ by absolute path with real spawned hook subprocesses (30 000 ms timeout — milliseconds) and real tool executions; per-cell workspaces, hook JSONL logs, and serialized confirmation details live in runs/<arm>-<cell>/. The mutation driver (harness/mutate.mjs) applies interface-preserving source edits with anchor-uniqueness checks and restores via git checkout — after every mutant the tree was git status-clean and the source sha256 byte-identical to the HEAD blob. origin/main was re-fetched anonymously read-only (the checkout's local ref had drifted stale; the actual tip is unchanged since round 11), and the shallow history was deepened (300), connecting the grafted PR history (16/16 snapshot commits reachable; merge-base 1fffa510… equals the snapshot's baseRefOid). Captures 01–03 are witness runs of identical harnesses via scripts/verify-capture.mjs. Raw logs in logs/, harnesses in harness/; assertions.json was computed by harness/finalize.mjs as scripted comparisons over these artifacts. Scratch worktrees (tmp/base-tree, tmp/merge-tree) removed after the round.
Flakiness gate log
rounds=5 files=6 skipped=0
file packages/cli/src/acp-integration/session/permissionUtils.test.ts: (cd packages/cli) npx --no-install vitest run ./src/acp-integration/session/permissionUtils.test.ts
file packages/cli/src/nonInteractive/permission-suggestions.test.ts: (cd packages/cli) npx --no-install vitest run ./src/nonInteractive/permission-suggestions.test.ts
file packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: (cd packages/cli) npx --no-install vitest run ./src/ui/components/messages/ToolConfirmationMessage.test.tsx
file packages/core/src/agents/workflow-run-registry.test.ts: (cd packages/core) npx --no-install vitest run ./src/agents/workflow-run-registry.test.ts
file packages/core/src/core/coreToolScheduler.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/coreToolScheduler.test.ts
file packages/core/src/tools/shell.test.ts: (cd packages/core) npx --no-install vitest run ./src/tools/shell.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/cli/src/acp-integration/session/permissionUtils.test.ts: PPPPP
packages/cli/src/nonInteractive/permission-suggestions.test.ts: PPPPP
packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: PPPPP
packages/core/src/agents/workflow-run-registry.test.ts: PPPPP
packages/core/src/core/coreToolScheduler.test.ts: PPPPP
packages/core/src/tools/shell.test.ts: PPPPP
verdict: pass
summary: 6 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 1 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 1 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 1 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 1 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 1 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 2 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 2 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 2 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 2 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 2 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 2 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 3 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 3 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 3 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 3 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 3 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 3 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 4 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 4 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 4 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 4 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 4 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 4 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 5 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 5 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 5 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 5 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 5 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 5 · packages/core/src/tools/shell.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 187 passed · 0 failed · 187 total Flakiness gate: ✅ 6 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：187 通过 · 0 失败 · 187 总计 抖动门：✅ 6 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #9441 Deep Verification (round 13) —
|
| previous item (round 12) | status at this head |
|---|---|
| edit/exec/write bounce A/B head vs base | re-measured: head 67/67 across 7 cells, base 50/50 across 6 (01-ab-head-cells.png, 02-ab-base-cells.png) |
| exec-branch live cell | re-measured: head exec view with command preview + hookAskReason + hideAlwaysAllow (10/10); base info (8/8) |
| WriteFile-with-diff case named by #9434 | re-measured: head edit-class diff view with +hello from verify r13 and hookAskReason (10/10); base info-only (8/8) |
| Trial merge into current main | re-run against the SAME main a82a11a0a4 (anonymous fetch this round confirmed the tip is unchanged since round 12 — and the container's local origin/main ref carried the correct value this time, unlike round 12's stale ref): conflict-free merge fec211cc72 with the identical 13-file/+3271 diff, and the 13 PR files are byte-identical to the CI merge tree (0 differing files); merged scheduler suite 404/404; the three sharpest live cells 32/32 on the merged build |
| R6-1c base audit divergence (PostToolUse told smuggled args) | re-measured: base args-mutation control cell 8/8 — the smuggled key leaks into the audit as the expected base failure (encoded as a passing control assertion); head 10/10 with the reviewed snapshot in both execution and audit |
| R6-1b ModifyWithEditor rejection while bounced | re-measured (editor-modify 11/11: ModifyWithEditor answered, zero execution/PostToolUse/file change after 3 s, PreToolUse still ×1; ProceedOnce then wrote exactly once) |
| R7-1 epoch guard (M6) | carries — head OID identical to rounds 8–12, so the guard hunks are byte-identical by construction; mutant not re-run (Not covered) |
| R8-2 fire-and-forget invalidation (M8) | carries — same basis as M6 |
| R5-5 hideModify guard (M0) | re-run: kills the same 3 tests; quoted failures include the hideModify object-match assertion (03-mutation-witness-matrix.png) |
| info-fallback hookAskReason (C1 positive control) | re-run: kills the same 2 tests, both in the mutated file, failing on the behavioral assertion expected undefined to be defined (waiting for hookAskReason) |
| sed-retention S1–S5 | suite-green (shell.test.ts re-run this round: 306/306) |
| M5/M9 survivors (coverage gaps) | stands — same head OID; round-4/8/9 adjudication carries; completeness reporting, not merge conditions |
| Observation 1: FORCE_COLOR=1 reds two pre-existing URL tests | re-measured: same two tests (should display urls if prompt and url are different, preserves urls when the prompt is rendered as plain text), 2 failed | 41 passed under FORCE_COLOR=1, 43/43 without it |
| Observation 2: base-side PostToolUse audit divergence | re-measured (args-mutation rows in the A/B table) |
Central claim + A/B
Central claim: when a PreToolUse hook returns ask, a tool call with a structured confirmation view is bounced back to awaiting_approval reusing that view — hook reason attached via hookAskReason — instead of the synthetic reason-only info prompt; approval executes exactly once; tools without a preparable view fall back to the reason prompt (keeping it on head).
Harness (harness/cell.mjs driven by harness/run-cells.mjs, per-cell logs in runs/<arm>-<cell>/): mock-free. Each cell boots the real compiled Config + HookSystem + MessageBus + CoreToolScheduler of the tree under test (absolute-path import of packages/core/dist/index.js; config.initialize({skipGeminiInitialization, skipMcpDiscovery}) builds the real registry — 33 tools asserted ≥30 per cell — plus the real HookSystem and its own MessageBus), with real PreToolUse + PostToolUse command hooks (spawned node child processes, JSON on stdin/stdout; every stdin payload appended to JSONL; PreToolUse answers {"decision":"ask","reason":…} for the cell's tool, {} otherwise), project hooks injected as ConfigParameters.projectHooks exactly where the CLI injects them, approval mode YOLO so only the hook gates the call, interactive: true, trustedFolder: true (required for getProjectHooks()), and a per-cell HOME/workspace so no ambient settings leak in. Edit/write cells seed the FileReadCache through a real read_file call scheduled through the same scheduler (both edit and write_file enforce prior read — without it the bounce falls back, which is itself a covered cell). Answers go through confirmationDetails.onConfirm(...) — the seam the TUI/stream-json responders use; the args-mutation cell reproduces the stream-json responder channel by rewriting waitingCall.request.args before answering.
Control-arm validity: base worktree at HEAD^1 (3a46420d59) rebuilt with tsc --build (exit 0, logs/build-base-core.txt), wired to the shared root node_modules plus the nested packages/core/node_modules symlink. package.json/package-lock.json diff between HEAD^1 and HEAD is 0 lines, so reusing the root node_modules is a faithful dependency control; grep of the base dist confirms zero production files import any @qwen-code/* workspace package (only compiled __tests__ files do, and no test file is imported by the harness path), so no symlink can smuggle head code into the base arm. Marker greps separate the arms: head dist hookAskReason=5 / bouncedHookReviewedArgs+confirmationEpochs=11; base dist 0/0. Hook timeout: 30000 (CommandHookConfig.timeout is milliseconds).
| cell | arm | key observables | result |
|---|---|---|---|
| edit-approve | head | edit view, real unified fileDiff (-beta version / +BETA-CONFIRMED), hookAskReason, hideModify:true, hideAlwaysAllow:true; success; file = confirmed; PreToolUse ×1, PostToolUse ×1, same tool_use_id |
12/12 |
| edit-approve | base | info view, reason-only prompt, no hookAskReason field; same execution observables |
12/12 |
| write-approve | head | prior-read write_file bounces to the edit-class diff view (+hello from verify r13) with hookAskReason; approved write lands byte-exactly once |
10/10 |
| write-approve | base | info view, no hookAskReason; executes once |
8/8 |
| edit-decline | head / base | Cancel → call cancelled, file unchanged, PostToolUse ×0 | 8/8 · 8/8 |
| write-fallback | head / base | unread overwrite: view prep fails prior-read enforcement → info fallback on both arms (head keeps hookAskReason, base has no field); approval does not write, call not success |
6/6 · 6/6 |
| exec-approve | head | exec view with the command preview (echo VERIFY-EXEC-R13), hookAskReason, hideAlwaysAllow; success; PreToolUse ×1, PostToolUse ×1; audited command reviewed |
10/10 |
| exec-approve | base | info view, reason-only, no hookAskReason field; executes once |
8/8 |
| args-mutation | head | responder rewrote request.args while waiting → file = confirmed content AND PostToolUse audit received the reviewed snapshot (new_string: BETA-CONFIRMED, no smuggled key) |
10/10 |
| args-mutation | base | file = confirmed but audit received new_string: "SMUGGLED-CONTENT" plus SMUGGLED_ARGS_CHANNEL: "responder" — the pre-existing divergence, encoded as the expected base failure (control cell passes) |
8/8 |
| editor-modify | head | ModifyWithEditor rejected while bounced (call still awaiting_approval 3 s later; zero execution/PostToolUse/file change; PreToolUse still ×1); ProceedOnce recovers and writes exactly once |
11/11 |
Head 67/67, base 50/50, merged-tree 32/32 — 149 cell assertions (runs/assertions-cells-head.json, runs/assertions-cells-base.json, runs-merged/; witness re-runs captured as 01-ab-head-cells.png and 02-ab-base-cells.png, whose assertion JSONs diffed byte-identical to the original runs).
Serialized bounced views (runs/*/result.json) confirm the table's shape per cell: head-edit-approve {type:"edit", title:"Confirm Edit: config.txt", fileDiff:<unified diff>, hookAskReason:"protected path: …", hideModify:true, hideAlwaysAllow:true} vs base-edit-approve {type:"info", title:"Hook requested confirmation to run edit", prompt:<reason>, hideAlwaysAllow:true} (no hookAskReason key); head-write-fallback {type:"info", prompt:<reason>, hookAskReason:<reason>} vs base-write-fallback {type:"info"} (no key) — the fallback keeps the reason on head exactly as the description claims; head-exec-approve {type:"exec", command:"echo VERIFY-EXEC-R13", rootCommand:"echo", hookAskReason:<reason>, hideAlwaysAllow:true} vs base info.
Mutation witnesses
Unmutated control first: the scheduler suite is 404/404 and the cli confirmation suite 43/43 at this head (logs/mutant-control.txt). Each mutant is an interface-preserving source edit with an anchor-uniqueness check (driver aborts unless the anchor occurs exactly once), the suite, then git checkout restore (harness/mutate.mjs; logs in logs/mutant-*.txt and logs/mutant-*-raw.txt).
| mutant | guard reverted | suite at mutant | killed tests |
|---|---|---|---|
| C1 (positive control) | info-fallback view loses hookAskReason |
2 failed | 402 passed | approves a bounced plan-shell call through the info fallback, cancels a stale plan-shell approval when the bounce lands in the info fallback — both in the SAME file, failing on the behavioral assertion expected undefined to be defined (waiting for hookAskReason) |
| M0 | hideModify dropped from bounced edit views (R5-5) |
3 failed | 401 passed | keeps the IDE diff closed when a PreToolUse ask bounces an edit call (fails on expected { type: 'edit', …(9) } to match object { type: 'edit', hideModify: true }), rejects ModifyWithEditor while a PreToolUse ask bounce is pending (to match object { hideModify: true }), closes the modify surface and drops modify payloads on a bounced edit confirmation (expected undefined to be true) |
| CLICAP (cli-side vacuity witness) | HOOK_ASK_REASON_MAX_LINES 5 → 8 |
1 failed | 42 passed | caps a verbose hook reason at five lines and reserves its height on a short terminal, failing on expected '…' not to contain 'hook reason line 6' |
C1 proves the harness can make the suite fail, and lands its reds in the mutated file. M0 and CLICAP kill exactly the test sets rounds 9–12 reported. After every mutant the source sha256 returned byte-identical to the HEAD blob (core scheduler 51be5a2d6c8a…, ToolConfirmationMessage bbd278ed4c99… — the same hashes recorded in round 12), git status --porcelain was clean, and the restored suites were green again (core 404/404, cli 43/43). Witness capture: 03-mutation-witness-matrix.png.
Targeted gates (all re-run this round)
- Changed test files — core:
coreToolScheduler.test.ts404,shell.test.ts306,workflow-run-registry.test.ts101 → 811/811, exit 0 (logs/mutant-control.txt,logs/gate-core-shell.txt,logs/gate-core-workflow.txt); cli:ToolConfirmationMessage.test.tsx43 +permissionUtils.test.ts+permission-suggestions.test.ts17 → 60/60, exit 0 (logs/mutant-control.txt,logs/gate-cli-extra.txt). Same counts as round 12. - Merged tree (trial merge
fec211cc72of2b43f0cf54intoorigin/maina82a11a0a4, conflict-free after deepening the shallow grafts; identical 13-file/+3271 diff; the 13 PR files byte-identical to the CI merge tree — 0 differing files): core scheduler suite 404/404 (logs/gate-merged-suite.txt); the three sharpest live cells (edit/write/exec approve) 32/32 on the merged build (runs-merged/,logs/cells-merged.txt). - tsc:
tsc --noEmitexit 0 forpackages/coreandpackages/cli(logs/gate-typecheck-core.txt,logs/gate-typecheck-cli.txt), plus a fresh exit-0 re-run after all probes (logs/gate-typecheck-final.txt). Liveness proven: a plantedhookAskReason: details.hookAskReason as unknown as numberinworkflow-run-registry.tsproduced exactlyerror TS2322: Type 'number' is not assignable to type 'string'at the carry site (line 1460), then removed and clean again (logs/gate-typecheck-live.txt). - eslint: all 13 changed files clean, exit 0 (
logs/gate-eslint.txt, re-verified post-probes inlogs/gate-eslint-final.txt). Liveness proven: a plantedanyintools.tsproduced exactly one@typescript-eslint/no-explicit-anyerror at 1030:19 (exit 1), then removed and clean again (logs/gate-eslint-live.txt). - FORCE_COLOR re-measurement (carried observation): under
FORCE_COLOR=1,ToolConfirmationMessage.test.tsxfails the same two pre-existing URL tests (2 failed | 41 passed); 43/43 without it (logs/gate-force-color.txt). - Validity checks (
logs/validity-checks.json, 7/7): base dist has no workspace imports in production files; head/base dist markers (5/0hookAskReason); lockfile unchanged across the PR;git merge-baseof main and the PR head equals the snapshot'sbaseRefOid(1fffa510…); merged-tree PR files byte-identical; trial merge conflict-free.
Corrections
None this round.
Findings
None new, none blocking. Observations, for the record:
- Pre-existing color-forcing sensitivity in two URL tests (carried, re-measured). Under
FORCE_COLOR=1,ToolConfirmationMessage.test.tsxfails exactly the same two tests as rounds 5–12; without it, 43/43. The PR's hook-reason tests pass under FORCE_COLOR (they are among the 41). Not a PR defect. Round note: theverify-capture.mjswrapper setsFORCE_COLOR=1on captured commands, so the mutation-matrix driver strips color-forcing env from its suite subprocesses to keep the witness runs in the same regime as the measured gates. - Base-side PostToolUse audit divergence (carried, pre-existing). Re-measured in the args-mutation control cell: base's audit receives the responder-mutated args (
new_string: SMUGGLED-CONTENT,SMUGGLED_ARGS_CHANNEL: responder) while executing the reviewed content; head restores the hook-reviewed snapshot for both execution and audit. The PR closes this on the bounced path.
Not covered
- TUI visual rendering. The hook-reason line and views are covered by the 43 green
ToolConfirmationMessagetests (plus the CLICAP witness); no rendered terminal frame was captured (verify-capture.mjscovers flat command output only). - Live IDE round-trip. R7-1/R8-2 guards carry rounds 8/9's M6/M8 kills — justified by the identical head OID (same commit ⇒ byte-identical guard hunks); no live IDE client this round.
- stream-json / ACP surfaces end-to-end. Covered by the 17 green cli tests (permissionUtils + permission-suggestions) and by the args-mutation cell, which reproduces the responder's
request.argsmutation channel; no live client round-trip. - Repo-wide suite. Only the affected workspaces' changed test files ran (plus the scheduler suite and the three live cells on the merged tree).
- Plan-shell policy cells were exercised through the suite (404 green incl. the Plan shell routing section), not as live cells — the live cells ran in YOLO mode where the policy is not applicable.
- M6/M8/M5/M9 mutants were not re-run: the head OID is identical to rounds 8–12, so those rounds' kill/adjudication carries by construction; completeness reporting, not merge conditions.
- Per-commit behavioral attribution. The checkout is depth 2 (grafted); the deepen (300) connected the histories enough to resolve
merge-baseto the snapshot'sbaseRefOid, but this round verified the aggregateHEAD^1..HEADdiff rather than re-attributing each of the snapshot's 16 commits. - Flakiness gate. Owned by the workflow's own lane.
- Merged-tree cli suite. Main is exactly one commit past the CI base (fix(core): report broadcast delivery failures in send_message #10081, a core-only change touching no file this PR touches), and all 13 PR files — including the 6 cli files — are byte-identical between the CI merge tree and the trial-merge tree, so the cli gates measured on the head tree apply to the merged tree unchanged.
Methodology
Verified on the CI merge-ref checkout (HEAD = merge 837f9825b4, HEAD^1 = base tip 3a46420d59, HEAD^2 = verified PR head 2b43f0cf54 — all three OIDs identical to rounds 11–12's recorded values; the sha256s of coreToolScheduler.ts (51be5a2d…) and ToolConfirmationMessage.tsx (bbd278ed…) match round 12's records byte-for-byte, so the tree under test is byte-identical and every measurement was nonetheless re-executed live); npm ci + npm run build had completed at HEAD. The base control worktree at HEAD^1 and the trial-merge worktree at origin/main+2b43f0cf54 were wired to the shared root node_modules (plus the nested packages/core/node_modules symlink, needed by the tsconfig paths mapping and the hoisted/nested dep split) and rebuilt with tsc --build (base build log exit 0). Harnesses drove each tree's compiled dist/ by absolute path with real spawned hook subprocesses (30 000 ms timeout — milliseconds) and real tool executions; per-cell workspaces, hook JSONL logs, and serialized confirmation details live in runs/<arm>-<cell>/ (merged-tree cells in runs-merged/). Round-13 harness notes, for reproducibility: an early head-arm run showed four assertion failures that were diagnosed as harness artifacts (a smoke-test workdir's hook JSONL was appended to by the first full run, and the payload field is hook_event_name, not eventName); both were fixed and all cells re-ran clean from wiped directories — none touched PR code. The mutation driver (harness/mutate.mjs) applies interface-preserving source edits with anchor-uniqueness checks and restores via git checkout — after every mutant the tree was git status-clean and the source sha256 byte-identical to the HEAD blob. origin/main was re-fetched anonymously read-only and confirmed unchanged since round 12 (a82a11a0a4); the shallow history was deepened 300 to connect the grafts so the trial merge could proceed (fatal: refusing to merge unrelated histories before, conflict-free merge fec211cc72 after). Captures 01–03 are witness runs of identical harnesses via scripts/verify-capture.mjs; the A/B captures' assertion JSONs diffed byte-identical to the original runs. assertions.json was computed by harness/finalize.mjs as scripted comparisons over these artifacts (149 cell assertions folded from the per-arm JSONs + 38 gate/mutation/validity checks). Scratch worktrees (tmp/base-tree, tmp/merge-tree) removed after the round.
Flakiness gate log
rounds=5 files=6 skipped=0
file packages/cli/src/acp-integration/session/permissionUtils.test.ts: (cd packages/cli) npx --no-install vitest run ./src/acp-integration/session/permissionUtils.test.ts
file packages/cli/src/nonInteractive/permission-suggestions.test.ts: (cd packages/cli) npx --no-install vitest run ./src/nonInteractive/permission-suggestions.test.ts
file packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: (cd packages/cli) npx --no-install vitest run ./src/ui/components/messages/ToolConfirmationMessage.test.tsx
file packages/core/src/agents/workflow-run-registry.test.ts: (cd packages/core) npx --no-install vitest run ./src/agents/workflow-run-registry.test.ts
file packages/core/src/core/coreToolScheduler.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/coreToolScheduler.test.ts
file packages/core/src/tools/shell.test.ts: (cd packages/core) npx --no-install vitest run ./src/tools/shell.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/cli/src/acp-integration/session/permissionUtils.test.ts: PPPPP
packages/cli/src/nonInteractive/permission-suggestions.test.ts: PPPPP
packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: PPPPP
packages/core/src/agents/workflow-run-registry.test.ts: PPPPP
packages/core/src/core/coreToolScheduler.test.ts: PPPPP
packages/core/src/tools/shell.test.ts: PPPPP
verdict: pass
summary: 6 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 1 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 1 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 1 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 1 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 1 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 2 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 2 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 2 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 2 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 2 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 2 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 3 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 3 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 3 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 3 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 3 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 3 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 4 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 4 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 4 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 4 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 4 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 4 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 5 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 5 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 5 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 5 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 5 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 5 · packages/core/src/tools/shell.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
@qwen-code /triage |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 206 passed · 0 failed · 206 total Flakiness gate: ✅ 6 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：206 通过 · 0 失败 · 206 总计 抖动门：✅ 6 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #9441 Deep Verification (round 14) —
|
| previous item (round 13) | status at this head |
|---|---|
| edit/exec/write bounce A/B head vs base | re-measured: head 76/76 across 7 cells, base 60/60 across 6 (01-ab-head-cells.png, 02-ab-base-cells.png) |
| exec-branch live cell | re-measured: head exec view with command preview + hookAskReason + hideAlwaysAllow (12/12); base info (11/11) |
| WriteFile-with-diff case named by #9434 | re-measured: head edit-class view Confirm Write: notes.txt with +hello from verify r14 and hookAskReason (14/14); base info-only (12/12) |
| Trial merge into current main | re-run against the SAME main a82a11a0a4 (anonymous git ls-remote this round confirmed the tip is unchanged): conflict-free merge, identical 13-file/+3271/−748 delta, 13 PR files byte-identical to the CI merge tree (0 differing); merged scheduler suite 404/404; the three sharpest live cells 40/40 on the merged build |
| R6-1c base audit divergence (PostToolUse told smuggled args) | re-measured: base args-mutation control cell 10/10 — the smuggled key + smuggled new_string leak into the audit as the expected base failure (encoded as passing control assertions); head 10/10 with the reviewed snapshot in both execution and audit |
| R6-1b ModifyWithEditor rejection while bounced | re-measured (editor-modify 11/11: ModifyWithEditor answered, zero execution/PostToolUse/file change after 3 s, PreToolUse still ×1; ProceedOnce then wrote exactly once — the reviewed content, not the editor content) |
| R7-1 epoch guard (M6) | carries — head OID identical to rounds 8–13, so the guard hunks are byte-identical by construction; mutant not re-run (Not covered) |
| R8-2 fire-and-forget invalidation (M8) | carries — same basis as M6 |
| R5-5 hideModify guard (M0) | re-run: kills the same 3 tests; quoted failures include the hideModify: true object-match assertions (03-mutation-witness-matrix.png) |
| info-fallback hookAskReason (C1 positive control) | re-run: kills the same 2 tests, both in the mutated file, failing on the behavioral assertion expected undefined to be defined (waiting for hookAskReason) |
| sed-retention S1–S5 | suite-green (shell.test.ts re-run this round: 306/306) |
| M5/M9 survivors (coverage gaps) | stands — same head OID; round-4/8/9 adjudication carries; completeness reporting, not merge conditions |
| Observation 1: FORCE_COLOR=1 reds two pre-existing URL tests | re-measured: same two tests (should display urls if prompt and url are different, preserves urls when the prompt is rendered as plain text), 2 failed | 41 passed under FORCE_COLOR=1, 43/43 without it |
| Observation 2: base-side PostToolUse audit divergence | re-measured (args-mutation rows in the A/B table) |
Central claim + A/B
Central claim: when a PreToolUse hook returns ask, a tool call with a structured confirmation view is bounced back to awaiting_approval reusing that view — hook reason attached via hookAskReason — instead of the synthetic reason-only info prompt; approval executes exactly once; decline cancels; tools without a preparable view fall back to the reason prompt (keeping it on head).
Harness (harness/cell.mjs driven by harness/run-cells.mjs, per-cell logs in runs/<arm>-<cell>/): mock-free. Each cell boots the real compiled Config + HookSystem + MessageBus + CoreToolScheduler of the tree under test (absolute-path import of packages/core/dist/index.js; config.initialize({skipGeminiInitialization, skipMcpDiscovery}) builds the real registry — ≥30 tools asserted per cell), with real PreToolUse + PostToolUse command hooks (harness/hook.mjs spawned node child processes, JSON on stdin/stdout; every stdin payload appended to JSONL; PreToolUse answers {"decision":"ask","reason":…} for the cell's tool, {} otherwise), project hooks injected as ConfigParameters.projectHooks with per-tool matchers exactly where the CLI injects them, approval mode YOLO so only the hook gates the call, interactive: true, trustedFolder: true (required for getProjectHooks()), and a per-cell HOME/workspace so no ambient settings leak in. Edit/write cells seed the FileReadCache through a real read_file call scheduled through the same scheduler (both edit and write_file enforce prior read — without it the bounce falls back, which is itself a covered cell). Answers go through confirmationDetails.onConfirm(...) — the seam the TUI/stream-json responders use; the args-mutation cell reproduces the stream-json responder channel by rewriting waitingCall.request.args before answering. Hook timeout: 30000 (milliseconds).
Control-arm validity: base worktree at HEAD^1 (3a46420d59) rebuilt with tsc --build (exit 0, logs/build-base-core.txt), wired to the shared root node_modules plus the nested packages/core/node_modules symlink. package.json/package-lock.json diff between HEAD^1 and HEAD is 0 lines, so reusing the root node_modules is a faithful dependency control. The skill's symlink hazard was checked directly: readlink -f tmp/base-tree/node_modules/@qwen-code/qwen-code-core resolves into the head tree — but grep of the rebuilt base dist shows zero production files import any @qwen-code/* workspace package (every remaining string mention is inside a comment; only compiled __tests__ files import workspace packages, and no test file is imported by the harness path), so no symlink can smuggle head code into the base arm. Marker greps separate the arms: head dist hookAskReason=5; base dist 0.
| cell | arm | key observables | result |
|---|---|---|---|
| edit-approve | head | edit view Confirm Edit: config.txt, real unified fileDiff (beta version → +BETA-CONFIRMED), hookAskReason, hideModify:true, hideAlwaysAllow:true; success; file = confirmed; PreToolUse ×1, PostToolUse ×1, same tool_use_id |
14/14 |
| edit-approve | base | info view Hook requested confirmation to run edit, reason-only prompt, no hookAskReason field; same execution observables |
12/12 |
| write-approve | head | prior-read write_file bounces to the edit-class diff view (Confirm Write: notes.txt, +hello from verify r14) with hookAskReason; approved write lands byte-exactly once |
14/14 |
| write-approve | base | info view, no hookAskReason; executes once |
12/12 |
| edit-decline | head / base | Cancel → call cancelled, file unchanged, PostToolUse ×0 | 8/8 · 8/8 |
| write-fallback | head / base | unread overwrite: view prep fails prior-read enforcement → info fallback on both arms (head keeps hookAskReason, base has no field); approval does not write, call not success |
7/7 · 7/7 |
| exec-approve | head | exec view with the command preview (echo VERIFY-EXEC-R14), hookAskReason, hideAlwaysAllow; success; PreToolUse ×1, PostToolUse ×1; PostToolUse tool_response shows the command output; audited command reviewed |
12/12 |
| exec-approve | base | info view, reason-only, no hookAskReason field; executes once |
11/11 |
| args-mutation | head | responder rewrote request.args while waiting → file = confirmed content AND PostToolUse audit received the reviewed snapshot (new_string: BETA-CONFIRMED, no smuggled key) |
10/10 |
| args-mutation | base | file = confirmed but audit received new_string: SMUGGLED-CONTENT plus SMUGGLED_ARGS_CHANNEL: responder — the pre-existing divergence, encoded as the expected base failure (control cell passes) |
10/10 |
| editor-modify | head | ModifyWithEditor rejected while bounced (call still awaiting_approval 3 s later; zero execution/PostToolUse/file change; PreToolUse still ×1); ProceedOnce recovers and writes exactly once (reviewed content, not editor content) |
11/11 |
Head 76/76, base 60/60, merged-tree 40/40 — 176 cell assertions (runs/assertions-cells-head.json, runs/assertions-cells-base.json, runs-merged/; witness re-runs captured as 01-ab-head-cells.png and 02-ab-base-cells.png, whose aggregate totals were re-checked identical to the original runs: 76/76 and 60/60).
Serialized bounced views (runs/*/details.json) confirm the table's shape per cell: head-edit-approve {type:"edit", title:"Confirm Edit: config.txt", fileDiff:<unified diff>, hookAskReason:"protected path: …", hideModify:true, hideAlwaysAllow:true} vs base-edit-approve {type:"info", title:"Hook requested confirmation to run edit", prompt:<reason>, hideAlwaysAllow:true} (no hookAskReason key); head-write-fallback {type:"info", prompt:<reason>, hookAskReason:<reason>} vs base-write-fallback {type:"info"} (no key) — the fallback keeps the reason on head exactly as the description claims; head-exec-approve {type:"exec", command:"echo VERIFY-EXEC-R14", hookAskReason:<reason>, hideAlwaysAllow:true} vs base info.
Corrections
None this round.
Findings
None new, none blocking. Observations, for the record:
- Pre-existing color-forcing sensitivity in two URL tests (carried, re-measured). Under
FORCE_COLOR=1,ToolConfirmationMessage.test.tsxfails exactly the same two tests as rounds 5–13 (should display urls if prompt and url are different,preserves urls when the prompt is rendered as plain text); without it, 43/43. The PR's hook-reason tests pass under FORCE_COLOR (they are among the 41). Not a PR defect. Round note:verify-capture.mjssetsFORCE_COLOR=1on captured commands; the suite runs cited here ran without it, matching the CI regime. - Base-side PostToolUse audit divergence (carried, pre-existing). Re-measured in the args-mutation control cell: base's audit receives the responder-mutated args (
new_string: SMUGGLED-CONTENT,SMUGGLED_ARGS_CHANNEL: responder) while executing the reviewed content; head restores the hook-reviewed snapshot for both execution and audit. The PR closes this on the bounced path.
Not covered
- TUI visual rendering. The hook-reason line and views are covered by the 43 green
ToolConfirmationMessagetests (plus the CLICAP witness); no rendered terminal frame was captured (verify-capture.mjscovers flat command output only). - Live IDE round-trip. R7-1/R8-2 guards carry rounds 8/9's M6/M8 kills — justified by the identical head OID (same commit ⇒ byte-identical guard hunks); no live IDE client this round.
- stream-json / ACP surfaces end-to-end. Covered by the 17 green cli tests (permissionUtils + permission-suggestions) and by the args-mutation cell, which reproduces the responder's
request.argsmutation channel; no live client round-trip. This round reproduces the wire shape of that channel, not a live stream-json client driving it. - Repo-wide suite. Only the affected workspaces' changed test files ran (plus the scheduler suite and three live cells on the merged tree).
- Plan-shell policy cells were exercised through the suite (404 green incl. the Plan shell routing section), not as live cells — the live cells ran in YOLO mode where the policy is not applicable.
- M6/M8/M5/M9 mutants were not re-run: the head OID is identical to rounds 8–13, so those rounds' kill/adjudication carries by construction; completeness reporting, not merge conditions.
- Per-commit behavioral attribution. The checkout is depth 2 (grafted);
git rev-list --count HEAD^1..HEAD^2returns 1 at the shallow boundary while the snapshot lists 16 commits. Deepening by 500 connected the histories enough to resolvemerge-baseto the snapshot'sbaseRefOid, but this round verified the aggregateHEAD^1..HEADdiff rather than attributing each commit. - Flakiness gate. Owned by the workflow's own lane.
- Merged-tree cli suite. Main is exactly one commit past the CI base (fix(core): report broadcast delivery failures in send_message #10081, touching
TeamManager.ts/send-message.*— zero file overlap with this PR's 13 files, verified bycomm), and all 13 PR files — including the 6 cli files — are byte-identical between the CI merge tree and the trial-merge tree, so the cli gates measured on the head tree apply to the merged tree unchanged.
Methodology
Verified on the CI merge-ref checkout (HEAD = merge 837f9825b4, HEAD^1 = base tip 3a46420d59, HEAD^2 = verified PR head 2b43f0cf54 — HEAD^2 equals the snapshot's headRefOid; the sha256s of coreToolScheduler.ts (51be5a2d6c8a…) and ToolConfirmationMessage.tsx (bbd278ed4c99…) match round 13's recorded prefixes, and git's content addressing makes identical commit OIDs byte-identical trees; npm ci + npm run build had completed at HEAD. Closure checks re-ran live inside harness/finalize.mjs. The base control worktree at HEAD^1 and the trial-merge worktree at FETCH_HEAD (main a82a11a0a4) + 2b43f0cf54 were wired to the shared root node_modules (plus the nested packages/core/node_modules symlink) and rebuilt with tsc --build (base and merged build logs exit 0); control-arm symlink audit as described under A/B. Harnesses drove each tree's compiled dist/ by absolute path with real spawned hook subprocesses (30 000 ms timeout) and real tool executions; per-cell workspaces, hook JSONL logs, and serialized confirmation details live in runs/<arm>-<cell>/ (merged-tree cells in runs-merged/). Round-14 harness notes, for reproducibility: one early head-arm run failed a single assertion diagnosed as a harness oracle-shape artifact (the exec cell read done.response.llmContent, while the shell tool's result travels in the PostToolUse tool_response payload — switched to that payload, which is the stronger wire oracle); no PR code was touched. The mutation driver (harness/mutate.mjs) applies interface-preserving source edits with anchor-uniqueness checks and restores via git checkout — after every mutant the tree was git status-clean and the source sha256 byte-identical to the HEAD blob. origin/main was re-fetched anonymously read-only and confirmed unchanged since round 13 (a82a11a0a4); the shallow history was deepened 500 so merge-base resolved to the snapshot's baseRefOid (1fffa510…) and the trial merge could proceed (conflict-free). Captures 01–03 are witness runs via scripts/verify-capture.mjs; the A/B captures' aggregate totals re-checked identical to the original runs. assertions.json was computed by harness/finalize.mjs as scripted comparisons over these artifacts (176 cell assertions folded from the per-arm/per-cell JSONs + 30 gate/mutation/validity checks). Scratch worktrees (tmp/base-tree, tmp/merge-tree) removed after the round.
Flakiness gate log
rounds=5 files=6 skipped=0
file packages/cli/src/acp-integration/session/permissionUtils.test.ts: (cd packages/cli) npx --no-install vitest run ./src/acp-integration/session/permissionUtils.test.ts
file packages/cli/src/nonInteractive/permission-suggestions.test.ts: (cd packages/cli) npx --no-install vitest run ./src/nonInteractive/permission-suggestions.test.ts
file packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: (cd packages/cli) npx --no-install vitest run ./src/ui/components/messages/ToolConfirmationMessage.test.tsx
file packages/core/src/agents/workflow-run-registry.test.ts: (cd packages/core) npx --no-install vitest run ./src/agents/workflow-run-registry.test.ts
file packages/core/src/core/coreToolScheduler.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/coreToolScheduler.test.ts
file packages/core/src/tools/shell.test.ts: (cd packages/core) npx --no-install vitest run ./src/tools/shell.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/cli/src/acp-integration/session/permissionUtils.test.ts: PPPPP
packages/cli/src/nonInteractive/permission-suggestions.test.ts: PPPPP
packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: PPPPP
packages/core/src/agents/workflow-run-registry.test.ts: PPPPP
packages/core/src/core/coreToolScheduler.test.ts: PPPPP
packages/core/src/tools/shell.test.ts: PPPPP
verdict: pass
summary: 6 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 1 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 1 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 1 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 1 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 1 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 2 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 2 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 2 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 2 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 2 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 2 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 3 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 3 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 3 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 3 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 3 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 3 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 4 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 4 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 4 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 4 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 4 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 4 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 5 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 5 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 5 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 5 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 5 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 5 · packages/core/src/tools/shell.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
@qwen-code /triage |
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 185 passed · 0 failed · 185 total Flakiness gate: ✅ 6 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：185 通过 · 0 失败 · 185 总计 抖动门：✅ 6 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #9441 Deep Verification (round 15) —
|
| previous item (round 14) | status at this head |
|---|---|
| edit/exec/write bounce A/B head vs base | re-measured: head 66/66 across 6 cells, base 57/57 across 6 (01-ab-matrix-head-vs-base.png) |
| exec-branch live cell | re-measured: head exec view with command preview + hookAskReason + hideAlwaysAllow (12/12); base info (9/9) |
| WriteFile-with-diff case named by #9434 | re-measured: head edit-class view with +hello from verify r15 and hookAskReason (12/12); base info-only (10/10) |
| Trial merge into current main | re-run against the NEW main 84c21b824e (advanced one commit since round 14): conflict-free, identical 13-file/+3271/−748 delta, 0 of 13 PR files differ from the CI merge tree, merged scheduler suite 404/404, three sharpest cells 37/37 on the merged build (02-merged-cells-on-current-main.png) |
| R6-1c base audit divergence (PostToolUse told smuggled args) | re-measured: base args-mutation cell 12/12 — probe fired at answer time, execution ran the reviewed content, audit received new_string: SMUGGLED-CONTENT-R15 + SMUGGLED_ARGS_CHANNEL: responder, no restore (encoded as passing control assertions); head 12/12 with reviewed snapshot in execution, audit, AND final request.args |
| R6-1b ModifyWithEditor rejection while bounced | suite re-measured (rejects ModifyWithEditor while a PreToolUse ask bounce is pending among the 404 green, on both head and merged trees); the live-cell form carries from round 14 on the identical closure — not re-run as a live cell this round (Not covered) |
| R7-1 epoch guard (M6) | carries — head OID identical to rounds 8–14, guard hunks byte-identical by construction; mutant not re-run (Not covered) |
| R8-2 fire-and-forget invalidation (M8) | carries — same basis as M6 |
| R5-5 hideModify guard (M0) | re-run: kills the same 3 tests with hideModify object-match assertions (03-mutation-witness-matrix.png) |
| info-fallback hookAskReason (C1) | re-run: kills the same 2 tests, both in the mutated file, failing on expected undefined to be defined while waiting for hookAskReason |
| CLICAP reason-line cap (cli) | re-run: 5→8 kills exactly caps a verbose hook reason at five lines… (not to contain 'hook reason line 6') |
| sed-retention S1–S5 | suite-green (shell.test.ts re-run this round: 306/306) |
| M5/M9 survivors (coverage gaps) | stands — same head OID; round-4/8/9 adjudication carries; completeness reporting, not merge conditions |
| Observation 1: FORCE_COLOR=1 reds two pre-existing URL tests | re-measured: same two tests (should display urls if prompt and url are different, preserves urls when the prompt is rendered as plain text), 2 failed | 41 passed under FORCE_COLOR=1, 43/43 without it |
| Observation 2: base-side PostToolUse audit divergence | re-measured (args-mutation rows in the A/B table) |
Central claim + A/B
Central claim: when a PreToolUse hook returns ask, a tool call with a structured confirmation view is bounced back to awaiting_approval reusing that view — hook reason attached via hookAskReason — instead of the synthetic reason-only info prompt; approval executes exactly once; decline cancels; tools without a preparable view fall back to the reason prompt (keeping it on head).
Harness (harness/cell.mjs driven by harness/run-cells.mjs, expectations in harness/expectations.mjs, per-cell logs in runs/<arm>-<cell>/): mock-free. Each cell boots the real compiled Config + HookSystem + MessageBus + CoreToolScheduler of the tree under test (absolute-path import of packages/core/dist/index.js; config.initialize({skipGeminiInitialization, skipMcpDiscovery}) builds the real registry — ≥30 tools asserted per cell), with real PreToolUse + PostToolUse command hooks (harness/hook.mjs spawned node child processes, JSON on stdin/stdout; every stdin payload appended to a JSONL wire log; PreToolUse answers {"decision":"ask","reason":…} for the cell's tool only) injected as ConfigParameters.projectHooks with per-tool matchers, approval mode YOLO so only the hook gates the call, interactive: true, trustedFolder: true (required for getProjectHooks()), and a per-cell HOME/workspace so no ambient settings leak in. Edit/write cells seed the FileReadCache through a real read_file call scheduled through the same scheduler (without the prior read the bounce falls back — itself a covered cell). Answers go through confirmationDetails.onConfirm(...) — the seam the TUI/stream-json responders use; the args-mutation cell reproduces the responder channel by rewriting waitingCall.request.args before answering and snapshotting the mutation at answer time.
Control-arm validity: base worktree at HEAD^1 (3a46420d59) rebuilt with tsc --build --force (exit 0, logs/build-base-core.log after fixing an initial harness-side wiring mistake — see Methodology), wired to the shared root node_modules (symlink, realpath verified) plus a real copy of the nested packages/core/node_modules (11 third-party dirs incl. ajv@8.20.0; zero symlinks among them). package.json/package-lock.json diff between HEAD^1 and HEAD is 0 lines, so the dependency closure is a faithful control. The skill's workspace-symlink hazard was audited directly: base dist's only non-test files mentioning @qwen-code/ are three comments (gitDiff.js, terminalSafe.js, scopes.js); grep for actual import syntax across non-test base dist returned zero lines, so nothing can smuggle head code into the base arm. Marker greps separate the arms: head dist hookAskReason=5; base dist 0 (both re-checked by harness/finalize.mjs).
| cell | arm | key observables | result |
|---|---|---|---|
| edit-approve | head | edit view Confirm Edit: config.txt, real unified fileDiff (-alpha version → +BETA-CONFIRMED-R15), hookAskReason, hideModify:true, hideAlwaysAllow:true; success; file = confirmed; PreToolUse ×1, PostToolUse ×1, same tool_use_id |
13/13 |
| edit-approve | base | info view, reason-only prompt, no hookAskReason key; same execution observables |
10/10 |
| write-approve | head | prior-read write_file bounces to the edit-class diff view for notes.txt with +hello from verify r15 and hookAskReason; approved write lands byte-exactly once |
12/12 |
| write-approve | base | info view, no hookAskReason key; executes once |
10/10 |
| exec-approve | head | exec view with command preview echo VERIFY-EXEC-R15, hookAskReason, hideAlwaysAllow; success; PostToolUse tool_response carries the command output; audited command is the reviewed one |
12/12 |
| exec-approve | base | info view, reason-only, no hookAskReason key; executes once |
9/9 |
| edit-decline | head / base | Cancel → call cancelled, file unchanged, PostToolUse ×0 | 9/9 · 9/9 |
| write-fallback | head / base | unread overwrite: view prep fails prior-read enforcement → info fallback on both arms (head keeps hookAskReason, base has no key); no write, call not success |
8/8 · 7/7 |
| args-mutation | head | probe fired at answer time (SMUGGLED-CONTENT-R15 + channel key in the at-answer snapshot) → execution ran the REVIEWED content, audit (PostToolUse tool_input) got the REVIEWED snapshot, final request.args restored (R6-1) |
12/12 |
| args-mutation | base | probe fired → execution ran the reviewed content but audit received the smuggled args and request.args was never restored — the pre-existing divergence, encoded as expected-base-failure assertions that pass when the leak is observed |
12/12 |
Head 66/66, base 57/57, merged-tree 37/37 — 160 cell assertions (runs/assertions-cells-head.json, runs/assertions-cells-base.json, runs/assertions-cells-merged.json; witness re-runs captured as 01-ab-matrix-head-vs-base.png and 02-merged-cells-on-current-main.png). Serialized bounced views (runs/*/details.json) confirm the shapes: head-edit-approve {type:"edit", title:"Confirm Edit: config.txt", fileDiff:<unified diff>, hookAskReason:"protected path: …", hideModify:true, hideAlwaysAllow:true} vs base-edit-approve {type:"info", prompt:<reason>} with no hookAskReason key; head-write-fallback {type:"info", prompt:<reason>, hookAskReason:<reason>} vs base-write-fallback {type:"info"} (no key) — the fallback keeps the reason on head exactly as the description claims; head-exec-approve {type:"exec", command:"echo VERIFY-EXEC-R15", hookAskReason:<reason>, hideAlwaysAllow:true} vs base info. The wire oracle (runs/*/hook.jsonl) shows the exact PreToolUse/PostToolUse payloads the hook system put on the wire, including tool_use_id pairing across the pair.
Mutation witnesses (harness/mutate.mjs, runs/mutation-results.json, 03-mutation-witness-matrix.png): unmutated controls green (scheduler suite 404/404, ToolConfirmationMessage 43/43). All three mutants killed by the intended assertions, each restored byte-identical to its HEAD blob with a clean tree afterwards:
| mutant | suite | killed tests (failure message) |
|---|---|---|
C1: info fallback drops hookAskReason |
coreToolScheduler.test.ts | approves a bounced plan-shell call through the info fallback, cancels a stale plan-shell approval when the bounce lands in the info fallback — both expected undefined to be defined at the hookAskReason wait |
M0: bounce drops hideModify |
coreToolScheduler.test.ts | keeps the IDE diff closed when a PreToolUse ask bounces an edit call, rejects ModifyWithEditor while a PreToolUse ask bounce is pending, closes the modify surface and drops modify payloads on a bounced edit confirmation — hideModify object-match assertions |
| CLICAP: reason cap 5→8 | ToolConfirmationMessage.test.tsx | exactly caps a verbose hook reason at five lines and reserves its height on a short terminal — not to contain 'hook reason line 6' |
Corrections
None this round.
Findings
None new, none blocking. Observations, for the record:
- Pre-existing color-forcing sensitivity in two URL tests (carried, re-measured). Under
FORCE_COLOR=1,ToolConfirmationMessage.test.tsxfails exactly the same two tests as rounds 5–14 (should display urls if prompt and url are different,preserves urls when the prompt is rendered as plain text); without it, 43/43. The PR's hook-reason tests pass under FORCE_COLOR (they are among the 41). Not a PR defect. - Base-side PostToolUse audit divergence (carried, pre-existing). Re-measured in the args-mutation control cell: base's audit receives the responder-mutated args (
new_string: SMUGGLED-CONTENT-R15,SMUGGLED_ARGS_CHANNEL: responder) while execution runs the reviewed content; head restores the hook-reviewed snapshot for execution, audit, and finalrequest.args. The PR closes this on the bounced path.
Not covered
- TUI visual rendering. The hook-reason line and views are covered by the 43 green
ToolConfirmationMessagetests plus the CLICAP witness; no rendered terminal frame was captured (verify-capture.mjscovers flat command output only). - Live IDE round-trip. R7-1/R8-2 guards carry rounds 8/9's M6/M8 kills — justified by the identical head OID (same commit ⇒ byte-identical guard hunks); no live IDE client this round.
- stream-json / ACP surfaces end-to-end. Covered by the 17 green cli tests (permissionUtils 13 + permission-suggestions 4) and by the args-mutation cell, which reproduces the responder's
request.argsmutation channel. This round reproduces the wire shape of that channel, not a live stream-json client driving it. - Repo-wide suite. Only the affected workspaces' changed test files ran (plus the scheduler suite and three live cells on the merged tree).
- Plan-shell policy cells were exercised through the suite (404 green incl. the Plan shell routing section on both head and merged trees), not as live cells — the live cells ran in YOLO mode where the policy is not applicable.
- ModifyWithEditor-rejection live cell ran in round 14 on this same closure; this round pins the same behavior through the suite (
rejects ModifyWithEditor while a PreToolUse ask bounce is pending, green on head and merged trees). - M6/M8/M5/M9 mutants were not re-run: the head OID is identical to rounds 8–14, so those rounds' kill/adjudication carries by construction; completeness reporting, not merge conditions.
- Per-commit behavioral attribution. The checkout is depth 2 (grafted);
git rev-list --count HEAD^1..HEAD^2returns 1 at the shallow boundary while the snapshot lists 16 commits.git merge-base(current main, PR head)resolves to the snapshot'sbaseRefOid(1fffa510…), confirming the aggregate diff base; this round verified the aggregateHEAD^1..HEADdiff rather than attributing each commit. - Flakiness gate. Owned by the workflow's own lane.
- Merged-tree cli suite. Main is two commits past the CI base (fix(core): report broadcast delivery failures in send_message #10081 touching
TeamManager.ts/send-message.*, feat(daemon): support local path extension installs #10131 touching daemon extension installs — zero file overlap with this PR's 13 files, verified bygit log HEAD^1..<main-tip> -- <13 files>returning empty), and all 13 PR files — including the 6 cli files — are byte-identical between the CI merge tree and the trial-merge tree (0 differing), so the cli gates measured on the head tree apply to the merged tree unchanged.
Methodology
Verified on the CI merge-ref checkout (HEAD = merge 837f9825b4, HEAD^1 = base tip 3a46420d59, HEAD^2 = verified PR head 2b43f0cf54 — HEAD^2 equals the snapshot's headRefOid; sha256 prefixes of coreToolScheduler.ts (51be5a2d6c8a…) and ToolConfirmationMessage.tsx (bbd278ed4c99…) match rounds 13–14; npm ci + npm run build had completed at HEAD). The head core dist was additionally rebuilt in-place once (tsc --build --force, exit 0) during control-arm diagnosis; the rebuild is from identical source and carries the 5 hookAskReason markers. The base control worktree at HEAD^1 and the trial-merge worktree at the fetched main tip 84c21b824e + 2b43f0cf54 were wired to the shared root node_modules (symlink realpath verified) plus a real copy of the nested packages/core/node_modules and rebuilt with tsc --build --force (exit 0 both; logs logs/build-base-core.log, logs/build-merged-core.log); the control-arm workspace-import audit is described under A/B. Harness notes, for reproducibility: two harness-side issues found and fixed during development, no PR code touched — (1) the hook wire field is hook_event_name, not event_name (visible in every JSONL entry in runs/*/hook.jsonl); (2) on head the R6-1 restore rewrites request.args back to the reviewed snapshot during re-execution, so the args-mutation probe must snapshot the mutation at answer time (a terminal-time reading witnesses the restore, not the mutation — that reading is now its own assertion). One base-build failure was also harness-side: an initial tmp/base-tree/node_modules symlink pointed one level too shallow (broken link), which silently defeated the @lydell/node-pty paths workaround and resolved ajv to the wrong root copy; fixing the link target produced the clean exit-0 build. The anonymous network probe (git ls-remote + git fetch of the main tip, read-only) succeeded this round, enabling the fresh trial merge against 84c21b824e. Harnesses drove each tree's compiled dist/ by absolute path with real spawned hook subprocesses (30 000 ms timeout) and real tool executions; per-cell workspaces, hook JSONL logs, and serialized confirmation details live in runs/<arm>-<cell>/. The mutation driver (harness/mutate.mjs) applies interface-preserving source edits with anchor-uniqueness checks and restores via git checkout — after every mutant the tree was git status-clean and the file git-hash byte-identical to the HEAD blob. Captures 01–03 are witness runs via scripts/verify-capture.mjs (which sets FORCE_COLOR=1 on captured commands; the cited suite runs ran without it, matching the CI regime). assertions.json was computed by harness/finalize.mjs re-checking every artifact: 160 cell assertions folded from the per-arm/per-cell JSONs + 9 mutation checks + 7 gate summaries + 2 typecheck exits + 1 eslint exit + 2 gate-liveness probes + 1 FORCE_COLOR pattern + 3 control-arm audit markers = 185. Scratch worktrees (tmp/base-tree, tmp/merge-tree) removed after the round.
Flakiness gate log
rounds=5 files=6 skipped=0
file packages/cli/src/acp-integration/session/permissionUtils.test.ts: (cd packages/cli) npx --no-install vitest run ./src/acp-integration/session/permissionUtils.test.ts
file packages/cli/src/nonInteractive/permission-suggestions.test.ts: (cd packages/cli) npx --no-install vitest run ./src/nonInteractive/permission-suggestions.test.ts
file packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: (cd packages/cli) npx --no-install vitest run ./src/ui/components/messages/ToolConfirmationMessage.test.tsx
file packages/core/src/agents/workflow-run-registry.test.ts: (cd packages/core) npx --no-install vitest run ./src/agents/workflow-run-registry.test.ts
file packages/core/src/core/coreToolScheduler.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/coreToolScheduler.test.ts
file packages/core/src/tools/shell.test.ts: (cd packages/core) npx --no-install vitest run ./src/tools/shell.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/cli/src/acp-integration/session/permissionUtils.test.ts: PPPPP
packages/cli/src/nonInteractive/permission-suggestions.test.ts: PPPPP
packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: PPPPP
packages/core/src/agents/workflow-run-registry.test.ts: PPPPP
packages/core/src/core/coreToolScheduler.test.ts: PPPPP
packages/core/src/tools/shell.test.ts: PPPPP
verdict: pass
summary: 6 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 1 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 1 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 1 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 1 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 1 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 2 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 2 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 2 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 2 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 2 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 2 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 3 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 3 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 3 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 3 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 3 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 3 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 4 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 4 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 4 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 4 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 4 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 4 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 5 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 5 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 5 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 5 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 5 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 5 · packages/core/src/tools/shell.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
@qwen-code /triage |
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 226 passed · 0 failed · 226 total Flakiness gate: ✅ 6 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：226 通过 · 0 失败 · 226 总计 抖动门：✅ 6 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #9441 Deep Verification (round 16) —
|
| previous item (round 15) | status at this head |
|---|---|
| edit/exec/write bounce A/B head vs base | re-measured: head 78/78 across 6 cells, base 73/73 across 6 (01-ab-matrix-head-vs-base.png) |
| exec-branch live cell | re-measured: head exec view with command preview + hookAskReason + hideAlwaysAllow (13/13); base info (10/10) |
| WriteFile-with-diff case named by #9434 | re-measured: head edit-class view with +hello from verify r16 and hookAskReason (13/13); base info-only (13/13) |
| Trial merge into current main | re-run against the NEW main fc0e827658 (advanced one commit since round 15): conflict-free, 0 of 13 PR files differ from the CI merge tree, merged scheduler suite 404/404, three sharpest cells 44/44 on the merged build (02-merged-cells-on-current-main.png) |
| R6-1c base audit divergence (PostToolUse told smuggled args) | re-measured: base args-mutation cell 15/15 — probe fired at answer time, execution ran the reviewed content, audit received new_string: SMUGGLED-CONTENT-R16 + SMUGGLED_ARGS_CHANNEL: responder, no restore (encoded as passing control assertions); head 15/15 with reviewed snapshot in execution, audit, AND final request.args |
| R6-1b ModifyWithEditor rejection while bounced | suite re-measured (rejects ModifyWithEditor while a PreToolUse ask bounce is pending among the 404 green; M0 kills it as one of its 3 tests on this round's fresh mutant run); the live-cell form carries from round 14/15 on the identical closure — not re-run as a live cell this round (Not covered) |
| R7-1 epoch guard (M6) | carries — head OID identical to rounds 8–15, guard hunks byte-identical by construction; mutant not re-run (Not covered) |
| R8-2 fire-and-forget invalidation (M8) | carries — same basis as M6 |
| R5-5 hideModify guard (M0) | re-run: kills the same 3 tests with hideModify object-match assertions (03-mutation-witness-matrix.png) |
| info-fallback hookAskReason (C1) | re-run: kills the same 2 tests, both in the mutated file, failing on expected undefined to be defined while waiting for hookAskReason |
| CLICAP reason-line cap (cli) | re-run: 5→8 kills exactly caps a verbose hook reason at five lines… (not to contain 'hook reason line 6') |
| sed-retention S1–S5 | suite-green (shell.test.ts re-run this round: 306/306) |
| M5/M9 survivors (coverage gaps) | stands — same head OID; round-4/8/9 adjudication carries; completeness reporting, not merge conditions |
| Observation 1: FORCE_COLOR=1 reds two pre-existing URL tests | re-measured: same two tests (should display urls if prompt and url are different, preserves urls when the prompt is rendered as plain text), 2 failed | 41 passed under FORCE_COLOR=1, 43/43 without it |
| Observation 2: base-side PostToolUse audit divergence | re-measured (args-mutation rows in the A/B table) |
Central claim + A/B
Central claim: when a PreToolUse hook returns ask, a tool call with a structured confirmation view is bounced back to awaiting_approval reusing that view — hook reason attached via hookAskReason — instead of the synthetic reason-only info prompt; approval executes exactly once; decline cancels; tools without a preparable view fall back to the reason prompt (keeping it on head).
Harness (harness/cell.mjs driven by harness/run-cells.mjs, expectations in harness/expectations.mjs, per-cell logs in runs/<arm>-<cell>/): mock-free. Each cell boots the real compiled Config + HookSystem + MessageBus + CoreToolScheduler of the tree under test (absolute-path import of packages/core/dist/index.js; config.initialize({skipGeminiInitialization, skipMcpDiscovery}) builds the real registry — 33 tools asserted per cell), with real PreToolUse + PostToolUse command hooks (harness/hook.mjs spawned node child processes, JSON on stdin/stdout; every stdin payload appended to a JSONL wire log; PreToolUse answers {"decision":"ask","reason":…} for the cell's tool only) injected as ConfigParameters.projectHooks with per-tool matchers, approval mode YOLO so only the hook gates the call, interactive: true, trustedFolder: true (required for getProjectHooks()), and a per-cell HOME/workspace so no ambient settings leak in. Edit/write cells seed the FileReadCache through a real read_file call scheduled through the same scheduler — both edit and write_file enforce a prior read this branch, so edit cells seed too (without the prior read the bounce falls back or the execution errors — the fallback itself is a covered cell). Answers go through confirmationDetails.onConfirm(...) — the seam the TUI/stream-json responders use; the args-mutation cell reproduces the responder channel by rewriting waitingCall.request.args before answering and snapshotting the mutation at answer time.
Control-arm validity: base worktree at HEAD^1 (3a46420d59) rebuilt with tsc --build --force (exit 0), wired to the shared root node_modules (symlink, realpath verified to /__w/qwen-code/qwen-code/node_modules) plus a real copy of the nested packages/core/node_modules (zero symlinks among them). package.json/package-lock.json diff between HEAD^1 and HEAD is 0 lines, so the dependency closure is a faithful control. The skill's workspace-symlink hazard was audited directly: grep for actual @qwen-code/ import syntax across non-test base dist returned zero lines, so nothing can smuggle head code into the base arm. Marker greps separate the arms: head dist hookAskReason=5; base dist 0; merged dist 5 (all re-checked by harness/finalize.mjs).
| cell | arm | key observables | result |
|---|---|---|---|
| edit-approve | head | edit view Confirm Edit: config.txt, real unified fileDiff (-alpha version → +BETA-CONFIRMED-R16), hookAskReason, hideModify:true, hideAlwaysAllow:true; success; file = confirmed; PreToolUse ×1, PostToolUse ×1, same tool_use_id |
16/16 |
| edit-approve | base | info view, reason-only prompt, no hookAskReason key; same execution observables |
14/14 |
| write-approve | head | prior-read write_file bounces to the edit-class diff view for notes.txt with +hello from verify r16 and hookAskReason; approved write lands byte-exactly once |
13/13 |
| write-approve | base | info view, no hookAskReason key; executes once |
13/13 |
| exec-approve | head | exec view Confirm Shell Command with command preview echo VERIFY-EXEC-R16, hookAskReason, hideAlwaysAllow; success; PostToolUse tool_response carries the command output; audited command is the reviewed one |
13/13 |
| exec-approve | base | info view, reason-only, no hookAskReason key; executes once |
10/10 |
| edit-decline | head / base | Cancel → call cancelled, file unchanged, PostToolUse ×0 | 10/10 · 10/10 |
| write-fallback | head / base | unread overwrite: view prep fails prior-read enforcement → info fallback on both arms (head keeps hookAskReason, base has no key); no write, call not success |
11/11 · 11/11 |
| args-mutation | head | probe fired at answer time (SMUGGLED-CONTENT-R16 + channel key in the at-answer snapshot) → execution ran the REVIEWED content, audit (PostToolUse tool_input) got the REVIEWED snapshot, final request.args restored (R6-1) |
15/15 |
| args-mutation | base | probe fired → execution ran the reviewed content but audit received the smuggled args and request.args was never restored — the pre-existing divergence, encoded as expected-base-failure assertions that pass when the leak is observed |
15/15 |
Head 78/78, base 73/73, merged-tree 44/44 — 195 cell assertions (runs/assertions-cells-head.json, runs/assertions-cells-base.json, runs/assertions-cells-merged.json; witness re-runs captured as 01-ab-matrix-head-vs-base.png and 02-merged-cells-on-current-main.png). Serialized bounced views (runs/*/details.json) confirm the shapes: head-edit-approve {type:"edit", title:"Confirm Edit: config.txt", fileDiff:<unified diff>, hookAskReason:"protected path: …", hideModify:true, hideAlwaysAllow:true} vs base-edit-approve {type:"info", title:"Hook requested confirmation to run edit", prompt:<reason>, hideAlwaysAllow:true} with no hookAskReason key; head-write-fallback {type:"info", prompt:<reason>, hookAskReason:<reason>} vs base-write-fallback {type:"info"} (no key) — the fallback keeps the reason on head exactly as the description claims; head-exec-approve {type:"exec", command:"echo VERIFY-EXEC-R16", hookAskReason:<reason>, hideAlwaysAllow:true} vs base info. The wire oracle (runs/*/hook.jsonl) shows the exact PreToolUse/PostToolUse payloads the hook system put on the wire, including tool_use_id pairing across the pair.
Trial merge into current main. Main advanced one commit since round 15 (84c21b824e → fc0e827658, #10055 — a CI-only change to the autofix lane); the three commits gained since the base tip (#10081, #10131, #10055) have zero file overlap with this PR's 13 files (git log 3a46420d59..fc0e827658 -- <13 files> empty), and base tip 3a46420d59 is a verified ancestor of fc0e827658. Trial merge of the CI merge commit into fc0e827658 in a scratch worktree: conflict-free; all 13 PR files byte-identical between the CI merge tree and the trial-merge tree (0 diff lines); merged core rebuilt tsc --build --force exit 0 with the 5 hookAskReason markers; merged scheduler suite 404/404; the three sharpest live cells (edit-approve, write-approve, args-mutation) 44/44 on the merged build.
Mutation witnesses (harness/mutate.mjs + harness/capture-mutations.mjs, runs/mutation-results.json, 03-mutation-witness-matrix.png): unmutated controls green (scheduler suite 404/404, ToolConfirmationMessage 43/43). All three mutants killed by the intended assertions, each restored byte-identical to its HEAD blob (git hash-object equality) with a clean tree afterwards:
| mutant | suite | killed tests (failure message) |
|---|---|---|
C1: info fallback drops hookAskReason |
coreToolScheduler.test.ts | approves a bounced plan-shell call through the info fallback, cancels a stale plan-shell approval when the bounce lands in the info fallback — both expected undefined to be defined at the hookAskReason wait |
M0: bounce drops hideModify |
coreToolScheduler.test.ts | keeps the IDE diff closed when a PreToolUse ask bounces an edit call, rejects ModifyWithEditor while a PreToolUse ask bounce is pending, closes the modify surface and drops modify payloads on a bounced edit confirmation — hideModify object-match assertions |
| CLICAP: reason cap 5→8 | ToolConfirmationMessage.test.tsx | exactly caps a verbose hook reason at five lines and reserves its height on a short terminal — not to contain 'hook reason line 6' |
Corrections
None this round.
Findings
None new, none blocking. Observations, for the record:
- Pre-existing color-forcing sensitivity in two URL tests (carried, re-measured). Under
FORCE_COLOR=1,ToolConfirmationMessage.test.tsxfails exactly the same two tests as rounds 5–15 (should display urls if prompt and url are different,preserves urls when the prompt is rendered as plain text); without it, 43/43. The PR's hook-reason tests pass under FORCE_COLOR (they are among the 41). Not a PR defect. - Base-side PostToolUse audit divergence (carried, pre-existing). Re-measured in the args-mutation control cell: base's audit receives the responder-mutated args (
new_string: SMUGGLED-CONTENT-R16,SMUGGLED_ARGS_CHANNEL: responder) while execution runs the reviewed content; head restores the hook-reviewed snapshot for execution, audit, and finalrequest.args. The PR closes this on the bounced path.
Not covered
- TUI visual rendering. The hook-reason line and views are covered by the 43 green
ToolConfirmationMessagetests plus the CLICAP witness; no rendered terminal frame was captured (verify-capture.mjscovers flat command output only). - Live IDE round-trip. R7-1/R8-2 guards carry rounds 8/9's M6/M8 kills — justified by the identical head OID (same commit ⇒ byte-identical guard hunks); no live IDE client this round.
- stream-json / ACP surfaces end-to-end. Covered by the 17 green cli tests (permissionUtils 13 + permission-suggestions 4) and by the args-mutation cell, which reproduces the responder's
request.argsmutation channel. This round reproduces the wire shape of that channel, not a live stream-json client driving it. - Repo-wide suite. Only the affected workspaces' changed test files ran (plus the scheduler suite and three live cells on the merged tree).
- Plan-shell policy cells were exercised through the suite (404 green incl. the Plan shell routing section, on head and merged trees), not as live cells — the live cells ran in YOLO mode where the policy is not applicable.
- ModifyWithEditor-rejection live cell last ran in round 14 on this same closure; this round pins the same behavior through the suite (green among 404, and killed by M0 as expected).
- M6/M8/M5/M9 mutants were not re-run: the head OID is identical to rounds 8–15, so those rounds' kill/adjudication carries by construction; completeness reporting, not merge conditions.
- Per-commit behavioral attribution. The checkout is depth 2 (grafted);
git rev-list --count HEAD^1..HEAD^2returns 1 at the shallow boundary while the snapshot lists 16 commits. This round verified the aggregateHEAD^1..HEADdiff rather than attributing each commit. - Flakiness gate. Owned by the workflow's own lane.
- Merged-tree cli suite. The three commits main gained since the CI base have zero file overlap with this PR's 13 files (verified by
git log), and all 13 PR files — including the 6 cli files — are byte-identical between the CI merge tree and the trial-merge tree (0 differing), so the cli gates measured on the head tree apply to the merged tree unchanged. - Windows/macOS behavior. Verified on the lane's Linux container only.
Methodology
Verified on the CI merge-ref checkout (HEAD = merge 837f9825b4, HEAD^1 = base tip 3a46420d59, HEAD^2 = verified PR head 2b43f0cf54 — HEAD^2 equals the snapshot's headRefOid; sha256 prefixes of coreToolScheduler.ts (51be5a2d6c8a…) and ToolConfirmationMessage.tsx (bbd278ed4c99…) match rounds 13–15; npm ci + npm run build had completed at HEAD; node v22.23.2). The base control worktree at HEAD^1 and the trial-merge worktree (CI merge commit merged with the fetched main tip fc0e827658 after setting a local git identity, conflict-free) were wired to the shared root node_modules (symlink realpath verified) plus a real copy of the nested packages/core/node_modules, and rebuilt with tsc --build --force (exit 0 both); the control-arm workspace-import audit is described under A/B. Harness notes, for reproducibility: three harness-side issues found and fixed during development this round, no PR code touched — (1) the command-hook timeout field is milliseconds, not seconds: an initial timeout: 25 created a 25 ms limit that raced node's startup and intermittently degraded ask decisions into allow-with-hookError (visible as non-deterministic bounce behavior during probe development); the harness now omits the field and uses the runner's 60 s default. (2) The edit tool enforces a prior read on this branch (edit_requires_prior_read), same as write_file, so edit cells seed the FileReadCache through a real scheduled read_file — without it the bounce falls back to the info view, which initially masked the structured view under test. (3) Relative harness paths leaked into tool args (File path must be absolute); the driver now resolves all paths. The anonymous network probe (git ls-remote + depth-300 git fetch of main, read-only) succeeded this round, enabling the fresh trial merge against fc0e827658. Harnesses drove each tree's compiled dist/ by absolute path with real spawned hook subprocesses and real tool executions; per-cell workspaces, hook JSONL logs, and serialized confirmation details live in runs/<arm>-<cell>/. The mutation driver applies interface-preserving source edits with anchor-uniqueness checks and restores via git checkout — after every mutant the tree was git status-clean and the file git-hash byte-identical to the HEAD blob. Captures 01–03 are witness re-runs via scripts/verify-capture.mjs (which sets FORCE_COLOR=1 on captured commands; the cited suite runs ran without it, matching the CI regime). assertions.json was computed by harness/finalize.mjs, which re-executed every gate/liveness/audit check and folded all artifacts: 195 cell assertions + 5 mutation checks + 6 changed-test-file gates + 2 typecheck exits + 2 typecheck-liveness checks + 1 eslint exit + 2 eslint-liveness checks + 1 FORCE_COLOR pattern + 8 control-arm/merge audits + 4 input-closure checks = 226. Scratch worktrees (tmp/base-tree, tmp/merge-tree) removed after the round.
Flakiness gate log
rounds=5 files=6 skipped=0
file packages/cli/src/acp-integration/session/permissionUtils.test.ts: (cd packages/cli) npx --no-install vitest run ./src/acp-integration/session/permissionUtils.test.ts
file packages/cli/src/nonInteractive/permission-suggestions.test.ts: (cd packages/cli) npx --no-install vitest run ./src/nonInteractive/permission-suggestions.test.ts
file packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: (cd packages/cli) npx --no-install vitest run ./src/ui/components/messages/ToolConfirmationMessage.test.tsx
file packages/core/src/agents/workflow-run-registry.test.ts: (cd packages/core) npx --no-install vitest run ./src/agents/workflow-run-registry.test.ts
file packages/core/src/core/coreToolScheduler.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/coreToolScheduler.test.ts
file packages/core/src/tools/shell.test.ts: (cd packages/core) npx --no-install vitest run ./src/tools/shell.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/cli/src/acp-integration/session/permissionUtils.test.ts: PPPPP
packages/cli/src/nonInteractive/permission-suggestions.test.ts: PPPPP
packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: PPPPP
packages/core/src/agents/workflow-run-registry.test.ts: PPPPP
packages/core/src/core/coreToolScheduler.test.ts: PPPPP
packages/core/src/tools/shell.test.ts: PPPPP
verdict: pass
summary: 6 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 1 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 1 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 1 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 1 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 1 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 2 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 2 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 2 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 2 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 2 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 2 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 3 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 3 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 3 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 3 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 3 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 3 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 4 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 4 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 4 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 4 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 4 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 4 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 5 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 5 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 5 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 5 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 5 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 5 · packages/core/src/tools/shell.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
|
@qwen-code /triage |
|
Triage re-run completed without a new review.
The stage comments above were updated with the latest result. View workflow run. 上方各阶段评论已更新为最新结果。查看工作流运行。 |
|
Sandboxed verification: ✅ passed — merge-ready (agent verdict) - workflow run Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check. Scripted assertions: 250 passed · 0 failed · 250 total Flakiness gate: ✅ 6 changed test file(s) x 5 identical rounds, no divergence 中文 — 判定：✅ 通过 · 可合入（agent 判定）沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码（与 base 构建 A/B 对照、无 mock harness 断言、定向门禁）。仅作为评审证据，不构成评审、批准或 CI 检查。 脚本断言：250 通过 · 0 失败 · 250 总计 抖动门：✅ 6 changed test file(s) x 5 identical rounds, no divergence Verification reportPR #9441 Deep Verification (round 17) —
|
| previous item (round 16) | status at this head |
|---|---|
| edit/exec/write bounce A/B head vs base | re-measured: head 76/76 across 6 cells, base 63/63 across 6 (01-ab-matrix-head-vs-base.png) |
| exec-branch live cell | re-measured: head exec view with command preview + hookAskReason + hideAlwaysAllow (12/12); base info (11/11) |
| WriteFile-with-diff case named by #9434 | re-measured: head edit-class view Confirm Write: notes.txt with +hello from verify r17 and hookAskReason (13/13); base info-only (10/10) |
| Trial merge into current main | re-run against the NEW main 1ca8cc5fdb (advanced two commits since round 16): conflict-free, 0 of 13 PR files differ from the CI merge tree, merged scheduler suite 404/404, all 6 live cells 76/76 on the merged build (02-merged-cells-on-new-main.png) |
| R6-1c base audit divergence (PostToolUse told smuggled args) | re-measured: base args-mutation cell 14/14 — probe fired at answer time, execution ran the reviewed content, audit received the smuggled args, no restore (encoded as passing control assertions); head 17/17 with reviewed snapshot in execution, audit, AND final request.args |
| R6-1b ModifyWithEditor rejection while bounced | suite re-measured (rejects ModifyWithEditor while a PreToolUse ask bounce is pending among the 404 green; M0 kills it as one of its 3 tests this round); the live-cell form carries from round 14/15 on the identical closure — not re-run as a live cell this round (Not covered) |
| R7-1 epoch guard (M6) | carries — head OID identical to rounds 8–16, guard hunks byte-identical by construction; mutant not re-run (Not covered) |
| R8-2 fire-and-forget invalidation (M8) | carries — same basis as M6 |
| R5-5 hideModify guard (M0) | re-run: kills the same 3 tests (03-mutation-witness-matrix.png) |
| info-fallback hookAskReason (C1) | re-run: kills the same 2 tests in the mutated file |
| CLICAP reason-line cap (cli) | re-run: 5→8 kills exactly caps a verbose hook reason at five lines…; the capture witness re-run additionally reds the two FORCE_COLOR-sensitive URL tests (see Findings obs. 1) |
| sed-retention S1–S5 | suite-green (shell.test.ts re-run this round: 306/306) |
| M5/M9 survivors (coverage gaps) | stands — same head OID; round-4/8/9 adjudication carries; completeness reporting, not merge conditions |
| Observation 1: FORCE_COLOR=1 reds two pre-existing URL tests | re-measured: same two tests (should display urls if prompt and url are different, preserves urls when the prompt is rendered as plain text), 2 failed | 41 passed under FORCE_COLOR=1, 43/43 without it |
| Observation 2: base-side PostToolUse audit divergence | re-measured (args-mutation rows in the A/B table) |
Central claim + A/B
Central claim: when a PreToolUse hook returns ask, a tool call with a structured confirmation view is bounced back to awaiting_approval reusing that view — hook reason attached via hookAskReason — instead of the synthetic reason-only info prompt; approval executes exactly once; decline cancels; tools without a preparable view fall back to the reason prompt (keeping it on head).
Harness (harness/cell.mjs driven by harness/run-cells.mjs, per-cell artifacts in runs/<arm>-<cell>/): mock-free. Each cell boots the real compiled Config + HookSystem + MessageBus + CoreToolScheduler of the tree under test (absolute-path import of packages/core/dist/index.js; config.initialize({skipGeminiInitialization, skipMcpDiscovery}) builds the real registry — 33 tools asserted per cell), with real PreToolUse + PostToolUse command hooks (harness/hook.mjs spawned node child processes, JSON on stdin/stdout; every stdin payload appended to a JSONL wire log; PreToolUse answers {"decision":"ask","reason":…} for the cell's tool only) injected as ConfigParameters.projectHooks with a per-tool matcher, approval mode YOLO so only the hook gates the call, interactive: true, trustedFolder: true (required for getProjectHooks()), and a per-cell HOME/workspace so no ambient settings leak in. Edit/write cells seed the FileReadCache through a real read_file call scheduled through the same scheduler — both edit and write_file enforce a prior read this branch, so edit cells seed too. Answers go through confirmationDetails.onConfirm(...) — the seam the TUI/stream-json responders use; the args-mutation cell reproduces the responder channel by rewriting waitingCall.request.args before answering (the update array shares request-object references with the scheduler's internal state, exactly the mutation path an off-TUI responder has). Terminal calls are removed from the scheduler's list after completion, so the driver tracks every call's latest state continuously across updates.
Control-arm validity: base worktree at HEAD^1 (3a46420d59) rebuilt with the package's own build script (exit 0), wired to the shared root node_modules (symlink, realpath verified to /__w/qwen-code/qwen-code/node_modules) plus a real copy of the nested packages/core/node_modules (zero non-.bin symlinks among them). package.json/package-lock.json diff between HEAD^1 and HEAD is 0 lines, so the dependency closure is a faithful control. The workspace-symlink hazard was audited directly: grep for actual @qwen-code/ import syntax across non-test base dist returned zero lines, so nothing can smuggle head code into the base arm. Marker greps separate the arms: head dist hookAskReason=5; base dist 0; merged dist 5 (re-checked by harness/finalize.mjs).
| cell | arm | key observables | result |
|---|---|---|---|
| edit-approve | head | edit view Confirm Edit: config.txt, real unified fileDiff (-alpha version → +BETA-CONFIRMED-R17), hookAskReason, hideModify:true, hideAlwaysAllow:true; success; file = confirmed; PreToolUse ×1, PostToolUse ×1, same tool_use_id |
13/13 |
| edit-approve | base | info view, reason-only prompt, no hookAskReason key; same execution observables |
10/10 |
| write-approve | head | prior-read write_file bounces to the edit-class diff view Confirm Write: notes.txt with +hello from verify r17 and hookAskReason; approved write lands byte-exactly once |
13/13 |
| write-approve | base | info view, no hookAskReason key; executes once |
10/10 |
| exec-approve | head | exec view with command preview echo VERIFY-EXEC-R17, hookAskReason, hideAlwaysAllow; success; PostToolUse tool_response carries the command output; audited command is the reviewed one |
12/12 |
| exec-approve | base | info view, reason-only, no hookAskReason key; executes once |
11/11 |
| edit-decline | head / base | Cancel → call cancelled, file unchanged, PostToolUse ×0 | 11/11 · 8/8 |
| write-fallback | head / base | unread overwrite: view prep fails prior-read enforcement → info fallback on both arms (head keeps hookAskReason, base has no key); approve → execution errors on the prior-read requirement, no write, no PostToolUse |
10/10 · 10/10 |
| args-mutation | head | probe fired at answer time (SMUGGLED-CONTENT-R17 + channel key in the at-answer snapshot) → execution ran the REVIEWED content, audit (PostToolUse tool_input) got the REVIEWED snapshot, final request.args restored (R6-1) |
17/17 |
| args-mutation | base | probe fired → execution ran the reviewed content but audit received the smuggled args and request.args was never restored — the pre-existing divergence, encoded as expected-base-failure assertions that pass when the leak is observed |
14/14 |
Head 76/76, base 63/63, merged-tree 76/76 — 215 cell assertions (runs/assertions-cells-{head,base,merged}.json; witness re-runs captured as 01-ab-matrix-head-vs-base.png and 02-merged-cells-on-new-main.png — both captures are live re-executions of the cells, not prints of saved logs). Per-cell serialized views (runs/*/obs.json) confirm the shapes: head-edit-approve {type:"edit", title:"Confirm Edit: config.txt", fileDiff:<unified diff>, hookAskReason:"protected path: …", hideModify:true, hideAlwaysAllow:true} vs base-edit-approve {type:"info", title:"Hook requested confirmation to run edit", prompt:<reason>, hideAlwaysAllow:true} with no hookAskReason key; head-write-fallback {type:"info", prompt:<reason>, hookAskReason:<reason>} vs base-write-fallback {type:"info"} (no key) — the fallback keeps the reason on head exactly as the description claims; head-exec-approve {type:"exec", command:"echo VERIFY-EXEC-R17", hookAskReason:<reason>, hideAlwaysAllow:true} vs base info. The wire oracle (runs/*/hook.jsonl) shows the exact PreToolUse/PostToolUse payloads the hook system put on the wire, including tool_use_id pairing across the pair.
Trial merge into current main. Main advanced two commits since round 16 (fc0e827658 → 1ca8cc5fdb: #10157 web-shell visuals, #10220 a telemetry-swap test-mock fix); the five commits gained since the base tip have zero file overlap with this PR's 13 files (git log 3a46420d59..1ca8cc5fdb -- <13 files> empty), and base tip 3a46420d59 is a verified ancestor of 1ca8cc5fdb. Trial merge of the CI merge commit into 1ca8cc5fdb in a scratch worktree: conflict-free (merge commit b9d5445516, parents 1ca8cc5fdb + 837f9825b4); all 13 PR files byte-identical between the CI merge tree and the trial-merge tree (0 differ); merged core rebuilt exit 0 with the 5 hookAskReason markers; merged scheduler suite 404/404; all 6 live cells 76/76 on the merged build.
Mutation witnesses (harness/mutate.mjs, summaries in runs/mutation-*-summary.json, witness 03-mutation-witness-matrix.png): unmutated controls green (scheduler suite 404/404, ToolConfirmationMessage 43/43). All three mutants killed by the intended assertions, each restored byte-identical to its HEAD blob (git hash-object equality) with a clean tree afterwards:
| mutant | suite | killed tests |
|---|---|---|
C1: info fallback drops hookAskReason |
coreToolScheduler.test.ts | approves a bounced plan-shell call through the info fallback, cancels a stale plan-shell approval when the bounce lands in the info fallback (402/404) |
M0: bounce drops hideModify |
coreToolScheduler.test.ts | keeps the IDE diff closed when a PreToolUse ask bounces an edit call, rejects ModifyWithEditor while a PreToolUse ask bounce is pending, closes the modify surface and drops modify payloads on a bounced edit confirmation (401/404) |
| CLICAP: reason cap 5→8 | ToolConfirmationMessage.test.tsx | exactly caps a verbose hook reason at five lines and reserves its height on a short terminal (42/43) |
Corrections
None this round.
Findings
None new, none blocking. Observations, for the record:
- Pre-existing color-forcing sensitivity in two URL tests (carried, re-measured, now also witnessed live). Under
FORCE_COLOR=1,ToolConfirmationMessage.test.tsxfails exactly the same two tests as rounds 5–16 (should display urls if prompt and url are different,preserves urls when the prompt is rendered as plain text); without it, 43/43. The PR's hook-reason tests pass under FORCE_COLOR (they are among the 41). A live corroboration landed in this round's capture witness: the CLICAP mutant re-run insideverify-capture.mjs(which sets FORCE_COLOR=1 on captured commands) showed the intended kill plus those same two URL tests red — seeruns/mutation-CLICAP-NOTE.mdand03-mutation-witness-matrix.png. Not a PR defect. - Base-side PostToolUse audit divergence (carried, pre-existing). Re-measured in the args-mutation control cell: base's audit receives the responder-mutated args (
new_string: SMUGGLED-CONTENT-R17,SMUGGLED_ARGS_CHANNEL: responder) while execution runs the reviewed content, andrequest.argsis never restored; head restores the hook-reviewed snapshot for execution, audit, and finalrequest.args. The PR closes this on the bounced path.
Not covered
- TUI visual rendering. The hook-reason line and views are covered by the 43 green
ToolConfirmationMessagetests plus the CLICAP witness; no rendered terminal frame was captured (verify-capture.mjscovers flat command output only). - Live IDE round-trip. R7-1/R8-2 guards carry rounds 8/9's M6/M8 kills — justified by the identical head OID (same commit ⇒ byte-identical guard hunks); no live IDE client this round.
- stream-json / ACP surfaces end-to-end. Covered by the 17 green cli tests (permissionUtils 13 + permission-suggestions 4) and by the args-mutation cell, which reproduces the responder's
request.argsmutation channel. This round reproduces the wire shape of that channel, not a live stream-json client driving it. - Repo-wide suite. Only the affected workspaces' changed test files ran (plus the scheduler suite and six live cells on the merged tree).
- Plan-shell policy cells were exercised through the suite (404 green incl. the Plan shell routing section, on head and merged trees), not as live cells — the live cells ran in YOLO mode where the policy is not applicable.
- ModifyWithEditor-rejection live cell last ran in round 14 on this same closure; this round pins the same behavior through the suite (green among 404, and killed by M0 as expected).
- M6/M8/M5/M9 mutants were not re-run: the head OID is identical to rounds 8–16, so those rounds' kill/adjudication carries by construction; completeness reporting, not merge conditions.
- Per-commit behavioral attribution. The checkout is depth 2 (grafted);
git rev-list --count HEAD^1..HEAD^2returns 1 at the shallow boundary while the snapshot lists 16 commits. This round verified the aggregateHEAD^1..HEADdiff rather than attributing each commit. - Flakiness gate. Owned by the workflow's own lane.
- Merged-tree cli suite. The two commits main gained since round 16 have zero file overlap with this PR's 13 files (verified by
git log), and all 13 PR files — including the 6 cli files — are byte-identical between the CI merge tree and the trial-merge tree (0 differing), so the cli gates measured on the head tree apply to the merged tree unchanged. - Windows/macOS behavior. Verified on the lane's Linux container only.
Methodology
Verified on the CI merge-ref checkout (HEAD = merge 837f9825b4, HEAD^1 = base tip 3a46420d59, HEAD^2 = verified PR head 2b43f0cf54 — HEAD^2 equals the snapshot's headRefOid; sha256 prefixes of coreToolScheduler.ts (51be5a2d6c8a…) and ToolConfirmationMessage.tsx (bbd278ed4c99…) match rounds 13–16; npm ci + npm run build had completed at HEAD; node v22.23.2). The harness was rebuilt from scratch this round (previous-round artifacts are not present in the checkout); its notes, for reproducibility: (1) terminal calls are removed from the scheduler's toolCalls list after completion, so the driver tracks per-call state continuously across onToolCallsUpdate snapshots instead of reading the list at the end; (2) the update snapshots share request-object references with the scheduler, which is what makes the responder-channel mutation realistic; (3) both edit and write_file enforce a prior read this branch, so structured cells seed the FileReadCache through a real scheduled read_file; (4) command-hook timeout is milliseconds — the harness omits it and uses the runner's 60 s default (round-16 harness lesson kept). The base control worktree at HEAD^1 and the trial-merge worktree (CI merge commit merged with the fetched main tip 1ca8cc5fdb after setting a local git identity, conflict-free) were wired to the shared root node_modules (symlink realpath verified) plus a real copy of the nested packages/core/node_modules, and rebuilt with the package's own build script (exit 0 both); the control-arm workspace-import audit is described under A/B. Harness-side bugs found and fixed during development this round, no PR code touched: a final-state lookup that missed terminal calls (fixed by continuous tracking), an arm-branch that graded the merged arm against base expectations (fixed: FIXED = head ∪ merged), ANSI escapes in vitest summaries (strip before matching), a mutation-restore baseline captured after mutation instead of before (the restore is verified independently by git hash-object equality + clean status, both quoted above), and a control-summary filename collision (the per-suite vitest JSONs are the unambiguous artifacts; large vitest JSONs were slimmed to the counters finalize reads — the mutation .txt logs keep the full output). Captures 01–03 are live witness re-executions via scripts/verify-capture.mjs (which sets FORCE_COLOR=1 on captured commands — see Findings obs. 1 for the visible consequence on the CLICAP re-run; the cited suite runs ran without it, matching the CI regime). Scratch worktrees (tmp/base-tree, tmp/merge-tree) were removed after the round; harness/finalize.mjs carries their seven tree-dependent audits from the recorded in-round results (labeled per-row) and is re-runnable, reproducing assertions.json = 250/250. assertions.json breakdown: 215 cell assertions + 5 mutation-matrix + 3 restore-integrity + 6 changed-test-file gates + 3 typecheck/eslint exits + 2 liveness plants + 1 tree-clean-after-liveness + 4 input-closure + 6 control-arm audits + 4 merge audits + 1 FORCE_COLOR pattern = 250.
Flakiness gate log
rounds=5 files=6 skipped=0
file packages/cli/src/acp-integration/session/permissionUtils.test.ts: (cd packages/cli) npx --no-install vitest run ./src/acp-integration/session/permissionUtils.test.ts
file packages/cli/src/nonInteractive/permission-suggestions.test.ts: (cd packages/cli) npx --no-install vitest run ./src/nonInteractive/permission-suggestions.test.ts
file packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: (cd packages/cli) npx --no-install vitest run ./src/ui/components/messages/ToolConfirmationMessage.test.tsx
file packages/core/src/agents/workflow-run-registry.test.ts: (cd packages/core) npx --no-install vitest run ./src/agents/workflow-run-registry.test.ts
file packages/core/src/core/coreToolScheduler.test.ts: (cd packages/core) npx --no-install vitest run ./src/core/coreToolScheduler.test.ts
file packages/core/src/tools/shell.test.ts: (cd packages/core) npx --no-install vitest run ./src/tools/shell.test.ts
per-file results (P=pass F=fail I=infra-exit, one letter per run):
packages/cli/src/acp-integration/session/permissionUtils.test.ts: PPPPP
packages/cli/src/nonInteractive/permission-suggestions.test.ts: PPPPP
packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: PPPPP
packages/core/src/agents/workflow-run-registry.test.ts: PPPPP
packages/core/src/core/coreToolScheduler.test.ts: PPPPP
packages/core/src/tools/shell.test.ts: PPPPP
verdict: pass
summary: 6 changed test file(s) x 5 identical rounds, no divergence
--- per-invocation detail (full copy in the artifact) ---
round 1 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 1 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 1 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 1 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 1 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 1 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 2 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 2 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 2 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 2 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 2 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 2 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 3 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 3 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 3 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 3 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 3 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 3 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 4 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 4 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 4 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 4 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 4 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 4 · packages/core/src/tools/shell.test.ts: P (exit 0)
round 5 · packages/cli/src/acp-integration/session/permissionUtils.test.ts: P (exit 0)
round 5 · packages/cli/src/nonInteractive/permission-suggestions.test.ts: P (exit 0)
round 5 · packages/cli/src/ui/components/messages/ToolConfirmationMessage.test.tsx: P (exit 0)
round 5 · packages/core/src/agents/workflow-run-registry.test.ts: P (exit 0)
round 5 · packages/core/src/core/coreToolScheduler.test.ts: P (exit 0)
round 5 · packages/core/src/tools/shell.test.ts: P (exit 0)
Evidence images
Harness scripts and raw logs are in the workflow run artifacts (7-day retention).
— Qwen Code · sandboxed verification
4 participants
What this PR does
When a PreToolUse hook returns an
askdecision, the affected tool call is bounced from the execution phase back toawaiting_approvalso the user can confirm it interactively. Until now that confirmation was always a synthetic plain-text prompt carrying only the hook's reason, so tools with a structured confirmation view — Edit/WriteFile diffs, shell command previews — lost it: the user was asked to approve an edit without ever seeing the diff (#9434).This change reuses the tool's own confirmation view when it provides a structured one (
editdiff orexeccommand), attaching the hook's reason via a new sharedhookAskReasonfield that the TUI renders as a warning line above the body. Tools without a structured view keep today's plain reason prompt.Why it's needed
Hook authors use PreToolUse
askto route selected paths/tools through human review. In that flow the reviewer must see what they are approving. Today an escalated Edit/WriteFile shows only the hook reason text, while the same call in the ordinary confirmation phase shows the full diff — an inconsistency the reporter compared unfavorably to Claude Code (#9434).Reviewer Test Plan
How to verify
Unit/integration (deterministic):
cd packages/core && npx vitest run src/core/coreToolScheduler.test.ts— includes two new cases in the PreToolUse-ask section: an edit tool bouncing to its diff confirmation withhookAskReasonattached (approve → executes exactly once through the tool's ownonConfirm), and a non-structured tool keeping the synthetic reason prompt. 370/370 pass.cd packages/cli && npx vitest run src/ui/components/messages/ToolConfirmationMessage.test.tsx— new cases assert the hook-reason line renders with the edit diff and is absent for ordinary confirmations. 42/42 pass.npm run typecheck --workspaces --if-presentclean; eslint + prettier clean on touched files.Manual E2E (how it was reproduced): a project
.qwen/settings.jsonwith a PreToolUse command hook onwrite_filereturning{"decision":"ask","reason":"protected directory: human review required (E2E #9434)"}, approval modeyoloso only the hook gates the call, then ask the model to overwrite an existing file (read first so the diff can be computed). Before: reason-only prompt. After: diff + hook reason. Approving runs the write exactly once (no re-ask loop); declining cancels.Evidence (Before & After)
Before (reason only, no diff — what any hook-
askon Edit/WriteFile showed):After (same hook, same call — tool's diff view reused, hook reason attached):
After approval:
✓ WriteFile Writing to config.txt, file content updated, no second prompt. When the tool's confirmation view cannot be prepared (e.g. WriteFile prior-read enforcement rejects the diff), the bounce falls back to the plain reason prompt instead of failing — same rendering as Before.Tested on
Environment (optional)
node packages/cli/dist/index.js(local build) in tmux, OpenAI-compatible endpoint (qwen3-coder-plus), project-level PreToolUse command hook,tools.approvalMode: yolo.Risk & Scope
getConfirmationDetails()(edit/write-file read the target file to compute the diff), adding one file read to hook-escalated calls; errors/aborts there fall back to the existing reason prompt (abort falls through to the deny path, mirroring the pre-bounce abort check).plan,mcp,ask_user_question, plaininfo) intentionally keep the reason prompt; IDE diff resolution for bounced calls mirrors the confirmation phase but was only exercised with IDE mode off.hookAskReasonis an additive optional field.Linked Issues
Fixes #9434
中文说明
本 PR 做了什么
当 PreToolUse hook 返回
ask决定时，受影响的工具调用会从执行阶段被弹回awaiting_approval，让用户交互式确认。此前该确认永远是一个只带 hook 原因的纯文本提示框，导致拥有结构化确认视图的工具（Edit/WriteFile 的 diff、shell 命令预览）丢失视图：用户被要求批准一次编辑却看不到 diff（#9434）。本改动在工具提供结构化确认视图（
editdiff 或exec命令）时复用该视图，并通过新增的共享字段hookAskReason附带 hook 原因，TUI 在正文上方以警告行渲染。没有结构化视图的工具保持现有的纯原因提示。为什么需要
Hook 作者用 PreToolUse
ask把特定路径/工具路由到人工审核。该流程里审核者必须看到自己要批准的内容。目前被升级的 Edit/WriteFile 只显示 hook 原因文本，而同样的调用在常规确认阶段会显示完整 diff——报告者认为这与 Claude Code 的行为不一致（#9434）。评审验证计划
验证方式：
packages/core下npx vitest run src/core/coreToolScheduler.test.ts，含 PreToolUse-ask 区块两个新用例：edit 工具弹回到其 diff 确认并附带hookAskReason（批准后经由工具自身onConfirm恰好执行一次）；非结构化工具保持合成原因提示。370/370 通过。packages/cli下npx vitest run src/ui/components/messages/ToolConfirmationMessage.test.tsx，新用例断言 hook 原因行与 edit diff 同屏渲染、常规确认不出现该行。42/42 通过。npm run typecheck --workspaces --if-present通过；eslint + prettier 对改动文件通过。.qwen/settings.json配置针对write_file的 PreToolUse command hook，返回{"decision":"ask","reason":"..."}，审批模式设为yolo（仅由 hook 把关），让模型覆写一个已存在文件（先读后写以便计算 diff）。修复前只有原因提示；修复后 diff + hook 原因同屏。批准恰好执行一次（无重复弹窗循环）；拒绝则取消。前后证据：修复前（仅原因、无 diff）；修复后（复用工具 diff 视图并附带 hook 原因，见英文部分代码块）。批准后写入成功且无第二次弹窗。当工具确认视图无法准备时（如 WriteFile 的 prior-read 强制拒绝 diff），弹回退到纯原因提示而非报错，与修复前渲染一致。
测试系统：Linux ✅；macOS / Windows 未测。
风险与范围
getConfirmationDetails()（edit/write-file 需读目标文件计算 diff），为 hook 升级的调用增加一次文件读取；该处出错/中断会退回到现有原因提示（中断则落入拒绝路径，与弹回前的中断检查一致）。plan、mcp、ask_user_question、纯info）有意保持原因提示；弹回调用的 IDE diff 解析与确认阶段保持一致，但仅在关闭 IDE 模式下验证。hookAskReason为纯新增可选字段。关联 Issue
Fixes #9434