Skip to content

fix(core): a never-delivered MCP call is a first delivery, not a replay - #8482

Closed
wenshao wants to merge 12 commits into
QwenLM:mainfrom
wenshao:fix/mcp-replay-never-delivered
Closed

fix(core): a never-delivered MCP call is a first delivery, not a replay#8482
wenshao wants to merge 12 commits into
QwenLM:mainfrom
wenshao:fix/mcp-replay-never-delivered

Conversation

@wenshao

@wenshao wenshao commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

mcp-tool.test.ts > auto-reconnect on connection error > reconnects instead of reporting a timeout when the server is known disconnected has been red on main since the replay-safety gate merged (#8387, 0cb109f513), and it fails deterministically — bisected: green at d1648b3af9, red from 0cb109f513 on. Every full-profile PR behind it fails the required Test (ubuntu-latest) job on this one test; #8468 and #8471 are both currently blocked by it.

The bug

#8387's gate protects against re-executing a call that may have completed before its connection failed — sound policy, keyed on the tool's idempotency annotations. But it also fires in the one case its own premise excludes: a call issued while the server was already known DISCONNECTED never reached the server. Nothing "may have completed"; retrying it is the first delivery of that call, not a replay. As landed, every dead transport became a permanent UNSAFE_REPLAY error for every unannotated tool — which is exactly the recovery path the existing (and now-red) test asserts.

The fix

  • The invocation snapshots getMCPServerStatus(serverName) before issuing its call — by the time the error surfaces, the status has been overwritten by the failure itself.
  • The gate (both the pre-reconnect check and the post-reconnect one) is skipped only when that snapshot says DISCONNECTED.
  • The fresh post-reconnect invocation snapshots its own pre-call status, so a call that dies mid-flight on the recovered connection faces the gate with no carve-out on the next hop.

All of #8387's refusal cases are unchanged — they run against a live-at-call-start connection. 91/91 in mcp-tool.test.ts.

Test plan

  • The previously-red test is the regression test; it passes with the fix and fails without it.
  • Full mcp-tool.test.ts: 91/91. eslint --max-warnings 0 clean.
中文说明

概述

mcp-tool.test.ts 的 "reconnects instead of reporting a timeout when the server is known disconnected" 自 #8387(0cb109f513)合入后在 main 上持续红灯,且为确定性失败——二分定位:d1648b3af9 绿,0cb109f513 起红。此后每个 full-profile PR 的必需 Test (ubuntu-latest) 任务都挂在这一个测试上;#8468#8471 当前均被其阻塞。

缺陷

#8387 的闸门防止重放"可能已经执行完成"的调用——策略本身正确,以工具的幂等性注解为准。但它同样拦截了其前提本身排除的场景:在服务器已知 DISCONNECTED 时发出的调用从未到达服务器,不存在"可能已完成";重试它是该调用的首次投递,不是重放。按合入的行为,每个断连的 transport 对所有无注解工具都变成永久性 UNSAFE_REPLAY 错误——而这正是现有(现已红灯)测试断言的恢复路径。

修复

  • invocation 在发出调用前快照 getMCPServerStatus(serverName)——等错误浮出时,状态早已被失败本身覆盖。
  • 仅当快照为 DISCONNECTED 时跳过闸门(重连前与重连后两处检查一致)。
  • 重连后的新 invocation 快照自己的调用前状态:在恢复的连接上中途死掉的调用,下一跳照常面对无豁免的闸门。

#8387 的全部拒绝用例不受影响——它们的场景都是调用时连接存活。mcp-tool.test.ts 91/91。

测试计划

  • 此前红灯的测试即回归测试;带修复通过,去掉修复失败。
  • 全套 mcp-tool.test.ts:91/91;eslint --max-warnings 0 无告警。

The replay-safety gate (QwenLM#8387) refuses to auto-replay a call that may
have completed before its connection failed, unless the tool's
annotations prove idempotency. Landed, it also refused the case its own
premise excludes: a call issued while the server was already known
DISCONNECTED never reached the server, so nothing 'may have completed'
— retrying it is the first delivery of the call, and refusing that
turns every dead transport into a permanent error for every unannotated
tool. The existing reconnect test asserts exactly this behaviour and
has been red on main since the gate merged, failing the required Test
job on every full-profile PR behind it.

The invocation now snapshots the server status before issuing its call;
the gate is skipped only when that snapshot says DISCONNECTED. The
fresh post-reconnect invocation snapshots its own status, so a call
that dies mid-flight on the recovered connection faces the gate with no
carve-out on the next hop. All QwenLM#8387 refusal cases are unchanged:
91/91 in mcp-tool.test.ts.
@wenshao

wenshao commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator Author

@qwen-code /takeover

@qwen-code-dev-bot qwen-code-dev-bot added the autofix/takeover Summon the autofix loop to manage this PR (remove to release; needs triage+) label Aug 3, 2026
@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤝 Takeover engaged: the autofix loop now manages this PR — it will address new review feedback and resolve base conflicts until the label is removed or the round cap is reached. This is a fork PR, so the first round comes from the next scheduled scan (usually within minutes). Remove the autofix/takeover label (or comment @qwen-code /takeover stop) to release.

中文说明

🤝 已接管:autofix 循环现在管理此 PR —— 将持续处理新的评审反馈与 base 冲突,直到移除标签或达到轮次上限。本 PR 来自 fork,首轮处理将由下一次定时扫描执行(通常几分钟内)。移除 autofix/takeover 标签(或评论 @qwen-code /takeover stop)即可释放。

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finished — CI landed green on fd5d806 and the deferred approval was posted. finalize run

Qwen Triage 已完成 —— fd5d806 的 CI 全绿,延迟审批已提交。查看 finalize 运行

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Re-run after the redesign — the PR has moved a long way since the first pass, so this updates all three staged comments against the current head.

  • Template: still off-template headings (Summary / The bug / The fix / Test plan), same call as before: the content is all there with a full Chinese translation, not gating on headings. What has changed since: the body is now also factually stale — it still describes the status-snapshot mechanism that was removed two autofix rounds ago in response to the Critical finding. The description needs a rewrite before merge, both to match the template and to describe what actually ships (see the Stage 3 note).
  • Problem: unchanged and still real — an observed, bisected regression. Since fix(core): Avoid replaying unsafe MCP tool calls #8387, a call that provably never reached the server was gated as UNSAFE_REPLAY for every unannotated tool. fix(core): align MCP reconnect timeout test with safe replay policy #8478 made the test live with that behavior via fixture annotations; it didn't fix the production path. The stale "red CI blocking fix(review): stop the reverse-audit loop while there is still time to report #8468/feat(review): a cost ledger from the records already on disk #8471" framing was already discussed in the first pass and doesn't change the verdict here.
  • Direction: aligned. Restores the recovery path fix(core): Avoid replaying unsafe MCP tool calls #8387's gate inadvertently removed, without weakening the gate's premise. The redesign goes further than the original PR: the carve-out now keys on delivery-level evidence (the SDK's pre-send rejection) instead of a status snapshot — which is exactly what the Critical review finding asked for.
  • Size: core path (packages/core/src/tools/mcp-tool.ts) — 41 production lines (39 + 2), 445 test lines, 0 generated/schema lines. Well under every threshold.
  • Approach: scope is right. One exact-match helper, two guard conditions, symmetric debug logging, documentation of the accepted trade-off — nothing else in production code. I don't see a simpler path, and every prior finding is reflected in the current shape (status reads gone, both gate sites consistent, next hop judged by its own evidence).
  • Risk: no high-risk-path match (mcp-tool.ts is not in the revert-correlated set), no elevated risk signals.

Moving on to code review. 🔍

中文说明

重设计后的重跑——自首轮 triage 以来 PR 变化很大,本次按当前 head 更新全部三条阶段评论。

  • 模板:标题仍与仓库模板不一致(Summary / The bug / The fix / Test plan),结论与上次相同:内容完整且附完整中文翻译,不以标题拦截。但与上次不同的是:正文现在事实上已过时——仍在描述两轮 autofix 前因 Critical 发现而移除的状态快照机制。合入前需要重写描述,既为对齐模板,也为准确描述实际合入的内容(见 Stage 3 说明)。
  • 问题:不变且仍然成立——已观测、经二分定位的回归。自 fix(core): Avoid replaying unsafe MCP tool calls #8387 起,确证从未到达服务器的调用对所有无注解工具都被闸门判为 UNSAFE_REPLAYfix(core): align MCP reconnect timeout test with safe replay policy #8478 只是用夹具注解让测试绕过了该行为,并未修复生产路径。「红灯 CI 阻塞 fix(review): stop the reverse-audit loop while there is still time to report #8468/feat(review): a cost ledger from the records already on disk #8471」的过时表述首轮已讨论,不影响此处结论。
  • 方向:对齐。恢复 fix(core): Avoid replaying unsafe MCP tool calls #8387 闸门误删的恢复路径,且不削弱闸门前提。重设计比原 PR 更进一步:豁免改为基于投递级证据(SDK 发送前拒绝)而非状态快照——正是 Critical 审查发现所要求的修复方向。
  • 规模:核心路径(packages/core/src/tools/mcp-tool.ts)——41 行生产代码(39 + 2),测试 445 行,生成/schema 0 行。远低于所有阈值。
  • 方案:范围合理。一个精确匹配辅助函数、两处守卫条件、对称的调试日志、对已接受取舍的文档说明——生产代码仅此而已。没有看到更简路径,且每条先前发现都体现在当前形态中(状态读取已移除、两处闸门一致、下一跳以自身证据受审)。
  • 风险:未命中高风险路径(mcp-tool.ts 不在与 revert 相关的路径集中),无升级风险信号。

进入代码审查 🔍

Qwen Code · qwen3.8-max

Reviewed at fd5d80672cc7cce72d7b61b015f350e252f111a3 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Code review

Independent proposal first, before the diff: given the Critical finding (a DISCONNECTED snapshot is not proof of non-delivery — teardown writes the status before the transport closes, and network transports report transient errors without closing), the only client-side signal that proves a call never left the machine is the SDK's own pre-send rejection — Protocol.request() rejects with a bare Error('Not connected') before writing when the transport is already gone. So: exact-match on that rejection, fail closed on everything else, remove the status snapshot entirely, keep both gate sites consistent, and let the retried call be judged by its own failure evidence on the next hop. That is exactly what this revision implements.

What I verified reading the diff against current main:

  • Reachability. Not connected matches the existing /not connected/i entry in MCP_CONNECTION_ERROR_PATTERNS, so shouldAttemptReconnect() returns true and the flow actually reaches the gate; the carve-out sits at the right point in handleReconnectOnError() (after abort / reconnect-eligibility checks, before reconnect), covering both gate sites.
  • No carve-out chaining. The post-reconnect replay runs via newInvocation.execute(), so a retry that dies mid-flight on the recovered connection faces the full gate with its own error evidence — pinned by the "dies mid-flight on the recovered connection" test.
  • Exact match is exact. getErrorMessage() returns error.message raw for a bare Error with no cause; any wrapper, cause suffix, or rewording falls through to the gate. The mutation-killing negative tests ('Not connected to database', both server statuses) pin the === boundary — loosening it to .includes() fails them.
  • The old design's holes are pinned shut by name. Two negative tests reproduce precisely the scenarios that sank the snapshot approach: an ambiguous failure that flips the status mid-flight, and an ambiguous failure issued while the status was already DISCONNECTED (the teardown window). Both stay gated.
  • The contract is pinned against the real dependency, not a mock's idea of it: a genuine never-connected SDK Client exercises the carve-out, and a real client/server pair over InMemoryTransport with an in-flight transport close stays gated. An SDK bump that rewords the pre-send rejection turns the first test red instead of silently losing the recovery path — and since the direction of drift is fail-closed (back to today's gated behavior), a bump can't regress safety, only the carve-out.

Two scenarios I considered and put down as non-issues: a malicious trusted server answering a delivered call with a crafted JSON-RPC error message of exactly Not connected — server-originated errors surface as McpError with a code prefix, so they don't exact-match; and even in the worst case that trick requires a trusted server and is strictly narrower than the annotation-spoofing surface the gate already accepts for trusted servers. And the ^1.30.0 caret range allowing a message reword — covered above (fail-closed drift, red test).

Everything raised in earlier rounds is resolved in this head: the Critical (R1-2) by the redesign itself; R2-1 (unversioned SDK string) and R3-1 (unpinned exact-match boundary) by the real-SDK and boundary tests; and both of my first-pass nits — the status read is gone entirely, and the carve-out now has tests of its own, so the merged suite pins it regardless of #8478's annotated fixture.

Testing

Unattended CI run — no PR code was built or executed here; the evidence below is the PR's own CI read through the API at the reviewed commit, plus the sandboxed verification running in parallel.

  • The required Test (ubuntu-latest, Node 22.x) job is still in progress on fd5d806; everything else is green or skipped (the normal fork-PR matrix). Table below; the finalize job will update it when CI lands.
  • The sandboxed verify job is running right now in this triage's workflow run — the A/B against the base build that settles the behavioral claim (never-delivered rejection → reconnect + first delivery; ambiguous failure → still gated). Its report posts to this PR on completion; treat it as pending until then. The author's mutation-testing numbers quoted in the thread are their claim, not independently re-run evidence.
  • Real-scenario TUI testing: N/A — MCP invocation internals, no TUI surface.

Final CI results for fd5d806 (auto-updated by the triage finalize job after CI completed):

Check Conclusion
Classify PR ✅ success
Desktop Shell (ubuntu-22.04) ✅ success
Test (ubuntu-latest, Node 22.x) ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success

One row per check name (latest run); skipped checks omitted; failures sort first. / 每个检查名一行(取最新一次运行),省略 skipped,失败项排在最前。

中文说明

代码审查:先独立给出方案再读 diff。既然 Critical 发现已证明 DISCONNECTED 快照不能作为未投递的证据(拆除流程先写状态后关 transport,网络传输会在不关闭的情况下报瞬时错误),客户端唯一能证明调用从未离开本机的信号就是 SDK 自己的发送前拒绝——transport 已消失时,Protocol.request() 在写入请求前以裸 Error('Not connected') 拒绝。因此:精确匹配该拒绝、其余一律失败关闭、彻底移除状态快照、两处闸门保持一致、重试调用在下一跳以自身错误证据受审。本版本正是这样实现的。

已核实:Not connected 命中现有 /not connected/i 模式,shouldAttemptReconnect() 返回 true,流程确实到达闸门;豁免位于 handleReconnectOnError() 的正确位置(abort/重连资格检查之后、重连之前),覆盖两处闸门。重连后的重放经由 newInvocation.execute() 执行,因此在恢复的连接上中途失败的调用以自己的错误证据面对完整闸门——由对应测试锚定。getErrorMessage() 对无 cause 的裸 Error 原样返回 error.message;任何包装、cause 后缀或措辞变化都会落入闸门。突变杀灭负例('Not connected to database',两种服务器状态)锚定了 === 边界——放宽为 .includes() 即失败。两个负例按名复现了击沉快照方案的确切场景:中途翻转状态的歧义失败、状态已为 DISCONNECTED 时发出的歧义失败(拆除窗口)——均仍被拦截。契约直接锚定在真实依赖上:真正从未连接的 SDK Client 走通豁免;真实 client/server 对经 InMemoryTransport 在途关闭 transport 则仍被拦截。SDK 升级若改写发送前拒绝措辞,第一个测试会变红而不是静默丢失恢复路径——且漂移方向是失败关闭(回到当前被拦截的行为),升级不可能回归安全性,最多丢失豁免本身。

两个考虑后判定不构成问题的场景:恶意 trusted 服务器对已投递的调用回以措辞恰为 Not connected 的 JSON-RPC 错误——服务器来源的错误以带 code 前缀的 McpError 浮出,不会精确匹配;即便最坏情形,该手法也需 trusted 服务器,且严格窄于闸门对 trusted 服务器已然接受的注解伪造面。以及 ^1.30.0 caret 范围允许措辞改写——见上(失败关闭的漂移 + 变红的测试)。

此前各轮的全部发现均已在当前 head 解决:Critical(R1-2)由重设计本身解决;R2-1(无版本锚定的 SDK 字符串)与 R3-1(未锚定的精确匹配边界)由真实 SDK 测试与边界测试解决;我首轮的两条 nit 也已解决——状态读取彻底移除,豁免现有自己的测试,合入后的测试套件无论如何都会锚定它(不依赖 #8478 的带注解夹具)。

测试:无人值守 CI 运行——未构建或执行任何 PR 代码;以上证据为通过 API 读取的该 PR 自身 CI,以及并行运行的沙箱验证。必需的 Test (ubuntu-latest, Node 22.x)fd5d806仍在运行;其余为绿色或跳过(fork PR 的正常矩阵)。沙箱 verify 任务正在本次 triage 工作流运行中执行——与 base 构建的 A/B 对比,用于落定行为性声明;完成后报告会发到本 PR,在此之前视为未决。线程中引用的作者突变测试数字为其声明,非独立复跑证据。真实场景 TUI 测试:N/A——MCP 调用内部,无 TUI 可见面。

Qwen Code · qwen3.8-max

Reviewed at fd5d80672cc7cce72d7b61b015f350e252f111a3 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Confidence: 4/5 — clean redesign that answers the Critical finding on its own terms, with genuinely good tests; the one real nit is that the PR description still documents the abandoned design, and CI hasn't landed on this head yet.

Stepping back: the arc of this PR is the review process working as intended. The original snapshot carve-out was the obvious fix and was wrong in exactly the subtle way the Critical finding and @wenshao's TOCTOU analysis both nailed down — a status map is a belief about the world, not evidence about a specific call. The redesign takes the only honest exit: key the carve-out on the one client-side event that constitutes non-delivery (the SDK rejecting before it writes), treat everything else as ambiguous, and fail closed. My independent proposal for this round was precisely that mechanism, and I couldn't find a simpler one. The implementation is 41 production lines with no drive-by edits, and the test suite is better than the fix: mutation-killed exact-match boundaries, named reproductions of both scenarios that sank the old design, and both directions of the carve-out pinned against the real SDK rather than a mock's opinion of it. Six months from now, this reads as a careful, self-contained gate refinement — not as debt.

The standing CHANGES_REQUESTED on this PR is from the /review run against the pre-redesign commit; the finding it cites (R1-2) is what drove the redesign, and this pass verifies the current head on its own merits.

The 4 and not 5, plainly named:

  1. The PR body is stale and must be rewritten before merge. It still describes the status-snapshot mechanism ("snapshots getMCPServerStatus(serverName) before issuing its call") that two autofix rounds removed. It also still carries the red-CI framing that fix(core): align MCP reconnect timeout test with safe replay policy #8478 made moot — @wenshao's own point pre-release: fix ci #1 asked for the summary to be rewritten as a deliberate policy loosening, and that's still the right ask: the merge record should say what actually ships.
  2. CI on fd5d806 is still in flight, and the sandboxed A/B verification hasn't posted its report yet. Neither is a doubt about the code — they're results that don't exist yet.

Approval deferred until CI lands green on fd5d80672cc7cce72d7b61b015f350e252f111a3 — the finalize step will post the commit-pinned approval if everything comes back green, and withhold it if anything lands red or the head moves.

中文说明

置信度:4/5 —— 干净的重设计,以 Critical 发现自身认可的方式回应了它,测试质量真正出色;唯一的实质 nit 是 PR 描述仍在记载已被放弃的设计,且 CI 尚未在当前 head 上落定。

退一步看:这个 PR 的演进正是审查流程按预期工作的样本。最初的快照豁免是显而易见的修复,也恰好以 Critical 发现和 @wenshao 的 TOCTOU 分析共同指出的那种微妙方式错了——状态表是对世界的认知,不是关于某次具体调用的证据。重设计选择了唯一诚实的出口:豁免锚定在唯一构成未投递的客户端事件上(SDK 在写入前拒绝),其余一切视为歧义,失败关闭。我本轮的独立方案正是这一机制,且没有找到更简的方案。实现为 41 行生产代码、无任何顺手改动;测试比修复本身更好:突变杀灭的精确匹配边界、按名复现击沉旧设计的两个场景、豁免的两个方向都锚定在真实 SDK 而非 mock 的看法上。六个月后回看,这是一次谨慎、自洽的闸门精化——不是债务。

本 PR 上悬挂的 CHANGES_REQUESTED 来自针对重设计前提交的 /review 运行;其引用的发现(R1-2)正是驱动本次重设计的原因,本次审查按当前 head 的自身表现独立验证。

给 4 而非 5,直说两点:

  1. PR 正文已过时,合入前必须重写。 它仍在描述两轮 autofix 前已移除的状态快照机制(「在发出调用前快照 getMCPServerStatus(serverName)」),也仍带着已被 fix(core): align MCP reconnect timeout test with safe replay policy #8478 化解的红灯 CI 表述——@wenshao 自己的第 1 点就要求把摘要重写为一次有意的策略放宽,这个要求仍然成立:合入记录应当描述实际合入的内容。
  2. fd5d806 的 CI 仍在运行,沙箱 A/B 验证尚未发布报告。两者都不是对代码的质疑——只是结果尚不存在。

审批推迟至 CI 在 fd5d80672cc7cce72d7b61b015f350e252f111a3 上全绿后执行——若全部转绿,finalize 步骤将发布锚定该提交的审批;若有红灯或 head 移动,则不予审批。

Qwen Code · qwen3.8-max

Reviewed at fd5d80672cc7cce72d7b61b015f350e252f111a3 · re-run with @qwen-code /triage

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🔀 Base updated: red check(s) [Test (ubuntu-latest, Node 22.x)] pass on current main — merged current main via update-branch; CI will re-run.

中文说明

🔀 已更新 base:红色检查 [Test (ubuntu-latest, Node 22.x)] 在当前 main 上通过 —— 已通过 update-branch 合入当前 main,CI 将重新运行。

@qwen-code-dev-bot

qwen-code-dev-bot commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

AutoFix round 5 finishedview run. See this round's report below.

中文说明

AutoFix 第 5 轮已完成 —— 查看运行。本轮报告见下方。

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下:

Autofix review round — no action needed

No new review feedback was found for this round:

  • Reviews: none since the last evaluation (2026-08-03T17:33:24Z)
  • Inline comments: none
  • Issue-level comments: none
  • Failed checks: none — all completed checks are passing (one CI test job was still in progress at evaluation time); none are red
  • Base conflicts: none (--conflict false); no merge was performed

No code changes were made, and no commit was created. The PR head remains unchanged.

中文说明

Autofix 审查轮次 —— 无需处理

本轮未发现新的审查反馈:

  • 审查(Reviews): 自上次评估(2026-08-03T17:33:24Z)以来没有新的审查
  • 行内评论:
  • Issue 级评论:
  • 失败的检查: 无 —— 所有已完成的检查均通过(评估时有一个 CI 测试任务仍在运行中);没有任何检查处于红色(失败)状态
  • 与基础分支的冲突: 无(--conflict false);未执行任何合并操作

未做任何代码修改,也未创建任何提交。PR 的 head 分支保持不变。

Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。


🧠 Handled by Qwen Code · model/模型 qwen3.8-max

@doudouOUC doudouOUC left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. Suggestions are inline. 1 Suggestion-level finding(s) could not be anchored to a changed line and were dropped; nothing further to act on here.

中文说明

已审查。 建议见行内评论。 1 条建议级发现无法锚定到改动行,已丢弃;此处无需进一步处理。

— qwen3.7-max via Qwen Code /review (v0.21.4)

Comment thread packages/core/src/tools/mcp-tool.ts Outdated
Comment on lines +394 to +395
const neverDelivered =
this.statusAtCallStart === MCPServerStatus.DISCONNECTED;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] Replay safety gate bypass is silent — no diagnostic logging when the neverDelivered carve-out fires.

Failure scenario: An oncall engineer investigating suspected double-execution of a non-idempotent MCP tool sees the same log output whether the replay gate was satisfied via canSafelyReplay() or bypassed via neverDelivered. The two code paths produce identical log output, making incident investigation require local reproduction with temporary logging.

Suggested change
const neverDelivered =
this.statusAtCallStart === MCPServerStatus.DISCONNECTED;
const neverDelivered =
this.statusAtCallStart === MCPServerStatus.DISCONNECTED;
if (neverDelivered) {
debugLogger.info(`Replay safety gate bypassed for '${this.serverName}': call was never delivered (status at call start: ${this.statusAtCallStart})`);
}
中文说明

[Suggestion] 重放安全闸门绕过时无诊断日志 — neverDelivered 豁免触发时没有记录。

失败场景:值班工程师调查疑似非幂等 MCP 工具双重执行时,无论重放闸门是通过 canSafelyReplay() 满足还是通过 neverDelivered 绕过,日志输出完全相同。两条代码路径产生相同的日志输出,使得事故调查需要在本地添加临时日志复现。

— qwen3.7-max via Qwen Code /review (v0.21.4)

@wenshao

wenshao commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator Author

Review

Verdict: the behavior change is correct and worth keeping — but as of the current head it ships with zero test coverage, and the PR description's justification is stale. Details and evidence below.

What this PR does

DiscoveredMCPToolInvocation now snapshots getMCPServerStatus(serverName) at the top of execute() and skips #8387's replay-safety gate (both the pre-reconnect and post-reconnect check) when that snapshot was DISCONNECTED. Rationale: a call issued against an already-dead transport was never delivered, so retrying it is a first delivery, not a replay — and refusing it turned every dead transport into a permanent UNSAFE_REPLAY error for every unannotated tool.

The reasoning is sound, the snapshot-before-the-call ordering is the right mechanism (a gate-time read is useless — the failure itself overwrites the status), and handleReconnectOnError is only reachable from execute(), so the field is always initialized before it's read.

Verification I ran

Worktree at head f04df0c, packages/core, vitest run src/tools/mcp-tool.test.ts:

build result
head as-is 91/91 pass
head with both !neverDelivered && guards deleted 91/91 pass

That second row is the finding.

1. 🔴 The change is untested — mutation-verified

Deleting the entire carve-out from this PR leaves the suite green. The reason: #8478 ("align MCP reconnect timeout test with safe replay policy", merged 2026-08-03T17:26Z) rewrote the fixtures of reconnects instead of reporting a timeout when the server is known disconnected to build its tools with trust: true + idempotentAnnotations. That test now takes the canSafelyReplay() === true path and never touches the carve-out. Since this branch merged main twice, that rewrite is in the head — so the test named in the PR body as "the regression test" no longer is one.

I wrote two probes that do cover it. Both pass on the head and both fail with the carve-out removed:

Probe A + B (fail without the fix, pass with it)
// A: an UNANNOTATED, untrusted tool on a known-disconnected server must
//    reconnect and deliver (this is the user-visible bug being fixed).
it('unannotated tool on a disconnected server reconnects and delivers', async () => {
  const deadClient = { callTool: vi.fn().mockRejectedValueOnce(new Error('Connection closed')) };
  const liveClient = { callTool: vi.fn().mockResolvedValueOnce({ content: [{ type: 'text', text: 'OK' }] }) };
  const newTool = new DiscoveredMCPTool(
    mockCallableToolInstance, serverName, serverToolName, baseDescription, inputSchema,
    undefined /* untrusted */, undefined, undefined, liveClient /* no annotations */,
  );
  const discoverToolsForServer = vi.fn().mockResolvedValue(undefined);
  const mockConfig = {
    isTrustedFolder: () => true,
    getToolRegistry: () => ({ discoverToolsForServer, ensureTool: vi.fn().mockResolvedValue(newTool) }),
    getTruncateToolOutputThreshold: () => 0,
    getTruncateToolOutputLines: () => 0,
  };
  updateMCPServerStatus(serverName, MCPServerStatus.DISCONNECTED);
  const tool = new DiscoveredMCPTool(
    mockCallableToolInstance, serverName, serverToolName, baseDescription, inputSchema,
    undefined, undefined, mockConfig as any, deadClient,
  );
  const result = await tool.build({ param: 'x' }).execute(new AbortController().signal);
  expect(discoverToolsForServer).toHaveBeenCalled();
  expect(liveClient.callTool).toHaveBeenCalledTimes(1);
  expect(result.llmContent).toEqual([{ text: 'OK' }]);
});

// B: the no-chaining claim — once the reconnect restores CONNECTED, a call
//    that dies mid-flight on the RECOVERED connection faces the full gate.
it('mid-flight death on the recovered connection is still gated', async () => {
  const deadClient = { callTool: vi.fn().mockRejectedValueOnce(new Error('Connection closed')) };
  const recoveredClient = { callTool: vi.fn().mockRejectedValue(new Error('Connection closed')) };
  const thirdClient = { callTool: vi.fn().mockResolvedValue({ content: [{ type: 'text', text: 'REPLAYED' }] }) };
  const mk = (c: any) => new DiscoveredMCPTool(
    mockCallableToolInstance, serverName, serverToolName, baseDescription, inputSchema,
    undefined, undefined, undefined, c,
  );
  const ensureTool = vi.fn().mockResolvedValueOnce(mk(recoveredClient)).mockResolvedValue(mk(thirdClient));
  // a real reconnect flips the global status back to CONNECTED
  const discoverToolsForServer = vi.fn().mockImplementation(async () => {
    updateMCPServerStatus(serverName, MCPServerStatus.CONNECTED);
  });
  const mockConfig = {
    isTrustedFolder: () => true,
    getToolRegistry: () => ({ discoverToolsForServer, ensureTool }),
    getTruncateToolOutputThreshold: () => 0,
    getTruncateToolOutputLines: () => 0,
  };
  updateMCPServerStatus(serverName, MCPServerStatus.DISCONNECTED);
  const tool = new DiscoveredMCPTool(
    mockCallableToolInstance, serverName, serverToolName, baseDescription, inputSchema,
    undefined, undefined, mockConfig as any, deadClient,
  );
  await expect(
    tool.build({ param: 'x' }).execute(new AbortController().signal),
  ).rejects.toThrow(/may have completed before the connection failed/);
  expect(recoveredClient.callTool).toHaveBeenCalledTimes(1);
  expect(thirdClient.callTool).not.toHaveBeenCalled();
});

Result with the fix: both pass (94/94 with the file). Result with !neverDelivered && removed: both fail, mcp-tool.test.ts still 91/91.

Probe B is the good news: the no-chaining property the PR claims actually holds — the fresh invocation does snapshot its own pre-call status and the second hop is gated. That's the subtlest part of the change and it works; it just isn't asserted anywhere.

2. 🟡 The PR description no longer matches reality

Worth rewriting so the merge rationale is the product bug (unannotated MCP tools can never auto-recover from a dead transport) rather than a CI-unblock that already happened. It also changes the relationship to #8478: that PR adjusted the test to the gate, this one adjusts the gate — reviewers should see both framed together.

3. 🟡 Residual risk: DISCONNECTED is a proxy for "not delivered", not proof

The carve-out's soundness rests entirely on the global status map being accurate at call time. Two places where it isn't, both narrow but both in the exact hazard class #8387 targets:

  • McpClient.disconnect() (packages/core/src/tools/mcp-client.ts:629-635) writes DISCONNECTED to the global registry before await this.transport.close(). A call issued in that window snapshots DISCONNECTED but can still go out over the live transport and execute. If it then fails as the transport tears down, the carve-out replays it.
  • getMCPServerStatus returns DISCONNECTED for names not in the registry (mcp-status.ts:92) — unknown and disconnected are the same value. Any invocation whose server name was dropped by removeMCPServerStatus (disable / hot-reload / runtime-add rollback) gets the carve-out unconditionally rather than by evidence.

Also worth quantifying: the carve-out doesn't chain across a successful reconnect (probe B), but it does chain while the status stays DISCONNECTED. Probe C — reconnect returns a tool but never restores CONNECTED — delivered the call ["hop0","hop1","hop2","hop3"], i.e. 4 deliveries, bounded only by MAX_RECONNECT_RETRIES. That's correct if the status is honest, and 4× the damage if it isn't. Consider tightening the comment on line 388 to say believed-never-delivered, so the next reader doesn't take neverDelivered as a fact.

4. 🟡 Possible gap: pooled servers

PoolEntry.updateGlobalStatus writes an any-CONNECTED-wins aggregate for the server name. For a pooled server with a live sibling entry, a genuinely dead per-session connection still reads CONNECTED, so an unannotated tool there keeps hitting UNSAFE_REPLAY — the bug this PR fixes would survive on that path. I didn't verify this end-to-end; worth a check before claiming the class of bug is closed.

5. 🔵 Nits

  • Three comment blocks (~20 lines) restate the same rationale for 8 lines of code. AGENTS.md says comments default to none and earn their place by explaining a non-obvious why — the why here genuinely is non-obvious, so one block deserves to stay, but the field docstring plus the two inline blocks is redundant. Suggest keeping the field docstring and reducing the two call-site blocks to a one-liner each.
  • statusAtCallStart is a mutable field written in execute() and read in handleReconnectOnError — an implicit ordering contract that happens to hold today. Threading it as a parameter (executeexecuteWith*handleReconnectOnError) would make it impossible to read an unset value if a future entry point is added.

Recommendation

Keep the fix; add probes A and B (they're the missing regression tests, and B pins the no-chaining invariant that's easy to break later); refresh the PR body. Items 3 and 4 are follow-up material, not blockers for this diff.

中文说明

结论: 行为变更本身是对的、值得合入,但按当前 head 它完全没有测试覆盖,且 PR 描述的理由已经过期。

验证: head f04df0c,packages/corevitest run src/tools/mcp-tool.test.ts——原样 91/91 通过;把两处 !neverDelivered && 全部删掉后仍然 91/91 通过。原因:#8478(2026-08-03T17:26Z 合入)把 "reconnects instead of reporting a timeout when the server is known disconnected" 的 fixture 改成了 trust: true + idempotentAnnotations,该用例已走 canSafelyReplay() === true 分支,不再触碰本次的豁免逻辑;而本分支两次合入 main,已经包含那次改写。所以 PR 里点名的"回归测试"如今并不是回归测试。

我补了两个探针(详见上方英文折叠块),带修复通过、去掉修复失败:A 覆盖"无注解 + 未授信工具在断连服务器上应重连并投递";B 覆盖 PR 最微妙的主张——重连恢复 CONNECTED 后,新 invocation 用自己的调用前快照,中途死掉照样被闸门拦住。好消息是 B 通过,该不变量确实成立,只是没有任何地方断言它。

其余问题:

  1. PR 描述里"main 持续红灯 / fix(review): stop the reverse-audit loop while there is still time to report #8468feat(review): a cost ledger from the records already on disk #8471 被阻塞 / 该测试去掉修复会失败"均已不成立(fix(core): align MCP reconnect timeout test with safe replay policy #8478 早 6 小时修好了),建议改写为以产品缺陷为理由。
  2. 豁免的正确性完全依赖全局状态图准确。两处窗口:McpClient.disconnect()(mcp-client.ts:629-635)先写全局 DISCONNECTEDawait transport.close(),该窗口内发出的调用仍可能真正投递并执行;getMCPServerStatus 对未登记的 server 名同样返回 DISCONNECTED(mcp-status.ts:92),"未知"与"断连"不可区分。另外探针 C 显示:重连返回了工具但状态未恢复 CONNECTED 时豁免会连锁,实测投递 4 次(hop0hop3),上限就是 MAX_RECONNECT_RETRIES。建议把注释里的 neverDelivered 措辞改为"推定未投递"。
  3. 池化路径可能未覆盖:PoolEntry.updateGlobalStatus 按 server 名写 any-CONNECTED-wins 聚合值,只要有存活的同名兄弟连接,真正死掉的会话级连接仍读到 CONNECTED,无注解工具依旧会撞 UNSAFE_REPLAY。这一条我没有端到端验证,建议确认。
  4. 小问题:约 20 行注释重复陈述同一理由(AGENTS.md 要求注释默认不写、只解释非显然的 why——这里的 why 确实非显然,保留字段文档注释即可);statusAtCallStart 是在 execute() 写、在 handleReconnectOnError 读的可变字段,改成参数透传更稳。

建议: 保留修复,补上探针 A、B,刷新 PR 描述;第 2、3 条作为后续跟进,不阻塞本次合入。

@wenshao

wenshao commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

Status note: main has since resolved the red test differently — #8478 rewrote the test to accept the gate's refusal (align MCP reconnect timeout test with safe replay policy), i.e. policy wins, the known-disconnected auto-reconnect behavior regression is accepted. This PR takes the opposite position: a call issued while the transport was already known-disconnected was never delivered, so nothing "may have completed" and the replay gate's own premise does not apply — the reconnect behavior every unannotated tool had is restored, safely.

The queue is unblocked either way, so this is now a pure design decision rather than a firefight:

Leaving the call to the maintainers; happy to do the rebase if the first option is preferred.

— via Claude Code (claude-fable-5)

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤖 Addressed the latest review feedback (round 1/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 1/100 轮)。改动内容与我反驳保留之处如下:

Autofix review round — PR #8482

Commit: fix(core): cover the never-delivered MCP replay carve-out with tests and logging

Feedback dispositions

1. 🔴 The change is untested — mutation-verified (@wenshao) — ADDRESSED.
Added the two missing regression tests (the reviewer's probes A and B, adapted to the file's conventions) in the auto-reconnect on connection error block:

  • reconnects and delivers an unannotated tool when the server was known disconnected — an untrusted, unannotated tool on a known-dead transport must reconnect and deliver (the user-visible bug this PR fixes).
  • still gates a call that dies mid-flight on the recovered connection — pins the no-chaining invariant: once the reconnect restores CONNECTED, the fresh invocation's own snapshot re-arms the gate for the next hop.

Mutation check reproduces the reviewer's finding and verifies the fix: with both !neverDelivered && guards deleted, exactly these two new tests fail (probe A with the UNSAFE_REPLAY error, probe B on the reconnect call count) while the previous 91 tests stay green; with the fix, 93/93 pass.

2. [Suggestion] rc:3707652692 — silent gate bypass (@doudouOUC / automated reviewer) — ADDRESSED.
handleReconnectOnError now emits a debugLogger.info line when the never-delivered carve-out fires (Replay safety gate bypassed for MCP server '<name>': call was never delivered (DISCONNECTED at call start)), consistent with the existing reconnect debug logging in the same path, so the two gate paths are distinguishable in incident investigation.

3. Nit — three comment blocks restate the same rationale (@wenshao) — ADDRESSED.
The field docstring on statusAtCallStart is now the single place carrying the full rationale (including the snapshot-before-the-attempt timing reason), the two call-site blocks are one line each, and the duplicated block in execute() is removed. Net −13 comment lines.

4. Item 3 sub-suggestion — say believed-never-delivered (@wenshao) — ADDRESSED.
Folded into the docstring rewrite: the status is now described as evidence that the call "is believed never to have reached" the server, not proof.

5. Nit — thread statusAtCallStart as a parameter (@wenshao) — DECLINED (not worth the diff growth).
handleReconnectOnError is private and reachable only through execute(), which writes the snapshot before dispatching (the reviewer independently confirmed the field is always initialized before it is read). Threading the value through three private signatures would defend against a future entry point that does not exist — the kind of speculative hardening AGENTS.md's Simplicity First rule argues against — while growing the diff. The ordering contract is now documented in the field docstring instead. Happy to revisit if the maintainer wants it.

6. Item 2 — PR description is stale (@wenshao) — DEFERRED (requires a GitHub write this mode does not perform).
Suggested refreshed framing for whoever updates the PR body: the merge rationale is the product bug — unannotated MCP tools can never auto-recover from a dead transport because the replay gate's "may have completed" premise does not apply to a call issued against an already-known-dead transport — not a CI unblock (which #8478 already provided). Relationship to #8478: that PR adjusted the test to the gate; this one adjusts the gate, and its new tests re-assert the original behavior expectation over #8478's fixture rewrite. The "red on main since #8387 / #8468 and #8471 blocked" claims should be dropped.

7. Item 3 residual risk windows + Item 4 pooled servers (@wenshao) — DEFERRED to follow-up.
Per the reviewer's own recommendation these are follow-up material, not blockers for this diff. For the record: (a) McpClient.disconnect() writing DISCONNECTED before transport.close() completes; (b) getMCPServerStatus returning DISCONNECTED for unknown names; (c) carve-out chaining while status stays DISCONNECTED, bounded by MAX_RECONNECT_RETRIES; (d) pooled servers where the any-CONNECTED-wins aggregate may mask a dead per-session connection. The new bypass log (disposition 2) makes occurrences of the carve-out observable, which should help size these in the follow-up.

8. Status note — merge vs. close (@wenshao) — MAINTAINER DECISION, left open.
The commenter explicitly leaves the design call (merge and re-assert the original test expectation vs. keep #8478's status quo) to the maintainers. This round implements the review's recommendation (keep the fix, add the missing tests); the decision itself is not this bot's to make.

No conflicts (--conflict false); origin/main was not merged.

Verification

  • cd packages/core && npx vitest run src/tools/mcp-tool.test.ts93/93 passed (91 existing + 2 new).
  • Mutation check (temporarily deleted both !neverDelivered && guards, then restored): the 2 new tests fail (probe A with UNSAFE_REPLAY, probe B on reconnect call count), remaining 91 green — confirms the tests cover the carve-out.
  • cd packages/core && npx vitest run src/tools/mcp-tool.test.ts src/tools/mcp-client.test.ts205/205 passed.
  • npm run buildpassed (exit 0).
  • npm run typecheckpassed (exit 0).
  • npm run lintpassed (exit 0).
  • npx prettier --write on the two touched files — clean (one long test line wrapped).
  • Full packages/core suite — 19093 passed; 82 failures in 12 files (logger, storage, editor, token storage, etc.). None MCP-related: the same suites fail identically at HEAD with this PR's diff removed (spot-checked 4 suites: identical failure counts), i.e. pre-existing environment-specific failures on this runner, not caused by this change.
  • Integration tests — not run: the touched behavior is exercised directly by the unit tests, not only through the bundled CLI or integration harness.
  • Settings schema — not regenerated: no settings source changed.
中文说明

Autofix 审查轮次 — PR #8482

提交:fix(core): cover the never-delivered MCP replay carve-out with tests and logging

反馈处理

1. 🔴 改动无测试覆盖(突变测试验证)(@wenshao) — 已处理。
auto-reconnect on connection error 块中补上了两条缺失的回归测试(即审查者的探针 A、B,按测试文件惯例改写):

  • reconnects and delivers an unannotated tool when the server was known disconnected —— 未授信、无注解的工具在已知断连的传输上必须重连并成功投递(本 PR 修复的用户可见缺陷)。
  • still gates a call that dies mid-flight on the recovered connection —— 钉住"不连锁"不变量:重连恢复 CONNECTED 后,新 invocation 用自己的调用前快照,下一跳照样被闸门拦住。

突变验证复现了审查者的发现并确认修复有效:删掉两处 !neverDelivered && 守卫后,恰好只有这两条新测试失败(探针 A 报 UNSAFE_REPLAY,探针 B 在重连调用次数断言上失败),原有 91 条仍全绿;带修复则 93/93 通过。

2. [Suggestion] rc:3707652692 — 闸门绕过无日志 (@doudouOUC / 自动审查器) — 已处理。
handleReconnectOnError 在 never-delivered 豁免触发时新增一条 debugLogger.infoReplay safety gate bypassed for MCP server '<name>': call was never delivered (DISCONNECTED at call start)),与同路径既有的重连调试日志风格一致,事故调查时两条闸门路径不再无法区分。

3. 小问题 — 三处注释块重复同一理由 (@wenshao) — 已处理。
statusAtCallStart 字段文档注释成为唯一承载完整理由的位置(含"先快照再尝试"的时序原因),两个调用处块各缩为一行,execute() 里的重复块删除。注释净减 13 行。

4. 第 2 条子建议 — 措辞改为"推定未投递" (@wenshao) — 已处理。
并入文档注释重写:状态现在表述为"推定调用未到达服务器"的证据,而非事实。

5. 小问题 — 把 statusAtCallStart 改为参数透传 (@wenshao) — 拒绝(不值得增加 diff)。
handleReconnectOnError 是私有方法,只能经由 execute() 到达,而 execute() 在分发前就写入了快照(审查者也独立确认该字段读取前必然已初始化)。把该值穿过三个私有方法签名是在为一个不存在的未来入口做防御——正是 AGENTS.md Simplicity First 原则反对的投机性加固——同时还会扩大 diff。时序契约已改由字段文档注释说明。若维护者坚持,可以再做。

6. 第 2 条 — PR 描述过期 (@wenshao) — 暂缓(需要本模式不执行的 GitHub 写操作)。
给更新 PR 描述的人的建议框架:合入理由应是产品缺陷——无注解 MCP 工具永远无法从死传输自动恢复,因为重放闸门的"可能已执行完成"前提对"发起时传输已确定死亡"的调用不成立——而不是 CI 解锁(#8478 已经完成)。与 #8478 的关系:那个 PR 让测试迁就闸门,本 PR 让闸门合理化,且新测试在 #8478 的 fixture 改写之上重新断言了原始行为预期。"main 自 #8387 起持续红灯 / #8468#8471 被阻塞"的说法应删除。

7. 第 3 条残余风险窗口 + 第 4 条池化服务器 (@wenshao) — 暂缓,转后续跟进。
按审查者自己的建议,这些是后续跟进材料,不阻塞本 diff。记录在案:(a) McpClient.disconnect()transport.close() 完成前就写全局 DISCONNECTED;(b) getMCPServerStatus 对未登记的名字也返回 DISCONNECTED;(c) 状态保持 DISCONNECTED 时豁免会连锁,上限为 MAX_RECONNECT_RETRIES;(d) 池化服务器中 any-CONNECTED-wins 聚合值可能掩盖真正死掉的会话级连接。新增的绕过日志(第 2 条)让豁免触发可观测,有助于在后续跟进中评估这些窗口的影响面。

8. 状态说明 — 合入还是关闭 (@wenshao) — 维护者决策,保持开放。
评论者明确把设计决策(合入并重新断言原始测试预期 vs. 维持 #8478 现状)留给维护者。本轮按审查建议执行(保留修复、补齐测试);决策本身不由本机器人做出。

无冲突(--conflict false);未合并 origin/main

验证

  • cd packages/core && npx vitest run src/tools/mcp-tool.test.ts93/93 通过(91 条原有 + 2 条新增)。
  • 突变验证(临时删除两处 !neverDelivered && 守卫后还原):2 条新测试失败(探针 A 报 UNSAFE_REPLAY,探针 B 在重连调用次数上失败),其余 91 条全绿——确认测试确实覆盖了豁免逻辑。
  • cd packages/core && npx vitest run src/tools/mcp-tool.test.ts src/tools/mcp-client.test.ts205/205 通过
  • npm run build通过(exit 0)。
  • npm run typecheck通过(exit 0)。
  • npm run lint通过(exit 0)。
  • 对两个改动文件运行 npx prettier --write — 干净(一条过长测试行被折行)。
  • packages/core 全量测试 — 19093 通过;12 个文件 82 条失败(logger、storage、editor、token storage 等)。均与 MCP 无关:在本 PR diff 移除后的 HEAD 上同样失败(抽查 4 个套件,失败数完全一致),属于本 runner 上既有的环境相关失败,非本次改动引入。
  • 集成测试 — 未运行:改动的行为由单元测试直接覆盖,并非只能通过打包后的 CLI 或集成测试框架验证。
  • Settings schema — 未重新生成:未改动任何 settings 源。

Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。

Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。


🧠 Handled by Qwen Code · model/模型 qwen3.8-max

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Critical] R1-2 packages/core/src/tools/mcp-tool.ts:389-391 (snapshot at :486) — the neverDelivered carve-out treats a DISCONNECTED status snapshot as proof the call never reached the server, but for streamable-HTTP/SSE servers the pinned MCP SDK (1.30.0) fires onerror on transient errors (SSE stream drop, POST failures such as a proxy 502/ECONNRESET) WITHOUT closing the transport — subsequent calls are still delivered, and nothing restores CONNECTED until a reconnect (~95s health-monitor window in non-pool mode, the standard interactive CLI path). Probe-verified double execution: driving the real shipped invocation with a delivered-then-failed call under a DISCONNECTED snapshot is automatically replayed (deliveries=2); with the carve-out removed the identical input throws UNSAFE_REPLAY (deliveries=1). The snapshot also keys on getMCPServerStatus(), which falls back to DISCONNECTED for servers with no registry entry — the exact comparison isExecutionTimeoutFailure() in this same file documents as unsafe and deliberately avoids. This re-opens the double-execution class #8387's gate closed, for unannotated (possibly non-idempotent) tools. Fix direction: require positive evidence of non-delivery — at minimum snapshot only a recorded status (this.statusAtCallStart = getAllMCPServerStatuses().get(this.serverName); fails closed for unregistered servers), and tie the bypass to delivery-level evidence (a local send-side rejection proving the call never left the client) rather than the global status map; network-level errors say nothing about whether the server received the request. (Relocated from inline: its anchor line overlaps an existing comment thread at mcp-tool.ts:391, which is a different — already addressed — finding.)

— qwen3.8-max via Qwen Code /review (v0.21.5)

Comment thread packages/core/src/tools/mcp-tool.ts Outdated
signal: AbortSignal,
updateOutput?: (output: ToolResultDisplay) => void,
): Promise<ToolResult> {
this.statusAtCallStart = getMCPServerStatus(this.serverName);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] No test pins the call-start snapshot timing — a gate-time live status read survives the entire 93-test suite (probe-verified: the mutant const neverDelivered = getMCPServerStatus(this.serverName) === MCPServerStatus.DISCONNECTED evaluated in handleReconnectOnError was applied and the suite run — 93/93 still pass). No failing mock in this file flips the server status on failure the way the real client's onerror wiring does (mcp-client.ts marks the server DISCONNECTED when the transport dies), so a future refactor collapsing statusAtCallStart into a gate-time read would ship green with the safety gate neutered for its core scenario: a call issued CONNECTED that dies mid-flight — exactly the case #8387's gate was built for. — Concrete cost: the snapshot's "before the attempt" invariant (which the adjacent comment calls load-bearing) is protected only by code review; the suite cannot see it. Add a test that discriminates: status CONNECTED before execute(); the failing callTool mock flips status to DISCONNECTED before rejecting; unannotated tool; expect unsafeReplayErrorMessage. Verified: the proposed test passes on shipped code and fails on the mutant (which replays the may-have-completed call instead of gating it).

const deadClient: McpDirectClient = {
  callTool: vi.fn().mockImplementation(async () => {
    // The real client marks the server DISCONNECTED when the transport
    // dies mid-flight; the snapshot at call start must decide, not this.
    updateMCPServerStatus(serverName, MCPServerStatus.DISCONNECTED);
    throw new Error('Connection closed');
  }),
};
// untrusted/unannotated tool; updateMCPServerStatus(serverName, CONNECTED) before execute()
await expect(
  reconnectTool.build(params).execute(new AbortController().signal),
).rejects.toThrow(unsafeReplayErrorMessage);
中文说明

[建议] 没有测试钉住「调用发起前快照」的时机——在闸门处实时读取状态的变体实现可以通过全部 93 个测试(已用探针验证:把 mutant const neverDelivered = getMCPServerStatus(this.serverName) === MCPServerStatus.DISCONNECTED 挪到 handleReconnectOnError 里实时求值后跑整套测试,仍然 93/93 通过)。本文件中没有任何失败 mock 会在失败时翻转服务器状态,而真实客户端的 onerror 接线会(transport 死掉时 mcp-client.ts 会把状态写成 DISCONNECTED)。因此未来若有重构把 statusAtCallStart 坍缩为闸门处实时读取,测试会全绿合入,而安全闸门在其核心场景(调用发起时 CONNECTED、中途死掉——正是 #8387 闸门要防的情形)里已形同虚设。——具体代价:快照「先于尝试」的不变量(旁边注释称其为关键选择)只靠代码评审保护,测试套件无法察觉。建议补一个能区分的测试:execute() 前状态为 CONNECTED;失败的 callTool mock 在 reject 前把状态翻成 DISCONNECTED;无注解工具;断言抛出 unsafeReplayErrorMessage。已验证:该测试在现有代码上通过,在上述 mutant 上失败(mutant 会重放可能已执行的调用而不是拦截它)。

— qwen3.8-max via Qwen Code /review (v0.21.5)

@wenshao

wenshao commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator Author

Review — fix(core): a never-delivered MCP call is a first delivery, not a replay

Overview. Adds a carve-out to #8387's replay-safety gate: DiscoveredMCPToolInvocation.execute() snapshots getMCPServerStatus(serverName) before issuing its call, and when that snapshot is DISCONNECTED both gate checks (pre-reconnect and post-reconnect) are skipped, on the premise that such a call never reached the server. Plus two tests and one debug log line. 139/-2 over 2 files. The reasoning is sound in the common case and the retry hop correctly re-snapshots, so a call that dies mid-flight on the recovered connection still faces the gate.

What I verified locally (worktree at 967f4741, packages/core/src/tools/mcp-tool.test.ts)

Run Result
PR head, full file 93/93 pass
PR head with mcp-tool.ts reverted to main (mutation) 2 failed — exactly the two new tests
main (d6f55a1c), unmodified file 91/91 pass
eslint --max-warnings 0 on both changed files clean

So the fix is necessary for its own tests, and both new tests are load-bearing (the second one fails on base because the gate short-circuits before recoveredClient is ever called).


1. The stated motivation is stale — main is already green

The description says the reconnect test "has been red on main since #8387" and is blocking #8468/#8471. That was true, but #8478 (e68f617b, "align MCP reconnect timeout test with safe replay policy") already landed on main and fixed it by giving the fixture server trust + idempotentAnnotations. I ran the file at d6f55a1c (current main): 91/91 green.

That changes what this PR is. It is no longer a CI unblock; it is a deliberate loosening of #8387's policy, and it should be reviewed and described as one. Please rewrite the summary accordingly — as written, a reviewer is being asked to approve a behavior change under the framing of an urgent red-CI fix. (It also means there's no rush, which is good, because of #2.)

2. Correctness: the status snapshot is a TOCTOU proxy for "never delivered"

McpClient.disconnect() writes the global DISCONNECTED before it closes the transport:

// packages/core/src/tools/mcp-client.ts:628-635
this.status = MCPServerStatus.DISCONNECTED;
updateMCPServerStatus(this.serverName, MCPServerStatus.DISCONNECTED);
this.isDisconnecting = true;
if (this.transport) {
  await this.transport.close();   // <-- transport is still live during this await
}

McpPoolEntry.forceShutdown() / doRestart() reach the same code via sweepAndDisconnect(), and the pid sweep + SIGTERM runs inside that window too. So there is a real interval in which the global map says DISCONNECTED while the SDK client still has a transport — a concurrent tool call issued in that window snapshots DISCONNECTED, is actually written to the server, may execute its side effect, then fails when the transport dies, and this PR replays it. That is precisely the double-execution #8387 exists to prevent. Idle-reaper sweeps, /mcp disable and pool restarts all run concurrently with tool calls, so the window is reachable, not theoretical.

Suggested fix — use a positive signal instead of an inferred one. The MCP SDK gives you exactly that: Protocol.request() (what client.callTool goes through) rejects before writing anything when the transport is gone:

// @modelcontextprotocol/sdk/dist/esm/shared/protocol.js:618-621
if (!this._transport) {
  earlyReject(new Error('Not connected'));
  return;
}

Not connected is proof the call never left the process; Connection closed / ECONNRESET are not. Keying the carve-out on the error identity (or on error identity and the status snapshot) closes the race by construction and makes the invariant self-evident rather than dependent on teardown ordering in two other files. If you keep the status-based version, please at least add a comment in mcp-client.ts:628 warning that the write ordering there is now load-bearing for replay safety.

3. The tests assert the carve-out with the error that doesn't prove it

Both new tests give the dead client new Error('Connection closed') — the ambiguous case. A genuinely never-delivered call throws Not connected. As written, the tests document the implementation (status-only) rather than the property the PR title claims ("never delivered"), and they'd pass unchanged even if the call had been delivered. Switching deadClient to reject with Not connected would make the fixture faithful; if you adopt #2 it becomes the actual mechanism.

4. Minor

  • Unknown server also reads as DISCONNECTED. getMCPServerStatus is serverStatuses.get(name) || DISCONNECTED (mcp-status.ts:92), so the carve-out fires for "no status entry at all", not just "known disconnected" — e.g. after removeMCPServerStatus() on a /mcp-disabled server. Tool removal is supposed to be paired with those calls so it's hard to hit today, but the doc comment on statusAtCallStart says "already known DISCONNECTED", which is stronger than what the code checks. Either tighten the check or soften the comment.
  • statusAtCallStart is a mutable field written inside execute(). Nothing else in this class carries per-execution state, and a re-entrant/reused invocation would clobber it. Threading the snapshot as a parameter (executeexecuteWith*ClienthandleReconnectOnError(error, statusAtCallStart, …)) keeps its lifetime obvious and makes the field unnecessary.
  • Test 1 can't distinguish the two carve-out sites. Its discoverToolsForServer is mockResolvedValue(undefined), so it never restores CONNECTED — the post-reconnect invocation re-snapshots DISCONNECTED and would be exempt on its own merits. Have it flip to CONNECTED (like test 2 does) and the test then genuinely pins the outer neverDelivered propagation at line 426.
  • The debugLogger.info on bypass is good; consider also logging when the gate fires, so the two branches are symmetric in a support log.

5. Worth deciding explicitly

After an UNSAFE_REPLAY error the model is told "verify the outcome before trying again". With this carve-out, the model's next attempt at the same tool now silently succeeds via reconnect — correct per the "first delivery" argument, but it means the guard degrades to a one-turn speed bump for unannotated tools. That's a reasonable product call; it just isn't stated anywhere in the PR or the code comments, and it's the thing a future reader will need.


Verdict: the direction is right and the implementation is clean, but I'd hold on merging until (a) the description is re-framed now that #8478 made main green, and (b) #2 is addressed or explicitly accepted — the status snapshot narrows #8387's guarantee from "always" to "except during teardown races", and the SDK already hands you a race-free signal.

中文小结

结论:方向正确,实现干净,但建议先处理两点再合入。

本地验证(worktree 967f4741):PR head 93/93 通过;把 mcp-tool.ts 回退到 main 后恰好挂掉新增的 2 个测试(变异测试成立);eslint --max-warnings 0 无告警。

  1. 动机已过期。 fix(core): align MCP reconnect timeout test with safe replay policy #8478(e68f617b)已经先落地并修好了那个红灯测试——我在 main(d6f55a1c)上实跑该文件是 91/91 全绿。所以本 PR 不再是"解 CI 阻塞",而是fix(core): Avoid replaying unsafe MCP tool calls #8387 策略的一次主动放宽,描述需要按这个定位重写。
  2. 状态快照只是"未投递"的近似,存在 TOCTOU。 McpClient.disconnect()await this.transport.close() 之前就写了全局 DISCONNECTED(mcp-client.ts:628-635),forceShutdown / doRestartsweepAndDisconnect 走的是同一条路。在这个窗口里发出的调用会快照到 DISCONNECTED实际写到了服务端、副作用可能已执行,然后随 transport 一起失败并被本 PR 重放——正是 fix(core): Avoid replaying unsafe MCP tool calls #8387 要防的重复执行。空闲回收、/mcp 禁用、连接池重启都会与工具调用并发,窗口是可达的。
    建议改用正向信号:MCP SDK 的 Protocol.request() 在 transport 缺失时在写出任何字节之前earlyReject(new Error('Not connected'))(protocol.js:618-621)。Not connected 才是"从未投递"的证据,Connection closed 不是。按错误身份判定可以从构造上消除竞态。
  3. 测试用的错误恰好是不能证明命题的那个。 两个新测试的 deadClient 抛的是 Connection closed(歧义态);真正未投递抛的是 Not connected。改成后者 fixture 才忠实于 PR 标题的命题。
  4. 细节:getMCPServerStatus 对未知 server 也返回 DISCONNECTED(mcp-status.ts:92),豁免范围比注释里写的"已知 DISCONNECTED"更宽;statusAtCallStart 作为在 execute() 里写的可变字段不如按参数下传;测试 1 的 discoverToolsForServer 没有恢复 CONNECTED,因此区分不出外层 neverDelivered 传递与内层重新快照。
  5. 需要明确表态:豁免之后,模型在收到 UNSAFE_REPLAY 后的下一次同工具调用会静默走通——按"首次投递"论是自洽的,但这等于该防护对无注解工具降级为"只挡一轮"。这个取舍本身可以接受,只是 PR 与代码注释里都没写。

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤖 Addressed the latest review feedback (round 2/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 2/100 轮)。改动内容与我反驳保留之处如下:

Autofix review round — PR #8482

All findings addressed in commit 465f163ca. No conflicts (--conflict false, no merge performed).

Core redesign: positive delivery evidence instead of a status snapshot

Both the automated reviewer's Critical R1-2 and @wenshao's #2 identified the same defect: the neverDelivered carve-out treated a DISCONNECTED status snapshot as proof a call never reached the server. It isn't — McpClient.disconnect() writes DISCONNECTED before await this.transport.close() (a teardown window in which calls are still delivered), and network transports report transient errors via onerror without closing. getMCPServerStatus() also falls back to DISCONNECTED for unregistered servers — the exact comparison isExecutionTimeoutFailure() in the same file documents as unsafe.

Fix: the carve-out now keys on the MCP SDK's own delivery-level evidence. In the pinned SDK (1.30.0), Protocol.request() rejects with exactly Error('Not connected') before writing the request when the transport is already gone (shared/protocol.js:618-621, verified in node_modules). A call in flight when the transport dies rejects with McpError('Connection closed') instead, and network failures (ECONNRESET, proxy 5xx, SSE drops) surface as their own errors — all ambiguous, all still gated. Verified the production mcpClient passed to DiscoveredMCPTool is the raw SDK Client (mcp-client.ts:1400), so callTool goes through exactly that path. This closes the race by construction; the status write ordering in mcp-client.ts is no longer load-bearing for replay safety, so no warning comment is needed there.

Changes in packages/core/src/tools/mcp-tool.ts:

  • New isNeverDeliveredError() helper (exact match on the SDK's pre-send rejection marker; fail-closed on anything else, including the genai wrapper fallback path).
  • Removed the statusAtCallStart field and its snapshot in execute() (also resolves @wenshao #4a and #4b — no status read, no mutable per-execution field, no "already known DISCONNECTED" comment that overstated the check).
  • Added symmetric debugLogger.info on both gate-fire sites (@wenshao #4d), next to the existing bypass log.
  • Documented the accepted trade-off above UNSAFE_REPLAY_ERROR_MESSAGE (@wenshao TypeError in Authentication Selection Interface #5): the gate blocks only the automatic same-turn replay; a fresh model-initiated attempt reconnects and delivers as usual, so for unannotated tools the guard is a one-turn speed bump by design.

Tests (packages/core/src/tools/mcp-tool.test.ts, 95 passing)

  • @wenshao 如何自定义密钥文件 .env可能与其他文件冲突 #3: both fixtures now reject with the faithful error — Not connected (provably never delivered) instead of Connection closed (the ambiguous case the old tests used).
  • @wenshao #4c: test 1's discoverToolsForServer now restores CONNECTED, matching a real reconnect.
  • Test 2 restructured to its stated property: the first attempt passes the gate on pre-send-rejection evidence, then the retried call dies mid-flight with an ambiguous error and the gate re-applies.
  • R1-1 (inline suggestion): added still gates a failure that flips the server status mid-flight — status CONNECTED before execute(), the failing callTool mock flips it to DISCONNECTED before rejecting (as the real onerror wiring does), unannotated tool, expects unsafeReplayErrorMessage.
  • R1-2 probe scenario: added still gates an ambiguous failure issued while the status was DISCONNECTED — a delivered-then-failed call under a DISCONNECTED snapshot must not replay (the double-execution the reviewer probe-verified).
  • Mutation check: temporarily replacing isNeverDeliveredError(error) with a gate-time status read made exactly the two new gating tests fail (2 failed | 93 passed); reverting restores 95/95. The suite now discriminates the status-based design in both its snapshot and live-read forms.

@wenshao #1 — stale PR description

Agreed: #8478 landed and made main green, so this PR is a deliberate loosening of #8387's policy, not a CI unblock. This flow cannot edit the PR body (no GitHub write access), so the description needs a maintainer update. Suggested framing:

This PR deliberately relaxes #8387's replay-safety gate (it is not a CI unblock — #8478 already fixed the red reconnect test on main). It allows automatic reconnect-and-retry only for calls proven never delivered — the MCP SDK's pre-send Not connected rejection — while every ambiguous failure (mid-flight disconnects, network errors, teardown races) still faces the gate.

Verification

  • npx vitest run src/tools/mcp-tool.test.ts (from packages/core) — 95/95 passed (93 pre-change; 2 restructured + 2 new)
  • Mutation check (gate-time status read applied then reverted) — mutant killed: 2 failed | 93 passed; reverted state 95/95
  • npx vitest run on all 10 MCP-related files + tool-registry.test.ts515/515 passed
  • npm run buildpassed
  • npm run typecheckpassed
  • npm run lintpassed
  • npx prettier --check on the two changed files — passed
中文说明

Autofix 评审轮次 — PR #8482

所有发现已在提交 465f163ca 中处理。无冲突(--conflict false,未做任何合并)。

核心重设计:用投递层面的正面证据取代状态快照

自动评审的 Critical R1-2@wenshao#2 指出了同一个缺陷:neverDelivered 豁免把 DISCONNECTED 状态快照当作"调用从未到达服务端"的证据。事实并非如此——McpClient.disconnect()await this.transport.close() 之前就写入了 DISCONNECTED(在这个拆除窗口内调用仍会被投递),且网络 transport 在遇到瞬时错误时只触发 onerror 而不关闭。getMCPServerStatus() 对未注册的 server 还会兜底返回 DISCONNECTED——这正是同文件里 isExecutionTimeoutFailure() 注释中明确记载为不安全、并刻意避免的比较方式。

修复: 豁免现在以 MCP SDK 自身的投递层面证据为准。在锁定的 SDK(1.30.0)中,当 transport 已不存在时,Protocol.request()在写出任何请求之前以恰好 Error('Not connected') 拒绝(shared/protocol.js:618-621,已在 node_modules 中核实)。而在途调用遇到 transport 关闭时收到的是 McpError('Connection closed'),网络错误(ECONNRESET、代理 5xx、SSE 掉线)则表现为各自的错误——这些都是歧义态,全部仍然被闸门拦截。已核实生产路径中传给 DiscoveredMCPToolmcpClient 就是原生 SDK Clientmcp-client.ts:1400),因此 callTool 恰好经过该路径。这在构造上消除了竞态;mcp-client.ts 中的状态写入顺序不再对重放安全构成关键依赖,因此也无需在那里添加警示注释。

packages/core/src/tools/mcp-tool.ts 的改动:

  • 新增 isNeverDeliveredError() 辅助函数(精确匹配 SDK 的"发送前拒绝"标记;其他任何错误一律失败关闭,包括 genai wrapper 回退路径)。
  • 移除 statusAtCallStart 字段及 execute() 中的快照(同时解决 @wenshao #4a 与 #4b——不再读状态、不再有可变的逐执行字段、也不再有那句强于实际检查的"已知 DISCONNECTED"注释)。
  • 在两处闸门触发点补充对称的 debugLogger.info@wenshao #4d),与已有的豁免日志对称。
  • UNSAFE_REPLAY_ERROR_MESSAGE 上方记录了已接受的取舍(@wenshao TypeError in Authentication Selection Interface #5):闸门只拦截同一轮次的自动重放;模型重新发起的调用会走正常重连路径并照常投递,因此对无注解工具而言该防护按设计就是"只挡一轮"的减速带。

测试(packages/core/src/tools/mcp-tool.test.ts,95 个通过)

  • @wenshao 如何自定义密钥文件 .env可能与其他文件冲突 #3 两个 fixture 现在抛出忠实的错误——Not connected(可证明从未投递),而不是旧测试所用的 Connection closed(歧义态)。
  • @wenshao #4c: 测试 1 的 discoverToolsForServer 现在会恢复 CONNECTED,与真实重连一致。
  • 测试 2 重构为其标题所述的性质:首次尝试凭"发送前拒绝"证据通过闸门,随后重试调用在途中以歧义错误死掉,闸门重新生效。
  • R1-1(内联建议): 新增 still gates a failure that flips the server status mid-flight——execute() 前状态为 CONNECTED,失败的 callTool mock 在 reject 前把状态翻成 DISCONNECTED(与真实 onerror 接线一致),无注解工具,断言抛出 unsafeReplayErrorMessage
  • R1-2 探针场景: 新增 still gates an ambiguous failure issued while the status was DISCONNECTED——在 DISCONNECTED 快照下"已投递后失败"的调用不得被重放(即评审者用探针验证出的重复执行)。
  • 变异测试:isNeverDeliveredError(error) 临时替换为闸门处实时读取状态后,恰好是新增的两个闸门测试失败(2 failed | 93 passed);还原后恢复 95/95。测试套件现在能够区分状态式设计的快照与实时读取两种形态。

@wenshao #1 — 过期的 PR 描述

同意:#8478 已落地且 main 已绿,所以本 PR 是#8387 策略的一次主动放宽,而不是解 CI 阻塞。本流程无法编辑 PR 正文(没有 GitHub 写权限),需要维护者更新描述。建议措辞:

本 PR 主动放宽 #8387 的重放安全闸门(并非解 CI 阻塞——#8478 已修复 main 上的红灯重连测试)。它仅允许对可证明从未投递的调用(MCP SDK 发送前的 Not connected 拒绝)自动重连重试;其余一切歧义失败(途中断连、网络错误、拆除竞态)仍然面对闸门。

验证

  • npx vitest run src/tools/mcp-tool.test.ts(于 packages/core)——95/95 通过(改动前 93 个;2 个重构 + 2 个新增)
  • 变异测试(施加闸门处实时读状态的变异后还原)——变异体被杀死:2 failed | 93 passed;还原后 95/95
  • npx vitest run 全部 10 个 MCP 相关文件 + tool-registry.test.ts——515/515 通过
  • npm run build ——通过
  • npm run typecheck ——通过
  • npm run lint ——通过
  • npx prettier --check 两个改动文件——通过

Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。

Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。


🧠 Handled by Qwen Code · model/模型 qwen3.8-max

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🔀 Base updated: red check(s) [Test (ubuntu-latest, Node 22.x)] pass on current main — merged current main via update-branch; CI will re-run.

中文说明

🔀 已更新 base:红色检查 [Test (ubuntu-latest, Node 22.x)] 在当前 main 上通过 —— 已通过 update-branch 合入当前 main,CI 将重新运行。

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Qwen Code review timed out. Qwen review timed out after 10800 seconds (of the 180-minute budget). For large PRs, retry with a longer timeout by commenting: @qwen-code /review --timeout=360. See workflow logs.

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🔀 Base updated: red check(s) [review-pr] pass on current main — merged current main via update-branch; CI will re-run.

中文说明

🔀 已更新 base:红色检查 [review-pr] 在当前 main 上通过 —— 已通过 update-branch 合入当前 main,CI 将重新运行。

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下:

Autofix review round — no action taken.

The only new feedback since the last evaluation is an automated notification that the Qwen Code review run timed out after 10800 seconds of its 180-minute budget. This is a review-harness event, not a code-level finding:

  • No new reviews or inline comments exist. All prior findings (verified against the raw review data) predate the last evaluation cutoff and were already addressed in earlier rounds — the current code keys the replay gate on the SDK's pre-send Not connected rejection and includes the requested diagnostic logging.
  • No failed CI checks. All checks on the current head are green (or skipped); the only non-green entry is the review workflow itself, which is already re-running. The CI test suite completed in ~15 minutes, so there is no evidence the PR's content is stalling the pipeline — the PR itself is small (2 files, ~284 lines).
  • The suggested remedy — retrying the review with a longer timeout by commenting @qwen-code /review --timeout=360 — is a GitHub comment action. The autofix agent has no GitHub write access; the workflow and maintainers own that decision.

No code changes were made this round and no commit was created.

中文说明

Autofix 审查轮次 — 未采取任何操作。

自上次评估以来唯一的新反馈是一条自动通知:Qwen Code 审查运行在 180 分钟预算中超时(10800 秒后终止)。这是审查流程(review harness)层面的事件,而非针对代码的发现:

  • 不存在新的 review 或行内评论。经与原始审查数据核对,所有先前的发现均早于上次评估的截止时间,且已在更早的轮次中处理完毕 —— 当前代码已将重放安全门(replay gate)改为基于 SDK 在发送前抛出的 Not connected 拒绝,并已包含此前要求添加的诊断日志。
  • 没有失败的 CI 检查。当前 head 上的所有检查均为绿色(或被跳过);唯一非绿的条目是审查工作流本身,而它已经在重新运行。CI 测试套件约 15 分钟即完成,因此没有证据表明本 PR 的内容拖慢了流水线 —— 本 PR 本身很小(2 个文件,约 284 行)。
  • 建议的补救方式 —— 通过评论 @qwen-code /review --timeout=360 以更长的超时时间重试审查 —— 是一个 GitHub 评论操作。Autofix 代理没有 GitHub 写入权限;该决定由工作流和维护者负责。

本轮未做任何代码更改,也未创建任何提交。

Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。


🧠 Handled by Qwen Code · model/模型 qwen3.8-max

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. Suggestions are inline. Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.

中文说明

已审查。 建议见行内评论。 未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。

— qwen3.8-max via Qwen Code /review (v0.21.5)

Comment on lines +116 to +118
function isNeverDeliveredError(error: unknown): boolean {
return getErrorMessage(error) === 'Not connected';
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] The never-delivered carve-out keys on an unversioned MCP SDK implementation-detail string, and all four new tests mock callTool with hand-rolled errors instead of exercising the real SDK, so SDK drift cannot fail CI.

Failure scenario: The dependency range is ^1.30.0; a routine minor bump that rewords Protocol.request()'s pre-send rejection — or wraps it with a cause, which getErrorMessage would suffix as (cause: ...) — makes isNeverDeliveredError return false forever after. All four new tests reject with their own new Error('Not connected'), so they keep passing; in production, never-delivered calls on unannotated tools silently revert to UNSAFE_REPLAY refusals — the exact failure this PR removes — with no red signal anywhere. (The pinned 1.30.0 behavior itself was probe-verified correct: a real SDK Client connected-then-closed yields exactly the bare Error('Not connected') pre-send rejection, and a delivered call can never surface that string — the McpError prefix MCP error ${code}: defeats the exact match.)

Suggested fix: add one test that exercises the real @modelcontextprotocol/sdk Client (never connected, or connected-then-closed): assert its genuine pre-send rejection is classified as never-delivered, and symmetrically that an in-flight transport close (McpError 'Connection closed') is not — pinning the contract at the real dependency so a rewording bump surfaces as a red test instead of a silent production regression.

中文说明

[建议] 未投递豁免依赖 MCP SDK 未固定版本的实现细节字符串,且四个新测试全部用手工 mock 的 callTool 错误代替真实 SDK,SDK 漂移无法让 CI 变红。

失败场景:依赖范围为 ^1.30.0;一次常规 minor 升级若改写 Protocol.request() 的发送前拒绝措辞——或给它附加 causegetErrorMessage 会追加 (cause: ...) 后缀)——isNeverDeliveredError 将永远返回 false。四个新测试都用自带的 new Error('Not connected') 拒绝,因此仍然全绿;生产环境中,无注解工具的未投递调用会静默退回 UNSAFE_REPLAY 拒绝——正是本 PR 要消除的故障——且没有任何红色信号。(已用探针验证 1.30.0 本身行为正确:真实 SDK Client 连接后关闭,产生的正是裸 Error('Not connected') 发送前拒绝;已投递的调用不可能浮出该字符串——McpError 前缀 MCP error ${code}: 会使精确匹配失败。)

建议修复:补一个使用真实 @modelcontextprotocol/sdk Client 的测试(从未连接或连接后关闭):断言其真实的发送前拒绝被判定为未投递,对称地断言传输中途关闭(McpError 'Connection closed')不被判定为未投递——把契约钉在真实依赖上,使措辞变更的升级以红灯测试显形,而非静默的生产回归。

— qwen3.8-max via Qwen Code /review (v0.21.5)

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤖 Addressed the latest review feedback (round 3/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 3/100 轮)。改动内容与我反驳保留之处如下:

Address review summary — PR #8482 (same-run verification repair)

This round is a same-run verification repair: the previous commit 47e024621 was REJECTED by the deterministic verification gate because the clean build failed with src/tools/mcp-tool.test.ts(2768,24): error TS2339: Property 'connect' does not exist on type 'McpDirectClient'. Per the repair instruction, that commit is preserved and one verified follow-up commit 3c2596b62 fixes the rejection.

Feedback points

[Suggestion] rc:3715798544 — never-delivered carve-out keyed on an unversioned SDK string; all tests mock callToolImplemented (re-verified after build repair)

The finding itself was implemented in the previous commit 47e024621 (two tests driving the real @modelcontextprotocol/sdk Client through the gate). This round repaired the compile error that commit introduced, so the implementation now survives the verification gate.

Root cause of the gate rejection: the second new test annotated the real SDK client as const realClient: McpDirectClient = new Client({...}) and then called realClient.connect(clientTransport). McpDirectClient is the minimal structural interface (only callTool) defined in mcp-tool.ts to keep the SDK import out of that file, so the annotation erased connect from the visible type and tsc --build failed. The annotation was also unnecessary: the DiscoveredMCPTool constructor parameter is typed McpDirectClient, so passing the inferred Client still type-checks via structural compatibility — exactly as production mcp-client.ts passes the real SDK client.

Fix (one line): remove the McpDirectClient annotation from realClient so its inferred type is the SDK Client, which exposes connect. Type annotations are erased at runtime, so the two pin tests behave exactly as described in the previous round's summary; as a side benefit the constructor argument now additionally pins, at compile time, that the real SDK Client still satisfies the McpDirectClient contract.

Review-body note ("Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally"): informational, not a finding. This round's change is a one-line type-annotation repair in a unit-test file inside packages/core; the behavior is exercised through focused Vitest, not the bundled CLI or integration harness, so no integration run is required by the verification policy.

Conflict

None (--conflict false); no merge performed.

Verification

Reproduction first: npx tsc --build in packages/core reproduced the exact gate error before the fix — src/tools/mcp-tool.test.ts(2768,24): error TS2339: Property 'connect' does not exist on type 'McpDirectClient'. After the fix:

  • npx tsc --build in packages/core (after removing dist/ and tsconfig.tsbuildinfo to match the gate's clean build) — passed
  • npm run build — passed
  • npm run typecheck — passed
  • npm run lint — passed
  • npx vitest run src/tools/mcp-tool.test.ts (in packages/core) — 98 passed (96 existing + 2 real-SDK pin tests)
  • npx prettier --check packages/core/src/tools/mcp-tool.test.ts — passed
中文说明

处理审查总结 — PR #8482(同轮验证修复)

本轮是一次同轮验证修复:上一个提交 47e024621 被确定性验证门禁拒绝,原因是干净构建失败,报错为 src/tools/mcp-tool.test.ts(2768,24): error TS2339: Property 'connect' does not exist on type 'McpDirectClient'。按照修复指令,该提交被保留,并以一个已验证的后续提交 3c2596b62 修复了被拒绝的问题。

反馈点

[建议] rc:3715798544 — 未投递豁免依赖未固定版本的 SDK 字符串,且所有测试都 mock 了 callTool已实现(构建修复后复核通过)

该发现项本身已在上一提交 47e024621 中实现(两个用真实 @modelcontextprotocol/sdk Client 驱动判定门的测试)。本轮修复了该提交引入的编译错误,使实现能够通过验证门禁。

门禁拒绝的根因:第二个新测试将真实 SDK 客户端标注为 const realClient: McpDirectClient = new Client({...}),随后调用了 realClient.connect(clientTransport)McpDirectClientmcp-tool.ts 中定义的最小结构化接口(只有 callTool),目的是让该文件不直接引入 SDK,因此这个类型标注把 connect 从可见类型上抹掉了,导致 tsc --build 失败。该标注本来也是多余的:DiscoveredMCPTool 构造函数的参数类型就是 McpDirectClient,传入推断出的 Client 类型会通过结构化兼容性完成类型检查——与生产代码 mcp-client.ts 传入真实 SDK 客户端的方式完全一致。

修复(一行):移除 realClient 上的 McpDirectClient 标注,使其推断类型为 SDK 的 Client,从而暴露 connect。类型标注在运行时会被擦除,因此两个钉扎测试的行为与上一轮总结中描述的完全一致;附带的好处是,构造函数实参现在额外在编译期钉住了真实 SDK Client 仍然满足 McpDirectClient 契约这一点。

审查正文备注("Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally"):仅为信息说明,不是发现项。本轮改动是 packages/core 内单元测试文件中的一行类型标注修复;该行为通过聚焦 Vitest 验证,而非打包后的 CLI 或集成测试框架,按验证策略无需运行集成测试。

冲突

无(--conflict false);未执行合并。

验证

先复现:修复前在 packages/core 中运行 npx tsc --build 复现了与门禁完全相同的错误 —— src/tools/mcp-tool.test.ts(2768,24): error TS2339: Property 'connect' does not exist on type 'McpDirectClient'。修复后:

  • npx tsc --build(在 packages/core 中,先删除 dist/tsconfig.tsbuildinfo 以匹配门禁的干净构建)— 通过
  • npm run build — 通过
  • npm run typecheck — 通过
  • npm run lint — 通过
  • npx vitest run src/tools/mcp-tool.test.ts(在 packages/core 中)— 98 通过(96 个既有 + 2 个真实 SDK 钉扎测试)
  • npx prettier --check packages/core/src/tools/mcp-tool.test.ts — 通过

Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。

Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。


🧠 Handled by Qwen Code · model/模型 qwen3.8-max

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🔀 Base updated: red check(s) [review-pr] pass on current main — merged current main via update-branch; CI will re-run.

中文说明

🔀 已更新 base:红色检查 [review-pr] 在当前 main 上通过 —— 已通过 update-branch 合入当前 main,CI 将重新运行。

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. Suggestions are inline. Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.

中文说明

已审查。 建议见行内评论。 未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。

— qwen3.8-max via Qwen Code /review (v0.21.5)

Comment on lines +116 to +118
function isNeverDeliveredError(error: unknown): boolean {
return getErrorMessage(error) === 'Not connected';
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R3-1: No test pins the exact-match boundary of isNeverDeliveredError — Failure scenario: a future "tolerate SDK wording variations" edit changes === to .includes(); the whole 98-test suite stays green (mutant verified by run), and a delivered call whose server-side error message merely contains Not connected (e.g. a tool handler throwing new Error('Not connected to database')) is then classified as never-delivered and auto-replayed — the exact double-execution hazard the gate exists to prevent. The current exact-match code is safe (the McpError constructor's MCP error <code>: prefix keeps every delivered rejection from matching today); this test pins the boundary against a future loosening. Add a negative-case test in the auto-reconnect describe block, e.g.:

it('still gates a rejection whose message only contains the pre-send wording', async () => {
  const deadClient: McpDirectClient = {
    callTool: vi
      .fn()
      .mockRejectedValueOnce(new Error('Not connected to database')),
  };
  // ...unannotated tool, DISCONNECTED or CONNECTED status alike...
  await expect(
    reconnectTool.build(params).execute(new AbortController().signal),
  ).rejects.toThrow(unsafeReplayErrorMessage);
  expect(discoverToolsForServer).not.toHaveBeenCalled();
  expect(liveClient.callTool).not.toHaveBeenCalled();
});
中文说明

[建议] 没有任何测试钉住 isNeverDeliveredError 的精确匹配边界 —— 失败场景:未来某次「兼容 SDK 措辞变化」的修改把 === 改成 .includes(),全部 98 个测试依然全绿(变异体已实跑验证),此后一个已投递、但服务端错误消息仅包含 Not connected 的调用(例如工具 handler 抛出 new Error('Not connected to database'))会被判定为「从未投递」并自动重放——这正是该闸门要防的重复执行风险。当前的精确匹配代码是安全的(McpError 构造器的 MCP error <code>: 前缀使任何已投递的错误响应在今天都不可能命中精确匹配);此测试用于钉住该边界,防止未来放宽。可在 auto-reconnect describe 块中补一个反例测试(形状见上方英文代码示例)。

— qwen3.8-max via Qwen Code /review (v0.21.5)

@wenshao

wenshao commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

@qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run

Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check.

Scripted assertions: 314 passed · 1 failed · 315 total

中文 — 判定:❌ 不通过 · 报告了发现(agent 判定)

沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查

脚本断言:314 通过 · 1 失败 · 315 总计

Verification report

PR 8482 Deep Verification — fix(core): a never-delivered MCP call is a first delivery, not a replay

Verdict: findings — scripted assertions 314 pass / 1 fail / 315 total. Verified head: 95a82fa74b1290091bccc314c74433cecd126637 (merge commit 488222b90, base 32e274157).

The code change is correct, load-bearing, and regression-free — but the PR's stated premise (a deterministically red test blocking main) is stale at this merge base, and the PR body describes a different mechanism than the one that landed. Details below.

中文摘要
  • 结论:findings(断言计数见 verdict 行)。代码本身正确、有效(load-bearing)、无回归,但有两处需要审阅者注意的描述性问题。
  • A/B 结论:central claim 的载荷证明成立(见「Central claim and A/B table」的 Cell C 一行):base 源码 + 本 PR 的新测试跑出 3 个红(其中 2 个精确抛出 UNSAFE_REPLAY),head 全绿;缺失豁免分支时 unannotated 工具的「从未投递」调用会被永久拒绝重试,本 PR 恢复了首次投递路径。变异矩阵(见「Mutation matrix」表)M1/M2/M3/M5 全部被测试杀死,正控制成立。
  • findings:
    1. PR 的核心前提(「某测试自 fix(core): Avoid replaying unsafe MCP tool calls #8387 起在 main 上确定性红灯、阻塞 fix(review): stop the reverse-audit loop while there is still time to report #8468/feat(review): a cost ledger from the records already on disk #8471」)在本 PR 自己的 base(32e274157)上不成立:该测试在 base 上为绿(92/92,测试体逐字节一致、单跑亦绿)。它在 fix(core): Avoid replaying unsafe MCP tool calls #8387 提交点(0cb109f513)确实是红的(已复现),但其后 main 上是靠给该测试补上 trust + idempotentAnnotations 让它绕过闸门而变绿的,并非修复了行为。因此「合并本 PR 即可解封 Test 任务」的说法对当前 main 已不适用;不过 fix(core): Avoid replaying unsafe MCP tool calls #8387 引入的回归在 base 上对无注解工具仍然存在,本 PR 的修复仍然有实际价值(已用 A/B 证明)。
    2. PR 正文(中英文)描述的机制是「调用前快照 getMCPServerStatus(),快照为 DISCONNECTED 时跳过闸门」;实际落地的代码是「以 SDK 发送前拒绝信息精确等于 'Not connected' 为准」。两者行为不同:正文描述的方案会豁免 DISCONNECTED 状态下的模糊失败,落地代码不豁免(有测试钉住)。建议合并前更新描述,避免把错误机制带进合并记录。
    3. 覆盖率建议(非阻塞):变异体 M4(===.includes())在测试套件中存活 —— 套件没有任何用例喂入服务端产生的 'Not connected' 错误。实测真 SDK 线上形态为 'MCP error -32603: Not connected',严格相等匹配正确地不予豁免,.includes() 则会错误豁免已投递的调用(正是 fix(core): Avoid replaying unsafe MCP tool calls #8387 要防的重放)。落地代码正确,缺的是钉住这一轴的 fixture(见 harness/sibling-sweep.mjs 的 H2/H3)。
  • 未覆盖:逐 commit 归因(depth-2,11 个 commit 不可达,只验证了聚合 diff);fix(review): stop the reverse-audit loop while there is still time to report #8468/feat(review): a cost ledger from the records already on disk #8471 的实际 CI 状态(无 token 无法查证);仓库级全量 gate(仅跑了 packages/core);core 套件中 63 个两臂完全一致的环境性历史失败未深究;真实 stdio/SSE 服务器的端到端 kill-reconnect 场景(线形态经真 SDK 复现,非完整生产流)。

Central claim and A/B table

Central claim (as verified, not as worded): the #8387 replay-safety gate refuses every failed call of an unannotated tool — including calls the SDK rejected before sending (new Error('Not connected'), Protocol.request() at @modelcontextprotocol/sdk@1.30.0 dist/esm/shared/protocol.js:620, transport already gone). Such a call provably never reached the server; the PR exempts exactly this shape from the gate and re-applies the gate in full on the retried hop.

Secondary claims: (a) all #8387 refusal cases remain gated; (b) the gate re-applies to the retried call's own failure evidence.

cell source test file oracle result
head own HEAD (PR) head (98 tests) vitest 98/98 green01-ab-head-mcp-suite-98-of-98.png
base own HEAD^1 = 32e274157 base (92 tests) vitest 92/92 green — incl. the PR's claimed-red test — 05-base-own-tests-green-92-of-92.png
C: base source + head tests HEAD^1 head (98 tests) vitest 3 red / 95 green: 2× exact UNSAFE_REPLAY (never-delivered delivery tests) + 1× "no reconnect attempted" — 02-ab-base-source-head-tests-3-red.png
premise probe 0cb109f513 (#8387 commit) the named test only vitest red with UNSAFE_REPLAY — the PR's historical claim reproduces at that commit04-premise-red-at-8387-commit.png
M5: head with mcp-tool.ts reverted to base mutant head (98 tests) vitest same 3 red — vacuity check: the new tests pin the fix

The load-bearing proof is cell C: the two delivery tests flip broken→fixed with the carve-out, failing on base with the exact UNSAFE_REPLAY_ERROR_MESSAGE — the permanent-error behavior the PR describes. The PR's own wording of the premise ("red on main since #8387") does not hold at this base — see Corrections.

Corrections (to the PR description, not code-change requests)

  1. The red-test premise is stale at the merge base. The named test (reconnects instead of reporting a timeout when the server is known disconnected) is green at HEAD^1 — 92/92 in the full file, and green run in isolation; its body is byte-identical between base and head (sha256 bcdee310…7431 both sides). It genuinely was red at 0cb109f513 (reproduced: UNSAFE_REPLAY from handleReconnectOnError), but between that commit and this base the test was edited to bypass the gatetrust: true and idempotentAnnotations were added to both its tool constructions (+46/−2 test-file diff in that window), making canSafelyReplay() return true. Nothing in mcp-tool.ts changed in that window except an unrelated guard addition. So: the behavior regression fix(core): Avoid replaying unsafe MCP tool calls #8387 introduced survived to this base only for unannotated tools (proven by cell C), while the test that once showed it was neutralized, and "every full-profile PR behind it fails the required Test job on this one test / fix(review): stop the reverse-audit loop while there is still time to report #8468 and feat(review): a cost ledger from the records already on disk #8471 are both currently blocked by it" is not reproducible from this tree.
  2. The body describes a mechanism that did not land. Both the English and 中文 sections say the invocation "snapshots getMCPServerStatus(serverName) before issuing its call" and the gate "is skipped only when that snapshot says DISCONNECTED". The landed code keys on getErrorMessage(error) === 'Not connected' (the SDK's pre-send rejection) — the approach was changed mid-PR (commit 465f163c, "key the MCP never-delivered carve-out on the SDK pre-send rejection") without a body update. The two designs differ behaviorally: under the described design an ambiguous failure (Connection closed) issued while the status was DISCONNECTED would have been exempted; the landed code keeps it gated, pinned by the new test still gates an ambiguous failure issued while the status was DISCONNECTED. The landed mechanism is the more conservative one. The body (and the "91/91" count — it is now 98/98) should be refreshed before merge so the merge record does not carry a wrong mechanism.
  3. One accuracy note in the PR's favor: the new doc-comment added on the UNSAFE_REPLAY_ERROR_MESSAGE static ("a deliberate retry is the model's own decision…") matches the landed behavior.

Findings

F1 — premise/description: the blocking red test does not exist at this base (see Correction 1)

Severity: reviewer-attention, not a code defect. The code fix remains justified on its own merits (cell C), but the merge rationale as written — "unblock the required Test job" — no longer matches the state of main at 32e274157: nothing in mcp-tool.test.ts is red there. Reproduce:

git worktree add /tmp/base 32e27415779226b23174a3b0aa6c04e094f1aca2  # wire node_modules
cd /tmp/base/packages/core && npx vitest run src/tools/mcp-tool.test.ts   # 92/92 green

F2 — description: body mechanism ≠ landed mechanism (see Correction 2)

Severity: reviewer-attention. Suggested action: update the PR body; no code change.

F3 — coverage gap: the suite cannot tell the strict matcher from a substring matcher (M4 survivor)

Severity: Suggestion. Mutation getErrorMessage(error) === 'Not connected'.includes('Not connected') leaves all 98 tests green, yet the sweep harness proves the two matchers differ on real wire shapes: a server handler that throws Error('Not connected') after receiving the call rejects the client with 'MCP error -32603: Not connected' (and an McpError throw double-wraps to 'MCP error -32603: MCP error -32603: Not connected') — the strict matcher correctly keeps these delivered calls gated; the includes-form would exempt them, i.e. auto-replay a call the server already saw, which is exactly the hazard #8387 exists to prevent. The landed code is correct; the fixture that would pin it (server-side 'Not connected' error after delivery → expect UNSAFE_REPLAY) is absent. Harness: harness/sibling-sweep.mjs (16/16), capture 03-sibling-sweep-real-sdk-wire-shapes.png. The assertions.json fail: 1 is this survivor, encoded as "the suite kills the includes-mutant" → false.

Mutation matrix (vacuity + load-bearing attribution)

All runs against head's test file (98 tests); full logs in logs/mutation-*.log; capture 06-mutation-matrix.png.

mutant result verdict
unmutated head 98/98 green control
M1 isNeverDeliveredError → true (always bypass) 14 failed killed — all 10 #8387 "should not replay" cases + all 4 new "still gates" tests (positive control: the harness can make the suite fail)
M2 isNeverDeliveredError → false (no carve-out) 3 failed killed — both delivery tests + the mid-flight test
M3 gate-2 back to strict canSafelyReplay() 3 failed killed — the post-rediscovery carve-out is independently load-bearing
M4 ===.includes() 98/98 green SURVIVOR — coverage gap (F3); behaviorally wrong against real wire shapes per the sweep
M5 full revert of mcp-tool.ts to base 3 failed killed — positive control, identical to cell C

No mutant regressed a passing test to green-while-wrong; M1's 14 kills show the #8387 era tests and the new tests pin the gate jointly. The new tests are not vacuous and are pinned by the carve-out itself (M2 disables only the predicate), not by an earlier branch.

Gates

  • packages/core typecheck (tsc --noEmit): clean, exit 0.
  • ESLint on both changed files with --max-warnings 0: clean, exit 0. Gate proven live: a planted no-unused-vars violation in a scratch file was reported (exit 1), then removed.
  • Full packages/core suite: head 19162 passed / 70 failed / 10 skipped (19242); base 19072 passed / 70 failed / 10 skipped (19152). All 63 distinct failing test names are byte-identical across arms (0 head-only failures): logger.test.ts (31), ide-client.test.ts (18), memoryDiscovery.test.ts (6), file-token-storage.test.ts (5), skill-manager.test.ts (4), subagent-manager.test.ts (3), installationManager.test.ts (2), rulesDiscovery.test.ts (1) — pre-existing, environment-sensitive, untouched by this PR. The base arm additionally shows 5 provider-preset suites as collection errors; that is an artifact of the base worktree harness (they import the @qwen-code/qwen-code-core package name, which resolves across the worktree's symlink boundary) and not a base-code state — they pass on head and the PR touches no provider code. The +90 total-test delta head-vs-base = +6 (this PR) + 84 (those five suites, uncollectable on the worktree).
  • Multi-commit: metadata lists 11 commits; git rev-list HEAD^1..HEAD^2 returns 1 (shallow boundary — the known false-small). Per-commit attribution was unreachable; the aggregate HEAD^1..HEAD diff (2 files) is what was verified.

Not covered

  • Per-commit verification of the 11 PR commits (depth-2 checkout); only the aggregate diff was exercised.
  • The actual CI status of fix(review): stop the reverse-audit loop while there is still time to report #8468 / feat(review): a cost ledger from the records already on disk #8471 — no GitHub token in this environment; the "currently blocked by it" claim could not be checked either way.
  • Repo-wide gates: other packages' unit suites, test:integration:*, e2e. The PR touches only packages/core/src/tools/mcp-tool{,.test}.ts, so scope was kept there.
  • The 63 pre-existing failing test names in the full core suite were attributed (identical on both arms) but not root-caused — they are orthogonal to this PR.
  • A live end-to-end MCP server kill/reconnect through mcp-client.ts discovery (real stdio/SSE transport): the harnesses reproduce the wire shapes through the real SDK client/server (shape), not the production reconnect orchestration (cause/trigger). The pre-send rejection itself lives in the transport-agnostic Protocol.request, so no per-transport variant is expected.
  • Boundary note, not a finding: getErrorMessage('Not connected') (a bare string rejection) would match the carve-out; the SDK Protocol always rejects with Error objects, and an Error with a cause fail-closes ('Not connected (cause: …)' ≠ matcher), so this is informational.

Methodology

Environment: the CI verify container (node:22-bookworm), working tree at refs/pull/8482/merge (depth 2); npm ci + npm run build pre-done at HEAD. A/B via scratch worktrees under tmp/ (base-tree at HEAD^1, red-tree at 0cb109f513, mut-tree at HEAD), each wired to the head install via node_modules symlinks — clean controls because the PR changes no package.json/lockfile, and the unit under test imports only relative paths plus third-party packages (@google/genai, vitest, @modelcontextprotocol/sdk@1.30.0); the workspace-link realpath (node_modules/@qwen-code/qwen-code-core → head tree) was checked and is not on any import path of mcp-tool{,.test}.ts (verified by import enumeration). SDK contract facts were read from the installed dist and re-proven behaviorally by harness/sibling-sweep.mjs (real Client/Server/InMemoryTransport, no mocks, assertions on exact client-side error strings). Mutations were applied with sed/python in mut-tree and restored by git checkout after each run (harness/mutation-matrix.sh). Raw per-cell logs: logs/ (full-suite-head.log, full-suite-base.log, mutation-*.log, sibling-sweep.log). All six evidence PNGs were produced by scripts/verify-capture.mjs from live runs. Scratch worktrees removed afterwards; head tree verified clean via git status.

Evidence images

01-ab-head-mcp-suite-98-of-98

02-ab-base-source-head-tests-3-red

03-sibling-sweep-real-sdk-wire-shapes

04-premise-red-at-8387-commit

05-base-own-tests-green-92-of-92

06-mutation-matrix

Harness scripts and raw logs are in the workflow run artifacts (7-day retention).

Qwen Code · sandboxed verification

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤖 Addressed the latest review feedback (round 4/100). What changed, and what I pushed back on: · 已处理最新评审反馈(第 4/100 轮)。改动内容与我反驳保留之处如下:

Round summary

PR #8482, branch fix/mcp-replay-never-delivered. No base conflict (--conflict false); no merge performed. One inline finding this round, resolved in code.

Feedback points

[Suggestion] R3-1 (rc:3717284179) — no test pins the exact-match boundary of isNeverDeliveredError — RESOLVED

Decision: Implement. The finding is valid and in scope: the reviewer mutation-verified that loosening the gate's === to .includes() keeps the entire existing suite green, after which a delivered call whose server-side error merely contains the pre-send wording (e.g. a tool handler throwing new Error('Not connected to database')) would be classified as never-delivered and auto-replayed — exactly the double-execution hazard the gate exists to prevent.

Changes: Added a parameterized negative-case test in the auto-reconnect on connection error describe block of packages/core/src/tools/mcp-tool.test.ts, placed directly after its positive never-delivered counterpart. It runs an unannotated, untrusted tool against both server statuses (it.each over CONNECTED and DISCONNECTED, as the reviewer suggested — both reachable the gate), rejects with Error('Not connected to database'), and asserts that the unsafe-replay error is thrown, no reconnect is attempted (discoverToolsForServer not called), and no replay reaches the live client.

Evidence: a temporary mutation (===.includes('Not connected') in isNeverDeliveredError) makes both new test cases fail while everything else passes — the mutant is killed; after reverting the mutation the full file is green again. Production code is unchanged; the diff is test-only (+67 lines).

No other feedback this round: no failed checks, no issue-level comments. The review-level note that "Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally" is informational about reviewer coverage; this round's change is a unit test in packages/core, fully covered by the focused Vitest run below.

Verification

  • npx vitest run src/tools/mcp-tool.test.ts (packages/core, touched) — 100 passed (98 existing + 2 new cases)
  • Mutation check: temporarily changed === to .includes(...) in isNeverDeliveredError, re-ran the new tests — both failed (mutant killed); mutation reverted, suite back to 100 passed
  • npx prettier --write packages/core/src/tools/mcp-tool.test.ts — unchanged (already formatted)
  • npm run build — passed
  • npm run typecheck — passed
  • npm run lint — passed
  • Integration tests — not run: the change is a unit test and the behavior it pins is exercised at the unit level, not only through the bundled CLI or integration harness

Commit: fd5d80672test(core): pin the never-delivered gate's exact-match boundary

中文说明

本轮摘要

PR #8482,分支 fix/mcp-replay-never-delivered。无基分支冲突(--conflict false),未执行任何合并。本轮共一条行内意见,已在代码中解决。

反馈意见

[建议] R3-1 (rc:3717284179) — 没有测试钉住 isNeverDeliveredError 的精确匹配边界 — 已解决

决定: 采纳实现。该意见有效且在范围内:审查者已通过变异测试验证,若把闸门的 === 放宽为 .includes(),现有全部测试依然全绿,此后一个已投递、但服务端错误消息仅包含 pre-send 措辞的调用(例如工具 handler 抛出 new Error('Not connected to database'))会被判定为「从未投递」并被自动重放——这正是该闸门要防止的重复执行风险。

改动:packages/core/src/tools/mcp-tool.test.tsauto-reconnect on connection error describe 块中新增一个参数化的反例测试,紧随其正向「从未投递」对照测试之后。该测试对一个无注解、非信任的工具,在两种服务器状态下(按审查者建议用 it.each 覆盖 CONNECTED 与 DISCONNECTED——两者都会走到闸门)以 Error('Not connected to database') 拒绝,并断言:抛出 unsafe-replay 错误、未尝试任何重连(discoverToolsForServer 未被调用)、且没有重放到达存活客户端。

证据: 临时变异(将 isNeverDeliveredError 中的 === 改为 .includes('Not connected'))使两个新测试用例失败、其余全部通过——变异体被杀死;还原变异后整个测试文件恢复全绿。生产代码未改动;diff 仅含测试(+67 行)。

本轮无其他反馈:无失败检查项,无 issue 级评论。审查摘要中「Integration Tests (CLI, No Sandbox) 在 CI 中被跳过且本地未运行」一条仅是对审查覆盖面的说明;本轮改动是 packages/core 中的单元测试,已由下方的定向 Vitest 运行完整覆盖。

验证

  • npx vitest run src/tools/mcp-tool.test.ts(packages/core,受影响包)— 100 个通过(98 个已有 + 2 个新用例)
  • 变异检查:临时将 isNeverDeliveredError 中的 === 改为 .includes(...) 后重跑新测试 — 两个均失败(变异体被杀死);还原变异后恢复 100 个通过
  • npx prettier --write packages/core/src/tools/mcp-tool.test.ts — 无变化(格式已符合要求)
  • npm run build — 通过
  • npm run typecheck — 通过
  • npm run lint — 通过
  • 集成测试 — 未运行:本次改动是单元测试,其钉住的行为在单元测试层面即可验证,并非只能通过打包后的 CLI 或集成测试框架验证

提交:fd5d80672test(core): pin the never-delivered gate's exact-match boundary

Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。

Re-review when you have a moment. After round 100 this bot stops and leaves the PR for a human. · 有空请复审;第 100 轮后本 bot 停止并将 PR 交给人工。


🧠 Handled by Qwen Code · model/模型 qwen3.8-max

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Triage re-run completed without a new review.

⚠️ The bot has neither a verdict nor a deferral on fd5d80672cc7cce72d7b61b015f350e252f111a3 — no APPROVED, CHANGES_REQUESTED, or COMMENTED review of its own. A DISMISSED one does not count: dismiss_stale_reviews voids the bot's approval on every push, which is exactly when a fresh one is needed. If this re-run was meant to review or approve, it did not, and an approval left by another account is a separate vote that does not count as the bot's own.

⚠️ 机器人在 fd5d80672cc7cce72d7b61b015f350e252f111a3既没有裁决也没有 defer —— 没有属于它自己的 APPROVEDCHANGES_REQUESTEDCOMMENTED 评审。DISMISSED 不算:dismiss_stale_reviews 会在每次推送时作废机器人的批准,而那恰恰是需要一次新批准的时刻。如果这次重跑本应评审或批准,那么它没有做到;而其他账号留下的批准是另一张票,不能算作机器人自己的。

The stage comments above were updated with the latest result. View workflow run.

上方各阶段评论已更新为最新结果。查看工作流运行

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship — CI landed green after the review. ✅

@qwen-code-dev-bot

Copy link
Copy Markdown
Collaborator

🤖 Reviewed the latest feedback — no changes needed. Why, point by point: · 已审阅最新反馈——无需改动。逐点说明原因如下:

AutoFix round — no action needed

No new actionable review feedback has arrived since the last evaluation (2026-08-05T01:20:48Z), so no changes were made this round.

  • Reviews: none new. The latest review on the branch is an APPROVE from the automated reviewer on the current head fd5d806 — "LGTM, looks ready to ship — CI landed green after the review."
  • Inline comments: none new since the last round. The last finding (R3-1: no test pins the exact-match boundary of isNeverDeliveredError) was already resolved in commit fd5d80672 and verified by mutation testing in the previous round.
  • Issue-level comments: the only new comments are workflow status updates (a /triage re-run trigger and the sandboxed-verification status notice), which carry no findings or requests.
  • Failed checks: none — all completed checks are green (11 success, 20 skipped; no failures).

No base conflict was reported, so no merge was performed. The branch remains at fd5d80672.

中文说明

AutoFix 本轮 —— 无需处理

自上次评估(2026-08-05T01:20:48Z)以来没有新的可处理评审反馈,因此本轮未做任何改动。

  • 评审: 无新评审。分支上最新的评审是自动审查者在当前 HEAD fd5d806 上给出的 APPROVE —— "LGTM, looks ready to ship — CI landed green after the review."(可以合入,审查之后 CI 全绿)。
  • 行内评论: 自上一轮以来无新评论。最后一条意见(R3-1:没有测试钉住 isNeverDeliveredError 的精确匹配边界)已在提交 fd5d80672 中解决,并在上一轮通过变异测试验证。
  • Issue 级评论: 新评论仅为工作流状态更新(/triage 重跑触发与沙箱验证状态通知),不包含任何意见或请求。
  • 失败检查项: 无 —— 所有已完成的检查均为绿色(11 个成功,20 个跳过;无失败)。

未报告基分支冲突,因此未执行合并。分支保持在 fd5d80672

Base-conflict check · 基分支冲突检查: no conflict with main. · 与 main 无冲突。


🧠 Handled by Qwen Code · model/模型 qwen3.8-max

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not reviewed: build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally.

Not reviewed: test-efficacy — harness baseline could not be green (nested ajv v8 not linked into the probe tree; environment artifact), so mutants/hunks did not run; the revert probe was inconclusive (no-tests).

[Critical] R4-2 (still-standing blocker — @wenshao's merge-hold condition (a), raised in two separate reviews on this PR): the PR description still documents the DISCARDED status-snapshot mechanism ('The invocation snapshots getMCPServerStatus(serverName) before issuing its call … The gate … is skipped only when that snapshot says DISCONNECTED') and a falsified regression-test claim ('The previously-red test is the regression test; it passes with the fix and fails without it'; '91/91'), while the shipped diff keys the carve-out solely on getErrorMessage(error) === 'Not connected' — no snapshot exists anywhere in the code, and the new JSDoc explicitly rejects the status-based approach. A/B-probed at this commit: the cited test passes with the carve-out reverted (#8478 already made it green on main via trust + idempotentAnnotations), and the file now has 100 tests. The merged-PR record would describe the very mechanism prior review rejected as double-execution-unsafe, inviting a future edit to restore snapshot semantics that do not exist. Fix: rewrite the Summary / The bug / The fix / Test plan sections (and the Chinese mirror) to describe the shipped error-identity mechanism (bypass only on the SDK's pre-send 'Not connected' rejection; ambiguous failures stay gated; the previously-red test was fixed by #8478), citing the new carve-out tests as the regression tests.

中文说明

未审查:build-and-test — Integration Tests (CLI, No Sandbox) was skipped in CI and its suite did not run locally。

未审查:test-efficacy — harness baseline could not be green (nested ajv v8 not linked into the probe tree; environment artifact), so mutants/hunks did not run; the revert probe was inconclusive (no-tests)。

[Critical] R4-2 (still-standing blocker — @wenshao's merge-hold condition (a), raised in two separate reviews on this PR): the PR description still documents the DISCARDED status-snapshot mechanism ('The invocation snapshots getMCPServerStatus(serverName) before issuing its call … The gate … is skipped only when that snapshot says DISCONNECTED') and a falsified regression-test claim ('The previously-red test is the regression test; it passes with the fix and fails without it'; '91/91'), while the shipped diff keys the carve-out solely on getErrorMessage(error) === 'Not connected' — no snapshot exists anywhere in the code, and the new JSDoc explicitly rejects the status-based approach. A/B-probed at this commit: the cited test passes with the carve-out reverted (#8478 already made it green on main via trust + idempotentAnnotations), and the file now has 100 tests. The merged-PR record would describe the very mechanism prior review rejected as double-execution-unsafe, inviting a future edit to restore snapshot semantics that do not exist. Fix: rewrite the Summary / The bug / The fix / Test plan sections (and the Chinese mirror) to describe the shipped error-identity mechanism (bypass only on the SDK's pre-send 'Not connected' rejection; ambiguous failures stay gated; the previously-red test was fixed by #8478), citing the new carve-out tests as the regression tests.

— qwen3.8-max via Qwen Code /review (v0.21.5)

Comment on lines +116 to +118
function isNeverDeliveredError(error: unknown): boolean {
return getErrorMessage(error) === 'Not connected';
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] No test pins the McpError-prefix invariant this carve-out's safety rests on. The exact match is sound today only because a server-originated rejection can never arrive as a bare 'Not connected': the SDK wraps every JSON-RPC error response as McpError, whose constructor prefixes the message (MCP error ${code}: ${message}), so a delivered call whose handler echoes 'Not connected' arrives as 'MCP error -32603: Not connected' and fails the match. The real-SDK tests pin the pre-send wording and the in-flight close, but not this third leg. Distinct from R3-1 (the .includes() near-miss), which the new 'Not connected to database' test kills. Failure scenario: a future "tolerate SDK wording variations" edit loosens the matcher (e.g. endsWith('Not connected'), unanchored regex, case-fold) → the whole 100-test suite stays green (mutant executed: the near-miss test does not kill it) → a delivered server-echo now matches and the gate auto-replays it — probe-verified against the pinned SDK: handler executed 4× through MAX_RECONNECT_RETRIES instead of UNSAFE_REPLAY. Suggested fix: add the symmetric real-SDK pin next to the in-flight test:

it('still gates a delivered call whose server handler echoes the pre-send wording', async () => {
  const params = { param: 'test' };
  const [clientTransport, serverTransport] =
    InMemoryTransport.createLinkedPair();
  const server = new Server(
    { name: 'real-sdk-server', version: '0.0.0' },
    { capabilities: { tools: {} } },
  );
  // Delivered: the handler runs, then throws the SDK's own guard wording.
  // The client must see the prefixed McpError and stay gated.
  server.setRequestHandler(CallToolRequestSchema, () => {
    throw new Error('Not connected');
  });
  await server.connect(serverTransport);
  const realClient = new Client({
    name: 'real-sdk-client',
    version: '0.0.0',
  });
  await realClient.connect(clientTransport);
  const discoverToolsForServer = vi.fn().mockResolvedValue(undefined);
  const mockConfig = {
    isTrustedFolder: () => true,
    getToolRegistry: () => ({ discoverToolsForServer, ensureTool: vi.fn() }),
    getTruncateToolOutputThreshold: () => 0,
    getTruncateToolOutputLines: () => 0,
  };
  updateMCPServerStatus(serverName, MCPServerStatus.CONNECTED);
  const reconnectTool = new DiscoveredMCPTool(
    mockCallableToolInstance,
    serverName,
    serverToolName,
    baseDescription,
    inputSchema,
    undefined,
    undefined,
    mockConfig as any,
    realClient, // unannotated
  );
  await expect(
    reconnectTool.build(params).execute(new AbortController().signal),
  ).rejects.toThrow(unsafeReplayErrorMessage);
  expect(discoverToolsForServer).not.toHaveBeenCalled();
});

(probe-verified: this exact test flips under the loosening mutant while the current suite stays green)

中文说明

没有测试钉住该豁免所依赖的 McpError 前缀不变量。当前精确匹配之所以可靠,仅仅是因为服务端产生的拒绝不可能以裸 'Not connected' 到达:SDK 把每个 JSON-RPC 错误响应包装为 McpError,其构造函数会给消息加前缀(MCP error ${code}: ${message}),因此已投递、handler 恰好回显 'Not connected' 的调用到达时是 'MCP error -32603: Not connected',无法命中精确匹配。真实 SDK 测试钉住了预发送措辞与中途断连,但没有钉住这第三条腿。这与 R3-1(.includes() 近似匹配)不同——那个变异体已被新增的 'Not connected to database' 测试杀死。失败场景:未来某次"容忍 SDK 措辞变化"的修改放宽匹配(如 endsWith('Not connected')、无锚正则、忽略大小写)→ 全部 100 个测试仍然全绿(已实测该变异体:近似匹配测试杀不死它)→ 已投递的服务端回显此时会命中豁免并被自动重放——针对锁定版本 SDK 的探针验证:handler 被执行 4 次(直到 MAX_RECONNECT_RETRIES),而不是抛出 UNSAFE_REPLAY。建议修复:在中途断连测试旁补一个对称的真实 SDK 钉桩(代码见上)。已验证:该测试在放宽变异体下会由绿转红,而现有套件保持全绿。

— qwen3.8-max via Qwen Code /review (v0.21.5)

@wenshao
wenshao disabled auto-merge August 5, 2026 05:16
@wenshao

wenshao commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

Closing by maintainer decision: the #8478 resolution stands — the replay gate refuses auto-replay for unannotated tools even when the server was known-disconnected at call start, and manual retry is the recovery path. The never-delivered carve-out documented here remains available in history if that trade is ever revisited.

@wenshao wenshao closed this Aug 5, 2026
@wenshao
wenshao deleted the fix/mcp-replay-never-delivered branch August 5, 2026 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autofix/takeover Summon the autofix loop to manage this PR (remove to release; needs triage+)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants