Skip to content

test(integration): remove flaky ACP recall timing check - #10703

Merged
yiliang114 merged 3 commits into
mainfrom
codex/fix-memory-recall-snapshot-10701
Sep 1, 2026
Merged

test(integration): remove flaky ACP recall timing check#10703
yiliang114 merged 3 commits into
mainfrom
codex/fix-memory-recall-snapshot-10701

Conversation

@yiliang114

@yiliang114 yiliang114 commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

This PR removes the bundled ACP auto-memory E2E that requires a real filesystem scan to publish a deterministic result inside the production 100 ms initial-recall window.

The production behavior remains covered at the two deterministic seams that own the contract: fake-timer client tests verify the 100 ms wait, early fast-result delivery, timeout behavior, cancellation, and deduplication; ACP Session tests verify recall before the initial send and refined recall after tool responses. No production memory, configuration, timeout, or test-concurrency code changes.

Why it's needed

Main E2E run 33501336475, job 99835166630 failed this assertion three times on ecs-qwen-hk3-21, while the same commit passed on macOS. The job already had one Vitest worker after #10567.

#9992 defines the first-request behavior conditionally: recalled memory is included when fast Recall completes inside the bounded initial window. The removed E2E turns that into an unconditional real-time guarantee and therefore measures shared-runner scheduling and filesystem tail latency rather than a stable product contract. Replacing it with another orchestrated E2E would duplicate the existing deterministic tests with more moving parts.

Reviewer Test Plan

How to verify

  1. Confirm the final diff only removes the timing-sensitive ACP E2E.
  2. Run the focused client tests and confirm the fake-timer cases still cover fast result, exact 100 ms expiry, cancellation, and cross-turn deduplication.
  3. Run the focused ACP Session tests and confirm initial recall and refined post-tool recall remain covered.
  4. Run the remaining bundled ACP integration suite.

Evidence (Before & After)

Before: the E2E failed whenever a loaded runner did not finish memory scanning within 100 ms, even though missing that bounded window is a supported path.

After: the wall-clock assertion is gone; the same behavior remains protected by deterministic tests at the client and ACP Session boundaries.

Tested on

OS Status
🍏 macOS ⚠️ focused Prettier and ESLint passed; full execution awaits clean-install PR CI
🪟 Windows N/A
🐧 Linux N/A

Environment (optional)

Local isolated worktree on Node.js 22.22.0. The reused dependency tree is stale against current main, so clean-install execution is delegated to PR CI.

Risk & Scope

  • Main risk or tradeoff: real-bundle coverage no longer asserts first-request auto-memory injection; that assertion cannot be made deterministic against the production 100 ms wall-clock budget.
  • Not validated / out of scope: production recall behavior and runner concurrency are unchanged.
  • Breaking changes / migration notes: none.

Linked Issues

Fixes #10701

Related: #9992, #10567

中文说明

本 PR 做了什么

本 PR 删除 bundled ACP auto-memory E2E 中依赖真实文件系统扫描在生产 100 ms 初始 Recall 窗口内发布 deterministic result 的用例。

生产行为仍由两个真正拥有该契约的确定性测试层覆盖:client fake-timer 单测验证 100 ms 等待、fast result 提前交付、超时、取消和去重;ACP Session 单测验证首轮发送前 recall,以及工具响应后的 refined recall。本 PR 不修改生产 memory、配置、timeout 或测试并发代码。

为什么需要

Main E2E run 33501336475 的 job 99835166630ecs-qwen-hk3-21 上连续三次失败,而同一 commit 在 macOS 上通过。该 job 在 #10567 之后已经只有一个 Vitest worker。

#9992 对首个请求的定义是有条件的:fast Recall 在有限初始窗口内完成时才注入 recalled memory。被删除的 E2E 把它变成无条件实时保证,因此实际测量的是共享 runner 调度和文件系统尾延迟,而不是稳定的产品契约。再用另一套编排式 E2E 替代,只会用更多状态重复已有的确定性测试。

Reviewer Test Plan

如何验证

  1. 确认最终 diff 只删除 timing-sensitive ACP E2E。
  2. 运行聚焦 client 测试,确认 fake-timer case 继续覆盖 fast result、精确 100 ms 到期、取消和跨 turn 去重。
  3. 运行聚焦 ACP Session 测试,确认 initial recall 和 refined post-tool recall 仍有覆盖。
  4. 运行剩余 bundled ACP integration suite。

前后证据

修改前:只要高负载 runner 没有在 100 ms 内完成 memory scan,E2E 就失败,即使错过有限窗口本身是受支持路径。

修改后:墙钟断言被删除;同一行为仍由 client 与 ACP Session 边界上的确定性测试保护。

测试环境

OS 状态
🍏 macOS ⚠️ 聚焦 Prettier 和 ESLint 已通过;完整执行等待 clean-install PR CI
🪟 Windows N/A
🐧 Linux N/A

环境(可选)

Node.js 22.22.0 的本地隔离 worktree。复用依赖树相对当前 main 已过期,因此 clean-install 执行交给 PR CI。

风险与范围

  • 主要风险或取舍:real bundle 不再断言首个请求 auto-memory injection;该断言无法在生产 100 ms 墙钟预算下做到确定性。
  • 未验证 / 不在范围内:生产 recall 行为和 runner 并发保持不变。
  • Breaking changes / 迁移说明:无。

关联 Issue

Fixes #10701

Related: #9992, #10567

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finishedview run. See the stage comments in this thread for the result.

Qwen Triage 已完成 —— 查看运行。结果见本线程中的各阶段评论。

@yiliang114 yiliang114 changed the title fix(memory): preload recall documents before the first prompt preload recall documents before the first prompt Sep 1, 2026
@yiliang114
yiliang114 enabled auto-merge September 1, 2026 12:38
@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

Note on the trajectory: this PR moved through three shapes — a production preload (f221b1da), a deterministic rewrite of the test (33ed51f1), and finally a straight removal (ffd4871f). The preload was fully reverted; this gate evaluates the final head, which is a pure test deletion (−109 net lines, one file).

Template looks good ✓

Problem: Observed failure — main E2E run 33501336475 / job 99835166630 failed the ACP first-request auto-memory assertion three times on ecs-qwen-hk3-21, same commit passing on macOS; linked issue #10701 is open. The gate question is whether that flake is a product bug or a test bug, and the contract says test bug: I verified in client.ts that the initial delivery point waits at most INITIAL_MEMORY_RECALL_WAIT_MS (100 ms) and that the model-selected result "reaches the model at the ToolResult delivery point instead". The removed E2E asserted the fast-path outcome in the first request unconditionally, under real directory-scan latency — a real-time guarantee the product never made. #10567 already pulled the environment lever (one worker per shard) and the flake survived.

Direction: Aligned, with one honest reservation named below. Removing an assertion that measures shared-runner scheduling rather than a stable contract is a legitimate flake fix. The reservation: this test was also the only one driving the real recall pipeline (scan → parse → select → inject) end-to-end through a real ACP process. I checked the replacement coverage the PR body claims, and it exists and bites: Session.test.ts ("managed auto-memory" block) asserts the memory prompt is actually prepended to the initial send and the refined prompt delivered at the tool_result point; client.test.ts fake-timer tests pin the 100 ms arbitration, fast result, timeout, and cancellation; recall.test.ts covers the real scan/select path. Interface drift between those layers is caught at compile time. So the residual gap is narrow — but the rewritten variant from 33ed51f1 would have kept the end-to-end execution without any wall-clock dependency, and it remains in this PR's history if that shape is preferred later.

Size: Test-only deletion, −109 net lines in one integration-test file. Stage 0 core-module gate does not apply.

Approach: Minimal — the diff excises exactly the one it block and nothing else. No production memory/config/concurrency changes, as claimed.

Risk: No high-risk-path matches (Stage 1e). Reviewers should know the changed suite never executes in PR CI (the full CLI integration lane is merge_group-only and no merge queue is enabled; the no-AK lane doesn't include this file) — the confirmation that the flake is gone lands with post-merge main E2E. The flake removal itself is by construction: the timing-sensitive assertion no longer exists.

Moving on to code review. 🔍

中文说明

感谢贡献!

关于演变过程: 本 PR 经历了三种形态——生产侧预载(f221b1da)、测试的确定性重写(33ed51f1)、以及最终的直接删除(ffd4871f)。预载已被完整回退;本门禁按最终 head 评估,即纯测试删除(净 −109 行,单个文件)。

模板完整 ✓

问题: 已观测到的失败——主 E2E run 33501336475 / job 99835166630 在 ecs-qwen-hk3-21 上连续三次未通过 ACP 首轮 auto-memory 断言,同一 commit 在 macOS 上通过;关联 issue #10701 处于打开状态。门禁问题在于这个 flake 是产品 bug 还是测试 bug,契约给出的答案是测试 bug:我在 client.ts 中核实,初始投递点最多等待 INITIAL_MEMORY_RECALL_WAIT_MS(100 ms),且模型选择结果"改在 ToolResult 投递点到达模型"。被删除的 E2E 在真实目录扫描延迟下无条件断言首个请求包含快速路径结果——产品从未做过这种实时保证。#10567 已经拉过环境侧操纵杆(每 shard 单 worker),flake 依然存在。

方向: 对齐,但有一个诚实的保留意见见下。删除一个实际测量共享 runner 调度、而非稳定产品契约的断言,是正当的 flake 修复。保留意见是:这个测试也是唯一一个在真实 ACP 进程中端到端驱动真实 recall 管线(扫描 → 解析 → 选择 → 注入)的测试。我核对了 PR 正文声称的替代覆盖,确实存在且有力度:Session.test.ts("managed auto-memory" 块)断言记忆 prompt 确实被前置到首次发送、且精化结果在 tool_result 点投递;client.test.ts 假计时器测试钉住 100 ms 仲裁、快速结果、超时与取消;recall.test.ts 覆盖真实的扫描/选择路径。层间接口漂移由编译期捕获。因此残余缺口很窄——但 33ed51f1 的重写版本本可以在没有任何墙钟依赖的情况下保留端到端执行,它仍留在本 PR 历史中,如果之后倾向那种形态可以取回。

规模: 纯测试删除,单个集成测试文件净 −109 行。Stage 0 核心模块门禁不适用。

方案: 最小化——diff 精确切除那一个 it 块,别无其他。如正文所述,无生产 memory/配置/并发变更。

风险: 无高风险路径命中(Stage 1e)。评审者应知晓:改动的套件不会在 PR CI 中执行(完整 CLI 集成通道仅 merge_group 且当前未启用合并队列;no-AK 通道不含此文件)——flake 已消除的确认将随合并后的 main E2E 落地。flake 消除本身是构造性的:时序敏感断言已不复存在。

进入代码审查 🔍

Qwen Code · qwen3.8-max

Reviewed at ffd4871fd56b0637b62c3a457dcca8a5f48bb181 · re-run with @qwen-code /triage

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
@yiliang114 yiliang114 changed the title preload recall documents before the first prompt test(integration): remove ACP recall wall-clock dependency Sep 1, 2026
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
@yiliang114 yiliang114 changed the title test(integration): remove ACP recall wall-clock dependency test(integration): remove flaky ACP recall timing check Sep 1, 2026

@yiliang114 yiliang114 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current head after the ponytail pass. The final diff only removes the flaky wall-clock E2E; fake-timer client tests still cover bounded initial delivery, and ACP Session tests cover initial and post-tool refined delivery. No production behavior or concurrency changes. Focused Prettier and ESLint pass; PR CI is pending.

@qwen-code-dev-bot qwen-code-dev-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head ffd4871f.

  • The final shape is a pure removal of the flaky assertion (+0/−78, one test): the E2E asserted the recalled-memory marker unconditionally on the first ACP model request, which encoded a real 100 ms wall-clock race with real filesystem scanning — exactly the kind of check that legitimately loses on loaded shared runners. The claim in the PR description and the review comment matches the code.
  • Contract coverage is not dropped: the initial-window fast path stays pinned by the fake-timer client unit tests in packages/core/src/core/client.test.ts, and both delivery paths of the conditional contract (recall before the initial send; refined recall after tool responses) have assertions in the ACP Session.test.ts. No production behavior or concurrency change; remaining imports in the trimmed test file are still used.
  • No prior open reviews or threads; CI on this head has no failures (Test and the review pipeline still running); per the channel convention the call is on the review itself.

@yiliang114 yiliang114 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review findings (self-PR, cannot self-approve):

  • The wall-clock race is genuinely removed, not papered over: the fake server now orchestrates the recall lifecycle deterministically — the selector request (non-stream) completes its 250ms budget and resolves a barrier; the first streamed main request awaits that barrier and answers with a read_file tool call for the actual memory file, so the marker enters the conversation through the recall path regardless of how fast the filesystem scan ran. The assertion relaxes from 'the first main request contains the marker' to 'at least two main requests and one of them contains the marker', which matches the new two-request shape.
  • Honest scoping in the comment: the 100ms scan-budget behavior itself stays covered by the fake-timer client tests, this E2E only pins delivery.
  • fakeToolCall(name, args, id) exists in fake-openai-server.ts with a matching signature; memoryFile is assigned before the prompt is submitted and only read at request time, so no initialization race.
  • Change is confined to the single it block; no product code touched.

Verified locally on 33ed51f: built the bundle and ran the target test — 'delivers managed auto-memory through the ACP recall lifecycle' passes (3.6s). No blocking issues found.

@chiga0 chiga0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — test(integration): remove flaky ACP recall timing check

Tier: Standard (test deletion — feature coverage impact)


What the change does at the new head (ffd4871)

Pure deletion of 'injects managed auto-memory into the first ACP model request' (+0 / −78). No replacement is added.

The deleted test:

// selector delayed 250ms (outside 100ms initial recall window)
if (body['stream'] !== true) await delay(250);
return { content: 'done' };
// …
const mainRequest = fakeServer.requests.find();
expect(JSON.stringify(mainRequest?.body['messages'])).toContain(marker);

The flakiness was structural: the selector was forced outside the 100ms budget, yet the assertion still required the marker to appear in the single first main request — a race the test could not win deterministically.

Removing it is the correct short-term fix; the test was negative-value CI signal.


Coverage impact

After this PR the ACP managed auto-memory recall lifecycle has no integration test coverage. The deleted test was the only E2E check on the path from selector → memory injection → model context. A replacement that drives the delivery deterministically (e.g. via Promise coordination rather than wall-clock) is worth a follow-up.

This is noted but is not a blocker — deleting a reliably-flaky test is a valid trade-off.


CI status

All jobs at ffd4871 are SKIPPED; review-pr is still in progress. No test evidence for or against the change (expected for a pure deletion).


Verdict

No blockers. The change is correct, the deletion matches the PR title exactly, and the flakiness root cause is sound. Approving with the recommendation to add a deterministic replacement in a follow-up.

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Code review

Independent proposal first: for a flaky E2E that asserts more than the product contract guarantees, the sane options are (a) make the production path fast enough — the reverted preload, which buys the guarantee at the cost of a scan on every refresh; (b) rewrite the test to be deterministic while keeping end-to-end coverage — the intermediate commit here; (c) remove the test and lean on the deterministic seams. I'd have landed on (b); the PR lands on (c). Having verified the seam coverage, (c) is defensible — the diff is honest about the tradeoff and the claims check out.

What I verified:

  • The contract the old test over-asserted. client.ts tryConsumeMemoryPrefetch: the initial delivery point waits at most INITIAL_MEMORY_RECALL_WAIT_MS; on budget expiry the fast result is injected if available, and the model-selected result lands at the tool_result delivery point. Missing the 100 ms window is a supported path, not a defect — the old E2E turned it into an unconditional real-time assertion against a real filesystem scan on shared runners.
  • The replacement coverage exists and asserts real injection, not just invocation. Session.test.ts "managed auto-memory": the initial-recall case asserts the memory prompt is prepended to the first sent message; the post-tool case asserts the refined prompt is the follow-up request content, consumed at 'tool_result'. client.test.ts fake-timer cases pin fast result, exact 100 ms expiry, cancellation, and dedup. recall.test.ts exercises the real scan/select path.
  • The deletion is surgical. The diff removes exactly one it block; delay (from node:timers/promises) is still used by two surviving tests, so no dangling import; nothing else references the removed test or its marker.
  • No production code touched — confirmed against the full PR diff at this head.

No blockers. Standing reservation (non-blocking, named in Stage 1): nothing now drives the real recall pipeline end-to-end through a real ACP process; the deterministic rewrite at 33ed51f1 is still in history if that coverage is wanted back without the wall-clock dependency.

Test evidence (PR's own CI, via API)

CI on ffd4871f at review time: no failures; the Ubuntu gate and the no-AK integration lane were still in flight (fetched once, no polling). Two structural notes so the table isn't over-read: the removed test never ran in PR CI anyway (full CLI integration lane is merge_group-only; the no-AK lane lists its files explicitly and this one isn't among them), so a green run proves no collateral breakage, and the flake fix itself is by construction — the timing-sensitive assertion is gone. Not verified: triage never executes PR code, and no live E2E run of the remaining ACP suite is quoted here; the author's local note says Prettier/ESLint passed on a stale dependency tree — that's the author's claim, and PR CI is the clean-install check.

Check Conclusion
Test (ubuntu-latest, Node 22.x) ⏳ pending (in progress)
Integration Tests (no-AK, No Sandbox) ⏳ pending (in progress)
Classify PR ✅ success
Desktop Shell (ubuntu-22.04) ✅ success
Desktop Shell (windows-2022) ✅ success
Dependency CVE audit ✅ success
Secret scan (TruffleHog) ✅ success
Test (macos-latest, Node 22.x) ⏭️ skipped (merge_group/schedule only)
Test (windows-latest, Node 22.x) ⏭️ skipped (merge_group/schedule only)
Integration Tests (CLI, No Sandbox) ⏭️ skipped (merge_group only)

No sandboxed-lane line this run: the PR carries no behavioural claim to settle — it removes a test, and CI (plus the structural argument above) is what there is to check. If a maintainer later wants the end-to-end coverage back, the rewrite at 33ed51f1 is the starting point.

中文说明

代码审查

先给独立方案:对于一个断言超出产品契约保证的 flaky E2E,合理选项是 (a) 让生产路径足够快——即被回退的预载,代价是每次刷新都要扫描;(b) 把测试重写为确定性、同时保留端到端覆盖——即本 PR 的中间版本;(c) 删除测试、依靠确定性接缝覆盖。我会选 (b);PR 落在 (c)。在核实接缝覆盖之后,(c) 站得住——diff 对取舍是诚实的,各项声称也都查证属实。

我核实的内容:

  • 旧测试过度断言的契约。 client.tstryConsumeMemoryPrefetchinitial 投递点最多等待 INITIAL_MEMORY_RECALL_WAIT_MS;预算耗尽时若有快速结果则注入,模型选择结果在 tool_result 投递点落地。错过 100 ms 窗口是受支持路径,不是缺陷——旧 E2E 把它变成了对共享 runner 上真实文件系统扫描的无条件实时断言。
  • 替代覆盖存在且断言的是真实注入,而非仅仅调用。 Session.test.ts "managed auto-memory":初始 recall 用例断言记忆 prompt 被前置到首条发送消息;工具后用例断言精化结果作为后续请求内容、在 'tool_result' 点被消费。client.test.ts 假计时器用例钉住快速结果、精确 100 ms 到期、取消与去重。recall.test.ts 走真实扫描/选择路径。
  • 删除是外科手术式的。 diff 只移除一个 it 块;delay(来自 node:timers/promises)仍被两个存留测试使用,无悬空 import;没有其他代码引用被删测试或其 marker。
  • 未触及生产代码——已按当前 head 的完整 PR diff 确认。

无阻塞问题。长期保留意见(非阻塞,已在 Stage 1 点名):现在没有任何测试在真实 ACP 进程中端到端驱动真实 recall 管线;如需在不要墙钟依赖的前提下取回该覆盖,33ed51f1 的重写仍在历史中。

测试证据(来自 PR 自身 CI,经 API 获取)

审查时 ffd4871f 的 CI 无失败;Ubuntu 门禁与 no-AK 集成通道仍在进行(一次抓取,未轮询)。两点结构性说明,避免误读表格:被删测试本来就不在 PR CI 中执行(完整 CLI 集成通道仅 merge_group;no-AK 通道显式列出文件且不含此项),因此绿色结果证明的是无附带破坏;flake 修复本身是构造性的——时序敏感断言已不存在。未验证:triage 不执行 PR 代码,此处也没有剩余 ACP 套件的实时 E2E 运行可引用;作者本地说明称 Prettier/ESLint 在过期依赖树上通过——这是作者的陈述,PR CI 是干净安装验证路径。

本次不给沙箱验证行:PR 没有需要了结的行为性论断——它删除一个测试,可核对的就是 CI 与上述结构性论证。若维护者之后想取回端到端覆盖,33ed51f1 的重写是起点。

Qwen Code · qwen3.8-max

Reviewed at ffd4871fd56b0637b62c3a457dcca8a5f48bb181 · re-run with @qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Confidence: 4/5 — the flake is real, the deleted assertion measured host scheduling rather than a product contract, and every coverage claim in the PR body checked out; the one reservation is a coverage-policy call the author already made with eyes open.

The honest reflection here is about the pivot, because it changes what this PR is. It opened as a production fix (preload the snapshot), then rewrote the test, then deleted it. Reading the final state against the code: the old E2E asserted an unconditional first-request guarantee that the recall design never promises — the initial delivery point is a bounded 100 ms wait, and the model-selected result has a second, guaranteed delivery point at the tool result. So the three-time failure on the loaded ECS host was the test measuring storage and event-loop tail latency, not the product breaking. Once that's established, the remaining question is only how much test coverage the fix should keep.

I'd have kept the deterministic rewrite (33ed51f1) — it preserved the only end-to-end exercise of the real scan → parse → select → inject path through a real ACP process, with no wall-clock dependency. The deletion is narrower than my preference, but not wrong: the seam tests genuinely assert real injection at both delivery points, the client fake-timer tests pin the budget arbitration, recall.test.ts covers the real scan/select, and compile-time wiring catches interface drift between the layers. Risk & Scope names exactly this tradeoff. That's a maintainer's coverage-policy call, documented and factually supported — not something the gate should overrule.

Standing follow-ups, none blocking: if auto-memory recall regresses end-to-end (real wiring, not unit seams), nothing will fail until it surfaces on main E2E or in telemetry; the rewrite is in this PR's history if that gap ever hurts. The flake fix itself is by construction — the timing-sensitive assertion no longer exists — and post-merge main E2E will confirm issue #10701 closes clean.

CI is still running on this commit, so approval is deferred until CI lands green on ffd4871f; the finalize step handles the commit-pinned approval.

中文说明

置信度:4/5 —— flake 真实存在,被删断言测量的是宿主机调度而非产品契约,PR 正文中的每一项覆盖声称都查证属实;唯一的保留意见是一个作者已在知情下做出的覆盖策略决定。

这里诚实的反思在于方案转变,因为它改变了这个 PR 的性质。它以生产侧修复(预载快照)开出,随后重写测试,最后删除测试。把最终状态对照代码来看:旧 E2E 断言的是无条件的首请求保证,而 recall 设计从不承诺这一点——初始投递点是有限的 100 ms 等待,模型选择结果还有第二个保证送达的投递点(工具结果处)。因此负载较高的 ECS 主机上连续三次失败,是测试在测量存储与事件循环尾延迟,而不是产品出了故障。确立这一点后,剩下的问题只是修复应保留多少测试覆盖。

我本会保留确定性重写(33ed51f1)——它在没有任何墙钟依赖的情况下,保留了唯一一次在真实 ACP 进程中端到端执行真实 扫描 → 解析 → 选择 → 注入 路径的测试。删除比我倾向的更窄,但并非错误:接缝测试确实断言了两个投递点的真实注入,client 假计时器测试钉住预算仲裁,recall.test.ts 覆盖真实扫描/选择,编译期接线捕获层间接口漂移。Risk & Scope 精确点名了这个取舍。这是维护者的覆盖策略决定,有文档、有事实支撑——门禁不应推翻。

遗留跟进事项,均非阻塞:如果 auto-memory recall 在端到端层面(真实接线,而非单测接缝)回归,在 main E2E 或遥测暴露之前不会有任何测试失败;如果这个缺口将来造成困扰,重写版本就在本 PR 历史里。flake 修复本身是构造性的——时序敏感断言已不存在——合并后的 main E2E 将确认 issue #10701 干净关闭。

该 commit 的 CI 仍在运行,因此批准推迟到 CI 在 ffd4871f 上变绿;收尾步骤会处理按 commit 钉住的批准。

Qwen Code · qwen3.8-max

Reviewed at ffd4871fd56b0637b62c3a457dcca8a5f48bb181 · re-run with @qwen-code /triage

@yiliang114
yiliang114 added this pull request to the merge queue Sep 1, 2026
Merged via the queue into main with commit 2774f08 Sep 1, 2026
96 of 98 checks passed
@qqqys

qqqys commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Review @ ffd4871fd5 — no Critical; approval pending the Test lane / 无 Critical,待 Test 通道转绿后批准

EN

Reviewed at head ffd4871fd5. The diff is a single-hunk deletion of one it block in integration-tests/cli/acp-integration.test.ts — nothing else.

Verified the PR's coverage claims against the tree rather than taking them on faith:

  • The deterministic seams it names are real: packages/core/src/core/client.test.ts pins the exact 100 ms initial-recall ceiling with fake timers ("should hold the main request for exactly the initial recall budget when recall never settles", "should end the initial wait early when recall settles inside the budget", plus the settled-at-consume-point injection case), and packages/cli/src/acp-integration/session/Session.test.ts covers recall before the initial send and refined recall after tool responses (beginManagedAutoMemoryRecall assertions).
  • The deleted E2E asserted something stronger than the product contract: that a real filesystem scan deterministically lands inside the production 100 ms window on a shared runner. Per feat(acp): enable managed auto-memory lifecycle #9992 that inclusion is conditional on completing within the bounded window, so the deleted assertion measured runner tail latency, not behavior. The CI failure signature in the PR body (three failures on ecs-qwen-hk3-21, same commit green on macOS) matches that diagnosis.
  • No dangling references after the deletion: mkdirSync/writeFileSync/join/delay all remain in use elsewhere in the file at head.

tmux e2e: not applicable — the change removes a test and touches no runtime code; the bundle-level check of "the remaining suite still passes" is exactly what the PR's own CI lanes are for.

No Critical found. I held approval only because Test (ubuntu-latest, Node 22.x) is still in progress at head (all other completed lanes are green; the review-pr cancellation is the superseded meta review lane). Will approve once that lane reports green.

中文

在 head ffd4871fd5 复核:diff 只删除了 acp-integration.test.ts 里一个 it 块,无其他改动。PR 声称的确定性覆盖已逐条对照源码核实存在(core client 假定时器测试锁定 100ms 初始 recall 窗口的确切语义,ACP Session 测试覆盖首发前 recall 与工具响应后的 refined recall)。被删的 E2E 断言强于产品契约(#9992 中该注入以"在有界窗口内完成"为条件),实测的是共享 runner 的尾部延迟。删除后无悬空的 import/辅助函数引用。tmux e2e 不适用(纯测试删除,无运行时代码变更)。未发现 Critical;唯一阻止批准的是 Test (ubuntu-latest) 仍在跑(review-pr 的 cancelled 是被取代的元评审通道,非产品通道)。该通道转绿后即批准。

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Released in v0.23.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Main CI failed: E2E Tests on 1f120c2253fe

6 participants