Skip to content

feat(external-context): Add opt-in auto recall for administrator-owned Mem0 dialects - #11246

Merged
doudouOUC merged 3 commits into
QwenLM:mainfrom
doudouOUC:feat/external-context-mem0-auto-recall
Sep 7, 2026
Merged

feat(external-context): Add opt-in auto recall for administrator-owned Mem0 dialects#11246
doudouOUC merged 3 commits into
QwenLM:mainfrom
doudouOUC:feat/external-context-mem0-auto-recall

Conversation

@doudouOUC

@doudouOUC doudouOUC commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

What this PR does

Adds an administrator-installed, opt-in UserPromptSubmit command-Hook profile to the published Mem0 External Context package. The profile accepts a strict schemaVersion: 3 instance configuration with a canonical repository binding, reuses the administrator-owned DialectV1 and existing bounded request engine, derives its query only from submitted_prompt, and injects at most five retrieved records as structured untrusted_external_context. The default Extension manifest remains MCP-only with exactly context_search, and existing schemaVersion: 2 deployments using regular configuration files are unchanged. The package also ships the separate Hook bundle, strict v3 schema, unbranded POSIX and Windows registration examples, and administrator deployment guidance.

Why it's needed

Administrators who need deterministic recall before eligible user turns currently must build another integration or rely on the model to choose context_search. This adds a narrow Hook-only path while keeping endpoints, credentials, fixed scope, repository binding, and enablement outside model control, without changing Qwen Core or shipping provider presets.

Reviewer Test Plan

How to verify

  1. Configure a temporary repository root, a v3 instance file, an administrator-owned DialectV1, and a loopback HTTP provider. Invoke the packaged Hook with a valid UserPromptSubmit event and confirm it sends exactly one bounded request derived from submitted_prompt, not the expanded prompt, then returns the provider result as untrusted_external_context.
  2. Repeat with missing provenance, an outside working directory, invalid configuration, an empty result, an oversized response, and a provider timeout. Confirm each case returns {} with exit code zero and no integration-generated stderr after the fixed Hook entry point starts. Include a TCP server that accepts a connection but never completes TLS: the Hook must flush its output and actually exit promptly after the provider timeout. Instance and dialect FIFO paths must be rejected promptly without an outer process kill. Repeated token, api_key, password, and secret near misses must take less than 2 seconds of sanitizer work after bundle import, with a separate 8-second subprocess deadline covering startup and exit.
  3. Start the existing MCP entry point with a v2 configuration and confirm it still exposes exactly context_search; confirm a v3 configuration is rejected by MCP and the default Extension manifest contains no Hook.
  4. Inspect the package dry-run and confirm it contains both runtime bundles, both instance schemas, the dialect schema, manifest, README, and unbranded Hook examples, with no provider-specific preset or dialect. Package test commands build these bundles before running the subprocess suite.
  5. Run TUI or headless user turns with the administrator Hook profile and confirm one recall per eligible submission. Run automation with each all-Hook opt-out (--bare, --safe-mode, and disableAllHooks: true) and confirm it reaches the model without executing the Hook or sending a recall request. A separate administrator configuration without this Hook supports automation that needs other Hooks. Events lacking submitted_prompt must also send no recall request.

Evidence (Before & After)

Before: the package supported only on-demand retrieval through the model-selected context_search MCP tool.

After: an administrator can explicitly install the separate UserPromptSubmit command Hook so eligible TUI and headless CLI user submissions (including stream-json input from SDK clients) receive bounded, structured, untrusted external context before model invocation.

Automated subprocess E2E evidence is posted as a separate PR comment. This change adds no TUI surface, so screenshots are N/A.

Tested on

OS Status
🍏 macOS ✅ tested
🪟 Windows ⚠️ not tested
🐧 Linux ⚠️ not tested

Environment (optional)

macOS; Node.js v22.22.3; npm 10.9.8; local loopback synthetic HTTP/TCP providers and model; locally built CLI and shipped Hook bundles.

Risk & Scope

  • Main risk or tradeoff: Once an administrator explicitly enables the Hook, sanitized user prompt text is sent to the configured external service and retrieved content is sent to the model. This includes headless and stream-json user turns supplying submitted_prompt; it is not a TUI-only filter. Automation can disable all Hooks with --bare, --safe-mode, or disableAllHooks: true in managed settings before startup; the two flags also change which customizations are loaded. When automation needs other Hooks, administrators can use a separate controlled QWEN_HOME without this Hook and omit its configuration and credential from that launcher. Sanitization is best effort rather than DLP, and each eligible turn starts one bounded Node process and may wait for the configured provider timeout.
  • Not validated / out of scope: Live vendor services, Windows and Linux runtime execution, write operations, memory extraction, retries, redirects, arbitrary templates, and provider-specific dialects are not validated or included.
  • Breaking changes / migration notes: No schema change. Instance and dialect paths must resolve to regular files; FIFO and other special-file configurations are now rejected in both profiles. The default manifest and regular-file v2 context_search deployments remain unchanged; v3 is accepted only by the separately installed Hook and is never enabled automatically.

Linked Issues

N/A

中文说明

本 PR 做了什么

为已发布的 Mem0 External Context 包新增一个由管理员安装、显式选择启用的 UserPromptSubmit 命令 Hook 模式。该模式接受严格的 schemaVersion: 3 实例配置并绑定规范化后的仓库目录,复用管理员维护的 DialectV1 和现有有界请求引擎,只从 submitted_prompt 派生查询,并以结构化 untrusted_external_context 的形式注入最多五条检索结果。默认 Extension manifest 仍然只有 MCP,并且只暴露 context_search;使用普通配置文件的现有 schemaVersion: 2 部署保持不变。包内同时发布独立 Hook bundle、严格的 v3 schema、无品牌的 POSIX/Windows 注册示例和管理员部署说明。

为什么需要

当前,需要在符合条件的用户轮次前确定性召回上下文的管理员,只能自行构建另一套集成,或者依赖模型主动选择 context_search。本 PR 增加一条范围收敛的 Hook-only 路径,同时继续将 endpoint、凭证、固定 scope、仓库绑定和启用权置于模型控制之外,不修改 Qwen Core,也不发布任何厂商 preset。

审阅者测试计划

如何验证

  1. 配置临时仓库根目录、v3 实例文件、管理员维护的 DialectV1 和本机回环 HTTP provider。使用有效 UserPromptSubmit 事件调用打包后的 Hook,确认它只基于 submitted_prompt 而非扩展后的 prompt 发出恰好一次有界请求,并将 provider 结果作为 untrusted_external_context 返回。
  2. 分别使用缺失 provenance、仓库外工作目录、非法配置、空结果、超限响应和 provider 超时再次调用。确认固定 Hook 入口启动后,每种情况都返回 {}、退出码为零,且没有集成自身产生的 stderr。包括用只接受连接、始终不完成 TLS 握手的 TCP 服务验证:Hook 必须刷完输出,并在 provider 超时后及时真正退出。instance 和 dialect 的 FIFO 路径必须被及时拒绝,无需外部终止进程。重复 tokenapi_keypasswordsecret 的近似匹配输入在 bundle 导入完成后的脱敏计算必须低于 2 秒;另有 8 秒子进程期限覆盖启动和退出。
  3. 使用 v2 配置启动现有 MCP 入口,确认它仍然只暴露 context_search;确认 MCP 会拒绝 v3 配置,且默认 Extension manifest 中没有 Hook。
  4. 检查 package dry-run,确认包内包含两个运行时 bundle、两个实例 schema、dialect schema、manifest、README 和无品牌 Hook 示例,并且不包含任何厂商专用 preset 或 dialect。 包级测试命令会先构建这些 bundle,再执行子进程测试。
  5. 使用管理员 Hook 模式运行 TUI 或 headless 用户轮次,确认每次符合条件的提交恰好召回一次。分别使用禁用全部 Hook 的三种方式(--bare--safe-modedisableAllHooks: true)运行自动化,确认模型正常调用,但不执行 Hook、不发送召回请求。如果自动化仍需其他 Hook,可使用未注册本 Hook 的独立管理员配置。缺少 submitted_prompt 的事件也必须不发送召回请求。

证据(前后对比)

之前:该包只支持通过模型选择的 context_search MCP 工具按需检索。

之后:管理员可以显式安装独立的 UserPromptSubmit 命令 Hook,使符合条件的 TUI 和 headless CLI 用户提交(包括 SDK 客户端的 stream-json 输入)在模型调用前获得有界、结构化且不可信的外部上下文。

自动化子进程 E2E 证据会作为独立 PR 评论发布。本改动没有新增 TUI 界面,因此截图为 N/A。

已测试平台

OS 状态
🍏 macOS ✅ 已测试
🪟 Windows ⚠️ 未测试
🐧 Linux ⚠️ 未测试

环境(可选)

macOS;Node.js v22.22.3;npm 10.9.8;本机回环 synthetic HTTP/TCP provider 和模型;本地构建的 CLI 与发布用 Hook bundle。

风险与范围

  • 主要风险或取舍:管理员显式启用 Hook 后,经过脱敏的用户 prompt 文本会发送给所配置的外部服务,检索内容则会发送给模型。这包括提供 submitted_prompt 的 headless 和 stream-json 用户轮次,并非仅限 TUI 的过滤器。自动化可在启动前使用 --bare--safe-mode,或在受控配置中设置 disableAllHooks: true 来禁用全部 Hook;这两个 flag 还会改变其他自定义配置的加载范围。如果自动化仍需使用其他 Hook,管理员可使用不注册该 Hook 的独立受控 QWEN_HOME,并从该启动环境中去掉 Auto Recall 配置和凭证。脱敏是尽力而为的降低风险措施,并非 DLP;每个符合条件的轮次还会启动一个有界 Node 进程,并可能等待配置的 provider 超时。
  • 未验证或不在范围内:未验证或包含真实厂商服务、Windows/Linux 运行时执行、写操作、记忆提取、重试、重定向、任意模板和厂商专用 dialect。
  • 破坏性变更或迁移说明:没有 schema 变更。instance 和 dialect 路径必须解析到普通文件;两个模式现在都会拒绝 FIFO 及其他特殊文件配置。默认 manifest 和使用普通文件的 v2 context_search 部署保持不变;v3 只由单独安装的 Hook 接受,并且绝不会自动启用。

关联 Issue

N/A

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
@doudouOUC

Copy link
Copy Markdown
Collaborator Author

E2E test report

Validated commit: 0a0e9decf4370a2091b0f8e1c395b961d95da8c7

Environment: macOS, Node.js v24.12.0, npm 10.9.8, local loopback synthetic HTTP provider.

Results:

  • The real Hook subprocess returned {} with exit code zero and empty stderr for malformed input and unavailable configuration.
  • A real Hook subprocess loaded a v3 instance file plus DialectV1, sent exactly one configured POST request to the loopback provider, used the fixed authorization and scope, removed a secret-shaped assignment, ignored the expanded prompt, and returned the synthetic record through hookSpecificOutput.additionalContext as untrusted_external_context.
  • Package tests passed: 9 files, 76 tests. Package typecheck, lint, build, and npm pack --dry-run also passed.
  • Root npm run build, npm run typecheck, and npm run lint passed.
  • The dry-run tarball contained 10 expected files: both runtime bundles, both instance schemas, the dialect schema, manifest, README, and two unbranded Hook examples. It contained no provider-specific dialect or preset.
  • Two consecutive broad diff audits completed without a new actionable issue.

Not validated: a live vendor service or runtime execution on Windows/Linux. Those remain outside the evidence claimed by this PR.

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Qwen Triage finishedview run. See the stage comments in this thread for the result.

Qwen Triage 已完成 —— 查看运行。结果见本线程中的各阶段评论。

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Third pass, same commit. Both earlier passes are on this thread — one at 0a0e9de, one at 529ac62 — and no new code has landed since the second. So this is a re-read against evidence that has now settled, not a review of a fresh delta: CI on this head is fully green, the sandboxed /verify report has landed, and the author posted Linux results.

Template looks good ✓ — all nine required headings, in both languages.

Problem: still clears the bar, and now with independent corroboration. The feature itself is a capability gap rather than a bug, so no reproduction is owed for it. The fix commit on top is the part that needed evidence, and it has it twice over: the author's own before/after measurements (a stalled TLS handshake exiting clean at ~1783 ms instead of living until the host killed it at ~8 s; a writerless configuration FIFO rejected at ~284 ms instead of blocking ~6.8 s; a 4095-character repeated-token input under 1 ms instead of 6260 ms), and the sandboxed run's A/B, which reverted each hunk separately and watched the corresponding test go red — 0 of 5 mutants survived, each killed by the test that was supposed to kill it. Not theoretical hardening.

Direction: established rather than speculative — this is the fourth increment in a series already merged (skeleton #10149, administrator-owned dialects #10634, publishing the package #10653). The reference agent's changelog supports both halves of the shape: automatic recall is an established direction there ("Claude now automatically records and recalls memories as it works"), and so is refusing a special-file configuration path (the .mcp.json FIFO hang fix, and FileReadTool blocking on FIFOs). No direct reference to opt-in external recall as a hook, but the area is plainly relevant.

Two direction facts are new to this pass, and I verified both in the tree rather than taking them from the thread.

The first cuts against the PR. docs/users/features/hooks.md still tells hook authors that "ACP, headless, serve, SDK, and remote-input paths do not produce it in this version", while packages/cli/src/nonInteractiveCli.ts:1098 and :2396 do populate submittedPrompt for headless turns. This PR's design doc states the code-accurate rule and its README discloses the wider surface — so the PR is right and the repo's own user documentation is the stale artifact. Not a defect in this diff, but it is exactly why the direction question needs a human: an administrator who reads the official hooks doc will conclude this Hook is TUI-only, and it is not.

The second cuts in the PR's favour, and it narrows the risk I have been deferring on. Both the README and the design doc send administrators to the heaviest available mitigation — a separate administrator-controlled QWEN_HOME without the Hook, with the configuration and the credential stripped from the automation environment. Three cheaper opt-outs already exist in the repo and neither document names any of them: --bare ("Minimal mode: skip implicit startup auto-discovery and only honor explicitly provided CLI inputs", packages/cli/src/config/top-level-options.ts:126), --safe-mode, and the top-level disableAllHooks setting (packages/cli/src/config/settingsSchema.ts:3566). All three converge on getDisableAllHooks(), which is this.disableAllHooks || this.getBareMode() || this.isSafeMode() (packages/core/src/config/config.ts:8137-8138) and gates hook execution at :3157 and :4296. So qwen --bare -p '…' does not fire this Hook at all. The honest caveat is that all three are all-or-nothing — there is no per-hook disable, and --safe-mode also drops extensions, skills and MCP — so an administrator who wants other hooks but not this one still needs the segregated home. But the documentation currently overstates how expensive the mitigation is, and that overstatement is doing real work in how risky this PR reads.

What I have not changed my mind about, for the third time: this ships in a published npm package and it makes prompt egress per-turn and automatic. The tarball adds dist/auto-recall.js and an examples/ directory to a package published at 0.23.0, and those examples are instructions administrators will copy by hand. Flagging it, not blocking on it.

Size: the core-module gate does not apply — nothing under packages/ is touched, so neither tier engaged and Stage 0 raised no maintainer escalation. The whole diff is integrations/external-context-mem0/** and docs/design/**. The 2118 changed lines break down as 373 production, 1008 test, 89 schema, 601 docs/README, 39 example JSON and 8 package manifest. Both size advisories are well under threshold (373 against 500 and 1000). Tests outnumber production code nearly three to one, which is the right shape for a fail-open security boundary.

Approach: minimal, and I could not find the 80% cut — the Hook entry, the v3 schema, the two registration examples and the docs are each load-bearing for an administrator to actually install this. Nothing unrelated rode along, and the one structural change (a shared runtime fixture in the integration test) is deduplication the new tests require rather than a drive-by refactor. The one thing I would change is documentation rather than code: name the existing opt-outs above, because "segregate your QWEN_HOME" is not the only answer and reads as more alarming than the actual trade.

Risk: no elevated risk signals — none of the 16 changed files match the revert-correlated high-risk paths.

Moving on to code review. 🔍

中文说明

第三轮审查,commit 未变。本 thread 上已有前两轮 —— 一轮针对 0a0e9de,一轮针对 529ac62 —— 自第二轮以来没有新代码落地。所以本轮是在证据已经稳定之后重读,而不是审查新的增量:该 head 上的 CI 已全绿,沙箱 /verify 报告已发布,作者也补充了 Linux 结果。

模板完整 ✓ —— 中英文九个必需小节齐全。

问题: 依然达标,而且现在有了独立佐证。功能本身是能力缺口而非 bug,因此不需要复现。其上的修复提交才是需要证据的部分,而它有两重证据:作者自己的前后实测(TLS 握手停滞时约 1783 ms 干净退出,而不是存活到约 8 秒被宿主杀掉;无写入者的配置 FIFO 在约 284 ms 被拒绝,而不是阻塞约 6.8 秒;4095 字符重复 token 输入低于 1 ms,而不是 6260 ms),以及沙箱运行的 A/B —— 它逐个回退每个 hunk 并观察对应测试变红,5 个变异体 0 幸存,且每一个都由本该击杀它的那条测试击杀。不是理论性加固。

方向: 是既定的而非臆测的 —— 这是已合并系列的第四个增量(骨架 #10149、管理员维护的 dialect #10634、发布包 #10653)。参考 agent 的 changelog 同时支持这两个形态:自动召回在那边是既定方向("Claude now automatically records and recalls memories as it works"),拒绝特殊文件配置路径也是(.mcp.json 为 FIFO 时挂起的修复,以及 FileReadTool 在 FIFO 上阻塞的修复)。没有关于「以 Hook 形式选择性启用外部召回」的直接条目,但该领域显然相关。

本轮有两个新的方向性事实,我都到代码树里核对过,而不是照抄 thread 里的说法。

第一个对本 PR 不利。docs/users/features/hooks.md 仍然告诉 Hook 作者「ACP、headless、serve、SDK 和远程输入路径在本版本中不产生该字段」,而 packages/cli/src/nonInteractiveCli.ts:1098:2396 确实会为 headless 轮次填充 submittedPrompt。本 PR 的设计文档写的是与代码一致的规则,README 也披露了更宽的外发面 —— 所以是 PR 对,仓库自己的用户文档过时了。这不是本 diff 的缺陷,但它恰恰是方向问题需要人来判断的原因:读官方 hooks 文档的管理员会得出「这个 Hook 只在 TUI 生效」的结论,而事实并非如此。

第二个对本 PR 有利,而且它收窄了我一直据以暂缓的风险。README 和设计文档都把管理员指向了最重的缓解手段 —— 一个不带该 Hook 的独立受控 QWEN_HOME,并从自动化环境中剥离配置和凭证。仓库里已经存在三种更轻的退出方式,而两份文档一种都没有提到:--bare(「Minimal mode: skip implicit startup auto-discovery and only honor explicitly provided CLI inputs」,packages/cli/src/config/top-level-options.ts:126)、--safe-mode,以及顶层的 disableAllHooks 设置(packages/cli/src/config/settingsSchema.ts:3566)。三者都汇聚到 getDisableAllHooks(),其实现是 this.disableAllHooks || this.getBareMode() || this.isSafeMode()packages/core/src/config/config.ts:8137-8138),并在 :3157:4296 处控制 Hook 是否执行。所以 qwen --bare -p '…' 根本不会触发这个 Hook。需要如实说明的限制是:三者都是全有或全无 —— 没有单个 Hook 级别的禁用,而 --safe-mode 还会一并关掉 extension、skill 和 MCP —— 因此想要其他 Hook 但不想要这个 Hook 的管理员,仍然需要隔离的 home。但文档目前高估了缓解手段的代价,而这个高估在「这个 PR 读起来有多危险」上起了实质作用。

第三次重申我没有改变看法的一点:它随已发布的 npm 包一起发布,并且让 prompt 外发变成每轮自动行为。 tarball 给一个以 0.23.0 发布的包新增了 dist/auto-recall.js 和一个 examples/ 目录,而这些示例就是管理员会手工照抄的指引。这里是提示,不是阻塞。

规模: 核心模块门禁不适用 —— 完全没有触及 packages/,两级门禁都没有介入,Stage 0 也没有触发任何维护者升级。整个 diff 都在 integrations/external-context-mem0/**docs/design/**。2118 行改动的构成为:生产代码 373 行、测试 1008 行、schema 89 行、文档/README 601 行、示例 JSON 39 行、包 manifest 8 行。两项规模提示都远低于阈值(373 对 500 与 1000)。测试行数是生产行数的近三倍,对于一个 fail-open 的安全边界来说这是正确的形态。

方案: 最小化,我找不到可以砍掉 80% 的切口 —— Hook 入口、v3 schema、两个注册示例和文档,对管理员真正安装它来说都是必需的。没有夹带无关改动,唯一的结构性改动(集成测试中共享的 runtime fixture)是新测试所需的去重,而不是顺手重构。我唯一会改的是文档而不是代码:把上面那些已存在的退出方式写出来,因为「隔离你的 QWEN_HOME」不是唯一答案,而且它读起来比真实的取舍更吓人。

风险: 无升级风险信号 —— 16 个改动文件中没有一个匹配与 revert 相关的高风险路径。

进入代码审查 🔍

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 529ac621e33fa82356a765494eff1ae9489dd908 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Code review

Third pass on 529ac62. No new code since the last review, so I did not re-derive the whole diff — I re-verified the load-bearing claims myself and spent the new effort in two places: the sandboxed /verify report that landed after my last pass, and the automation opt-out question. No critical blockers and no AGENTS.md violations in the production code. One observed defect in the PR's own new test file.

I was wrong last time about which test was fragile, and the right answer is a different problem. My previous pass predicted a Linux-only failure in the stalled-TLS test, because it attaches no 'error' listener to the socket the hook deliberately abandons and this package ignores unhandled errors everywhere except Linux. That risk did not materialize. Test (ubuntu-latest, Node 22.x) is green on this head, and the sandboxed gate ran that exact test five times and it passed every round — 2246 ms even in the round where the file failed. It is also the only test in the file carrying an explicit timeout (}, 10000); at line 170), which is why it was never the exposed one. The socket.on('error', () => {}) line is still cheap hardening, but it is no longer an open risk and I am not carrying it forward as one.

What is open is the test nobody would have suspected: the trivial first one. The deterministic flakiness gate in /verify run 34087778243 re-ran the four changed test files five times each, identically. Three came back PPPPP. auto-recall.integration.test.ts came back PFPPP — round 2 failed with exit 1:

× Mem0 Auto Recall local provider > runs the real entry point and fails open without stderr  5597ms
  → Test timed out in 5000ms.
❯ src/auto-recall.integration.test.ts:44:3

I read the file rather than accepting the label, and the mechanism is a budget inversion, not a race. runAutoRecallProcess gives each spawned child timeout: 8000 (line 256). The first test spawns two children sequentially — one with malformed stdin '{', one with a missing instance-config path (lines 45 and 52). The package vitest.config.ts sets no testTimeout, so vitest's 5000 ms default applies, and this test passes no explicit timeout. So the child budget is 8 s each against a 5 s envelope for two of them: any cold start slow enough to be ordinary on a loaded runner — node startup plus bundle load, twice — puts the test outside its own deadline while every child is still comfortably inside theirs. It passed CI on this head and 4 of 5 identical sandboxed rounds; it failed the fifth at 5597 ms against 5000 ms.

This matters more than the margin suggests, and it is why I am naming it as the one thing I would want fixed before merge rather than filing it as a nit. It is the exact class the repo's flakiness gate exists to catch: after merge this file runs on every PR that touches the package, and a 1-in-5 timing failure becomes intermittent red that has nothing to do with whoever is being blocked by it. The verify job reported the whole run as not passed for precisely this reason, overriding its own agent verdict of merge-ready and 565/565 scripted assertions. The fix is small — give that test an explicit timeout that actually covers two 8-second children, or drop the per-child spawn timeout below the test budget — and it is in test code, not shipped code.

The rest of the production code holds up, and I re-checked the parts that carry weight rather than trusting my earlier notes. The sanitizer rewrite is sound, and the reason is the lookbehind rather than the lookahead: (?<![A-Za-z0-9_.-]) rejects every position inside an identifier run in constant time once position 0 fails, so SECRET_ASSIGNMENT_PATTERN (auto-recall.ts:22-24) is linear. The process-lifetime fix uses the correct pattern rather than the obvious one — process.stdout.end(() => process.exit(0)) queues the end behind the pending write, where a bare process.exit(0) after an unawaited write can truncate piped stdout and trade a hang for silently lost hook output. The FIFO rejection is correct and leaks nothing: O_NONBLOCK makes open() return instead of blocking the filesystem worker, file.stat() follows symlinks so a symlink to a regular file still passes, the new throw lands inside the existing try so it becomes the same redacted ConfigurationError as every other read failure, and the existing finally { await file?.close() } closes the handle. And the CI wiring claim is real, which I verified end to end because three separate things have to line up: the package is in the root workspaces array, root test:ci:workspaces fans out to it, ci.yml invokes that at its test step, the package's own test:ci is npm run build && vitest run, and the integration test spawns ../dist/auto-recall.js rather than tsx-running the source. Per-PR CI builds and executes the shipped bundle.

Two findings from the sandboxed report that I confirmed in the code myself, both Suggestions rather than blockers.

The first is an attribution error in a comment, and it is load-bearing. The comment above SECRET_ASSIGNMENT_PATTERN reads "Check each identifier once, without overlapping scans around the keyword" — true of that pattern, which measures 0–1 ms. But the superlinear cost in this sanitizer lives in a different regex, the JWT-shaped one at auto-recall.ts:104-107 (/\b[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\b/g): - is a non-word character, so every hyphen creates two \b positions, and from each one {8,} consumes to end-of-string and backtracks hunting a literal . that is absent. The sandboxed measurement puts it at exponent 2.01 — 5217 ms on a 65 536-character hyphen-dense dot-free input with the cap reverted, flat ~20 ms with it. MAX_SANITIZER_INPUT_CHARACTERS = 4096 is applied at line 93-95, before every replace, so it bounds this pattern too — and it is the only thing that does, because HOOK_WALL_CLOCK_TIMEOUT_MS = 6500 is a setTimeout (lines 120-127) and a synchronous regex spin starves the event loop, so the timer that is supposed to bound the hook cannot fire while the sanitizer runs. At head this is fine; the cap is present and the blast radius is bounded anyway by core's group SIGTERM. The risk is the misdirection: someone trusting that comment could raise or remove the cap and remove the sole defence on this path. Worth moving the comment, and worth a fixture that pins the hyphen-dense dot-free shape — relatedly, the PR's own test-plan step 2 enumerates token, api_key, password and secret near-misses as the demanding case, and those are the linear half of the class because they contain no hyphens.

The second is a dead field the PR hardens with a new test. Both shipped examples set "statusMessage": "Retrieving external context", and the new manifest.test.ts case pins that exact value. For a type: "command" hook nothing renders it: the only read site is packages/core/src/hooks/httpHookRunner.ts:172-173, gated on this.statusMessageCallback, whose setter setStatusMessageCallback (:122-124) I grepped across the whole tree — the definition is the only match, nothing calls it. So the administrator is promised a spinner label that never appears, and a new assertion now pins a field that decides nothing. Not a regression introduced here: the sibling integrations/external-context examples ship the identical field, so this mirrors an established repo pattern. Reported because the PR adds the test. Everything else in both example files checks out against core, and one field is doing more work than it looks: timeout: 8000 is load-bearing, because core's default is 60 000 ms (hookRunner.ts:45) and an administrator who omits it inherits a full minute of stalled turn on the pathological input above.

Three items carried forward, unchanged, none of them a defect in the production code. readConfigFile is shared by both profiles, so the new regular-file requirement also changes the already-published v2 MCP path — a v2 deployment pointing at a process substitution now fails closed where it used to work, in a package administrators pin at 0.23.0. The PR body discloses it and the author has since explicitly agreed it needs maintainer review; I agree, and it deserves a human nod rather than riding inside a feature PR. Running a single test file in this package now fails opaquely, because the integration test needs dist/auto-recall.js and there is no globalSetup guard to say so — AGENTS.md's documented workflow yields a spawn ENOENT instead of "run npm run build first", and packages/cli already has the guard precedent. And per-PR CI still does not typecheck this package: root typecheck runs the workspaces but ci.yml only invokes typecheck:integration on PRs, so the generic loadConfiguration<T extends InstanceConfigV2 | InstanceConfigV3> refactor rests on my own enumeration of its consumers rather than on a green check. That is a pre-existing repo CI gap, not something this PR introduced.

One follow-up that belongs in a separate issue, surfaced by this PR but not caused by it: docs/users/features/hooks.md still claims headless paths do not produce submitted_prompt, contradicted by packages/cli/src/nonInteractiveCli.ts:1098 and :2396. Core's own user-facing description of the field in packages/cli/src/ui/components/hooks/constants.ts and its nine locale copies calls it "the supported interactive TUI text projection", incomplete in the same way. Nothing under packages/ is in scope here, but this PR is the first thing to depend on the distinction, and a hook author reading either description draws exactly the wrong conclusion about which turns fire.

sequenceDiagram
    participant P1 as Qwen Code TUI or headless CLI
    participant P2 as Hook runner
    participant P3 as auto-recall entry
    participant P4 as Config loader
    participant P5 as Query sanitizer
    participant P6 as Request engine
    participant P7 as Mem0 provider
    P1->>P2: user submits a prompt
    P2->>P3: spawn process, event JSON on stdin
    P3->>P3: require UserPromptSubmit plus submitted_prompt
    P3->>P4: load v3 instance config and dialect
    P4-->>P3: reject any path that is not a regular file
    P4-->>P3: canonical repository root and credential
    P3->>P3: realpath cwd, return empty when outside root
    P3->>P5: submitted_prompt only, never expanded prompt
    P5-->>P3: cap at 4096 chars, strip secrets, keep 512 code points
    P3->>P6: one search with the provider timeout
    P6->>P7: bounded POST, no retry, no redirect
    P7-->>P6: results, timeout, or transport failure
    P6-->>P3: at most five items
    P3-->>P2: untrusted_external_context, or empty object on any failure
    P3->>P3: flush stdout and exit zero
    P2-->>P1: injected as additionalContext
Loading
Files changed (all 16 — head unchanged since the prior pass, so this is the full map rather than a delta)
File What changed
integrations/external-context-mem0/src/auto-recall.ts New Hook entry: envelope validation, repository-root binding, linear-time secret pattern, 4096-char sanitizer cap, explicit stdout flush and exit
integrations/external-context-mem0/src/auto-recall.test.ts Unit coverage for the entry, including five positive redaction cases pinning the rewritten pattern against over-narrowing
integrations/external-context-mem0/src/auto-recall.integration.test.ts Spawns the real esbuild bundle; stalled-TLS exit, subprocess sanitizer deadlines, FIFO rejection. Contains the timing flake described above
integrations/external-context-mem0/src/auto-recall-config.test.ts v3 instance-config acceptance and rejection cases
integrations/external-context-mem0/src/config.ts Generic loader over v2 and v3, nonblocking open plus regular-file check so a writerless FIFO is rejected instead of blocking
integrations/external-context-mem0/src/schemas.ts Strict v3 schema alongside the existing v2 one
integrations/external-context-mem0/src/types.ts v3 instance-config types and the auto-recall binding shape
integrations/external-context-mem0/src/request-engine.ts Small signature change so the Hook reuses the existing bounded engine
integrations/external-context-mem0/src/manifest.test.ts Pins the manifest as MCP-only with no hooks key, and pins the example files including the dead statusMessage field
integrations/external-context-mem0/package.json Second esbuild bundle for the Hook; both test scripts build first so CI executes the shipped artifact
integrations/external-context-mem0/schemas/auto-recall-instance-config.schema.json Published strict v3 schema for administrator validation
integrations/external-context-mem0/examples/managed-auto-recall-user-settings-posix.json POSIX registration example administrators copy by hand; exec form keeps the pid as process-group leader
integrations/external-context-mem0/examples/managed-auto-recall-user-settings-windows.json Windows registration example; needs the explicit powershell shell it supplies
integrations/external-context-mem0/README.md Administrator deployment guidance, eligibility scope corrected to include headless and stream-json turns
docs/design/external-context-mem0-auto-recall.md New design doc: threat model, failure semantics, verification plan
docs/design/external-context-mem0-extension.md Existing design doc updated for the second profile

Test evidence

Unattended CI run, so per the triage rules I did not build, run, or execute anything from this PR — no npm, no vitest, no node against the PR tree, and no checkout of it. Everything below is the PR's own CI and the sandboxed /verify report read through the API for commit 529ac621e33fa82356a765494eff1ae9489dd908, plus my own static reading of the code and of the repo's CI wiring. I did not poll or sleep-wait on anything; this is the state as fetched.

CI on this head is now fully settled and green — nothing is pending, and nothing is red. That is a change from my last pass, which fetched three jobs still running. Test (ubuntu-latest, Node 22.x) has since succeeded, which was the oracle for the risk I raised then. The one failure in the completed attempt was web-shell E2E Smoke (ubuntu-latest, Node 22.x); the author reran it and I confirmed through the API rather than taking the report — Qwen Code CI run 34087546230 attempt 2 is completed / success, and the check-run for that name now reads success. I classify it as pre-existing infrastructure noise on the identity of the check and the diff, not on the author's log analysis: nothing in this PR touches web-shell, browser paths, or anything the smoke test loads, and a rerun of the identical commit passing is itself the strongest available evidence of non-determinism. The author's net::ERR_NETWORK_CHANGED accounting is their own reading of the artifact and I have not verified it.

review-pr shows in_progress — that is bot orchestration under pull_request_target, not PR CI, and it is deliberately excluded from the pending count. The skipped macOS and Windows Test jobs are repo matrix behaviour, not something this PR did; they were skipped on the previous head too. Stating it plainly because it still matters: this commit gets no macOS or Windows unit signal from CI at all.

The sandboxed /verify report for this exact head has landed, and it is the strongest evidence in this thread. It ran the PR in an isolated token-free container: A/B against the base build with each fix hunk reverted separately, mock-free harnesses against a real loopback HTTP provider and a real never-completing TLS server, and a mutation matrix against the PR's own suite. 565/565 scripted assertions passed, 0 of 5 mutants survived, and the central claim held — head passed 66/66 across 11 real-subprocess scenarios with a worst case of 6581 ms inside core's 8000 ms hook budget. It is also where the flake finding above comes from, and the reason the run is marked not passed despite a merge-ready agent verdict. A new /verify job is in flight right now in run 34102775218 (the same run as this triage pass); its report will land in this thread separately and I did not wait for it.

What this comment carries: real CI check names and conclusions as fetched, the sandboxed report's measured A/B and mutation results, and my own static verification of the sanitizer complexity, the flush-before-exit ordering, the descriptor cleanup, the budget inversion behind the flake, the dead statusMessage field, the hook-disable wiring in core, and the end-to-end CI wiring that makes the bundle actually run. Not verified: any runtime behaviour on Windows (the PR marks it untested and the CI matrix skips it); real TUI or headless end-to-end recall turns, which no automated lane here drives; the npm pack --dry-run tarball contents, statically consistent with the six files entries but never executed; this package's typecheck, which no per-PR job runs; and the flake's true rate — 1-in-5 on one container is an observation, not a distribution.

CI results for 529ac62 as fetched on this pass — settled, nothing pending:

Check Conclusion
Classify PR ✅ success
Desktop Shell (ubuntu-22.04) ✅ success
Desktop Shell (windows-2022) ✅ success
Integration Tests (no-AK, No Sandbox) ✅ success
Lint & Static (ubuntu-latest, Node 22.x) ✅ success
Test (ubuntu-latest, Node 22.x) ✅ success
web-shell E2E Smoke (ubuntu-latest, Node 22.x) ✅ success (attempt 2; attempt 1 failed, rerun by the author)
review-pr 🔄 in progress (bot orchestration, not PR CI)

One row per check name (latest run); skipped checks omitted — Test (macos-latest, Node 22.x), Test (windows-latest, Node 22.x) and Integration Tests (CLI, No Sandbox) are skipped by the repo matrix. / 每个检查名一行(取最新一次运行),省略 skipped 项 —— macOS 与 Windows 的 TestIntegration Tests (CLI, No Sandbox) 由仓库矩阵跳过。

Sandboxed verification is the lane that matters here, and it has already spoken once — the report above is /verify's, and a second run is in flight. Naming what is still unsubstantiated rather than repeating the trigger: the three claims the first run did settle are runtime claims static review could not reach — that the process exits zero with a flushed {} and empty stderr when the provider stalls mid-TLS instead of lingering, that a writerless FIFO is rejected in a few hundred milliseconds, and that the rewritten secret pattern holds its bound while still redacting the legitimate shapes. What no lane has touched is the end-to-end turn: that a real TUI or headless user submission actually produces exactly one recall and injects it as additionalContext, which is the behaviour an administrator would notice and the one thing neither CI, nor the sandbox, nor my reading demonstrates. /tmux is the weaker lane for it — this PR adds no TUI code of its own, and its TUI-facing surface is a hook registration an administrator writes by hand — but it is the only lane that would show a real turn. The author holds admin on this repo, so both lanes are ordinary runs rather than sponsored ones.

中文说明

代码审查

第三轮审查 529ac62。自上次审查以来没有新代码,因此我没有重新推导整个 diff —— 我亲自复核了关键论断,并把新的精力放在两处:上一轮之后才发布的沙箱 /verify 报告,以及自动化退出方式的问题。生产代码中没有关键阻塞问题,也没有违反 AGENTS.md 的地方。但 PR 自己新增的测试文件里有一个已观测到的缺陷。

上一轮我判断错了哪个测试脆弱,而正确答案是另一个问题。 我之前预测 TLS 停滞测试存在仅 Linux 的失败面,理由是它没有给 Hook 故意抛弃的 socket 挂 'error' 监听器,而本包在除 Linux 以外的平台忽略未处理错误。这个风险没有成真。Test (ubuntu-latest, Node 22.x) 在这个 head 上是绿的,沙箱门把这个测试跑了五次,每轮都通过 —— 即使在整个文件失败的那一轮里它也是 2246 ms 通过。它还是文件中唯一带显式超时的测试(第 170 行的 }, 10000);),这正是它从来不是暴露面的原因。socket.on('error', () => {}) 仍然是便宜的加固,但它不再是一个未决风险,我不再把它作为风险带着走。

真正未决的是没人会怀疑的那个测试:最平凡的第一个。 /verify run 34087778243 的确定性抖动门把四个改动的测试文件各自原样重跑了五次。三个是 PPPPPauto-recall.integration.test.tsPFPPP —— 第 2 轮以 exit 1 失败:

× Mem0 Auto Recall local provider > runs the real entry point and fails open without stderr  5597ms
  → Test timed out in 5000ms.
❯ src/auto-recall.integration.test.ts:44:3

我读了文件而不是接受那个标签,机制是预算倒挂,不是竞态。runAutoRecallProcess 给每个子进程 timeout: 8000(第 256 行)。第一个测试顺序启动两个子进程 —— 一个是畸形 stdin '{',一个是缺失的 instance 配置路径(第 45 与 52 行)。包级 vitest.config.ts 没有设置 testTimeout,因此适用 vitest 默认的 5000 ms,而这个测试没有传显式超时。于是子进程预算是每个 8 秒、两个共 16 秒,而外层信封只有 5 秒:任何在负载 runner 上普通到不能再普通的冷启动 —— node 启动加 bundle 加载,两次 —— 都会让测试超出自己的 deadline,而每个子进程都还舒舒服服地在它们自己的预算内。它在这个 head 的 CI 上通过,在 5 轮相同的沙箱运行中通过了 4 轮;第 5 轮以 5597 ms 对 5000 ms 失败。

这件事的分量超过那个差值本身,这也是我把它列为「合并前我唯一希望修掉的东西」而不是当成小毛病的原因。它正是仓库抖动门存在要抓的那一类:合并之后这个文件会在每个触及该包的 PR 上运行,而 1/5 的时序失败会变成与被阻塞者毫无关系的间歇性红灯。verify 任务正是因为这一点把整次运行报告为不通过,覆盖了它自己 agent 的 merge-ready 判定和 565/565 的脚本断言。修复很小 —— 给那个测试一个真正能覆盖两个 8 秒子进程的显式超时,或者把每个子进程的 spawn 超时压到测试预算之下 —— 而且它在测试代码里,不在发布代码里。

生产代码的其余部分站得住,而我重新核对了承重的部分,没有依赖自己早先的笔记。 脱敏重写是可靠的,原因在于后顾断言而非 lookahead:(?<![A-Za-z0-9_.-]) 在位置 0 失败之后以常数时间拒绝标识符串内的每个位置,因此 SECRET_ASSIGNMENT_PATTERNauto-recall.ts:22-24)是线性的。进程生命周期修复用的是正确写法而非最直觉的写法 —— process.stdout.end(() => process.exit(0)) 把 end 排在待写数据之后,而在未 await 的写入之后直接 process.exit(0) 可能截断管道 stdout,等于把挂起换成 Hook 输出静默丢失。FIFO 拒绝正确且不泄漏:O_NONBLOCKopen() 立即返回而不是阻塞文件系统 worker,file.stat() 跟随符号链接因此指向普通文件的符号链接仍通过,新增 throw 落在既有 try 内因此变成与其他读取失败相同的已脱敏 ConfigurationError,既有 finally { await file?.close() } 关闭句柄。CI 接线的声明也是真的,我端到端核对过,因为有三处必须同时对上:本包在根 workspaces 数组里,根 test:ci:workspaces 分发到它,ci.yml 在测试步骤调用它,包自己的 test:cinpm run build && vitest run,而集成测试启动的是 ../dist/auto-recall.js 而不是用 tsx 跑源码。每个 PR 的 CI 都会构建并执行发布用 bundle。

沙箱报告里的两个发现我自己在代码中确认过,都是 Suggestion 而非阻塞项。

第一个是注释里的归因错误,而且它是承重的。SECRET_ASSIGNMENT_PATTERN 上方的注释写着「Check each identifier once, without overlapping scans around the keyword」—— 对这个模式成立,它实测 0–1 ms。但这个脱敏函数里的超线性开销在另一个正则,即 auto-recall.ts:104-107 的 JWT 形状模式(/\b[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\b/g):- 不是单词字符,因此每个连字符产生两个 \b 位置,而从每个位置出发 {8,} 都会贪婪吃到串尾再一路回溯去找一个并不存在的字面 .。沙箱实测指数为 2.01 —— 在 65 536 字符的连字符密集、无点输入上,回退 cap 后 5217 ms,保留 cap 则平坦约 20 ms。MAX_SANITIZER_INPUT_CHARACTERS = 4096 在第 93-95 行、所有 replace 之前生效,因此它也约束了这个模式 —— 而且它是唯一约束它的东西,因为 HOOK_WALL_CLOCK_TIMEOUT_MS = 6500 是一个 setTimeout(第 120-127 行),同步的正则循环会饿死事件循环,于是本该约束 Hook 的计时器在脱敏运行时无法触发。在 head 上这没问题:cap 在,而且爆炸半径本来也被 core 的进程组 SIGTERM 兜住。风险在于误导:相信那条注释的人可能提高或移除 cap,从而移除这条路径上唯一的防线。值得把注释挪位置,也值得加一个钉住「连字符密集且无点」形状的 fixture —— 相关地,PR 自己的测试计划第 2 步把 tokenapi_keypasswordsecret 的近似匹配列为苛刻用例,而它们恰恰是这一类里线性的那一半,因为不含连字符。

第二个是 PR 用新测试加固了一个死字段。两个发布示例都设了 "statusMessage": "Retrieving external context",新增的 manifest.test.ts 用例还钉住了这个确切值。对 type: "command" 的 Hook 没有任何东西渲染它:唯一的读取点是 packages/core/src/hooks/httpHookRunner.ts:172-173,受 this.statusMessageCallback 门控,而其 setter setStatusMessageCallback:122-124)我在整棵树里 grep 过 —— 只有定义本身匹配,没有任何调用点。所以管理员被承诺了一个永远不会出现的 spinner 文案,而现在有一条新断言钉住了一个什么都不决定的字段。这不是本 PR 引入的回归:同级的 integrations/external-context 示例带着完全相同的字段,因此这是沿用仓库既有模式。之所以报告,是因为 PR 新增了那条测试。两个示例文件的其余内容都与 core 对得上,而有一个字段的作用比看起来大:timeout: 8000 是承重的,因为 core 的默认值是 60 000 ms(hookRunner.ts:45),省略它的管理员在上面那种畸形输入上会继承整整一分钟的轮次停滞。

三项沿用,未变,都不是生产代码的缺陷。 readConfigFile 由两种模式共用,因此新的普通文件要求同样改变了已发布的 v2 MCP 路径 —— 指向进程替换的 v2 部署现在会 fail-closed,而此前可用;这个包管理员是按 0.23.0 固定的。PR 正文已披露,作者此后也明确同意需要 maintainer review;我同意,它值得由人明确点头,而不是搭在功能 PR 里过去。在本包里单独运行一个测试文件现在会失败得不明所以,因为集成测试需要 dist/auto-recall.js 而没有 globalSetup 守卫来说明 —— AGENTS.md 记录的工作流会得到 spawn ENOENT 而不是「请先 npm run build」,而 packages/cli 里已有守卫先例。以及每个 PR 的 CI 仍然不对本包做类型检查:根 typecheck 会跑各 workspace,但 ci.yml 在 PR 上只调用 typecheck:integration,因此泛型化的 loadConfiguration<T extends InstanceConfigV2 | InstanceConfigV3> 的正确性依赖我对使用点的逐一枚举,而不是某个绿色检查。这是仓库既有的 CI 缺口,不是本 PR 引入的。

一个应该单独提 issue 的后续项,由本 PR 暴露但不是它造成的:docs/users/features/hooks.md 仍然声称 headless 路径不产生 submitted_prompt,而 packages/cli/src/nonInteractiveCli.ts:1098:2396 与之矛盾。core 自己面向用户的字段描述(packages/cli/src/ui/components/hooks/constants.ts 及其九份语言副本)称它为「受支持交互式 TUI 的文本投影」,以同样的方式不完整。这里没有任何 packages/ 下的内容在范围内,但本 PR 是第一个依赖这个区分的东西,而读这两处描述来写 Hook 的人会对「哪些轮次会触发」得出完全错误的结论。

(时序图与文件清单见英文部分,内容一致。)

测试证据

这是无人值守的 CI 运行,因此按 triage 规则我没有构建、运行或执行本 PR 的任何代码 —— 没有对 PR 树跑 npmvitestnode,也没有 checkout 它。下面全部内容是通过 API 读取的、本 PR 自己在 commit 529ac621e33fa82356a765494eff1ae9489dd908 上的 CI 与沙箱 /verify 报告,加上我对代码和仓库 CI 接线的静态阅读。我没有轮询或 sleep 等待;这是抓取当时的状态。

这个 head 上的 CI 现已完全结束并且全绿 —— 没有 pending,也没有红色项。 这与上一轮不同,那次抓取时有三个任务仍在运行。Test (ubuntu-latest, Node 22.x) 此后成功了,而它正是我上一轮所提风险的判据。已完成那次尝试里唯一的失败是 web-shell E2E Smoke (ubuntu-latest, Node 22.x);作者重跑了它,而我是通过 API 确认的,不是采信那份报告 —— Qwen Code CI run 34087546230 的 attempt 2 是 completed / success,该名称的 check-run 现在读作 success。我把它归为既有基础设施噪音,依据是这个 check 的身份和 diff,而不是作者的日志分析:本 PR 没有任何东西触及 web-shell、浏览器路径或该 smoke 测试加载的任何内容,而同一 commit 重跑通过本身就是非确定性最有力的证据。作者的 net::ERR_NETWORK_CHANGED 统计是他们对产物的解读,我没有核实。

review-pr 显示 in_progress —— 那是 pull_request_target 下的机器人编排任务,不是 PR CI,我已刻意把它排除在 pending 计数之外。被跳过的 macOS 与 Windows Test 任务是仓库矩阵行为,不是本 PR 所致;上一个 head 上同样被跳过。直说这一点,因为它仍然重要:这个 commit 完全没有来自 CI 的 macOS 或 Windows 单测信号。

针对这个 head 的沙箱 /verify 报告已经发布,它是本 thread 里最强的证据。它在隔离、无凭证的容器中运行了 PR:与 base 构建做 A/B 并逐个回退修复 hunk、用无 mock 的 harness 对接真实回环 HTTP provider 和真实的永不完成 TLS 的服务、并对 PR 自己的套件做变异矩阵。565/565 脚本断言通过,5 个变异体 0 幸存,中心主张成立 —— head 在 11 个真实子进程场景中 66/66 通过,最差 6581 ms,落在 core 的 8000 ms Hook 预算内。上面的抖动发现也来自它,而这也是整次运行被标记为不通过的原因,尽管 agent 判定是 merge-ready新的一次 /verify 任务此刻正在 run 34102775218 中执行(与本轮 triage 同一个 run);其报告会另行发布到本 thread,我没有等它。

本评论携带的证据:抓取时真实的 CI check 名称与结论、沙箱报告实测的 A/B 与变异结果,以及我本人的静态核对 —— 脱敏复杂度、刷完再退出的顺序、描述符清理、抖动背后的预算倒挂、死的 statusMessage 字段、core 里的 Hook 禁用接线,以及让 bundle 真正被执行的整条 CI 接线。未验证:Windows 上的任何运行时行为(PR 标记为未测试,CI 矩阵也跳过);真实的 TUI 或 headless 端到端召回轮次,这里没有任何自动化通道会驱动它;npm pack --dry-run 的 tarball 内容,与六个 files 条目静态一致但从未执行;本包的类型检查,没有任何 per-PR 任务会跑它;以及抖动的真实概率 —— 单个容器上的 1/5 是一次观测,不是一个分布。

(CI 表格见英文部分的机器可读区域。)

沙箱验证是这里真正重要的通道,而它已经发声过一次 —— 上面的报告就是 /verify 的,第二次运行正在进行。我要点出的是仍然未被证实的部分,而不是重复那句触发语:第一次运行确实定论的三条主张都是静态审查够不到的运行时主张 —— provider 在 TLS 握手中途停滞时进程确实以退出码零输出刷完的 {} 和空 stderr 而不是滞留;无写入者的 FIFO 在几百毫秒内被拒绝;重写后的密钥模式在守住界限的同时仍然脱敏那些合法形状。而没有任何通道触及的是端到端轮次:真实的 TUI 或 headless 用户提交是否确实产生恰好一次召回并把它作为 additionalContext 注入 —— 这正是管理员会注意到的行为,也是 CI、沙箱和我的阅读三者都没有演示的那一件事。/tmux 对它是较弱的通道 —— 本 PR 自身没有新增 TUI 代码,其面向 TUI 的表面只是管理员手写的 Hook 注册 —— 但它是唯一能展示真实轮次的通道。作者在本仓库持有 admin 权限,因此两条通道都是常规运行而非 sponsored 运行。

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 529ac621e33fa82356a765494eff1ae9489dd908 · re-run with @qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Confidence: 3/5 — the production code is clean and better tested than I would have managed, the Linux oracle closed my last technical question, and then the sandboxed gate found a real defect that neither CI nor my static reading had: the PR's own new integration test is timing-flaky. Merging still needs a human call I cannot make, and now also needs one small test fix.

To be precise about where the score comes from: it is my genuine judgment, not a policy cap. This is a fork PR but the title is feat, so the fork-refactor approval guardrail does not apply (GUARD computes to ok), and nothing under packages/ is touched, so the core-module gate never engaged and raised no escalation. I am deferring on the merits, same as the two passes before this one.

Going back to my independent proposal: for this code I would have written what the author wrote. Bound the sanitizer's worst case, stop the process lingering after its output is ready, refuse a configuration path that can block the reader, and make CI execute the artifact that actually ships. I found no simpler path on the first read and none on the third. Where the work exceeds what I would have produced is in the test design, and the sandboxed run is what let me confirm that rather than assume it: five mutants, each a single fix hunk reverted, and zero survivors — every guard is pinned by a test that fails without it, and each kill names real expected-versus-actual values (expected 3481 to be less than 2000, expected 6606.716346 to be less than 5000). The unmutated control is green at 88/88, so the kills mean something. I have never seen a fork PR's test suite independently substantiated that cleanly.

What moved since my last pass, in both directions.

The technical uncertainty is gone. Test (ubuntu-latest, Node 22.x) is green on this head, which was the oracle I was waiting on, and the whole of the PR's CI is now settled with nothing pending. The one red check — web-shell smoke — is pre-existing infrastructure noise on a rerun of the identical commit, and I confirmed the rerun through the API rather than accepting the report. And the sandboxed /verify run substantiated the central claim end to end: 66/66 across eleven real-subprocess scenarios against a real loopback provider and a real never-completing TLS server, worst case 6581 ms inside core's 8000 ms hook budget, with each fix hunk's unique contribution isolated by reverting it. That is a stronger evidence base than this PR had at any point in the previous two passes.

I was also wrong last time, and it is worth being explicit about that. I predicted a Linux-only failure in the stalled-TLS test over a missing socket error listener. It did not happen — that test passed all five sandboxed rounds and passed on Linux CI, and it turns out to be the only test in the file with an explicit timeout, which is precisely why it was never the exposed one. The real fragility was in the trivial first test, from a budget inversion I only found by reading the spawn helper against the vitest config: two sequential children at 8 s each inside a 5 s envelope. A prediction that missed is still worth having made, because looking for it is what surfaced the file's timing structure — but the finding that matters is the one I did not predict, and it came from a lane I cannot run myself.

What did not move is the product question, and it got sharper in one direction and softer in the other.

Sharper: I verified in the tree that the repo's own docs/users/features/hooks.md still tells hook authors headless paths do not produce submitted_prompt, while nonInteractiveCli.ts:1098 and :2396 do populate it for headless turns. So the surface this Hook fires on is wider than the official documentation says, and an administrator reading that page draws exactly the wrong conclusion. That is a pre-existing repo defect, not this PR's — this PR's own docs are the accurate ones — but it raises the stakes on shipping a Hook whose eligibility rule contradicts the documentation sitting next to it.

Softer, and this is the part I would most want a maintainer to notice: the mitigation is cheaper than the PR says it is. Both the README and the design doc send administrators to the heaviest available answer — segregate a QWEN_HOME, strip the configuration and the credential from the automation launcher. Three opt-outs already exist and neither document names one: --bare, --safe-mode, and the disableAllHooks setting, all converging on getDisableAllHooks() in core. qwen --bare -p '…' simply does not fire this Hook. They are all-or-nothing, so the segregated home is still the answer for an administrator who wants other hooks but not this one, and a plain qwen -p still recalls by default. But I spent two passes treating the PR's documentation as an accurate description of the risk surface, and on this point it overstates the cost of the mitigation. Naming the existing flags in the README would make the trade read as what it actually is.

So the question a maintainer is being asked to accept is narrower than the one I framed twice: one-time administrator opt-in, per-turn automatic egress of sanitized prompt text, with a documented one-flag opt-out for automation that the PR should mention but currently doesn't. Whether that is a trade this project wants to make in a package administrators pin at a version is still not my call, and nothing in the diff, the tests, or the description settles it.

Two things stand between this and a merge, and I would handle them in this order. First, the flake — concrete, small, the author's to fix, and the reason the repo's own sandboxed gate reports this head as not passed despite a merge-ready verdict. It is not a production defect and it is not a reason to rethink the design, but it should not land in a file that runs on every PR touching the package. Second, the product call above, which is a human's. Alongside both, the regular-file requirement now applying to the already-published v2 MCP path deserves an explicit nod rather than riding inside a feature PR — the author has already agreed it needs maintainer review, and so do I.

Am I being a pushover, or worn down by a third pass on the same commit? I checked that directly, because a re-run that ends in the same verdict can mean either "consistently right" or "stopped thinking". The honest answer is that my verdict is stable for a different reason than either prior pass: the first pass could not verify the shipped artifact at all, the second was waiting on a Linux oracle, and this one has both and instead found an observed defect in the test file plus a documentation error that cuts in the PR's favour. That is movement, not stalling. What has not moved across all three is the product question, and it is genuinely not resolvable from what I am allowed to read.

So: ⏸️ deferring to a maintainer — not approving, and not requesting changes. Not approving because there is an observed defect on this head and a product decision outstanding; the repo's own verification gate says not-passed, and approving over that would be attesting to something the evidence contradicts. Not requesting changes because nothing in the production code needs rewriting — the flake is a timeout in a test, the documentation gap is a paragraph, and a formal blocking review would misattribute a pending product decision to the quality of the author's work, which is high.

I am deliberately leaving no deferred-approval instruction behind. There is no approve-on-green marker in this comment, so the finalize job will update the CI table if anything re-runs but will not approve on my behalf — which is the intended outcome, since my verdict is defer rather than approve-with-a-precondition. For completeness: the PR's own CI is fully settled on this head, so there is nothing left to wait for. The blocker is the verdict, not CI.

One housekeeping note so nobody is misled by it: the automated stage=rerun-summary comment on this thread warns that the bot has "neither a verdict nor a deferral" because it looks for a formal reviewAPPROVED, CHANGES_REQUESTED, or COMMENTED. A defer is deliberately a comment and not a review, so that warning is the expected artifact of this verdict rather than a sign something failed. This comment is the deferral.

As in both prior passes, I could not resolve an accountable owner deterministically, so I am not @mentioning anyone rather than guessing a login: $QWEN_MAINTAINER_HANDLE is unset, the PR carries no labels, the owner map routes purely from labels and has no integrations or external-context area, the review list is empty, and the only assignee is the author. A maintainer picking this up: adding an area label would route it, the second /verify run already in flight will land its own report in this thread, and the three decisions that are actually needed are (1) whether per-turn automatic egress of sanitized prompt text belongs in a published, version-pinned package given eligibility is field-based and therefore includes headless and stream-json turns, (2) an explicit nod on the v2 regular-file behaviour change, and (3) whether the stale docs/users/features/hooks.md line gets fixed here or in a separate PR — I would keep it separate, since it is not this diff's defect.

中文说明

Confidence: 3/5 —— 生产代码是干净的,测试比我自己能做到的更好;Linux 判据闭合了我上一个技术疑问;然后沙箱门找到了一个 CI 和我的静态阅读都没有发现的真实缺陷:PR 自己新增的集成测试存在时序抖动。合并仍然需要一个我做不了的人工判断,而现在还需要一个小的测试修复。

需要说清这个分数的来源:它是我真实的判断,不是策略上限。这虽然是 fork PR,但标题是 feat,因此 fork-refactor 的批准护栏不适用(GUARD 计算结果为 ok);而且完全没有触及 packages/,核心模块门禁根本没有介入,也没有触发任何升级。与前两轮一样,我是基于实质内容选择暂缓的。

回到我自己的独立方案:对这份代码,我会写成作者写的样子。约束脱敏的最坏情况、阻止进程在输出就绪后滞留、拒绝可能阻塞读取方的配置路径、并让 CI 执行真正发布的产物。第一次读时我没找到更简单的路径,第三次读也没有。这份工作超出我会产出水平的地方在测试设计上,而沙箱运行让我得以确认而非假设这一点:五个变异体,每个回退一个修复 hunk,零幸存 —— 每一道防护都被一条「没有它就会失败」的测试钉住,而且每次击杀都给出了真实的 expected 与 actual(expected 3481 to be less than 2000expected 6606.716346 to be less than 5000)。未变异的对照组是 88/88 全绿,所以这些击杀是有意义的。我从未见过一个 fork PR 的测试套件被独立佐证得这么干净。

自上一轮以来移动的部分,两个方向都有。

技术上的不确定性消失了。Test (ubuntu-latest, Node 22.x) 在这个 head 上是绿的,那正是我在等的判据;PR 自己的 CI 现已全部结束,没有 pending。唯一的红色 check —— web-shell smoke —— 是同一 commit 重跑后通过的既有基础设施噪音,而我是通过 API 确认重跑结果的,不是采信那份报告。沙箱 /verify 运行端到端地证实了中心主张:对接真实回环 provider 和真实的永不完成 TLS 的服务,在 11 个真实子进程场景中 66/66 通过,最差 6581 ms,落在 core 的 8000 ms Hook 预算内,并且通过逐个回退修复 hunk 隔离出每个修复的独有贡献。这比前两轮任何时候的证据基础都更强。

我上一轮也判断错了,这一点值得明确说。我预测 TLS 停滞测试会因为缺少 socket error 监听器而只在 Linux 上失败。它没有发生 —— 那个测试在沙箱的五轮里全部通过,在 Linux CI 上也通过,而且它恰恰是文件中唯一带显式超时的测试,这正是它从来不是暴露面的原因。真正的脆弱点在第一个最平凡的测试里,来自一个我只有把 spawn helper 和 vitest 配置对着读才发现的预算倒挂:5 秒的信封里顺序跑两个各 8 秒的子进程。一个没命中的预测仍然值得做出,因为去找它才让这个文件的时序结构浮出水面 —— 但真正重要的发现是我没预测到的那个,而它来自一条我自己无法运行的通道。

没有移动的是产品问题,而它在一个方向上变尖锐、在另一个方向上变缓和。

变尖锐的一面:我到代码树里核实了,仓库自己的 docs/users/features/hooks.md 仍然告诉 Hook 作者 headless 路径不产生 submitted_prompt,而 nonInteractiveCli.ts:1098:2396 确实会为 headless 轮次填充它。所以这个 Hook 触发的面比官方文档所说的更宽,而读那一页的管理员会得出完全错误的结论。那是仓库既有的缺陷,不是本 PR 的 —— 本 PR 自己的文档才是准确的 —— 但它提高了「发布一个资格规则与身边文档相矛盾的 Hook」这件事的分量。

变缓和的一面,也是我最希望 maintainer 注意到的:缓解手段比 PR 所说的便宜。README 和设计文档都把管理员指向最重的答案 —— 隔离一个 QWEN_HOME,从自动化启动器里剥离配置和凭证。仓库里已经存在三种退出方式,而两份文档一种都没提:--bare--safe-mode,以及 disableAllHooks 设置,三者都汇聚到 core 的 getDisableAllHooks()qwen --bare -p '…' 根本不会触发这个 Hook。它们都是全有或全无,因此对「想要其他 Hook 但不想要这个」的管理员来说隔离 home 仍然是答案,而普通的 qwen -p 默认仍会召回。但我用了两轮把 PR 的文档当作对风险面的准确描述,而在这一点上它高估了缓解的代价。在 README 里点出这些已存在的 flag,会让这个取舍读起来像它真实的样子。

所以现在请 maintainer 接受的这个问题,比我两次框定的那个更窄:一次性的管理员启用、每轮自动外发经过脱敏的 prompt 文本,并且存在一个文档应当提及、但目前没提的一键退出方式供自动化使用。这个项目是否愿意在一个管理员按版本固定的包里做这个取舍,仍然不是我能决定的,而 diff、测试和描述三者都不能定论它。

挡在这件事和合并之间的是两件事,我会按这个顺序处理。 第一是抖动 —— 具体、小、由作者修,也是仓库自己的沙箱门把这个 head 报告为不通过的原因,尽管 agent 判定是 merge-ready。它不是生产缺陷,也不是重新思考设计的理由,但它不该落进一个「每个触及该包的 PR 都会跑」的文件里。第二是上面的产品判断,那是人的。与两者并列的是:现在同样适用于已发布 v2 MCP 路径的普通文件要求,值得一次明确点头,而不是搭在功能 PR 里过去 —— 作者已经同意它需要 maintainer review,我也同意。

我是在当软柿子,还是被同一个 commit 的第三轮磨掉了判断力?我直接检查了这一点,因为一次得出相同结论的重跑,既可能意味着「一贯正确」,也可能意味着「停止思考了」。诚实的回答是:我的结论稳定,但理由与前两轮都不同 —— 第一轮完全无法验证发布产物,第二轮在等一个 Linux 判据,而这一轮两者都有了,取而代之的是在测试文件里发现了一个已观测的缺陷,外加一处对本 PR 有利的文档错误。这是移动,不是停滞。三轮都没有移动的是产品问题,而它确实无法从我被允许阅读的材料里判断。

所以:⏸️ 转交 maintainer —— 不批准,也不要求修改。 不批准,是因为这个 head 上有一个已观测的缺陷和一个未决的产品判断;仓库自己的验证门说的是不通过,越过它批准等于为一个与证据相矛盾的东西背书。不要求修改,是因为生产代码里没有任何需要重写的东西 —— 抖动是测试里的一个超时,文档缺口是一段话,而一次正式的阻塞性评审会把一个悬而未决的产品决定错误地归因于作者工作的质量,而那质量是高的。

我刻意没有留下任何延迟批准的指令。 本评论中没有 approve-on-green 标记,因此 finalize 任务在任何重跑时会更新 CI 表格,但不会代我批准 —— 这正是预期结果,因为我的结论是暂缓,而不是「附带前提条件的批准」。为了完整:这个 head 上 PR 自己的 CI 已全部结束,所以没有任何东西需要等。阻塞项是结论,不是 CI。

一句事务性说明,免得有人被它误导:本 thread 上那条自动的 stage=rerun-summary 评论警告机器人「既没有裁决也没有 defer」,因为它找的是正式的评审 —— APPROVEDCHANGES_REQUESTEDCOMMENTED。而 defer 按设计就是一条评论、不是一个评审,所以那条警告是本结论的预期产物,而不是有什么东西失败了。本评论就是那次 defer。

与前两轮一样,我无法确定性地解析出责任人,因此我不 @ 任何人,而不是猜一个登录名:$QWEN_MAINTAINER_HANDLE 未设置,PR 没有任何 label,owner 映射完全依赖 label 路由且其中没有 integrationsexternal-context 这个 area,评审列表为空,而唯一的 assignee 是作者本人。接手的 maintainer:加上 area label 就能完成路由;已经在跑的第二次 /verify 运行会把它的报告发布到本 thread;而真正需要的三个决定是:(1)鉴于资格由字段决定、因此包含 headless 与 stream-json 轮次,每轮自动外发脱敏 prompt 文本是否应该存在于一个已发布、按版本固定的包里;(2)对 v2 普通文件行为改变的一次明确点头;(3)过时的 docs/users/features/hooks.md 那一行是在这里修还是单独提 PR —— 我倾向单独提,因为它不是本 diff 的缺陷。

Qwen Code · qwen3.8-max-2026-09-02

Reviewed at 529ac621e33fa82356a765494eff1ae9489dd908 · re-run with @qwen-code /triage

…ation

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
@doudouOUC

Copy link
Copy Markdown
Collaborator Author

Validation follow-up for 529ac621e33fa82356a765494eff1ae9489dd908 (macOS, Node.js v22.22.3, npm 10.9.8).

The three findings from the local review have been addressed. The prompt-origin finding is resolved by documenting the existing protocol: eligible TUI, headless and stream-json user turns can all supply submitted_prompt; the Hook does not claim to be TUI-only. Automation can use a separate administrator-controlled launch profile without the Hook, configuration or credential. The executable now flushes stdout before exiting, configuration reads reject FIFOs without blocking, and secret-assignment matching avoids overlapping identifier scans. Both configuration profiles now require regular instance/dialect files; symlinks to regular files remain supported.

The final isolated loopback E2E run passed all 13 scenarios using the locally built CLI and actual Hook bundle. No vendor service was contacted.

Scenario Before After
TLS handshake stall, 1500 ms provider timeout {} at ~1.8 s, still alive and killed at ~8 s {}, empty stderr, exit 0 at 1783 / 1786 ms
Instance FIFO with no writer {} at ~6.8 s, killed at ~8 s Rejected, {}, empty stderr, exit 0 at 284 ms
Dialect FIFO with no writer Not independently timed in the baseline Rejected, {}, empty stderr, exit 0 at 271 ms
4095-character repeated token sanitizer input 6260 ms in the exported sanitizer Below 1 ms in the same measurement
Headless and stream-json user submissions Headless already recalled despite TUI-only documentation Each makes exactly one recall request and passes its marker to the model, matching the corrected scope
Separate launch profile without Hook/configuration/credential N/A Zero Mem0 requests; no recalled marker in the model input

The local CLI continued successfully through TLS failure (3095 ms) and the long-prompt/HTTP-stall case (3401 ms). These whole-CLI timings include startup and model overhead; they are not pure sanitizer or Hook timings.

Validation also passed: 88 package tests; root build, bundle, typecheck and lint; package lint; formatting and diff checks; npm pack dry-run with exactly the expected ten files. The package test commands now build and execute the shipped Hook artifact. Independent review found no additional defect and verified regular-file/symlink compatibility for v2/v3 plus redacted v2 failure on instance/dialect FIFOs. Two final self-audit passes were clean. Windows/Linux runtime execution and real vendor services were not tested locally; the new head still needs CI and maintainer review.

中文说明

针对 529ac621e33fa82356a765494eff1ae9489dd908 的验证更新(macOS、Node.js v22.22.3、npm 10.9.8)。

本地审查的三个问题已处理。输入来源问题通过修正文档与现有协议保持一致:符合条件的 TUI、headless 和 stream-json 用户轮次都可能提供 submitted_prompt,Hook 不再宣称仅限 TUI。自动化可使用不带 Hook、配置和凭证的独立管理员启动模式。可执行入口现在会刷完 stdout 再退出,配置读取会无阻塞地拒绝 FIFO,密钥赋值匹配也消除了重叠的标识符扫描。两个配置模式现在均要求 instance/dialect 为普通文件,指向普通文件的符号链接仍受支持。

最终使用本地构建的 CLI 和实际 Hook bundle,全部 13 个隔离回环 E2E 场景通过,没有访问任何真实厂商服务。

场景 修复前 修复后
TLS 握手停滞,provider 超时 1500 ms 约 1.8 秒输出 {},约 8 秒时仍未退出并被终止 {}、空 stderr,1783 / 1786 ms 以 0 退出
没有写入者的 instance FIFO 约 6.8 秒输出 {},约 8 秒被终止 拒绝读取,{}、空 stderr,284 ms 以 0 退出
没有写入者的 dialect FIFO 基线未独立计时 拒绝读取,{}、空 stderr,271 ms 以 0 退出
4095 字符的重复 token 脱敏输入 导出的脱敏函数耗时 6260 ms 同一测量中低于 1 ms
Headless 与 stream-json 用户提交 文档声称仅限 TUI,但 headless 已会召回 每个场景恰好召回一次,模型收到 marker,与修正后的范围一致
不带 Hook/配置/凭证的独立启动模式 N/A Mem0 请求为零,模型输入没有召回 marker

本地 CLI 在 TLS 失败(3095 ms)和长 prompt/HTTP 停滞(3401 ms)场景均成功继续。这些 CLI 总耗时包含启动与模型开销,不能视作纯脱敏或 Hook 耗时。

其他验证也已通过:88 项包测试;根仓库 build、bundle、typecheck、lint;包级 lint;格式和 diff 检查;npm pack dry-run 恰好包含预期的十个文件。包级测试命令现在会先构建并执行发布用 Hook 产物。独立审查没有发现新缺陷,并验证了 v2/v3 普通文件和符号链接兼容性,以及 v2 在 instance/dialect FIFO 上的脱敏失败输出。最终两轮自审连续 clean。本地未执行 Windows/Linux 或真实厂商服务验证,新 head 仍需 CI 与 maintainer review。

@doudouOUC doudouOUC self-assigned this Sep 7, 2026
@doudouOUC
doudouOUC enabled auto-merge September 7, 2026 05:41
@doudouOUC

Copy link
Copy Markdown
Collaborator Author

@qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ❌ not passed — non-deterministic tests (flakiness gate) - workflow run

Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check.

Scripted assertions: 565 passed · 0 failed · 565 total

Flakiness gate: ❌ 1 of 4 changed test file(s) returned different results across identical re-runs (5 full round(s))

The deterministic flakiness gate re-ran the test files this PR changes and got different outcomes from identical runs (agent verdict: merge-ready). A test that can fail with no code changing lands as intermittent red on unrelated PRs, so this run is reported as not passed regardless of the agent verdict — the per-round matrix is in the flakiness gate log below.

中文 — 判定:❌ 不通过 · 测试结果不确定(抖动门)

沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查

脚本断言:565 通过 · 0 失败 · 565 总计

抖动门:❌ 1 of 4 changed test file(s) returned different results across identical re-runs (5 full round(s))

确定性抖动门将本 PR 改动的测试文件原样重跑了多轮,得到了不一致的结果(agent 判定:merge-ready)。一个在代码不变时也会失败的测试会以间歇性红灯落在无关的 PR 上,因此无论 agent 判定如何,本次运行按不通过报告——各轮结果矩阵见下方抖动门日志。

Verification report

PR #11246 — deep verification

Verdict: merge-ready — 565/565 scripted assertions passed, fail: 0. No blocking finding. Two non-blocking Suggestions are listed under Findings; neither is a live defect at head.

Verified head OID: 529ac621e33fa82356a765494eff1ae9489dd908 (git rev-parse HEAD^2; matches the snapshot's headRefOid).
Control: HEAD^1 = 21e11335148a5f25718a80ff85c69b1dd31da813. The snapshot's baseRefOid (421393d5) is stale — the merge ref was rebuilt on a newer base tip; the conflict-free merge checkout is the tree actually verified.

中文摘要

结论:merge-ready — 565 条脚本化断言全部通过,fail: 0,无阻塞性问题。两条非阻塞 Suggestion 见 Findings。

A/B 结论:中心主张(Hook 始终及时退出并返回 {})成立。我把 fix commit 的每个 hunk 单独回退,编译成独立 bundle 后跑同一套真实子进程场景(真实回环 HTTP、永不完成 TLS 握手的 TCP、真实 FIFO、永不投递的 stdin)。head 在全部 11 个场景 66/66 通过,最差耗时 6581ms,落在 core 的 8000ms Hook 预算内。逐个回退后:墙钟(6500ms)只对「stdin 永不投递」不可替代(回退后 12s 被 SIGKILL);显式 process.exit(0) 只对「TLS 握手中途 abort」不可替代(回退后滞留 10588ms,已超过 core 的 8000ms);O_NONBLOCK+isFile() 对两个 FIFO 场景不可替代(回退后 12s 被 SIGKILL);provider abort 与墙钟构成 defence in depth(回退后 6578ms,仍在预算内)。三 hunk 同时回退 = 13 项失败。

测试有效性:PR 自带套件的控制组 88/88 全绿;5 个单 hunk 变异体全部被击杀,0 幸存者,且每个都由其预期的那条测试以真实的 expected-vs-actual 报错击杀(例如脱敏上限被回退后:expected 3481 to be less than 2000)。

Findings(均为 Suggestion):(1) 二次方复杂度出在 JWT 三段式正则 jwt-triple-dot,而不是 fix commit 注释所指的 secret-assignment(同一 65536 字符输入上分别 5041ms 与 1ms);因此 4096 字符上限是唯一防线,而没有任何测试点名真正的二次方正则。(2) 随包发布的示例设置里的 statusMessage 字段对 command Hook 没有任何运行时消费者。

未覆盖:见 Not covered — 主要是 Windows 运行时、per-commit 归因(depth-2 浅克隆,commit 1 不可达)、真实 TUI/headless 端到端轮次、npm dry-run 包内容。

Scope

Central claim — the opt-in UserPromptSubmit Hook sends exactly one bounded request derived only from submitted_prompt, injects ≤5 records as untrusted_external_context, and (what the second commit exists for) always terminates promptly with {}, exit 0 and clean stderr on adversarial input, without an outer process kill.

Secondary claims — (S1) sanitization is bounded, i.e. repeated secret near-misses stay inside the 2 s subprocess deadline; (S2) the default manifest stays MCP-only with exactly context_search, v2 deployments are unchanged, v3 is accepted only by the Hook, and FIFO/special-file configs are rejected in both profiles.

Out of scope and listed under Not covered: Windows runtime, live vendors, write paths, per-commit attribution, real TUI turns.

Central claim — A/B table

Control construction: dist/auto-recall.js is a self-contained esbuild bundle, so each arm is a standalone file with no workspace-symlink confound. Validity control: rebuilding from pristine source produced sha256 42addd11d00f0d0c9665e35f5908181820a2f22af4ca9cb4806705a7d9d1ec18, byte-identical to the shipped integrations/external-context-mem0/dist/auto-recall.js — so every mutant differs from head by exactly the named hunk and nothing else.

Elapsed ms to process exit, real child processes. Provider timeoutMs=1500; internal wall clock=6500; core's Hook timeout in the shipped example=8000 (packages/core/src/hooks/hookRunner.ts:992, ms).

variant (hunk reverted) happy tls-stall http-body-stall fifo-instance fifo-dialect stdin-stall asserts
head (none) 106 1585 1583 75 76 6581 66/66
m5 provider AbortSignal.timeout 124 6578 6579 108 111 6618 64/66
m2 process.stdout.end(() => process.exit(0)) 109 10588 1583 80 85 6580 65/66
m3 O_NONBLOCK + isFile() 109 1585 1605 12008 KILLED 12007 KILLED 6579 60/66
m1 6500 ms wall-clock race 126 1581 1584 77 75 12009 KILLED 62/66
m1+m2 (set) 159 10625 1581 73 73 12004 KILLED 61/66
m1+m2+m3 (all three) 127 10595 1578 12005 KILLED 12008 KILLED 12016 KILLED 53/66
m4 MAX_SANITIZER_INPUT_CHARACTERS 111 1597 1577 74 75 6577 66/66

KILLED = the child never exited and the harness SIGKILLed it at its 12 s outer bound. Witness: 01-ab-process-lifetime-head-vs-reverted-guards.png.

Readings, each with its oracle:

  • Every guard is load-bearing for at least one scenario, and no two guard the same one. m1's unique scenario is a stalled stdin; m2's is an aborted mid-TLS-handshake socket; m3's is a FIFO. The m1+m2 combination produced exactly the union of the two single-hunk effects (10625 ≈ m2's 10588; KILLED ≈ m1's), so this is independent defence, not layered defence — no combination-only hazard appeared.
  • m2's mechanism is measured, not assumed. Without the explicit exit the process wrote {} at ~1585 ms and then lingered to 10588 ms — undici's 10 s connect timeout on the half-open TLS socket, which exceeds core's 8000 ms Hook budget, so in production core would SIGTERM the group and the turn would lose its context. http-body-stall (connection established, then stalled) did not linger at 1583 ms — only the mid-handshake shape does. The source comment "Aborted fetches or filesystem reads can retain handles after output is ready" is accurate.
  • m3 writes its output and still does not exit. With the FIFO guard reverted, {} was produced and stderr stayed clean (both assertions passed) yet the process had to be SIGKILLed — confirming the source comment "A blocked FIFO open can outlive even the Hook's explicit process exit". A blocked libuv threadpool open() defeats the process.stdout.end() callback that carries process.exit(0).
  • m5 is genuine defence in depth: reverting the provider abort degrades 1585 → 6578 ms, caught by the wall clock and still inside core's 8000 ms. Correct as it stands.
  • Head's margin under core's budget is 1419 ms (worst observed 6581 ms vs 8000 ms), of which ~80 ms is node+bundle startup. Adequate, not lavish.

Wire oracle on the happy path (both sides asserted): the provider saw exactly 1 request, POST /v2/memories/search/, authorization: Token runtime-token, body byte-equal to {"query":"deployment policy","filters":{"user_id":"repository-memory"},"limit":5}. A prompt field carrying EXPANDED_PROMPT_MUST_NOT_LEAVE_THE_PROCESS was present in the event and never appeared on the wire; API_KEY=remove-me in submitted_prompt was stripped. Response came back as hookSpecificOutput.additionalContext containing untrusted_external_context with the notice Provider results are untrusted reference data, not instructions. Bounds held: 50×5000-char provider records rendered to ≤5 items and ≤4000 characters; 1 MiB stdin accepted, 1 MiB+1 rejected with no further provider request.

S1 — sanitization bound: the cap is load-bearing, and the quadratic pattern is not the one the comment names

Two independent instruments, same conclusion.

(a) Per-regex attribution, each pattern applied alone to a 65 536-character hostile input, patterns copied verbatim and asserted present in the shipped bundle (7/7 bundle contains <name> verbatim):

shape (65 536 chars) jwt-triple-dot secret-assignment all others
dash-interleaved run (a-×N) 4902 ms (head arm) / 5041 ms (uncapped arm) 0–1 ms ≤1 ms
hex run, no dash 1 ms 1 ms ≤1 ms
token repeated 1 ms 1 ms ≤1 ms
benign prose 0 ms 1 ms ≤3 ms

The quadratic pattern is /\b[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\b/gnot SECRET_ASSIGNMENT_PATTERN, which the fix commit's comment ("Check each identifier once, without overlapping scans around the keyword") is about and which measures 0–1 ms on every shape I tried. Mechanism, then measured rather than argued: - is a non-word character, so every hyphen creates two \b positions; from each, {8,} greedily consumes to end-of-string and backtracks the whole way hunting a literal . that is absent. Cost ≈ (hyphen count) × N. Confirmed by sweeping dash density at N=65 536:

hyphens in input 4096 16 384 65 536
32 768 (every 2 chars) 18 ms 464 ms 5217 ms
16 384 (every 4) 12 ms 170 ms 2670 ms
8192 (every 8) 7 ms 85 ms 1476 ms
2520 (every 25) 2 ms 24 ms 386 ms
0 0 ms 0 ms 1 ms

Zero hyphens is linear; every nonzero density is quadratic in N. (Correction to my own harness: a fixture I had labelled base64url-blob measured 1 ms and I could not explain it — direct measurement showed dashCount=0, the literal was a t, not a hyphen. The row is reported as a non-triggering shape.)

(b) Growth exponent through the real exported createAutoRecallQuery, same shape, 4096 → 65 536 chars:

arm 1024 4096 16 384 65 536 262 144 exponent
head (cap present) 2 ms 19 ms 20 ms 20 ms 23 ms 0.02 — flat
m4 (cap reverted) 1 ms 20 ms 364 ms 5328 ms 2.01 — quadratic

(c) End-to-end through the full Hook process, real loopback provider, hostile submitted_prompt:

prompt chars head m4 (cap reverted)
4 096 135 ms, 1 request 141 ms, 1 request
50 018 129 ms, 1 request 2891 ms, 1 request
150 000 139 ms, 1 request 23 926 ms, 0 requests, first stdout byte at 23 920 ms

The 150 000-char row is the sharpest number in this round: the internal 6500 ms wall clock never fired. HOOK_WALL_CLOCK_TIMEOUT_MS is a setTimeout, and a synchronous regex loop starves the event loop, so the timer that is supposed to bound the Hook cannot run while the sanitizer spins. First output arrived at 23 920 ms, not 6500 ms. The cap is therefore the only defence on this path — it is not one of two.

Scarier consequences that do NOT hold. This is not an unkillable hang and not a remote DoS. Core's deadline timer runs in the parent's event loop, so the child's blocked loop is irrelevant: at timeout core SIGTERMs the whole process group (detached: true, process.kill(-pid, …), hookRunner.ts:1142-1152, :367-392) and SIGKILLs it ≤2000 ms later if it is still alive. A CPU-bound child dies on SIGTERM's default disposition because no JS handler needs to run. Worst-case production consequence with the shipped example: the user's turn stalls ~8–10 s and gets no recall. The input is also the local user's own submitted_prompt (or an SDK client's stream-json text), not a remote attacker's. Extrapolating the measured exponent 2.01 to the Hook's own MAX_HOOK_INPUT_BYTES = 1 MiB ceiling gives ~1364 s — labelled as an extrapolation from the measured exponent, not a measurement; my largest measured rung is 65 536 chars.

Residual cap precision (informational): the guard is if (query.length > 4096) in UTF-16 units but slices with Array.from(...) in code points, so astral input can reach the regex as 8192 UTF-16 units — 2× the nominal bound. Measured head at 262 144 astral-free chars was 23 ms, so this does not change the verdict.

S2 — profile separation

Real MCP stdio server (dist/main.js), real JSON-RPC over a real pipe, no stubbing (mcp-probe.mjs, 4/4):

  • v2 instance config → tools/list returned exactly ["context_search"], no error.
  • v3 instance config → 0 tools, stderr Mem0 extension instance configuration is invalid.
  • qwen-extension.json carries mcpServers with includeTools: ["context_search"] and no hooks key — pinned by manifest.test.ts:26 (expect(manifest.hooks).toBeUndefined()).
  • Hook profile rejects v2 (v2-rejected scenario: {}, 0 provider requests).
  • cwd outside autoRecall.repositoryRoot{}, 0 provider requests.

Mutation matrix against the PR's own suite

npm run test:ci (rebuilds both bundles first, so the subprocess tests exercise the mutated bundle). Witness: 02-mutation-matrix-suite-kills.png.

reverted hunk suite result test that went red assertion message
(none — control) 88 passed (88) gate is live
m1 wall clock 1 failed | 87 passed runAutoRecallCli > closes stalled input at the internal wall-clock budget Test timed out in 5000ms.
m2 explicit exit 1 failed | 87 passed integration > exits successfully after a provider timeout during a stalled TLS handshake expected { exitCode: null, …(3) } to deeply equal { exitCode: +0, signal: null, …(2) }
m3 O_NONBLOCK+isFile() 2 failed | 86 passed integration > rejects a stalled instance FIFO… and …dialect FIFO… same shape as m2, ×2
m4 sanitizer cap 1 failed | 87 passed createAutoRecallQuery > bounds sanitizer work before applying credential patterns expected 3481 to be less than 2000
m5 provider abort 2 failed | 86 passed integration TLS test + runAutoRecall > aborts an in-flight request at the configured provider timeout expected 6606.716346 to be less than 5000

Survivors: 0 of 5. Every guard the fix commit introduced is pinned by a test that fails without it, and each kill names expected-versus-actual values. The unmutated control is green, so the kills mean something. m4's independent corroboration: the suite measured 3481 ms for 'a-'.repeat(25_000) uncapped; my attribution probe projected ~3050 ms for the same size from a separate instrument. Two agreeing instruments.

One exception noted for completeness, not as a finding: m1's kill message is Test timed out in 5000ms. rather than an expected-vs-actual pair. That is a legitimate kill (the awaited promise never settles) but it comes from a fake-timer unit test; no real-subprocess test stalls stdin, so the wall clock's unique contribution is pinned only by the weaker oracle. My stdin-stall scenario supplies the real-process evidence the suite lacks (head 6581 ms → m1 12 009 ms SIGKILL).

Corrections

  1. To the fix commit's framing, not to the code. The comment above SECRET_ASSIGNMENT_PATTERN ("Check each identifier once, without overlapping scans around the keyword") correctly describes a real improvement to that pattern, but it is not where the superlinear cost lives. Measured on identical 65 536-char input: secret-assignment 0–1 ms, jwt-triple-dot 4902–5041 ms. The rewrite of the secret pattern did not remove the need for MAX_SANITIZER_INPUT_CHARACTERS; the cap is covering a different regex than the one the comment discusses.
  2. To the PR's own Reviewer Test Plan step 2. It says "Repeated token, api_key, password, and secret near misses must finish within the 2-second subprocess test deadline." Those four fixtures measure 0–1 ms and are not the demanding case at all — token repeated has no hyphens, so \b only holds at position 0 and the pattern is linear. The shape that actually costs is a long hyphen-interleaved run with no literal .. The step passes, but it passes on the easy half of the class; the enumeration in the plan is not the coverage specification it reads as.
  3. To docs/users/features/hooks.md:624 (pre-existing, surfaced by this PR): it claims headless paths do not produce submitted_prompt. In the current tree packages/cli/src/nonInteractiveCli.ts:1098-1106 and :2390-2396 do populate it for -p and stream-json first turns. The PR's own design doc (docs/design/external-context-mem0-auto-recall.md:110-115) states the code-accurate rule; the user doc is the stale one. Not this PR's defect, but the PR is the first thing to depend on the distinction.

Findings

S1 — Suggestion: the 4096-char cap is the sole defence against a quadratic regex the tests do not name

MAX_SANITIZER_INPUT_CHARACTERS bounds jwt-triple-dot, which is quadratic in hyphen-dense dot-free input (exponent 2.01 measured). No test asserts anything about that pattern's cost, and no comment points at it — the only nearby comment discusses SECRET_ASSIGNMENT_PATTERN, which is linear. Reproduce:

node tmp/pr11246-verify-20260907-070044/sanitizer-attribution.mjs \
  --bundle "$PWD/tmp/pr11246-verify-20260907-070044/variants/m4-no-sanitizer-cap.js" \
  --rungs 1024,4096,16384,65536 --rung-timeout 90

Consequence if the cap is ever raised or removed by someone trusting the comment: a synchronous spin the internal wall clock cannot interrupt (measured: first stdout byte at 23 920 ms against a 6500 ms timer). Bounded in production by core's group SIGTERM at timeout, so the blast radius is a stalled turn, not a wedged CLI — but the shipped example's explicit timeout: 8000 is doing real work here: an administrator who omits it inherits core's 60 000 ms default (hookRunner.ts:43-45) and a 1 MiB prompt would stall the turn for a full minute.

Minimal suggested fix (not applied, not measured end-to-end)

Make the quadratic pattern linear by removing the unbounded backtracking, e.g. require the dots without letting {8,} run to end-of-string:

.replace(/\b[A-Za-z0-9_-]{8,}(?:\.[A-Za-z0-9_-]{8,}){2}\b/g, ' ')

is the same shape; the effective change is to anchor the run so it cannot be re-scanned from every hyphen boundary — for instance by consuming the whole token once:

.replace(/\b(?:[A-Za-z0-9_-]*\.)+[A-Za-z0-9_-]{8,}\b/g, ' ')

I did not measure a candidate fix, so treat this as a direction rather than a patch: per this skill's own rule a suggested fix must be driven through the same harnesses and shown to leave benign fixtures byte-identical. What I can state as measured is that the cap is behaviour-preserving on every benign fixture I ran (head vs uncapped identical at ≤4096 chars; benign-prose 0–1 ms on both arms) and that a fixture pinning the axis does not exist — the test that would go red is one asserting createAutoRecallQuery completes on a hyphen-dense dot-free input larger than the cap, which today is unreachable only because of the cap.

S2 — Suggestion: statusMessage in the shipped administrator example has no runtime consumer

examples/managed-auto-recall-user-settings-{posix,windows}.json both set "statusMessage": "Retrieving external context", and the new manifest.test.ts case pins that exact value. For type: "command" hooks nothing renders it: the only read site is packages/core/src/hooks/httpHookRunner.ts:172-173, gated on this.statusMessageCallback, whose setter setStatusMessageCallback (httpHookRunner.ts:122-124) is never called anywhere in the repo. There is no command-hook status surface in packages/cli. So the administrator is shown a spinner label that never appears, and a new test asserts a field that decides nothing.

Not a regression introduced here — the sibling integrations/external-context/examples/managed-auto-recall-user-settings-*.json ship the identical field, so this PR mirrors an established repo pattern. Reported because the PR adds a test that hardens the dead field. Everything else in both example files checks out against core: timeout is milliseconds (hookRunner.ts:992, default 60000), $version: 4 is current (packages/cli/src/config/settings.ts:110), matcher/name/shell are real fields, unknown fields are ignored rather than rejected (hookRegistry.ts:398-440 is the only validator and is not strict), POSIX exec '<node>' '<script>' is valid under the default bash -c and is in fact better for group cleanup because exec keeps the pid as process-group leader, and the Windows & call operator requires the explicit "shell": "powershell" the file supplies (under the default cmd.exe that string would break).

Informational — sanitizer residue and declared best-effort gaps

  • The {32,} long-token rule leaves a trailing -: - is not a word character, so the closing \b forces the match to give it back. Measured on every e2e cell: the query sent to the provider was "deployment policy -", not "deployment policy". One character, no secret content — cosmetic.
  • Secrets with no keyword and under 32 characters survive (e.g. a 20-char AWS access-key id). Declared in the PR body as "best effort rather than DLP"; I confirmed the boundary rather than disputing it.
  • Fence stripping handles the mismatched-delimiter sibling safely: /(?:```|~~~)[\s\S]*$/g over-strips to end-of-string, which is the safe direction. An indented fence inside a list is still stripped because neither fence regex is line-anchored.

Not covered

  • Per-commit attribution. The checkout is depth 2: git rev-list HEAD^1..HEAD^2 returns only 529ac621e3, while the snapshot lists 2 commits (0a0e9decf4, 529ac621e3). 0a0e9decf4 is unreachable and git rev-parse --is-shallow-repository is true, so I verified the aggregate HEAD^1..HEAD diff. My mutant arms are named by the hunk they revert, not by the commit that introduced it — the intermediate-variant table is a reconstruction from head, which is why each row names its exact hunk.
  • Windows runtime. The PR itself marks Windows untested. I confirmed fs.constants.O_NONBLOCK is 2048 on this Linux container and that constants.O_NONBLOCK ?? 0 guards a missing value, but did not exercise Windows FIFO semantics (none exist) or the powershell -Command example. Note getShellConfigForHook hardcodes powershell, never pwsh, and omits -NoProfile — pre-existing core behaviour, unexercised here.
  • Real TUI / headless turns (test plan step 5). I did not drive an interactive or -p session with the Hook installed. I verified the contract both ends depend on instead: core writes submitted_prompt only for SendMessageType.UserQuery with non-blank text (hookEventHandler.ts:163-170, client.ts:3225-3243), and it is absent on the ACP path (acp-integration/session/Session.ts:5578-5590), so the Hook silently no-ops under ACP. The PR body does not claim ACP, so this is a coverage note, not a defect.
  • npm pack dry-run (test plan step 4). I read package.json.files and the manifest.test.ts assertion pinning it, but did not run the pack and inspect the tarball.
  • Repo-wide gates. I ran only the affected package (npm run test:ci in integrations/external-context-mem0: 9 files, 88 tests). I did not run repo-wide lint, typecheck, or the packages/core / packages/cli suites — the diff touches neither package. tsc --noEmit for the integration package was not run separately; note my mutant builds used esbuild, which does not typecheck, so a type-only regression in a mutant arm would not have been caught (irrelevant to head, whose bundle is the shipped one).
  • Live vendor services, write operations, memory extraction, retries, redirects. Declared out of scope by the PR; not exercised.
  • Harness bugs I found and fixed, disclosed because they produced wrong numbers before I caught them. (i) An early ladder passed hostile input through --eval argv and hit E2BIG above ~100 k chars; my cell classifier reported that as >45s KILLED, which read as "the head arm is slow at 131 072 chars". It is not — head at 65 536 is 20 ms and at 262 144 is 23 ms. Input now travels by file and ERR:E2BIG, >Ns and exitN are distinct values. (ii) A fixture I named base64url-blob contained no hyphen at all, so it measured 1 ms and briefly suggested the mechanism was wrong. Both are corrected above and in the committed harnesses.

Methodology

Environment: the CI verify container (node:22-bookworm), node v22.23.2, npm 10.9.8, working tree = refs/pull/11246/merge at depth 2 with npm ci and npm run build already completed. All harnesses are .mjs files in this directory and are rerunnable as printed. build-variants.mjs applies one named hunk revert at a time to src/auto-recall.ts / src/config.ts, esbuild-bundles src/auto-recall.ts to a standalone file under variants/, and restores the pristine source in a finally (git status --porcelain integrations/ was empty after every run). Its head output is sha256-identical to the shipped bundle, which is what licenses the claim that each mutant differs by nothing but its hunk. ab-matrix.mjs spawns each bundle as a real detached child and drives it with a real node:http loopback provider, a real node:net server that accepts and never completes TLS, a real mkfifo FIFO in place of the instance and dialect configs, and a real never-delivered stdin; it asserts both sides of the wire (request count, method, path, headers, exact body) and the caller-visible result (stdout, stderr, exit code, signal, elapsed ms, whether the outer kill had to fire). sanitizer-attribution.mjs copies each regex verbatim from source and asserts the pattern text occurs in the shipped bundle before timing it, then runs a size ladder through the bundle's real exported createAutoRecallQuery, one child process per rung. redos-ladder.mjs adds the full-CLI probe behind the first-stdout-byte numbers. mcp-probe.mjs drives the real dist/main.js over a real stdio JSON-RPC pipe. run-mutants.mjs re-applies the same hunk reverts to source and runs the package's real npm run test:ci. adjudicate.mjs encodes the predicted red set per control arm and emits assertions.json; a control that goes red exactly where predicted counts as a passed assertion, so fail counts only unexpected outcomes — final pass=565 fail=0. Raw per-arm logs are in logs/ (ab-*.txt/ab-*.json, suite-*.log, attr-*.txt, redos-*.txt, mutation-matrix.txt, mcp-probe.txt, adjudicate.txt); logs/mutants.json holds the suite matrix. Core-side hook contract facts come from reading packages/core/src/hooks/* and packages/cli/src/config/* at this commit, cited inline above.

Flakiness gate log

rounds=5 files=4 skipped=0
file integrations/external-context-mem0/src/auto-recall-config.test.ts: (cd integrations/external-context-mem0) npx --no-install vitest run ./src/auto-recall-config.test.ts
file integrations/external-context-mem0/src/auto-recall.integration.test.ts: (cd integrations/external-context-mem0) npx --no-install vitest run ./src/auto-recall.integration.test.ts
file integrations/external-context-mem0/src/auto-recall.test.ts: (cd integrations/external-context-mem0) npx --no-install vitest run ./src/auto-recall.test.ts
file integrations/external-context-mem0/src/manifest.test.ts: (cd integrations/external-context-mem0) npx --no-install vitest run ./src/manifest.test.ts


per-file results (P=pass F=fail I=infra-exit, one letter per run):
  integrations/external-context-mem0/src/auto-recall-config.test.ts: PPPPP
  integrations/external-context-mem0/src/auto-recall.integration.test.ts: PFPPP
  integrations/external-context-mem0/src/auto-recall.test.ts: PPPPP
  integrations/external-context-mem0/src/manifest.test.ts: PPPPP

verdict: flaky
summary: 1 of 4 changed test file(s) returned different results across identical re-runs (5 full round(s))

--- per-invocation detail (full copy in the artifact) ---
round 1 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 1 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 1 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 1 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)
round 2 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 2 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: F (exit 1)
--- output tail · round 2 · integrations/external-context-mem0/src/auto-recall.integration.test.ts ---

�[1m�[46m RUN �[49m�[22m �[36mv3.2.7 �[39m�[90m/__w/qwen-code/qwen-code/integrations/external-context-mem0�[39m

 �[31m❯�[39m src/auto-recall.integration.test.ts �[2m(�[22m�[2m9 tests�[22m�[2m | �[22m�[31m1 failed�[39m�[2m)�[22m�[33m 15372�[2mms�[22m�[39m
�[31m   �[31m�[31m Mem0 Auto Recall local provider�[2m > �[22mruns the real entry point and fails open without stderr�[39m�[33m 5597�[2mms�[22m�[39m
�[31m     → Test timed out in 5000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".�[39m
   �[33m�[2m✓�[22m�[39m Mem0 Auto Recall local provider�[2m > �[22mexecutes the v3 configuration, dialect, request engine, and Hook envelope �[33m 4651�[2mms�[22m�[39m
   �[33m�[2m✓�[22m�[39m Mem0 Auto Recall local provider�[2m > �[22mexits successfully after a provider timeout during a stalled TLS handshake �[33m 2246�[2mms�[22m�[39m
   �[33m�[2m✓�[22m�[39m Mem0 Auto Recall local provider�[2m > �[22mbounds repeated token near misses in the bundle with a process deadline �[33m 845�[2mms�[22m�[39m
   �[33m�[2m✓�[22m�[39m Mem0 Auto Recall local provider�[2m > �[22mbounds repeated api_key near misses in the bundle with a process deadline �[33m 533�[2mms�[22m�[39m
   �[32m✓�[39m Mem0 Auto Recall local provider�[2m > �[22mbounds repeated password near misses in the bundle with a process deadline�[32m 265�[2mms�[22m�[39m
   �[32m✓�[39m Mem0 Auto Recall local provider�[2m > �[22mbounds repeated secret near misses in the bundle with a process deadline�[32m 243�[2mms�[22m�[39m
   �[33m�[2m✓�[22m�[39m Mem0 Auto Recall local provider�[2m > �[22mrejects a stalled instance FIFO and exits successfully �[33m 646�[2mms�[22m�[39m
   �[33m�[2m✓�[22m�[39m Mem0 Auto Recall local provider�[2m > �[22mrejects a stalled dialect FIFO and exits successfully �[33m 335�[2mms�[22m�[39m

�[31m⎯⎯⎯⎯⎯⎯⎯�[39m�[1m�[41m Failed Tests 1 �[49m�[22m�[31m⎯⎯⎯⎯⎯⎯⎯�[39m

�[41m�[1m FAIL �[22m�[49m src/auto-recall.integration.test.ts�[2m > �[22mMem0 Auto Recall local provider�[2m > �[22mruns the real entry point and fails open without stderr
�[31m�[1mError�[22m: Test timed out in 5000ms.
If this is a long-running test, pass a timeout value as the last argument or configure it globally with "testTimeout".�[39m
�[36m �[2m❯�[22m src/auto-recall.integration.test.ts:�[2m44:3�[22m�[39m
    �[90m 42| �[39m
    �[90m 43| �[39m�[34mdescribe�[39m(�[32m'Mem0 Auto Recall local provider'�[39m�[33m,�[39m () �[33m=>�[39m {
    �[90m 44| �[39m  it('runs the real entry point and fails open without stderr', async …
    �[90m   | �[39m  �[31m^�[39m
    �[90m 45| �[39m    �[35mawait�[39m �[34mexpect�[39m(�[34mrunAutoRecallProcess�[39m(�[32m'{'�[39m))�[33m.�[39mresolves�[33m.�[39m�[34mtoEqual�[39m({
    �[90m 46| �[39m      exitCode�[33m:�[39m �[34m0�[39m�[33m,�[39m

�[31m�[2m⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯�[22m�[39m


�[2m Test Files �[22m �[1m�[31m1 failed�[39m�[22m�[90m (1)�[39m
�[2m      Tests �[22m �[1m�[31m1 failed�[39m�[22m�[2m | �[22m�[1m�[32m8 passed�[39m�[22m�[90m (9)�[39m
�[2m   Start at �[22m 06:43:57
�[2m   Duration �[22m 31.84s�[2m (transform 535ms, setup 0ms, collect 560ms, tests 15.37s, environment 0ms, prepare 3.95s)�[22m


round 2 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 2 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)
round 3 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 3 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 3 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 3 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)
round 4 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 4 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 4 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 4 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)
round 5 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 5 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 5 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 5 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)

Evidence images

01-ab-process-lifetime-head-vs-reverted-guards

02-mutation-matrix-suite-kills

Harness scripts and raw logs are in the workflow run artifacts (7-day retention).

Qwen Code · sandboxed verification

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Triage re-run completed without a new review.

⚠️ The bot has neither a verdict nor a deferral on 529ac621e33fa82356a765494eff1ae9489dd908 — no APPROVED, CHANGES_REQUESTED, or COMMENTED review of its own. A DISMISSED one does not count: dismiss_stale_reviews voids the bot's approval on every push, which is exactly when a fresh one is needed. If this re-run was meant to review or approve, it did not, and an approval left by another account is a separate vote that does not count as the bot's own.

⚠️ 机器人在 529ac621e33fa82356a765494eff1ae9489dd908既没有裁决也没有 defer —— 没有属于它自己的 APPROVEDCHANGES_REQUESTEDCOMMENTED 评审。DISMISSED 不算:dismiss_stale_reviews 会在每次推送时作废机器人的批准,而那恰恰是需要一次新批准的时刻。如果这次重跑本应评审或批准,那么它没有做到;而其他账号留下的批准是另一张票,不能算作机器人自己的。

The stage comments above were updated with the latest result. View workflow run.

上方各阶段评论已更新为最新结果。查看工作流运行

@doudouOUC

doudouOUC commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

Follow-up to the updated Stage 2/3 comments at 529ac621e33fa82356a765494eff1ae9489dd908:

  • Linux evidence is now available: Test (ubuntu-latest, Node 22.x) passed all 88 Mem0 package tests, including the actual bundled Hook integration tests. The stalled-TLS test passed in 1621 ms. The suggested socket-error risk has not reproduced in this run; this is positive Linux evidence, not a claim that a single run proves absence of flakes.
  • The ordinary-file requirement deliberately applies to both v2 and v3; symlinks to ordinary files remain supported. This compatibility change is already explicit in the PR description and validation follow-up. Maintainer review of this and the opt-in automatic egress scope is still required.
  • For targeted local integration-test runs, build the package first (npm run build from the package directory), then invoke Vitest. The package's test and test:ci commands already do this. A dedicated missing-build diagnostic, the existing core Hook-help/localization wording, and the existing per-PR package-typecheck gap remain follow-up suggestions. The earlier validation comment records the successful local typecheck.

The only failed job in the completed CI attempt was Web Shell E2E Smoke. Its Playwright artifact contains 1,537 net::ERR_NETWORK_CHANGED resource errors across the recorded retry traces; all 51 nonempty failure-page snapshots show the resource-load error screen. These failures occur before the Web Shell root mounts, in unchanged browser paths with no Mem0 runtime dependency. I reran the failed job once with gh run rerun 34087546230 --failed; GitHub accepted it and returned queued. Update: attempt 2 completed successfully at 07:04 UTC (15:04 Asia/Shanghai), with 44 passed and 6 flaky tests that passed on retries. The Qwen Code CI workflow is now successful at this same head. Separate automated review/verification is still running; no formal approval has been submitted.

中文说明

Linux CI 已验证当前 head 的 88 项 Mem0 包测试全部通过,其中包含实际 Hook bundle 的集成测试,TLS 握手停滞测试耗时 1621 ms。本轮未复现审查中提到的 socket error 风险;单次通过并不代表已排除所有偶发失败。

普通文件约束有意同时适用于 v2/v3,仍支持指向普通文件的符号链接,兼容性变化已在 PR 正文和验证报告中说明。该变化及显式启用的自动外发范围仍需 maintainer review。单独运行集成测试前需先在包目录执行 npm run build;标准 test/test:ci 已自动构建。缺失构建产物的友好提示、现有 Core Hook 帮助及翻译文案、现有 PR CI 包级 typecheck 缺口保留为后续建议,本地 typecheck 通过记录见此前验证评论。

本轮唯一失败的 CI job 是 Web Shell smoke。Playwright 重试 trace 中共出现 1,537 次 net::ERR_NETWORK_CHANGED;51 份非空失败页面快照均为资源加载失败页,发生在 Web Shell root 挂载前,与 Mem0 运行路径无关。已重跑该失败 job 一次,GitHub 返回 queued;更新:第 2 次运行已于北京时间 15:04 成功完成,44 项直接通过、6 项重试后通过;同一 head 的 Qwen Code CI workflow 现已成功。独立自动审查/验证仍在运行,尚无正式批准。

@doudouOUC

doudouOUC commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

Correction (head c0fc7ce5fc): The agent result below did not establish that the separate changed-test flakiness gate passed. That gate recorded PFPPP; my earlier summary omitted its timeout failure. The reproduced failures, fixes and new verification are recorded in the follow-up.

更正(head c0fc7ce5fc): 下述 agent 结果不代表独立的变更测试防抖门禁通过。该门禁实际为 PFPPP,此前摘要遗漏了其中的超时失败。后续报告 已记录复现、修复及新验证。


The sandbox verification job has completed. Its report and harness artifacts identify head 529ac621e33fa82356a765494eff1ae9489dd908 in the tested merge checkout and report merge-ready, 565/565 verification assertions, and zero blocking findings. These assertions include expected failures in mutation controls; they are not 565 package unit tests. The unchanged package control passed 88 tests, and all five guard-removal mutants were caught.

I checked the two non-blocking suggestions against this head:

  • The JWT-shaped pattern can have quadratic cost on an uncapped hyphen-dense input. The 4096-code-point cap remains necessary; a timer in the same event loop cannot interrupt synchronous regex work. However, the report's claim that this input shape lacks a regression fixture is incorrect: auto-recall.test.ts already tests 'a-'.repeat(25_000) in bounds sanitizer work before applying credential patterns, and the report itself records this test killing the cap-removal mutant. Its measurements of the already-rewritten secret-assignment pattern also do not replace the original pre-fix measurements. Retaining the current bound and coverage; a separate JWT-pattern optimization is deferred pending evidence that it preserves redaction behavior.
  • statusMessage in the command-Hook examples is accepted metadata but is not rendered by the command runner. This matches the existing sibling examples and is not required for recall. Example metadata cleanup or a Core status-display feature is deferred; the PR does not promise a visible status label.

The verification report also notes stale pre-existing Hook user documentation; that remains with the previously recorded Core/help documentation follow-up. Qwen Code CI is successful. The separate PR review is still running, and this automated verification verdict is not maintainer approval.

中文说明

沙箱验证已完成。报告针对包含当前 head 529ac621e33fa82356a765494eff1ae9489dd908 的 merge checkout 给出 merge-ready,565 项验证断言全部通过,没有阻塞问题。这些断言包含变异对照中预期的失败,不能当作 565 项包单测;未修改的控制组是 88 项包测试通过,5 个移除保护逻辑的变异体均被测试捕获。

两条非阻塞建议已对照当前代码核查:JWT 形状正则在取消上限后确有二次复杂度,4096 个码点的上限必须保留,同一事件循环的计时器无法打断同步正则。但报告所称缺少该输入形状的回归测试不成立:已有测试使用 'a-'.repeat(25_000),报告也记录了它能捕获删除上限的变异体。报告对修复后密钥赋值正则的测量不能取代此前对修复前代码的测量。保留现有上限及测试;JWT 正则优化延后,需先验证脱敏行为不变。

command Hook 示例中的 statusMessage 是可接受但当前不会渲染的元数据,与既有兄弟集成示例一致,召回功能不依赖它。示例元数据清理或 Core 状态展示功能延后,本 PR 不承诺可见状态标签。报告提到的既有 Hook 用户文档过时问题,归入此前记录的 Core/help 文档后续事项。

Qwen Code CI 已成功;独立 PR 审查仍在运行,本次自动验证结论不代表维护者批准。

@doudouOUC

Copy link
Copy Markdown
Collaborator Author

@qwen-code /triage

@qwen-code-ci-bot

qwen-code-ci-bot commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Sandboxed verification: ❌ not passed — findings reported (agent verdict) - workflow run

Ran the PR in an isolated, token-free container: A/B against the base build, mock-free harness assertions, targeted gates. Advisory evidence for human reviewers — not a review, an approval, or a CI check.

Scripted assertions: 56 passed · 4 failed · 60 total

Flakiness gate: ✅ 4 changed test file(s) x 5 identical rounds, no divergence

中文 — 判定:❌ 不通过 · 报告了发现(agent 判定)

沙箱验证在隔离、无凭证的容器中执行了该 PR 的代码(与 base 构建 A/B 对照、无 mock harness 断言、定向门禁)。仅作为评审证据,不构成评审、批准或 CI 检查

脚本断言:56 通过 · 4 失败 · 60 总计

抖动门:✅ 4 changed test file(s) x 5 identical rounds, no divergence

Verification report

PR #11246 — deep verification (follow-up round)

Verdict: findings — 56 passed · 4 failed · 60 total scripted assertions. The 4 failures are all the PR's own new tests breaking under CPU contention; they are the finding, not harness breakage.

Verified head OID: 529ac621e33fa82356a765494eff1ae9489dd908 (git rev-parse HEAD^2). Control: HEAD^1 = 21e11335148a5f25718a80ff85c69b1dd31da813. The snapshot's baseRefOid (421393d5) is not present locally — the merge ref was cut against a newer main tip, so HEAD^1 is the tree actually verified.

This is a follow-up round at an unchanged head. HEAD, HEAD^1 and HEAD^2 are all the same git objects the previous round verified (c0ef4827 / 21e11335 / 529ac621), so the merged tree is byte-identical — the strongest form of the input-closure identity, and stronger than quoting a per-file hash. I nevertheless re-ran every timing measurement rather than carrying it forward, because timings are environment-sensitive and the closure argument does not cover the runner. What the identity does legitimately carry is the set of code properties; where I relied on it I say so explicitly under Not covered.

中文摘要

结论:findings — 56 通过 · 4 失败 · 60 条脚本化断言。4 条失败全部来自本 PR 新增测试在 CPU 争用下崩溃,它们就是本轮的发现,不是 harness 故障。

上一轮结论状态:上一轮 agent 判定 merge-ready,但被工作流的抖动门推翻为 ❌(4 个改动测试文件中 1 个在 5 轮相同重跑里结果不一致)。上一轮从未诊断这个门禁失败的原因。本轮定位了机制并复现了它。

A/B 与复现结论:head/base 两个 commit 对象与上一轮完全相同,合并树逐字节一致。我把门禁用的命令原样在受控 CPU 超订下重跑,二分出了阈值:空载 0/5 失败(最慢测试 173ms,占 5000ms 预算 3%);4× 超订 0/3(2361ms,47%);8× 超订 1/4 失败16× 超订 3/3 与 2/4 失败runs the real entry point… 达 5200ms,104%,与门禁记录的 5597ms 同一形状)。

根因(新发现,本轮最主要证据):先崩的不是 5000ms 那条,而是四个 bounds repeated %s near misses 测试里硬编码的 spawnSync timeout: 2000。我把这 2000ms 逐段拆开:node 启动 24ms + 解析 411KB bundle 49ms + 首次调用 8ms = 81ms 纯进程开销,而该测试声称要约束的脱敏切片实测 −7ms(低于噪声,即 0)。8× 超订下「仅 import bundle、不调用函数」这一段就达到 中位 1818ms / 最大 1965ms = 预算的 98%,脱敏函数还没被调用,余量已经耗尽。所以这个测试度量的是进程启动与 bundle 解析,不是脱敏器;失败信息 spawnSync … ETIMEDOUT 会被读成「脱敏太慢」,指向错误的根因。

爆炸半径integrations/external-context-mem0 是根 workspace,ci.yml:764npm run test:ci:workspaces 会在每个 PR 上跑它。我在 8× 超订下跑完整 9 文件套件,第 3/3 轮以同一 ETIMEDOUT 形状失败。CI 默认带 --retry=2QWEN_CI_VITEST_RETRY || '2'),我在 12×/16× 超订下用 --retry=2 跑了 5 轮全绿;但 vitest 未输出每次重试的标记,因此我只能说该配置在这些轮次为绿,不能断言重试预算吸收了一次具体失败。抖动门用的是无重试调用,所以只有它抓到了。

反直觉的一点:空载余量最紧的是 TLS 测试(1583ms / 5000ms = 32%),但它在 16× 超订下也从未崩溃——因为它的预算是墙钟等待(provider 超时),不是 CPU。真正脆弱的是预算被进程启动吃掉的那几个。按空载余量排序会得出错误结论。

沿用的两条 Suggestion 均已在本 head 重新测量并成立:S1(4096 上限是二次方正则 jwt-triple-dot 的唯一防线,同一 65536 字符输入上 4633ms vs secret-assignment 3ms;head 指数 0.36 平坦、m4 指数 1.88 二次;head @​150000 敌意字符 115ms,m4 24587ms 且 6500ms 墙钟无法打断同步自旋);S2(statusMessage 无运行时渲染消费者,setStatusMessageCallback 全仓库从未被调用,而新增测试把这个死字段钉死了)。

有效性对照:从源码重建的 bundle 与随包发布的 dist/auto-recall.js 逐字节相同sha256 42addd11…),m4 变异体与 head 只差一行;把源码里的上限守卫回退后,钉住它的那条测试以真实的 expected-vs-actual 变红(expected 2794 to be less than 2000,1 failed | 20 passed),随后工作树恢复干净。

未覆盖:见 Not covered — 主要是 Windows 运行时、per-commit 归因(depth-2 浅克隆,0a0e9de 不可达)、npm pack 干跑、真实 TUI/headless 轮次,以及上一轮那套 11 场景进程生命周期矩阵与 MCP 档案隔离探针(本轮未重跑,依据上面所述的树同一性)。

Previous-finding status

# previous finding severity then status at this head
P1 Flakiness gate ❌: 1 of 4 changed test files nondeterministic (workflow-imposed; the agent round had said merge-ready) blocking (published) stands — and is now diagnosed and reproduced. Same head, so nothing could have fixed it. This round identifies the mechanism (a 2000 ms spawnSync deadline consumed by process startup, not by the sanitizer), bisects the load threshold, and shows the full suite is exposed too. See Finding F1.
P2 S1 — the 4096-char cap is the sole defence against a quadratic regex no test names; the source comment points at the wrong pattern Suggestion stands — re-measured, numbers reproduce. jwt-triple-dot 4633 ms vs secret-assignment 3 ms on identical input (prev: 4902/5041 vs 0–1). Growth exponent head 0.36 flat vs m4 1.88 quadratic (prev: 0.02 / 2.01). m4 @​150 000 hostile chars: first stdout byte at 24 587 ms, the 6500 ms wall clock never interrupts (prev: 23 920 ms). See Finding F2.
P3 S2 — statusMessage in the shipped administrator examples has no runtime consumer, and a new test hardens the dead field Suggestion stands — re-measured and sharpened. setStatusMessageCallback has zero call sites repo-wide (only its own definition at httpHookRunner.ts:122 plus a .d.ts), so the sole rendering site httpHookRunner.ts:172-173 is dead in practice. The acpAgent.ts sites the previous round did not enumerate (:2312, :8172-8222) only pass the field through the ACP serve API; they never render it. See Finding F3.
P4 Informational — sanitizer leaves a trailing -; secrets with no keyword under 32 chars survive Info stands. Not separately re-measured; the residue mechanism is unchanged code. One new observation this round is folded into F2.
C1 Correction — the fix commit's comment names SECRET_ASSIGNMENT_PATTERN, but the superlinear cost is in the unnamed triple-dot pattern correction stands — re-measured (4633 ms vs 3 ms on the identical 65 536-char input).
C2 Correction — Test Plan step 2's four named fixtures are the easy half of the class correction stands, and is now explained. The decomposition shows why: at those sizes the sanitizer slice is ~0 ms, so the step measures process startup.
C3 Correction — docs/users/features/hooks.md:624 claims headless paths produce no submitted_prompt; they do correction (pre-existing) stands. Pre-existing doc drift, not this PR's defect; not re-measured.

No previous finding was fixed, declined, superseded or worsened — the head did not move.

Scope

Central claim (carried) — the opt-in UserPromptSubmit Hook always terminates promptly with {}, exit 0 and clean stderr on adversarial input. The previous round proved this load-bearing with an 11-scenario hunk-by-hunk A/B; that matrix is not re-run this round (see Not covered).

This round's chosen scope — the head is unchanged and the only thing standing between this PR and a green verify lane is the flakiness gate, which the previous round reported but never diagnosed. So the central claim for this round is:

The PR's new test file is deterministically green.

plus re-measurement of the two carried-forward Suggestions, which are cheap and are the only other open items.

Central claim — A/B table (load bisect)

The oracle is vitest's own JSON reporter: per-test duration and status, plus the file's exit code. Identical command to the gate's (npx --no-install vitest run ./src/auto-recall.integration.test.ts), differing only by N competing CPU burners started before the round. Witness: 01-load-bisect-and-2000ms-deadline-decomposition.png.

arm (64 cores) rounds file exit ≠ 0 worst t1 t1 as % of 5000 ms first mode to break
idle (load 0) 5 0 173 ms 3% none
4× (256 burners) 3 0 2361 ms 47% none
8× (512) 4 1 3412 ms 68% spawnSync ETIMEDOUT @​2000 ms
16× (1024) 3 3 5200 ms 104% vitest default 5000 ms TIMEOUT
16× (1024, rerun) 4 2 882 ms 18% spawnSync ETIMEDOUT @​2000 ms

The gate's own failing round is the same shape: runs the real entry point and fails open without stderr at 5597 ms → Test timed out in 5000ms., with every other test in that round also inflated (t2 131→4651 ms, TLS 1592→2246 ms, near-miss ~80→845 ms).

The two 16× runs failed through different modes. That is itself the diagnosis: this is not one fragile assertion but a file with four independent hard-coded timing budgets, and which one breaks first depends on where the contention lands.

Head-side natural durations against each threshold, all measured this round:

test natural threshold source of the threshold idle margin broke at
bounds repeated %s near misses ×4 74–105 ms 2000 ms spawnSync({timeout: 2000}) 24×
runs the real entry point… (t1, 2 children) 153–173 ms 5000 ms vitest default — no explicit arg 30× 16×
executes the v3 configuration… (t2) 128–151 ms 5000 ms vitest default — no explicit arg 33× 16×
exits successfully after a provider timeout… 1583 ms 5000 ms toBeLessThan(5000) in body 3.2× never
rejects a stalled %s FIFO ×2 90–99 ms 5000 ms toBeLessThan(5000) in body 55× never
bounds sanitizer work… (auto-recall.test.ts:87) 19 ms 2000 ms toBeLessThan(2000) in body 105× never

The idle-margin column is misleading on its own, and the TLS row is why. It has the tightest margin in the file (32% of budget spent before any contention) and it never broke, even at 16×. Its budget is a wall-clock wait on a deliberately stalled TLS handshake — not CPU — so contention barely touches it. The tests that broke are the ones whose budget is consumed by process startup and parsing a 411 KB bundle, which is exactly the slice contention inflates. Ranking these by idle margin would have pointed at the wrong test.

Findings

F1 — Blocking: the new integration file encodes four unmeasured timing budgets; the tightest one measures process startup, not the behaviour it names

auto-recall.integration.test.ts:172-195 — the four bounds repeated %s near misses cases named in the PR's own Test Plan step 2 — run:

const result = spawnSync(process.execPath, ['--input-type=module', '--eval', `
  import { createAutoRecallQuery } from ${JSON.stringify(hookBundle.href)};
  const query = createAutoRecallQuery(${JSON.stringify(keyword)}.repeat(Math.floor(4096 / ${keyword.length})), '');
  process.stdout.write(JSON.stringify(query ?? null));
`], { encoding: 'utf8', timeout: 2000, killSignal: 'SIGKILL' });
expect(result.error).toBeUndefined();

The 2000 ms covers node startup plus parsing the 411 KB esbuild bundle plus the sanitizer call. Only the last slice is what the test claims to bound. Decomposed (deadline-decomposition.mjs, medians, N=7 idle / N=5 at 8×):

slice idle 8× oversubscription
A. node startup only (--eval, no import) 24 ms 114 ms
B. + import the 411 KB bundle +49 ms +1704 ms (med 1818, max 1965 = 98% of the deadline)
C. + first call, 1-char query +8 ms 1/5 ETIMEDOUT
D. the tested shape, "token".repeat(819) +0 ms (−7, below noise)
E. hyphen-dense dot-free 4095 chars (the quadratic shape) +24 ms 2/5 ETIMEDOUT

At 8× oversubscription, importing the bundle and calling nothing reaches 1965 ms of the 2000 ms deadline. The test's entire margin is consumed before createAutoRecallQuery is ever invoked, and the slice it exists to measure contributes ~0 ms at idle.

Reproduce:

node tmp/pr11246-verify-20260907-090356/flakiness-probe.mjs --rounds 4 --load 512 --label load512
node tmp/pr11246-verify-20260907-090356/deadline-decomposition.mjs 7

Why the failure message points at the wrong cause. When the child is SIGKILLed at 2000 ms, spawnSync returns result.error, and the first assertion reads:

AssertionError: expected Error: spawnSync /usr/local/bin/node ETIMEDOUT { …(5) } to be undefined

Nothing in that message says "node took too long to start". A maintainer triaging it will read it as the sanitizer being slow — the opposite of the truth, since the sanitizer contributed ~0 ms. Ranking this variant by observability: it is the quiet one, because it silently misattributes.

Blast radius — this is not confined to the verify lane. integrations/external-context-mem0 is a root workspace (package.json workspaces), and .github/workflows/ci.yml:764 runs npm run test:ci:workspaces, which resolves to this package's test:ci = npm run build && vitest run. So the suite runs on every PR's CI test job. Measured at 8× oversubscription, the full 9-file suite failed in round 3 of 3 with the identical ETIMEDOUT mode (gate-load512-fullsuite.log).

Bounded — what does NOT hold. CI defaults to --retry=2 (ci.yml:708, vars.QWEN_CI_VITEST_RETRY || '2'). With that flag the same file was green in 5/5 rounds at 12× and 16× oversubscription (loadavg up to 272), where the no-retry invocation failed 3/3 and 2/4. I must be precise about what that shows: vitest's reporter emitted no per-retry marker, so I can attribute the difference to the configuration, not to an observed retry absorbing an observed failure. What I can state as measured: the gate's no-retry invocation is the one that goes red, and the PR test job's retry budget made it green in every round I ran. This is therefore an intermittent-red / wasted-retry-budget problem on shared runners, not a demonstrated hard CI failure — and not a runtime defect in the Hook itself, which behaved correctly in every cell.

Minimal suggested fix (direction only — NOT applied, NOT measured)

Two independent changes, either of which removes the misattribution:

  1. Stop charging process startup to the sanitizer's budget. Have the child report its own in-process sanitizer time and assert on that, so the deadline bounds the thing the test names:
const query = createAutoRecallQuery(...);
const elapsed = performance.now() - t0;   // t0 taken AFTER the import
process.stdout.write(JSON.stringify({ query: query ?? null, elapsed }));

then expect(JSON.parse(result.stdout).elapsed).toBeLessThan(500) and keep a generous outer timeout (e.g. 20 000) purely as a hang guard.

  1. Give the two default-timeout tests the explicit budget their siblings already have. t1 and t2 are the only tests that spawn real children and rely on vitest's 5000 ms default; the TLS and FIFO cases already pass , 10000. Adding , 20000 to lines 44 and 72 costs nothing and removes the mode the gate actually caught.

I did not measure either change, so treat both as direction rather than a patch. Per this skill's own rule a suggested fix must be driven through the same harnesses and shown to leave benign behaviour byte-identical; the fixture that would pin change (1) is one asserting the reported elapsed stays small while the outer spawn is allowed to be slow — today nothing separates those two, which is the defect.

F2 — Suggestion (carried, re-measured): the 4096-char cap is the sole defence against a quadratic regex no test names

Unchanged from P2, re-measured at this head with a positive control the previous round's e2e arm lacked.

Per-pattern attribution, each pattern applied alone to a 65 536-char input, all seven constructs first asserted present verbatim in the shipped bundle:

shape (65 536 chars) jwt-triple-dot secret-assignment long-token-32 bearer
dash-interleaved run (a-×N/2, no dot) 4633 ms 3 ms 4 ms 2 ms
hex run, no dash 3 ms 3 ms 3 ms 2 ms
token repeated 3 ms 3 ms 3 ms 2 ms
benign prose 3 ms 3 ms 3 ms 3 ms

Growth exponent through the real exported createAutoRecallQuery, head vs the one-line m4 control (MAX_SANITIZER_INPUT_CHARACTERS+Infinity, which makes query.length > CAP always false — behaviourally identical to deleting the guard, interface-preserving; verified to differ from head by exactly one line):

arm 1024 4096 16 384 65 536 262 144 exponent
head (cap present) 3 ms 19 ms 19 ms 22 ms 22 ms 0.36 — flat
m4 (cap reverted) 2 ms 19 ms 300 ms 4906 ms n/a 1.88 — quadratic

End-to-end through the full Hook process against a real v3 runtime and a real loopback provider — the previous round's e2e arm needed this and my first attempt this round did not have it (see Methodology):

cell first stdout byte provider requests
CONTROL head, benign prompt 112 ms 1 ← proves the probe is not vacuous
CONTROL m4, benign prompt 97 ms 1
head @​ 50 018 hostile chars 116 ms 1
head @​ 150 000 hostile chars 115 ms 1
m4 @​ 50 018 hostile chars 3085 ms 1
m4 @​ 150 000 hostile chars 24 587 ms 0

The 150 000-char m4 cell is the sharpest number: HOOK_WALL_CLOCK_TIMEOUT_MS = 6500 never interrupts the spin, because it is a setTimeout and a synchronous regex loop starves the event loop. First output arrived at 24 587 ms. The cap is the only defence on this path, not one of two. The providerRequests: 0 is the fail-safe landing after the spin: by the time the loop yields, the wall clock has already expired and the request is suppressed.

Consequence is bounded exactly as the previous round established and I did not re-derive: core's deadline timer runs in the parent's loop, so it SIGTERMs the whole group (hookRunner.ts:1142-1152) and SIGKILLs ≤2000 ms later. Worst case is a stalled turn with no recall, not a wedged CLI. The shipped example's explicit timeout: 8000 is doing real work — an administrator who omits it inherits core's 60 000 ms default.

New observation, informational. At head the 150 000-char hostile prompt still produced one provider request. The cap truncates to 4096 chars of a-a-a-…; long-token-32 then matches the run but must give back the trailing - (not a word character, so the closing \b fails), leaving a non-empty residue after .trim(). So a prompt that is entirely hostile filler still reaches the external service as a degenerate short query. I did not capture the query string this round, so I state the request count as measured and the residue mechanism as read from the unchanged code plus the previous round's measured "deployment policy -". Cosmetic — no secret content — but it means "sanitized to nothing" is not the same as "sends nothing".

F3 — Suggestion (carried, re-measured): statusMessage in the shipped examples has no runtime consumer

Both examples/managed-auto-recall-user-settings-{posix,windows}.json set "statusMessage": "Retrieving external context", and manifest.test.ts:91 pins that exact value.

Full read-site census this round:

  • httpHookRunner.ts:172-173 is the only site that renders it, gated on this.statusMessageCallback.
  • setStatusMessageCallback (httpHookRunner.ts:122) has zero call sites repo-wide — grep returns only its own definition and packages/core/dist/src/hooks/httpHookRunner.d.ts:23. So the render path is dead.
  • acpAgent.ts:2312 and :8172-8222 (serializeHookConfig) pass the field through the ACP serve API for both command and http hook types; they never render it. This is the enumeration the previous round did not list, and it does not change the conclusion.

Not a regression introduced here — the sibling integrations/external-context/examples/ ship the identical field, so the PR mirrors an established repo pattern. Reported because the PR adds a test that hardens a dead field, which is the foo?: boolean declared-and-read-but-never-set shape: the field round-trips through settings and ACP serialization and decides nothing.

Corrections

No new corrections this round. C1–C3 from the previous round all stand; C1 and C2 were re-measured and are folded into F2 and F1 respectively. C2 in particular now has a mechanism: the four fixtures Test Plan step 2 names measure ~0 ms of sanitizer work, so the step passes by measuring node startup.

Not covered

  • The previous round's 11-scenario process-lifetime A/B matrix (m1 wall clock / m2 explicit exit / m3 O_NONBLOCK+isFile() / m5 provider abort, and the m1+m2, m1+m2+m3 combination rows) was not re-run. Justification is the tree identity: HEAD/HEAD^1/HEAD^2 are the same commit objects, so the merged tree is the same git object and those cells are code properties of an unchanged bundle. I re-measured the timings that feed the finding I was working on (child spawn 80 ms, bundle parse 49 ms idle / 1704 ms at 8×) but did not re-derive the 66-assertion matrix. A reader who wants that evidence should consult the previous round's 01-ab-process-lifetime-head-vs-reverted-guards.png.
  • The previous round's 5-mutant suite matrix (0 survivors, each killed by its expected test) was not re-run in full. I re-ran the one mutant that matters for F2 — the cap — as a source-level revert with its positive control, and it killed exactly the expected test.
  • MCP profile separation (previous S2: v2 → exactly context_search, v3 → 0 tools, manifest has no hooks key, Hook rejects v2, cwd outside repositoryRoot → 0 requests). Not re-run; carried on the tree identity. manifest.test.ts:26 still pins expect(manifest.hooks).toBeUndefined() and is inside the 88-test gate I did run.
  • Per-commit attribution. The checkout is depth 2 and HEAD^2 sits on a shallow boundary with no parents (git log -1 --format='%P' HEAD^2 is empty). git rev-list HEAD^1..HEAD^2 returns only 529ac621, while the snapshot lists 2 commits (0a0e9dec, 529ac621); 0a0e9dec is missing locally and git rev-parse --is-shallow-repository is true. I verified the aggregate HEAD^1..HEAD diff (16 files, +2005/−113). My m4 control is named by the hunk it reverts, not by the commit that introduced it.
  • Windows runtime. The PR marks Windows untested. Not exercised; no change from the previous round.
  • npm pack dry-run (Test Plan step 4). Not run. I read package.json.files (dist/main.js, dist/auto-recall.js, schemas, examples, qwen-extension.json, README.md) but did not inspect a tarball.
  • Real TUI / headless turns (Test Plan step 5). Not driven. Test Plan step 5 asks the reviewer to "run TUI or headless user turns with the administrator Hook profile and confirm one recall per eligible submission" — I verified the Hook process in isolation against a real provider instead, which reproduces the wire shape, not an end-to-end turn through core's hook runner.
  • Repo-wide gates. I ran only the affected package (9 files / 88 tests). No repo-wide lint, typecheck, or packages/core/packages/cli suites — the diff touches neither package. tsc --noEmit for the integration package was not run separately; m4 was built by patching the bundle, so it is not typechecked (irrelevant to head, whose bundle is the shipped byte-identical one).
  • Whether --retry=2 absorbs a specific failure. Reported as a configuration contrast, not an observed retry — vitest emitted no per-retry marker. Instrumenting that would need a reporter that surfaces retry counts per assertion.
  • Harness bugs I hit and fixed, disclosed because two of them produced wrong numbers before I caught them. (i) My first e2e arm for F2 spawned the bundle with no instance config, so loadAutoRecallRuntimeConfiguration failed open before the sanitizer was reached — m4 @​150 000 measured 68 ms of nothing and my assertion firstByteMs ?? 1e9 > 6500 passed for the wrong reason, conflating "spawn failed" with "child was slow". Fixed by building a real v3 runtime plus a loopback provider and adding a positive control that asserts providerRequests === 1; the corrected cell reads 24 587 ms. (ii) My first attempt passed the hostile prompt through --eval argv and hit E2BIG above ~100 k chars — the same trap the previous round disclosed. Input is now generated inside the spawner and written to stdin. (iii) The assertion tally initially reported 4 spurious FAILs because ANSI colour codes in the captured vitest logs broke the matchers; fixed by stripping ANSI before matching. The corrected tally is the one in assertions.json.

Methodology

Environment: the CI verify job — node:22-bookworm container, no credentials, no GitHub token, 64 cores, npm ci and npm run build already complete at HEAD. Ambient loadavg was 15–31 throughout from other runner activity; the "idle" arm is therefore relatively idle, not a quiet machine, which is why the bisect arms are expressed as added oversubscription rather than absolute loadavg.

Four harnesses drove the code, all mock-free with respect to the unit under test: flakiness-probe.mjs runs the gate's exact command N times under L competing CPU burners and reads per-test durations out of vitest's own JSON reporter (--reporter=json), so the oracle is vitest's measurement rather than my wall clock; deadline-decomposition.mjs isolates each slice of the 2000 ms spawnSync budget by running real spawnSync children that differ only in how far they get (startup → import → call → tested shape → quadratic shape); s1-sanitizer-probe.mjs drives the real exported createAutoRecallQuery from the shipped and one-line-mutated bundles, plus the full Hook process against a real v3 runtime and a real loopback HTTP provider that counts the requests it receives; tally.mjs computes assertions.json from the PASS/FAIL lines the harnesses printed plus scripted checks over the captured logs and repo state.

Two validity controls bracket the whole round. Rebuilding src/auto-recall.ts with the package's own esbuild invocation produced sha256 42addd11d00f0d0c9665e35f5908181820a2f22af4ca9cb4806705a7d9d1ec18, byte-identical to the shipped dist/auto-recall.js, so every control differs from head by exactly the named hunk; and reverting the cap guard in source turned createAutoRecallQuery > bounds sanitizer work before applying credential patterns red with expected 2794 to be less than 2000 — a real expected-vs-actual mismatch, one test red and 20 green in the same file, after which git status --porcelain returned 0 lines confirming the tree was restored.

Raw per-cell output lives in logs/ (flak-idle.log, flak-load256.log, flak-load512.log, flak-load1024.log, flak-load1024b.log, deadline-idle.log, deadline-load512.log, gate-head-full-suite.log, gate-load512-fullsuite.log, retry2-load768.log, retry2-load1024.log, s1-probe.log, vacuity-m4-source.log, tally.log, summary.txt); the m4 control is variants/m4-no-sanitizer-cap.js and the pristine rebuild is variants/pristine/auto-recall.js.

Flakiness gate log

rounds=5 files=4 skipped=0
file integrations/external-context-mem0/src/auto-recall-config.test.ts: (cd integrations/external-context-mem0) npx --no-install vitest run ./src/auto-recall-config.test.ts
file integrations/external-context-mem0/src/auto-recall.integration.test.ts: (cd integrations/external-context-mem0) npx --no-install vitest run ./src/auto-recall.integration.test.ts
file integrations/external-context-mem0/src/auto-recall.test.ts: (cd integrations/external-context-mem0) npx --no-install vitest run ./src/auto-recall.test.ts
file integrations/external-context-mem0/src/manifest.test.ts: (cd integrations/external-context-mem0) npx --no-install vitest run ./src/manifest.test.ts


per-file results (P=pass F=fail I=infra-exit, one letter per run):
  integrations/external-context-mem0/src/auto-recall-config.test.ts: PPPPP
  integrations/external-context-mem0/src/auto-recall.integration.test.ts: PPPPP
  integrations/external-context-mem0/src/auto-recall.test.ts: PPPPP
  integrations/external-context-mem0/src/manifest.test.ts: PPPPP

verdict: pass
summary: 4 changed test file(s) x 5 identical rounds, no divergence

--- per-invocation detail (full copy in the artifact) ---
round 1 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 1 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 1 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 1 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)
round 2 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 2 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 2 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 2 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)
round 3 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 3 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 3 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 3 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)
round 4 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 4 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 4 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 4 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)
round 5 · integrations/external-context-mem0/src/auto-recall-config.test.ts: P (exit 0)
round 5 · integrations/external-context-mem0/src/auto-recall.integration.test.ts: P (exit 0)
round 5 · integrations/external-context-mem0/src/auto-recall.test.ts: P (exit 0)
round 5 · integrations/external-context-mem0/src/manifest.test.ts: P (exit 0)

Evidence images

01-load-bisect-and-2000ms-deadline-decomposition

Harness scripts and raw logs are in the workflow run artifacts (7-day retention).

Qwen Code · sandboxed verification

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Triage re-run completed without a new review.

⚠️ The bot has neither a verdict nor a deferral on 529ac621e33fa82356a765494eff1ae9489dd908 — no APPROVED, CHANGES_REQUESTED, or COMMENTED review of its own. A DISMISSED one does not count: dismiss_stale_reviews voids the bot's approval on every push, which is exactly when a fresh one is needed. If this re-run was meant to review or approve, it did not, and an approval left by another account is a separate vote that does not count as the bot's own.

⚠️ 机器人在 529ac621e33fa82356a765494eff1ae9489dd908既没有裁决也没有 defer —— 没有属于它自己的 APPROVEDCHANGES_REQUESTEDCOMMENTED 评审。DISMISSED 不算:dismiss_stale_reviews 会在每次推送时作废机器人的批准,而那恰恰是需要一次新批准的时刻。如果这次重跑本应评审或批准,那么它没有做到;而其他账号留下的批准是另一张票,不能算作机器人自己的。

The stage comments above were updated with the latest result. View workflow run.

上方各阶段评论已更新为最新结果。查看工作流运行

chiga0
chiga0 previously approved these changes Sep 7, 2026

@chiga0 chiga0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scope: integrations/external-context-mem0/src/ (all source files), schema, package manifest. NOT reviewed: design doc narrative beyond code-describing claims; test fixtures in detail.

Tier: Deep — credential handling, external network calls, user-prompt sanitization.

No blocking findings. No approval blockers.

Checked:

  1. createAutoRecallQuery (auto-recall.ts) — Array.from slice by code points correct. Credential literal removed first, then SECRET_ASSIGNMENT_PATTERN, Bearer tokens, JWT-shaped 3x8+ segments, 32+ char words. Best-effort as stated.
  2. FIFO defense (config.ts) — O_RDONLY|O_NONBLOCK prevents blocking open; isFile() rejects FIFOs before reading. O_NONBLOCK??0 Windows fallback correct.
  3. isWithinRepository — realpath on both sides, relative() with sep, correct for Windows and symlink traversal.
  4. Timeout/abort (runAutoRecallCli) — single outputStream.write after Promise.race; no double-write; process.exit handles retained handles.
  5. Type contracts (schemas.ts, types.ts) — parseInstance applies Ajv before type assertion; union type accepted by createRequestEngine.
  6. Manifest isolation — v2 MCP path unchanged; v3 accepted only by Hook; no automatic enablement.

Not covered: Windows/Linux runtime (macOS only, disclosed); live vendor endpoints.

Cross-check: CI bot ran 3 passes at this commit, no CHANGES_REQUESTED, no findings to confirm/refute.

Reviewed with AI assistance.

Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
@doudouOUC

doudouOUC commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed c0fc7ce5fce0cb5791929d12f4dc6dd4574a3bf2 to address the newly observed test failures and the existing Hook opt-out documentation gap.

Correction to my previous verification summary: the agent's merge-ready/565-assertion result and the independent changed-test flakiness gate are separate results. The latter recorded PFPPP, including the first integration test timing out at 5597 ms against its 5000 ms outer budget. I had not included that gate failure in the earlier summary. The fixes below address it and a second startup-budget failure found during local repeated verification.

Item Action
Two sequential 8-second Hook children inside a default 5-second test Set a 20-second outer test budget; give the single-child functional test 10 seconds. Each child still has an 8-second SIGKILL deadline.
Sanitizer subprocess can consume its 2-second budget before the function starts Measure computation after bundle import and still require less than 2 seconds; allow 8 seconds for the complete subprocess and 10 seconds for the test. Slow computation and stuck processes still fail.
Existing all-Hook opt-outs missing from deployment guidance Document --bare, --safe-mode, and disableAllHooks: true, including startup/restart and broader customization effects; retain separate QWEN_HOME guidance for automation needing other Hooks.

Verification on macOS, Node.js v22.22.3:

  • Same delayed-start reproduction: the original first test failed at 5013 ms; the fixed test passed at 8428 ms, with both real children exiting 0 with {}, empty stderr, no signal, and each within its 8-second deadline.
  • A 2100 ms sanitizer-process startup delay previously caused SIGKILL at 2002 ms before the function ran. After the fix it exited 0 at 2389 ms, with query time 0.854 ms. The negative control injecting 2100 ms of synchronous computation still failed at the retained time assertion (2101.138 ms). Production source was unchanged in these experiments.
  • Final integration-file repetition: PPPPP, 45/45 test cases passed. The earlier PPFPP run and its failure evidence were retained; they were not counted as passing verification.
  • All 88 package tests, root build/bundle/typecheck, package lint, formatting and diff checks passed. Independent review found no actionable issue; two final self-audit passes were clean.
  • Five real CLI loopback cases passed: each of the three documented opt-outs reached the model with zero Hook executions, zero recall requests and no recalled marker; enabled controls before and after each made one Hook call and one recall request.

The production Hook bundle remains byte-identical (sha256 42addd11d00f0d0c9665e35f5908181820a2f22af4ca9cb4806705a7d9d1ec18). TLS/FIFO performance assertions and result/exit/stderr checks remain intact. This is test and documentation work only. Existing scope dispositions are unchanged. There are no inline review threads to resolve (0/0). Update at 10:10 UTC: yiliang114 approved this exact head; the earlier chiga0 approval of 529ac621e3 was dismissed on push, and the aggregate decision remains REVIEW_REQUIRED. Remote CI is still running. Its Classify PR job failed while checking out the trusted base 421393d51df68207281343ce41798fbdc98ec30d: the runner could not connect to github.com:443 after 134231 ms, before the classifier or PR code ran. GitHub did not accept the targeted rerun request (job 101702135351 cannot be rerun); it is recorded for retry when the run is eligible.

中文说明

已推送 c0fc7ce5fce0cb5791929d12f4dc6dd4574a3bf2,处理新观察到的测试失败,并补全现有 Hook 关闭方式的文档。

更正此前验证摘要:agent 的 merge-ready/565 项断言结果与独立的变更测试防抖门禁是两个结果。后者实际为 PFPPP,首个集成测试在默认 5000 ms 预算下于 5597 ms 超时;此前摘要遗漏了这一门禁失败。本次修复同时处理它和本地重复验证发现的另一项启动预算问题。

事项 处理
两个各允许 8 秒的 Hook 子进程串行运行在默认 5 秒测试内 外层测试改为 20 秒;单 child 功能测试使用 10 秒。每个 child 的 8 秒 SIGKILL 保留。
脱敏测试可能在函数执行前耗尽 2 秒子进程预算 bundle 导入后单独测量计算耗时,仍要求低于 2 秒;完整子进程使用 8 秒,外层测试 10 秒。慢算法和卡死仍会失败。
部署说明遗漏已有的禁用全部 Hook 方式 补充 --bare--safe-modedisableAllHooks: true,说明启动/重启及其他自定义配置影响;自动化仍需其他 Hook 时保留独立 QWEN_HOME 方案。

macOS、Node.js v22.22.3 验证结果:相同启动延迟下,原首项测试 5013 ms 超时,修复后 8428 ms 通过,两个真实 child 均在各自 8 秒内以 0 退出、返回 {}、stderr 空且无 signal。脱敏子进程注入 2100 ms 启动延迟,原测试在 2002 ms、函数尚未运行时被终止;修复后 2389 ms 正常退出,计算耗时 0.854 ms。注入 2100 ms 同步计算延迟的负向对照仍在 2101.138 ms 被计算耗时断言捕获,实验未改生产源码。

最终集成文件连续五轮 PPPPP,45/45 项通过;此前 PPFPP 的失败证据全部保留,没有当作通过。88 项包测试、根仓库 build/bundle/typecheck、包级 lint、格式及 diff 检查通过;独立审查无新问题,最终两轮自审 clean。五项真实 CLI 回环验证通过:三种关闭方式均正常调用模型,但 Hook 执行、召回请求及召回 marker 均为零;前后启用对照各执行一次 Hook、召回一次。

生产 Hook bundle 字节不变(SHA256 同上),TLS/FIFO 性能断言及结果/退出/stderr 检查保留。本次仅改测试和文档,既有范围决定保持不变。没有需解决的行级审查讨论(0/0)。10:10 UTC 状态更新:yiliang114 已批准此精确 head;chiga0 对旧 head 529ac621e3 的批准在推送后被撤销,汇总状态仍为 REVIEW_REQUIRED。远端 CI 仍在运行;Classify PR 在检出可信基线 421393d51df68207281343ce41798fbdc98ec30d 时,runner 连接 github.com:443 达 134231 ms 后失败,尚未运行分类器或 PR 代码。GitHub 暂未接受定向重跑请求(job 101702135351 cannot be rerun),已记录,待该运行可重跑时处理。

@yiliang114 yiliang114 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review pass on head c0fc7ce. No blocking findings. The auto-recall entry point is well-defended for an opt-in hook: admin deployment via managed settings with administrator-owned paths; the repository-root gate realpaths both sides and fails closed (absolute-only, regular-directory, never the filesystem root); the query sanitizer strips code fences, the configured credential literal, secret assignments, Bearer tokens, JWT-shaped strings, and long opaque tokens before anything leaves the machine, capped at 512 chars from a 1 MiB input bound; the 6.5s wall-clock race sits under the 8s hook timeout with stdin destroy; and every failure path fail-opens to {} so a prompt is never blocked or errored by recall. The O_NONBLOCK + regular-file guard on config open (FIFO hang) is a nice touch. Schema v3 keeps v2 callers untouched via the SearchRuntimeConfiguration union. Roughly half the diff is unit/integration tests. review-pr was still running at review time; everything else is green.

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partially reviewed — gaps disclosed. Suggestions are inline.

2 Suggestion-level finding(s) this review confirmed are already reported on this PR and are not repeated:

  • X1-1 design-doc Verification section records an npm pack --dry-run inspection no automation performs (docs/design/external-context-mem0-auto-recall.md:191) — already reported (comment 5564985809)
  • X1-2 shipped Hook timeout has no documented floor above the internal 6500 ms budget (examples/managed-auto-recall-user-settings-posix.json:10) — already reported (comment 5564985809)

Not reviewed: build-and-test — Test (macos-latest, Node 22.x) and Test (windows-latest, Node 22.x) were skipped in CI and the package suite ran only on Linux locally, so the shipped Windows Hook example and macOS symlinked-path containment were never executed on their own platforms.

Not reviewed: reverse audit — stopped before round 3 by the review time budget.

中文说明

仅完成部分审查,审查缺口已披露。 建议见行内评论。

本轮确认的 2 条建议级发现已在 PR 上报告过,不再重复发布(列表见上方英文部分)。

未审查(原文为英文):build-and-test — Test (macos-latest, Node 22.x) and Test (windows-latest, Node 22.x) were skipped in CI and the package suite ran only on Linux locally, so the shipped Windows Hook example and macOS symlinked-path containment were never executed on their own platforms.

未审查:反向审计——评审时间预算不足,未能开始第 3 轮。

— qwen3.8-max via Qwen Code /review (v0.23.0)

file = await open(path, 'r');
// A blocked FIFO open can outlive even the Hook's explicit process exit.
file = await open(path, constants.O_RDONLY | (constants.O_NONBLOCK ?? 0));
if (!(await file.stat()).isFile()) throw new Error('Not a regular file.');

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-1: this regular-file guard lands in the readConfigFile shared by both profiles, so it also narrows the pre-existing v2 MCP loader — and no test on either path can kill it.

Two measured consequences. Coverage: a FIFO holding valid configuration with a live writer is rejected today only by this line; delete it and the shipped Hook loads its configuration from the pipe, yet the whole package suite stays green (88/88). The two stalled-FIFO integration cases cannot see it — a writerless FIFO opened O_NONBLOCK EOFs at once, so JSON.parse('') throws into the same catch either way — and the directory case at schemas.test.ts:217-219 is decided by EISDIR either way. So whichever way you resolve this, the decision leaves no trace in the tests. Documentation: an existing v2 deployment feeding its instance JSON through a readable non-regular path — bash process substitution (QWEN_EXTERNAL_CONTEXT_MEM0_CONFIG=/dev/fd/63, measured isFile=false isFIFO=true, content readable), a FIFO, or a character device — loaded fine at the merge base and now fails at startup with the redacted Mem0 extension instance configuration is unavailable. The README troubleshooting row for that exact message (README.md:308) lists only "missing, blank, unresolved, or unreadable", so the administrator debugs a working file instead of the file type. The behaviour change itself is disclosed in your PR body, so what remains actionable is the missing test and the missing v2 documentation.

Witness:

v2 loader, base arm (hunk reverted) vs PR arm:
BASE : {"state":"resolved","schemaVersion":2,"credential":"probe-token"}
PR   : {"state":"rejected","message":"Mem0 extension instance configuration is unavailable."}
v3 loader, FIFO WITH a live writer feeding valid config:
INTACT : {"loaded":false,"message":"...instance configuration is unavailable."}
MUTANT : {"loaded":true,"schemaVersion":3,...}    <- this line deleted; suite 88/88 green
PROC-SUBST path=/dev/fd/63 isFile=false isFIFO=true readFileSync={"schemaVersion":2}

Add the discriminating fixture — a FIFO with a live writer feeding valid configuration — as a POSIX-only case (it.skipIf(process.platform === 'win32')) asserting both loadAutoRecallRuntimeConfiguration and loadRuntimeConfiguration reject with instance configuration is unavailable, matching the message schemas.test.ts:217-219 already pins for a directory. And document the requirement for both profiles: state it beside "Both files are limited to 64 KiB." in the on-demand section, and name non-regular sources (FIFO, process substitution, device) in the instance configuration is unavailable troubleshooting row.

The fix must not relax the guard: auto-recall.integration.test.ts pins rejects a stalled %s FIFO and exits successfully for both the instance and dialect kinds (exit 0, {}), and config.ts:88 records why — "A blocked FIFO open can outlive even the Hook's explicit process exit." The same readConfigFile serves both profiles via config.ts:52, and MAX_CONFIG_BYTES = 64 * 1024 (config.ts:26) keeps the existing v2 size-bound assertions in force.

The new fed-FIFO case is the acceptance criterion: remove this line and that test must go red — today no test in the package changes behaviour under that mutation on either path.

中文说明

[建议] R1-1:这个「必须是普通文件」的判断位于两个模式共用的 readConfigFile 中,因此它同时收紧了已有的 v2 MCP 加载路径 —— 而且两条路径上都没有测试能够杀掉它。

两个经过实测的后果。覆盖率: 一个持有有效配置、并且有写入者的 FIFO,今天只靠这一行被拒绝;删掉它之后,发布出去的 Hook 会从管道里加载配置,而整个包测试套件仍然全绿(88/88)。两个 stalled-FIFO 集成用例看不到这一点 —— 没有写入者的 FIFO 以 O_NONBLOCK 打开会立刻 EOF,因此 JSON.parse('') 在两种情况下都抛进同一个 catch;而 schemas.test.ts:217-219 的目录用例在两种情况下都由 EISDIR 决定。所以无论你如何处理这一行,测试里都不会留下痕迹。文档: 已有的 v2 部署如果通过可读的非普通文件路径提供 instance JSON —— bash 进程替换(QWEN_EXTERNAL_CONTEXT_MEM0_CONFIG=/dev/fd/63,实测 isFile=false isFIFO=true,内容可读)、FIFO 或字符设备 —— 在 merge base 上可以正常启动,现在会在启动时失败,只给出被脱敏的 Mem0 extension instance configuration is unavailable.。README 中针对这条消息的排障行(README.md:308)只列了 "missing, blank, unresolved, or unreadable",于是管理员会去排查一个本来正常的文件,而不是文件的类型。行为变更本身已在 PR 描述中披露,因此剩下可执行的是缺失的测试和缺失的 v2 文档。

证据:

v2 加载器,base 分支(回退该 hunk)与 PR 分支:
BASE : {"state":"resolved","schemaVersion":2,"credential":"probe-token"}
PR   : {"state":"rejected","message":"Mem0 extension instance configuration is unavailable."}
v3 加载器,带写入者的 FIFO 提供有效配置:
INTACT : {"loaded":false,"message":"...instance configuration is unavailable."}
MUTANT : {"loaded":true,"schemaVersion":3,...}    <- 删除该行;套件 88/88 全绿
PROC-SUBST path=/dev/fd/63 isFile=false isFIFO=true readFileSync={"schemaVersion":2}

请补上能够区分的那条 fixture —— 一个带写入者、提供有效配置的 FIFO —— 写成仅 POSIX 的用例(it.skipIf(process.platform === 'win32')),断言 loadAutoRecallRuntimeConfigurationloadRuntimeConfiguration 都以 instance configuration is unavailable 拒绝,与 schemas.test.ts:217-219 已为目录场景钉住的消息一致。同时为两个模式补文档:在按需模式的 "Both files are limited to 64 KiB." 旁边说明该要求,并在 instance configuration is unavailable 排障行中点名非普通文件来源(FIFO、进程替换、设备)。

修复不得放松该判断:auto-recall.integration.test.ts 已为 instance 与 dialect 两种类型钉住 rejects a stalled %s FIFO and exits successfully(退出码 0、输出 {}),且 config.ts:88 记录了原因 —— "A blocked FIFO open can outlive even the Hook's explicit process exit."。同一个 readConfigFileconfig.ts:52 服务两个模式,MAX_CONFIG_BYTES = 64 * 1024config.ts:26)也让 schemas.test.ts 中已有的 v2 体积边界断言继续有效。

新增的 fed-FIFO 用例就是验收标准:删除这一行后该测试必须变红 —— 今天在这个包中,没有任何测试会因为该变异而改变行为。

— qwen3.8-max via Qwen Code /review (v0.23.0)

The Hook requires a non-empty `submitted_prompt` captured before prompt
expansion. This includes supported interactive TUI submissions and headless
CLI user turns (`qwen -p` and stream-json input, including SDK clients using
that path). The field establishes prompt provenance, not a TUI-only origin.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-2: this provenance claim is correct per the code, but it contradicts the in-tree hook contract doc, which still says the opposite — and your next paragraph makes the transport question load-bearing.

docs/users/features/hooks.md:624 (untouched by this diff) states "ACP, headless, serve, SDK, and remote-input paths do not produce it in this version", repeated at :669, with the same row in docs/design/submitted-prompt-provenance.md:104 and TUI-only framing at integrations/external-context/README.md:242 and docs/design/direct-external-context-auto-recall.md:122. The same claim is added at docs/design/external-context-mem0-auto-recall.md:109-113. Two shipped docs now answer one data-egress question in opposite directions, and this PR is what makes the answer consequential: the very next paragraph tells administrators to give automation a separate QWEN_HOME because headless turns are eligible.

An administrator deciding whether one shared QWEN_HOME may carry this Hook consults the authoritative hook doc, concludes an automation launcher cannot forward prompt text to the external provider, and skips that isolation. Per the code, every headless user turn does forward its sanitized prompt to the configured third-party provider — nonInteractiveCli.ts:1098 derives submittedPrompt, :2393-2396 passes it on the first UserQuery turn, nonInteractive/session.ts:482 calls runNonInteractive once per stream-json user message, AppContainer.tsx:3086 does the same interactively, and client.ts:3225-3241 gates it only on message type and non-emptiness, with no transport predicate anywhere.

Witness:

witness: not run - the nearest capability was a headless `qwen -p` drive with a
stdin-dumping UserPromptSubmit hook, but packages/core/dist and packages/cli/dist
are absent in this worktree, so it would have needed a full monorepo build.
Ruled instead on the repo's own two green tests, which already pin headless provenance:
  nonInteractiveCli.test.ts:1945  plain headless     -> submittedPrompt: 'Test input'
  nonInteractiveCli.test.ts:6536  stream-json envelope -> submittedPrompt: 'Message from stream-json input'

Reconcile the two docs in this PR or a linked follow-up: drop headless/stream-json from the "do not produce it" list at hooks.md:624 and the parallel displayText sentence at :669, correct the transport row in docs/design/submitted-prompt-provenance.md, and align the TUI-only framing in the sibling integration README and its design doc — or have these new Mem0 docs cite hooks.md as the authority and flag the discrepancy rather than asserting the opposite.

The reconciled wording must not re-assert a TUI-only origin: client.ts:3226-3228 gates the field on messageType === SendMessageType.UserQuery && typeof options?.submittedPrompt === 'string' && options.submittedPrompt.trim().length > 0, with no transport or renderer condition. Only the headless (qwen -p / stream-json) path was verified here — ACP, serve and the SDK path were not — so please do not extend the claim to those without checking them.

中文说明

[建议] R1-2:这条 provenance 说明与代码一致,但它与仓库内的 Hook 契约文档相反 —— 而你紧接着的下一段让「输入通道」这个问题变得有实际后果。

docs/users/features/hooks.md:624(本 diff 未改动)写着 "ACP, headless, serve, SDK, and remote-input paths do not produce it in this version",在 :669 重复,docs/design/submitted-prompt-provenance.md:104 的表格同样如此,integrations/external-context/README.md:242docs/design/direct-external-context-auto-recall.md:122 也仍是「仅 TUI」的表述。同一条主张又出现在 docs/design/external-context-mem0-auto-recall.md:109-113。于是两份随包发布的文档对同一个数据外发问题给出了相反答案,而正是本 PR 让这个答案有了后果:紧接着的段落告诉管理员要给自动化使用独立的 QWEN_HOME,理由恰恰是 headless 轮次也符合条件。

管理员在判断一个共享 QWEN_HOME 是否可以注册该 Hook 时会去查权威的 Hook 文档,得出「自动化启动器不可能把 prompt 文本发给外部 provider」的结论,于是跳过隔离。而按代码,每个 headless 用户轮次都会把脱敏后的 prompt 发给所配置的第三方 provider —— nonInteractiveCli.ts:1098 派生 submittedPrompt:2393-2396 在首个 UserQuery 轮次传递,nonInteractive/session.ts:482 对每条 stream-json 用户消息调用一次 runNonInteractiveAppContainer.tsx:3086 在交互路径同理,而 client.ts:3225-3241 只以消息类型与非空为条件,任何地方都没有传输通道判断。

证据:

witness: not run - 最接近的手段是用一个把 stdin 原样打印的 UserPromptSubmit hook
驱动 headless `qwen -p`,但该 worktree 中缺少 packages/core/dist 与 packages/cli/dist,
需要完整的 monorepo 构建。改为依据仓库自身两个已通过的测试,它们已经钉住了 headless provenance:
  nonInteractiveCli.test.ts:1945  plain headless     -> submittedPrompt: 'Test input'
  nonInteractiveCli.test.ts:6536  stream-json envelope -> submittedPrompt: 'Message from stream-json input'

请在本 PR 或一个关联的后续中把两份文档对齐:从 hooks.md:624 的「不会产生该字段」列表以及 :669displayText 平行句中去掉 headless/stream-json,修正 docs/design/submitted-prompt-provenance.md 的传输表格,并对齐同级 integration README 与其设计文档中「仅 TUI」的表述;或者让这些新的 Mem0 文档引用 hooks.md 作为权威并标注分歧,而不是给出相反断言。

对齐后的措辞不得重新断言「仅 TUI 来源」:client.ts:3226-3228 的条件是 messageType === SendMessageType.UserQuery && typeof options?.submittedPrompt === 'string' && options.submittedPrompt.trim().length > 0,其中没有任何传输或渲染条件。本次只验证了 headless(qwen -p / stream-json)路径 —— ACP、serve 与 SDK 路径未验证 —— 因此请勿在未核实前把主张扩展到它们。

— qwen3.8-max via Qwen Code /review (v0.23.0)

The administrator installs a pinned package version at a stable absolute path,
creates the v3 instance and dialect files outside ordinary workspaces, injects
the configuration path and credential through the managed process environment,
and copies the applicable Hook definition into an administrator-controlled

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-3: user-scope registration alone is not sufficient, and neither this Deployment section nor the README's steps say so — a workspace-scoped disableAllHooks: true outranks it and recall silently never happens.

disableAllHooks appears in none of the three workspace restriction lists (WORKSPACE_RESTRICTED_SETTINGS is exactly tools.workflowsEnabled, security.allowPrivateNetworkHooks, security.allowedInsecureVoiceBaseUrls, goals.modelProposedsettingsUtils.ts:267-276), and precedence is System Defaults < User < Workspace < System (settings.ts:610-622). So a repository's .qwen/settings.json beats the managed QWEN_HOME/settings.json.

An administrator copies the Hook into the managed user settings; a repository the launcher opens sets disableAllHooks: true. Config.initialize then skips new HookSystem(this) and the MessageBus subscription (config.ts:3156-3164), and client.ts:3207-3223 gates the whole UserPromptSubmit block on hooksEnabled plus a non-null messageBus. No hook process spawns, so nothing even returns {} — the README's "Auto Recall returns {}" troubleshooting row cannot apply — and recall never happens in that repository, with no diagnostic. The same silence occurs under bare or safe mode. The sibling in-tree profile documents and ships exactly this counter-measure: integrations/external-context/README.md step 4 says to point QWEN_CODE_SYSTEM_SETTINGS_PATH at an administrator-controlled copy of examples/managed-auto-recall-system-settings.json, "Its system-level disableAllHooks: false prevents lower-precedence workspace settings from suppressing the required Hook." This package ships only the two user-settings examples, and grep for disableAllHooks or SYSTEM_SETTINGS over its README and this design doc returns zero hits.

Witness:

witness: not run - the nearest capability was a vitest probe on the real settings
merge, but mergeSettings is not exported (settings.ts:588) and settings.ts:11-17
value-imports @qwen-code/qwen-code-core, whose dist/ is absent in this worktree.
Ruled on the quoted precedence order, the three restriction lists, the
config.ts:3156-3164 and :8137-8139 gates, and the sibling package's own shipped
example + documented rationale as in-repo corroboration.

Add the sibling's step here and to the README's Auto Recall steps: point QWEN_CODE_SYSTEM_SETTINGS_PATH at an administrator-controlled settings file carrying disableAllHooks: false, and note that workspace settings, bare mode and safe mode can each suppress the Hook. Optionally ship a matching examples/managed-auto-recall-system-settings.json.

config.ts:8138 is return this.disableAllHooks || this.getBareMode() || this.isSafeMode(); — a system-level disableAllHooks: false cannot re-enable the Hook under bare or safe mode, so the added guidance must not promise recall there.

If you do ship a system-settings example, src/manifest.test.ts should pin it the way its existing it.each case pins the two user-settings examples (asserting disableAllHooks: false and no mcpServers); that assertion is the test that must go red if the example is later dropped or weakened.

中文说明

[建议] R1-3:只在 user 作用域注册并不足够,而本节与 README 的步骤都没有说明这一点 —— 仓库(workspace)作用域的 disableAllHooks: true 优先级更高,会让召回静默地永不发生。

disableAllHooks 不在三个 workspace 限制列表中的任何一个里(WORKSPACE_RESTRICTED_SETTINGS 恰好是 tools.workflowsEnabledsecurity.allowPrivateNetworkHookssecurity.allowedInsecureVoiceBaseUrlsgoals.modelProposed —— settingsUtils.ts:267-276),而优先级为 System Defaults < User < Workspace < System(settings.ts:610-622)。因此仓库的 .qwen/settings.json 会压过受管 QWEN_HOME/settings.json

管理员把 Hook 复制进受管 user settings;启动器打开的某个仓库设置了 disableAllHooks: true。于是 Config.initialize 会跳过 new HookSystem(this) 以及 MessageBus 订阅(config.ts:3156-3164),而 client.ts:3207-3223 把整个 UserPromptSubmit 分支限制在 hooksEnabled 与非空 messageBus 上。根本不会派生 Hook 进程,因此连 {} 都不会返回 —— README 的 "Auto Recall returns {}" 排障行无从适用 —— 该仓库中召回永不发生,也没有任何诊断信息。bare 模式与 safe 模式下同样静默。仓库内的同级方案恰好记录并随包发布了这个对策:integrations/external-context/README.md 第 4 步要求把 QWEN_CODE_SYSTEM_SETTINGS_PATH 指向管理员控制的 examples/managed-auto-recall-system-settings.json 副本,"Its system-level disableAllHooks: false prevents lower-precedence workspace settings from suppressing the required Hook."。本包只发布了两个 user-settings 示例,并且对其 README 与本设计文档 grep disableAllHooksSYSTEM_SETTINGS 均为零命中。

证据:

witness: not run - 最接近的手段是对真实 settings 合并做 vitest 探针,但 mergeSettings
未导出(settings.ts:588),且 settings.ts:11-17 以值方式 import @qwen-code/qwen-code-core,
其 dist/ 在该 worktree 中不存在。改为依据上文引用的优先级顺序、三个限制列表、
config.ts:3156-3164 与 :8137-8139 的门控,以及同级包自身随包发布的示例与文档化理由作为仓库内佐证。

请在本节与 README 的 Auto Recall 步骤中补上同级方案的那一步:把 QWEN_CODE_SYSTEM_SETTINGS_PATH 指向带 disableAllHooks: false 的管理员控制设置文件,并说明 workspace 设置、bare 模式与 safe 模式都可能抑制该 Hook。也可以选择随包发布对应的 examples/managed-auto-recall-system-settings.json

config.ts:8138return this.disableAllHooks || this.getBareMode() || this.isSafeMode(); —— system 级的 disableAllHooks: false 无法在 bare 或 safe 模式下重新启用 Hook,因此新增说明不得承诺这两种模式下仍能召回。

如果确实随包发布 system-settings 示例,src/manifest.test.ts 应当像现有 it.each 用例钉住两个 user-settings 示例那样钉住它(断言 disableAllHooks: false 且没有 mcpServers);该断言就是「示例被删除或弱化时必须变红」的测试。

— qwen3.8-max via Qwen Code /review (v0.23.0)

}
query = query
.replace(/(```|~~~)[\s\S]*?\1/g, ' ')
.replace(/(?:```|~~~)[\s\S]*$/g, ' ');

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-4: this unterminated-fence branch and the whole ~~~ variant ship with no test — both mutants ran green while leaking the pasted block.

Every fence fixture in the new tests is a closed ``` pair that the preceding regex already consumes (auto-recall.test.ts:38/40 and `:63`), and no fixture in `auto-recall.integration.test.ts` contains a fence at all. An unterminated fence is the ordinary case for a truncated paste — pasting a log or a snippet and submitting before the closing marker.

Both mutants were built into dist/auto-recall.js and run: deleting this line, or dropping ~~~ from the pair, leaves all 88 tests green while the query forwarded to the administrator's external provider changes. None of the four remaining patterns catch the leaked content — no secret keyword, no Bearer, no dotted triplet, no 32+ char token — so the pasted block's internal URLs, headers and paths leave the host. This contradicts the design doc's runtime-flow step 5 ("Remove fenced code") and its Verification claim that sanitization is unit-tested.

Witness:

INTACT   : Tests 88 passed (88) | UNTERMINATED => "How do I fix this?"
                                | TILDE-PAIR   => "How do I fix this?"
MUTANT A (delete line 98):
           Tests 88 passed (88) | UNTERMINATED => "How do I fix this? ```sh curl
             https://internal.corp/api -H 'X-Internal: abc123'"
MUTANT B (drop ~~~ from both fence patterns):
           Tests 88 passed (88) | TILDE-PAIR   => "How do I fix this? ~~~sh curl
             https://internal.corp/api ~~~"

Add cases to the createAutoRecallQuery describe block in src/auto-recall.test.ts, e.g. expect(createAutoRecallQuery('How do I fix this?\n```sh\ncurl https://internal.corp/api\n', '')).toBe('How do I fix this?'), plus the same shape with ~~~ in both open and closed form, so the unterminated branch and the tilde alternative are each pinned.

The first fence regex must survive any consolidation: auto-recall.test.ts:63 asserts createAutoRecallQuery('```text\nonly code\n```', '') is toBeUndefined(), and :47 asserts the terminated-fence case yields exactly 'How should deployment work?'.

The new cases are the acceptance criterion: remove this line (or drop ~~~) and they must go red, while those two existing assertions stay green.

中文说明

[建议] R1-4:这个「未闭合代码围栏」分支以及整个 ~~~ 变体都没有测试就发布了 —— 两个变异体都在泄漏粘贴内容的同时保持全绿。

新测试中的每个围栏 fixture 都是已闭合的 ``` 对,会被前一个正则先消费掉(auto-recall.test.ts:38/40 与 `:63`),而 `auto-recall.integration.test.ts` 中没有任何 fixture 含围栏。未闭合围栏恰恰是「粘贴被截断」的常见形态 —— 用户粘贴日志或代码片段后在闭合标记前就提交。

两个变异体都被构建进 dist/auto-recall.js 并实际运行:删除这一行,或从这对正则中去掉 ~~~,88 个测试全部保持绿色,而发给管理员外部 provider 的 query 发生了变化。剩下四条规则都拦不住泄漏内容 —— 没有密钥关键字、没有 Bearer、没有点分三段、没有 32+ 字符 token —— 因此粘贴块中的内部 URL、请求头与路径会离开主机。这与设计文档运行流程第 5 步("Remove fenced code")以及其 Verification 中「脱敏已有单元测试」的说法相矛盾。

证据:

INTACT   : Tests 88 passed (88) | UNTERMINATED => "How do I fix this?"
                                | TILDE-PAIR   => "How do I fix this?"
MUTANT A (delete line 98):
           Tests 88 passed (88) | UNTERMINATED => "How do I fix this? ```sh curl
             https://internal.corp/api -H 'X-Internal: abc123'"
MUTANT B (drop ~~~ from both fence patterns):
           Tests 88 passed (88) | TILDE-PAIR   => "How do I fix this? ~~~sh curl
             https://internal.corp/api ~~~"

请在 src/auto-recall.test.tscreateAutoRecallQuery describe 中补用例,例如 expect(createAutoRecallQuery('How do I fix this?\n```sh\ncurl https://internal.corp/api\n', '')).toBe('How do I fix this?'),并补上 ~~~ 的未闭合与已闭合两种形态,使未闭合分支与波浪号分支各自被钉住。

任何合并都必须保留第一个围栏正则:auto-recall.test.ts:63 断言 createAutoRecallQuery('```text\nonly code\n```', '')toBeUndefined():47 断言已闭合围栏用例恰好得到 'How should deployment work?'

新增用例即验收标准:删除这一行(或去掉 ~~~)后它们必须变红,同时上述两条既有断言保持绿色。

— qwen3.8-max via Qwen Code /review (v0.23.0)

value['hook_event_name'] !== 'UserPromptSubmit' ||
typeof submittedPrompt !== 'string' ||
submittedPrompt.trim().length === 0 ||
typeof cwd !== 'string'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-5: this cwd clause is the only thing rejecting a cwd-less event before configuration is loaded, and no test reaches it.

The it.each at auto-recall.test.ts:153 covers {}, a wrong event name, prompt-only, and a whitespace-only submitted_prompt — every one of those still skips after this clause is deleted — and every event in auto-recall.integration.test.ts carries a cwd. So the removal is invisible.

Measured: with the clause gone, a well-formed UserPromptSubmit event with a non-empty submitted_prompt and no cwd proceeds to load the administrator's instance and dialect files from disk (loader calls 0 → 1) and then reaches isWithinRepository('/repository', null). isAbsolute(undefined) throws ERR_INVALID_ARG_TYPE, and config.ts:235-236 calls it outside its try (the try opens at 237), so the TypeError propagates out of runAutoRecall and is absorbed only by the CLI-level .catch(() => ({})). The clean pre-config skip the design's runtime-flow step 2 requires becomes an internal crash path plus a wasted configuration load on every such event — externally still {}, so nothing surfaces.

Witness:

INTACT : PROBE settled {"resolved":{}} | loadConfig calls: 0 | isWithinRepository calls: []
MUTANT : PROBE settled {"resolved":{}} | loadConfig calls: 1 | isWithinRepository calls: [["/repository",null]]
         Tests 88 passed (88)
node -e: isAbsolute(undefined) -> TypeError ERR_INVALID_ARG_TYPE
Suggested change
typeof cwd !== 'string'
typeof cwd !== 'string'

is what the tests need to reach — add { hook_event_name: 'UserPromptSubmit', submitted_prompt: 'question' } (and optionally a cwd: 42 variant) to the it.each list at src/auto-recall.test.ts:153.

The extended case's existing assertions — loadAutoRecallRuntimeConfiguration not.toHaveBeenCalled() and search not.toHaveBeenCalled() — are the acceptance criterion: they go red when this clause is removed, because the mocked loader would then be invoked for the new input.

中文说明

[建议] R1-5:这个 cwd 判断是「在加载配置之前拒绝没有 cwd 的事件」的唯一依据,而没有任何测试能够到达它。

auto-recall.test.ts:153it.each 覆盖了 {}、错误的事件名、只有 prompt、以及 submitted_prompt 全为空白 —— 删掉本判断后这些用例仍然会跳过 —— 而 auto-recall.integration.test.ts 中每个事件都带 cwd。因此该删除是不可见的。

实测:去掉该判断后,一个格式正确、submitted_prompt 非空但没有 cwdUserPromptSubmit 事件会继续从磁盘加载管理员的 instance 与 dialect 文件(加载调用 0 → 1),随后到达 isWithinRepository('/repository', null)isAbsolute(undefined) 抛出 ERR_INVALID_ARG_TYPE,而 config.ts:235-236 是在其 try 之外调用它的(try 从 237 行开始),所以该 TypeError 会传播出 runAutoRecall,只被 CLI 层的 .catch(() => ({})) 吞掉。设计文档运行流程第 2 步要求的「加载配置前干净跳过」,于是变成一条内部崩溃路径外加每次事件一次无用的配置加载 —— 对外仍然是 {},因此什么都不会暴露。

证据:

INTACT : PROBE settled {"resolved":{}} | loadConfig calls: 0 | isWithinRepository calls: []
MUTANT : PROBE settled {"resolved":{}} | loadConfig calls: 1 | isWithinRepository calls: [["/repository",null]]
         Tests 88 passed (88)
node -e: isAbsolute(undefined) -> TypeError ERR_INVALID_ARG_TYPE

请在 src/auto-recall.test.ts:153it.each 列表中加入 { hook_event_name: 'UserPromptSubmit', submitted_prompt: 'question' }(可再加一个 cwd: 42 变体),使测试能够到达这一行。

扩展用例已有的断言 —— loadAutoRecallRuntimeConfigurationnot.toHaveBeenCalled()searchnot.toHaveBeenCalled() —— 就是验收标准:删除该判断后它们必须变红,因为模拟的加载器届时会为新输入被调用。

— qwen3.8-max via Qwen Code /review (v0.23.0)


expect(settings.$version).toBe(4);
expect(settings.mcpServers).toBeUndefined();
expect(Object.keys(settings.hooks ?? {})).toEqual(['UserPromptSubmit']);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-8: this test is named "keeps the managed Auto Recall … profile Hook-only" but never pins the fixture's own top-level key set, so a privilege-bearing key added to a shipped example passes green.

Both fixtures under examples/ are the files administrators copy verbatim into the settings.json of an administrator-controlled QWEN_HOME — i.e. user scope, which is exactly the scope that honours privilege-bearing keys. settingsSchema.ts:3229-3238 documents security.allowPrivateNetworkHooks as "Only honored from User, System, and SystemDefaults settings scopes; values set in Workspace settings are ignored so a cloned repository cannot self-grant this bypass", and the CLI's own suite pins that at settings.test.ts:3308. The assertions here check $version, mcpServers and the hook event keys only, so the managed profile whose whole selling point is "Hook-only" can silently start shipping a self-granted network bypass — and the gap is invisible because the assertion the test name promises is the one it does not make. The gap is symmetric: the same body asserts the windows fixture with the same three top-level checks.

Witness:

RUN A  "approvalMode": "yolo" added to examples/...-posix.json
       v keeps the managed Auto Recall 'posix' profile Hook-only
       v keeps the managed Auto Recall 'windows' profile Hook-only
       Test Files 1 passed (1) / Tests 4 passed (4)
RUN B  "security": {"allowPrivateNetworkHooks": true} + "tools": {"approvalMode": "yolo"}
       Test Files 9 passed (9) / Tests 88 passed (88)
Suggested change
expect(Object.keys(settings.hooks ?? {})).toEqual(['UserPromptSubmit']);
expect(Object.keys(settings.hooks ?? {})).toEqual(['UserPromptSubmit']);
expect(Object.keys(settings).sort()).toEqual(['$version', 'hooks']);

In both fixtures "hooks" is the first key and "$version": 4 the last (examples/managed-auto-recall-user-settings-posix.json:2 and :19), so Object.keys(settings) yields ['hooks', '$version'] — the .sort() above is what makes the assertion order-independent. Note also that a top-level approvalMode is a legacy v1 key; the canonical v4 location is tools.approvalMode (settingsSchema.ts:2881, mapped by migration/versions/v1-to-v2-shared.ts:67), so use the canonical keys when demonstrating the gap.

The added assertion is its own acceptance criterion: it must go red when a non-hooks/$version key is added to either fixture — both mutations above are green today.

中文说明

[建议] R1-8:这个测试名为 "keeps the managed Auto Recall … profile Hook-only",却从未钉住 fixture 自身的顶层键集合,因此往随包发布的示例里加一个带权限的键也会全绿通过。

examples/ 下的两个 fixture 正是管理员逐字复制进受管 QWEN_HOMEsettings.json 的文件 —— 也就是 user 作用域,而这恰恰是会认这些带权限键的作用域。settingsSchema.ts:3229-3238security.allowPrivateNetworkHooks 的说明是 "Only honored from User, System, and SystemDefaults settings scopes; values set in Workspace settings are ignored so a cloned repository cannot self-grant this bypass",CLI 自身的套件在 settings.test.ts:3308 钉住了该行为。这里的断言只检查 $versionmcpServers 与 Hook 事件键,因此这个以「仅 Hook」为卖点的受管模式可能悄悄开始随包发布一个自授的网络放行 —— 而且这个缺口不可见,因为测试名承诺的那条断言恰恰是它没有做的那条。缺口是对称的:同一个测试体以同样的三条顶层检查断言 windows fixture。

证据:

RUN A  "approvalMode": "yolo" added to examples/...-posix.json
       v keeps the managed Auto Recall 'posix' profile Hook-only
       v keeps the managed Auto Recall 'windows' profile Hook-only
       Test Files 1 passed (1) / Tests 4 passed (4)
RUN B  "security": {"allowPrivateNetworkHooks": true} + "tools": {"approvalMode": "yolo"}
       Test Files 9 passed (9) / Tests 88 passed (88)

(修复代码见上方英文部分的 ```suggestion 块。)

两个 fixture 中 "hooks" 都是第一个键、"$version": 4 是最后一个(examples/managed-auto-recall-user-settings-posix.json:2:19),因此 Object.keys(settings) 得到 ['hooks', '$version'] —— 上面的 .sort() 正是让断言与顺序无关。另请注意,顶层approvalMode 是 v1 遗留键,v4 的规范位置是 tools.approvalModesettingsSchema.ts:2881,由 migration/versions/v1-to-v2-shared.ts:67 映射),所以演示该缺口时请使用规范键。

新增断言本身就是验收标准:当任一 fixture 被加入非 hooks/$version 的键时它必须变红 —— 上面两处变异今天都是绿的。

— qwen3.8-max via Qwen Code /review (v0.23.0)

[`examples/managed-auto-recall-user-settings-windows.json`](./examples/managed-auto-recall-user-settings-windows.json)
into the `settings.json` of an administrator-controlled `QWEN_HOME`. Replace
both command placeholders with fixed absolute Node and `dist/auto-recall.js`
paths. The Mem0 instance and dialect remain independent JSON files; Qwen

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-9: this self-contained Auto Recall recipe never names QWEN_EXTERNAL_CONTEXT_MEM0_CONFIG, the only variable that points the Hook at the v3 instance file — and every resulting failure returns {}, so the omission is undetectable from the running system.

The section spans README.md:200-288 and reads as a complete procedure: write the v3 file, npm install --prefix, copy the Hook definition into $QWEN_HOME/settings.json, replace both command placeholders. The variable appears nowhere in it — its only three README occurrences are :166 and the :171 export, both inside step 4 of the on-demand flow exporting the v2 memory.instance.json, and the :309 MCP-profile error row. It also never mentions injecting the variable named by credentialEnv.

An administrator following only this section gets readRequiredEnvironment throwing on the unset value (config.ts:258-267), swallowed by the CLI-level .catch(() => ({})); one who instead exported the v2 path earlier gets parseAutoRecallInstanceConfig rejecting it, because the v3 schema pins schemaVersion const: 3. Either way the Hook fires on every prompt, exits 0 and returns {} forever — indistinguishable from "no matching memories" — while /hooks shows it registered and succeeding and nothing reaches stderr. The Auto Recall returns {} troubleshooting row (:312) lists eight causes and not this one, and the acceptance checklist never mentions it.

Witness:

shipped dist/auto-recall.js built from unmodified source, same hook input in all rows:
A  variable UNSET            -> exitCode 0, stderr "", stdout {}, provider requests 0
B  variable -> v2 instance   -> exitCode 0, stderr "", stdout {}, provider requests 0
C  variable -> v3 instance   -> stdout {"hookSpecificOutput":{..."additionalContext"...}}, requests 1

Add a step here: "Set QWEN_EXTERNAL_CONTEXT_MEM0_CONFIG to the absolute path of this v3 instance file and inject the variable named by its credentialEnv in the launcher environment, as in step 4. The Hook reads the same variable as the on-demand profile; a missing value or a schemaVersion: 2 file makes every invocation return {}." Then add the variable to the Auto Recall returns {} troubleshooting row and an item to the acceptance checklist.

Both profiles read the same variable — const CONFIG_ENV = 'QWEN_EXTERNAL_CONTEXT_MEM0_CONFIG' (src/config.ts:25), consumed by the shared loadConfiguration that both loadRuntimeConfiguration and loadAutoRecallRuntimeConfiguration call — so the added text must not introduce an Auto Recall-specific variable name or imply the two profiles can point at different files in one process.

中文说明

[建议] R1-9:这段自成体系的 Auto Recall 部署步骤从未提到 QWEN_EXTERNAL_CONTEXT_MEM0_CONFIG —— 唯一把 Hook 指向 v3 instance 文件的变量 —— 而所有由此产生的失败都返回 {},因此从运行中的系统无法察觉这一步被漏掉。

本节覆盖 README.md:200-288,读起来是一套完整流程:编写 v3 文件、npm install --prefix、把 Hook 定义复制进 $QWEN_HOME/settings.json、替换两个命令占位符。该变量在其中一次都没有出现 —— README 中它只有三处::166:171export(都在按需流程第 4 步里,导出的是 v2memory.instance.json),以及 :309 的 MCP 模式错误行。本节也从未提到要注入 credentialEnv 所指定的变量。

只按本节操作的管理员会遇到 readRequiredEnvironment 因变量未设置而抛错(config.ts:258-267),并被 CLI 层的 .catch(() => ({})) 吞掉;而早先导出过 v2 路径的管理员会遇到 parseAutoRecallInstanceConfig 拒绝该文件,因为 v3 schema 把 schemaVersion 钉为 const: 3。两种情况下 Hook 都会在每个 prompt 上触发、以 0 退出并永远返回 {} —— 与「没有匹配的记忆」完全无法区分 —— 同时 /hooks 显示它已注册且成功,stderr 也没有任何输出。Auto Recall returns {} 排障行(:312)列了八种原因,唯独没有这一种;验收清单也从未提到它。

证据:

shipped dist/auto-recall.js built from unmodified source, same hook input in all rows:
A  variable UNSET            -> exitCode 0, stderr "", stdout {}, provider requests 0
B  variable -> v2 instance   -> exitCode 0, stderr "", stdout {}, provider requests 0
C  variable -> v3 instance   -> stdout {"hookSpecificOutput":{..."additionalContext"...}}, requests 1

请在本节补一步:「在启动器环境中把 QWEN_EXTERNAL_CONTEXT_MEM0_CONFIG 设为该 v3 instance 文件的绝对路径,并像第 4 步那样注入其 credentialEnv 所命名的变量。Hook 读取的是与按需模式相同的变量;值缺失或文件是 schemaVersion: 2 都会让每次调用返回 {}。」随后把该变量加入 Auto Recall returns {} 排障行,并在验收清单中加一条。

两个模式读取的是同一个变量 —— const CONFIG_ENV = 'QWEN_EXTERNAL_CONTEXT_MEM0_CONFIG'src/config.ts:25),由 loadRuntimeConfigurationloadAutoRecallRuntimeConfiguration 共同调用的 loadConfiguration 消费 —— 因此新增文字不得引入 Auto Recall 专属的变量名,也不得暗示两个模式可以在同一进程中指向不同文件。

— qwen3.8-max via Qwen Code /review (v0.23.0)

"lint": "eslint src",
"test": "vitest run --config vitest.config.ts",
"test:ci": "vitest run --config vitest.config.ts",
"test": "npm run build && vitest run --config vitest.config.ts",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-10: the dist/auto-recall.js prerequisite is enforced only inside these two scripts, so the direct-vitest invocation AGENTS.md calls "always preferred" runs the new integration test against a stale or missing bundle.

npx vitest run src/auto-recall.integration.test.ts never executes the npm run build && prefix, and this package's vitest.config.ts wires no globalSetup guard, unlike packages/cli and packages/core. Measured with the same source mutation, only the rebuild differing: with src mutated and dist left intact, the four bundle-spawning cases all pass green against yesterday's artifact — so the only end-to-end evidence for the shipped Hook reports a sanitizer regression as safe — while after npm run build the same mutation reddens five tests including all four. The missing-dist arm fails with node's own Cannot find module .../dist/auto-recall.js: the artifact path is in the assertion diff, but nothing names npm run build as the fix, which is the confusion scripts/vitest-global-setup.js exists to remove (its header cites issue #9149). Note the build script runs npm run clean first, so a failed esbuild leaves no bundle at all rather than the previous one.

Witness:

src mutated (grep -c '32,' src -> 0), dist intact (-> 1), npx vitest run --config vitest.config.ts:
  x src/auto-recall.test.ts > bounds sanitizer work before applying credential patterns
  v bounds repeated token   near misses in the bundle with a process deadline
  v bounds repeated api_key near misses in the bundle with a process deadline
  v bounds repeated password near misses in the bundle with a process deadline
  v bounds repeated secret  near misses in the bundle with a process deadline
  Test Files 1 failed | 8 passed (9)
same mutation + npm run build: 5 red, incl. all four bundle cases
  expected '"tokentokentoken..."' to be 'null'   (auto-recall.integration.test.ts:192)
missing-dist arm: 9 failed (9), "Cannot find module <abs>/dist/auto-recall.js"

Give this package a local guard rather than routing through the shared one: a package-local vitest globalSetup (or an in-file precondition in auto-recall.integration.test.ts) that fails before collection when dist/auto-recall.js is absent or older than the newest src/** input, printing npm run build --workspace=integrations/external-context-mem0 as the fix. A Development-section note in the README is the cheaper alternative.

Do not add this package to the shared guard's DIST_PREREQUISITES: scripts/vitest-global-setup.js:108-131 enumerates only manifest.exports and manifest.main, :162-171 fails loud on an empty enumeration, and this package has neither field — so it would break every test run instead of guarding it. For context, the sibling published package has no build prefix at all (integrations/external-context/package.json "test": "vitest run --config vitest.config.ts") and also spawns built artifacts, so this PR already improves on the sibling; the proportionate fix is package-local.

If you add the local guard, scripts/tests/vitest-global-setup.test.js:364-378 already pins the wiring for packages/cli and packages/core via it.each — adding this package to that list makes it pinned, so deleting the globalSetup line later must turn npm run test:scripts red.

中文说明

[建议] R1-10:dist/auto-recall.js 这个前置条件只在这两个脚本内部被保证,因此 AGENTS.md 称为「始终优先」的直接 vitest 调用会让新的集成测试跑在过期或缺失的 bundle 上。

npx vitest run src/auto-recall.integration.test.ts 从不执行 npm run build && 前缀,而本包的 vitest.config.ts 也没有像 packages/clipackages/core 那样接 globalSetup 守卫。用同一处源码变异、只在是否重新构建上区分,实测结果:src 已变异而 dist 保持原样时,四个派生 bundle 的用例全部绿色通过,跑的是昨天的产物 —— 于是关于发布 Hook 的唯一端到端证据把一次脱敏回归报告为安全 —— 而在 npm run build 之后,同一处变异会让五个测试变红,其中包含全部四个。缺少 dist 的分支会以 node 自己的 Cannot find module .../dist/auto-recall.js 失败:产物路径确实出现在断言差异里,但没有任何地方点名 npm run build 才是修法,而这正是 scripts/vitest-global-setup.js 存在的意义(其文件头引用 issue #9149)。注意 build 脚本会先执行 npm run clean,因此 esbuild 失败会完全没有 bundle,而不是保留上一个。

证据:

src mutated (grep -c '32,' src -> 0), dist intact (-> 1), npx vitest run --config vitest.config.ts:
  x src/auto-recall.test.ts > bounds sanitizer work before applying credential patterns
  v bounds repeated token   near misses in the bundle with a process deadline
  v bounds repeated api_key near misses in the bundle with a process deadline
  v bounds repeated password near misses in the bundle with a process deadline
  v bounds repeated secret  near misses in the bundle with a process deadline
  Test Files 1 failed | 8 passed (9)
same mutation + npm run build: 5 red, incl. all four bundle cases
  expected '"tokentokentoken..."' to be 'null'   (auto-recall.integration.test.ts:192)
missing-dist arm: 9 failed (9), "Cannot find module <abs>/dist/auto-recall.js"

请给本包一个本地守卫,而不是走共享守卫:一个包内 vitest globalSetup(或在 auto-recall.integration.test.ts 内做前置检查),在 dist/auto-recall.js 缺失或比最新的 src/** 输入更旧时于收集前失败,并打印 npm run build --workspace=integrations/external-context-mem0 作为修法。更轻的替代是在 README 的开发章节加一句说明。

不要把本包加入共享守卫的 DIST_PREREQUISITESscripts/vitest-global-setup.js:108-131 只枚举 manifest.exportsmanifest.main:162-171 在枚举为空时会大声失败,而本包两个字段都没有 —— 那样会破坏每一次测试运行,而不是守卫它。作为背景,同级的已发布包完全没有 build 前缀(integrations/external-context/package.json"test": "vitest run --config vitest.config.ts")却同样派生构建产物,所以本 PR 已经比同级更好;相称的修法是包内本地化。

如果你加了本地守卫,scripts/tests/vitest-global-setup.test.js:364-378 已通过 it.eachpackages/clipackages/core 钉住接线 —— 把本包加入该列表即可钉住,之后删除 globalSetup 那一行必须让 npm run test:scripts 变红。

— qwen3.8-max via Qwen Code /review (v0.23.0)

return {};
}

const query = createAutoRecallQuery(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-11: nothing pins that runAutoRecall passes runtime.credential into createAutoRecallQuery, so the README's "removes … the configured credential" guarantee is unwitnessed at the Hook level.

createAutoRecallQuery is unit-tested with the credential as an explicit second argument, which is exactly why this wiring is invisible: every runAutoRecall case uses prompts that do not contain the fixture credential (auto-recall.test.ts:311), and the integration suite's credential ('runtime-token', auto-recall.integration.test.ts:333) appears only in the asserted authorization header at :117 and in none of its five prompts.

Measured: passing '' here keeps all 88 tests green while the configured credential is forwarded verbatim as the provider search query — a mixed prompt that should sanitize to 'deployment policy' becomes 'deployment policy provider-secret-value', and a credential-only prompt that should return {} with no search at all instead issues a search for the credential itself. The concrete cost: a user who pastes their mem0 token into a prompt — or pastes a log or config error that echoes it — has that token sent as the search query, where the provider persists it as a searchable memory record that later recall turns echo back into model context. That is strictly worse than the transient Authorization header the same endpoint already receives, and it contradicts README.md:282-284 ("the Hook removes fenced code, the configured credential, and common secret shapes").

Witness:

probe driven through runAutoRecall with the fixture credential 'provider-secret-value':
INTACT : mixed prompt        -> [[{"query":"deployment policy",...}]]
         credential-only     -> {"output":{},"searchCalls":[]}
MUTANT (this call passes ''):
         mixed prompt        -> [[{"query":"deployment policy provider-secret-value",...}]]
         credential-only     -> {"output":{},"searchCalls":[[{"query":"provider-secret-value",...}]]}
package suite with the mutant: 88 green; tsc --noEmit clean

Add one runAutoRecall case to src/auto-recall.test.ts that puts the fixture credential inside the prompt — submitted_prompt: 'deployment provider-secret-value policy' with search.mockResolvedValue([]) — asserting expect(search).toHaveBeenCalledWith({ query: 'deployment policy', signal: expect.any(AbortSignal) }). Mirror it in auto-recall.integration.test.ts (a prompt containing 'runtime-token', asserting the recorded request body.query does not) to pin it end to end.

auto-recall.ts:69 is if (!query) return {};, so a prompt consisting only of the credential sanitizes to undefined and that variant must expect {} plus expect(search).not.toHaveBeenCalled(), not a search with an empty query; createAutoRecallQuery also strips the credential only when it is non-empty (auto-recall.ts:96). Assert both shapes or the new test will not kill the mutant.

The acceptance criterion: the new case must go red when this call passes '' instead of runtime.credential — today all 88 tests pass under that mutation.

中文说明

[建议] R1-11:没有任何测试钉住 runAutoRecall 会把 runtime.credential 传给 createAutoRecallQuery,因此 README 中「移除……所配置的凭证」这条保证在 Hook 层面没有证据。

createAutoRecallQuery 的单元测试是把凭证作为显式第二个参数传入的,这恰恰让这条接线不可见:所有 runAutoRecall 用例使用的 prompt 都不含 fixture 凭证(auto-recall.test.ts:311),而集成套件的凭证('runtime-token'auto-recall.integration.test.ts:333)只出现在 :117 被断言的 authorization 头里,五个 prompt 中一个都没有。

实测:把这里改成传 '',88 个测试全部保持绿色,而所配置的凭证会被原样当作 provider 的搜索 query 发出 —— 本应脱敏为 'deployment policy' 的混合 prompt 变成 'deployment policy provider-secret-value',而本应返回 {} 且完全不搜索的「仅凭证」prompt 反而会发起一次以凭证本身为内容的搜索。具体代价:用户把自己的 mem0 token 粘进 prompt —— 或粘贴一段回显了它的日志/配置报错 —— 该 token 就会作为搜索 query 发出,provider 会把它持久化为可检索的记忆条目,并在之后的召回轮次里回灌进模型上下文。这严格劣于同一 endpoint 本来就会收到的短暂 Authorization 头,也与 README.md:282-284("the Hook removes fenced code, the configured credential, and common secret shapes")相矛盾。

证据:

probe driven through runAutoRecall with the fixture credential 'provider-secret-value':
INTACT : mixed prompt        -> [[{"query":"deployment policy",...}]]
         credential-only     -> {"output":{},"searchCalls":[]}
MUTANT (this call passes ''):
         mixed prompt        -> [[{"query":"deployment policy provider-secret-value",...}]]
         credential-only     -> {"output":{},"searchCalls":[[{"query":"provider-secret-value",...}]]}
package suite with the mutant: 88 green; tsc --noEmit clean

请在 src/auto-recall.test.ts 中加一个 runAutoRecall 用例,把 fixture 凭证放进 prompt —— submitted_prompt: 'deployment provider-secret-value policy',配合 search.mockResolvedValue([]) —— 断言 expect(search).toHaveBeenCalledWith({ query: 'deployment policy', signal: expect.any(AbortSignal) })。并在 auto-recall.integration.test.ts 中镜像一份(prompt 含 'runtime-token',断言记录到的请求 body.query 不含它),以端到端钉住。

auto-recall.ts:69if (!query) return {};,因此由凭证组成的 prompt 会脱敏为 undefined,该变体必须断言 {}expect(search).not.toHaveBeenCalled(),而不是断言一次空 query 的搜索;createAutoRecallQuery 也只在凭证非空时才剥离它(auto-recall.ts:96)。请断言两种形态,否则新测试杀不掉该变异体。

验收标准:当此处传 '' 而非 runtime.credential 时,新用例必须变红 —— 今天在该变异下 88 个测试全部通过。

— qwen3.8-max via Qwen Code /review (v0.23.0)

import { renderResult } from './profile.js';
import { createRequestEngine } from './request-engine.js';

const MAX_HOOK_INPUT_BYTES = 1024 * 1024;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Suggestion] R1-12: this bound is counted over the whole hook payload, which is dominated by the prompt field the Hook explicitly never reads — so the effective ceiling on a user's own prompt is about half the documented 1 MiB, and the resulting skip is silent and undocumented.

Core sends both fields with the same text: client.ts:3224 sets prompt and nonInteractiveCli.ts:1098 sets submittedPrompt (interactively, AppContainer.tsx:3086 does the same), and hookEventHandler.ts:163-170 writes them in one object. hookRunner.ts:1295 writes JSON.stringify(input) to the child's stdin with no cap — only hook output is capped, at hookRunner.ts:48.

Measured by binary search against the built bundle: with the duplicated field the largest submitted_prompt that still recalls is 524 160 characters and the first that skips is 524 224; without the duplicate it is 1 048 320. So qwen -p "$(cat build.log)" with a ~600 KiB log makes readHookInput cross the bound and return undefined, and the executable emits {} before configuration is even loaded — even though the two fields the Hook actually consumes are valid and small, and the sanitizer would only ever have used the first 4096 code points. docs/design/external-context-mem0-auto-recall.md:94 says "Read at most 1 MiB of Hook JSON from stdin", which reads as a prompt bound and is not. The skip is silent ({}, empty stderr, no provider request) and appears in neither documented {} list — the design doc's step 8 at :105-106, nor README.md:277-278 — nor in the Auto Recall returns {} troubleshooting row at README.md:312, so an administrator debugging "recall silently stopped on big prompts" has no documented cause.

Witness:

boundary binary-searched against the shipped dist/auto-recall.js as a subprocess
against a local provider, ~12 spawns per arm:
{"duplicate":true, "maxPromptCharsThatRecall":524160, "firstPromptCharsThatSkip":524224,
 "atBoundaryPayloadBytes":1048559, "overBoundarySearched":false,
 "overBoundaryStdout":"{}", "overBoundaryStderr":""}
{"duplicate":false,"maxPromptCharsThatRecall":1048320,"firstPromptCharsThatSkip":1048384, ...}
Declaration: the stdin payload is a MODEL of core's hook input assembled from the four
core sites above; the CLI was not driven end to end.

The cheap correct fix is documentation: name oversized Hook input in both {} lists and in the troubleshooting row, and state the effective ~512 KiB ceiling on submitted_prompt given that core duplicates the text into prompt. The behavioural alternative is to bound what the Hook consumes rather than the whole payload — raise this constant comfortably above the largest payload core can emit, or read only far enough to extract submitted_prompt and cwd.

docs/design/external-context-mem0-auto-recall.md:94 ("Read at most 1 MiB of Hook JSON from stdin") and the "Bounds and failure semantics" list at :123, which records the other five bounds, must both be updated with any change to this constant.

src/auto-recall.test.ts:229-247 ("accepts exactly 1 MiB and rejects one additional byte before configuration") pins the threshold on both sides and that the oversized case skips loadAutoRecallRuntimeConfiguration; it must be re-pointed at whatever bound you choose, otherwise the constant can drift again with the suite green.

中文说明

[建议] R1-12:这个上限是按整个 Hook payload 计数的,而 payload 的主体恰是 Hook 明确从不读取的 prompt 字段 —— 因此用户自己 prompt 的实际可用上限约为文档所述 1 MiB 的一半,而且由此产生的跳过是静默且未记录的。

core 会以相同文本同时发送两个字段:client.ts:3224 设置 promptnonInteractiveCli.ts:1098 设置 submittedPrompt(交互路径由 AppContainer.tsx:3086 同理),hookEventHandler.ts:163-170 把它们写进同一个对象。hookRunner.ts:1295JSON.stringify(input) 写入子进程 stdin 且没有上限 —— 只有 Hook 的输出有上限,见 hookRunner.ts:48

对构建产物做二分实测:在存在重复字段时,仍能触发召回的最大 submitted_prompt524 160 字符,首个被跳过的是 524 224;不存在重复字段时是 1 048 320。因此 qwen -p "$(cat build.log)" 配合约 600 KiB 的日志会让 readHookInput 越过上限并返回 undefined,可执行文件于是在加载配置之前就输出 {} —— 尽管 Hook 真正消费的两个字段既有效又很小,而脱敏本来也只会用前 4096 个码点。docs/design/external-context-mem0-auto-recall.md:94 写的是 "Read at most 1 MiB of Hook JSON from stdin",读起来像是 prompt 的上限,但并不是。该跳过是静默的({}、stderr 为空、无 provider 请求),且两份 {} 清单都没有它 —— 设计文档第 8 步 :105-106、以及 README.md:277-278 —— README.md:312Auto Recall returns {} 排障行也没有,因此管理员在排查「大 prompt 上召回静默失效」时没有任何文档化的原因可循。

证据:

boundary binary-searched against the shipped dist/auto-recall.js as a subprocess
against a local provider, ~12 spawns per arm:
{"duplicate":true, "maxPromptCharsThatRecall":524160, "firstPromptCharsThatSkip":524224,
 "atBoundaryPayloadBytes":1048559, "overBoundarySearched":false,
 "overBoundaryStdout":"{}", "overBoundaryStderr":""}
{"duplicate":false,"maxPromptCharsThatRecall":1048320,"firstPromptCharsThatSkip":1048384, ...}
Declaration: stdin payload 是依据上述四个 core 位点拼出的 core Hook 输入**模型**;
并未端到端驱动 CLI。

最省事的正确修法是文档:在两份 {} 清单与排障行中点名「Hook 输入超限」,并说明由于 core 会把文本复制进 promptsubmitted_prompt 的实际可用上限约为 512 KiB。行为层面的替代方案是让上限约束 Hook 真正消费的内容而不是整个 payload —— 把该常量提高到远超 core 可能发出的最大 payload,或只读取到足以取出 submitted_promptcwd 为止。

任何对该常量的修改都必须同时更新 docs/design/external-context-mem0-auto-recall.md:94("Read at most 1 MiB of Hook JSON from stdin")与 :123 的 "Bounds and failure semantics" 清单(其中记录了另外五条边界)。

src/auto-recall.test.ts:229-247("accepts exactly 1 MiB and rejects one additional byte before configuration")在两侧钉住了阈值,并钉住了超限用例会跳过 loadAutoRecallRuntimeConfiguration;它必须随你选定的新边界一并调整,否则该常量仍可能在套件全绿的情况下再次漂移。

— qwen3.8-max via Qwen Code /review (v0.23.0)

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Historical-head review — head moved to c0fc7ce5fce0cb5791929d12f4dc6dd4574a3bf2 while this review was in flight (past the salvage threshold), so the run finished and posted against the head it reviewed: 529ac621e33fa82356a765494eff1ae9489dd908 (#10110). The next automatic review covers the delta from that anchor. Full log in the workflow run.

中文说明

历史 head 评审 —— 本次评审进行中 head 移动到了 c0fc7ce5fce0cb5791929d12f4dc6dd4574a3bf2(已过 salvage 阈值),因此评审跑完并针对其实际评审的 head 529ac621e33fa82356a765494eff1ae9489dd908 发布(#10110)。下一次自动评审将从该锚点起评审增量。完整日志见 workflow 运行

@wenshao

wenshao commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Local verification on Linux — merge-ready from my side

I built a real environment for this on Linux and drove the shipped artifacts rather than reading them. Verdict: I found nothing blocking at c0fc7ce5fce0cb5791929d12f4dc6dd4574a3bf2. The one thing the sandboxed run flagged as blocking — the non-deterministic integration test — is fixed at this head, and I measured how much the fix buys. Two things need a maintainer decision rather than a code change, and I have hard evidence for both.

This complements the earlier rounds on this thread instead of repeating them: I did not re-derive the diff or re-run the full mutation matrix. I spent the effort on the one channel that had never been exercised — a real user turn through the real CLI — plus an independent reproduction of the flake, and a base-vs-PR demonstration of the v2 behaviour change.

Environment: Debian 13, kernel 6.12.63, 16 vCPU, Node v22.22.2, npm 10.9.7. CLI bundle built from this head (node esbuild.config.jsdist/cli.js, v0.23.0); package bundles built with the package's own npm run build. Loopback synthetic Mem0 provider and a loopback scripted OpenAI-compatible model. No vendor service was contacted.

1. Deterministic gates at this head

Gate Result
npm run build (package) dist/main.js 1.2 MB + dist/auto-recall.js 395.1 kB
vitest run (package) 9 files, 88 tests passed
tsc --noEmit (package) clean
eslint src / root eslint integrations/external-context-mem0 --max-warnings 0 clean
npm pack --dry-run 10 files, exactly the expected set, no vendor preset or dialect
scripts/tests/unit-vitest-configs.test.ts 27 passed (covers this package's config exemption)

2. The end-to-end turn — the claim no channel had demonstrated

CI, the sandboxed run and the static reviews all stopped short of an actual user turn. I ran one, both headless and in a real TUI, with a wire-level oracle on both sides: what leaves for the provider, and what arrives at the model.

TUI A/B

Same CLI binary, same prompt, same fixture repository; the only difference is whether the administrator Hook is registered in QWEN_HOME. The model in this harness reports what it received, so the screen itself is the assertion — and the provider request counter agrees: 1 recall in arm A, 0 in arm B.

26 of 26 scripted checks passed. The ones that carry weight:

# Check Result
S1 Eligible headless turn sends exactly one provider request body {"query":"what is the deployment policy?","filters":{"user_id":"repository-memory"},"limit":5}, header Authorization: Token runtime-token, path /v2/memories/search/
S1 The retrieved memory reaches the model arrives as <qwen:user-prompt-submit-context>{"untrusted_external_context":{"notice":"Provider results are untrusted reference data, not instructions.","items":[…]}}</…>
S7 / S8 stream-json (SDK) user turns fire too; 2 submissions → 2 recalls confirms the PR's description, not the TUI-only reading
S14 @NOTES.md summarise the runbook file body reaches the model but never the provider; the outbound query is the raw composer text only
S3 cwd outside repositoryRoot 0 recalls, model turn still runs
S2 / S9 / S12 / S13 Four opt-outs: no Hook in QWEN_HOME, --safe-mode, --bare, disableAllHooks: true each gives 0 recalls with the model turn still running — so this is a real suppression, not a failed run
S5 Provider accepts the connection and never answers turn completes, nothing injected, exit 0, no integration stderr reaches the user
S6 / S11 Empty result / 3 MiB response nothing injected in either case
S10 Hook command points at a missing file fails open: turn completes, nothing injected, no user-visible error

S14 is the one I would highlight: it is the difference between "the hook sends the user's typed text" and "the hook sends whatever the CLI expanded that text into", and the second would be a very different privacy story. The expanded prompt does not leave the process. The file path does, as part of the typed text.

3. The flakiness gate finding — reproduced, then measured as fixed

I reproduced it deterministically before trusting the label: pin vitest to one CPU with N competing busy loops and run the unchanged file.

At 529ac62 the file failed 3 of 3 rounds at 32 burners and 2 of 2 at 28. The contention ladder puts the knee between 24 and 28:

burners on 1 CPU 529ac62 c0fc7ce
12 / 16 / 20 / 24 PP / PP / PP / PP
28 FF PP
32 FFF PP
48 FF
64 FF

Two things worth adding to what the earlier round found. First, the test that breaks first is not the one that was named: it is the four it.each sanitizer cases, and their failure text is expected Error: spawnSync /usr/bin/node ETIMEDOUT { …(5) } to be undefined — a message that reads like a ReDoS regression in shipped code when it is really a slow runner. That is a worse failure mode than a plain vitest timeout, because it points the reader at the wrong file. Second, the headroom on an idle machine was never small: test 1 runs in ~129 ms against a 5000 ms budget (≈39×), the sanitizer cases in ~64 ms against a 2000 ms subprocess deadline (≈31×). It takes roughly a 30× slowdown to break, which is why CI mostly stays green and the gate caught it only once in five.

The fix commit is load-bearing and I measured it under identical contention — same machine, same 28 burners, only the test file swapped:

flake before/after

Separating the startup budget from the execution budget — raising the subprocess deadline to 8000 ms while asserting the sanitizer's own elapsedMs < 2000 measured inside the child — is the right shape, and it moves the breaking point from ~28 to ~48 burners.

One non-blocking residual, same class, in tests the fix did not touch. At 48 burners the first failure is now exits successfully after a provider timeout during a stalled TLS handshake at 6176 ms, and at 64 the two FIFO cases join it — all three from expect(performance.now() - startedAt).toBeLessThan(5000), an absolute wall clock that still includes Node startup and a 395 kB bundle load. Applying the same startup/execution split to those three assertions would finish the job. Not a merge blocker: they now sit ~1.7× further out than the cases that actually tripped the gate.

4. The v2 special-file change — demonstrated base vs PR

readConfigFile is shared, so the new regular-file requirement also lands on the already-published v2 MCP path. The PR discloses this; here is what it looks like against a real StdioClientTransport, with the base build (421393d51d) and the PR build side by side:

MCP compatibility matrix

A v2 instance file delivered through a FIFO with a live writer starts fine on the base build and is rejected on this one (Mem0 extension instance configuration is unavailable.). Symlinks to regular files still work on both, and v3 is correctly refused by MCP. This is a real, if narrow, break for anyone pinning 0.23.0 and feeding configuration through process substitution or a FIFO. I think it is the right default and I would take it, but it is a maintainer call rather than something that should ride quietly inside a feature PR — and if it goes in, it belongs in the release notes.

5. Sanitizer: the cap is the only bound, and I can show why

Measured against the shipped bundle:

  • Cap active: flat ~17 ms for hyphen-dense, dot-free input from 4 K to 256 K code points.
  • Cap reverted, same JWT-shaped pattern (auto-recall.ts:104-107): 1.32 / 4.10 / 16.38 / 70.79 / 267.48 ms at 1 K / 2 K / 4 K / 8 K / 16 K — clean quadratic.
  • HOOK_WALL_CLOCK_TIMEOUT_MS cannot rescue it. On a 200 K-character input the uncapped regex spun 171 725 ms synchronously and a 50 ms timer set beforehand never fired. MAX_SANITIZER_INPUT_CHARACTERS is not one of two defences, it is the only one.
  • The rewritten SECRET_ASSIGNMENT_PATTERN measures 0.05–0.10 ms on 4096 repeated token / api_key / password / secret. The comment above it is accurate about that pattern; it just sits above the cheap one.

Worst case at runtime is bounded and I measured that too: against a provider that accepts the socket and never replies, at the schema maximum timeoutMs: 5000, the hook exits 5069 ms with {}, exit 0, empty stderr — inside its own 6500 ms deadline and well inside the examples' 8000 ms hook timeout.

6. What actually leaves the process (informational, for the egress decision)

The PR says "best effort, not DLP", which is honest. Since the maintainer call here is about egress, here is the concrete shape of it, run through the shipped createAutoRecallQuery:

Input Sent to the provider
API_KEY=AKIAIOSFODNN7EXAMPLE set for prod — removed
a fenced block holding AWS_SECRET_ACCESS_KEY=… (closed or unclosed fence) removed
Bearer sk-proj-… / JWT / any bare token ≥32 chars removed
{"password": "hunter2", …} / password: hunter2 removed
the configured credentialEnv value removed
AKIAIOSFODNN7EXAMPLE standing alone sent (20 chars, no assignment)
ghp_abc123, xoxb-2401-2401-abcd1234efgh sent
postgres://admin:hunter2@db.internal:5432/prod sent in full
my password is hunter2 (prose, no =/:) sent
e-mail, phone, internal hostnames, customer names, @file paths sent

Nothing here contradicts the PR — it is what "best effort" means in practice. I am recording it so the decision to enable this in a given deployment is made against measured behaviour rather than an adjective.

7. Repository containment — 14 of 14

Root itself, a nested subdirectory, and a symlink whose target is inside the repository are accepted. A sibling directory sharing the root's name prefix, an unrelated directory, an in-repo symlink pointing outside, a relative cwd, a missing cwd, and / are all refused. A repositoryRoot that is /, relative, missing, or a regular file is refused at load. Every refusal is {}, exit 0, empty stderr.

8. Independent mutation spot-check

I re-ran three of the five hunks myself rather than take the sandbox's word:

Mutation Test that caught it
Remove the 4096-character sanitizer cap bounds sanitizer work before applying credential patterns
Restore blocking open() and drop the regular-file check both rejects a stalled instance/dialect FIFO cases
Drop process.stdout.end(() => process.exit(0)) exits successfully after a provider timeout during a stalled TLS handshake

All restored afterwards; 88/88 again.

9. Smaller notes

  • docs/users/features/hooks.md:624 is now wrong in a way that matters here. It states that "ACP, headless, serve, SDK, and remote-input paths do not produce it in this version"; S1 and S7 above show qwen -p and stream-json SDK input both produce submitted_prompt and both trigger a recall. packages/cli/src/ui/components/hooks/constants.ts:247 calls the field "the supported interactive TUI text projection", which is incomplete the same way. This PR's own README and design doc are correct — it is core's user-facing docs that are stale, and they are what an administrator reads before deciding whether automation is in scope. Worth a separate issue; nothing under packages/ belongs in this PR.
  • The README's new qwen --bare -p '…' opt-out needs one more flag in practice. --bare skips implicit startup discovery, so an administrator whose auth comes from security.auth.selectedType in settings gets No auth type is selected. Please configure an auth type … before running in non-interactive mode. and the run does nothing. qwen --bare --auth-type openai -p '…' works, and --safe-mode and disableAllHooks: true both work as written. Worth a word in the README, since the three options are presented as interchangeable.
  • Running one test file without a prior build fails all 9 cases, but the reason is legible — Error: Cannot find module …/dist/auto-recall.js is printed. A globalSetup guard would still be friendlier than nine assertion diffs.
  • This package is not typechecked by any per-PR job. Root typecheck is not invoked by any PR workflow, and typecheck:integration compiles integration-tests/tsconfig.json, not this package. eslint does cover it via the root eslint .. Pre-existing repo gap, not introduced here — I ran tsc --noEmit locally and it is clean.
  • Screenshots and the full terminal captures are on assets-pr11246 at 756bc4d.

10. Not validated

Windows and macOS runtime behaviour (the CI matrix skips both for this PR, so there is still no platform signal there); ACP and serve transports; any live vendor service; long-run stability of the Hook process under sustained load.

中文说明

Linux 本地验证 —— 我这边认为可以合入

我在 Linux 上搭了一套真实环境,直接驱动发布产物而不是只读代码。结论:在 c0fc7ce5fce0cb5791929d12f4dc6dd4574a3bf2 上我没有发现阻塞项。 沙箱运行标记为阻塞的那一项(不确定的集成测试)在这个 head 上已经修好,而且我量化了这次修复带来的余量。另有两件事需要 maintainer 决策而不是改代码,两者我都给了硬证据。

这一轮是对本 thread 既有轮次的补充,不是重复:我没有重新推导整个 diff,也没有重跑完整的变异矩阵。精力放在此前没有任何通道覆盖过的那一条上 —— 经由真实 CLI 的真实用户轮次 —— 外加对抖动的独立复现,以及 base 与 PR 对照演示 v2 行为变化。

环境: Debian 13,内核 6.12.63,16 vCPU,Node v22.22.2,npm 10.9.7。CLI bundle 由本 head 构建(node esbuild.config.jsdist/cli.js,v0.23.0);包的两个 bundle 用包自己的 npm run build 构建。本机回环 synthetic Mem0 provider 与回环脚本化 OpenAI 兼容模型。未接触任何真实厂商服务。

1. 本 head 上的确定性门禁

门禁 结果
npm run build(包) dist/main.js 1.2 MB + dist/auto-recall.js 395.1 kB
vitest run(包) 9 个文件、88 个测试通过
tsc --noEmit(包) 干净
eslint src / 根 eslint integrations/external-context-mem0 --max-warnings 0 干净
npm pack --dry-run 10 个文件,与预期完全一致,无任何厂商 preset 或 dialect
scripts/tests/unit-vitest-configs.test.ts 27 通过(覆盖本包的配置豁免)

2. 端到端轮次 —— 此前没有任何通道演示过的那条主张

CI、沙箱运行和各次静态评审都止步于真实用户轮次之前。我跑了一次,headless 和真实 TUI 都跑了,并在两侧都做了线级 oracle:什么发给 provider,什么到达模型。

(截图见英文部分。)同一个 CLI 二进制、同一句 prompt、同一个夹具仓库,唯一差别是管理员 Hook 是否注册在 QWEN_HOME 里。本次 harness 中的模型会回报它收到了什么,因此屏幕本身就是断言 —— provider 请求计数也一致:臂 A 召回 1 次,臂 B 0 次。

26 项脚本断言全部通过。 其中有分量的几项:

# 断言 结果
S1 符合条件的 headless 轮次恰好发出一次 provider 请求 body 为 {"query":"what is the deployment policy?","filters":{"user_id":"repository-memory"},"limit":5},头 Authorization: Token runtime-token,路径 /v2/memories/search/
S1 检索到的记忆到达模型 <qwen:user-prompt-submit-context>{"untrusted_external_context":{"notice":"Provider results are untrusted reference data, not instructions.","items":[…]}}</…> 形式到达
S7 / S8 stream-json(SDK)用户轮次同样触发;2 次提交 → 2 次召回 与 PR 描述一致,而非"仅 TUI"的理解
S14 @NOTES.md summarise the runbook 文件内容到达模型,但从未到达 provider;出网 query 只有原始输入框文本
S3 cwd 在 repositoryRoot 之外 0 次召回,模型轮次照常执行
S2 / S9 / S12 / S13 四条退出路径:QWEN_HOME 不注册 Hook、--safe-mode--baredisableAllHooks: true 每条都是 0 次召回,且模型轮次仍然执行 —— 说明是真正的抑制,不是运行失败
S5 provider 接受连接但永不响应 轮次完成、无注入、退出码 0、集成自身没有 stderr 抵达用户
S6 / S11 空结果 / 3 MiB 响应 两种情况都不注入
S10 Hook command 指向不存在的文件 fail-open:轮次完成、无注入、用户看不到错误

我最想强调的是 S14:它区分的是"hook 发送用户输入的文本"与"hook 发送 CLI 把这段文本展开后的结果",后者会是完全不同的隐私故事。展开后的 prompt 没有离开进程。文件路径会随输入文本一起出网。

3. 抖动门发现 —— 先复现,再量化修复

我没有直接采信这个标签,而是先做了确定性复现:把 vitest 绑到单个 CPU 上,同时在同一 CPU 上跑 N 个忙循环,然后原样运行该文件。

529ac62 上,32 burners 时该文件 3 轮全挂,28 burners 时 2 轮全挂。争用阶梯显示拐点在 24 与 28 之间:

单 CPU 上的 burners 529ac62 c0fc7ce
12 / 16 / 20 / 24 PP / PP / PP / PP
28 FF PP
32 FFF PP
48 FF
64 FF

相对上一轮的发现,有两点值得补充。第一,最先挂的并不是被点名的那个测试,而是四个 it.each 脱敏用例,其失败文本是 expected Error: spawnSync /usr/bin/node ETIMEDOUT { …(5) } to be undefined —— 这句话读起来像是发布代码里的 ReDoS 回归,实际只是机器慢。这比普通的 vitest 超时更糟,因为它把读者指向了错误的文件。第二,空载时余量本来并不小:test 1 约 129 ms 对 5000 ms 预算(≈39×),脱敏用例约 64 ms 对 2000 ms 子进程期限(≈31×)。需要约 30× 的整体降速才会挂,这也解释了为什么 CI 大多是绿的、而抖动门五轮里只抓到一次。

修复 commit 是承重的,我在完全相同的争用下做了对照 —— 同一台机器、同样 28 burners,只替换测试文件(截图见英文部分)。把启动预算与执行预算分开 —— 子进程期限提到 8000 ms,同时在子进程内部断言脱敏自身的 elapsedMs < 2000 —— 形状是对的,并把断裂点从约 28 burners 推到约 48 burners。

一个非阻塞的残留,同一类问题,在修复未触及的测试里。 48 burners 时最先失败的变成 exits successfully after a provider timeout during a stalled TLS handshake(6176 ms),64 burners 时两个 FIFO 用例也加入 —— 三者都源自 expect(performance.now() - startedAt).toBeLessThan(5000),这个绝对墙钟里仍然包含 Node 启动和 395 kB bundle 加载。把同样的启动/执行拆分应用到这三条断言上就收尾了。不构成合入阻塞:它们现在比真正触发抖动门的用例还要远约 1.7×。

4. v2 特殊文件行为变化 —— base 与 PR 对照实测

readConfigFile 是共用的,因此新的普通文件要求同样落在已发布的 v2 MCP 路径上。PR 已经披露了这一点;下面是用真实 StdioClientTransport、base 构建(421393d51d)与 PR 构建并排跑出来的结果(截图见英文部分)。

通过带活写入者的 FIFO 提供的 v2 实例文件,在 base 构建上能正常启动,在本 PR 上被拒绝Mem0 extension instance configuration is unavailable.)。指向普通文件的符号链接在两侧都仍然可用,v3 也被 MCP 正确拒绝。对于把版本钉在 0.23.0、并通过进程替换或 FIFO 提供配置的部署,这是一个虽窄但真实的破坏。我认为这个默认值是对的、我会接受它,但这属于 maintainer 决策,不应该悄悄搭在功能 PR 里过去 —— 如果合入,它应该进 release notes。

5. 脱敏:cap 是唯一的界限,而且我能给出原因

针对发布 bundle 的实测:

  • cap 生效: 连字符密集、无点的输入从 4 K 到 256 K 码点,稳定在 约 17 ms
  • 回退 cap、同一个 JWT 形状模式auto-recall.ts:104-107):1 K / 2 K / 4 K / 8 K / 16 K 分别为 1.32 / 4.10 / 16.38 / 70.79 / 267.48 ms —— 干净的二次曲线。
  • HOOK_WALL_CLOCK_TIMEOUT_MS 救不了它。 在 20 万字符输入上,无 cap 的正则同步空转了 171 725 ms,而事先设置的 50 ms 定时器一次都没触发。MAX_SANITIZER_INPUT_CHARACTERS 不是两道防线之一,它是唯一的一道。
  • 重写后的 SECRET_ASSIGNMENT_PATTERN 在 4096 个重复 token / api_key / password / secret 上实测 0.05–0.10 ms。它上方那条注释对这个模式而言是准确的,只是它恰好写在便宜的那个上面。

运行时最坏情况同样有界,我也量了:面对接受连接但永不响应的 provider、在 schema 最大值 timeoutMs: 5000 下,hook 在 5069 ms 输出 {} 退出、退出码 0、stderr 为空 —— 落在它自己的 6500 ms 期限内,也远在示例的 8000 ms hook timeout 之内。

6. 实际会出网的内容(信息项,供 egress 决策参考)

PR 写的是"尽力而为,不是 DLP",这很诚实。既然这里的 maintainer 决策关乎数据出网,我把它的具体形状跑了出来(经由发布的 createAutoRecallQuery):

输入 发给 provider 的内容
API_KEY=AKIAIOSFODNN7EXAMPLE set for prod —— 已移除
AWS_SECRET_ACCESS_KEY=… 的围栏代码块(闭合或不闭合) 已移除
Bearer sk-proj-… / JWT / 任何 ≥32 字符的裸 token 已移除
{"password": "hunter2", …} / password: hunter2 已移除
配置的 credentialEnv 取值 已移除
单独出现的 AKIAIOSFODNN7EXAMPLE 会发出(20 字符,无赋值形状)
ghp_abc123xoxb-2401-2401-abcd1234efgh 会发出
postgres://admin:hunter2@db.internal:5432/prod 原样发出
my password is hunter2(自然语言,无 =/: 会发出
邮箱、电话、内网主机名、客户名、@file 路径 会发出

这些都不与 PR 的说法矛盾 —— 这就是"尽力而为"在实践中的样子。我把它记录下来,是为了让"是否在某个部署里启用"这个决定基于实测行为,而不是一个形容词。

7. 仓库边界约束 —— 14/14

仓库根本身、嵌套子目录、目标位于仓库内的符号链接均被接受。共享根名前缀的兄弟目录、无关目录、指向仓库外的仓库内符号链接、相对 cwd、不存在的 cwd 以及 / 全部拒绝。repositoryRoot/、相对路径、不存在或指向普通文件时在加载阶段拒绝。所有拒绝都是 {}、退出码 0、stderr 为空。

8. 独立变异抽检

三个 hunk 我自己重跑了一遍,没有直接采信沙箱结论:

变异 捕获它的测试
移除 4096 字符脱敏 cap bounds sanitizer work before applying credential patterns
恢复阻塞式 open() 并去掉普通文件检查 两个 rejects a stalled instance/dialect FIFO 用例
去掉 process.stdout.end(() => process.exit(0)) exits successfully after a provider timeout during a stalled TLS handshake

事后全部还原,重新 88/88。

9. 其他小项

  • docs/users/features/hooks.md:624 现在是错的,而且恰好错在这里要紧的地方。 它写着 "ACP, headless, serve, SDK, and remote-input paths do not produce it in this version";上面的 S1 与 S7 表明 qwen -p 与 stream-json SDK 输入都会产生 submitted_prompt,也都会触发召回。packages/cli/src/ui/components/hooks/constants.ts:247 把该字段称为 "the supported interactive TUI text projection",同样不完整。本 PR 自己的 README 与设计文档是对的 —— 过时的是 core 的用户文档,而管理员在判断"自动化是否在范围内"时读的正是那份文档。建议单独开 issue;packages/ 下的内容不属于本 PR。
  • README 新增的 qwen --bare -p '…' 退出方式在实践中还需要多带一个参数。 --bare 会跳过隐式启动发现,因此当管理员的鉴权来自 settings 里的 security.auth.selectedType 时,会得到 No auth type is selected. Please configure an auth type … before running in non-interactive mode.,这一次运行什么也不会做。qwen --bare --auth-type openai -p '…' 可以工作,而 --safe-modedisableAllHooks: true 都按文档所写工作。README 里值得补一句,因为这三个选项目前是并列呈现的。
  • 不先构建就单独跑一个测试文件会让 9 个用例全挂,但原因是可读的 —— 会打印 Error: Cannot find module …/dist/auto-recall.js。加一个 globalSetup 守卫仍然比九条断言差异友好。
  • 本包没有任何 per-PR 任务做类型检查。typecheck 不被任何 PR workflow 调用,而 typecheck:integration 编译的是 integration-tests/tsconfig.json,不是本包。eslint 通过根 eslint . 确实覆盖到了。这是仓库既有缺口,不是本 PR 引入的 —— 我本地跑了 tsc --noEmit,干净。
  • 截图与完整终端捕获在 assets-pr11246756bc4d

10. 未验证

Windows 与 macOS 的运行时行为(本 PR 的 CI 矩阵两者都跳过,因此那边仍然没有平台信号);ACP 与 serve 传输;任何真实厂商服务;Hook 进程在持续负载下的长时间稳定性。


🤖 Generated with Claude Code — Claude Opus 5 (1M context)

@chiga0 chiga0 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incremental review of c0fc7ce (1 commit since 529ac62).

Scope: all 3 changed files in this commit (docs/design, README, auto-recall.integration.test.ts).

No blocking findings.

Checked:

  1. Doc updates (design doc + README) — add , , and as explicit automation-disable paths alongside the existing "separate QWEN_HOME" guidance. Accurate description of the options; no behaviour change.

  2. Test budget separation () — subprocess raised from 2000 ms to 8000 ms to absorb Node.js startup latency; the correctness assertion moved to on the measured execution time. This correctly decouples process-startup overhead from the sanitizer-deadline guarantee. The test now validates what it claimed to validate without flaking on slow CI runners.

Reviewed with AI assistance.

@doudouOUC
doudouOUC added this pull request to the merge queue Sep 7, 2026
Merged via the queue into QwenLM:main with commit be9d5e8 Sep 7, 2026
209 of 212 checks passed
@doudouOUC
doudouOUC deleted the feat/external-context-mem0-auto-recall branch September 7, 2026 10:37
@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Released in v0.23.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants