fix(memory): don't advance AutoMemory extract cursor when the agent makes zero tool calls - #6398
Conversation
…o tool calls Fixes QwenLM#6311 The extract cursor previously advanced unconditionally after the forked extractor agent reported 'completed', even when it made zero real tool calls (e.g. a small/local model hallucinating a bash command instead of calling write_file). This silently and permanently skipped those history messages from being reprocessed. Also fixes extractionAgentPlanner.ts using filesTouched (attempted paths, unconfirmed) instead of filesWritten (confirmed successful writes) when deriving touchedTopics, matching the pattern already used in remember.ts. touchedTopics.length > 0 alone is not sufficient to gate the cursor advance: a legitimate 'nothing durable to save' outcome also produces an empty touchedTopics array and would otherwise be treated the same as a hallucinated run. A new hasToolActivity signal (derived from filesTouched, which includes read-only calls like read_file) distinguishes 'agent engaged with the task and found nothing new to save' (legitimate noop, cursor still advances) from 'agent made zero tool calls at all' (hallucination, cursor held for retry).
|
Thanks for the PR, @lcheng321! Template looks good ✓ Problem: This is an observed bug with solid evidence. Issue #6311 includes a full extractor subagent log showing a local model (liquid/lfm2-1.2b) hallucinating a bash script as plain text output instead of calling Direction: AutoMemory extraction reliability is squarely within core scope. The fix targets the exact mechanism described in the issue: unconditional cursor advance on agent completion regardless of whether the agent actually did anything. No CHANGELOG reference, but the area is clearly relevant. Approach: The scope feels right — 2 source files, 4 test files, all tightly focused. The Moving on to code review. 🔍 中文说明感谢 PR，@lcheng321！ 模板完整 ✓ 问题： 这是一个已观测到的 bug，有充分证据。6311 号 issue 包含完整的 extractor 子代理日志，显示一个本地模型（liquid/lfm2-1.2b）将 bash 脚本作为纯文本输出，而不是调用 方向： AutoMemory 提取可靠性完全在核心范围内。修复精确针对 issue 中描述的机制：agent 完成时无条件推进 cursor，不管 agent 实际做了什么。CHANGELOG 中无直接参考，但该领域明显相关。 方案： 范围合理——2 个源文件，4 个测试文件，全部紧密聚焦。 进入代码审查 🔍 — Qwen Code · qwen3.7-max |
Code ReviewIndependent proposal (before reading diff): I would add a boolean signal to the extraction result indicating whether the agent made any real tool calls, then gate cursor advance on that signal. The signal should distinguish hallucination (zero tool activity) from legitimate noop (agent checked existing memory, found nothing new). I'd also fix the Comparison: The PR's approach matches my proposal closely. It adds No correctness bugs, security holes, or regressions found. The code follows project conventions. Two test cases — one for the hallucination bug ( Real-Scenario TestingThis is a backend logic fix in the AutoMemory extraction pipeline. The bug only triggers when a local/small model hallucinating tool calls as plain text — cannot be reproduced on-demand with the available model. The evidence is the unit test before/after. Installed build (v0.19.1) — basic sanityPR code — auto-memory extraction pathExtraction pipeline runs without errors. No regressions in the happy path. Unit Test EvidenceAll 5 test files pass (40 tests total): Bug regression verified: Temporarily reverted the fix in This confirms the test actually catches the bug — Lint: clean (eslint on both changed source files). 中文说明代码审查独立方案（阅读 diff 前）： 我会在提取结果中添加一个布尔信号，表示 agent 是否做过真正的工具调用，然后根据该信号来控制 cursor 推进。该信号需要区分幻觉（零工具活动）和合法 noop（agent 检查了现有 memory 发现没有新内容）。同时修复 对比： PR 的方案与我的独立方案高度一致。添加了 未发现正确性 bug、安全漏洞或回归。代码遵循项目规范。两个测试用例——一个针对幻觉 bug（ 真实场景测试这是 AutoMemory 提取管道中的后端逻辑修复。该 bug 仅在本地/小模型将工具调用幻觉为纯文本时触发——无法用可用模型按需复现。证据来自单元测试的 before/after。 安装版本（v0.19.1）基本功能正常，PR 代码的 auto-memory 提取路径运行无错误，正常路径无回归。 单元测试证据5 个测试文件全部通过（共 40 个测试）。 Bug 回归已验证： 临时撤销 Lint： 干净。 — Qwen Code · qwen3.7-max |
|
This is a clean, well-targeted fix for a real bug. The evidence chain is solid: issue #6311 provides observed log data showing a local model hallucinating bash as plain text during AutoMemory extraction, the cursor JSON confirming silent advancement, and the PR's unit test correctly reproduces the failure without the fix ( The Switching 40 tests pass across all 5 affected test files. Lint clean. No scope creep, no drive-by refactors, no over-engineering. Ships what it promises. Approving. ✅ 中文说明这是一个干净、精准的修复，针对一个真实的 bug。证据链完整：6311 号 issue 提供了观测到的日志数据，显示本地模型在 AutoMemory 提取过程中将 bash 幻觉为纯文本，cursor JSON 确认了静默推进，PR 的单元测试在没有修复的情况下正确复现了失败（
将 5 个受影响的测试文件共 40 个测试全部通过。Lint 干净。无范围蔓延、无顺手重构、无过度工程。交付了承诺的内容。 批准 ✅ — Qwen Code · qwen3.7-max |
qwen-code-ci-bot
left a comment
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
| await Promise.all([projectRebuild, userRebuild]); | ||
| } | ||
|
|
||
| const madeGenuineProgress = |
There was a problem hiding this comment.
[Suggestion] touchedTopics.length > 0 is redundant with hasToolActivity. Since touchedTopics is derived from filesWritten (a strict subset of filesTouched — see forkedAgent.ts lines 568→582), touchedTopics.length > 0 always implies hasToolActivity is already true. The left side of the || can never be the deciding factor.
This could mislead future maintainers into thinking there are two independent paths to cursor advancement. Either simplify:
| const madeGenuineProgress = | |
| const madeGenuineProgress = agentResult.hasToolActivity; |
Or add a comment explaining the defensive intent:
| const madeGenuineProgress = | |
| // touchedTopics.length > 0 implies hasToolActivity today (filesWritten ⊆ filesTouched), | |
| // but the explicit check guards against a future planner refactor that decouples them. | |
| const madeGenuineProgress = | |
| agentResult.touchedTopics.length > 0 || agentResult.hasToolActivity; |
— qwen3.7-max via Qwen Code /review
| status: 'completed', | ||
| finalText: '', | ||
| filesTouched: ['/tmp/auto-memory/user/prefs.md'], | ||
| filesWritten: ['/tmp/auto-memory/user/prefs.md'], |
There was a problem hiding this comment.
[Suggestion] All tests in this file set filesTouched and filesWritten to identical arrays. No test exercises the scenario where they diverge (e.g., agent reads a memory file but decides not to write to it). The PR's core semantic split — filesWritten for topic classification, filesTouched for hasToolActivity — is unverified at the unit level.
Consider adding a test like:
it('derives touchedTopics from filesWritten, not filesTouched', async () => {
vi.mocked(runForkedAgent).mockResolvedValue({
status: 'completed',
finalText: '',
filesTouched: ['/tmp/auto-memory/user/prefs.md', '/tmp/auto-memory/user/readonly.md'],
filesWritten: ['/tmp/auto-memory/user/prefs.md'],
});
const result = await runAutoMemoryExtractionByAgent(mockConfig, '/tmp');
expect(result.touchedTopics).toEqual(['user']);
expect(result.hasToolActivity).toBe(true);
});This proves touchedTopics reflects only filesWritten while hasToolActivity reflects filesTouched.
— qwen3.7-max via Qwen Code /review
✅ Independent verification — merge referenceI independently verified this PR against a real build of the PR worktree ( 1 · Bug reproduction — A/B in a real buildReverting only the 2 · No regressionsAll 5 test files the PR touches pass, and the entire 3 · Real end‑to‑end integration (added coverage) 🔬The PR's unit tests mock the whole planner and inject
The decisive rows are the first two: both produce Why the fix is sound
Minor observation (non‑blocking)
Verified locally in a 🇨🇳 中文版验证报告（点击展开）✅ 独立验证 —— 供合并参考我在 PR 分支的真实构建（worktree 1 · Bug 复现 —— 真实构建下的 A/B仅还原 2 · 无回归PR 改动的 5 个测试文件全部通过，整个 3 · 真实端到端集成（新增覆盖）🔬PR 自带单测把整个 planner mock 掉、直接注入
关键在前两行：两者的 修复为何成立
一点非阻塞性观察
以上均在 PR HEAD |
3 participants
Fixes #6311
The extract cursor previously advanced unconditionally after the forked extractor agent reported completed, even when it made zero real tool calls, for example a small or local model hallucinating a bash command instead of calling write_file. This silently and permanently skipped those history messages from being reprocessed.
Also fixes extractionAgentPlanner.ts using filesTouched, which is attempted and unconfirmed paths, instead of filesWritten, which is confirmed successful writes, when deriving touchedTopics. This matches the pattern already used in remember.ts.
touchedTopics.length greater than 0 alone is not sufficient to gate the cursor advance. A legitimate nothing durable to save outcome also produces an empty touchedTopics array and would otherwise be treated the same as a hallucinated run. A new hasToolActivity signal, derived from filesTouched which includes read only calls like read_file, distinguishes agent engaged with the task and found nothing new to save, which is a legitimate noop where the cursor still advances, from agent made zero tool calls at all, which is the hallucination case where the cursor is held for retry.
What this PR does
This PR changes two files in packages/core/src/memory. extract.ts now only advances processedOffset to the full history length when the extractor agent made genuine progress, meaning touchedTopics.length is greater than 0 or hasToolActivity is true. Otherwise it holds the cursor at its previous offset so the same slice of history gets retried on the next extraction pass instead of being silently skipped forever. extractionAgentPlanner.ts adds a new hasToolActivity field to the AutoMemoryExtractionExecutionResult interface, which is true whenever the agent made at least one real tool call, and switches touchedTopics derivation from filesTouched to filesWritten so only confirmed writes count toward topics touched.
Why it's needed
Issue 6311 shows a local model called liquid slash lfm2 1.2b hallucinating a bash script as plain text instead of calling write_file during managed AutoMemory extraction. The extractor agent still reported status completed, so the cursor advanced past those messages unconditionally. The user's remember this instruction was silently dropped and could never be retried, even in a later session with a stronger model. The triage comment suggested gating the cursor advance on touchedTopics.length greater than 0, but that alone cannot distinguish a legitimate agent checked existing memory and decided there is nothing new to save outcome from the hallucination case, since both produce an empty touchedTopics array. The hasToolActivity signal fixes that ambiguity by checking whether the agent made any tool call at all, including read only ones like read_file, which is the actual signature of a hallucinated run in the issue log.
Reviewer Test Plan
How to verify
This is a pure unit test level fix. runAutoMemoryExtractionByAgent is fully mocked in extract.test.ts at packages/core/src/memory/extract.test.ts, so no real LLM calls are needed to reproduce or verify this.
Step one, reproduce the bug before the fix. With only the new test added and the source fix not yet applied, run this single test in isolation.
cd packages/core
npx vitest run src/memory/extract.test.ts -t "BUG #6311"
This produces a failing test with the following output.
FAIL src/memory/extract.test.ts
BUG #6311: should NOT advance cursor when agent makes zero tool calls (hallucination)
AssertionError: expected 1 to be +0
Expected: 0
Received: 1
Received 1 is the bug itself. Even though the agent made zero tool calls, the cursor still advanced past the single unprocessed message, meaning the instruction Remember that I prefer pnpm over npm would never be retried and would be lost forever.
Step two, apply the fix in extract.ts and extractionAgentPlanner.ts. Rerun the same test.
npx vitest run src/memory/extract.test.ts -t "BUG #6311"
This now passes.
PASS src/memory/extract.test.ts
BUG #6311: should NOT advance cursor when agent makes zero tool calls (hallucination)
Step three, confirm no regressions across every file touched by this change.
npx vitest run src/memory/extract.test.ts
Result: 16 passed, 16 total.
npx vitest run src/memory/extractionAgentPlanner.test.ts
Result: 11 passed, 11 total.
npx vitest run src/memory/extractAgent.test.ts
Result: 1 passed, 1 total.
npx vitest run src/memory/memoryLifecycle.integration.test.ts
Result: 1 passed, 1 total.
npx vitest run src/memory/remember.test.ts
Result: 11 passed, 11 total.
A companion regression test named should advance cursor on legitimate noop confirms the fix does not overcorrect. When the agent genuinely checked memory and found nothing new, meaning hasToolActivity is true and touchedTopics is empty, the cursor still advances normally to the full history length.
Step four, confirm lint and typecheck are clean.
npm run lint
Result: no errors.
npm run typecheck --workspace=packages/core
Result: no errors.
Evidence (Before & After)
This is a non UI, backend only logic fix, so there are no screenshots. The evidence is the exact numeric value of processedOffset before and after the fix, shown through the test assertion output above.
Before the fix, for the hallucination scenario where the agent completes with zero tool calls, processedOffset was 1 when it should have stayed at 0. This is a silent numeric error, not a crash and not a visible error message to the end user, which is exactly why it went unnoticed until someone inspected the cursor file directly, as described in the original issue.
After the fix, for that same hallucination scenario, processedOffset correctly stays at 0, so the unprocessed message will be retried on the next extraction pass.
For the legitimate noop scenario, where the agent has hasToolActivity true but touchedTopics empty, processedOffset correctly advances to the full history length both before and after the fix, confirming the fix only changes behavior for the true hallucination case and does not regress the normal skip logic.
Summary table.
Tested on
Environment (optional)
Local run using npx vitest against packages/core on Windows, Node v24.17.0. runAutoMemoryExtractionByAgent is mocked at the unit test layer throughout, so no real model or sandbox environment is involved in this verification.
Risk & Scope
Main risk or tradeoff: if hasToolActivity is ever computed incorrectly for a legitimate run, for example an agent that only reads memory but genuinely has nothing to save, the cursor could fail to advance and the same history slice would be retried indefinitely. The included legitimate noop regression test guards against this specific case.
Not validated or out of scope: behavior against a real local or small model actually hallucinating, as in the original issue log, was not re verified end to end. This fix targets the exact mechanism described in the issue, meaning zero tool calls on a completed run, at the unit test level. The companion issue 6308 about configurable extractor timeouts is related but not addressed here.
Breaking changes or migration notes: AutoMemoryExtractionExecutionResult gains a new required field hasToolActivity. Any other caller constructing this type, whether in test mocks or elsewhere, will need to supply it. This PR updates all in repo mocks accordingly.
Linked Issues
Fixes #6311
中文说明
修复 6311 号问题
之前 extract cursor 在 forked extractor agent 报告 completed 状态后会无条件推进，即使 agent 实际上没有做任何真正的工具调用，例如本地或小模型把 bash 命令幻觉成了纯文本，而不是调用 write_file。这会导致这些历史消息被永久性地静默跳过，再也不会被重新处理。
同时修复了 extractionAgentPlanner.ts 中使用 filesTouched，也就是尝试写入但未确认成功的路径，而不是 filesWritten，也就是确认成功写入的路径，来推导 touchedTopics 的问题，使其与 remember.ts 中已有的写法保持一致。
仅凭 touchedTopics.length 大于 0 不足以作为 cursor 推进的判断条件。合法的没有新东西需要保存的结果同样会产生空的 touchedTopics 数组，如果只看这个字段，会和幻觉场景混为一谈。新增的 hasToolActivity 信号，从 filesTouched 派生，包含只读调用如 read_file，能够区分 agent 认真检查过确实没有新东西这种合法 noop，此时 cursor 正常推进，和 agent 完全没有调用任何工具这种幻觉场景，此时 cursor 保持不变以便重试。
这个 PR 做了什么
在 packages 下 core 下 src 下 memory 目录中改动了两个文件。extract.ts 现在只有在 extractor agent 确实取得了实质性进展时，也就是 touchedTopics.length 大于 0 或者 hasToolActivity 为 true，才会将 processedOffset 推进到完整的 history 长度。否则 cursor 保持在之前的 offset，让同一段历史消息在下一次提取时被重新处理，而不是被永久跳过。extractionAgentPlanner.ts 在 AutoMemoryExtractionExecutionResult 接口中新增了 hasToolActivity 字段，只要 agent 做过至少一次真正的工具调用就为 true，并将 touchedTopics 的推导来源从 filesTouched 切换为 filesWritten，确保只有确认成功的写入才会被计入涉及的主题。
为什么需要这个改动
6311 号问题中展示了一个本地模型在 managed AutoMemory 提取过程中，把一段 bash 脚本当作纯文本吐出来，而不是调用 write_file 工具。extractor agent 依然报告了 completed 状态，导致 cursor 无条件跳过了这些消息，用户记住这件事的指令被静默丢弃，而且永远无法重试，即使之后换成更强的模型也不行。分类评论建议仅凭 touchedTopics.length 大于 0 来判断，但这无法区分 agent 认真检查了现有记忆判断没有新东西需要保存这种合法结果和幻觉场景，两者产生的 touchedTopics 都是空数组。hasToolActivity 信号通过检查 agent 是否做过任何工具调用，包括只读调用如 read_file，解决了这个歧义，这正是问题日志中幻觉场景的真实特征。
Reviewer 测试方案
这是一个纯单元测试层面的修复，extract.test.ts 中 runAutoMemoryExtractionByAgent 被完全 mock 掉，因此复现和验证都不需要真实的 LLM 调用。
第一步，在修复前复现 bug。只加上新测试但源码还没改的情况下，单独跑这一个测试，命令是进入 packages 下 core 目录，然后跑 npx vitest run，指定 extract.test.ts 文件，参数加上 t 和 BUG 6311。这会产生一个失败的测试，输出显示期望是 0，实际收到的是 1。收到 1 就是 bug 本身，即使 agent 零工具调用，cursor 依然跳过了这条唯一未处理的消息，意味着记住我更喜欢用 pnpm 而不是 npm 这条指令永远不会被重试，会永久丢失。
第二步，应用 extract.ts 和 extractionAgentPlanner.ts 里的修复，重新跑同一个测试，这次测试通过。
第三步，确认所有涉及文件都没有回归。分别跑 extract.test.ts，结果 16 个测试全部通过。跑 extractionAgentPlanner.test.ts，结果 11 个测试全部通过。跑 extractAgent.test.ts，结果 1 个测试通过。跑 memoryLifecycle.integration.test.ts，结果 1 个测试通过。跑 remember.test.ts，结果 11 个测试全部通过。配套新增的回归测试确认这个修复没有矫枉过正，当 agent 确实检查过记忆判断没有新东西时，cursor 依然正常推进。
第四步，确认 lint 和 typecheck 都干净，两者均无报错通过。
证据，修复前和修复后
这是一个非 UI、纯后端逻辑修复，所以没有截图。证据就是 processedOffset 这个数值在修复前后的具体变化。
修复前，对于 agent 完成时零工具调用的幻觉场景，processedOffset 是 1，但本应保持为 0。这是一个静默的数值错误，不是崩溃，也不是给最终用户看到的报错提示，这正是为什么这个问题一直没被发现，直到有人直接检查 cursor 文件才发现，正如原始 issue 里描述的那样。
修复后，同样的幻觉场景下，processedOffset 正确地保持为 0，这样未处理的消息会在下一次提取时被重试。
对于合法 noop 场景，也就是 agent 有工具活动但 touchedTopics 为空，修复前后 processedOffset 都正确地推进到完整长度，确认这个修复只改变了真正幻觉场景下的行为，没有破坏正常的跳过逻辑。
测试环境
Windows 已测试，macOS 和 Linux 未测试。本地在 Windows 上通过 npx vitest run 针对 packages 下 core 跑测试，Node 版本 v24.17.0。全程 runAutoMemoryExtractionByAgent 都在单元测试层被 mock，不涉及真实模型或沙箱环境。
风险与范围
主要风险，如果 hasToolActivity 在某个合法场景下被错误计算，比如 agent 只是读取了记忆但确实没有需要保存的东西却被误判，cursor 可能无法推进，导致同一段历史消息被无限重试。已包含的合法 noop 回归测试专门针对这种情况做了防护。
未验证或超出范围，没有针对真实的本地或小模型实际产生幻觉的场景做端到端复测，本修复在单元测试层面针对 issue 中描述的确切机制进行了修复，也就是 completed 状态下零工具调用。相关的 6308 号问题，关于可配置 extractor 超时，与本修复相关但未在此一并处理。
破坏性改动或迁移说明，AutoMemoryExtractionExecutionResult 新增了一个必填字段 hasToolActivity，任何其他构造该类型的调用方，无论是测试 mock 还是其他地方，都需要补上这个字段，本 PR 已同步更新了仓库内所有相关 mock。