Skip to content

fix(core): record auto-memory index reads in FileReadCache - #7468

Merged
wenshao merged 2 commits into
QwenLM:mainfrom
han-dreamer:fix/7287-record-auto-memory-index-read
Jul 23, 2026
Merged

fix(core): record auto-memory index reads in FileReadCache#7468
wenshao merged 2 commits into
QwenLM:mainfrom
han-dreamer:fix/7287-record-auto-memory-index-read

Conversation

@han-dreamer

Copy link
Copy Markdown
Contributor

What this PR does

Registers project-level and user-level auto-memory MEMORY.md reads in the session FileReadCache when those indexes are loaded into the system prompt.

Why it's needed

The model already sees these index files through the auto-memory prompt, but the prior-read guard did not know that. As a result, the first write_file update to MEMORY.md in a fresh session was rejected until the model spent an extra read_file round-trip.

Reviewer Test Plan

How to verify

Run npm test --workspace=@qwen-code/qwen-code-core -- src/config/config.test.ts. The regression test creates project-level and user-level MEMORY.md files, refreshes hierarchical memory, and verifies that checkPriorRead() allows overwriting both index files without an explicit read_file call.

The test also uses a temporary memory base directory and restores the environment after completion, so it does not touch a real user memory directory.

Evidence (Before & After)

N/A — non-user-visible core behavior and regression test change.

Tested on

OS Status
🍏 macOS ⚠️ not tested
🪟 Windows ✅ tested
🐧 Linux ⚠️ not tested

Environment (optional)

Local npm workspace on Windows. config.test.ts passed with 419 tests. Changed files also passed ESLint and Prettier checks.

Risk & Scope

  • Main risk or tradeoff: if the file disappears or stat fails after the prompt-loading read, the cache entry is not seeded and the existing prior-read protection remains in effect; the refresh itself continues without failing.
  • Not validated / out of scope: QWEN.md, AGENTS.md, team-memory indexes, and broader system-prompt file tracking.
  • Breaking changes / migration notes: none.

Linked Issues

Fixes #7287

中文说明

What this PR does

当项目级和用户级自动记忆 MEMORY.md 被加载到 system prompt 时,将这两次读取登记到当前 session 的 FileReadCache 中。

Why it's needed

模型实际上已经通过 auto-memory prompt 看到了这些索引文件,但 prior-read guard 并不知道这次读取。因此,在全新 session 中第一次使用 write_file 更新 MEMORY.md 时,必须额外调用一次 read_file,否则写入会被拒绝。

Reviewer Test Plan

How to verify

运行 npm test --workspace=@qwen-code/qwen-code-core -- src/config/config.test.ts。回归测试会创建项目级和用户级 MEMORY.md,刷新 hierarchical memory,并验证 checkPriorRead() 可以在没有显式调用 read_file 的情况下允许覆盖这两个索引文件。

测试使用临时 memory base directory,并在结束后恢复环境变量,不会修改真实用户 memory 目录。

Evidence (Before & After)

不适用:这是不可见的 core 行为和回归测试改动。

Tested on

OS 状态
🍏 macOS ⚠️ 未测试
🪟 Windows ✅ 已测试
🐧 Linux ⚠️ 未测试

Environment (optional)

Windows 本地 npm workspace。config.test.ts 的 419 个测试全部通过,改动文件也通过了 ESLint 和 Prettier 检查。

Risk & Scope

  • 主要风险或取舍:如果 prompt 加载后文件消失或 stat 失败,cache 不会登记该文件,但原有 prior-read 保护仍然生效;refresh 本身不会失败。
  • 未验证或不在范围内:QWEN.mdAGENTS.md、team-memory 索引以及更广泛的 system-prompt 文件追踪。
  • 破坏性改动或迁移说明:无。

Linked Issues

Fixes #7287

@qwen-code-ci-bot

qwen-code-ci-bot commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Qwen precheck requires maintainer approval before automated triage/review.

Head SHA: 6a6d18d290402df9a1fee6db989401f9e4b9dc5f

Reason:

  • prompt_injection:system_prompt

A maintainer with write access can inspect the PR and manually request a run with @qwen-code /triage or @qwen-code /review. A new push requires a fresh precheck.

ZijianZhang989
ZijianZhang989 previously approved these changes Jul 22, 2026

@ZijianZhang989 ZijianZhang989 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found. LGTM! ✅

— qwen3.7-max via Qwen Code /review

@gwinthis gwinthis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Architecture Review — PR #7468

Verdict: ✅ Approve — small, focused, correct fix for a real UX friction bug.

What it does

Registers auto-memory MEMORY.md reads (project-level + user-level) in the session FileReadCache during refreshHierarchicalMemory(), so the model's first write_file to update the index isn't rejected by the prior-read guard.

Code quality

Aspect Assessment
Scope Minimal — 1 new private method, 1 call site, 1 regression test
Error handling stat failure → warn log, no crash. Prior-read guard stays in effect as fallback. Correct.
Parallelism Promise.all for both indexes — non-blocking, independent
Test isolation Temp dirs + env restore in finally + clearAutoMemoryRootCache() — clean
Guard ordering indexContent === null check before stat — avoids unnecessary I/O

Design insight: Implicit Read Registration

This PR surfaces a broader architectural pattern worth documenting:

The system reads files on the model's behalf (system prompt assembly, memory loading, QWEN.md injection) but the prior-read guard only tracks model-initiated reads (via read_file tool). This creates a "phantom read" gap: the model has seen the content in its context, but the guard doesn't know it.

The fix pattern — register implicit reads in the same cache that guards explicit writes — is the right approach. It maintains the invariant: "if the model can see a file's content in its context, the guard should know about it."

Minor observations (non-blocking)

  1. Scope boundary is correct: PR explicitly excludes QWEN.md, AGENTS.md, and team-memory indexes. These are loaded via loadServerHierarchicalMemory which has a different code path. Extending to those would be a separate PR.

  2. full: true is correct here: The auto-memory index is read in its entirety (readAutoMemoryIndex returns the full file content), so full: true accurately reflects what the model sees.

  3. Race window: Between readAutoMemoryIndex() (content read) and fsPromises.stat() (cache seeding), the file could change. The PR handles this correctly — if stat fails, the cache entry is simply not seeded, and the model gets the standard "read first" error on write. No data corruption possible.

@yiliang114

Copy link
Copy Markdown
Collaborator

E2E verification and review note

I ran a deterministic fake-model CLI E2E against the PR base (80863c0858) and this head (6dc708f12f65ba6ee270669536c8d55294e06c15).

Coverage:

  • Fresh isolated HOME, QWEN_HOME, and QWEN_CODE_MEMORY_BASE_DIR.
  • Real npm run dev CLI with --auth-type=openai --model fake-model --prompt ... --yolo --no-chat-recording.
  • Real first-turn write_file tool call to the user-level auto-memory MEMORY.md.
  • Fake OpenAI server verified the first model request contained the user memory index, then inspected the second request's tool result.

Result:

  • Base: the first write_file returned the prior-read rejection (has not been read in this session).
  • PR head: the same first write_file no longer returned that rejection.
  • A direct core-flow check also verified both project-level and user-level auto-memory index reads are seeded into FileReadCache.

Screenshots:

Before: CLI first write rejected

After: CLI first write allowed

Core comparison: project and user indexes

Review note: I also did a full diff review. The fix is small and matches the issue, but there is one residual prior-read race to consider: readAutoMemoryIndex() reads the content first, while recordAutoMemoryIndexRead() records a later fs.stat() result. If the file changes between those two operations, the cache can record the newer fingerprint even though the model only saw the older content. The regular read_file path avoids this exact post-read stat pattern and documents why. I did not change the PR; this is the only remaining review concern I found.

@han-dreamer

Copy link
Copy Markdown
Contributor Author

Thanks for the careful E2E check and the race note. I updated the PR so auto-memory index reads now return the stats captured before the content read, and FileReadCache records that same fingerprint instead of performing a later post-read stat(). I also added regression coverage showing a file changed after the captured read stats is treated as stale rather than authorized.

@wenshao

wenshao commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Maintainer local verification — merge reference

Verified locally on macOS (Darwin 24.6.0) against the current head 6a6d18d — the commit that adds the race hardening, which the earlier E2E (run on the pre-fix 6dc708f) predated. This fills the macOS gap in the PR's OS matrix and confirms the follow-up fix on the head that would actually merge.

1. Real-flow A/B (unmocked)

A standalone harness drives the actual Config.refreshHierarchicalMemory() → real storeFileReadCachecheckPriorRead() chain (no vitest mocks), run identically against base b054d831c and head 6a6d18d — only the source tree differs. It uses a temporary memory base dir and never touches a real user memory directory.

Check base (before) head (#7468)
first write_file to project MEMORY.md edit_requires_prior_read ✅ allowed
first write_file to user MEMORY.md edit_requires_prior_read ✅ allowed
write_file after on-disk drift file_changed_since_read (still caught)

Real-flow A/B: base rejects both indexes, head allows both and still catches drift

This reproduces #7287 on base (the model must burn an extra read_file round-trip before it can update its own memory index) and confirms both index tiers — project and user — are writable on the very first turn after the PR, while the stale-read safety net stays intact (Scenario B). The earlier E2E only exercised the user-level index; this covers both.

2. Regression + full suites

packages/core config.test.ts (420) and store.test.ts (13) → 433/433 pass, including the four new tests. Changed files also pass prettier, eslint, and tsc --noEmit.

433 tests pass; prettier / eslint / tsc clean

3. On the race concern from the earlier review

The follow-up commit "seed memory read cache from read stats" resolves it correctly. readMemoryIndexWithStats() now captures fs.stat() before reading the content, and the cache is seeded with that same fingerprint (no later post-read stat()). If the file changes between the captured stat and a subsequent write, the recorded fingerprint is older than disk, so checkPriorRead() returns FILE_CHANGED_SINCE_READ — it fails safe (conservative rejection) rather than authorizing a write against content the model never saw. There is no ordering under which it can falsely authorize. Scenario B above exercises exactly this on real code.

Verdict

LGTM — merge-ready. Behavior matches the linked issue, the earlier race note is addressed and independently verified on the current head, and macOS is now covered alongside the author's Windows testing.

中文版本

维护者本地验证 — 合并参考

macOS(Darwin 24.6.0)上针对当前 head 6a6d18d 完成本地验证。该 commit 增加了对竞态的加固,而之前的 E2E 评审是在修复前的 6dc708f 上进行的。本次验证补齐了 PR OS 矩阵中缺失的 macOS,并在真正会被合并的 head 上确认了后续修复。

1. 真实流程 A/B(未打桩)

一个独立脚本驱动真实Config.refreshHierarchicalMemory() → 真实 storeFileReadCachecheckPriorRead() 链路(不使用 vitest mock),在 base b054d831c 和 head 6a6d18d 上以相同方式运行——唯一变量是源码树。脚本使用临时 memory base 目录,不会触碰真实的用户 memory 目录。

检查项 base(修复前) head(#7468
首次 write_file项目级 MEMORY.md edit_requires_prior_read ✅ 允许
首次 write_file用户级 MEMORY.md edit_requires_prior_read ✅ 允许
磁盘内容漂移后再 write_file file_changed_since_read(仍被拦截)

base 上复现了 #7287(模型必须先额外调用一次 read_file 才能更新自己的 memory 索引);PR 之后,项目级和用户级两个索引在第一轮就都可写,同时“过期读取”的安全网仍然有效(场景 B)。之前的 E2E 只覆盖了用户级索引,本次两者都覆盖。

2. 回归测试 + 完整套件

packages/coreconfig.test.ts(420)和 store.test.ts(13)→ 433/433 全部通过,其中包含 4 个新增测试。改动文件同时通过 prettier、eslint 和 tsc --noEmit

3. 关于上一轮评审提出的竞态问题

后续 commit “seed memory read cache from read stats” 正确地解决了它。readMemoryIndexWithStats() 现在在读取内容之前先捕获 fs.stat(),并用同一份 fingerprint 登记到 cache(不再做读取后的 stat())。如果文件在捕获的 stat 与后续写入之间发生变化,登记的 fingerprint 会早于磁盘,于是 checkPriorRead() 返回 FILE_CHANGED_SINCE_READ——即保守失败(拒绝),而不会对模型从未看到过的内容授权写入。不存在会误授权的顺序。上面的场景 B 正是在真实代码上验证了这一点。

结论

LGTM,可以合并。 行为符合关联 issue,上一轮的竞态提醒已被解决并在当前 head 上独立验证,macOS 也已随作者的 Windows 测试一并覆盖。

@yiliang114 yiliang114 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Clean fix — auto-memory MEMORY.md indexes now register in FileReadCache on load, eliminating the spurious write rejection on first save. The stat+read TOCTOU is correctly handled (test verifies FILE_CHANGED_SINCE_READ when file changes between stat and read). Backward compatible — old readAutoMemoryIndex still exists. New readAutoMemoryIndexWithStats returns both content and fs.Stats. Tests cover cache seeding and race detection.

— qwen3.8-max-preview via Qwen Code /review

@wenshao
wenshao added this pull request to the merge queue Jul 23, 2026
Merged via the queue into QwenLM:main with commit fcc250b Jul 23, 2026
46 checks passed
@yiliang114

Copy link
Copy Markdown
Collaborator

Post-merge correctness follow-up: I found one lifecycle gap that is separate from the stat-before-read race fixed here.

refreshHierarchicalMemory() can update Config.userMemory and seed FileReadCache without updating the active GeminiChat system instruction. A reachable sequence is:

  1. The live chat starts with MEMORY.md version A.
  2. The file is updated externally to version B.
  3. Enabling, disabling, installing, or uninstalling an extension calls refreshExtensionRuntime()refreshHierarchicalMemory().
  4. The cache now records B, but this path does not call refreshSystemInstruction(), so the live chat still sees A.

A later write based on A can therefore pass checkPriorRead() against B and overwrite entries the model never saw. I reproduced this on 6a6d18d: the live prompt remained on A, Config.userMemory and the cache moved to B, and checkPriorRead() returned ok: true.

The inverse also occurs after /clear, resume, compaction, cache eviction, or creating a per-agent Config: the prompt can still contain MEMORY.md while its cache entry is gone, so the original false rejection returns.

I think the implicit-read snapshot needs to be tied to the lifecycle of the prompt/cache instance that actually receives it: grant it only when the corresponding system instruction is installed, and preserve or reseed it while that instruction remains active.

@han-dreamer
han-dreamer deleted the fix/7287-record-auto-memory-index-read branch July 23, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(core): auto-memory MEMORY.md loaded into system prompt but not registered in FileReadCache — write_file always rejected on first update

6 participants