Memory v3 — storage, read loop, and write path (P2–P4), all flag-gated#31990
Conversation
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
…1978) Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
…#31980) Co-authored-by: Vellum Assistant <assistant@vellum.ai>
…hness) (#31981) Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
…31984) Co-authored-by: Vellum Assistant <assistant@vellum.ai>
…ains standing-context files (#31985) Co-authored-by: Vellum Assistant <assistant@vellum.ai>
…31986) Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Co-authored-by: Vellum Assistant <assistant@vellum.ai>
…2, log v3) (#31989) Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Plan papertrail: Memory v3 — Storage, Read Loop, and Write Path (P2–P4)Memory v3 — Storage, Read Loop, and Write Path (P2–P4)OverviewBuilds memory v3 — the retrieval-loop redesign — in-codebase as a Out of scope (do NOT build here): P5 (the pilot-assistant cutover + deleting v2's retrieval/write path — the only hard gate) and the later/optional plugin extraction. Also out of scope: the v2→v3 page migration (reclassifying the ~1,600 pages into the tree, authoring Shared standing-context substrate (preserved, NOT forked). The files Grounding anchors (verified against current
Standing verify for every PR (from CLAUDE.md; run in PR 1: v3 tree-node format + node storeDepends onNone Branchmemory-v3/pr-1-node-format Titlefeat(memory-v3): tree-node on-disk format + node store Files
Implementation steps
Acceptance criteria
PR 2: v3 tree index (DAG build + cache)Depends onPR 1 Branchmemory-v3/pr-2-tree-index Titlefeat(memory-v3): tree index with DAG adjacency + cache Files
Implementation steps
Acceptance criteria
PR 3: compositional index renderingDepends onPR 2 Branchmemory-v3/pr-3-index-composition Titlefeat(memory-v3): compose node index from children + routing hints Files
Implementation steps
Acceptance criteria
PR 4: traversal primitives + cycle/visited guardsDepends onPR 2 Branchmemory-v3/pr-4-traversal Titlefeat(memory-v3): parallel-fan-out traversal with cycle/visited guards Files
Implementation steps
Acceptance criteria
PR 5: tree validationDepends onPR 3, PR 4 Branchmemory-v3/pr-5-validate Titlefeat(memory-v3): tree validator (orphans, cycles, dangling refs, freshness) Files
Implementation steps
Acceptance criteria
PR 6: CLI
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 93da857fb2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | "memory_v3_consolidate" | ||
| | "memory_v3_index_maintenance" | ||
| | "memory_v3_edge_learning" |
There was a problem hiding this comment.
Classify memory_v3_consolidate as a slow LLM job
memory_v3_consolidate is added as a new job type but not added to SLOW_LLM_JOB_TYPES, so it falls into the fast lane. This job can run up to 15 minutes and performs background LLM work, so putting it in the fast lane breaks lane isolation and can block short fast jobs whenever memory.v3.write.enabled is on. It should be classified with other slow LLM jobs (like memory_v2_consolidate) to preserve intended scheduling behavior.
Useful? React with 👍 / 👎.
| consolidateIntervalMs: z | ||
| .number({ | ||
| error: "memory.v3.write.consolidateIntervalMs must be a number", | ||
| }) | ||
| .int("memory.v3.write.consolidateIntervalMs must be an integer") | ||
| .default(3600000) |
There was a problem hiding this comment.
Enforce positive v3 consolidation interval
memory.v3.write.consolidateIntervalMs is only constrained to an integer, so 0 or negative values are accepted. In maybeEnqueueGraphMaintenanceJobs, the scheduler checks nowMs - lastRun >= intervalMs; with a non-positive interval that condition is always true, causing a consolidation job to be enqueued on every worker pass and rapidly flooding the queue. Add a positive constraint to this schema field (matching the v2 interval validation).
Useful? React with 👍 / 👎.
The live-shadow middleware runs on every turn and read `config.memory.v3.enabled` unguarded. Configs built outside the Zod schema (agent-loop test fixtures) have no `memory.v3` block, so the gate threw `TypeError: undefined is not an object` and aborted the turn — cascading across ~13 agent-loop test files. Guard with optional chaining (matches the loop's existing `write?.coactivation` pattern) and add a regression test for the absent-v3 config. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PR #31983 registered the two read-only v3 routes but never added their ACTOR_ENDPOINTS entries in route-policy.ts; the per-PR run skipped CI so the route-policy coverage guard never ran. Add both as settings.read (mirroring the v2 read routes), satisfying guard-tests.test.ts. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Classify memory_v3_consolidate as a slow LLM job (it hands off to a background agent for up to 15 min like memory_v2_consolidate); leaving it in the fast lane broke lane isolation when memory.v3.write.enabled is on. - Constrain memory.v3.write.consolidateIntervalMs to positive: 0/negative made the scheduler's `now - lastRun >= interval` always true, flooding the queue. - Tests for both. Co-authored-by: Vellum Assistant <assistant@vellum.ai>
Summary
Builds memory v3 — the retrieval-loop redesign — as a new
assistant/src/memory/v3/namespace besidememory/v2/. Everything is additive and flag-gated; withconfig.memory.v3.*at defaults (all false) nothing changes in production. v2 is byte-for-byte untouched (empty diff onmemory/v2/).assistant memory v3 validate|treeCLI/routes.runRetrievalLoop; exposed as a P1-harnessRetriever(compare route, gated onv3.enabled) and a live-shadowmemoryRetrievalmiddleware (gated onv3.enabled && v3.shadow, injects v2, logs v3 asmode='v3_shadow').memory/buffer.mdinto the tree while preserving essentials/threads/recent exactly as v2 (scheduler retargets only whenv3.write.enabled).Out of scope (per plan): P5 cutover + v2 retirement, plugin extraction, and the by-hand v2→v3 page migration.
Production-safety (verified by self-review)
With
v3.enabled/shadow/write.*all false: shadow middleware is a pure pass-through, the compare route excludes v3, the consolidation scheduler still enqueuesmemory_v2_consolidate, and nothing auto-enqueues v3 jobs. Migrations 262/263 only create empty tables.Self-review result
generate:openapino-diff, v3 tests pass per-file).memory_v3_edge_learningis dispatched but never enqueued; co-activationusedis never reconciled to true; the loop callsexpandEdgeswithoutaboveThreshold'sextraAdjacency; consolidation doesn't yet consume promotion candidates.filter.ts/gate.ts/tree-walk.ts; v3 consolidation prompt-override resolver is a verbatim fork of v2's. Behavior-preserving consolidation deferred to avoid regression risk on tested inert code.memoryRetrievalpipeline); post-compaction cached-block reinjection bypasses the pipeline but performs no new retrieval, so nothing is missed.PRs merged into this feature branch
#31971, #31972, #31973, #31974, #31975, #31976, #31977, #31978, #31979, #31980, #31981, #31982, #31983, #31984, #31985, #31986, #31987, #31988, #31989 (19 PRs).
Part of plan: memory-v3-build.md