diff --git a/openspec/changes/memory-core-redesign/.openspec.yaml b/openspec/changes/memory-core-redesign/.openspec.yaml new file mode 100644 index 000000000..43e65ca6e --- /dev/null +++ b/openspec/changes/memory-core-redesign/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-03 diff --git a/openspec/changes/memory-core-redesign/design.md b/openspec/changes/memory-core-redesign/design.md new file mode 100644 index 000000000..86b14695f --- /dev/null +++ b/openspec/changes/memory-core-redesign/design.md @@ -0,0 +1,298 @@ +# Design: memory-core-redesign + +## Context + +The July 2026 audit (`docs/research/memory-audit-2026-07.md`) measured the +memory system end-to-end against the live 1,216-document corpus: 46% of +auto-injected memories are pollution (relevance judged, κ=0.754); the lexical +composite score carries no relevance signal (precision flat across every +floor); the LLM curation tier had **zero** successful decisions in its +production lifetime; consolidation fired twice ever while redundancy reached +14% and doubled in five weeks; the checkpoint worker drops 95% of its intake; +`Searchable` recall mode secretly participates in automatic recall; the Trace +class and expiry mechanics are fully vestigial (0 traces ever, no deletion, +204 expired records mummified on disk); `memory_edges` has 0 rows and the +facet planner knows 4 demo facets against ~900 real ones. + +The quick-win slice (PR #1568) revived the LLM tier, adopted the balanced +curation prompt, re-tuned lexical scoring, and added an injection budget. +This change is the structural remainder: add the missing **judgment** +(embeddings), add the missing **metabolism** (lifecycle: lossless merge, +expiry, consolidation), and **subtract** the dead structure that three +redesign cycles accreted. Prior art constraints come from the May 2026 +autoresearch (`docs/research/memory-recall-findings-2026-05.md`): nominate-by-cosine/decide-by-LLM is +ratified (no cosine threshold separates duplicates from siblings — siblings +live at 0.905–0.941 inside the duplicate band), and the nominator model is +snowflake-arctic-embed 137M (33M-class models measured inadequate for +doc-to-doc dedup). + +Actor/persistence context: memory lives in the daemon's single SQLite file +(`NetclawPaths.MemorySqliteDbPath == SqliteDbPath`), whose memory tables are +owned by `SQLiteMemoryStore.InitializeAsync` (idempotent DDL), NOT by the +daemon's `SchemaMigrator`. Two write pipelines exist today: the inline +per-session path (`SessionMemoryObserverActor` → `MemoryProposalGate` → +`MemoryCurationActor`, a per-session child of `LlmSessionActor`) and the +daemon checkpoint worker (`MemoryCurationWorkerService` → +`MemoryCurationEngine`). Recall runs on the session actor's turn path under a +hard latency budget (`Memory.RecallTimeoutMs`, default 300 ms). + +## Goals / Non-Goals + +**Goals** + +1. Fewer, more comprehensive memories: near-duplicates are detected + semantically at write time and merged losslessly. +2. Fewer, more accurate injections: recall is gated by an absolute semantic + relevance floor; most turns inject nothing (measured correct outcome for + 65% of real queries). +3. A real lifecycle: expired rows are deleted, redundant clusters are + consolidated under operator control, short-lived (≈72 h) memories exist + and work. +4. Tool-use lessons are captured and surface exactly when the relevant tool + is used. +5. Less machinery: taxonomy and pipelines shrink to the behaviors that + actually exist; every metadata field written is consumed by a reader. + +**Non-Goals** + +- Multilingual embeddings (model swap later; vectors keyed by `model_id`). +- ANN indexes (brute force is sub-ms at this corpus scale; revisit ≥50k). +- Automatic (code-level) detection of tool-use corrections. +- Applying consolidation to any corpus as part of implementation (tooling + ships; each apply run is an operator decision). +- Multi-node/cluster memory; this remains single-process MVP. + +## Decisions + +### D1. Embedding runtime: in-process ONNX, new `src/Netclaw.Embeddings` project + +`Microsoft.ML.OnnxRuntime` (CPU EP; linux-x64 + linux-arm64 ship in 1.25+) + +`FastBertTokenizer` (pure managed WordPiece — the chosen model is BERT-class) ++ `System.Numerics.Tensors` for SIMD cosine. The consumer-defined seam +`IMemoryEmbedder` lives in `Netclaw.Actors/Memory` so actor code never +references OnnxRuntime; `Netclaw.Embeddings` is referenced by Daemon and CLI +only. A singleton `OnnxMemoryEmbedder` holds one `InferenceSession` +(`IntraOpNumThreads` bounded, concurrency semaphore ≤2); an +`UnavailableMemoryEmbedder` stub carries `IsAvailable=false` for degraded +mode. + +*Alternative considered*: Ollama sidecar — rejected: violates the +single-process constitution, adds a network hop inside the recall budget, and +creates a second silent-failure surface. *Alternative*: embedding via the +existing chat-provider plugins — rejected: recall must work when no provider +is reachable, and provider embedding APIs are not uniformly available. + +### D2. Model provisioning: pinned allowlist, download at initialization, never embedded + +`Memory.Embeddings.ModelId` selects from an **in-code allowlist manifest** +(model id → URL + byte size + SHA-256); arbitrary URLs are rejected +(supply-chain boundary). An `EmbeddingWarmupHostedService` provisions at +daemon start when `AutoDownload=true` (atomic temp+rename download, hash +verify, then one warm-up inference), or the operator runs +`netclaw memory backfill-embeddings`. The ~90–140 MB artifact is never an +embedded resource (would bloat every RID publish). Default model: +snowflake-arctic-embed 137M int8 (May-ratified; mxbai-embed-large 335M is the +allowlisted fallback). Post-PoC decision deferred: mirroring artifacts into +the existing R2 feeds channel vs pinned upstream URLs. + +### D3. Vector storage: separate `memory_embeddings` table, owned by the store + +`memory_embeddings(item_id, item_kind, model_id, content_hash, dims, vector +BLOB, created_at, PRIMARY KEY(item_id, model_id))`, created in +`SQLiteMemoryStore.InitializeAsync` alongside the other memory DDL — not a +daemon migration, preserving the store's standalone-initialization contract +(doctor and tests construct it without the migrator). Content hash = +SHA-256 of normalized title+body; re-embed is skip-if-hash-match, so backfill +re-runs are free. Model change = new `model_id` rows + `--force` backfill; no +rewrite of the 224 MB documents table. Backfill state is **derived** +(LEFT JOIN on current model + hash), never a progress table. kNN executes as +a brute-force scan over an in-memory `MemoryVectorIndex` (flat float[] per +model, ~1.8 MB at current scale, invalidated by a store version counter). No +sqlite-vec/native extensions (ARM64 + deployment liability for zero benefit +at this scale). + +*Failure/recovery*: a crash between document commit and embedding upsert +leaves a missing-embedding row; the warmup service's gap-repair sweep and the +embedding doctor check both surface and heal it. Vectors are derived data — +loss is always recoverable by re-embedding. + +### D4. Write-side: one evaluator; kNN nominates, LLM decides; no cosine auto-merge + +The duplicated evaluation logic in `MemoryCurationActor.EvaluateSingleAsync` +and `MemoryCurationEngine` collapses into one shared `MemoryCurationEvaluator` +used by both the inline actor and the daemon worker (today's guards diverge — +`GuardDestructiveUpdate` exists on one path only). Evaluation order: + +1. Exact-anchor + near-identical body → deterministic SKIP (cheap fast path). +2. Embedding kNN nomination at `NominatorSimilarityThreshold` (default 0.86) + / `NominatorK` (default 5). **Any nominee forces the LLM tier** — the May + measurement stands: no cosine threshold separates duplicates from siblings, + so cosine never auto-merges and never auto-skips. +3. No nominee and no anchor match → CREATE without an LLM call (the common + case stays cheap; median nominee count on a random write is 0). +4. Embedder unavailable → the current lexical candidate search runs as the + explicitly-logged degraded path. + +*Alternative considered*: cosine auto-merge tier above 0.95 — rejected: the +measured sample shows ~3 pairs there, not worth a data-loss risk surface. + +### D5. Lossless merge: LLM-synthesized body + deterministic MergeGuard + append fallback + +CONSOLIDATE/UPDATE decisions now carry a merged body +(`CurationDecision.MergedBody`) synthesized by the curation LLM from +full-content previews. A deterministic `MergeGuard` validates it: load-bearing +tokens (URLs, numbers, versions, dates, code identifiers) from every source +body must survive (≥95%), and length must not collapse. On failure the write +degrades to a **structural append** (existing body + dated separator + +proposal — finally producing the `AppendDocument` semantics that have existed +unused since the enums were written). The raw +`markdown_body = excluded.markdown_body` overwrite becomes unreachable from +curation decisions. Records remain immutable and curation-bypassing. + +*Why not prompt-only*: the May decider eval measured the balanced prompt at +~27% wrong-merge on hard near-duplicates. The guard turns a wrong merge from +silent data loss into recoverable over-consolidation. + +### D6. Read-side: hybrid recall with an absolute cosine floor + +Per turn: embed the query once (sub-budget inside `RecallTimeoutMs`; on +timeout or unavailable → lexical-only + rate-limited +`memory_recall_vector_degraded` log). Candidates = FTS5 top-k ∪ vector top-k, +deduplicated, **all candidates passing the identical policy gates** +(audience/boundary/sensitivity/recall-mode) regardless of source — a +correctness requirement with its own scenario. Scoring = weighted fusion +(`VectorWeight` 0.7 × cosine + `LexicalWeight` 0.3 × squashed selector score ++ dampened class prior), then an **absolute floor**: `MinCosineSimilarity` +(default 0.55, calibrated against the real-traffic gold set +`gold-prod-2026-07`). Nothing above the floor → inject nothing, and the +volatile `[memory-recall]` block is omitted entirely (zero tokens). Recency +decay (`RecencyHalfLifeDays`, floor-bounded multiplier) breaks ties toward +fresh knowledge. The quick-win char budget and `AutoRecallMaxItems` remain +the outer bounds. + +*Alternative considered*: RRF fusion — rejected: rank-only fusion always +admits the top item even when nothing is relevant; the zero-injection +behavior requires an absolute score. *Latency risk is explicit*: Ollama +measurements ran far above the 10–50 ms/query assumption; the ONNX int8 +short-query latency MUST be measured before this slice ships (mitigations: +raise `RecallTimeoutMs`, pre-warmed session, or skip-vector-under-pressure — +all loud, none silent). + +### D7. Taxonomy rebalance: recall modes mean what they say + +- **BREAKING (semantic fix)**: `Searchable` leaves the automatic recall pool + (`SearchByPlanAsync` admits `auto` only). `Searchable` = find_memories + surface; `Manual` = explicit-id access; `Never` = policy-hidden. The 22 + legacy compaction rows were already repaired in the quick-win slice; a + startup data-repair re-asserts invariants idempotently. +- Formation: the observer sidecar proposes a recall mode; the policy gate + honors it for durable facts with **default `searchable`** — `auto` is + reserved for standing facts that should color every conversation (identity, + durable preferences, environment). This breaks the measured 97%-auto + monoculture at the source. The distillation prompt is rewritten for fewer, + more comprehensive proposals (consolidate related observations into one + document; propose fewer atomic fragments). +- **Trace revival**: the sidecar may propose `trace` (short-lived operational + state, TTL 72 h) — the class becomes reachable, recallable while fresh + (recall mode `auto` with its TTL as the guard, weighted below durable + facts), and actually deleted by the expiry sweep (D8). +- **Tool lessons**: new `MemoryClass.ToolLesson` → Document/MergeDocument/ + Searchable, anchored `anchor_type="tool"`, `canonical_name=`. + Captured explicitly (`store_memory` accepts the class; the `netclaw-memory` + skill instructs saving a lesson when the user corrects tool usage) and by + the sidecar distillation prompt (correction-hunting instruction). Recall is + **per-tool context injection**: on a tool's first use in a session, the + tool-execution pipeline appends a compact `[tool-lessons:]` block + (top 2 by `updated_at`, bounded chars) to the tool result — an exact + anchor-id lookup, no embedding, outside the pre-turn recall budget, reset + on compaction. The dead `verified-tool-finding` +25 recall bonus is + removed; `store_memory` with the class becomes the first real producer of + the `VerifiedToolFinding` checkpoint flag. + +*Alternative considered*: overloading Evidence for lessons — rejected: +Evidence is policy-forced to immutable Record + searchable, so lessons could +never be refined by curation and would never surface unprompted. + +### D8. Metabolism: expiry sweep + operator-gated consolidation + +- **Expiry sweep**: a daemon maintenance step (piggybacking the checkpoint + worker's idle loop) DELETEs rows whose `expires_at` has passed beyond a + grace window — they are already invisible to every read path, so deletion + is behavior-neutral by construction; each sweep logs counts. (Audit: 204 of + 384 evidence records currently mummified.) +- **Consolidation**: `netclaw memory consolidate --dry-run` builds the kNN + cluster graph, runs the merge-synthesis prompt per cluster, and writes a + human-editable `plan.jsonl` + report — no mutation. `--apply --plan ` + executes a reviewed plan verbatim: refuses a live daemon by default, takes + a `VACUUM INTO` backup first, applies in batched transactions, re-embeds + merged bodies, rebuilds FTS rows, and records a `memory_maintenance_runs` + ledger row. `netclaw memory status` reports class/recall-mode/embedding + coverage. CLI-owned rather than a daemon job because the ratification gate + is inherently interactive. + +### D9. Subtraction + +Removed with evidence they carry no load (audit): `memory_edges` table and +its DDL/spec requirement (0 rows ever; anchors remain as flat grouping keys); +the facet/soft-scope *inference* in `DeterministicRetrievalPlanning` (4 +hardcoded demo facets; stopword-hygiene and lexical-term extraction remain); +the checkpoint worker's unconditional turn-complete enqueue (gated at enqueue +by the same project-fact precondition the extractor applies — eliminating +~95% wasted enqueue/lease/deserialize cycles; the freed lane is where the +expiry sweep lives). Wire enums keep their values for serialization +compatibility; only dead *behavior* is deleted. + +## Risks / Trade-offs + +- [Model download unavailable offline at first run] → loud degraded mode: + doctor Error, daemon status `embeddings: degraded`, rate-limited logs; + lexical recall keeps serving. Never silent. +- [Query-embedding latency blows the 300 ms recall budget on CPU] → measured + gate before Slice 4 ships; warmup inference at start; per-turn vector + sub-budget with logged lexical fallback; `RecallTimeoutMs` already + operator-tunable. +- [LLM merge synthesis loses information] → MergeGuard token-retention check + + structural-append fallback; consolidation applies only via human-ratified + plan files with a backup taken first. +- [Cosine floor calibrated on one corpus generalizes poorly] → floor lives in + config next to `ModelId`; gold-set eval (real traffic) pins the calibration; + doctor warns on mixed-model embedding rows. +- [Searchable-out-of-auto surprises users who relied on incidental recall] → + BREAKING is called out; `find_memories` covers the tail; formation default + changes only affect NEW memories; consolidation plans may propose + recall-mode changes but only under ratification. +- [ARM64 native OnnxRuntime regression] → CI publish smoke leg on linux-arm64; + FastBertTokenizer is pure managed. +- [Two write paths drift again during the transition] → shared + `MemoryCurationEvaluator` lands as its own slice before any nominator work; + divergence becomes structurally impossible rather than reviewed-for. + +## Migration Plan + +1. Slices are independently shippable, in order: (1) shared evaluator + extraction (behavior-neutral refactor), (2) embedding foundation (writes + vectors, nothing reads them — zero behavior risk), (3) write-side + nominate→decide + lossless merge, (4) read-side hybrid + cosine floor, + (5) taxonomy rebalance + trace revival + tool lessons, (6) maintenance + CLI + expiry sweep + subtraction. +2. Existing corpora: `backfill-embeddings` (measured: minutes) is required + before slices 3–4 activate their vector paths; both paths degrade loudly + to lexical when coverage is incomplete rather than misbehaving. +3. Rollback: each slice is config-gated (`Memory.Embeddings.Enabled`, + nominator/recall thresholds) — disabling returns to the quick-win + behavior. Vectors are derived data; dropping `memory_embeddings` is safe. +4. Schema: new tables via idempotent `InitializeAsync` DDL; config surface + added to `netclaw-config.v1.schema.json` with defaults (migration-friendly + per the constitution's schema rules); `netclaw-memory` system skill updated + in the same PR as each behavior slice. + +## Open Questions + +- ONNX int8 query-embedding latency on reference hardware (measure in Slice 2; + gates Slice 4's sub-budget design). +- Final `MinCosineSimilarity` default (calibrate against `gold-prod-2026-07` + during Slice 4; 0.55 is the working hypothesis). +- Whether the R2 feeds channel should mirror model artifacts (post-PoC + operational decision; allowlist design is unaffected). +- Trace auto-recall weighting while fresh (small prior vs durable-fact parity) + — decide with eval cases in Slice 5. diff --git a/openspec/changes/memory-core-redesign/proposal.md b/openspec/changes/memory-core-redesign/proposal.md new file mode 100644 index 000000000..73e0fb898 --- /dev/null +++ b/openspec/changes/memory-core-redesign/proposal.md @@ -0,0 +1,149 @@ +# Proposal: memory-core-redesign + +Source PRD: PRD-007 (agent personality and local memory). Evidence base: +`docs/research/memory-audit-2026-07.md` (July 2026 measured audit) and +`docs/research/memory-recall-findings-2026-05.md` (May 2026 autoresearch). + +## Why + +The memory system's two intelligence layers have never functioned — the LLM +curation tier has zero successful decisions in its production lifetime and +recall ranks by a lexical score measured to carry no relevance signal — so the +corpus accretes near-duplicates (14% redundant, doubling in five weeks) while +automatic recall injects 46% pollution into live turns (19% of recall events +actively misleading). The quick-win slice (July 2026) stopped the worst +bleeding; this change adds the missing semantic judgment and lifecycle, and +removes the dead structure that three redesign cycles left behind. + +## What Changes + +- **Semantic judgment (embeddings).** In-process ONNX embedding infrastructure + (snowflake-arctic-embed 137M int8, CPU, zero sidecars): embed-on-write, a + brute-force in-memory vector index, and model provisioning with a pinned + hash-verified allowlist, downloaded at daemon initialization. +- **Write-side dedup becomes nominate→decide.** Embedding kNN nominates + near-duplicates (τ≈0.86, k=5, config); any nominee forces the LLM curation + tier to decide merge/enrich/keep. No cosine auto-merge tier (measured: no + threshold separates duplicates from siblings). One shared curation evaluator + replaces the two divergent pipelines. +- **Lossless merges.** CONSOLIDATE/UPDATE produce an LLM-synthesized merged + body validated by a deterministic MergeGuard (load-bearing-token retention) + with a structural-append fallback — the raw `markdown_body` overwrite path + becomes unreachable. **BREAKING** for any consumer that assumed merge == + replace. +- **Read-side hybrid recall with an absolute relevance floor.** Query + embedding + FTS5 union, weighted fusion, and a cosine floor calibrated + against the real-traffic gold set — turns where nothing is relevant inject + nothing (measured: 65% of real queries). +- **Taxonomy rebalance around real behaviors.** `Searchable` recall mode is + removed from the automatic pool (**BREAKING** semantic fix: today + searchable ⊂ auto); durable-fact formation defaults to searchable with auto + reserved for identity/preferences/environment; `Trace` (72 h short-lived + memory) gets a reachable producer and a recallable-while-fresh mode; an + expiry sweep actually deletes expired rows. +- **Tool-use lessons.** New `tool_lesson` memory class (Document/merge/ + searchable) anchored per tool, captured explicitly (`store_memory`) and by + the sidecar distillation prompt; recalled via per-tool context injection on + first tool use per session — outside the pre-turn recall budget. +- **Maintenance tooling.** `netclaw memory` CLI group: `backfill-embeddings`, + `consolidate --dry-run` (ratification plan file) / `--apply --plan` (gated, + backup-first), `status`; maintenance-run ledger table. +- **Subtraction.** Remove: the unused `memory_edges` graph, the inert + 4-demo-facet planner inference, the dead `verified-tool-finding` +25 recall + bonus, and gate the checkpoint worker's turn-complete lane at enqueue time + (95% of its intake is dropped by design today). **BREAKING** only at the + schema-surface level; no functional behavior depends on any of these + (verified by audit). + +## Capabilities + +### New Capabilities + +- `memory-embeddings` — ONNX embedding runtime: model provisioning + (allowlist, SHA-256, atomic download), embed-on-write, vector index, and + loud-degradation semantics (doctor check, daemon status, structured logs; + lexical recall keeps serving but never silently). +- `memory-maintenance` — operator-driven corpus lifecycle: embedding + backfill, consolidation dry-run/apply with human ratification and + backup-first apply, expiry sweep, `netclaw memory status`, maintenance + ledger. + +### Modified Capabilities + +- `netclaw-agent-memory` — hybrid recall + absolute cosine floor and + injection semantics; nominate→decide curation with lossless merge; recall- + mode semantics fix (searchable leaves the automatic pool); trace revival + (producer, fresh-recall, deletion); tool-lesson class + per-tool context + injection; formation-side recall-mode assignment; removal of graph-edge and + facet-inference requirements (the spec's flagged open decisions — "keyword + vs vector search, embedding strategy, injection budgets" — are resolved by + this change). + +## Impact + +- **Code**: `src/Netclaw.Embeddings` (new project), `Netclaw.Actors/Memory` + (curation evaluator, store schema + vector queries, policy gates, enums), + `Netclaw.Actors/Sessions` (recall coordinator, tool-execution pipeline for + lessons), `Netclaw.Daemon` (DI, warmup hosted service, checkpoint gating), + `Netclaw.Cli` (memory command group, doctor checks), + `Netclaw.Configuration` (Memory.Embeddings/Recall/Curation config objects + + schema sync), observer sidecar distillation prompt, `netclaw-memory` system + skill. +- **Dependencies**: `Microsoft.ML.OnnxRuntime` (CPU; linux-x64 + linux-arm64), + `FastBertTokenizer`, `System.Numerics.Tensors`. Model artifact (~90–140 MB) + distributed at runtime, never embedded in the binary. +- **Data**: new `memory_embeddings` and `memory_maintenance_runs` tables + (owned by `SQLiteMemoryStore.InitializeAsync`, not daemon migrations); + one-time ratified consolidation pass over the existing corpus (operator- + gated; out of automatic paths); expiry sweep begins deleting expired + records. +- **Evals**: recall-quality gold-set regression suite (real-traffic gold from + the audit); eval cases for tool lessons and zero-injection behavior; the + scenario suite's paraphrase-gap case (P09) flips back to expected-recall. + +### In scope (MVP) + +Slices 2–6 as designed: embedding foundation; write-side nominate→decide + +lossless merge; read-side hybrid + cosine floor; taxonomy rebalance + tool +lessons + trace revival + expiry sweep; maintenance CLI + subtraction items. + +### Out of scope + +- Multilingual embedding models (future pass; vectors are keyed by + `model_id`, thresholds live in config next to `ModelId`, so a model swap is + `config change + backfill --force`; re-evaluate .NET SentencePiece + tokenizer support then). +- ANN indexes (brute-force cosine is sub-ms at ≤50k vectors). +- Mirroring the model artifact into the R2 feeds infra (post-PoC decision; + pinned HF URLs first). +- Structural (code-level) detection of tool-use corrections (explicit + + sidecar capture only). +- Applying consolidation to any live corpus as part of this change's + implementation (tooling ships; each apply run remains an operator decision). + +## Security and Operational Impact + +- **Model supply chain**: `ModelId` selects from a pinned in-code allowlist + (id → URL + size + SHA-256); arbitrary URLs are rejected; downloads are + atomic (temp + rename) and hash-verified before load. A failed or missing + model is a **loud** degraded state (doctor Error, daemon status + `embeddings: degraded`, rate-limited structured logs) — lexical recall + keeps serving; no silent fallback. +- **Policy parity**: vector-sourced recall candidates pass the identical + audience/boundary/sensitivity/recall-mode gates as lexical candidates + (scenario-tested requirement, not an implementation detail). +- **Destructive-operation gating**: consolidation `--apply` executes only a + previously written, human-editable plan file, refuses a live daemon by + default, and takes a `VACUUM INTO` backup first. The expiry sweep deletes + only rows already invisible to every recall/search path. +- **Resource envelope**: measured on the reference box (i9-9900K) — full + 1,216-doc backfill 4.5–8.3 min, <0.5 GB RSS; steady-state embed-on-write + ~13 docs/day. The recall-time query-embedding sub-budget must be + re-measured on the ONNX int8 path before the hybrid slice ships (Ollama + measurements ran 4–30× above the design assumption for full documents; + queries are far shorter). +- **Operations**: new doctor checks (embedding provisioning/coverage, + curation-LLM health — the latter shipped with the quick-win slice); + `netclaw memory status` becomes the corpus-health surface; runbook + `docs/runbooks/memory-health-and-evals.md` gains embedding/consolidation + sections. diff --git a/openspec/changes/memory-core-redesign/specs/memory-embeddings/spec.md b/openspec/changes/memory-core-redesign/specs/memory-embeddings/spec.md new file mode 100644 index 000000000..05fb16b25 --- /dev/null +++ b/openspec/changes/memory-core-redesign/specs/memory-embeddings/spec.md @@ -0,0 +1,100 @@ +# Spec: memory-embeddings (new capability) + +## ADDED Requirements + +### Requirement: In-process embedding runtime + +The system SHALL compute memory embeddings in-process with a CPU ONNX runtime +and a managed tokenizer — no sidecar processes, no network inference hop. +Embedding components SHALL sit behind a narrow interface owned by the memory +subsystem so actor code carries no ONNX dependency, and the runtime SHALL +support both linux-x64 and linux-arm64. + +#### Scenario: Embeddings compute without external services + +- **GIVEN** a healthy daemon with the embedding model provisioned +- **WHEN** a memory document is written +- **THEN** its embedding is computed in-process +- **AND** no network call or child process is involved in inference + +### Requirement: Pinned model provisioning + +The embedding model SHALL be selected by id from a pinned in-code allowlist +mapping model id to download URL, byte size, and SHA-256. Arbitrary model URLs +SHALL be rejected. Provisioning SHALL download atomically (temporary file then +rename), verify the hash before load, and run at daemon initialization when +auto-download is enabled or on explicit operator command. The model artifact +SHALL NOT be embedded in the application binary. + +#### Scenario: Hash mismatch refuses the model + +- **GIVEN** a downloaded model artifact whose SHA-256 does not match the + allowlist entry +- **WHEN** provisioning verifies the artifact +- **THEN** the artifact is discarded and not loaded +- **AND** the failure is surfaced as a doctor-visible error + +#### Scenario: Unknown model id is rejected + +- **GIVEN** configuration naming a model id absent from the allowlist +- **WHEN** the daemon initializes embeddings +- **THEN** provisioning refuses with a configuration error identifying the + allowlisted ids + +### Requirement: Embed-on-write with derived backfill state + +Every recallable memory document SHALL receive an embedding keyed by +`(item id, model id)` with a content hash of its normalized text. Writes SHALL +embed after commit; a startup gap-repair sweep SHALL embed any item missing a +current-model embedding. Re-embedding SHALL be skipped when the content hash +is unchanged. Backfill progress SHALL be derived from the store (items lacking +a current-model embedding), never tracked in separate mutable state. Vectors +are derived data: loss or deletion of embeddings SHALL be recoverable by +re-embedding without any loss of memory content. + +#### Scenario: Crash between write and embed self-heals + +- **GIVEN** a document committed whose embedding upsert was interrupted +- **WHEN** the daemon next starts and the gap-repair sweep runs +- **THEN** the missing embedding is computed and stored +- **AND** the embedding doctor check reports full coverage afterward + +#### Scenario: Model change re-embeds without data loss + +- **GIVEN** a corpus embedded under model A +- **WHEN** the operator switches configuration to allowlisted model B and runs + a forced backfill +- **THEN** embeddings for model B are created alongside or replacing model A's +- **AND** memory content is unmodified + +### Requirement: Loud degradation without silent fallback + +Memory recall and curation SHALL continue on their lexical paths when the +embedding model is missing, corrupt, or the runtime fails, and the degraded +state SHALL be loud: a doctor check reports the cause, the daemon runtime +status reports embeddings as degraded, and recall/curation log structured +degradation events. The system SHALL NOT silently revert to lexical behavior +without these signals. + +#### Scenario: Missing model degrades loudly + +- **GIVEN** auto-download is disabled and no model artifact is present +- **WHEN** the daemon starts and a turn triggers recall +- **THEN** recall serves lexical-only results +- **AND** daemon status reports embeddings degraded +- **AND** `netclaw doctor` reports the missing model as an error with + remediation + +### Requirement: Embedding coverage diagnostics + +A doctor check SHALL report embedding provisioning state and corpus coverage: +model present and hash-valid, count of items lacking current-model embeddings, +and a warning when embeddings exist under multiple model ids (mixed-model +corpus invalidates similarity thresholds). + +#### Scenario: Mixed-model corpus warns + +- **GIVEN** embeddings stored under two different model ids +- **WHEN** the embedding doctor check runs +- **THEN** it warns that similarity thresholds are calibrated per model +- **AND** recommends a forced backfill under the active model diff --git a/openspec/changes/memory-core-redesign/specs/memory-maintenance/spec.md b/openspec/changes/memory-core-redesign/specs/memory-maintenance/spec.md new file mode 100644 index 000000000..dd878a220 --- /dev/null +++ b/openspec/changes/memory-core-redesign/specs/memory-maintenance/spec.md @@ -0,0 +1,83 @@ +# Spec: memory-maintenance (new capability) + +## ADDED Requirements + +### Requirement: Embedding backfill command + +The CLI SHALL provide `netclaw memory backfill-embeddings` to provision the +model if needed and embed every item lacking a current-model embedding, with a +`--force` mode that re-embeds everything under the active model. Backfill +SHALL be safe against a live daemon (small batched writes under WAL) and SHALL +report progress and a final coverage summary. + +#### Scenario: Backfill completes coverage + +- **GIVEN** a corpus with items lacking current-model embeddings +- **WHEN** the operator runs the backfill command +- **THEN** all recallable items receive embeddings under the active model +- **AND** the command reports counts embedded, skipped (hash-unchanged), and + failed + +### Requirement: Ratified consolidation with dry-run plan files + +Corpus consolidation SHALL be two-phase and operator-gated. A dry-run SHALL +build near-duplicate clusters by embedding similarity, synthesize a proposed +lossless merge per cluster, and write a human-editable plan file plus a +readable report — with no database mutation. An apply run SHALL execute a +previously written plan file verbatim (operators veto by editing or deleting +plan entries), SHALL refuse to run against a live daemon by default, SHALL +take a database backup before mutating, and SHALL re-embed merged results and +rebuild affected search rows. Every apply SHALL be recorded in a maintenance +ledger. + +#### Scenario: Dry-run mutates nothing + +- **GIVEN** a corpus containing near-duplicate clusters +- **WHEN** the operator runs consolidation in dry-run mode +- **THEN** a plan file and report are produced +- **AND** the database bytes are unchanged + +#### Scenario: Apply executes only the ratified plan + +- **GIVEN** a reviewed plan file with one cluster entry deleted by the + operator +- **WHEN** the operator runs apply with that plan +- **THEN** a backup of the database is created first +- **AND** the deleted entry's cluster is left untouched +- **AND** the remaining entries are applied and recorded in the maintenance + ledger + +#### Scenario: Apply refuses a live daemon + +- **GIVEN** the daemon is running +- **WHEN** the operator runs consolidation apply without the explicit + live-override flag +- **THEN** the command refuses and names the running daemon + +### Requirement: Expiry sweep deletes expired rows + +The system SHALL periodically delete memory rows whose expiry has passed +beyond a grace window. Expired rows are already excluded from every recall and +search surface, so deletion SHALL be behavior-neutral for reads; each sweep +SHALL log the number of rows removed per class. + +#### Scenario: Expired evidence is physically removed + +- **GIVEN** evidence records whose expiry passed beyond the grace window +- **WHEN** the maintenance sweep runs +- **THEN** those rows are deleted from the store +- **AND** the sweep logs the per-class deletion counts + +### Requirement: Memory status surface + +The CLI SHALL provide `netclaw memory status` reporting corpus composition +(counts by class and recall mode), embedding coverage for the active model, +pending checkpoints, expired-row counts awaiting sweep, and the most recent +maintenance-ledger entries. + +#### Scenario: Operator inspects corpus health + +- **GIVEN** a daemon with a populated memory store +- **WHEN** the operator runs the status command +- **THEN** it reports class/recall-mode counts, embedding coverage, pending + checkpoints, and recent maintenance runs diff --git a/openspec/changes/memory-core-redesign/specs/netclaw-agent-memory/spec.md b/openspec/changes/memory-core-redesign/specs/netclaw-agent-memory/spec.md new file mode 100644 index 000000000..f66f780f9 --- /dev/null +++ b/openspec/changes/memory-core-redesign/specs/netclaw-agent-memory/spec.md @@ -0,0 +1,259 @@ +# Delta: netclaw-agent-memory (memory-core-redesign) + +## MODIFIED Requirements + +### Requirement: Automatic pre-turn recall + +The system SHALL execute automatic recall before each user-facing model turn +using the latest user message, recent session context, active anchors, and +policy scope. Recall SHALL be hybrid: lexical (FTS5) and semantic (embedding +cosine) candidates are merged, and every candidate SHALL pass the identical +audience/boundary/sensitivity/recall-mode policy gates regardless of which +retriever surfaced it. Injection SHALL be gated by an absolute relevance +floor: when no candidate clears the configured minimum semantic similarity, +the turn SHALL inject nothing and the recall context block SHALL be omitted +entirely. Automatic recall SHALL be bounded by a latency budget and SHALL +degrade safely — to lexical-only scoring with a structured degradation log +when the embedder is unavailable or over its sub-budget, and to no injection +when the memory substrate is unavailable. + +#### Scenario: Recall completes within budget + +- **GIVEN** the memory substrate is healthy +- **WHEN** a new turn begins +- **THEN** the session retrieves and injects a bounded recall bundle before the + model call +- **AND** the recall operation completes within the configured time budget or + degrades safely + +#### Scenario: Nothing relevant means nothing injected + +- **GIVEN** the memory store contains no memory semantically related to the + user's message +- **WHEN** automatic recall runs for the turn +- **THEN** no memory items are injected +- **AND** no recall context block is added to the prompt +- **AND** the retrieval log records zero injected items with the applied floor + +#### Scenario: Vector-sourced candidates obey policy gates + +- **GIVEN** a memory item excluded by the session's audience or sensitivity + policy +- **WHEN** the semantic retriever surfaces that item as a top cosine candidate +- **THEN** the item is filtered before scoring exactly as a lexical candidate + would be + +#### Scenario: Embedder degradation is loud, not silent + +- **GIVEN** the embedding runtime is unavailable or exceeds its per-turn + sub-budget +- **WHEN** automatic recall runs +- **THEN** recall proceeds lexical-only within the same latency budget +- **AND** a structured vector-degradation event is logged for diagnostics + +#### Scenario: Recall failure degrades without blocking the turn + +- **GIVEN** the memory database is temporarily unavailable +- **WHEN** the session starts automatic recall for a turn +- **THEN** the user-facing turn continues without durable recall injection +- **AND** the session records degraded memory status for diagnostics + +### Requirement: Rules-first candidate extraction + +The system SHALL run deterministic rules before any curator LLM call when +converting checkpoints into durable memory. These rules SHALL reject ephemeral +chatter, policy-violating content, and low-confidence candidates before +invoking the curator. Duplicate detection SHALL be semantic: an embedding +nearest-neighbor nomination step SHALL shortlist existing memories above a +configured similarity threshold, and any nomination SHALL force the curator +LLM to decide the relationship (skip, update, consolidate, or create). +Similarity SHALL only nominate — the system SHALL NOT auto-merge or auto-skip +on a similarity threshold alone. Both write pipelines (inline session curation +and daemon checkpoint curation) SHALL evaluate candidates through one shared +evaluator with identical guards. When the embedding runtime is unavailable, +extraction SHALL fall back to lexical candidate search and SHALL log the +degradation. + +#### Scenario: Trivial chatter is filtered before curation + +- **GIVEN** a checkpoint contains both stable project facts and casual + acknowledgments +- **WHEN** rules-first extraction runs +- **THEN** the stable facts survive as candidates +- **AND** the casual acknowledgments are dropped without calling the curator for + them + +#### Scenario: Paraphrased duplicate is nominated and adjudicated + +- **GIVEN** an existing memory states a fact and a new proposal states the + same fact in different words with low word overlap +- **WHEN** extraction runs with a healthy embedding runtime +- **THEN** the existing memory is nominated by embedding similarity +- **AND** the curator LLM decides the relationship +- **AND** no automatic merge occurs from the similarity score alone + +#### Scenario: Novel proposal skips the curator + +- **GIVEN** a proposal with no embedding nomination above the threshold and no + matching anchor +- **WHEN** extraction runs +- **THEN** the proposal is stored as a new memory without a curator LLM call + +### Requirement: Documents versus records semantics + +The system SHALL distinguish mutable `documents` from immutable `records`. +Documents SHALL represent living, mergeable knowledge. A curator merge +decision (consolidate or update) SHALL produce a merged body that preserves +the information of every source document; a deterministic merge guard SHALL +verify load-bearing content (identifiers, numbers, dates, URLs) survives, and +on guard failure the system SHALL fall back to a lossless structural append +with provenance rather than overwriting. Destructive whole-body replacement +SHALL NOT be reachable from curation decisions. Records SHALL represent +time-bound observations that are immutable once written and can only be +superseded, expired, or tombstoned by subsequent operations; dated +observations SHALL be stored as new entries, never overwritten by newer +readings. + +#### Scenario: Preference update modifies a document + +- **GIVEN** an operator preference is stored as a document on a `person` anchor +- **WHEN** the operator corrects that preference later +- **THEN** the system updates the document according to its merge semantics +- **AND** preserves version lineage for auditability + +#### Scenario: Lossy merge output falls back to append + +- **GIVEN** the curator produces a merged body missing load-bearing content + from a source document +- **WHEN** the merge guard validates the merge +- **THEN** the merge is rejected +- **AND** the proposal is appended to the existing document with a dated + separator instead + +#### Scenario: Historical event becomes a superseded record + +- **GIVEN** a host IP change is stored as a record on a `host` anchor +- **WHEN** a newer verified IP change is persisted +- **THEN** the new fact is stored as a new record +- **AND** the older record is marked as superseded rather than overwritten + +### Requirement: Durable memory policy envelope + +Every durable anchor, document, and record SHALL carry policy metadata +including `audience`, `sensitivity`, `recallMode`, `confidence`, `freshness`, +and `updateSemantics`. The write path SHALL assign or reject these values +before persistence, and the recall path SHALL filter by them before prompt +injection. Recall modes SHALL mean what they name: `auto` items are eligible +for automatic pre-turn recall; `searchable` items surface only through +explicit search tools; `manual` items are reachable only by explicit id; +`never` items are hidden from all recall surfaces. Formation SHALL default +newly distilled durable facts to `searchable`, reserving `auto` for standing +facts intended to color every conversation (identity, durable preferences, +environment). + +#### Scenario: Sensitive memory is blocked from auto recall + +- **GIVEN** a stored memory item is marked `audience=personal`, + `sensitivity=secret`, and `recallMode=manual` +- **WHEN** a session whose audience does not include `personal` runs automatic + pre-turn recall +- **THEN** the item is excluded from the automatic recall bundle +- **AND** it remains available only to explicit authorized workflows if policy + allows + +#### Scenario: Searchable items stay out of automatic recall + +- **GIVEN** a memory item with `recallMode=searchable` +- **WHEN** automatic pre-turn recall runs on a strongly matching query +- **THEN** the item is not injected automatically +- **AND** an explicit `find_memories` search for the same terms returns it + +#### Scenario: Topical distillate defaults to searchable + +- **GIVEN** the observation sidecar distills a topical project fact without an + explicit recall-mode proposal +- **WHEN** the proposal passes the policy gate +- **THEN** it persists with `recallMode=searchable` + +## REMOVED Requirements + +### Requirement: Hierarchical anchor graph memory model + +**Reason**: Measured vestigial — the `memory_edges` table has held zero rows +across the system's production lifetime, no recall path traverses hierarchy or +typed edges, and semantic (embedding) retrieval supersedes graph expansion as +the mechanism for surfacing related memories. Maintaining the graph schema and +its write-side metadata is carrying cost without a reader. + +**Migration**: Anchors remain as flat grouping keys (including the per-tool +anchors used by tool lessons). The `memory_edges` table and its DDL are +dropped; no data migration is required because no data exists. Related-memory +discovery is served by embedding similarity (`memory-embeddings` capability). + +## ADDED Requirements + +### Requirement: Short-lived trace memories + +The system SHALL support a short-lived memory class for operational state +that is useful for roughly its TTL (default 72 hours) and worthless after — +deploy states, in-flight incident context, temporary environment quirks. The +observation sidecar SHALL be able to propose this class; fresh (unexpired) +trace memories SHALL be eligible for automatic recall weighted below durable +facts; expired trace memories SHALL be deleted by the maintenance sweep, not +merely hidden. + +#### Scenario: Fresh trace surfaces, expired trace is gone + +- **GIVEN** a trace memory recording an in-flight deployment state, created + one hour ago +- **WHEN** the user asks about the deployment +- **THEN** the trace is eligible for the automatic recall bundle +- **WHEN** the trace's TTL elapses and the maintenance sweep runs +- **THEN** the row is deleted from the store + +### Requirement: Tool-use lesson memories + +The system SHALL support a tool-lesson memory class capturing durable lessons +about correct tool usage (conventions, flags, pitfalls), stored as mergeable +documents anchored to the tool they concern. Lessons SHALL be capturable +explicitly through the memory tools and proposable by the observation sidecar +when a transcript shows the user correcting the agent's tool usage. Lessons +SHALL surface through per-tool context injection: on a tool's first use in a +session, a bounded lessons block for that tool SHALL be appended to the tool +result, outside the pre-turn recall budget, at most once per tool per session +(reset on compaction). Lessons for one tool SHALL deduplicate through the +standard curation pipeline so they consolidate into few comprehensive +documents per tool. + +#### Scenario: Correction becomes a lesson and surfaces on next use + +- **GIVEN** the user corrects the agent's usage of a tool (e.g., a release + tag must not carry a `v` prefix) +- **WHEN** the lesson is stored as a tool-lesson memory anchored to that tool +- **AND** a later session uses that tool for the first time +- **THEN** the tool result carries a bounded lessons block containing the + lesson +- **AND** subsequent uses of the same tool in that session do not repeat the + block + +#### Scenario: Repeated lessons consolidate per tool + +- **GIVEN** an existing lesson document for a tool +- **WHEN** a semantically overlapping lesson for the same tool is proposed +- **THEN** the curation pipeline nominates the existing lesson and the curator + merges losslessly rather than accumulating near-duplicates + +### Requirement: Checkpoint enqueue gating + +The daemon checkpoint pipeline SHALL NOT enqueue work it will deterministically +discard: turn-complete checkpoints SHALL be gated at enqueue time by the same +precondition the extractor applies, so the worker's intake consists of +checkpoints that can produce memory operations. + +#### Scenario: Unextractable turn produces no checkpoint + +- **GIVEN** a completed turn whose content contains no extractable memory + candidate +- **WHEN** the session evaluates checkpoint enqueue +- **THEN** no turn-complete checkpoint is enqueued +- **AND** explicit memory requests and compaction boundaries are unaffected diff --git a/openspec/changes/memory-core-redesign/tasks.md b/openspec/changes/memory-core-redesign/tasks.md new file mode 100644 index 000000000..fa2cf7207 --- /dev/null +++ b/openspec/changes/memory-core-redesign/tasks.md @@ -0,0 +1,76 @@ +# Tasks: memory-core-redesign + +Slices are independently shippable in order; each slice's final tasks are its +constitution gates (tests, evals where mapped, schema/skill sync, slopwatch). + +## 1. Shared curation evaluator (behavior-neutral refactor) + +- [ ] 1.1 Extract `MemoryCurationEvaluator` from `MemoryCurationActor.EvaluateSingleAsync` and wire the actor through it +- [ ] 1.2 Route `MemoryCurationEngine` (daemon worker) through the same evaluator, including `GuardDestructiveUpdate`, deleting the divergent inline logic +- [ ] 1.3 Characterization tests proving inline and daemon paths produce identical decisions for the same inputs +- [ ] 1.4 Run full memory test suites + slopwatch; no behavior change expected (decision-mix log fields unchanged) + +## 2. Embedding foundation + +- [ ] 2.1 Create `src/Netclaw.Embeddings` project (Microsoft.ML.OnnxRuntime CPU, FastBertTokenizer, System.Numerics.Tensors) and `IMemoryEmbedder` seam in `Netclaw.Actors/Memory` +- [ ] 2.2 Implement `OnnxMemoryEmbedder` (single InferenceSession, bounded intra-op threads, concurrency semaphore) + `UnavailableMemoryEmbedder` +- [ ] 2.3 Implement `EmbeddingModelProvisioner`: pinned allowlist (id → URL, size, SHA-256), atomic download, hash verification, rejection of unknown ids +- [ ] 2.4 Add `memory_embeddings` table + `UpsertEmbeddingAsync`/`FindNearestByEmbeddingAsync`/coverage queries to `SQLiteMemoryStore.InitializeAsync` (idempotent DDL) +- [ ] 2.5 Implement `MemoryContentHasher` (normalized title+body SHA-256) and hash-skip on re-embed +- [ ] 2.6 Implement `MemoryVectorIndex` (per-model flat float[] brute-force cosine, store-version invalidation) +- [ ] 2.7 `EmbeddingWarmupHostedService`: provision-or-degrade at startup, warm-up inference, gap-repair sweep; register `IMemoryEmbedder` in daemon DI +- [ ] 2.8 Embed-on-write after both curation batch commit paths +- [ ] 2.9 `netclaw memory backfill-embeddings [--force]` CLI command +- [ ] 2.10 `MemoryEmbeddingDoctorCheck` (model presence/hash, coverage, mixed-model warning) + daemon status `embeddings: degraded` surface + rate-limited degradation logs +- [ ] 2.11 Config: `Memory.Embeddings { Enabled, ModelId, AutoDownload }` + schema sync with defaults +- [ ] 2.12 Tests: provisioner hash-rejection/unknown-id, hash-skip, gap repair, vector index invalidation, degraded stub; CI uses a tiny fixture ONNX model (no downloads in tests) +- [ ] 2.13 **Measure ONNX int8 short-query embedding latency on reference hardware; record the number in design.md and gate Slice 4's sub-budget on it** +- [ ] 2.14 ARM64 publish smoke leg exercising OnnxRuntime load +- [ ] 2.15 Update `netclaw-memory` + `netclaw-operations` skills (backfill command, degraded mode); eval suite run + +## 3. Write-side nominate→decide + lossless merge + +- [ ] 3.1 Nominator in the shared evaluator: kNN shortlist at `Memory.Curation.NominatorSimilarityThreshold`/`NominatorK`; any nominee forces the LLM tier; no-nominee-no-anchor creates without LLM; lexical candidate search becomes the logged degraded path +- [ ] 3.2 Extend `CurationPromptBuilder` response protocol: CONSOLIDATE/UPDATE emit a merged body; `CurationDecision.MergedBody`; full-content previews for nominated candidates +- [ ] 3.3 Implement `MergeGuard` (load-bearing-token retention ≥95%, length collapse check) with structural-append fallback producing `AppendDocument` semantics +- [ ] 3.4 Route all curation UPDATE/CONSOLIDATE writes through guard-validated merged bodies; make raw whole-body overwrite unreachable from curation decisions +- [ ] 3.5 Config: `Memory.Curation { NominatorSimilarityThreshold, NominatorK, LlmMaxOutputTokens, LlmTimeoutSeconds }` (replacing hardcoded constants) + schema sync +- [ ] 3.6 Tests: paraphrase-dupe nomination (fixture pairs from the audit corpus shape), sibling pairs never auto-merge, MergeGuard property tests, append fallback, both-pipelines parity +- [ ] 3.7 Eval suite (memory category) + skill sync; update decision-mix expectations (consolidate share should rise from ~0.1%) + +## 4. Read-side hybrid recall + absolute floor + +- [ ] 4.1 Query embedding per turn with a vector sub-budget inside `RecallTimeoutMs`; lexical-only fallback + `memory_recall_vector_degraded` log on miss +- [ ] 4.2 Candidate union (FTS5 ∪ vector top-k) with policy-gate parity for vector-sourced hits +- [ ] 4.3 Weighted fusion scoring + `MinCosineSimilarity` absolute floor; omit the `[memory-recall]` block entirely on zero injections +- [ ] 4.4 Recency half-life decay (floor-bounded multiplier) on composite scores +- [ ] 4.5 Config: `Memory.Recall { VectorWeight, LexicalWeight, MinCosineSimilarity, RecencyHalfLifeDays }` + schema sync +- [ ] 4.6 Calibrate the floor against `gold-prod-2026-07` (local gold set); record calibration numbers in design.md +- [ ] 4.7 Gold-set recall regression suite (fixture corpus + labeled queries asserting injected/withheld ids, MRR/precision floors, zero-injection cases) +- [ ] 4.8 Flip scenario P09 (paraphrase-gap) back to expected-recall; policy-parity scenario test; latency budget test with warm embedder +- [ ] 4.9 Eval suite + `netclaw-memory` skill update (hybrid recall, zero-injection normality) + +## 5. Taxonomy rebalance, trace revival, tool lessons + +- [ ] 5.1 **BREAKING**: restrict automatic recall to `recall_mode='auto'` in `SearchByPlanAsync` (searchable leaves the auto pool); update `MemoryIndexContextLayer` guidance +- [ ] 5.2 Formation: policy gate honors sidecar-proposed recall mode for durable facts, defaulting to `searchable`; observer distillation prompt rewritten for fewer, more comprehensive proposals with an explicit auto-mode whitelist (identity/preferences/environment) +- [ ] 5.3 Trace revival: reachable producer (sidecar may propose `trace` with 72 h TTL), fresh-trace auto-recall eligibility weighted below durable facts, removal of the unreachable turn-complete Trace dead code +- [ ] 5.4 `MemoryClass.ToolLesson` (`tool_lesson`) → Document/MergeDocument/Searchable with per-tool anchors; `store_memory` accepts the class and sets the `VerifiedToolFinding` checkpoint flag +- [ ] 5.5 Sidecar distillation prompt: correction-hunting instruction producing tool-lesson proposals +- [ ] 5.6 Per-tool context injection in the tool-execution pipeline: `[tool-lessons:]` block on first use per session (bounded, once per tool, reset on compaction); remove the dead `verified-tool-finding` +25 recall bonus +- [ ] 5.7 Tests: searchable-out-of-auto regression, formation default, trace TTL round-trip, lesson capture→injection end-to-end, once-per-session + compaction reset +- [ ] 5.8 Eval cases: tool-lesson store→new-session→first-tool-use surfaces lesson; must-auto-recall identity facts still auto-recall after rebalance +- [ ] 5.9 Skill sync (`netclaw-memory`: classes table, lessons guidance) + schema sync for any new wire values + +## 6. Maintenance CLI, expiry sweep, subtraction + +- [ ] 6.1 `memory_maintenance_runs` ledger table (store DDL) +- [ ] 6.2 `netclaw memory consolidate --dry-run`: kNN cluster graph → merge synthesis → `plan.jsonl` + report, zero mutation (byte-identical DB test) +- [ ] 6.3 `netclaw memory consolidate --apply --plan `: live-daemon refusal (override flag), `VACUUM INTO` backup, batched apply, re-embed + FTS rebuild, ledger row +- [ ] 6.4 Expiry sweep in the daemon maintenance loop (grace window, per-class deletion logging) +- [ ] 6.5 `netclaw memory status` (composition, coverage, pending checkpoints, expired-awaiting-sweep, recent ledger) +- [ ] 6.6 Checkpoint enqueue gating: turn-complete lane gated by the extractor's precondition at enqueue time +- [ ] 6.7 Subtraction: drop `memory_edges` DDL, remove facet/soft-scope inference from `DeterministicRetrievalPlanning` (keep stopword hygiene + lexical terms), delete dead Trace path remnants +- [ ] 6.8 Integration tests on a seeded corpus: backfill→dry-run→edited-plan apply→status round-trip; sweep deletes only past-grace rows +- [ ] 6.9 Runbook update (`docs/runbooks/memory-health-and-evals.md`): embedding, consolidation, sweep operations +- [ ] 6.10 Final gates: full test suites, eval suite, slopwatch, headers, schema doctor round-trip on a real config