feat(memory): add observer-driven recall planning and retrieval prototype

Netclaw.slnx

@@ -16,6 +16,7 @@
     <Project Path="src/Netclaw.Cli.Tests/Netclaw.Cli.Tests.csproj" />
     <Project Path="src/Netclaw.Cli/Netclaw.Cli.csproj" />
     <Project Path="src/Netclaw.Actors.Tests/Netclaw.Actors.Tests.csproj" />
+    <Project Path="src/Netclaw.MemoryRetrievalPoC.Tests/Netclaw.MemoryRetrievalPoC.Tests.csproj" />
     <Project Path="src/Netclaw.Tools.Abstractions/Netclaw.Tools.Abstractions.csproj" />
     <Project Path="src/Netclaw.Tools.Generators/Netclaw.Tools.Generators.csproj" />
     <Project Path="src/Netclaw.Search/Netclaw.Search.csproj" />

docs/runbooks/memory-health-and-evals.md

@@ -52,13 +52,24 @@ Only `accepted` findings are enqueued into the memory checkpoint pipeline.
 
 ## Eval Execution
 
-Run the seeded memory quality tests:
+Run the provider-independent memory quality tests:
 
 ```bash
-dotnet test src/Netclaw.Actors.Tests/Netclaw.Actors.Tests.csproj --filter "FullyQualifiedName~SubAgentActorTests"
+dotnet test src/Netclaw.Actors.Tests/Netclaw.Actors.Tests.csproj --filter "FullyQualifiedName~MemoryRedesignedEvalSuiteTests|FullyQualifiedName~MemoryEvalSeedSuiteTests"
 dotnet test src/Netclaw.Cli.Tests/Netclaw.Cli.Tests.csproj --filter "FullyQualifiedName~MemoryCheckpointHealthDoctorCheckTests|FullyQualifiedName~DaemonClientMappingTests"
 ```
 
+Redesigned eval coverage now includes:
+
+- `formation_then_auto_recall`
+- `formation_then_intentional_search`
+- `evidence_vs_durable_separation`
+- `proposal_gate_rejection`
+- `soul_boundary`
+- `expiry_and_staleness`
+
+These suites are synthetic/sanitized and do not require live provider credentials.
+
 Run quality gate checks:
 
 ```bash

feeds/skills/.system/files/identity-management/1.0.4.md

@@ -0,0 +1,69 @@
+# Identity Management
+
+<!-- description: How to keep SOUL.md narrowly scoped to identity/profile updates while project facts and evidence stay in SQLite memory -->
+<!-- triggers: learn user preference | update personality | identity profile | save durable fact | soul update -->
+
+## Overview
+
+Your identity is defined by three files in `~/.netclaw/identity/`. These files
+are loaded into every system prompt, so keep them concise and high-signal.
+
+## Identity Files
+
+| File | Purpose | What Belongs Here |
+|------|---------|-------------------|
+| `SOUL.md` | Who you serve | User's name, family, key relationships, stable communication preferences, timezone. |
+| `AGENTS.md` | How you operate | Behavioral rules, workflow preferences, operating guidelines. |
+| `TOOLING.md` | What you can do | Environment capabilities, installed tools, MCP server notes. |
+
+## SOUL Boundary
+
+`SOUL.md` is a narrow identity/profile surface, not a general memory sink.
+
+Allowed in `SOUL.md`:
+
+- name and relationship facts
+- tone / style / voice preferences
+- standing communication preferences
+- explicit identity/profile updates
+
+Do not put these in `SOUL.md`:
+
+- project facts
+- research passages
+- tool findings
+- troubleshooting evidence
+- execution trace or turn-local breadcrumbs
+
+Those belong in SQLite memory via the memory pipeline, not in identity files.
+
+## How to Edit
+
+1. Always read first.
+2. Only edit identity files for true identity/profile changes.
+3. Keep entries short and durable.
+4. Put project and world knowledge in memory, not `SOUL.md`.
+
+## Progressive Disclosure
+
+Top-level files should be concise summaries. When a topic needs more depth,
+create a detail file in the matching subdirectory:
+
+- `~/.netclaw/identity/soul/`
+- `~/.netclaw/identity/agents/`
+- `~/.netclaw/identity/tooling/`
+
+## Memory Triage
+
+| Information Type | Destination |
+|-----------------|-------------|
+| Personal facts and stable communication preferences | `SOUL.md` |
+| Behavioral and workflow rules | `AGENTS.md` |
+| Environment capabilities and tool configuration | `TOOLING.md` |
+| Project facts, solutions, research, evidence | SQLite memory (`store_memory`, automatic memory, `find_memories`) |
+
+## Rule Of Thumb
+
+If the information should be injected into every prompt forever, it may belong in
+an identity file. If it should only be recalled or searched when relevant, it
+belongs in memory.

feeds/skills/.system/files/memory-usage/1.3.0.md

@@ -0,0 +1,105 @@
+# Memory Usage
+
+<!-- description: How SQLite-backed memory now splits automatic durable recall from intentional evidence search -->
+<!-- triggers: conversation start | user shares durable fact | need prior knowledge | memory error | save information -->
+
+## Default Model
+
+Netclaw memory is SQLite-first.
+
+- Automatic recall runs before each user-facing turn.
+- Automatic recall injects `durable_fact` only.
+- Explicit tools are a deliberate manual-control layer.
+
+Available tools:
+
+- `find_memories`
+- `get_memories`
+- `store_memory`
+- `update_memory`
+
+## Automatic Recall
+
+- Runs before each user-facing turn.
+- Uses bounded recall planning plus deterministic gates.
+- Injects `durable_fact` only.
+- Never injects `evidence` or `trace` into the automatic recall bundle.
+- If degraded, continue the turn and treat memory as partial for that turn.
+
+## Intentional Search
+
+Use `find_memories` + `get_memories` when:
+
+- the user explicitly asks what Netclaw remembers
+- the automatic recall bundle seems insufficient
+- you need targeted retrieval beyond the injected bundle
+
+Normal `find_memories` behavior:
+
+- searches `durable_fact` plus current `evidence`
+- excludes `trace`
+- hides expired evidence by default
+
+Audit/debug search:
+
+- `find_memories(query, include_stale: true)` may surface expired evidence
+- stale evidence is clearly marked with `stale=true`
+
+Two-phase retrieval pattern:
+
+1. `find_memories("query")`
+2. `get_memories("id1, id2")`
+
+## Explicit Writes
+
+### `store_memory`
+
+Use only for deliberate remember/save actions:
+
+- explicit remember requests
+- intentionally pinning a high-value durable fact, decision, or preference
+
+Do not call `store_memory` reflexively on routine turns.
+
+### `update_memory`
+
+Use only to correct or supersede existing memory.
+
+## What The System Stores
+
+- `durable_fact`: stable facts and preferences
+- `evidence`: supporting research, tool findings, and time-bound notes
+- `trace`: short-lived execution breadcrumbs
+
+Freshness rules:
+
+- `durable_fact` is non-expiring by default
+- `evidence` expires and is excluded from auto recall after expiry
+- `trace` is short-lived and never part of normal recall/search behavior
+
+## SOUL Boundary
+
+Do not use identity files as a sink for project facts, research passages, tool
+findings, or evidence. `SOUL.md` is only for narrow identity/profile updates.
+
+## Diagnostics
+
+When memory behavior looks wrong:
+
+1. `netclaw status`
+2. `netclaw doctor`
+3. read `self-diagnostics`
+4. read `docs/runbooks/memory-health-and-evals.md`
+
+Useful log events:
+
+- `memory_recall_plan_resolved`
+- `memory_recall_plan_fallback`
+- `memory_observation_sidecar_completed`
+- `memory_observation_gate_result`
+- `turn_memory_recall`
+
+## Eval Gate
+
+Before rollout, run the redesigned provider-independent eval suites first, then
+optional live smoke checks with local Ollama models.

feeds/skills/.system/manifest.json

@@ -1,17 +1,17 @@
 {
   "schemaVersion": 1,
   "feedType": "system",
-  "updatedAt": "2026-03-07T22:08:39Z",
+  "updatedAt": "2026-03-10T00:41:24Z",
   "skills": [
     {
       "name": "identity-management",
-      "version": "1.0.3",
+      "version": "1.0.4",
       "minimumDaemonVersion": "0.1.0",
-      "sha256": "dc15ea53fbd70077b05f02a00454f7373950681dd58d4e8b23e677bb3681113e",
-      "sizeBytes": 4336,
-      "url": "https://feeds.netclaw.dev/skills/.system/files/identity-management/1.0.3.md",
+      "sha256": "0d3c700ea75d60d4cc258c57419cf6372e2778c92751d6903c0b740c2e5c70be",
+      "sizeBytes": 2383,
+      "url": "https://feeds.netclaw.dev/skills/.system/files/identity-management/1.0.4.md",
       "category": null,
-      "description": "Update SOUL.md/AGENTS.md/TOOLING.md when learning durable user preferences or environment facts. Read for file structure and triage."
+      "description": "How to keep SOUL.md narrowly scoped to identity/profile updates while project facts and evidence stay in SQLite memory"
     },
     {
       "name": "memorizer-usage",
@@ -25,13 +25,13 @@
     },
     {
       "name": "memory-usage",
-      "version": "1.2.0",
+      "version": "1.3.0",
       "minimumDaemonVersion": "0.1.0",
-      "sha256": "b6544a153294d4d44a02d815373b92b150c5030419c673ae60a594f7e8260cf4",
-      "sizeBytes": 2791,
-      "url": "https://feeds.netclaw.dev/skills/.system/files/memory-usage/1.2.0.md",
+      "sha256": "ff84917195f292b999fbfec9c648bbd56649c2e15e9d39967fa77844583f40b2",
+      "sizeBytes": 2846,
+      "url": "https://feeds.netclaw.dev/skills/.system/files/memory-usage/1.3.0.md",
       "category": null,
-      "description": "How SQLite-backed durable memory works with automatic recall and manual memory tools"
+      "description": "How SQLite-backed memory now splits automatic durable recall from intentional evidence search"
     },
     {
       "name": "self-diagnostics",

openspec/changes/add-memory-observer-and-recall-planner/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-03-09