Skip to content

refactor(memory): delete dead sidecar infrastructure and HardScope plumbing#586

Merged
Aaronontheweb merged 2 commits into
devfrom
memory-system-cleanup
Apr 10, 2026
Merged

refactor(memory): delete dead sidecar infrastructure and HardScope plumbing#586
Aaronontheweb merged 2 commits into
devfrom
memory-system-cleanup

Conversation

@Aaronontheweb

Copy link
Copy Markdown
Collaborator

Summary

Part one of a multi-step cleanup of the Netclaw memory subsystem. This PR deletes dead code only — no behavior changes for any test that was already passing, no schema migrations, no API surface that users could reach. It's a pure net-removal refactor that leaves the memory system easier to reason about before checkpoint 2 (the Domain field + schema migration) lands.

Background

While closing #582 I discovered that the memory subsystem had two large pieces of dead infrastructure sitting as "scheduled for removal" tombstones since late March:

  1. Recall / observation sidecars. Introduced as a prototype in PR feat(memory): add observer-driven recall planning and retrieval prototype #198 on 2026-03-12, replaced by DeterministicCandidateSelector for recall and by SessionMemoryObserverActor.RunDistillationAsync for memory formation within ~2 weeks. The infrastructure got an explicit "Scheduled for removal" annotation on 2026-03-25 (PR refactor: consolidate SessionConfig — reduce 16 properties to ~5 meaningful settings #414) but the deletion never shipped. Zero production code paths reach any of it.
  2. HardScope / HardScopeOverride / domain-derived boundary inference. DeterministicRetrievalRequestPlanner.ResolveHardScope parsed a session ID prefix into a per-project domain, but the coordinator always populated HardScopeOverride first via Protocol.SessionId.ToMemoryDomain(), which unconditionally returns SecurityPolicyDefaults.DefaultMemoryDomain. The session-ID parsing branch was unreachable. Domain affinity in DeterministicCandidateSelector was disabled in fix(memory): add composite-score floor to recall ranker (#582) #585 for the same reason.

Beyond those two, the code review pass also flagged a handful of small residuals: dead parameters on MemoryPolicyScopeResolver.ResolveBoundary, a duplicate private helper in the coordinator, dead message records in LlmMessages, and two WHERE 1 = 1 SQL placeholders. Those are cleaned up in the second commit.

Design decision: "audience + lexical/facet/anchor matching, nothing else"

Before touching anything I verified with the project owner that the mental model going forward is:

That framing is the basis for every deletion in this PR. If a concept only existed to prop up one of the three things above (hard scope, LLM-driven sidecars, domain-derived boundaries) it's gone.

What this PR does

Sidecar infrastructure deleted

File Disposition
src/Netclaw.Actors/Sessions/SessionSidecarRunner.cs Deleted entire file (321 lines) — LLM sidecar call executor, only reachable from the dead recall fallback path
src/Netclaw.Actors/Sessions/MemorySidecarPromptBuilder.cs Reduced from 151 lines to the single BuildClassificationRules helper still used by the live SessionMemoryObserverActor distillation path
src/Netclaw.Actors/Sessions/SidecarRecallPlanner.cs SidecarMemoryObserver class deleted (observation sidecar). SidecarRecallPlanner class retained — still used by the live SqliteFindMemoriesTool to build structural request plans without any LLM involvement. domain parameter dropped from BuildRequest signature.
src/Netclaw.Actors/Sessions/MemorySidecarContracts.cs MemoryObservationRequest, MemoryObservationCurrentTurn, MemoryObservationRecentContext, MemoryObservationPolicyScope, MemoryObservationCompleted, RecallPlanningCompleted all deleted. MemoryProposal, MemoryAnchor, MemoryRelation, MemoryProposalOperation, RecallPlanningRequest, RecallQueryPlan retained — all live via the distillation path or the find_memories tool.
src/Netclaw.Actors/Sessions/SQLiteMemoryRecallCoordinator.cs The entire if (!_sessionTuning.MemorySidecarsEnabled) … fallback branch in RecallAsync, plus BuildPlanAsync, LogRecallTrace, TokenizeTerms, FallbackSearchTerms, NormalizeRequest. Constructor parameters reduced from 6 (store, logger, clientProvider, sidecarPlanner, recallPlanGate, sessionTuning, sessionConfig) to 3 (store, logger, sessionTuning).
src/Netclaw.Actors/Sessions/LlmSessionActor.cs Command<MemoryObservationCompleted> handler deleted — registered handler with zero senders.
src/Netclaw.Actors/Sessions/LlmMessages.cs MemoryObservationFailed and RecallPlanningFailed records deleted — also had no senders.
src/Netclaw.Configuration/SessionTuning.cs MemorySidecarsEnabled flag deleted. Config binding in SessionConfig, JSON schema entry in netclaw-config.v1.schema.json, and the Memory_sidecars_enabled_by_default default-check test in SessionConfigDefaultsTests all removed to match.

HardScope / domain plumbing deleted

File Disposition
src/Netclaw.Actors/Sessions/DeterministicRetrievalPlanning.cs ResolveHardScope and IsTransportScopedSession deleted. HardScope property removed from DeterministicRetrievalRequestPlan.
src/Netclaw.Actors/Sessions/IMemoryRecallCoordinator.cs HardScopeOverride field removed from AutomaticRecallRequest.
src/Netclaw.Actors/Sessions/Pipelines/SessionRecallManager.cs Dropped the HardScopeOverride: sessionId.ToMemoryDomain() argument from the recall request construction.

Boundary resolution simplified

All Netclaw memory runs inside a single trusted-instance daemon. InferLegacyBoundaryFromDomain ultimately returned TrustedInstanceBoundary for every input the system actually produces. Rather than thread that fiction through 7 call sites:

  • Every call to SecurityPolicyDefaults.InferLegacyBoundaryFromDomain(...) in the recall path replaced with SecurityPolicyDefaults.TrustedInstanceBoundary directly. Curation-pipeline call sites are still on the old function and are deferred to checkpoint 2.
  • MemoryPolicyScopeResolver.ResolveBoundary simplified to the single-value case: if a caller-supplied boundary is present, trim and return it; otherwise return TrustedInstanceBoundary. The audience and sessionId parameters became dead weight and were dropped from the signature and every call site (curation pipeline, policy gates, all four SQLite memory tools).
  • The private SQLiteMemoryRecallCoordinator.ResolveBoundary helper was byte-identical to the scope-resolver method after simplification — inlined at the call site and deleted.

Search method collapse

SQLiteMemoryStore.SearchByPlanAsync (the domain-scoped variant) and SearchAcrossDomainsByPlanAsync were merged into the single cross-domain method. The domain parameter is gone, the WHERE d.domain = $domain / WHERE r.domain = $domain SQL clauses are gone, and two WHERE 1 = 1 scaffolding artifacts left behind by the inlining were also removed.

Test fallout

  • Deleted entire files: MemorySidecarPromptBuilderTests.cs (tested the deleted prompt builder methods) and SessionSidecarRunnerTests.cs (tested the deleted runner).
  • Deleted specific tests: Sidecar_observation_promotes_strong_user_assertion_into_memory from LlmSessionIntegrationTests, plus the "memory observation sidecar" branch of the fake chat client that only existed to serve that test.
  • Parameter cleanup in ~17 test files: MemorySidecarsEnabled = false/true literals stripped from every SessionTuning constructor call. HardScopeOverride: "user:aaron" dropped from DeterministicRetrievalPlanningTests, MemoryRedesignedEvalSuiteTests. plan.HardScope assertions removed.
  • DeterministicCandidateSelectorTests fixture update: MakePlan lost its hardScope parameter. The Score_geometry_domain_affinity_boosts_same_domain_match test from fix(memory): add composite-score floor to recall ranker (#582) #585 was replaced with Domain_is_not_a_scoring_signal which asserts same-domain and cross-domain candidates score identically.
  • SQL call site updates: SQLiteMemoryStoreTests and MemoryRedesignedEvalSuiteTests switched from SearchByPlanAsync(..., "project:test", ..., InferLegacyBoundaryFromDomain(...), ...) to SearchAcrossDomainsByPlanAsync(..., TrustedInstanceBoundary, ...).

What this PR deliberately does NOT do

These are all tracked as checkpoint 2 work:

  • Protocol.SessionId.ToMemoryDomain() is still a trivial constant passthrough with ~14 call sites. Inlining it is mechanical churn across LlmSessionActor, SessionMemoryObserverActor, the SQLite memory tools, SubAgentActor, and SessionToolExecutionPipeline.
  • The Domain field on SQLiteMemoryDocument, SQLiteMemoryHydratedItem, SQLiteMemorySearchResult, SQLiteMemoryCurationOperation, SQLiteMemoryAnchor, AutomaticRecallItem, and MemoryCheckpointPayload stays for now. Removing it touches every writer and reader of every memory record type.
  • The domain column on memory_documents, memory_records, memory_anchors, memory_edges stays. Dropping it needs an idempotent schema migration path, and schema migration risk is better decoupled from this deletion-heavy PR.
  • NormalizeLegacyBoundariesAsync still reads the domain column to bucket pre-existing data. Dead once the column is gone.
  • SecurityPolicyDefaults.InferLegacyBoundaryFromDomain still has two callers in MemoryCurationPipeline (lines 319, 508) that haven't been migrated yet. SecurityPolicyDefaults.DefaultMemoryDomain stays as the constant the column receives.
  • openspec/specs/netclaw-agent-memory/spec.md still references domain as part of the memory policy envelope and has a scenario about domain=business filtering. The OpenSpec update will land alongside the checkpoint 2 PR so the spec describes the final landed state in one pass.

Net effect

~1,450 lines deleted, ~80 added across two commits. All 1,967 tests across 7 test projects pass locally. Slopwatch clean (0 issues). dotnet build clean (0 warnings).

Closes #584 in spirit — the decision captured there is "domain scoping is the wrong mental model; audience handles it." The formal issue close will follow the checkpoint 2 PR that finishes removing the vestigial column and field.

Test plan

  • dotnet build green across all projects (0 warnings, 0 errors)
  • dotnet test src/Netclaw.Actors.Tests/Netclaw.Actors.Tests.csproj — 840 passed / 0 failed
  • dotnet test full solution — 1,967 passed / 0 failed across 7 test projects
  • dotnet slopwatch analyze — 0 issues
  • CI green on this PR

…umbing

Audience is the only security axis for memory access. Lexical/facet/anchor
matching is the only relevance signal. Everything else (recall sidecar,
observation sidecar, HardScope, domain-derived boundary inference) was dead
or vestigial and has been removed in a single pass.

Sidecar infrastructure deleted (dead in production since ~March 12, 2026):
- SessionSidecarRunner.cs (entire file) — LLM sidecar call executor, only
  invoked by the recall fallback path that production never hits
- SidecarMemoryObserver class and MemoryObservationRequest + subtypes —
  replaced by SessionMemoryObserverActor.RunDistillationAsync ~March 24
- MemoryObservationCompleted + its handler in LlmSessionActor — registered
  but no senders existed
- RecallPlanningCompleted — zero callers
- MemorySidecarPromptBuilder reduced to just BuildClassificationRules
  (the only method still used by the live distillation path)
- The entire sidecar fallback path in SQLiteMemoryRecallCoordinator.RecallAsync
  (BuildPlanAsync, LogRecallTrace, TokenizeTerms, FallbackSearchTerms),
  including the IChatClientProvider / SidecarRecallPlanner / RecallPlanGate /
  SessionConfig constructor parameters that were only used by that path
- SessionTuning.MemorySidecarsEnabled flag + its config binding + JSON schema
  entry + the SessionConfigDefaultsTests.Memory_sidecars_enabled_by_default
  assertion

HardScope / domain-derived-boundary plumbing deleted:
- DeterministicRetrievalRequestPlanner.ResolveHardScope and
  IsTransportScopedSession — session-ID prefix parsing that the coordinator
  bypassed because it populated HardScopeOverride first
- HardScope field on DeterministicRetrievalRequestPlan, HardScopeOverride
  field on AutomaticRecallRequest, NormalizeRequest on the coordinator
- All SecurityPolicyDefaults.InferLegacyBoundaryFromDomain call sites
  replaced with SecurityPolicyDefaults.TrustedInstanceBoundary. Netclaw is a
  single-instance trusted-boundary system; the "derive boundary from domain
  prefix" logic was dead code because ToMemoryDomain always returns
  project:default anyway
- MemoryPolicyScopeResolver.ResolveBoundary simplified — no session-ID or
  domain heuristics, always returns TrustedInstanceBoundary when no explicit
  boundary is provided

Search method cleanup:
- SQLiteMemoryStore.SearchByPlanAsync (domain-scoped variant) collapsed into
  SearchAcrossDomainsByPlanAsync — the sole remaining search method
- WHERE d.domain = ? / WHERE r.domain = ? filters removed from the SQL
- domain parameter dropped from SidecarRecallPlanner.BuildRequest and
  RecallPlanningRequest

Planner anchor-hint filter:
- The filter that strips sentence-start capitalized stopwords from anchor
  hints (landed in #582) stays and expands slightly

Test fallout cleaned up across ~20 test files:
- MemorySidecarPromptBuilderTests.cs and SessionSidecarRunnerTests.cs
  deleted outright
- Sidecar_observation_promotes_strong_user_assertion_into_memory test and
  the memory-observation-sidecar branch of the fake chat client removed
  from LlmSessionIntegrationTests
- MemorySidecarsEnabled literals stripped from 17 test SessionTuning
  constructions
- HardScopeOverride / plan.HardScope references removed from
  DeterministicRetrievalPlanningTests, DeterministicCandidateSelectorTests,
  MemoryRedesignedEvalSuiteTests, MemoryRecallScenarioTests
- InferLegacyBoundaryFromDomain call sites in SQLiteMemoryStoreTests and
  MemoryRedesignedEvalSuiteTests updated to TrustedInstanceBoundary
- Score_geometry_domain_affinity_boosts_same_domain_match test replaced
  with Domain_is_not_a_scoring_signal assertion

Intentionally NOT touched (deferred to a follow-up PR):
- ToMemoryDomain() method and its ~14 call sites — still returns the
  constant DefaultMemoryDomain, which is still written to the domain column
- The Domain field on SQLiteMemoryDocument, SQLiteMemoryHydratedItem,
  SQLiteMemorySearchResult, SQLiteMemoryCurationOperation, SQLiteMemoryAnchor
- The domain column on memory_documents, memory_records, memory_anchors,
  memory_edges tables
- NormalizeLegacyBoundariesAsync (still reads the domain column)
- SecurityPolicyDefaults.DefaultMemoryDomain and InferLegacyBoundaryFromDomain
  (the latter is now dead code but kept alongside DefaultMemoryDomain for the
  followup)

Those are a separate, larger cleanup that requires schema-migration work and
is riskier to land in the same commit. Tracked as the next step in #584.

Net: ~1,400 lines deleted, ~60 added. All 1,967 tests green across every
test project. Slopwatch clean.
Follow-on cleanup pass from code review of the previous commit.

- Drop dead audience and sessionId parameters from
  MemoryPolicyScopeResolver.ResolveBoundary. Boundary is a single-valued
  constant (audience is the security axis), so neither parameter was read.
  Updated all 7 call sites across the curation pipeline, policy gates, and
  the SQLite memory tools.
- Inline the private ResolveBoundary helper in SQLiteMemoryRecallCoordinator
  — it duplicated MemoryPolicyScopeResolver.ResolveBoundary after the
  simplification. Coordinator now uses the shared resolver directly.
- Delete the MemoryObservationFailed and RecallPlanningFailed message
  records from LlmMessages.cs. Zero senders and zero receivers after the
  previous commit removed the sidecar handler.
- Delete two WHERE 1 = 1 placeholder artifacts in
  SearchAcrossDomainsByPlanAsync — leftover scaffolding from the inlined
  domain-filter clauses that got collapsed in the previous commit.
- Trim narrating-history comments down to the non-obvious WHY, or remove
  outright where the code already says what the comment restated.

Net another ~30 lines deleted. All 840 Actors tests and the full 1,967-test
solution suite still pass. Slopwatch clean.
@Aaronontheweb Aaronontheweb enabled auto-merge (squash) April 10, 2026 22:12
@Aaronontheweb Aaronontheweb merged commit 4736471 into dev Apr 10, 2026
3 checks passed
@Aaronontheweb Aaronontheweb deleted the memory-system-cleanup branch April 10, 2026 22:17
@Aaronontheweb Aaronontheweb added memory Memory formation, recall, curation pipeline cleanup Code quality improvements and tech debt reduction labels Apr 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cleanup Code quality improvements and tech debt reduction memory Memory formation, recall, curation pipeline

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Memory domain scoping is half-implemented: all sessions collapse to project:default

1 participant