Feat/hindsight smart retain pipeline pluggable

[McClean]

What does this PR do?

Adds a configurable client-side smart retain pipeline to the Hindsight memory plugin to reduce write amplification and improve recall quality at scale. Without this, every hindsight_retain call hits the bank — duplicates, near-duplicates, and noisy raw-text retains all land as separate units, the bank grows linearly with chatter, and recall quality degrades. With this, retains are deduped, extracted, and delta-merged client-side before they reach the API, and the whole pipeline runs through a dedicated auxiliary task slot so the classifier model can be pinned cheaply (gpt-oss-20b / gemma-3-flash) without dragging the main chat model into pre-retain work.

The pipeline is opt-in via plugin config and degrades cleanly: if the auxiliary call fails repeatedly, the circuit breaker opens and (per aux_fallback_to_main knob) either falls back to the main model or skips the smart step and writes raw.

Depends on #29817 (feat(plugins): add register_auxiliary_task()). That PR adds the PluginContext.register_auxiliary_task() API, which this plugin uses to declare its memory_retain_filter slot without modifying core files. Don't merge this until the parent lands; once it does, this branch rebases cleanly onto the merge commit, and the dependency commit drops out.

Related Issue

N/A — companion PR to #29817.

Fixes #

Type of Change

• ✨ New feature (non-breaking change that adds functionality)

Changes Made

• plugins/memory/hindsight/__init__.py — adds the smart retain pipeline:
  • Pre-retain dedup via recall + auxiliary LLM classifier (skips if a near-duplicate already exists)
  • Client-side extraction (retain_extract) — auxiliary LLM rewrites raw retain text into a normalised memory unit before it hits the bank
  • Delta retain mode — appends only the diff against the most-recent unit on the same scope, keeping bank growth linear
  • Dynamic scope filtering on both recall and retain, so units carry consistent scope tags
  • Auxiliary circuit breaker with aux_fallback_to_main knob — opens after N consecutive aux failures, optionally re-routes to main
  • Smart pipeline applied to both tool retains and the flush-on-switch path (so memory bridging across conversation switches uses the same dedup logic)
  • Bank config fetch from API for richer pre-filter prompts (when api_url is configured)
  • Warning on incomplete bank config when the smart pipeline is active, so misconfiguration fails loudly
  • register(ctx) calls ctx.register_auxiliary_task(key="memory_retain_filter", display_name="Memory retain filter", description="pre-retain content classification (Hindsight smart pipeline)", defaults={"timeout": 30, ...}) — declares the aux task slot through the plugin surface, no core edits
  • Default change: retain_every_n_turns 5 → 1 (smart pipeline is opt-in via the smart-pipeline config block; sampling default tightened so the pipeline sees every turn when it's on)
  • Recall-type filtering in the dedup check (only compares against retained units, not extracted-snippet hits)
  • Bank config fetch gated on api_url presence only (was previously coupled to other flags)
• tests/agent/test_memory_session_switch.py — adds 11 lines covering test fixture restoration for the new retain-pipeline attributes on the memory provider, so cross-session switch tests don't leak state between runs
• tests/plugins/memory/test_hindsight_provider.py — adds 8 lines covering the new pipeline knobs

How to Test

1. Land feat(plugins): add register_auxiliary_task() to PluginContext API #29817 first, then rebase this branch onto the merge commit (git rebase main — the dependency commit 2d94abe9a will drop out cleanly)
2. pytest tests/plugins/memory/test_hindsight_provider.py tests/agent/test_memory_session_switch.py -q — all pass
3. Configure the smart pipeline in ~/.hermes/config.yaml:

yaml
plugins:
memory_provider:
hindsight:
smart_pipeline:
enabled: true
dedup: true
extract: true
delta: true
aux_fallback_to_main: false

4. Run hermes model → Configure auxiliary models and confirm memory_retain_filter appears in the picker (provided by the registration API in the parent PR). Pin it to a cheap classifier model
5. Start a chat, retain something, then retain a near-duplicate of it — confirm the second retain is skipped (look for dedup: skipped in the agent log)
6. Retain a long raw paragraph — confirm it lands as a normalised extracted unit, not the raw text
7. Retain twice on the same scope with overlapping content — confirm the second unit is a delta against the first, not a full copy
8. Force the auxiliary endpoint to fail (point it at an invalid base_url) and confirm the circuit breaker opens after the configured threshold and aux_fallback_to_main is honoured
9. Trigger a session switch with pending retains and confirm the flush path runs through the same smart pipeline (not bypassed)

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (feat(hindsight):)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits — once rebased on the parent merge commit)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (test fixture restoration + provider knobs)
• I've tested on my platform: Ubuntu 24.04

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — pipeline knobs are documented inline in the plugin's config schema and docstrings
• I've updated cli-config.yaml.example if I added/changed config keys — N/A (plugin-owned config block under plugins.memory_provider.hindsight.smart_pipeline, schema documented in the plugin)
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A (additive, plugin-internal)
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — pure Python, no path/process assumptions
• I've updated tool descriptions/schemas if I changed tool behaviour — N/A (hindsight_retain / hindsight_recall tool surface is unchanged; the pipeline is a client-side wrapper)

Backwards Compatibility

Non-breaking and opt-in. The smart pipeline is gated behind plugins.memory_provider.hindsight.smart_pipeline.enabled (default off). With it off, the plugin's retain/recall paths are byte-identical to the current behaviour. The retain_every_n_turns default change (5 → 1) only affects users who already have the smart pipeline on, since the pipeline's whole point is to make every-turn retains cheap.

The memory_retain_filter aux task is declared via the parent PR's plugin registration API — no core files are modified; the slot exists only when the Hindsight plugin is loaded.

Screenshots / Logs

N/A — internal pipeline, no UI surface beyond hermes model (where memory_retain_filter appears as another row in the auxiliary picker, contributed by the parent PR).

[teknium1]

Thanks for the work here, @McClean-Edison! The parent PR #29817 has landed on main (via salvage #31177), so this is unblocked.

Before we proceed, this needs @nicolo-esposito's review first — could you ping Nicolo in the Nous Discord and ask him to take a look at this PR? Once he approves the design, Teknium will do the final review and we'll salvage it onto current main with your authorship preserved.

(FYI the cherry-pick onto current main is clean, and the targeted memory-plugin tests pass — just waiting on Nicolo's signoff on the architecture before merging.)

[teknium1]

Thanks for the plugin-scoped implementation; the parent auxiliary-task registration API is now on main, and current main still has no equivalent smart-retain pipeline.

Problems

• aux_fallback_to_main is not a main-model fallback: PR plugins/memory/hindsight/__init__.py:1592 calls get_text_auxiliary_client(""), while current agent/auxiliary_client.py:5148-5156 resolves a task-less auxiliary client with no active main_runtime.
• The registered timeout: 30 at PR plugins/memory/hindsight/__init__.py:2652-2658 is never passed to the direct client.chat.completions.create() calls (for example lines 1620-1625). The same bypass skips the task's extra_body handling.
• The documented plugins.memory_provider.hindsight.smart_pipeline config is not read by the provider loader (plugins/memory/hindsight/__init__.py:349-392); the implementation instead adds flat provider-config fields.
• The new pipeline paths have no behavioral tests; the provider-test change only disables bank-config fetches.

Suggested changes

• Use the auxiliary execution path that applies per-task routing/timeout/extra-body and pass an explicit active-main runtime for the fallback case.
• Align documentation with the actual configuration surface and add hermetic coverage for classifier/dedup/extraction/breaker/scope/flush paths.
• Preserve current append-mode retention behavior added in 09d66037 when salvaging.

This is an automated hermes-sweeper review.

[teknium1]

get_text_auxiliary_client("") does not select the active main-model runtime: it resolves a task-less auxiliary client and receives no main_runtime. Please pass an explicit main runtime into this provider path, or remove the aux_fallback_to_main guarantee.

[teknium1]

The registered memory_retain_filter timeout and extra_body are not applied here: this direct SDK call has neither. Route this through the task-aware auxiliary execution helper (and use it at the other smart-pipeline call sites) so the advertised per-task configuration takes effect.