Skip to content

fix(hindsight): bound the append-mode session turn buffer - #62977

Open
Vissirexa wants to merge 1 commit into
NousResearch:mainfrom
Vissirexa:fix/bound-tts-and-session-turns-caches
Open

fix(hindsight): bound the append-mode session turn buffer#62977
Vissirexa wants to merge 1 commit into
NousResearch:mainfrom
Vissirexa:fix/bound-tts-and-session-turns-caches

Conversation

@Vissirexa

@Vissirexa Vissirexa commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

Part of #62950. Rebased onto current main and narrowed to the hindsight half only — the TTS model-cache half of this PR landed via #73512 (commit 15673dfce7, authorship preserved), so it has dropped out of the diff. The contributor email mapping commit also dropped: contributors/emails/tikkanadityajyothi@gmail.com now exists upstream, and contributors/README.md explicitly forbids new AUTHOR_MAP entries in scripts/release.py.

What's left is one commit in one subsystem.

What

_session_turns (plugins/memory/hindsight/__init__.py) accumulated every turn's text for the whole session, reset only at session boundaries — so a never-ending session grew without bound, independent of context compaction.

The fix is scoped to append mode only:

  • In append mode each retain ships just the delta since the last watermark (_session_turns[_last_retained_turn_count:]), and once retained a turn is never read again — sync_turn always slices from the watermark, and flush-on-switch flushes what's left. So after an append retain the buffer now drops the retained prefix (clear() + reset the watermark), bounding it to the un-retained tail.
  • Overwrite mode is deliberately untouched. Legacy/overwrite APIs resend the whole session each retain (list(self._session_turns)) because each retain replaces the document, so that path must keep every turn.

This is distinct from #40605 (which fixed the quadratic re-send on append); the in-RAM list itself is what this bounds.

How to test

scripts/run_tests.sh tests/plugins/memory/test_hindsight_provider.py

TestSessionTurnsBufferBounding covers: append trims the retained prefix while shipping every turn exactly once (no loss, no duplication), the buffer stays bounded across 100+ turns, and overwrite mode keeps the full buffer.

Full tests/plugins/memory/ passes on the rebased branch — 18 files, 555 tests, 0 failures under the canonical scripts/run_tests.sh.

Platforms tested: macOS (Apple Silicon), Python 3.11.

@alt-glitch alt-glitch added type/bug Something isn't working comp/plugins Plugin system and bundled plugins tool/tts Text-to-speech and transcription tool/memory Memory tool and memory providers P3 Low — cosmetic, nice to have labels Jul 12, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused memory-bound fixes. Current checkout HEAD still retains all Piper/KittenTTS cache entries (tools/tts_tool.py:2020-2025, tools/tts_tool.py:2117-2123) and keeps append-retained Hindsight turns in _session_turns after advancing only a watermark (plugins/memory/hindsight/__init__.py:1637-1696). Commit 8be65e07d0fa addresses those exact paths with bounded LRU caches and append-only buffer trimming while preserving the overwrite path.

No verified correctness issue found in this review. Existing integration tests already exercise both provider call paths, and the added tests cover LRU order plus append-mode no-loss/no-duplication semantics.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 12, 2026
@teknium1 teknium1 added the area/memory Memory subsystem: store, providers, sync, background reviews label Jul 19, 2026
teknium1 pushed a commit that referenced this pull request Jul 28, 2026
Salvaged from PR #62977 (@Vissirexa) — TTS model-cache half only (the
hindsight turn-buffer half is a different subsystem and was dropped).

_piper_voice_cache and _kittentts_model_cache were keyed by voice/model
with no eviction, and each entry is a whole loaded model (tens of MB).
A surface that sweeps voices pinned one model per voice for the process
lifetime. New _tts_cache_get_or_load() get-or-loads through a small LRU
(_TTS_MODEL_CACHE_MAX=3), refreshing recency on a hit and evicting the
least-recently-used model on a cold miss.
@teknium1

Copy link
Copy Markdown
Contributor

The TTS model-cache half of this PR landed on main via #73512 (merge f3cc2bc) as commit 15673df with your authorship preserved — _piper_voice_cache/_kittentts_model_cache now run through a small LRU. The hindsight turn-buffer half is a different subsystem and remains for the memory-subsystem triage, so this PR stays open for that half.

@Vissirexa
Vissirexa force-pushed the fix/bound-tts-and-session-turns-caches branch from 7f9b0f3 to 874a4c0 Compare July 29, 2026 01:42
@Vissirexa Vissirexa changed the title fix: bound the TTS model caches and hindsight append-mode turn buffer fix(hindsight): bound the append-mode session turn buffer Jul 29, 2026
@Vissirexa

Copy link
Copy Markdown
Contributor Author

Thanks — rebased onto current main and narrowed this PR to the hindsight half, so the remaining diff is scoped to the memory subsystem for triage.

What's left is a single commit touching plugins/memory/hindsight/__init__.py (+9/-1) and its tests. scripts/run_tests.sh tests/plugins/memory/ — 18 files, 555 tests, 0 failures. Title and description updated to match.

_session_turns accumulated every turn's text for the whole session, reset
only at session boundaries, so a never-ending session grew without bound
independent of context compaction.

In append mode each retain ships only the delta since the last watermark
(_session_turns[_last_retained_turn_count:]), and a retained turn is never
read again — sync_turn always slices from the watermark and flush-on-switch
flushes what's left. So after an append retain the buffer drops the retained
prefix (clear() + reset the watermark), bounding it to the un-retained tail.

Overwrite mode is deliberately untouched: legacy/overwrite APIs resend the
whole session each retain because each retain replaces the document, so that
path must keep every turn.

Tests: append trims the retained prefix while shipping every turn exactly
once (no loss, no duplication), stays bounded across 100+ turns, and
overwrite mode keeps the full buffer.
@Vissirexa
Vissirexa force-pushed the fix/bound-tts-and-session-turns-caches branch from 874a4c0 to cb77e00 Compare August 1, 2026 00:41
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
Salvaged from PR NousResearch#62977 (@Vissirexa) — TTS model-cache half only (the
hindsight turn-buffer half is a different subsystem and was dropped).

_piper_voice_cache and _kittentts_model_cache were keyed by voice/model
with no eviction, and each entry is a whole loaded model (tens of MB).
A surface that sweeps voices pinned one model per voice for the process
lifetime. New _tts_cache_get_or_load() get-or-loads through a small LRU
(_TTS_MODEL_CACHE_MAX=3), refreshing recency on a hit and evicting the
least-recently-used model on a cold miss.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/memory Memory subsystem: store, providers, sync, background reviews comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state tool/memory Memory tool and memory providers tool/tts Text-to-speech and transcription type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants