fix(hindsight): bound the append-mode session turn buffer - #62977
fix(hindsight): bound the append-mode session turn buffer#62977Vissirexa wants to merge 1 commit into
Conversation
|
Thanks for the focused memory-bound fixes. Current checkout HEAD still retains all Piper/KittenTTS cache entries ( No verified correctness issue found in this review. Existing integration tests already exercise both provider call paths, and the added tests cover LRU order plus append-mode no-loss/no-duplication semantics. Automated hermes-sweeper review. |
Salvaged from PR #62977 (@Vissirexa) — TTS model-cache half only (the hindsight turn-buffer half is a different subsystem and was dropped). _piper_voice_cache and _kittentts_model_cache were keyed by voice/model with no eviction, and each entry is a whole loaded model (tens of MB). A surface that sweeps voices pinned one model per voice for the process lifetime. New _tts_cache_get_or_load() get-or-loads through a small LRU (_TTS_MODEL_CACHE_MAX=3), refreshing recency on a hit and evicting the least-recently-used model on a cold miss.
|
The TTS model-cache half of this PR landed on main via #73512 (merge f3cc2bc) as commit 15673df with your authorship preserved — |
7f9b0f3 to
874a4c0
Compare
|
Thanks — rebased onto current
What's left is a single commit touching |
_session_turns accumulated every turn's text for the whole session, reset only at session boundaries, so a never-ending session grew without bound independent of context compaction. In append mode each retain ships only the delta since the last watermark (_session_turns[_last_retained_turn_count:]), and a retained turn is never read again — sync_turn always slices from the watermark and flush-on-switch flushes what's left. So after an append retain the buffer drops the retained prefix (clear() + reset the watermark), bounding it to the un-retained tail. Overwrite mode is deliberately untouched: legacy/overwrite APIs resend the whole session each retain because each retain replaces the document, so that path must keep every turn. Tests: append trims the retained prefix while shipping every turn exactly once (no loss, no duplication), stays bounded across 100+ turns, and overwrite mode keeps the full buffer.
874a4c0 to
cb77e00
Compare
Salvaged from PR NousResearch#62977 (@Vissirexa) — TTS model-cache half only (the hindsight turn-buffer half is a different subsystem and was dropped). _piper_voice_cache and _kittentts_model_cache were keyed by voice/model with no eviction, and each entry is a whole loaded model (tens of MB). A surface that sweeps voices pinned one model per voice for the process lifetime. New _tts_cache_get_or_load() get-or-loads through a small LRU (_TTS_MODEL_CACHE_MAX=3), refreshing recency on a hit and evicting the least-recently-used model on a cold miss.
Part of #62950. Rebased onto current
mainand narrowed to the hindsight half only — the TTS model-cache half of this PR landed via #73512 (commit15673dfce7, authorship preserved), so it has dropped out of the diff. The contributor email mapping commit also dropped:contributors/emails/tikkanadityajyothi@gmail.comnow exists upstream, andcontributors/README.mdexplicitly forbids newAUTHOR_MAPentries inscripts/release.py.What's left is one commit in one subsystem.
What
_session_turns(plugins/memory/hindsight/__init__.py) accumulated every turn's text for the whole session, reset only at session boundaries — so a never-ending session grew without bound, independent of context compaction.The fix is scoped to append mode only:
_session_turns[_last_retained_turn_count:]), and once retained a turn is never read again —sync_turnalways slices from the watermark, and flush-on-switch flushes what's left. So after an append retain the buffer now drops the retained prefix (clear()+ reset the watermark), bounding it to the un-retained tail.list(self._session_turns)) because each retain replaces the document, so that path must keep every turn.This is distinct from #40605 (which fixed the quadratic re-send on append); the in-RAM list itself is what this bounds.
How to test
TestSessionTurnsBufferBoundingcovers: append trims the retained prefix while shipping every turn exactly once (no loss, no duplication), the buffer stays bounded across 100+ turns, and overwrite mode keeps the full buffer.Full
tests/plugins/memory/passes on the rebased branch — 18 files, 555 tests, 0 failures under the canonicalscripts/run_tests.sh.Platforms tested: macOS (Apple Silicon), Python 3.11.