Skip to content

fix(hindsight): guard prefetch results by query and flush buffered turns on shutdown - #77454

Closed
zayn-0101 wants to merge 1 commit into
NousResearch:mainfrom
zayn-0101:fix/hindsight-prefetch-stale-flush
Closed

zayn-0101 wants to merge 1 commit into
NousResearch:mainfrom
zayn-0101:fix/hindsight-prefetch-stale-flush

Conversation

@zayn-0101

Copy link
Copy Markdown

Summary

Two small consumer-side fixes in plugins/memory/hindsight/__init__.py:

  1. Prefetch result can leak across turns. prefetch() unconditionally
    consumes _prefetch_result, so a slow background recall queued at the end
    of turn N can surface in turn N+1's context as stale memory. This PR binds
    the cached result to the exact query that produced it (_prefetch_query +
    _prefetch_done under the existing lock) and truncates the query
    identically in both prefetch() and queue_prefetch() so the match holds.
    Results from an earlier turn are simply not consumed.

  2. Buffered turns can be lost at shutdown. When retain_every_n_turns > 1
    and a session ends below the retain cadence, the turns sitting in
    _session_turns are never handed to the retain writer. shutdown() now
    calls a new _flush_pending_turns() before setting the shutdown flag, so
    the final partial batch is retained. The append watermark is advanced for
    both update modes so legacy (overwrite) servers do not re-ship turns that
    were already submitted by sync_turn().

Related upstream work:

Verification

  • tests/plugins/memory/test_hindsight_provider.py: 57 passed
  • python -m py_compile plugins/memory/hindsight/__init__.py: passed
  • git diff --check: passed

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins tool/memory Memory tool and memory providers sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Aug 3, 2026
@teknium1

Copy link
Copy Markdown
Collaborator

Closing: the bundled Hindsight provider this PR patches has moved out of this repo.

Thanks @zayn-0101 for this contribution. In #119888 (merge 9d799e0531c; removal commit 4cbf862abe4) the in-tree plugins/memory/hindsight/ provider was removed — Hindsight now installs from the plugin catalog and its code lives in vectorize-io/hindsight hindsight-integrations/hermes (maintained by @nicoloboschi). There is no longer any code in this repo for the Hindsight half of this PR to patch, so we are closing every open PR against the bundled provider rather than leaving them stranded.

Triage notes:

If you believe this was closed in error, comment and we will reopen.

(Bulk-closed in the hindsight-move close pass.)

@teknium1 teknium1 closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants