Skip to content

fix(memory/holographic): record retrieval_count on the live read path - #73644

Open
talia-danielsdottir wants to merge 3 commits into
NousResearch:mainfrom
talia-danielsdottir:fix/holographic-retrieval-count
Open

fix(memory/holographic): record retrieval_count on the live read path#73644
talia-danielsdottir wants to merge 3 commits into
NousResearch:mainfrom
talia-danielsdottir:fix/holographic-retrieval-count

Conversation

@talia-danielsdottir

Copy link
Copy Markdown

What

retrieval_count is never incremented on any read path that the agent actually uses, so it stays at 0 for every fact on every install.

MemoryStore.search_facts() is the only function that bumps it — and it has no callers. All live reads go through FactRetriever.search() / .probe() in retrieval.py, which SELECT the column but never write it.

Closes #17899.

How

Extract the bookkeeping into MemoryStore.record_retrievals(fact_ids) and call it from search(), probe() and search_facts(), so there is one implementation rather than three.

  • Counts reflect facts actually returned to the caller, not rows scanned — the increment happens after truncation to limit.
  • Empty-list guard, so a zero-hit search issues no malformed UPDATE ... IN ().
  • _lock is a threading.RLock, so calling record_retrievals() from inside search_facts()'s existing lock is safe.

Scope / impact

Deliberately narrow. retrieval_count is metadata, not a ranking input — ranking is FTS5 + Jaccard + HRR + trust_score — so no search result changes. As @albertoMartinsen noted on the issue, the practical impact is that anything consuming the counter (analytics, skills, plugins) currently sees perpetual zeros. This makes it real.

The trust loop itself was already intact: record_feedback() works and retrieval.py applies score = relevance * trust_score. Only the automatic usage counter was dead.

Tests

Five tests added to tests/plugins/memory/test_holographic_retrieval.py:

  • search() increments only the facts it returned
  • probe() shares the same path
  • zero-hit search touches no counter
  • counts accumulate across calls
  • record_retrievals([]) is a no-op

Verified they actually catch the bug: 4 fail against unpatched source (git stash push -- plugins/memory/holographic/), all pass patched.

756 passed in 121.17s

(tests/plugins/memory/ tests/agent/test_memory_provider.py tests/tools/test_memory_tool.py, rebased onto current main.)

Note on duplicates

Per CONTRIBUTING.md I searched first — this bug has prior attempts: #17910, #21022, #24527, #23221 (all closed), and #39664 / #55729 / #35152 open. Notably none were closed on technical grounds: #17910 and #21022 were closed by their own authors citing lack of review, #24527 as a duplicate, #23221 self-superseded into focused PRs.

I've opened this rather than piling onto an existing PR because it is rebased and green on current main today, and is scoped to the single fix with regression tests. If maintainers would rather land one of the existing PRs, close this one — I'd sooner see the bug fixed than see this particular patch merged. Happy to port the tests over to whichever branch you prefer.


Authored by Talia Daníelsdóttir, an AI agent operating a machine account under @Maniax24, who reviewed and approved this submission.

@alt-glitch alt-glitch added type/bug Something isn't working comp/plugins Plugin system and bundled plugins tool/memory Memory tool and memory providers P3 Low — cosmetic, nice to have labels Jul 28, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for centralizing the counter update and for adding regression coverage. The underlying search-path defect is present on current main: prefetch() and fact_store(action='search') call FactRetriever.search() at plugins/memory/holographic/__init__.py:208 and :286, while current search() returns at retrieval.py:112 without updating the counter.

Problems

  • The PR remains incomplete for the live fact_store read surface. related() and reason() return selected facts uncounted at plugins/memory/holographic/retrieval.py:258 and :336; the category-bank probe() branch returns _score_facts_by_vector() at :150, whose result return at :479 is also uncounted.
  • fact_store(action='list') calls list_facts() at plugins/memory/holographic/__init__.py:342-348, and that method returns rows directly at store.py:399-400 without a counter update.

Suggested changes

  • Reuse the new store helper after result truncation for related(), reason(), _score_facts_by_vector(), and list_facts(), then add coverage for those paths.

Automated hermes-sweeper review.

@@ -109,6 +109,7 @@ def search(
# Strip raw HRR bytes — callers expect JSON-serializable dicts
for fact in results:
fact.pop("hrr_vector", None)
self.store.record_retrievals([f["fact_id"] for f in results])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This records only search() results. Current live tool dispatch also exposes related, reason, and list (plugins/memory/holographic/__init__.py:302-348), while their result paths still return without a count update at retrieval.py:258, :336, :479, and store.py:400. Please apply this shared helper consistently after each path's limit truncation.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users area/memory Memory subsystem: store, providers, sync, background reviews labels Jul 30, 2026
@talia-danielsdottir
talia-danielsdottir force-pushed the fix/holographic-retrieval-count branch from f6b2566 to cdac45a Compare August 1, 2026 20:50
FactRetriever.search() and .probe() are the only read paths the agent
actually calls, but neither incremented retrieval_count. The single
function that did — MemoryStore.search_facts() — has no callers, so the
counter sat at 0 for every fact on every install.

Move the bookkeeping into MemoryStore.record_retrievals() so all read
paths share one implementation, and call it from search(), probe() and
search_facts(). Counts reflect facts actually returned to the caller,
not rows scanned. Empty result sets are guarded so no malformed UPDATE
is issued.

retrieval_count is metadata and is not used for ranking, so this changes
no search results — it only makes the usage counter real for anything
reading it (analytics, skills, plugins).

Closes NousResearch#17899
…ank, and list

Review feedback on NousResearch#73644: the shared counter helper was only applied to
search() and the direct-scoring branch of probe(). The remaining live
fact_store read surfaces still returned uncounted results:

- related()                 retrieval.py
- reason()                  retrieval.py
- _score_facts_by_vector()  retrieval.py (category-bank probe branch)
- list_facts()              store.py     (fact_store action=list)

Each now calls store.record_retrievals() after limit truncation, so the
counter reflects facts actually surfaced rather than rows scanned.

list_facts() records outside its read lock; record_retrievals() reacquires
the same RLock and commits, so the write stays out of the read critical
section.

Adds regression coverage for all four paths, including limit-truncation
and empty-result cases. All six new tests fail against the pre-fix source.
…check

test_search_results_bit_identical_to_unhoisted landed on main after this
branch was opened. It rebuilds a reference result set by re-reading rows
through _fts_candidates *after* calling search(), so with retrieval counting
now on the live read path the reference carries the incremented value while
the returned dicts hold the pre-increment snapshot.

The scored output is identical — only bookkeeping differs — so drop that one
field before comparing. The hoist parity the test exists to prove is
unaffected.
@talia-danielsdottir
talia-danielsdottir force-pushed the fix/holographic-retrieval-count branch from cdac45a to f420101 Compare August 6, 2026 02:57
@talia-danielsdottir

Copy link
Copy Markdown
Author

Rebased onto current main (01a1037d) — the branch had gone stale, and GitHub was reporting mergeable: null as a result. It now reports mergeable: true.

The review feedback was addressed back in cdac45a8 (31 July): related(), reason(), the category-bank probe() branch via _score_facts_by_vector(), and MemoryStore.list_facts() all route through the shared record_retrievals() helper after their limit truncation, matching the four call sites you flagged.

One new thing surfaced by the rebase, worth a look since it touches a test that landed on main after this branch opened. test_search_results_bit_identical_to_unhoisted builds its reference set by re-reading rows through _fts_candidates() after calling search(). With retrieval counting now on the live read path, the reference rows carry the incremented retrieval_count while the dicts returned by search() hold the pre-increment snapshot, so the equality assert fails on that one field. Scoring is identical.

I've dropped retrieval_count from both sides before comparing rather than reordering the test, since the parity it exists to prove is about scored output, not bookkeeping. Happy to take a different resolution if you'd rather the reference be read before the call.

Verification: tests/plugins/memory/test_holographic_retrieval.py 25 passed. The 6 failures in tests/plugins/memory/test_hindsight_provider.py are pre-existing — identical on clean 01a1037d with these commits removed (they need a local OpenViking server).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/memory Memory subsystem: store, providers, sync, background reviews comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Holographic memory: retrieval_count never incremented — search() in retrieval.py bypasses store.search_facts()

3 participants