Skip to content

fix(hindsight): complete recall tool methods - #18268

Open
ThomasMarcelis wants to merge 4 commits into
NousResearch:mainfrom
ThomasMarcelis:jd/hindsight-recall-methods-clean
Open

fix(hindsight): complete recall tool methods#18268
ThomasMarcelis wants to merge 4 commits into
NousResearch:mainfrom
ThomasMarcelis:jd/hindsight-recall-methods-clean

Conversation

@ThomasMarcelis

@ThomasMarcelis ThomasMarcelis commented May 1, 2026

Copy link
Copy Markdown

Summary

  • Completes hindsight_recall support for method="recall", method="list", and method="entity".
  • Adds validated per-call controls for budget, max_tokens, types, tags, tags_match, tag_groups, metadata filtering, list pagination, and entity token budget.
  • Uses the current provider operation wrapper and the public Hindsight client.memory.list_memories(...) API.
  • Fixes entity formatting for the current Hindsight client model shape and narrows scoped list/entity output so unrelated or untagged results do not leak through filtered calls.
  • Falls back to default recall for unknown method values, matching existing tolerant control handling.
  • Updates the Hindsight plugin README and user-facing memory-provider docs.

Investigation

  • Current main uses _run_hindsight_operation(...) for Hindsight calls, so this keeps recall/list/entity behavior inside that provider operation wrapper.
  • Raw/list retrieval uses the public client.memory.list_memories(...) path instead of private client internals.
  • Current Hindsight entity responses expose RecallResponse.entities as a mapping of entity ids to entity state objects with entity_id, canonical_name, and observations; entity output now formats those fields directly.
  • Metadata, tag, and type filters are applied after retrieval where the public API cannot express the full filter, including exact metadata scoping and exclusion of untagged list items when tags are requested.
  • List item type detection accepts type, fact_type, and factType so current and older response shapes filter consistently.
  • Invalid per-call controls fall back to configured defaults or safe bounds instead of breaking recall.

Validation

  • python3 -m py_compile plugins/memory/hindsight/__init__.py tests/plugins/memory/test_hindsight_provider.py -> passed
  • HERMES_HOME=/tmp/hermes-pr18268-provider PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/plugins/memory/test_hindsight_provider.py -q -o addopts= -> 110 passed
  • HERMES_HOME=/tmp/hermes-pr18268-memory PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/plugins/memory -q -o addopts= -> 156 passed
  • python3 -m ruff check plugins/memory/hindsight/__init__.py tests/plugins/memory/test_hindsight_provider.py -> passed
  • git diff --check -> passed
  • HERMES_HOME=/tmp/hermes-pr18268-full PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/ -q -> blocked before collection because this local environment lacks pytest-xdist for the repo default -n auto.
  • HERMES_HOME=/tmp/hermes-pr18268-full PYTHONDONTWRITEBYTECODE=1 python3 -m pytest tests/ -q -o addopts= -> 29 collection errors, 4 skipped; errors are missing local optional/dev dependencies (acp, fire).
  • Baseline comparison: the same full-suite command on clean origin/main also reports 29 collection errors and 4 skipped for the same missing dependencies, so this PR does not introduce those local full-suite failures.

Notes / Caveats

  • Live PR head after this update: 8040bd900714855407972f66fb5bc1c2662af7f6.
  • The PR branch is mergeable but still diverged from current main (ahead_by=4, behind_by=34 at verification time).
  • This PR does not modify workflow files. If maintainers require the branch to be updated onto current main, an owner/maintainer token with workflow scope may be needed because upstream main has workflow-file changes.

@ThomasMarcelis
ThomasMarcelis force-pushed the jd/hindsight-recall-methods-clean branch from 5ee1b3e to 09176b2 Compare May 1, 2026 06:08
@ThomasMarcelis
ThomasMarcelis force-pushed the jd/hindsight-recall-methods-clean branch from 09176b2 to 691f873 Compare May 1, 2026 06:09
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/plugins Plugin system and bundled plugins tool/memory Memory tool and memory providers labels May 1, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for expanding the Hindsight recall surface. The premise remains valid on current main: hindsight_recall still exposes only query and directly calls arecall (plugins/memory/hindsight/__init__.py:314-326, :1729-1754).

Problems

  • The PR omits Hindsight 0.6.1's supported opinion fact type from _VALID_RECALL_TYPES (plugins/memory/hindsight/__init__.py:57 in PR head), so _normalize_recall_types drops a valid request at :373. Main pins that client version in pyproject.toml:159.
  • The list filter rejects every untagged item at proposed plugins/memory/hindsight/__init__.py:437, including tags_match="any" and "all". Hindsight 0.6.1 documents those modes as including untagged entries; only the strict modes exclude them. This makes list behavior differ from normal recall.
  • The entity default_types=["world"] at proposed :1724 is shadowed by current main's non-empty observation-only default (plugins/memory/hindsight/__init__.py:1344-1351), so the PR's base-era world-default test needs reconciliation during salvage.

Suggested changes

  • Preserve the client-supported opinion type and add a regression test.
  • Make list post-filtering preserve the documented strict/non-strict tag semantics, with coverage for all four modes.
  • Resolve entity type defaults against current main's recall-type policy before transplanting the tests.

Automated hermes-sweeper review.

_DEFAULT_IDLE_TIMEOUT = 300 # seconds — Hindsight embedded daemon default
_VALID_BUDGETS = {"low", "mid", "high"}
_VALID_RECALL_METHODS = {"recall", "list", "entity"}
_VALID_RECALL_TYPES = {"world", "experience", "observation"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hindsight-client==0.6.1 also supports the opinion fact type for arecall. Dropping it here means a valid per-call types: ["opinion"] silently falls back; please include it or intentionally document and test a Hermes-level restriction.

if not tags:
return True
item_tags = _item_tags(item)
if not item_tags:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes any and all reject untagged memories, but Hindsight defines only any_strict and all_strict as excluding untagged results. Preserve those non-strict semantics so list filtering matches normal recall.

recall_kwargs, metadata_filter, _, _ = self._build_tool_recall_kwargs(
args,
query,
default_types=["world"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On current main, _recall_types defaults to ["observation"], so _build_tool_recall_kwargs() selects that before this fallback. Reconcile this entity default and its test with the current observation-only policy during salvage.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users area/memory Memory subsystem: store, providers, sync, background reviews labels Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/memory Memory subsystem: store, providers, sync, background reviews comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/memory Memory tool and memory providers type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants