feat(skill): memory-enhanced-retrieval — entity-indexed two-stage retrieval - #49542
feat(skill): memory-enhanced-retrieval — entity-indexed two-stage retrieval#49542luxuguang-leo wants to merge 1 commit into
Conversation
Entity-indexed two-stage retrieval for Hermes. Uses jieba + SQLite as a sidecar to improve multi-hop and cross-session recall. Based on A/B testing against MRAgent (ICML 2026). Results: 100% correct+partial (vs MRAgent 30%), 12s avg (vs 33s). Zero Hermes core changes. Full report and data at https://github.com/luxuguang-leo/hermes-memory-ab-test
alt-glitch
added
type/feature
teknium1
left a comment
teknium1
left a comment
There was a problem hiding this comment.
Thanks for contributing a skill-only approach. The current patch does not yet implement the retrieval behavior it describes.
Problems
optional-skills/research/hermes-memory-ab-test/scripts/build_index.py:32-48only extracts and writes entities. It contains no index query, second retrieval pass, merge, or invocation ofsession_search; current main discovery instead callsdb.search_messages()attools/session_search_tool.py:513.build_index()clearsentitiesatscripts/build_index.py:35, while each--addcall passes a one-item list at line 48. Repeated indexing therefore removes all prior turns.- The default path at
scripts/build_index.py:44hardcodes~/.hermes, conflicting with the profile-safe state-path rule inAGENTS.md:1169-1178.
Suggested changes
- Implement and document a complete standalone two-stage retrieval command, or re-scope the skill so it does not claim to improve
session_search. - Make indexing incremental and profile-aware, then add tests for multi-turn retention and retrieval.
- Add the standard runnable procedure, pitfalls, verification, and script tests required for an optional skill.
Automated hermes-sweeper review.
|
|
||
| # Memory Enhanced Retrieval | ||
|
|
||
| Improves `session_search` for multi-hop and cross-session queries using entity |
There was a problem hiding this comment.
The patch does not currently improve session_search: the only script writes an index and has no query, two-stage retrieval, result merge, or invocation of the session-search path. Please either provide a complete standalone retrieval workflow here or re-scope this claim.
| def build_index(conversation, db_path): | ||
| conn = sqlite3.connect(db_path) | ||
| conn.execute("CREATE TABLE IF NOT EXISTS entities (entity TEXT, turn_idx INT, content TEXT, tag TEXT)") | ||
| conn.execute("DELETE FROM entities") |
There was a problem hiding this comment.
This deletes all indexed rows on every call. Because the CLI passes one --add value as a one-turn conversation at line 48, each subsequent indexing operation drops all prior conversation history. Use an incremental/idempotent update keyed by source message instead.
|
|
||
| if __name__ == "__main__": | ||
| p = argparse.ArgumentParser() | ||
| p.add_argument("--db", default=str(Path.home()/".hermes/entity_index/index.db")) |
There was a problem hiding this comment.
This hardcodes the default Hermes home and bypasses named-profile isolation. Resolve this from the profile-aware Hermes home mechanism (or at least HERMES_HOME with the normal fallback) and ensure the parent directory exists before opening SQLite.
teknium1
added
sweeper:risk-compatibility
3 participants
Summary
Adds
memory-enhanced-retrievalas an optional skill underresearch/. This skill implements entity-indexed two-stage retrieval to improve multi-hop and cross-session recall in Hermes' session search.Background
Based on A/B testing against MRAgent (ICML 2026, "Memory is Reconstructed, Not Retrieved") on 10 real-world conversation scenarios:
Design
hermes skills removefor full rollbackFull report: https://github.com/luxuguang-leo/hermes-memory-ab-test