fix(holographic): keep hyphenated identifiers searchable (split on -/_)

[dsitmilis]

fix(holographic): hyphenated-identifier search returned 0 results

_Sanitize FTS query glued hyphenated tokens (vibe-trading -> vibetrading)
but FTS5's default tokenizer splits stored text on the same separators, so
the glued token never existed in the index and search returned 0 hits for
any hyphenated identifier (MCP server names, package names, domains).

Split each token on '-' and '_' before sanitizing so identifiers expand
into OR-joined sub-tokens (vibe-trading -> "vibe" OR "trading"). Phrase
quoting preserved; non-hyphenated queries unaffected.

Re-submission of closed PR #64209. Upstream main still uses plain query.lower().split() which glues vibe-trading into vibetrading; FTS5 tokenizes on -/_ so the glued form never matches. This splits tokens on -/_ into OR-joined sub-tokens.

[alt-glitch]

This was generated by AI during triage.

Duplicate of #14033 (earliest-open canonical for the holographic-memory FTS5 hyphenated-query cluster; same plugins/memory/holographic/retrieval.py sanitize mechanism). This is a re-submission of closed #64209, which was already part of the same cluster.

[teknium1]

Thanks for isolating the hyphen-loss path. The premise remains valid on current main: plugins/memory/holographic/retrieval.py:603-615 removes - before its FTS5 MATCH query at line 503.

Problems

• The added loops in plugins/memory/holographic/retrieval.py:611-612 turn vibe-trading into "vibe" OR "trading", which broadens an identifier lookup to either component. SessionDB instead quotes identifiers to preserve phrase semantics (hermes_state.py:4821-4827).
• _ is not removed by the current _FTS_SPECIAL set (retrieval.py:603), so splitting it newly broadens existing underscore queries without a demonstrated failure.
• The PR changes no tests. Existing coverage at tests/plugins/memory/test_holographic_retrieval.py:22-129 has prose-recall checks but no identifier retrieval regression.

Suggested changes

• Preserve an identifier as one quoted phrase, or demonstrate and test why OR-expanded matching is intended; avoid splitting underscores without a failing case.
• Add sanitizer and integration coverage for hyphenated identifiers through both FactRetriever.search() and MemoryStore.search_facts().

This is an automated hermes-sweeper review.

[teknium1]

Splitting here turns vibe-trading into "vibe" OR "trading" and also broadens underscore queries even though current _FTS_SPECIAL does not remove _. Please preserve a single quoted identifier phrase (as SessionDB does at hermes_state.py:4821-4827) or add a regression demonstrating why the broader OR semantics are required.

[GottZ]

This was generated by AI during triage.

Summary

Twenty PRs address or reference this issue complex: the session-search branch was implemented by #1776, while the unresolved holographic branch concerns preserving hyphenated identifiers through the shared FTS5 sanitizer rather than deleting or broadly OR-splitting their components; several other PRs are adjacent, superseded, contaminated, or unrelated changes.

Related pull requests

• fix: handle hyphenated FTS5 queries and preserve quoted literals #1773 [closed] related — (+55/-12) — n/a: The visible diff contains provider, gateway, authentication, and timeout changes but no session-search FTS5 fix, so this closed PR is relevant only as the contaminated predecessor to fix: handle hyphenated FTS5 queries and preserve quoted literals #1776.
• fix: handle hyphenated FTS5 queries and preserve quoted literals #1776 [merged] best fix — (+95/-10) — n/a: This merged reference implementation preserves balanced quotes, phrase-quotes hyphenated session queries, and adds unit and real-FTS5 regression coverage; it is also a recorded best fix for session_search mishandles hyphenated FTS5 queries and strips quoted literals #1770 and Session search broad FTS query can fail on hyphenated OR terms #7824.
• fix(search): handle hyphenated queries and preserve quoted literals in FTS5 #1777 [closed] fixes — (+92/-6) — n/a: This alternative preserves quoted phrases, converts inter-word hyphens to spaces, and logs FTS5 failures, but it was closed as superseded by the already-merged fix: handle hyphenated FTS5 queries and preserve quoted literals #1776.
• fix(holographic): sanitize FTS5 queries, fix entity wildcard matching, close DB on shutdown #6667 related — (+22/-5) — n/a: Its FTS and shutdown hunks are superseded on main, but the open diff still contains the distinct literal entity-name and alias matching fix that the keep_open review says remains salvageable with regression tests.
• fix: sanitize hyphenated session-search FTS queries #7820 [closed] best fix — (+26/-4) — n/a: This closed narrow session-search follow-up normalizes duplicated Boolean operators and tests the broad hyphenated OR query, remaining useful as regression evidence alongside merged fix: handle hyphenated FTS5 queries and preserve quoted literals #1776.
• fix(memory): sanitize holographic fact_store FTS queries #14033 best fix — (+68/-2) — n/a: This recorded best fix covers both holographic retriever and store paths with result-bearing hyphen tests, but the keep_open verdict requires adapting it to the current shared prose sanitizer, retaining stopword/OR behavior, handling colons, and preserving existing recall tests.
• fix(memory/holographic): sanitize hyphenated tokens in FTS5 queries #14262 partial — (+24/-2) — n/a: This competing fix quotes hyphenated tokens only in the older retriever path and bundles unrelated gateway and classifier changes, so its keep_open salvage path is to rebase, split those hunks, and cover the shared store path.
• fix(tools): auto-quote hyphenated tokens in fact_store FTS5 queries #14794 partial — (+176/-0) — n/a: This focused alternative provides strong real-SQLite ASCII-hyphen and Unicode-dash evidence, but its keep_open review requires integration into the current shared sanitizer plus result-bearing tests through both public search paths.
• feat(memory): add Episode hierarchy with graph propagation and pronoun resolution #16040 related — (+594/-9) — n/a: This open episode-hierarchy feature is not a fix for the reported FTS5 cause, and its keep_open review instead identifies session-switch, limit, ranking, episode-fact access, and current-connection integration work.
• fix(search): quote underscored terms in FTS5 query sanitization #16755 [closed] related — (+97/-13) — n/a: The underscore-quoting change is buried in a contaminated twelve-file diff, and this closed attempt was superseded by cleaner fix(search): quote underscored terms in FTS5 query sanitization #16827 and merged salvage fix(search): quote underscored terms in FTS5 query sanitization #16915.
• fix(search): quote underscored terms in FTS5 query sanitization #16765 [closed] related — (+245/-35) — n/a: Despite claiming a two-file scope, the visible diff remains broadly contaminated, so this closed retry is relevant only as another predecessor to fix(search): quote underscored terms in FTS5 query sanitization #16827 and fix(search): quote underscored terms in FTS5 query sanitization #16915.
• fix(search): quote underscored terms in FTS5 query sanitization #16827 [closed] related — (+26/-2) — n/a: This clean underscore-query patch and tests were closed after contributor-confirmed salvage into fix(search): quote underscored terms in FTS5 query sanitization #16915 with authorship preserved.
• fix(search): quote underscored terms in FTS5 query sanitization #16915 [merged] related — (+27/-2) — n/a: This merged reference implementation salvages fix(search): quote underscored terms in FTS5 query sanitization #16827 and adds underscore phrase-quoting coverage, addressing an adjacent tokenizer mismatch rather than the holographic hyphen-loss path.
• fix(memory): 4 holographic memory bug fixes (#14024 #17899 #17350 #22907) #23221 [closed] partial — (+474/-30) — n/a: This closed four-fix bundle includes holographic FTS sanitization but was explicitly superseded by focused PRs, with its FTS portion split into fix(memory): sanitize FTS5 queries to prevent crashes on hyphenated tokens (#14024) #39663.
• fix(holographic-memory): surface init failure instead of cryptic AttributeError #32545 related — (+149/-0) — n/a: This open PR addresses actionable errors after holographic initialization failure rather than FTS query handling, and its keep_open concern about configured database paths was addressed in commit 4bd794c.
• fix(memory): sanitize FTS5 queries to prevent crashes on hyphenated tokens (#14024) #39663 partial — (+174/-6) — n/a: This focused descendant of fix(memory): 4 holographic memory bug fixes (#14024 #17899 #17350 #22907) #23221 retrieves hyphenated facts and adds fuzz coverage, but its keep_open verdict requires extending rather than replacing the current shared sanitizer, removing the duplicate fallback, and testing both retriever and store paths.
• fix(holographic memory): sanitize FTS5 MATCH query to prevent silent empty results #51124 [closed] best fix — (+353/-4) — n/a: This closed recorded best fix sanitizes both holographic search paths and adds broad punctuation plus result-bearing store tests, remaining a strong reference even though contributor discussion classifies it as duplicating fix(memory): sanitize holographic fact_store FTS queries #14033.
• fix(holographic): hyphenated-identifier search returned 0 results #64209 [closed] duplicate — (+16/-10) — n/a: This closed predecessor changes the current sanitizer from glued identifiers to OR-expanded components and was superseded by the re-submitted fix(holographic): keep hyphenated identifiers searchable (split on -/_) #64256 within the fix(memory): sanitize holographic fact_store FTS queries #14033 duplicate cluster.
• fix(holographic): keep hyphenated identifiers searchable (split on -/_) #64256 partial — (+16/-10) — n/a: The diff fixes hyphen deletion by OR-splitting hyphen and underscore components, but the keep_open review finds that this broadens identifier semantics, changes underscores without a demonstrated failure, and adds no sanitizer or integration tests.
• fix(memory/holographic): sanitize FTS5 MATCH queries #75095 [closed] partial — (+22/-1) — n/a: This closed store-only sanitizer is already implemented on main by 638d2e7 at plugins/memory/holographic/store.py:254-257, as documented by the automated close verdict.

Duplicates

#1777 duplicates the session-search core of merged #1776, with #7820 a narrower follow-up; #14033, #14262, #14794, #23221/#39663, #51124, #64209/#64256, and #75095 overlap on holographic FTS5 sanitization, while #16755, #16765, and #16827 are superseded predecessors of merged #16915.

Suggested consolidation

Keep #14033 open as the canonical salvage path because it is a recorded best fix and covers both holographic search paths, but require it to extend the current shared sanitizer without losing prose recall, add colon handling, and retain both-path integration coverage; keep #6667, #16040, and #32545 separate because their surviving scopes are distinct. Despite the keep_open review on #64256, its visible diff OR-expands identifier components, unnecessarily splits underscores, and adds no tests, so close it as a duplicate of #14033 together with #14262, #14794, and #39663 once their useful test cases are carried over; leave closed #51124 as the recorded best-fix reference rather than reopening it, and treat #75095 as already implemented on main via 638d2e7.

Complex graph

flowchart TD
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I14024(["issue #14024 (open)"])
    I21102(["issue #21102 (closed)"])
    subgraph Dup14033 ["PRs duplicating each other"]
        P14033["PR #14033 (open)"]
        P14262["PR #14262 (open)"]
        P14794["PR #14794 (open)"]
        P23221["PR #23221 (closed)"]
        P39663["PR #39663 (open)"]
        P51124["PR #51124 (closed)"]
        P64209["PR #64209 (closed)"]
        P64256["PR #64256 (open)"]
        P75095["PR #75095 (closed)"]
    end
    P64256 -.->|partial| I14024
    P64256 -.->|partial| I21102
    class I14024 open
    class I21102 closed
    class P14033 open
    class P14262 open
    class P14794 open
    class P23221 closed
    class P39663 open
    class P51124 closed
    class P64209 closed
    class P64256 open
    class P75095 closed
    class P14033 best
    class P51124 best
    class P51124 best
    class P64256 target
    click I14024 "https://github.com/NousResearch/hermes-agent/issues/14024"
    click I21102 "https://github.com/NousResearch/hermes-agent/issues/21102"
    click P14033 "https://github.com/NousResearch/hermes-agent/pull/14033"
    click P14262 "https://github.com/NousResearch/hermes-agent/pull/14262"
    click P14794 "https://github.com/NousResearch/hermes-agent/pull/14794"
    click P23221 "https://github.com/NousResearch/hermes-agent/pull/23221"
    click P39663 "https://github.com/NousResearch/hermes-agent/pull/39663"
    click P51124 "https://github.com/NousResearch/hermes-agent/pull/51124"
    click P64209 "https://github.com/NousResearch/hermes-agent/pull/64209"
    click P64256 "https://github.com/NousResearch/hermes-agent/pull/64256"
    click P75095 "https://github.com/NousResearch/hermes-agent/pull/75095"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 20 pull requests and 4 issues in this complex. Each diff was read against this issue; Assessment working set: 190 kB of PR diffs, 43 kB of issue/PR text, 23 kB of discussion (40 comments), 31 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.