Skip to content

fix(hermes-state): quote colon-delimited fts5 literals - #19952

Closed
Que0x wants to merge 1 commit into
NousResearch:mainfrom
Que0x:fix/hermes-state-fts5-colon-quoting
Closed

fix(hermes-state): quote colon-delimited fts5 literals#19952
Que0x wants to merge 1 commit into
NousResearch:mainfrom
Que0x:fix/hermes-state-fts5-colon-quoting

Conversation

@Que0x

@Que0x Que0x commented May 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes an FTS5 search bug in SessionDB.search_messages() where colon-delimited literals and Windows-style paths were misparsed as FTS5 column filters instead of searched as plain text.

Queries like path:config.yaml and C:\Users\demo\project could trigger no such column inside SQLite FTS5. search_messages() caught that OperationalError and returned [], so valid searches silently looked like misses.

Problem

Hermes recently hardened FTS5 query sanitization for quotes, hyphens, dots, underscores, and boolean operators, but : remained untreated.

That left a real edge case:

  • path:config.yaml was sanitized into path:"config.yaml", which FTS5 interpreted as a column filter on a non-existent path column.
  • C:\Users\demo was interpreted as a column filter on C.
  • The resulting sqlite3.OperationalError was swallowed by search_messages(), so users saw empty results instead of matches.

This is especially user-visible on Windows and in any workflow that searches config-style tokens, paths, or log fragments containing :.

Fix

Updated _sanitize_fts5_query() in hermes_state.py to quote literal-like tokens containing punctuation that FTS5 would otherwise tokenize or interpret as syntax, including:

  • .
  • _
  • -
  • :
  • /
  • \

The implementation uses a single token-based pass so we preserve existing behavior for:

  • quoted phrases
  • boolean operators
  • hyphenated/dotted/underscored terms

and avoid double-quoting mixed forms like my-app.config.ts.

Tests

Added regression coverage in tests/test_hermes_state.py for:

  • literal search of path:config.yaml
  • literal search of C:\Users\demo\project
  • sanitizer output for colon-delimited tokens and Windows paths

Validation

Ran:

scripts/run_tests.sh tests/test_hermes_state.py -k 'colon_and_windows_path'
scripts/run_tests.sh tests/test_hermes_state.py -k 'FTS5Search'


Results:

targeted regression tests: passed
broader FTS5 search suite: passed

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 5, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #18692 (FTS5 operator stripping) and earlier FTS5 sanitization fixes (#1776, #1777). Extends the same _sanitize_fts5_query() code path.

1 similar comment
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #18692 (FTS5 operator stripping) and earlier FTS5 sanitization fixes (#1776, #1777). Extends the same _sanitize_fts5_query() code path.

@Que0x

Que0x commented May 5, 2026

Copy link
Copy Markdown
Contributor Author

Closing already fixed.

@Que0x Que0x closed this May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants