Skip to content

fix(search): quote underscored terms in FTS5 query sanitization - #16765

Closed
crayfish-ai wants to merge 4 commits into
NousResearch:mainfrom
crayfish-ai:fix-fts5-underscore-search-v2
Closed

fix(search): quote underscored terms in FTS5 query sanitization#16765
crayfish-ai wants to merge 4 commits into
NousResearch:mainfrom
crayfish-ai:fix-fts5-underscore-search-v2

Conversation

@crayfish-ai

Copy link
Copy Markdown
Contributor

Summary

FTS5 default tokenizer splits 'sp_new1' into tokens 'sp' and 'new1'. Without quoting, a search for 'sp_new' becomes an AND query ('sp AND new') that fails to match rows indexed as 'sp_new1'.

Fix

Add underscore to the character class in Step 5 regex ([.-] -> [._-]) so underscored terms are wrapped in double quotes and treated as exact phrases.

Test

  • 27/27 sanitize tests pass (26 existing + 1 new underscore test)
  • 184/184 full test suite pass
  • Functional verification: sp_new search now returns results (previously returned 0)

Files changed

  • hermes_state.py:1285 — regex [.-] -> [._-]
  • tests/test_hermes_state.py — new test_sanitize_fts5_quotes_underscored_terms

Scope note

This PR contains only the 2 files necessary for the fix. Previous PR #16755 had a large diff due to branch contamination.

Hermes Agent added 4 commits April 23, 2026 17:30
- holographic/__init__.py: 并行FTS+HRR双路检索,2s timeout,score fusion取top5
- holographic/retrieval.py: FTS查询改用OR+前缀匹配,支持中文单字分词和英文前缀
- holographic/store.py: 新增remove_fact_by_content()按内容删除fact
- run_agent.py: memory bridge增加remove action,传递deleted_entry给外置provider
- tools/memory_tool.py: remove返回被删entry内容
- agent/memory_manager.py: 文档澄清BuiltinMemoryProvider不走插件系统
- agent/auxiliary_client.py: MiniMax custom endpoint的base_url补充_to_openai_base_url转换
- whatsapp-bridge/package-lock.json: 依赖锁文件更新
- docs/memory-providers.md: honcho_sync工具文档更新
…rite

- resolve_provider_client: apply _to_openai_base_url() to custom base_url
  (fixes /anthropic → /v1 rewrite that was missing for provider="custom")
- resolve_provider_client: use main_runtime["model"] instead of
  _read_main_model() so auxiliary tasks follow system default changes
- title_generator: thread main_runtime through generate_title →
  auto_title_session → maybe_auto_title (both CLI and gateway call sites)
- flush_memories (run_agent.py): pass main_runtime to call_llm so it
  uses the same model as the main conversation
- update test assertion to match new auto_title_session signature
The /restart command triggered a drain that waited for ALL agents including
the caller agent itself, which was waiting for the HTTP response — creating
a deadlock that always timed out after 60 seconds.

Changes:
- _drain_active_agents(): add exclude_key param to filter out the caller
- _interrupt_running_agents(): add exclude_key param to skip the caller
- _stop_impl(): pass _restart_caller_key to both functions
- _handle_restart_command(): record caller's session_key before draining
- GatewayRunner.__init__: initialize _restart_caller_key

Test update:
- tests/gateway/restart_test_helpers.py: add _restart_caller_key to runner setup
- tests/gateway/test_clean_shutdown_marker.py: add _restart_caller_key to runner
FTS5 default tokenizer splits 'sp_new1' into tokens 'sp' and 'new1'.
Without quoting, a search for 'sp_new' becomes an AND query
('sp AND new') that fails to match rows indexed as 'sp_new1'.

Fix: add underscore to the character class in Step 5 regex
([.-] -> [._-]) so underscored terms are wrapped in double quotes.

Also adds test_sanitize_fts5_quotes_underscored_terms.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/memory Memory tool and memory providers labels Apr 28, 2026
@crayfish-ai

Copy link
Copy Markdown
Contributor Author

Recreated as fix-fts5-v3 on current upstream/main

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants