feat(tool-search): one search call finds many tools — multi-query, stemming, exact-name ranking, batched describe (salvage #92766) - #95119
Merged
Conversation
…mming tool_search now takes queries: string[] (searched independently against the same catalog, limit applies per query, default 5 / max 25) and returns the split shape: per-query groups carry tool names only, one shared tools map holds each matched tool's source, description (400-char cap) and required parameter names once. When some queries miss, a single top-level available_sources + hint block replaces the old per-response fallback. tool_describe now takes names: string[] and returns a map keyed by name; unknown names collect in not_found (with the refresh hint) and non-deferrable names keep their per-name spelling-check error in errors, so one bad name no longer fails the whole call. Duplicates dedupe silently. The shared tokenizer now applies Snowball stemming (english, exact-pinned snowballstemmer) at both index and query time, closing the measured plural/singular miss where 'issues' failed to return create_issue. The inline BM25 is unchanged. Stemmer instances are thread-local (they carry mutable parse state and bridge dispatch can run on parallel tool-call threads). New config knobs under tools.tool_search: max_queries / max_describe_names (default 10 each, floor 1, no upper clamp) bound the per-call array inputs; over-cap calls error so the model repairs in one round-trip. No backward compatibility with the single query/name shapes, by decision. scripts/analyze_livetest.py renders both shapes since transcripts on disk may predate this change.
…input The parallel determinism test warms _stem's lru_cache after ~11 distinct stems, so almost no iterations reach the underlying stemmer and a shared (non-thread-local) instance survives it. New test bypasses the cache with per-iteration unique tokens via _stem.__wrapped__, so thousands of stems run concurrently: a shared stemmer's mutable parse state fails it within 2,000 calls (verified — the mutant dies 8/8 runs; healthy runs stay green).
Two interaction seams between the #92693 salvage (merged as #95050) and this branch: the source-label indexing test now compares in token space (the stemmer shortens 'catalogsource' to 'catalogsourc'), and the unregistered-core-name describe test forces the unregistered condition via monkeypatch instead of depending on which sibling test file imported model_tools first.
Contributor
૮ >ﻌ< ა ci reviewran on 448bab2 — test(tool-search): make salvage-seam tests order- and stem-r
|
alt-glitch
added
type/feature
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Finding a deferred tool now takes one bridge call instead of several, and search stops missing tools over trivial wording differences.
tool_searchaccepts multiple queries in a single call; plural queries match singular tool names via Snowball stemming; a query that IS a tool's exact name ranks that tool first;tool_describebatches multiple names and reports misses innot_foundinstead of failing the whole call.Salvaged from #92766 by @alt-glitch (8 commits, authorship preserved), rebased onto current main on top of the merged #95050 with the one predicted docs conflict resolved as the author specified (single Retrieval bullet covering both source-name indexing and stemming).
Changes
tools/tool_search.py:tool_searchtakesqueries: string[](per-querylimit, grouped results, deduped sharedtoolsmap that now includes each tool'srequiredparams);tool_describetakesnames: string[]withnot_found/errorssplit; Snowball stemming in the shared_tokenize(thread-local stemmer instances, process-wide stem memo); exact-name fast path over BM25 length normalization; internal per-call caps (10 queries / 10 names), bare strings coerced.pyproject.toml/uv.lock:snowballstemmer==3.1.1— official Snowball project package, pure Python, zero transitive deps, exact-pinned.hermes_cli/config_defaults.py:max_search_limitdefault 20 → 25.tests/tools/test_tool_search_multiquery.py(26 tests);test_tool_search.pyandtest_a2a_schema_registration.pyupdated to the new shapes; follow-up commit makes two salvage-seam tests order- and stem-robust against the just-merged fix(tool-search): parallel batching, listing truncation, and service-name search work again under deferral (salvage #92693, part 1) #95050 (token-space comparison; monkeypatched unregistered-name condition).Validation
tool_search "issues"(live, 215 deferred tools)*_issuefamily absentget_issue,create_issue_label,list_issue_statuses, … returnedtool_describewith one bogus namenot_foundCache impact: bridge-tool schema/description wording changes — one-time prefix bust on the release that ships it, same as any tool-description edit; byte-stable across assemblies after that.
Infographic