fix(tool-search): parallel-execution barrier, listing truncation, source indexing, availability-cache staleness

[alt-glitch]

Summary

Five bugs in the tool-search deferral layer, one PR. The headline: supports_parallel_tool_calls silently stopped working the moment any MCP server was attached, because the batch planner treated the literal name tool_call as a sequential barrier. The rest: catalog listing lines truncated at "e.g.", service-name searches missing their own tools, a docstring describing impossible math, and a tool-availability cache that never noticed when a credential appeared.

No new tools, no schema changes, no cache-shape changes beyond a one-time prefix bust on release (same as any tool-description edit).

Background: how deferred tool dispatch works

With many MCP servers attached, sending every tool's JSON schema to the model on every turn gets expensive. So Hermes defers them: the model sees three bridge tools instead, and loads schemas on demand.

flowchart LR
    subgraph model_sees [what the model sees]
        TS[tool_search]
        TD[tool_describe]
        TC[tool_call]
    end
    subgraph hidden [deferred catalog]
        L1[mcp__linear__get_issue]
        L2[mcp__linear__create_issue]
        G1[mcp__granola__get_meetings]
        DOTS[... 200 more]
    end
    TS -- "BM25 search" --> hidden
    TD -- "load one schema" --> hidden
    TC -- "unwrap + dispatch" --> hidden

Loading

A deferred call arrives as tool_call(name="mcp__linear__get_issue", arguments={...}). The executor unwraps it and dispatches the real tool, so hooks, approvals, and guardrails all see the underlying name. That part worked. The problem was everything around dispatch that keyed on the literal string tool_call instead of unwrapping first.

Fix 1: the parallel-execution barrier (the big one)

When the model emits several tool calls in one turn, a planner splits them into segments: runs of read-only/safe calls execute concurrently on a thread pool, everything else runs in order. Admission is by tool name.

The planner never unwrapped the bridge. tool_call is not in any safe list, so it always fell through to "sequential barrier" — for every deferred tool, regardless of what was inside:

flowchart TD
    subgraph before [before — bridge active]
        A1["tool_call(linear get_issue)"] --> P1{planner sees name tool_call}
        A2["tool_call(granola folders)"] --> P1
        P1 -- "not in safe list" --> S1[sequential: one at a time]
    end
    subgraph after [after — planner peels the wrapper]
        B1["tool_call(linear get_issue)"] --> P2{peel to underlying tool}
        B2["tool_call(granola folders)"] --> P2
        P2 -- "server opted in to parallel" --> R1[parallel: both at once]
        P2 -- "not opted in / unparseable" --> S2[sequential, as before]
    end

Loading

The practical effect: setting supports_parallel_tool_calls: true on an MCP server did nothing once deferral was active — which is always, since any MCP server activates the bridge. The config option and the bridge were mutually exclusive and nothing said so.

The planner now peels the wrapper with the same parser the executor uses (resolve_underlying_call) and decides admission on the underlying tool and its real arguments:

• A bridged call gets exactly the admission the same call gets direct — parity, not extra permissiveness.
• A wrapper that fails to parse (bad JSON, missing name, core tool smuggled through, bridge recursion) stays a sequential barrier and fails at dispatch exactly as before.
• Path-conflict analysis sees the underlying arguments, so overlap detection works on what will actually run.
• tool_search and tool_describe are stateless catalog reads, so a batch of them runs concurrently too.

Reproduction

Live agent, two MCP servers with supports_parallel_tool_calls: true, one prompt asking for both calls in a single batch. Tool start time = completed timestamp minus logged duration:

	linear call	granola call	batch wall time
before	starts 09:11:24.08, runs 0.54s	starts 09:11:24.63 — after linear finished	1.31s (sum)
after	starts 09:12:37.10, runs 0.65s	starts 09:12:37.10 — same instant	1.18s (max)

Two tools make a small difference; a fan-out of eight I/O-bound calls pays the full serialization tax.

Fix 2: listing lines truncated at the first period

Tier-1 disclosure embeds a catalog listing: one line per tool, name: first sentence of description. The "first sentence" cut searched for the first ., !, or ? anywhere:

description	listing line before	after
Fetch a page from api.github.com and return the JSON body.	Fetch a page from api.	full sentence
Create an issue (e.g. a bug report) in a repository.	Create an issue (e.	full sentence
Upgrade to v1.2 of the schema and migrate all rows.	Upgrade to v1.	full sentence
List repos! Supports pagination.	List repos (terminator dropped)	List repos!

The listing is the model's only view of a deferred tool until it loads the schema, so garbled lines directly hurt tool selection. The boundary is now one regex: a terminator must be followed by whitespace or end-of-string, and e.g., i.e., etc. don't count. Linear-time on hostile input (MCP descriptions are third-party), and byte-identical output on the 211 real MCP descriptions in a live install except the one e.g. case this exists to fix.

Fix 3: searching for a service missed the service's tools

How the search works, in one paragraph

Each deferred tool becomes a small text document: its name split into words, its description, its parameter names. Queries score against these documents with BM25 — a term that appears in few documents is worth more than one that appears everywhere (IDF), a term matching a short document is worth more than the same match in a long one (length normalization). Top scores win.

What was wrong

Two things about how the documents were built:

1. The source wasn't in the document. A tool named create_issue living on the linear server had no "linear" anywhere in its indexed text — so the obvious query "linear" returned nothing. Native MCP tools dodged this only because their names happen to embed the server (mcp__linear__create_issue).
2. The mcp prefix was indexed. Every native MCP tool document carried the token mcp, worth nearly zero information (it's in every document, so IDF collapses) but still able to match every tool at once and drown out the term the model actually meant.

flowchart LR
    subgraph doc_before [indexed text — before]
        D1["mcp linear create issue + description + params"]
    end
    subgraph doc_after [indexed text — after]
        D2["linear create issue + description + params"]
    end
    Q1["query: linear"] -.->|"only matches if name contains it"| D1
    Q1 ==>|"always matches via source label"| D2

Loading

Now the group label (linear, not mcp-linear) is indexed with every tool from that source — exactly once, whether or not the tool's own name already carries it, so native and plugin tools get the same term weight — and the dead prefix is stripped before tokenizing.

What this deliberately does not do

No stemming, no synonyms, no field weights, no multi-query — those are a separate, larger change to the ranking itself. This fix only makes the document contain what it always should have.

Fix 4: a docstring describing impossible math

The search's substring fallback claimed to protect against "zero IDF" — a term appearing in every document scoring zero. With the IDF variant actually used, log(1 + (N - df + 0.5)/(df + 0.5)), that cannot happen: the value is strictly positive for every df ≤ N. The fallback's real (and useful) job is total token misses — "hub" never tokenizes out of github_create_issue, so nothing scores, and the substring pass catches it. The docstring and the user docs now describe the mechanism that exists.

No behavior change. This matters because the wrong rationale invites a "fix" for a case that can't occur.

Fix 5: the tool list couldn't notice a credential appearing

get_tool_definitions memoizes its result. The cache key captured registry mutations (a counter that bumps on register/deregister) and config-file edits (mtime fingerprint) — but not the availability probes. Tools gated on external state (is Docker running? is this credential set?) run a check_fn, and those verdicts are TTL-cached for 30 seconds precisely so they can flip when the world changes. Except a flip changed nothing in the memo key:

sequenceDiagram
    participant W as world
    participant M as memo (get_tool_definitions)
    participant P as availability probes
    Note over M: before
    W->>M: request tool list
    M->>P: probe (first call, cache miss)
    P-->>M: unavailable → tool omitted
    W->>W: credential lands / daemon starts
    W->>M: request tool list (any time later)
    M-->>W: cache HIT — stale list, probes never re-run
    Note over M: after
    W->>M: request tool list
    M->>P: TTL-cached verdict snapshot in the key
    P-->>M: verdict flipped → key changed → recompute
    M-->>W: fresh list within ~35s worst case

Loading

The key now includes an aggregate snapshot of every probe's verdict. The snapshot has its own short-TTL cache (keyed on registry scope, probe scope, and registry generation; cleared by invalidate_check_fn_cache()), so a hot-path hit is one dictionary lookup and a flaky probe coalesces to at most one live re-probe per window instead of one per rebuild. A flip propagates within the probe TTL plus the snapshot TTL (~35s worst case), or immediately on explicit invalidation. Probes marked no-cache (local config reads) are excluded — their inputs are covered by the config fingerprint.

The same staleness class existed at a second site: the executor's cache of which tools a session may reach through the bridge. It now keys on the verdict snapshot and the config fingerprint too. Both cache sites re-check the snapshot after computing and skip the store on mismatch, so a verdict flipping mid-compute can't park a fresh result under a stale key.

Also in this PR

enabled: auto and enabled: on do the same thing today. That is now stated at all three surfaces (dataclass, shipped config comment, docs) along with why auto remains the default: it reserves room for a future mode that inlines schemas when they fit and defers only when they don't, without breaking anyone who pinned on or off.

Validation

• tests/tools/test_deferral_fixes.py — 24 behavior-level tests at public seams (planner segment shapes, search results, listing lines, get_tool_definitions / scope-cache output), not internals. Each fix's tests verified red against the unfixed code; the invariant tests (unparseable wrappers stay barriers, non-opted-in servers stay sequential, direct/bridged parity, fallback on token misses, memo still hits when verdicts are stable) hold on both sides.
• Sibling suites green: test_tool_search.py, test_tool_batch_segmentation.py, test_builtin_memory_disabled_surface.py, test_get_tool_definitions_cache_isolation.py, test_terminal_tool_requirements.py, test_mcp_reload_refreshes_cached_agents.py, test_discord_tool.py, test_kanban_tools.py, test_plugins_hub_perf_guard.py, test_x_search_tool.py, test_delegate_kanban_isolation.py.
• Live red→green on a real agent with two OAuth MCP servers (table in Fix 1); search results and listing lines re-checked against 211 real MCP descriptions.

Cache impact

Fixes 1, 3, 4, 5 don't touch the request prefix at all (planner/dispatch/memo internals). Fix 2 changes listing bytes: one-time prefix bust on the release that ships it, identical to any listing edit; lines get slightly longer (real sentences instead of fragments), which can shift a borderline catalog's listing form — the byte-stability suite stayed green.

[github-actions]

૮ >ﻌ< ა ci review

_{ran on bf32ea0 — fix(tool-search): re-key the verdict-snapshot cache on regis}

⚠️ Warnings

OSV vulnerability scan · View job

7 known vulnerabilities found in pinned dependencies.

• CVE-2026-67213 in package-lock.json
• CVE-2026-67213 in website/package-lock.json
• CVE-2026-71554 in uv.lock
• CVE-2026-70608 in package-lock.json
• CVE-2026-56876 in package-lock.json
• CVE-2026-70606 in package-lock.json
• CVE-2026-71554 in uv.lock

How to fix:

Review the findings in the Security tab. Update the affected dependencies if a patched version is available.

---

debug info

CI timings

CI timings · View report · View job

Wall time 2m41s vs 2m49s (-4.7%). 6 job(s) slower, 5 faster, 2 unchanged.

• OS-specific tests / Windows-only tests: -41.0s
• Python tests / Run tests: -5.0s
• OS-specific tests / macOS-only tests: +5.0s
• OSV scan / Scan lockfiles / osv-scan: +3.0s
• Docs Site / docs-site-checks: -3.0s

[Enough1122]

AI code review — automated review for reference, author can ignore or act on any point.

Overall: a thoughtful quartet of fixes — deciding parallel admission on the underlying tool after peeling the tool_call wrapper (closing a real concurrency regression for bridge-enabled servers), putting external availability drift into memo keys via an aggregate verdict snapshot with explicit TOCTOU guards on both caches, indexing source labels so service-name queries recall tools whose names omit the vendor, and the mcp__ prefix strip (dead-weight IDF term) are all well-reasoned. The comments documenting why each guard exists are exemplary. Points:

1. tools/registry.py:520-543 — on a snapshot miss, every distinct probe runs serially through _check_fn_cached. For a process's first assembly with many network-backed MCP tools, that's a potentially multi-second stall on the hot path (each probe is TTL-cached afterwards, so it's once per horizon). Consider logging slow snapshot builds above debug, or pre-warming the snapshot right after registry load rather than inside the first get_tool_definitions call.

2. tools/tool_search.py:1060 — (?<!\be\.g)(?<!\bi\.e)(?<!\betc)[.!?](?=\s|$) is fixed-width lookbehind (valid), but note "e.g." etc. at end-of-string are now also treated as non-terminals, so a description ending "…uses OAuth, e.g." keeps the whole tail until the 60-char clip. Acceptable tradeoff; just confirming it's intended since the clip will then cut mid-word more often on abbreviation-heavy descriptions.

3. tools/tool_search.py:1078-1079 — switching to text[:m.end()] now retains !/? terminators where the old code kept only .; listings will subtly change. Tests appear updated, just flagging the user-visible diff.

4. agent/tool_dispatch_helpers.py:29-38 — the broad except Exception → treat as opaque wrapper is the right fail-closed direction (sequential barrier), but a malformed tool_call payload will now also silently lose parallelism with no signal; a logger.debug inside the except would help diagnose models that emit broken bridge args repeatedly.

The auto ≡ on aliasing contract documented in both config_defaults and should_activate is exactly how to reserve semantics without breaking pinned configs.

[teknium1]

Closing — all five fixes were salvaged onto current main with your authorship preserved:

• Fixes 1–4 (parallel batch barrier, listing truncation, source indexing, docstring) merged via fix(tool-search): parallel batching, listing truncation, and service-name search work again under deferral (salvage #92693, part 1) #95050 (commit 62b2d78).
• Fix 5 (availability-cache staleness) lives in fix(tools): availability-gated tools appear when their credential/daemon does (salvage #92693, part 2) #95053, split out at maintainer request so the caching trade-off can be reviewed on its own; it is green and pending a decision there.

Your multi-query/stemming PR #92766 also landed via #95119. Excellent work across the set — the red-verified tests and live E2E evidence made these the easy kind of salvage. Thanks!