feat(tool_search): config-gated builtin tool deferral (#6839)

[tgmerritt]

What does this PR do?

Implements the builtin-toolset half of #6839 (lazy tool schema loading) by extending the existing in-tree Tool Search progressive-disclosure layer — strictly opt-in, off by default.

Today tools/tool_search.py can defer MCP and non-core plugin tools behind the tool_search / tool_describe / tool_call bridge, but builtin tools are categorically excluded ("Core tools are never deferred. No exceptions."). That covers MCP-heavy installs — yet most of the hard measurements in #6839 are about installs with few or no MCP servers, where Hermes' own builtin schemas are the per-turn overhead:

• 42 builtin tools ≈ 61K chars of schema per call with zero MCP servers configured (#6839 comment)
• 56-tool hermes-cli toolset: a trivial no-tool turn on a local model takes 558s vs 31s with a 2-tool toolset — the schema preamble dominates time-to-first-token on self-hosted backends with no provider prompt caching
• ~6,500 of 10,600 system-prompt tokens (61%) being tool schemas on a stock install

This PR adds two config keys under the existing tools.tool_search block:

tools:
  tool_search:
    include_builtin: true          # default: false — opt-in
    # always_include: [terminal, read_file, web_search]   # optional override

• include_builtin (default false): builtin tools outside always_include become deferrable, joining the same catalog, threshold gate, BM25 retrieval, and bridge dispatch as MCP/plugin tools. With the flag off, classification behavior is byte-for-byte identical to today.
• always_include: names that never defer. Defaults to a lean hot set (terminal/process, file tools, web tools, execute_code, skill tools). A user-provided list replaces the default hot set but is always unioned with a hard floor — todo, memory, session_search, delegate_task, clarify — because those are serviced by the agent loop itself (model_tools._AGENT_LOOP_TOOLS) and deferring them would break the loop. The pin also works for MCP/plugin names (keep one hot MCP tool while the rest defer); pinning is exclusion-only and can never add tools outside the session's toolset scope.

Design decisions worth flagging for review:

1. Extension, not a parallel mechanism. Feature: Lazy Tool Schema Loading — Two-Pass Tool Injection to Reduce Token Overhead #6839's thread has two MCP-only implementation attempts (feat(mcp): add tool_search for on-demand MCP schema fetching #27257, closed; feat(mcp): add config-gated lazy MCP schema loading #33052, open) and a third-party BM25 pre-selection plugin whose top-k misses are reported to break webhook tasks. This PR deliberately builds on the in-tree Tool Search instead: the stateless-catalog design already solves the tool-dropout failure mode, and the scoping gate (scoped_deferrable_names) already prevents the bridge from widening a restricted session's tool surface. The diff is small because everything funnels through is_deferrable_tool_name().
2. The "core tools never defer" invariant becomes "never defer by default". The original invariant exists to prevent silent capability loss; the floor + always_include + explicit double-opt-in (include_builtin AND tool-search enabled/threshold) preserve that intent while unlocking the Feature: Lazy Tool Schema Loading — Two-Pass Tool Injection to Reduce Token Overhead #6839 use case. The existing regression tests for the invariant still pass unmodified.
3. always_include extends — never replaces — the agent-loop floor. A config typo can make the hot set smaller than intended, but it cannot produce an agent that can't todo/remember/delegate/clarify.
4. Threshold gate unchanged. In auto mode nothing activates until the (now larger) deferrable surface crosses threshold_pct of the model's context window, so small setups stay pass-through even with include_builtin: true.

Complementary to #33052 (MCP-side stub/promotion plugin) — different tool population, composable config surfaces. If maintainers prefer consolidating the #6839/#13332 design space first, happy to adjust scope; this was announced in #6839 before implementation.

Related Issue

Addresses the builtin-toolset portion of #6839

Type of Change

• ✨ New feature (non-breaking change that adds functionality)

Changes Made

• tools/tool_search.py — ToolSearchConfig gains include_builtin + always_include (with ALWAYS_INCLUDE_FLOOR / DEFAULT_ALWAYS_INCLUDE); is_deferrable_tool_name() / classify_tools() / scoped_deferrable_names() / dispatch_tool_describe() accept an optional config (default = user config, resolved once per assembly); _classify_source() reports a builtin source kind for catalog hits
• model_tools.py — assembly comment + activation message updated (no logic change)
• hermes_cli/config.py — DEFAULT_CONFIG documents the two new keys
• tests/tools/test_tool_search.py — 18 new tests (config parsing, floor enforcement, classification, assembly, describe gating, scoping), all with explicit configs so results never depend on the developer's ~/.hermes/config.yaml
• website/docs/user-guide/features/tool-search.md — new "Deferring builtin tools (opt-in)" section + config table rows

How to Test

1. scripts/run_tests.sh tests/tools/ tests/test_model_tools.py tests/hermes_cli/ — tests/tools/ and tests/test_model_tools.py fully green, including the pre-existing test_core_tools_never_defer invariant test unmodified. (5 tests/hermes_cli/ failures on my machine reproduce identically on clean main — they live-query localhost:11434, where I run a real Ollama, and collide with a researcher shim in my PATH; unrelated to this diff.)
2. Default behavior unchanged: with no config (or include_builtin: false), classify_tools() output is identical to main for any input — builtin tools never enter the deferred catalog
3. Opt-in behavior: set tools.tool_search: {enabled: on, include_builtin: true}, start a CLI session, and observe the activation line — browser/cron/kanban schemas leave the tools array while terminal, file tools, and web tools stay direct; tool_search("click a button") → tool_describe("browser_click") → tool_call(...) round-trips through the standard bridge dispatch with hooks/guardrails firing against the real tool name

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate (feat(mcp): add tool_search for on-demand MCP schema fetching #27257 closed, feat(mcp): add config-gated lazy MCP schema loading #33052 is MCP-only — see above)
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run the relevant test suites via scripts/run_tests.sh (see How to Test)
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: Ubuntu 24.04 (aarch64)

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — website/docs/user-guide/features/tool-search.md
• I've updated cli-config.yaml.example if I added/changed config keys — N/A (the example file doesn't carry the tools.tool_search block; keys documented in DEFAULT_CONFIG and the docs page)
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — pure-Python set/config logic, no OS-specific paths
• I've updated tool descriptions/schemas if I changed tool behavior — N/A (bridge tool schemas unchanged; only which tools sit behind them)

---

I had Claude (Fable 5) do this work — same arrangement as my previous PRs (#43045, #43058): testing how far the model gets on an unfamiliar codebase with minimal direction from me, and using my Claude-time to give something back to the project. The scoping decision (extend in-tree Tool Search rather than add a parallel lazy-loading mechanism), the implementation, and the tests are its work; I reviewed and take responsibility for the submission.

🤖 Generated with Claude Code

[teknium1]

Thanks for extending the existing Tool Search bridge rather than adding another discovery path. The builtin-schema premise is real on current main: tools/tool_search.py:163-184 still keeps every core tool direct.

Problems

• The resolved deferral policy is not session-stable. Assembly uses an explicit config (tools/tool_search.py:632), but tool_describe reloads it (tools/tool_search.py:715) and tool_call reaches is_deferrable_tool_name() without passing one (tools/tool_search.py:791). After a config edit, an existing agent can retain bridge-only schemas from agent/agent_init.py:1189-1198 while refusing to describe/call the tool it deferred. Tool configuration must remain stable for the session to preserve cache behavior.
• A custom always_include can defer skills, kanban, or computer use, but their system-prompt guidance is gated on model-visible valid_tool_names in agent/system_prompt.py:221-249. Preserve a pre-assembly available-name set for guidance and add an initialized-agent regression test.

Suggested changes

• Snapshot and thread one Tool Search config through assembly and all bridge dispatch/scope paths.
• Add available-vs-visible tool-name handling plus lifecycle and prompt-guidance coverage.

Automated hermes-sweeper review.

[teknium1]

This re-loads mutable user config after assembly. An existing agent can still expose the bridge schemas assembled with include_builtin: true, then reject a deferred builtin after config changes because resolve_underlying_call() also resolves the new config. Snapshot the resolved policy with the agent/tool definition snapshot and thread it through describe/call/scope checks so a session's tool surface remains cache-stable.