feat(tool_search): defer_toolsets — opt built-in toolsets into deferral - #60182
Draft
lEWFkRAD wants to merge 2 commits into
Draft
feat(tool_search): defer_toolsets — opt built-in toolsets into deferral#60182lEWFkRAD wants to merge 2 commits into
lEWFkRAD wants to merge 2 commits into
Conversation
…deferral Tool search today defers only MCP and plugin tools; built-in toolsets are core-protected without exception. On installs with few or no MCP servers that makes the feature a near no-op while the built-in schemas themselves are the entry tax — 14k+ tokens re-prefixed onto every session's first prefill (terminal, session_search, file, delegation, skills, computer_use alone are ~8k). Add tools.tool_search.defer_toolsets: an explicit, default-empty list of built-in toolset names whose tools ride the existing bridge (tool_search/tool_describe/tool_call) like MCP/plugin tools do. Opt-in only — nothing changes for existing configs, and the classifier's core protection stays the default for everything not listed. The opt-in is honored consistently on both sides of the bridge: assembly (classify/strip) and dispatch (tool_describe, tool_call unwrap, session scoping) resolve the same set, so an opted-in tool stripped from the visible array remains reachable through the bridge with guardrails, approvals, and hooks firing identically. Also documents the tools.tool_search block in cli-config.yaml.example (previously unlisted).
lEWFkRAD
marked this pull request as draft
July 7, 2026 13:36
6 tasks
13 tasks
teknium1
reviewed
Jul 15, 2026
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the focused opt-in design and for threading the configured set through assembly and bridge dispatch.
Problems
tools/tool_search.py:589removes opted-in core tools from the assembled array, andagent/agent_init.py:1195-1204derivesagent.valid_tool_namesfrom that array. This dropsSESSION_SEARCH_GUIDANCE,SKILLS_GUIDANCE, and computer-use guidance becauseagent/system_prompt.py:222-252gates them onvalid_tool_names; direct calls to deferred names also fail validation inagent/conversation_loop.py:4567-4575.- The patch updates only
cli-config.yaml.example;website/docs/user-guide/features/tool-search.md:18-25andhermes_cli/config.py:2824-2846still state that core tools never defer.
Suggested changes
- Preserve a session-static pre-assembly availability set for guidance and deliberate deferred-tool routing, distinct from the model-visible bridge schema set.
- Add the default-empty key to
DEFAULT_CONFIG, update the user guide, and cover an AIAgent-level opted-in core-tool path.
Automated hermes-sweeper review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Tool search (progressive tool disclosure) currently defers only MCP and plugin tools; built-in ("core") toolsets are always loaded. On installs with few or no MCP servers this makes the feature a near no-op, because there the built-in schemas are the entry tax — 14k+ tokens re-prefixed onto every session's first prefill.
This adds
tools.tool_search.defer_toolsets: an explicit, default-empty list of built-in toolset names whose tools ride the existingtool_search/tool_describe/tool_callbridge, exactly like MCP/plugin tools already do. It's opt-in — nothing changes for existing configs, and core protection stays the default for everything not listed.The opt-in is honored consistently on both sides of the bridge — assembly (classify/strip) and dispatch (
tool_describe,tool_callunwrap, session scoping) resolve the same set — so an opted-in tool stripped from the visible array stays reachable through the bridge with guardrails, approvals, and hooks firing identically.Related Issue
Fixes #60181
Type of Change
Changes Made
tools/tool_search.py— resolve a per-install "deferrable built-in" set fromdefer_toolsetsand apply it symmetrically in assembly and dispatch.tests/tools/test_tool_search.py— coverage for the opt-in set: classification/strip, describe/call reachability, session scoping, and the default (empty) no-op.cli-config.yaml.example— document thetools.tool_searchblock (previously unlisted), includingdefer_toolsetsand its round-trip trade-off.How to Test
tools.tool_search.defer_toolsets: [session_search, delegation, skills, computer_use].tool_searchand invokes it through the bridge; guardrails/approvals/hooks fire as normal.pytest tests/tools/test_tool_search.py -q→ 47 passed.Checklist
Code
feat(tool_search):)pytest tests/ -qand the tool_search tests pass (47 passed)Documentation & Housekeeping
cli-config.yaml.examplecli-config.yaml.example(added thetools.tool_searchblock incl.defer_toolsets)