Skip to content

feat(tool_search): defer_toolsets — opt built-in toolsets into deferral - #60182

Draft
lEWFkRAD wants to merge 2 commits into
NousResearch:mainfrom
lEWFkRAD:feat/tool-search-defer-builtin
Draft

feat(tool_search): defer_toolsets — opt built-in toolsets into deferral#60182
lEWFkRAD wants to merge 2 commits into
NousResearch:mainfrom
lEWFkRAD:feat/tool-search-defer-builtin

Conversation

@lEWFkRAD

@lEWFkRAD lEWFkRAD commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Tool search (progressive tool disclosure) currently defers only MCP and plugin tools; built-in ("core") toolsets are always loaded. On installs with few or no MCP servers this makes the feature a near no-op, because there the built-in schemas are the entry tax — 14k+ tokens re-prefixed onto every session's first prefill.

This adds tools.tool_search.defer_toolsets: an explicit, default-empty list of built-in toolset names whose tools ride the existing tool_search / tool_describe / tool_call bridge, exactly like MCP/plugin tools already do. It's opt-in — nothing changes for existing configs, and core protection stays the default for everything not listed.

The opt-in is honored consistently on both sides of the bridge — assembly (classify/strip) and dispatch (tool_describe, tool_call unwrap, session scoping) resolve the same set — so an opted-in tool stripped from the visible array stays reachable through the bridge with guardrails, approvals, and hooks firing identically.

Related Issue

Fixes #60181

Type of Change

  • ✨ New feature (non-breaking change that adds functionality)

Changes Made

  • tools/tool_search.py — resolve a per-install "deferrable built-in" set from defer_toolsets and apply it symmetrically in assembly and dispatch.
  • tests/tools/test_tool_search.py — coverage for the opt-in set: classification/strip, describe/call reachability, session scoping, and the default (empty) no-op.
  • cli-config.yaml.example — document the tools.tool_search block (previously unlisted), including defer_toolsets and its round-trip trade-off.

How to Test

  1. On an MCP-light install, set tools.tool_search.defer_toolsets: [session_search, delegation, skills, computer_use].
  2. Start a session — the deferred toolsets no longer appear in the model-facing tools array (measured ~7.2k-token / ~49% cut to the first-prefill tool-schema tax).
  3. Ask for something that needs a deferred tool — the model discovers it via tool_search and invokes it through the bridge; guardrails/approvals/hooks fire as normal.
  4. pytest tests/tools/test_tool_search.py -q → 47 passed.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (feat(tool_search):)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this feature (no unrelated commits)
  • I've run pytest tests/ -q and the tool_search tests pass (47 passed)
  • I've added tests for my changes
  • I've tested on my platform: Windows 11

Documentation & Housekeeping

  • I've updated relevant documentation (docstrings) — and cli-config.yaml.example
  • I've updated cli-config.yaml.example (added the tools.tool_search block incl. defer_toolsets)
  • N/A — no architecture/workflow change to CONTRIBUTING/AGENTS
  • I've considered cross-platform impact — pure config/assembly logic, no platform-specific paths
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A (bridge behavior unchanged; only which tools ride it)

…deferral

Tool search today defers only MCP and plugin tools; built-in toolsets are
core-protected without exception. On installs with few or no MCP servers
that makes the feature a near no-op while the built-in schemas themselves
are the entry tax — 14k+ tokens re-prefixed onto every session's first
prefill (terminal, session_search, file, delegation, skills, computer_use
alone are ~8k).

Add tools.tool_search.defer_toolsets: an explicit, default-empty list of
built-in toolset names whose tools ride the existing bridge
(tool_search/tool_describe/tool_call) like MCP/plugin tools do. Opt-in
only — nothing changes for existing configs, and the classifier's core
protection stays the default for everything not listed.

The opt-in is honored consistently on both sides of the bridge: assembly
(classify/strip) and dispatch (tool_describe, tool_call unwrap, session
scoping) resolve the same set, so an opted-in tool stripped from the
visible array remains reachable through the bridge with guardrails,
approvals, and hooks firing identically.

Also documents the tools.tool_search block in cli-config.yaml.example
(previously unlisted).

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused opt-in design and for threading the configured set through assembly and bridge dispatch.

Problems

  • tools/tool_search.py:589 removes opted-in core tools from the assembled array, and agent/agent_init.py:1195-1204 derives agent.valid_tool_names from that array. This drops SESSION_SEARCH_GUIDANCE, SKILLS_GUIDANCE, and computer-use guidance because agent/system_prompt.py:222-252 gates them on valid_tool_names; direct calls to deferred names also fail validation in agent/conversation_loop.py:4567-4575.
  • The patch updates only cli-config.yaml.example; website/docs/user-guide/features/tool-search.md:18-25 and hermes_cli/config.py:2824-2846 still state that core tools never defer.

Suggested changes

  • Preserve a session-static pre-assembly availability set for guidance and deliberate deferred-tool routing, distinct from the model-visible bridge schema set.
  • Add the default-empty key to DEFAULT_CONFIG, update the user guide, and cover an AIAgent-level opted-in core-tool path.

Automated hermes-sweeper review.

Comment thread tools/tool_search.py
@teknium1 teknium1 added sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users labels Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users sweeper:risk-caching Sweeper risk: may break/degrade prompt caching or cache-key stability (invariant) sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Let built-in toolsets opt into tool-search deferral (defer_toolsets)

3 participants