feat: per-MCP-server tool_injection control (description_only mode)

[WeilaiSun]

Summary

Adds per-MCP-server tool_injection config option, supporting "full" (default, backwards-compatible) and "description_only" modes. Description-only tools are always deferred and discovered via tool_search, mirroring Claude Code's defer_loading pattern — the agent sees tool names + descriptions in the system prompt (stable tier, cached prefix) while full schemas are loaded on demand.

Closes #66736 · Inspired by #6839 (Lazy Tool Schema Loading) · Addresses Writer Harness paper critique (arXiv 2607.06906: Hermes tools not in cached prefix)

Motivation

Currently, all MCP tool schemas are loaded into every API call irrespective of frequency. For low-frequency MCP servers (e.g., firecrawl's 26 tools used 2-3 times per session), this wastes tokens on every turn. The tool_search mechanism already supports deferral, but only at a global threshold (>10% context) — there's no per-server control.

Real-world impact: one user moved 5 low-frequency MCP servers (64 tools) to mcporter CLI workarounds, saving ~60+ tool schemas from the tools array per turn. This feature makes that pattern native.

Design

Mirrors Claude Code's three-tier tool pool and Hermes's own Skill frontmatter discovery:

mcp_servers:
  firecrawl:
    command: npx
    args: [-y, firecrawl-mcp]
    tool_injection: description_only  # NEW: name+desc in prompt, schema on demand

Flow:

1. System prompt stable tier ← tool names + one-line descriptions (cached prefix)
2. Tools array ← bridge tools only (tool_search, tool_describe, tool_call)
3. Agent calls tool_search → tool_describe → tool_call to use deferred tools
4. Catalog stores full schemas → loaded from BM25 search on demand

Changes (3 files, +90/-1 lines)

• tools/tool_search.py: mark_description_only_tool() registry + force-defer in assemble_tools() even under threshold
• tools/mcp_tool.py: read tool_injection config, mark tools during registration
• agent/system_prompt.py: inject MCP tool inventory into stable tier, mirroring Skill index block

Backwards Compatibility

• tool_injection defaults to "full" — no behavior change for existing configs
• All existing tests pass without modification
• tool_search must be enabled (tools.tool_search) for description_only tools to be usable

Testing

• Unit: mark_description_only_tool / is_description_only_tool round-trip
• Unit: classify_tools correctly routes description_only tools to deferrable
• Unit: assemble_tools forces bridge injection when description_only tools exist
• Integration: description_only MCP server → tools in stable tier, not in tools array
• Integration: tool_search → tool_describe → tool_call works end-to-end

References

• Feature: Per-MCP-server tool injection control — description-only mode for low-frequency MCPs #66736 (this feature request)
• Feature: Lazy Tool Schema Loading — Two-Pass Tool Injection to Reduce Token Overhead #6839 (Lazy Tool Schema Loading — broader two-pass approach)
• Feature: Hybrid Tool Pre-Selection (Semantic + Keyword) — RAG-style schema injection to reduce token overhead without extra LLM round trips #13332 (Hybrid Tool Pre-Selection — complementary)
• Claude Code Blog: Prompt caching is everything (defer_loading pattern)
• Writer Agent Harness paper: arXiv 2607.06906 — Section 4.4: "Hermes does not place its tool schemas inside a cached prefix"

[alt-glitch]

This was generated by AI during triage.

Related to #66736 and the broader #66481 smart-loading work. This is an independently scoped per-server description-only policy, so it is not marked duplicate; maintainer selection is needed for the MCP lazy-schema design.

[teknium1]

Thanks for the focused per-server proposal. The global-threshold premise is real on current main (model_tools.py:548-563), but this implementation needs scope and lifecycle work before it is safe to salvage.

Problems

• agent/system_prompt.py enumerates the process-global marked set, not the agent's filtered toolset. This conflicts with the restricted-session isolation contract documented in tests/tools/test_tool_search.py:419-428.
• The marking added in tools/mcp_tool.py covers only server._tools; resource/prompt utility schemas are registered separately at tools/mcp_tool.py:5088-5123 and remain eager.
• Current model_tools.py:550 skips assembly when global tool search is off, while the new inventory still instructs the model to use tool_search.
• The PR adds no tests despite changing MCP registration, tool assembly, and cached prompt construction.

Suggested changes

• Derive the inventory from agent-scoped pre-assembly definitions, define utility-tool semantics, and add coverage for scoped sessions, tool-search-off, late MCP registration, and bridge dispatch.

Automated hermes-sweeper review.

[teknium1]

get_description_only_tool_names() is process-global. This inventory is not filtered through the agent's enabled/disabled toolsets, so a restricted session can be told about description-only tools from another MCP server. Please derive the inventory from the same session-scoped pre-assembly definitions used by the bridge catalog.

[teknium1]

This marker is applied only in the server-native server._tools registration loop. The resource/prompt utility schemas registered below remain eager, so the advertised per-server mode does not apply to every tool from this server. Please either cover those registrations or narrow and document the policy.

[WeilaiSun]

Review feedback fixes (3 commits)

Addressed all 4 problems identified by @teknium1 in sweeper review:

P1: Session-scoped inventory ✅

• agent_init.py: Capture _pre_assembly_tool_names from model_tools._last_resolved_tool_names
• system_prompt.py: Filter get_description_only_tool_names() against session-scoped names instead of process-global set

P2: Utility tool marking ✅

• mcp_tool.py L4855: Mark resource/prompt utility schemas as description_only when server has tool_injection: description_only

P3: tool_search=off gate ✅

• system_prompt.py: Skip inventory block when ts_cfg.enabled == "off" — no misleading tool_search hints

P4: Test coverage ✅

• TestDescriptionOnly class: 11 tests, all 5 scenarios covered:
  • mark/is round-trip + copy semantics
  • deferrable classification + classify_tools
  • bridge activation below threshold
  • tool_search=off gate
  • session-scoped inventory (restricted session isolation)
  • lazy MCP registration persistence
  • bridge dispatch compatibility
  • system prompt inventory verification (names+descriptions, no full schemas)

50/50 tests pass | Changes match suggested approach: agent-scoped definitions, utility-tool semantics, comprehensive coverage.

[WeilaiSun]

Hey @teknium1, gentle ping — all 4 issues from the sweeper review have been addressed and pushed (see previous comment).

Summary of fixes:

• P1: Session-scoped inventory derived from agent's filtered toolset
• P2: Resource/prompt utility schemas now marked description_only
• P3: tool_search=off gate — no misleading hints when search disabled
• P4: 11 tests covering all 5 scenarios, all passing

Ready for re-review when you have a chance. Thanks!

[GottZ]

This was generated by AI during triage.

Summary

One PR addresses issue #66736. #66826 implements the requested per-server description_only policy by deferring marked MCP tools, publishing a scoped name-and-description catalog, and exposing full schemas through the existing tool-search bridge.

Related pull requests

• feat: per-MCP-server tool_injection control (description_only mode) #66826 best fix — (+828/-15) — n/a: The diff covers regular and generated MCP utility tools, tool-search-disabled behavior, cache hits, late registration, deregistration, scoped inventory, and bridge discovery with added tests. Consistent with the contributor's COMMENTED keep_open review, the implementation remains salvageable, but agent_init.py still copies session scope from the process-global model_tools._last_pre_assembly_tool_names, leaving a possible cross-agent race between tool-definition assembly and agent capture.

Suggested consolidation

Keep #66826 open with a salvage path: preserve its per-server policy, lifecycle handling, prompt inventory, and test coverage, while replacing the process-global pre-assembly handoff with state returned directly from tool-definition assembly or otherwise bound to the receiving agent, then add a concurrent-agent isolation regression test. There are no duplicate PRs to close.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I66736(["issue #66736 (open)"])
    P66826["PR #66826 (open)"]
    P66826 -->|best fix| I66736
    class I66736 open
    class P66826 open
    class P66826 best
    class P66826 target
    click I66736 "https://github.com/NousResearch/hermes-agent/issues/66736"
    click P66826 "https://github.com/NousResearch/hermes-agent/pull/66826"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 1 pull request and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 49 kB of PR diffs, 6 kB of issue/PR text, 6 kB of discussion (7 comments), 2 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[WeilaiSun]

Thanks for the triage review @GottZ — the race is fixed.

What changed (3 commits on top of the P1/P2 fixes):

1. get_tool_definitions_with_meta() — the tool-definition assembly now returns (tools, pre_assembly_names) directly from the call that produced them (both cache-hit and fresh paths). agent_init binds agent._pre_assembly_tool_names from that return value instead of copying the process-global _last_pre_assembly_tool_names — no cross-agent handoff, no race window. The legacy global remains only for the remaining non-agent consumers (documented).
2. Concurrent-agent isolation regression test — two agents with disjoint toolsets interleaved through init; each agent's pre-assembly inventory stays its own (RED on the old implementation).
3. Same-class race in refresh_agent_mcp_tools (mcp_tool.py) also switched to the with_meta return value instead of reading the process global.
4. Full mock adaptation across the test suite (conftest forwarding fixture + per-site patches), so the CI surface is green for the affected paths.

Verified: test_model_tools + cache_isolation + delegate + refresh + turn_context + acp suites all pass; ruff clean. (Two pre-existing environment-only failures in TestVisionDispatchLoopSafety and a Windows path-separator test are unrelated and reproduce on the parent commit.)

Happy to adjust if you'd prefer the state bound differently.