Skip to content

fix(agent): repair mcp_ prefix drop in _repair_tool_call - #21696

Closed
real-wimpSquad wants to merge 1 commit into
NousResearch:mainfrom
real-wimpSquad:fix/repair-mcp-prefix-drop
Closed

fix(agent): repair mcp_ prefix drop in _repair_tool_call#21696
real-wimpSquad wants to merge 1 commit into
NousResearch:mainfrom
real-wimpSquad:fix/repair-mcp-prefix-drop

Conversation

@real-wimpSquad

@real-wimpSquad real-wimpSquad commented May 8, 2026

Copy link
Copy Markdown

What

Models sometimes call MCP server tools without the mcp_ prefix that Hermes adds at registration time — e.g. APE_search instead of mcp_APE_search, or discord_send instead of mcp_discord_send. The call fails with Tool does not exist and the agent burns retries before giving up.

Why it happens

Hermes registers MCP tools as mcp_{server}_{tool} (line ~995 in mcp_tool.py). The existing _repair_tool_call fuzzy matcher uses difflib with cutoff=0.7, but APE_search scores ~0.60 against mcp_APE_search — just below the threshold. The fuzzy path misses it every time.

Fix

Add an explicit mcp_ prepend check before the candidate-set / fuzzy-match path. Four candidates tried in order:

  1. mcp_<original>
  2. mcp_<lowered>
  3. mcp_<normalized>
  4. Case-folded linear scan (handles APE_SEARCH → mcp_APE_search where server name has mixed case)

The check is unambiguous — only resolves on an exact match in valid_tool_names, cannot mis-route non-MCP tools.

How to reproduce

  1. Configure any MCP server (e.g. named APE)
  2. Use a model that tends to drop the mcp_ prefix (observed with several models including Gemma and Claude variants)
  3. Model calls APE_search → Tool 'APE_search' does not exist

Tests

Added TestMcpPrefixDrop (10 tests) to the existing test_repair_tool_call_name.py:

  • All APE tool variants without prefix
  • Other MCP server names
  • Already-correct names (no double-prefix)
  • Uppercase input
  • Non-MCP tools in valid set unaffected
  • No-invent guard (unknown tools still return None)

All 28 tests pass. The 2 pre-existing failures in test_concurrent_interrupt.py are present on main before this change and unrelated.

Platform

Tested on macOS 14 / Python 3.11.

@real-wimpSquad
fix(agent): repair mcp_ prefix drop in _repair_tool_call
82ecff5 
Models sometimes call MCP server tools without the mcp_ prefix that
Hermes adds at registration time — e.g. APE_search instead of
mcp_APE_search, or discord_send instead of mcp_discord_send.

difflib scores ~0.60 for this pattern (just below the 0.7 cutoff),
so the existing fuzzy-match path misses it and the call fails with
"Tool does not exist".

Fix: add an explicit mcp_ prepend check before the candidate-set /
fuzzy-match path. Three candidates are tried in order:
  mcp_<original>
  mcp_<lowered>
  mcp_<normalized>
then a case-folded linear scan for mixed-case server names
(e.g. APE_SEARCH -> mcp_APE_search).

The check is unambiguous — only resolves on an exact match in
valid_tool_names, so it cannot mis-route non-MCP tools.

Adds TestMcpPrefixDrop (10 tests) to the existing repair test file,
covering: all APE tool variants, other MCP servers, already-correct
names, uppercase input, non-MCP tools unaffected, and the no-invent
guard. All 28 tests pass; pre-existing test_concurrent_interrupt
failures are unrelated and present on main before this change.

Discovered while working with an APE MCP server where the model
consistently emitted APE_meta / APE_search instead of the prefixed
names.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/mcp MCP client and OAuth P2 Medium — degraded but workaround exists labels May 8, 2026
@alt-glitch alt-glitch mentioned this pull request Jul 3, 2026
Open
@yingliang-zhang yingliang-zhang mentioned this pull request Jul 9, 2026
Closed
yingliang-zhang added a commit to yingliang-zhang/hermes-agent that referenced this pull request Jul 9, 2026
@yingliang-zhang
fix(agent): repair MCP tool names with dropped mcp__server__ prefix
c464de1 
Models (notably GLM) sometimes emit MCP tool names without the full
mcp__<server>__ prefix — e.g. list_memory_projects instead of
mcp__basic_memory__list_memory_projects. The existing repair pipeline
(case-folding, CamelCase, suffix stripping, fuzzy match at 0.7 cutoff)
cannot map the bare suffix back to the full prefixed name, so the call
fails with "Unknown tool" and the agent burns 3 retries before aborting
with "Model generated invalid tool call".

Add two repair paths in repair_tool_call() before the fuzzy-match
fallback:

1. Bare suffix (no __ in emitted name): scan registered mcp__ tools
   and match on the tool-suffix portion. Skip when ambiguous (multiple
   servers expose the same suffix) to avoid non-deterministic routing.

2. Partial prefix (__ in emitted name but missing mcp__): prepend
   mcp__ and check for an exact match.

The mcp__server__tool naming convention was introduced in NousResearch#52750
(merged 2026-07-05). Prior PRs NousResearch#21696, NousResearch#37100, NousResearch#33359 target the old
single-underscore mcp_ format and do not apply to the new convention.

9 new tests in TestMcpPrefixRepair covering both cases, ambiguity
guard, and non-MCP tool passthrough. All 38 tests in
test_repair_tool_call_name.py pass.
yingliang-zhang added a commit to yingliang-zhang/hermes-agent that referenced this pull request Jul 11, 2026
@yingliang-zhang
fix(agent): repair MCP tool names with dropped mcp__server__ prefix
e87d254 
Models (notably GLM) sometimes emit MCP tool names without the full
mcp__<server>__ prefix — e.g. list_memory_projects instead of
mcp__basic_memory__list_memory_projects. The existing repair pipeline
(case-folding, CamelCase, suffix stripping, fuzzy match at 0.7 cutoff)
cannot map the bare suffix back to the full prefixed name, so the call
fails with "Unknown tool" and the agent burns 3 retries before aborting
with "Model generated invalid tool call".

Add two repair paths in repair_tool_call() before the fuzzy-match
fallback:

1. Bare suffix (no __ in emitted name): scan registered mcp__ tools
   and match on the tool-suffix portion. Skip when ambiguous (multiple
   servers expose the same suffix) to avoid non-deterministic routing.

2. Partial prefix (__ in emitted name but missing mcp__): prepend
   mcp__ and check for an exact match.

The mcp__server__tool naming convention was introduced in NousResearch#52750
(merged 2026-07-05). Prior PRs NousResearch#21696, NousResearch#37100, NousResearch#33359 target the old
single-underscore mcp_ format and do not apply to the new convention.

9 new tests in TestMcpPrefixRepair covering both cases, ambiguity
guard, and non-MCP tool passthrough. All 38 tests in
test_repair_tool_call_name.py pass.
@swissly swissly mentioned this pull request Jul 11, 2026
Closed
6 tasks
swissly added a commit to swissly/hermes-agent that referenced this pull request Jul 11, 2026
@swissly
feat(agent): add tool-call repair observability
bb3b43d 
Structured stats collection for the existing tool-call repair pipeline.
Records RepairEvent (pattern, tool, model, timestamp) at each repair
pass in message_sanitization.py and model_tools.py coerce_tool_args.

New module: agent/tool_repair_stats.py
- RepairPattern enum (20 known failure patterns)
- ToolRepairStats singleton: thread-safe, ring-buffer (10k events)
- record_repair() convenience function
- summary() for CLI display

Hooks added (1-2 lines each, zero-overhead when unused):
- message_sanitization.py: 6 hooks in _repair_tool_call_arguments
  (empty_args, none_literal, control_char_escape, trailing_comma,
   unrepairable)
- model_tools.py: 2 hooks in coerce_tool_args
  (bare_string_wrap, bare_object_wrap)

Design constraints:
- No new model tools (zero API cost impact)
- No prompt caching impact
- No new config keys
- Import failure → no-op (never breaks repair pipeline)
- Thread-safe with threading.Lock
- Bounded memory (ring buffer caps at 10k events)

Tests: 19 new tests (stats, thread-safety, ring-buffer, resilience)
Regression: 82 existing repair/coercion tests still pass

Complementary to existing repair PRs (NousResearch#62578, NousResearch#56399, NousResearch#61550, NousResearch#59267,
NousResearch#52747, NousResearch#55620, NousResearch#56557, NousResearch#21696) — adds observability, not repairs.
@teknium1

teknium1 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Thanks for the focused reproduction and regression coverage.

This automated hermes-sweeper review is closing this under the standing model-output-repair policy: Hermes does not add new reconstruction passes for incorrect model-emitted tool names.

  • The PR adds a new mcp_ prefix reconstruction before the existing unknown-tool correction path.
  • Current MCP registration instead uses mcp__<server>__<tool> at tools/mcp_tool.py:4466-4484, shipped by e01f58ff1fdebbb6f7af971f04825d071f3f09da (v2026.7.7).
  • The repair implementation has also moved: run_agent.py:3834-3837 is now a forwarder to agent/agent_runtime_helpers.py:2325-2416.
  • The linked PR fix(agent): repair MCP tool names with dropped mcp__server__ prefix #61336 was closed under the same model-output-repair policy for the corresponding current-format reconstruction.

Please re-scope any follow-up to the model/provider or MCP schema/prompt source that emits the incorrect tool name, rather than adding another core output-repair branch.

Closed as not-planned per standing maintainer policy (model-output-repair). This is a design-direction decision, not a code-quality judgment — see the Contribution Rubric in AGENTS.md for what the project is looking for. If you believe this policy was misapplied to your change, comment here and a maintainer will take a look.

@teknium1 teknium1 closed this Jul 13, 2026
@teknium1 teknium1 added the sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) label Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:not-planned Sweeper: closed per standing maintainer policy (design direction) tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@real-wimpSquad @teknium1 @alt-glitch