Skip to content

fix(mcp): don't trip circuit breaker on tool errors - #74795

Draft
renga-kogahara wants to merge 1 commit into
NousResearch:mainfrom
renga-kogahara:hotfix/mcp-tool-error-circuit-breaker
Draft

fix(mcp): don't trip circuit breaker on tool errors#74795
renga-kogahara wants to merge 1 commit into
NousResearch:mainfrom
renga-kogahara:hotfix/mcp-tool-error-circuit-breaker

Conversation

@renga-kogahara

Copy link
Copy Markdown

What does this PR do?

Treats completed MCP CallToolResult(isError=True) responses as tool/domain errors rather than server connectivity failures.

A completed RPC proves the MCP transport is reachable. Counting a tool-level validation error toward the server circuit breaker can incorrectly mark a healthy MCP server as unreachable and prevent subsequent valid calls.

Related Issue

Supersedes #74718.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • Reset the server connectivity breaker after any completed tools/call RPC, including isError=True results.
  • Preserve completed tool/domain errors returned after OAuth recovery or session reconnect instead of reclassifying them as auth/transport failures.
  • Continue counting raised timeout, transport, and session failures toward the connectivity breaker.
  • Add regression coverage for normal calls, OAuth retries, and session-reconnect retries.

How to Test

  1. Run .venv/bin/python -m pytest tests/tools/test_mcp*.py -q -o 'addopts='.
  2. Confirm repeated CallToolResult(isError=True) responses do not open the server connectivity breaker.
  3. Confirm OAuth and session reconnect retries return completed tool errors unchanged.

Focused result: 385 passed.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Linux

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — docstrings updated
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — no platform-specific APIs added
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Screenshots / Logs

N/A

@alt-glitch alt-glitch added type/bug Something isn't working comp/tools Tool registry, model_tools, toolsets tool/mcp MCP client and OAuth P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jul 30, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #74045. Both reset the MCP connectivity breaker after a completed tool/domain-error RPC; #74045 also covers the auth and session-recovery paths, so this narrower patch is redundant.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused MCP classification fix. Current main still converts completed CallToolResult.isError responses into an error envelope at tools/mcp_tool.py:4812-4827 and then increments the server-wide breaker for that envelope at tools/mcp_tool.py:4896-4905, so the reported behavior is present.

Problems

  • This patch is redundant with the still-open #74045: its production changes cover the same auth retry, session-reconnect retry, and completed-call classification sites. #74045 also exercises recovery through _make_tool_handler, rather than only calling the shared retry helpers directly.

Suggested changes

  • If retained separately, test the auth and session recovery paths through _make_tool_handler with a real CallToolResult(isError=True) fixture, covering the conversion path at tools/mcp_tool.py:4812-4827 as well as the recovery helper.

This is an automated hermes-sweeper review.

server = MagicMock()
server._reconnect_event = MagicMock()
mcp_tool._servers["srv"] = server
mcp_tool._server_error_counts["srv"] = 2

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This invokes the private recovery helper with a prebuilt JSON error, so it does not exercise the real CallToolResult.isErrortool_error(...) conversion in _make_tool_handler. Consider a handler-level regression like #74045's recovery coverage.


try:
result = mcp_tool._handle_session_expired_and_retry(
"srv-domain-error",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This likewise verifies the helper return value but not the actual post-reconnect CallToolResult.isError conversion path. A handler-level fixture would cover the end-to-end behavior this PR changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants