Skip to content

fix(mcp): catch RuntimeError in _wait_for_reconnect_or_shutdown cleanup - #71846

Closed
Arasz wants to merge 1 commit into
NousResearch:mainfrom
Arasz:fix/mcp-event-loop-closed-race
Closed

fix(mcp): catch RuntimeError in _wait_for_reconnect_or_shutdown cleanup#71846
Arasz wants to merge 1 commit into
NousResearch:mainfrom
Arasz:fix/mcp-event-loop-closed-race

Conversation

@Arasz

@Arasz Arasz commented Jul 26, 2026

Copy link
Copy Markdown

What

Catches RuntimeError in the finally block of _wait_for_reconnect_or_shutdown at both cleanup sites in mcp_tool.py.

Why

When the event loop closes during MCP server reload or Ctrl+C exit, parked MCPServerTask coroutines are garbage-collected. Their finally blocks call t.cancel() which invokes loop.call_soon() on the already-closed loop, raising RuntimeError: Event loop is closed.

This produces 14+ ignored exception tracebacks on every MCP reload — noisy and confusing for users, though functionally harmless.

How

Wrap t.cancel() in try/except RuntimeError at both _wait_for_reconnect_or_shutdown cleanup sites (lines ~2297 and ~2341). When the loop is already closed, the tasks are dead — breaking out of the cleanup loop is the correct behavior.

The fix is purely defensive: no behavior change during normal operation, just graceful handling of the shutdown race.

Test plan

  • New test: test_wait_for_reconnect_or_shutdown_no_error_on_closed_loop — verifies no RuntimeError escapes when the loop closes during park
  • New test: test_wait_for_reconnect_or_shutdown_returns_on_shutdown_event — baseline: normal shutdown path still works
  • Existing tests: test_mcp_reconnect_retry_reset.py (2/2 pass)
  • Manual: hermes chat -q "hello" then Ctrl+C — no traceback

Platforms tested

macOS (Apple Silicon, Python 3.11.15)

Related issues

Closes #60197
Closes #66228

When the event loop closes during MCP server reload or Ctrl+C exit,
parked MCPServerTask coroutines are garbage-collected. Their finally
blocks call t.cancel() which invokes loop.call_soon() on the already-
closed loop, raising RuntimeError: Event loop is closed.

This produced 14+ ignored exception tracebacks on every MCP reload.

Fix: wrap t.cancel() in try/except RuntimeError at both cleanup sites
in _wait_for_reconnect_or_shutdown. When the loop is already closed,
the tasks are dead — breaking out is the correct cleanup.

Closes NousResearch#60197
Closes NousResearch#66228
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/mcp MCP client and OAuth P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jul 26, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #60032. The current diff applies the same RuntimeError guard to the same two MCP cleanup loops; its added regression test does not introduce a distinct repair mechanism.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for isolating the closed-loop cancellation symptom and adding coverage.

Automated hermes-sweeper review found this is already implemented on current main by the stronger merged consolidation in #74139:

  • tools/mcp_tool.py:6717-6766 drains and reaps pending MCP-loop tasks before scheduling loop.stop().
  • tools/mcp_tool.py:6781-6823 keeps drain and stop ordered on the owning loop before loop.close().
  • tests/tools/test_mcp_stability.py:606-612 verifies cleanup completes without a closed-loop error before stop, including the blocked-loop ordering case at :652-697.
  • Merged commit 1f70ba6bca71caa2c440607b06b74e596969e1b1 completed fix(mcp): drain the MCP loop and retain parked startup tasks for clean shutdown #74139's consolidation.

The current diff is therefore redundant; thank you for the focused investigation.

@teknium1 teknium1 closed this Jul 30, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

3 participants