Skip to content

fix: suppress RuntimeError in MCP task shutdown when event loop is closed - #80955

Open
shaiful-hisham wants to merge 1 commit into
NousResearch:mainfrom
shaiful-hisham:fix/mcp-shutdown-event-loop-closed
Open

fix: suppress RuntimeError in MCP task shutdown when event loop is closed#80955
shaiful-hisham wants to merge 1 commit into
NousResearch:mainfrom
shaiful-hisham:fix/mcp-shutdown-event-loop-closed

Conversation

@shaiful-hisham

@shaiful-hisham shaiful-hisham commented Aug 7, 2026

Copy link
Copy Markdown

Problem

When exiting Hermes, MCP server tasks that are still pending can encounter an already-closed event loop during cancellation. Calling t.cancel() after the loop is closed raises RuntimeError('Event loop is closed'), which surfaces as Exception ignored in: <coroutine object MCPServerTask.run> during garbage collection.

Symptom

Exception ignored in: <coroutine object MCPServerTask.run at 0x10d3f5d20>
RuntimeError: Event loop is closed

Fix (4 call sites in tools/mcp_tool.py)

  1. _wait_for_lifecycle_event finally — wrap t.cancel() in try/except RuntimeError so cancelled tasks clean up silently when the loop is already gone
  2. _wait_for_reconnect_or_shutdown finally — same pattern
  3. _wait_for_lazy_reconnect finally — same pattern
  4. _stop_mcp_loop drain path — change elif not loop.is_closed() to if not stop_owned_by_loop and not loop.is_closed() so drain is always attempted regardless of schedule success

Tested

Verified on macOS — the "Exception ignored" traceback no longer appears on hermes exit.

Consolidation

This PR is the consolidated fix for the closed-loop cancellation family, superseding #60032, #60104, and #60380. Coverage comparison:

Call site This PR #60032 #60104 #60380
_wait_for_lifecycle_event finally
_wait_for_reconnect_or_shutdown finally
_wait_for_lazy_reconnect finally
_stop_mcp_loop drain path

This PR fixes the whole bug class — every finally block that calls t.cancel() on a possibly-closed loop, plus the drain path in _stop_mcp_loop so tasks are always cancelled before loop.close(). Happy to coordinate with #60032's author (open) so we land one shared shutdown strategy rather than overlapping fixes.

…osed

When exiting Hermes, MCP server tasks that are still pending can
encounter an already-closed event loop during cancellation. Calling
t.cancel() after the loop is closed raises RuntimeError('Event loop
is closed'), which surfaces as 'Exception ignored' during garbage
collection.

Four call sites patched:
- _wait_for_lifecycle_event finally block (3 sites): wrap t.cancel()
  in try/except RuntimeError so cancelled tasks clean up silently
  when the loop is already gone
- _stop_mcp_loop drain path: always attempt drain regardless of
  safe_schedule_threadsafe success, so tasks are cancelled before
  loop.close() is called
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/mcp MCP client and OAuth P2 Medium — degraded but workaround exists labels Aug 7, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to #60032, #60104, and #60380: this uses the same closed-loop cancellation family but adds lazy-reconnect coverage and a drain-path change. Please consolidate the preferred shutdown strategy.

@shaiful-hisham

Copy link
Copy Markdown
Author

Thanks for the triage note. I've updated the PR description with a Consolidation section — this PR is intentionally the consolidated fix for the closed-loop cancellation family:

Happy to coordinate with the #60032 author so we land one shared shutdown strategy rather than overlapping fixes — this PR is ready to serve as the consolidation point.

zons-zhaozhy pushed a commit to zons-zhaozhy/hermes-agent that referenced this pull request Aug 10, 2026
…+ #68270精华

mcp_tool.py (3 call sites):
- NousResearch#80955 三段式: t.cancel() in try → except RuntimeError skip await → else await
- 消除 TOCTOU 竞态 (is_closed() check + await gap)
- 统一所有 3 个 _wait_*_reconnect call sites

cli.py:
- NousResearch#68270 提取 _suppress_closed_loop_errors 到模块级 (line ~1330)
- interactive + single-query 两条路径共享同一 handler
- 消除 interactive 路径的闭包重复定义
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants