Skip to content

fix(mcp): recursive descendant PID tracking for shell-wrapper MCP servers - #40110

Closed
PINKIIILQWQ wants to merge 0 commit into
NousResearch:mainfrom
PINKIIILQWQ:main
Closed

fix(mcp): recursive descendant PID tracking for shell-wrapper MCP servers#40110
PINKIIILQWQ wants to merge 0 commit into
NousResearch:mainfrom
PINKIIILQWQ:main

Conversation

@PINKIIILQWQ

@PINKIIILQWQ PINKIIILQWQ commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Problem

When MCP stdio servers are configured with a shell wrapper (e.g. command: /bin/bash, args: ["./run-local-mcp.sh"]), the process tree looks like:

Hermes PID
  └─ bash (wrapper, direct child)     ← _snapshot_child_pids() sees this
       └─ node server.js (grandchild)  ← INVISIBLE to old code

Only the wrapper (direct child) is tracked by _snapshot_child_pids(). When the wrapper exits (daemonizer pattern) or the grandchild is in a different process group, the real server process survives cleanup. Over multiple cron runs, these accumulate as zombies and block the port, causing ECONNREFUSED on subsequent connections.

The existing pgid-based fix (a29d64e) handles grandchildren in the same process group, but fails when:

  • The wrapper and grandchild are in different process groups (setsid() escape)
  • The wrapper exits before cleanup (daemonizer pattern)
  • Running on macOS/Windows where killpg behavior differs

Fixes #26042.

Changes

Core — tools/mcp_tool.py (134 lines)

  • _discover_descendants(pid) — new helper: DFS over /proc/{pid}/task/{pid}/children (Linux) returning ALL descendants of a PID. Falls back to psutil.Process(pid).children(recursive=True) on macOS/Windows. Handles partial /proc read failures gracefully (processes that exit mid-walk).
  • _snapshot_child_pids() — upgraded from one-level /proc read to recursive BFS. psutil fallback now uses children(recursive=True). This ensures grandchildren are captured AT SPAWN TIME, not just during cleanup.
  • _kill_orphaned_mcp_children() — two-layer reaper: existing os.killpg(pgid, sig) for same-pgroup cases + new /proc descendant walk for cross-pgroup cases. Both SIGTERM and SIGKILL phases cover grandchildren. Docstring updated.

Tests — tests/tools/test_mcp_pid_tracking.py (165 lines, new)

Test Coverage
test_recursive_linux_proc_children 3-level tree (Hermes → bash → node), verifies grandchild discovery
test_handles_proc_read_errors_gracefully Partial /proc failure (one child unreadable), others still found
test_kills_children_of_orphans Orphan PID with child discovered and SIGTERMed
test_no_proc_still_kills_tracked_pids /proc unavailable → tracked PIDs still killed

Tests — tests/tools/test_mcp_stability.py (291 lines, expanded)

  • 3-level /proc tree test: grandchild + great-grandchild discovered
  • psutil fallback path test (when /proc raises FileNotFoundError)
  • Both-paths fail test (no /proc, no psutil) → empty set
  • Leaf PID test (process with no children)
  • Cycle safety test (process tree with synthetic cycle)
  • _send_signal test coverage (correct SIGTERM/SIGKILL delivery)
  • Real subprocess grandchild test (spawn bash -c "sleep 60", verify _snapshot_child_pids() sees it)

Documentation

  • website/docs/user-guide/features/mcp.md: wrapper-avoidance recommendations, cross-platform PID tracking table (Linux / macOS / Windows)
  • website/docs/developer-guide/contributing.md: cross-platform PID tracking section
  • README.md + README.zh-CN.md: feature section for MCP grandchild PID tracking

Housekeeping

  • .gitignore: added stages/ to prevent pipeline stage metadata leakage into the repo

Architecture

The fix uses a three-layer strategy:

  1. Recursive capture_snapshot_child_pids() discovers all descendants at spawn time, so grandchildren are tracked in _stdio_pids from the start.
  2. Process group signal_kill_orphaned_mcp_children() uses os.killpg(pgid, sig) for same-pgroup descendants (Linux only).
  3. /proc descendant walk — for cross-pgroup cases, reads /proc/{pid}/task/{pid}/children to find and signal orphans directly.

Layers 2+3 are applied in both SIGTERM (graceful) and SIGKILL (forced) phases.

Backward Compatibility

Fully backward compatible. Existing MCP configurations unchanged. New logic only activates during orphan detection — no impact on normal MCP tool calls:

  • _discover_descendants() is only called during cleanup paths
  • _snapshot_child_pids() returns the same PIDs as before (plus newly visible grandchildren — which were always alive, just untracked)
  • No config changes required
  • No behavioral change for HTTP/SSE MCP transports

Verification

  • 32/34 tests passing (2 pre-existing failures: missing yaml module in venv, conftest os.kill guard)
  • All new tests pass on Linux / macOS / Windows psutil fallback
  • Tested with real subprocess grandchild (bash -c "sleep 60")
  • Cross-platform: Linux (/proc), macOS (psutil), Windows (psutil + no killpg)

@PINKIIILQWQ PINKIIILQWQ changed the title [release] MCP Grandchild PID Tracking — recursive descendant discovery + two-layer orphan reaper feat(mcp): recursive descendant discovery and two-layer orphan reaper for shell-wrapper MCP servers Jun 5, 2026
@PINKIIILQWQ PINKIIILQWQ changed the title feat(mcp): recursive descendant discovery and two-layer orphan reaper for shell-wrapper MCP servers fix(mcp): recursive descendant PID tracking for shell-wrapper MCP servers Jun 5, 2026
@alt-glitch alt-glitch added type/bug Something isn't working tool/mcp MCP client and OAuth P2 Medium — degraded but workaround exists labels Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Medium — degraded but workaround exists tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCP servers unavailable in cron sessions — ECONNREFUSED when accessing MCP tools from scheduled jobs

2 participants