Skip to content

fix: wait for MCP discovery in one-shot mode before building agent (#68137) - #71277

Closed
rkfshakti wants to merge 1 commit into
NousResearch:mainfrom
rkfshakti:fix/oneshot-mcp-discovery-wait
Closed

fix: wait for MCP discovery in one-shot mode before building agent (#68137)#71277
rkfshakti wants to merge 1 commit into
NousResearch:mainfrom
rkfshakti:fix/oneshot-mcp-discovery-wait

Conversation

@rkfshakti

Copy link
Copy Markdown
Contributor

Problem

hermes -z (one-shot) snapshots the tool registry before background MCP discovery finishes. Fast servers (e.g. a compiled binary like mcp-grafana) usually make it in; slower stdio servers (a Python-based MCP that takes 2-12s to import/boot) are silently dropped — the agent runs without their tools, with no error anywhere.

Worse: since the model never sees the tools, it sometimes refuses the task and sometimes fabricates a plausible answer as if it had called the tool.

Root cause

  • hermes_cli/main.py: _should_background_mcp_startup returns True for command in {None, "chat", "rl"} → discovery runs in the cli-mcp-discovery background thread.
  • hermes_cli/oneshot.py: run_oneshot / _run_agent build AIAgent directly and never call hermes_cli.mcp_startup.wait_for_mcp_discovery() (unlike the interactive path in cli_agent_setup_mixin.py, which does).

Fix

Call wait_for_mcp_discovery() in _run_agent before constructing AIAgent, bounded by mcp_discovery_timeout so a dead server can't hang the one-shot. The wait is guarded with try/except so a discovery failure can't crash the one-shot run itself — mirroring the interactive path's resilience.

This mirrors the interactive path at cli_agent_setup_mixin.py.

Tests

Added tests/hermes_cli/test_oneshot_mcp_discovery.py with 2 test cases:

  • test_waits_before_building_agent — records call order and asserts wait_for_mcp_discovery is called before AIAgent construction. Fails on pre-fix code (agent built with no wait).
  • test_proceeds_when_discovery_wait_raises — asserts the agent is still built and a result returned when wait_for_mcp_discovery raises, proving the guard works.

I verified both tests fail on the pre-fix code and pass with the fix. Ruff is clean on the changed files.

Fixes #68137

…ousResearch#68137)

hermes -z (one-shot) snapshots the tool registry before background MCP
discovery finishes. Fast servers (e.g. compiled binaries) usually make
it in; slower stdio servers (Python-based MCP that takes 2-12s to boot)
are silently dropped — the agent runs without their tools, with no
error anywhere. Since the model never sees the tools, it sometimes
fabricates a plausible answer as if it had called the tool.

The interactive path (cli_agent_setup_mixin.py) already calls
wait_for_mcp_discovery() before constructing the agent. The one-shot
path (_run_agent in hermes_cli/oneshot.py) never called it.

Fix: call wait_for_mcp_discovery() in _run_agent before constructing
AIAgent, bounded by mcp_discovery_timeout so a dead server can't hang
the one-shot. The wait is guarded with try/except so a discovery failure
can't crash the one-shot run itself — mirroring the interactive path's
resilience.

Regression tests (tests/hermes_cli/test_oneshot_mcp_discovery.py):
- test_waits_before_building_agent: records call order and asserts the
  discovery wait happens before AIAgent construction (fails on pre-fix
  code where the agent is built with no wait).
- test_proceeds_when_discovery_wait_raises: asserts the agent is still
  built and a result returned when wait_for_mcp_discovery raises.
@rkfshakti

Copy link
Copy Markdown
Contributor Author

Reopening this fix as a fresh PR (the earlier #68168 was closed as a duplicate of #68162, but that one has stalled with no review activity and the bug is still live on main).

I've rebased on current main, hardened the wait with a try/except guard so a discovery failure can't crash the one-shot run, and rewrote the regression tests to patch collaborators at their source modules (the previous version patched them as attributes of hermes_cli.oneshot, which broke because _run_agent imports them locally). Both tests fail on the pre-fix code and pass with the fix; Ruff is clean.

Can you review once again?

@alt-glitch alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard tool/mcp MCP client and OAuth sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists labels Jul 25, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of open #68162: both wait for MCP discovery before one-shot AIAgent construction snapshots the tool registry, with the same bounded failure-tolerant behavior. #68162 is the earlier open implementation.

@rkfshakti

Copy link
Copy Markdown
Contributor Author

Thanks for the triage note — #68162 is the earlier fix for the same issue. Closing this in favor of the original PR. Apologies for the duplicate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

One-shot mode (-z) builds the agent without waiting for background MCP discovery — slow stdio servers are silently dropped

2 participants