feat: auto-refresh MCP tools for slow servers via async late-binding

[397239396]

Problem

When the TUI or CLI starts, it waits only 0.75 seconds for all MCP servers to connect. Slow servers like lark (~8s), redis (~10s), and ssh (~4s) miss this window, so their tools never get registered with the agent.

Users had to manually /reload-mcp after startup — a poor experience.

main branch has a partial fix: _schedule_mcp_late_refresh() in tui_gateway/server.py. But it has several problems:

Problems with main's _schedule_mcp_late_refresh

1. TUI-only — lives inside server.py, CLI has no equivalent. CLI users with slow MCP servers are still stuck.

2. No thread safety — directly mutates agent.tools and agent.valid_tool_names without any lock. The main thread reads these during tool iteration; a concurrent write mid-iteration can produce inconsistent state.

3. No thread deduplication — spawns a new daemon thread per session. If multiple sessions are created rapidly, multiple background threads race to update the same agent.

4. Fragile tool-change detection — compares tool counts (len(new_defs) == len(agent.tools)), not tool names. If a slow server connects with 5 tools and the agent already has 5 from other servers, the refresh is silently skipped.

5. TUI thread not synced to mcp_startup — entry.py creates its own _mcp_discovery_thread but never publishes it to hermes_cli.mcp_startup._mcp_discovery_thread. Any shared module trying to check "is MCP discovery still running?" gets None, even though TUI discovery is active. This is a latent bug that blocks any future shared MCP infrastructure.

Solution

Shared two-phase async MCP discovery in hermes_cli/mcp_startup.py:

Phase	When	Behavior	Blocking
1 (startup)	At agent build time	Wait 0.75s for fast servers, build agent with available tools	0.75s
2 (background)	After agent created	Daemon thread waits for discovery to complete, then auto-merges new tools into the running agent	None

Changes

File	Change
hermes_cli/mcp_startup.py	Added spawn_late_mcp_refresh() — shared background thread that waits for _mcp_discovery_event, then merges tools from newly-connected servers into the agent. Thread-safe (_agent_tools_lock), deduplicates threads, detects changes by tool name not count.
hermes_cli/cli_agent_setup_mixin.py	Call spawn_late_mcp_refresh(agent) after CLI agent creation — CLI now also gets late-bound tools.
tui_gateway/server.py	Replace TUI-local _schedule_mcp_late_refresh with shared spawn_late_mcp_refresh + on_refreshed callback that emits session.info. Removes ~60 lines of duplicate TUI-only logic.
tui_gateway/entry.py	Bug fix: Sync TUI's _mcp_discovery_thread to mcp_startup._mcp_discovery_thread so shared code can see TUI discovery state. Without this, spawn_late_mcp_refresh would skip the background path and only do an inline check.
tests/hermes_cli/test_mcp_startup.py	183 lines of new tests: inline refresh, background thread, dedup, timeout, callback failure.
tests/test_tui_mcp_late_refresh.py	Deleted — replaced by the shared tests above.

Why this is better than main's approach

Aspect	main (_schedule_mcp_late_refresh)	This PR (spawn_late_mcp_refresh)
Scope	TUI only	CLI + TUI (shared)
Thread safety	No lock on agent.tools	_agent_tools_lock guards mutations
Thread dedup	None (one thread per session)	Single late-refresh thread, guarded by _mcp_discovery_lock
Change detection	Compares tool count	Compares tool name set
Discovery-already-done	No-op (skips entirely)	Inline refresh check right away
Callback system	Hardcoded _emit	Pluggable on_refreshed callback
Entry.py sync	Discovery thread invisible to shared code	Bug fix: publishes thread to mcp_startup

What this replaces

• ❌ _schedule_mcp_late_refresh in server.py — replaced by shared spawn_late_mcp_refresh
• ❌ HERMES_MCP_DISCOVERY_TIMEOUT environment variable — no longer needed

Benefits

• Startup is instant (~0.75s wait, no more)
• All tools are available — slow servers auto-merge once connected, no user action needed
• No configuration required — works out of the box
• Thread-safe — no race conditions on tool list mutation
• Works for both CLI and TUI — single implementation, consistent behavior
• Tested — 6 test scenarios covering inline, background, dedup, timeout, and error paths

[397239396]

Superseded by #48431 — resolved merge conflicts with latest main and squashed into a clean single commit.

[teknium1]

Thanks for the MCP late-binding work. An automated hermes-sweeper review found that the requested behavior is already implemented on current main by a stronger, cache-safe path.

• agent/turn_context.py:176-200 refreshes late-registered MCP tools before each turn's first API call, covering the CLI/TUI/gateway agent path without changing an in-flight cached prefix.
• tools/mcp_tool.py:5231-5350 provides the shared name-diffed, agent-scoped, additive-preserving, generation-safe tool snapshot refresh.
• tui_gateway/server.py:4370-4441 retains late TUI refresh only while the session is pre-first-turn; tui_gateway/entry.py:235-290 covers both stdio and desktop/dashboard discovery-thread owners.
• This landed through 93d6e730288e4ffab8076a0539f25e37a71f238f and follow-up hardening b6e2a54a94f58f9ebafa79f45d45b0ccb2b17043 (released in v2026.6.19).

The PR's own spawn_late_mcp_refresh() lacks the current pre-turn cache gate, so it would regress the repository rule against changing toolsets mid-conversation. The contributor's note that this was superseded by #48431 is also consistent with this review.