feat(mcp): smart loading — connect_strategy, tool budget, per-session scoping, meta-tools

[prismatic7]

Summary

Four levers to control MCP server loading, reducing startup time and context-window bloat for users with many MCP servers. Closes umbrella issue #66473.

Changes

Lever 1: connect_strategy (per-server)

Three modes — startup (default, backward compatible), lazy (defer subprocess spawn until first call), on_demand (invisible until explicitly loaded).

Lever 2: mcp_tool_budget (global)

Caps MCP tool schemas in the LLM prompt. LRU eviction drops least-used servers' tools. Meta-tools always survive. Default: 0 (unlimited).

Lever 3: /mcp slash commands

/mcp list, /mcp enable <name>, /mcp disable <name>, /mcp load <name> — per-session server management. CLI-only (no gateway dispatcher yet).

Lever 4: Meta-tools (always available)

mcp_list_servers and mcp_load_server — agent-driven discovery and activation.

Bug fixes (discovered during audit)

1. Lazy server limbo — _activate_lazy_server removed from _lazy_servers before connecting; failed connections left unrecoverable
2. /mcp disable nuked everything — called _clear_session_loaded_servers() which cleared ALL on_demand servers, not just the named one
3. Stub tools invisible — lazy server stubs had no toolset alias registered
4. Meta-tools in wrong toolset — registered into "mcp" which doesn't exist in any platform composite

Maintainer review fixes (teknium1's hermes-sweeper)

1. Session isolation — _session_loaded_servers changed from a process-global set to Dict[session_id, set], keyed by a contextvars.ContextVar. Each asyncio task gets its own session identity, so one session's load/disable no longer affects every session.

2. Lazy stub deregistration — after _activate_lazy_server() succeeds, the mcp__activate_* stub tool is deregistered via registry.deregister(). A subsequent stub call no longer fails silently.

3. Agent tool refresh — added _refresh_active_agent_tools() helper that calls refresh_agent_mcp_tools() on the active agent after both lazy activation and on_demand load. Best-effort: if no active agent is found, tools are picked up on the next agent build (next-turn activation contract).

4. /mcp CLI-only guard — added cli_only=True to the /mcp CommandDef. Prevents gateway/TUI from trying to dispatch an unimplemented route.

Files modified

File	Changes
tools/mcp_tool.py	connect_strategy parsing, lazy/on_demand tracking, meta-tools, session isolation, stub deregistration, agent refresh (+54/-14)
model_tools.py	mcp_tool_budget enforcement, LRU tracking (+64)
toolsets.py	Meta-tools added to _HERMES_CORE_TOOLS (+4/-1)
hermes_cli/config.py	mcp_tool_budget default (+6)
hermes_cli/commands.py	/mcp slash command registration, CLI-only guard (+4)
cli.py	/mcp command handler (+104)

Backward Compatibility

• connect_strategy defaults to startup — existing configs unchanged
• mcp_tool_budget defaults to 0 (unlimited) — existing behavior preserved
• enabled: false still works as before
• idle_timeout_seconds / max_lifetime_seconds still work for recycling connected servers
• Session identity contextvar defaults to "" — sessions that don't set it share a global namespace (backward compatible)

Related Issues

• Feature: MCP Smart Loading — Lazy Connection, Tool Budget, Per-Session Scoping & On-Demand Servers #66473 (umbrella)
• Feature: support 'lazy: true' for MCP servers (connect on first tool call, not at startup) #63626 (lazy MCP)
• Feature: Lazy Tool Schema Loading — Two-Pass Tool Injection to Reduce Token Overhead #6839 (tool schema deferral)
• ACP: honor --toolsets and add per-session tool scoping (_make_agent hardcodes enabled_toolsets) #45955 (per-session scoping)

[teknium1]

Thanks for tackling the MCP startup and schema-footprint problem. This needs revision before it can safely land.

Problems

• tools/mcp_tool.py:447-501 calls the state per-session, but _session_loaded_servers is one process-global set keyed only by server name. _make_check_fn() then uses it for visibility, so one session's load/disable operation changes every session's MCP surface.
• tools/mcp_tool.py:5188 activates the lazy server, while :520-522 removes it from _lazy_servers; the mcp__activate_* stub registered at :5198-5206 is never deregistered. A later stub call therefore fails instead of being replaced as the docstring promises.
• tools/mcp_tool.py:6271-6275 says the loaded tools are now available but does not refresh the active agent's snapshot. The existing helper documents that agent.tools is snapshotted once and that refresh callers own the cache contract (tools/mcp_tool.py:5524-5556).
• hermes_cli/commands.py:217-219 makes /mcp gateway-visible, but gateway/run.py has no matching dispatcher branch.

Suggested changes

• Key visibility by real session identity and add multi-session isolation tests.
• Atomically replace/deregister the lazy stub after activation.
• Define a cache-safe next-turn activation contract or add an explicitly consented refresh path.
• Make /mcp CLI-only or implement its gateway/TUI routes and tests.

Automated hermes-sweeper review.

[prismatic7]

All resolved. Thanks for the feedback!

[carljborg]

Strong +1 on connect_strategy: lazy from a live deployment — and a use case that I think sharpens the motivation beyond "many servers": MCP servers backed by a slow/serverless upstream.

We run an agentmail-mcp stdio bridge whose upstream is a hosted, serverless MCP endpoint (mcp.agentmail.to) that cold-starts in ~7–18s and occasionally times out entirely. Under the current eager startup strategy it fails the initial handshake, burns its 3 retries, and parks — silently disabling the email tools for the whole session, even though the endpoint answers fine a few seconds later when actually used. (It also happened to trip the kanban-reaper race #63728 in the gateway, but that's orthogonal — even with that fixed it still parks, because a cold serverless endpoint just can't complete a handshake inside the startup window.)

connect_strategy: lazy is the correct fix here: register the stub tools at startup so the model can call them, and only pay the connection cost on first use — when the endpoint is far more likely to be warm, and a slow/failed connect degrades a single tool call instead of killing the whole server for the session. That's the difference between "email tools dead all session" and "email tools work when the user asks for email."

So this lever isn't only a startup-time / context-footprint optimization for large fleets — it's arguably the right default handling for any MCP server whose upstream is on-demand/serverless. Thanks for building it; happy to test a revised revision against our (reliably-cold) setup if that's useful for the needs-decision.

[teknium1]

Thanks for the thorough audit and the four-lever build here — real effort went into this, and the bug fixes you surfaced along the way (lazy-server limbo, /mcp disable over-clearing, stub deregistration) are genuinely useful findings.

Closing this without merging, because the design has diverged from where we're taking MCP loading:

Context bloat (Levers 2, 3, 4 — tool budget, /mcp session scoping, meta-tools): already solved on main by the tool_search progressive-disclosure system (tools/tool_search.py, config tools.tool_search, auto mode). When deferrable MCP/plugin schemas exceed the context threshold they're replaced with tool_search / tool_describe / tool_call bridge tools and surfaced on demand. That fully covers the schema-footprint problem the tool budget + LRU eviction + mcp_list_servers/mcp_load_server meta-tools were built for, without the per-session registry-visibility toggling (which is cache- and session-state-hostile — the sweeper flagged exactly that).

Startup time (Lever 1 — connect_strategy: lazy): we're going a different direction than lazy. Rather than deferring connection to first tool call (which trades startup cost for a slow/park-prone first call), the plan is to keep enabled MCP servers always connected but owned by the gateway process instead of the per-session agent — pay the npx/handshake cost once at gateway boot, and have every session reuse the already-live connections. That eliminates per-session respawn cost without the lazy tradeoff.

I've re-scoped the umbrella issue #66473 to track that gateway-attached-persistent-MCP design.

Your commits keep their authorship in your branch. If you want to take a run at the gateway-attached approach, #66473 is the place — happy to have you on it.