Skip to content

perf(mcp): lazy server startup from fingerprint-keyed schema cache (#56832 extract-salvage) - #77511

Merged
kshitijk4poor merged 3 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/56832-lazy-mcp
Aug 3, 2026
Merged

perf(mcp): lazy server startup from fingerprint-keyed schema cache (#56832 extract-salvage)#77511
kshitijk4poor merged 3 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/56832-lazy-mcp

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Extract-salvage of #56832 by @Vansh5632 — the PR's still-novel core (lazy MCP startup), with the schema-cache module cherry-picked under their authorship and the wiring re-derived onto main's current connect machinery.

Context — what this changes for users

Every desktop/dashboard start spawns EVERY configured stdio MCP child (Playwright, etc.) and fetches schemas, even for servers the session never touches — main's idle recycle only reclaims them after opt-in idle windows, never avoids the initial spawn. With mcp_servers..lazy: true (default OFF), a server whose config fingerprint matches a valid on-disk schema-cache entry registers its tools WITHOUT spawning; the child starts on first actual tool use.

Scope honesty — the full 4-group map of the original PR

  • Lazy MCP startup: extracted here (the PR's real value; main had nothing equivalent).
  • Runtime trim group: superseded by feat(mem): config-driven allocator trim with telemetry (#66355 salvage) #76905 (context.memory_trim — richer telemetry + lifecycle coverage).
  • Session-DB perf group: superseded on main (read-only opens, session_count_by_source, schema-v23 last_active SQL helpers).
  • mimalloc preload group: genuinely NOT covered by anything on main (allocator substitution at spawn ≠ runtime trim) but requires a maintainer security decision (allow-dyld-environment-variables entitlement + build-host mimalloc staging) and a CJS→ESM port — deliberately left out; flagged for human review.

Design (from #56832) + re-derivation notes

  • Registration-from-cache in register_mcp_servers; miss/stale falls back to eager connect with write-through refresh.
  • First tool use routes through _ensure_lazy_server_connected, composing with main's connect cooldown (A single failing stdio MCP server churns the whole MCP bridge — healthy servers' tools become "Unknown tool" at runtime #50394) and _server_connecting dedup — no duplicated connect path.
  • resource/prompt utility handlers (list_resources/get_prompt) also connect-on-first-use — closes the gap flagged in the original sweeper review.
  • check_fn stays a pure dict-membership: tools list as available pre-connect without triggering a spawn.

Review follow-ups folded (simplify pass)

  • Phantom-tool reconciliation: cached tools the live server no longer offers are deregistered after first-use connect (were permanent registry ghosts burning circuit-breaker strikes).
  • atomic_json_write + cache/ location + 0o600 (registry discovery-cache precedent); no secrets persisted (command/args/env only hashed into the fingerprint, never stored).
  • Cache-load path runs the same prompt-injection description scan as the eager path (the cache file is user-writable JSON).
  • Write-through skips byte-identical rewrites (flapping stdio servers no longer hammer the disk).

Verification

  • 444 mcp tests green (28 lazy/cache incl. 4 new guards); ruff clean.
  • Mutation checks: cache-read disabled → registration test fails; connect bypassed → 3 first-use tests fail; phantom-dereg disabled → its test fails; write-skip disabled → its test fails; all restored green.

Attribution mapping for Vansh5632 landed in #77492.

Closes #56832.

Vansh5632 and others added 3 commits August 3, 2026 13:39
Stores per-server tool manifests in ~/.hermes/mcp_schema_cache.json so
tools can be registered into the agent snapshot without spawning the
stdio child at startup. Entries are keyed by server name plus a
fingerprint of the connection-defining config (command/args/url/
transport/tool filters), so any config change invalidates the entry.

Extracted from NousResearch#56832.
…earch#56832)

Wires the fingerprint-keyed schema cache (previous commit, @Vansh5632's
design from NousResearch#56832) into the startup path, re-derived onto main's
current connect machinery:

- register_mcp_servers: servers with mcp_servers.<name>.lazy=true whose
  config fingerprint matches a valid cache entry register tools from
  cache WITHOUT spawning; miss/stale falls back to eager connect.
- First tool use routes through _ensure_lazy_server_connected, which
  composes with the connect cooldown (NousResearch#50394) and _server_connecting
  dedup rather than duplicating the connect path.
- resource/prompt utility handlers (list_resources/get_prompt) also
  connect-on-first-use — closes the gap flagged in the original
  sweeper review.
- Write-through: a live connect refreshes the cache entry.

Config gate is per-server, default OFF, matching the
idle_timeout_seconds key pattern. 24 lazy/cache tests + 440 mcp-wide
green; mutation-checked (cache-read disabled -> registration test
fails; connect bypassed -> 3 first-use tests fail).
Five review findings folded:
- schema cache writes via utils.atomic_json_write (fsync; was bare
  tmp+replace), file moved to cache/mcp_schema_cache.json with 0o600
  (sibling precedent: registry discovery cache)
- phantom-tool reconciliation: after a lazy server's first-use connect,
  cached tools the live server no longer offers are deregistered (were
  permanent registry ghosts burning circuit-breaker strikes on every
  'Unknown tool' round-trip); stale fingerprint logged
- cache-load path now runs _scan_mcp_description like the eager path
  (cache file is user-writable JSON; defense-in-depth)
- write-through skips the disk rewrite when the entry is unchanged
  (a flapping stdio server was rewriting byte-identical JSON per
  revival)
- _lazy_server_fingerprints no longer write-only dead state (consumed
  by the reconciliation logging)

444 mcp tests green (440 pre-fold + 4 new guards); phantom-dereg and
write-skip mutation-checked.
@kshitijk4poor
kshitijk4poor enabled auto-merge (rebase) August 3, 2026 08:46
@kshitijk4poor
kshitijk4poor merged commit ebf967f into NousResearch:main Aug 3, 2026
35 checks passed
@alt-glitch alt-glitch added type/perf Performance improvement or optimization P3 Low — cosmetic, nice to have tool/mcp MCP client and OAuth labels Aug 3, 2026
@kshitijk4poor
kshitijk4poor deleted the salvage/56832-lazy-mcp branch August 5, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low — cosmetic, nice to have tool/mcp MCP client and OAuth type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants