feat(mcp): add config-gated lazy MCP schema loading - #33052
Closed
Interstellar-code wants to merge 1 commit into
Closed
feat(mcp): add config-gated lazy MCP schema loading#33052Interstellar-code wants to merge 1 commit into
Interstellar-code wants to merge 1 commit into
Conversation
Constraint: upstream already tracks broader lazy-loading variants in NousResearch#6839 and NousResearch#13332, so this PR is intentionally scoped to the smallest plugin-first Phase 1 slice: request-time stubs, explicit promotion, and per-session state only. Rejected: Include baseline telemetry and reporting | expands scope beyond the core lazy-loading loop and adds unrelated review surface Rejected: Include server-level discovery and eager-threshold behavior | better handled as a follow-up once the core stub/promotion loop lands Confidence: high Scope-risk: narrow Directive: Keep future extensions (server discovery, eager promotion, telemetry) layered on top of this Phase 1 contract instead of broadening the initial API semantics. Tested: scripts/run_tests.sh tests/plugins/mcp_lazy/test_stubs.py tests/plugins/mcp_lazy/test_pool.py tests/plugins/mcp_lazy/test_promote_and_hook.py tests/hermes_cli/test_config_validation.py Not-tested: Full Hermes suite; live TPM/token measurements against a large MCP installation on upstream main
This was referenced Jun 10, 2026
12 tasks
Contributor
|
Thanks for the focused MCP-heavy-install proposal. This is an automated hermes-sweeper review; current
Closing as implemented on main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #6839
Summary
This PR adds a narrow, config-gated Phase 1 implementation of lazy MCP schema loading to reduce per-turn tool-schema token overhead for MCP-heavy installs.
Scope in this PR:
This PR is intentionally limited to the core lazy-loading loop. It does not include later expansion ideas like server-level discovery, eager server promotion, or broader telemetry/reporting.
Problem
Large MCP installations inject hundreds of full tool schemas into every API call, even when the conversation does not need them. This creates large fixed prompt overhead and can block lower-TPM providers entirely.
Approach
When
mcp.lazy_loadingis enabled:load_mcp_toolsto request full schemas for selected toolsThe hook is plugin-based and fail-open:
transform_toolsrewrites the outgoing tool list just before API dispatchBackward compatibility
Included config keys
mcp.lazy_loadingmcp.lazy_stub_max_descNot included in this PR
Tests
Ran the narrowed Phase 1 suite:
tests/plugins/mcp_lazy/test_stubs.pytests/plugins/mcp_lazy/test_pool.pytests/plugins/mcp_lazy/test_promote_and_hook.pytests/hermes_cli/test_config_validation.pyCommand: