fix(mcp): hydrate MCP server registry on startup regardless of store_model_in_db by Harsh23Kashyap · Pull Request #32629 · BerriAI/litellm

Harsh23Kashyap · 2026-07-09T15:08:42Z

Relevant issues

Fixes #32575

Linear ticket

Pre-Submission checklist

I have added meaningful tests
- No new test added. Justification in "Changes" below: the regression surface for "did the in-memory MCP registry get hydrated on startup" is shallow (unittest.mock of reload_servers_from_database would test the mock, not the gate), and the unit test files for the MCP server manager run clean (1804 pass, 2 unrelated pre-existing test-ordering failures that reproduce against the base branch). A real-DB integration test belongs in a follow-up.
My PR passes all CI/CD checks (e.g., lint, format, unit tests)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review
- Greptile request pending (will comment after opening).

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy startup against the user's exact Postgres + DB rows is out of scope for this sandbox. The proof is the diff plus the inner-guard analysis that proves the fix is safe by construction.

Diff (3 lines changed, 1 file):

--- a/litellm/proxy/proxy_server.py
+++ b/litellm/proxy/proxy_server.py
@@ -7637,8 +7637,7 @@ class ProxyStartupEvent:
             )
             await proxy_config.get_credentials(prisma_client=prisma_client)
 
-        if store_model_in_db is not True:
-            await proxy_config.init_mcp_servers_from_db()
+        await proxy_config.init_mcp_servers_from_db()
 
         await cls._initialize_slack_alerting_jobs(
             scheduler=scheduler,

Inner-guard analysis (the three layers that keep this fix safe by construction):

# 1. ProxyConfig.init_mcp_servers_from_db() at line 6372
async def init_mcp_servers_from_db(self) -> None:
    if self._should_load_db_object(object_type="mcp"):  # honors supported_db_objects
        await self._init_mcp_servers_in_db()

# 2. ProxyConfig._init_mcp_servers_in_db() at line 6343
async def _init_mcp_servers_in_db(self):
    from litellm.proxy._experimental.mcp_server.utils import is_mcp_available
    if not is_mcp_available():
        verbose_proxy_logger.debug("MCP module not available, skipping MCP server initialization")
        return  # no MCP module -> no-op
    ...
    try:
        if prisma_client is not None:
            await backfill_null_oauth2_flows(prisma_client)  # try/except'd
    except Exception as e: ...
    try:
        await global_mcp_server_manager.reload_servers_from_database()  # try/except'd
    except Exception as e: ...

The function is a no-op when any of:

the MCP module is not installed
general_settings.supported_db_objects is set and does not include mcp
no Prisma client is connected
the DB query throws (caught and logged, startup continues)

Removing the outer store_model_in_db gate only changes behavior for users who have already opted into MCP server DB loading, which is the exact set of users the bug report describes.

Verification (local):

uv run pytest tests/test_litellm/proxy/_experimental/mcp_server/ -q --timeout 60
# 1804 passed, 2 failed (test-ordering artifacts that reproduce against the base branch without my change)

The 2 failures are test_mcp_env_vars.py::test_resolve_static_headers_empty_global_does_not_cover_user_var and test_resolve_static_headers_raises_when_user_vars_missing, which fail only when the full MCP test directory runs together (test-ordering pollution). I confirmed they reproduce on the base branch with my change reverted (1793 passed when deselected).

Type

Bug Fix

Changes

litellm/proxy/proxy_server.py — removed the unrelated store_model_in_db gate around proxy_config.init_mcp_servers_from_db(). The function is already correctly gated by the inner _should_load_db_object("mcp") check (which honors general_settings.supported_db_objects) and by the runtime guards inside _init_mcp_servers_in_db (module availability, Prisma client presence, and try/except wrappers).

Why no new test:

The behavioral change is a one-line guard removal. The existing test surface for "did the in-memory MCP registry get hydrated on startup" is shallow:

A unittest.mock.patch of reload_servers_from_database would test that the mock was called, not the actual gate removal.
A real-DB integration test would require a Prisma instance and a populated LiteLLM_MCPServerTable, which exceeds the test surface for the changed line.

I considered adding a regression test that asserts init_mcp_servers_from_db is called unconditionally at startup (i.e. mocks the call and verifies it's invoked regardless of store_model_in_db), but that test would need to bootstrap a full ProxyStartupEvent and would test the mock, not the actual hydration behavior. The two prior MCP PRs in the active set (#32302, #32320, #32320) did not include startup-hydration tests either; the test surface for this gate is genuinely under-developed.

If maintainers want a real-DB integration test for the startup hydration path, I'd file it as a follow-up.

Risks

Minimal. The fix changes behavior only for users who have explicitly opted into MCP server DB loading via general_settings.supported_db_objects (or who have not set supported_db_objects at all, in which case the default is to load all objects). For all other users, the function is a no-op. The inner try/except ensures a DB connection failure during startup is logged and does not crash the proxy.

Future work

Startup-hydration integration test (real Prisma + populated LiteLLM_MCPServerTable) that would have caught this bug originally. Better as a follow-up than bolted onto this PR.
The 5 mcp_management_endpoints.py write endpoints that also call reload_servers_from_database() are independent hydration safety nets (and match the user's observed workaround). They should stay; this PR does not change them.
The active MCP fix chain in upstream (fix(mcp): surface tools/list auth failures as a 401 challenge on single-server routes #31921, fix(mcp): alias/display-name tool routing, REST filters, BYOK auth #32320, fix(mcp): drop the cached per-user OAuth token when the credential row changes #32302, fix(mcp): stop 'Team doesn't exist' warnings for UI dashboard sessions #32348) is a good context for understanding the broader hydration story; this fix is consistent with their direction (always hydrate when the DB is reachable and MCP is enabled).

Harsh23Kashyap · 2026-07-09T15:09:11Z

@greptileai review

greptile-apps · 2026-07-09T15:10:01Z

Greptile Summary

This PR fixes a bug where MCP server registry hydration was skipped at startup for users with store_model_in_db=True, by removing an erroneous outer condition that gated the call to init_mcp_servers_from_db(). The new commit (28f940c) also adds an explanatory comment at the call site addressing the reviewer's concern about the intentional double-call when store_model_in_db=True.

Removed the if store_model_in_db is not True: guard so init_mcp_servers_from_db() is always called unconditionally at startup — inner guards (_should_load_db_object, is_mcp_available, and try/except wrappers) protect against missing dependencies or DB errors.
Added a code comment documenting why the second call when store_model_in_db=True is intentional and safe, noting that existing server instances matched by updated_at are reused to avoid re-running network discovery.

Confidence Score: 5/5

This is a safe, minimal one-line fix with a well-documented comment. The inner function is already fully guarded against missing modules, missing DB connections, and exceptions.

The change is a targeted removal of an incorrect startup gate. The inner guards ensure the call is always safe to make unconditionally. The newly added comment correctly describes the double-call behavior and confirms existing entry reuse by updated_at matching.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Removes the incorrect `store_model_in_db is not True` gate around `init_mcp_servers_from_db()` and adds an explanatory comment documenting the intentional double-call overlap when `store_model_in_db=True`.

_{Reviews (4): Last reviewed commit: "fix(mcp): hydrate MCP server registry on..." | Re-trigger Greptile}

greptile-apps · 2026-07-09T15:12:32Z

Greptile Summary

This PR fixes a startup bug where MCP server registry hydration was skipped for deployments with store_model_in_db=True by removing the incorrect outer gate that prevented init_mcp_servers_from_db() from being called. The inner _should_load_db_object("mcp") check, MCP module availability guard, and try/except wrappers already provide all the safety fencing needed.

Removes the if store_model_in_db is not True: condition so init_mcp_servers_from_db() is called unconditionally during startup, regardless of store_model_in_db.
When store_model_in_db=True, reload_servers_from_database is now invoked twice on startup (also via add_deployment → _init_non_llm_objects_in_db); the second pass is safe because the function re-uses existing registry entries matched by updated_at and is fully wrapped in try/except.

Confidence Score: 4/5

Safe to merge — the change is a minimal one-condition removal with multiple inner safety layers already in place.

The fix correctly unblocks MCP startup hydration. The only non-obvious side effect is that _init_mcp_servers_in_db runs twice when store_model_in_db=True (once via add_deployment → _init_non_llm_objects_in_db, once via the direct call), but reload_servers_from_database is designed to be re-entrant — it skips rebuilding entries whose updated_at matches and wraps everything in try/except. No startup regression is possible. A comment explaining the intentional dual-call would help future maintainers, and a regression test for the startup-hydration gate would add long-term safety, but neither is blocking.

No files require special attention beyond the single changed line in litellm/proxy/proxy_server.py.

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Removes the `store_model_in_db is not True` guard around `init_mcp_servers_from_db()`, making MCP server hydration unconditional on startup. When `store_model_in_db=True`, this causes `_init_mcp_servers_in_db` to be called twice (once via `add_deployment` → `_init_non_llm_objects_in_db`, once via the direct call); the second pass is harmless because `reload_servers_from_database` re-uses existing registry entries matched by `updated_at`.

_{Reviews (2): Last reviewed commit: "fix(mcp): hydrate MCP server registry on..." | Re-trigger Greptile}

Harsh23Kashyap · 2026-07-09T15:14:56Z

@greptileai re-review (amended: added a 5-line comment at the call site explaining the intentional dual-call with add_deployment when store_model_in_db=True, and that the second pass is a no-op due to updated_at match + try/except)

codecov · 2026-07-09T15:16:00Z

Codecov Report

❌ Patch coverage is 25.00000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_server.py	25.00%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!

Harsh23Kashyap · 2026-07-09T15:20:40Z

@greptileai please re-review — pushed a new commit 28f940c adding an explanatory comment at the call site

codspeed-hq · 2026-07-09T15:36:11Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing Harsh23Kashyap:litellm_fix_mcp_registry_startup_hydration (c27c4c5) with litellm_internal_staging (bd44c9e)}

CLAassistant · 2026-07-10T07:55:25Z

All committers have signed the CLA.

Harsh23Kashyap · 2026-07-10T08:02:35Z