Repository navigation
feat(mcp)!: disable stdio MCP servers by default - #44066
Conversation
stdio MCP servers now only run when the proxy is started with LITELLM_ENABLE_MCP_STDIO=true. While it is off, existing stdio servers stay registered but never start: tool listings skip them quietly, direct tool calls and health checks return a 403 naming the env var, and creating or updating a stdio server is rejected. The flag is read from the process environment only, so DB-stored environment_variables cannot turn it on. The UI reads mcp_stdio_enabled from /.well-known/litellm-ui-config to grey out the stdio transport, show a banner on stdio forms, and badge stdio server cards. BREAKING CHANGE: stdio MCP servers are off by default. Set LITELLM_ENABLE_MCP_STDIO=true in the proxy environment and restart to keep using them.
|
Docs for this change, covering the new LITELLM_ENABLE_MCP_STDIO env var and the upgrade note: BerriAI/litellm-docs#1991 |
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…e selected worker LITELLM_ENABLE_MCP_STDIO set under environment_variables in config.yaml is now skipped like the DB-stored value, so only the process environment can enable stdio. The dashboard reads mcp_stdio_enabled from the proxy it is managing, so a control plane shows each worker's own setting.
|
@greptileai please re-review: the config.yaml path and the worker-aware UI flag are fixed in the latest commits |
…p the UI unchanged until the flag loads Prompt, resource and resource-template listings now skip a blocked stdio server at debug level like tool listing does, instead of logging a warning per server on every call. The dashboard only treats stdio as disabled once the proxy explicitly reports mcp_stdio_enabled false, so a proxy with the flag on, or an older one without the field, renders exactly as before with no flicker while loading.
…nce per server A gateway tools/call naming a blocked stdio server's tool now returns the LITELLM_ENABLE_MCP_STDIO message instead of "Tool not found". The "will not start" warning moves out of build_mcp_server_from_table, which DB reload re-runs on every cycle for rows with a NULL updated_at and which drafts and test-connection also call. It now fires when a row first enters the registry or changes transport.
The Overview and MCP Tools tabs showed "No tools available" with no reason while stdio is disabled. The detail page now shows the same warning banner as the edit form, and hands off to the form's banner once editing starts.
|
@greptileai please re-review: blocked stdio tool calls now return the flag error, and the startup warning fires once per server |
Keeps local/no-long-condition-chain within its budget
|
QA PASS at tip At the merge base stdio starts unconditionally. At the tip with the flag unset, stdio servers stay listed but are marked disabled, return no tools, calls get 403, create and update get 422, and HTTP MCP is unaffected. Setting
Both instances of each leg returned identical results After, flag unset (instance A1, A2 identical)After, flag set in process envBefore (merge base, instance B1, B2 identical)Config file and DB values are ignoredInstance started with the flag unset in env and Value written to the DB through the API, checked after a 70s sync on both instances Mixed fleet: A1 with flag, A2 without, same DBDashboard built from the tip. Flag unset: stdio servers show a Two non-blocking observations. With the flag unset, Responses, Chat Completions and Messages requests that reference a stdio server succeed with an empty tool list, so the client only sees the model saying it has no tool rather than an explicit error. The DB ignore warning is logged on every DB config sync per worker (27 lines in about 3 minutes on one instance), which may be noisy in production logs |
…g once The DB config sync re-reads environment_variables on every cycle, so a stored flag logged the warning on each sync per worker
|
Thanks. 077e1df now logs the DB warning once per process. Empty tool lists for blocked stdio servers are the intended quiet behavior |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 077e1df. Configure here.
* feat(mcp)!: disable stdio MCP servers by default (#44066) * feat(mcp)!: disable stdio MCP servers by default stdio MCP servers now only run when the proxy is started with LITELLM_ENABLE_MCP_STDIO=true. While it is off, existing stdio servers stay registered but never start: tool listings skip them quietly, direct tool calls and health checks return a 403 naming the env var, and creating or updating a stdio server is rejected. The flag is read from the process environment only, so DB-stored environment_variables cannot turn it on. The UI reads mcp_stdio_enabled from /.well-known/litellm-ui-config to grey out the stdio transport, show a banner on stdio forms, and badge stdio server cards. BREAKING CHANGE: stdio MCP servers are off by default. Set LITELLM_ENABLE_MCP_STDIO=true in the proxy environment and restart to keep using them. * fix(mcp): ignore stdio flag from config file and read UI flag from the selected worker LITELLM_ENABLE_MCP_STDIO set under environment_variables in config.yaml is now skipped like the DB-stored value, so only the process environment can enable stdio. The dashboard reads mcp_stdio_enabled from the proxy it is managing, so a control plane shows each worker's own setting. * test(mcp): cover non-mapping payloads in the shared transport validator * fix(mcp): skip blocked stdio servers quietly in every listing and keep the UI unchanged until the flag loads Prompt, resource and resource-template listings now skip a blocked stdio server at debug level like tool listing does, instead of logging a warning per server on every call. The dashboard only treats stdio as disabled once the proxy explicitly reports mcp_stdio_enabled false, so a proxy with the flag on, or an older one without the field, renders exactly as before with no flicker while loading. * fix(mcp): route blocked stdio tool calls to the flag error and warn once per server A gateway tools/call naming a blocked stdio server's tool now returns the LITELLM_ENABLE_MCP_STDIO message instead of "Tool not found". The "will not start" warning moves out of build_mcp_server_from_table, which DB reload re-runs on every cycle for rows with a NULL updated_at and which drafts and test-connection also call. It now fires when a row first enters the registry or changes transport. * fix(ui): explain on the server detail page why a stdio server is inert The Overview and MCP Tools tabs showed "No tools available" with no reason while stdio is disabled. The detail page now shows the same warning banner as the edit form, and hands off to the form's banner once editing starts. * refactor(ui): name the stdio banner conditions on the server detail page Keeps local/no-long-condition-chain within its budget * fix(proxy): log the ignored DB-stored LITELLM_ENABLE_MCP_STDIO warning once The DB config sync re-reads environment_variables on every cycle, so a stored flag logged the warning on each sync per worker (cherry picked from commit 2b85808) * chore: update Next.js build artifacts (2026-10-02 17:40 UTC, node v24.19.0)
* feat(mcp)!: disable stdio MCP servers by default (#44066) * feat(mcp)!: disable stdio MCP servers by default stdio MCP servers now only run when the proxy is started with LITELLM_ENABLE_MCP_STDIO=true. While it is off, existing stdio servers stay registered but never start: tool listings skip them quietly, direct tool calls and health checks return a 403 naming the env var, and creating or updating a stdio server is rejected. The flag is read from the process environment only, so DB-stored environment_variables cannot turn it on. The UI reads mcp_stdio_enabled from /.well-known/litellm-ui-config to grey out the stdio transport, show a banner on stdio forms, and badge stdio server cards. BREAKING CHANGE: stdio MCP servers are off by default. Set LITELLM_ENABLE_MCP_STDIO=true in the proxy environment and restart to keep using them. * fix(mcp): ignore stdio flag from config file and read UI flag from the selected worker LITELLM_ENABLE_MCP_STDIO set under environment_variables in config.yaml is now skipped like the DB-stored value, so only the process environment can enable stdio. The dashboard reads mcp_stdio_enabled from the proxy it is managing, so a control plane shows each worker's own setting. * test(mcp): cover non-mapping payloads in the shared transport validator * fix(mcp): skip blocked stdio servers quietly in every listing and keep the UI unchanged until the flag loads Prompt, resource and resource-template listings now skip a blocked stdio server at debug level like tool listing does, instead of logging a warning per server on every call. The dashboard only treats stdio as disabled once the proxy explicitly reports mcp_stdio_enabled false, so a proxy with the flag on, or an older one without the field, renders exactly as before with no flicker while loading. * fix(mcp): route blocked stdio tool calls to the flag error and warn once per server A gateway tools/call naming a blocked stdio server's tool now returns the LITELLM_ENABLE_MCP_STDIO message instead of "Tool not found". The "will not start" warning moves out of build_mcp_server_from_table, which DB reload re-runs on every cycle for rows with a NULL updated_at and which drafts and test-connection also call. It now fires when a row first enters the registry or changes transport. * fix(ui): explain on the server detail page why a stdio server is inert The Overview and MCP Tools tabs showed "No tools available" with no reason while stdio is disabled. The detail page now shows the same warning banner as the edit form, and hands off to the form's banner once editing starts. * refactor(ui): name the stdio banner conditions on the server detail page Keeps local/no-long-condition-chain within its budget * fix(proxy): log the ignored DB-stored LITELLM_ENABLE_MCP_STDIO warning once The DB config sync re-reads environment_variables on every cycle, so a stored flag logged the warning on each sync per worker (cherry picked from commit 2b85808) * chore(deps): bump pyjwt to 2.15.0 * chore(deps): bump pypdf to 6.19.0 * chore(deps): bump tornado to 6.5.9 * chore(deps): bump urllib3 to 2.8.0 * chore(deps): bump gitpython to 3.1.62 * chore: update Next.js build artifacts (2026-10-02 18:00 UTC, node v24.19.0)
TLDR
Problem this solves:
How it solves it:
LITELLM_ENABLE_MCP_STDIO, off by defaultIntentional product change: proxies that already run stdio MCP servers stop starting them after upgrade until the admin sets
LITELLM_ENABLE_MCP_STDIO=trueand restarts. Users lose those servers' tools in the meantime, and the proxy logs one warning per server naming the env varUser Flow
Before: an admin with a stdio MCP server in config.yaml upgrades and everything keeps working, with no way to opt out of stdio
["echo"]echo_stdio-echoreturnsecho: hi"transport": "stdio"creates a new stdio serverAfter: the same admin upgrades without setting the env var, and stdio stays off until they opt in
MCP server 'echo_stdio' will not start: stdio MCP servers are disabled on this proxy. Set LITELLM_ENABLE_MCP_STDIO=true on the proxy and restart to enable them[], and POST /mcp-rest/tools/call returns 403 with the message"transport": "stdio"returns 422 with the message, and the Create/Edit forms grey out stdioLITELLM_ENABLE_MCP_STDIO=true, restart, and steps 1 to 4 of Before work againServer cards with the flag off (badges and tooltip boxed):
Edit form transport select with the flag off, stdio greyed out with the tooltip:
Edit page of an existing stdio server with the flag off:
Create form prefilled from a stdio catalog entry with the flag off:
Detail page of an existing stdio server with the flag off, which used to say only "No tools available":
Same page with
LITELLM_ENABLE_MCP_STDIO=true, which looks the same as before this PR (no badges, stdio server healthy):Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Setup: a local proxy on port 4071 with Postgres and
STORE_MODEL_IN_DB=True, launched underenv -iso only the listed env reaches it.echo_stdiois a stdio server in config.yaml (python echo_server.py, a one-tool MCP server that echoes its input) with idebabfe6d53267166b95d507599c04f10.new_stdiowas created through the API in the Before run.KEYis the proxy master key. No LLM provider is involved, since the change is in MCP server handlingThe UI screenshots for each case are embedded under User Flow above
Before (a76b59d)
stdio server works
curl -s 'localhost:4071/mcp-rest/tools/list?server_id=ebabfe6d53267166b95d507599c04f10' -H "Authorization: Bearer $KEY" | jq -c '[.tools[].name]'curl -s localhost:4071/mcp-rest/tools/call -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"server_id":"ebabfe6d53267166b95d507599c04f10","name":"echo_stdio-echo","arguments":{"text":"hi"}}' | jq -c '.content'creating a stdio server
curl -s localhost:4071/v1/mcp/server -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"server_name":"new_stdio","transport":"stdio","command":"python","args":["echo_server.py"]}' | jq -c '{server_name,transport}'UI config
curl -s localhost:4071/.well-known/litellm-ui-configflag stored in the database
flag set in config.yaml
After (ac4a835)
The cases below ran at ac4a835. The full end-to-end run at 4351da2 is at the end of this section and covers them again. Later commits: 94712b1 names the detail-page banner condition for a lint budget, and 077e1df logs the "stored in the database" warning once per process instead of on every DB config sync (found by Devin QA)
stdio server works
Flag off (env var unset):
curl -s 'localhost:4071/mcp-rest/tools/list?server_id=ebabfe6d53267166b95d507599c04f10' -H "Authorization: Bearer $KEY" | jq -c '[.tools[].name]'curl -s -w ' HTTP %{http_code}' localhost:4071/mcp-rest/tools/call -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"server_id":"ebabfe6d53267166b95d507599c04f10","name":"echo_stdio-echo","arguments":{"text":"hi"}}'curl -s localhost:4071/v1/mcp/server/ebabfe6d53267166b95d507599c04f10 -H "Authorization: Bearer $KEY" | jq -c '{server_name,status,health_check_error}'Flag on (
LITELLM_ENABLE_MCP_STDIO=true), no "will not start" lines in the boot log:creating a stdio server
curl -s -w ' HTTP %{http_code}' localhost:4071/v1/mcp/server -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"server_name":"stdio_off","transport":"stdio","command":"python","args":["echo_server.py"]}'"server_name":"stdio_on"UI config
curl -s localhost:4071/.well-known/litellm-ui-config | jq -c '{mcp_stdio_enabled}'flag stored in the database
curl -s -w ' HTTP %{http_code}' localhost:4071/config/update -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' -d '{"environment_variables":{"LITELLM_ENABLE_MCP_STDIO":"true"}}'curl -s localhost:4071/.well-known/litellm-ui-config | jq -c '{mcp_stdio_enabled}', and again after a restartHTTP 403with the message, and the proxy log showsWithout the database guard, the restart in step 2 returned
{"mcp_stdio_enabled":true}, which is why it is part of this PRflag set in config.yaml
environment_variablescurl -s localhost:4071/.well-known/litellm-ui-config | jq -c '{mcp_stdio_enabled}'gateway call to a blocked stdio tool (flag off, 4351da2)
tools/callon/mcp/namingcanary_cfg-echo, afterinitializeError: Tool 'echo' not foundDB reload with a row whose updated_at is NULL (flag off, 4 workers, 4351da2)
psql -c "update \"LiteLLM_MCPServerTable\" set updated_at=NULL where server_name='canary_db'", boot with--num_workers 4, wait 3 minutesgrep "will not start" proxy.log | grep -oE "server '[a-z_]+'" | sort | uniq -ccanary_dblogged 70 times in 9.5 minutesfull end-to-end run (4351da2)
What ran and what came back
The stdio test server appends a line to a log file every time it starts, so a spawn count per case shows whether stdio ever ran. A script of 47 customer cases (list, get, create, update, import, test connection, health, REST tools list and call, MCP gateway tools, prompts and resources on
/mcp/and per-server paths,/v1/responses,/v1/chat/completionsand/v1/messageswithlitellm_proxyMCP tools against gpt-5.4-mini and claude-haiku-4-5) ran against three proxies, each on a fresh Postgres database:mainat the merge base, this branch with the flag on, and this branch with the flag offFlag on vs
main: every response is identical exceptmcp_stdio_enabledin ui-config (nullon main,truehere). Spawn counts match except two adjacent cases where a background health check moved 3 spawns from one case to the next (6 in total on both). The${X-CANARY-TOKEN}header reaches the stdio env the same way on bothFlag off: 0 spawns across all 47 cases. Every stdio create, update, import and test connection is a 422 with the message, REST calls are a 403, gateway listings leave stdio out, gateway calls return the message, and HTTP servers and the three LLM endpoints behave the same as with the flag on
Transitions on one database: restarting on, off, then on again leaves the MCP rows byte-identical and stdio works again with no repair. With an on and an off proxy sharing the database, the off proxy picks up a stdio server created on the on proxy within 48 seconds, refuses calls to it with 0 spawns and warns once. Booting
mainon that database afterwards runs both stdio servers. With 4 workers and the flag off, 40 of 40 ui-config reads sayfalse, 60 of 60 calls are 403 and nothing spawnsAccess: a key allowed only a stdio server sees no tools and gets the message on a gateway call. A key allowed a stdio and an HTTP server sees only the HTTP tools
Flag sources: only the process env (including a
.envfile next to the proxy) turns stdio on. config.yaml (direct, lowercase key,os.environ/reference) and/config/update(at runtime and after a restart, either case) do not. Valuestrue,TRUE,trueandtrue\nturn it on; unset, empty,false,0,1,yes,onandTrue1do notThe SDK
litellm.experimental_mcp_clientstill starts stdio servers with the flag unset. With the flag on, the MCP Servers page was sampled every 25 ms for the first 10 seconds of load (352 samples) and never showed any disabled textUnit suites for the MCP directories on
mainand this branch, with the flag on and off, show no outcome differences (0 failures in all four runs)Type
🆕 New Feature
Caveats (if any)
Severe
LITELLM_ENABLE_MCP_STDIO=trueon the proxy and restartLow
true(any case, trimmed) enables stdio;1andyesdo not/v1/mcp/server/healthendpoint returns status only, the reason is on/v1/mcp/server/{id}.envfile next to the proxy counts as the process env and can turn stdio on/v1/responses,/v1/chat/completionsand/v1/messagesrequests that point at a blocked stdio server succeed with no tools offered, so the client sees the model say it has no tool rather than an error/v1/mcp/server/registeris now refused with the flag message instead of "use POST /v1/mcp/server"server_outcomesin the gateway's list meta reports them as oklitellm.experimental_mcp_client(SDK) is not gated, only the proxy istests/mcp_testsand the integration suite now enable the flag; onlytests/mcp_testswas run locally (30 passed)Final Attestation
Note
High Risk
Breaking change: upgrades disable local subprocess MCP unless admins set
LITELLM_ENABLE_MCP_STDIO=trueand restart; touches security-sensitive command execution and broad MCP/API/UI surfaces.Overview
stdio MCP servers are now opt-in via process env
LITELLM_ENABLE_MCP_STDIO=true(default off). When disabled, stdio servers remain registered but do not spawn processes: tool/prompt/resource listings return empty, tool calls return 403, create/update validation fails, and boot/reload logs a one-time warning per server.Proxy hardening:
LITELLM_ENABLE_MCP_STDIOcannot be turned on throughconfig.yamlenvironment_variablesor DB-stored env vars—only the proxy process environment counts. UI discovery (/.well-known/litellm-ui-config) addsmcp_stdio_enabledso the dashboard can grey out stdio transport, show banners/badges, and explain how to enable it.CI/tests: Integration and MCP e2e runs set the flag to
trueso existing stdio coverage keeps working; new unit/UI tests cover the off path and ignored flag sources.Reviewed by Cursor Bugbot for commit 077e1df. Bugbot is set up for automated code reviews on this repo. Configure here.