fix(mcp): stop preemptively 401-challenging client_credentials (M2M) MCP servers

[tin-berri]

Relevant issues

Ships the product fix that PR #33263 documents as a known failure against current main: its test_gateway_exchanges_client_credentials_and_sends_minted_token e2e test is the protocol-level regression guard for this bug and goes green once this lands

Linear ticket

Part of LIT-4263 (migrate client_credentials M2M into v2); this ships the connect-time gate fix that track requires

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Rig: a local stub serving an OAuth token endpoint (client_credentials grant, counts every mint) plus an MCP streamable-HTTP mount that 401s any request whose bearer is not a token that endpoint minted. A served MCP call is therefore proof the gateway ran the grant and attached the minted token. Proxy runs against local Postgres with --port 4001

Register the M2M server the way a production host does:

curl -s -X POST http://127.0.0.1:4001/v1/mcp/server -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{
  "alias": "m2mrig", "url": "http://127.0.0.1:8765/mcp", "transport": "http", "allow_all_keys": true,
  "auth_type": "oauth2", "oauth2_flow": "client_credentials", "token_url": "http://127.0.0.1:8765/token",
  "credentials": {"client_id": "m2m-rig-client-id", "client_secret": "m2m-rig-client-secret"}}'

{"server_id": "e314c2f6-43a3-43d8-8231-0206615fdf65", "alias": "m2mrig", "auth_type": "oauth2", "oauth2_flow": "client_credentials", ...}

Before the fix, connecting over the MCP protocol path is dead on arrival on both routes; the gateway pushes the client into an interactive OAuth flow it can never complete, and the stub IdP records zero mint attempts:

curl -si -X POST http://127.0.0.1:4001/m2mrig/mcp -H "x-litellm-api-key: Bearer sk-1234" \
  -H "Content-Type: application/json" -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{...}}'

HTTP/1.1 401 Unauthorized
www-authenticate: Bearer authorization_uri="http://127.0.0.1:4001/.well-known/oauth-authorization-server/m2mrig"

curl -si -X POST http://127.0.0.1:4001/mcp/ -H "x-litellm-api-key: Bearer sk-1234" -H "x-mcp-servers: m2mrig" ...same initialize...

HTTP/1.1 401 Unauthorized
www-authenticate: Bearer authorization_uri="http://127.0.0.1:4001/.well-known/oauth-authorization-server/m2mrig"

After the fix, the same commands complete the full flow. initialize returns a session, tools list, a tool call round-trips, and the stub proves the gateway minted exactly one token and attached it (the caller's virtual key never crossed the gateway boundary):

curl -si -X POST http://127.0.0.1:4001/m2mrig/mcp ...same initialize...
HTTP/1.1 200 OK
mcp-session-id: a47f24bfebdb4f19928b50e66bb63a65

curl -s ... -d '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}'
"name":"m2mrig-echo"
"name":"m2mrig-last_auth"

curl -s ... -d '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"m2mrig-echo","arguments":{"text":"live-m2m-proof"}}}'
data: {"jsonrpc":"2.0","id":3,"result":{"content":[{"type":"text","text":"echo:live-m2m-proof"}],...,"isError":false}}

curl -s http://127.0.0.1:8765/stats
{"mints":2,"last_mcp_auth":"Bearer m2m-token-2","denied":12}   # was mints:1 before the flow; exactly one gateway mint

curl -s -X POST http://127.0.0.1:4001/mcp/ -H "x-mcp-servers: m2mrig" ...initialize... -w "%{http_code}"
200                                                            # aggregate route; mints stays 2, cached token reused

The interactive flow is untouched; a per-user oauth2 server with no stored token still gets the challenge:

curl -si -X POST http://127.0.0.1:4001/interactiverig/mcp ...same initialize...
HTTP/1.1 401 Unauthorized
www-authenticate: Bearer authorization_uri="http://127.0.0.1:4001/.well-known/oauth-authorization-server/interactiverig"

Legacy rows are covered too. Nulling the stored flow on an M2M-shape row (UPDATE "LiteLLM_MCPServerTable" SET oauth2_flow = NULL WHERE alias = 'm2mlegacy') and restarting the proxy, initialize on /m2mlegacy/mcp returns 200 instead of the spurious challenge

Type

🐛 Bug Fix

Changes

_raise_preemptive_401_for_unauthenticated_servers in litellm/proxy/_experimental/mcp_server/server.py raised the interactive OAuth 401 for every oauth2 server that received no per-request oauth header. The interactive logic (delegate challenge, stored-token check) was correctly nested under needs_user_oauth_token, but the raise sat at the outer level, so a client_credentials (M2M) server, for which the gateway mints its own upstream token and no user flow exists, fell through into a challenge it can never satisfy. Both the /{alias}/mcp route and the /mcp aggregate go through this function, so M2M servers were unusable over the MCP protocol path

Two changes, one invariant: a server whose credential mode requires no user participation is never preemptively challenged for user auth, and the challenge classifies a server's flow exactly as egress does. First, the needs_user_oauth_token branch is inverted into an early continue, so the 401 raise is only reachable for flows that actually need a user token. Second, the loop now applies MCPServerManager.resolve_oauth2_flow_for_request right after resolving the server, the same request-time backstop the listing and tool-call paths already apply, so a legacy null-flow row with the M2M field shape is treated as M2M at the challenge gate instead of being pushed into a user OAuth flow that egress would never look for

The regression test (test_client_credentials_server_is_not_preemptively_challenged in tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_stale_session.py) drives handle_streamable_http_mcp with a real MCPServer object, parametrized over a stamped oauth2_flow="client_credentials" row and an unstamped M2M-shape row; both must reach the session manager without the per-user token store ever being consulted. Both cases fail on the unfixed code (the stamped case with the exact 401 from the bug, the unstamped case by consulting the per-user store) and pass with the fix. The pre-existing tests pinning the interactive, delegate, and passthrough challenges all still pass

QA runbook

1. Start a proxy with a Postgres-backed config and register an MCP server with auth_type: oauth2, oauth2_flow: client_credentials, a token_url, and client credentials for a real M2M upstream (any IdP-guarded MCP server works)
2. As a regular virtual key, POST an MCP initialize to /{alias}/mcp with x-litellm-api-key: Bearer <key>; expect HTTP 200 and an mcp-session-id header, not a 401 with a gateway authorization_uri
3. Complete the handshake (notifications/initialized), then tools/list and a tools/call; both must succeed and the upstream must receive the gateway-minted bearer, never your virtual key
4. Repeat step 2 against the aggregate /mcp/ route with x-mcp-servers: <alias>; expect 200
5. Register a second oauth2 server with oauth2_flow: authorization_code and repeat step 2 against it without authorizing first; expect the 401 challenge with authorization_uri (interactive flow unchanged)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR fixes a bug where the connect-time 401 challenge gate in _raise_preemptive_401_for_unauthenticated_servers was firing for client_credentials (M2M) servers even though those servers have no interactive user OAuth flow to complete. The 401 raise was at the outer auth_type == oauth2 level rather than gated on needs_user_oauth_token, so M2M servers were pushed into a user OAuth flow that egress would never look for, making them completely unusable over the MCP protocol path.

• server.py: Adds resolve_oauth2_flow_for_request right after server lookup (matching what listing and tool-call paths already do) to correctly classify legacy null-flow M2M-shaped rows, then inverts the needs_user_oauth_token check into an early continue so the 401 raise is unreachable for M2M servers on both the /{alias}/mcp and /mcp aggregate routes.
• Test: A new parametrized test drives handle_streamable_http_mcp with a stamped oauth2_flow="client_credentials" row and an unstamped M2M-shape row, asserting the session manager is reached and the per-user token store is never consulted for either case.

Confidence Score: 5/5

The change is safe to merge — it removes a spurious 401 raise for a specific server class without touching any authentication logic for other flows.

The inversion of the needs_user_oauth_token guard is logically equivalent to the old code for interactive OAuth servers, and the added resolve_oauth2_flow_for_request call is already applied on every other code path that resolves a server for a request. The fix is narrow, the test covers both the stamped and unstamped M2M cases with direct assertions on the session manager and token store, and all pre-existing tests for interactive, delegate, and passthrough challenges remain untouched.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/_experimental/mcp_server/server.py	Fixes _raise_preemptive_401_for_unauthenticated_servers to skip the interactive-OAuth 401 challenge for M2M (client_credentials) servers by adding an early continue when needs_user_oauth_token is False, and normalizes legacy null-flow M2M-shaped rows via resolve_oauth2_flow_for_request before the challenge gate — matching what listing and tool-call paths already do.
tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_stale_session.py	Adds a parametrized test covering both a stamped oauth2_flow="client_credentials" row and an unstamped M2M-shape row; verifies the session manager is reached and has_user_oauth_token is never consulted for either case.

_{Reviews (1): Last reviewed commit: "fix(mcp): stop preemptively 401-challeng..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_mcp_m2m_preemptive_401 (bba044a) with litellm_internal_staging (fba7ac4)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (03e7dc4) during the generation of this report, so fba7ac4 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[tin-berri]

Superseded by #33586, which fixes the same M2M spurious challenge plus the bearer-in-Authorization masking bug in the same gate. The transport-level regression tests from this PR are grafted there (commit 9b19f8b) and the live before/after evidence in this description remains valid for the M2M half