Skip to content

fix(mcp): stop phantom 401 trace on OAuth2 passthrough success - #30411

Closed
ryan-crabbe-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_fix_mcp_oauth2_passthrough_phantom_401
Closed

fix(mcp): stop phantom 401 trace on OAuth2 passthrough success#30411
ryan-crabbe-berri wants to merge 1 commit into
litellm_internal_stagingfrom
litellm_fix_mcp_oauth2_passthrough_phantom_401

Conversation

@ryan-crabbe-berri

@ryan-crabbe-berri ryan-crabbe-berri commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Relevant issues

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible; it only solves 1 specific problem

Proof of Fix

For an OAuth2-mode MCP server such as Atlassian, an MCP client authenticates by sending its upstream OAuth token as Authorization: Bearer <token>. Before this change process_mcp_request validated that bearer as a LiteLLM key first; that always fails for an opaque OAuth token, and the auth exception handler logged the failure to the observability callbacks as a 401 error span before the OAuth2 passthrough recovered. Every successful OAuth2 tool call therefore carried a phantom 401 in its traces

Repro against a live proxy that has an OAuth2 MCP server configured and callbacks: ["otel"]:

curl -sL -N http://localhost:4000/<server>/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "Authorization: Bearer <upstream-oauth-token>" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"<tool>","arguments":{}}}'

Before: the proxy logs 401: LiteLLM Virtual Key expected. Received=... and emits a 401 OTel span for the request even though the tool returns a result

After: no Virtual Key expected line is logged and no 401 span is emitted; the opaque bearer takes the passthrough path and the tool result is unchanged

Type

Bug Fix

Changes

process_mcp_request forwards an opaque upstream OAuth2 token straight to an OAuth2-mode target instead of validating it as a LiteLLM key first. A bearer that LiteLLM can actually authenticate still validates normally so the user is resolved: an sk- key always, and a JWT-shaped bearer when enable_jwt_auth is on. A failed validation on a non-OAuth2 server still propagates instead of granting an anonymous session. The branch is factored into _resolve_oauth2_header_auth with early returns

Tests added in test_user_api_key_auth_mcp.py: opaque token skips validation (no phantom 401), sk- and JWT (when enabled) still validate, a JWT-shaped token passes through when JWT auth is off, and a non-sk- bearer on a non-OAuth2 target still raises

For an OAuth2-mode MCP server the client sends its upstream OAuth token as
`Authorization: Bearer <token>`. process_mcp_request validated that bearer as a
LiteLLM key first, which always fails (virtual keys must start with `sk-`), and
the auth exception handler logged that failure to the observability callbacks as
a 401 error span before the OAuth2 passthrough recovered. Every successful
OAuth2 tool call therefore carried a phantom 401 in its traces.

Pass the token through directly when the target is OAuth2-mode and the bearer is
not `sk-`-shaped, skipping the validation that could only fail. An `sk-`-shaped
bearer still validates normally so a LiteLLM key supplied via Authorization keeps
resolving the user.
@greptile-apps

greptile-apps Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a phantom 401 OTel trace emitted on every successful OAuth2 passthrough tool call. The root cause was that process_mcp_request unconditionally attempted LiteLLM key validation on any Authorization bearer — a validation that always failed for non-sk- tokens (since LiteLLM virtual keys must start with sk-), triggering the 401 error span before the OAuth2 fallback path recovered silently.

  • Core fix: Before calling user_api_key_auth, the handler now checks _target_servers_use_oauth2 and whether the stripped bearer starts with sk-. A non-sk- bearer on an OAuth2-mode target is passed through immediately as UserAPIKeyAuth(), skipping the doomed validation and its side-effectful 401 span entirely.
  • Backward compat preserved: An sk--shaped bearer on an OAuth2 server still goes through full LiteLLM key validation; the exception-handler fallback (for an sk--shaped token that isn't actually a valid key) is also kept intact.
  • Tests added: test_oauth2_token_skips_litellm_validation_no_phantom_401 asserts user_api_key_auth is never called for the fixed path; test_sk_bearer_on_oauth2_target_still_validates pins the backward-compat case.

Confidence Score: 5/5

Safe to merge — the change only eliminates a doomed, side-effectful validation call; the granted access level for non-sk- bearers on OAuth2 servers is identical to what the old code produced after the exception-handler fallback.

The early-return path for non-sk- bearers on OAuth2 targets produces exactly the same UserAPIKeyAuth() result as the old try/catch path did after validation invariably failed; no new access is granted. The sk- bearer path and the cold-start passthrough paths are structurally unchanged. _target_servers_use_oauth2 fails closed (returns False on any unresolved or non-OAuth2 target), so the shortcut cannot be triggered against a non-OAuth2 server. New tests pin both the fixed path and the backward-compat case with mocks only.

No files require special attention.

Important Files Changed

Filename Overview
litellm/proxy/_experimental/mcp_server/auth/user_api_key_auth_mcp.py Refactors the OAuth2 bearer handling in process_mcp_request to short-circuit LiteLLM key validation for non-sk- bearers on OAuth2-mode targets, eliminating the phantom 401 OTel span; backward-compat sk- bearer path and exception-handler fallback are both preserved.
tests/test_litellm/proxy/_experimental/mcp_server/auth/test_user_api_key_auth_mcp.py Adds two new unit tests: one asserting user_api_key_auth is never invoked for a non-sk- bearer on an OAuth2 target, and one confirming an sk--shaped bearer still goes through LiteLLM key validation; both use only mocks with no real network calls.

Reviews (1): Last reviewed commit: "fix(mcp): stop OAuth2 passthrough from l..." | Re-trigger Greptile

@codecov

codecov Bot commented Jun 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.77419% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...erimental/mcp_server/auth/user_api_key_auth_mcp.py 96.77% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

client_ip=client_ip,
)
bearer_token = _get_bearer_token(litellm_api_key)
if targets_oauth2 and not bearer_token.startswith("sk-"):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium: JWT proxy credentials bypass MCP identity checks

user_api_key_auth accepts more than sk- virtual keys: when JWT or OAuth2 proxy auth is enabled, a non-sk- bearer can be the caller's LiteLLM credential. This shortcut admits that request as anonymous on OAuth2 MCP targets, so a user whose token would normally resolve to key/team MCP permissions, spend/rate limits, and stored-token scoping can call targets through the anonymous OAuth2 passthrough path instead. Keep validating bearer tokens that can be LiteLLM credentials, or gate the no-validation passthrough behind an explicit upstream-delegated-auth signal rather than only not startswith("sk-").

@veria-ai

veria-ai Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

PR overview

This pull request updates MCP OAuth2 passthrough authentication handling to avoid emitting a misleading 401 trace when passthrough succeeds. The touched code is in the MCP server user API key authentication path.

There is one open security issue remaining in the MCP authentication flow. The current shortcut for OAuth2 passthrough can let JWT or OAuth2 proxy credentials skip normal LiteLLM identity validation and proceed as anonymous, bypassing key/team MCP permissions, spend or rate limits, and stored-token scoping. No issues have been addressed yet, so the PR still needs a guardrail before this authentication path is safe.

Open issues (1)

Fixed/addressed: 0 · PR risk: 7/10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant