fix(mcp): challenge and scope gateway-owned server authentication

[joshua-berri]

TLDR

Problem this solves:

• Gateway-owned MCP endpoints omit authentication challenges and discovery metadata
• Newly advertised modes lose the requested server's authorization scope

How it solves it:

• Share eligibility for gateway challenges and protected-resource metadata
• Preserve named-server scope through consent, tokens, and refresh
• Keep delegated and client-forwarded authentication behavior unchanged

User Flow

Before: a client cannot discover gateway sign-in for a named no-auth MCP server

1. POST an unauthenticated initialization to /aws_knowledge_mcp/mcp or /mcp/aws_knowledge_mcp
2. Receive 401 without WWW-Authenticate
3. GET the matching protected-resource metadata and receive 404; sign-in stops

After: the client discovers sign-in and grants access to only the requested server

1. POST the same unauthenticated initialization to either URL format
2. Receive 401 with WWW-Authenticate pointing to matching protected-resource metadata
3. GET that metadata and receive 200, identifying the resource and gateway authorization server
4. Complete registration, sign-in, consent for that server, and S256 PKCE token exchange
5. List tools and make a real tool call on that server
6. Attempt to list or call another server using the access token or a refreshed token; receive an MCP authorization error even when the user can access both servers

Relevant issues

LIT-6634 tracks the separate root-metadata fallback defect

Linear ticket

Resolves LIT-6635

https://linear.app/litellm-ai/issue/LIT-6635

Pre-Submission checklist

• Meaningful mapped regression tests added
• Affected backend tests and applicable local lint/budget gates pass
• Required CI passes on the new commit
• Scope remains limited to gateway-owned MCP authentication
• Greptile confidence at least 4/5 on the new commit
• Bugbot has no outstanding findings on the new commit
• Veria confirms zero unresolved concerns on the new commit
• Contributor previously confirmed signing the CLA

Local affected-suite result: 897 passed, 10 warnings in 75.58s (0:01:15)

The 16 new scope cases fail against the preceding PR tip and pass with this fix. They exercise both URL formats across None/none, API key, bearer, basic, authorization, token, and SigV4 modes. They verify scoped consent, rejection when the server becomes inaccessible, no unnecessary per-user vendor authorization, and scope preservation through token redemption and refresh

Test command and local gate results

python -m pytest tests/test_litellm/proxy/_experimental/mcp_server/test_gateway_dcr_flow.py tests/test_litellm/proxy/_experimental/mcp_server/auth/test_user_api_key_auth_mcp.py tests/test_litellm/proxy/_experimental/mcp_server/test_discoverable_endpoints.py -q --tb=short

897 passed, 10 warnings in 75.58s (0:01:15)
2 files already formatted
All checks passed!
All checks passed!
OK: every strict rule is within its codebase ceiling (base f529d6d6bd8f9bdc3177caf319446bf5a07b304a)
OK: every LIT rule is within its codebase ceiling (base f529d6d6bd8f9bdc3177caf319446bf5a07b304a)
OK: every TQ rule is within its test-suite ceiling (base f529d6d6bd8f9bdc3177caf319446bf5a07b304a)

OK: every rule is within its basedpyright limit or no higher than base (137951 errors total)

The first combined test run was killed with exit 137 during the Docker build. The complete successful run above was repeated after the build finished. No budgets or suppressions were changed

Screenshots / Proof of Fix

Isolated Docker Compose gateway at http://127.0.0.1:46635, private PostgreSQL, the real public AWS Knowledge MCP, and a running MCP SDK server enforcing a gateway-held bearer. Both runtime images were built from clean commit archives with the repository Dockerfile. All four changed installed production files match the current commit by SHA-256. No LLM calls are needed for these MCP tools

Save these reproducible scripts/configs together. All shown credentials are disposable local-only values; session tokens are withheld from published traces. Build the images from the named commits, then run LIT6635_IMAGE=<image-tag> docker compose up -d --no-build. Tags used: litellm-6635:f529d6d6bd and litellm-6635:scope-fixed

config.yaml

model_list: []
mcp_servers:
  aws_knowledge_mcp:
    url: https://knowledge-mcp.global.api.aws
    transport: http
    auth_type: none
  credential_mcp:
    url: http://upstream:8000/mcp
    transport: http
    auth_type: bearer_token
    authentication_token: upstream-only-lit6635
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

compose.yaml

name: litellm-6635-acceptance
services:
  gateway:
    image: ${LIT6635_IMAGE:-litellm-6635:f529d6d6bd}
    entrypoint: ["/app/docker/prod_entrypoint.sh"]
    ports:
      - "127.0.0.1:46635:4000"
    volumes:
      - ./config.yaml:/app/config.yaml:ro
    command: ["--config", "/app/config.yaml", "--port", "4000"]
    environment:
      DATABASE_URL: postgresql://lit6635:local6635@db:5432/lit6635
      LITELLM_MASTER_KEY: sk-local-lit6635-verification
      LITELLM_SALT_KEY: local-lit6635-verification-salt
      STORE_MODEL_IN_DB: "True"
      PRISMA_BINARY_CACHE_DIR: /opt/prisma/binaries
      PRISMA_CLI_PATH: /opt/prisma/binaries/node_modules/.bin/prisma
      PRISMA_CLI_QUERY_ENGINE_TYPE: binary
      PRISMA_OFFLINE_MODE: "true"
    depends_on:
      db:
        condition: service_healthy
  upstream:
    image: litellm-6635:1bcb587d33
    entrypoint: ["python", "/verification/upstream.py"]
    volumes:
      - ./:/verification
  db:
    image: postgres:16
    environment:
      POSTGRES_DB: lit6635
      POSTGRES_USER: lit6635
      POSTGRES_PASSWORD: local6635
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d lit6635 -U lit6635"]
      interval: 2s
      timeout: 5s
      retries: 20
volumes:
  db_data:

upstream.py

import json
import hashlib
from pathlib import Path
from mcp.server.fastmcp import FastMCP
from mcp.server.transport_security import TransportSecuritySettings
import uvicorn
mcp = FastMCP("credential-check", stateless_http=True, json_response=True, transport_security=TransportSecuritySettings(enable_dns_rebinding_protection=False))
@mcp.tool()
def verify_credentials() -> str:
    return "upstream credential accepted"
inner = mcp.streamable_http_app()
async def app(scope, receive, send):
    if scope["type"] == "http":
        headers = dict(scope["headers"])
        accepted = headers.get(b"authorization") == b"Bearer upstream-only-lit6635"
        known = Path("/verification/ingress-hashes").read_text().splitlines() if Path("/verification/ingress-hashes").exists() else []
        leaked = any(hashlib.sha256(v).hexdigest() in known for v in headers.values()) or any(b"sk-local-lit6635" in v or b"litellm-session" in v for v in headers.values())
        with Path("/verification/upstream-audit.jsonl").open("a") as out:
            out.write(json.dumps({"path":scope["path"], "accepted":accepted, "ingress_marker_leaked":leaked})+"\n")
        if not accepted or leaked:
            await send({"type":"http.response.start", "status":401, "headers":[(b"www-authenticate",b"Bearer")]})
            await send({"type":"http.response.body", "body":b"upstream credential rejected"})
            return
    await inner(scope, receive, send)
uvicorn.run(app, host="0.0.0.0", port=8000)

matrix.py

import json, subprocess, sys
from pathlib import Path
revision=sys.argv[1]
out=Path(revision);out.mkdir(exist_ok=True)
base='http://127.0.0.1:46635'
for server in ['aws_knowledge_mcp','credential_mcp']:
 for path in [f'/{server}/mcp',f'/mcp/{server}']:
  for mode in ['missing','invalid','valid','metadata']:
   name=path.strip('/').replace('/','_')+'-'+mode
   url=base+('/.well-known/oauth-protected-resource' if mode=='metadata' else '')+path
   cmd=['curl','--max-time','45','-sS','-D',str(out/(name+'.headers')),'-o',str(out/(name+'.body')),url]
   if mode!='metadata':
    cmd+=['-H','Content-Type: application/json','-H','Accept: application/json, text/event-stream','-d',json.dumps({'jsonrpc':'2.0','id':1,'method':'tools/list' if mode=='valid' else 'initialize','params':{} if mode=='valid' else {'protocolVersion':'2025-06-18','capabilities':{},'clientInfo':{'name':'acceptance','version':'1'}}})]
   if mode in ['invalid','valid']:cmd+=['-H','Authorization: Bearer '+('invalid-local-token' if mode=='invalid' else 'sk-local-lit6635-verification')]
   subprocess.run(cmd,check=True)
   h=(out/(name+'.headers')).read_text();b=(out/(name+'.body')).read_text();status=int(h.splitlines()[0].split()[1]);challenge=next((l for l in h.splitlines() if l.lower().startswith('www-authenticate:')), '')
   expected=200 if mode=='valid' or (mode=='metadata' and revision=='after') else 404 if mode=='metadata' else 401
   assert status==expected,(name,status,b)
   if mode in ['missing','invalid']:
    assert bool(challenge)==(revision=='after'),(name,challenge)
    if revision=='after':
     assert base+'/.well-known/oauth-protected-resource'+path in challenge
     assert ('invalid_token' in challenge)==(mode=='invalid')
   if mode=='metadata' and revision=='after':
    m=json.loads(b);assert m['resource']==base+path and m['authorization_servers']==[base+'/mcp']
   if mode=='valid':
    m=json.loads(next((l[6:] for l in b.splitlines() if l.startswith('data: ')),b));assert len(m['result']['tools'])==(5 if server=='aws_knowledge_mcp' else 1)
   print(f'PASS {path} {mode}: {status} {challenge}',flush=True)

scope_probe.py

import base64
import hashlib
import json
import re
import secrets
import subprocess
import sys
import tempfile
from pathlib import Path
from urllib.parse import parse_qs, urlencode, urlparse

base = 'http://127.0.0.1:46635'
resource = base + sys.argv[1]
work = Path(tempfile.mkdtemp(prefix='lit6635-dcr-'))
jar = work / 'cookies'

def call(url, payload=None, form=None, bearer=None):
    cmd = ['curl', '--max-time', '90', '-sS', '-D', str(work/'headers'), '-o', str(work/'body'), '-b', str(jar), '-c', str(jar), url]
    if payload is not None:
        cmd += ['-H', 'Content-Type: application/json', '-H', 'Accept: application/json, text/event-stream', '-d', json.dumps(payload)]
    if form is not None:
        cmd += ['-H', 'Content-Type: application/x-www-form-urlencoded', '-d', urlencode(form)]
    if bearer is not None:
        cmd += ['-H', 'Authorization: Bearer '+bearer]
    subprocess.run(cmd, check=True)
    lines = (work/'headers').read_text().splitlines()
    status = int(lines[0].split()[1])
    headers = dict(line.split(': ', 1) for line in lines[1:] if ': ' in line)
    return status, {k.lower(): v for k,v in headers.items()}, (work/'body').read_text()

def result(body):
    return json.loads(next((line[6:] for line in body.splitlines() if line.startswith('data: ')), body))

status, headers, body = call(resource, {'jsonrpc':'2.0','id':1,'method':'initialize','params':{'protocolVersion':'2025-06-18','capabilities':{},'clientInfo':{'name':'lit6635-dcr','version':'1'}}})
assert status == 401, (status, body)
challenge = headers.get('www-authenticate', '')
print('Unauthenticated initialize:', status, challenge or '(no WWW-Authenticate)', flush=True)
match = re.search('resource_metadata="([^"]+)"', challenge)
if not match:
    print('FAIL: client cannot discover sign-in from the 401 challenge', flush=True)
    sys.exit(1)
status, _, body = call(match[1])
metadata = json.loads(body)
assert status == 200 and metadata['resource'] == resource, (status, body)
issuer = metadata['authorization_servers'][0]
as_url = base + '/.well-known/oauth-authorization-server' + urlparse(issuer).path
status, _, body = call(as_url)
as_meta = json.loads(body)
assert status == 200 and as_meta['issuer'] == issuer, (status, body)
print('Resource and authorization-server metadata: matching resource and issuer', flush=True)
redirect = 'http://127.0.0.1:46636/callback'
status, _, body = call(as_meta['registration_endpoint'], {'client_name':'lit6635-verification','redirect_uris':[redirect],'grant_types':['authorization_code','refresh_token'],'response_types':['code'],'token_endpoint_auth_method':'none'})
assert status == 201, (status, body)
client_id = json.loads(body)['client_id']
status, _, body = call(base+'/login', form={'username':'admin','password':'sk-local-lit6635-verification'})
assert status == 303, (status, body)
verifier = secrets.token_urlsafe(48)
challenge = base64.urlsafe_b64encode(hashlib.sha256(verifier.encode()).digest()).decode().rstrip('=')
url = as_meta['authorization_endpoint']+'?'+urlencode({'client_id':client_id,'redirect_uri':redirect,'response_type':'code','state':'lit6635','code_challenge':challenge,'code_challenge_method':'S256','resource':resource})
status, headers, body = call(url)
assert status == 303, (status, body)
flow = parse_qs(urlparse(headers['location']).query)['connect_flow'][0]
status, _, body = call(base+'/authorize/flow?'+urlencode({'flow':flow}))
assert status == 200, (status, body)
description=json.loads(body)
if '--expect-denied' in sys.argv:
    expected_name='aws_knowledge_mcp' if 'aws_knowledge_mcp' in resource else 'credential_mcp'
    assert description['state']=='m2m' and description['server_name']==expected_name and description['connected'] is True,description
print('Consent scope: '+json.dumps(description),flush=True)
print('DCR registration, local user sign-in, and consent flow: successful', flush=True)
status, headers, body = call(base+'/authorize/complete', form={'flow':flow,'decision':'approve'})
assert status == 303, (status, body)
code = parse_qs(urlparse(headers['location']).query)['code'][0]
status, _, body = call(as_meta['token_endpoint'], form={'grant_type':'authorization_code','client_id':client_id,'code':code,'redirect_uri':redirect,'code_verifier':verifier,'resource':resource})
assert status == 200, (status, body)
bearer = json.loads(body)['access_token']
refresh_token = json.loads(body)['refresh_token']
with (Path(__file__).parent/'ingress-hashes').open('a') as out:
    out.write(hashlib.sha256(('Bearer '+bearer).encode()).hexdigest()+'\n')
print('Authorization-code exchange with S256 PKCE: successful (token withheld)', flush=True)
for method, params in [('initialize', {'protocolVersion':'2025-06-18','capabilities':{},'clientInfo':{'name':'acceptance','version':'1'}}), ('tools/list', {}), ('tools/call', {'name':('credential_mcp-verify_credentials' if 'credential_mcp' in resource else 'aws_knowledge_mcp-aws___list_regions'),'arguments':{}})]:
    status, _, body = call(resource, {'jsonrpc':'2.0','id':2,'method':method,'params':params}, bearer=bearer)
    obj = result(body)
    evidence = Path(__file__).parent/'after'
    evidence.mkdir(exist_ok=True)
    (evidence/(urlparse(resource).path.strip('/').replace('/','_')+'-'+method.replace('/','_')+'.json')).write_text(json.dumps(obj, indent=2))
    assert status == 200 and 'result' in obj and not obj['result'].get('isError', False), (status, body)
    if method == 'tools/list':
        assert len(obj['result']['tools']) == (1 if 'credential_mcp' in resource else 5), obj
    print('Gateway-session '+method+': 200, '+(str(len(obj['result']['tools']))+' tools' if method=='tools/list' else ('protocol negotiated' if method=='initialize' else 'tool result, isError=false')), flush=True)
other_name = 'credential_mcp' if 'aws_knowledge_mcp' in resource else 'aws_knowledge_mcp'
for token_kind in ['access','refresh']:
    if token_kind == 'refresh':
        status,_,body=call(as_meta['token_endpoint'],form={'grant_type':'refresh_token','client_id':client_id,'refresh_token':refresh_token})
        assert status==200,(status,body)
        bearer=json.loads(body)['access_token']
    status,_,own_body=call(resource,{'jsonrpc':'2.0','id':8,'method':'tools/list','params':{}},bearer=bearer)
    assert status==200 and len(result(own_body)['result']['tools'])==(5 if 'aws_knowledge_mcp' in resource else 1),(status,own_body)
    print('OWN-SERVER '+token_kind+' tools/list: 200, expected tools present',flush=True)
    for other in ['/'+other_name+'/mcp','/mcp/'+other_name]:
        for method,params in [('tools/list',{}),('tools/call',{'name':other_name+('-verify_credentials' if other_name=='credential_mcp' else '-aws___list_regions'),'arguments':{}})]:
            status,_,body=call(base+other,{'jsonrpc':'2.0','id':9,'method':method,'params':params},bearer=bearer)
            print('CROSS-SERVER '+token_kind+' '+other+' '+method+': HTTP '+str(status)+' '+body,flush=True)
            obj=result(body)
            if '--expect-denied' in sys.argv:
                assert status in (401,403,404) or 'error' in obj or obj.get('result',{}).get('isError') is True or (method=='tools/list' and obj.get('result',{}).get('tools')==[]),(status,body)
print('PASS scoped token and refresh cannot access the other server' if '--expect-denied' in sys.argv else 'Completed scope probe',flush=True)

Before (f529d6d)

Authentication and discovery matrix

1. Run python3 matrix.py before. This runs actual curl requests for missing, invalid, and valid gateway credentials, plus metadata, on both URL formats for both upstreams
2. Captured output:

PASS /aws_knowledge_mcp/mcp missing: 401 
PASS /aws_knowledge_mcp/mcp invalid: 401 
PASS /aws_knowledge_mcp/mcp valid: 200 
PASS /aws_knowledge_mcp/mcp metadata: 404 
PASS /mcp/aws_knowledge_mcp missing: 401 
PASS /mcp/aws_knowledge_mcp invalid: 401 
PASS /mcp/aws_knowledge_mcp valid: 200 
PASS /mcp/aws_knowledge_mcp metadata: 404 
PASS /credential_mcp/mcp missing: 401 
PASS /credential_mcp/mcp invalid: 401 
PASS /credential_mcp/mcp valid: 200 
PASS /credential_mcp/mcp metadata: 404 
PASS /mcp/credential_mcp missing: 401 
PASS /mcp/credential_mcp invalid: 401 
PASS /mcp/credential_mcp valid: 200 
PASS /mcp/credential_mcp metadata: 404 

OAuth consent and cross-server authorization

1. Run the same challenge-driven flow for each server and URL format:

python3 scope_probe.py /aws_knowledge_mcp/mcp
python3 scope_probe.py /mcp/aws_knowledge_mcp
python3 scope_probe.py /credential_mcp/mcp
python3 scope_probe.py /mcp/credential_mcp

2. Captured results:

Unauthenticated initialize: 401 (no WWW-Authenticate)
FAIL: client cannot discover sign-in from the 401 challenge

Unauthenticated initialize: 401 (no WWW-Authenticate)
FAIL: client cannot discover sign-in from the 401 challenge

Unauthenticated initialize: 401 (no WWW-Authenticate)
FAIL: client cannot discover sign-in from the 401 challenge

Unauthenticated initialize: 401 (no WWW-Authenticate)
FAIL: client cannot discover sign-in from the 401 challenge

After (a0bb021)

Authentication and discovery matrix

1. Run python3 matrix.py after. This runs actual curl requests for missing, invalid, and valid gateway credentials, plus metadata, on both URL formats for both upstreams
2. Captured output:

PASS /aws_knowledge_mcp/mcp missing: 401 www-authenticate: Bearer resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/aws_knowledge_mcp/mcp"
PASS /aws_knowledge_mcp/mcp invalid: 401 www-authenticate: Bearer error="invalid_token", resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/aws_knowledge_mcp/mcp"
PASS /aws_knowledge_mcp/mcp valid: 200 
PASS /aws_knowledge_mcp/mcp metadata: 200 
PASS /mcp/aws_knowledge_mcp missing: 401 www-authenticate: Bearer resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/mcp/aws_knowledge_mcp"
PASS /mcp/aws_knowledge_mcp invalid: 401 www-authenticate: Bearer error="invalid_token", resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/mcp/aws_knowledge_mcp"
PASS /mcp/aws_knowledge_mcp valid: 200 
PASS /mcp/aws_knowledge_mcp metadata: 200 
PASS /credential_mcp/mcp missing: 401 www-authenticate: Bearer resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/credential_mcp/mcp"
PASS /credential_mcp/mcp invalid: 401 www-authenticate: Bearer error="invalid_token", resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/credential_mcp/mcp"
PASS /credential_mcp/mcp valid: 200 
PASS /credential_mcp/mcp metadata: 200 
PASS /mcp/credential_mcp missing: 401 www-authenticate: Bearer resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/mcp/credential_mcp"
PASS /mcp/credential_mcp invalid: 401 www-authenticate: Bearer error="invalid_token", resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/mcp/credential_mcp"
PASS /mcp/credential_mcp valid: 200 
PASS /mcp/credential_mcp metadata: 200 

OAuth consent and cross-server authorization

1. Run the same challenge-driven flow for each server and URL format:

python3 scope_probe.py /aws_knowledge_mcp/mcp --expect-denied
python3 scope_probe.py /mcp/aws_knowledge_mcp --expect-denied
python3 scope_probe.py /credential_mcp/mcp --expect-denied
python3 scope_probe.py /mcp/credential_mcp --expect-denied

2. Captured results:

aws-legacy full live trace

Unauthenticated initialize: 401 Bearer resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/aws_knowledge_mcp/mcp"
Resource and authorization-server metadata: matching resource and issuer
Consent scope: {"state": "m2m", "client_origin": "http://127.0.0.1:46636", "server_id": "d4940e113c44b70e4708564777381b1c", "server_name": "aws_knowledge_mcp", "connected": true}
DCR registration, local user sign-in, and consent flow: successful
Authorization-code exchange with S256 PKCE: successful (token withheld)
Gateway-session initialize: 200, protocol negotiated
Gateway-session tools/list: 200, 5 tools
Gateway-session tools/call: 200, tool result, isError=false
OWN-SERVER access tools/list: 200, expected tools present
CROSS-SERVER access /credential_mcp/mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: credential_mcp"}}


CROSS-SERVER access /credential_mcp/mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"The key is not allowed to access the requested MCP servers: credential_mcp"}],"isError":true}}


CROSS-SERVER access /mcp/credential_mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: credential_mcp"}}


CROSS-SERVER access /mcp/credential_mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"The key is not allowed to access the requested MCP servers: credential_mcp"}],"isError":true}}


OWN-SERVER refresh tools/list: 200, expected tools present
CROSS-SERVER refresh /credential_mcp/mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: credential_mcp"}}


CROSS-SERVER refresh /credential_mcp/mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"The key is not allowed to access the requested MCP servers: credential_mcp"}],"isError":true}}


CROSS-SERVER refresh /mcp/credential_mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: credential_mcp"}}


CROSS-SERVER refresh /mcp/credential_mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"The key is not allowed to access the requested MCP servers: credential_mcp"}],"isError":true}}


PASS scoped token and refresh cannot access the other server

aws-standard full live trace

Unauthenticated initialize: 401 Bearer resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/mcp/aws_knowledge_mcp"
Resource and authorization-server metadata: matching resource and issuer
Consent scope: {"state": "m2m", "client_origin": "http://127.0.0.1:46636", "server_id": "d4940e113c44b70e4708564777381b1c", "server_name": "aws_knowledge_mcp", "connected": true}
DCR registration, local user sign-in, and consent flow: successful
Authorization-code exchange with S256 PKCE: successful (token withheld)
Gateway-session initialize: 200, protocol negotiated
Gateway-session tools/list: 200, 5 tools
Gateway-session tools/call: 200, tool result, isError=false
OWN-SERVER access tools/list: 200, expected tools present
CROSS-SERVER access /credential_mcp/mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: credential_mcp"}}


CROSS-SERVER access /credential_mcp/mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"The key is not allowed to access the requested MCP servers: credential_mcp"}],"isError":true}}


CROSS-SERVER access /mcp/credential_mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: credential_mcp"}}


CROSS-SERVER access /mcp/credential_mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"The key is not allowed to access the requested MCP servers: credential_mcp"}],"isError":true}}


OWN-SERVER refresh tools/list: 200, expected tools present
CROSS-SERVER refresh /credential_mcp/mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: credential_mcp"}}


CROSS-SERVER refresh /credential_mcp/mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"The key is not allowed to access the requested MCP servers: credential_mcp"}],"isError":true}}


CROSS-SERVER refresh /mcp/credential_mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: credential_mcp"}}


CROSS-SERVER refresh /mcp/credential_mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"The key is not allowed to access the requested MCP servers: credential_mcp"}],"isError":true}}


PASS scoped token and refresh cannot access the other server

credential-legacy full live trace

Unauthenticated initialize: 401 Bearer resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/credential_mcp/mcp"
Resource and authorization-server metadata: matching resource and issuer
Consent scope: {"state": "m2m", "client_origin": "http://127.0.0.1:46636", "server_id": "089ad23966ec2b8cbba9e3cd44b09db1", "server_name": "credential_mcp", "connected": true}
DCR registration, local user sign-in, and consent flow: successful
Authorization-code exchange with S256 PKCE: successful (token withheld)
Gateway-session initialize: 200, protocol negotiated
Gateway-session tools/list: 200, 1 tools
Gateway-session tools/call: 200, tool result, isError=false
OWN-SERVER access tools/list: 200, expected tools present
CROSS-SERVER access /aws_knowledge_mcp/mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}}


CROSS-SERVER access /aws_knowledge_mcp/mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"Error: The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}],"isError":true}}


CROSS-SERVER access /mcp/aws_knowledge_mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}}


CROSS-SERVER access /mcp/aws_knowledge_mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"Error: The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}],"isError":true}}


OWN-SERVER refresh tools/list: 200, expected tools present
CROSS-SERVER refresh /aws_knowledge_mcp/mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}}


CROSS-SERVER refresh /aws_knowledge_mcp/mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"Error: The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}],"isError":true}}


CROSS-SERVER refresh /mcp/aws_knowledge_mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}}


CROSS-SERVER refresh /mcp/aws_knowledge_mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"Error: The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}],"isError":true}}


PASS scoped token and refresh cannot access the other server

credential-standard full live trace

Unauthenticated initialize: 401 Bearer resource_metadata="http://127.0.0.1:46635/.well-known/oauth-protected-resource/mcp/credential_mcp"
Resource and authorization-server metadata: matching resource and issuer
Consent scope: {"state": "m2m", "client_origin": "http://127.0.0.1:46636", "server_id": "089ad23966ec2b8cbba9e3cd44b09db1", "server_name": "credential_mcp", "connected": true}
DCR registration, local user sign-in, and consent flow: successful
Authorization-code exchange with S256 PKCE: successful (token withheld)
Gateway-session initialize: 200, protocol negotiated
Gateway-session tools/list: 200, 1 tools
Gateway-session tools/call: 200, tool result, isError=false
OWN-SERVER access tools/list: 200, expected tools present
CROSS-SERVER access /aws_knowledge_mcp/mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}}


CROSS-SERVER access /aws_knowledge_mcp/mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"Error: The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}],"isError":true}}


CROSS-SERVER access /mcp/aws_knowledge_mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}}


CROSS-SERVER access /mcp/aws_knowledge_mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"Error: The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}],"isError":true}}


OWN-SERVER refresh tools/list: 200, expected tools present
CROSS-SERVER refresh /aws_knowledge_mcp/mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}}


CROSS-SERVER refresh /aws_knowledge_mcp/mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"Error: The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}],"isError":true}}


CROSS-SERVER refresh /mcp/aws_knowledge_mcp tools/list: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"error":{"code":-32600,"message":"The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}}


CROSS-SERVER refresh /mcp/aws_knowledge_mcp tools/call: HTTP 200 event: message
data: {"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"Error: The key is not allowed to access the requested MCP servers: aws_knowledge_mcp"}],"isError":true}}


PASS scoped token and refresh cannot access the other server

3. All 32 cross-server attempts were denied: two source servers × two source URL formats × access/refreshed tokens × two destination formats × list/call. The approved server remained usable. Denials are MCP errors carried inside HTTP 200; the assertions inspect the MCP result, not just HTTP status

Veria finding verification

Veria identified a scope gap in the preceding PR tip, 1bcb587d336c8a141809ca0fe165d1b305f1122f: after approving AWS MCP, both the access and refreshed token could list and call the credential-protected server. This was reproduced locally using the same scope probe before changing the code. Representative captured response:

{"jsonrpc":"2.0","id":9,"result":{"content":[{"type":"text","text":"upstream credential accepted"}],"structuredContent":{"result":"upstream credential accepted"},"isError":false}}

The current-tip traces above demonstrate the same cross-server requests denied. Consent also now names only the requested server. Gateway-owned modes enter the existing ready-to-connect state, while interactive OAuth still requires its vendor credential and existing gateway-managed OAuth scopes remain supported

Type

Bug Fix

Caveats (if any)

Low

• Historical bootstrap retry did not reproduce on two fresh databases
• Pre-existing smol-toml OSV alert is not a required check
• Fresh automated reviews and CI are pending
• Terraform state regression from fix(terraform): keep prior state when a virtual key update is rejected #40512 fails CI; rerun accepted

Final Attestation

• Current-tip CI and automated reviews have completed successfully

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_mcp_auth_challenge_6635 (a0bb021) with litellm_internal_staging (87e2026)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (bf39643) during the generation of this report, so cea2f10 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[joshua-berri]

@BugBot review

Please review this MCP authentication discovery fix for correctness, security, regressions, and preservation of delegated authentication behavior.

[cursor]

Stale Bugbot comment from a previous run.

[joshua-berri]

@greptileai Please review this MCP authentication discovery fix, including gateway-owned modes, delegated-mode exclusions, and the regression coverage and live verification.

[greptile-apps]

Greptile Summary

This PR enables gateway sign-in discovery for per-server MCP endpoints whose upstream authentication is owned by the gateway, then preserves the selected server scope through consent, token exchange, refresh, and request admission

• Uses one eligibility predicate for authentication challenges and protected-resource metadata
• Extends scoped gateway authorization to no-auth and gateway-held credential modes
• Retains exclusions for delegated and client-forwarded authentication
• Preserves server binding across authorization codes and refresh-token rotation
• Adds coverage for both supported per-server URL forms and upstream authentication modes

Confidence Score: 5/5

The PR appears safe to merge, with server scope preserved through consent, token refresh, and cross-server admission

No actionable failure remains. The latest changes retain the selected server ID through authorization-code and refresh-token grants, reject mismatched resource targets, enforce live user reachability, and constrain admitted sessions to the selected server

Important Files Changed

Filename	Overview
litellm/types/mcp_server/mcp_server_manager.py	Defines the shared gateway-authorization eligibility predicate while excluding delegated and explicitly client-forwarded credential modes
litellm/proxy/_experimental/mcp_server/auth/user_api_key_auth_mcp.py	Uses the shared predicate to add matching per-server discovery challenges to gateway admission failures
litellm/proxy/_experimental/mcp_server/discoverable_endpoints.py	Returns gateway authorization metadata for explicitly named servers eligible for gateway-owned sign-in
litellm/proxy/_experimental/mcp_server/gateway_dcr_flow.py	Extends scoped consent and grant handling to gateway-owned non-OAuth upstream modes while retaining reachability and resource checks
tests/test_litellm/proxy/_experimental/mcp_server/auth/test_user_api_key_auth_mcp.py	Covers missing and invalid ingress credentials across newly eligible and excluded authentication modes
tests/test_litellm/proxy/_experimental/mcp_server/test_discoverable_endpoints.py	Covers protected-resource metadata for both per-server URL formats and supported gateway-owned modes
tests/test_litellm/proxy/_experimental/mcp_server/test_gateway_dcr_flow.py	Verifies scoped consent, code redemption, reachability rejection, and refresh-token scope preservation

_{Reviews (3): Last reviewed commit: "fix(mcp): preserve gateway-owned resourc..." | Re-trigger Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[joshua-berri]

@cursor review Please check the resource-scope fix and regression tests at the latest commit; local cross-server and refreshed-token verification is attached.

[joshua-berri]

@greptileai Please re-review the latest resource-scope fix, including consent, token refresh, and cross-server denial; updated local evidence is in the PR description.

[joshua-berri]

@veria-ai Please re-review the resource-scope fix at a0bb021; local tests confirm cross-server denial for access and refreshed tokens on both URL formats.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit a0bb021. Configure here.}

[mateo-berri]

LGTM. Thanks!