fix(proxy): stop treating upstream model body field as a LiteLLM model on auth-enforced pass-through routes

[yassin-berriai]

Relevant issues

Linear ticket

Resolves LIT-4299

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy (config-registered pass-through endpoint with auth: true forwarding to the real OpenAI chat completions API, Postgres-backed proxy, real LLM calls costing real $). Config:

model_list:
  - model_name: gpt-5-mini
    litellm_params:
      model: openai/gpt-5-mini
      api_key: os.environ/OPENAI_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  pass_through_endpoints:
    - path: "/my-custom-endpoint"
      target: "https://api.openai.com/v1/chat/completions"
      auth: true
      headers:
        Authorization: "Bearer os.environ/OPENAI_API_KEY"
        content-type: "application/json"

Key minted with a model allowlist that does not contain the upstream model, plus access to the route:

curl -s -X POST http://127.0.0.1:4299/key/generate -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"models": ["gpt-5-mini"], "metadata": {"allowed_passthrough_routes": ["/my-custom-endpoint"]}}'

Before (captured with this PR's model-extraction skip reverted, i.e. auth behavior identical to base 4d33964): the pass-through request is rejected because the upstream-only model body field is checked against the key's LiteLLM model allowlist

curl -s -X POST http://127.0.0.1:4299/my-custom-endpoint -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-5-nano", "messages": [{"role": "user", "content": "Say PASSTHROUGH-OK"}]}'

{
    "error": {
        "message": "key not allowed to access model. This key can only access models=['gpt-5-mini']. Tried to access gpt-5-nano",
        "type": "key_model_access_denied",
        "param": "model",
        "code": "403"
    }
}

After (captured at 783454e): the same curl is forwarded to OpenAI and returns a real completion

{
  "model": "gpt-5-nano-2025-08-07",
  "content": "PASSTHROUGH-OK",
  "usage_total_tokens": 284
}

Every other auth control stays enforced (same fixed proxy, same run):

# bad key still rejected
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://127.0.0.1:4299/my-custom-endpoint \
  -H "Authorization: Bearer sk-bogus" -H "Content-Type: application/json" \
  -d '{"model": "gpt-5-nano", "messages": [{"role": "user", "content": "hi"}]}'
# -> 401

# key without allowed_passthrough_routes still rejected
# -> {"error":{"message":"Key/team not allowed to access passthrough route /my-custom-endpoint. Configure `allowed_passthrough_routes` on the team or key.","type":"auth_error","param":"None","code":"403"}}

# model allowlist still enforced on managed LLM routes with the same key
curl -s -X POST http://127.0.0.1:4299/v1/chat/completions -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d '{"model": "gpt-5-nano", "messages": [{"role": "user", "content": "hi"}]}'
# -> {"error":{"message":"key not allowed to access model. This key can only access models=['gpt-5-mini']. Tried to access gpt-5-nano","type":"key_model_access_denied","param":"model","code":"403"}}

# allowed model on managed routes still works
curl -s -X POST http://127.0.0.1:4299/v1/chat/completions -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d '{"model": "gpt-5-mini", "messages": [{"role": "user", "content": "Say CHAT-OK"}]}'
# -> "CHAT-OK"

Independent e2e verification (Devin)

An independent Devin session verified the PR end to end on a fresh VM: two live proxies against the same Postgres and real OpenAI upstream, PR branch (783454e) on :4000 and the base merge-base (4d33964) on :4001, driven with one restricted virtual key. All contract items passed: the base control returned key_model_access_denied for gpt-5-nano on /my-custom-endpoint, the PR branch returned a real 200 completion for the identical request, and every negative control held (bogus key 401, key without allowed_passthrough_routes 403, model allowlist still enforced on /v1/chat/completions)

The recording was captured at 783454e. The follow-up commit 5296b05 rebuilt the internal mechanism (keying the skip off the dispatched FastAPI endpoint instead of the request path) to close a path-collision bypass; the observable behavior for every case shown here is identical, and the new collision case is covered by unit tests with mutation checks

Full terminal recording of the flow:

Base branch (pre-fix) denies the upstream-only model:

PR branch forwards the identical request and returns a real completion:

Negative controls (401 bogus key, 403 route, 403 model on managed route, 200 allowed model):

Full-resolution video: https://raw.githubusercontent.com/yassin-berriai/litellm-pr-media/main/pr-33710/rec.mp4

Type

🐛 Bug Fix

Changes

A user-defined pass-through endpoint configured with auth: true runs full virtual-key auth, and get_model_from_request unconditionally extracted the model field from the request body, so the key/team/user/project model allowlist checks rejected any request whose model value only exists at the upstream (key_model_access_denied), even when the key was explicitly granted the route via allowed_passthrough_routes. Pass-through bodies are forwarded verbatim to the configured target, so that field names an upstream model, not a LiteLLM-managed one

The skip is keyed off the FastAPI-resolved endpoint, not the request path. create_pass_through_route tags its handler with a marker attribute, and get_model_from_request returns None only when request.scope["endpoint"] carries that marker. Because routing runs before auth dependencies, this reflects the handler that actually serves the request, so it is correct with respect to the HTTP method and to path collisions: a custom pass-through path that overlaps a built-in route (for example /v1/chat/completions, or an include_subpath prefix of one) resolves to the built-in handler, which carries no marker, so model enforcement stays on for that request. That single choke point covers common_checks, the key-level can_key_call_model path, and per-model budget checks, while key auth, allowed_passthrough_routes enforcement, and spend/budget tracking are untouched. Built-in provider passthrough routes (/vertex_ai, /gemini, ...) are separate handlers and keep model enforcement

Regression tests: get_model_from_request returns None when the request was dispatched to a pass-through handler, still extracts and enforces the model when the same path resolves to a built-in handler (the collision case), and still extracts the model on the no-request budget path (tests/test_litellm/proxy/auth/test_auth_utils.py); common_checks accepts an upstream-only model on a pass-through-dispatched request while still rejecting the same body when a built-in handler is dispatched (tests/test_litellm/proxy/auth/test_auth_checks.py). Each fails without the fix (verified by mutation)

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes a bug where auth-enforced user-defined pass-through routes (auth: true) incorrectly rejected requests because get_model_from_request extracted the model field from the forwarded body and checked it against the key/team LiteLLM model allowlist. For pass-through routes, that field names an upstream model (e.g., OpenAI's model), not a LiteLLM-managed one.

• A LITELLM_PASS_THROUGH_ENDPOINT_MARKER constant is defined in the types module and stamped as an attribute onto the handler function returned by create_pass_through_route. get_model_from_request reads this marker from request.scope[\"endpoint\"] (the FastAPI-resolved handler) and returns None early, skipping all model allowlist and per-model budget checks for pass-through-dispatched requests.
• The fix keys off the FastAPI-resolved endpoint object rather than the request path, so a custom pass-through path that collides with a built-in route still gets the built-in handler (which carries no marker), keeping model enforcement intact for that collision case.
• New unit tests cover: model suppression on pass-through dispatch, model enforcement on builtin dispatch (collision), no-request callers (budget reservation), and integration through common_checks — each verified by mutation to fail without the fix.

Confidence Score: 5/5

Safe to merge. The fix is surgical and backward-compatible: new parameters all have None defaults, every existing call site is unaffected, and all other auth controls (key validity, allowed_passthrough_routes, model allowlist on managed routes, spend tracking) remain fully enforced.

The change threads the existing request object into get_model_from_request and uses a function-level attribute marker — set at route-registration time by the server — to detect pass-through dispatch. The identity-check (is True, not truthiness) correctly handles Mock objects in tests. Keying off the FastAPI-resolved endpoint rather than the request path handles the path-collision edge case correctly. Tests cover the core behavior, the collision case, and callers with no request object, each verified by mutation. No logic is removed from any other auth gate.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/types/passthrough_endpoints/pass_through_endpoints.py	Adds the LITELLM_PASS_THROUGH_ENDPOINT_MARKER constant that bridges create_pass_through_route and auth logic; clean addition with no side-effects.
litellm/proxy/pass_through_endpoints/pass_through_endpoints.py	Stamps LITELLM_PASS_THROUGH_ENDPOINT_MARKER onto the returned endpoint function exactly once, after the closure is built, so the attribute persists across requests.
litellm/proxy/auth/auth_utils.py	Adds _request_dispatched_to_pass_through_endpoint with a strict is True identity check, and threads request through get_model_from_request with a safe None default for backward compatibility.
litellm/proxy/auth/auth_checks.py	Threads the existing request parameter into get_model_from_request; request was already present in common_checks so the change is a one-liner with no signature impact.
litellm/proxy/auth/user_api_key_auth.py	Threads request into _get_model_from_request_context so per-model budget checks also skip model extraction for pass-through dispatched requests.
tests/test_litellm/proxy/auth/test_auth_utils.py	Adds three focused unit tests covering: pass-through dispatch suppresses model, builtin dispatch enforces model, and no-request caller still extracts model.
tests/test_litellm/proxy/auth/test_auth_checks.py	Adds an async integration test that exercises common_checks end-to-end: pass-through dispatch accepts upstream-only model, builtin dispatch rejects the same model against the team allowlist.

_{Reviews (2): Last reviewed commit: "fix(proxy): key pass-through model-acces..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yassin-berriai]

Note on the red osv-scan check: it flags mcp 1.26.0 in uv.lock (GHSA-hvrp-rf83-w775, GHSA-jpw9-pfvf-9f58, GHSA-vj7q-gjh5-988w; fixed in 1.27.2/1.28.1). This PR does not touch uv.lock or pyproject.toml, and the same check fails identically on every currently open PR against litellm_internal_staging, so it is base-branch drift. The dependency bump belongs in its own PR on the base branch

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_passthrough_auth_model_conflict (5296b05) with litellm_internal_staging (561b679)}

[yassin-berriai]

@greptileai please review the current head 5296b05

[yassin-berriai]

CI note: the one red job besides osv-scan is litellm_router_testing, which is unrelated to this diff and looks like real-provider-API flake. This PR touches only proxy auth model extraction and adds no router files (git diff origin/litellm_internal_staging...HEAD lists no router file). The full router test selection collects cleanly with no import error, the sibling litellm_router_unit_testing job passes on this same commit, and locally every offline router test passes with the only failure being test_usage_based_routing needing AZURE_API_KEY (a credential CI provides). Could a maintainer rerun that job. osv-scan is separate base-branch drift (mcp 1.26.0 in the base uv.lock), failing every open PR and fixable only in its own dependency bump

[shivamrawat1]

available to be tested in 1.94.0.rc1