fix(passthrough): stop leaking the caller's virtual key on credential-less Vertex passthrough - #38114
Conversation
…-less Vertex passthrough When no Vertex credential is configured (no default_vertex_config, no matching use_in_pass_through deployment, no vector-store credential), the Vertex passthrough took the bring-your-own-credentials branch and forwarded the entire incoming header set upstream to Google. That set included whichever header carried the caller's LiteLLM virtual key: x-litellm-api-key, or Authorization when get_litellm_virtual_key read the key from there. The proxy's own secret was sent to a third-party provider. The credential-less branch now drops x-litellm-api-key and the Authorization value that equals the virtual key, keeping a genuine bring-your-own Google credential (an OAuth token in Authorization, or x-goog-api-key) so real BYO passthrough still works. When neither survives, the request fails with a clean 401 telling the operator no credential is configured, instead of forwarding the virtual key. Regression coverage in the mapped test path asserts the 401-and-never-forwarded behavior for both leak vectors and that a real Google credential still passes through with the virtual key stripped.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR hardens credential-less Vertex passthrough handling and rejects requests that lack a usable upstream credential
Confidence Score: 5/5The PR appears safe to merge No blocking failure remains
|
| Filename | Overview |
|---|---|
| litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py | Adds credential-aware filtering and an early rejection path; the fixes address all displayed prior findings |
| tests/test_litellm/proxy/pass_through_endpoints/test_llm_pass_through_endpoints.py | Adds regression tests for credential source precedence, configured headers, supported schemes, and bring-your-own credentials |
Reviews (11): Last reviewed commit: "fix(vertex-passthrough): cover the mappe..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
…ss Vertex forward The credential-less Vertex passthrough dropped the caller's LiteLLM virtual key only from Authorization by exact match. A caller who sent the same key in x-goog-api-key (which doubles as a real Google credential) had it accepted as a credential and forwarded upstream. Drop the virtual key by value across every forwarded header, normalizing any Bearer prefix, so no header name carries it to Google.
Adds a regression asserting the value-based strip also drops the caller's virtual key when it is duplicated into the api-key and x-api-key headers, while a genuine bring-your-own Google credential still forwards.
On the credential-less Vertex passthrough branch, drop every header that can only carry LiteLLM caller auth (x-litellm-api-key, api-key, x-api-key) by name, since Google never consumes them, and strip the virtual key by value from Authorization / x-goog-api-key, which may instead hold a genuine bring-your-own Google credential. This closes the residual leak where a distinct caller secret in api-key or x-api-key still reached upstream.
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 2 · PR risk: 0/10 |
user_api_key_auth also authenticates a caller from the operator-configured general_settings.litellm_key_header_name, reading that header straight off the request, so a virtual key sent there survived the credential-less Vertex forwarding filter and reached Google alongside a real bring-your-own credential. Value-strip every header whose value matches the caller's key from any accepted source, including that custom header.
…alHeaders The hand-rolled drop set missed Ocp-Apim-Subscription-Key, so a caller Azure APIM secret in that header was forwarded to Google on the credential-less branch. Derive the name-drop set from the canonical SpecialHeaders.litellm_credential_header_names(), minus Authorization and x-goog-api-key which double as real Google credentials and are value-stripped instead. New credential headers added there are now dropped automatically.
The credential-less filter derived the caller key only from x-litellm-api-key, Authorization, and the custom header, but the route authenticates through Depends(user_api_key_auth), which also accepts the key from x-goog-api-key. A virtual key sent only in x-goog-api-key therefore authenticated yet was kept as a preserved upstream header and forwarded to Google. Resolve the caller key by the same precedence get_api_key uses and value-strip exactly that, so a key in x-goog-api-key is stripped while a real Google key alongside a higher-precedence virtual key is preserved.
…er_token The filter's own Bearer-only stripping missed the other schemes user_api_key_auth accepts, so a virtual key echoed as `Authorization: Basic <key>` alongside a higher-precedence auth header did not match the caller key and was forwarded to Google. Reuse the auth module's _get_bearer_token so the comparison strips exactly what authentication does (Bearer / bearer / Basic / AWS4-HMAC-SHA256), falling back to the raw value for a bare token.
…in streaming tests The LIT-4761 streaming-classification tests passed only the bring-your-own Google OAuth token in Authorization and mocked get_litellm_virtual_key, a shape that cannot authenticate in production. The credential-less filter now resolves the caller key by auth precedence, so a lone Authorization value reads as the key and is stripped. Send the virtual key in x-litellm-api-key, matching a real request, so Authorization is preserved and the classification assertions run.
…eaders user_api_key_auth also accepts the caller key from a pass_through_endpoints entry's headers.litellm_user_api_key, not just litellm_key_header_name. Drop every operator-configured caller-key header by name and treat them as top-precedence caller-key sources, so a virtual key sent through one is never forwarded to Google.
…key headers The resolver placed both operator-configured key headers at the top of its precedence, but user_api_key_auth only overrides with litellm_key_header_name; a pass_through_endpoints litellm_user_api_key is checked last. So a request that authenticated via Authorization while also sending a pass-through header could have the wrong value chosen, leaving the authenticated Authorization key forwarded. Order the resolver exactly like get_api_key: override first, built-in headers next, pass-through header last.
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
…header On mapped pass-through routes, of which /vertex_ai is one, user_api_key_auth accepts the caller key from a header literally named litellm_user_api_key and applies it last, so it overrides every other source. The credential-less filter neither dropped it nor resolved the caller key from it, so a virtual key there reached Google past a real x-goog-api-key, and a bring-your-own Authorization could be stripped when auth actually came from that header. Drop it by name and resolve it at highest precedence.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 16a81c9. Configure here.
tin-berri
left a comment
tin-berri
left a comment
There was a problem hiding this comment.
Approving — real leak, right fix. Forwarding _safe_get_request_headers(request).copy() wholesale meant the caller's virtual key went to Google on every credential-less Vertex passthrough, and splitting it into "drop the proxy-only auth headers by name, drop the value that actually authenticated wherever it appears, 401 if nothing usable survives" is the correct decomposition. Reusing _get_bearer_token from user_api_key_auth rather than re-deriving the scheme stripping is exactly right — that comparison has to agree with authentication or it strips the wrong thing.
Traced the cases that decide whether this is correct or just looks correct:
- virtual key in
Authorizationonly → stripped by value, nothing survives, 401 with the actionable message. Right. - virtual key in
x-litellm-api-key+ real Google token inAuthorization→ key dropped by name,Authorizationvalue ≠ caller key so it's preserved. Right. - virtual key in
x-goog-api-key→ resolved as the authenticated value and dropped by value, not kept just because Google consumes that header. This is the case a by-name-only fix would have missed.
Also checked the dict → Mapping return-type change for runtime breakage: create_pass_through_route does dict(param_custom_headers) at the pass_through_request call, and forward_headers_from_request rebinds ({**request_headers, **headers}) rather than mutating in place, so the MappingProxyType never reaches anything that writes to it.
One thing worth a guard. The whole fix rests on forward_headers being False on this route, and nothing here says so. If it's ever True, forward_headers_from_request merges the raw incoming headers back in — and it only pops a request header when that name is already present in headers, so every header this PR dropped by name (x-litellm-api-key, api-key, x-api-key, Ocp-Apim-Subscription-Key, litellm_user_api_key, the operator-configured ones) gets re-added verbatim and the leak is back. Today _base_vertex_proxy_route doesn't pass _forward_headers and the default is False, so it's safe — but that's an invisible dependency holding up a security fix. Either assert it at the call site or note it where _forward_headers_for_credentialless_vertex_passthrough is defined, so someone enabling header forwarding on this route trips over it.
code-quality is the recursive_detector red on llm_request_utils.py — base drift, and #38149 fixes it.
…ertex passthrough PR BerriAI#38114 dropped whichever header user_api_key_auth would read the caller's key from, by precedence. Under custom_auth, JWT auth, or no master key that header is the caller's own Google token, so the bring-your-own-credentials Vertex branch answered 401 to every valid request. A header value is now dropped only when it is the master key or when its hash is the api_key that authenticated the request, so a Google token that auth never consumed keeps flowing while a LiteLLM key still never reaches Google. test_passthrough_post_call_guardrails.py no longer plants a MagicMock proxy_server module in sys.modules at import, which poisoned sibling tests that read module globals at call time.
2 participants
TLDR
Problem this solves:
Authorization,x-litellm-api-key,x-goog-api-key,api-key,x-api-key,Ocp-Apim-Subscription-Key, the mapped-routelitellm_user_api_keyheader, and the operator-configuredlitellm_key_header_nameHow it solves it:
SpecialHeaders.litellm_credential_header_names(), so every proxy-only auth header Google never consumes (x-litellm-api-key,api-key,x-api-key,Ocp-Apim-Subscription-Key) is dropped, plus any operator-configured caller-key header (litellm_key_header_nameand eachpass_through_endpointsentry'slitellm_user_api_key), and future additions are covered automaticallyuser_api_key_authuses and value-strip exactly that value fromAuthorization/x-goog-api-key(which can instead hold a real Google credential) and from the operator-configured custom key header, normalizing the value with the auth module's own_get_bearer_tokenso every scheme it accepts (Bearer/bearer/Basic/AWS4-HMAC-SHA256) is matchedUser Flow
Before: a developer calls Vertex passthrough on a proxy with no Vertex credential configured, and their LiteLLM key is handed to Google
https://litellm-domain/vertex_ai/v1/projects/my-proj/locations/global/publishers/google/models/gemini-2.5-pro:generateContentwithAuthorization: Bearer sk-...(their LiteLLM virtual key) and a JSON bodyhttps://aiplatform.googleapis.com/v1/projects/my-proj/...:generateContentwith that sameAuthorization: Bearer sk-...still on itx-litellm-api-key,x-goog-api-key,api-key,x-api-key, or the operator's configured key header instead leaks it the same wayAfter: the same call fails fast with a clean 401, and the key is never forwarded
Authorization: Bearer sk-...and the same body401saying no Vertex credential is configured and the virtual key is not forwardedhttps://aiplatform.googleapis.com; sending the key inx-litellm-api-keyorx-goog-api-keygives the same 401, and anyapi-key/x-api-key/ configured custom key header is stripped before forwardingAuthorization, or a real Google API key inx-goog-api-key) still has it forwarded, now with the LiteLLM key and the other proxy auth headers stripped outRelevant issues
Linear ticket
Resolves LIT-5997
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Shared setup (no real virtual key is ever sent to real Google: a local sink intercepts every egress and the capture is the proof)
DEFAULT_VERTEXAI_*env vars, and nouse_in_pass_throughmodel, so the passthrough takes the credential-less branchgoogleapis.com, records the exact headers that would have gone to Google, and returns a synthetic599so nothing reaches Google. A leak shows up as the virtual key (or any caller secret) appearing in that capturecurl -sX POST http://127.0.0.1:PORT/key/generate -H "<auth header>: Bearer $MASTER_KEY" -d '{"duration":"2h"}'returnssk-…{"contents":[{"role":"user","parts":[{"text":"hi"}]}]}general_settings.litellm_key_header_name: x-company-key; every other case runs against a default-config proxyBefore (28b433a)
Case A: virtual key in Authorization
curl -sX POST http://127.0.0.1:49346/vertex_ai/v1/projects/my-proj/locations/global/publishers/google/models/gemini-2.5-pro:generateContent -H "Authorization: Bearer sk-…" -d '<payload>'aiplatform.googleapis.com:authorization: Bearer sk-…, the virtual key was on its way to GoogleCase B: virtual key in x-litellm-api-key
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -d '<payload>'aiplatform.googleapis.com:x-litellm-api-key: sk-…, the virtual key was on its way to GoogleCase D: virtual key in x-goog-api-key
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "x-goog-api-key: sk-…" -d '<payload>'aiplatform.googleapis.com:x-goog-api-key: sk-…andx-litellm-api-key: sk-…. The virtual key was on its way to Google, this time posing as a Google API keyCase F: distinct caller secrets in api-key and x-api-key
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -H "api-key: azure-secret-abc123" -H "x-api-key: anthropic-secret-def456" -d '<payload>'aiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauth,api-key: azure-secret-abc123,x-api-key: anthropic-secret-def456, andx-litellm-api-key: sk-…. Every proxy auth header, including two unrelated caller secrets and the virtual key, went to GoogleCase G: virtual key in the operator-configured custom key header
Against a merge-base proxy configured with
general_settings.litellm_key_header_name: x-company-keycurl -sX POST http://127.0.0.1:42826/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-company-key: Bearer sk-…" -H "x-goog-api-key: AIzaSyReal-Google-Api-Key-000" -d '<payload>'aiplatform.googleapis.com:x-company-key: Bearer sk-…andx-goog-api-key: AIzaSyReal-Google-Api-Key-000. The caller authenticated with the custom header, and that same header carrying the virtual key was forwarded to Google alongside the real Google keyCase H: caller Azure APIM secret in Ocp-Apim-Subscription-Key
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -H "Ocp-Apim-Subscription-Key: azure-apim-secret-xyz789" -d '<payload>'aiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauthandOcp-Apim-Subscription-Key: azure-apim-secret-xyz789. The caller's Azure APIM subscription key, an auth header the proxy accepts but Google never consumes, was forwarded to GoogleCase J: virtual key echoed into Authorization with a Basic scheme
curl -sX POST http://127.0.0.1:49346/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Basic sk-…" -d '<payload>'aiplatform.googleapis.com:authorization: Basic sk-…. The caller authenticated withx-litellm-api-keyand echoed the same virtual key intoAuthorizationunder aBasicscheme, and that header carrying the key was forwarded to GoogleAfter (16a81c9)
The header-filtering behavior in Cases A-H is stable across the hardening commits and was captured against the default-config proxy on port 41337 and the custom-key-header proxy on port 41779. Case I was run against a fresh default-config proxy on port 40923, and Case J plus the no-regression re-check of Cases C and E against a fresh default-config proxy on port 40611 at this exact tip.
Case A: virtual key in Authorization
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "Authorization: Bearer sk-…" -d '<payload>'HTTP 401with{"detail":"No Vertex AI credential is configured on this proxy and the request carried no upstream Google credential. The LiteLLM virtual key is not forwarded to Google. ..."}. The sink recorded nothing new: no request reachedgoogleapis.comCase B: virtual key in x-litellm-api-key
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -d '<payload>'HTTP 401with the same "no Vertex AI credential is configured" detail. The sink recorded nothing new: no request reachedgoogleapis.comCase C: bring-your-own Google token, virtual key in x-litellm-api-key for proxy auth
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauth,x-litellm-api-key: <absent>, and the virtual key is nowhere in the outbound. Real bring-your-own passthrough still works, with the key strippedCase D: virtual key in x-goog-api-key
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "x-goog-api-key: sk-…" -d '<payload>'HTTP 401with the same "no Vertex AI credential is configured" detail. The sink recorded nothing new: no request reachedgoogleapis.com. The key posing as a Google API key no longer satisfies the gateCase E: bring-your-own real Google API key in x-goog-api-key, virtual key in x-litellm-api-key for proxy auth
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "x-goog-api-key: AIzaSyReal-Google-Api-Key-000" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:x-goog-api-key: AIzaSyReal-Google-Api-Key-000,x-litellm-api-key: <absent>, and the virtual key is nowhere in the outbound. A real Google API key that differs from the virtual key still forwards, with the key strippedCase F: distinct caller secrets in api-key and x-api-key, bring-your-own Google token for the real credential
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -H "api-key: azure-secret-abc123" -H "x-api-key: anthropic-secret-def456" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauthonly.x-litellm-api-key,api-key, andx-api-keyare all<absent>, and neitherazure-secret-abc123,anthropic-secret-def456, nor the virtual key appears anywhere. The Google token still forwards, every proxy auth header is droppedCase G: virtual key in the operator-configured custom key header
Against the proxy configured with
general_settings.litellm_key_header_name: x-company-keycurl -sw '%{http_code}' -X POST http://127.0.0.1:41779/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-company-key: Bearer sk-…" -H "x-goog-api-key: AIzaSyReal-Google-Api-Key-000" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:x-goog-api-key: AIzaSyReal-Google-Api-Key-000only,x-company-key: <absent>, and the virtual key is nowhere in the outbound. The caller still authenticates with the custom header, the real Google key still forwards, and the virtual key is strippedx-company-keywith no real Google credential returnsHTTP 401and nothing reachesgoogleapis.comCase H: caller Azure APIM secret in Ocp-Apim-Subscription-Key, legitimate Google header preserved
curl -sw '%{http_code}' -X POST http://127.0.0.1:41337/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Bearer ya29.fake-google-oauth" -H "Ocp-Apim-Subscription-Key: azure-apim-secret-xyz789" -H "X-Goog-User-Project: my-billing-proj" -d '<payload>'HTTP 599from the sink (the request was forwarded). Sink capture of the outbound toaiplatform.googleapis.com:authorization: Bearer ya29.fake-google-oauthandX-Goog-User-Project: my-billing-proj, whileOcp-Apim-Subscription-Keyis<absent>andazure-apim-secret-xyz789appears nowhere. The caller's Azure APIM secret is dropped, and the genuine GoogleX-Goog-User-Projectheader is preserved so real Vertex requests keep workingCase I: virtual key resolved through the full auth precedence, and no regression to real credentials
The route authenticates through
Depends(user_api_key_auth), which accepts the caller key from every header inSpecialHeaders.litellm_credential_header_names(),x-goog-api-keyincluded. The filter now resolves the caller key by that same precedence and value-strips exactly the value that authenticated, sox-goog-api-keyis stripped when it carried the key and preserved when it carried a real Google key alongside a higher-precedence virtual key. All against the port 40923 proxy at this tip:curl -sw '%{http_code}' -X POST http://127.0.0.1:40923/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-goog-api-key: sk-…" -d '<payload>'returnsHTTP 401and the sink records nothing new (the proxy's own auth rejects a virtual key inx-goog-api-keyon this route today, and the filter would strip it regardless, closing the boundary that a mocked unit test exercises directly)x-litellm-api-key: sk-…+Authorization: Bearer ya29.fake-google-oauth) still returnsHTTP 599withauthorization: Bearer ya29.fake-google-oauthforwarded and the virtual key absentx-litellm-api-key: sk-…+x-goog-api-key: AIzaSyReal-Google-Api-Key-000) still returnsHTTP 599withx-goog-api-key: AIzaSyReal-Google-Api-Key-000forwarded and the virtual key absent, so resolving by precedence does not strip a genuine Google keyCase J: virtual key echoed into Authorization with a Basic scheme, and no regression to real credentials
Against a fresh default-config proxy on port 40611 at this tip:
curl -sw '%{http_code}' -X POST http://127.0.0.1:40611/vertex_ai/.../gemini-2.5-pro:generateContent -H "x-litellm-api-key: sk-…" -H "Authorization: Basic sk-…" -d '<payload>'returnsHTTP 401and the sink records nothing new. Normalizing the value with the auth module's own_get_bearer_tokenstrips theBasicscheme just as authentication does, so the echoed virtual key matches the authenticated key andAuthorizationis dropped, leaving no upstream credentialx-litellm-api-key: sk-…+Authorization: Bearer ya29.fake-google-oauth) still returnsHTTP 599withauthorization: Bearer ya29.fake-google-oauthforwarded and the virtual key absentx-litellm-api-key: sk-…+x-goog-api-key: AIzaSyReal-Google-Api-Key-000) still returnsHTTP 599withx-goog-api-key: AIzaSyReal-Google-Api-Key-000forwarded and the virtual key absentCase K: virtual key in the mapped-route litellm_user_api_key header, with real credentials preserved
The
/vertex_aiprefix is a mapped pass-through route, souser_api_key_authaccepts the caller key from a header literally namedlitellm_user_api_keyand applies it last, overriding every other source. Against fresh default-config proxies on ports 40611 (before this commit) and 40337 (this tip), same request each side:litellm_user_api_key: sk-…(auth) +Authorization: Bearer ya29.fake-google-oauth+x-goog-api-key: AIzaSyReal-Google-Api-Key-000.HTTP 599, and the sink capture toaiplatform.googleapis.comshowslitellm_user_api_key: sk-…forwarded with the virtual key, while the realAuthorizationwas dropped. The virtual key reached Google and the bring-your-own token was lostHTTP 599, and the sink capture showsauthorization: Bearer ya29.fake-google-oauthandx-goog-api-key: AIzaSyReal-Google-Api-Key-000both forwarded,litellm_user_api_key<absent>, and the virtual key nowhere. The key that authenticated is stripped, and both real credentials are preservedType
🐛 Bug Fix
Caveats (if any)
Scope of this PR is the credential-less Vertex passthrough leaking the caller's LiteLLM credential through request headers. One adjacent vector is intentionally left for a separate, focused change: a caller can also send a virtual key in the
?key=URL query parameter (the Google AI Studio auth convention thatuser_api_key_authreads on generateContent routes). A client that authenticates only with?key=is already rejected with a 401 on this branch (no surviving upstream Google credential), so that common case does not leak. The virtual key does still ride the forwarded URL query when the caller both authenticates with a header and brings a real Google credential, but stripping credential query params belongs in the shared pass-through URL-forwarding path (it affects every provider, not just Vertex) and is being tracked as its own follow-up rather than widening this PR's blast radius.Final Attestation
Live PR risk
/live-pr-risk CHECKED e6eb6a4: SAFE. The graph is tiny and fully local:
_forwarded_headers_for_credentialless_vertex_passthroughis called only by_prepare_vertex_auth_headers, which is called only by_base_vertex_proxy_route, which is reached by the two public routesvertex_proxy_routeandvertex_discovery_proxy_route. The re-signed_prepare_vertex_auth_headersnow returnsMapping[str, str]and its credential-less branch raisesHTTPException(401); nothing downstream mutates the returned headers, andcreate_pass_through_routecopies them withdict(...), so the immutableMappingProxyTypeis safe on every path.vertex_proxy_routewas driven live in the proof above;vertex_discovery_proxy_routeruns the identical credential-less branch and is covered by unit tests. Direct-caller and related passthrough tests pass on the fixed head (160 passed), includingtest_vertex_passthrough_load_balancing.py, which unpacks the new tuple directly./live-pr-risk CHECKED 4bc0977: SAFE. This commit hardens the same branch to strip the virtual key from every forwarded header by value (normalizing any
Bearerprefix) instead of only fromAuthorizationby name, closing thex-goog-api-keyvector Greptile flagged. Re-walked the graph for the added code: the new_bearer_strippedis a pure module-private helper referenced only inside_forwarded_headers_for_credentialless_vertex_passthrough, and that function still has exactly one caller, so the blast radius is identical to the commit above. No new dependents, side effects, or raises./live-pr-risk CHECKED e7c2ede: SAFE. This commit additionally drops the proxy-only auth headers Google never consumes (
x-litellm-api-key,api-key,x-api-key) by name via a new module-private_HEADERS_NEVER_FORWARDED_TO_VERTEXfrozenset, keeping the by-value virtual-key strip forAuthorization/x-goog-api-key. The frozenset is referenced only inside_forwarded_headers_for_credentialless_vertex_passthrough, which still has exactly one caller, so the blast radius is unchanged; no new dependents, side effects, or raises, and the gate is untouched./live-pr-risk CHECKED ee03632: SAFE. This commit closes the last vector Greptile flagged:
user_api_key_authalso authenticates a caller from the operator-configuredgeneral_settings.litellm_key_header_name, read straight off the request, so a virtual key sent there survived the filter. The new module-private_credentialless_caller_key_valuesreadsgeneral_settings(read-only, via the same lazy import already used elsewhere in the module) and returns the set of accepted key values; the filter now value-strips any header matching one of them. Blast radius unchanged: both the helper and the filter are called only from the single existing caller, no new raises (the 401 gate is untouched), no mutation, no signature change. Verified live on both sides: at the merge-base withlitellm_key_header_name: x-company-keyset,x-company-key: Bearer <vkey>forwarded toaiplatform.googleapis.comalongside a real Google key; on this tip the same request forwards only the real Google key with the custom header and the virtual key stripped, and the custom header alone returns 401. Cases A-F re-verified unchanged on this tip./live-pr-risk CHECKED ab93636: SAFE. This commit replaces the hand-rolled name-drop set with one derived from the canonical
SpecialHeaders.litellm_credential_header_names()minus the two headers that double as real Google credentials (Authorization,x-goog-api-key), which are value-stripped instead. This is the same sourceuser_api_key_authreads the caller's key from, so the drop set now cannot drift out of sync with what authenticates, and it picked upOcp-Apim-Subscription-Key, which the hand-rolled set missed.SpecialHeadersis a pure enum already imported into the module via its_typesstar import; the derived frozenset is evaluated once at import with no side effects, and both module-level constants are referenced only inside the single-caller filter, so the blast radius is unchanged. Verified live on both sides: at the merge-base a distinct caller Azure APIM secret inOcp-Apim-Subscription-Keyforwarded toaiplatform.googleapis.com; on this tip it is dropped while the genuine GoogleX-Goog-User-Projectheader is preserved, and all of Cases A-G re-verified unchanged./live-pr-risk CHECKED f3dc339: SAFE. This commit resolves the caller key by the same precedence
get_api_keyuses (customlitellm_key_header_name, thenx-litellm-api-key,Authorization,api-key,x-api-key,x-goog-api-key,Ocp-Apim-Subscription-Key) and value-strips exactly the one value that authenticated, instead of only the value fromx-litellm-api-key/Authorization/ the custom header. This closes the structural gap Greptile flagged:x-goog-api-keyis an accepted auth source that the old caller-key set omitted while keeping the header, so a virtual key authenticated through it would have been forwarded. The precedence tuple is built once at import from the sameSpecialHeadersenum;_authenticated_caller_key_valuesreads request headers andgeneral_settingsread-only and is still called only by the single-caller filter, so the blast radius is unchanged, no new raises, no mutation. Verified live at this tip: a real Google key inx-goog-api-keyalongside a higher-precedence virtual key inx-litellm-api-keyis still forwarded (Case E, 599), the bring-your-own OAuth token is still forwarded (Case C, 599), and a virtual key sent only inx-goog-api-keyis rejected. Note the proxy's own auth currently rejects a virtual key presented inx-goog-api-keyon this route before the filter runs, so this commit is defense-in-depth on the forwarding boundary, proven directly by the added unit tests./live-pr-risk CHECKED 5d8286c: SAFE. This commit swaps the filter's own
Bearer-only stripping for the auth module's_get_bearer_token, so the caller-key comparison normalizes exactly the schemes authentication accepts (Bearer/bearer/Basic/AWS4-HMAC-SHA256), with a raw-value fallback for a bare token. It closes the case Greptile flagged: a virtual key echoed asAuthorization: Basic <key>alongside a higher-precedence auth header did not match the caller key under the old normalization and was forwarded._get_bearer_tokenis a pure function inuser_api_key_auth(already imported into this module foruser_api_key_auth), with no side effects; the new_normalize_credential_valuewrapper is referenced only by the single-caller resolver and filter, so the blast radius is unchanged. Verified live on both sides: at the merge-baseAuthorization: Basic <vkey>forwarded toaiplatform.googleapis.com; on this tip it returns 401 with nothing forwarded, while the bring-your-own OAuth token (Case C) and a real Google key (Case E) still forward with the virtual key absent./live-pr-risk CHECKED 2fe1e7e: SAFE.
user_api_key_authalso accepts the caller key from apass_through_endpointsentry'sheaders.litellm_user_api_key, not onlylitellm_key_header_name. This commit adds_operator_configured_caller_key_header_names, which reads both fromgeneral_settings(read-only), drops every configured caller-key header by name, and feeds them as top-precedence caller-key sources into the resolver. The helper is pure and referenced only by the single-caller resolver and filter, so the blast radius is unchanged, no new raises, no mutation. Config values are read defensively withisinstanceguards. Covered by unit tests that configure each source throughgeneral_settingsand assert the configured header is dropped while a real Google key inx-goog-api-keyis preserved; the full passthrough test file (160 tests) passes, including the LIT-4761 streaming-classification suite whose fixture now sends the virtual key inx-litellm-api-key, matching a real request./live-pr-risk CHECKED fcc047b: SAFE. Corrects the precedence of the operator-configured key headers to match
get_api_keyexactly:litellm_key_header_nameoverrides everything so it resolves first, then the built-in headers inget_api_keyorder, then apass_through_endpointslitellm_user_api_keyheader whichget_api_keychecks last. The prior commit had lumped both configured sources at the top, so a request that authenticated viaAuthorizationwhile also carrying a pass-through header could pick the wrong value and leave the authenticatedAuthorizationkey forwarded._operator_configured_caller_key_header_namesnow returns(override, pass_through)and both the resolver ordering and the name-drop consume it; still pure, still called only by the single-caller resolver and filter, no new raises or mutation. Covered by a new unit test whereAuthorizationholds the authenticated key, a pass-through header holds a decoy, and a realx-goog-api-keyis present: theAuthorizationkey is stripped, the pass-through header dropped, and the real Google key preserved. Full passthrough test file (161 tests) green./live-pr-risk CHECKED 16a81c9: SAFE. Closes a high-severity vector Cursor Bugbot flagged: on mapped pass-through routes (of which
/vertex_aiis one),user_api_key_authaccepts the caller key from a header literally namedlitellm_user_api_keyviacheck_api_key_for_custom_headers_or_pass_through_endpoints, applied last so it overrides every other source. The filter now drops that header by name and resolves it at highest precedence. Adds a module constant plus a prepend to the resolver order and a union into the name-drop set; still pure, still called only by the single-caller resolver and filter, no new raises or mutation. Verified live on both sides: at the previous tip a virtual key inlitellm_user_api_keyforwarded toaiplatform.googleapis.comwhile the realAuthorizationwas wrongly stripped; on this tip the virtual key is dropped and both the bring-your-own OAuth token and a real Google key are preserved. Full passthrough test file (163 tests) green.Note
High Risk
Security-sensitive change to Vertex passthrough auth and header forwarding; wrong filtering could break BYO-Google flows or still leak secrets upstream.
Overview
Fixes LIT-5997: when the proxy has no Vertex credential, the bring-your-own-credentials branch no longer forwards the full incoming header set to Google.
Credential-less Vertex passthrough now builds upstream headers via
_forwarded_headers_for_credentialless_vertex_passthroughinstead of copying all request headers. Proxy-only auth headers (fromSpecialHeaders.litellm_credential_header_names()exceptAuthorization/x-goog-api-key, plus operatorlitellm_key_header_nameand pass-through key headers) are dropped by name. The caller key is resolved with the same precedence asuser_api_key_authand stripped by value from any remaining header (using_get_bearer_tokenfor scheme normalization). If neither a survivingAuthorizationnorx-goog-api-keyremains, the route returns 401 with guidance instead of calling Google.Bring-your-own Google OAuth or API keys still forward; LiteLLM virtual keys in
Authorization,x-litellm-api-key,x-goog-api-key, or custom headers no longer reach upstream. Tests were updated and expanded (TestVertexCredentiallessPassthroughVirtualKeyLeak) for these cases.Reviewed by Cursor Bugbot for commit 16a81c9. Bugbot is set up for automated code reviews on this repo. Configure here.