Repository navigation
fix(jwt): accept a team alias in x-litellm-team-id - #42445
Conversation
The header only matched canonical team ids, so a JWT caller selecting one of their teams by its alias got a 403 even though they belonged to it. The header value is now resolved through the existing alias lookup before the JWT allowed-team check and the DB membership fallback, while a value that is already a team id never costs an alias lookup and denials keep naming the value the caller sent Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
|
There was a problem hiding this comment.
Devin Review found 1 potential issue.
2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Under fallback_to_db_teams a header value whose team row read fails for any reason other than TeamNotFoundError now keeps the membership denial instead of falling through to the alias lookup, so a degraded read cannot select a different team that carries the value as an alias. Drops the HeaderTeam docstring that only restated its fields Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 1071db1. Configure here.
…election to rc/1.103.0 (#43343) * feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans (#42486) * feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): keep the caller's header session under missing_session_id: generate and read replayed payload session ids Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): drop only the proxy-minted session id so a caller id on the other metadata key survives Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): keep a replayed session id hidden when it only echoes the payload trace id Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): keep a replayed session id even when the payload trace id fell back to it Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): stop reading the replayed payload's session id, the generated marker does not survive replay Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): audit gen_ai.conversation.id on otel v2 spans through a real proxy, sink and postgres Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): keep otel conversation rigs alive for the whole session so shuffled shards do not reboot the proxy per test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(otel): stop the audit rig proxies from probing sibling test peers for model info Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(otel): record accepted OTLP batches in the sink instead of mutating the collector Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(otel): guard the accepted batch deque so snapshots cannot race sink appends Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mrinal <mrinal@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: yucheng <yucheng@berri.ai> (cherry picked from commit a319690) * fix(jwt): accept a team alias in x-litellm-team-id (#42445) * fix(jwt): accept a team alias in x-litellm-team-id The header only matched canonical team ids, so a JWT caller selecting one of their teams by its alias got a 403 even though they belonged to it. The header value is now resolved through the existing alias lookup before the JWT allowed-team check and the DB membership fallback, while a value that is already a team id never costs an alias lookup and denials keep naming the value the caller sent Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(jwt): only alias a header team id the database provably lacks Under fallback_to_db_teams a header value whose team row read fails for any reason other than TeamNotFoundError now keeps the membership denial instead of falling through to the alias lookup, so a degraded read cannot select a different team that carries the value as an alias. Drops the HeaderTeam docstring that only restated its fields Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 071cb49) * fix(jwt): say x-litellm-team-id matched no team id or alias in the 403 (#42495) * fix(jwt): say x-litellm-team-id matched no team id or alias in the 403 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(jwt): tell the caller when x-litellm-team-id names an alias shared by several teams Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(jwt): deny a shared x-litellm-team-id alias exactly like an unknown value A distinct 403 for an alias several teams share was raised before the allowed-teams check, so any JWT could probe which aliases exist. The alias lookup now treats the duplicate as a miss, and both denials say the value does not resolve to a team id or a unique team alias, which is true for unknown, unauthorized and duplicate values alike Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 08639fc) * fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim (#43206) * fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs(jwt): describe header team selection under fallback_to_db_teams Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yassin <yassin@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> (cherry picked from commit 7b4fd47) --------- Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mrinal <mrinal@berri.ai> Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: yassin <yassin@berri.ai>
TLDR
Problem this solves:
x-litellm-team-idon JWT requests only matched a team idHow it solves it:
User Flow
Before: a JWT caller who belongs to a team selects it by alias and is refused
{"team_id": "team-eng", "team_alias": "engineering"}team-engsends POST http://localhost:4000/v1/chat/completions withx-litellm-team-id: team-engand gets 200x-litellm-team-id: engineeringand gets 403Team 'engineering' from x-litellm-team-id header is not in your JWT's allowed teamsfallback_to_db_teamson, worded asis not in your team membershipsAfter: the same request with the alias is served and billed to the same team
{"team_id": "team-eng", "team_alias": "engineering"}team-engsends POST http://localhost:4000/v1/chat/completions withx-litellm-team-id: team-engand gets 200x-litellm-team-id: engineeringand gets 200fallback_to_db_teamson for a developer whose DB membership includesteam-engteam_id: team-engRelevant issues
Follow-up to #31438
Affected release
Linear ticket
Resolves LIT-7181
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Setup shared by both sides: two proxies per commit, each with 4 uvicorn workers on one Postgres, JWT auth against a real Keycloak realm (
JWT_PUBLIC_KEY_URLpointing at its JWKS,team_ids_jwt_field: groups,user_id_jwt_field: sub), realgpt-5.5calls through OpenAI and realclaude-sonnet-5calls through Anthropic. The claims rig carries the team id in the JWT'sgroupsclaim; the fallback rig setsfallback_to_db_teams: true, the JWT carries no team claim and the user is a DB member of the team. Base proxies were built from25af172b85on ports 20183 (claims) and 20184 (fallback), head proxies from1071db1119on ports 20181 and 20182. Teamaud7181-ok-92fa5fhas aliasaud7181-ok-alias-92fa5f;aud7181-nonmember-alias-92fa5fis the alias of a team the caller is not in;aud7181-dup1-alias-92fa5fis an alias shared by two teams. Every case ran with the same three requests,$JWTbeing the Keycloak access token and$TEAMthe header value named in the caseSpend attribution was read back from Postgres by the call id the proxy stamps on each request
Before (25af172)
Claims mode, header is the team id
BASE=http://127.0.0.1:20183 TEAM=aud7181-ok-92fa5f/v1/chat/completions:{"id": "chatcmpl-EQqSj9rzB6qftAjHqOFGUSNoqIPPB", "model": "gpt-5.5", "object": "chat.completion", ...}HTTP 200/v1/messages:{"id": "msg_011CfJBu2aYxeuCeyhov3mvV", "model": "claude-sonnet-5", "type": "message", ...}HTTP 200/v1/responses:{"id": "resp_ko1DAZzVGvYuS2y7S3WQtMeMnF2t9CRjK...", "object": "response", ...}HTTP 200Claims mode, header is the team alias, /v1/chat/completions
BASE=http://127.0.0.1:20183 TEAM=aud7181-ok-alias-92fa5f{"error":{"message":"Team 'aud7181-ok-alias-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. Allowed teams: ['aud7181-ok-92fa5f', ...]","type":"auth_error","param":"None","code":"403"}}HTTP 403Claims mode, header is the team alias, /v1/messages
BASEandTEAM{"error":{"message":"Team 'aud7181-ok-alias-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. ..."}}HTTP 403Claims mode, header is the team alias, /v1/responses
BASEandTEAM{"error":{"message":"Team 'aud7181-ok-alias-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. ..."}}HTTP 403DB fallback mode, header is the team alias
BASE=http://127.0.0.1:20184 TEAM=aud7181-ok-alias-92fa5f, user is a DB member ofaud7181-ok-92fa5f/v1/chat/completions:{"error":{"message":"Team 'aud7181-ok-alias-92fa5f' (from x-litellm-team-id header) is not in your team memberships.","type":"auth_error","param":"None","code":"403"}}HTTP 403/v1/messagesand/v1/responses: same 403 bodyAlias of a team the caller is not in
TEAM=aud7181-nonmember-alias-92fa5f{"error":{"message":"Team 'aud7181-nonmember-alias-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. ..."}}HTTP 403{"error":{"message":"Team 'aud7181-nonmember-alias-92fa5f' (from x-litellm-team-id header) is not in your team memberships." ...}}HTTP 403Unknown value
TEAM=aud7181-does-not-exist-92fa5f{"error":{"message":"Team 'aud7181-does-not-exist-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. ..."}}HTTP 403{"error":{"message":"Team 'aud7181-does-not-exist-92fa5f' (from x-litellm-team-id header) is not in your team memberships." ...}}HTTP 403Alias shared by two teams
TEAM=aud7181-dup1-alias-92fa5f{"error":{"message":"Team 'aud7181-dup1-alias-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. ..."}}HTTP 403{"error":{"message":"Team 'aud7181-dup1-alias-92fa5f' (from x-litellm-team-id header) is not in your team memberships." ...}}HTTP 403Spend attribution
8aef3df8-ad9d-47f8-9143-48f72c037dc1(claims),28fe73c8-051c-4960-8c34-af3d495f0978(fallback)aud7181-ok-92fa5f|successfor each; the alias requests were refused and wrote no spend rowAfter (1071db1)
Claims mode, header is the team id
BASE=http://127.0.0.1:20181 TEAM=aud7181-ok-92fa5f/v1/chat/completions:{"id": "chatcmpl-EQqS3ZbP28zx8tqQ0gAuCEtO0UAXZ", "model": "gpt-5.5", "object": "chat.completion", ...}HTTP 200/v1/messages:{"id": "msg_011CfJBr2iQmEbxzM5ZkHhdT", "model": "claude-sonnet-5", "type": "message", ...}HTTP 200/v1/responses:{"id": "resp__bSlQu5k_ROBMpTtLIviRRPvtbikWGV...", "object": "response", ...}HTTP 200Claims mode, header is the team alias, /v1/chat/completions
BASE=http://127.0.0.1:20181 TEAM=aud7181-ok-alias-92fa5f{"id": "chatcmpl-EQqSNMKgRhlbX2ns7ugtHxwIWG0Me", "model": "gpt-5.5", "object": "chat.completion", ...}HTTP 200Claims mode, header is the team alias, /v1/messages
BASEandTEAM{"id": "msg_011CfJBsWM6wttw9rPsHDVeY", "model": "claude-sonnet-5", "type": "message", ...}HTTP 200Claims mode, header is the team alias, /v1/responses
BASEandTEAM{"id": "resp_pcQVZJT6FwbQiFXXjDjzqoEhFSPsYv1DcyQVOJtzVr2B...", "object": "response", ...}HTTP 200DB fallback mode, header is the team alias
BASE=http://127.0.0.1:20182 TEAM=aud7181-ok-alias-92fa5f, user is a DB member ofaud7181-ok-92fa5f/v1/chat/completions:{"id": "chatcmpl-EQqTTL8Krl4uURcDVMt7gQpjW93r8", "model": "gpt-5.5", "object": "chat.completion", ...}HTTP 200/v1/messages:{"id": "msg_011CfJBxX49py8keq4CefLhz", ...}HTTP 200,/v1/responses:{"id": "resp_SiVHHJicldJzKVkRvkUBilvNCnyluIRy...", ...}HTTP 200Alias of a team the caller is not in
TEAM=aud7181-nonmember-alias-92fa5f{"error":{"message":"Team 'aud7181-nonmember-alias-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. ..."}}HTTP 403{"error":{"message":"Team 'aud7181-nonmember-alias-92fa5f' (from x-litellm-team-id header) is not in your team memberships." ...}}HTTP 403Unknown value
TEAM=aud7181-does-not-exist-92fa5f{"error":{"message":"Team 'aud7181-does-not-exist-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. ..."}}HTTP 403{"error":{"message":"Team 'aud7181-does-not-exist-92fa5f' (from x-litellm-team-id header) is not in your team memberships." ...}}HTTP 403Alias shared by two teams
TEAM=aud7181-dup1-alias-92fa5f{"error":{"message":"Team 'aud7181-dup1-alias-92fa5f' from x-litellm-team-id header is not in your JWT's allowed teams. ..."}}HTTP 403{"error":{"message":"Team 'aud7181-dup1-alias-92fa5f' (from x-litellm-team-id header) is not in your team memberships." ...}}HTTP 403Spend attribution
1198e013-e94a-4fcc-8074-0e1353805521(claims chat),006c9701-6936-4a79-a31a-5b17288f9e6a(claims messages),21381d3f-13e9-42e5-9dce-7c6b18e945b0(claims responses),2a1baf16-77ae-4ac6-b8d3-f80d1fba69fd(fallback chat), plus the by-id requestc83f0884-c909-462a-809b-61e242769622aud7181-ok-92fa5f|successfor every one of them: the alias requests are billed to the canonical team id, the same as the by-id requestselect count(*) from "LiteLLM_SpendLogs" where team_id like '%-alias-%'returned 0 over the whole runFull audit run at the tip (1071db1)
The cases above are the curl slice of a larger run on the same four proxies. The happy matrix drove all three endpoints, streaming and not, through the OpenAI and Anthropic SDKs (sync and async) and raw curl, in both modes, with the id and the alias: 138 rows, every alias row 200 on head and 403 on base, every successful row correlated to exactly one spend row under
aud7181-ok-92fa5f(112/112). The sad and edge cells covered the blocked team's alias, a team without the model (403team_model_access_deniedby alias and by id alike), an exhausted budget by alias (422 like the id header), upper-cased and 8 KB values, an alias renamed mid-run, and a route the team may not call. A 40-request mixed burst at the tip landed 40/40 spend rows under the canonical id. Chaos on the head rig: pausing Postgres mid-burst turned/health/readinessinto 503db: disconnectedand it recovered to 200 on unpause, killing a worker got it replaced by the supervisor, and a clean restart served the alias again on the first request. The Admin UI logs and teams pages at the tip show the alias requests under the canonical team with non-zero spend. Not covered: the proxy-admin path of the same header, which is unchanged and still takes an id only, and JWTs that carry a singleteam_idclaim instead ofgroupsCI on the tip
Every required check passes on
1071db1119. The non-requiredcode-qualityjob fails in the recursive-function detector onlitellm/proxy/bug_report_config.py::_render_json, a file this PR does not touch; main fixed that detector allowlist in #42442 about twenty minutes after this PR's last run started, and the same job passes on main's current head1a714548a4. Re-running the job would reuse the stale merge ref, and a rebase would move the tip and invalidate the Bugbot, live-pr-risk and audit evidence for nothing, so it is left as isRecording
A no-mocks screen recording at the tip shows the alias request refused on main and served on this branch with real gpt-5.5 traffic, plus the Admin UI logs page attributing both requests to the canonical team id. It was posted to the Slack thread that opened this ticket; no permalink is available to this session
Type
🐛 Bug Fix
Caveats (if any)
Low
team_aliasread before the 403auth_builderalready fails when the DB is down. A request that would have been served fails the same way on both sides in that state; only a request that was going to be refused anyway changes status. Discussed and settled in the Greptile thread on this PRx-litellm-team-idpath (team upsert for proxy admins) is unchanged and still takes an id onlyQA runbook
Environment: a proxy booted with a JWT auth block (
JWT_PUBLIC_KEY_URLat the Keycloak realm's JWKS,team_ids_jwt_field: groups,user_id_jwt_field: sub) and theE2E_KEYCLOAK_*variables fromtests/e2e/other/idp.py, plusOPENAI_API_KEYfor the chat calltests/e2e/other/test_jwt_auth_e2e.py::TestJwtTeamHeader::test_team_header_with_the_team_alias_binds_the_same_team_as_the_team_id - a JWT caller can select their team by alias and the request is attributed to the same team id as when selecting it by id
{"team_id": "<id>", "team_alias": "<alias>", "models": ["<model>"]}<id>and fetch an access token for themAuthorization: Bearer <jwt>andx-litellm-team-id: <id>, expect 200x-litellm-team-id: <alias>, expect 200team_idequal to<id>on both rowstests/e2e/other/test_jwt_auth_e2e.py::TestJwtTeamHeader::test_team_header_with_the_alias_of_a_team_the_caller_is_not_in_is_rejected_like_an_unknown_value - the alias of a team the caller is not in is refused with the same 403 shape as a value that names no team at all
x-litellm-team-id: <other alias>, expect 403x-litellm-team-id: <random unknown value>, expect 403Final Attestation
ran /live-pr-risk and found no regressions/backward incompatible risks
Link to Devin session: https://app.devin.ai/sessions/326d1a76c8f840d8ae7a72e594d7bb84
Open in Devin Desktop: https://app.devin.ai/desktop/session/326d1a76c8f840d8ae7a72e594d7bb84?variant=devin
Note
Medium Risk
Changes JWT team selection and 403 behavior on a security-sensitive path, but scope is limited to header resolution with explicit anti-oracle rules and broad test coverage.
Overview
JWT callers can pin team context with
x-litellm-team-idusing a team alias, not only the canonical team id.get_team_id_from_headeris replaced by asyncresolve_team_from_header, which returns aHeaderTeam(raw header value + resolvedteam_id) after optionalget_team_object_by_aliaslookup.When the value is already an allowed JWT team id, resolution stays id-only (no alias DB read). Under
fallback_to_db_teams, the header tries id lookup first and only falls through to alias when the team row is provably missing (TeamNotFoundError), so transient DB errors are not treated as alias misses.403 responses and route/membership denials name only what the client sent in the header, not the resolved id, so non-member aliases and unknown values stay indistinguishable. Alias lookup 5xx errors propagate instead of being masked as membership denials.
auth_builderwires the resolvedteam_idfor spend/limits while keeping provisional header teams from upserting under DB fallback. E2E and unit tests cover alias binding, denial shape, DB-fallback ordering, andteam_allowed_routeswith aliases.Reviewed by Cursor Bugbot for commit 1071db1. Bugbot is set up for automated code reviews on this repo. Configure here.