feat(jwt): fall back to DB team memberships when JWT has no team claims - #31356
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR adds an opt-in
Confidence Score: 5/5Safe to merge; the opt-in flag defaults to false so all existing deployments are unchanged, and the active path enforces model-access, route, and DB-membership gates equivalent to the existing claim-based path. All previously raised concerns (DB membership destruction during sync, missing membership-budget row on the fallback path, RBAC-asserted team being re-checked against DB membership, header team upsert before validation) are addressed in this revision. The only new finding is an edge-case misleading error message when team_allowed_routes is restrictive — the request is correctly denied in all cases, the message is just imprecise about the reason. No incorrect authorization or data-integrity risk was identified. litellm/proxy/auth/handle_jwt.py — specifically _resolve_db_team_fallback's post-loop error branch, which conflates route-blocked and model-blocked failure reasons in one error message.
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/handle_jwt.py | Core auth logic: adds DB-team fallback path, preserves DB memberships during sync when JWT has no claims, and validates header team against DB membership before attributing usage; one edge-case error message is misleading when team_allowed_routes blocks all candidates |
| tests/test_litellm/proxy/auth/test_handle_jwt.py | Adds ~1600 lines of new parametrized and scenario tests covering the DB-fallback path; existing test modifications are pure style reformats (same assertions); no weakening of coverage |
| litellm/proxy/_types.py | Adds opt-in fallback_to_db_teams: bool = False field to LiteLLM_JWTAuth with a clear docstring; default-off preserves existing behavior |
| codecov.yaml | Disables carryforward for three dead CI flags that were polluting patch coverage on unrelated PRs with stale line maps |
Reviews (21): Last reviewed commit: "fix(jwt): collapse provisional header te..." | Re-trigger Greptile
Greptile SummaryThis PR adds an opt-in
Confidence Score: 3/5Not safe to merge as-is: the DB fallback feature has two functional gaps that can cause silent auth misattribution and persistent data corruption in deployments that combine it with other existing flags. The The interaction between
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/handle_jwt.py | Core auth change adding DB team fallback; contains two P1 logic bugs: sync_user_role_and_teams can wipe DB memberships before the fallback reads them, and _resolve_db_team_fallback picks the first team without checking per-team model access restrictions. |
| litellm/proxy/_types.py | Adds fallback_to_db_teams: bool = False field to LiteLLM_JWTAuth; clean, opt-in, backward-compatible addition. |
| tests/test_litellm/proxy/auth/test_handle_jwt.py | Adds new mock-only tests covering header deferral, DB fallback resolution, and an end-to-end matrix; existing tests are reformatted (assertion parenthesization) with no logic change. Tests do not cover the sync_user_role_and_teams conflict scenario. |
Reviews (2): Last reviewed commit: "style(jwt): use PEP 585/604 annotations ..." | Re-trigger Greptile
… stop team-id leak When fallback_to_db_teams is enabled and a JWT carries no team claims, sync_user_role_and_teams previously computed teams_to_remove as every existing DB membership and wiped the user out of all their teams on each request, which also left the DB fallback nothing to resolve. Skip team removal in that case so memberships survive and the fallback can attribute usage. Apply the same per-team model-access check the claim-based path enforces when selecting a DB fallback team, so a team's models restriction is no longer bypassed; a team that cannot serve the requested model is skipped in favor of one that can. Drop the user's team-id list from the x-litellm-team-id membership 403 detail so a valid-JWT caller can no longer enumerate team IDs.
|
Addressed all three findings in c1f6695:
|
…provisional teams The DB-team fallback resolved a team but never loaded its team membership row, so per-team membership budget limits were silently skipped on that path. _resolve_db_team_fallback now fetches the resolved team's membership when a user_id is known and returns it, matching the claim-based path so downstream LiteLLM_TeamMembership budget enforcement works there too. The provisional x-litellm-team-id validation also fired on any non-None team_id, including an RBAC role-derived one, which 403'd RBAC team flows when the asserted team was not also a DB membership. It now runs only when team_id actually came from the header (team_id == header_team_id).
|
Addressed both remaining findings in 9911998: The DB team fallback now loads the resolved team's membership row. The provisional Generated by Claude Code |
…evel A transient get_team_membership failure on the DB team fallback path is recoverable: the team is still resolved and the request proceeds, just without per-team membership budget enforcement for that request. Logging that at debug hid a silent budget-enforcement gap from operators, so it now logs at warning and states that enforcement was skipped. Behavior is otherwise unchanged: the resolved team is returned with a None membership rather than failing the request, covered by test_resolve_db_team_fallback_survives_membership_lookup_error.
|
Addressed the remaining visibility note in 4a2b0c0: a transient Generated by Claude Code |
|
bugbot run Generated by Claude Code |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 4 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for all 4 issues found in the latest run.
- ✅ Fixed: Wrong 403 when model denied
- _resolve_db_team_fallback now tracks whether any DB membership resolved and raises a model-access denial (matching find_team_with_model_access) when memberships exist but none can access the requested model, only falling back to the no-membership message when the user actually has no memberships.
- ✅ Fixed: Default team disables DB fallback
- db_team_fallback is now derived from get_all_jwt_team_ids (which ignores team_id_default), computed once near the start of auth_builder, and used to skip both the specific_team_id injection and find_and_validate_specific_team_id so a configured default no longer hides claimless tokens from the DB fallback.
- ✅ Fixed: Passthrough check skipped after fallback
- After _resolve_db_team_fallback assigns a team_id, auth_builder re-runs _team_has_passthrough_route_access and raises _raise_team_passthrough_route_denial when the fallback-selected team is not allowed, mirroring the claim-based enforcement.
- ✅ Fixed: Sync preserve ignores singular team claims
- sync_user_role_and_teams now reads jwt_team_ids via get_all_jwt_team_ids so singular-only IdP setups are no longer treated as claimless, preventing stale DB memberships from persisting and later being attributed by _resolve_db_team_fallback.
You can send follow-ups to the cloud agent here.
…ement Resolves four issues in the fallback_to_db_teams path: - _resolve_db_team_fallback now surfaces a model-access denial when memberships exist but none can access the requested model, instead of always returning the no-membership message - auth_builder gates the fallback on real JWT team claims via get_all_jwt_team_ids so a configured team_id_default does not silently route claimless tokens to the default team - A team selected only via _resolve_db_team_fallback is re-validated against the team's allowed_passthrough_routes; the earlier gate ran while team_id was still None - sync_user_role_and_teams considers both plural and singular team claim shapes when reconciling DB memberships so singular-only tokens (Okta/Auth0 defaults) no longer leave stale teams behind
|
|
|
Bugbot's four findings are addressed in e897495, each with a regression test: A model-denied DB-fallback request under Generated by Claude Code |
…hip check When fallback_to_db_teams is on and the JWT carries no team claims, an x-litellm-team-id header is accepted provisionally and only validated against the user's DB memberships later in auth_builder. With team_id_upsert also enabled, get_team_object ran the upsert on that unvalidated header team first, so an attacker-supplied header could create an orphaned team row before the 403 membership check. Suppress the upsert whenever the team is provisional (db_team_fallback), since a genuine membership team already exists and an invalid one must not be created. Regression: test_auth_builder_provisional_header_team_is_not_upserted.
|
Addressed the upsert-before-membership-check note in 08194af. When Generated by Claude Code |
…ride When a JWT carries an RBAC team role but no group claims, auth_builder already sets team_id from the RBAC object_id. db_team_fallback still evaluated true there, so the provisional x-litellm-team-id path accepted a header team and silently overrode the RBAC-asserted team with any team the caller belonged to. Gate db_team_fallback on team_id being unset, and drive the header's provisional acceptance off db_team_fallback rather than the raw flag, so an RBAC token plus a non-claim header team is rejected with 403 instead of substituting the team. Regression: test_auth_builder_header_cannot_override_rbac_team_under_db_fallback.
|
Addressed the RBAC-team override note in 3522123. Generated by Claude Code |
The membership sync read both plural and singular JWT team claims via get_all_jwt_team_ids unconditionally, which silently changed reconciliation for every deployment using sync_user_role_and_teams, not just those opting into fallback_to_db_teams: a singular-only IdP token that previously stripped all DB teams would now be recognized. Gate the dual-claim read on fallback_to_db_teams so flag-off deployments keep the upstream plural-only behavior, honoring the PR's contract that existing deployments are unchanged. Regression: test_sync_user_role_and_teams_singular_claim_only_recognized_under_flag.
|
Scoped the dual-claim membership sync to Generated by Claude Code |
The model-access-denied 403 in _resolve_db_team_fallback echoed the user's full DB team-id list in its detail. It is only the caller's own memberships, but it is inconsistent with the membership-validation 403 in the same feature that was deliberately scrubbed of team IDs. Replace the enumerated list with a generic "no team you are a member of has access" message. Regression extends test_resolve_db_team_fallback_distinguishes_no_membership_vs_model_denied to assert the team id is absent from the detail.
|
Dropped the user's team-id list from the model-access-denied 403 in Generated by Claude Code |
|
bugbot run Generated by Claude Code |
|
On the "Comments Outside Diff" P1 about a claimless non-member header loading the attacker team before membership validation: the exploitable part of this, creating an orphaned team row from an attacker-supplied header, is already closed and stays closed, and the residual read is side-effect-free, so there is no open security issue here
A read of a team the caller is not a member of grants nothing. The only consumer of that Deferring the read until after membership validation would require resolving @greptileai review d55aa14 Generated by Claude Code |
|
Both points on the 3/5 are pre-existing or deliberate behavior, not regressions this PR introduces; neither is an open issue On the P1 "fallback_to_db_teams=False still attributes claimless JWTs to a single DB team": that is the upstream single-team DB fallback, and this PR preserves it byte-for-byte when the flag is off. The diff against base shows the only change to that branch is gating, upstream's On the membership-lookup note in the score rationale: that is the intended behavior added in 4a2b0c0, not a gap. When @greptileai review d55aa14 Generated by Claude Code |
|
The flag-off P1 is a false positive that comes from a mocked The gate is in So flag-off behavior is unchanged from upstream in both enforce modes: enforce-on claimless raises 403, enforce-off claimless resolves the single DB team exactly as before. Gating @greptileai review d55aa14 Generated by Claude Code |
|
bugbot run Generated by Claude Code |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Header bypasses route gate
- Extracted the team_allowed_routes gate from _resolve_db_team_fallback into a shared _is_team_route_allowed helper and now apply it in auth_builder's header-team validation branch under db_team_fallback, so a claimless JWT presenting x-litellm-team-id can no longer reach routes the JWT config narrows for team-role callers.
You can send follow-ups to the cloud agent here.
The auto-pick DB-team fallback already gates against team_allowed_routes, but a claimless JWT presenting x-litellm-team-id under fallback_to_db_teams set team_id directly from the header and only re-validated DB membership afterwards, skipping the route gate. A caller could reach management/info routes that the JWT config narrowed for team-role callers by supplying the header even though the auto-pick path on the same route returns no team.
|
5d7d41b closes the header-path gap Bugbot flagged: the @greptileai review 5d7d41b Generated by Claude Code |
| # provisional x-litellm-team-id header could override an RBAC-asserted team. | ||
| db_team_fallback = ( | ||
| jwt_handler.litellm_jwtauth.fallback_to_db_teams | ||
| and not jwt_handler.get_all_jwt_team_ids(token=jwt_valid_token) |
There was a problem hiding this comment.
High: Empty team claims bypass team revocation
get_all_jwt_team_ids() returns [] both when the configured team claim is absent and when it is present but explicitly empty. With fallback_to_db_teams enabled, a user whose IdP now issues teams: [] can still fall back to DB memberships and keep using those teams' model access; sync_user_role_and_teams() also preserves DB teams in the same empty-list case. Please distinguish claim absence from an empty claim value, and only run/preserve the DB fallback when none of the configured team ID or alias claim fields are present.
There was a problem hiding this comment.
This is the intended semantics of fallback_to_db_teams, and distinguishing an absent claim from an empty one the way suggested would break the feature's primary use case rather than close a bypass.
fallback_to_db_teams is opt-in and shifts the source of team truth from the JWT to LiteLLM's own membership table for deployments whose IdP does not carry team membership in the token. In that mode an absent team claim and an empty team claim both mean the same thing: the JWT does not assert teams, so the DB is authoritative. Many IdPs that omit team membership emit the configured claim as an empty array for a user with no groups rather than dropping the key entirely, so treating teams: [] as "claims present, therefore revoked, 403 under enforcement" would reject exactly the no-team-claim tokens this flag exists to support, including the Entra-style case that motivated it. That is why db_team_fallback keys off get_all_jwt_team_ids being empty and sync_user_role_and_teams preserves DB memberships in the no-claim case; preserving is required or the fallback that runs immediately after would have nothing to resolve.
Revocation in this mode is performed by removing the user's DB team membership, not by the IdP emitting an empty array; the JWT team claim is deliberately not the revocation channel once an operator opts into DB-backed teams. An operator who wants JWT-driven team revocation simply does not enable fallback_to_db_teams: with the flag off, the strict claim-based path and the upstream sync reconciliation are preserved exactly, so an empty or shrinking team claim strips DB memberships as before. The flag changes behavior only for tokens that carry no team claims, and only for deployments that opted in, which is the documented contract. Leaving the behavior as-is
Generated by Claude Code
|
@veria-ai review Generated by Claude Code |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 5d7d41b. Configure here.
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 9584918. Configure here.
…ship denial A caller holding a valid claimless JWT under fallback_to_db_teams could distinguish nonexistent teams (404 from get_team_object) from existing teams they do not belong to (membership 403) by varying x-litellm-team-id, giving an authenticated team-id existence oracle. The provisional header path now rewrites the lookup failure into the exact 403 the membership check raises, while claim-backed header teams keep the upstream 404. Also drop the unreachable falsy-team guard in _resolve_db_team_fallback (get_team_object returns a team or raises, never None) and stop codecov carryforward for three dead flags whose stale sessions were measured against old file revisions and sank patch coverage with phantom executable lines
|
Generated by Claude Code |
|
bugbot run |
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 12e7547. Configure here.
📝 WalkthroughWalkthroughAdds a ChangesJWT DB Team Fallback
Codecov Flag Configuration
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Client
participant AuthBuilder
participant JWTAuthManager
participant DB
Client->>AuthBuilder: Request with JWT (no team claims) + optional x-litellm-team-id header
AuthBuilder->>AuthBuilder: Compute db_team_fallback flag
AuthBuilder->>JWTAuthManager: get_team_id_from_header(fallback_to_db_teams=true)
JWTAuthManager-->>AuthBuilder: Provisionally deferred header team_id
AuthBuilder->>JWTAuthManager: _resolve_db_team_fallback(user_id)
JWTAuthManager->>DB: Load user's team memberships
DB-->>JWTAuthManager: Team membership rows
JWTAuthManager-->>AuthBuilder: Selected team_id (model/route gated)
alt Header team matches resolved team
AuthBuilder->>JWTAuthManager: _validate_header_team_in_db_membership
JWTAuthManager->>DB: Check membership
DB-->>JWTAuthManager: Membership result
JWTAuthManager->>JWTAuthManager: _is_team_route_allowed
end
AuthBuilder-->>Client: Authorized (team_id, membership) or 403 denial
Related PRs: None mentioned. Suggested labels: enhancement, security, tests Suggested reviewers: None specified. 🐰 A JWT arrives with claims all bare, 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
|
Verified working on the last commit |
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
make test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Screenshots / Proof of Fix
Re-verified end to end on a licensed proxy (real
LITELLM_LICENSEfrom the environment, accepted at startup withCHECKING PREMIUM USER - Truein both runs) backed by a real Postgres DB overDATABASE_URLand real OpenAI spend. This run supersedes the earlier proof in this section. Exact commits: before =7f991481cc069d7a069a8a50c140bcfaec9a4e6c(merge-base withlitellm_internal_staging), after =12e7547c7181fa3faf4409afcd9d4c34a0059a94(current head). Each proxy ran from its own detached git worktree with its own venv (uv venv --python 3.12,pip install -e ".[proxy]" prisma,prisma generate --schema litellm/proxy/schema.prisma), launched aspython litellm/proxy/proxy_cli.py --config <config> --port <port> --detailed_debug; no source edits, mocks, or shims of any kindJWT auth uses a locally generated RS256 keypair with a static JWKS served over HTTP (
python3 -m http.server 29873in a dir containingjwks.json); env for both proxies:JWT_PUBLIC_KEY_URL=http://localhost:29873/jwks.json,JWT_AUDIENCE=litellm-qa-31356,JWT_ISSUER=http://localhost:29873. Both runs use this config; the before run drops thefallback_to_db_teamsline because pre-PR code rejects it at startup, verified live on the merge-base:DB setup via the admin API: team
qa-jwt-fb-team-7204withmodels: ["gpt-5.5"]and userqa-jwt-fb-user-7204added to it via/team/member_add, plus a second teamqa-jwt-fb-team-7204-otherthe user is not a member of, and a second userqa-jwt-fb-user-7204-noteamwith no memberships.$JWTis an RS256 token whose only claims aresub=qa-jwt-fb-user-7204,aud,iss,iat, andexp; no team claimsBefore (
7f991481cc, port 28517): the claimless JWT is rejected even though the user has a DB team membership, and the header cannot rescue itAfter (
12e7547c71, port 27431,fallback_to_db_teams: true): the same claimless JWT resolves the team from DB membership and the request reaches OpenAIwith the proxy log confirming the attribution:
JWT DB team fallback: resolved team_id=qa-jwt-fb-team-7204 from user DB membershipThe rest of the matrix on the after proxy
The disposable teams and users were deleted from the DB afterwards
Type
🆕 New Feature
Changes
Entra-backed JWT flows can mint valid user tokens that carry no LiteLLM team claims. With
enforce_team_based_model_accesson, vanilla LiteLLM rejects those tokens with HTTP 403 before it ever loads the user's records, even when the user already has valid team memberships in the proxy database. This adds an opt-infallback_to_db_teamsflag onLiteLLM_JWTAuththat shifts the source of team truth from JWT claims to LiteLLM's own team membership table for deployments where the IdP does not carry the team listWhen the flag is enabled and the JWT has no team claims,
JWTAuthManager.auth_builderdefers the early "no teams in token" 403 until after the user and membership records are resolved, then attributes usage to the user's first resolvable DB team. Anx-litellm-team-idheader is accepted provisionally and validated against the user's DB memberships before it becomes request context, so a caller cannot select a team they do not belong to. If the user has no DB team membership and team model access is enforced, the request still fails with 403The flag defaults to false, so existing deployments are unchanged: the upstream single-team DB fallback and strict claim-based authorization are preserved exactly. The behavior only diverges when an operator opts in, and even then only for tokens that carry no team claims
Tests extend
tests/test_litellm/proxy/auth/test_handle_jwt.pyand cover the header-deferral logic, the early-403 deferral infind_team_with_model_access, the DB membership resolver skipping orphaned memberships, and an end-to-endauth_buildermatrix spanning single-team, multi-team, valid header, invalid header, no-membership-under-enforcement, and the flag-off control that proves upstream behavior is unchangedA follow-up hardens the provisional header path: when the header team fails to load, the 404 from
get_team_objectis rewritten into the same 403 the membership check raises, so a caller cannot probe which team ids exist by varyingx-litellm-team-id. Claim-backed header teams keep the upstream 404. The same commit drops an unreachable falsy-team guard in the fallback resolver and stops codecov carryforward for three dead flags whose stale line maps were sinking patch coverage on PRs touching since-edited filesNote
High Risk
Changes proxy JWT authentication, team attribution, and spend tracking paths; misconfiguration or edge-case bugs could allow wrong-team access or bypass route/model gates despite added checks.
Overview
Adds opt-in
fallback_to_db_teamsonLiteLLM_JWTAuth(default false) so JWTs with no team claims can attribute usage from database team memberships instead of failing early with HTTP 403 whenenforce_team_based_model_accessis on.When enabled,
JWTAuthManager.auth_builderdefers the “no teams in token” rejection, picks the first resolvable DB membership (with the same model access,team_allowed_routes, membership budget, and passthrough checks as the claim path), and acceptsx-litellm-team-idonly provisionally—validated against DB membership, with no team upsert on that header. RBAC-asserted teams are not overridden by the header or re-checked against DB membership.Related JWT fixes in the same flow: team alias resolution wins over
team_id_defaultwhen there is no real team-id claim; sync preserves DB teams on claimless tokens and reconciles singular team claims when the flag is on (plural-only behavior unchanged when off).Reviewed by Cursor Bugbot for commit 9584918. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
High Risk
Changes proxy JWT authentication, team attribution, and spend tracking; misconfiguration or edge-case bugs could allow wrong-team access or bypass route/model gates despite added checks.
Overview
Adds opt-in
fallback_to_db_teamsonLiteLLM_JWTAuth(default false). When enabled and the JWT has no team claims, proxy JWT auth can attribute requests from database team memberships instead of failing early with HTTP 403 underenforce_team_based_model_access.JWTAuthManager.auth_builderdefers the empty-token-teams rejection, runs_resolve_db_team_fallback(first resolvable membership with the same model access,team_allowed_routes, membership budget, and passthrough checks as the claim path), and treatsx-litellm-team-idas provisional—validated against DB membership, with no team upsert on that header. RBAC-pinned teams are not overridden by the header; invalid header teams get a uniform 403 (no team-existence oracle).Related JWT behavior in the same flow: team alias wins over
team_id_defaultwhen there is no real team-id claim; sync preserves DB teams on claimless tokens when the flag is on and reconciles singular team claims only in that mode.codecov.yamldisables carryforward for three unused CI flags so stale patch coverage does not penalize unrelated PRs.Tests in
test_handle_jwt.pycover the fallback matrix, header security, and regressions.Reviewed by Cursor Bugbot for commit 12e7547. Bugbot is set up for automated code reviews on this repo. Configure here.
Summary by CodeRabbit
New Features
Bug Fixes