Skip to content

feat(team): report per-user spend within a team for JWT traffic - #39771

Merged
yassin-berriai merged 3 commits into
litellm_internal_stagingfrom
litellm_team_spend_by_user
Sep 4, 2026
Merged

yassin-berriai merged 3 commits into
litellm_internal_stagingfrom
litellm_team_spend_by_user

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Team spend cannot be broken down per user
  • User spend merges traffic across every team the user is in
  • JWT/SSO requests have no virtual key, so key breakdowns miss them

How it solves it:

  • New GET /team/spend/by_user grouping spend logs by team and user
  • Team admins see every member, plain members see only themselves
  • Team Usage page gets a per-user card with CSV download

User Flow

Before: a team admin whose members authenticate with JWTs cannot see how much each member spent inside that team

  1. Alice (admin of Team Alpha, also a member of Team Beta) sends POST http://localhost:4000/v1/chat/completions with a JWT for Team Alpha and gets 200 with a completion
  2. Bob (member of Team Alpha) does the same with his own JWT, 200
  3. Alice sends one more with a JWT for Team Beta, 200
  4. The admin opens http://localhost:3000/usage, picks Team Usage, and the Spend Per Team table shows Team Alpha with 3 requests and no per-user split
  5. The admin calls GET http://localhost:4000/user/daily/activity?user_id=alice@example.com and gets 3 requests, which mixes her Team Alpha and Team Beta traffic together
  6. The admin calls GET http://localhost:4000/team/spend/by_user?team_ids=team-alpha&start_date=...&end_date=... and gets 404 Not Found

After: the same traffic is attributed per user inside each team, in the API, the UI, and a CSV

  1. Alice, Bob, and Alice again send the same three chat completions, all 200
  2. The admin opens http://localhost:3000/usage, picks Team Usage, and a new Spend Per User Within Team card lists Team Alpha / alice@example.com (2 requests), Team Alpha / bob@example.com (1), Team Beta / alice@example.com (1) with spend, success, failure, and token counts
  3. Clicking Download CSV saves team_user_spend_to.csv with one row per team and user
  4. GET http://localhost:4000/team/spend/by_user?team_ids=team-alpha,team-beta&start_date=...&end_date=... returns the same three rows as JSON
  5. Alice calls the endpoint with her Team Alpha JWT and, as team admin, sees both her row and Bob's
  6. Bob calls it with his Team Alpha JWT and sees only his own row
  7. Bob asks for team-beta, which he is not in, and gets 404

Relevant issues

Linear ticket

Resolves LIT-6948

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests: endpoint unit tests in tests/test_litellm/proxy/management_endpoints/test_team_endpoints.py, route access in tests/test_litellm/proxy/auth/test_route_checks.py, the actor x team authz matrix in tests/proxy_behavior/management/test_team_spend_by_user.py (27 cases against the ASGI proxy with a real Postgres), and dashboard unit tests for the CSV/label helpers plus the EntityUsage wiring. Each was mutation-checked (dropping the route from the access lists fails 24/27 behavior cases, dropping the litellm-dashboard filter fails the EntityUsage test)
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Setup shared by both arms. Postgres + proxy on localhost:4000 started from this checkout (PYTHONPATH=$PWD), config enabling JWT auth against a local JWKS with team_id_jwt_field: billable_id, user_id_jwt_field: email, user_id_upsert: true, plus an Anthropic haiku deployment. Teams team-alpha (alias Team Alpha, alice admin, bob member) and team-beta (alias Team Beta, alice member). Three real chat completions were sent with JWTs and no virtual key: alice in team-alpha twice, bob in team-alpha once, alice in team-beta once. $MASTER is the master key, $JWT_* are the minted JWTs

Before (2e73400)

Team view has no per-user split

  1. curl -s "localhost:4000/team/daily/activity/aggregated?team_ids=team-alpha&start_date=2026-09-04&end_date=2026-09-04" -H "Authorization: Bearer $MASTER" | jq ".results[0].breakdown | {api_keys, entities}"
  2. Observed: api_keys: {} (JWT traffic has no key) and a single team-alpha entity with api_requests: 3, spend: 0.000102, no user dimension

User view merges teams

  1. curl -s "localhost:4000/user/daily/activity?user_id=alice@example.com&start_date=2026-09-04&end_date=2026-09-04" -H "Authorization: Bearer $MASTER" | jq ".metadata | {total_spend, total_api_requests}"
  2. Observed: {"total_spend": 0.000102, "total_api_requests": 3}, which is alice's Team Alpha and Team Beta traffic added together

Per-user-within-team endpoint

  1. curl -s -w "\nHTTP %{http_code}\n" "localhost:4000/team/spend/by_user?team_ids=team-alpha&start_date=2026-09-04&end_date=2026-09-04" -H "Authorization: Bearer $MASTER"
  2. Observed: {"detail":"Not Found"} HTTP 404

Admin UI

  1. Open http://localhost:3000/usage, choose Team Usage in the Usage View dropdown
  2. Observed: Spend Per Team, Top Virtual Keys (empty, JWT traffic), Top Public Model Names; no per-user card

After (5f4496c)

Team view has no per-user split

  1. Same team/daily/activity/aggregated call as Before
  2. Observed: unchanged, one team-alpha entity, 3 requests (this endpoint is out of scope)

User view merges teams

  1. Same user/daily/activity call as Before
  2. Observed: unchanged, 3 requests (out of scope)

Per-user-within-team endpoint

  1. curl -s "localhost:4000/team/spend/by_user?team_ids=team-alpha,team-beta&start_date=2026-09-04&end_date=2026-09-04" -H "Authorization: Bearer $MASTER" | jq -c ".results[]|{team_id,team_alias,user_id,spend,api_requests,successful_requests,failed_requests,total_tokens}"
  2. Observed:
    {"team_id":"team-alpha","team_alias":"Team Alpha","user_id":"alice@example.com","spend":6.8e-05,"api_requests":2,"successful_requests":2,"failed_requests":0,"total_tokens":28}
    {"team_id":"team-alpha","team_alias":"Team Alpha","user_id":"bob@example.com","spend":3.4e-05,"api_requests":1,"successful_requests":1,"failed_requests":0,"total_tokens":14}
    {"team_id":"team-beta","team_alias":"Team Beta","user_id":"alice@example.com","spend":3.4e-05,"api_requests":1,"successful_requests":1,"failed_requests":0,"total_tokens":14}
    
  3. Alice's Team Alpha JWT (team admin): curl -s -w "\nHTTP %{http_code}\n" "localhost:4000/team/spend/by_user?team_ids=team-alpha&start_date=2026-09-04&end_date=2026-09-04" -H "Authorization: Bearer $JWT_ALICE_ALPHA" | jq -c ".results[]|{user_id,spend,api_requests}"
  4. Observed: {"user_id":"alice@example.com","spend":6.8e-05,"api_requests":2} and {"user_id":"bob@example.com","spend":3.4e-05,"api_requests":1}, HTTP 200
  5. Bob's Team Alpha JWT (plain member): same call with $JWT_BOB_ALPHA
  6. Observed: only {"user_id":"bob@example.com","spend":3.4e-05,"api_requests":1}, HTTP 200
  7. Alice's Team Beta JWT (plain member there): team_ids=team-beta with $JWT_ALICE_BETA
  8. Observed: only {"user_id":"alice@example.com","spend":3.4e-05,"api_requests":1}, HTTP 200
  9. Bob asks for team-beta: team_ids=team-beta with $JWT_BOB_ALPHA
  10. Observed: {"detail":{"error":"User does not belong to Team= team-beta. Call /user/info to see user's teams"}} HTTP 404
  11. Missing team_ids: curl -s -w "\nHTTP %{http_code}\n" "localhost:4000/team/spend/by_user?start_date=2026-09-04&end_date=2026-09-04" -H "Authorization: Bearer $MASTER"
  12. Observed: {"detail":{"error":"Please provide team_ids"}} HTTP 400
  13. Range with no traffic: team_ids=team-alpha&start_date=2026-08-01&end_date=2026-08-31
  14. Observed: {"start_date":"2026-08-01","end_date":"2026-08-31","results":[]} HTTP 200

Admin UI

  1. npm run dev in ui/litellm-dashboard, open http://localhost:3000/usage, choose Team Usage in the Usage View dropdown, scroll below Spend Per Team
  2. Observed: a Spend Per User Within Team card with rows Team Alpha / alice@example.com ($0.0001, 2 requests, 2 successful, 0 failed, 28 tokens), Team Alpha / bob@example.com (1, 1, 0, 14), Team Beta / alice@example.com (1, 1, 0, 14), while Spend Per Team above it shows Team Alpha 3 / 42 tokens and Team Beta 1 / 14

Spend Per User Within Team card listing all three team and user rows under the Spend Per Team table

  1. Click Filter by team and pick Team Beta
  2. Observed: the card refetches with team_ids=team-beta (proxy log shows GET /team/spend/by_user?...&team_ids=team-beta) and shows only Team Beta / alice@example.com (1 request), Spend Per Team drops to Team Beta only

Spend Per User Within Team card showing only the Team Beta alice row after filtering

  1. Click Download CSV
  2. Observed: team_user_spend_2026-08-28_to_2026-09-04.csv downloaded with header Start Date,End Date,Team,Team ID,User,User ID,User Email,Spend (USD),Requests,Successful,Failed,Prompt Tokens,Completion Tokens,Total Tokens and the same three rows, e.g. 2026-08-28,2026-09-04,Team Alpha,team-alpha,alice@example.com,alice@example.com,alice@example.com,0.000068,2,2,0,18,10,28

Terminal showing the downloaded CSV with one row per team and user

Type

🆕 New Feature

Caveats (if any)

Medium

  • Aggregates raw spend logs per request, not a daily rollup table
    • Bounded by the existing team daily activity max date range
  • Team Usage card sends every visible team when no team filter is picked

Low

  • Rows where a JWT carried no user id show as (no user) in the UI

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/b39d9e85349e45e795ff3979df4acf2d
Open in Devin Desktop: https://app.devin.ai/desktop/session/b39d9e85349e45e795ff3979df4acf2d?variant=devin
Requested by: @yassin-berriai

Add GET /team/spend/by_user, which groups raw spend logs by (team_id, user)
so JWT/SSO requests with no virtual key are attributed to the user inside
each selected team. Team admins see every member, plain members see only
their own row. The Team Usage page gets a Spend Per User Within Team card
with CSV export backed by the same endpoint.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai please review, this adds GET /team/spend/by_user plus the Team Usage per-user card and CSV export

@codspeed

codspeed Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_team_spend_by_user (5f4496c) with litellm_internal_staging (04a198e)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (f74bc94) during the generation of this report, so 04a198e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@greptile-apps

greptile-apps Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds team-scoped, per-user spend reporting for JWT/SSO traffic and exposes it through the management API and Team Usage dashboard.

  • Adds an authorization-scoped /team/spend/by_user endpoint with date-range validation and per-team/user aggregation.
  • Adds a dashboard card and formula-safe CSV export for team-member spend.
  • Adds route, authorization-matrix, endpoint, dashboard-wiring, and CSV-helper coverage.
  • The changes since the previous review remove excess explanatory docstrings and add endpoint-audit and dashboard wiring coverage.

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness, security, or repository-rule issue identified.

No new actionable failure remains after the latest changes. The raw-query and timezone threads were manually resolved after the author supplied implementation context. The source-comment thread was also manually resolved after devin-ai-integration[bot] removed the helper, model, and test docstrings, leaving only the endpoint’s OpenAPI-facing description.

Important Files Changed

Filename Overview
litellm/proxy/management_endpoints/team_endpoints.py Adds the scoped per-user team-spend endpoint and removes the explanatory docstrings identified previously.
litellm/types/proxy/management_endpoints/team_endpoints.py Defines the response models for per-user team spend while removing unnecessary model docstrings.
ui/litellm-dashboard/src/app/(dashboard)/usage/_components/components/EntityUsage/EntityUsage.tsx Wires the per-user spend card into Team Usage with the selected or visible team IDs.
ui/litellm-dashboard/src/app/(dashboard)/usage/_components/components/EntityUsage/TeamUserSpendCard.tsx Displays authorized team-member spend results and provides CSV download.
ui/litellm-dashboard/src/app/(dashboard)/usage/_components/components/EntityUsage/teamUserSpend.ts Implements labels, stable row IDs, sorting, formula-safe CSV generation, and download handling.
ui/litellm-dashboard/src/components/networking.tsx Adds the typed client call using the same date boundaries as the existing team activity request.
tests/proxy_behavior/management/test_team_spend_by_user.py Covers the endpoint authorization matrix against the proxy behavior harness.
terraform/provider/tools/endpointaudit/coverage_allowlist.txt Classifies the new read-only reporting endpoint alongside comparable analytics APIs that are not Terraform-managed.

Reviews (2): Last reviewed commit: "refactor(team): drop explanatory docstri..." | Re-trigger Greptile

Comment thread litellm/proxy/management_endpoints/team_endpoints.py
Comment thread ui/litellm-dashboard/src/components/networking.tsx
Comment thread litellm/proxy/management_endpoints/team_endpoints.py Outdated
@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.36364% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...tellm/proxy/management_endpoints/team_endpoints.py 94.73% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

yassin-berriai and others added 2 commits September 4, 2026 18:04
…owlist and EntityUsage unit test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…and regen schema.d.ts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai please re-review at 5f4496c: docstrings dropped, timezone and raw-query threads answered inline with code references

@yassin-berriai
yassin-berriai merged commit dd01abc into litellm_internal_staging Sep 4, 2026
183 of 186 checks passed
@yassin-berriai
yassin-berriai deleted the litellm_team_spend_by_user branch September 4, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants