Skip to content

fix(ui): show user attribution in Top Virtual Keys usage tables - #40729

Merged
ryan-crabbe-berri merged 6 commits into
mainfrom
litellm_lit7572_top_keys_user_column
Sep 19, 2026
Merged

ryan-crabbe-berri merged 6 commits into
mainfrom
litellm_lit7572_top_keys_user_column

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Top Virtual Keys shows only key ids for CLI/SSO session keys
  • No way to see which user a spend row belongs to
  • Clicking such a row fails with "Key not found in database"

How it solves it:

  • Carry the owner (user_email, falling back to user_id) from the daily activity breakdown into the top keys rows
  • Add a User column to Top Virtual Keys (global and entity usage), shown only when a row carries a user
  • Type the row shape (TopKeyItem) so both producers must supply it
  • Move the global top keys aggregation into getGlobalTopKeys so it is unit tested directly
  • Daily activity reports key_exists per key, true only for keys /key/info can open
  • Key IDs and chart bars for keys that no longer exist are not clickable

User Flow

Before: an admin whose users log in through the CLI cannot tell who is spending on the Usage page

  1. They open https://litellm-domain/ui/?page=new_usage
  2. The Top Virtual Keys table lists rows with a Key ID and a Key Alias column only; session key rows show an opaque hash and the email fallback at best
  3. They click a Key ID to find the owner and get the toast "Failed to fetch key info - Key not found in database"
  4. They have no column that tells them which user the spend belongs to

After: the same table shows the owning user on every row, and only offers a key lookup that can succeed

  1. They open https://litellm-domain/ui/?page=new_usage
  2. The Top Virtual Keys table now has a User column between Key Alias and Spend, showing the user's email, or the user id when the owner has no email on file ("-" for rows with no owner); the column is hidden when no row has a user, so the Old Usage page is unchanged
  3. The Key ID of a session key or a deleted key is plain text instead of a blue clickable pill, and hovering it says the key is no longer in the database; clicking it, or its bar in Chart View, does nothing and shows no error. Keys that still exist open their details as before
  4. They read the owner directly from the User column
  5. The same User column and Key ID behavior appear under Team, Organization, Tag and other entity usage views

Relevant issues

Follow-up to #40275 (email label fallback). Reported by a customer via Pylon #8500

Linear ticket

Resolves LIT-7572

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Focused Vitest runs pass for TopKeyView.test.tsx, EntityUsage.test.tsx and tests/top_key_view.test.tsx (the new cases cover getGlobalTopKeys summing spend across days and carrying the owner, the user_id fallback in both aggregators, a null alias with a user email, the User header appearing only when a row has a user, and keys with key_exists: false being skipped by both the table click and the chart bar click). test_common_daily_activity.py covers the backend flag for an active, a deleted and a never-stored key

Live run against a local proxy on this branch, with real gpt-5.5 spend from four keys: two alias-less keys owned by users with an email, one alias-less key owned by a user with no email, and one aliased key. These are regular user-owned keys without an alias, which hit the same render path as CLI session keys (no alias, owner in the breakdown metadata), since a real CLI SSO login needs an IdP

Before (base), 1440px wide

before

After (this PR), 1440px wide. The User column shows the email, or the user id for the owner with no email. Note that at this width the table is wider than its card (556px in 494px), so Spend (USD) is only reachable by scrolling the table sideways

after 1440

After (this PR), 1920px wide, where all four columns fit

after 1920

Key lookup. On the same proxy I deleted one of the seeded keys with POST /key/delete and sent one more gpt-5.5 request with the master key, which is never stored as a key row. The activity endpoint now marks both as not existing:

$ curl -s "http://localhost:4729/user/daily/activity?start_date=2026-09-19&end_date=2026-09-19&page_size=100" -H "Authorization: Bearer sk-1234" \
    | jq -c '.results[].breakdown.api_keys | to_entries[] | {key:(.key[0:12]), key_alias:.value.metadata.key_alias, user_id:.value.metadata.user_id, key_exists:.value.metadata.key_exists}'
{"key":"ae79113bcd08","key_alias":null,"user_id":"lit7572-bob","key_exists":false}
{"key":"1995b0f933e2","key_alias":null,"user_id":"lit7572-no-email-user","key_exists":true}
{"key":"2bdb5005e52e","key_alias":"lit7572-alice-prod-key","user_id":"lit7572-alice","key_exists":true}
{"key":"litellm_prox","key_alias":null,"user_id":null,"key_exists":false}
{"key":"cda114dbd1d8","key_alias":null,"user_id":"lit7572-alice","key_exists":true}

Before (previous commit on this branch): every Key ID is a clickable pill, and clicking the deleted key fires GET /key/info and toasts "Failed to fetch key info - Key not found in database"

before key click

After: the deleted key and the master key row are plain text with an explanatory tooltip. Clicking the deleted key sent zero /key/info requests and raised no toast, while the keys that still exist stay clickable

after key hover

To reproduce:

  1. Create a user with an email and a key for them with no alias: POST /user/new with user_id and user_email, then POST /key/generate with that user_id
  2. Send a chat completion with that key, delete the key with POST /key/delete, and wait about a minute for the daily spend flush
  3. Open http://localhost:4000/ui/?page=new_usage (?page=usage is the Old Usage page)
  4. Open the date picker, pick Today, click Apply
  5. Scroll to the Top Virtual Keys card and read the User column
  6. Hover and click the deleted key's Key ID, then switch to Chart View and click its bar

Type

Bug Fix

Caveats (if any)

Medium

  • At 1440px wide the User column pushes Spend (USD) behind a sideways scroll

Low

  • key_exists is a new optional response field, so existing clients are unaffected
  • Rows with no key_exists value stay clickable, matching the old behavior
  • Live proof used a deleted key and the master key, not a real CLI SSO session
    • Both miss the key table the same way a session key does
  • Key Alias repeats the email next to the User column for alias-less keys
  • Linking the User cell to the user's page is follow-up work

Link to Devin session: https://app.devin.ai/sessions/d7dab4487de544488c1871e08c8aa0d7
Open in Devin Desktop: https://app.devin.ai/desktop/session/d7dab4487de544488c1871e08c8aa0d7?variant=devin


Note

Low Risk
Adds an optional API field and UI-only gating on key detail lookup; spend aggregation logic is unchanged aside from metadata enrichment.

Overview
Adds key_exists to daily-activity KeyMetadata so the API marks keys as present only when they still exist in the active verification-token table (deleted, session, and master keys default to false while historical alias/owner metadata can remain).

On the Usage dashboard, Top Virtual Keys now shows a conditional User column (user_email, then user_id), centralizes global top-key aggregation in getGlobalTopKeys, and types rows as TopKeyItem. TopKeyView disables /key/info for keys with key_exists === false (plain Key ID + tooltip; chart clicks ignored) while keeping existing keys clickable.

Reviewed by Cursor Bugbot for commit 82fd632. Bugbot is set up for automated code reviews on this repo. Configure here.

@devin-ai-integration
devin-ai-integration Bot requested a review from a team September 11, 2026 14:15
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@greptile-apps

greptile-apps Bot commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with no outstanding correctness, security, or repository-rule failures

Summary

This PR adds user attribution to Top Virtual Keys in New Usage and entity views, and prevents failed key-detail lookups for deleted or session keys

  • Adds key_exists to daily activity key metadata and generated API schemas
  • Propagates email or user-ID attribution through global and entity key aggregation
  • Conditionally displays the User column and disables table and chart interactions for unresolvable keys
  • Adds backend, aggregation, and component regression coverage

Reviews (4) · Last reviewed commit: "test(ui): share one chart bar lookup acr..."

@yuneng-berri
yuneng-berri deleted the branch main September 13, 2026 04:52
@yuneng-berri yuneng-berri reopened this Sep 13, 2026
@CLAassistant

CLAassistant commented Sep 13, 2026 •

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ ryan-crabbe-berri
❌ devin-ai-integration[bot]
You have signed the CLA already but the status is still pending? Let us recheck it.

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai please re-review after the follow-up commit that addresses both earlier threads, CI is green on the new tip

@jesus-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@ryan-crabbe-berri ryan-crabbe-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UsagePageView's new user_email line is fully mocked out in tests, and the User column disappears for owners that have a user_id but no email

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@ryan-crabbe-berri addressed in 142766d: aggregation moved to getGlobalTopKeys with direct unit tests, and the User column now falls back to user_id

@ryan-crabbe-berri

Copy link
Copy Markdown
Contributor

@greptile re review

@ryan-crabbe-berri

Copy link
Copy Markdown
Contributor

bugbot run

@codspeed

codspeed Bot commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_lit7572_top_keys_user_column (82fd632) with main (fc5f6c4)

Open in CodSpeed

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@codecov

codecov Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@ryan-crabbe-berri
ryan-crabbe-berri changed the base branch from litellm_internal_staging to main September 19, 2026 18:02
devin-ai-integration Bot and others added 5 commits September 19, 2026 11:04
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…database

/user/daily/activity now reports key_exists on each api key's metadata, true
only when the key is in the active key table that /key/info reads. Top Virtual
Keys renders the Key ID as plain text with an explanatory tooltip and ignores
chart bar clicks when key_exists is false, so deleted keys and CLI/SSO session
keys no longer dead-end on a "Key not found in database" toast
@ryan-crabbe-berri
ryan-crabbe-berri force-pushed the litellm_lit7572_top_keys_user_column branch from 9379463 to 89bf870 Compare September 19, 2026 18:08
The key_exists chart test added a second direct DOM lookup for the Recharts bars, which exposes no role or label, and pushed testing-library/no-node-access over its budget (709 > 707). Both chart tests now go through one helper
@ryan-crabbe-berri

Copy link
Copy Markdown
Contributor

@greptile re review

@ryan-crabbe-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 82fd632. Configure here.

@ryan-crabbe-berri
ryan-crabbe-berri dismissed their stale review September 19, 2026 18:25

Both requested changes landed: the aggregation has direct unit tests and the User column falls back to the user id when there is no email

@ryan-crabbe-berri
ryan-crabbe-berri merged commit 659cef0 into main Sep 19, 2026
88 checks passed
@ryan-crabbe-berri
ryan-crabbe-berri deleted the litellm_lit7572_top_keys_user_column branch September 19, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants