fix(usage): rank per-model top keys server-side over all keys

[devin-ai-integration]

TLDR

Problem this solves:

• Usage model rows show Top Virtual Keys from the globally capped key subset
• A model whose biggest spender is outside the global top 100 ranks it wrong
• Client-side ranking cannot recover keys the aggregated response dropped

How it solves it:

• New GET /user|team/daily/activity/aggregated/model_top_keys ranks keys per model over all keys
• Dashboard fetches it lazily when a model row is expanded
• Payload stays bounded at 5 keys per request, and only open rows fetch

User Flow

Before: an admin opens a model row in Usage and the Top Virtual Keys card misses the key that spent the most on it

1. They open http://localhost:4000/ui/usage/ on a deployment with more distinct keys than the top keys cap, and pick Model Activity
2. They expand the claude-sonnet-4-5 row
3. The Top Virtual Keys by Spend card lists only keys in the global top spenders: lit8618-heavy-a (1 request) is shown, lit8618-claude-only (2 requests, the model's biggest spender) is absent
4. The Team Usage tab shows the same wrong list for the team's model rows

After: the same row lists the real top spenders for that model

1. They open http://localhost:3000/usage/ (dashboard dev server against the same proxy and data) and pick Model Activity
2. They expand the claude-sonnet-4-5 row
3. The card shows a short loading state, then lists lit8618-claude-only (2 requests, 60 tokens) first and lit8618-heavy-a (1 request, 30 tokens) second, in both the Models and Model Groups views
4. The Team Usage tab does the same for the team's model rows, still scoped to the teams and keys the caller may see

Linear ticket

Resolves LIT-8618

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Setup shared by both legs: local Postgres, proxy started with PYTHONPATH=<checkout> USAGE_TOP_API_KEYS_LIMIT=2 LITELLM_LOCAL_MODEL_COST_MAP=True python litellm/proxy/proxy_cli.py --config ~/lit8618/config.yaml --port 4000 --use_v2_migration_resolver, models gpt-4.1-mini (openai) and claude-sonnet-4-5 (anthropic), three keys created through POST /key/generate with aliases lit8618-heavy-a, lit8618-heavy-b, lit8618-claude-only, and real traffic sent through POST /v1/chat/completions once, so the same LiteLLM_DailyUserSpend rows back both legs:

gpt-4.1-mini:      heavy-a $0.001296, heavy-b $0.00126
claude-sonnet-4-5: claude-only $0.00066, heavy-a $0.00033

Global top 2 keys by total spend are heavy-a ($0.001626) and heavy-b ($0.00126). claude-only ($0.00066) is outside the cap although it is the top key on claude-sonnet-4-5

Before (1e6c983, origin/main)

Aggregated model breakdown drops the model's top key

1. grep -c model_top_keys litellm/proxy/management_endpoints/internal_user_endpoints.py prints 0
2. curl -s -H "Authorization: Bearer sk-1234" "http://localhost:4000/user/daily/activity/aggregated?start_date=2026-09-19&end_date=2026-09-27" then reading .results[0].breakdown.models["anthropic/claude-sonnet-4-5"].api_key_breakdown gives one key only:

   anthropic/claude-sonnet-4-5 {'9a87394760': 0.00033}
   

   That hash is lit8618-heavy-a. lit8618-claude-only is missing, so the dashboard ranks from one key

Per-model route does not exist

1. curl -s -o /dev/null -w "HTTP %{http_code}\n" -H "Authorization: Bearer sk-1234" "http://localhost:4000/user/daily/activity/aggregated/model_top_keys?start_date=2026-09-19&end_date=2026-09-27&model=anthropic/claude-sonnet-4-5&group_by=model"

   HTTP 404
   

Admin UI

1. Screenshot under User Flow, Before: the claude-sonnet-4-5 row lists lit8618-heavy-a only

After (949123b)

Aggregated model breakdown drops the model's top key

1. grep -c model_top_keys litellm/proxy/management_endpoints/internal_user_endpoints.py prints 3
2. The same aggregated curl still returns the globally capped breakdown ({'9a87394760': 0.00033}), by design: the dashboard no longer reads api_key_breakdown for the Top Virtual Keys card on model rows

Per-model route does not exist

1. curl -s -H "Authorization: Bearer sk-1234" "http://localhost:4000/user/daily/activity/aggregated/model_top_keys?start_date=2026-09-19&end_date=2026-09-27&model=anthropic/claude-sonnet-4-5&group_by=model" returns (key_alias, spend, api_requests):

   [('lit8618-claude-only', 0.00066, 2), ('lit8618-heavy-a', 0.00033, 1)]
   

2. Same request with model=claude-sonnet-4-5&group_by=model_group returns the same two keys in the same order

Admin UI

1. Screenshot under User Flow, After: the claude-sonnet-4-5 row lists lit8618-claude-only (2 requests, 60 tokens) first, then lit8618-heavy-a (1 request, 30 tokens)

After, re-run on 4c199bb (main merged in)

Same proxy, USAGE_TOP_API_KEYS_LIMIT=2, fresh keys lit8618v2-heavy-a (3 requests on openai/gpt-4.1-mini, 1 on openai/gpt-4.1), lit8618v2-heavy-b (3 on gpt-4.1-mini) and lit8618v2-b-only (2 on gpt-4.1). The database also holds older keys that outspend all three, so the global top two contain none of them

1. curl -s -H "Authorization: Bearer $KEY" "http://localhost:4000/user/daily/activity/aggregated?start_date=2026-09-26&end_date=2026-09-28" | jq '.results[0].breakdown.models["openai/gpt-4.1"].api_key_breakdown | map_values(.metrics.spend)' prints {}: the capped aggregate has no key at all for this model, so the old client-side card would be empty
2. curl -s -H "Authorization: Bearer $KEY" "http://localhost:4000/user/daily/activity/aggregated/model_top_keys?start_date=2026-09-26&end_date=2026-09-28&model=openai/gpt-4.1&group_by=model" | jq '[.api_keys[] | [.key_alias, .spend, .api_requests]]' prints [["lit8618v2-b-only",0.12,2],["lit8618v2-heavy-a",0.06,1]]
3. Same request with model=<public model group name>&group_by=model_group prints the same two keys in the same order

Regression test

tests/integration/spend/test_team_daily_activity_aggregated.py::test_model_top_keys_ranks_keys_by_spend_on_the_model_not_total_spend drives the real proxy against Postgres: a key that spends more in total but less on the queried model ranks second on /team/daily/activity/aggregated/model_top_keys, with exact spend values. The two scenario models now use distinct underlying model values (openai/gpt-4o-mini vs openai/gpt-4o), since the default group_by=model filters the underlying model column, not the public alias

tests/test_litellm/proxy/management_endpoints/test_common_daily_activity.py::test_model_top_api_keys_ranks_a_key_the_global_cap_dropped seeds USAGE_TOP_API_KEYS_LIMIT + 2 keys into a real Postgres, asserts the aggregated breakdown drops the outlier and the per-model query ranks it first with exact spend, request and token values. Mutation check: adding a global top-N api_key IN (...) filter to the per-model query fails it with [('key-101', 0.1)] == [('key-outlier', 0.5), ('key-101', 0.1)]

Type

🐛 Bug Fix

Caveats (if any)

Low

• Non-team entity Usage views (tag, organization, customer, agent) and the MCP view keep the client-side derivation, since their data does not go through the capped aggregated query
• Each expanded model row costs one request per expand; collapsing and reopening refetches
• The prebuilt bundle under litellm/proxy/_experimental/out/ is not rebuilt here; the weekly chore: update Next.js build artifacts job owns it
• Taxonomy audit (A-BB): H7 not applicable per the point above; H9 CI covers ui/ and tests/test_litellm/ without a paths: filter; X4 userId || undefined and empty team_ids match sibling calls and the backend's empty-list-means-nothing contract; D1 the new route sits in every route group its /aggregated sibling already does; no new module-level proxy_server import, cyclic-import check clean

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/8dbd3ca55e154740a7e27c87544f9082
Open in Devin Desktop: https://app.devin.ai/desktop/session/8dbd3ca55e154740a7e27c87544f9082?variant=devin
Requested by: @yassin-berriai

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds per-model top API keys ranking to usage analytics.

This PR appears safe to merge; no outstanding findings remain.

Summary

The PR adds server-side per-model top-key ranking and fetches it when a Usage model row is expanded, so the card can include keys omitted by the globally capped aggregate.

• The latest revision gives the integration test distinct underlying models.
• Both previous Greptile threads are resolved.

_{Reviews (4) · Last reviewed commit: "test(integration): rank model top keys b..."}

[codecov]

Codecov Report

❌ Patch coverage is 68.11594% with 22 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...xy/management_endpoints/internal_user_endpoints.py	18.75%	13 Missing ⚠️
...tellm/proxy/management_endpoints/team_endpoints.py	20.00%	8 Missing ⚠️
...roxy/management_endpoints/common_daily_activity.py	96.29%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

@greptileai addressed both threads: closed rows no longer refetch, and a database-backed per-model ranking test was added

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit_8618_model_top_keys (4c199bb) with main (f4308bc)}

[devin-ai-integration]

@greptileai merged main and added a database-backed per-model ranking regression test at 949123b, please re-review

[devin-ai-integration]

@greptileai please re-review at 4c199bb (main merged in, integration test now uses distinct underlying models)

[devin-ai-integration]

Superseded by the redesigned bounded-usage stack: repository #43398, routes #43408, UI #43409.