Repository navigation
feat: add model leaderboard page - #43649
Conversation
|
|
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
@greptile review |
…to ranked deployments
|
@greptile review |
|
@greptile review |
| await table.group_by( | ||
| by=["task_type", "model_group", "model", "custom_llm_provider"], | ||
| sum=_SUM_FIELDS, | ||
| where=date_window, | ||
| order={"_sum": {"completion_tokens": "desc"}}, | ||
| ) |
There was a problem hiding this comment.
Task responses grow with deployments
Removing the 500-row cap makes each /model-insights response include every task/deployment/provider combination in the selected period. On gateways with many deployments, the default 365-day view can return a large amount of data, increasing response and browser processing costs. Switching the chart metric fetches the same task breakdown again even though it does not depend on that metric.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
|
@greptile review |
[TLDR](feat: add model leaderboard page)
Problem this solves:
How it solves it:
User Flow
Before: a proxy admin has no page that shows which models the gateway is using
After: the same admin sees model usage ranked and charted in one place
Relevant issues
Linear ticket
Pre-Submission checklist
Screenshots / Proof of Fix
Before (98c710c)
After (ebcedb6)
--use_v2_migration_resolver, then open http://localhost:3000/model-insightscurl http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" -d '{"model":"claude-sonnet-4-5","messages":[{"role":"user","content":"Reply with the word ok"}],"max_tokens":10,"metadata":{"tags":["task:debugging"]}}'debuggingrequest forclaude-sonnet-4-5appears inLiteLLM_DailyModelUsage, with 12 prompt and 4 completion tokensType
🆕 New Feature
Caveats (if any)
Medium
task:<name>request tagsuncategorizedLow
Final Attestation