Skip to content

fix(ui): drive auto-router usage from the shared cost-optimization time picker - #37871

Merged
tin-berri merged 2 commits into
litellm_internal_stagingfrom
litellm_autorouter_shared_time_picker
Aug 21, 2026
Merged

fix(ui): drive auto-router usage from the shared cost-optimization time picker#37871
tin-berri merged 2 commits into
litellm_internal_stagingfrom
litellm_autorouter_shared_time_picker

Conversation

@tin-berri

@tin-berri tin-berri commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Overall tab and Auto-Router tab showed different auto-router savings
  • Overall used the time range picker, Auto-Router used its own 24h/7d/30d toggle

How it solves it:

  • Auto-Router usage now reads the page's shared Select Time Range picker
  • Removes the tab-local 24h/7d/30d toggle and its window math
  • A range ending today extends to the current UTC day, like the Overall reads

User Flow

Before: an admin comparing savings across tabs sees numbers that look contradictory

  1. They open https://litellm-domain/ui/cost-optimization and the Overall tab shows Auto-router savings of $1,447.76 for the picked range of the last 30 days
  2. They click the Auto-Router tab, which defaults to its own "24h" toggle and shows Total estimated savings of $156.37
  3. Nothing on either tab says the two figures cover different windows, so the dashboard looks broken

After: both tabs answer for the same window, picked in one place

  1. They open https://litellm-domain/ui/cost-optimization and the Overall tab shows Auto-router savings for the picked range
  2. They click the Auto-Router tab and see the same Select Time Range picker with the same range already applied, and the totals now line up with the Overall card
  3. Changing the range on either tab changes it everywhere, and fine print on the Auto-Router tab explains the remaining small difference: its totals count whole sessions overlapping the range while the Overall tab buckets by UTC day

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Setup for the parity case: proxy on this branch at localhost:4200, a complexity auto-router with tiers claude-haiku-4-5 (SIMPLE), claude-sonnet-4-5 (MEDIUM), claude-opus-4-5 (COMPLEX and REASONING) at public per-token prices, llm classifier on haiku. Upstream is an internal gateway fronting the real Anthropic-served models rather than api.anthropic.com directly (the local Anthropic key is out of credit; the calls are real completions with real spend). The QA database had zero auto-router traffic for the day before the run

Before (8122cfc)

Shared range across tabs

  1. Open http://localhost:4000/ui/cost-optimization, note the Auto-router savings card and the picked range on the Overall tab
  2. Click the Auto-Router tab: it shows its own 24h/7d/30d toggle defaulting to 24h, so its Total estimated savings covers a different window than the Overall card and the two figures disagree

Savings parity on real traffic

  1. The Auto-Router tab cannot be set to the Overall tab's picked range at all, so the two savings figures cannot be compared like for like

After (aaa9757)

Shared range across tabs

  1. Open http://localhost:4000/ui/cost-optimization, note the Auto-router savings card and the picked range on the Overall tab
  2. Click the Auto-Router tab: the same Select Time Range picker appears with the same range already applied and the toggle is gone
  3. Change the range on the Auto-Router tab, return to Overall, and see the same range applied there too
  4. West of UTC after 5pm local, start a fresh session through an auto-router and see it counted on both tabs without waiting for local midnight

Savings parity on real traffic

  1. Send 9 chat completions of mixed complexity through the router: for p in "what is 2+2?" ... "architect a multi-region active-active postgres deployment..."; do curl -s http://localhost:4200/v1/chat/completions -H "Authorization: Bearer $KEY" -d '{"model":"auto_router_real","max_tokens":150,"messages":[{"role":"user","content":"'"$p"'"}]}'; done (all 9 return real completions; the classifier spreads them SIMPLE 3, MEDIUM 2, COMPLEX 2, REASONING 2)
  2. curl -s "http://localhost:4200/auto_router/benchmarks?start_date=2026-08-21&end_date=2026-08-21" -H "Authorization: Bearer $KEY" returns sessions 9, spend 0.020592, saved_spend 0.004138, baseline_spend 0.024730 (this feeds the Auto-Router tab's Total estimated savings)
  3. curl -s "http://localhost:4200/user/daily/activity?start_date=2026-08-21&end_date=2026-08-21&timezone=420&include_current_utc_day=true" -H "Authorization: Bearer $KEY" returns autorouter_savings_spend 0.004138 for the day (this feeds the Overall tab's Auto-router savings card)
  4. Parity: 0.004138 - 0.004138 = 0.000000000, an exact match on the same window; the per-model breakdown agrees too (opus turns save 0 because opus is the baseline, sonnet saves 0.003090, haiku saves 0.001048)
  5. The Overall day's total spend (0.025839) exceeds the router's own spend (0.020592) by exactly the internal classifier calls' cost, which both savings figures correctly exclude

Type

🐛 Bug Fix

Caveats (if any)

  • Small residual difference stays by design: sessions overlapping the range count whole, per-day UTC buckets do not; fine print on the tab now says so
  • The benchmarks endpoint has no timezone param, so the live-end UTC extension mirrors _adjust_dates_for_timezone client side; historical dates stay pass-through, matching the daily-activity policy
  • The parity run's upstream was the internal gateway rather than api.anthropic.com; a direct-key re-run is owed once the Anthropic account has credit

@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR makes Auto-Router usage consume the shared cost-optimization date range and removes its independent window selector.

  • Passes the shared activity range through the Auto-Router tab.
  • Aligns benchmark query dates with the Overall tab’s current-UTC-day behavior.
  • Adds focused component and date-boundary tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useAutoRouterBenchmarks.ts Converts the shared picker range into benchmark query dates while matching the Overall tab’s UTC-day extension.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/AutoRouterBenchmarksTab.tsx Replaces the tab-local duration selector with the shared date picker and documents the intentional session-overlap difference.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/CostOptimizationView.tsx Supplies the existing shared activity range to the Auto-Router tab.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useAutoRouterBenchmarks.test.ts Covers historical ranges, incomplete ranges, and current-day behavior east and west of UTC.

Reviews (2): Last reviewed commit: "fix(ui): extend a live-ending benchmarks..." | Re-trigger Greptile

@tin-berri

Copy link
Copy Markdown
Contributor Author

@greptileai can you review here and reply with score

@tin-berri

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit aaa9757. Configure here.

@tin-berri
tin-berri merged commit 9821b45 into litellm_internal_staging Aug 21, 2026
70 of 71 checks passed
@tin-berri
tin-berri deleted the litellm_autorouter_shared_time_picker branch August 21, 2026 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants