feat(ui): add the auto-router usage tab to cost optimization

[tin-berri]

TLDR

Problem this solves:

• The per-session auto-router benchmarks backend (feat(spend): rebuild the auto-router benchmarks backend as a per-session rollup #35910) is merged but nothing in the dashboard renders it, so the savings evidence customers asked for is only reachable by curl
• Earlier tab attempts (feat(ui): rebuild the auto-router benchmarks tab around savings and cache behaviour #35488, feat(ui): lead the benchmarks tab with savings and the sessions it was earned over #35741) targeted the closed v2 contract and never landed

How it solves it:

• Adds an Auto-Router Usage tab to Cost Optimization that reads GET /auto_router/benchmarks through the typed openapi client and renders a savings hero (actual routed spend against the estimated highest-cost-model spend, sessions, turns, avg saved per session), three session-shape tiles and a prompt-caching card with a share-of-turns breakdown
• A 30d / 7d / 24h window picker drives the endpoint's start_date / end_date params; the router selector maps to the server totals or one group's slice, so the client re-aggregates nothing and cannot disagree with the backend or the Usage tab

Relevant issues

• New dashboard tab rendering the v3 auto-router benchmarks rollup, base shadcn components only
• Supersedes the UI half of feat(proxy): auto-router session benchmarks endpoint and UI #35190, whose spend-log-scan backend was replaced by feat(spend): rebuild the auto-router benchmarks backend as a per-session rollup #35910
• No backend change; schema.d.ts already carries the endpoint types so networking.tsx is untouched

Linear ticket

Resolves LIT-4712

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live rig: worktree proxy on localhost:4790, fresh Postgres with the #35910 schema, two auto_router/complexity_router routes (auto, claude-auto) with a Bedrock haiku/sonnet/opus tier ladder. 22 real Bedrock turns were driven across 5 sessions with x-litellm-session-id headers and a cached system prefix, costing real money and producing real cache reads. One session was then aged 10 days back in the rollup so the window picker visibly changes the data

Empty state before traffic (no params; the server defaults the 30 day window):

curl -s -H "Authorization: Bearer sk-1234" "http://localhost:4790/auto_router/benchmarks"
{"start_date": "2026-07-06", "end_date": "2026-08-05", "routers_in_scope": 0,
 "totals": {"sessions": 0, "turns": 0, ... all zeros ...}, "groups": []}

After the turns landed in the rollup, the same curl reports 2 routers in scope with 5 sessions over 22 turns, spend 0.0299 against a 0.1305 baseline (77.1 percent saved), and per-bucket cache stats; narrowing the window excludes the aged session:

curl -s -H "Authorization: Bearer sk-1234" "http://localhost:4790/auto_router/benchmarks?start_date=2026-08-04&end_date=2026-08-05"
-> 4 sessions, 19 turns

UI walkthrough (the rig serves the rebuilt bundle; the bundle itself is deliberately not committed):

1. Open http://localhost:4790/ui and log in as admin with the rig master key
2. Go to http://localhost:4790/ui/cost-optimization and open the Auto-Router Usage tab
3. The hero card shows total estimated savings with a green delta badge, the actual auto-router spend against the estimated spend at the highest-cost model, total sessions and turns, and avg saved per session; below it the three tiles and the prompt caching card with the share-of-turns bar, per-bucket rows and the expired-miss stat with its tooltip
4. Flip the 30d / 7d / 24h picker: the subtitle follows and the numbers narrow (24h drops the aged session). Switch the selector to claude-auto: every number narrows to that router's slice

Type

🆕 New Feature

Changes

• New AutoRouterBenchmarksTab under cost-optimization _components: savings hero (spend vs highest-cost baseline, total sessions/turns, avg saved per session), session tiles, prompt-caching card (share-of-turns bar with per-segment shares, bucket table, expired-miss stat with tooltip), all base shadcn
• 30d / 7d / 24h window picker wired to the endpoint's start_date/end_date (UTC calendar days; server owns the default window); header and controls stay reachable in loading, error and empty states
• autoRouterBenchmarks.ts view helpers: selection (viewFor, duplicate-alias safe), bucket rows with shares, expired-miss share, window derivation; types come from the generated schema.d.ts and useAutoRouterBenchmarks.ts wraps $api.useQuery
• Tab registered in CostOptimizationView as Auto-Router Usage; the page alert now links the auto-router discussion 32168; a 403 renders an admin-only message since the nav also admits internal users
• ~470 test lines: helper unit tests (selection, shares, window math incl. UTC edge, zero guards) and component state tests (loading, 403 vs generic error, empty with reachable picker, headline numbers, bucket rows, picker refetch)

QA runbook

1. Run a proxy with a DB and at least one auto_router/complexity_router model, e.g. the config in the proof section
2. Send a few multi-turn chat completions to the router alias with an x-litellm-session-id: <session> header per conversation
3. Wait for the spend writer flush, then open the dashboard's Cost Optimization page and the Auto-Router Usage tab
4. Verify the hero, tiles and caching numbers against curl /auto_router/benchmarks, flip the window picker and verify against the same curl with explicit start_date/end_date, then filter by one router and verify against that group's entry
5. Log in as a non proxy-admin internal user and confirm the tab shows the admin-only message instead of a raw error

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Low Risk
Dashboard-only feature behind existing admin-gated API; no backend or auth changes, with broad unit/component test coverage.

Overview
Adds an Auto-Router Usage tab on Cost Optimization that surfaces GET /auto_router/benchmarks via the typed OpenAPI client, so savings and cache evidence from the merged backend is visible in the dashboard instead of only via curl.

The tab includes a savings hero (estimated savings vs actual spend and highest-cost baseline, sessions/turns, avg saved per session), session-shape metrics, and a prompt-caching breakdown (bucket shares, hit rates, optional expired-miss). 30d / 7d / 24h windows map to start_date / end_date; the router selector uses server totals or a single group slice through viewFor so the UI does not re-aggregate. Loading, 403 (admin-only), error, and empty states keep header controls usable.

CostOptimizationView registers the fourth tab and points the experimental alert at discussion 32168. Helper and component tests cover selection, bucket math, window UTC behavior, and UI states.

^{Reviewed by Cursor Bugbot for commit c10d695. Bugbot is set up for automated code reviews on this repo. Configure here.}

[tin-berri]

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

Score: 4/5

The implementation is clean and well-scoped. Here's the reasoning:

What's good:

• Types come directly from schema.d.ts — no manual type duplication
• Client never re-aggregates; viewFor maps to data.totals or a group's slice directly from the server response
• 403 vs generic error handling is correct and the admin-only message is a nice UX touch
• Test coverage is solid: helper unit tests + component state tests covering loading, error, empty, and all headline numbers
• No backend changes, no fastapi imports, no provider-specific code in wrong places

What keeps it from a 5:

1. returnMissShare silent case: when !withinTtl && !unknown (all non-expired misses are fully accounted for by return_misses_expired), the function falls into the catch-all and renders "the rest missed because the prefix changed or carried no TTL telemetry" even though neither count is > 0. The "rest" is 0 misses but the sentence still renders. A guard for this case — either returning null when expiredPct rounds to 100, or an explicit !withinTtl && !unknown branch — would fix it. This case also has no test coverage.

2. onValueChange type annotation: (value: string | null) on the Select is technically wrong — shadcn/radix types onValueChange as (value: string) => void, so the ?? ALL_ROUTERS null-coalesce is dead code. Minor, but slightly misleading to future readers.

[greptile-apps]

Greptile Summary

Adds an Auto-Router Benchmarks tab to Cost Optimization using the generated OpenAPI query client

• Displays backend-provided savings, session, router, and prompt-cache aggregates
• Supports selecting an individual router group without re-aggregating data in the client
• Handles loading, authorization, generic error, and empty-result states
• Adds focused unit and component coverage for selection, formatting, cache attribution, and rendering states

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code failure identified

The selector uses server-provided aggregate slices, backend invariants protect the displayed calculations and empty states, and the feature uses the dashboard’s generated HTTP client with explicit authorization and error handling

Important Files Changed

Filename	Overview
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/AutoRouterBenchmarksTab.tsx	Adds the benchmark presentation, router selector, and explicit loading, authorization, error, and empty states without an accepted correctness issue
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/autoRouterBenchmarks.ts	Adds typed selection and display helpers that align with the backend aggregation and cache-attribution contracts
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useAutoRouterBenchmarks.ts	Adds a generated OpenAPI query hook gated on the supplied access token
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/CostOptimizationView.tsx	Registers the new benchmark tab while preserving Usage as the default tab

_{Reviews (1): Last reviewed commit: "feat(ui): add the auto-router benchmarks..." | Re-trigger Greptile}

[tin-berri]

Fixed the all-expired case: restReason is now null there and the sentence ends at the TTL, with a unit and a component test pinning it

On the Select typing: this dashboard's Select is base-ui, not radix, and its onValueChange is typed (value: string | null, eventDetails) => void; dropping the null coalesce fails tsc with TS2322, so the annotation stays

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

4/5

Both fixes check out:

• restReason: null in the all-expired branch is correct — the !withinTtl && !unknown guard lands there cleanly, and both the unit test (drops the rest clause when every miss expired past the TTL) and the component test (ends the miss sentence at the TTL when every miss expired) pin the exact rendered output. No issues there.
• The value ?? ALL_ROUTERS null coalesce is justified if base-ui's onValueChange types value as string | null; the annotation is load-bearing, not cosmetic.

The one remaining reason I'm holding at 4 rather than 5: the "Last 30 days" label in the heading is hardcoded even though the server returns start_date / end_date in the response. If the server is ever called with explicit date params, the label will lie. It's minor (the 30-day default is correct for the current call), but it's the kind of thing that silently misfires when someone wires up a date-range picker later. Not a blocker, just a note for when that comes.

[cursor]

Disabled query stuck on loading

Low Severity

The tab gates the query with enabled: Boolean(accessToken) but treats isPending as loading. In TanStack Query v5 a disabled query with no cache stays isPending, so a missing token never reaches the unavailable path and the tab can remain on the loading message.

Additional Locations (1)

• ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/useAutoRouterBenchmarks.ts#L2-L4

 

^{Reviewed by Cursor Bugbot for commit 89fb62f. Configure here.}

[tin-berri]

Null token cannot persist here: useAuthorized reads the cookie synchronously and redirects to login (useAuthorized.ts:30-39), so loading renders only for the pre-navigation instant

[tin-berri]

Pushed two follow-ups from design review: the hero and caching cards now use fractional grid columns (5fr 3fr and 1fr 3fr) so their proportions match the design at any viewport width instead of fixed rem columns, and the experimental dashboard alert now links the auto-router discussion 32168

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

4 / 5

The two follow-up changes are both correct and improve the PR:

• 5fr 3fr / 1fr 3fr grid columns — this is the right call. Arbitrary fractional columns scale proportionally at any viewport width; fixed rem columns would overflow on narrow screens or leave awkward whitespace on wide ones. The breakpoints chosen (md: for hero, lg: for caching) also match the relative width of each card sensibly.
• Link 32172 → 32168 — straightforward correction, verified against the PR description.

One minor thing I notice in the component that predates these follow-ups but is worth a note: the onValueChange callback in the router selector is typed (value: string | null) => ..., but shadcn/radix's Select.onValueChange always passes string, never null. The ?? ALL_ROUTERS guard is harmless but the type annotation is technically wrong and could suppress a real type mismatch if the API changes. Not a blocker, but worth tightening to (value: string) => setSelectedKey(value).

Everything else — the viewFor server-authoritative aggregation, the 403 vs generic error split, the retry: false on the hook, test coverage on edge cases — is solid. The follow-ups don't introduce any new issues.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 8fa894e. Configure here.}

[tin-berri]

The Select here is base-ui (components.json style base-vega), not radix: SelectPrimitive.Root types onValueChange as (value: string | null, eventDetails) => void, and changing the annotation to (value: string) fails tsc with TS2322 (verified locally against this branch). The null coalesce and annotation are required by the actual library, so this stays as is

@greptileai can you reply here with a score and why that score?

[greptile-apps]

That's a valid correction. If the project's Select wraps a base-ui primitive whose onValueChange is typed (value: string | null, eventDetails?) => void rather than Radix's (value: string) => void, then both the string | null annotation and the ?? ALL_ROUTERS coalesce are required — removing them would break tsc exactly as you described.

I was assuming the standard shadcn/Radix contract; the local type definition overrides that assumption. The code is correct as written.

Score: 5/5 — the implementation is sound. The tab reads server aggregates without client re-aggregation, handles all four states (loading, 403, error, empty), the selector logic falls back cleanly on stale keys, and the test suite covers the edge cases that matter (duplicate router aliases, negative savings, all-expired misses, zero-session empty state). No concerns remain.

Shall I remember this as a rule: "In this repo, the Select component comes from base-ui (base-vega), not Radix, so onValueChange is typed (value: string | null) => void and null coalescing in the handler is correct and required"?

[tin-berri]

Pushed the design iteration round: the tab is now titled Auto-Router Usage, the hero shows actual vs highest-cost-model spend with total sessions/turns and avg saved per session, a 30d/7d/24h window picker drives the endpoint's date params, the caching card gained per-segment shares and an expired-miss stat with a tooltip, and the page alert links discussion 32168

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

4/5

The implementation is clean and well-tested. Here is the breakdown:

What's solid:

• ~470 lines of tests cover loading, 403 vs generic error, empty state, window picker state, headline numbers, bucket rows, and the UTC midnight edge case — this is genuine coverage, not theatre
• viewFor never re-aggregates client-side: it either returns data.totals or a single group's pre-computed slice, so it cannot disagree with the backend or the Usage tab
• 403 / generic error / empty state are all handled before the data branch, so the picker stays reachable in all non-data states
• groupKey deduplication by router_name + router_type is correct and tested

Minor issues that keep it from 5/5:

1. sharePct vs bar width discrepancy — bucketRows pre-rounds sharePct with Math.round, but StackedTurnBar recalculates the actual float for the CSS width. The coloured bar and the label below it can differ by up to 1 percentage point. Consider using b.sharePct for both the bar width and the label to guarantee they match.

2. Redundant view === null guard — view can only be null when data is falsy, which is already caught by !data on the same line in BenchmarksBody. It does not cause a bug, but it signals uncertainty about the invariant.

3. windowFor(range, new Date()) in the hook — called at render time, so the query key silently changes at UTC midnight without user action. For a benchmarks tab this is acceptable, but worth a comment.

None of these are regressions or security issues. The tab is a net positive — it surfaces the #35910 backend data through a correctly wired, well-tested UI with no backend changes required.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 498cade. Configure here.}

[tin-berri]

Both minor findings addressed: the share bar segments and their labels now render from the same rounded sharePct so they cannot diverge, and BenchmarksBody takes selectedKey and derives the view after the data guard, removing the nullable and the redundant check. On the midnight query-key note: agreed it is acceptable for this tab, and the repo bans explanatory comments, so noting it here instead

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

5/5.

Both findings from the prior round are confirmed resolved:

• Share bar / label alignment — StackedTurnBar and BucketTable both source segment widths and label text from b.sharePct (the same Math.round-ed value from bucketRows), so a segment and its label are guaranteed to agree. The overflow-hidden container handles the cosmetic case where rounded shares sum to 101%.
• BenchmarksBody nullable — selectedKey is a plain string prop derived from useState; the view is computed after all the early-return guards, so viewFor always receives a non-null data argument and the redundant null-check is gone.

Everything else holds: the logic helpers are pure and fully unit-tested (selection, fallback, bucket shares, zero-division, expired-miss, UTC window math), the component tests cover all five render states plus picker interactions, the ALL_ROUTERS sentinel never leaks into the UI, and there are no backend, auth, or SDK surface changes. The midnight query-key behaviour is a known, acceptable trade-off for this tab.

Clean implementation, thorough tests, the review feedback was taken seriously — good to go.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit c10d695. Configure here.}