Skip to content

fix: expired-miss share over all measured turns + cost-optimization tab labels - #36037

Merged
tin-berri merged 2 commits into
litellm_internal_stagingfrom
litellm_expired_miss_tooltip
Aug 6, 2026
Merged

tin-berri merged 2 commits into
litellm_internal_stagingfrom
litellm_expired_miss_tooltip

Conversation

@tin-berri

@tin-berri tin-berri commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • expired-miss percentage denominator was showing only return-to-tier misses, hiding the fact that rapid tier flips become cache hits
  • cost-optimization tab names were verbose

How it solves it:

  • change expired-miss denominator to all measured turns (same_model + first_visit + return_to_tier), contextualizing expired misses as a share of overall coverage
  • rename 'Usage' tab to 'Overall' and 'Auto-Router Usage' tab to 'Auto-Router'
  • update all unit and component tests to match new semantics

Relevant issues

Follow-up to #35995, which introduced the caching card and expired-miss stat

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live verification at http://localhost:3000/cost-optimization/ (Auto-Router tab):

  • Expired-miss tooltip appears on hover over the whole row (label plus percentage)
  • Percentage shows 30.0% (6 expired / 20 measured turns from seeded test sessions)
  • Tab bar shows Overall, Prompt Compression, Prompt Caching, Auto-Router

Unit tests: 91/91 pass, mutation check confirms the formula change is locked by tests

Type

Bug Fix

Changes

  • autoRouterBenchmarks.ts: expired-miss percentage now computed as (expired_misses / all_measured_turns) instead of (expired_misses / return_to_tier_misses)
  • CostOptimizationView.tsx: tab labels renamed to 'Overall' and 'Auto-Router'
  • autoRouterBenchmarks.test.ts and AutoRouterBenchmarksTab.test.tsx: updated assertions to match new denominator and label names

QA runbook

  1. Run the proxy (python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debug --reload --use_v2_migration_resolver) and the dashboard (npm run dev in ui/litellm-dashboard)
  2. Open http://localhost:3000/cost-optimization/ and switch to the Auto-Router tab
  3. Hover over the 'Expired-miss' row: the tooltip should appear
  4. Confirm the percentage reflects a share of all measured turns, not just return-to-tier misses

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Note

Low Risk
Dashboard-only analytics display and label changes with unit tests; no API or auth changes.

Overview
Auto-router cache “Expired-miss” now uses all measured turns (same model + first visit + return to tier) as the denominator instead of only return-to-tier misses, so the percentage reflects TTL-related misses as a share of overall coverage (e.g. fixture drops from ~27% to ~2.3%). The row stays visible at 0.0% when there are measured turns but no expired misses, and is hidden only when no turns were measured.

The expired-miss row is a single focusable tooltip trigger (button) for the whole label + value, with tooltip copy updated to describe the new definition.

Cost Optimization tab labels are shortened: Usage → Overall and Auto-Router Usage → Auto-Router, with matching test updates in expiredMissShare, AutoRouterBenchmarksTab, and CostOptimizationView.

Reviewed by Cursor Bugbot for commit e8cd1b5. Bugbot is set up for automated code reviews on this repo. Configure here.

@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR changes the expired-miss percentage to use all bucketed measured turns and shortens two cost-optimization tab labels.

  • Computes expired misses over the combined same-model, first-visit, and return-to-tier turn total.
  • Keeps the zero-percent row visible whenever turns were measured.
  • Makes the full expired-miss row a focusable tooltip trigger.
  • Renames the tabs to “Overall” and “Auto-Router” and updates corresponding tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/autoRouterBenchmarks.ts Updates the expired-miss denominator to the combined measured-turn bucket total and preserves null only for an empty dataset.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/AutoRouterBenchmarksTab.tsx Makes the full expired-miss row a focusable tooltip trigger and updates its explanation to match the new metric.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/CostOptimizationView.tsx Shortens the visible Usage and Auto-Router Usage tab labels without changing their stable keys or content.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/autoRouterBenchmarks.test.ts Covers the revised denominator, zero-expired-miss behavior, and no-measured-turn case.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/AutoRouterBenchmarksTab.test.tsx Verifies the revised displayed percentage, zero row, empty state, and focusable full-row trigger.
ui/litellm-dashboard/src/app/(dashboard)/cost-optimization/_components/CostOptimizationView.test.tsx Updates tab rendering and selection assertions for the shortened labels.

Reviews (2): Last reviewed commit: "fix: auto-router expired-miss percentage..." | Re-trigger Greptile

@tin-berri tin-berri changed the title fix(ui): make the expired-miss stat row a focusable tooltip trigger fix: auto-router expired-miss tooltip scope and cost-optimization tab labels Aug 6, 2026
@tin-berri tin-berri changed the title fix: auto-router expired-miss tooltip scope and cost-optimization tab labels fix: expired-miss share over all measured turns + cost-optimization tab labels Aug 6, 2026
@tin-berri
tin-berri force-pushed the litellm_expired_miss_tooltip branch from 9c00692 to 7d40067 Compare August 6, 2026 04:49
@tin-berri
tin-berri enabled auto-merge (squash) August 6, 2026 04:58
…bels

- change expired-miss percentage denominator from return-to-tier misses to
  all measured turns (same_model + first_visit + return_to_tier). when
  auto-routers flip tiers rapidly within TTL, return-to-tier turns become
  hits and disappear from the miss count; the old metric reported only the
  rare failure population. the new metric contextualizes that population as
  a share of overall coverage
- rename usage tab from 'Usage' to 'Overall'
- rename auto-router-usage tab from 'Auto-Router Usage' to 'Auto-Router'
- update component and unit tests to match new semantics
@tin-berri
tin-berri force-pushed the litellm_expired_miss_tooltip branch from 7d40067 to e8cd1b5 Compare August 6, 2026 05:09
@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@mateo-berri

Copy link
Copy Markdown
Contributor

@greptileai

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit e8cd1b5. Configure here.

@mateo-berri mateo-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@tin-berri
tin-berri merged commit 34fc8d2 into litellm_internal_staging Aug 6, 2026
78 checks passed
@tin-berri
tin-berri deleted the litellm_expired_miss_tooltip branch August 6, 2026 05:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants