fix(autorouter): compare historical and new savings consistently

[tin-berri]

TLDR

Problem this solves:

• Historical savings disappear from the cost comparison
• Unknown-estimate requests inflate the displayed actual spend

How it solves it:

• Combine historical and newer savings with matching eligible costs
• Recover historical costs from bounded, reconciled retained logs

Intentional product change: the savings card compares eligible requests across both estimation methods. Unknown estimates are excluded, the duplicate actual-spend row is removed, and LLM/classification subtotals use the same eligible requests. Partial sessions with multiple baseline models omit the ambiguous model label

User Flow

Before: historical savings disappear and the card mixes different requests

1. Open http://127.0.0.1:48182/cost-optimization/ and select Auto-Router, then Usage
2. Select combined-savings-complete over the last 30 days
3. See $0.50 saved, $109 actual, a second $1 actual row, and $1.50 baseline at 33%

After: old and new savings, actual spend, baseline and percentage describe the same requests

1. Open http://127.0.0.1:48182/cost-optimization/ and select Auto-Router, then Usage
2. Select combined-savings-complete over the last 30 days
3. See $30 saved, $10 actual, $9.90 LLM, $0.10 classification, $40 baseline, and 75% savings

Validation

The restored breakdown passes 41 endpoint tests, 37 PostgreSQL tests and 63 dashboard tests. Earlier validation passed 277 backend tests. Recovery covers positive, zero and negative savings, user scope, missing/corrupt logs, changed request counts and skipped classification. Mutating the classifier subtotal to include unknown requests fails six regressions; removing the partial-session baseline-label guard also fails its regression. Earlier history and unknown-spend mutations also failed

Scoped Python/dashboard lint, type and budget gates, and generated API-type synchronization pass on the committed tip. Live benchmark and session endpoints agree on $30 saved, $10 eligible actual and $40 baseline. The dashboard shows $9.90 LLM plus $0.10 classification and uses eligible requests for the classification rate

142 added test lines versus 318 production lines, excluding generated types (0.45:1). No migrations or spend-write changes

CI and patch coverage pass on 5a40335. Veria reports no security issues on that commit. The earlier Greptile finding is fixed; fresh Greptile and Bugbot reviews have been requested

Pre-Submission checklist

• I have added meaningful tests
• The affected test files pass locally
• My PR passes all required CI/CD checks
• My PR solves one specific problem
• Greptile confidence is at least 4/5 before maintainer review

Screenshots / Proof of Fix

Real localhost proxies and browser, using the same synthetic PostgreSQL records on both revisions. Historical request: $9 actual including $0.10 classification, $29.50 recorded savings. New request: $1 actual, $0.50 saved. Unknown request: $99 actual with no estimate. A second identical session omits the retained historical log. This reads recorded charges, so provider inference does not apply

The requests below use an admin key for benchmarks and the owning virtual key for sessions

curl -sS -H "Authorization: Bearer $ADMIN_KEY" "$PROXY/auto_router/benchmarks?start_date=2026-09-26&end_date=2026-09-26"
curl -sS -H "Authorization: Bearer $VIRTUAL_KEY" "$PROXY/auto_router/session?session_id=combined-savings-complete"

Before (40297e6)

Historical, new and unknown requests

1. Run both requests with PROXY=http://127.0.0.1:48282
2. Both report $0.50 savings, $1 eligible actual and $1.50 eligible baseline; the dashboard shows 33% and $109 actual, as pictured above

Missing historical log

1. Repeat for combined-savings-missing
2. Recorded historical savings remain absent: $0.50 saved, $1.50 baseline and 33%

After (5a40335)

Historical, new and unknown requests

1. Run both requests with PROXY=http://127.0.0.1:48284
2. Both report $30 savings, $10 eligible actual, $40 eligible baseline and two estimated requests; benchmarks also return $0.10 eligible classification. The dashboard shows $9.90 LLM, $0.10 classification and 75%, as pictured above

Missing historical log

1. Repeat for combined-savings-missing
2. Recorded savings remain $30; actual, LLM, classification and baseline rows show Unavailable, and the percentage is omitted

Type

Bug Fix

Caveats

Medium

• Missing retained logs can leave historical comparisons unavailable
• Recovery stops at two seconds or the configured row cap

Low

• Date ranges still count whole overlapping sessions

Final Attestation

• Regressions cover history, unknown costs, missing logs and changed counts

---

Note

Medium Risk
Benchmark and session savings math now depends on bounded spend-log scans and reconciliation; incorrect recovery could misstate baselines, though mismatches suppress breakdown fields and preserve recorded savings totals.

Overview
Auto-router benchmarks and session APIs now keep recorded historical savings visible while aligning actual spend, baseline, classifier subtotals, and savings % to the same eligible requests—historical estimates plus newer tracked estimates, excluding turns with no savings estimate (including high-spend “unknown” requests that previously inflated actual spend).

When session rollups have partial estimate coverage, a new bounded spend-log reconciliation (historical_session_comparisons) joins retained LiteLLM_SpendLogs to rollups, verifies totals, and backfills comparison fields; failures degrade gracefully (savings dollars preserved, cost breakdown marked unavailable). Benchmark aggregation adds savings_estimated_classifier_cost and a savings_comparison_complete gate before exposing baseline math.

baseline_model now derives from baseline_models and is suppressed when partial coverage and multiple baselines would mislabel the counterfactual. The cost-optimization dashboard updates copy and layout: single “total estimated savings” hero, eligible-request spend rows, and a message when historical savings exist without matching cost details.

^{Reviewed by Cursor Bugbot for commit 5a40335. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov]

Codecov Report

❌ Patch coverage is 83.33333% with 12 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/db/autorouter_savings_comparison.py	73.80%	11 Missing ⚠️
litellm/models/autorouter_session.py	66.66%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_combined_router_savings (5a40335) with main (b248b1c)}

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Refactors how router savings estimates are calculated and compared.

The PR appears safe to merge; no new actionable issue or outstanding previous finding was established.

Summary

The PR combines historical and newer auto-router savings with costs from the same estimated requests.

• It adds bounded reconciliation against retained spend logs and exposes an eligible classification-cost subtotal.
• The dashboard uses that subtotal for its spend breakdown and marks costs unavailable when a comparison cannot be completed.
• The previously reported ambiguous baseline label is now omitted for partial sessions with multiple recorded baselines.

_{Reviews (2) · Last reviewed commit: "fix(router): avoid ambiguous baseline la..."}

[tin-berri]

@greptileai Please re-review 5a40335; the baseline-label issue is fixed, with a regression that fails when the guard is removed

[tin-berri]

bugbot run

Please review the latest commit, 5a40335, including eligible spend breakdowns and the fix for ambiguous baseline labels

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 5a40335. Configure here.}