fix(auto-router): count usage savings by selected UTC request day

[tin-berri]

TLDR

Problem this solves:

• Auto-Router usage showed whole-session savings for any overlapping session
• Its total disagreed with the Overall tab for the same dates

How it solves it:

• Record auto-routed spend and savings per UTC request day and router
• Read the all-router total from the same daily rows Overall reads
• Flag savings recorded before this change as unattributed, with no baseline

Intentional product change: the "Avg saved per session" tile is removed, because it divided selected-day money by whole-session counts. Session averages and caching stay whole-session and are now labelled that way

User Flow

Before: an admin picking a date range sees more Auto-Router savings than Overall reports

1. Open https://litellm-domain/ui/?page=cost-optimization and pick Aug 2 – Sep 1
2. Overall shows $10.00 auto-router savings
3. Auto-Router shows $21.00, because a session that ran past Sep 1 counts whole

After: both tabs report the savings earned on the selected days

1. Open https://litellm-domain/ui/?page=cost-optimization and pick Aug 2 – Sep 1
2. Overall shows $10.00 auto-router savings
3. Auto-Router shows $10.00, with session metrics labelled as whole-session

Linear ticket

Resolves LIT-9171

Follow-up: LIT-9172 moves the per-router daily write into the daily spend writer, so per-router totals also add up with disable_spend_logs on and for sessionless requests

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Rig: local proxy with Postgres 16 and a complexity auto-router whose tiers are real Anthropic models reached through the sandbox gateway. The September rows are seeded fixture records (one session earning 7, 3 and 11 on Aug 31, Sep 1 and Sep 2), not provider traffic. Each step compares GET /user/daily/activity/aggregated (Overall) with GET /auto_router/benchmarks (Auto-Router) for the same dates

Before (d7e6154, merge base before rebase)

Selected-day windows

1. Sep 1: Overall 3.0, Auto-Router 21.0
2. Aug 31: Overall 7.0, Auto-Router 21.0
3. Sep 2: Overall 11.0, Auto-Router 21.0
4. Aug 31 – Sep 2: Overall 21.0, Auto-Router 21.0

Dashboard

1. Aug 2 – Sep 1: Overall $10.00, Auto-Router $21.00 (screenshot above)

After (33eea0b)

Selected-day windows

1. Sep 1: Overall 3.0, Auto-Router 3.0
2. Aug 31: Overall 7.0, Auto-Router 7.0
3. Sep 2: Overall 11.0, Auto-Router 11.0
4. Aug 31 – Sep 2: Overall 21.0, Auto-Router 21.0, so adjacent windows add up
5. History with no per-router day rows: same totals, unattributed_saved_spend set, baseline and percentage null on totals and on each router group
6. Same session with per-router day rows: router group Sep 1 saved 3.0, baseline 4.0, 75%
7. Empty key filter: GET /auto_router/benchmarks?api_key= returns 422
8. Sep 1 scoped by key, by user, and by both: Auto-Router 3.0 each, Overall with the same user filter 3.0
9. One alias changing router type mid-session: the session stays with its first type (1 session, 2.0 turns, 3600s); the new type's group shows its own $2.00 with session averages unavailable rather than 0

Real traffic

1. Three POST /v1/chat/completions to smart-router in one session: 200s routed to haiku, sonnet, haiku
2. Today's window: Overall 0.031958, Auto-Router 0.031958, baseline 0.05284, 60.5%
3. GET /auto_router/session still reports the whole session: 3 turns, saved 0.031958

Dashboard

1. Aug 2 – Sep 1: Overall $10.00, Auto-Router $10.00 (screenshots above)

Type

🐛 Bug Fix

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes spend accounting SQL, a new migration, and a breaking API shape for benchmarks; correctness depends on daily upserts staying in sync with session rollups and baseline corrections.

Overview
Fixes Auto-Router usage reporting so spend and savings match the selected UTC date range (and the Overall tab), instead of summing whole overlapping sessions.

Adds LiteLLM_AutoRouterDailySpend, written in the same upsert as session rollups and corrected when late baselines publish. AUTOROUTER_BENCHMARKS_SQL now joins session rows (shape, cache, session_turns) with daily rows (money filtered by $5–$6). The benchmarks API sets the headline saved_spend from LiteLLM_DailyUserSpend.autorouter_savings_spend, exposes unattributed_saved_spend when that total disagrees with per-router daily sums, and drops saved_per_session. Session averages can be null when routed requests have no matching session row; cache coverage uses session_turns.

Retention cleanup deletes expired daily rollup rows with session retention. The cost-optimization UI removes the avg-saved-per-session tile and clarifies selected-day vs whole-session metrics.

^{Reviewed by Cursor Bugbot for commit 33eea0b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[tin-berri]

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

Score: 4/5

This is a well-scoped fix with a sound accounting model: per-UTC-day/router rows are written atomically with the session rollup, late baseline corrections update those rows, and the endpoint’s headline total now reads the same daily spend source as Overall. The cross-midnight, user/key scoping, legacy/unattributed history, failed projection rollback, and late-replay tests provide strong coverage. The UI also clearly distinguishes selected-day money from whole-session metrics and removes the misleading average-savings tile.

I’m stopping at 4 rather than 5 because the required CI/e2e status is still pending, so the migration and concurrent read/write behavior have not yet been validated by the full pipeline. Subject to those checks passing, this looks ready to merge.

[greptile-apps]

Confidence Score: 5/5

[Critical risk] Adds database table and modifies spend accounting logic.

The PR appears safe to merge based on the reviewed changes

Findings

1.  New code mutates containers ▶

Summary

The PR records auto-router spend by UTC request day and uses those rows for date-selected benchmarks

• The latest revision combines session metrics and daily money in one benchmark query
• Router groups without session rows now report unavailable session averages rather than zero
• The earlier mutating merge and test-call collection have been removed

_{Reviews (3) · Last reviewed commit: "fix(auto-router): count usage savings by..."}

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai can you reply here with a score and why that score? All four findings are addressed in 4619950 (router_type in the day key, router-group baselines null under an unattributed residual, day-row retention on the session horizon, empty api_key rejected)

[tin-berri]

bugbot run

[greptile-apps]

Score: 5/5

The follow-up commit addresses all four findings:

• router_type is included in the daily-rollup conflict key, so same-day reconfiguration keeps spend, savings, and tier data separated.
• When recorded savings exceed the router-day rows, the residual is exposed as unattributed_saved_spend, and baseline and percentage comparisons are cleared for both totals and groups.
• Daily rows use the same retention horizon as session rollups, preventing selected-day money from disappearing while the corresponding session data remains available.
• Empty api_key filters are rejected during validation before any database query, matching the existing user_id behavior.

The added regression tests cover these cases, and the latest e2e status is passing. I have no remaining blocking concerns.

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai can you reply here with a score and why that score? d8a2425 only changes the unattributed-savings note wording to be accurate for a negative residual (Bugbot finding)

[tin-berri]

bugbot run

[greptile-apps]

Score: 5/5

This remains a complete fix. The latest commit only adjusts the dashboard note so the residual is described by its magnitude (abs(...)), which is accurate for both positive and negative differences while still making clear that the baseline comparison is unavailable. It does not alter the accounting, aggregation, attribution, validation, retention, or test-covered behavior previously reviewed.

I have no remaining blocking concerns; the current e2e check is in progress rather than failing.

[cursor]

Stale Bugbot comment from a previous run.

[tin-berri]

@greptileai can you reply here with a score and why that score? 6d4f699 rebases onto main (the PR was conflicting, so pull_request CI never ran). main's #43398/#43408 deleted the private daily-activity where builder this PR had made public, so the Auto-Router headline now reads through main's public daily_activity_scope + build_where_clause, the same owner every daily-activity route uses; this PR no longer touches common_daily_activity.py

[tin-berri]

bugbot run

[greptile-apps]

Score: 4/5

The rebase addresses the integration concern: the Auto-Router headline now uses main’s public daily_activity_scope and build_where_clause, matching the ownership and filtering path used by the other daily-activity routes, and the PR no longer modifies common_daily_activity.py. The previously reviewed accounting fixes remain intact: UTC-day/router attribution, unattributed legacy residual handling, retention alignment, validation, and regression coverage.

I’m keeping this at 4/5 only because the required post-rebase e2e/CI check is currently pending, so the migration and end-to-end behavior have not yet been validated by CI. I have no new code-level blocking concern from the stated rebase changes; subject to that check passing, this is ready to merge.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_autorouter_period_savings (33eea0b) with main (e5873ad)}

[cursor]

Stale Bugbot comment from a previous run.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[tin-berri]

@greptileai review

[greptile-apps]

New code mutates containers This merge assigns into a dictionary, and the new test helper appends to a list. Both violate the repository's no-mutation directive, which must be satisfied before merging

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[tin-berri]

Fixed in 33eea0b by deleting the merge rather than rewriting it: session shape and day money now come from one SQL statement (sessions FULL OUTER JOIN days USING (router_name, router_type) in AUTOROUTER_BENCHMARKS_SQL), so _window_rows and its dict assignment are gone and the endpoint makes one query fewer. The test fake records calls through AsyncMock.await_args_list instead of appending to a list

[tin-berri]

@greptileai can you reply here with a score and why that score? 33eea0b addresses both P2s: the benchmark read is now one SQL statement (sessions FULL OUTER JOIN days), removing the mutating merge, and router groups with requests but no session rows report session averages as null instead of 0

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 33eea0b. Configure here.}

[tin-berri]

@greptileai can you reply here with a score and why that score? Re-requesting on 33eea0b: the benchmark read is one SQL statement now (sessions FULL OUTER JOIN days, removing the mutating merge), and groups with requests but no session rows report session averages as null. Full CI is green apart from Analyze (python), whose LogInjection.ql 2GiB failure also occurs on main at 71788fe and e5873ad

[greptile-apps]

Score: 5/5

The latest revision resolves the remaining concerns. The benchmark read is now a single SQL query combining session metrics with daily spend via FULL OUTER JOIN, so selected-day totals are computed without a mutating application-side merge and cannot lose router-day rows that have no corresponding session row. Those router groups now correctly expose unavailable (null) whole-session averages rather than presenting misleading zeroes.

The previously reviewed safeguards remain intact: UTC-day/router attribution, unattributed legacy residual handling, retention alignment, empty-filter validation, and regression coverage. Full CI is green aside from the Python Analyze LogInjection.ql 2 GiB failure, which also reproduces on main (71788fe1c5 and e5873adbc4) and is therefore not introduced by this PR. I have no remaining blocking concerns.