feat(ptu): daily rollup writes per-model PTU flat cost by active hour

[yucheng-berri]

TLDR

Problem this solves:

• PTU config on a model does not turn into any cost yet
• Flat cost must prorate by the exact active hours, not whole days
• A day that a once-daily job never prices stays unpriced forever

How it solves it:

• A daily rollup reads model PTU config and writes ptu_count * rate * active_hours
• Active hours is the day's overlap with the optional effective window, clamped to 24
• Each run then catches up any day inside a declared window that carries no row yet

Relevant issues

Linear ticket

Resolves LIT-4077

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Captured live against a proxy on this branch (commit acf106b), real DB, no mocks. A model with ptu_count 5, cost_per_ptu_per_hour 2.0 and ptu_effective_from 2026-07-31T23:00:00Z is rolled up for 2026-07-31; the window opens at 23:00 so the day has one active hour and the flat cost is 5 * 2.0 * 1 = 10.00

run_ptu_flat_cost_rollup(prisma, target_date=2026-07-31)
-> models_processed=4 rows_written=4

# sentinel row written to LiteLLM_DailyTeamSpend (api_key='__ptu_flat_cost__')
SELECT ptu_flat_cost, ptu_source_model_id FROM "LiteLLM_DailyTeamSpend"
  WHERE api_key='__ptu_flat_cost__' AND date='2026-07-31' AND team_id=<pr2-proof team>;
-> ptu_flat_cost = 10   ptu_source_model_id = 030ac71e-... (the deployment id)

A separate always-on model in the same run wrote 480 (10 PTU * 2.0 * 24h), confirming the full-day path

The rollup's output, surfaced

One deployment at 10 PTU x $2.00/hr with no effective window accrues a full day each run. Rolling up four consecutive UTC days and viewing the team gives 4 x $480 = $1,920, one violet bar per day:

Multi-pod

Three separate proxy processes against one Postgres and one Redis, each constructing its own PodLockManager and so its own pod_id, all released at the same wall-clock instant the way a cron fires everywhere at once. Ten PTU deployments, four sentinel rows

With Redis healthy the lock elects a single writer and the other two return without touching the database:

{"pod": "pod-1", "ran": false, "ms": 7.5}
{"pod": "pod-2", "ran": false, "ms": 8.2}
{"pod": "pod-3", "ran": true,  "rows_written": 4, "rows_failed": 0, "ms": 37.4}
-> 4 sentinel rows, total_flat_cost 1299.4985416666666

With Redis stopped every pod proceeds unguarded rather than skipping the day, and the reconcile is still exactly once:

{"pod": "pod-1", "ran": true, "rows_written": 4, "rows_failed": 0, "ms": 77.4}
{"pod": "pod-2", "ran": true, "rows_written": 4, "rows_failed": 0, "ms": 77.4}
{"pod": "pod-3", "ran": true, "rows_written": 4, "rows_failed": 0, "ms": 75.9}

before: 4 rows, total_flat_cost 1299.4985416666666
after:  4 rows, total_flat_cost 1299.4985416666666

Three concurrent full reconciles, twelve upserts, and the day is byte-identical afterwards. That is the property the design rests on: the lock avoids duplicate work, while correctness comes from the idempotent upsert on the sentinel key and a prune that deletes only rows this run did not refresh, so losing the lock costs effort rather than money

Renaming a deployment's public name at the barrier instant, with all three pods running unguarded, was attempted six times and never produced a duplicate charge; one row per model on every trial. The interleaving that would produce one is narrow, the config read measured at about 5 ms, so this bounds the failure rather than proving it impossible

This is the second of a stack, on top of the model-config PR. The read path that surfaces this on /team/daily/activity and the UI land in follow-up PRs

Catch-up proof

Live against a proxy on this branch (commit 67c8d40), real Postgres, no mocks.

Rebased onto litellm_internal_staging at f6587fa on 2026-08-06, which had moved 441 commits past the original branch point. The whole matrix below was re-run against the rebased head on a live proxy and a real Postgres, so nothing here is carried over from the pre-rebase run
An operator configures PTU on a deployment whose window opened 30 days ago, which the form accepts and the backend validates and stores

POST /model/new   ptu_count 10, cost_per_ptu_per_hour 2.0, ptu_effective_from 2026-07-07T00:00:00Z
  -> HTTP 200

GET /team/daily/activity?start_date=2026-07-07&end_date=2026-08-05&page_size=1000
  total_flat_cost: 0.0

One scheduled run, called exactly as the 00:15 UTC cron calls it, with no target_date

run_scheduled_ptu_rollup(prisma, alert=alert)
  -> RollupResult(day=2026-08-05, models_processed=1, rows_written=1, rows_failed=0)
  -> alerts: []

GET /team/daily/activity?start_date=2026-07-07&end_date=2026-08-05&page_size=1000
  total_flat_cost: 14400.0
  days priced    : 30
  first / last   : 2026-07-07 480.0 / 2026-08-05 480.0

30 days at 10 PTU and 2.00 an hour is 480 a day, so the declared window is billed in full. A second run finds nothing to do and leaves the figure at 14400.0

Deleting one day's row, which is what a pod restart across 00:15 leaves behind, drops the total to 13920.0; the next run restores it to 14400.0 without touching any other day

The property that matters most for a billing job is that catching up never restates a closed day. Doubling the rate to 4.00 and running again

PATCH /model/{id}/update   cost_per_ptu_per_hour 4.0   -> HTTP 200

closed day 2026-07-22 before the run : 480
closed day 2026-07-22 after the run  : 480
day the run owns, 2026-08-05         : 960
total_flat_cost                      : 14880.0

The 480 delta is the one day the daily reconcile owns. Every earlier day keeps the amount it was billed

Rename safety

A review pass found that matching gaps on the model name alone double charges a renamed deployment: the historical rows stay filed under the old name, since the catch-up never prunes, and every one of those days then reads as an unpriced gap under the new name. The gap check now also matches on ptu_source_model_id, which survives a rename

Proven live on the same database, ten days of a backdated window at 10 PTU and 2.00 an hour. The deployment is renamed through the dashboard's own PATCH shape, which rewrites team_public_model_name, and the rollup is then run again

with the deployment-id match (this PR)
  2026-07-27 .. 2026-08-04   1 row each, old-name,     480 each
  2026-08-05                 1 row,      renamed-ptu,  480
  total_flat_cost                                      4800.0

without it, same database, same rename
  2026-07-27 .. 2026-08-04   2 rows each, old-name + renamed-ptu, 960 each
  2026-08-05                 1 row,       renamed-ptu,            480
  total                                                           9120

Only the day the daily reconcile owns moves to the new name, because that path does prune. Every earlier day keeps the single charge it was billed

Concurrency across a rename

A four-pod rig found that two runs holding config views from either side of a rename wrote
the same day under two different composite keys, so nothing collided and both charges
survived. It never healed: no later run repaired it and none alerted. The sentinel row now
keys on the deployment id with the display name in model_group, outside the key

Same rig, same forced interleave, before and after

keyed on the display name        keyed on the deployment id
  2026-08-03  2 rows  960          2026-08-03  1 row   480
  total_flat_cost   2880.0         total_flat_cost   2400.0

Four pods firing the cron at once still elect exactly one runner, a second run writes
nothing, and deleting a mid-window day is repaired by the next catch-up back to 2400.00

The prune and clock skew

A third pass found that the prune compares a cutoff and an updated_at stamped on different
hosts. On the rig, a pod whose clock ran ten minutes ahead swept the charge a concurrent
pod had just written, and because both pods key the same row now, the day was left at zero

The prune is the only destructive step, so it now runs only when the pod took the
cross-pod lock. The upserts stay unguarded, since they are idempotent and no lock problem
may cost a day. The cutoff also allows PTU_PRUNE_SKEW_GRACE_SECONDS of slack, which
separates the two populations without requiring clocks to agree: a stale row is hours old
and a concurrently written one is seconds old

before                            after
  1 row before A's prune            1 row before A's prune
  0 rows, $0 after                  1 row, $480 after

The catch-up deletes nothing

An earlier revision had the catch-up delete sentinel rows for deployments that no longer
carry PTU config, reasoning that the single-day prune sweeps a date only once and a charge
it missed would be billed forever. That went too far: it also erased days a deployment was
legitimately billed for, so deleting a deployment wiped its history out of every usage view
that reports it. Removing a deployment or narrowing its window now stops it accruing new
charges and leaves the closed days standing, since those days were incurred

Covered by unit tests that assert the catch-up issues no delete at all, for a removed
deployment, for a narrowed window, and after the last PTU deployment is gone. The live
multi-pod matrix in this description was captured before this change and has not been
re-run against it

Live run 2026-08-08, commit c81b5a4

Fresh Postgres, real Gemini traffic, no mocks. One deployment at 15 PTU and $2.00/hr from 2026-08-01, a second at 10 PTU windowed 2026-08-06 23:00 to 2026-08-08 00:00

Pricing and identity

rollup 2026-08-05        1 row written   $720.00   = 15 * 2.00 * 24h
  row: model = e009a429-...  (deployment id, in the unique key)
       model_group = ptu-prod  (display name, outside the key)
re-run the same day      still 1 row, still $720.00      idempotent

Rename, which used to file a second charge

PATCH model_name -> ptu-prod-renamed      200
re-run 2026-08-05                          1 row, $720.00
  row: model = e009a429-...  unchanged     model_group = ptu-prod-renamed  updated

Hourly proration across a window boundary

2026-08-06  window opens 23:00   $20.00    = 10 * 2.00 * 1h
2026-08-07  full day             $480.00   = 10 * 2.00 * 24h
2026-08-08  window closed        $0.00

The catch-up, after deleting the windowed deployment

before delete            6 rows   $3380.00
POST /model/delete       200
after the scheduled run  10 rows  $6260.00
  the deleted deployment keeps 2026-08-06 $20.00 and 2026-08-07 $480.00
  and the catch-up gap-fills 2026-08-01..04 for the surviving deployment, 4 x $720.00

Deleting a deployment stops it accruing and leaves its closed days standing. The earlier revision of this PR deleted those rows

Known limitations

The effective window's start is required, not optional. ptu_effective_from must be set whenever ptu_count and cost_per_ptu_per_hour are, because flat cost accrues from that instant and an inferred start would bill days the deployment did not exist. Only ptu_effective_to is optional, and leaving it blank means open-ended. Earlier revisions of this description called the whole window optional, which was wrong

Single-writer election needs Redis. run_scheduled_ptu_rollup takes the cross-pod lock through PodLockManager, and when no Redis cache is configured it skips election and runs on every pod. That path deliberately passes may_prune=False, so the destructive step never runs unelected and the remaining work is idempotent upserts against the sentinel key: the cost is duplicated effort, not a wrong number. A fleet that wants one writer per day needs general_settings.use_redis_transaction_buffer and a reachable Redis

The catch-up reaches back at most PTU_ROLLUP_MAX_BACKFILL_DAYS, currently 90. A window whose start is older than that never has its earliest days priced by any run, and nothing reports the omission. The cap exists so an open-ended window cannot scan the whole table; a deployment needing a longer history has to be priced by an explicit single-day run per date

Type

New Feature

Changes

run_ptu_flat_cost_rollup enumerates model deployments, parses each model_info into a validated PTUModel (positive count, non-negative rate, team_id present, effective dates coerced to UTC), computes the day's active hours as the clamped overlap of the day with the effective window, and upserts one sentinel-api_key row per model into LiteLLM_DailyTeamSpend keyed by (team_id, model, date). The charge is keyed on the operator-facing name: creating a team-scoped deployment rewrites model_name to a synthetic routing key and keeps the chosen name in model_info.team_public_model_name, and PTU is only accepted alongside a team_id, so reading model_name directly would file every charge under a UUID no usage view can resolve. Deployments that share a public name inside a team are summed into one sentinel row, the current charges are written before the day's unrefreshed rows are pruned so a removed or expired config never leaves a positive charge behind, and a failing write is retried with backoff before the batch moves on; the run reports rows_failed so a day that still needs a manual rerun is visible. The composite-key upsert makes re-runs idempotent

The cron is registered unconditionally at startup and runs daily at 00:15 UTC; a model without PTU config simply produces no row. Because every proxy process schedules it, the run takes the shared PodLockManager lock first, so one pod reconciles the day and a loser cannot prune rows the winner just wrote; a deployment with no Redis-backed lock manager runs unguarded, as SpendLogCleanup does. acquire_lock reports contention and a Redis outage identically, so a failed acquire is followed by a read of the lock key: if nobody actually holds it the run proceeds unguarded rather than skipping, since an unreachable Redis would otherwise drop the whole day's charges on every pod at once. The lock exists to avoid duplicate work, so no lock problem is allowed to cost a day

The registration itself is deliberately not wrapped in a try/except, unlike the spend-log-cleanup and batch-cost registrations that follow it. Those guard values that can legitimately be bad at runtime, such as an operator-supplied retention string; the only way this one raises is a code defect, since the module imports nothing beyond the standard library and two litellm modules already loaded by that point, with PrismaClient and PodLockManager behind TYPE_CHECKING and no module-level executable code. Swallowing that would leave the rollup silently unregistered and the flat cost quietly missing until someone reads an invoice, which is the more expensive failure for a billing job than refusing to start

The prune deletes by updated_at < run_started rather than by "the keys I just computed". Whether a sentinel row is garbage is then a property of the row, not of one run's in-memory config snapshot, which is the same shape SpendLogCleanup deletes retired spend logs by (WHERE startTime < cutoff). It matters under concurrency: with a key-list predicate, two pods that read the config either side of a model rename each compute a correct-for-them list, and whichever deletes last removes the charge the other just wrote. With the timestamp predicate a row written after a run began is out of reach of that run's delete, so the worst case is a renamed-away charge lingering until the next run sweeps it. The lease can therefore lapse mid-run without risking the data; it costs duplicate work, not correctness

The prune is still skipped when any charge failed to write, since a row whose replacement never landed would otherwise look unrefreshed and be deleted

Charges that exhaust their retries raise a failed_tracking_spend alert naming the date and the count, so the gap surfaces through whatever alerting the deployment already has rather than only in proxy logs

Pricing one day per fire leaves two ways for a day to end up unpriced and stay that way. A window backdated at configuration time is a validated, supported input, and no fire ever revisits the days it already covers, so declaring a window that opened a month ago silently bills one day of it. A fire that is missed or lands late is worse in kind: the next one does not replay it, because the billed day comes from the wall clock rather than the scheduled time, which is also why no scheduler tuning recovers it. Neither raises anything, since the failure alert only fires for a charge that was attempted, and a day never attempted leaves rows_failed at zero

So each scheduled run follows the day's reconcile with a catch-up pass. It reads the sentinel rows already present across the scanned range, computes the charges every declared window implies over that range, and writes only the (team_id, model name, date) charges with no row, bounded at the earliest ptu_effective_from and floored at PTU_ROLLUP_MAX_BACKFILL_DAYS so an open-ended window does not scan the whole table. Three properties keep it from being worse than the gap it closes. It never rewrites an existing row, so a day already priced keeps the amount it was billed however the config has changed since; re-pricing it would silently restate a bill that was correct for that day, which is the failure this design is most careful to avoid. It runs no prune, so deciding a sentinel row is stale stays the single-day path's job and a deployment removed after a day was billed does not lose that day's charge. And zero-cost days write nothing, which keeps absence of a row from meaning "priced at zero" and leaves an out-of-window day reconsidered on each run rather than recorded as done

The pass runs inside the same pod lock as the day's reconcile, and only on the scheduled shape: passing an explicit target_date still means reconcile exactly that day. Its failure is contained, since the day's own rollup has already committed and its result must reach the caller whatever the catch-up does. A charge the catch-up cannot write raises its own alert naming the range

Behavior changes

A deployment whose model_info carries an unparseable or inverted PTU effective window now accrues no flat cost until the config is fixed, where an earlier revision of this PR read the bad bound as "no bound" and billed the full day

A scheduled run now writes rows for elapsed in-window days beyond yesterday, where an earlier revision of this PR wrote yesterday only. It adds rows and never changes or deletes one, so a team whose days are all priced sees no difference, and a proxy with no PTU config anywhere does no extra database work beyond one model-table read the rollup already performed. Nothing else in the stack changes for a well-formed config

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

---

Note

Medium Risk
Writes directly to daily team spend aggregates on a schedule; incorrect proration or upsert keys could misstate billing, though upserts are idempotent and failures are isolated per model.

Overview
Adds a daily PTU flat-cost rollup so provisioned-throughput settings on model deployments turn into team spend rows, prorated by how many UTC hours the PTU window overlaps each day.

A new run_ptu_flat_cost_rollup job scans LiteLLM_ProxyModelTable, validates model_info (ptu_count, cost_per_ptu_per_hour, team_id, optional effective window), computes ptu_count × rate × active_hours, and idempotently upserts into LiteLLM_DailyTeamSpend using the sentinel api key __ptu_flat_cost__ plus ptu_flat_cost / ptu_source_model_id. Per-model upsert failures are logged and skipped so one bad deployment does not stop the batch.

Proxy startup registers an APScheduler cron at 00:15 UTC (defaults to rolling up yesterday). Unit tests cover hour-window edge cases, parsing, rollup behavior, and that the job is registered at startup.

^{Reviewed by Cursor Bugbot for commit 2678380. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

Adds scheduled PTU flat-cost accounting and recovery.

• Computes per-deployment daily charges from UTC effective-window overlap.
• Stores idempotent sentinel spend rows keyed by deployment ID.
• Adds cross-pod scheduling, guarded pruning, write retries, alerts, and bounded historical catch-up.
• Adds unit coverage for pricing, reconciliation, locking, backfill, and startup registration.

Confidence Score: 4/5

The PR is not yet safe to merge because stale charges, lease-expiry concurrency, and unrecoverable failed charges can still misstate team spend.

The five-minute cutoff can preserve an obsolete row beyond its only scheduled reconciliation, the fixed lock can expire while a rollup is still pruning, and a failed charge becomes impossible to reconstruct after its deployment configuration is removed.

Files Needing Attention: litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py

Important Files Changed

Filename	Overview
litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py	Implements PTU pricing, reconciliation, locking, pruning, and catch-up, but previously reported lifecycle and recovery failures remain.
litellm/proxy/proxy_server.py	Registers the PTU rollup as a daily UTC cron and connects high-priority spend-tracking alerts.
litellm/constants.py	Defines the sentinel, lock duration, backfill bound, and pruning grace constants used by the rollup.
tests/test_litellm/proxy/spend_tracking/test_ptu_flat_cost_rollup.py	Adds extensive rollup coverage, including proration, retries, concurrency, pruning, and historical catch-up.
tests/test_litellm/proxy/test_proxy_server.py	Verifies registration of the scheduled PTU rollup job.

_{Reviews (24): Last reviewed commit: "feat(ptu): daily rollup writes per-model..." | Re-trigger Greptile}

[devin-ai-integration]

Devin Review found 4 potential issues.

View 3 additional findings in Devin Review.

[devin-ai-integration]

🟡 Model rows are read with untyped attribute access instead of validated typed input

Deployment rows are accepted as a bare untyped value and picked apart with dynamic attribute reads (_parse_ptu_model(row: object) at litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:66-95), which the repository guidelines prohibit in favour of validating the input into a typed model first.
Impact: The new code violates a mandatory typing rule of the repository, and silently yields empty model ids/names when the shape differs.

Rule reference and effect

CLAUDE.md requires "Fully typed; no Any or coarse types like dict[str, Any] or just dict. Every function parameter must be strongly typed" and: "If you're trying to create a new function that relies on untyped stuff ... just validate it in the caller with Pydantic (a model or TypeAdapter that returns the typed thing or raises will do) and then pass the now typed variable in". Here row: object plus getattr(row, "model_id", "") or "" (litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:88-90) also means a missing model_name silently becomes "", which then becomes the row's model key.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 New tests contain explanatory comments, which the repository guidelines forbid

The new test file adds inline explanatory comments (for example the arithmetic note at tests/test_litellm/proxy/spend_tracking/test_ptu_flat_cost_rollup.py:36), which the repository's guidelines explicitly prohibit for new code.
Impact: The change violates a mandatory repository convention and will be flagged in review.

CLAUDE.md comment rule

CLAUDE.md: "Do not write any comments (existing comments can stay) unless explicitly asked to in a user (not system) prompt". The new test file adds comments at lines 36, 43, and 109 (# missing team_id). Docstrings are fine; these inline comments should be dropped or folded into test names/docstrings.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[codecov]

Codecov Report

❌ Patch coverage is 98.71795% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/proxy_server.py	75.00%	2 Missing ⚠️
...tellm/proxy/spend_tracking/ptu_flat_cost_rollup.py	99.55%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[veria-ai]

Low: Mutable model state allows flat-cost evasion

This query only sees the deployment's current state. A team administrator can delete their PTU deployment or set ptu_effective_to before the target day prior to the 00:15 rollup, causing the previous day's flat cost to be skipped. Persist immutable PTU configuration intervals when models are created or updated, and calculate the rollup from that history rather than the live model table.

[veria-ai]

PR overview

This pull request adds a daily PTU cost rollup that records each model’s flat cost according to its active hours. It also supports backfilling charges for prior effective dates.

Three issues have been addressed, but three remain open. Most significantly, PTU flat-cost charges are not included in authoritative team spend or budget counters, allowing continued requests after the budget should be exhausted. Mutable deployment state can also omit prior charges, while unbounded backfills may consume excessive worker memory and database capacity.

Open issues (3)

• Low: Mutable model state allows flat-cost evasion — litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:175
• Medium: PTU charges bypass team budgets — litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:208
• Low: Unbounded PTU backfill work — litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:454

Fixed/addressed: 3 · PR risk: 7/10

[devin-ai-integration]

Devin Review found 3 new potential issues.

View 4 additional findings in Devin Review.

[devin-ai-integration]

🟡 Team usage reports now show a fake API key entry for reserved-capacity rows

Reserved-capacity rows are written into the same team daily-spend table under a made-up key name (api_key=PTU_SENTINEL_API_KEY at litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:136-156) that existing usage reports do not filter out, so teams see a phantom key with zero usage in their activity breakdowns.
Impact: Team daily activity responses and the usage UI display a bogus key entry and extra records for every reserved-capacity model.

Read path has no sentinel filter yet

/team/daily/activity queries litellm_dailyteamspend via get_daily_activity with no api_key exclusion (litellm/proxy/management_endpoints/team_endpoints.py:5235-5248), and litellm/proxy/management_endpoints/common_daily_activity.py:339-347 buckets every record by record.api_key, looking up alias metadata that will be missing for __ptu_flat_cost__. The sentinel rows also carry custom_llm_provider=""/endpoint="", adding empty-provider and empty-endpoint buckets and consuming pagination slots. The PR defers the read path to a follow-up, so on this commit alone the write pollutes existing responses.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[yucheng-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 2678380. Configure here.}

[cursor]

Rollup skips string model_info

High Severity

_parse_ptu_model only accepts model_info as a dict and returns None otherwise. Elsewhere the codebase documents that LiteLLM_ProxyModelTable.model_info can arrive as a JSON string depending on the query path, and model_info_as_mapping / ModelRepository already normalize that. Raw find_many() rows can therefore make every PTU deployment look invalid, so the rollup silently writes no flat-cost rows.

Additional Locations (1)

• litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py#L175-L177

 

^{Reviewed by Cursor Bugbot for commit 2678380. Configure here.}

[cursor]

Cron not pinned to UTC

Medium Severity

The PTU rollup cron is logged as running at 00:15 UTC, but add_job does not set timezone. The shared AsyncIOScheduler is created with timezone=None, which uses the host local zone. On non-UTC hosts the job fires at local midnight instead of UTC, so rollups run at the wrong wall-clock time relative to the UTC day the cost math uses.

 

^{Reviewed by Cursor Bugbot for commit 2678380. Configure here.}

[yucheng-berri]

@greptileai fixed: the rollup now parses JSON-string model_info, and the daily cron is pinned to UTC to match the cost day. Re-review please

[yucheng-berri]

@greptileai the rollup now clears each day's PTU sentinel rows before rewriting, so a removed or expired config leaves no stale charge. Re-review please

[yucheng-berri]

@greptileai the rollup now writes current charges before pruning stale rows (no delete-before-write underbilling window) and sums same-name deployments per team into one row. Re-review please

[yucheng-berri]

@greptileai a failed sentinel upsert now retries with backoff before the batch moves on, and the run reports rows_failed so a lost day is visible. Re-review please

[yucheng-berri]

@greptileai review latest head and the comment i gave you

[yucheng-berri]

@greptileai review

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 4 additional findings in Devin Review.

[devin-ai-integration]

🟡 Deleting a provisioned-throughput model loses that day's charge for good

Charges are computed only from the deployments that still exist when the nightly job runs (_load_ptu_models at litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:318-321), so a model removed before the next run is never billed for the hours it was active that day and nothing later recovers it.
Impact: A team can use provisioned capacity all day and be charged nothing for it simply because the model was deleted before midnight.

Why no later pass recovers the day

The scheduled job at 00:15 UTC prices only the previous UTC day, and both the day's rollup and the catch-up pass derive charges from the live LiteLLM_ProxyModelTable rows (litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:350-351 and litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:456-468). Once the deployment row (or just its ptu_count / cost_per_ptu_per_hour / ptu_effective_from fields) is gone, _parse_ptu_model returns None for it forever, so the last unpriced day (and any earlier days the job never got to, e.g. after a multi-day outage) produce no charge and no alert, since rows_failed stays 0 for a charge that was never attempted. The PR description claims deletion "stops it accruing new charges and leaves the closed days standing", which holds for already-written rows but silently drops the in-progress and any still-unpriced days.

Prompt for agents

The rollup and its catch-up pass both read PTU config from the live LiteLLM_ProxyModelTable, so any day not yet priced when a deployment is deleted (or has its PTU fields cleared) is lost permanently, including the day the deletion happened. Consider pricing the elapsed portion of the current day at deletion time in the model-delete/update path, or persisting a small immutable record of PTU config windows that the rollup can read even after the deployment row is gone. At minimum, document/alert on the gap so operators know deleting a PTU deployment forfeits the unbilled hours.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[yucheng-berri]

@greptileai review

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 5 additional findings in Devin Review.

[devin-ai-integration]

🟡 A single malformed model record can abort the whole nightly PTU cost run

A stored model configuration that decodes to something other than a key/value object (json.loads at litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:113) is returned as-is instead of being rejected, so reading it later crashes and the entire nightly charge run stops before pricing anyone.
Impact: One bad model entry can silently prevent every team's provisioned-throughput charge from being recorded for that day.

Mechanism: the JSON branch of _decode_model_info does not verify the decoded value is a dict

_decode_model_info (litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:109-118) returns json.loads(raw) directly when raw is a string. Valid JSON that is not an object (e.g. "123", "[]", "true") decodes to an int/list/bool, which is then returned as dict[str, object]. _parse_ptu_model immediately calls model_info.get("ptu_count") (:131) and raises AttributeError. _load_ptu_models (:318-321) has no per-row guard, and neither run_ptu_flat_cost_rollup nor run_ptu_flat_cost_backfill catches it, so the exception propagates all the way out of the scheduled job (and, on the locked path, only the finally release runs). The dict branch at :116 shows the intent that only mappings are accepted; the string branch skips that check. Note the test test_a_bad_row_does_not_abort_pricing_for_other_teams asserts exactly the property this breaks, but only for a dict-shaped bad row.

Suggested change

	if isinstance(raw, str):
	try:
	return json.loads(raw)
	except (TypeError, ValueError):
	return None
	if isinstance(raw, str):
	try:
	decoded: Final = json.loads(raw)
	except (TypeError, ValueError):
	return None
	return decoded if isinstance(decoded, dict) else None

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_ptu_flat_cost_rollup (803a577) with litellm_internal_staging (f6b9518)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (b7da09e) during the generation of this report, so f6b9518 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩