feat(ptu): accrue flat cost for PTU deployments declared in config.yaml

[yucheng-berri]

TLDR

Problem this solves:

• A PTU deployment in config.yaml accrues no flat cost
• The rollup only reads the database model table
• Its traffic is billed per token instead

How it solves it:

• The rollup also reads config-declared deployments off the router
• Registering one zeroes its pricing, since capacity is prepaid
• Both halves apply the same rules from one module

User Flow

Before: a team on reserved capacity is billed for its tokens and never for the capacity

1. An admin declares a provisioned-throughput deployment in config.yaml with team_id, ptu_count, cost_per_ptu_per_hour and ptu_effective_from, then starts the proxy with LITELLM_ENABLE_PTU_COST_ATTRIBUTION=True
2. Someone on that team sends POST https://litellm-domain/v1/chat/completions and gets a normal answer
3. https://litellm-domain/ui/?page=logs shows that request charged at the provider's per-token rate
4. The nightly attribution job runs at 00:15 UTC and writes nothing for that deployment
5. https://litellm-domain/ui/?page=usage shows the team's token spend and no reserved-capacity cost, so the hourly charge the provider is invoicing appears nowhere

After: the same team is billed for the capacity it reserved, and not again for the traffic that capacity serves

1. The admin declares the same deployment and starts the same proxy
2. Someone on that team sends the same POST https://litellm-domain/v1/chat/completions and gets the same answer
3. https://litellm-domain/ui/?page=logs shows that request at $0, because the reservation already paid for it
4. The nightly job runs and attributes the reservation to the team
5. https://litellm-domain/ui/?page=usage shows the team's reserved-capacity cost for the day, charted separately from request cost

An admin who wants the deployment billed per token instead removes ptu_count and cost_per_ptu_per_hour, exactly as on the API path

Relevant issues

Linear ticket

Resolves LIT-5809

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Real Postgres, real Gemini traffic, a proxy booted from the config below on each side, no mocks. Gemini stands in for Azure because PTU capacity cannot be provisioned here; the fields the feature reads are provider-agnostic, and what is being proven is the cost path. The only thing not exercised is the cron trigger itself, since the job has no on-demand entry point, so each side calls the same scheduled function directly against its own database.

model_list:
  - model_name: gpt-4o-ptu
    litellm_params:
      model: gemini/gemini-2.5-flash
      api_key: os.environ/GEMINI_API_KEY
      input_cost_per_token: 0.000005
      output_cost_per_token: 0.000015
    model_info:
      team_id: ptu-team-0001
      ptu_count: 100
      cost_per_ptu_per_hour: 0.02
      ptu_effective_from: "2026-01-01T00:00:00Z"

general_settings:
  master_key: sk-lit5809

export LITELLM_ENABLE_PTU_COST_ATTRIBUTION=True
python -m litellm.proxy.proxy_cli --config ptu.yaml --port <port>
KEY=$(curl -s -X POST http://127.0.0.1:<port>/key/generate -H "Authorization: Bearer sk-lit5809" \
  -H 'Content-Type: application/json' -d '{"team_id":"ptu-team-0001","models":["gpt-4o-ptu"]}' | jq -r .key)

Before (eecb226)

a real request through the config.yaml PTU deployment

1. Send it

curl -s -X POST http://127.0.0.1:4811/v1/chat/completions -H "Authorization: Bearer $KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-ptu","messages":[{"role":"user","content":"Say hi."}],"max_tokens":25}'

tokens: 4 / 21

2. The deployment is priced at what the operator declared

router pricing: input=5e-06 output=1.5e-05

the nightly attribution job

1. Run it for that day

rollup: RollupResult(day=2026-08-18, models_processed=0, rows_written=0, rows_failed=0, lapsed=())
flat-cost rows: []

2. Read the team's day back out of Postgres

                             api_key                              | model_group |  spend   | prompt_tokens | completion_tokens | ptu_flat_cost
------------------------------------------------------------------+-------------+----------+---------------+-------------------+---------------
 291e0dcf9ff6ecfbb84b5306a9cd6fb2575ca8a2fa992d859ad81593d58ba670 | gpt-4o-ptu  | 0.000335 |             4 |                21 |             0

The tokens are billed at 4 x 5e-06 plus 21 x 1.5e-05, and the reservation the provider is invoicing hourly is recorded nowhere

After (286bbc9)

a real request through the config.yaml PTU deployment

1. Send the same request

curl -s -X POST http://127.0.0.1:4812/v1/chat/completions -H "Authorization: Bearer $KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-ptu","messages":[{"role":"user","content":"Say hi."}],"max_tokens":25}'

tokens: 4 / 21

2. The deployment carries no per-token rate

router pricing: input=0.0 output=0.0

the nightly attribution job

1. Run it for that day

rollup: RollupResult(day=2026-08-18, models_processed=1, rows_written=1, rows_failed=0, lapsed=())
flat-cost rows: [('ptu-team-0001', 'gpt-4o-ptu', 48.0)]

2. Read the team's day back out of Postgres

                             api_key                              | model_group | spend | prompt_tokens | completion_tokens | ptu_flat_cost
------------------------------------------------------------------+-------------+-------+---------------+-------------------+---------------
 8d808bffd673ab86d6f0e3a2addf8f5933ed7f26fd5a6b9d2666ed5aa52c0864 | gpt-4o-ptu  |     0 |             4 |                21 |             0
 __ptu_flat_cost__                                                | gpt-4o-ptu  |     0 |             0 |                 0 |            48

100 units at $0.02 an hour over a full day is $48, the same request records 4 and 21 real tokens at no per-token charge, and the deployment id is unchanged from the before run, so cooldowns, deployment budgets and existing spend rows all still key on it

Re-run at the current tip, exit code checked and the log grepped for a traceback, after an earlier capture turned out to be a partially completed run

Type

🆕 New Feature

Changes

The rollup reads the router's deployments alongside the table, keeping the ones no database row owns and dropping the per-request credential clones, which carry original_model_id and would otherwise bill one reservation once per distinct caller key.

Registering such a deployment zeroes its pricing. The rules for what accrues and what that zeroes live in litellm/litellm_core_utils/ptu_pricing.py, which the write endpoints now import as well, because a deployment the rollup declines to charge and the router prices at zero would serve its traffic for free. tiered_pricing is emptied rather than zeroed, since its tiers outrank the rates beside them; the search context table is written zeroed because an absent one means the provider's default rather than free; and any further rate the deployment declares is zeroed alongside the standing set.

The prune is bounded to the deployments a run scanned, but only for a run that priced a config-declared deployment, so a database-only proxy sweeps exactly as it does today.

The nightly catch-up shares that loader, so a config-declared reservation is priced for its elapsed days too. Verified live: a reservation started five days ago writes five days at $240, one started two hundred days ago writes 91 days at $4,368, and a second run of either writes nothing further.

QA runbook

Caveats (if any)

• A zeroed deployment wins QualityRouter's cost tiebreak where an unset rate lost it
• Attribution needs every replica booted from the same config.yaml
• Outside its window a PTU deployment bills nothing, matching the database path
• Enabling this prices up to 91 elapsed days on the first nightly run

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

[greptile-apps]

Greptile Summary

The PR adds flat-cost attribution for config-declared PTU deployments while zeroing their per-request pricing to avoid double billing.

• Centralizes PTU term validation and zero-pricing rules.
• Includes router-backed deployments in daily rollup and backfill processing.
• Bounds reconciliation when config-sourced deployments participate.
• Adds focused pricing, rollup, and router-isolation tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/ptu_pricing.py	Centralizes PTU eligibility, date validation, feature-flag handling, and zero-pricing behavior.
litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py	Extends attribution to router-backed config deployments and scopes pruning to deployments observed by config-aware runs.
litellm/router.py	Applies zeroed pricing when eligible config-declared PTU deployments are registered.
litellm/proxy/management_endpoints/model_management_endpoints.py	Reuses centralized PTU pricing-field definitions in model-management validation.
tests/test_litellm/proxy/spend_tracking/test_ptu_flat_cost_rollup.py	Adds coverage for loading, charging, deduplicating, and reconciling config-declared PTU deployments.
tests/test_litellm/test_router_model_cost_isolation.py	Verifies PTU pricing overrides remain isolated between router deployments.

_{Reviews (2): Last reviewed commit: "refactor(ptu): drop a Final rebind and t..." | Re-trigger Greptile}

[veria-ai]

Medium: Budget bypass outside the PTU effective window

This zeroes per-request pricing without checking whether the current time is within ptu_effective_from and ptu_effective_to, while _active_hours_on_day drops the flat charge outside that interval. An authenticated caller can keep invoking a routable config deployment before its start or after its end without increasing tracked spend, so budget enforcement never catches up. Keep the deployment unavailable outside the interval, or reevaluate and restore its normal pricing when the interval is inactive.

[yucheng-berri]

Pre-existing: #36829 zeroes on carrying PTU config, not window state, so the database path already does this. Gating here alone would split the paths.

[veria-ai]

Thanks for the detail. I can't automatically confirm this is safe to dismiss, so I'm leaving the thread open for a maintainer to make the call.

[yucheng-berri]

#36829 exercised a closed window live and states it: "A closed window alerts; it does not resume per-token billing."

[veria-ai]

Thanks for the detail. I can't automatically confirm this is safe to dismiss, so I'm leaving the thread open for a maintainer to make the call.

[veria-ai]

PR overview

This PR adds flat-cost accrual for provisioned throughput unit (PTU) deployments declared in config.yaml, including time-bounded effective periods and corresponding router pricing behavior.

One security issue remains open: authenticated callers can use a routable PTU deployment outside its configured effective window without request costs being added to tracked spend. This allows budget enforcement to be bypassed for that deployment before the start time or after the end time; no issues have yet been addressed.

Open issues (1)

• Medium: Budget bypass outside the PTU effective window — litellm/router.py:7700

Fixed/addressed: 0 · PR risk: 6/10

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

Devin Review found 1 potential issue.

View 1 additional finding in Devin Review.

[devin-ai-integration]

🟡 Reserved-capacity deployment serves traffic for free before its start date

A PTU deployment whose reservation start is in the future has its per-token pricing zeroed at registration (zeroed_ptu_pricing at litellm/litellm_core_utils/ptu_pricing.py:118-146) even though flat cost only begins accruing from that future start, so the team is charged nothing at all for traffic served in the gap between when the proxy boots and when the reservation begins.
Impact: A team can send requests through a not-yet-active reserved deployment and be billed neither per token nor for reserved capacity, silently underbilling until the start date arrives.

How the future-dated window escapes both charge paths

ptu_terms accepts a reservation with an effective_from in the future (litellm/litellm_core_utils/ptu_pricing.py:101-115 only rejects an inverted or unparseable window, not a future one), so zeroed_ptu_pricing returns the zeroed pricing map and the router registers the deployment priced at 0. Meanwhile the rollup's _active_hours_on_day (litellm/proxy/spend_tracking/ptu_flat_cost_rollup.py:213-221) clamps start to effective_from, so any day before the start yields 0 active hours and no flat charge. The result is that requests served before effective_from cost the team nothing on either path. This extends the pre-existing DB-path behavior to config.yaml deployments and is acknowledged as a caveat in the PR description.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[yucheng-berri]

Agreed, and as you note this is the database path's existing behaviour from #36829. Changing it here alone would split the two paths.

[yucheng-berri]

Confirmed deliberate: #36829 demonstrated a closed window live and documented it as "A closed window alerts; it does not resume per-token billing."

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit5809_config_yaml_ptu (c7ca8f4) with litellm_internal_staging (f22eeb2)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (26841da) during the generation of this report, so f22eeb2 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yucheng-berri]

@greptileai please review the current head c7ca8f4. Divergent replica configs break serving before billing; documented as a caveat, and unfixable without persisting config models.

[devin-ai-integration]

Live verification on staging: the customer's config.yaml PTU model now accrues flat cost

Ran the reported repro end to end on d542c82f0e (staging, with #37501, #37556 and #37571 all merged) against real Postgres, real Redis for the pod lock, a real proxy booted from a config.yaml, a dashboard rebuilt from that tree, and real billed provider calls. Azure is not available here, so the PTU deployments are backed by OpenAI gpt-4o-mini and the fresh calls are OpenAI.

A config.yaml model carrying ptu_count: 10, cost_per_ptu_per_hour: 0.50, ptu_effective_from and a team_id, with no row in LiteLLM_ProxyModelTable, now gets a sentinel row of 120.00 (10 × 0.50 × 24) keyed api_key = __ptu_flat_cost__, spend = 0, model_group = azure-ptu-fixed, and it shows on the Usage page under the readable name. The 390.00 total for that day is 240 from the DB-configured deployment plus 120 from the config-declared one plus 30 from a deployment starting at 18:00Z, so proration, the exclusive ptu_effective_to and lapsed windows all land where they should.

Usage page, config deployment included	Real request through the PTU deployment

Pricing zeroing holds and does not double bill: the same backend model reached through a non-PTU config entry billed $0.00000495 while the PTU entry logged $0.00000000. Precedence holds too — a shadow config entry declaring 99 PTU at $99/h under an existing DB row's id produced exactly one sentinel row valued from the database, and neither a db_model: true entry nor a credential clone carrying original_model_id multiplied the charge. A malformed stored model_info was skipped with exit code 0 and no traceback while every valid deployment was still priced, and the scheduled run under a real Redis pod lock completed with no mappingproxy serialization error, so the prune, the catch-up pass and the lapsed-window alert all ran.

One thing a user will still hit

The customer's snippet as written has no team_id, and _parse_ptu_model requires one. In that state the deployment accrues nothing and nothing explains why: no startup error, no warning, no log line naming the model or the missing field. Because ptu_terms returns None, its per-token pricing is also left intact, so the deployment quietly bills per token ($0.00000315 on a real call) with no flat accrual — which is exactly the behaviour originally reported. Adding team_id fixes it completely. A single warning at load time naming the deployment and the missing field would turn this from a silent no-op into a self-service fix. The bare date form "2025-08-17" parses fine as UTC midnight, so it is not implicated.

Separately, the Usage export mode "Day-by-day by team and model" has no Flat Cost ($) or Total Cost ($) columns and omits the flat-cost rows entirely, while "Day-by-day by team" carries both and matches the tiles exactly.

Carried and unchanged on this head: some invalid PTU configs answer HTTP 422 rather than the documented 400; the aggregated GROUPING SETS path emits an all-zero unknown provider bucket the per-row path omits, and reports raw model ids where the per-row path reports readable names; EntityUsage.tsx hard-codes Grid numItems={5} while seven tiles render when expanded.

Suites

The PTU backend suites plus test_proxy_server.py, test_proxy_setting_endpoints.py, test_ptu_model_settings.py and test_common_daily_activity.py: 750 passed, exit 0, with MICROSOFT_CLIENT_ID, MICROSOFT_CLIENT_SECRET and MICROSOFT_TENANT cleared for the known environment leak. Dashboard vitest over the eight PTU-touching files: 249 passed on Node v24.19.0.