feat(ptu): configure provisioned-throughput flat cost on a model deployment

[yucheng-berri]

TLDR

Problem this solves:

• No way to attach provisioned-throughput cost inputs to a model
• Teams cannot see PTU flat cost alongside per-request spend

How it solves it:

• ModelInfo gains ptu_count, cost_per_ptu_per_hour, and an effective window
• model/new and model/{id}/update validate and store them; model/info returns them

Relevant issues

Linear ticket

Resolves LIT-4077

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Captured live against a proxy on this branch (commit 277f096), real endpoints, no mocks.

Rebased onto litellm_internal_staging at f6587fa on 2026-08-06, which had moved 441 commits past the original branch point. The whole matrix below was re-run against the rebased head on a live proxy and a real Postgres, so nothing here is carried over from the pre-rebase run

The validation matrix, re-run on the rebased head

POST /model/new  ptu_count 10, no cost_per_ptu_per_hour
  -> 400 "ptu_count and cost_per_ptu_per_hour must be set together"

POST /model/new  ptu_count 2.5, cost_per_ptu_per_hour 2.0
  -> 422 "Input should be a valid integer, got a number with a fractional part"

POST /model/new  ptu_count 10, cost_per_ptu_per_hour -2.0
  -> 422 "Value error, cost_per_ptu_per_hour must be non-negative"

POST /model/new  ptu_count 10, cost_per_ptu_per_hour 2.0, no team_id
  -> 400 "team_id is required when PTU fields are set (one model maps to one team)"

POST /model/new  ptu_count 10, cost_per_ptu_per_hour 2.0, team_id set,
                 ptu_effective_from 2026-07-07T00:00:00Z
  -> HTTP 200

GET /model/info
  -> {'ptu_count': 10, 'cost_per_ptu_per_hour': 2.0,
      'ptu_effective_from': '2026-07-07T00:00:00+00:00', 'team_id': '71cb34c9-...'}

The stored window comes back as the same UTC wall clock it went in as

# 1. model/new with PTU config is persisted and returned
POST /model/new  model_info={team_id, ptu_count:5, cost_per_ptu_per_hour:2.0, ptu_effective_from:"2026-07-31T23:00:00Z"}
-> model_info: ptu_count=5 cost_per_ptu_per_hour=2.0 effective_from=2026-07-31T23:00:00Z

# 2. only one of the two PTU fields is rejected
POST /model/new  model_info={team_id, ptu_count:5}
-> HTTP 400 "ptu_count and cost_per_ptu_per_hour must be set together"

# 3. PTU fields without a team_id are rejected
POST /model/new  model_info={ptu_count:5, cost_per_ptu_per_hour:2.0}
-> HTTP 400 "team_id is required when PTU fields are set (one model maps to one team)"

Screens

The four fields on the model form, and the cross-field invariant rejecting a half-set pair before it reaches the API:

Per-field bounds, rejected in the form rather than surfacing as a backend error:

This is the first of a stack. The daily rollup that writes ptu_flat_cost, the read path that surfaces it, and the UI land in follow-up PRs

Live run 2026-08-08, commit c0ab5d2

Fresh Postgres, migrations deployed from litellm-proxy-extras, real Gemini traffic, no mocks

POST /model/new   ptu_count 15, cost_per_ptu_per_hour 2.0, ptu_effective_from 2026-08-01   -> 200
POST /model/new   count without rate                                                        -> 400 must be set together
POST /model/new   pair without ptu_effective_from                                           -> 400 ptu_effective_from is required
POST /model/new   pair without team_id                                                      -> 400 team_id is required
POST /model/new   ptu_count 0                                                               -> 422 must be a positive integer
POST /model/new   cost_per_ptu_per_hour -1.0                                                -> 422 must be non-negative
POST /model/new   ptu_effective_to before ptu_effective_from                                -> 422 must be after

The merged-view fix, driven against a deployment storing count 15 / rate 3.0 / start 2026-08-01

PATCH  rate only, no start in the body      -> 200   stored rate 3.0, start preserved
PATCH  ptu_count: null (clears one half)    -> 400   must be set together
       stored after the refusal              = count 15, rate 3.0   unchanged
PATCH  count and rate both null             -> 200   both keys gone, PTU turned off
PATCH  reconfigure                          -> 200   count 15, rate 2.0 restored

Before this fix the half-clear passed validation on the stored count and committed the team write before update_db_model's backstop rejected it

The same config as seen by an operator, on Models + Endpoints for the deployment created above

Type

New Feature

Changes

ModelInfo gains ptu_count, cost_per_ptu_per_hour, ptu_effective_from and ptu_effective_to. A model_validator enforces per-field bounds: a positive count, a non-negative rate, and effective_to strictly after effective_from. Bounds run at request-parse time on both model/new and model/{id}/update since both carry a ModelInfo

The cross-field invariant (count and rate set together, team_id required when either is set) is enforced by a helper applied to the effective model_info: on POST /model/new against the create payload, and on PATCH /model/{model_id}/update against the merged stored-plus-incoming model_info, so a partial patch that changes only one field validates the final state rather than the delta. Note the verb: /model/{model_id}/update is PATCH-only and answers POST with a 405, while the older POST /model/update is a different route that does not write model_info at all

The two invariant failures above answer 400 from the handler. The per-field bounds live on the ModelInfo validator instead, so they are reported at request-parse time as FastAPI's usual 422 rather than 400; both reject the request and persist nothing, but the status codes differ by design and the SDK/Router keep the bounds even when the endpoint is bypassed

The merged view mirrors what the write will store, including the clears. update_db_model drops a PTU field a patch sends as an explicit null, so carrying the stored value into validation approved a deployment that never forms: clearing a count while leaving a rate passed the pair check and then committed the team ACL write before the backstop validator inside update_db_model rejected it. Both paths now derive the cleared set from one helper, so they cannot drift again

LiteLLM_DailyTeamSpend gains a ptu_flat_cost column and a sentinel api_key constant, ready for the rollup PR; nothing writes them yet in this PR

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

---

Note

Medium Risk
Requires a DB migration and changes model deployment validation and spend schema; rollup/read paths are not in this PR, so misconfiguration is limited to rejected API payloads until follow-ups land.

Overview
Adds provisioned throughput (PTU) settings on proxy model deployments so flat reservation cost can be configured per team-scoped model before a follow-up rollup writes spend.

ModelInfo now accepts ptu_count, cost_per_ptu_per_hour, and optional ptu_effective_from / ptu_effective_to, with bounds enforced at parse time (positive count, non-negative rate, effective window ordering). /model/new and /model/{id}/update (via merged model_info in update_db_model) call _validate_ptu_model_info: PTU count and hourly rate must be set together, and team_id is required when either is present.

LiteLLM_DailyTeamSpend gains ptu_flat_cost and ptu_source_model_id (migration + Prisma); PTU_SENTINEL_API_KEY and PTU_ROLLUP_JOB_ID are defined for future rollup rows—nothing writes these columns in this PR. Dashboard OpenAPI types include the new fields.

^{Reviewed by Cursor Bugbot for commit f46ef05. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

Adds validated, team-scoped provisioned-throughput configuration to model deployments and prepares daily team spend storage for a later rollup.

• Adds PTU count, hourly cost, and effective-window fields to ModelInfo.
• Validates create and merged update state before model or team-assignment writes.
• Adds the ptu_flat_cost database column and PTU rollup constants.
• Updates generated dashboard types and adds PTU validation/update tests.

Confidence Score: 5/5

The PR appears safe to merge because the previously reported model-validation and explicit-clear failures are fixed and no blocking failure remains.

No blocking failure remains; mixed timezone bounds are normalized before comparison, explicit PTU nulls are removed consistently from prevalidation and persistence, and merged validation runs before team assignment writes.

Important Files Changed

Filename	Overview
litellm/types/router.py	Adds PTU fields and bounds/window validation; UTC normalization fixes the previously reported mixed naive/aware timestamp comparison failure.
litellm/proxy/management_endpoints/model_management_endpoints.py	Adds merged-state PTU validation and explicit-null handling before team assignment writes, resolving the previously reported partial-write and failed-clear paths.
tests/test_litellm/proxy/management_endpoints/test_ptu_model_settings.py	Covers PTU bounds, effective windows, mixed timezones, merged partial updates, explicit clears, and validation-before-write behavior.
schema.prisma	Adds the ptu_flat_cost field consistently with the other Prisma schema copies.
litellm-proxy-extras/litellm_proxy_extras/migrations/20260713010000_add_ptu_columns_to_daily_team_spend/migration.sql	Adds the non-null daily PTU flat-cost column with a zero default for existing rows.
litellm/constants.py	Defines stable sentinel identifiers for the planned PTU rollup.
ui/litellm-dashboard/src/lib/http/schema.d.ts	Exposes the new PTU model information fields through generated dashboard API types.

_{Reviews (12): Last reviewed commit: "feat(ptu): configure provisioned-through..." | Re-trigger Greptile}

[devin-ai-integration]

Devin Review found 4 potential issues.

View 2 additional findings in Devin Review.

[devin-ai-integration]

🟡 New explanatory notes added in the model update path despite the project ban on writing comments

A seven-line explanatory comment block was newly added before the provisioned-throughput validation call (litellm/proxy/management_endpoints/model_management_endpoints.py:815-822), which the repository's coding guidelines forbid for new code.
Impact: The change violates an explicit repository rule that no new comments be written.

Rule source and location

CLAUDE.md (mandatory per AGENTS.md) states: "Do not write any comments (existing comments can stay) unless explicitly asked to in a user (not system) prompt". The rationale here largely duplicates the docstrings already on _merged_ptu_model_info and _validate_ptu_model_info, so it can be dropped or folded into those docstrings.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Two unused settings are added for a feature that does not exist yet

Two constants for a future reservation rollup are introduced (PTU_SENTINEL_API_KEY and PTU_ROLLUP_JOB_ID at litellm/constants.py:1723-1724) even though nothing in this change reads them, which the repository guidelines forbid.
Impact: Dead, speculative configuration ships to users and can drift out of sync with the follow-up work that was supposed to use it.

Rule reference and verification

CLAUDE.md requires "Minimum code that solves the problem. Nothing speculative" and "No features beyond what was asked". A repo-wide search finds no reader of either constant; the PR description itself states nothing writes these yet.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[codecov]

Codecov Report

❌ Patch coverage is 94.66667% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...management_endpoints/model_management_endpoints.py	91.83%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_ptu_model_config (d2be1ce) with litellm_internal_staging (b0fd3e1)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (334e6da) during the generation of this report, so 4c1fb9b was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[yucheng-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f46ef05. Configure here.}

[cursor]

Merged PTU window not validated

High Severity

The _validate_ptu_model_info function doesn't check that ptu_effective_to is after ptu_effective_from. This allows invalid PTU date ranges to be persisted. When these models are later loaded, ModelInfo validation fails, potentially dropping models from the live router or blocking updates.

Additional Locations (1)

• litellm/proxy/management_endpoints/model_management_endpoints.py#L144-L161

 

^{Reviewed by Cursor Bugbot for commit f46ef05. Configure here.}

[yucheng-berri]

@greptileai fixed: the effective window now rejects to <= from with a 400, and clearing a PTU field removes it from model_info. Re-review please

[yucheng-berri]

@greptileai the effective-window check now coerces naive and offset-aware timestamps to UTC before comparing, so mixed-awareness inputs no longer crash parsing. Re-review please

[yucheng-berri]

@greptileai rebased onto the updated base with no content change; the naive/aware timestamp-comparison fix is unchanged. Re-review please

[yucheng-berri]

@greptileai rebased onto current staging; the PTU window is now validated before the count/rate gate so a one-ended patch cannot persist an inverted window. Re-review please

[yucheng-berri]

@greptileai review Rebased onto current staging, resolving conflicts in constants, router types, and the model update endpoint. Please re-review

[yucheng-berri]

@greptileai review Annotated the new PTU locals as Final to clear the type-discipline budget. Rebased onto current staging

[yucheng-berri]

@greptileai review The update path validates the merged model_info now, so raising the rate on a configured model is no longer rejected for carrying no start

[yucheng-berri]

@greptileai review

[yucheng-berri]

@greptileai review latest head

[yucheng-berri]

@greptileai review

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 4 additional findings in Devin Review.

[devin-ai-integration]

🟡 New tests replace parts of a class at runtime instead of injecting dependencies

The new tests swap out an authorization method on a shared class at runtime (monkeypatch.setattr(mme.ModelManagementAuthChecks, ...) at tests/test_litellm/proxy/management_endpoints/test_ptu_model_settings.py:325), which the repository's guidelines call an anti-pattern.
Impact: The test approach violates an explicit repository testing rule.

Rule source

CLAUDE.md states: "Monkeypatching attributes of a class to do testing is an anti-pattern. Prefer dependency-injecting things into classes. That way, at unit test time, you can pass a mocked dependency in". The helper at tests/test_litellm/proxy/management_endpoints/test_ptu_model_settings.py:315-332 patches both module-level functions and the ModelManagementAuthChecks.allow_team_model_action class attribute; the class-attribute patch is the one the rule names directly.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[yucheng-berri]

@greptileai review