feat(ptu): runtime UI flag for enable_ptu_cost_attribution + always-registered rollup - #33439
feat(ptu): runtime UI flag for enable_ptu_cost_attribution + always-registered rollup#33439yucheng-berri wants to merge 1 commit into
Conversation
Greptile SummaryThis PR wires
Confidence Score: 4/5Safe to merge — the core logic change (unconditional job registration + flag-gated execution) is correct, the sync path is consistent with other runtime flags, and the UI follows established patterns. The implementation is clean and well-tested. The one minor concern is that test_ptu_rollup_job_registered_regardless_of_flag asserts against the module-level ps.scheduler object after the with block exits, which relies on that global not being reset between tests. If test ordering changes, a later test could see a scheduler in an unexpected state. It is not a production defect but is worth noting. tests/test_litellm/proxy/test_proxy_server.py — the new test reads module-level scheduler state after context-manager patches exit; worth confirming test isolation holds across the full suite run.
|
| Filename | Overview |
|---|---|
| litellm/proxy/proxy_server.py | Moves PTU rollup job registration outside the feature-flag gate; import and scheduler.add_job now run unconditionally. The job's own flag check in run_ptu_reservation_rollup handles the no-op path cleanly. |
| litellm/proxy/ui_crud_endpoints/proxy_setting_endpoints.py | Adds enable_ptu_cost_attribution to UISettings model, ALLOWED_UI_SETTINGS_FIELDS, and _RUNTIME_GENERAL_SETTINGS_FLAGS. The flag correctly lands in general_settings on both write and startup sync, which is required for the rollup's runtime flag check. |
| tests/test_litellm/proxy/test_proxy_server.py | Adds test asserting PTU rollup job is registered even when enable_ptu_cost_attribution is False at startup. Test relies on module-level scheduler state; has a minor concern around test isolation but is otherwise well-structured. |
| tests/test_litellm/proxy/ui_crud_endpoints/test_proxy_setting_endpoints.py | Adds two tests: one verifying PATCH persists and syncs enable_ptu_cost_attribution to general_settings, and one asserting membership in both ALLOWED_UI_SETTINGS_FIELDS and _RUNTIME_GENERAL_SETTINGS_FLAGS. Good coverage of the sync path. |
| ui/litellm-dashboard/src/components/Settings/AdminSettings/UISettings/UISettings.tsx | Adds the enable_ptu_cost_attribution Switch toggle with page-refresh behavior, matching the existing enable_chat_ui and enable_projects_ui patterns. aria-label defaults to the schema description. |
| ui/litellm-dashboard/src/components/Settings/AdminSettings/UISettings/UISettings.test.tsx | Adds test verifying the PTU toggle calls updateSettings with the correct payload and triggers the page-refresh success notification. Mock schema uses 'Enable PTU cost attribution' as the description (matching the aria-label query), so the test is internally consistent. |
Reviews (1): Last reviewed commit: "feat(ptu): runtime UI flag for enable_pt..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
4100e34 to
06d8b45
Compare
a06bcf7 to
0ffa82f
Compare
…egistered rollup Add enable_ptu_cost_attribution to UISettings so admins can flip it from the UI's Admin Settings page without a proxy restart. Register the daily rollup job unconditionally at startup; it already no-ops on flag-off, so the runtime toggle takes effect on the next scheduled fire without any restart. Existing per-request guards in the /ptu_reservation endpoints and in run_ptu_reservation_rollup remain the source of truth. Persisted settings sync into general_settings on both PATCH and GET paths as well as on server startup via _sync_ui_settings_to_general_settings.
0ffa82f to
1b85426
Compare
|
Closing as superseded. This is part of the v1 PTU design, which stored PTU config in a separate reservation table. The shipped design puts that config on the model deployment instead, merged as #35341, #35343, #35391, #35393 and #36829. The runtime gate landed as litellm/proxy/spend_tracking/ptu_feature_flag.py. The branch is kept, so nothing here is lost. |
Relevant issues
Stage 5 of the LIT-1697 stack, sitting on top of #33302
Linear ticket
Resolves LIT-1697
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
QA runbook below has the concrete steps against a live proxy hitting real Azure PTU + gpt-4. Paste output from steps 1 through 5 back once run
Type
New Feature
Changes
Wires
enable_ptu_cost_attributioninto the runtime-flippable UI settings pipeline so an admin can turn the whole PTU cost attribution feature on and off from the Admin Settings page without a proxy restart. The flag is now declared onUISettings, added toALLOWED_UI_SETTINGS_FIELDS, and added to_RUNTIME_GENERAL_SETTINGS_FLAGS, which is the list/update/ui_settingsand/get/ui_settingssync intogeneral_settings; the same sync also runs on startup via_sync_ui_settings_to_general_settings, so a value persisted through the UI survives a restart.Moves the daily rollup job's
scheduler.add_jobcall outside the flag gate.run_ptu_reservation_rollupalready returns early when the flag is off (added in #33137 asskipped_flag_off=True), so scheduling the job unconditionally is a no-op until the flag flips. This is what makes the flag actually live-flippable; before this PR, flipping the flag would only affect the endpoints and UI, and the rollup would still need a restart to attach.Adds a Switch on the Admin Settings > UI Settings page ("Enable PTU cost attribution") next to the existing
enable_chat_uitoggle, with the same page-refresh notice pattern used for enable_projects_ui and enable_chat_ui. The page-refresh handles reflowing the leftnav to show the PTU Reservations item and the Usage page column.Behavior changes
PATCH /update/ui_settingsnow acceptsenable_ptu_cost_attribution: bool; unknown-field validation keeps the same shape it did for the other UI flagsGET /get/ui_settingsreturnsenable_ptu_cost_attributioninvaluesand its description infield_schemaptu_reservation_rollup_jobcron is registered on every startup regardless of flag state; it fires daily at 00:15 UTC and no-ops (returningRollupResult(skipped_flag_off=True)) while the flag is off. Zero-write, zero-read while disableduseIsPtuCostAttributionEnabled; no config-file change required to unlock them once the UI toggle is onNothing else changes for tenants that never touch the flag. The per-request spend hot path is not modified in this PR (or any earlier stage in the stack).
QA runbook
Prereq: real Azure PTU deployment. Run against a live proxy started with
python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debug --reload --use_v2_migration_resolver 2>&1 | tee litellm.logand a fresh DB.Confirm the flag is off by default.
curl -H "Authorization: Bearer sk-1234" http://localhost:4000/get/ui_settings | jq '.values.enable_ptu_cost_attribution'returnsfalse(or absent).curl -H "Authorization: Bearer sk-1234" -X POST http://localhost:4000/ptu_reservation/new -H "Content-Type: application/json" -d '{"team_id":"any","model":"gpt-4","ptu_count":1,"cost_per_ptu":100.0,"effective_from":"2026-08-01T00:00:00Z"}'returns 403 withPTU cost attribution is not enabled.Flip it on via the UI. Open http://localhost:4000/ui/ > Admin Settings > UI Settings. Toggle "Enable PTU cost attribution" on. The page reloads. The leftnav now shows "PTU Reservations".
Confirm the sync happened without a restart.
curl -H "Authorization: Bearer sk-1234" http://localhost:4000/get/ui_settings | jq '.values.enable_ptu_cost_attribution'returnstrue.Create a reservation and see the flat cost land. Follow steps 2 through 5 of
05-stage-rollout.md's original runbook (create reservation for a real Azure gpt-4 team, runpython scripts/ptu_reservation_backfill.py --date 2026-07-15, thenGET /team/daily/activityfor that team). Confirmmetrics.flat_costis nonzero for that day.Flip it off. Toggle back off in the UI, refresh. Nav item disappears.
POST /ptu_reservation/newreturns 403 again.GET /team/daily/activitystill returns the historical row from step 4 unchanged (flag-off does not delete data).Restart the proxy after step 2 (with the flag still on in the DB). Confirm on restart that the log shows
Synced UI settings to general_settings on startup: ['enable_ptu_cost_attribution']and step 3's curl still returnstrue. Persistence survives restart.Final Attestation