feat(proxy): add GET /management/v1/budgets - #35310
Conversation
Paging, sorting, filtering and search for an entity collection, declared once as a ListSpec and served by handle_list. The route injects a ListExecutor that owns its table, so this module never imports Prisma. The caller's scope is derived from the caller alone and ANDed with whatever they filtered on, so a query parameter can only narrow what they may read. This is the shared half of the budgets list; it lands here so the endpoint has something to register against, and drops out when the framework arrives on its own branch.
The Budgets page reads /budget/list, which returns the whole table as a bare array with no way to page, sort or filter it. A customer with enough budgets to fill the page has no way to find one. Registers LiteLLM_BudgetTable against the management/v1 list contract: sortable on budget_id, max_budget, tpm_limit, rpm_limit and created_at, default order newest-first with budget_id breaking ties, search on budget_id, and filters for budget_duration, max_budget and created_at. budget_duration is deliberately not sortable; the column holds "7d"/"30d" strings, so a lexicographic ORDER BY puts "30d" ahead of "7d". tpm_limit and rpm_limit are BigInt? in Prisma, so rows validate through a pydantic model on the way out and serialize as JSON numbers. A caller without admin view is refused 403 as a problem document rather than served an empty page. /budget/list is untouched.
Greptile SummaryAdds a paginated management API for listing budgets.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains; offset-less datetime filters are normalized to aware UTC values before binding, and the SQL converts the resulting instant to the naive UTC representation used by the database column.
|
| Filename | Overview |
|---|---|
| litellm/proxy/management_endpoints/management_v1/list_framework.py | Normalizes parsed datetimes to aware UTC values and renders timestamp-compatible SQL binds, resolving the previously reported naive-datetime boundary shift. |
| litellm/proxy/management_endpoints/management_v1/budgets.py | Implements the authenticated paginated budget-list endpoint with restricted fields, filtering, sorting, and deterministic pagination. |
| litellm/proxy/_types.py | Grants the new read-only route to the existing admin-viewer access tier. |
| tests/test_litellm/proxy/management_endpoints/management_v1/test_budgets.py | Covers response shape, authorization, query planning, timestamp normalization, SQL generation, pagination, and serialization. |
| tests/e2e/management/test_budget_customer_user_org_e2e.py | Adds live-proxy coverage for budget sorting, paging, filtering, numeric limits, and non-admin rejection. |
| ui/litellm-dashboard/src/lib/http/schema.d.ts | Adds generated TypeScript declarations for the new budget-list operation and response models. |
Reviews (2): Last reviewed commit: "refactor(proxy): fold the predicate rend..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…itellm_/management-v1-budgets # Conflicts: # litellm/proxy/management_endpoints/management_v1/list_framework.py # litellm/types/proxy/management_endpoints/management_v1.py
PR #35308 landed a different shape than this branch was written against: `where` is a tuple of frozen predicates rather than a Prisma-shaped mapping, `ListSpec` carries both the row and the wire type, and `where_sql` / `order_by_sql` render for a raw-SQL executor. The budgets executor now queries through `query_raw` the way the spend logs facet does, selecting only the columns it serves. Also casts datetime binds in `where_sql`. They cross into the query engine as JSON, so an uncast placeholder arrives as text and Postgres refuses `timestamp >= text` outright; every `filter[created_at][gte|lte]` was answering 500. The cast reads the bind as an instant and drops it to naive UTC to match Prisma's TIMESTAMP(3) column, the same one /spend/logs/ui applies.
recursive_detector flags `_render_all`, and the flag is fair: it recursed once per predicate, so the stack grew with the number of filters on the request for no reason. Walking a predicate list is a running bind index, which is a fold. `_render` still re-enters for `AnyOf`, but its clauses are plain comparisons built by `?q=`, so that nesting is one level deep and no caller can drive it deeper.
QA verdict: PASSQA'd this against a live proxy on localhost:4000 with a real Postgres, driving the route with curl only. Seeded seven budgets through Happy path: sort, page, envelope, and numeric tpm/rpm
Nulls stay last in both directions, which is the run a missing Paging partitions the set with no duplicates, a page past the end is a 200 with empty data, and created_at filters, the 500 this PR fixesAll three spellings resolve to the same instant and none 500s; the boundary genuinely splits the set (1 row vs the other 6), so the comparison is not a no-op. A malformed value is a 400 problem document rather than a driver error Nothing in the proxy log mentions Problem documents, and the old route left aloneThe side-by-side is the useful part: for the same non-admin caller the new route answers 403 q escaping and injectionThe discriminating row is the second one: Authorization tiersWorth noting for the description: a plain non-admin key is stopped 401 by the route check before reaching the handler, so the handler's 403 is only observable for a key explicitly permitted to call the route. Both are safe refusals and neither is ever a 200 with an empty page Three small observations, none blocking. The PR's own tests also pass against this proxy (392 unit, and the 4 Tested by this Devin session |
QA verdict: PASSA second, independent pass on top of the earlier QA run, on a larger seeded table and covering ground that one did not: the no-database 503 branch, QA'd against a live proxy on Proxy started with: docker run -d --name litellm-pg -e POSTGRES_PASSWORD=litellm -e POSTGRES_USER=litellm -e POSTGRES_DB=litellm -p 5432:5432 postgres:16
DATABASE_URL="postgresql://litellm:litellm@localhost:5432/litellm" STORE_MODEL_IN_DB=True \
uv run --no-sync python litellm/proxy/proxy_cli.py --config qa_budgets_config.yaml --detailed_debug --use_v2_migration_resolver 2>&1 | tee litellm.log
# second proxy, no DB, for the 503 branch
env -u DATABASE_URL uv run --no-sync python litellm/proxy/proxy_cli.py --config qa_budgets_config.yaml --port 4001
model_list:
- model_name: fake-openai
litellm_params:
model: openai/fake
api_key: sk-fake
api_base: http://localhost:8080
general_settings:
master_key: sk-1234
litellm_settings:
drop_params: trueSeeded through The core contract: sorting, paging, links and the projected columns
The response carries exactly the nine serving columns, Filters and the always-appended Walking the whole table at The 403-vs-400 improvementTwo independent gates. No key and a bogus key are 401 at the route gate. A non-admin key that lists Sad path: problem documents, validation, and the no-DB 503
Eight malformed inputs, eight 400s with distinct, actionable details, no 500s: bad datetime, bad float, The proxy started without Edge cases: clamping, timezones, injection safety, unicode, deep paging
Injection payloads in Unicode and CJK substrings both find Regression suites
Two nits, neither blockingOut-of-range pages return 200 with empty Separately, a caller has to percent-encode |
2 participants
TLDR
Problem this solves:
/budget/listreturns the whole table, unpagedHow it solves it:
GET /management/v1/budgetswith paging, sorting, filteringbudget_id, RFC 9457 errors, realtotal_count/budget/listuntouched; the dashboard moves over separatelyRelevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
All of the following was captured against a live proxy at commit
78c756dff9, started withand seeded with four budgets created through
/budget/new:demo-prod-monthly(500, 30d, tpm 60000, rpm 1200),demo-staging-weekly(50, 7d),demo-eval-daily(5, 24h) anddemo-internal-unlimited(no cap, no window).--globoffkeeps curl from eating thefilter[...]bracketsThe last two calls are the same key against both endpoints: the new route answers 403 as a problem document,
/budget/liststill answers 400 in the proxy's OpenAI error shape. That endpoint is deliberately unchanged here; deprecating it is a follow-up once the dashboard stops calling itThe final pair covers the datetime cast. Before it, every
filter[created_at][gte|lte]answered 500 withoperator does not exist: timestamp without time zone >= textType
🆕 New Feature
Changes
GET /management/v1/budgetsregistersLiteLLM_BudgetTableagainst the generic list contract from #35308. The budgets side declares aListSpecand injects aListExecutorthat owns the table, so the planner stays free of any database dependency and the executor is the only place that knows SQLThe spec sorts on
budget_id,max_budget,tpm_limit,rpm_limitandcreated_at, defaulting to-created_at, withbudget_idappended as the tiebreaker on every query.budget_durationis deliberately not sortable: the column holds strings like"7d"and"30d", so a lexicographicORDER BYwould put"30d"ahead of"7d"and quietly mis-order the pageSearch is a case-insensitive contains on
budget_id, the only text identity on the row. Filters arefilter[budget_duration][in|is_null],filter[max_budget][gte|lte|is_null]andfilter[created_at][gte|lte]. Anything else, including an operator a filter does not declare, is refused 400 with the accepted set inallowed, since a silently ignored filter over-returns budgets and that is worse than a rejected request.page_sizedefaults to 50 and clamps at 100The scope is decided from the caller and never from the query string, and the planner puts it ahead of the caller's filters as conjuncts they sit behind, so a filter can only narrow what they may read. A caller without admin view is refused 403 rather than served an empty page, which would read as "this proxy has no budgets". The population is the same one
/budget/listgates on today; only the status code differsThe executor reads through
query_raw, the way the spend logs facet does, selecting only the nine columns it serves rather thanSELECT *, socreated_byandmodel_max_budgetnever reach the browser. Rows validate through a pydantic model on the way out, which is what makestpm_limitandrpm_limitJSON numbers: they areBigInt?in Prisma and the query engine hands them back as decimal stringsOne change lands in the shared
where_sql: datetime binds are now cast. They cross into the query engine as JSON, so an uncast placeholder reaches Postgres as text andtimestamp >= textis rejected outright rather than answered wrongly, which made everycreated_atfilter a 500. The cast reads the bind as an instant and drops it to naive UTC to match Prisma'sTIMESTAMP(3)column, matching what/spend/logs/uialready does. Budgets is the first list with a datetime filter, so nothing exercised this beforeQA runbook
Run against a live proxy on
:4000with the master key. Every step below uses--globoffso curl leaves thefilter[...]brackets alonetests/e2e/management/test_budget_customer_user_org_e2e.py::TestBudgetListV1::test_sorts_pages_and_filters_the_budgets_it_created- three budgets tagged with one marker come back sorted, paged and filtered, with a correct totalcurl -X POST http://localhost:4000/budget/new -H "Authorization: Bearer sk-1234" -H 'Content-Type: application/json' -d '{"budget_id": "qa1-small", "max_budget": 1.0, "budget_duration": "7d", "tpm_limit": 60000}', then the same forqa1-medium(2.0, 30d) andqa1-large(3.0, 30d)curl --globoff 'http://localhost:4000/management/v1/budgets?q=qa1&sort=-max_budget' -H "Authorization: Bearer sk-1234"and expect exactlyqa1-large,qa1-medium,qa1-smallin that ordercurl --globoff 'http://localhost:4000/management/v1/budgets?q=qa1&sort=-max_budget&page=2&page_size=1' -H "Authorization: Bearer sk-1234"and expect onlyqa1-medium,meta.total_count3,meta.total_pages3, and bothlinks.prevandlinks.nextpopulatedcurl --globoff 'http://localhost:4000/management/v1/budgets?q=qa1&filter[budget_duration][in]=30d' -H "Authorization: Bearer sk-1234"and expectqa1-mediumandqa1-largeonlytpm_limitto render as60000, an unquoted JSON number, on every rowtests/e2e/management/test_budget_customer_user_org_e2e.py::TestBudgetListV1::test_is_null_finds_the_budget_left_uncapped-is_nullseparates an uncapped budget from a capped oneqa2-uncappedwith nomax_budgetandqa2-cappedwithmax_budget4.0curl --globoff 'http://localhost:4000/management/v1/budgets?q=qa2&filter[max_budget][is_null]=true' -H "Authorization: Bearer sk-1234"and expect onlyqa2-uncapped, withmax_budgetnullis_null=falseand expect onlyqa2-cappedtests/e2e/management/test_budget_customer_user_org_e2e.py::TestBudgetListV1::test_refuses_a_sort_field_and_a_parameter_it_does_not_support- an unsortable column and an undeclared parameter are both refused 400curl --globoff -i 'http://localhost:4000/management/v1/budgets?sort=budget_duration' -H "Authorization: Bearer sk-1234"and expect 400,application/problem+json, and the five sortable fields inallowedcurl --globoff -i 'http://localhost:4000/management/v1/budgets?filter[budget_id][eq]=x' -H "Authorization: Bearer sk-1234"and expect 400 namingfilter[budget_id][eq]indetailtests/e2e/management/test_budget_customer_user_org_e2e.py::TestBudgetListV1::test_is_refused_for_a_non_admin_key- a key without admin view cannot read the budget listcurl -X POST http://localhost:4000/key/generate -H "Authorization: Bearer sk-1234" -H 'Content-Type: application/json' -d '{}'and keep the returned keycurl --globoff -i http://localhost:4000/management/v1/budgets -H "Authorization: Bearer <that key>"and expect 401 or 403, never a 200 carrying an emptydata{"allowed_routes": ["/management/v1/budgets"]}and expecturn:litellm:error:forbiddenPrerequisites are a proxy with a database attached and the master key; no provider credentials are needed, since nothing here reaches an LLM. Clean up the
qa1-*andqa2-*budgets with/budget/deletewhen you are doneFinal Attestation