test(e2e): assert an org budget block is a 429 naming the organization

[ryan-crabbe-berri]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Captured at c1ef5a5 (pre-rebase; the branch was later rebased onto the updated base as 8f96c00 and both budget cases re-ran green there) against a live proxy (docker compose stack in tests/e2e) with real paid llama-3.3-70b-versatile calls through the claude-haiku-4-5 model group. Create an org with a tiny max_budget, a team under it with no budget, and a key with only team_id:

$ curl -s -X POST http://localhost:4000/organization/new -H 'Authorization: Bearer sk-1234' \
    -H 'Content-Type: application/json' -d '{"organization_alias":"e2e3-proof","max_budget":0.000003}'
$ curl -s -X POST http://localhost:4000/team/new -H 'Authorization: Bearer sk-1234' \
    -H 'Content-Type: application/json' -d '{"team_alias":"e2e3-team","organization_id":"<org_id>"}'
$ curl -s -X POST http://localhost:4000/key/generate -H 'Authorization: Bearer sk-1234' \
    -H 'Content-Type: application/json' -d '{"team_id":"<team_id>"}'

Drive /v1/chat/completions with that key. Call 1 succeeds and burns through the org cap; call 2 onward is a 429 budget_exceeded that names the organization as the blocker:

--- call 1 ---
... "usage":{"completion_tokens":16,"prompt_tokens":38,"total_tokens":54, ...
HTTP 200
--- call 2 ---
{"error":{"message":"Budget has been exceeded! Organization=6aad3043-3761-478d-862c-091afb68442c Current cost: 3.506e-05, Max budget: 3e-06","type":"budget_exceeded","param":null,"code":"429"}}
HTTP 429
--- call 3 ---
{"error":{"message":"Budget has been exceeded! Organization=6aad3043-3761-478d-862c-091afb68442c Current cost: 3.506e-05, Max budget: 3e-06","type":"budget_exceeded","param":null,"code":"429"}}
HTTP 429

All 6 enforcement cases pass against the same live proxy (6 passed in 19.81s) and basedpyright tests/e2e reports 0 errors

Type

✅ Test

Changes

The org budget case in tests/e2e/quota_management/budgets/test_budget_enforcement_e2e.py (covering quota_management.budget.organization.blocks_over_limit) already sets up the scenario structurally: the org carries a tiny max_budget while the team under it and the key carry none, so only the org can block. What it never asserted is that the block is attributable to the org from the customer's side. OrganizationBudgetCase now gets its own run() that asserts the refusal is HTTP 429 with error type budget_exceeded AND that the error message names the blocking entity (Organization=<org_id>), so a block that happens to come from some other layer can no longer pass as org-level enforcement

Stacked on #33632 since it strengthens the same file; the diff here is only the org case

QA runbook

• tests/e2e/quota_management/budgets/test_budget_enforcement_e2e.py::test_budget_enforcement[OrganizationBudgetCase] - an org's max_budget stops a team key under it even though neither the team nor the key carries a budget, and the refusal is a 429 budget_exceeded naming the org (needs ANTHROPIC_API_KEY for the claude-haiku-4-5 model in the compose config)
  • Create a capped org: curl -X POST http://localhost:4000/organization/new -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"organization_alias": "qa-org-budget", "max_budget": 0.000003}'
  • Create a team inside it with no budget: curl -X POST http://localhost:4000/team/new -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"team_alias": "qa-org-team", "organization_id": "<org_id>"}'
  • Mint a key with only team_id: curl -X POST http://localhost:4000/key/generate -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"team_id": "<team_id>"}'
  • Send /v1/chat/completions requests with that key ({"model": "claude-haiku-4-5", "messages": [{"role": "user", "content": "spend"}], "max_tokens": 16}) a couple of seconds apart until one is refused
  • Expect the refusal to be HTTP 429 with body type "budget_exceeded" and message "Budget has been exceeded! Organization=<org_id> ..."; any other status, error type, or blocking entity is a failure
  • Sanity check: this test makes sense to add and is not hand-wavey (e.g., assert actual expected spend instead of just spend > 0) or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

This PR strengthens the existing OrganizationBudgetCase e2e test by overriding run() with two explicit assertions: the blocked response must be HTTP 429, and the error body must contain Organization=<org_id> to confirm the org (and not some other layer) is the entity that triggered the refusal. A companion refactor promotes org_id from a local variable to self._org_id so the new run() can reference it.

• OrganizationBudgetCase.init(): org_id renamed to self._org_id so the teardown lambda and the new run() share the same value.
• OrganizationBudgetCase.run(): new override — calls _assert_budget_blocks, then asserts status_code == 429 and f"Organization={self._org_id}" in blocked.body; a block from any other budget layer (team, key) now causes the test to fail explicitly.

Confidence Score: 5/5

Safe to merge — only touches an e2e test file, adds stronger assertions, and leaves all other test cases untouched.

The change is a net improvement: it replaces a weaker 'budget blocks eventually' check with explicit assertions on HTTP status and which entity triggered the block. No existing assertion is relaxed, no production code is touched, and the teardown cleanup is preserved correctly with self._org_id.

No files require special attention.

Important Files Changed

Filename	Overview
tests/e2e/quota_management/budgets/test_budget_enforcement_e2e.py	Adds a custom run() to OrganizationBudgetCase that asserts HTTP 429 and Organization=<id> in the blocked response body; also promotes org_id from a local variable to self._org_id so run() can reference it.

_{Reviews (2): Last reviewed commit: "test(e2e): assert an org budget block is..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[ryan-crabbe-berri]

@greptileai re review