test(e2e): a member's team budget cuts off only that member's key - #33718
Conversation
Greptile SummaryThis PR adds isolation coverage to
Confidence Score: 5/5Test-only change that adds a well-scoped isolation assertion; no production code is touched and teardown ordering is correct. The diff is confined to a single e2e test file and only adds a teammate B fixture plus a run() override. Teardown registration follows the same LIFO pattern used by every other case in the file (keys appended after users, so deleted first in reverse). Lambda closures over teammate_id are safe because the variable is never reassigned after the lambda is created. The _teammate_key attribute is set in init() before run() can reference it, consistent with the framework contract. No production paths are affected. No files require special attention.
|
| Filename | Overview |
|---|---|
| tests/e2e/quota_management/budgets/test_budget_enforcement_e2e.py | Extends TeamMemberBudgetCase with an uncapped teammate B and a run() method that verifies member A's block doesn't bleed onto B; teardown ordering, closure semantics, and structural attribution are all correct. |
Reviews (3): Last reviewed commit: "test(e2e): drop the float-formatted cap ..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
@greptileai re review |
300c844 to
e9ab684
Compare
|
@greptileai re review |
0e03795
into
litellm_internal_staging
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Captured at f83b515, before the branch was rebased onto litellm_internal_staging once #33632 squash-merged; the same assertions now sit at e9ab684 and the member case re-ran green after each revision against a live proxy (docker compose stack in tests/e2e) with real paid llama-3.3-70b-versatile calls through the
claude-haiku-4-5model group. A team with a roomy max_budget (100.0), members A and B (both users with roomy 100.0 budgets), A capped with max_budget_in_team 3e-06, B uncapped, one team-member key each (team_id + user_id):With team and user budgets at 100.0 and the key uncapped, only A's member cap can block, and B keeps serving after A is cut off. All 6 enforcement cases pass against the same live proxy (6 passed in 14.82s) and
basedpyright tests/e2ereports 0 errorsType
✅ Test
Changes
The team-member budget case in
tests/e2e/quota_management/budgets/test_budget_enforcement_e2e.py(coveringquota_management.budget.team_member.blocks_over_limit) already proved the enforcement half: member A with a tinymax_budget_in_teamunder a roomy team gets blocked. What nothing asserted is the isolation half of the same customer story: when A burns through their member cap, their teammates must keep working.TeamMemberBudgetCasenow adds an uncapped member B with their own team-member key; after A's key is refused, one call on B's key must return a successful completion, proving A's cap did not leak onto the team or its other membersThe refusal shape is also pinned: A's block must be HTTP 429
budget_exceeded. Attribution to the member cap is structural rather than message-based: with the team and both user budgets at 100.0 and the key uncapped, no other cap can block. An earlier revision also matched the cap value in the error message, but that couples the test to the proxy's float formatting, so it was droppedTargets litellm_internal_staging directly; the
_assert_budget_blocksreturn value it builds on landed with #33632. The diff here is only the team-member caseQA runbook
{"team_alias":"qa-member-budget","max_budget":100.0}and save the team_id{"max_budget":100.0}twice and save user_a and user_b{"team_id":"<team_id>","member":{"role":"user","user_id":"<user_a>"},"max_budget_in_team":0.000003}, then again for user_b without max_budget_in_team{"team_id":"<team_id>","user_id":"<user_a>"}, then the same for user_bFinal Attestation