test(e2e): cover key budget_limits multi-window on personal, team, and team-member keys - #33898
Conversation
…d team-member keys
Greptile SummaryThis PR extends e2e coverage for key-level
Confidence Score: 5/5Safe to merge — test-only changes with no production code touched Both files are e2e tests living entirely under tests/e2e/. The refactoring is mechanical (class wrap, helper extraction) and does not remove or weaken any existing assertions — it adds one (block attribution must name the 30s window). The new parametrized test follows the same flow as the pre-existing bare-key test but mints three different key contexts. Cleanup ordering via resources.defer is consistent with the LIFO pattern already used throughout the suite. No production code is changed. No files require special attention
|
| Filename | Overview |
|---|---|
| tests/e2e/quota_management/budgets/test_multi_window_budget_e2e.py | Adds parametrized test covering personal/team/team-member key kinds, extracts shared helpers, wraps tests into class, and strengthens attribution assertion; no logic regressions found |
| tests/e2e/quota_management/budgets/test_team_multi_window_budget_e2e.py | Pure mechanical wrap of existing tests into TestTeamMultiWindowBudget class; no logic changes |
Reviews (2): Last reviewed commit: "refactor(e2e): group multi-window budget..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Pure mechanical wrap: the tests move into TestKeyMultiWindowBudget and TestTeamMultiWindowBudget per the suite's lay-the-pattern-down-in-a-class convention; helpers stay module-level and bodies are unchanged besides indentation and the two signatures that would cross 120 chars
|
@greptileai re review |
Relevant issues
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
Originally written on top of #33832; that PR merged mid-flight, so this branch is rebased onto staging past the squash merge
All runs are against a live proxy on localhost:4000 running staging code with a throwaway Postgres and Redis, the fast budget rescheduler, and real provider calls (
gpt-5.5andclaude-haiku-4-5are backed bygroq/llama-3.3-70b-versatilelocally since that is the only live provider key on this machine)The new sweep ran five consecutive times at commit 734e633 with cooldowns: 3 passed in 221.61s, 239.20s, 194.12s, 249.34s, 259.16s; zero failures (the spread is 30s wall-clock window alignment)
Proof restamped after the class-wrap commit at head 2ad30c0: three consecutive full-file runs of each touched file, zero failures
basedpyright tests/e2ereports 0 errors andpython -m coverage_registry.collector --strictpassesType
✅ Test
Changes
Two commits. The first covers the "Key
budget_limits(multi-window) blocks + resets" row of the budgets coverage matrix for the three unexercised key kinds. The existingtest_short_window_blocks_then_resetsproves a bare key's tight 30s window trips and self-heals while a roomy 1m window rides along; the new parametrizedtest_short_window_blocks_then_resets_across_key_kindsproves the same key-level windows still enforce and reset when the key is minted to a person (user_id), a team (team_id), or a team membership (both), each with roomy (100.0) surrounding budgets so only the key's own windows can block. The shared body is hoisted into_assert_short_window_blocks_then_resetsand now also asserts the block is attributed to the 30s window ("over 30s budget" in the refusal body, mirroring the long-window test's attribution check), so a future change that lets a surrounding budget produce the block fails instead of passing silentlyThe second commit is a pure mechanical wrap: the tests of this file and its team sibling move into
TestKeyMultiWindowBudgetandTestTeamMultiWindowBudgetspec classes, per the suite's lay-the-pattern-down-in-a-class convention (these two files were the only budgets files with loose module-level tests). Helpers stay module-level; bodies are unchanged besides indentation,self, and the two def signatures that would cross 120 chars. Node ids gain the class prefixAll three parametrized cases intentionally mark the existing registry cell
quota_management.budget.key_multi_window.blocks_then_resets(shared with the bare-key test): the matrix tracks the per-key-kind split while the registry keeps one row per budget level x behavior. The long-window direction stays covered on the bare-key shape by #33832's test and is not repeated per key kindQA runbook
TestKeyMultiWindowBudget::test_short_window_blocks_then_resets_across_key_kinds[personal]creates a user withmax_budget100, mints that user a personal key whosebudget_limitsare 30s at 1e-9 and 1m at 1.0, drives/chat/completionsuntil refused, asserts the refusal names the 30s window, then polls until calls flow again within the elapsed bound, holding every refusal during the wait to a budget block; teardown deletes the key, then the userTestKeyMultiWindowBudget::test_short_window_blocks_then_resets_across_key_kinds[team]does the same with a team atmax_budget100 and a team key carrying the window pair; teardown deletes the key, then the teamTestKeyMultiWindowBudget::test_short_window_blocks_then_resets_across_key_kinds[team_member]does the same with a team and user atmax_budget100 each, the membership atmax_budget_in_team100, and a team-member key carrying the window pair; teardown deletes the key, the user, then the teamTestKeyMultiWindowBudget::test_short_window_blocks_then_resetsstill passes through the hoisted helper with its original bare-key mint shapeTestKeyMultiWindowBudget::test_long_window_blocks_after_short_window_resetsand bothTestTeamMultiWindowBudgettests still pass unchanged inside their new classespython -m coverage_registry.collector --strictpasses with the parametrized cases sharingquota_management.budget.key_multi_window.blocks_then_resetsFinal Attestation