test(e2e): drop xfail markers for now-fixed team-budget-JSON and custom-pricing-leak bugs - #31249
Conversation
…custom-pricing-leak bugs Both tests were xfail(strict=True) for known proxy bugs: /team/new writing budget_limits as a raw list (Prisma 500) and custom per-token pricing leaking into the shared cost map for sibling deployments. Both are fixed, so the tests pass and strict mode reports the unexpected pass as a failure. Remove the markers (as their reasons instructed) so they run as plain regression guards; docstrings updated to describe the regression each now pins.
Greptile SummaryThis PR promotes two previously-skipped e2e tests from
Confidence Score: 5/5Safe to merge — only xfail markers and docstrings are changed, no test logic or assertions are touched. Both tests retain all their original assertions and now run unconditionally as regression guards. The PR description references the concrete XPASS output that triggered this change, satisfying the team's evidence-of-fix rule. No production code is modified. No files require special attention.
|
| Filename | Overview |
|---|---|
| tests/e2e/budgets/test_team_multi_window_budget_e2e.py | Removes xfail(strict=True) marker from test_team_short_window_blocks_then_resets and updates the module docstring to describe it as a regression guard for the now-fixed /team/new json.dumps bug; no assertions changed. |
| tests/e2e/llm_translation/test_custom_pricing_e2e.py | Removes xfail(strict=True) marker from test_custom_pricing_is_isolated_from_sibling_deployment and updates the module docstring to describe it as a regression guard for the now-fixed pricing isolation bug; no assertions changed. |
Reviews (1): Last reviewed commit: "test(e2e): drop xfail markers for the no..." | Re-trigger Greptile
|
these two tests were internally found bugs that are now fixed on litellm_internal_staging - removing xfail markers |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Relevant issues
Two e2e tests were
xfail(strict=True)for known proxy bugs that are now fixed onlitellm_internal_staging:/team/newwrotebudget_limitsas a raw list into theJson?column, which Prisma rejected with a 500 (the key path and/team/updatejson.dumps first). Related: feat: multiple concurrent budget windows per API key and team (#24883) #25109Pre-Submission checklist
Screenshots / Proof of Fix
Both fixes already shipped to
litellm_internal_staging, so the two tests pass their assertions there. Because they were still markedxfail(strict=True), the run reported the unexpected passes as failures:With the strict markers removed, both run as ordinary passing tests, taking the suite to all-green; their assertions are unchanged, so they now stand as regression guards for the two fixes
Type
🧹 Refactoring
✅ Test
Changes
Removed the
@pytest.mark.xfail(strict=True, ...)decorator fromtest_team_short_window_blocks_then_resetsandtest_custom_pricing_is_isolated_from_sibling_deployment, as each marker's reason instructed once the underlying bug was fixed. Updated both docstrings to describe the regression each test now pins rather than the bug it used to tolerate. No assertion changed