Skip to content

test(e2e): drop xfail markers for now-fixed team-budget-JSON and custom-pricing-leak bugs #31249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mubashir1osmani merged 1 commit into from
Jun 25, 2026
+6 −20
Merged

test(e2e): drop xfail markers for now-fixed team-budget-JSON and custom-pricing-leak bugs #31249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 4 additions & 11 deletions tests/e2e/budgets/test_team_multi_window_budget_e2e.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,10 @@
and calls flow again. This exercises the reset_budget_windows TEAM branch (raw SQL over
LiteLLM_TeamTable.budget_limits, the literal #25109 path), which had no live coverage.

Fails at team creation today: /team/new writes the raw window list straight to the
Json? column, where Prisma rejects it (500), unlike the key path and /team/update which
json.dumps it first. Marked xfail(strict=True) so the suite stays green while the bug
persists and flips to a failure the moment the write is fixed and the marker should be
removed.
This also guards the /team/new write path: it must json.dumps the window list into
the Json? column. A raw list there made Prisma reject the create with a 500 (the key
path and /team/update already json.dumps first); a regression would fail team creation
here.
"""

import time
Expand All @@ -33,12 +32,6 @@ def _call(client: BudgetClient, key: str):
return client.chat(key, "claude-haiku-4-5", f"team-window {unique_marker()}", max_tokens=16)


@pytest.mark.xfail(
strict=True,
reason="known proxy bug: /team/new writes budget_limits straight to the Json? "
"column and Prisma rejects it (500), unlike the key path and /team/update which "
"json.dumps first; remove this marker once that write is fixed",
)
def test_team_short_window_blocks_then_resets(client: BudgetClient, resources: ResourceManager) -> None:
team_id = client.create_team(
alias=f"e2e-team-window-{unique_marker()}",
Expand Down
11 changes: 2 additions & 9 deletions tests/e2e/llm_translation/test_custom_pricing_e2e.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,8 @@
- reporting: /model/info surfaces those rates for the model
- isolation: gemini-2.5-flash shares the same underlying gemini/gemini-2.5-flash
but sets no override, so it must keep its own price; an override that leaks into
the shared cost map misprices it. This fails on a real proxy gap today, so it is
marked xfail(strict=True): the suite stays green while the leak persists and
flips to a failure the moment isolation is fixed and the marker should be removed.
the shared cost map misprices it. A regression that reintroduces that leak makes
the sibling's rate match the custom one and fails the isolation check.
"""

import time
Expand Down Expand Up @@ -192,12 +191,6 @@ def test_model_info_reports_custom_pricing(client: PassthroughClient) -> None:
)


@pytest.mark.xfail(
strict=True,
reason="known proxy bug: a deployment's custom per-token pricing leaks into the "
"shared cost map for sibling deployments of the same underlying model; remove "
"this marker once isolation is fixed",
)
def test_custom_pricing_is_isolated_from_sibling_deployment(
client: PassthroughClient,
) -> None:
Expand Down
Loading