fix(e2e): migrate load suite from e2e_gateway to ProxyClient by mubashir1osmani · Pull Request #33839 · BerriAI/litellm

mubashir1osmani · 2026-07-18T19:30:36Z

Relevant issues

Linear ticket

Pre-Submission checklist

I have added meaningful tests
My PR passes all CI/CD checks (e.g., lint, format, unit tests)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Stage e2e collection failed with:

ERROR load - ModuleNotFoundError: No module named 'e2e_gateway'
load/conftest.py: from e2e_gateway import Gateway

After this change, pytest load/ --collect-only collects the load suite with no import error

Type

Bug Fix
Test

Changes

Gateway was renamed to ProxyClient and e2e_gateway.py was removed. The load suite landed on staging still importing the old module, which aborted the whole e2e job at collection. Wire load/conftest and LoadClient to the shared session proxy fixture and ProxyClient like every other suite

QA runbook

tests/e2e/load/test_chat_completions_throughput_e2e.py::TestChatCompletionsThroughput::test_sustains_throughput_slo_under_load - load suite still registers the mock load model and drives Locust against it
- Confirm from proxy_client import ProxyClient and no e2e_gateway under tests/e2e/load
- From tests/e2e: uv run pytest load/ --collect-only -q expects 1 test, no ModuleNotFoundError
- Against a live proxy with master key: run the load test (needs DD/provider env only as the rest of e2e; load model is mock_response via /model/new)
- Sanity check: this is a rename/DI fix only; no load SLO change

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Unique batch model names avoid load-balancing onto stale azure-batch deployments that still pointed at the retired gpt-4.1-mini-batch, which only the managed/unified path was hitting. Retry batch retrieve on 500 and /ui/api-keys navigation on ERR_ABORTED. Skip the MCP key-access suite when the compose-only mcp-upstream is unreachable on stage k8s

Register the regional Datadog MCP endpoint with DD-API-KEY / DD-APPLICATION-KEY static headers (CI-safe header auth; browser OAuth is not headless-automatable). Seed a chat completion marked e2e-datadog-mcp-*, assert the proxy shipped it, list tools, call search_datadog_logs for the marker, and delete the server on teardown. Math-upstream key-access tests only skip when that compose service is unreachable

Key-access denial and happy-path MCP e2e both register the real regional Datadog remote MCP server with DD-API-KEY / DD-APPLICATION-KEY headers. Remove the mcp-upstream compose service and FastMCP add/multiply fixture

Document that tests/e2e/mcp must register via datadog_mcp helpers against mcp.<site>/v1/mcp and must not introduce compose or fake MCP upstreams

Keep the FastMCP fixture file; e2e no longer wires it in compose, but the module itself is not part of the Datadog-only cleanup

pytest on the host never inherited compose env_file keys, so DD_API_KEY stayed empty. load_dotenv tests/e2e/.env in e2e_config. Register the dynamically loaded datadog_reader module in sys.modules so dataclasses do not crash under Python 3.12

Put the provider deployment name in every JSONL body so Azure does not depend on a perfect model rewrite. Retry create/retrieve/cancel on transient statuses with backoff. Drop cancel assertions for azure and vertex (registry only has a shared basic cell; create+retrieve prove routing, cancel stays best-effort cleanup)

Post-login client redirects abort the first /ui/api-keys/ goto on stage. Wait off /ui/login after cookie set, then accept the page once Create New Key is visible even if goto raised ERR_ABORTED

API management e2e already covers key generate/update persistence. The UI Models-dropdown sentinel cases only added SPA ERR_ABORTED noise and no unique product signal. Remove the suite and unused browser fixtures

…down Resolve conflicts in management/mcp conftest and mcp_client by keeping our Datadog-only MCP suite, ProxyClient DI from staging, and the removed Playwright key-models-dropdown tests

Replace bare next() over PROVIDERS with _model_for that raises ValueError naming the missing provider and the known list, instead of StopIteration

…tellm_e2e_stage_green_fixes

Stage collection failed with ModuleNotFoundError: e2e_gateway after the Gateway rename. Wire load/conftest and LoadClient to the shared ProxyClient fixture like every other suite

greptile-apps · 2026-07-18T19:32:29Z

Greptile Summary

This PR fixes a collection-time ModuleNotFoundError in the load test suite caused by a stale import of the removed e2e_gateway module. The fix wires load/conftest.py and LoadClient to the existing session-scoped proxy fixture and ProxyClient, consistent with every other e2e suite.

Removes the from e2e_gateway import Gateway import and replaces all gateway references with proxy: ProxyClient in both changed files.
Changes build_client() from calling the now-deleted build_gateway() internally to accepting an injected ProxyClient, aligning with the dependency-injection pattern used elsewhere in the e2e harness.

Confidence Score: 5/5

Safe to merge — the change is a pure rename/DI fix in test infrastructure with no logic alterations.

Both files change only symbol names and the DI wiring; the load SLO logic, fixture scopes, teardown ordering, and cleanup paths are unchanged. LoadClient.proxy satisfies the ProxyClientProvider protocol (both statically and at runtime via hasattr), so the shared resources fixture continues to work correctly. No new code paths are introduced.

No files require special attention.

Important Files Changed

Filename	Overview
tests/e2e/load/conftest.py	Replaces Gateway/e2e_gateway with ProxyClient/proxy_client throughout, and injects the session-scoped proxy fixture into client rather than calling the removed build_gateway() internally.
tests/e2e/load/load_client.py	Renames the gateway: Gateway field to proxy: ProxyClient and updates build_client to accept a ProxyClient argument, matching the ProxyClientProvider protocol used by the shared resources fixture.

_{Reviews (1): Last reviewed commit: "fix(e2e): drop duplicate datadog_mcp_url..." | Re-trigger Greptile}

ishaan-berri

GREAT WORK!!!!!!! YOUR THE GOAT

codecov · 2026-07-18T19:36:13Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

codspeed-hq · 2026-07-18T19:49:25Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_e2e_stage_green_fixes (824f177) with litellm_internal_staging (fdf380d)}

mubashir1osmani added 14 commits July 18, 2026 11:14

test(e2e): drop compose math MCP upstream; use Datadog only

ae5b67e

Key-access denial and happy-path MCP e2e both register the real regional Datadog remote MCP server with DD-API-KEY / DD-APPLICATION-KEY headers. Remove the mcp-upstream compose service and FastMCP add/multiply fixture

docs(e2e): require real Datadog MCP for all mcp suite tests

5054af8

Document that tests/e2e/mcp must register via datadog_mcp helpers against mcp.<site>/v1/mcp and must not introduce compose or fake MCP upstreams

chore: restore mcp_e2e_upstream_server.py

8ff5edd

Keep the FastMCP fixture file; e2e no longer wires it in compose, but the module itself is not part of the Datadog-only cleanup

test(e2e/ui): treat api-keys shell as success after SPA ERR_ABORTED

80d21ae

Post-login client redirects abort the first /ui/api-keys/ goto on stage. Wait off /ui/login after cookie set, then accept the page once Create New Key is visible even if goto raised ERR_ABORTED

test(e2e): drop flaky key models dropdown Playwright suite

498a8d9

API management e2e already covers key generate/update persistence. The UI Models-dropdown sentinel cases only added SPA ERR_ABORTED noise and no unique product signal. Remove the suite and unused browser fixtures

merge litellm_internal_staging; keep e2e Datadog MCP and drop UI drop…

f26c0b1

…down Resolve conflicts in management/mcp conftest and mcp_client by keeping our Datadog-only MCP suite, ProxyClient DI from staging, and the removed Playwright key-models-dropdown tests

test(e2e/batches): fail clearly when OPENAI/AZURE provider is missing

4a17be4

Replace bare next() over PROVIDERS with _model_for that raises ValueError naming the missing provider and the known list, instead of StopIteration

Merge remote-tracking branch 'berri/litellm_internal_staging' into li…

a568b5a

…tellm_e2e_stage_green_fixes

fix(e2e): migrate load suite from e2e_gateway to ProxyClient

78a78c1

Stage collection failed with ModuleNotFoundError: e2e_gateway after the Gateway rename. Wire load/conftest and LoadClient to the shared ProxyClient fixture like every other suite

fix(e2e): drop duplicate datadog_mcp_url and CLAUDE section after merge

824f177

mubashir1osmani enabled auto-merge (squash) July 18, 2026 19:31

ishaan-berri approved these changes Jul 18, 2026

View reviewed changes

mubashir1osmani merged commit 6a2e0a8 into litellm_internal_staging Jul 18, 2026
77 checks passed

mubashir1osmani deleted the litellm_e2e_stage_green_fixes branch July 18, 2026 19:41

yassin-berriai mentioned this pull request Jul 18, 2026

test(e2e): spendlog cost for streaming /v1/messages via responses bridge #33753

Merged

10 tasks

Navigation Menu

Uh oh!

Uh oh!

fix(e2e): migrate load suite from e2e_gateway to ProxyClient - #33839

fix(e2e): migrate load suite from e2e_gateway to ProxyClient#33839
mubashir1osmani merged 14 commits into
litellm_internal_stagingfrom
litellm_e2e_stage_green_fixes

mubashir1osmani commented Jul 18, 2026

Uh oh!

greptile-apps Bot commented Jul 18, 2026

Important Files Changed

Uh oh!

ishaan-berri left a comment

Uh oh!

codecov Bot commented Jul 18, 2026

Uh oh!

Uh oh!

codspeed-hq Bot commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

mubashir1osmani commented Jul 18, 2026

Relevant issues

Linear ticket

Pre-Submission checklist

Delays in PR merge?

Screenshots / Proof of Fix

Type

Changes

QA runbook

Final Attestation

Uh oh!

greptile-apps Bot commented Jul 18, 2026

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Uh oh!

ishaan-berri left a comment

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jul 18, 2026

Codecov Report

Uh oh!

Uh oh!

codspeed-hq Bot commented Jul 18, 2026

Merging this PR will not alter performance

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants