fix(e2e): route model management to the control plane

[mubashir1osmani]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

The 2026-07-06 00:59 UTC stage run failed 46 tests and errored 17 more. Every test that registers a deployment at runtime died on one of the two bugs this PR fixes

Proof against the live split deployment on berrie-litellm-stage, with kubectl port-forward svc/litellm-gateway 14000:4000 (data plane) and kubectl port-forward svc/litellm-backend 14001:4001 (control plane). First /model/new against the data plane, which is where the old routing table sent it:

$ curl -s -w "\nhttp_code: %{http_code}\n" -X POST http://localhost:14000/model/new \
    -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
    -d '{"model_name": "e2e-routing-proof", "litellm_params": {"model": "openai/gpt-4o-mini", "api_key": "os.environ/OPENAI_API_KEY"}, "model_info": {"id": "e2e-routing-proof"}}'
{"detail":"Not Found"}
http_code: 404

The identical request against the control plane, which is where this PR routes it, registers the deployment:

$ curl -s -X POST http://localhost:14001/model/new \
    -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
    -d '{"model_name": "e2e-routing-proof", "litellm_params": {"model": "openai/gpt-4o-mini", "api_key": "os.environ/OPENAI_API_KEY"}, "model_info": {"id": "e2e-routing-proof"}}'
{"model_id":"e2e-routing-proof","model_name":"e2e-routing-proof","litellm_params":{...},"mode...

Same split for the teardown route:

$ curl -s -w "\nhttp_code: %{http_code}\n" -X POST http://localhost:14000/model/delete \
    -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" -d '{"id": "e2e-routing-proof"}'
{"detail":"Not Found"}
http_code: 404

$ curl -s -w "\nhttp_code: %{http_code}\n" -X POST http://localhost:14001/model/delete \
    -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" -d '{"id": "e2e-routing-proof"}'
{"message":"Model: e2e-routing-proof deleted successfully"}
http_code: 200

The new regression tests fail on the previous code (6 failed) and pass with the fix

Type

Bug Fix

Changes

Two harness bugs, one routing and one API surface, together account for 33 of the 63 failures in the stage run (the rest are exhausted Anthropic credits plus a handful still under investigation)

SplitTransport routes each path to the data plane or the control plane by prefix, but CONTROL_PLANE_PREFIXES listed only /model/info. POST /model/new and POST /model/delete therefore fell through to the data-plane gateway, which serves no management routes and answers 404. Every llm_translation test that registers its deployment at runtime failed at setup, and the access_control 403 test failed because its probe of /model/new never reached the control plane's auth check. The prefix is now /model/, which covers all model-management routes while /models, the OpenAI-compatible list-models route, stays on the data plane

batch_client.py, test_provider_features_e2e.py, and test_deepseek_reasoning_e2e.py call gateway.create_model, but Gateway never had that method, so all 17 batch tests errored at fixture setup with AttributeError before a single request was sent. Gateway now owns create_model (with the optional mode the batches suite passes) and delete_model, and EndpointsClient delegates to them instead of carrying its own copy, per the harness convention that shared routes live on Gateway

Regression coverage: test_transport.py pins the management-vs-LLM routing predicate, and test_e2e_gateway.py pins the Gateway model-management surface through a typed fake Transport, including the batch-mode delegation that was the exact failing call chain

A follow-up commit also excludes the master key from HttpTransport's repr: pytest prints fixture values verbatim into failure output, so every failed e2e test was embedding the proxy master key in CI logs and any aggregator they ship to (the 2026-07-06 stage run's Loki export contains it verbatim; rotating the stage master key is recommended). A canary test pins the fixture repr chain

[mubashir1osmani]

@greptileai

[greptile-apps]

Greptile Summary

Fixes two test harness bugs that caused 33+ stage failures: SplitTransport was only routing /model/info to the control plane (leaving /model/new and /model/delete to 404 on the data plane), and Gateway was missing the create_model/delete_model methods that batch and llm_translation tests expected. New regression tests in test_transport.py and test_e2e_gateway.py pin both the routing predicate and the Gateway method surface.

• transport.py: Broadens CONTROL_PLANE_PREFIXES entry from "/model/info" to "/model/" so all model-management paths route to the control plane; /models (the OpenAI list-models route) is unaffected because it does not start with /model/.
• e2e_gateway.py: Adds create_model (with the optional mode the batches suite uses) and delete_model to Gateway; EndpointsClient and BatchClient now delegate to these instead of duplicating the logic.
• models.py: Extracts ModelMode as a named type alias, eliminating the inline Literal duplicate across the codebase.

Confidence Score: 4/5

All changes are confined to the e2e test harness; no production code is touched. The routing fix and Gateway surface additions are correct, well-tested, and narrowly scoped.

The two core fixes (routing prefix and Gateway method) are straightforward and backed by new passing tests. The only minor concern is that _RecordingTransport.post hard-codes a model_id payload for every response type, including NoBody — this relies on Pydantic's default extra-field ignore behaviour and would break if NoBody ever tightens its config. Everything else is clean.

tests/e2e/test_e2e_gateway.py — the fake transport's post method could be more defensive about response types that have no fields.

Important Files Changed

Filename	Overview
tests/e2e/transport.py	Changed CONTROL_PLANE_PREFIXES from "/model/info" to "/model/" so all model-management routes (/model/new, /model/delete, /model/update, /model/info) route to the control plane while /models stays on the data plane. Change is correct and complete.
tests/e2e/e2e_gateway.py	Added create_model and delete_model to Gateway, centralizing model management per harness convention. delete_model issues a warning on failure (appropriate for teardown). create_model correctly threads the optional mode argument that batch tests require.
tests/e2e/llm_translation/endpoints_client.py	Removed the duplicate create_model/delete_model implementations and delegated to Gateway. Removed now-unused imports. Clean refactor with no functional change from the caller's perspective.
tests/e2e/models.py	Extracted ModelMode as a named type alias so the mode literal is DRY across ModelInfoBody and Gateway.create_model's signature. Straightforward improvement.
tests/e2e/test_e2e_gateway.py	New unit tests for Gateway.create_model and delete_model via a typed fake transport (_RecordingTransport). The fake always returns {"model_id": "registered-id"} for every response_type including NoBody — this works today because Pydantic ignores extra fields by default, but is implicitly fragile.
tests/e2e/test_transport.py	New parametrized tests pinning is_control_plane_path for both management routes (must go to control plane) and LLM routes (must stay on data plane). Good regression coverage for the routing predicate.

_{Reviews (1): Last reviewed commit: "fix(e2e): route model management to the ..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR fixes two bugs in the e2e test harness that together caused 33+ test failures in the stage run: CONTROL_PLANE_PREFIXES in transport.py listed only /model/info, so POST /model/new and POST /model/delete fell through to the data-plane gateway (which returns 404 for management routes); and Gateway lacked create_model/delete_model methods that several test clients were already calling, causing AttributeError at fixture setup.

• Routing fix (transport.py): changes the prefix from /model/info to /model/, routing all model-management paths to the control plane while correctly leaving /models (OpenAI list-models, no trailing slash) on the data plane.
• Gateway API fix (e2e_gateway.py): adds create_model (with optional mode for batch deployments) and delete_model; EndpointsClient now delegates to them instead of duplicating transport logic.
• Regression tests (test_transport.py, test_e2e_gateway.py): pin the is_control_plane_path predicate and the Gateway model-management surface through a typed fake Transport, so routing drift or signature changes fail in CI rather than in a live stage run.

Confidence Score: 4/5

All changes are confined to the e2e test harness; no production code is touched. The routing fix is a single-character change with clear before/after evidence from curl proofs.

The two targeted fixes are straightforward and well-evidenced. The new regression tests directly pin the exact failure conditions. The only concern is a fragile assumption in _RecordingTransport.post that works today because Pydantic ignores extra fields on NoBody by default.

test_e2e_gateway.py — the _RecordingTransport.post canned response relies on Pydantic's default extra-field handling for the delete path.

Important Files Changed

Filename	Overview
tests/e2e/transport.py	One-line fix: CONTROL_PLANE_PREFIXES entry changed from "/model/info" to "/model/" so that /model/new and /model/delete are routed to the control plane; /models (OpenAI list-models) correctly falls through to data plane since it doesn't start with "/model/"
tests/e2e/e2e_gateway.py	Adds create_model and delete_model to Gateway, moving the model-management surface out of EndpointsClient; create_model accepts an optional mode parameter to support batch deployments
tests/e2e/llm_translation/endpoints_client.py	create_model and delete_model now delegate to Gateway instead of duplicating the transport logic; unused imports cleaned up accordingly
tests/e2e/models.py	Extracts ModelMode type alias from the inline Literal in ModelInfoBody; purely cosmetic refactor enabling reuse in Gateway.create_model
tests/e2e/test_e2e_gateway.py	New unit tests for Gateway model-management surface using a typed fake Transport; _RecordingTransport.post always validates with {"model_id": "registered-id"} regardless of response_type, which silently succeeds for NoBody (delete path) only because Pydantic ignores extra fields by default
tests/e2e/test_transport.py	New parametrized tests pinning the is_control_plane_path predicate for management routes (/model/new, /model/delete, etc.) vs data-plane routes (/models, /chat/completions, etc.)

_{Reviews (2): Last reviewed commit: "fix(e2e): route model management to the ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!