test(e2e): add models_mgmt suite covering add/update/delete persistence and route permissions

[mubashir1osmani]

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Stacked on #32261 (needs Gateway.create_model and the /model/ control-plane routing); merge that first

Screenshots / Proof of Fix

Run live against the split stage deployment (kubectl port-forward litellm-gateway:4000 and litellm-backend:4001, master key from the cluster secret via env):

$ LITELLM_PROXY_URL=http://localhost:14000 LITELLM_CONTROL_PLANE_URL=http://localhost:14001 \
    uv run pytest tests/e2e/models_mgmt/ -v
tests/e2e/models_mgmt/test_models_mgmt_e2e.py::TestModelManagementRoutes::test_add_persists_to_model_info_and_serves_chat PASSED [ 25%]
tests/e2e/models_mgmt/test_models_mgmt_e2e.py::TestModelManagementRoutes::test_update_tpm_persists_and_deployment_still_serves PASSED [ 50%]
tests/e2e/models_mgmt/test_models_mgmt_e2e.py::TestModelManagementRoutes::test_delete_removes_from_model_info_and_rejects_chat PASSED [ 75%]
tests/e2e/models_mgmt/test_models_mgmt_e2e.py::TestModelManagementRoutes::test_add_forbidden_for_llm_only_key PASSED [100%]
======================== 4 passed in 123.69s (0:02:03) =========================

Post-run /model/info leak check confirmed zero deployments named e2e-mgmt* or e2e-probe* remained

Type

Test

Changes

tests/e2e/CLAUDE.md documents a models_mgmt suite for the model-management routes, but the directory never existed; the mgmt.* registry rows had no live coverage and the routes were only exercised as fixture plumbing by other suites. This adds the suite with four tests covering the add/update/delete lifecycle contract and the route-permission contract: add persists to /model/info and the deployment actually serves chat through the data plane; updating tpm persists without the merge clobbering the backend model or key; delete removes the deployment from /model/info and traffic is rejected with a 400 naming the model; and a key restricted to llm_api_routes gets exactly 403 with the route-permission marker from /model/new, with the model verifiably not created

Two live contracts the tests encode: control-plane writes reach the data plane on its periodic DB sync (roughly 30s), so traffic-facing assertions poll to a deadline; and a deleted model is rejected with the router's no-healthy-deployments 400 rather than the unknown-model shape, so the client accepts either form as long as it names the model

models.py gains the typed pieces the suite needs: tpm on LiteLLMParamsBody, a DeploymentParams model for /model/info litellm_params (superset of the pricing fields the custom-pricing tests read), and ModelUpdateParams/ModelUpdateBody for /model/update

[mubashir1osmani]

@greptileai

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR adds a new models_mgmt e2e suite covering the add/update/delete lifecycle for model deployments, validates management writes propagate to the data plane, and enforces the route-permission contract for restricted API keys. The PR also promotes create_model/delete_model into Gateway as shared helpers and widens CONTROL_PLANE_PREFIXES from /model/info to all of /model/*.

• transport.py: routing change to "/model/" correctly covers all management sub-paths without matching the OpenAI-compatible /models endpoint.
• models.py: replaces CustomPricing with the new DeploymentParams type for ModelInfoEntry.litellm_params, adding model and tpm while preserving pricing fields; cache_read_input_token_cost and cache_creation_input_token_cost are silently dropped via extra="ignore".
• test_models_mgmt_e2e.py: four live tests with polling for data-plane propagation; test_e2e_gateway.py pins the Gateway surface via a typed recording transport.

Confidence Score: 4/5

Safe to merge — all changes are confined to the e2e test tree with no production code modifications.

The core logic is sound and backed by parametrized unit tests plus a live stage run. Two items warrant attention: the immediate (non-polled) tpm assertion after create assumes the control-plane /model/info has no caching layer, and the deliberate omission of cache token-cost fields from DeploymentParams could cause a hard-to-diagnose miss if a future test reads them through litellm_params. Neither is a blocker.

tests/e2e/models_mgmt/test_models_mgmt_e2e.py (immediate tpm assertion), tests/e2e/models.py (DeploymentParams field coverage)

Important Files Changed

Filename	Overview
tests/e2e/transport.py	Replaces the narrow /model/info control-plane prefix with /model/ so all management endpoints (new/delete/update/info) are routed to the control plane in split deployments; correct and well-tested.
tests/e2e/models.py	Adds DeploymentParams as the new type for ModelInfoEntry.litellm_params; safe for existing callers but silently drops cache token-cost fields.
tests/e2e/e2e_gateway.py	Promotes create_model/delete_model into Gateway as canonical shared helpers; structurally clean.
tests/e2e/models_mgmt/test_models_mgmt_e2e.py	New suite covering add/update/delete lifecycle; tpm assertion after create is immediate (no poll), relying on control-plane synchrony.
tests/e2e/models_mgmt/models_mgmt_client.py	Well-structured client with strict delete, update_model_tpm, and raw-status helpers.
tests/e2e/test_e2e_gateway.py	Unit regression tests for Gateway.create_model/delete_model via a typed recording transport.
tests/e2e/test_transport.py	Comprehensive parametrized tests for is_control_plane_path covering all management and LLM routes.
tests/e2e/llm_translation/endpoints_client.py	Delegates create_model/delete_model to Gateway helpers, eliminating duplicate implementation.
tests/e2e/models_mgmt/conftest.py	Minimal conftest registering the covers marker and session-scoped client fixture.

_{Reviews (1): Last reviewed commit: "test(e2e): add models_mgmt suite coverin..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR adds the models_mgmt e2e suite, covering the full add/update/delete lifecycle for model deployments and the route-permission contract, along with supporting infrastructure (centralised Gateway.create_model/delete_model, typed DeploymentParams/ModelUpdateBody models, and transport routing broadened to cover all /model/* paths).

• New models_mgmt suite – four live tests: add persists to /model/info and serves chat; update-tpm persists without clobbering the backend model; delete removes from /model/info and traffic is rejected with a 400 naming the model; and a key restricted to llm_api_routes gets exactly 403 from /model/new.
• Centralisation of model-management calls – Gateway.create_model/delete_model are moved from endpoints_client.py into e2e_gateway.py, with BatchClient and EndpointsClient delegating to them; unit tests in test_e2e_gateway.py pin the surface against rename/signature drift.
• Transport and model-type housekeeping – CONTROL_PLANE_PREFIXES is broadened from /model/info to /model/ so all management routes are correctly dispatched in a split deployment; ModelInfoEntry.litellm_params moves from CustomPricing to the new DeploymentParams type.

Confidence Score: 4/5

Safe to merge; all changes are confined to the test harness and do not touch production code.

The new DeploymentParams type drops cache_read_input_token_cost and cache_creation_input_token_cost that were present in the CustomPricing it replaces for ModelInfoEntry.litellm_params. No existing test currently reads those fields from litellm_params, so nothing breaks today, but any future cache-pricing assertion will silently receive None instead of the actual rate. All other changes — the new suite, the gateway centralisation, and the transport routing fix — are straightforward and well-covered by the new unit tests.

tests/e2e/models.py — the DeploymentParams type warrants a second look to confirm the dropped cache-pricing fields are intentional.

Important Files Changed

Filename	Overview
tests/e2e/models.py	Introduces DeploymentParams for ModelInfoEntry.litellm_params; drops cache_read_input_token_cost and cache_creation_input_token_cost that were present in the old CustomPricing type, and adds ModelMode, ModelUpdateParams, and ModelUpdateBody.
tests/e2e/transport.py	Broadens /model/info control-plane prefix to /model/ so all model management routes route to the control plane; /models (OpenAI list-models route) is unaffected.
tests/e2e/e2e_gateway.py	Centralises create_model and delete_model into Gateway; delete_model warns on failure (teardown-safe).
tests/e2e/models_mgmt/models_mgmt_client.py	New client for the models-mgmt suite; delete_model uses unwrap (hard-fail during act phase); create_model_status uses send to surface raw HTTP status for permission tests.
tests/e2e/models_mgmt/test_models_mgmt_e2e.py	Four live e2e tests covering add/update/delete lifecycle and route-permission contract; polling pattern is correct; teardown uses warn-only delete.
tests/e2e/test_e2e_gateway.py	Unit tests for Gateway.create_model/delete_model using a typed fake transport; pins the surface so signature drift is caught before a live run.
tests/e2e/test_transport.py	New unit tests for is_control_plane_path; verifies management routes route to control plane and LLM routes stay on data plane.
tests/e2e/models_mgmt/conftest.py	Minimal conftest: session-scoped client fixture and covers marker registration; lifecycle fixtures come from the parent conftest.
tests/e2e/llm_translation/endpoints_client.py	Delegates create_model/delete_model to the newly centralised Gateway methods, removing duplicated raw-transport calls.

Comments Outside Diff (2)

1. tests/e2e/models.py, line 301-311 (link)

   DeploymentParams silently drops cache_read_input_token_cost and cache_creation_input_token_cost that existed in CustomPricing. With extra="ignore", any cache-pricing values the proxy returns in litellm_params are silently swallowed, making it impossible for a test to assert on them. The PR description calls DeploymentParams a "superset of the pricing fields the custom-pricing tests read", but it's actually a strict subset of CustomPricing on those two fields.

2. tests/e2e/models_mgmt/models_mgmt_client.py, line 25 (link)

   Unused import: ModelNewBody

   ModelNewBody is imported here but constructed inline inside create_model_status. Static analysers may flag this depending on how they trace usage through inline construction — worth a double-check to confirm it is load-bearing.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "test(e2e): add models_mgmt suite coverin..." | Re-trigger Greptile}

[mubashir1osmani]

Folded into the management suite instead of a separate models_mgmt folder, per review direction; the four tests, the ManagementClient model methods, and the models.py types now land via the fold PR referenced above, re-verified live on stage as part of the full 12-test management run