fix(router): use forwarded model_id for native Azure container IDs - #27921
Merged
mateo-berri merged 11 commits intoMay 20, 2026
Conversation
… _init_containers_api_endpoints Azure code-interpreter containers return provider-native IDs (cntr_ + hex) that carry no LiteLLM routing payload, so _decode_container_id returns model_id=None. The router was falling through to call the handler directly, bypassing _ageneric_api_call_with_fallbacks and leaving api_base=None for Azure deployments. Fall back to the model_id forwarded from the proxy ownership check so deployment credentials are always applied. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Greptile SummaryThis PR fixes Azure container routing failures when provider-native IDs (
Confidence Score: 5/5Safe to merge — the change is well-scoped, all production call paths have been updated to await the async conversion, and the security boundary is pinned by a regression test. The ownership cache is warmed by assert_user_can_access_container before get_container_forwarding_params in every handler path, so the new DB fallback is only reachable on cold start. The params or None fix is mechanical and documented by a self-contained test. No existing tests were weakened. No files require special attention. ownership.py is the most complex change but caching is correct and all code paths are covered by new tests.
|
| Filename | Overview |
|---|---|
| litellm/proxy/container_endpoints/ownership.py | Converts get_container_forwarding_params to async, adds _get_stored_container_id for recovering model_id from the DB-stored encoded ID, and warms _CONTAINER_STORED_ID_CACHE in both record_container_owner and _get_container_owner. |
| litellm/router.py | Adds _forwarded_model_id fallback when decoded model_id is None; logic is correct and narrow in scope. |
| litellm/llms/azure/containers/transformation.py | Adds _normalize_api_base and _extract_api_version; logic is clean and well-tested. |
| litellm/llms/custom_httpx/container_handler.py | Applies params or None guard in both sync and async HTTP dispatch paths. |
| litellm/llms/custom_httpx/llm_http_handler.py | Same params or None guard applied across nine container call sites. |
| litellm/proxy/container_endpoints/handler_factory.py | All call sites of get_container_forwarding_params updated to await the now-async function. |
| litellm/proxy/container_endpoints/endpoints.py | Two call sites updated to await get_container_forwarding_params. |
| tests/test_litellm/containers/test_azure_container_transformation.py | 251 lines of new mock-only tests covering all fixed scenarios including the security boundary regression. |
Reviews (8): Last reviewed commit: "Merge branch 'litellm_internal_staging' ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…AzureContainerConfig.get_complete_url When a deployment's api_base is the responses endpoint URL (e.g. .../openai/responses?api-version=...), AzureContainerConfig was appending /openai/containers on top of it, producing the broken path .../openai/responses/openai/containers. Azure returns 404 for that URL while the correct path is .../openai/containers. Strip any /openai/responses suffix from api_base before constructing the containers URL so the resource root is always used as the starting point. Co-authored-by: Cursor <cursoragent@cursor.com>
…oyment's api_version The deployment's api_version (e.g. 2024-08-01-preview) targets the chat/responses API and is too old for the containers API, which requires 2025-04-01-preview. The responses endpoint api_base already carries the correct api-version in its query string. Extract it and use it for the containers URL, overriding the stale deployment-level version. Fixes DELETE and file-upload operations returning 404 due to wrong api-version. Co-authored-by: Cursor <cursoragent@cursor.com>
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
Contributor
Azure container routing and URL handling updateThis PR adjusts Azure container URL construction, preserves query strings on container HTTP calls, and routes native Azure container IDs using the forwarded model identifier. I checked the proxy container ownership gates, router call path, and provider request construction and did not find a new externally reachable security issue in the changed lines. Status: 0 open |
…eserve api-version
httpx erases a URL's query-string when params={} (empty dict) is passed,
silently stripping ?api-version=2025-04-01-preview from every container
POST/DELETE request. Azure's GET endpoints tolerate a missing api-version;
POST (upload) and DELETE are strict, so those returned 404.
Fix: use `params or None` in container_handler._async_handle and
llm_http_handler.async_container_delete_handler (and all sibling container
handlers) so that an empty params dict falls back to None, leaving httpx to
preserve the URL's existing query string intact.
Adds a regression test that directly documents the httpx behaviour.
Co-authored-by: Cursor <cursoragent@cursor.com>
…dpoints Two reviewer findings addressed: 1. Truncated comment on the model_id fallback line — now complete. 2. Security: the elif branch that fired when container_id was absent allowed any authenticated caller to supply model_id in a POST /v1/containers body and route the request through an arbitrary deployment UUID, bypassing the model-level access checks that only validate `model`. Removed the elif branch; operations without container_id (create, list) route by the caller-supplied `model` field as before. model_id forwarding is kept only inside the container_id block, where the proxy ownership check has already validated the container before forwarding the deployment ID. Adds a regression test pinning the security boundary: no-container-id path calls original_function directly even when model_id is in kwargs. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
@greptile re review |
…naged IDs Add test_regression_get_container_forwarding_params_sets_model_id_for_managed_id to verify that get_container_forwarding_params (the proxy-side half of the Azure routing fix) correctly extracts and forwards model_id from a LiteLLM-managed encoded container ID. This closes the gap identified by Greptile P1: the previous regression test only injected model_id as a direct kwarg, validating the router in isolation. The new test exercises the actual proxy-to-router data flow through ownership.get_container_forwarding_params, confirming that kwargs["model_id"] is populated before _init_containers_api_endpoints is reached. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
@greptile re review |
mateo-berri
requested changes
May 15, 2026
mateo-berri
left a comment
mateo-berri
left a comment
Contributor
There was a problem hiding this comment.
#27921 (comment) isn't this still a valid concern?
Contributor
Author
That was already fixed @mateo-berri . Sorry forgot to resolve |
Use path.endswith() instead of path.find() for _AZURE_ENDPOINT_PATHS so the suffix strip only fires when api_base actually ends with one of the endpoint-specific path suffixes. This is the more precise check greptile flagged on the original find()-based implementation.
|
|
Contributor
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high mode and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Sync container handler missing empty params guard fix
- Applied the same
effective_params = query_params or Noneguard in_sync_handleand used it in GET/DELETE/POST calls, mirroring the async fix so the URL's?api-version=...is not stripped by httpx.
- Applied the same
Preview (52a8b167cc)
diff --git a/litellm/llms/azure/containers/transformation.py b/litellm/llms/azure/containers/transformation.py
--- a/litellm/llms/azure/containers/transformation.py
+++ b/litellm/llms/azure/containers/transformation.py
@@ -1,10 +1,17 @@
from typing import Optional
+from urllib.parse import parse_qs, urlparse, urlunparse
from litellm.llms.azure.common_utils import BaseAzureLLM
from litellm.llms.openai.containers.transformation import OpenAIContainerConfig
from litellm.types.router import GenericLiteLLMParams
+# Endpoint-specific path suffixes that may appear in a deployment's api_base
+# (e.g. the responses endpoint URL is stored as api_base for Azure models).
+# Strip these before building the containers URL so we always start from the
+# resource root (https://resource.cognitiveservices.azure.com).
+_AZURE_ENDPOINT_PATHS = ("/openai/responses",)
+
class AzureContainerConfig(OpenAIContainerConfig):
"""
Configuration class for Azure OpenAI container API.
@@ -27,6 +34,27 @@
litellm_params=GenericLiteLLMParams(api_key=api_key),
)
+ @staticmethod
+ def _normalize_api_base(api_base: Optional[str]) -> Optional[str]:
+ """Strip endpoint-specific path suffixes from api_base to get the resource root."""
+ if not api_base:
+ return api_base
+ parsed = urlparse(api_base)
+ path = parsed.path
+ for ep in _AZURE_ENDPOINT_PATHS:
+ if path.endswith(ep):
+ return urlunparse(
+ (parsed.scheme, parsed.netloc, path[: -len(ep)], "", "", "")
+ )
+ return api_base
+
+ @staticmethod
+ def _extract_api_version(api_base: Optional[str]) -> Optional[str]:
+ """Return the api-version query param from api_base if present."""
+ if not api_base:
+ return None
+ return parse_qs(urlparse(api_base).query).get("api-version", [None])[0]
+
def get_complete_url(
self,
api_base: Optional[str],
@@ -39,10 +67,19 @@
{endpoint}/openai/v1/containers
when api_version is 'v1', 'latest', or 'preview'; otherwise:
{endpoint}/openai/containers
+
+ The deployment's api_base may be the responses endpoint URL
+ (e.g. .../openai/responses?api-version=2025-04-01-preview). We
+ prefer the api-version embedded there over the deployment's
+ api_version field, which may point to an older chat API version.
"""
+ effective_params = dict(litellm_params)
+ api_version_from_base = self._extract_api_version(api_base)
+ if api_version_from_base:
+ effective_params["api_version"] = api_version_from_base
return BaseAzureLLM._get_base_azure_url(
- api_base=api_base,
- litellm_params=litellm_params,
+ api_base=self._normalize_api_base(api_base),
+ litellm_params=effective_params,
route="/openai/containers",
default_api_version="v1",
)
diff --git a/litellm/llms/custom_httpx/container_handler.py b/litellm/llms/custom_httpx/container_handler.py
--- a/litellm/llms/custom_httpx/container_handler.py
+++ b/litellm/llms/custom_httpx/container_handler.py
@@ -257,14 +257,19 @@
returns_binary = endpoint_config.get("returns_binary", False)
is_multipart = endpoint_config.get("is_multipart", False)
+ # An empty dict passed as `params` to httpx strips any existing query
+ # string from the URL (e.g. ?api-version=...). Use None instead so
+ # httpx leaves the URL's own query string intact.
+ effective_params = query_params or None
+
try:
if method == "GET":
response = http_client.get(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
elif method == "DELETE":
response = http_client.delete(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
elif method == "POST":
if is_multipart and "file" in kwargs:
@@ -272,11 +277,11 @@
kwargs["file"], headers
)
response = http_client.post(
- url=url, headers=headers, params=query_params, files=files
+ url=url, headers=headers, params=effective_params, files=files
)
else:
response = http_client.post(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
else:
raise ValueError(f"Unsupported HTTP method: {method}")
@@ -376,14 +381,19 @@
returns_binary = endpoint_config.get("returns_binary", False)
is_multipart = endpoint_config.get("is_multipart", False)
+ # An empty dict passed as `params` to httpx strips any existing query
+ # string from the URL (e.g. ?api-version=...). Use None instead so
+ # httpx leaves the URL's own query string intact.
+ effective_params = query_params or None
+
try:
if method == "GET":
response = await http_client.get(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
elif method == "DELETE":
response = await http_client.delete(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
elif method == "POST":
if is_multipart and "file" in kwargs:
@@ -391,11 +401,11 @@
kwargs["file"], headers
)
response = await http_client.post(
- url=url, headers=headers, params=query_params, files=files
+ url=url, headers=headers, params=effective_params, files=files
)
else:
response = await http_client.post(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
else:
raise ValueError(f"Unsupported HTTP method: {method}")
diff --git a/litellm/llms/custom_httpx/llm_http_handler.py b/litellm/llms/custom_httpx/llm_http_handler.py
--- a/litellm/llms/custom_httpx/llm_http_handler.py
+++ b/litellm/llms/custom_httpx/llm_http_handler.py
@@ -7815,7 +7815,7 @@
response = sync_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_list_response(
@@ -7892,7 +7892,7 @@
response = await async_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_list_response(
@@ -7982,7 +7982,7 @@
response = sync_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_retrieve_response(
@@ -8059,7 +8059,7 @@
response = await async_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_retrieve_response(
@@ -8149,7 +8149,7 @@
response = sync_httpx_client.delete(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_delete_response(
@@ -8226,7 +8226,7 @@
response = await async_httpx_client.delete(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_delete_response(
@@ -8322,7 +8322,7 @@
response = sync_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_file_list_response(
@@ -8401,7 +8401,7 @@
response = await async_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_file_list_response(
@@ -8489,7 +8489,7 @@
response = sync_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_file_content_response(
@@ -8565,7 +8565,7 @@
response = await async_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_file_content_response(
diff --git a/litellm/router.py b/litellm/router.py
--- a/litellm/router.py
+++ b/litellm/router.py
@@ -5551,6 +5551,7 @@
from litellm.responses.utils import ResponsesAPIRequestUtils
container_id = kwargs.get("container_id")
+ _forwarded_model_id = kwargs.get("model_id")
if isinstance(container_id, str):
decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
original_id = decoded.get("response_id", container_id)
@@ -5559,7 +5560,14 @@
decoded_provider = decoded.get("custom_llm_provider")
if decoded_provider and kwargs.get("custom_llm_provider") == "openai":
kwargs["custom_llm_provider"] = decoded_provider
- model_id = decoded.get("model_id")
+ # Fall back to the model_id forwarded by the proxy when the container_id
+ # is a native upstream ID (e.g. Azure hex cntr_) that carries no LiteLLM
+ # routing payload, so deployment credentials (api_base, api_key) are applied.
+ model_id = decoded.get("model_id") or (
+ _forwarded_model_id.strip()
+ if isinstance(_forwarded_model_id, str) and _forwarded_model_id.strip()
+ else None
+ )
if model_id:
kwargs["model"] = model_id
return await self._ageneric_api_call_with_fallbacks(
diff --git a/tests/test_litellm/containers/test_azure_container_transformation.py b/tests/test_litellm/containers/test_azure_container_transformation.py
--- a/tests/test_litellm/containers/test_azure_container_transformation.py
+++ b/tests/test_litellm/containers/test_azure_container_transformation.py
@@ -109,6 +109,31 @@
assert "/openai/v1/containers" in url
+ def test_get_complete_url_strips_responses_path_and_preserves_api_version(self):
+ """When api_base is the responses endpoint URL, get_complete_url must:
+ - strip /openai/responses (no double-path)
+ - use the api-version from api_base query string, NOT the deployment's
+ older api_version (e.g. 2024-08-01-preview → containers need 2025-04-01-preview)
+ """
+ api_base = "https://my-resource.cognitiveservices.azure.com/openai/responses?api-version=2025-04-01-preview"
+
+ url = self.config.get_complete_url(
+ api_base=api_base,
+ litellm_params={"api_version": "2024-08-01-preview"},
+ )
+
+ assert (
+ "/openai/responses/openai/containers" not in url
+ ), "path must not double /openai/responses"
+ assert "my-resource.cognitiveservices.azure.com" in url
+ assert "/openai/containers" in url or "/openai/v1/containers" in url
+ assert (
+ "2025-04-01-preview" in url
+ ), "must use version from api_base, not litellm_params"
+ assert (
+ "2024-08-01-preview" not in url
+ ), "must not fall back to older chat api_version"
+
def test_get_complete_url_raises_without_api_base(self, monkeypatch):
monkeypatch.delenv("AZURE_API_BASE", raising=False)
monkeypatch.setattr(litellm, "api_base", None)
@@ -531,6 +556,92 @@
assert qs.get("api-version") == ["v1"]
assert qs.get("foo") == ["bar"]
+ @pytest.mark.asyncio
+ async def test_regression_no_container_id_does_not_use_user_supplied_model_id(
+ self, monkeypatch
+ ):
+ """Operations without container_id (create, list) must NOT route via
+ _ageneric_api_call_with_fallbacks using a caller-supplied model_id.
+
+ Security boundary: only the path that holds a validated container_id
+ is trusted to fall back to the forwarded model_id. A caller setting
+ model_id without container_id on POST /v1/containers must not gain
+ access to an arbitrary deployment UUID.
+ """
+ from litellm.router import Router
+
+ router = Router(
+ model_list=[
+ {
+ "model_name": "azure-model",
+ "litellm_params": {
+ "model": "azure/gpt-4",
+ "api_base": "https://my-resource.cognitiveservices.azure.com",
+ "api_key": "test-key",
+ "api_version": "2025-04-01-preview",
+ },
+ "model_info": {"id": "deployment-uuid-123"},
+ }
+ ]
+ )
+
+ fallback_called = {"called": False}
+
+ async def _mock_fallback(original_function, **kwargs):
+ fallback_called["called"] = True
+ return {}
+
+ monkeypatch.setattr(router, "_ageneric_api_call_with_fallbacks", _mock_fallback)
+
+ original_called = {"called": False}
+
+ async def _noop(**kwargs):
+ original_called["called"] = True
+ return {}
+
+ # No container_id — simulates create/list; caller injects a model_id
+ await router._init_containers_api_endpoints(
+ original_function=_noop,
+ model_id="deployment-uuid-123",
+ custom_llm_provider="azure",
+ )
+
+ assert not fallback_called["called"], (
+ "_ageneric_api_call_with_fallbacks must NOT be called when "
+ "container_id is absent, even if model_id is supplied"
+ )
+ assert original_called["called"], "original_function must be called directly"
+
+ def test_regression_httpx_empty_params_strips_query_string(self):
+ """httpx erases the URL query-string when params={} (empty dict) is passed.
+
+ Root cause of the Azure container 404s on POST/DELETE:
+ _build_query_params returns {} when the endpoint has no extra params;
+ passing that {} as params= to httpx wiped ?api-version=2025-04-01-preview.
+
+ Fix: every container httpx call now uses `params or None` so an empty
+ dict falls back to None, which tells httpx to leave the URL untouched.
+ """
+ url = (
+ "https://resource.cognitiveservices.azure.com"
+ "/openai/containers/cntr_123?api-version=2025-04-01-preview"
+ )
+ client = httpx.AsyncClient()
+
+ req_none = client.build_request("DELETE", url, params=None)
+ assert "api-version=2025-04-01-preview" in str(req_none.url)
+
+ req_empty = client.build_request("DELETE", url, params={})
+ assert "api-version" not in str(
+ req_empty.url
+ ), "Documents root cause: params={} strips the query string"
+
+ effective: dict = {}
+ req_guarded = client.build_request("DELETE", url, params=effective or None)
+ assert "api-version=2025-04-01-preview" in str(
+ req_guarded.url
+ ), "`params or None` must preserve ?api-version"
+
def test_regression_proxy_resolves_azure_text_same_as_azure(self):
"""Router/proxy treat azure_text like azure for container config."""
from litellm.proxy.container_endpoints.handler_factory import (
@@ -770,3 +881,86 @@
assert captured["data"]["container_id"] == "cntr_123"
assert captured["data"]["custom_llm_provider"] == "azure"
assert captured["data"]["model_id"] == "model_abc123"
+
+ def test_regression_get_container_forwarding_params_sets_model_id_for_managed_id(
+ self,
+ ):
+ """get_container_forwarding_params must extract model_id from a
+ LiteLLM-managed encoded container ID and include it in the forwarding
+ dict. This is the proxy-side half of the native-Azure-ID routing fix:
+ the router's _init_containers_api_endpoints reads kwargs["model_id"]
+ which is set here.
+ """
+ from litellm.proxy.container_endpoints.ownership import (
+ get_container_forwarding_params,
+ )
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="deployment-uuid-123",
+ container_id="cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df",
+ )
+
+ params = get_container_forwarding_params(
+ container_id=encoded_id,
+ original_container_id="cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df",
+ custom_llm_provider="azure",
+ )
+
+ assert params.get("model_id") == "deployment-uuid-123", (
+ "model_id must be forwarded to the router for managed container IDs"
+ )
+ assert params.get("container_id") == (
+ "cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df"
+ )
+ assert params.get("custom_llm_provider") == "azure"
+
+ @pytest.mark.asyncio
+ async def test_regression_native_azure_container_id_uses_forwarded_model_id(
+ self, monkeypatch
+ ):
+ """Native Azure container IDs (cntr_ + hex, no LiteLLM payload) must
+ still route through _ageneric_api_call_with_fallbacks using the
+ model_id forwarded from the proxy ownership check so that deployment
+ credentials (api_base) are applied."""
+ from litellm.router import Router
+
+ router = Router(
+ model_list=[
+ {
+ "model_name": "azure-model",
+ "litellm_params": {
+ "model": "azure/gpt-4",
+ "api_base": "https://my-resource.cognitiveservices.azure.com",
+ "api_key": "test-key",
+ "api_version": "2025-04-01-preview",
+ },
+ "model_info": {"id": "deployment-uuid-123"},
+ }
+ ]
+ )
+
+ called_with: dict = {}
+
+ async def _mock_fallback(original_function, **kwargs):
+ called_with.update(kwargs)
+ return {}
+
+ monkeypatch.setattr(router, "_ageneric_api_call_with_fallbacks", _mock_fallback)
+
+ native_azure_id = "cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df"
+
+ async def _noop(**kwargs):
+ return {}
+
+ await router._init_containers_api_endpoints(
+ original_function=_noop,
+ container_id=native_azure_id,
+ model_id="deployment-uuid-123",
+ custom_llm_provider="azure",
+ )
+
+ assert called_with.get("model") == "deployment-uuid-123", (
+ "_ageneric_api_call_with_fallbacks must be called with the forwarded "
+ "model_id when the container_id carries no LiteLLM routing payload"
+ )You can send follow-ups to the cloud agent here.
Mirror the async path fix: pass None instead of an empty params dict so httpx does not strip the URL's existing query string (e.g. ?api-version=...), which is required for Azure container routing. Co-authored-by: Yassin Kortam <yassin@berri.ai>
Contributor
Co-authored-by: Yassin Kortam <yassin@berri.ai>
Contributor
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high mode and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Forwarded model_id is never set for native Azure IDs
- Made get_container_forwarding_params async and fall back to decoding the stored unified_object_id on the ownership row when the user-supplied native cntr_ id carries no model_id, so the router-side fallback in _init_containers_api_endpoints is actually exercised in production.
Preview (386bf74790)
diff --git a/litellm/llms/azure/containers/transformation.py b/litellm/llms/azure/containers/transformation.py
--- a/litellm/llms/azure/containers/transformation.py
+++ b/litellm/llms/azure/containers/transformation.py
@@ -1,10 +1,17 @@
from typing import Optional
+from urllib.parse import parse_qs, urlparse, urlunparse
from litellm.llms.azure.common_utils import BaseAzureLLM
from litellm.llms.openai.containers.transformation import OpenAIContainerConfig
from litellm.types.router import GenericLiteLLMParams
+# Endpoint-specific path suffixes that may appear in a deployment's api_base
+# (e.g. the responses endpoint URL is stored as api_base for Azure models).
+# Strip these before building the containers URL so we always start from the
+# resource root (https://resource.cognitiveservices.azure.com).
+_AZURE_ENDPOINT_PATHS = ("/openai/responses",)
+
class AzureContainerConfig(OpenAIContainerConfig):
"""
Configuration class for Azure OpenAI container API.
@@ -27,6 +34,27 @@
litellm_params=GenericLiteLLMParams(api_key=api_key),
)
+ @staticmethod
+ def _normalize_api_base(api_base: Optional[str]) -> Optional[str]:
+ """Strip endpoint-specific path suffixes from api_base to get the resource root."""
+ if not api_base:
+ return api_base
+ parsed = urlparse(api_base)
+ path = parsed.path.rstrip("/")
+ for ep in _AZURE_ENDPOINT_PATHS:
+ if path.endswith(ep):
+ return urlunparse(
+ (parsed.scheme, parsed.netloc, path[: -len(ep)], "", "", "")
+ )
+ return api_base
+
+ @staticmethod
+ def _extract_api_version(api_base: Optional[str]) -> Optional[str]:
+ """Return the api-version query param from api_base if present."""
+ if not api_base:
+ return None
+ return parse_qs(urlparse(api_base).query).get("api-version", [None])[0]
+
def get_complete_url(
self,
api_base: Optional[str],
@@ -39,10 +67,19 @@
{endpoint}/openai/v1/containers
when api_version is 'v1', 'latest', or 'preview'; otherwise:
{endpoint}/openai/containers
+
+ The deployment's api_base may be the responses endpoint URL
+ (e.g. .../openai/responses?api-version=2025-04-01-preview). We
+ prefer the api-version embedded there over the deployment's
+ api_version field, which may point to an older chat API version.
"""
+ effective_params = dict(litellm_params)
+ api_version_from_base = self._extract_api_version(api_base)
+ if api_version_from_base:
+ effective_params["api_version"] = api_version_from_base
return BaseAzureLLM._get_base_azure_url(
- api_base=api_base,
- litellm_params=litellm_params,
+ api_base=self._normalize_api_base(api_base),
+ litellm_params=effective_params,
route="/openai/containers",
default_api_version="v1",
)
diff --git a/litellm/llms/custom_httpx/container_handler.py b/litellm/llms/custom_httpx/container_handler.py
--- a/litellm/llms/custom_httpx/container_handler.py
+++ b/litellm/llms/custom_httpx/container_handler.py
@@ -257,14 +257,19 @@
returns_binary = endpoint_config.get("returns_binary", False)
is_multipart = endpoint_config.get("is_multipart", False)
+ # An empty dict passed as `params` to httpx strips any existing query
+ # string from the URL (e.g. ?api-version=...). Use None instead so
+ # httpx leaves the URL's own query string intact.
+ effective_params = query_params or None
+
try:
if method == "GET":
response = http_client.get(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
elif method == "DELETE":
response = http_client.delete(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
elif method == "POST":
if is_multipart and "file" in kwargs:
@@ -272,11 +277,11 @@
kwargs["file"], headers
)
response = http_client.post(
- url=url, headers=headers, params=query_params, files=files
+ url=url, headers=headers, params=effective_params, files=files
)
else:
response = http_client.post(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
else:
raise ValueError(f"Unsupported HTTP method: {method}")
@@ -376,14 +381,19 @@
returns_binary = endpoint_config.get("returns_binary", False)
is_multipart = endpoint_config.get("is_multipart", False)
+ # An empty dict passed as `params` to httpx strips any existing query
+ # string from the URL (e.g. ?api-version=...). Use None instead so
+ # httpx leaves the URL's own query string intact.
+ effective_params = query_params or None
+
try:
if method == "GET":
response = await http_client.get(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
elif method == "DELETE":
response = await http_client.delete(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
elif method == "POST":
if is_multipart and "file" in kwargs:
@@ -391,11 +401,11 @@
kwargs["file"], headers
)
response = await http_client.post(
- url=url, headers=headers, params=query_params, files=files
+ url=url, headers=headers, params=effective_params, files=files
)
else:
response = await http_client.post(
- url=url, headers=headers, params=query_params
+ url=url, headers=headers, params=effective_params
)
else:
raise ValueError(f"Unsupported HTTP method: {method}")
diff --git a/litellm/llms/custom_httpx/llm_http_handler.py b/litellm/llms/custom_httpx/llm_http_handler.py
--- a/litellm/llms/custom_httpx/llm_http_handler.py
+++ b/litellm/llms/custom_httpx/llm_http_handler.py
@@ -7815,7 +7815,7 @@
response = sync_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_list_response(
@@ -7892,7 +7892,7 @@
response = await async_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_list_response(
@@ -7982,7 +7982,7 @@
response = sync_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_retrieve_response(
@@ -8059,7 +8059,7 @@
response = await async_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_retrieve_response(
@@ -8149,7 +8149,7 @@
response = sync_httpx_client.delete(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_delete_response(
@@ -8226,7 +8226,7 @@
response = await async_httpx_client.delete(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_delete_response(
@@ -8322,7 +8322,7 @@
response = sync_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_file_list_response(
@@ -8401,7 +8401,7 @@
response = await async_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_file_list_response(
@@ -8489,7 +8489,7 @@
response = sync_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_file_content_response(
@@ -8565,7 +8565,7 @@
response = await async_httpx_client.get(
url=url,
headers=headers,
- params=params,
+ params=params or None,
)
return container_provider_config.transform_container_file_content_response(
diff --git a/litellm/proxy/container_endpoints/endpoints.py b/litellm/proxy/container_endpoints/endpoints.py
--- a/litellm/proxy/container_endpoints/endpoints.py
+++ b/litellm/proxy/container_endpoints/endpoints.py
@@ -328,7 +328,7 @@
custom_llm_provider=custom_llm_provider,
)
data.update(
- get_container_forwarding_params(
+ await get_container_forwarding_params(
container_id,
original_container_id,
custom_llm_provider,
@@ -433,7 +433,7 @@
custom_llm_provider=custom_llm_provider,
)
data.update(
- get_container_forwarding_params(
+ await get_container_forwarding_params(
container_id,
original_container_id,
custom_llm_provider,
diff --git a/litellm/proxy/container_endpoints/handler_factory.py b/litellm/proxy/container_endpoints/handler_factory.py
--- a/litellm/proxy/container_endpoints/handler_factory.py
+++ b/litellm/proxy/container_endpoints/handler_factory.py
@@ -196,10 +196,12 @@
)
data: Dict[str, Any] = {
"file_id": file_id,
- **get_container_forwarding_params(
- container_id=container_id,
- original_container_id=original_container_id,
- custom_llm_provider=resolved_provider,
+ **(
+ await get_container_forwarding_params(
+ container_id=container_id,
+ original_container_id=original_container_id,
+ custom_llm_provider=resolved_provider,
+ )
),
}
processor = ProxyBaseLLMRequestProcessing(data=data)
@@ -316,7 +318,7 @@
)
data.update(
- get_container_forwarding_params(
+ await get_container_forwarding_params(
container_id=container_id,
original_container_id=original_container_id,
custom_llm_provider=resolved_provider,
@@ -396,7 +398,7 @@
)
)
data.update(
- get_container_forwarding_params(
+ await get_container_forwarding_params(
container_id=path_params["container_id"],
original_container_id=original_container_id,
custom_llm_provider=resolved_provider,
diff --git a/litellm/proxy/container_endpoints/ownership.py b/litellm/proxy/container_endpoints/ownership.py
--- a/litellm/proxy/container_endpoints/ownership.py
+++ b/litellm/proxy/container_endpoints/ownership.py
@@ -23,6 +23,13 @@
_NEGATIVE_OWNER_SENTINEL = "__litellm_container_no_owner__"
_CONTAINER_OWNER_CACHE = InMemoryCache(max_size_in_memory=10000, default_ttl=60)
+# Caches the stored ``unified_object_id`` (the encoded container ID
+# captured at create time) so ``get_container_forwarding_params`` can
+# recover the deployment ``model_id`` for native upstream IDs without
+# re-hitting Prisma on every retrieve/delete.
+_NEGATIVE_STORED_ID_SENTINEL = "__litellm_container_no_stored_id__"
+_CONTAINER_STORED_ID_CACHE = InMemoryCache(max_size_in_memory=10000, default_ttl=60)
+
# Per-caller-scope cache for ``GET /v1/containers`` list filtering. Without
# this, every list call issues a fresh ``find_many`` against
# ``litellm_managedobjecttable``. The cache key is the sorted owner-scope
@@ -56,7 +63,7 @@
return original_container_id, custom_llm_provider
-def get_container_forwarding_params(
+async def get_container_forwarding_params(
container_id: str, original_container_id: str, custom_llm_provider: str
) -> Dict[str, str]:
params = {
@@ -65,6 +72,20 @@
}
decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
model_id = decoded.get("model_id")
+ if not (isinstance(model_id, str) and model_id):
+ # Native upstream IDs (e.g. Azure ``cntr_<hex>``) carry no LiteLLM
+ # routing payload, so decoding the user-supplied id yields no
+ # ``model_id``. Recover it from the encoded ``unified_object_id``
+ # captured on the ownership row at create time — when the router
+ # selected a specific deployment that ID embeds the model_id.
+ stored_id = await _get_stored_container_id(
+ original_container_id, custom_llm_provider
+ )
+ if stored_id and stored_id != container_id:
+ stored_decoded = ResponsesAPIRequestUtils._decode_container_id(stored_id)
+ stored_model_id = stored_decoded.get("model_id")
+ if isinstance(stored_model_id, str) and stored_model_id:
+ model_id = stored_model_id
if isinstance(model_id, str) and model_id:
params["model_id"] = model_id
return params
@@ -168,6 +189,7 @@
)
_CONTAINER_OWNER_CACHE.set_cache(model_object_id, owner)
+ _CONTAINER_STORED_ID_CACHE.set_cache(model_object_id, container_id)
# Drop the caller's own list-cache entry so the just-created container
# shows up on their next ``GET /v1/containers``. Other callers with
# disjoint scope tuples have their own entries; intersecting-scope
@@ -207,9 +229,60 @@
_CONTAINER_OWNER_CACHE.set_cache(
model_object_id, owner if owner is not None else _NEGATIVE_OWNER_SENTINEL
)
+ stored_id = getattr(row, "unified_object_id", None) if row is not None else None
+ _CONTAINER_STORED_ID_CACHE.set_cache(
+ model_object_id,
+ (
+ stored_id
+ if isinstance(stored_id, str) and stored_id
+ else _NEGATIVE_STORED_ID_SENTINEL
+ ),
+ )
return owner
+async def _get_stored_container_id(
+ original_container_id: str, custom_llm_provider: str
+) -> Optional[str]:
+ """Return the ``unified_object_id`` stored at create time, if any.
+
+ Used by :func:`get_container_forwarding_params` to recover the
+ deployment ``model_id`` for native upstream container IDs: the stored
+ value is the encoded form produced by ``encode_container_id_in_response``
+ when the router selected a specific deployment.
+ """
+ model_object_id = _container_model_object_id(
+ original_container_id, custom_llm_provider
+ )
+
+ cached = _CONTAINER_STORED_ID_CACHE.get_cache(model_object_id)
+ if cached == _NEGATIVE_STORED_ID_SENTINEL:
+ return None
+ if isinstance(cached, str) and cached:
+ return cached
+
+ prisma_client = await _get_prisma_client()
+ if prisma_client is None:
+ return None
+
+ row = await prisma_client.db.litellm_managedobjecttable.find_first(
+ where={
+ "model_object_id": model_object_id,
+ "file_purpose": CONTAINER_OBJECT_PURPOSE,
+ }
+ )
+ stored_id = getattr(row, "unified_object_id", None) if row is not None else None
+ _CONTAINER_STORED_ID_CACHE.set_cache(
+ model_object_id,
+ (
+ stored_id
+ if isinstance(stored_id, str) and stored_id
+ else _NEGATIVE_STORED_ID_SENTINEL
+ ),
+ )
+ return stored_id if isinstance(stored_id, str) and stored_id else None
+
+
async def assert_user_can_access_container(
container_id: str,
user_api_key_dict: UserAPIKeyAuth,
diff --git a/litellm/router.py b/litellm/router.py
--- a/litellm/router.py
+++ b/litellm/router.py
@@ -5551,6 +5551,7 @@
from litellm.responses.utils import ResponsesAPIRequestUtils
container_id = kwargs.get("container_id")
+ _forwarded_model_id = kwargs.get("model_id")
if isinstance(container_id, str):
decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
original_id = decoded.get("response_id", container_id)
@@ -5559,7 +5560,14 @@
decoded_provider = decoded.get("custom_llm_provider")
if decoded_provider and kwargs.get("custom_llm_provider") == "openai":
kwargs["custom_llm_provider"] = decoded_provider
- model_id = decoded.get("model_id")
+ # Fall back to the model_id forwarded by the proxy when the container_id
+ # is a native upstream ID (e.g. Azure hex cntr_) that carries no LiteLLM
+ # routing payload, so deployment credentials (api_base, api_key) are applied.
+ model_id = decoded.get("model_id") or (
+ _forwarded_model_id.strip()
+ if isinstance(_forwarded_model_id, str) and _forwarded_model_id.strip()
+ else None
+ )
if model_id:
kwargs["model"] = model_id
return await self._ageneric_api_call_with_fallbacks(
diff --git a/tests/test_litellm/containers/test_azure_container_transformation.py b/tests/test_litellm/containers/test_azure_container_transformation.py
--- a/tests/test_litellm/containers/test_azure_container_transformation.py
+++ b/tests/test_litellm/containers/test_azure_container_transformation.py
@@ -109,6 +109,31 @@
assert "/openai/v1/containers" in url
+ def test_get_complete_url_strips_responses_path_and_preserves_api_version(self):
+ """When api_base is the responses endpoint URL, get_complete_url must:
+ - strip /openai/responses (no double-path)
+ - use the api-version from api_base query string, NOT the deployment's
+ older api_version (e.g. 2024-08-01-preview → containers need 2025-04-01-preview)
+ """
+ api_base = "https://my-resource.cognitiveservices.azure.com/openai/responses?api-version=2025-04-01-preview"
+
+ url = self.config.get_complete_url(
+ api_base=api_base,
+ litellm_params={"api_version": "2024-08-01-preview"},
+ )
+
+ assert (
+ "/openai/responses/openai/containers" not in url
+ ), "path must not double /openai/responses"
+ assert "my-resource.cognitiveservices.azure.com" in url
+ assert "/openai/containers" in url or "/openai/v1/containers" in url
+ assert (
+ "2025-04-01-preview" in url
+ ), "must use version from api_base, not litellm_params"
+ assert (
+ "2024-08-01-preview" not in url
+ ), "must not fall back to older chat api_version"
+
def test_get_complete_url_raises_without_api_base(self, monkeypatch):
monkeypatch.delenv("AZURE_API_BASE", raising=False)
monkeypatch.setattr(litellm, "api_base", None)
@@ -531,6 +556,92 @@
assert qs.get("api-version") == ["v1"]
assert qs.get("foo") == ["bar"]
+ @pytest.mark.asyncio
+ async def test_regression_no_container_id_does_not_use_user_supplied_model_id(
+ self, monkeypatch
+ ):
+ """Operations without container_id (create, list) must NOT route via
+ _ageneric_api_call_with_fallbacks using a caller-supplied model_id.
+
+ Security boundary: only the path that holds a validated container_id
+ is trusted to fall back to the forwarded model_id. A caller setting
+ model_id without container_id on POST /v1/containers must not gain
+ access to an arbitrary deployment UUID.
+ """
+ from litellm.router import Router
+
+ router = Router(
+ model_list=[
+ {
+ "model_name": "azure-model",
+ "litellm_params": {
+ "model": "azure/gpt-4",
+ "api_base": "https://my-resource.cognitiveservices.azure.com",
+ "api_key": "test-key",
+ "api_version": "2025-04-01-preview",
+ },
+ "model_info": {"id": "deployment-uuid-123"},
+ }
+ ]
+ )
+
+ fallback_called = {"called": False}
+
+ async def _mock_fallback(original_function, **kwargs):
+ fallback_called["called"] = True
+ return {}
+
+ monkeypatch.setattr(router, "_ageneric_api_call_with_fallbacks", _mock_fallback)
+
+ original_called = {"called": False}
+
+ async def _noop(**kwargs):
+ original_called["called"] = True
+ return {}
+
+ # No container_id — simulates create/list; caller injects a model_id
+ await router._init_containers_api_endpoints(
+ original_function=_noop,
+ model_id="deployment-uuid-123",
+ custom_llm_provider="azure",
+ )
+
+ assert not fallback_called["called"], (
+ "_ageneric_api_call_with_fallbacks must NOT be called when "
+ "container_id is absent, even if model_id is supplied"
+ )
+ assert original_called["called"], "original_function must be called directly"
+
+ def test_regression_httpx_empty_params_strips_query_string(self):
+ """httpx erases the URL query-string when params={} (empty dict) is passed.
+
+ Root cause of the Azure container 404s on POST/DELETE:
+ _build_query_params returns {} when the endpoint has no extra params;
+ passing that {} as params= to httpx wiped ?api-version=2025-04-01-preview.
+
+ Fix: every container httpx call now uses `params or None` so an empty
+ dict falls back to None, which tells httpx to leave the URL untouched.
+ """
+ url = (
+ "https://resource.cognitiveservices.azure.com"
+ "/openai/containers/cntr_123?api-version=2025-04-01-preview"
+ )
+ client = httpx.AsyncClient()
+
+ req_none = client.build_request("DELETE", url, params=None)
+ assert "api-version=2025-04-01-preview" in str(req_none.url)
+
+ req_empty = client.build_request("DELETE", url, params={})
+ assert "api-version" not in str(
+ req_empty.url
+ ), "Documents root cause: params={} strips the query string"
+
+ effective: dict = {}
+ req_guarded = client.build_request("DELETE", url, params=effective or None)
+ assert "api-version=2025-04-01-preview" in str(
+ req_guarded.url
+ ), "`params or None` must preserve ?api-version"
+
def test_regression_proxy_resolves_azure_text_same_as_azure(self):
"""Router/proxy treat azure_text like azure for container config."""
from litellm.proxy.container_endpoints.handler_factory import (
@@ -770,3 +881,143 @@
assert captured["data"]["container_id"] == "cntr_123"
assert captured["data"]["custom_llm_provider"] == "azure"
assert captured["data"]["model_id"] == "model_abc123"
+
+ @pytest.mark.asyncio
+ async def test_regression_get_container_forwarding_params_sets_model_id_for_managed_id(
+ self,
+ ):
+ """get_container_forwarding_params must extract model_id from a
+ LiteLLM-managed encoded container ID and include it in the forwarding
+ dict. This is the proxy-side half of the native-Azure-ID routing fix:
+ the router's _init_containers_api_endpoints reads kwargs["model_id"]
+ which is set here.
+ """
+ from litellm.proxy.container_endpoints.ownership import (
+ get_container_forwarding_params,
+ )
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="deployment-uuid-123",
+ container_id="cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df",
+ )
+
+ params = await get_container_forwarding_params(
+ container_id=encoded_id,
+ original_container_id="cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df",
+ custom_llm_provider="azure",
+ )
+
+ assert (
+ params.get("model_id") == "deployment-uuid-123"
+ ), "model_id must be forwarded to the router for managed container IDs"
+ assert params.get("container_id") == (
+ "cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df"
+ )
+ assert params.get("custom_llm_provider") == "azure"
+
+ @pytest.mark.asyncio
+ async def test_regression_get_container_forwarding_params_recovers_model_id_for_native_id(
+ self, monkeypatch
+ ):
+ """Native Azure IDs (``cntr_<hex>``) cannot be decoded, so model_id
+ must be recovered from the ownership row's ``unified_object_id`` —
+ the encoded form captured at create time when the router selected a
+ specific deployment. Without this, the router-side fallback for
+ native IDs in ``_init_containers_api_endpoints`` is dead code.
+ """
+ from types import SimpleNamespace
+ from unittest.mock import AsyncMock
+
+ from litellm.proxy.container_endpoints import ownership
+ from litellm.proxy.container_endpoints.ownership import (
+ get_container_forwarding_params,
+ )
+
+ native_id = "cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df"
+ encoded_stored_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="deployment-uuid-123",
+ container_id=native_id,
+ )
+
+ ownership._CONTAINER_STORED_ID_CACHE.flush_cache()
+ ownership._CONTAINER_OWNER_CACHE.flush_cache()
+
+ table = AsyncMock()
+ table.find_first.return_value = SimpleNamespace(
+ created_by="user-1",
+ file_purpose=ownership.CONTAINER_OBJECT_PURPOSE,
+ unified_object_id=encoded_stored_id,
+ )
+ prisma_client = SimpleNamespace(
+ db=SimpleNamespace(litellm_managedobjecttable=table)
+ )
+ monkeypatch.setattr(
+ ownership,
+ "_get_prisma_client",
+ AsyncMock(return_value=prisma_client),
+ )
+
+ params = await get_container_forwarding_params(
+ container_id=native_id,
+ original_container_id=native_id,
+ custom_llm_provider="azure",
+ )
+
+ assert params.get("model_id") == "deployment-uuid-123", (
+ "model_id must be recovered from the stored unified_object_id "
+ "for native upstream container IDs"
+ )
+ assert params.get("container_id") == native_id
+ assert params.get("custom_llm_provider") == "azure"
+
+ @pytest.mark.asyncio
+ async def test_regression_native_azure_container_id_uses_forwarded_model_id(
+ self, monkeypatch
+ ):
+ """Native Azure container IDs (cntr_ + hex, no LiteLLM payload) must
+ still route through _ageneric_api_call_with_fallbacks using the
+ model_id forwarded from the proxy ownership check so that deployment
+ credentials (api_base) are applied."""
+ from litellm.router import Router
+
+ router = Router(
+ model_list=[
+ {
+ "model_name": "azure-model",
+ "litellm_params": {
+ "model": "azure/gpt-4",
+ "api_base": "https://my-resource.cognitiveservices.azure.com",
+ "api_key": "test-key",
+ "api_version": "2025-04-01-preview",
+ },
+ "model_info": {"id": "deployment-uuid-123"},
+ }
+ ]
+ )
+
+ called_with: dict = {}
+
+ async def _mock_fallback(original_function, **kwargs):
+ called_with.update(kwargs)
+ return {}
+
+ monkeypatch.setattr(router, "_ageneric_api_call_with_fallbacks", _mock_fallback)
+
+ native_azure_id = "cntr_6a058b43d24c8190a226cfb1d35405b20115fb7875ff11df"
+
+ async def _noop(**kwargs):
+ return {}
+
+ await router._init_containers_api_endpoints(
+ original_function=_noop,
+ container_id=native_azure_id,
+ model_id="deployment-uuid-123",
+ custom_llm_provider="azure",
+ )
+
+ assert called_with.get("model") == "deployment-uuid-123", (
+ "_ageneric_api_call_with_fallbacks must be called with the forwarded "
+ "model_id when the container_id carries no LiteLLM routing payload"
+ )You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit 57c281f. Configure here.
…zure container IDs get_container_forwarding_params previously only set model_id when the user-supplied container_id was a LiteLLM-managed encoded id. For native upstream IDs (e.g. Azure 'cntr_<hex>') the decode fails and model_id was never forwarded — making the router-side fallback in _init_containers_api_endpoints unreachable in production. Fall back to the stored 'unified_object_id' on the ownership row, which is the encoded form captured at create time when the router selected a specific deployment. Decoding that yields the deployment model_id and restores router-based credential application (api_base, api_key) for retrieve/delete and container-file operations on native IDs. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
…ner_routing Pull in base-branch fixes that resolve the CI failures on this PR's HEAD: - 92de742 test(stream-chunk-builder): replace shut-down gpt-4o-audio-preview with gpt-audio-1.5; also fixes test_standard_logging_payload_audio - ce87c41 test(realtime): migrate realtime tests off shut-down upstream models (resolves realtime_translation_testing job failures) - 39a1d43 + b5db7ed + 9770efe + 2b00ea9 test(fireworks): replace deprecated llama-v3p3-70b-instruct and mock remaining live smoke tests - 99a63d5 model_cost_map: add mistral/ministral-8b-2512 entry, fixing test_completion_mistral_api after Mistral's tiny->ministral alias Resolve textual conflicts by taking the base-branch version for: - litellm/proxy/_experimental/mcp_server/{auth/user_api_key_auth_mcp.py, mcp_server_manager.py} and litellm/proxy/management_endpoints/ mcp_management_endpoints.py — base intentionally removes the available_on_public_internet gating (5aabfcc) that this branch added, along with the matching test updates. - pyproject.toml + litellm-proxy-extras/pyproject.toml + uv.lock — version bumps (litellm-proxy-extras 0.4.73, litellm-enterprise 0.1.41). - litellm/_redis.py — add socket_timeout / socket_connect_timeout to the allowed cluster kwargs (additive merge). - Generated Next.js artifacts under litellm/proxy/_experimental/out/ — take the regenerated build from the base branch. Co-authored-by: Claude <claude@anthropic.com>
Contributor
This was referenced Jun 14, 2026
Merged
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
Fixes LIT-3064
Azure code-interpreter containers return provider-native IDs in the format
cntr_<hex>. These carry no LiteLLM routing payload, so_decode_container_idreturnsmodel_id=None._init_containers_api_endpointswas then falling through to call the handler directly, bypassing_ageneric_api_call_with_fallbacks. This leftapi_base=Nonefor Azure deployments, causing:OpenAI worked because
OpenAIContainerConfig.get_complete_urlhas a hardcoded fallback tohttps://api.openai.com/v1. Azure's equivalent raises immediately onapi_base=None.Fix
Fall back to the
model_idforwarded from the proxy ownership check when_decode_container_idyields no model. Also handles the case wherecontainer_idis absent butmodel_idis forwarded.Changed files:
litellm/router.py—_init_containers_api_endpoints: use_forwarded_model_idwhen decoded yieldsNonetests/test_litellm/containers/test_azure_container_transformation.py— regression test for native hex container ID routingNote
Medium Risk
Touches shared HTTP request construction and Azure URL building; mistakes could drop or duplicate query params (notably
api-version) and break container API calls across providers.Overview
Improves Azure container URL generation to handle deployments whose
api_basepoints at an endpoint URL like/openai/responses?api-version=...: it now strips endpoint suffixes, extractsapi-versionfrom the URL, and prefers it over the configuredapi_versionwhen building/openai/containers.Updates container HTTP handlers to pass
params=None(instead of{}) when there are no query params so httpx doesn’t drop query strings already present in the URL (e.g. Azure?api-version=...) for sync/async container and container-file requests.Reviewed by Cursor Bugbot for commit 4713d9f. Bugbot is set up for automated code reviews on this repo. Configure here.