fix(proxy): route azure container file requests by decoded deployment - #26402
Merged
mateo-berri merged 12 commits intoMay 1, 2026
Merged
Conversation
Use decoded managed container model_id to resolve deployment credentials for container file calls and add regressions to verify provider/model metadata decoding and api_base selection. Made-with: Cursor
Contributor
|
No new security concerns identified in this PR. Status: 1 open |
Contributor
Greptile SummaryThis PR shifts container ID decoding responsibility from the proxy handler layer ( Confidence Score: 4/5Safe to merge; the architectural change is sound and regression tests cover the key paths. One P2 style issue in a test name. No P0/P1 findings beyond those already raised in prior review threads. The only new finding is a misleading test name (P2). The critical decoding logic in the router is well-tested by the new unit tests.
|
| Filename | Overview |
|---|---|
| litellm/router.py | Adds managed container ID decoding in _init_containers_api_endpoints; routes through _ageneric_api_call_with_fallbacks when a model_id is present, otherwise falls back to a direct provider call after unwrapping the ID. |
| litellm/proxy/container_endpoints/handler_factory.py | Removes proxy-layer container ID decoding from all three request paths; routes binary responses through ProxyBaseLLMRequestProcessing; adds fastapi_response propagation and a runtime bytes-type guard. |
| litellm/proxy/common_request_processing.py | Registers five new container-file route types in the allowed route-type lists so base_process_llm_request correctly dispatches them. |
| tests/router_unit_tests/test_router_endpoints.py | Updates the existing no-managed-ID test description; adds three new tests for managed-ID decoding, provider override, and the empty-model_id unwrap path in the router. |
| tests/test_litellm/containers/test_azure_container_transformation.py | Adds three regression tests; the multipart-upload test has a misleading name — it asserts the proxy passes custom_llm_provider="openai" (not the decoded "azure"), which contradicts the test name's claim that the provider from the managed ID is used. |
Reviews (9): Last reviewed commit: "Fix greptile review" | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Remove redundant model_id guard assignment and drop duplicate provider-aware fallback lookup that repeated earlier router checks. Made-with: Cursor
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
|
Hey @Sameerlite - thanks for the fix on this. Still a few gaps we've identified. We tested the PR branch against our multi-region Azure setup and can confirm that binary file content download now works correctly. The managed container ID decodes to the right However, we're seeing that the fix only covers the Looks like async def _init_containers_api_endpoints(self, original_function, custom_llm_provider=None, **kwargs):
if custom_llm_provider and "custom_llm_provider" not in kwargs:
kwargs["custom_llm_provider"] = custom_llm_provider
return await original_function(**kwargs) # no model_id decodeMaybe we can apply a similar pattern in Still looking for a comprehensive fix to intelligently route all Containers API endpoints to the correct region. |
4 tasks
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Contributor
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Autofix Details
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Binary response wrapped twice with conflicting headers
- Binary container file responses now copy the proxy-populated FastAPI response headers onto the returned binary Response.
- ✅ Fixed: Multipart upload no longer decodes provider from container ID
- Multipart uploads now decode LiteLLM-managed container IDs and use the embedded provider when the request did not explicitly override it.
Preview (3e540e1d81)
diff --git a/litellm/proxy/common_request_processing.py b/litellm/proxy/common_request_processing.py
--- a/litellm/proxy/common_request_processing.py
+++ b/litellm/proxy/common_request_processing.py
@@ -683,6 +683,11 @@
"aingest",
"aretrieve_container",
"adelete_container",
+ "aupload_container_file",
+ "alist_container_files",
+ "aretrieve_container_file",
+ "adelete_container_file",
+ "aretrieve_container_file_content",
"acreate_skill",
"alist_skills",
"aget_skill",
@@ -940,6 +945,11 @@
"aingest",
"aretrieve_container",
"adelete_container",
+ "aupload_container_file",
+ "alist_container_files",
+ "aretrieve_container_file",
+ "adelete_container_file",
+ "aretrieve_container_file_content",
"acreate_skill",
"alist_skills",
"aget_skill",
diff --git a/litellm/proxy/container_endpoints/handler_factory.py b/litellm/proxy/container_endpoints/handler_factory.py
--- a/litellm/proxy/container_endpoints/handler_factory.py
+++ b/litellm/proxy/container_endpoints/handler_factory.py
@@ -64,10 +64,12 @@
request: Request,
container_id: str,
file_id: str,
+ fastapi_response: Response,
user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
):
return await _process_binary_request(
request=request,
+ fastapi_response=fastapi_response,
container_id=container_id,
file_id=file_id,
user_api_key_dict=user_api_key_dict,
@@ -152,63 +154,61 @@
async def _process_binary_request(
request: Request,
+ fastapi_response: Response,
container_id: str,
file_id: str,
user_api_key_dict: UserAPIKeyAuth,
):
"""
- Process binary content requests using the proper transformation pattern.
+ Process binary content requests through the standard proxy/router pipeline.
- This uses the provider config transformations and llm_http_handler
- to maintain consistency with the established pattern.
+ The router owns managed container ID decoding and deployment selection. This
+ handler only adapts the byte response to FastAPI.
"""
- from litellm.litellm_core_utils.litellm_logging import Logging
- from litellm.llms.custom_httpx.llm_http_handler import BaseLLMHTTPHandler
- from litellm.types.router import GenericLiteLLMParams
+ from litellm.proxy.proxy_server import (
+ general_settings,
+ llm_router,
+ proxy_config,
+ proxy_logging_obj,
+ select_data_generator,
+ user_api_base,
+ user_max_tokens,
+ user_model,
+ user_request_timeout,
+ user_temperature,
+ version,
+ )
- # Extract custom_llm_provider
custom_llm_provider = (
get_custom_llm_provider_from_request_headers(request=request)
or get_custom_llm_provider_from_request_query(request=request)
or "openai"
)
+ data: Dict[str, Any] = {
+ "container_id": container_id,
+ "file_id": file_id,
+ "custom_llm_provider": custom_llm_provider,
+ }
+ processor = ProxyBaseLLMRequestProcessing(data=data)
- # Build litellm_params - credentials are resolved by provider config from env
- litellm_params = GenericLiteLLMParams()
-
- # Decode container ID and extract provider info
- decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
- original_container_id = decoded.get("response_id", container_id)
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
- if decoded_provider and custom_llm_provider == "openai":
- custom_llm_provider = decoded_provider
-
- # Get the provider config
- container_provider_config = _get_container_provider_config(custom_llm_provider)
-
- # Create logging object
- logging_obj = Logging(
- model="container-file-content",
- messages=[],
- stream=False,
- call_type="container_file_content",
- start_time=None,
- litellm_call_id="",
- function_id="",
- )
-
- # Use the HTTP handler to make the request
- handler = BaseLLMHTTPHandler()
-
try:
- content = await handler.async_container_file_content_handler(
- container_id=original_container_id, # Use decoded original ID
- file_id=file_id,
- container_provider_config=container_provider_config,
- litellm_params=litellm_params,
- logging_obj=logging_obj,
+ content = await processor.base_process_llm_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ user_api_key_dict=user_api_key_dict,
+ route_type="aretrieve_container_file_content",
+ proxy_logging_obj=proxy_logging_obj,
+ llm_router=llm_router,
+ general_settings=general_settings,
+ proxy_config=proxy_config,
+ select_data_generator=select_data_generator,
+ model=None,
+ user_model=user_model,
+ user_temperature=user_temperature,
+ user_request_timeout=user_request_timeout,
+ user_max_tokens=user_max_tokens,
+ user_api_base=user_api_base,
+ version=version,
)
# Determine content type based on common file extensions in the file_id
@@ -231,11 +231,17 @@
return Response(
content=content,
+ headers=dict(fastapi_response.headers),
media_type=content_type,
)
except Exception as e:
- raise e
+ raise await processor._handle_llm_api_exception(
+ e=e,
+ user_api_key_dict=user_api_key_dict,
+ proxy_logging_obj=proxy_logging_obj,
+ version=version,
+ )
async def _process_multipart_upload_request(
@@ -283,17 +289,12 @@
or get_custom_llm_provider_from_request_query(request=request)
or "openai"
)
-
- # Decode container ID and extract provider info
- decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
- original_container_id = decoded.get("response_id", container_id)
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
+ decoded_container_id = ResponsesAPIRequestUtils._decode_container_id(container_id)
+ decoded_provider = decoded_container_id.get("custom_llm_provider")
if decoded_provider and custom_llm_provider == "openai":
custom_llm_provider = decoded_provider
- data["container_id"] = original_container_id # Use decoded original ID
+ data["container_id"] = container_id
data["custom_llm_provider"] = custom_llm_provider
processor = ProxyBaseLLMRequestProcessing(data=data)
@@ -359,21 +360,6 @@
or "openai"
)
- # Decode container_id if present in path_params
- if "container_id" in path_params:
- decoded = ResponsesAPIRequestUtils._decode_container_id(
- path_params["container_id"]
- )
- original_container_id = decoded.get("response_id", path_params["container_id"])
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
- if decoded_provider and custom_llm_provider == "openai":
- custom_llm_provider = decoded_provider
-
- # Update path_params with decoded original ID
- data["container_id"] = original_container_id
-
data["custom_llm_provider"] = custom_llm_provider
processor = ProxyBaseLLMRequestProcessing(data=data)
diff --git a/litellm/router.py b/litellm/router.py
--- a/litellm/router.py
+++ b/litellm/router.py
@@ -5261,11 +5261,32 @@
"""
Initialize the Containers API endpoints on the router.
- Container operations don't need model-based routing, so we call the
- original function directly with the custom_llm_provider.
+ LiteLLM-managed container IDs (``cntr_...``) encode ``model_id`` and provider
+ metadata. When present, decode the ID, replace ``container_id`` with the
+ upstream value, and route through ``_ageneric_api_call_with_fallbacks`` so
+ deployment credentials (e.g. regional ``api_base`` for Azure) match
+ :meth:`_init_responses_api_endpoints`. Otherwise call the handler directly.
"""
if custom_llm_provider and "custom_llm_provider" not in kwargs:
kwargs["custom_llm_provider"] = custom_llm_provider
+
+ from litellm.responses.utils import ResponsesAPIRequestUtils
+
+ container_id = kwargs.get("container_id")
+ if isinstance(container_id, str):
+ decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
+ model_id = decoded.get("model_id")
+ if model_id:
+ kwargs["container_id"] = decoded.get("response_id", container_id)
+ kwargs["model"] = model_id
+ decoded_provider = decoded.get("custom_llm_provider")
+ if decoded_provider and kwargs.get("custom_llm_provider") == "openai":
+ kwargs["custom_llm_provider"] = decoded_provider
+ return await self._ageneric_api_call_with_fallbacks(
+ original_function=original_function,
+ **kwargs,
+ )
+
return await original_function(**kwargs)
async def _init_responses_api_endpoints(
diff --git a/tests/router_unit_tests/test_router_endpoints.py b/tests/router_unit_tests/test_router_endpoints.py
--- a/tests/router_unit_tests/test_router_endpoints.py
+++ b/tests/router_unit_tests/test_router_endpoints.py
@@ -1110,7 +1110,7 @@
async def test_init_containers_api_endpoints():
"""
Test that _init_containers_api_endpoints calls the original function
- directly without model-based routing.
+ directly when there is no managed container ID (no embedded model_id).
"""
router = Router(model_list=[])
@@ -1127,3 +1127,46 @@
custom_llm_provider="openai", name="Test Container"
)
assert result == mock_response
+
+
+@pytest.mark.asyncio
+async def test_init_containers_api_endpoints_managed_id_routes_via_generic_fallbacks():
+ """
+ Managed ``cntr_`` IDs embed ``model_id``; router should decode and use
+ ``_ageneric_api_call_with_fallbacks`` so deployment credentials apply.
+ """
+ from litellm.responses.utils import ResponsesAPIRequestUtils
+
+ router = Router(
+ model_list=[
+ {
+ "model_name": "azure-router-model",
+ "litellm_params": {
+ "model": "azure/gpt-4",
+ "api_key": "fake-key",
+ "api_base": "https://westus.api.cognitive.microsoft.com",
+ },
+ }
+ ]
+ )
+ router._ageneric_api_call_with_fallbacks = AsyncMock()
+
+ managed_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="azure-router-model",
+ container_id="cfile_upstream_abc",
+ )
+
+ await router._init_containers_api_endpoints(
+ original_function=AsyncMock(),
+ custom_llm_provider="openai",
+ container_id=managed_id,
+ file_id="cfile_xyz",
+ )
+
+ router._ageneric_api_call_with_fallbacks.assert_called_once()
+ call_kw = router._ageneric_api_call_with_fallbacks.call_args.kwargs
+ assert call_kw["model"] == "azure-router-model"
+ assert call_kw["container_id"] == "cfile_upstream_abc"
+ assert call_kw["file_id"] == "cfile_xyz"
+ assert call_kw["custom_llm_provider"] == "azure"
diff --git a/tests/test_litellm/containers/test_azure_container_transformation.py b/tests/test_litellm/containers/test_azure_container_transformation.py
--- a/tests/test_litellm/containers/test_azure_container_transformation.py
+++ b/tests/test_litellm/containers/test_azure_container_transformation.py
@@ -11,6 +11,7 @@
import litellm
from litellm.llms.azure.containers.transformation import AzureContainerConfig
from litellm.llms.base_llm.containers.transformation import BaseContainerConfig
+from litellm.responses.utils import ResponsesAPIRequestUtils
from litellm.types.containers.main import (
ContainerFileListResponse,
ContainerListResponse,
@@ -518,3 +519,206 @@
c2 = _get_container_provider_config("azure_text")
assert type(c1) is type(c2)
assert isinstance(c1, AzureContainerConfig)
+
+ @pytest.mark.asyncio
+ async def test_proxy_process_request_preserves_managed_container_id(
+ self, monkeypatch
+ ):
+ from starlette.requests import Request
+
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ return {"id": "cfile_abc"}
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "GET",
+ "path": "/v1/containers/id/files/id/content",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+ fastapi_response = MagicMock()
+
+ await handler_factory._process_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ user_api_key_dict=MagicMock(),
+ route_type="alist_container_files",
+ path_params={"container_id": encoded_id},
+ )
+
+ assert captured["route_type"] == "alist_container_files"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["custom_llm_provider"] == "openai"
+ assert "model_id" not in captured["data"]
+ assert "api_base" not in captured["data"]
+
+ @pytest.mark.asyncio
+ async def test_regression_binary_file_request_routes_through_proxy_processor(
+ self, monkeypatch
+ ):
+ from fastapi import Response
+ from starlette.requests import Request
+
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ fastapi_response.headers["x-litellm-call-id"] = "call-123"
+ return b"csv-bytes"
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "GET",
+ "path": "/v1/containers/id/files/id/content",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+ fastapi_response = Response()
+
+ response = await handler_factory._process_binary_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ container_id=encoded_id,
+ file_id="cfile_abc",
+ user_api_key_dict=MagicMock(),
+ )
+
+ assert captured["route_type"] == "aretrieve_container_file_content"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["file_id"] == "cfile_abc"
+ assert captured["data"]["custom_llm_provider"] == "openai"
+ assert response.status_code == 200
+ assert response.body == b"csv-bytes"
+ assert response.headers["x-litellm-call-id"] == "call-123"
+
+ @pytest.mark.asyncio
+ async def test_regression_multipart_upload_request_uses_provider_from_managed_id(
+ self, monkeypatch
+ ):
+ from starlette.requests import Request
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+ from litellm.proxy.common_utils import http_parsing_utils
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_get_form_data(request):
+ return {"file": "ignored"}
+
+ async def _mock_convert_upload_files_to_file_data(form_data):
+ return {"file": [("data.csv", b"csv-bytes", "text/csv")]}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ return {"id": "cfile_abc"}
+
+ monkeypatch.setattr(
+ http_parsing_utils,
+ "get_form_data",
+ _mock_get_form_data,
+ )
+ monkeypatch.setattr(
+ http_parsing_utils,
+ "convert_upload_files_to_file_data",
+ _mock_convert_upload_files_to_file_data,
+ )
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "POST",
+ "path": "/v1/containers/id/files",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+
+ await handler_factory._process_multipart_upload_request(
+ request=request,
+ fastapi_response=MagicMock(),
+ user_api_key_dict=MagicMock(),
+ route_type="aupload_container_file",
+ container_id=encoded_id,
+ )
+
+ assert captured["route_type"] == "aupload_container_file"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["custom_llm_provider"] == "azure"You can send follow-ups to the cloud agent here.
Contributor
|
bugbot run |
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Inconsistent provider decoding only in multipart handler
- Removed multipart handler-level managed container ID decoding so all container handlers delegate provider extraction to the router consistently.
Preview (02ee8e2e44)
diff --git a/litellm/proxy/common_request_processing.py b/litellm/proxy/common_request_processing.py
--- a/litellm/proxy/common_request_processing.py
+++ b/litellm/proxy/common_request_processing.py
@@ -683,6 +683,11 @@
"aingest",
"aretrieve_container",
"adelete_container",
+ "aupload_container_file",
+ "alist_container_files",
+ "aretrieve_container_file",
+ "adelete_container_file",
+ "aretrieve_container_file_content",
"acreate_skill",
"alist_skills",
"aget_skill",
@@ -940,6 +945,11 @@
"aingest",
"aretrieve_container",
"adelete_container",
+ "aupload_container_file",
+ "alist_container_files",
+ "aretrieve_container_file",
+ "adelete_container_file",
+ "aretrieve_container_file_content",
"acreate_skill",
"alist_skills",
"aget_skill",
diff --git a/litellm/proxy/container_endpoints/handler_factory.py b/litellm/proxy/container_endpoints/handler_factory.py
--- a/litellm/proxy/container_endpoints/handler_factory.py
+++ b/litellm/proxy/container_endpoints/handler_factory.py
@@ -19,7 +19,6 @@
get_custom_llm_provider_from_request_headers,
get_custom_llm_provider_from_request_query,
)
-from litellm.responses.utils import ResponsesAPIRequestUtils
def _load_endpoints_config() -> Dict:
@@ -64,10 +63,12 @@
request: Request,
container_id: str,
file_id: str,
+ fastapi_response: Response,
user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
):
return await _process_binary_request(
request=request,
+ fastapi_response=fastapi_response,
container_id=container_id,
file_id=file_id,
user_api_key_dict=user_api_key_dict,
@@ -152,63 +153,61 @@
async def _process_binary_request(
request: Request,
+ fastapi_response: Response,
container_id: str,
file_id: str,
user_api_key_dict: UserAPIKeyAuth,
):
"""
- Process binary content requests using the proper transformation pattern.
+ Process binary content requests through the standard proxy/router pipeline.
- This uses the provider config transformations and llm_http_handler
- to maintain consistency with the established pattern.
+ The router owns managed container ID decoding and deployment selection. This
+ handler only adapts the byte response to FastAPI.
"""
- from litellm.litellm_core_utils.litellm_logging import Logging
- from litellm.llms.custom_httpx.llm_http_handler import BaseLLMHTTPHandler
- from litellm.types.router import GenericLiteLLMParams
+ from litellm.proxy.proxy_server import (
+ general_settings,
+ llm_router,
+ proxy_config,
+ proxy_logging_obj,
+ select_data_generator,
+ user_api_base,
+ user_max_tokens,
+ user_model,
+ user_request_timeout,
+ user_temperature,
+ version,
+ )
- # Extract custom_llm_provider
custom_llm_provider = (
get_custom_llm_provider_from_request_headers(request=request)
or get_custom_llm_provider_from_request_query(request=request)
or "openai"
)
+ data: Dict[str, Any] = {
+ "container_id": container_id,
+ "file_id": file_id,
+ "custom_llm_provider": custom_llm_provider,
+ }
+ processor = ProxyBaseLLMRequestProcessing(data=data)
- # Build litellm_params - credentials are resolved by provider config from env
- litellm_params = GenericLiteLLMParams()
-
- # Decode container ID and extract provider info
- decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
- original_container_id = decoded.get("response_id", container_id)
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
- if decoded_provider and custom_llm_provider == "openai":
- custom_llm_provider = decoded_provider
-
- # Get the provider config
- container_provider_config = _get_container_provider_config(custom_llm_provider)
-
- # Create logging object
- logging_obj = Logging(
- model="container-file-content",
- messages=[],
- stream=False,
- call_type="container_file_content",
- start_time=None,
- litellm_call_id="",
- function_id="",
- )
-
- # Use the HTTP handler to make the request
- handler = BaseLLMHTTPHandler()
-
try:
- content = await handler.async_container_file_content_handler(
- container_id=original_container_id, # Use decoded original ID
- file_id=file_id,
- container_provider_config=container_provider_config,
- litellm_params=litellm_params,
- logging_obj=logging_obj,
+ content = await processor.base_process_llm_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ user_api_key_dict=user_api_key_dict,
+ route_type="aretrieve_container_file_content",
+ proxy_logging_obj=proxy_logging_obj,
+ llm_router=llm_router,
+ general_settings=general_settings,
+ proxy_config=proxy_config,
+ select_data_generator=select_data_generator,
+ model=None,
+ user_model=user_model,
+ user_temperature=user_temperature,
+ user_request_timeout=user_request_timeout,
+ user_max_tokens=user_max_tokens,
+ user_api_base=user_api_base,
+ version=version,
)
# Determine content type based on common file extensions in the file_id
@@ -231,11 +230,17 @@
return Response(
content=content,
+ headers=dict(fastapi_response.headers),
media_type=content_type,
)
except Exception as e:
- raise e
+ raise await processor._handle_llm_api_exception(
+ e=e,
+ user_api_key_dict=user_api_key_dict,
+ proxy_logging_obj=proxy_logging_obj,
+ version=version,
+ )
async def _process_multipart_upload_request(
@@ -284,16 +289,7 @@
or "openai"
)
- # Decode container ID and extract provider info
- decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
- original_container_id = decoded.get("response_id", container_id)
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
- if decoded_provider and custom_llm_provider == "openai":
- custom_llm_provider = decoded_provider
-
- data["container_id"] = original_container_id # Use decoded original ID
+ data["container_id"] = container_id
data["custom_llm_provider"] = custom_llm_provider
processor = ProxyBaseLLMRequestProcessing(data=data)
@@ -359,21 +355,6 @@
or "openai"
)
- # Decode container_id if present in path_params
- if "container_id" in path_params:
- decoded = ResponsesAPIRequestUtils._decode_container_id(
- path_params["container_id"]
- )
- original_container_id = decoded.get("response_id", path_params["container_id"])
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
- if decoded_provider and custom_llm_provider == "openai":
- custom_llm_provider = decoded_provider
-
- # Update path_params with decoded original ID
- data["container_id"] = original_container_id
-
data["custom_llm_provider"] = custom_llm_provider
processor = ProxyBaseLLMRequestProcessing(data=data)
diff --git a/litellm/router.py b/litellm/router.py
--- a/litellm/router.py
+++ b/litellm/router.py
@@ -5261,11 +5261,32 @@
"""
Initialize the Containers API endpoints on the router.
- Container operations don't need model-based routing, so we call the
- original function directly with the custom_llm_provider.
+ LiteLLM-managed container IDs (``cntr_...``) encode ``model_id`` and provider
+ metadata. When present, decode the ID, replace ``container_id`` with the
+ upstream value, and route through ``_ageneric_api_call_with_fallbacks`` so
+ deployment credentials (e.g. regional ``api_base`` for Azure) match
+ :meth:`_init_responses_api_endpoints`. Otherwise call the handler directly.
"""
if custom_llm_provider and "custom_llm_provider" not in kwargs:
kwargs["custom_llm_provider"] = custom_llm_provider
+
+ from litellm.responses.utils import ResponsesAPIRequestUtils
+
+ container_id = kwargs.get("container_id")
+ if isinstance(container_id, str):
+ decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
+ model_id = decoded.get("model_id")
+ if model_id:
+ kwargs["container_id"] = decoded.get("response_id", container_id)
+ kwargs["model"] = model_id
+ decoded_provider = decoded.get("custom_llm_provider")
+ if decoded_provider and kwargs.get("custom_llm_provider") == "openai":
+ kwargs["custom_llm_provider"] = decoded_provider
+ return await self._ageneric_api_call_with_fallbacks(
+ original_function=original_function,
+ **kwargs,
+ )
+
return await original_function(**kwargs)
async def _init_responses_api_endpoints(
diff --git a/tests/router_unit_tests/test_router_endpoints.py b/tests/router_unit_tests/test_router_endpoints.py
--- a/tests/router_unit_tests/test_router_endpoints.py
+++ b/tests/router_unit_tests/test_router_endpoints.py
@@ -1110,7 +1110,7 @@
async def test_init_containers_api_endpoints():
"""
Test that _init_containers_api_endpoints calls the original function
- directly without model-based routing.
+ directly when there is no managed container ID (no embedded model_id).
"""
router = Router(model_list=[])
@@ -1127,3 +1127,46 @@
custom_llm_provider="openai", name="Test Container"
)
assert result == mock_response
+
+
+@pytest.mark.asyncio
+async def test_init_containers_api_endpoints_managed_id_routes_via_generic_fallbacks():
+ """
+ Managed ``cntr_`` IDs embed ``model_id``; router should decode and use
+ ``_ageneric_api_call_with_fallbacks`` so deployment credentials apply.
+ """
+ from litellm.responses.utils import ResponsesAPIRequestUtils
+
+ router = Router(
+ model_list=[
+ {
+ "model_name": "azure-router-model",
+ "litellm_params": {
+ "model": "azure/gpt-4",
+ "api_key": "fake-key",
+ "api_base": "https://westus.api.cognitive.microsoft.com",
+ },
+ }
+ ]
+ )
+ router._ageneric_api_call_with_fallbacks = AsyncMock()
+
+ managed_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="azure-router-model",
+ container_id="cfile_upstream_abc",
+ )
+
+ await router._init_containers_api_endpoints(
+ original_function=AsyncMock(),
+ custom_llm_provider="openai",
+ container_id=managed_id,
+ file_id="cfile_xyz",
+ )
+
+ router._ageneric_api_call_with_fallbacks.assert_called_once()
+ call_kw = router._ageneric_api_call_with_fallbacks.call_args.kwargs
+ assert call_kw["model"] == "azure-router-model"
+ assert call_kw["container_id"] == "cfile_upstream_abc"
+ assert call_kw["file_id"] == "cfile_xyz"
+ assert call_kw["custom_llm_provider"] == "azure"
diff --git a/tests/test_litellm/containers/test_azure_container_transformation.py b/tests/test_litellm/containers/test_azure_container_transformation.py
--- a/tests/test_litellm/containers/test_azure_container_transformation.py
+++ b/tests/test_litellm/containers/test_azure_container_transformation.py
@@ -11,6 +11,7 @@
import litellm
from litellm.llms.azure.containers.transformation import AzureContainerConfig
from litellm.llms.base_llm.containers.transformation import BaseContainerConfig
+from litellm.responses.utils import ResponsesAPIRequestUtils
from litellm.types.containers.main import (
ContainerFileListResponse,
ContainerListResponse,
@@ -518,3 +519,206 @@
c2 = _get_container_provider_config("azure_text")
assert type(c1) is type(c2)
assert isinstance(c1, AzureContainerConfig)
+
+ @pytest.mark.asyncio
+ async def test_proxy_process_request_preserves_managed_container_id(
+ self, monkeypatch
+ ):
+ from starlette.requests import Request
+
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ return {"id": "cfile_abc"}
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "GET",
+ "path": "/v1/containers/id/files/id/content",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+ fastapi_response = MagicMock()
+
+ await handler_factory._process_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ user_api_key_dict=MagicMock(),
+ route_type="alist_container_files",
+ path_params={"container_id": encoded_id},
+ )
+
+ assert captured["route_type"] == "alist_container_files"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["custom_llm_provider"] == "openai"
+ assert "model_id" not in captured["data"]
+ assert "api_base" not in captured["data"]
+
+ @pytest.mark.asyncio
+ async def test_regression_binary_file_request_routes_through_proxy_processor(
+ self, monkeypatch
+ ):
+ from fastapi import Response
+ from starlette.requests import Request
+
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ fastapi_response.headers["x-litellm-call-id"] = "call-123"
+ return b"csv-bytes"
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "GET",
+ "path": "/v1/containers/id/files/id/content",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+ fastapi_response = Response()
+
+ response = await handler_factory._process_binary_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ container_id=encoded_id,
+ file_id="cfile_abc",
+ user_api_key_dict=MagicMock(),
+ )
+
+ assert captured["route_type"] == "aretrieve_container_file_content"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["file_id"] == "cfile_abc"
+ assert captured["data"]["custom_llm_provider"] == "openai"
+ assert response.status_code == 200
+ assert response.body == b"csv-bytes"
+ assert response.headers["x-litellm-call-id"] == "call-123"
+
+ @pytest.mark.asyncio
+ async def test_regression_multipart_upload_request_uses_provider_from_managed_id(
+ self, monkeypatch
+ ):
+ from starlette.requests import Request
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+ from litellm.proxy.common_utils import http_parsing_utils
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_get_form_data(request):
+ return {"file": "ignored"}
+
+ async def _mock_convert_upload_files_to_file_data(form_data):
+ return {"file": [("data.csv", b"csv-bytes", "text/csv")]}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ return {"id": "cfile_abc"}
+
+ monkeypatch.setattr(
+ http_parsing_utils,
+ "get_form_data",
+ _mock_get_form_data,
+ )
+ monkeypatch.setattr(
+ http_parsing_utils,
+ "convert_upload_files_to_file_data",
+ _mock_convert_upload_files_to_file_data,
+ )
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "POST",
+ "path": "/v1/containers/id/files",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+
+ await handler_factory._process_multipart_upload_request(
+ request=request,
+ fastapi_response=MagicMock(),
+ user_api_key_dict=MagicMock(),
+ route_type="aupload_container_file",
+ container_id=encoded_id,
+ )
+
+ assert captured["route_type"] == "aupload_container_file"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["custom_llm_provider"] == "azure"You can send follow-ups to the cloud agent here.
|
|
Contributor
|
bugbot run |
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Test asserts wrong provider after decoding removal
- Updated the multipart upload regression test to expect the handler-layer default provider of openai after managed ID decoding moved to the router.
Preview (2af55422f6)
diff --git a/litellm/proxy/common_request_processing.py b/litellm/proxy/common_request_processing.py
--- a/litellm/proxy/common_request_processing.py
+++ b/litellm/proxy/common_request_processing.py
@@ -683,6 +683,11 @@
"aingest",
"aretrieve_container",
"adelete_container",
+ "aupload_container_file",
+ "alist_container_files",
+ "aretrieve_container_file",
+ "adelete_container_file",
+ "aretrieve_container_file_content",
"acreate_skill",
"alist_skills",
"aget_skill",
@@ -940,6 +945,11 @@
"aingest",
"aretrieve_container",
"adelete_container",
+ "aupload_container_file",
+ "alist_container_files",
+ "aretrieve_container_file",
+ "adelete_container_file",
+ "aretrieve_container_file_content",
"acreate_skill",
"alist_skills",
"aget_skill",
diff --git a/litellm/proxy/container_endpoints/handler_factory.py b/litellm/proxy/container_endpoints/handler_factory.py
--- a/litellm/proxy/container_endpoints/handler_factory.py
+++ b/litellm/proxy/container_endpoints/handler_factory.py
@@ -19,7 +19,6 @@
get_custom_llm_provider_from_request_headers,
get_custom_llm_provider_from_request_query,
)
-from litellm.responses.utils import ResponsesAPIRequestUtils
def _load_endpoints_config() -> Dict:
@@ -64,10 +63,12 @@
request: Request,
container_id: str,
file_id: str,
+ fastapi_response: Response,
user_api_key_dict: UserAPIKeyAuth = Depends(user_api_key_auth),
):
return await _process_binary_request(
request=request,
+ fastapi_response=fastapi_response,
container_id=container_id,
file_id=file_id,
user_api_key_dict=user_api_key_dict,
@@ -152,63 +153,61 @@
async def _process_binary_request(
request: Request,
+ fastapi_response: Response,
container_id: str,
file_id: str,
user_api_key_dict: UserAPIKeyAuth,
):
"""
- Process binary content requests using the proper transformation pattern.
+ Process binary content requests through the standard proxy/router pipeline.
- This uses the provider config transformations and llm_http_handler
- to maintain consistency with the established pattern.
+ The router owns managed container ID decoding and deployment selection. This
+ handler only adapts the byte response to FastAPI.
"""
- from litellm.litellm_core_utils.litellm_logging import Logging
- from litellm.llms.custom_httpx.llm_http_handler import BaseLLMHTTPHandler
- from litellm.types.router import GenericLiteLLMParams
+ from litellm.proxy.proxy_server import (
+ general_settings,
+ llm_router,
+ proxy_config,
+ proxy_logging_obj,
+ select_data_generator,
+ user_api_base,
+ user_max_tokens,
+ user_model,
+ user_request_timeout,
+ user_temperature,
+ version,
+ )
- # Extract custom_llm_provider
custom_llm_provider = (
get_custom_llm_provider_from_request_headers(request=request)
or get_custom_llm_provider_from_request_query(request=request)
or "openai"
)
+ data: Dict[str, Any] = {
+ "container_id": container_id,
+ "file_id": file_id,
+ "custom_llm_provider": custom_llm_provider,
+ }
+ processor = ProxyBaseLLMRequestProcessing(data=data)
- # Build litellm_params - credentials are resolved by provider config from env
- litellm_params = GenericLiteLLMParams()
-
- # Decode container ID and extract provider info
- decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
- original_container_id = decoded.get("response_id", container_id)
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
- if decoded_provider and custom_llm_provider == "openai":
- custom_llm_provider = decoded_provider
-
- # Get the provider config
- container_provider_config = _get_container_provider_config(custom_llm_provider)
-
- # Create logging object
- logging_obj = Logging(
- model="container-file-content",
- messages=[],
- stream=False,
- call_type="container_file_content",
- start_time=None,
- litellm_call_id="",
- function_id="",
- )
-
- # Use the HTTP handler to make the request
- handler = BaseLLMHTTPHandler()
-
try:
- content = await handler.async_container_file_content_handler(
- container_id=original_container_id, # Use decoded original ID
- file_id=file_id,
- container_provider_config=container_provider_config,
- litellm_params=litellm_params,
- logging_obj=logging_obj,
+ content = await processor.base_process_llm_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ user_api_key_dict=user_api_key_dict,
+ route_type="aretrieve_container_file_content",
+ proxy_logging_obj=proxy_logging_obj,
+ llm_router=llm_router,
+ general_settings=general_settings,
+ proxy_config=proxy_config,
+ select_data_generator=select_data_generator,
+ model=None,
+ user_model=user_model,
+ user_temperature=user_temperature,
+ user_request_timeout=user_request_timeout,
+ user_max_tokens=user_max_tokens,
+ user_api_base=user_api_base,
+ version=version,
)
# Determine content type based on common file extensions in the file_id
@@ -231,11 +230,17 @@
return Response(
content=content,
+ headers=dict(fastapi_response.headers),
media_type=content_type,
)
except Exception as e:
- raise e
+ raise await processor._handle_llm_api_exception(
+ e=e,
+ user_api_key_dict=user_api_key_dict,
+ proxy_logging_obj=proxy_logging_obj,
+ version=version,
+ )
async def _process_multipart_upload_request(
@@ -284,16 +289,7 @@
or "openai"
)
- # Decode container ID and extract provider info
- decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
- original_container_id = decoded.get("response_id", container_id)
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
- if decoded_provider and custom_llm_provider == "openai":
- custom_llm_provider = decoded_provider
-
- data["container_id"] = original_container_id # Use decoded original ID
+ data["container_id"] = container_id
data["custom_llm_provider"] = custom_llm_provider
processor = ProxyBaseLLMRequestProcessing(data=data)
@@ -359,21 +355,6 @@
or "openai"
)
- # Decode container_id if present in path_params
- if "container_id" in path_params:
- decoded = ResponsesAPIRequestUtils._decode_container_id(
- path_params["container_id"]
- )
- original_container_id = decoded.get("response_id", path_params["container_id"])
-
- # If container ID has encoded provider info and user didn't explicitly set provider, use it
- decoded_provider = decoded.get("custom_llm_provider")
- if decoded_provider and custom_llm_provider == "openai":
- custom_llm_provider = decoded_provider
-
- # Update path_params with decoded original ID
- data["container_id"] = original_container_id
-
data["custom_llm_provider"] = custom_llm_provider
processor = ProxyBaseLLMRequestProcessing(data=data)
diff --git a/litellm/router.py b/litellm/router.py
--- a/litellm/router.py
+++ b/litellm/router.py
@@ -5261,11 +5261,32 @@
"""
Initialize the Containers API endpoints on the router.
- Container operations don't need model-based routing, so we call the
- original function directly with the custom_llm_provider.
+ LiteLLM-managed container IDs (``cntr_...``) encode ``model_id`` and provider
+ metadata. When present, decode the ID, replace ``container_id`` with the
+ upstream value, and route through ``_ageneric_api_call_with_fallbacks`` so
+ deployment credentials (e.g. regional ``api_base`` for Azure) match
+ :meth:`_init_responses_api_endpoints`. Otherwise call the handler directly.
"""
if custom_llm_provider and "custom_llm_provider" not in kwargs:
kwargs["custom_llm_provider"] = custom_llm_provider
+
+ from litellm.responses.utils import ResponsesAPIRequestUtils
+
+ container_id = kwargs.get("container_id")
+ if isinstance(container_id, str):
+ decoded = ResponsesAPIRequestUtils._decode_container_id(container_id)
+ model_id = decoded.get("model_id")
+ if model_id:
+ kwargs["container_id"] = decoded.get("response_id", container_id)
+ kwargs["model"] = model_id
+ decoded_provider = decoded.get("custom_llm_provider")
+ if decoded_provider and kwargs.get("custom_llm_provider") == "openai":
+ kwargs["custom_llm_provider"] = decoded_provider
+ return await self._ageneric_api_call_with_fallbacks(
+ original_function=original_function,
+ **kwargs,
+ )
+
return await original_function(**kwargs)
async def _init_responses_api_endpoints(
diff --git a/tests/router_unit_tests/test_router_endpoints.py b/tests/router_unit_tests/test_router_endpoints.py
--- a/tests/router_unit_tests/test_router_endpoints.py
+++ b/tests/router_unit_tests/test_router_endpoints.py
@@ -1110,7 +1110,7 @@
async def test_init_containers_api_endpoints():
"""
Test that _init_containers_api_endpoints calls the original function
- directly without model-based routing.
+ directly when there is no managed container ID (no embedded model_id).
"""
router = Router(model_list=[])
@@ -1127,3 +1127,46 @@
custom_llm_provider="openai", name="Test Container"
)
assert result == mock_response
+
+
+@pytest.mark.asyncio
+async def test_init_containers_api_endpoints_managed_id_routes_via_generic_fallbacks():
+ """
+ Managed ``cntr_`` IDs embed ``model_id``; router should decode and use
+ ``_ageneric_api_call_with_fallbacks`` so deployment credentials apply.
+ """
+ from litellm.responses.utils import ResponsesAPIRequestUtils
+
+ router = Router(
+ model_list=[
+ {
+ "model_name": "azure-router-model",
+ "litellm_params": {
+ "model": "azure/gpt-4",
+ "api_key": "fake-key",
+ "api_base": "https://westus.api.cognitive.microsoft.com",
+ },
+ }
+ ]
+ )
+ router._ageneric_api_call_with_fallbacks = AsyncMock()
+
+ managed_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="azure-router-model",
+ container_id="cfile_upstream_abc",
+ )
+
+ await router._init_containers_api_endpoints(
+ original_function=AsyncMock(),
+ custom_llm_provider="openai",
+ container_id=managed_id,
+ file_id="cfile_xyz",
+ )
+
+ router._ageneric_api_call_with_fallbacks.assert_called_once()
+ call_kw = router._ageneric_api_call_with_fallbacks.call_args.kwargs
+ assert call_kw["model"] == "azure-router-model"
+ assert call_kw["container_id"] == "cfile_upstream_abc"
+ assert call_kw["file_id"] == "cfile_xyz"
+ assert call_kw["custom_llm_provider"] == "azure"
diff --git a/tests/test_litellm/containers/test_azure_container_transformation.py b/tests/test_litellm/containers/test_azure_container_transformation.py
--- a/tests/test_litellm/containers/test_azure_container_transformation.py
+++ b/tests/test_litellm/containers/test_azure_container_transformation.py
@@ -11,6 +11,7 @@
import litellm
from litellm.llms.azure.containers.transformation import AzureContainerConfig
from litellm.llms.base_llm.containers.transformation import BaseContainerConfig
+from litellm.responses.utils import ResponsesAPIRequestUtils
from litellm.types.containers.main import (
ContainerFileListResponse,
ContainerListResponse,
@@ -518,3 +519,206 @@
c2 = _get_container_provider_config("azure_text")
assert type(c1) is type(c2)
assert isinstance(c1, AzureContainerConfig)
+
+ @pytest.mark.asyncio
+ async def test_proxy_process_request_preserves_managed_container_id(
+ self, monkeypatch
+ ):
+ from starlette.requests import Request
+
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ return {"id": "cfile_abc"}
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "GET",
+ "path": "/v1/containers/id/files/id/content",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+ fastapi_response = MagicMock()
+
+ await handler_factory._process_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ user_api_key_dict=MagicMock(),
+ route_type="alist_container_files",
+ path_params={"container_id": encoded_id},
+ )
+
+ assert captured["route_type"] == "alist_container_files"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["custom_llm_provider"] == "openai"
+ assert "model_id" not in captured["data"]
+ assert "api_base" not in captured["data"]
+
+ @pytest.mark.asyncio
+ async def test_regression_binary_file_request_routes_through_proxy_processor(
+ self, monkeypatch
+ ):
+ from fastapi import Response
+ from starlette.requests import Request
+
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ fastapi_response.headers["x-litellm-call-id"] = "call-123"
+ return b"csv-bytes"
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "GET",
+ "path": "/v1/containers/id/files/id/content",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+ fastapi_response = Response()
+
+ response = await handler_factory._process_binary_request(
+ request=request,
+ fastapi_response=fastapi_response,
+ container_id=encoded_id,
+ file_id="cfile_abc",
+ user_api_key_dict=MagicMock(),
+ )
+
+ assert captured["route_type"] == "aretrieve_container_file_content"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["file_id"] == "cfile_abc"
+ assert captured["data"]["custom_llm_provider"] == "openai"
+ assert response.status_code == 200
+ assert response.body == b"csv-bytes"
+ assert response.headers["x-litellm-call-id"] == "call-123"
+
+ @pytest.mark.asyncio
+ async def test_regression_multipart_upload_request_uses_provider_from_managed_id(
+ self, monkeypatch
+ ):
+ from starlette.requests import Request
+
+ from litellm.proxy.common_request_processing import (
+ ProxyBaseLLMRequestProcessing,
+ )
+ from litellm.proxy.common_utils import http_parsing_utils
+ from litellm.proxy.container_endpoints import handler_factory
+
+ encoded_id = ResponsesAPIRequestUtils._build_container_id(
+ custom_llm_provider="azure",
+ model_id="model_abc123",
+ container_id="cntr_123",
+ )
+ captured = {}
+
+ async def _mock_get_form_data(request):
+ return {"file": "ignored"}
+
+ async def _mock_convert_upload_files_to_file_data(form_data):
+ return {"file": [("data.csv", b"csv-bytes", "text/csv")]}
+
+ async def _mock_base_process_llm_request(
+ self,
+ request,
+ fastapi_response,
+ user_api_key_dict,
+ route_type,
+ **kwargs,
+ ):
+ captured["data"] = self.data
+ captured["route_type"] = route_type
+ return {"id": "cfile_abc"}
+
+ monkeypatch.setattr(
+ http_parsing_utils,
+ "get_form_data",
+ _mock_get_form_data,
+ )
+ monkeypatch.setattr(
+ http_parsing_utils,
+ "convert_upload_files_to_file_data",
+ _mock_convert_upload_files_to_file_data,
+ )
+ monkeypatch.setattr(
+ ProxyBaseLLMRequestProcessing,
+ "base_process_llm_request",
+ _mock_base_process_llm_request,
+ )
+
+ request = Request(
+ {
+ "type": "http",
+ "method": "POST",
+ "path": "/v1/containers/id/files",
+ "headers": [],
+ "query_string": b"",
+ }
+ )
+
+ await handler_factory._process_multipart_upload_request(
+ request=request,
+ fastapi_response=MagicMock(),
+ user_api_key_dict=MagicMock(),
+ route_type="aupload_container_file",
+ container_id=encoded_id,
+ )
+
+ assert captured["route_type"] == "aupload_container_file"
+ assert captured["data"]["container_id"] == encoded_id
+ assert captured["data"]["custom_llm_provider"] == "openai"You can send follow-ups to the cloud agent here.
Contributor
|
bugbot run |
mateo-berri
self-requested a review
April 30, 2026 17:55
Contributor
… empty Managed cntr_... IDs can be encoded with an empty model_id (e.g. streaming responses without router metadata, or target_model_names=[]). The previous guard only unwrapped when model_id was truthy, so the raw cntr_... token leaked to the upstream provider, which rejects it. Always swap in decoded["response_id"] when it differs from the input, and keep the model_id check only for deciding whether to fan out via _ageneric_api_call_with_fallbacks.
Contributor
Contributor
|
bugbot run |
… model_id A managed cntr_ ID can encode a non-OpenAI provider (e.g. azure) with an empty model_id when streaming events have no router model_info.id. The provider override was nested inside 'if model_id:', so such IDs unwrapped the container_id but kept custom_llm_provider='openai', routing the request to the wrong upstream. Hoist the override out of the model_id guard.
Contributor
Contributor
|
bugbot run |
Contributor
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 46ba48e. Configure here.
Contributor
Author
Contributor
|
bugbot run |
Contributor
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit a94ae62. Configure here.
fzowl
pushed a commit
to fzowl/litellm
that referenced
this pull request
Jun 24, 2026
…-file-routing-fix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes LIT-2456
Test plan
poetry run pytest tests/test_litellm/containers/test_azure_container_transformation.py -k \"decode_container_routing_metadata_includes_model_id or binary_file_request_uses_deployment_api_base\" -v/openai/responses/openai/containers/...URL for container file content requestscontainer_id+file_idNote
Medium Risk
Changes container file request routing and binary content handling to flow through the shared proxy/router pipeline and to decode LiteLLM-managed
cntr_...IDs for deployment selection; mistakes here could misroute requests or break file operations for non-OpenAI providers (notably Azure).Overview
Fixes container file endpoints (including binary
.../content) to route through the standard proxy/router pipeline so LiteLLM-managedcntr_...container IDs can be decoded for provider +model_idmetadata and the correct deployment credentials are applied (e.g. Azure regionalapi_base).Moves managed-ID decoding/routing responsibility into
Router._init_containers_api_endpoints(using_ageneric_api_call_with_fallbackswhenmodel_idis present), removes ad-hoc decoding inhandler_factory, and adds regression tests covering managed IDs with/withoutmodel_id, provider override, multipart upload, and binary content header/bytes passthrough.Reviewed by Cursor Bugbot for commit a94ae62. Bugbot is set up for automated code reviews on this repo. Configure here.