feat: support Azure OpenAI container API - #24685
Conversation
- Register AzureOpenAIContainerConfig in ProviderConfigManager and proxy handler_factory - Add AZURE_DEFAULT_CONTAINERS_API_VERSION constant - Update custom_llm_provider type hints to accept "azure" in containers/main.py - Update containers and container_files docs with Azure examples
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR registers a new Confidence Score: 4/5Not safe to merge as-is — the The PR correctly addresses the two previously flagged issues (hardcoded OpenAI cost provider and sub-resource URL malformation for most operations). However,
|
| Filename | Overview |
|---|---|
| litellm/llms/azure/containers/transformation.py | New AzureOpenAIContainerConfig class correctly overrides auth, URL construction, cost tracking, and most sub-resource URL methods, but the get_complete_url result (which embeds ?api-version=… in the URL) is incompatible with the string-concatenation _build_url used by upload_container_file via generic_container_handler, causing malformed URLs for that operation. |
| litellm/containers/main.py | Type hints updated from Literal["openai"] to Literal["openai", "azure"] across all container and container-file overloads; functional logic unchanged. |
| litellm/proxy/container_endpoints/handler_factory.py | Clean registration of AzureOpenAIContainerConfig alongside the existing OpenAI case; no regressions to the OpenAI path. |
| litellm/utils.py | AzureOpenAIContainerConfig registered in ProviderConfigManager.get_provider_container_config() under LlmProviders.AZURE; mirrors the existing OpenAI registration pattern correctly. |
| litellm/constants.py | Adds AZURE_DEFAULT_CONTAINERS_API_VERSION constant configurable via env var, defaulting to "preview"; consistent with the existing AZURE_DEFAULT_RESPONSES_API_VERSION pattern. |
| tests/test_litellm/containers/test_azure_container_transformation.py | 17 mock-only unit tests covering inheritance, auth headers, URL construction, and transform methods; the end-to-end upload_container_file path (which uses _build_url) is not exercised, so the malformed-URL regression is not caught. |
Sequence Diagram
sequenceDiagram
participant U as User
participant M as containers/main.py
participant PCM as ProviderConfigManager
participant AZCONF as AzureOpenAIContainerConfig
participant BH as BaseLLMHTTPHandler
participant GCH as GenericContainerHandler
participant AZ as Azure OpenAI API
Note over U,AZ: Container CRUD (create/list/retrieve/delete)
U->>M: create_container(custom_llm_provider="azure")
M->>PCM: get_provider_container_config(AZURE)
PCM-->>M: AzureOpenAIContainerConfig
M->>BH: container_create_handler(config)
BH->>AZCONF: get_complete_url() → URL+?api-version=preview
BH->>AZCONF: validate_environment() → {api-key: ...}
BH->>AZCONF: transform_container_create_response()
BH->>AZ: POST /openai/v1/containers?api-version=preview
AZ-->>U: ContainerObject
Note over U,AZ: list_container_files (works correctly)
U->>M: list_container_files(container_id, provider="azure")
M->>BH: container_file_list_handler(config)
BH->>AZCONF: transform_container_file_list_request() [overridden]
BH->>AZ: GET /openai/v1/containers/cntr_123/files?api-version=preview
AZ-->>U: ContainerFileListResponse
Note over U,AZ: upload_container_file — BROKEN for Azure
U->>M: upload_container_file(container_id, file, provider="azure")
M->>GCH: generic_container_handler.handle()
GCH->>AZCONF: get_complete_url() → URL+?api-version=preview
GCH->>GCH: _build_url() [string concat → malformed URL]
Note right of GCH: URL becomes …/containers?api-version=preview/cntr_123/files
GCH->>AZ: POST malformed URL → 4xx error
Reviews (3): Last reviewed commit: "fix: handle Azure URL query params in co..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| class AzureOpenAIContainerConfig(OpenAIContainerConfig): | ||
| """Azure OpenAI Container Config. | ||
|
|
||
| Inherits from OpenAIContainerConfig and overrides only Azure-specific methods. | ||
| Request/response transformations are identical to OpenAI. | ||
| """ | ||
|
|
||
| def get_complete_url( | ||
| self, | ||
| api_base: Optional[str], | ||
| litellm_params: dict, | ||
| ) -> str: | ||
| """Get the complete URL for Azure container API. | ||
|
|
||
| Constructs Azure-specific URLs like: | ||
| https://{resource}.openai.azure.com/openai/v1/containers?api-version=xxx | ||
| """ | ||
| return BaseAzureLLM._get_base_azure_url( | ||
| api_base=api_base, | ||
| litellm_params=litellm_params, | ||
| route="/openai/v1/containers", | ||
| default_api_version=AZURE_DEFAULT_CONTAINERS_API_VERSION, | ||
| ) | ||
|
|
||
| def validate_environment( | ||
| self, | ||
| headers: dict, | ||
| api_key: Optional[str] = None, | ||
| ) -> dict: | ||
| """Validate and set up Azure authentication headers. | ||
|
|
||
| Uses Azure api-key header (not Bearer token like OpenAI). | ||
| """ | ||
| # Create a GenericLiteLLMParams with the api_key if provided | ||
| litellm_params = GenericLiteLLMParams(api_key=api_key) if api_key else None | ||
|
|
||
| # Azure uses BaseAzureLLM's validation which handles api-key header | ||
| return BaseAzureLLM._base_validate_azure_environment( | ||
| headers=headers, litellm_params=litellm_params | ||
| ) |
There was a problem hiding this comment.
AzureOpenAIContainerConfig inherits transform_container_create_response from OpenAIContainerConfig without overriding it. That inherited method calls:
container_cost = StandardBuiltInToolCostTracking.get_cost_for_code_interpreter(
sessions=1,
provider="openai", # ← hardcoded OpenAI, even when called on Azure
)get_cost_for_code_interpreter uses the provider argument to look up pricing from the model-cost map (azure/container vs openai/container). Running under an Azure provider will silently attribute every container-creation cost to "openai", and if the two providers' per-session prices ever diverge the calculated cost will be wrong.
Override the method in AzureOpenAIContainerConfig to pass provider="azure":
def transform_container_create_response(
self,
raw_response: httpx.Response,
logging_obj: LiteLLMLoggingObj,
) -> ContainerObject:
"""Transform the Azure OpenAI container creation response."""
response_data = raw_response.json()
container_obj = ContainerObject(**response_data)
container_cost = StandardBuiltInToolCostTracking.get_cost_for_code_interpreter(
sessions=1,
provider="azure", # ← correct provider
)
if not hasattr(container_obj, "_hidden_params") or container_obj._hidden_params is None:
container_obj._hidden_params = {}
if "additional_headers" not in container_obj._hidden_params:
container_obj._hidden_params["additional_headers"] = {}
container_obj._hidden_params["additional_headers"][
"llm_provider-x-litellm-response-cost"
] = container_cost
return container_obj
|
|
Override transform_container_create_response in AzureOpenAIContainerConfig to pass provider="azure" instead of inheriting the hardcoded "openai" value. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| def get_complete_url( | ||
| self, | ||
| api_base: Optional[str], | ||
| litellm_params: dict, | ||
| ) -> str: | ||
| """Get the complete URL for Azure container API. | ||
|
|
||
| Constructs Azure-specific URLs like: | ||
| https://{resource}.openai.azure.com/openai/v1/containers?api-version=xxx | ||
| """ | ||
| return BaseAzureLLM._get_base_azure_url( | ||
| api_base=api_base, | ||
| litellm_params=litellm_params, | ||
| route="/openai/v1/containers", | ||
| default_api_version=AZURE_DEFAULT_CONTAINERS_API_VERSION, | ||
| ) |
There was a problem hiding this comment.
get_complete_url() returns a URL with ?api-version=preview already embedded as a query parameter (e.g. https://resource.openai.azure.com/openai/v1/containers?api-version=preview). This URL is then passed as api_base into the inherited OpenAI file-operation transform methods, which append path segments via naïve string concatenation:
# From OpenAIContainerConfig (inherited, not overridden):
url = f"{api_base.rstrip('/')}/{container_id}/files"
# → "https://.../containers?api-version=preview/cntr_123/files"
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# cntr_123 and /files end up as part
# of the api-version query value!Because rstrip('/') only strips trailing slashes, the resulting URL has the container_id and file path embedded inside the query string value instead of in the URL path. This affects all sub-resource operations:
transform_container_retrieve_request→f"{api_base.rstrip('/')}/{container_id}"transform_container_delete_request→ same patterntransform_container_file_list_request→f"{api_base.rstrip('/')}/{container_id}/files"transform_container_file_content_request→f"{api_base.rstrip('/')}/{container_id}/files/{file_id}/content"
Only the container list (url = api_base) and create (POST body, no path appending) operations are unaffected.
The fix is to override these four methods in AzureOpenAIContainerConfig and strip the query string before appending the path, then re-attach it. Or more simply, build the path URL first and then use httpx.URL(...).copy_with(params=...) to attach api-version cleanly.
The tests in test_azure_container_transformation.py do not catch this because they pass hardcoded api_base values without the ?api-version=... query string (e.g. api_base = "https://my-resource.openai.azure.com/openai/v1/containers"), bypassing the get_complete_url() → transform pipeline entirely.
Override transform methods that append path segments to api_base, using URL parsing to insert subpaths before query parameters. Without this, Azure's ?api-version=... query param causes malformed URLs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Looks like another PR with this functionality was merged in #25287 |
|
thanks @david-kuehn , closes this PR. |
|
More work on this going on here: #26402 |
3 participants
Relevant issues
Fixes #22996
Pre-Submission checklist
tests/test_litellm/directorymake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
🆕 New Feature
Changes
Summary
LiteLLM's Container endpoints currently only route to OpenAI. When using Code Interpreter via Azure OpenAI, container file retrieval requests fail because LiteLLM forwards them to OpenAI instead of Azure. This PR registers the existing
AzureOpenAIContainerConfigclass so that all container operations (CRUD + file operations) work with Azure OpenAI.Code Changes
litellm/constants.py— AddAZURE_DEFAULT_CONTAINERS_API_VERSIONconstantlitellm/utils.py— RegisterAzureOpenAIContainerConfiginProviderConfigManager.get_provider_container_config()litellm/proxy/container_endpoints/handler_factory.py— Register Azure in_get_container_provider_config()litellm/containers/main.py— Updatecustom_llm_providertype hints fromLiteral["openai"]toLiteral["openai", "azure"]Tests
tests/test_litellm/containers/test_azure_container_transformation.py— 17 tests covering Azure container config inheritance, authentication headers, URL construction, request/response transformations, and provider registrationDocs
docs/my-website/docs/containers.md— Add Azure to supported providers, add OpenAI/Azure tab examplesdocs/my-website/docs/container_files.md— Add Azure to supported providers, add OpenAI/Azure tab examples