fix(azure): preserve content_policy_violation error details from Azure OpenAI#20883
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile OverviewGreptile SummaryThis PR improves Azure OpenAI content-policy violation handling, especially for Images/DALL-E flows.
Confidence Score: 4/5
|
| Filename | Overview |
|---|---|
| litellm/litellm_core_utils/exception_mapping_utils.py | Refines Azure content-policy detection by removing broad substring matching and additionally checking nested inner_error.code for ResponsibleAIPolicyViolation. Main concern: nested-body shape handling is still limited to body['error'].* variants. |
| litellm/llms/azure/azure.py | Updates Azure DALL-E polling failure handling to preserve structured error payloads by setting body on AzureOpenAIError and using a clearer message extracted from the payload. |
| tests/test_litellm/llms/azure/test_azure_exception_mapping.py | Adds regression tests for Azure Images content-policy violations and polling error body preservation. Issue: contains a stray debug print() that should be removed to keep tests quiet. |
Sequence Diagram
sequenceDiagram
participant Caller as LiteLLM caller
participant AzurePy as litellm/llms/azure/azure.py
participant HTTP as HTTPHandler/httpx
participant ExMap as exception_mapping_utils.exception_type
participant AzureMap as AzureOpenAIExceptionMapping
Caller->>AzurePy: aimage_generation()/sync image generation
AzurePy->>HTTP: POST submit image generation
HTTP-->>AzurePy: 202 + operation-location
loop Poll until done
AzurePy->>HTTP: GET operation-location
HTTP-->>AzurePy: {status: succeeded|failed, ...}
end
alt status == failed
AzurePy->>AzurePy: raise AzureOpenAIError(message, body=error_data)
Caller->>ExMap: exception_type(original_exception)
ExMap->>ExMap: extract body['error'].code
ExMap->>ExMap: check body['error'].inner_error.code
alt content policy detected
ExMap->>AzureMap: create_content_policy_violation_error(...)
AzureMap-->>Caller: ContentPolicyViolationError (preserves provider fields)
else other bad request
ExMap-->>Caller: BadRequestError
end
else status == succeeded
AzurePy-->>Caller: Image result
end
| azure_error_code = body_dict["error"].get("code") # type: ignore[index] | ||
| # Also check inner_error for | ||
| # ResponsibleAIPolicyViolation which indicates a | ||
| # content policy violation even when the top-level |
There was a problem hiding this comment.
Inner error check too narrow
exception_type() only inspects body["error"]["inner_error"] / innererror for ResponsibleAIPolicyViolation. Azure error payloads can also include the inner object under the top-level body (e.g. body["inner_error"]) or use different casing, so those cases will still fall through to BadRequestError and drop structured details. This matters when the exception body mirrors the Images API error without the error wrapper.
Also appears in: tests/test_litellm/llms/azure/test_azure_exception_mapping.py fixtures only cover the body["error"]["inner_error"] and body["innererror"] shapes, so the missing shapes are untested.
Additional Comments (1)
This |
[Infra] UI - E2E Tests: Key Delete, Regenerate, and Update TPM/RPM Limits
* fix(cloudzero): update CBF field mappings per LIT-1907 Phase 1 field updates for CloudZero integration: ADD/UPDATE: - resource/account: Send concat(api_key_alias, '|', api_key_prefix) - resource/service: Send model_group instead of service_type - resource/usage_family: Send provider instead of hardcoded 'llm-usage' - action/operation: NEW - Send team_id - resource/id: Send model name instead of CZRN - resource/tag:organization_alias: Add if exists - resource/tag:project_alias: Add if exists - resource/tag:user_alias: Add if exists REMOVE: - resource/tag:total_tokens: Removed - resource/tag:team_id: Removed (team_id now in action/operation) Fixes LIT-1907 * Update litellm/integrations/cloudzero/transform.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: define api_key_alias variable, update CBFRecord docstring - Fix F821 lint error: api_key_alias was used but not defined - Update CBFRecord docstring to reflect LIT-1907 field mappings - Remove unused Optional import --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…ies, Viewing how many keys, teams it applies on (BerriAI#20904) * init schema with TAGS * ui: add policy test * resolvePoliciesCall * add_policy_sources_to_metadata + headers * types Policy * preview Impact * def _describe_match_reason( * match based on TAGs * TestTagBasedAttachments * test fixes * add policy_resolve_router * add_guardrails_from_policy_engine * TestMatchAttribution * refactor * fix * fix: address Greptile review feedback on policy resolve endpoints - Track unnamed keys/teams as separate counts instead of inflating affected_keys_count with duplicate "(unnamed key)" placeholders. Added unnamed_keys_count and unnamed_teams_count to response. - Push alias pattern matching to DB via _build_alias_where() which converts exact patterns to Prisma "in" and suffix wildcards to "startsWith" filters. - Gate sync_policies_from_db/sync_attachments_from_db behind force_sync query param (default false) to avoid 2 DB round-trips on every /policies/resolve request. - Remove worktree-only conftest.py that cleared sys.modules at import time — no longer needed since code moved to main repo. - Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: eliminate duplicate DB queries and fix header delimiter ambiguity - Fetch teams table once in estimate_attachment_impact and reuse for both tag-based and alias-based lookups (was querying teams twice when both tag_patterns and team_patterns were provided). - Convert tag/team filter functions from async DB queries to sync filters that operate on pre-fetched data (_filter_keys_by_tags, _filter_teams_by_tags). - Fix comma ambiguity in x-litellm-policy-sources header: use '; ' as entry delimiter since matched_via values can contain commas. - Use '+' as the within-value separator in matched_via reason strings (e.g. "tag:healthcare+team:health-team") to avoid conflict with header delimiters. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update litellm/proxy/policy_engine/policy_resolve_endpoints.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* init schema with TAGS * ui: add policy test * resolvePoliciesCall * add_policy_sources_to_metadata + headers * types Policy * preview Impact * def _describe_match_reason( * match based on TAGs * TestTagBasedAttachments * test fixes * add policy_resolve_router * add_guardrails_from_policy_engine * TestMatchAttribution * refactor * fix * fix: address Greptile review feedback on policy resolve endpoints - Track unnamed keys/teams as separate counts instead of inflating affected_keys_count with duplicate "(unnamed key)" placeholders. Added unnamed_keys_count and unnamed_teams_count to response. - Push alias pattern matching to DB via _build_alias_where() which converts exact patterns to Prisma "in" and suffix wildcards to "startsWith" filters. - Gate sync_policies_from_db/sync_attachments_from_db behind force_sync query param (default false) to avoid 2 DB round-trips on every /policies/resolve request. - Remove worktree-only conftest.py that cleared sys.modules at import time — no longer needed since code moved to main repo. - Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: eliminate duplicate DB queries and fix header delimiter ambiguity - Fetch teams table once in estimate_attachment_impact and reuse for both tag-based and alias-based lookups (was querying teams twice when both tag_patterns and team_patterns were provided). - Convert tag/team filter functions from async DB queries to sync filters that operate on pre-fetched data (_filter_keys_by_tags, _filter_teams_by_tags). - Fix comma ambiguity in x-litellm-policy-sources header: use '; ' as entry delimiter since matched_via values can contain commas. - Use '+' as the within-value separator in matched_via reason strings (e.g. "tag:healthcare+team:health-team") to avoid conflict with header delimiters. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs v1 guide with UI imgs * docs fix --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add support for Alibaba Cloud's Qwen3-Max model with: - 258K input tokens, 65K output tokens - Tiered pricing based on context window usage (0-32K, 32K-128K, 128K-252K) - Function calling and tool choice support - Reasoning capabilities enabled Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
…-6-structured-outputs feat: enable support for non-tool structured outputs on Anthropic Claude Opus 4.5 and 4.6 (use `output_format` param)
[Feature] UI - Login: New Login With SSO Button
…odal [Feature] UI - Navbar: Option to hide Usage Popup
…e OpenAI Closes BerriAI#20811 Azure OpenAI returns rich error payloads for content policy violations (inner_error with ResponsibleAIPolicyViolation, content_filter_results, revised_prompt). Previously these details were lost when: 1. The top-level error code was not "content_policy_violation" but the inner_error.code was "ResponsibleAIPolicyViolation" -- the structured check only examined the top-level code. 2. The DALL-E image generation polling path stringified the error JSON into the message field instead of setting the structured body, making it impossible for exception_type() to extract error details. 3. The string-based fallback detector used "invalid_request_error" as a content-policy indicator, which is too broad and could misclassify regular bad-request errors. Changes: - exception_mapping_utils.py: Check inner_error.code for ResponsibleAIPolicyViolation when top-level code is not content_policy_violation. Replace overly broad "invalid_request_error" string match with specific Azure safety-system messages. - azure.py: Set structured body on AzureOpenAIError in both async and sync DALL-E polling paths so exception_type() can inspect error details. - test_azure_exception_mapping.py: Add regression tests covering the exact error payloads from issue BerriAI#20811. - Fix pre-existing lint: duplicate PerplexityResponsesConfig dict key, unused RouteChecks top-level import.
08ca908 to
00328ba
Compare
ce421df
into
BerriAI:litellm_oss_staging_02_11_2026
Relevant issues
Closes #20811
What this PR does
Azure OpenAI returns rich error payloads for content policy violations including
inner_errorwithResponsibleAIPolicyViolation,content_filter_results, andrevised_prompt. These details were being lost and replaced by a generic HTTP 400 response.Root causes fixed
Incomplete
azure_error_codeextraction: The structured check inexception_mapping_utils.pyonly examined the top-levelbody["error"]["code"]. When Azure returns a generic top-level code but nestsResponsibleAIPolicyViolationininner_error.code, the content-policy path was skipped entirely, falling through to genericBadRequestErrorhandlers that discard error details.Image generation polling path loses structured body: The DALL-E polling path in
azure.pystringified the entire error JSON into themessagefield ofAzureOpenAIErrorwithout setting thebodyattribute. This made it impossible forexception_type()to extract structured error details.Overly broad string-based detection:
is_azure_content_policy_violation_error()used"invalid_request_error"as a content-policy indicator, which could misclassify regular bad-request errors. Replaced with specific Azure safety-system message patterns.Changes
litellm/litellm_core_utils/exception_mapping_utils.pyinner_error.codeforResponsibleAIPolicyViolationwhen top-level code differs; replace broad string match with specific patternslitellm/llms/azure/azure.pybodyonAzureOpenAIErrorin both async and sync DALL-E polling pathstests/test_litellm/llms/azure/test_azure_exception_mapping.pyPre-Submission checklist
tests/litellm/directorymake test-unit