Skip to content

fix(azure): preserve content_policy_violation error details from Azure OpenAI#20883

Merged
krrishdholakia merged 21 commits intoBerriAI:litellm_oss_staging_02_11_2026from
skylarkoo7:fix-20811-azure-error-detail-preservation
Feb 11, 2026
Merged

fix(azure): preserve content_policy_violation error details from Azure OpenAI#20883
krrishdholakia merged 21 commits intoBerriAI:litellm_oss_staging_02_11_2026from
skylarkoo7:fix-20811-azure-error-detail-preservation

Conversation

@skylarkoo7
Copy link
Contributor

Relevant issues

Closes #20811

What this PR does

Azure OpenAI returns rich error payloads for content policy violations including inner_error with ResponsibleAIPolicyViolation, content_filter_results, and revised_prompt. These details were being lost and replaced by a generic HTTP 400 response.

Root causes fixed

  1. Incomplete azure_error_code extraction: The structured check in exception_mapping_utils.py only examined the top-level body["error"]["code"]. When Azure returns a generic top-level code but nests ResponsibleAIPolicyViolation in inner_error.code, the content-policy path was skipped entirely, falling through to generic BadRequestError handlers that discard error details.

  2. Image generation polling path loses structured body: The DALL-E polling path in azure.py stringified the entire error JSON into the message field of AzureOpenAIError without setting the body attribute. This made it impossible for exception_type() to extract structured error details.

  3. Overly broad string-based detection: is_azure_content_policy_violation_error() used "invalid_request_error" as a content-policy indicator, which could misclassify regular bad-request errors. Replaced with specific Azure safety-system message patterns.

Changes

File Change
litellm/litellm_core_utils/exception_mapping_utils.py Check inner_error.code for ResponsibleAIPolicyViolation when top-level code differs; replace broad string match with specific patterns
litellm/llms/azure/azure.py Set structured body on AzureOpenAIError in both async and sync DALL-E polling paths
tests/test_litellm/llms/azure/test_azure_exception_mapping.py Add 5 regression tests covering the exact error payloads from issue #20811

Pre-Submission checklist

  • I have Added testing in the tests/litellm/ directory
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

@vercel
Copy link

vercel bot commented Feb 10, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 11, 2026 6:44am

Request Review

@CLAassistant
Copy link

CLAassistant commented Feb 10, 2026

CLA assistant check
All committers have signed the CLA.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 10, 2026

Greptile Overview

Greptile Summary

This PR improves Azure OpenAI content-policy violation handling, especially for Images/DALL-E flows.

  • In litellm_core_utils/exception_mapping_utils.py, Azure content-policy detection now prefers structured error payloads and can treat error.inner_error.code == ResponsibleAIPolicyViolation as a content-policy violation even when the top-level code is generic. It also narrows string-based detection to more specific Azure safety-system phrases.
  • In litellm/llms/azure/azure.py, the DALL-E operation polling failure path now raises AzureOpenAIError with a structured body (instead of only a JSON-stringified message), enabling exception_type() to preserve and route rich error details.
  • Adds regression tests to ensure these Azure Images payload shapes map to ContentPolicyViolationError and that nested policy fields are preserved.

Confidence Score: 4/5

  • Generally safe to merge after addressing small correctness/test hygiene issues.
  • The main behavioral changes are narrowly scoped and backed by targeted regression tests, but there are still unhandled Azure error-body shape variants that will deterministically bypass the new inner_error-based detection, plus a stray debug print in tests.
  • litellm/litellm_core_utils/exception_mapping_utils.py; tests/test_litellm/llms/azure/test_azure_exception_mapping.py

Important Files Changed

Filename Overview
litellm/litellm_core_utils/exception_mapping_utils.py Refines Azure content-policy detection by removing broad substring matching and additionally checking nested inner_error.code for ResponsibleAIPolicyViolation. Main concern: nested-body shape handling is still limited to body['error'].* variants.
litellm/llms/azure/azure.py Updates Azure DALL-E polling failure handling to preserve structured error payloads by setting body on AzureOpenAIError and using a clearer message extracted from the payload.
tests/test_litellm/llms/azure/test_azure_exception_mapping.py Adds regression tests for Azure Images content-policy violations and polling error body preservation. Issue: contains a stray debug print() that should be removed to keep tests quiet.

Sequence Diagram

sequenceDiagram
  participant Caller as LiteLLM caller
  participant AzurePy as litellm/llms/azure/azure.py
  participant HTTP as HTTPHandler/httpx
  participant ExMap as exception_mapping_utils.exception_type
  participant AzureMap as AzureOpenAIExceptionMapping

  Caller->>AzurePy: aimage_generation()/sync image generation
  AzurePy->>HTTP: POST submit image generation
  HTTP-->>AzurePy: 202 + operation-location
  loop Poll until done
    AzurePy->>HTTP: GET operation-location
    HTTP-->>AzurePy: {status: succeeded|failed, ...}
  end
  alt status == failed
    AzurePy->>AzurePy: raise AzureOpenAIError(message, body=error_data)
    Caller->>ExMap: exception_type(original_exception)
    ExMap->>ExMap: extract body['error'].code
    ExMap->>ExMap: check body['error'].inner_error.code
    alt content policy detected
      ExMap->>AzureMap: create_content_policy_violation_error(...)
      AzureMap-->>Caller: ContentPolicyViolationError (preserves provider fields)
    else other bad request
      ExMap-->>Caller: BadRequestError
    end
  else status == succeeded
    AzurePy-->>Caller: Image result
  end
Loading

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Comment on lines 2064 to +2067
azure_error_code = body_dict["error"].get("code") # type: ignore[index]
# Also check inner_error for
# ResponsibleAIPolicyViolation which indicates a
# content policy violation even when the top-level
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inner error check too narrow

exception_type() only inspects body["error"]["inner_error"] / innererror for ResponsibleAIPolicyViolation. Azure error payloads can also include the inner object under the top-level body (e.g. body["inner_error"]) or use different casing, so those cases will still fall through to BadRequestError and drop structured details. This matters when the exception body mirrors the Images API error without the error wrapper.

Also appears in: tests/test_litellm/llms/azure/test_azure_exception_mapping.py fixtures only cover the body["error"]["inner_error"] and body["innererror"] shapes, so the missing shapes are untested.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 10, 2026

Additional Comments (1)

tests/test_litellm/llms/azure/test_azure_exception_mapping.py
Debug print in test

This print("got provider_specific_fields=", ...) will pollute test output (and may fail in setups that treat unexpected stdout as noise). It looks like leftover debugging and should be removed.

yuneng-jiang and others added 16 commits February 10, 2026 16:30
[Infra] UI - E2E Tests: Key Delete, Regenerate, and Update TPM/RPM Limits
* fix(cloudzero): update CBF field mappings per LIT-1907

Phase 1 field updates for CloudZero integration:

ADD/UPDATE:
- resource/account: Send concat(api_key_alias, '|', api_key_prefix)
- resource/service: Send model_group instead of service_type
- resource/usage_family: Send provider instead of hardcoded 'llm-usage'
- action/operation: NEW - Send team_id
- resource/id: Send model name instead of CZRN
- resource/tag:organization_alias: Add if exists
- resource/tag:project_alias: Add if exists
- resource/tag:user_alias: Add if exists

REMOVE:
- resource/tag:total_tokens: Removed
- resource/tag:team_id: Removed (team_id now in action/operation)

Fixes LIT-1907

* Update litellm/integrations/cloudzero/transform.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix: define api_key_alias variable, update CBFRecord docstring

- Fix F821 lint error: api_key_alias was used but not defined
- Update CBFRecord docstring to reflect LIT-1907 field mappings
- Remove unused Optional import

---------

Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…ies, Viewing how many keys, teams it applies on (BerriAI#20904)

* init schema with TAGS

* ui: add policy test

* resolvePoliciesCall

* add_policy_sources_to_metadata + headers

* types Policy

* preview Impact

* def _describe_match_reason(

* match based on TAGs

* TestTagBasedAttachments

* test fixes

* add policy_resolve_router

* add_guardrails_from_policy_engine

* TestMatchAttribution

* refactor

* fix

* fix: address Greptile review feedback on policy resolve endpoints

- Track unnamed keys/teams as separate counts instead of inflating
  affected_keys_count with duplicate "(unnamed key)" placeholders.
  Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
  converts exact patterns to Prisma "in" and suffix wildcards to
  "startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
  force_sync query param (default false) to avoid 2 DB round-trips
  on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
  time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: eliminate duplicate DB queries and fix header delimiter ambiguity

- Fetch teams table once in estimate_attachment_impact and reuse for
  both tag-based and alias-based lookups (was querying teams twice when
  both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
  filters that operate on pre-fetched data (_filter_keys_by_tags,
  _filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
  as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
  (e.g. "tag:healthcare+team:health-team") to avoid conflict with
  header delimiters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update litellm/proxy/policy_engine/policy_resolve_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* init schema with TAGS

* ui: add policy test

* resolvePoliciesCall

* add_policy_sources_to_metadata + headers

* types Policy

* preview Impact

* def _describe_match_reason(

* match based on TAGs

* TestTagBasedAttachments

* test fixes

* add policy_resolve_router

* add_guardrails_from_policy_engine

* TestMatchAttribution

* refactor

* fix

* fix: address Greptile review feedback on policy resolve endpoints

- Track unnamed keys/teams as separate counts instead of inflating
  affected_keys_count with duplicate "(unnamed key)" placeholders.
  Added unnamed_keys_count and unnamed_teams_count to response.
- Push alias pattern matching to DB via _build_alias_where() which
  converts exact patterns to Prisma "in" and suffix wildcards to
  "startsWith" filters.
- Gate sync_policies_from_db/sync_attachments_from_db behind
  force_sync query param (default false) to avoid 2 DB round-trips
  on every /policies/resolve request.
- Remove worktree-only conftest.py that cleared sys.modules at import
  time — no longer needed since code moved to main repo.
- Rename MAX_ESTIMATE_IMPACT_ROWS → MAX_POLICY_ESTIMATE_IMPACT_ROWS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: eliminate duplicate DB queries and fix header delimiter ambiguity

- Fetch teams table once in estimate_attachment_impact and reuse for
  both tag-based and alias-based lookups (was querying teams twice when
  both tag_patterns and team_patterns were provided).
- Convert tag/team filter functions from async DB queries to sync
  filters that operate on pre-fetched data (_filter_keys_by_tags,
  _filter_teams_by_tags).
- Fix comma ambiguity in x-litellm-policy-sources header: use '; '
  as entry delimiter since matched_via values can contain commas.
- Use '+' as the within-value separator in matched_via reason strings
  (e.g. "tag:healthcare+team:health-team") to avoid conflict with
  header delimiters.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs v1 guide with UI imgs

* docs fix

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Add support for Alibaba Cloud's Qwen3-Max model with:
- 258K input tokens, 65K output tokens
- Tiered pricing based on context window usage (0-32K, 32K-128K, 128K-252K)
- Function calling and tool choice support
- Reasoning capabilities enabled

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
…-6-structured-outputs

feat: enable support for non-tool structured outputs on Anthropic Claude Opus 4.5 and 4.6 (use `output_format` param)
[Feature] UI - Login: New Login With SSO Button
…odal

[Feature] UI - Navbar: Option to hide Usage Popup
…e OpenAI

Closes BerriAI#20811

Azure OpenAI returns rich error payloads for content policy violations
(inner_error with ResponsibleAIPolicyViolation, content_filter_results,
revised_prompt). Previously these details were lost when:

1. The top-level error code was not "content_policy_violation" but the
   inner_error.code was "ResponsibleAIPolicyViolation" -- the structured
   check only examined the top-level code.

2. The DALL-E image generation polling path stringified the error JSON
   into the message field instead of setting the structured body, making
   it impossible for exception_type() to extract error details.

3. The string-based fallback detector used "invalid_request_error" as a
   content-policy indicator, which is too broad and could misclassify
   regular bad-request errors.

Changes:
- exception_mapping_utils.py: Check inner_error.code for
  ResponsibleAIPolicyViolation when top-level code is not
  content_policy_violation. Replace overly broad "invalid_request_error"
  string match with specific Azure safety-system messages.
- azure.py: Set structured body on AzureOpenAIError in both async and
  sync DALL-E polling paths so exception_type() can inspect error details.
- test_azure_exception_mapping.py: Add regression tests covering the
  exact error payloads from issue BerriAI#20811.
- Fix pre-existing lint: duplicate PerplexityResponsesConfig dict key,
  unused RouteChecks top-level import.
@skylarkoo7 skylarkoo7 force-pushed the fix-20811-azure-error-detail-preservation branch from 08ca908 to 00328ba Compare February 11, 2026 06:43
@krrishdholakia krrishdholakia changed the base branch from main to litellm_oss_staging_02_11_2026 February 11, 2026 06:46
@krrishdholakia krrishdholakia merged commit ce421df into BerriAI:litellm_oss_staging_02_11_2026 Feb 11, 2026
6 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Reopen [19328] Azure OpenAI error details (content_policy_violation) are replaced by generic HTTP 400 in LiteLLM (v1.81.3)