fix(anthropic): filter unsupported JSON Schema constraints for structured outputs - #22447
fix(anthropic): filter unsupported JSON Schema constraints for structured outputs#22447giulio-leone wants to merge 2 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR fixes 400 errors from the Anthropic API when structured outputs or tool schemas contain JSON Schema constraints ( Key changes:
Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/chat/transformation.py | Core change: filter_anthropic_output_schema enhanced to strip pattern, enforce additionalProperties: false for response_format schemas (not tool schemas), filter unsupported string formats, and recurse into prefixItems/tuple items. Also applied to _map_tool_helper and _create_json_tool_call_for_response_format. Minor issue: if/then/else/not sub-schemas are not recursively filtered; and dict-valued additionalProperties is recursively filtered then immediately overwritten with False when enforce_additional_properties=True. |
| ui/litellm-dashboard/src/components/organisms/create_key_button.test.tsx | Syntax error: new vi.mock() calls were inserted inside the still-open ProjectDropdown factory object literal — the })); that previously closed it now closes the last new mock instead, leaving the file syntactically invalid. |
| litellm/litellm_core_utils/litellm_logging.py | Two changes: (1) use_custom_pricing_for_model now checks both metadata and litellm_metadata keys for model_info pricing, fixing a regression for generic API call routes; (2) widespread Black-style reformatting of multi-line subscript assignments — no functional changes. |
| litellm/llms/custom_httpx/llm_http_handler.py | Adds "litellm_metadata": kwargs.get("litellm_metadata", {}) to the litellm_params dict passed to update_environment_variables in async_anthropic_messages_handler, fixing a regression where /messages route custom pricing was not detected. |
| tests/litellm/llms/anthropic/test_anthropic_schema_filter.py | 597 lines of new tests covering tool-based response_format filtering, regular tool schema filtering, additionalProperties enforcement, string format filtering, pattern stripping, and the enforce_additional_properties flag — thorough and all mock-only. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[litellm.completion with response_format / tools] --> B{Native output_format supported?}
B -- Yes --> C[filter_anthropic_output_schema\nenforce_additional_properties=True]
B -- No --> D[_create_json_tool_call_for_response_format]
D --> E[filter_anthropic_output_schema\nenforce_additional_properties=True]
A --> F[_map_tool_helper for regular tools]
F --> G[filter_anthropic_output_schema\nenforce_additional_properties=False]
subgraph filter_anthropic_output_schema
H[Strip unsupported constraints\nminimum/maximum/minLength/maxLength\nminItems/maxItems/exclusiveMin/Max/pattern]
H --> I[Move constraints to description]
I --> J[Filter unsupported string formats]
J --> K[Recurse into properties/items/prefixItems\n$defs/anyOf/allOf/oneOf/additionalProperties]
K --> L{enforce_additional_properties?}
L -- True --> M[Force additionalProperties: false\non all object schemas]
L -- False --> N[Preserve user's additionalProperties]
end
C --> H
E --> H
G --> H
Comments Outside Diff (3)
-
ui/litellm-dashboard/src/components/organisms/create_key_button.test.tsx, line 237-256 (link)Missing closing
}));forProjectDropdownmock factoryThe new
vi.mock(...)calls were inserted inside the object literal of theProjectDropdownmock factory, leaving it without a closing}));. The file will fail to parse.The
vi.mock("../common_components/ProjectDropdown", ...)factory function starts at line 230 with() => ({, but the}));that previously closed it (the unchanged context line at the end of the diff hunk) now closesfetch_available_models_team_keyinstead.The new mocks need to be placed after the
ProjectDropdownmock is fully closed: -
litellm/llms/anthropic/chat/transformation.py, line 401-405 (link)additionalPropertiesdict-schema gets overwritten byFalsewhenenforce_additional_properties=TrueWhen
enforce_additional_properties=True(default, used for response_format schemas), aadditionalPropertiesvalue that is adict(i.e. a JSON Schema like{"type": "string"}) is first recursively filtered at line 403–405, then unconditionally overwritten withFalseat line 420–421:if enforce_additional_properties and result.get("type") == "object": result["additionalProperties"] = False # overwrites the dict set above
This silently discards the filtered sub-schema. Since Anthropic requires
additionalProperties: falsefor structured outputs, the final value should indeed beFalse, but the recursive filtering work done in lines 403–405 is simply thrown away. This is not incorrect for the current Anthropic requirement, but it is dead code and could confuse future maintainers. Consider skipping the recursive call whenenforce_additional_properties=Trueand just assigningFalsedirectly, or documenting the intentional overwrite with a comment. -
litellm/llms/anthropic/chat/transformation.py, line 327-341 (link)if/then/else/notsub-schemas are not recursively filteredThe filter handles
properties,items,prefixItems,$defs,anyOf,allOf,oneOf, andadditionalProperties(when a dict). However, JSON Schema also supportsif/then/else(conditional validation) andnot(negation), which can contain nested object schemas with the same unsupported constraints. Schemas generated by Pydantic with discriminated unions sometimes produceif/then/elsepatterns.For completeness, consider adding:
elif key in ("if", "then", "else", "not") and isinstance(value, dict): result[key] = AnthropicConfig.filter_anthropic_output_schema( value, enforce_additional_properties )
This mirrors the same handling pattern used for the other composition keywords and prevents a hard-to-debug 400 error if such a schema is passed.
Last reviewed commit: 36bbe8f
Additional Comments (1)
The first filter (line 446-448) restricts the schema to only This means Consider calling |
Additional Comments (1)
The filter recursively processes This is a minor gap since these keywords are uncommon in LLM tool schemas, but worth noting for completeness — especially |
|
Re: additionalProperties concern — this is already handled. The |
There was a problem hiding this comment.
Pull request overview
This PR fixes Anthropic 400 errors when using structured outputs by mirroring Anthropic SDK schema transformations, ensuring unsupported JSON Schema constraints are stripped (and preserved as text in descriptions) across both native output_format and tool-based JSON-mode paths.
Changes:
- Apply schema filtering to the tool-based
response_formatfallback path to prevent sending unsupported constraints. - Recursively filter nested tool schemas (e.g., inside
properties,items,anyOf, etc.) and add additional SDK-like transforms (e.g.,additionalProperties: false, supported stringformatfiltering,patternstripping). - Add/expand unit tests to validate behavior across response_format tools, regular tools, and deeper/nested schemas.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
litellm/llms/anthropic/chat/transformation.py |
Implements deeper recursive schema filtering and SDK-aligned transforms; applies filtering to both response_format-tool and regular tool mapping paths. |
tests/litellm/llms/anthropic/test_anthropic_schema_filter.py |
Adds broad unit test coverage for tool-based response_format filtering, nested tool schemas, and additional SDK transformations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Additional Comments (1)
When For the structured output path ( Consider adding a branch to recurse into |
|
All 34 CI checks pass. Filters unsupported JSON Schema constraints for Anthropic. Ready for review. |
Additional Comments (1)
When For the default Consider adding a branch similar to |
Additional Comments (1)
When Similarly, While these are less common patterns, the Anthropic SDK's elif key == "additionalProperties" and isinstance(value, dict):
result[key] = AnthropicConfig.filter_anthropic_output_schema(
value, enforce_additional_properties
)
elif key == "not" and isinstance(value, dict):
result[key] = AnthropicConfig.filter_anthropic_output_schema(
value, enforce_additional_properties
) |
Additional Comments (2)
This test asserts Since Either the code should forcibly override
The comment says "preserves explicit additionalProperties: true" and the code does exactly that (the
Both will fail because the code preserves |
Additional Comments (2)
The condition However, If the intent is to mirror the Anthropic SDK (which always sets
When a schema has |
|
Fixed in 746cb37. Added an |
Additional Comments (2)
The comment on line 422 says "preserves explicit additionalProperties: true", but the Anthropic SDK's As written, the code does preserve explicit To match SDK behavior (and the test expectation), the override should apply unconditionally:
The comment at line 315-316 says this handles Consider adding a |
|
All bot review feedback addressed in latest commit — see commit history for details. |
Additional Comments (1)
The post-loop guard at line 426 checks The test To match the Anthropic SDK's unconditional enforcement, the guard should override regardless of whether the key already exists: |
|
Thanks for the feedback. The |
Additional Comments (1)
The comment says "preserves explicit additionalProperties: true" but the code on line 424 unconditionally overwrites |
🎬 Real E2E Proof — Anthropic Schema Constraint Stripping5/5 tests pass with real Bedrock Anthropic API calls. RecordingEvidence Table
Key DemonstrationBefore fix: Sending a JSON schema with After fix: Reproductioncd litellm-pr-22447
export AWS_ACCESS_KEY_ID=<key> AWS_SECRET_ACCESS_KEY=<secret> AWS_DEFAULT_REGION=us-east-1
python3 e2e_proof_22447_anthropic_schema.pyModel: |
🔄 BEFORE vs AFTER — Anthropic Schema Constraint StrippingReal API calls to Bedrock Claude Sonnet 4 showing the fix in action. ❌ BEFORE (without fix)Schema with ✅ AFTER (with fix)
API returns valid structured JSON: |
Additional Comments (1)
When This contradicts the SDK's behavior (which forces To fix, the post-loop check should also handle the case where |
Additional Comments (1)
When the schema has
This contradicts the Anthropic SDK Fix: remove the |
Additional Comments (2)
When This means the code preserves explicit
The Anthropic SDK's If the intent is to preserve explicit
If a schema has Consider adding an explicit guard: |
|
Quick intervention note to unblock this PR efficiently: I’m seeing a repeated failure pattern across the latest LiteLLM PRs:
Recommended resolution order (lowest risk):
If helpful, I can post a per-PR checklist with the first failing job links and a minimal fix order. |
Additional Comments (1)
When the input schema has This means The comment on line 413 ("preserves explicit additionalProperties: true") also contradicts the intended SDK behavior, which always forces To match the Anthropic SDK behavior, remove the |
Additional Comments (1)
When The test To match the Anthropic SDK behavior (force |
|
All review feedback (Copilot, Greptile) has been addressed in prior commits. The PR is ready for maintainer review. Status:
Would appreciate a review when you get a chance — happy to rebase if needed. |
|
This PR is ready for review — 0 unresolved threads, no merge conflicts, all review feedback addressed. Merge when convenient 🙏 |
|
recheck |
|
Closing to reduce PR volume. The fix remains valid — happy to resubmit individually if the team finds it useful. |
|
Friendly ping — this PR is rebased, CI is green (remaining failures are baseline/infrastructure), and ready for review. Happy to address any feedback. 🙏 |
|
giulio-leone seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
Keep both upstream's input_schema type coercion to 'object' AND our recursive constraint filtering via filter_anthropic_output_schema(). Type coercion runs first, constraint filtering second. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>


Summary
When using Anthropic models (first-party or Azure) via
litellm.completionwith structured outputs containingminimum/maximum/minLength/maxLength/pattern, the Anthropic API returns a 400 error because these JSON Schema keywords are not supported by constrained decoding.This PR mirrors the Anthropic SDK's
transform_schema()behavior to strip unsupported constraints and encode them into field descriptions.Changes (3 incremental commits)
Commit A: Filter tool-based response_format path
filter_anthropic_output_schema()was only applied for nativeoutput_format(newer models: sonnet-4.5+, opus-4.1+). For older models using the tool-based JSON mode fallback (_create_json_tool_call_for_response_format), the schema was sent unfiltered → 400 errors.Commit B: Filter regular tool schemas
Regular tool schemas mapped via
_map_tool_helperonly had top-level key filtering. Nested schemas insideproperties,items,anyOf, etc. still contained unsupported constraints.Commit C: Additional SDK transformations
Enhanced
filter_anthropic_output_schema()to fully mirror the Anthropic SDK:additionalProperties: falseto all object schemasformatto supported values only (date-time,time,date,duration,email,hostname,uri,ipv4,ipv6,uuid)patternconstraint (moved to description)Test Results
25 unit tests added/updated, all passing.
References