fix(anthropic): place strict flag at tool level, not inside input_schema - #27542
fix(anthropic): place strict flag at tool level, not inside input_schema#27542Jwrede wants to merge 3 commits into
Conversation
[Infra] Promote Internal Staging to main
Anthropic enforces strict tool use only when `strict` is set at the tool top level (sibling of name/input_schema). LiteLLM was placing it inside input_schema where Anthropic silently ignores it, so callers thought strict was engaged but the model emitted unconstrained tool arguments. - Remove `strict` from AnthropicInputSchema (not part of the schema) - Add `strict` to AnthropicMessagesTool (where Anthropic expects it) - Extract strict from tool.function.strict (OpenAI canonical) or from parameters.strict (legacy), and set it on the tool object Fixes BerriAI#27490
Greptile SummaryThis PR fixes the placement of the
Confidence Score: 4/5Safe to merge; the fix is straightforward and well-tested, with one minor edge case in the fallback logic worth addressing. The litellm/llms/anthropic/chat/transformation.py line 679 — the strict-extraction expression.
|
| Filename | Overview |
|---|---|
| litellm/llms/anthropic/chat/transformation.py | Extracts strict from either tool.function.strict or parameters.strict and places it at the tool top level; minor or-based falsy evaluation could override an explicit strict=False in an edge case. |
| litellm/types/llms/anthropic.py | Removes strict from AnthropicInputSchema (where it doesn't belong per Anthropic's spec) and adds it to AnthropicMessagesTool (correct location); clean and correct type change. |
| tests/test_litellm/llms/anthropic/chat/test_anthropic_chat_transformation.py | Adds 4 unit tests covering function-level strict, parameters-level strict, no-strict omission, and strict=False omission; all are mocked and don't make real network calls, in line with the folder's test rules. |
Reviews (1): Last reviewed commit: "fix(anthropic): place strict flag at too..." | Re-trigger Greptile
| **input_schema_filtered | ||
| ) | ||
|
|
||
| _strict = tool["function"].get("strict") or _input_schema.get("strict") |
There was a problem hiding this comment.
or swallows an explicit strict=False at the function level
tool["function"].get("strict") or _input_schema.get("strict") uses short-circuit or, so if a caller explicitly sets strict=False on the function object the expression falls through to _input_schema.get("strict"). If the parameters dict still has a leftover strict=True (e.g., legacy placement that wasn't cleaned up by the caller), the result is True even though the caller explicitly opted out at the canonical location. Using is None chaining avoids this ambiguity.
| _strict = tool["function"].get("strict") or _input_schema.get("strict") | |
| _function_strict = tool["function"].get("strict") | |
| _strict = _function_strict if _function_strict is not None else _input_schema.get("strict") |
There was a problem hiding this comment.
The code already uses the is not None pattern -- see lines 679-684. _function_strict is checked with if _function_strict is not None before falling back to _input_schema.get("strict"), so an explicit strict=False at the function level is preserved correctly.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
When strict=False is set at the function level and strict=True remains in parameters, the or operator treats False as falsy and falls through to the parameters value. Use is-None chaining so the function-level value always takes precedence when explicitly set.
|
Friendly ping @ishaan-jaff @krrish-berri-2 -- this fixes strict flag being silently ignored on Anthropic tools (placed inside input_schema where Anthropic drops it). Data correctness bug. Ready for review. |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. |
Relevant issues
Fixes #27490
Pre-Submission checklist
tests/test_litellm/directory, Adding at least 1 test is a hard requirementmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewType
Bug Fix
Changes
Anthropic enforces
stricttool use only when the flag is set at the tool top level (sibling ofname/input_schema). LiteLLM was placing it insideinput_schemawhere Anthropic silently ignores it, so callers thought strict was engaged but the model emitted unconstrained tool arguments.The fix:
strictfromAnthropicInputSchema(not part of the JSON Schema object)stricttoAnthropicMessagesTool(where Anthropic expects it)_map_tool_helper, extractstrictfrom eithertool.function.strict(OpenAI canonical location) orparameters.strict(legacy placement) and set it on the tool objectTest plan
Added 4 tests in
test_anthropic_chat_transformation.py:strict: trueontool.functionis placed at tool top levelstrict: trueinsideparametersis moved to tool top levelstrictflag means the field is omitted entirelystrict: falseis not forwarded to AnthropicScreenshots / Proof of Fix
Before (on main) --
strictends up nested insideinput_schema, Anthropic ignores it:{"name": "get_weather", "input_schema": {"type": "object", "strict": true, ...}, "type": "custom"}After (this PR) --
strictis at the tool top level, Anthropic enforces it:{"name": "get_weather", "input_schema": {"type": "object", ...}, "type": "custom", "strict": true}