fix(bedrock): drop strict/additionalProperties from toolSpec for Sonnet 4.6 and Haiku 4.5 - #35617
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
50be12d to
5cf5902
Compare
|
Note for reviewer: Greptile initially could not summarise this PR because the branch base was stale (fork not synced to upstream). That is now fixed — the diff is clean: 3 files, 75 insertions, 7 deletions, identical in shape to #31943 (Sonnet 4 fix). What this PR does in one sentence: extends the existing Files changed:
Test result: 43 passed (was 26 before this PR), 0 failed. |
Greptile SummaryThe PR disables unsupported strict tool schemas for Claude Sonnet 4.6 and Haiku 4.5 on Bedrock Converse.
Confidence Score: 5/5The PR appears safe to merge, with the affected model variants consistently configured and covered by focused regression tests. The canonical and fallback model maps remain synchronized, model-ID normalization reaches the new flags, and the changed tests verify both capability lookup and emitted Bedrock tool schemas.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds the strict-tools capability override consistently to all relevant Sonnet 4.6 and Haiku 4.5 Bedrock Converse entries. |
| litellm/model_prices_and_context_window_backup.json | Mirrors the canonical model metadata changes so local fallback lookups preserve the corrected behavior. |
| tests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py | Extends meaningful regression coverage without weakening the remaining supported-model assertions. |
Reviews (1): Last reviewed commit: "fix(bedrock): drop strict/additionalProp..." | Re-trigger Greptile
…et 4.6 and Haiku 4.5 Bedrock Converse rejects toolSpec.strict for Claude Sonnet 4.6 and Claude Haiku 4.5, causing a grammar-size error (>300MB compiled grammar) that results in a 504 Gateway Timeout for callers. The same fix was already applied for Opus 4.7/4.8 (BerriAI#31923) and Sonnet 4 (BerriAI#31943). This extends it to the missing model variants by setting bedrock_converse_supports_strict_tools: false on all regional Bedrock Converse entries for these two models. Fixes BerriAI#34388 Co-authored-by: Hermes Agent <hermes-agent@nousresearch.com>
5cf5902 to
7c146bf
Compare
TLDR
Problem this solves:
toolSpec.strict, producing a compiled-grammar error (>300MB) and a 504 Gateway Timeout for the callerstrictandadditionalPropertiesto these models since v1.90.0How it solves it:
bedrock_converse_supports_strict_tools: falseon all regional Bedrock Converse entries for Sonnet 4.6 and Haiku 4.5 (14 entries each file)Relevant issues
Closes #34388
Type
Changes
model_prices_and_context_window.json— addbedrock_converse_supports_strict_tools: falseto 14 regional Bedrock entries for Sonnet 4.6 and Haiku 4.5litellm/model_prices_and_context_window_backup.json— same change to backup filetests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py— extend existing regression tests to cover Sonnet 4.6 and Haiku 4.5; correct a wrong assertion that previously claimed Sonnet 4.6 keepsstrictforwardedTesting
pytest tests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py -v # 43 passed (was 26 before this fix)The 17 new test cases confirm:
strictis dropped fromtoolSpecfor all Sonnet 4.6 and Haiku 4.5 model IDsadditionalPropertiesis dropped from the input schema for these modelsbedrock_converse_supports_strict_toolsflag is correctly set in the model cost mapbedrock_converse_supports_strict_tools()returnsFalsefor these modelsScreenshots / Proof of Fix
Before this fix,
_bedrock_tools_ptwith a strict tool and modelanthropic.claude-sonnet-4-6returns:{'inputSchema': {'json': {'additionalProperties': False, ...}}, 'name': 'get_weather', 'strict': True} # strict leaks through → Bedrock returns grammar-size error → 504 timeoutAfter this fix:
{'inputSchema': {'json': {...}}, 'name': 'get_weather', 'description': '...'} # strict and additionalProperties stripped → Bedrock accepts the requestPre-submission checklist
pytest— 43 passed, 0 failed)model_prices_and_context_window.jsonand backup file updatedAI Assistance Disclosure
This contribution was AI-assisted using Hermes Agent (Nous Research).
Co-authored-by: Hermes Agent hermes-agent@nousresearch.com