fix(bedrock): drop toolSpec.strict for Claude Sonnet 5 and Fable 5 - #32455
Conversation
Bedrock Converse routes Claude Sonnet 5 and Fable 5 through the same
Anthropic-compatible validator as Opus 4.7/4.8 and Sonnet 4, which rejects
`toolSpec.strict` even though Anthropic's native API accepts `strict` as a
top-level tool field:
tools.0.custom.strict: Extra inputs are not permitted
Both were missing the `bedrock_converse_supports_strict_tools: false`
opt-out flag, so the blanket strict-forwarding path 400s any tool-calling
request against `bedrock/*.claude-sonnet-5` / `bedrock/*.claude-fable-5`.
Verified against live Bedrock Converse (us-east-1): with `strict` present
→ 400 (above error); with `strict` stripped → 200. Confirmed for both the
`us.` and `global.` inference-profile variants of each model.
Add the flag to all Bedrock (`bedrock_converse`) Sonnet 5 and Fable 5
entries in model_prices_and_context_window.json and its backup, and extend
the strict-tools regression tests to cover them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Greptile SummaryThis PR adds
Confidence Score: 5/5Safe to merge — the change is additive JSON data only, touching no runtime logic, and every new entry is covered by tests. The fix follows an established, tested pattern from two prior PRs. Both JSON files are kept in sync, all 10 new cost-map keys have direct test coverage across three test functions, and the live-call evidence in the PR description confirms the before/after behaviour on real Bedrock endpoints. No files require special attention.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds bedrock_converse_supports_strict_tools: false to 10 model entries (6 Sonnet 5, 4 Fable 5); consistent with existing flag placement for Opus 4.7/4.8 and Sonnet 4. |
| litellm/model_prices_and_context_window_backup.json | Identical flag additions as the primary JSON; the two files are kept in sync. |
| tests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py | Extends parametrized tests for all three test functions to cover Sonnet 5 and Fable 5; function docstring updated; module-level docstring still only mentions Opus 4.7/4.8 and Sonnet 4. |
Reviews (2): Last reviewed commit: "test(bedrock): cover all Sonnet 5 / Fabl..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Merging this PR will improve performance by 13.29%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ⚡ | test_completion_simple_message |
4.6 ms | 4.1 ms | +13.29% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing seanturner83:fix/bedrock-converse-drop-strict-sonnet5-fable5 (70ee6ff) with litellm_internal_staging (cd6e8cd)
…ocstring Address Greptile review on the strict-tools regression test: - Add the 4 regional variants that were in the cost-map flag test but missing from the end-to-end _bedrock_tools_pt strip test (au./jp. Sonnet 5, global./eu. Fable 5) — that test is the only one exercising the full strip pipeline, so those profiles had partial coverage. - Add the bare anthropic.claude-fable-5 key to the helper assertion. - Update the stale docstring to include Sonnet 5 and Fable 5. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks @greptileai — addressed both gaps in
All 10 cost-map keys now have end-to-end |
Relevant issues
Follow-up to #31923 (Opus 4.7/4.8) and #31943 (Sonnet 4) — same root cause, two more models.
Pre-Submission checklist
What this fixes
Bedrock Converse routes Claude Sonnet 5 and Fable 5 through the same Anthropic-compatible validator as Opus 4.7/4.8 and Sonnet 4, which rejects
toolSpec.stricteven though Anthropic's native API acceptsstrictas a top-level tool field:Both models were missing the
bedrock_converse_supports_strict_tools: falseopt-out flag, so the default strict-forwarding path 400s any tool-calling request againstbedrock/*.claude-sonnet-5orbedrock/*.claude-fable-5(SDKs like openai-agents stampstricton every tool, so it fires universally at the first call).The change
bedrock_converse_supports_strict_tools: falseto allbedrock_converseSonnet 5 (6) and Fable 5 (4) inference-profile entries inmodel_prices_and_context_window.jsonand its backup.test_bedrock_converse_strict_tools_opus_47_48.py) to cover Sonnet 5 / Fable 5 in the dropped-list, the helper check, and the cost-map-flag check.No logic change — this reuses the exact mechanism from #31923/#31943; the models just needed the flag.
Screenshots / Proof of Fix
Live Bedrock Converse calls (us-east-1, real $), captured at commit
bbb8af5:Confirmed for both
us.andglobal.inference-profile variants of each model. Unit tests:pytest tests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py→ 44 passed.Type
🐛 Bug Fix
Co-authored with Claude Opus 4.8.