Skip to content

fix(bedrock): drop toolSpec.strict for four more Claude Converse ids - #39115

Open
pdecat wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
pdecat:fix/bedrock-strict-tools-unflagged-claude-ids
Open

fix(bedrock): drop toolSpec.strict for four more Claude Converse ids#39115
pdecat wants to merge 1 commit into
BerriAI:litellm_internal_stagingfrom
pdecat:fix/bedrock-strict-tools-unflagged-claude-ids

Conversation

@pdecat

@pdecat pdecat commented Sep 1, 2026

Copy link
Copy Markdown

Bedrock Converse routes Haiku 4.5, Sonnet 4.5/4.6 and Opus 4.6 through the same Anthropic-compatible validator as Opus 4.7/4.8 and Sonnet 4/5 (#31582), but their cost-map entries were never flagged, so toolSpec.strict is still forwarded for them.

Forwarding strict: true makes the model compile the tool schema into a constrained-decoding grammar, and that grammar rejects ordinary JSON Schema keywords. A tool carrying maxItems on an array property then fails the whole request:

BedrockException - {"message":"The model returned the following errors: tools.1.custom: For 'array' type, property 'maxItems' is not supported"}

Neither field fails on its own, only the pair. Measured through a proxy against Sonnet 4.6, two tools sent so the failing one is tools.1:

strict maxItems result
false absent 200
false present 200
true absent 200
true present 400

So it surfaces as a sudden breakage for tool-calling clients that set strict on their tools, on any upgrade crossing v1.90.0, where toolSpec.strict forwarding to Converse was introduced. Reproduced against Bedrock for Haiku 4.5, Sonnet 4.6 and Opus 4.6.

Change

bedrock_converse_supports_strict_tools: false on all 30 Bedrock entries for those four ids (bare key, every region prefix, the us-gov keys), in model_prices_and_context_window.json and its backup, matching how Opus 4.7/4.8, Sonnet 4 and Sonnet 5 are already flagged. No code changes: the gate is data, so the fix reaches deployments through the cost map rather than another hardcoded model pattern.

test_bedrock_converse_strict_tools_opus_47_48.py asserted the pre-fix behaviour for Sonnet 4.5/4.6 and Opus 4.6, listing them as models that accept strict, so those cases move to the "strict dropped" side and the cost-map assertions cover the new keys. 66 passed locally, plus tests/test_litellm/test_model_prices_schema.py and ci_cd/check_files_match.py.

Sonnet 4.5 has no separate repro of its own and is flagged as the remaining member of the same cohort. Happy to drop it if you would rather ship only reproduced ids.

Fixes #34388. Overlaps #34609, which flags two of these ids; this one covers all four and updates the test that contradicts them, so close whichever fits better.

@pdecat
pdecat requested a review from mateo-berri as a code owner September 1, 2026 14:26
@codspeed-hq

codspeed-hq Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing pdecat:fix/bedrock-strict-tools-unflagged-claude-ids (dde0e60) with litellm_internal_staging (ec3f818)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR marks additional Claude Haiku 4.5, Sonnet 4.5/4.6, and Opus 4.6 Bedrock entries as not supporting Converse strict tools.

  • Keeps the primary and backup model cost maps synchronized.
  • Updates transformation, helper, and cost-map tests to verify that toolSpec.strict is removed for the affected model identifiers.

Confidence Score: 5/5

The PR appears safe to merge, with the new model metadata consistently applied and exercised by focused regression coverage.

The Bedrock Converse helper consumes these capability flags on the intended transformation path, affected prefixed identifiers resolve to flagged entries, non-Anthropic-prefixed forms are already guarded as unsupported, and both cost maps remain synchronized.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds the strict-tools capability override to the intended Bedrock model and regional entries without leaving a reachable affected identifier uncovered.
litellm/model_prices_and_context_window_backup.json Mirrors the primary cost-map updates used by packaged local model metadata.
tests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py Moves the newly unsupported models to strict-removal coverage and verifies representative transformations and cost-map entries without weakening existing supported-model checks.

Reviews (1): Last reviewed commit: "fix(bedrock): drop toolSpec.strict for f..." | Re-trigger Greptile

@codecov

codecov Bot commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Bedrock Converse routes Haiku 4.5, Sonnet 4.5/4.6 and Opus 4.6 through the
same Anthropic-compatible validator as Opus 4.7/4.8 and Sonnet 4/5, but their
cost-map entries were never flagged, so `toolSpec.strict` is still forwarded
for them.

Forwarding `strict: true` makes the model compile the tool schema into a
constrained-decoding grammar, and that grammar rejects ordinary JSON Schema
keywords. A tool carrying `maxItems` on an array property then fails the whole
request:

    BedrockException - {"message":"The model returned the following errors:
    tools.1.custom: For 'array' type, property 'maxItems' is not supported"}

Neither field fails on its own, only the pair, which is why this surfaced as
a sudden breakage for tool-calling clients (Cline forks, Codex, MCP toolsets)
after upgrading across v1.90.0, where forwarding `toolSpec.strict` to Converse
was introduced. Reproduced against Bedrock for Haiku 4.5, Sonnet 4.6 and
Opus 4.6; Sonnet 4.5 is flagged as the remaining member of the same cohort.

Sets `bedrock_converse_supports_strict_tools: false` on all 30 Bedrock entries
for those four ids (bare, every region prefix, and the us-gov keys), matching
how Opus 4.7/4.8, Sonnet 4 and Sonnet 5 are already flagged, so the fix is data
rather than another hardcoded model pattern.

Two tests asserted the pre-fix behaviour and are updated: the strict-tools
regression suite listed Sonnet 4.5/4.6 and Opus 4.6 as models that keep the
field, and test_bedrock_tools_pt_strict_parameter used Sonnet 4.5 as its
example of a Claude id that forwards it, which is now Opus 4.5.

Fixes BerriAI#34388
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LiteLLM forwards unsupported strict tool fields to Bedrock Claude models, causing 400 errors and hanging requests

1 participant