Skip to content

fix(bedrock): drop strict/additionalProperties from toolSpec for Sonnet 4.6 and Haiku 4.5 - #35617

Open
nickhac wants to merge 1 commit into
BerriAI:mainfrom
nickhac:fix/bedrock-strict-tools-sonnet-4-6-haiku-4-5
Open

fix(bedrock): drop strict/additionalProperties from toolSpec for Sonnet 4.6 and Haiku 4.5#35617
nickhac wants to merge 1 commit into
BerriAI:mainfrom
nickhac:fix/bedrock-strict-tools-sonnet-4-6-haiku-4-5

Conversation

@nickhac

@nickhac nickhac commented Aug 3, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

  • Claude Sonnet 4.6 and Haiku 4.5 on Bedrock Converse reject toolSpec.strict, producing a compiled-grammar error (>300MB) and a 504 Gateway Timeout for the caller
  • LiteLLM has been forwarding strict and additionalProperties to these models since v1.90.0

How it solves it:

Relevant issues

Closes #34388

Type

  • Bug Fix

Changes

  • model_prices_and_context_window.json — add bedrock_converse_supports_strict_tools: false to 14 regional Bedrock entries for Sonnet 4.6 and Haiku 4.5
  • litellm/model_prices_and_context_window_backup.json — same change to backup file
  • tests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py — extend existing regression tests to cover Sonnet 4.6 and Haiku 4.5; correct a wrong assertion that previously claimed Sonnet 4.6 keeps strict forwarded

Testing

pytest tests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py -v
# 43 passed (was 26 before this fix)

The 17 new test cases confirm:

  • strict is dropped from toolSpec for all Sonnet 4.6 and Haiku 4.5 model IDs
  • additionalProperties is dropped from the input schema for these models
  • bedrock_converse_supports_strict_tools flag is correctly set in the model cost map
  • The helper function bedrock_converse_supports_strict_tools() returns False for these models

Screenshots / Proof of Fix

Before this fix, _bedrock_tools_pt with a strict tool and model anthropic.claude-sonnet-4-6 returns:

{'inputSchema': {'json': {'additionalProperties': False, ...}}, 'name': 'get_weather', 'strict': True}
# strict leaks through → Bedrock returns grammar-size error → 504 timeout

After this fix:

{'inputSchema': {'json': {...}}, 'name': 'get_weather', 'description': '...'}
# strict and additionalProperties stripped → Bedrock accepts the request

Pre-submission checklist

  • Tests added (17 new parametrized test cases in existing regression test file)
  • CI passes locally (pytest — 43 passed, 0 failed)
  • Scope isolated: extends existing fix pattern, no new abstractions
  • Both model_prices_and_context_window.json and backup file updated

AI Assistance Disclosure

This contribution was AI-assisted using Hermes Agent (Nous Research).

Co-authored-by: Hermes Agent hermes-agent@nousresearch.com

@CLAassistant

CLAassistant commented Aug 3, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@nickhac
nickhac force-pushed the fix/bedrock-strict-tools-sonnet-4-6-haiku-4-5 branch from 50be12d to 5cf5902 Compare August 3, 2026 01:08
@nickhac

nickhac commented Aug 3, 2026

Copy link
Copy Markdown
Author

Note for reviewer: Greptile initially could not summarise this PR because the branch base was stale (fork not synced to upstream). That is now fixed — the diff is clean: 3 files, 75 insertions, 7 deletions, identical in shape to #31943 (Sonnet 4 fix).

What this PR does in one sentence: extends the existing bedrock_converse_supports_strict_tools: false opt-out (already applied in #31923 for Opus 4.7/4.8 and #31943 for Sonnet 4) to Claude Sonnet 4.6 and Claude Haiku 4.5, which exhibit the same compiled-grammar overflow error when toolSpec.strict is forwarded.

Files changed:

  • model_prices_and_context_window.json — adds flag to 6 Sonnet 4.6 regional entries + 8 Haiku 4.5 regional entries
  • litellm/model_prices_and_context_window_backup.json — same
  • test_bedrock_converse_strict_tools_opus_47_48.py — extends existing regression test parametrize lists; corrects a wrong assertion that previously claimed bedrock/us.anthropic.claude-sonnet-4-6 keeps strict forwarded (it does not — that is the bug)

Test result: 43 passed (was 26 before this PR), 0 failed.

@codspeed-hq

codspeed-hq Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing nickhac:fix/bedrock-strict-tools-sonnet-4-6-haiku-4-5 (7c146bf) with main (714fff6)

Open in CodSpeed

@nickhac

nickhac commented Aug 3, 2026

Copy link
Copy Markdown
Author

@greptileai

@greptile-apps

greptile-apps Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR disables unsupported strict tool schemas for Claude Sonnet 4.6 and Haiku 4.5 on Bedrock Converse.

  • Adds the capability flag to every relevant regional model entry in both canonical and backup metadata.
  • Extends regression coverage for model lookup, strict removal, and additionalProperties removal.
  • Corrects the prior Sonnet 4.6 expectation while retaining positive coverage for supported Anthropic models.

Confidence Score: 5/5

The PR appears safe to merge, with the affected model variants consistently configured and covered by focused regression tests.

The canonical and fallback model maps remain synchronized, model-ID normalization reaches the new flags, and the changed tests verify both capability lookup and emitted Bedrock tool schemas.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds the strict-tools capability override consistently to all relevant Sonnet 4.6 and Haiku 4.5 Bedrock Converse entries.
litellm/model_prices_and_context_window_backup.json Mirrors the canonical model metadata changes so local fallback lookups preserve the corrected behavior.
tests/test_litellm/litellm_core_utils/prompt_templates/test_bedrock_converse_strict_tools_opus_47_48.py Extends meaningful regression coverage without weakening the remaining supported-model assertions.

Reviews (1): Last reviewed commit: "fix(bedrock): drop strict/additionalProp..." | Re-trigger Greptile

…et 4.6 and Haiku 4.5

Bedrock Converse rejects toolSpec.strict for Claude Sonnet 4.6 and
Claude Haiku 4.5, causing a grammar-size error (>300MB compiled grammar)
that results in a 504 Gateway Timeout for callers.

The same fix was already applied for Opus 4.7/4.8 (BerriAI#31923) and Sonnet 4
(BerriAI#31943). This extends it to the missing model variants by setting
bedrock_converse_supports_strict_tools: false on all regional Bedrock
Converse entries for these two models.

Fixes BerriAI#34388

Co-authored-by: Hermes Agent <hermes-agent@nousresearch.com>
@nickhac
nickhac force-pushed the fix/bedrock-strict-tools-sonnet-4-6-haiku-4-5 branch from 5cf5902 to 7c146bf Compare August 8, 2026 06:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants