fix(bedrock): honor ttl for tool_config cache injection points

[shivamrawat1]

Resolves LIT-3892

Issue
Bedrock Converse cache_control_injection_points with location: tool_config did not honor control.ttl. Callers could request ttl: 1h on tool definitions, but LiteLLM always emitted a bare {"cachePoint": {"type": "default"}} in toolConfig.tools, so Bedrock silently fell back to the 5-minute default. Message and system cache points already respected ttl via _get_cache_point_block; the tool_config path skipped that logic entirely

Separately, is_claude_4_5_on_bedrock used a hardcoded model-name pattern list to decide whether 5m/1h TTL was allowed. That list needed manual updates for every new Claude release and already missed models like Sonnet 5 and Fable 5 that have 1h TTL pricing in model_prices_and_context_window.json

Two Claude 3.5 Sonnet entries in the pricing JSON incorrectly carried cache_creation_input_token_cost_above_1hr, which would have made a JSON-driven check wrongly grant them 1h TTL support (their own us./eu./apac. regional variants did not have the field)

Cause
In converse_transformation.py, the tool_config branch hardcoded the cache point and never read point["control"]:

bedrock_tools.append({"cachePoint": {"type": "default"}})
The message/system path already went through _get_cache_point_block, which mapped cache_control.ttl onto the Bedrock cachePoint when the model was eligible. The type for tool_config injection points also lacked a control field, so ttl was not even modeled for that location

For model eligibility, is_claude_4_5_on_bedrock matched substring patterns like sonnet-4-5 and opus-4-7 instead of reading capability data from the pricing JSON. New models only worked after someone updated the regex list, and the JSON already had a better signal: cache_creation_input_token_cost_above_1hr is present only on models AWS bills for 1-hour cache writes

Fix
Pass ttl through for tool_config cache injection

Extracted shared _build_cache_point_block(control, model) and wired both message/system and tool_config paths through it. Added control: Optional[ChatCompletionCachedContent] to CacheControlToolConfigInjectionPoint. Added regression tests asserting ttl: 1h survives for Claude Sonnet 4.5 and is dropped for unsupported models

Drive 1h TTL eligibility from pricing JSON

Replaced the hardcoded pattern list in is_claude_4_5_on_bedrock with a lookup against cache_creation_input_token_cost_above_1hr in litellm.model_cost (with get_bedrock_base_model fallback). New Claude releases pick up 1h TTL support automatically when their pricing entry ships

Fix bad pricing data

Removed cache_creation_input_token_cost_above_1hr (and its above-200k variant) from anthropic.claude-3-5-sonnet-20240620-v1:0 and anthropic.claude-3-5-sonnet-20241022-v2:0 in both model_prices_and_context_window.json and the bundled backup, matching their regional variants and AWS's documented 1h-TTL model set

SCRIPT:

from litellm.llms.bedrock.chat.converse_transformation import AmazonConverseConfig

config = AmazonConverseConfig()
messages = [
    {"role": "system", "content": "You are a helpful assistant.", "cache_control": {"type": "ephemeral", "ttl": "1h"}},
    {"role": "user", "content": "What's the weather in SF?"},
]
optional_params = {
    "tools": [
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get weather for a location",
                "parameters": {
                    "type": "object",
                    "properties": {"location": {"type": "string"}},
                    "required": ["location"],
                },
            },
        }
    ],
    "cache_control_injection_points": [
        {"location": "tool_config", "control": {"type": "ephemeral", "ttl": "1h"}},
    ],
}

result = config._transform_request(
    model="us.anthropic.claude-sonnet-4-5-20250929-v1:0",  # claude-4.5, supports ttl
    messages=messages,
    optional_params=optional_params,
    litellm_params={},
)

print(result["system"]) # -> cachePoint DOES carry {'ttl': '1h'} (message path works)
print(result["toolConfig"]) # -> {'cachePoint': {'type': 'default'}} <-- ttl silently dropped

Before:

After:

---

Note

Medium Risk
Changes Bedrock request shaping for prompt caching and parallel tools; wrong pricing flags could omit TTL or parallel config for some model IDs, but scope is limited to capability gating and cache metadata.

Overview
Fixes LIT-3892: Bedrock Converse cache_control_injection_points with location: tool_config now forward control.ttl (e.g. 1h) into toolConfig.tools cache points, using the same _build_cache_point_block path as message/system blocks. CacheControlToolConfigInjectionPoint gains an optional control field.

Model capability checks no longer rely on hardcoded Claude name patterns. is_claude_4_5_on_bedrock (extended cache TTL) keys off cache_creation_input_token_cost_above_1hr in pricing data, with regional → base-model lookup. bedrock_converse_supports_parallel_tool_use_config is a separate JSON flag so parallel-tool config is not tied to TTL eligibility.

Pricing updates: supports_parallel_tool_use_config on eligible Bedrock Anthropic models; removed erroneous cache_creation_input_token_cost_above_1hr from two Claude 3.5 Sonnet base entries so they are not treated as 1h-TTL models.

^{Reviewed by Cursor Bugbot for commit 4e53edc. Bugbot is set up for automated code reviews on this repo. Configure here.}

Summary by CodeRabbit

• New Features

  • Added broader support for Bedrock/Anthropic model capabilities, including parallel tool use and improved cache control handling.
  • Expanded model metadata so more variants can use the right request settings automatically.

• Bug Fixes

  • Improved cache TTL behavior for supported models, including regional variants.
  • Fixed fallback behavior so capability detection works even when a regional entry is missing fields.

• Chores

  • Updated model pricing and capability data formatting for consistency.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 3 committers have signed the CLA.

✅ mateo-berri
❌ Shivam Rawat
❌ cursoragent

---

Shivam Rawat seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes Bedrock Converse cache_control_injection_points with location: "tool_config" so they correctly honor control.ttl, matching the existing behavior for message and system cache points. It also decouples 1h-TTL eligibility and parallel-tool-use config eligibility from hardcoded model-name pattern lists, instead reading cache_creation_input_token_cost_above_1hr and supports_parallel_tool_use_config directly from the pricing JSON.

• Shared _build_cache_point_block helper now handles both message/system and tool-config paths, eliminating the bare {"cachePoint": {"type": "default"}} that silently dropped TTL for tool-config injection points.
• is_claude_4_5_on_bedrock is rewritten to read cache_creation_input_token_cost_above_1hr from litellm.model_cost (with get_bedrock_base_model fallback) instead of matching a hardcoded substring list; bedrock_converse_supports_parallel_tool_use_config is added as a separate, data-driven gate for disable_parallel_tool_use.
• Incorrect cache_creation_input_token_cost_above_1hr fields were removed from the two Claude 3.5 Sonnet Bedrock entries that previously caused those models to be incorrectly treated as 1h-TTL-capable.

Confidence Score: 5/5

Safe to merge. The core transformation logic change is narrow and correct, the pricing JSON edits are verified by tests that assert on specific model entries, and all new tests use local cost maps or monkeypatch to avoid network calls.

The tool-config TTL fix routes through the same helper already trusted for message/system cache points. Model-eligibility checks now read from the pricing JSON, which is the established pattern. The only noteworthy item is a TypedDict annotation precision issue with NotRequired that has no runtime impact. No behavioral regressions were found.

No files require special attention. The pricing JSON changes are symmetric across the main and backup files, and the models that lost cache_creation_input_token_cost_above_1hr match their regional variants and AWS documentation.

Important Files Changed

Filename	Overview
litellm/llms/bedrock/chat/converse_transformation.py	Extracted shared _build_cache_point_block helper and wired both message/system and tool_config injection paths through it, fixing the TTL drop. Parallel tool use guard now uses bedrock_converse_supports_parallel_tool_use_config instead of is_claude_4_5_on_bedrock.
litellm/llms/bedrock/common_utils.py	Replaced the hardcoded Claude-4.5 pattern list in is_claude_4_5_on_bedrock with a lookup against cache_creation_input_token_cost_above_1hr in litellm.model_cost. Added bedrock_converse_supports_parallel_tool_use_config using a separate supports_parallel_tool_use_config flag.
litellm/types/integrations/anthropic_cache_control_hook.py	Added control: Optional[ChatCompletionCachedContent] to CacheControlToolConfigInjectionPoint. The field is technically required (TypedDict default total=True) but callers that omit it still work at runtime via .get("control"); NotRequired would be more accurate.
litellm/model_prices_and_context_window_backup.json	Removed incorrect cache_creation_input_token_cost_above_1hr from Claude 3.5 Sonnet Bedrock entries; added supports_parallel_tool_use_config: true to haiku-4.5, opus-4.6 variants, opus-4.7 variants, and claude-sonnet-5 entries.
model_prices_and_context_window.json	Mirrors backup JSON changes: removes erroneous 1h-TTL pricing from Claude 3.5 Sonnet Bedrock entries and adds supports_parallel_tool_use_config to relevant Claude 4.5+/4.6/4.7 model entries.
tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py	Adds regression tests for TTL honoring on tool_config injection points (supported model, regional fallback, unsupported model) and for decoupled parallel tool use vs TTL eligibility. All tests use monkeypatch or LITELLM_LOCAL_MODEL_COST_MAP to avoid network calls.
tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py	Adds a regression test confirming that a regional model entry lacking a capability field correctly falls back to the base model entry via the any() pattern.

_{Reviews (9): Last reviewed commit: "fix(bedrock): fall back to base model en..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[shivamrawat1]

@greptile review again with new commit that resolves the p2 issue

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Autofix Details

Bugbot Autofix prepared fixes for both issues found in the latest run.

• ✅ Fixed: ToolBlock constructor on Python 3.10
  • Replaced the tool-config cachePoint TypedDict keyword constructor with a plain dict append so the path works on Python 3.10.
• ✅ Fixed: Parallel tools use TTL gate
  • Separated the Bedrock parallel-tool capability check from the extended-cache TTL pricing predicate and covered unpriced Claude 4.7-style model IDs with a regression test.

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Parallel tools gate drops Sonnet 5
  • The parallel tool-use gate now also accepts Bedrock models that qualify through the pricing JSON, with a regression test for Sonnet 5 request transformation.

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: TTL gate enables parallel tools
  • Removed the extended-cache TTL pricing helper from the parallel-tool capability gate and added a regression covering TTL-only Bedrock models.

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

@greptileai

---

Generated by Claude Code

[mateo-berri]

@greptileai

---

Generated by Claude Code

[mateo-berri]

bugbot run

---

Generated by Claude Code

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Regional cost map blocks TTL
  • Updated the Bedrock TTL eligibility check to fall back to the stripped base model when a regional cost entry lacks 1-hour cache pricing, with JP Opus 4.7 regression coverage.

_{You can send follow-ups to the cloud agent here.}

[mateo-berri]

@greptileai

---

Generated by Claude Code

[mateo-berri]

LGTM; thanks!

[mateo-berri]

bugbot run

[mateo-berri]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 4e53edc. Configure here.}

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 50b4c18d-0f3a-4500-b8fa-81e7620e2f48

📥 Commits

Reviewing files that changed from the base of the PR and between bea8c93 and 4e53edc.

📒 Files selected for processing (11)

• litellm/llms/bedrock/chat/converse_transformation.py
• litellm/llms/bedrock/common_utils.py
• litellm/model_prices_and_context_window_backup.json
• litellm/types/integrations/anthropic_cache_control_hook.py
• model_prices_and_context_window.json
• ruff-strict-budget.json
• tests/test_litellm/litellm_core_utils/prompt_templates/test_litellm_core_utils_prompt_templates_factory.py
• tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py
• tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py
• tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py
• tests/test_litellm/test_utils.py

---

📝 Walkthrough

Walkthrough

Bedrock Converse gating for parallel tool-use and extended cache TTL support switches from hardcoded Claude 4.5 name matching to data-driven lookups against litellm.model_cost, with base-model fallback. cachePoint construction is refactored into a shared helper, and pricing JSON files gain a new capability flag alongside formatting normalization.

Changes

Capability-driven parallel tool-use and cache TTL gating

Layer / File(s)	Summary
Capability helper functions litellm/llms/bedrock/common_utils.py, tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py	Adds bedrock_converse_supports_parallel_tool_use_config and rewrites is_claude_4_5_on_bedrock to check litellm.model_cost fields (with base-model fallback) instead of hardcoded name/version matching; adds a fallback regression test.
Converse transformation wiring and cachePoint refactor litellm/llms/bedrock/chat/converse_transformation.py, litellm/types/integrations/anthropic_cache_control_hook.py	Imports and uses the new capability helper to gate parallel tool-use config, factors cachePoint TTL logic into _build_cache_point_block, uses it in tool_config cache injection, and adds an optional control field to the cache injection point type.
Regression tests for gating and TTL behavior tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py, tests/test_litellm/litellm_core_utils/prompt_templates/*, tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py	Adds tests for parallel tool-use gating decoupled from TTL pricing, TTL injection/fallback in tool_config cachePoints, and TTL stripping/preservation for tool cache_control, forcing local model cost maps for determinism.

Pricing/context window JSON data updates

Layer / File(s)	Summary
Capability flags and pricing normalization litellm/model_prices_and_context_window_backup.json, model_prices_and_context_window.json, tests/test_litellm/test_utils.py, ruff-strict-budget.json	Adds supports_parallel_tool_use_config capability flag across many Anthropic/Bedrock model entries, normalizes numeric formatting to scientific notation, reformats arrays, adjusts regional uplift multipliers, inserts/reorganizes Snowflake and pinstripes entries, extends the JSON schema test, and lowers the ruff SIM101 budget limit.

Estimated code review effort: 3 (Moderate) | ~30 minutes

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch litellm_bedrock_cache_control_default

⚔️ Resolve merge conflicts

• Resolve merge conflict in branch litellm_bedrock_cache_control_default

---

_{Comment @coderabbitai help to get the list of available commands.}