fix(bedrock): honor cache_control ttl on message-level cachePoint blocks

[arashne]

Relevant issues

Fixes #32154. Completes the ttl support added for system messages in #20338 (#19848, #20326).

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review (received 5/5: "Safe to merge")

Screenshots / Proof of Fix

Repro before the fix (v1.90.0 and main): a message-level cache_control with ttl: "1h" on a supported model emits cachePoint: {"type": "default"} — the ttl is silently dropped (see #32154 for the snippet). After the fix, the same input emits {"type": "default", "ttl": "1h"} for user content, tool-result, and per-tool-call placements; unsupported models keep emitting the plain default block.

E2e proof with real Bedrock calls (cacheWriteInputTokens billed at the 1h rate for global.anthropic.claude-opus-4-7) to follow as a comment — we run this patch in production via a sitecustomize overlay and will attach the live usage output.

tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py -k "ttl_for_supported or ttl_for_unsupported"
8 passed

Adjacent suites (test_converse_transformation.py, test_litellm_core_utils_prompt_templates_factory.py, test_anthropic_cache_control_hook.py) have byte-identical failure sets on base and patched (pre-existing env failures only): 241 passed → 247 passed, no new failures.

Type

🐛 Bug Fix

Changes

Bedrock Converse supports cachePoint.ttl and _get_cache_point_block maps cache_control.ttl → cachePoint.ttl, but the model argument its allow-list gate requires was only threaded through the system-message path — every message-level path silently degraded a requested 1h to the 5-minute default, exactly on the conversation-tail breakpoint that long-running agents set (cache_control_injection_points with index: -1) to survive tool calls longer than 5 minutes.

• factory.py: pass model= at the 8 _get_cache_point_block call sites in _bedrock_converse_messages_pt / _pt_async.
• Tool-result blocks: capture the cache_control dict (previously reduced to a boolean) and route through _get_cache_point_block so ttl survives.
• _convert_to_bedrock_tool_call_invoke: accept optional model and route per-tool-call cache_control through _get_cache_point_block; both callers pass model=model.
• Tests: parameterized over user / tool-call / tool-result placements × supported (opus-4-7) and unsupported (claude-3-5-sonnet) models, sync and async paths asserted equal.

Out of scope (pre-existing, orthogonal): message-level cache_control on a content-less assistant message emits no cachePoint at all; per-tool-call placement covers that message.

[CLAassistant]

All committers have signed the CLA.

[arashne]

@greptileai

[greptile-apps]

Greptile Summary

This PR fixes a bug where cache_control.ttl was silently dropped on message-level and tool-call-level cachePoint blocks for Bedrock Converse, even though the system-message path already honored it. The root cause was that _get_cache_point_block was called without the model argument (or not called at all, with a hardcoded {"type": "default"} block) in all eight message-level invocations.

• factory.py: Threads model= through all eight _get_cache_point_block call sites in _bedrock_converse_messages_pt, BedrockConverseMessagesProcessor._bedrock_converse_messages_pt_async, and _convert_to_bedrock_tool_call_invoke; also refactors the tool-result path to capture the actual cache_control dict (instead of a boolean flag) so the TTL value reaches _build_cache_point_block.
• test_converse_transformation.py: Adds two parametrized tests covering user, tool-call, and tool-result placements across supported (claude-opus-4-7) and unsupported (claude-3-5-sonnet) models, with sync and async paths asserted to be equal.

Confidence Score: 5/5

Safe to merge. The change is a targeted, additive fix that threads an existing parameter to call sites that were already present; it does not alter any API surface, add new branching logic, or touch any auth/security paths.

The refactoring is minimal and correct: the tool-result boolean flag is replaced by capturing the actual dict (no behavioral change when no TTL is set, correct behavior when TTL is set), and the model argument is simply plumbed through to existing helper calls. The is_claude_4_5_on_bedrock gate is driven by model_prices_and_context_window.json, so newly-supported models will pick it up automatically without code changes. The new tests cover all three placement types and both sync/async paths, and make no real network calls.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/prompt_templates/factory.py	All eight _get_cache_point_block call sites in the message and tool-call paths now receive model=model; tool-result path now captures the actual cache_control dict instead of a boolean, preserving the TTL value correctly.
tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py	New parametrized tests for supported/unsupported model TTL behavior across all three placement types (user, tool_call, tool-result); pure transformation tests with no real network calls.

_{Reviews (1): Last reviewed commit: "fix(bedrock): honor cache_control ttl on..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR fixes a silent TTL downgrade in Bedrock Converse prompt caching: message-level and per-tool-call cache_control blocks with ttl: "1h" were emitting a plain {"type": "default"} cachePoint because the model argument needed by _get_cache_point_block's allow-list gate was never threaded through the message-level paths (only the system-message path had it wired). The fix is a minimal, mechanical thread-through: the model parameter is now passed at all 8 call sites in _bedrock_converse_messages_pt / _pt_async, and tool-result blocks now capture the actual cache_control dict instead of reducing it to a boolean.

• _convert_to_bedrock_tool_call_invoke gains an optional model parameter and delegates per-tool-call cache_control to _get_cache_point_block, replacing two hardcoded CachePointBlock(type="default") constructions; both callers (sync and async) pass model=model.
• Tool-result blocks in both sync and async paths now preserve the full cache_control dict (tool_msg_cache_control) instead of a boolean flag, enabling _get_cache_point_block to include ttl when the model's pricing entry carries cache_creation_input_token_cost_above_1hr.
• New parameterized tests cover user, tool-call, and tool-result placements for both a supported (global.anthropic.claude-opus-4-7) and unsupported (anthropic.claude-3-5-sonnet-20240620-v1:0) model, asserting sync/async parity and the presence or absence of ttl in the emitted cachePoint.

Confidence Score: 5/5

Safe to merge — the change is a mechanical parameter thread-through with no new logic, and the allow-list gate already uses model_prices_and_context_window.json rather than hardcoded patterns.

All eight call sites are updated consistently across both sync and async paths. The tool-result boolean-to-dict refactor preserves existing behavior for non-TTL cache_control while correctly forwarding TTL when present. The new tests verify both the happy path and the guard, and confirm sync/async parity. No existing tests are weakened or removed.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/litellm_core_utils/prompt_templates/factory.py	Threads model parameter through all 8 _get_cache_point_block call sites in the Bedrock Converse message transformation paths (sync + async), and captures the actual cache_control dict (instead of a boolean sentinel) for tool-result blocks so TTL survives into the cachePoint block
tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py	Adds parameterized tests (user / tool_call / tool placements × supported / unsupported models) covering the sync and async transformation paths; pure in-memory tests with no network calls

_{Reviews (2): Last reviewed commit: "fix(bedrock): honor cache_control ttl on..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[arashne]

Gentle nudge on this one, and some fresh precedent for it.

Since I opened this, #31929 ("honor ttl for tool_config cache injection points") merged and shipped in 1.90.3 / 1.91.0. That threads ttl through the location: "tool_config" path. This PR is the same class of fix for the two paths #31929 didn't touch: message-level cache_control blocks and per-tool-call blocks, where _get_cache_point_block is still called without model= (or replaced by a hardcoded CachePointBlock(type="default")), so ttl: "1h" silently degrades to 5m while the system-message path honors it.

I re-checked against v1.91.0: all eight message-path call sites still omit model, and the tool-call/tool-result paths still hardcode {"type": "default"}. Root cause + repro are in #32154.

CI is green (Codecov reports full coverage on the changed lines), Greptile scored it 5/5, and the CLA is signed. @mateo-berri, since you merged the sibling tool_config fix, would you be up for reviewing this one?

[arashne]

Landed upstream as #32551 (142d5aa, with co-author credit) — the message-level paths via #32538, and the tool-result / per-tool-call paths from here. Full ttl fix is in; closing this fork PR. Thanks @mateo-berri for carrying it across CI.