fix(bedrock): split Converse 1h vs 5m cache writes by cacheDetails TTL

[Anuj7411]

TLDR

Problem this solves:

• Bedrock Converse 1-hour cache writes get billed at the 5-minute rate
• 1h cache writes undercounted by roughly 31 to 37 percent

How it solves it:

• parse the per-TTL cacheDetails array Bedrock returns on TokenUsage
• populate cache_creation_token_details so the cost calc applies the 1h rate to 1h writes

User Flow

Before: a developer running Claude with a 1-hour prompt cache TTL on the Bedrock Converse path sees cache-write spend that is too low

1. They send a request with a 1h cache checkpoint and get a response whose usage reports, say, 3000 cache-write tokens at 1h TTL and 1000 at 5m
2. The proxy bills all 4000 tokens at the 5-minute cache-write rate
3. Their logged spend for that request is about 31 percent under the real AWS charge

After: the same request bills each TTL bucket at its own rate

1. They send the same request with the same 1h and 5m cache-write split
2. The proxy bills the 3000 1h tokens at the 1h rate and the 1000 5m tokens at the 5m rate
3. Their logged spend matches what AWS actually charges

Relevant issues

Fixes #36760

Linear ticket

Type

🐛 Bug Fix

Changes

AmazonConverseConfig._transform_usage only read the aggregate cacheWriteInputTokens field, so cache_creation_token_details stayed unset and calculate_cache_writing_cost billed the whole cache-write count at the 5-minute rate. Bedrock's Converse TokenUsage actually returns a cacheDetails array with a per-TTL breakdown (inputTokens plus ttl of 5m or 1h, sorted 1h first), which ConverseTokenUsageBlock did not declare, so it was dropped before it reached the transform. This declares cacheDetails on the usage block, adds a ConverseCacheDetailBlock type, and sums the per-TTL buckets into CacheCreationTokenDetails(ephemeral_5m_input_tokens, ephemeral_1h_input_tokens), the same shape the Anthropic direct provider and Vertex already produce. When cacheDetails is absent the split stays None and behavior is unchanged.

Screenshots / Proof of Fix

Verified against the real model_cost map at commit 860fa15 (Bedrock anthropic.claude-haiku-4-5: 5m cache-write 1.25e-6, 1h 2e-6), running a Converse usage with a 3000 token 1h write and a 1000 token 5m write through _transform_usage then completion_cost:

cache-write cost NEW (split): 0.00725000   OLD (all 5m): 0.00500000   -> 31.0% undercount fixed
total completion_cost: 0.00760000

Regression tests in tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py: test_transform_usage_splits_cache_writes_by_ttl (asserts the 1h/5m split) and test_transform_usage_without_cache_details_leaves_ttl_split_unset (guards the no-regression path). Both pass; they fail if the parse is removed.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing Anuj7411:litellm_bedrock_converse_1hr_cache_write_ttl (e454dd9) with litellm_internal_staging (973329e)}

[mateo-berri]

Closing as superseded: #36762 landed the cacheDetails split with a reconciliation guard and merged today. Thanks for the contribution!