fix(bedrock): price 1h Converse cache writes at the 1h rate

[devin-ai-integration]

TLDR

Problem this solves:

• Bedrock Converse 1h cache writes billed at 5m rate
• Roughly 37% undercount on every 1h cache write

How it solves it:

• Parse Bedrock's per-TTL cacheDetails cache write breakdown
• Feed the 5m/1h split into the existing tiered pricing

User Flow

Before: an app that caches a large system prompt with a 1 hour TTL on Bedrock Converse is charged the 5 minute cache write rate, so its LiteLLM spend reads about 37% under what AWS bills

1. They send POST http://localhost:51423/v1/chat/completions with "model": "bedrock-converse-opus-4-8" and an 11.6k token system block carrying "cache_control": {"type": "ephemeral", "ttl": "1h"}
2. The 200 response reports "cache_creation_input_tokens": 11632 and a prompt_tokens_details object with no per-TTL breakdown at all
3. The x-litellm-response-cost header on that response reads 0.07288, which is 11632 tokens priced at the 5 minute rate of $6.25/M
4. They open http://localhost:51423/ui/?page=logs and see the same understated spend on the request, while their AWS bill charges the 1 hour rate of $10/M, so $0.11648

After: the same request is charged the 1 hour rate, so LiteLLM spend lines up with the AWS bill

1. They send the same POST http://localhost:51423/v1/chat/completions with the same 1h cache_control block
2. The 200 response reports "cache_creation_input_tokens": 11630 plus a "cache_creation_token_details" object showing "ephemeral_1h_input_tokens": 11630 and "ephemeral_5m_input_tokens": 0
3. The x-litellm-response-cost header now reads 0.11648, matching the 1 hour rate
4. http://localhost:51423/ui/?page=logs shows that request at the corrected spend, and an otherwise identical request with a "ttl": "5m" block still comes back at 0.07288

Relevant issues

Fixes #36760

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Real Bedrock Converse calls against global.anthropic.claude-opus-4-8 in us-east-1, no mocks, real spend. Each run uses a unique nonce inside the cached system prompt so it is a fresh cache write rather than a read. Proxy booted with

python litellm/proxy/proxy_cli.py --config repro_config.yaml --port <port> --detailed_debug

where the config declares one deployment, bedrock/converse/global.anthropic.claude-opus-4-8 in us-east-1

Request sent in both runs

curl -sS -D hdr.txt http://127.0.0.1:<port>/v1/chat/completions \
  -H 'Content-Type: application/json' -H 'Authorization: Bearer sk-1234' \
  --data '{"model":"bedrock-converse-opus-4-8","max_tokens":16,
    "messages":[{"role":"system","content":[{"type":"text","text":"You are a terse assistant. Reference corpus follows.\n<~11.6k tokens>","cache_control":{"type":"ephemeral","ttl":"1h"}}]},
                {"role":"user","content":"Reply with exactly the word OK."}]}'
grep -i x-litellm-response-cost hdr.txt

Before, at merge base 5f2986a1f3c9c4aee6dec7ac36d02d7ca96110d5

    "usage": {
        "prompt_tokens": 11648,
        "completion_tokens": 4,
        "prompt_tokens_details": {
            "cached_tokens": 0,
            "text_tokens": 16,
            "cache_write_tokens": 11632,
            "cache_creation_tokens": 11632
        },
        "cache_creation_input_tokens": 11632,
        "cache_read_input_tokens": 0
    }
--- response headers (cost) ---
x-litellm-response-cost: 0.07288

$0.07288 is 16*5e-6 + 4*2.5e-5 + 11632*6.25e-6, the 5 minute rate, while the correct 1 hour figure is 16*5e-6 + 4*2.5e-5 + 11632*1.0e-5 = 0.11650

After, at PR tip 7a42dccbc79b6be0bea5b5a2b4e92f241182874d

    "usage": {
        "prompt_tokens": 11646,
        "completion_tokens": 4,
        "prompt_tokens_details": {
            "cached_tokens": 0,
            "text_tokens": 16,
            "cache_write_tokens": 11630,
            "cache_creation_tokens": 11630,
            "cache_creation_token_details": {
                "ephemeral_5m_input_tokens": 0,
                "ephemeral_1h_input_tokens": 11630
            }
        },
        "cache_creation_input_tokens": 11630,
        "cache_read_input_tokens": 0
    }
--- response headers (cost) ---
x-litellm-response-cost: 0.11648000000000001

$0.11648 is 16*5e-6 + 4*2.5e-5 + 11630*1.0e-5, the 1 hour rate

Same tip, same script but with "ttl": "5m", to show 5 minute writes are untouched

        "prompt_tokens_details": {
            "cache_creation_tokens": 11632,
            "cache_creation_token_details": {
                "ephemeral_5m_input_tokens": 11632,
                "ephemeral_1h_input_tokens": 0
            }
        }
--- response headers (cost) ---
x-litellm-response-cost: 0.07288

Independent second run at the same tip, and the recorded spend matches the header

$ curl -s "http://127.0.0.1:51423/spend/logs?request_id=chatcmpl-06c3f063-..." -H 'Authorization: Bearer sk-1234'
spend 0.11651    cache_creation_token_details {"ephemeral_1h_input_tokens": 11633, "ephemeral_5m_input_tokens": 0}
$ curl -s "http://127.0.0.1:51423/spend/logs?request_id=chatcmpl-84b1970c-..." -H 'Authorization: Bearer sk-1234'
spend 0.07288625 cache_creation_token_details {"ephemeral_1h_input_tokens": 0, "ephemeral_5m_input_tokens": 11633}

The only commit after 7a42dcc is a ruff format reflow of one line, no behavior change

A direct boto3 converse call confirms Bedrock reports the split, including mixed cache points in one request, which the fix maps straight through

"cacheDetails": [{"ttl": "5m", "inputTokens": 10200}, {"ttl": "1h", "inputTokens": 15622}]

Type

🐛 Bug Fix

Caveats (if any)

• Unknown future TTLs fall back to the aggregate rate
• Bedrock passthrough route still undercounts, see [Bug]: Bedrock passthrough bills 1-hour cache writes at the 5-minute rate (cache_creation.ephemeral_1h_input_tokens not propagated) #29432

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/2279c9dc9acb461db7a4393ba5ce6d45
Requested by: @mateo-berri

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

The PR maps Bedrock Converse’s per-TTL cache-write usage into LiteLLM’s existing tiered cache-pricing fields.

• Adds normalized 5-minute and 1-hour cache-creation token details with aggregate-pricing fallback when the breakdown is unavailable or incomplete.
• Extends Bedrock usage types to represent cacheDetails.
• Adds transformation and cost regression tests for 5-minute, 1-hour, mixed, missing, and unknown TTL details.

Confidence Score: 5/5

The PR appears safe to merge, with the changed usage fields flowing correctly into existing tiered cache-write pricing.

The implementation reconciles known TTL buckets against aggregate cache-write tokens before using the detailed pricing path, preserves aggregate fallback behavior for incomplete data, and exercises the corrected cost calculation with focused tests.

Important Files Changed

Filename	Overview
litellm/llms/bedrock/chat/converse_transformation.py	Parses and validates Bedrock’s per-TTL cache-write breakdown before attaching it to normalized prompt usage.
litellm/types/llms/bedrock.py	Adds the optional typed representation of Bedrock Converse cache-detail entries.
tests/test_litellm/llms/bedrock/chat/test_converse_transformation.py	Covers TTL splitting, fallback cases, mixed writes, and corrected 1-hour cache-write pricing.

_{Reviews (1): Last reviewed commit: "fix(bedrock): price 1h Converse cache wr..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_bedrock_converse_cache_details (ac63216) with litellm_internal_staging (add095b)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (7fcca52) during the generation of this report, so add095b was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

Closing as superseded: #36762 landed this fix (merged as abf6ef9), with this PR's cost regression test ported onto it.