fix(bedrock): map Invoke streaming cacheRead/cacheWrite token counts into usage

[devin-ai-integration]

TLDR

Problem this solves:

• Bedrock Invoke streaming dropped cacheRead/cacheWrite token counts
• Cache-heavy traffic was billed as fresh input
• Usage-only logging fallback ignored cache-write tokens

How it solves it:

• Map both cache fields from amazon-bedrock-invocationMetrics into Anthropic usage
• Merge metrics into existing chunk usage instead of overwriting it
• Read cache_creation_input_tokens from message_delta in the fallback

Relevant issues

Fixes #34497

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Pending: a live streaming call against a Bedrock Invoke Anthropic model with a large cached system prompt, showing usage.cache_read_input_tokens / usage.cache_creation_input_tokens and the resulting spend before and after the fix. Will be added to this PR once captured

Type

🐛 Bug Fix

Changes

On the Invoke path the cache breakdown exists only in the camelCase amazon-bedrock-invocationMetrics block on the final chunk, so the decoder was the last place it could be recovered; it read inputTokenCount / outputTokenCount only and replaced the chunk's usage wholesale, which meant the promotion step added for LIT-2411 (which works on snake_case fields already on message_start / message_stop) never saw any cache tokens to promote. The decoder now merges the full metric set into whatever usage the chunk already carries:

_INVOCATION_METRICS_TO_ANTHROPIC_USAGE = {
    "inputTokenCount": "input_tokens",
    "outputTokenCount": "output_tokens",
    "cacheReadInputTokenCount": "cache_read_input_tokens",
    "cacheWriteInputTokenCount": "cache_creation_input_tokens",
}

chunk_data["usage"] = {**(chunk_data.get("usage") or {}), **mapped_metrics}

Since the cache counts now land on the final message_delta, _build_usage_only_response_from_chunks (the recovery path used when stream_chunk_builder cannot reassemble a large agentic stream) also had to stop reading cache-write tokens from message_start only; it now picks up cache_creation_input_tokens and the 5m/1h cache_creation split from message_delta too, so those streams are no longer billed with cache-write at zero.

For the reported numbers (1 uncached input, 421714 cache read, 1139 cache write, 162 output) reconstructed usage goes from prompt_tokens=1 to prompt_tokens=422854 with the cached tokens priced at cache rates rather than as fresh input.

Regression coverage: decoder mapping and non-clobbering of existing usage, an end-to-end Invoke stream (decoder -> bedrock_sse_wrapper -> logging reconstruction -> completion_cost) asserting cache tokens survive and the cached cost is below the equivalent all-fresh-input cost, plus a usage-only fallback case with the cache breakdown on message_delta. All four fail on the parent commit

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/9f4c540664624f00925465fcdc0ce405

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

Maps Bedrock Invoke streaming cache metrics into Anthropic usage and preserves them through fallback reconstruction.

• Adds cache-read and cache-write token mappings for Bedrock invocation metrics while retaining existing usage fields.
• Recovers cache-creation totals and 5-minute/1-hour details from message_delta events in usage-only logging fallback.
• Adds hermetic regression coverage for metric mapping, usage merging, stream reconstruction, and cached-token cost calculation.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness or security issues identified.

The mapped cache metrics retain the expected Anthropic usage semantics, flow through streaming reconstruction, and are covered by hermetic accounting and cost regression tests.

Important Files Changed

Filename	Overview
litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py	Correctly maps Bedrock cache metrics into Anthropic usage while preserving non-conflicting usage fields.
litellm/proxy/pass_through_endpoints/llm_provider_handlers/anthropic_passthrough_logging_handler.py	Extends usage-only reconstruction to retain cache-write totals and tier details reported on the final delta.
tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py	Adds hermetic unit and end-to-end coverage proving cache metrics survive decoding, reconstruction, and cost calculation.
tests/test_litellm/proxy/pass_through_endpoints/llm_provider_handlers/test_anthropic_passthrough_logging_handler.py	Adds focused regression coverage for recovering cache-creation usage from message-delta events.

_{Reviews (1): Last reviewed commit: "fix(bedrock): map invoke streaming cache..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_bedrock_invoke_cache_token_metrics (be15986) with litellm_internal_staging (8177230)}

[tin-berri]

Closing as stale: no activity in the last 30 days. Feel free to reopen if this is still relevant.