fix(cost): bill gpt-5.6 prompt cache reads at the cache read rate

[devin-ai-integration]

TLDR

Problem this solves:

• gpt-5.6 cache reads billed at full input rate
• /v1/messages usage dropped cache read/write tokens

How it solves it:

• Map Responses API cache tokens into Anthropic usage
• Price Anthropic-shaped usage with the Anthropic usage mapper
• Pin gpt-5.6 flex/priority cache write rates in tests

Relevant issues

Fixes #34801

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live proxy against real OpenAI openai/gpt-5.6-sol with a ~4.5k token system prompt sent twice so the second call hits OpenAI's prompt cache; the deployment advertises /v1/responses, so /v1/messages goes through the Anthropic-to-Responses bridge

PREFIX=$(python3 -c "print('Pack my box with five dozen liquor jugs. ' * 450)")
for run in warm cached; do
  curl -sS -D /tmp/h.txt http://localhost:4000/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d "$(jq -n --arg p "$PREFIX" '{model:"openai-gpt-5.6-sol",max_tokens:16,system:$p,messages:[{role:"user",content:"Reply with the number 1"}]}')" | jq -c '.usage'
  grep -i "^x-litellm-response-cost:" /tmp/h.txt
done

Before, on the base branch at bcce83a17e; the cache fields are dropped from usage and all 4517 prompt tokens are billed at the uncached input rate ($5.00/M) even though 4514 of them were cache reads ($0.50/M), a 9.4x overstatement on this request

{"input_tokens":4517,"output_tokens":5}
x-litellm-response-cost: 0.022735000000000002

After, at 69a8043667; the warm call surfaces the cache write and prices it at the cache write rate (3 * $5.00/M + 4514 * $6.25/M + 5 * $30.00/M) and the cached call prices the reads at the cache read rate (3 * $5.00/M + 4514 * $0.50/M + 5 * $30.00/M), matching what OpenAI bills

{"input_tokens":3,"output_tokens":5,"cache_creation_input_tokens":4514}
x-litellm-response-cost: 0.028377500000000003
{"input_tokens":3,"output_tokens":5,"cache_read_input_tokens":4514}
x-litellm-response-cost: 0.0024219999999999997

Streaming /v1/messages on the same deployment reports the same split in message_delta, also at 69a8043667

curl -sS http://localhost:4000/v1/messages -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d "$(jq -n --arg p "$PREFIX" '{model:"openai-gpt-5.6-sol",max_tokens:16,stream:true,system:$p,messages:[{role:"user",content:"Reply with the number 1"}]}')" | grep -o '{"type": "message_delta".*'

{"type": "message_delta", "delta": {"stop_reason": "end_turn", "stop_sequence": null}, "usage": {"input_tokens": 3, "output_tokens": 5, "cache_read_input_tokens": 4514}}

Type

🐛 Bug Fix

Changes

Two things went wrong on the way from OpenAI usage to spend

The Anthropic Messages to Responses API adapter only copied the aggregate input_tokens and output_tokens, so input_tokens_details.cached_tokens and cache_write_tokens never reached the Anthropic usage object. Both the non-streaming and streaming paths now share one conversion that reuses the existing ResponseAPILoggingUtils and Anthropic usage helpers, so input_tokens excludes cache tokens and the cache counts are reported as cache_read_input_tokens / cache_creation_input_tokens

Cost calculation then misread that Anthropic-shaped usage. _get_usage_object and completion_cost detect Responses API usage by the presence of input_tokens plus output_tokens, which Anthropic usage also has, and that converter only understands nested input_tokens_details; the top-level cache_read_input_tokens and cache_creation_input_tokens were dropped and the whole prompt was billed as fresh input. AnthropicConfig.is_anthropic_usage_object now identifies that shape by its cache keys (and the absence of chat-shaped prompt_tokens), and those usage objects go through AnthropicConfig.calculate_usage, which already splits cache reads, cache writes and fresh input correctly. This also fixes the response cost header and cost_breakdown for native Anthropic /v1/messages traffic, which previously disagreed with the response_cost computed during success logging. Following review feedback, the detection heuristic now documents why requiring a cache key is deliberate, and tests pin that Responses-shaped usage with nested cached tokens is never classified as Anthropic and that a missing usage object maps to zeroed Anthropic usage

The branch originally also added cache_creation_input_token_cost_flex / cache_creation_input_token_cost_priority to ModelInfo, but the base branch has since shipped those fields on its own, so after the merge this PR only keeps a regression test pinning the gpt-5.6 flex and priority cache write rates

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/f8089b5a5c7f4c55ad317c56d8a632fa

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

This PR corrects prompt-cache usage propagation and cost calculation for Responses API traffic exposed through Anthropic Messages

• Converts Responses API cache-read and cache-write token details into Anthropic usage fields for streaming and non-streaming responses
• Routes Anthropic-shaped usage through the provider-owned usage mapper before calculating cost
• Adds regression coverage for usage-shape detection, missing usage, cached-token pricing, and GPT-5.6 service-tier cache-write rates

Confidence Score: 5/5

The PR appears safe to merge

No blocking failure remains

Important Files Changed

Filename	Overview
litellm/cost_calculator.py	Dispatches Anthropic-shaped cache usage to the provider-owned mapper in both cost normalization paths
litellm/llms/anthropic/chat/transformation.py	Adds shape detection that distinguishes Anthropic cache usage from chat and Responses API usage
litellm/llms/anthropic/experimental_pass_through/responses_adapters/transformation.py	Centralizes conversion of Responses API usage into Anthropic cache-aware usage
litellm/llms/anthropic/experimental_pass_through/responses_adapters/streaming_iterator.py	Reuses the shared usage conversion for final streaming message usage

_{Reviews (2): Last reviewed commit: "chore: keep base field ordering for serv..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[ErenAta16]

The billing bug is worth fixing and routing Anthropic-shaped usage through the Anthropic mapper is the right call. One thing about the detection heuristic that I'd want settled before this lands, because it decides which pricing path a request takes.

is_anthropic_usage_object requires a cache key to be present:

if "prompt_tokens" in usage_object or "input_tokens" not in usage_object:
    return False
return any(key in usage_object for key in ("cache_read_input_tokens", "cache_creation_input_tokens"))

Ran it over the shapes it has to separate:

anthropic w/ cache read      -> True
anthropic w/ cache write     -> True
anthropic, no cache activity -> False
openai chat usage            -> False
responses api usage          -> False

The first four rows are what you want. The third is the one I'd question: an Anthropic response that had no cache activity carries no cache_* keys, so it's classified as not-Anthropic and falls through to the Responses mapping, which is the path this PR exists to avoid. That's probably harmless today if the two mappings agree on plain input_tokens/output_tokens, but it means the function's name promises more than it delivers, and the classification depends on a pricing-relevant field being non-empty rather than on the payload's shape.

Concretely it makes the routing state-dependent: the same model on the same endpoint takes one path on a cache hit and the other on a cache miss. If the two mappings ever diverge on anything besides cache tokens, that shows up as an intermittent cost discrepancy which is unpleasant to chase.

"input_tokens" in u and "prompt_tokens" not in u alone would classify on shape rather than content. If the cache-key requirement is deliberate (say, because Responses usage also uses input_tokens and the cache keys are the only discriminator) then that's fine, but it's worth a line in the docstring saying so, since as written the docstring explains why the cache keys matter rather than why they're required for detection.

Second, smaller: completion_cost now has two sequential checks that both rewrite _usage, with the Anthropic one first. Worth confirming a Responses payload that happens to carry cache_read_input_tokens can't be captured by the first branch, since after this change the elif means it never reaches _is_response_api_usage. My table above suggests it can't with current shapes, but it's the kind of ordering that's easy to invalidate later.

The ModelInfo additions for flex/priority cache_creation look mechanical and consistent with the existing input_cost_per_token_flex pattern, no concerns there.

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_gpt56_cache_token_pricing (69a8043) with litellm_internal_staging (732bba0)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (69a8043) during the generation of this report, so 732bba0 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai

[mateo-berri]

If the cache-key requirement is deliberate (say, because Responses usage also uses input_tokens and the cache keys are the only discriminator) then that's fine, but it's worth a line in the docstring saying so

Deliberate for exactly that reason. The docstring now says so, and a test pins that Responses-shaped usage never matches the Anthropic branch (e8a80b9)

[mateo-berri]

LGTM