fix(cost): bill cache-read tokens at the input rate when the map has no cache-read rate

[devin-ai-integration]

TLDR

Problem this solves:

• Cache-read tokens bill at $0 when the map has no cache-read rate
• Cache-write tokens on the same entry already bill at the input rate
• The caching-savings report priced the same reads at the input rate

How it solves it:

• Absent cache-read rate now resolves to the input rate, like cache-creation
• Explicit cache_read_input_token_cost: 0.0 still means free
• Drops the missing_cache_read_uses_input flag, so both paths agree

User Flow

Before: a team routing Bedrock traffic to a model whose map entry lists only input and output prices sees cached prompt tokens billed as free

1. They send POST https://litellm-domain/v1/chat/completions for a deployment whose cost map entry has input_cost_per_token and output_cost_per_token but no cache_read_input_token_cost
2. The provider reports 640 fresh input tokens, 12288 cache-read tokens and 380 output tokens
3. The response carries x-litellm-response-cost: 0.0005222, which is the fresh input plus output only, and no cache-read header
4. https://litellm-domain/ui/?page=logs shows 12928 prompt tokens at a spend that covers 640 of them

After: the same request bills the cached tokens as ordinary input

1. They send the same POST https://litellm-domain/v1/chat/completions for the same deployment
2. The provider reports the same 640 fresh, 12288 cached and 380 output tokens
3. The response carries x-litellm-response-cost: 0.00347132 with x-litellm-response-cost-cache-read: 0.00294912
4. https://litellm-domain/ui/?page=logs shows 12928 prompt tokens at a spend that covers all of them

Relevant issues

Surfaced by the deterministic cost suite in #41328, whose two meta.llama4-maverick-17b-instruct-v1:0 cache-read cells go green with this fix

Affected release

Linear ticket

Resolves LIT-8111

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Two live proxies on the same Postgres and the same config (tests/e2e/gateway/cost_calculation_ci_config.yml, LITELLM_MODEL_COST_MAP_URL pointing at the test-owned tests/e2e/cost_map.json, MODEL_COST_MAP_MIN_MODEL_COUNT=1, MODEL_COST_MAP_MAX_SHRINK_RATIO=0). The map entry for meta.llama4-maverick-17b-instruct-v1:0 has input_cost_per_token 2.4e-7 and output_cost_per_token 9.7e-7 and no cache keys. The upstream is the scripted Bedrock Converse sidecar from #41328 replaying a response with 640 fresh input, 12288 cache-read and 380 output tokens, because Bedrock does not offer prompt caching on that model, which is exactly why the real map has no cache-read rate for it

Shared setup on each proxy:

curl -s http://localhost:PORT/model/new -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
  -d '{"model_name":"lit8111-llama","litellm_params":{"model":"bedrock/converse/meta.llama4-maverick-17b-instruct-v1:0","api_key":"sk-scripted-provider","api_base":"http://127.0.0.1:9100/lit8111/bedrock","aws_region_name":"us-east-1"}}'

Before (0bd8b7f)

1. curl -i http://localhost:4000/v1/chat/completions -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' -d '{"model":"lit8111-llama","messages":[{"role":"user","content":"say hi"}]}'
2. Observed headers:

x-litellm-response-cost: 0.0005222
x-litellm-response-cost-input: 0.0001536
x-litellm-response-cost-output: 0.0003686

3. Spend row: spend=0.00052220 prompt_tokens=12928 completion_tokens=380, so 12288 of the 12928 prompt tokens were billed at $0

After (af61d92)

1. Same curl against http://localhost:4002/v1/chat/completions
2. Observed headers:

x-litellm-response-cost: 0.0034713199999999995
x-litellm-response-cost-input: 0.00015359999999999983
x-litellm-response-cost-output: 0.0003686
x-litellm-response-cost-cache-read: 0.0029491199999999995

3. Spend row: spend=0.00347132 prompt_tokens=12928 completion_tokens=380. The difference from Before is exactly 12288 * 2.4e-7 = 0.00294912

Type

🐛 Bug Fix

Caveats (if any)

Medium

• Spend goes up for any map entry that reports cache reads without a cache-read price
  • Today that is the intended fallback for cache writes, so this aligns reads with it
  • An entry that really is free for reads must say cache_read_input_token_cost: 0.0

Low

• test_missing_cache_read_policy_preserves_billing pinned the $0 behavior and was rewritten to the new invariant
• The /cost/estimate endpoint now reports the input rate as the effective cache-read rate for such entries; its test that pinned 0.0 was updated
• Proof uses a scripted upstream rather than a paid call, since no Bedrock model both lacks a cache-read price and returns cache reads

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/977f8f8050c749b2b52ca0d077a3196e
Open in Devin Desktop: https://app.devin.ai/desktop/session/977f8f8050c749b2b52ca0d077a3196e?variant=devin

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[cursor]

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_lit_8111_cache_read_missing_rate (af61d92) with main (0bd8b7f)}

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge, with the implementation and tests consistently applying the intended cache-read pricing fallback.

Summary

This PR makes missing cache-read prices fall back to the resolved input-token rate while preserving explicit zero-priced cache reads.

• Removes the internal alternate cache-read fallback policy so billing and caching-savings calculations agree.
• Preserves tiered, service-tier, and off-peak input-rate resolution.
• Adds direct billing coverage and updates cost-estimation tests for the new fallback behavior.

_{Reviews (2) · Last reviewed commit: "test(cost_tracking): expect cache reads ..."}

[cursor]

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

[devin-ai-integration]

Pushed af61d92 updating the cost estimate test that pinned $0 cache reads. Rebutted the ungated billing finding as intentional

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

LGTM. Thanks!