fix(databricks): bill cached tokens at cache rates and add missing Claude pricing

[mateo-berri]

TLDR

Problem this solves:

• Databricks billed cache reads at the full input rate
• Databricks registry entries carried no cache read or write rates
• Claude Opus 4.7, 4.8, 5, Sonnet 5 and Fable 5 were unpriced

How it solves it:

• Databricks cost calculation now runs through the cache-aware calculator
• Adds cache read and write rates to 42 existing entries
• Adds the five missing Claude entries at published DBU rates

User Flow

Before: a developer tracking Databricks Claude spend pays full input price for every cached token, and sees no spend at all on the newest models

1. They configure databricks/databricks-claude-sonnet-4-5 and send POST https://litellm-domain/v1/chat/completions with a prompt long enough to be cached
2. The response comes back 200, reporting 1,500 prompt tokens of which 800 are cache reads and 500 are cache writes, plus 100 completion tokens
3. They send that usage to POST https://litellm-domain/spend/calculate and get back {"cost": 0.005999987}, the same amount an entirely uncached request of that size costs
4. They switch to databricks/databricks-claude-opus-5 and repeat, and https://litellm-domain/spend/calculate returns {"cost": 0.0}, so the request lands on https://litellm-domain/ui/?page=logs at $0 spend

After: cache reads and cache writes bill at their own rates, and the newest models report real spend

1. They configure databricks/databricks-claude-sonnet-4-5 and send POST https://litellm-domain/v1/chat/completions with a prompt long enough to be cached
2. The response comes back 200, reporting 1,500 prompt tokens of which 800 are cache reads and 500 are cache writes, plus 100 completion tokens
3. They send that usage to POST https://litellm-domain/spend/calculate and get back {"cost": 0.004215001}, with cache reads and cache writes each billed at their own published rate
4. They switch to databricks/databricks-claude-opus-5 and repeat, and https://litellm-domain/spend/calculate returns {"cost": 0.007025025}, which is what https://litellm-domain/ui/?page=logs now shows for the request

Relevant issues

Linear ticket

Resolves LIT-5977

Pricing sources

Rates come from the published Databricks DBU table for proprietary foundation model serving, converted at the documented $0.070 per DBU that the existing entries already use.

Model	Input DBU/1M	Output DBU/1M	Cache write DBU/1M	Cache read DBU/1M
Opus 4.7	71.429	357.143	89.286	7.143
Opus 4.8	71.429	357.143	89.286	7.143
Opus 5	71.429	357.143	89.286	7.143
Sonnet 5	42.857	214.286	53.571	4.286
Fable 5	142.858	714.286	178.572	14.286

Each dollar figure is that DBU rate multiplied by $0.070, written to the significant digits the multiplication yields. Databricks publishes rates like 71.429 DBU per million tokens rather than round numbers, so the dollar rates come out non-round as well: Opus 5 input is 71.429 x $0.070 = $5.00003 per million tokens, not an even $5.00. A literal such as 3.0002e-07 is therefore the intended cache read rate for Sonnet 5, taken from its published 4.286 cache read DBU, and not a rounding artifact. The values are stored at exactly that precision, so none of them carry the trailing digits a raw float product would leave behind

Cache rates are derived exactly the way input and output are, from the published cache write and cache read DBU for that model, never as a ratio of the input rate. That distinction is worth stating because Databricks does not publish cache rates as clean multiples: Opus 5 lists 7.143 cache read DBU against 71.429 input DBU, giving $0.50001 per million tokens where a flat tenth of input would have given $0.500003. Of the 47 Databricks entries, 33 carry cache rates and the remaining 14 publish none at all and are covered two paragraphs below. Of those 33, 29 take their cache DBU straight from the published table, while databricks-claude-3-7-sonnet and databricks-claude-sonnet-4-1 are absent from it and instead inherit the Sonnet 4 family's rates, which they already match on an implied 42.857 input DBU

Two of those 33 deliberately sit below the published table. databricks-gemini-2-5-pro and databricks-gemini-2-5-flash carry a 20% promotional discount that the published DBU figures exclude and that runs through 2027-01-31, which is exactly why their stored rates land a factor of 1.25 under the table. They are correct as they stand rather than stale, so this PR does not touch them. Their cache rates follow their own discounted input rate instead of the undiscounted published cache DBU, keeping each entry on a single price list. A test pins both the discount and the cache relationship, so whoever drops the 1.25x when the promotion ends has to move the cache rates with it

That promotion is marked on every Gemini row, not just the 2.5 pair, and six covered models have registry entries. The other four, databricks-gemini-3-pro, databricks-gemini-3-1-pro, databricks-gemini-3-flash and databricks-gemini-3-1-flash-lite, store the full list rate and therefore bill about 25% above what Databricks actually charges. Those input and output literals predate this branch and it does not change them. The cache rates it adds for the four are derived from each entry's own stored input rate, which is what Databricks bills a Gemini cache write at, so every field on those entries stays on one price list and a later correction can move all of them together. The tests name the two groups separately for what they store rather than implying the four are exempt from the promotion, and the overcharge is tracked on its own ticket

The change is not uniformly a refund, and it is worth being explicit about the direction. Cache reads fall a long way, 9.7x on the request in the proof below, because they used to bill at the full input rate. Cache writes move the opposite way: they also used to fall back to the input rate, and they now bill the published write premium, 1.25x input on every Claude model here. A workload that writes the cache more often than it reads will see its reported spend go up, and the proof below shows both directions on the same pair of requests.

Sonnet 5 is listed at its standard rates rather than the introductory launch rates (28.571 input and 142.857 output DBU), which run only through 2026-08-31. Pricing entries carry no expiry date, so the choice is which direction to be wrong in. Standard rates overcharge Sonnet 5 by 50% for the nine days left in the introductory window, then become correct on their own. Introductory rates would read correctly for nine days and then undercharge by a third indefinitely, letting real spend outrun an enforced budget with no signal. The introductory numbers are kept in the entry's metadata.notes.

Opus 4.7 and Opus 4.8 are both priced from their own published rate of 71.429 input DBU, matching Opus 4.5 and 4.6. Neither had an entry before, so both fell back to the original Opus 4 at 214.286 DBU and billed $15 per million input tokens where Databricks charges $5. Correcting that drops their billed rate threefold, to $5 in and $25 out per million. It is a correction rather than a discount, but it is a visible decrease for anyone already calling either model. Opus 5, Sonnet 5 and Fable 5 had no fallback to land on at all and billed $0, so for those three the change is a rise from nothing to a real rate

The 14 Databricks models that publish no cache pricing at all, the Llama, Mixtral, Gemma, GPT OSS, MPT and embedding entries, get cache rates equal to their input rate. Routing them through the shared calculator otherwise bills a cached token at zero, because that calculator treats a missing cache rate as free, and a model with no caching discount should charge the ordinary input price instead. On a 10,000 token prompt with 8,000 cache reads this is the difference between $0.0050001 and $0.0010000, a fivefold undercharge, and a test now pins every priced Databricks entry to declaring cache rates so no future entry can regress into it.

Registry changes are additive and confined to databricks/ keys: no entry is removed and no existing rate is changed. Alongside the cache rates, 28 of the 42 existing entries newly declare supports_prompt_caching, which they had left off despite the provider supporting it. An old-formula versus new-formula comparison over an uncached usage block came back identical for all 47 Databricks models, and all seven legacy endpoint aliases kept their existing behaviour.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Shared setup, identical on both legs: two separate proxy processes each booted litellm --config <cfg> --num_workers 2 on two random free high ports, sharing one Postgres database and one Redis, matching the multi-pod shape customers run. Every request is sent through one instance and its spend is read back through the other. LITELLM_LOCAL_MODEL_COST_MAP=True so pricing comes from the checked-out registry. Clients are the OpenAI Python SDK for /v1/chat/completions and /v1/responses and the Anthropic Python SDK for /v1/messages. Cached cases send a large system block carrying cache_control: {"type": "ephemeral"} twice, so the second call is a real cache read against Databricks. Model deployments are databricks/databricks-claude-sonnet-4-5 and databricks/databricks-claude-opus-5 on a live Databricks workspace.

Before (a44bb47)

Cached request on /v1/chat/completions

1. Send the priming call, then the identical call through the other instance with the OpenAI SDK. The second response reports prompt_tokens 9895, cache_read_input_tokens 9882, completion_tokens 4, so only 13 prompt tokens were fresh
2. Read the spend back through the first instance:

$ curl -s -H "Authorization: Bearer $KEY" "http://127.0.0.1:18317/spend/logs?request_id=msg_bdrk_013s2cRNWXJKioF38CXe3bd1"
"spend": 0.02974490113, "prompt_tokens": 9895, "completion_tokens": 4, "call_type": "acompletion"
metadata.cost_breakdown = {"input_cost": 0.029684901050000004, "output_cost": 6.000008000000001e-05, "total_cost": 0.029744901130000004}

3. That is 9895 * 3.00/1M + 4 * 15.00/1M, so all 9882 cache-read tokens were billed at the full input rate. At the published cache-read rate the same request costs 0.00306379759, so the bill is 9.7x too high

Unpriced new model on /v1/chat/completions

1. Send a short completion to databricks/databricks-claude-opus-5. It returns 200 with content ok and usage prompt_tokens 16, completion_tokens 4, so Databricks really served and really billed it
2. Read the spend back through the other instance:

$ curl -s -H "Authorization: Bearer $KEY" "http://127.0.0.1:18317/spend/logs?request_id=msg_bdrk_qas2gynkl4sykjumrcvmh2zoxwov2uricuzt4vlnqv4j7u6t3gfq"
"model": "databricks/databricks-claude-opus-5", "spend": 0.0, "prompt_tokens": 16, "completion_tokens": 4
metadata.cost_breakdown = {"input_cost": 0, "output_cost": 0, "total_cost": 0.0}

3. Spend is exactly 0.0 on a real billable request

Cached request on /v1/messages

1. Run the same two-call cache flow with the Anthropic SDK. The second response reports input_tokens 13, cache_read_input_tokens 9882, output_tokens 4
2. Read the spend back through the other instance:

$ curl -s -H "Authorization: Bearer $KEY" "http://127.0.0.1:19483/spend/logs?request_id=msg_bdrk_01SCsmwARE1jzRz6zykE7pPg"
"spend": 0.02974490113, "prompt_tokens": 9895, "completion_tokens": 4, "call_type": "anthropic_messages"

3. Identical to the chat-completions case, so the route makes no difference

Cached request on /v1/responses

1. Run the same two-call cache flow with the OpenAI SDK responses API. The second response reports input_tokens 9895, input_tokens_details.cached_tokens 9882, output_tokens 4
2. Read the spend back through the other instance:

$ curl -s -H "Authorization: Bearer $KEY" "http://127.0.0.1:19483/spend/logs?request_id=msg_bdrk_016oxJmfGqvkyP7T2BZ28X2t"
"spend": 0.02974490113, "prompt_tokens": 9895, "completion_tokens": 4, "call_type": "aresponses"

3. All three cached routes land on the same 0.02974490113, because the cost calculation never looks at the cache-read split

After (0a869b7)

Commits since this hash (112224a, 9f1191e, 77cae92, 15e8a35) net to test-only changes: the head's tree differs from it solely in tests/test_litellm/llms/databricks/test_databricks_cost_calculator.py, so proxy behavior at the tip is identical to this proven run

Two proxy instances, each started with --num_workers 2, sharing one Postgres and one Redis. Every case is sent through one instance and the spend read back through the other, so the numbers below survive a process boundary. Instance A is 127.0.0.1:39346, instance B is 127.0.0.1:51566.

Cached request on /v1/chat/completions

1. Send a priming call carrying a ~10k-token cache_control: {"type": "ephemeral"} system block through A, then send the identical call again. The second response reports prompt_tokens 10184, cache_read_input_tokens 10170, completion_tokens 4
2. Read the spend back through B:

$ curl -s -X GET "http://127.0.0.1:51566/spend/logs?request_id=msg_bdrk_017bQ6PSke7u8NQrc8DUZBgj" -H "Authorization: Bearer $KEY"
"spend": 0.00315320334, "cache_read_input_tokens": 10170,
"cost_breakdown": {"cache_read_cost": 0.0030512034000000003, "total_cost": 0.00315320334}

3. That is 14 * 2.99999/1M + 10170 * 0.30002/1M + 4 * 15.00002/1M, exact to the last digit, against 0.03061189824 under the old full-rate arithmetic, so 9.71x cheaper. The 14 text tokens are prompt_tokens minus the cache tokens, because Databricks reports prompt_tokens inclusive of them
4. The paired cache write msg_bdrk_01VhKmwPEhWYxD5SFKhyWvqN billed 0.038239194840000004, exactly 14 * 2.99999/1M + 10170 * 3.74997/1M + 4 * 15.00002/1M, so the write rate is live too

Unpriced new model on /v1/chat/completions

1. Send a short completion to databricks/databricks-claude-opus-5 through B. It returns 200 with usage prompt_tokens 22, completion_tokens 7
2. Read the spend back through A:

$ curl -s -X GET "http://127.0.0.1:39346/spend/logs?request_id=msg_bdrk_pvlfwlgc7ukqwutwf5r5ijxy6bveqbptkgmwxr5d2jfcroftxqta" -H "Authorization: Bearer $KEY"
"spend": 0.00028500073,
"cost_breakdown": {"input_cost": 0.00011000066, "output_cost": 0.00017500007}

3. That is 22 * 5.00003/1M + 7 * 25.00001/1M, non-zero where the same call previously billed nothing, and $5.00003/1M is exactly the published 71.429 input DBU at $0.070 per DBU

Cached request on /v1/messages

1. Run the same two-call cache flow with the Anthropic SDK through A. The response reports input_tokens 14, cache_read_input_tokens 10170, output_tokens 4
2. Read the spend back through B:

$ curl -s -X GET "http://127.0.0.1:51566/spend/logs?request_id=msg_bdrk_01My8JVDdoCuYuG5tcppUZES" -H "Authorization: Bearer $KEY"
"spend": 0.00315320334, "call_type": "anthropic_messages",
"cost_breakdown": {"cache_read_cost": 0.0030512034000000003}

3. Identical to the chat-completions case, down to the cache-read component

Cached request on /v1/responses

1. Run the same two-call cache flow with the OpenAI responses SDK through B. The response reports input_tokens 10184, input_tokens_details.cached_tokens 10170
2. Read the spend back through A:

$ curl -s -X GET "http://127.0.0.1:39346/spend/logs?request_id=msg_bdrk_01U4P6UsvoRjADf5shyxg5RC" -H "Authorization: Bearer $KEY"
"spend": 0.00315320334, "call_type": "aresponses",
"cost_breakdown": {"cache_read_cost": 0.0030512034000000003}

3. All three cached routes agree on the cache-aware figure, and the previously unpriced model bills at its published rate

Observations from the run, none of them caused or worsened by this PR:

• spend stores one ULP below cost_breakdown.total_cost on the largest of the values, a float round-trip in that column
• Opus 5 rejects temperature upstream, and drop_params does not strip it
• /spend/logs keys on the provider message id, not the resp_... id the responses SDK hands back
• Databricks returns Anthropic-shaped cache fields on all three routes

Type

🐛 Bug Fix
🆕 New Feature

Caveats (if any)

• Sonnet 5 ships at standard rates, not the expiring introductory ones
• Opus 4.7 and 4.8 now bill at their own rate, threefold below Opus 4's
• Fable 5 rates come from the published table; no endpoint served to verify
• Fable 5 is the one Claude endpoint Databricks documents as text-only, so it ships without vision
• Cache writes now bill the 1.25x premium instead of falling back to input, so write-heavy spend rises
• Gemini 2.5 Pro and Flash carry a 20% promotion through 2027-01-31, so their cache rates track the discount
• Four Gemini 3 entries store the pre-existing list rate and bill 25% high; untouched here, tracked separately
• Rates assume the US $0.070 per DBU on Global endpoints; other regions and In-geo tiers differ
• Long-context tiers above 200k tokens publish separate higher rates that the registry does not model
• New entries shadow future minor versions, so Opus 5.1 inherits Opus 5 pricing
• Opus 4.9 is still unlisted, so it keeps inheriting Opus 4's threefold-higher rate
• No entry carries a 1h-TTL cache write rate, which would bill zero if Databricks ever returned one
• 28 entries newly declare cache support, but auto cache-control injection stays off for Databricks
• The 42 pre-existing entries keep the older metadata.notes wording about the DBU fields
• A pre-existing test swapped opus-5-1 for haiku-5-1, since the new entry makes the former mapped
• Legacy dbrx-instruct and meta-llama-3.1-70b aliases stay unpriced, unchanged
• Custom per-token pricing double-counts cache tokens, a separate pre-existing bug

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

• 15e8a35 passes /live-pr-risk

---

Note

Medium Risk
Changes affect reported customer spend and billing for all Databricks traffic, including large corrections for newly mapped Opus models and cache-heavy workloads; risk is mitigated by extensive tests but pricing JSON mistakes would propagate widely.

Overview
Databricks spend calculation now routes through generic_cost_per_token instead of multiplying raw prompt_tokens by the input rate. Legacy endpoint aliases (e.g. mixtral-8x7b-instruct-v0.1) are normalized via a small prefix map before registry lookup, so cache read/write usage fields can bill at their own rates.

The model price registry (main + backup JSON) is updated in bulk for Databricks: cache_creation_input_token_cost and cache_read_input_token_cost on existing entries, supports_prompt_caching on many chat models, and new priced entries for Claude Opus 4.7/4.8/5, Sonnet 5, and Fable 5 at published DBU-derived dollar rates.

Tests lock in cached vs uncached math, DBU conversion for new models, legacy alias resolution, promotional vs list Gemini rates, and that Databricks Claude models do not receive automatic cache_control injection even when caching is marked supported.

^{Reviewed by Cursor Bugbot for commit 15e8a35. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR makes Databricks spend calculation cache-aware and adds pricing metadata for newer Claude models.

• Routes Databricks token costs through the shared cache-aware calculator.
• Adds cache read/write rates and prompt-caching capabilities to Databricks registry entries.
• Adds pricing and capabilities for Claude Opus 4.7, Opus 4.8, Opus 5, Sonnet 5, and Fable 5.
• Expands regression coverage for cache billing, promotional rates, endpoint aliases, and fallback matching.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains, and the previously reported test typing issue is resolved at the current head.

Important Files Changed

Filename	Overview
litellm/llms/databricks/cost_calculator.py	Replaces direct input/output arithmetic with the shared cache-aware calculator while preserving legacy endpoint normalization.
model_prices_and_context_window.json	Adds Databricks cache rates, caching capability metadata, and five Claude pricing entries.
litellm/model_prices_and_context_window_backup.json	Mirrors the Databricks pricing and capability updates in the backup registry.
tests/test_litellm/llms/databricks/test_databricks_cost_calculator.py	Adds comprehensive billing and registry tests; the previously reported missing type annotations are now present.
tests/test_litellm/integrations/test_anthropic_cache_control_hook.py	Verifies that advertising Databricks prompt caching does not enable automatic Anthropic cache-control injection.
tests/test_litellm/litellm_core_utils/test_fallback_generalizations.py	Updates fallback generalization coverage for newly registered Claude model families.

_{Reviews (6): Last reviewed commit: "test: separate gemini entries storing th..." | Re-trigger Greptile}

[veria-ai]

PR overview

This pull request updates Databricks model pricing so cached tokens are billed at cache-specific rates and adds missing Claude pricing. It also modifies the primary and backup model pricing maps.

Two pricing issues remain open despite one issue having been addressed. Long-context Gemini Pro cache usage is underbilled, allowing users to consume more resources than proxy budgets reflect, while Sonnet 5 rates are applied before their effective date and can prematurely exhaust shared budgets. The tiered cache-pricing gap is the more significant remaining concern because it permits repeatable budget-control circumvention.

Open issues (2)

• Low: Premature rate increase enables shared-budget exhaustion — model_prices_and_context_window.json:14900
• Medium: Long-context cache usage is underbilled — model_prices_and_context_window.json:14948

Fixed/addressed: 1 · PR risk: 6/10

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[veria-ai]

Low: Premature rate increase enables shared-budget exhaustion

These standard rates do not take effect until September 1, 2026, but the calculator and reservation path consume this value immediately. Until then, a team member can make Sonnet 5 requests count at roughly 150% of their actual cost and prematurely exhaust shared team, key, or model budgets; retain the introductory rates until the transition date or add date-aware pricing. Apply the same correction to litellm/model_prices_and_context_window_backup.json.

[mateo-berri]

Your earlier thread asked for exactly these standard rates. No date-aware pricing exists, and undercharging past August 31 breaks budgets worse.

[veria-ai]

Thanks for the detail. I can't automatically confirm this is safe to dismiss, so I'm leaving the thread open for a maintainer to make the call.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_databricks_cache_token_pricing (15e8a35) with litellm_internal_staging (aae36f4)}

[veria-ai]

Medium: Long-context cache usage is underbilled

Databricks increases Gemini Pro cache rates above 200k input tokens, but this entry only defines the short-context rate. Because generic_cost_per_token activates tiered cache pricing only when an input_cost_per_token_above_200k_tokens field exists, a user can repeatedly submit cached prompts above 200k tokens and be charged the lower rate against their proxy budget. The same issue affects databricks-gemini-3-1-pro and databricks-gemini-3-pro; add the input, output, cache-creation, and cache-read above-200k fields to both price maps.

[mateo-berri]

These entries never modeled 200k tiers; input and output share that pre-existing gap. A cache-rate fix is the wrong place to introduce tiered pricing.

[veria-ai]

Thanks for the detail. I can't automatically confirm this is safe to dismiss, so I'm leaving the thread open for a maintainer to make the call.

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 15e8a35. Configure here.}