fix(register_model): preserve built-in cache pricing when registering custom overrides under unmapped keys

[yassin-berriai]

Relevant issues

Resolves LIT-3683

Custom-priced models with only input_cost_per_token and output_cost_per_token set in model_info (no cache pricing) silently bill cache_read and cache_creation tokens at 0. For cache-heavy Anthropic traffic this drops the bulk of the real cost from spend reporting. The customer-reported workarounds align with the mechanism; either spell out the cache costs in the override, or remove the override entirely so the cost calculator falls back to the matching built-in entry.

Linear ticket

LIT-3683

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

What's the bug

Two parallel registration paths in litellm drop cache cost when the user provides partial custom pricing.

The first is in litellm.utils.register_model. When a config registers a custom-priced model whose key shape litellm.get_model_info does not recognize (for example bedrock/bedrock/us.anthropic.claude-sonnet-4-6), the lookup fails and existing_model is set to an empty dict. The merged entry written into litellm.model_cost[<key>] then contains only the fields the user set explicitly; built-in cache pricing fields are absent.

The second is in litellm.Router._create_deployment and Router._add_deployment_to_router. Each deployment is registered under its model_info.id (a UUID-like string). That key is never present in the built-in cost map, so register_model again falls through to an empty base and the resulting entry has no cache fields. This is the path the proxy actually uses at runtime when a deployment has input_cost_per_token/output_cost_per_token set under model_info; _select_model_name_for_cost_calc picks the deploy-id entry over the (correctly-cached) shared backend alias.

In both cases, _get_cost_per_unit later returns the default_value=0.0 for the missing cache_creation_input_token_cost and cache_read_input_token_cost keys, so cache tokens are billed at 0.

The fix

Two pairs of changes, one per registration path.

In register_model (litellm/utils.py): when get_model_info(model=key) fails, before falling back to an empty entry, try to resolve a canonical built-in entry by stripping leading provider prefixes (bedrock/bedrock/x -> x), region prefixes (us., eu., apac., and so on), and the provider-specific suffix via _strip_model_name. The first variant that resolves in litellm.model_cost is inherited as the base so cache pricing and other defaults flow through, while the user's explicit overrides still win on merge. No cross-provider auto-detection is introduced; a variant is only accepted when it exactly matches an existing model_cost key. When nothing resolves and the user supplied no cache pricing, a warning is logged so the 0-cache-cost behavior is no longer silent.

In the router (litellm/router.py): before registering a deployment under its model_info.id, if the deployment has custom input/output pricing but no cache pricing, look up the backend litellm_params.model in the built-in cost map and copy over cache_creation_input_token_cost, cache_read_input_token_cost, and the tiered variants. User-specified cache fields still win; only None/missing entries are inherited. No-op when the backend model has no canonical entry.

Screenshots / Proof of Fix

Live proxy run hitting the real Anthropic API with prompt caching, using the exact config and script attached to LIT-3683 (4 worker proxy, postgres + redis in docker, anthropic/claude-sonnet-4-5-20250929 backend with two deployments that both have only input/output pricing in model_info).

The repro fires three calls per run; one cache write, one cache read against claude-custom-broken, one cache read against claude-custom. Both deployments have the same partial pricing override.

Before the fix, querying the LiteLLM_SpendLogs table after the run:

                  request_id                   |        model_id         |  spend   | prompt_tokens | completion_tokens | cache_read | cache_create
-----------------------------------------------+-------------------------+----------+---------------+-------------------+------------+--------------
 chatcmpl-d68c2aa1-d4d5-4905-b8da-d37665146056 | my-claude-deploy        | 0.000126 |         45014 |                 6 | 45002      | 0
 chatcmpl-af18bbdc-2e74-4a31-8807-91f796789c82 | my-claude-deploy-broken | 0.000186 |         45014 |                10 | 45002      | 0
 chatcmpl-7dc249a8-a51d-480e-a4e7-23fe15feee75 | my-claude-deploy-broken | 0.000153 |         45013 |                 8 | 0          | 45002

The cost_breakdown metadata on the cache-read row reads input_cost=3.6e-05, output_cost=1.5e-04, total=0.000186, i.e. only the 12 regular text tokens were billed; 45002 cache_read tokens contributed 0 to the cost.

After the fix, the same script with the same backend usage:

                  request_id                   |        model_id         |   spend   | prompt_tokens | completion_tokens | cache_read | cache_create
-----------------------------------------------+-------------------------+-----------+---------------+-------------------+------------+--------------
 chatcmpl-c25224ef-40aa-4136-93ed-1c3c8971a125 | my-claude-deploy        | 0.0136716 |         45014 |                 9 | 45002      | 0
 chatcmpl-237e550f-e30f-4509-804a-cbd0182b8a34 | my-claude-deploy-broken | 0.0136716 |         45014 |                 9 | 45002      | 0
 chatcmpl-3e54275c-579f-44f4-8d44-f2f7d42d853d | my-claude-deploy-broken | 0.1689105 |         45013 |                 8 | 0          | 45002

0.0136716 = 12 * 3e-6 + 45002 * 3e-7 + 9 * 1.5e-5. 0.1689105 = 11 * 3e-6 + 45002 * 3.75e-6 + 8 * 1.5e-5. Both match the expected cost when cache rates come from the canonical Claude entry.

Type

Bug Fix

Changes

litellm/utils.py: in register_model, fall back to a prefix-stripped canonical lookup when get_model_info cannot resolve the registered key, so cache pricing survives the merge. Warn when no canonical match is found and no cache pricing was supplied.

litellm/router.py: before registering a deployment under its model_info.id, inherit cache pricing fields from the backend litellm_params.model's built-in entry when the deployment overrides only input/output cost. User-specified cache fields always win.

tests/test_litellm/test_register_model_custom_pricing.py: two new regression tests covering the prefix-strip case and the no-match warning case.

tests/test_litellm/test_router_model_cost_isolation.py: two new regression tests covering the deploy-id case (partial pricing inherits backend cache rates) and the explicit-override case (user-supplied cache rates are preserved).

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 97.91667% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/router.py	94.44%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[yassin-berriai]

Verified the proof-of-fix locally end-to-end. The same Python snippet from the PR body, run against the parent litellm checkout vs the worktree checkout (so we can compare the exact same code paths with and without the fix). Output:

=== BEFORE ===
litellm module: /Users/yassinkortam/Documents/Github/berriai/litellm/litellm/utils.py
registered entry input_cost_per_token = 3.3e-06
registered entry cache_creation_input_token_cost = None
registered entry cache_read_input_token_cost = None

Usage: 100 text + 200 cache_creation + 800 cache_read input tokens, 100 output tokens
computed input_cost  = 0.000330
computed output_cost = 0.001650
computed TOTAL       = 0.001980
correct TOTAL        = 0.003069  (using built-in cache rates)
drop                 = 35.5%

=== AFTER ===
litellm module: /Users/yassinkortam/Documents/Github/berriai/litellm/.claude/worktrees/lucky-mixing-cat/litellm/utils.py
registered entry input_cost_per_token = 3.3e-06
registered entry cache_creation_input_token_cost = 4.125e-06
registered entry cache_read_input_token_cost = 3.3e-07

Usage: 100 text + 200 cache_creation + 800 cache_read input tokens, 100 output tokens
computed input_cost  = 0.001419
computed output_cost = 0.001650
computed TOTAL       = 0.003069
correct TOTAL        = 0.003069  (using built-in cache rates)
drop                 = 0.0%

A live Bedrock proxy run with real cache_control prompt-caching traffic and a database-backed spend log is the next step; I do not have AWS Bedrock credentials in this environment, so this is held over until the next live-proof pass (the inline Python above exercises the exact register_model + generic_cost_per_token path the proxy hits, so the numbers should match end-to-end).

[greptile-apps]

Greptile Summary

This PR fixes two independent bugs where custom-priced deployments silently billed cache tokens at $0: one in register_model (when get_model_info cannot resolve the key) and one in the Router's deployment registration path (where the deploy-id entry was registered without any cache fields). A third unrelated fix corrects _reconcile_budget_reservation_for_counter_update to return set() on failure so the direct spend-counter increment always runs.

• litellm/utils.py: adds _resolve_builtin_model_cost_entry to strip provider/region prefixes and inherit cache pricing fields when register_model falls back to an empty base entry; logs a warning when no match can be found.
• litellm/router.py: adds _inherit_builtin_cache_pricing and calls it from both _create_deployment and _add_deployment_to_router so deploy-id entries inherit built-in cache rates when the user only set input/output cost.
• litellm/proxy/proxy_server.py: adds return set() inside the reconcile-failure handler so the caller's direct increment is not skipped when Redis fails during budget-reservation reconciliation.

Confidence Score: 5/5

Safe to merge; the fix is well-scoped and backed by targeted regression tests covering all four changed code paths.

Both registration paths inherit cache pricing only for missing fields — user-set values always win — and a no-op guard prevents the helper from touching deployments with no custom pricing. The budget-reservation fix is additive (a missing return statement inside an existing except block) with a direct end-to-end test confirming the counter reaches the expected value.

litellm/utils.py — the hardcoded _BEDROCK_REGION_PREFIXES constant is Bedrock-specific and lives outside the llms/ directory; it is also incomplete for several AWS regions.

Important Files Changed

Filename	Overview
litellm/utils.py	Adds _resolve_builtin_model_cost_entry to inherit cache pricing when register_model falls back to an empty base entry; logic is correct but _BEDROCK_REGION_PREFIXES is provider-specific code placed outside llms/ and is incomplete for non-listed AWS regions.
litellm/router.py	Adds _inherit_builtin_cache_pricing static method and calls it from both _create_deployment and _add_deployment_to_router before registering a deploy-id entry; cache fields are inherited only when user has not set them, and the method is a clean no-op when the backend model has no canonical entry.
litellm/proxy/proxy_server.py	Fixes _reconcile_budget_reservation_for_counter_update to return set() on failure instead of returning the original reserved_counter_keys, ensuring the direct spend-counter increment always runs when reconciliation fails.
tests/test_litellm/test_register_model_custom_pricing.py	Adds two regression tests for the prefix-strip path (verifying cache pricing is inherited) and the no-match warning path; existing assertion formatting was reformatted without changing behaviour.
tests/test_litellm/test_router_model_cost_isolation.py	Adds four tests covering partial-pricing inheritance, explicit-override preservation, the unit helper directly, and a no-op for unknown backend models; all new tests and no existing assertions weakened.
tests/test_litellm/proxy/proxy_server/test_spend_counters.py	Updates assertion from {"spend:key:abc"} to set() to match the corrected return value of _reconcile_budget_reservation_for_counter_update on failure; this is the expected post-fix behavior.
tests/test_litellm/proxy/spend_tracking/test_budget_reservation_redis_failure.py	New regression test that exercises the flaky-Redis path end-to-end and asserts the enforced counter equals response_cost after a reconcile failure; mocked-only, no real network calls.
tests/test_litellm/proxy/test_proxy_server.py	Renames test and updates assertion from is None to == 0.25 to verify the direct increment now runs after a reconcile failure; the change tests the correct fixed behaviour.

_{Reviews (3): Last reviewed commit: "fix(register_model): inherit only cache ..." | Re-trigger Greptile}

[greptile-apps]

Cross-provider inheritance when provider prefix is stripped

_resolve_builtin_model_cost_entry does not verify that the resolved entry's litellm_provider matches the user's intended provider. Stripping a leading provider segment produces a bare model name that could collide with an entry from a completely different provider. For example, a user registering bedrock/claude-3-5-sonnet-20241022 with litellm_provider: "bedrock" would have its provider prefix stripped to claude-3-5-sonnet-20241022, which is present in model_cost as an Anthropic direct-API entry. If Bedrock and Anthropic happen to carry the same cache rates this is benign, but the guard entry.get("litellm_provider") is not None only checks that the field exists – it does not assert the provider matches. A simple if provider and entry.get("litellm_provider") not in (provider, None): continue inside the lookup loop would prevent accidental inheritance from a different provider's cost table.

[greptile-apps]

Early-exit guard requires all five cache fields, not just the two canonical ones

_inherit_builtin_cache_pricing skips the lookup only when all five cache_fields are already non-None. If a user sets cache_creation_input_token_cost and cache_read_input_token_cost (the two the cost calculator actually reads) but leaves the three tiered variants None, the function still performs the backend lookup. The behaviour is actually correct because the per-field if model_info.get(field) is None check inside the loop protects user-set values, but the early-return guard is misleading and causes an unnecessary get_model_info call. Consider tightening the early exit to just the two canonical fields to make the intent clearer.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[oss-agent-shin]

@greptileai

[yassin-berriai]

@greptileai