fix(router): isolate all per-deployment pricing overrides from sibling deployments

[mubashir1osmani]

Relevant issues

Linear ticket

LIT-3897

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Screenshots / Proof of Fix

Run a proxy with two deployments on the same backend, where one overrides a tiered rate that the previous denylist did not strip. Save this as lit3897.yaml

model_list:
  - model_name: custom-priced-flash
    litellm_params:
      model: gemini/gemini-2.5-flash
      api_key: os.environ/GEMINI_API_KEY
      input_cost_per_token: 5e-05
      output_cost_per_token: 1e-04
      input_cost_per_token_above_272k_tokens: 9.99e-04
  - model_name: gemini-2.5-flash
    litellm_params:
      model: gemini/gemini-2.5-flash
      api_key: os.environ/GEMINI_API_KEY

Then

python litellm/proxy/proxy_cli.py --config lit3897.yaml --detailed_debug --reload --use_v2_migration_resolver 2>&1 | tee litellm.log

curl -s http://localhost:4000/model/info -H "Authorization: Bearer sk-1234" \
  | jq -r '.data[] | select(.model_name=="custom-priced-flash" or .model_name=="gemini-2.5-flash") | "\(.model_name)\tinput=\(.model_info.input_cost_per_token)\tinput_above_272k=\(.model_info.input_cost_per_token_above_272k_tokens)"'

Expected output after the fix, where the override stays on custom-priced-flash only and the sibling keeps the canonical gemini rate

custom-priced-flash	input=5e-05	input_above_272k=0.000999
gemini-2.5-flash	input=3e-07	input_above_272k=null

Before the fix the sibling gemini-2.5-flash reported input_above_272k=0.000999, the leaked override

Type

🐛 Bug Fix

Changes

Two router deployments that share the same backend model (for example gemini/gemini-2.5-flash) are supposed to get isolated custom pricing. The router already strips per-deployment pricing fields from the shared backend-alias key before registering it into litellm.model_cost, using CustomPricingLiteLLMParams as the list of fields to strip. That denylist had drifted from ModelInfoBase, so tiered and per-unit cost fields like input_cost_per_token_above_272k_tokens, input_cost_per_token_above_512k_tokens, cache_read_input_token_cost_above_272k_tokens, output_vector_size, ocr_cost_per_page, and the regional uplift multipliers were never stripped. A deployment overriding any of them leaked the override into the shared key, and every sibling then read the wrong rate via /model/info

The fix adds the missing cost fields to CustomPricingLiteLLMParams so it covers every pricing field in ModelInfoBase, which closes the leak at all the sites that consume this single source of truth (router stripping, the litellm_params to model_info copy, pre-call utils, auth, and logging). A guard test asserts the denylist never drifts from ModelInfoBase again, and a regression test confirms a tiered override stays isolated to its own deployment model_id key and does not pollute the shared backend key a sibling resolves through

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR fixes a pricing override leak between sibling router deployments that share the same backend model. When two deployments point at the same backend (e.g. gemini/gemini-2.5-flash) and one carries a custom pricing override, the router strips those fields from the shared backend key using CustomPricingLiteLLMParams as a denylist — but that list had drifted from ModelInfoBase, so tiered, regional-uplift, and OCR cost fields were never stripped and leaked into the shared key.

• litellm/types/utils.py: Adds the 13 previously-missing pricing fields to CustomPricingLiteLLMParams (input/output_cost_per_token_above_272k/512k_tokens, cache_read_input_token_cost_above_272k/512k_tokens, input_cost_per_image_token, output_vector_size, ocr_cost_per_page, ocr_cost_per_credit, annotation_cost_per_page, and the two regional uplift multipliers), fully aligning the denylist with ModelInfoBase.
• tests/test_litellm/test_router_model_cost_isolation.py: Adds a structural guard test (test_custom_pricing_field_denylist_covers_all_builtin_pricing_fields) that will fail immediately if future additions to ModelInfoBase are not mirrored in CustomPricingLiteLLMParams, plus two regression tests covering the isolation through litellm.get_model_info and through the proxy /model/info resolution helper.

Confidence Score: 5/5

Safe to merge — the change is additive (new fields on an existing Pydantic model) and is fully covered by both a structural guard test and two regression tests.

The fix is minimal and targeted: 13 Optional fields are appended to CustomPricingLiteLLMParams with None defaults, which is non-breaking. All newly added fields are verified to be present in ModelInfoBase and their absence was the root cause of the described leak. The guard test using typing.get_type_hints(ModelInfoBase) will catch any future drift. The regression tests reproduce the exact scenario from the ticket and verify both the per-deployment custom key and the shared backend key simultaneously.

No files require special attention. litellm/types/utils.py is the only production file changed and the modification is purely additive.

Important Files Changed

Filename	Overview
litellm/types/utils.py	Adds 13 missing pricing fields to CustomPricingLiteLLMParams so the denylist fully matches ModelInfoBase, closing the per-deployment override leak for tiered, regional-uplift, and OCR cost fields.
tests/test_litellm/test_router_model_cost_isolation.py	Adds a guard test that uses typing.get_type_hints to ensure CustomPricingLiteLLMParams always covers every pricing field in ModelInfoBase, plus two regression tests validating tiered-override isolation via model_cost lookup and the proxy /model/info path.

_{Reviews (1): Last reviewed commit: "fix(router): isolate all per-deployment ..." | Re-trigger Greptile}

[mubashir1osmani]

before the fix: input_above_272k=null is not mapped for custom pricing model

after the fix:
custom-priced-flash input=5e-05 input_above_272k=0.000999

[mubashir1osmani]

Superseded by in-repo PR #31021 (pushed to BerriAI/litellm directly so full CI runs; cross-fork PRs skip the secret-gated jobs).