[Fix] prevent shared backend model key from being polluted by per-deployment custom pricing

[shivamrawat1]

Relevant issues

Closes #20546

Problem
When the proxy model_list contains two deployments that use the same backend model (e.g., both pointing to vertex_ai/gemini-2.5-flash), and one deployment has explicit zero-cost pricing in model_info while the other relies on built-in pricing, both models incorrectly reported $0 cost.
Example config:

model_list:  
  - model_name: gcp/google/gemini-2.5-flash.custom 
    litellm_params:      
      model: vertex_ai/gemini-2.5-flash    
      model_info:      
      input_cost_per_token: 0.0      
      output_cost_per_token: 0.0  
    model_info:
      id: gcp/google/gemini-2.5-flash.custom
  - model_name: gcp/google/gemini-2.5-flash.same   
    litellm_params:      
      model: vertex_ai/gemini-2.5-flash    # no custom pricing — should use built-in pricing
    model_info:
      id: gcp/google/gemini-2.5-flash.same

With both models present, the second model showed no (equivalent to 0) cost. With only the second model in the config, cost was correct.

Before:

After:

Model uses the custom pricing.
Note: Both models should have distintive id under model info,

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🐛 Bug Fix

Changes

Root Cause
In Router._create_deployment(), each deployment's model_info was registered in litellm.model_cost under two keys:
The deployment's unique model_id (safe — unique per deployment)
The shared backend model name (e.g., vertex_ai/gemini-2.5-flash) — this key is global and shared across all deployments using the same underlying model
When the first deployment (with input_cost_per_token: 0) was processed, its zero-cost pricing was written to the shared vertex_ai/gemini-2.5-flash key, overwriting the built-in pricing. The second deployment then picked up that zero-cost entry as its base, resulting in both reporting $0.
Fix
Strip custom pricing fields from the model info before registering under the shared backend model name. Each deployment's full pricing (including custom overrides) is still stored under its unique model_id. This prevents one deployment's pricing from polluting another deployment that shares the same backend.
The fix is backward compatible because:
The shared key is still always registered (preserving lookups by backend name)
Built-in pricing in the shared key is never overwritten by per-deployment overrides
The cost calculator already uses model_id for lookups when custom_pricing=True

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 8, 2026 0:09am

[greptile-apps]

Greptile Overview

Greptile Summary

Fixed pricing pollution bug where multiple deployments sharing the same backend model (e.g., vertex_ai/gemini-2.5-flash) would incorrectly inherit custom pricing from each other. The issue occurred because custom pricing fields (like input_cost_per_token: 0.0) from one deployment were being registered under the shared backend model key, affecting all other deployments using that model.

Key Changes:

• Strips custom pricing fields from model_info before registering under the shared backend model name (vertex_ai/gemini-2.5-flash)
• Each deployment's full pricing (including custom overrides) is still stored under its unique model_id
• The shared backend key now only contains base model information and built-in pricing

Impact:

• Prevents zero-cost deployments from making other deployments report $0 cost
• Maintains backward compatibility since the shared key is still registered
• Cost calculator already uses model_id for lookups when custom pricing is present

Confidence Score: 3/5

• Safe to merge with test coverage recommendation
• The fix correctly addresses the root cause by filtering custom pricing fields from the shared backend model registration while preserving them in the per-deployment registration. The logic is sound and backward compatible. However, the PR lacks test coverage for this specific scenario (multiple deployments with the same backend model but different custom pricing), which is a hard requirement according to the PR checklist. The fix is minimal and isolated, but without tests it's difficult to verify it fully resolves the issue and prevents regression.
• No files require special attention beyond adding test coverage

Important Files Changed

Filename	Overview
litellm/router.py	Strips custom pricing fields from shared backend model registration to prevent pollution across deployments with same backend model

Sequence Diagram

sequenceDiagram
    participant Config as Router Config
    participant Router as Router._create_deployment
    participant Deployment as Deployment Object
    participant ModelCost as litellm.model_cost
    
    Note over Config,ModelCost: Deployment 1: custom zero-cost pricing
    Config->>Router: deployment_info, model_info (empty or base)
    Router->>Router: Extract custom pricing from litellm_params
    Router->>Router: Add custom pricing to _model_info<br/>(input_cost_per_token: 0.0)
    Router->>Deployment: Create deployment with model_info
    Router->>ModelCost: register_model(model_id: _model_info)<br/>[includes custom pricing]
    Router->>Router: Strip custom pricing fields<br/>_shared_model_info = filter(_model_info)
    Router->>ModelCost: register_model(vertex_ai/gemini: _shared_model_info)<br/>[no custom pricing, preserves built-in]
    
    Note over Config,ModelCost: Deployment 2: no custom pricing
    Config->>Router: deployment_info, model_info (empty or base)
    Router->>Router: No custom pricing in litellm_params
    Router->>Deployment: Create deployment with model_info
    Router->>ModelCost: register_model(model_id: _model_info)<br/>[uses built-in pricing]
    Router->>Router: Strip custom pricing fields<br/>_shared_model_info = filter(_model_info)
    Router->>ModelCost: register_model(vertex_ai/gemini: _shared_model_info)<br/>[preserves built-in, not polluted by Deployment 1]
    
    Note over ModelCost: Result: Each deployment has correct pricing<br/>Shared backend key has built-in pricing

Loading

[greptile-apps]

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Check that _model_info doesn't become empty after filtering. If a deployment only has custom pricing fields and no other model info, _shared_model_info could be empty or contain no useful data for the shared backend key.

[shivamrawat1]

@greptile If _shared_model_info ends up empty (because _model_info only contained custom pricing fields), calling register_model with an empty dict is a harmless no-op — _update_dictionary(existing_model, {}) iterates over zero keys and leaves the built-in pricing untouched.

[shivamrawat1]

@greptile review the pr again with the tests added

[greptile-apps]

Greptile Overview

Greptile Summary

This PR fixes a critical pricing bug where per-deployment custom pricing was polluting the shared backend model key in litellm.model_cost, causing deployments without custom pricing to incorrectly inherit zero or custom costs from other deployments sharing the same backend model.

Changes:

• Modified Router._create_deployment() in litellm/router.py to filter out custom pricing fields before registering under the shared backend model name
• Each deployment's full pricing (including custom overrides) is still stored under its unique model_id key
• The shared backend key now only contains non-pricing metadata, allowing register_model() to preserve built-in pricing
• Added comprehensive test suite with 4 test cases covering zero-cost pricing, non-zero custom pricing, model_id storage, and deployment order independence

Technical correctness:

• The fix correctly identifies all custom pricing fields from CustomPricingLiteLLMParams.model_fields.keys()
• Empty _shared_model_info dict is safe because register_model() fetches fresh built-in pricing and merges with it
• The lookup mechanism in get_deployment_model_info() correctly merges deployment-specific info with built-in pricing
• Backward compatible because shared key registration is preserved and deployment-specific pricing is still fully accessible via model_id

Confidence Score: 5/5

• This PR is safe to merge with high confidence
• The fix is surgically targeted, well-tested, and solves a clear bug without introducing new issues. The change correctly strips only custom pricing fields before shared key registration while preserving deployment-specific pricing under unique model_id keys. The comprehensive test suite validates the fix across multiple scenarios including zero-cost, non-zero custom pricing, and deployment order independence. The implementation is backward compatible and aligns with the existing cost lookup mechanism in get_deployment_model_info().
• No files require special attention

Important Files Changed

Filename	Overview
litellm/router.py	Correctly strips custom pricing fields before registering shared backend model key, preventing pricing pollution across deployments
tests/test_litellm/test_router_model_cost_isolation.py	Comprehensive test coverage with 4 test cases covering zero-cost pricing, non-zero custom pricing, model_id storage, and deployment order independence

Sequence Diagram

sequenceDiagram
    participant Router
    participant DeployA as Deployment A<br/>(custom pricing)
    participant DeployB as Deployment B<br/>(built-in pricing)
    participant ModelCost as litellm.model_cost
    participant BuiltIn as Built-in Pricing

    Note over Router: Process Deployment A
    Router->>DeployA: Create with custom pricing
    Router->>ModelCost: register(deployment-a, full pricing)
    Note over ModelCost: Deployment-specific key stored
    
    Router->>Router: Strip custom pricing fields
    Router->>BuiltIn: get_model_info(backend_model)
    BuiltIn-->>Router: Return built-in pricing
    Router->>ModelCost: register(backend_model, stripped info)
    Note over ModelCost: Shared key keeps built-in pricing

    Note over Router: Process Deployment B
    Router->>DeployB: Create without custom pricing
    Router->>ModelCost: register(deployment-b, model_info)
    Note over ModelCost: Deployment-specific key stored
    
    Router->>Router: Strip custom pricing (none exist)
    Router->>BuiltIn: get_model_info(backend_model)
    BuiltIn-->>Router: Return built-in pricing
    Router->>ModelCost: register(backend_model, stripped info)
    Note over ModelCost: Shared key maintains built-in pricing

    Note over Router,ModelCost: Lookup Phase
    Router->>ModelCost: get_deployment_model_info(deployment-a)
    ModelCost-->>Router: Custom pricing returned
    
    Router->>ModelCost: get_deployment_model_info(deployment-b)
    ModelCost-->>Router: Non-pricing fields only
    Router->>BuiltIn: get_model_info(backend_model)
    BuiltIn-->>Router: Built-in pricing
    Router->>Router: Merge deployment-b + built-in
    Note over Router: Deployment B correctly uses built-in pricing

Loading

[greptile-apps]

_{2 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[ishaan-jaff]

lgtm