fix(main): stop per-request custom pricing from clobbering shared model_cost pricing

[mateo-berri]

Relevant issues

Linear ticket

Resolves LIT-3991

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

End-to-end demonstration against a live proxy hitting the real OpenAI API; no mocks, no database, no caching. Two deployments share the openai/text-embedding-3-small backend: the named text-embedding-3-small with no explicit pricing (built-in pricing applies) and an openai/* wildcard with explicit zero pricing. Cost is read from the x-litellm-response-cost* response headers; note the proxy omits the plain x-litellm-response-cost header when the computed cost is 0, in which case x-litellm-response-cost-original: 0.0 is the signal. Every request uses a distinct input string to rule out any caching

qa_config.yaml:

model_list:
  - model_name: text-embedding-3-small
    litellm_params:
      model: openai/text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY
  - model_name: openai/*
    litellm_params:
      model: openai/*
      api_key: os.environ/OPENAI_API_KEY
      input_cost_per_token: 0
      output_cost_per_token: 0
general_settings:
  master_key: sk-1234

Setup: a fresh checkout with its own venv (uv venv .venv && uv pip install -e ".[proxy]"), OPENAI_API_KEY exported by sourcing a local .env, and a random free port

$ python -c 'import socket; s=socket.socket(); s.bind(("127.0.0.1",0)); print(s.getsockname()[1])'
50609
$ set -a; . ./.env; set +a
$ .venv/bin/python litellm/proxy/proxy_cli.py --config qa_config.yaml --port 50609 2>&1 | tee proxy_before.log

Before (base 26c0c93)

Step 1, named deployment. Built-in pricing applies, cost is 2.8e-07

$ curl -sS -D - -o /dev/null http://127.0.0.1:50609/v1/embeddings \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "text-embedding-3-small", "input": "before-run step1 named deployment baseline cost check 7f3a"}' \
    | grep -i x-litellm-response-cost
x-litellm-response-cost: 2.8e-07
x-litellm-response-cost-original: 2.8e-07
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0

Step 2, wildcard route. Cost is 0.0 as configured on the wildcard

$ curl -sS -D - -o /dev/null http://127.0.0.1:50609/v1/embeddings \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "openai/text-embedding-3-small", "input": "before-run step2 wildcard route zero cost expected 9c1d"}' \
    | grep -i x-litellm-response-cost
x-litellm-response-cost-original: 0.0
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0

Step 3, named deployment again. This is the bug: one wildcard request poisoned the shared pricing entry and the named deployment now reports 0.0

$ curl -sS -D - -o /dev/null http://127.0.0.1:50609/v1/embeddings \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "text-embedding-3-small", "input": "before-run step3 named deployment after wildcard poisoning check 4e8b"}' \
    | grep -i x-litellm-response-cost
x-litellm-response-cost-original: 0.0
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0

After (this PR)

Same worktree after git checkout origin/litellm_lit3991_wildcard_zero_cost_poisoning, proxy restarted with the same command (logged to proxy_after.log)

Step 1, named deployment. Cost is 2.6e-07, built-in pricing as before

$ curl -sS -D - -o /dev/null http://127.0.0.1:50609/v1/embeddings \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "text-embedding-3-small", "input": "after-run step1 named deployment baseline cost check a2f6"}' \
    | grep -i x-litellm-response-cost
x-litellm-response-cost: 2.6e-07
x-litellm-response-cost-original: 2.6e-07
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0

Step 2, wildcard route. Still 0.0 as configured

$ curl -sS -D - -o /dev/null http://127.0.0.1:50609/v1/embeddings \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "openai/text-embedding-3-small", "input": "after-run step2 wildcard route zero cost expected d5c3"}' \
    | grep -i x-litellm-response-cost
x-litellm-response-cost-original: 0.0
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0

Step 3, named deployment again. Fixed: cost stays at built-in pricing (3e-07, same magnitude as step 1) instead of dropping to 0

$ curl -sS -D - -o /dev/null http://127.0.0.1:50609/v1/embeddings \
    -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
    -d '{"model": "text-embedding-3-small", "input": "after-run step3 named deployment stays priced post wildcard 1b9e"}' \
    | grep -i x-litellm-response-cost
x-litellm-response-cost: 3e-07
x-litellm-response-cost-original: 3e-07
x-litellm-response-cost-discount-amount: 0.0
x-litellm-response-cost-margin-amount: 0.0
x-litellm-response-cost-margin-percent: 0.0

Type

🐛 Bug Fix

Changes

When a proxy has a named deployment that relies on built-in pricing (e.g. model_name text-embedding-3-small backed by openai/text-embedding-3-small) plus an openai/* wildcard deployment with explicit input_cost_per_token: 0 and output_cost_per_token: 0, a single call routed through the wildcard rewrote the shared openai/text-embedding-3-small entry in litellm.model_cost with the zero pricing. Every subsequent call to the named deployment was then logged at $0 cost until process restart

Router startup already isolates per-deployment custom pricing: _create_deployment registers the full pricing under the deployment's unique model id and strips custom pricing fields from the shared {provider}/{model} key. The request-time registration in completion() and embedding() did not follow that isolation; it wrote the full custom pricing entry onto the shared key on every request carrying input/output cost overrides

This PR aligns the request-time registration with the startup isolation. Requests that carry a router deployment id (in metadata model_info, the same signal the cost calculator uses for router_model_id resolution) now register the full pricing under that unique id only, while the shared key receives the entry with custom pricing fields stripped. The request itself is still costed with its own pricing because the cost calculator resolves custom pricing through the deployment id entry. Direct SDK calls without a router deployment id keep the existing behavior of registering the shared key with the given pricing, so litellm.completion / litellm.embedding users passing input_cost_per_token / output_cost_per_token are unaffected, and _is_model_cost_zero (which reads router deployment info) is untouched

The stripping logic is shared through CustomPricingLiteLLMParams.strip_custom_pricing_fields and reused by Router._create_deployment and Router.add_deployment instead of being duplicated inline

New regression tests in tests/test_litellm/test_register_model_custom_pricing.py cover the embedding and completion paths (zero pricing must not clobber the built-in entry, the custom-priced request is still costed via its deployment id, direct SDK behavior preserved), and tests/test_litellm/test_router_model_cost_isolation.py adds the full scenario through a Router with a named deployment plus a zero-cost openai/* wildcard. They fail on the base commit (except the direct SDK guard, which protects existing behavior) and pass with the fix

Two legacy tests encoded the leak as expected behavior and are updated to the corrected semantics. tests/local_testing/test_router_fallbacks.py::test_router_fallbacks_with_custom_model_costs asserted that calling a deployment with 30/60 per-token pricing rewrote the shared claude-sonnet-4-5-20250929 entry, contradicting its own stated goal ("make sure custom model doesn't override default model costs"); it now asserts the shared entry keeps the built-in pricing while the request itself is still costed at the deployment's rate. tests/local_testing/test_cost_calc.py::test_run recomputed the streaming cost with a bare completion_cost(response), which only matched the non-stream cost while the shared gpt-3.5-turbo entry was poisoned by the per-request pricing; it now passes that pricing explicitly via custom_cost_per_token

---

Note

Medium Risk
Changes global litellm.model_cost registration and billing-related paths for router-originated requests; behavior is intentionally different from the buggy legacy path but direct SDK pricing registration is preserved.

Overview
Fixes LIT-3991: router/proxy requests that register custom per-token pricing during completion() / embedding() no longer overwrite the shared {provider}/{model} entry in litellm.model_cost, so a zero-cost wildcard deployment cannot zero out built-in pricing for sibling deployments on the same backend.

Request-time registration now mirrors router startup: _register_custom_pricing_for_request detects a router deployment id in metadata, registers full pricing under that id, and updates the shared key with CustomPricingLiteLLMParams.strip_custom_pricing_fields. Direct SDK calls without a deployment id still register custom pricing on the shared key unchanged. Router _create_deployment / add_deployment use the same strip helper instead of inline field filtering.

Tests add LIT-3991 regression coverage (embedding/completion/router wildcard scenarios) and adjust legacy tests that previously treated shared-key pollution as expected behavior.

^{Reviewed by Cursor Bugbot for commit 58df116. Bugbot is set up for automated code reviews on this repo. Configure here.}

[mateo-berri]

@greptileai

[greptile-apps]

Greptile Summary

This PR fixes a pricing isolation bug where a router deployment with explicit zero-cost pricing (e.g. an openai/* wildcard) permanently clobbered the shared openai/text-embedding-3-small entry in litellm.model_cost, causing all subsequent requests to sibling deployments relying on built-in pricing to be logged at $0 until process restart.

• Introduces _register_custom_pricing_for_request in main.py that routes router-originated requests (detected via the deployment id in metadata.model_info.id) to register full pricing only under the deployment's unique id, while the shared {provider}/{model} key receives a stripped entry with no pricing fields — mirroring the isolation already done at startup in Router._create_deployment.
• Extracts the shared-key stripping logic into CustomPricingLiteLLMParams.strip_custom_pricing_fields and reuses it in router.py (two call sites) and the new request-time registration path, eliminating duplication.
• Updates two legacy tests that were asserting the bug as expected behavior, and adds new regression tests covering both embedding and completion paths.

Confidence Score: 5/5

Safe to merge — the fix is narrowly scoped, direct SDK callers are unaffected, and the behaviour is exercised by both unit and end-to-end router tests that fail on the base commit and pass with the fix.

The core change routes custom pricing registration through the deployment id rather than the shared key, which is well-contained in two call sites and follows the same pattern already applied at router startup in _create_deployment. The strip_custom_pricing_fields extraction removes duplication without changing logic. The two updated legacy tests were encoding the bug as expected behaviour; the corrected assertions still verify that the custom pricing is applied to the request, so no coverage is lost. No regressions found in the direct-SDK path or the router path.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/main.py	Adds _get_router_deployment_id and _register_custom_pricing_for_request; both completion() and embedding() now use the latter, correctly isolating per-deployment pricing from the shared model_cost key for router requests while preserving legacy SDK behavior.
litellm/types/utils.py	Adds strip_custom_pricing_fields classmethod to CustomPricingLiteLLMParams; clean extraction of the inline dict-comprehension used in router.py, no logic changes.
litellm/router.py	Refactors two identical inline stripping dict-comprehensions in _create_deployment and add_deployment to use CustomPricingLiteLLMParams.strip_custom_pricing_fields; behavior is identical.
tests/local_testing/test_router_fallbacks.py	Updates test_router_fallbacks_with_custom_model_costs to assert shared model key retains built-in pricing instead of the custom pricing — correctly encodes the intended contract stated in the test docstring, and still checks the request itself is costed at the custom rate via response_cost > 10.
tests/local_testing/test_cost_calc.py	Updates streaming cost assertion to pass custom_cost_per_token explicitly rather than relying on the shared gpt-3.5-turbo key having been poisoned with custom pricing; correct fix for the changed isolation behavior.
tests/test_litellm/test_register_model_custom_pricing.py	Adds four new regression tests covering the embedding zero-pricing isolation, deployment-id-based cost resolution, completion path isolation, and direct SDK legacy behavior; all use mock_response (no real network calls).
tests/test_litellm/test_router_model_cost_isolation.py	Adds end-to-end regression test through a real Router with named + wildcard zero-cost deployments; verifies the wildcard request doesn't poison the named deployment's built-in pricing.

_{Reviews (3): Last reviewed commit: "test: update legacy tests that asserted ..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 92.59259% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/main.py	90.90%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

This PR fixes a production bug where per-request custom pricing on router deployments clobbered the shared litellm.model_cost entry for the same backend model. A wildcard deployment with input_cost_per_token: 0 would overwrite the built-in pricing for sibling named deployments on every request, causing all subsequent sibling calls to log $0 cost until process restart.

• Core fix (main.py): Replaces inline litellm.register_model calls in completion() and embedding() with _register_custom_pricing_for_request, which detects router-originated requests via metadata.model_info.id and routes full custom pricing to the deployment-id key only, while updating the shared provider/model key with pricing fields stripped — mirroring the isolation already done by Router._create_deployment at startup.
• Shared helper (types/utils.py): Extracts CustomPricingLiteLLMParams.strip_custom_pricing_fields as a classmethod, eliminating duplicated dict-comprehension logic from two places in router.py.
• Tests: New regression tests in both test files cover the embedding and completion paths, the router end-to-end scenario, and the preserved direct-SDK behavior; all use mock responses with no real network calls.

Confidence Score: 5/5

Safe to merge — the change is tightly scoped to the pricing-registration path, direct SDK behaviour is explicitly preserved, and the isolation logic mirrors what the Router already does at startup.

The fix correctly identifies router-originated requests via the deployment id already placed in metadata by _update_kwargs_with_deployment, isolates custom pricing to the deployment-id key (matching startup-time behaviour), and leaves the shared provider/model key unmodified. The helper that strips pricing fields is a clean extraction of existing logic in two router.py sites. Tests cover zero-pricing non-clobber, correct cost attribution via deployment id, and direct SDK guard — all using mocked responses. No regressions are introduced for existing callers.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/main.py	Adds _get_router_deployment_id and _register_custom_pricing_for_request helpers; replaces inline register_model calls in completion() and embedding() with the new isolation-aware function. Logic is correct — router calls get full pricing under deployment id only, direct SDK calls keep legacy behavior.
litellm/types/utils.py	Adds strip_custom_pricing_fields classmethod to CustomPricingLiteLLMParams — pure refactoring of duplicate dict-comprehension logic already in router.py, behaviour is identical.
litellm/router.py	Two call sites in _create_deployment and add_deployment that hand-rolled the custom-pricing-field stripping now delegate to strip_custom_pricing_fields — equivalent transformation, no behaviour change.
tests/test_litellm/test_register_model_custom_pricing.py	Four new tests added: zero-pricing non-clobber for embedding and completion, cost-via-deployment-id, and direct SDK guard; all use mock_response and fake keys — no network calls.
tests/test_litellm/test_router_model_cost_isolation.py	End-to-end Router test covering the full LIT-3991 scenario: named deployment + zero-cost wildcard; verifies shared key pricing survives a wildcard call and named deployment still accrues non-zero cost.

_{Reviews (2): Last reviewed commit: "fix(main): stop per-request custom prici..." | Re-trigger Greptile}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 58df116. Configure here.}