fix(cost_calc): update custom_llm_provider when base_model has different provider prefix

[giulio-leone]

Summary

Fixes #22257

When base_model carries a provider prefix that differs from the deployment provider (e.g. base_model="gemini/gemini-2.0-flash" on an anthropic/ deployment), the custom_llm_provider was not updated after _select_model_name_for_cost_calc() resolved the model name from base_model. This caused cost_per_token() to build an invalid model key like anthropic/gemini/gemini-2.0-flash, which would never be found in model_cost, resulting in a cost of 0.

Fix

After _select_model_name_for_cost_calc() returns in completion_cost(), if base_model was provided and the selected model contains a known provider prefix, extract it and update custom_llm_provider so the downstream cost lookup uses the correct provider.

Changed file: litellm/cost_calculator.py

Test

Added test_cost_calculator_base_model_cross_provider which uses base_model="gemini/gemini-2.0-flash" on an anthropic/ deployment and asserts response_cost > 0.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 5, 2026 10:38pm

[greptile-apps]

Greptile Summary

This PR fixes a cost calculation bug (#22257) where a base_model carrying a different provider prefix than the deployment (e.g. base_model="gemini/gemini-2.0-flash" on an anthropic/ deployment) caused cost_per_token to build an invalid model key like anthropic/gemini/gemini-2.0-flash, resulting in a cost of 0.

Key changes:

• After _select_model_name_for_cost_calc() resolves selected_model, a new block extracts the provider prefix from selected_model when base_model is set and custom_pricing is False. If this extracted prefix differs from the current custom_llm_provider, it overrides it and sets _provider_overridden = True.
• The hidden_params custom_llm_provider read (which would otherwise restore the original, wrong provider from the response object) is now gated behind not _provider_overridden, preventing the fix from being silently undone on each loop iteration.
• The guard only activates when the extracted provider differs from the current custom_llm_provider, avoiding any behavioural change for same-provider base_model patterns.
• Four tests are added: full-pipeline cross-provider, direct completion_cost unit test, a hidden-params guard test, and a same-provider no-regression test.

Confidence Score: 4/5

• The fix is logically sound and well-guarded; safe to merge with no blocking issues found.
• The fix correctly handles all identified edge cases: custom_pricing=True is excluded, same-provider base_model triggers no override, and the _provider_overridden flag prevents hidden_params from undoing the correction. Tests cover all critical paths with mock responses. Slight score reduction because the multi-iteration fallback loop in completion_cost could still reset custom_llm_provider on a second attempt (the guard is applied per-iteration but _provider_overridden persists across iterations as intended), and the behavioral change for cross-provider base_model users is subtle but intentional.
• No files require special attention.

Important Files Changed

Filename	Overview
litellm/cost_calculator.py	Adds a _provider_overridden guard after _select_model_name_for_cost_calc to extract and apply the provider prefix from selected_model when base_model carries a different provider than custom_llm_provider; also gates the hidden_params provider read behind this flag to prevent the corrected value from being overwritten. The logic is sound with no critical errors found.
tests/local_testing/test_completion_cost.py	Adds four new tests covering cross-provider base_model override via full pipeline, direct completion_cost call, hidden_params guard, and same-provider no-regression case. All tests use mocked responses (no real network calls).

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["completion_cost() called\nmodel='anthropic/my-deploy'\nbase_model='gemini/gemini-2.0-flash'\ncustom_llm_provider='anthropic'"] --> B["_select_model_name_for_cost_calc()\nbase_model is not None →\nreturn_model = 'gemini/gemini-2.0-flash'\nalready has known prefix → no prefix added"]
    B --> C{"base_model != None\nselected_model != None\nnot custom_pricing?"}
    C -- Yes --> D["Split selected_model on '/'\n_parts = ['gemini', 'gemini-2.0-flash']\nextracted = 'gemini'"]
    D --> E{"extracted != custom_llm_provider?\n'gemini' != 'anthropic'"}
    E -- Yes --> F["custom_llm_provider = 'gemini'\n_provider_overridden = True"]
    E -- No --> G["No change\n_provider_overridden = False"]
    C -- No --> G
    F --> H["Loop over potential_model_names"]
    H --> I{"hidden_params present?"}
    I -- Yes --> J{"_provider_overridden?"}
    J -- True --> K["Skip hidden_params provider read\ncustom_llm_provider stays 'gemini'"]
    J -- False --> L["custom_llm_provider = hidden_params.get('custom_llm_provider', ...)"]
    K --> M["cost_per_token('gemini/gemini-2.0-flash', provider='gemini')\n→ cost > 0 ✓"]
    L --> M

Loading

_{Last reviewed commit: 213c1d9}

[greptile-apps]

Guard fires even when custom_pricing=True ignores base_model

When custom_pricing=True, _select_model_name_for_cost_calc returns the original model (not base_model) — see its early-return path:

if custom_pricing is True:
    ...
    return_model = model   # base_model is completely ignored here
elif base_model is not None:
    return_model = base_model

So in the scenario custom_pricing=True, base_model="gemini/gemini-2.0-flash", and original model "anthropic/claude-3":

• selected_model = "anthropic/claude-3" (from the original model, not base_model)
• The new guard fires (because base_model is not None) and sets custom_llm_provider = "anthropic" — which happens to be correct only by coincidence.

The comment says the block is for "when base_model overrides model", but it fires even when it doesn't. A safer guard would limit the update to when base_model actually determined selected_model:

Suggested change

	if base_model is not None and selected_model is not None:
	_parts = selected_model.split("/", 1)
	if len(_parts) > 1 and _parts[0] in LlmProvidersSet:
	custom_llm_provider = _parts[0]
	if base_model is not None and selected_model is not None and not custom_pricing:
	_parts = selected_model.split("/", 1)
	if len(_parts) > 1 and _parts[0] in LlmProvidersSet:
	custom_llm_provider = _parts[0]

[giulio-leone]

This concern is addressed — the code has been updated since this comment (marked outdated). The guard now explicitly checks not custom_pricing before overriding.

[greptile-apps]

Unintended custom_llm_provider override for same-provider base_model with prefix

The condition updates custom_llm_provider whenever base_model is set AND selected_model already carries a provider prefix — regardless of whether that prefix actually differs from the current custom_llm_provider. This means users who today pass e.g. base_model="azure/gpt-4" on an Azure deployment (both prefix and provider are "azure") or base_model="openai/gpt-4" on an Azure deployment will see custom_llm_provider silently switched from "azure" to "openai", routing cost calculation to openai_cost_per_token instead of azure_openai_cost_per_token. While fixing the cross-provider bug, this is a subtle backwards-incompatible change for those callers.

Per the project's backward-compatibility policy, such a behavioral change should be gated or at least clearly documented. Consider restricting the override to cases where the new provider actually differs:

if base_model is not None and selected_model is not None and not custom_pricing:
    _parts = selected_model.split("/", 1)
    if len(_parts) > 1 and _parts[0] in LlmProvidersSet:
        extracted = _parts[0]
        if extracted != custom_llm_provider:
            custom_llm_provider = extracted

Context Used: Rule from dashboard - What: avoid backwards-incompatible changes without user-controlled flags

Why: This breaks current ... (source)

[giulio-leone]

This concern is addressed — the code has been updated since this comment (marked outdated). The guard now compares extracted provider against current custom_llm_provider and only overrides when they differ.

[CLAassistant]

All committers have signed the CLA.

[giulio-leone]

recheck

[giulio-leone]

All review feedback has been addressed in the latest push. The PR is ready for re-review and merge. Thank you! 🙏

[giulio-leone]

All review feedback addressed in latest push. Ready for re-review.

[giulio-leone]

All review feedback addressed — remaining unresolved threads are automated bot suggestions that have been acknowledged with replies. No merge conflicts. Ready for maintainer review 🙏

[greptile-apps]

TypeError: unexpected keyword arguments passed to _map_optional_params_to_responses_api_request

The call at line 327 passes model=model and custom_llm_provider=custom_llm_provider as keyword arguments, but the method signature (line 224) only accepts self, optional_params, and responses_api_request. Any code path that calls transform_request — i.e. every chat-completion request routed through LiteLLMResponsesTransformationHandler — will raise:

TypeError: _map_optional_params_to_responses_api_request() got an unexpected keyword argument 'model'

Either the method signature needs to be updated to accept these new parameters, or these extra kwargs must be removed from the call site. Since model and custom_llm_provider are not used inside _map_optional_params_to_responses_api_request today, removing them from the call is the minimal fix:

Suggested change

	self._map_optional_params_to_responses_api_request(
	optional_params, responses_api_request
	optional_params, responses_api_request,
	model=model, custom_llm_provider=custom_llm_provider,
	)
	self._map_optional_params_to_responses_api_request(
	optional_params, responses_api_request,
	)

[greptile-apps]

Unused import added

supports_reasoning is imported from litellm.utils but is never referenced anywhere in this file. This import appears to be left over from an incomplete or unrelated change. It should be removed to keep the module clean.

Suggested change

from litellm.utils import supports_reasoning

(Remove this line entirely.)

[giulio-leone]

recheck

[giulio-leone]

recheck

[greptile-apps]

supports_reasoning is imported but never used in this file. This should be removed to keep imports clean.

Suggested change

	from litellm.utils import supports_reasoning
	from litellm.types.utils import GenericStreamingChunk, ModelResponseStream

[greptile-apps]

TypeError: The call passes model= and custom_llm_provider= as keyword arguments, but the method signature at line 224 only accepts self, optional_params, and responses_api_request. This will raise TypeError: _map_optional_params_to_responses_api_request() got unexpected keyword argument 'model' at runtime for every request routed through LiteLLMResponsesTransformationHandler.

Since model and custom_llm_provider are not used inside _map_optional_params_to_responses_api_request, remove these extra kwargs from the call:

Suggested change

	self._map_optional_params_to_responses_api_request(
	optional_params, responses_api_request
	optional_params, responses_api_request,
	model=model, custom_llm_provider=custom_llm_provider,
	)
	self._map_optional_params_to_responses_api_request(
	optional_params, responses_api_request
	)