fix(proxy): forward resolved provider and deployment pricing in /cost/estimate

[devin-ai-integration]

TLDR

Problem this solves:

• /cost/estimate 500s for on-prem models like nvidia/zai-org/glm-5.2
• Configured per-token pricing on such deployments was ignored

How it solves it:

• Forward the router-resolved provider into the cost calculation
• Forward the deployment's input/output_cost_per_token as custom pricing, read from litellm_params or model_info (litellm_params wins, matching the router's cost-map registration precedence)
• Return that configured per-token pricing in the estimate response

User Flow

Before: an admin sizing costs for a self-hosted deployment gets a 500 instead of an estimate

1. The proxy config lists model_name: nvidia/zai-org/glm-5.2 pointing at a self-hosted OpenAI-compatible server, with input_cost_per_token/output_cost_per_token optionally set
2. They send POST http://litellm-domain/cost/estimate with {"model": "nvidia/zai-org/glm-5.2", "input_tokens": 1000, "output_tokens": 500}
3. The response is a 500: "Could not calculate cost for model 'nvidia/zai-org/glm-5.2' (resolved to 'zai-org/GLM-5.2'): litellm.BadRequestError: LLM Provider NOT provided"

After: the same request returns a real estimate that honors the deployment's configured pricing

1. The proxy config lists the same nvidia/zai-org/glm-5.2 deployment
2. They send the same POST http://litellm-domain/cost/estimate request
3. The response is a 200 naming the resolved provider, and when the deployment configures per-token pricing (under litellm_params or model_info) the estimate reflects it (cost_per_request, daily_cost, and the per-token rates)

Relevant issues

Linear ticket

Resolves LIT-5210

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

/cost/estimate is a pure pricing computation; it never dispatches a request to a provider, so there is no real LLM call or spend involved. The realistic end-user action is the curl below, run against a live proxy configured exactly as the ticket describes. Both legs booted a live proxy from the same config: the before leg at the merge base 7e80e094c4, the after leg at this PR's head 0fdbe03c50

model_list:
  - model_name: nvidia/zai-org/glm-5.2
    litellm_params:
      model: zai-org/GLM-5.2
      custom_llm_provider: openai
      api_base: http://localhost:9999/v1
      api_key: sk-fake
  - model_name: nvidia-priced/zai-org/glm-5.2
    litellm_params:
      model: zai-org/GLM-5.2
      custom_llm_provider: openai
      api_base: http://localhost:9999/v1
      api_key: sk-fake
      input_cost_per_token: 0.000001
      output_cost_per_token: 0.000002
  - model_name: nvidia-mi-priced/zai-org/glm-5.2
    litellm_params:
      model: zai-org/GLM-5.2
      custom_llm_provider: openai
      api_base: http://localhost:9999/v1
      api_key: sk-fake
    model_info:
      input_cost_per_token: 0.000003
      output_cost_per_token: 0.000004
  - model_name: gpt-4
    litellm_params:
      model: openai/gpt-4
      api_key: os.environ/OPENAI_API_KEY
general_settings:
  master_key: sk-1234

Before (merge base 7e80e094c4): every on-prem deployment 500s, priced or not, wherever the pricing lives

$ curl -s -X POST http://localhost:43117/cost/estimate -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d '{"model":"nvidia/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500}'
{"detail":{"error":"Could not calculate cost for model 'nvidia/zai-org/glm-5.2' (resolved to 'zai-org/GLM-5.2'): litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=zai-org/GLM-5.2\n Pass model as E.g. For 'Huggingface' inference endpoints pass in `completion(model='huggingface/starcoder',..)` Learn more: https://docs.litellm.ai/docs/providers"}}

$ curl -s -X POST http://localhost:43117/cost/estimate -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d '{"model":"nvidia-priced/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500,"num_requests_per_day":100}'
{"detail":{"error":"Could not calculate cost for model 'nvidia-priced/zai-org/glm-5.2' (resolved to 'zai-org/GLM-5.2'): litellm.BadRequestError: LLM Provider NOT provided. ..."}}

$ curl -s -X POST http://localhost:43117/cost/estimate -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d '{"model":"nvidia-mi-priced/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500}'
{"detail":{"error":"Could not calculate cost for model 'nvidia-mi-priced/zai-org/glm-5.2' (resolved to 'zai-org/GLM-5.2'): litellm.BadRequestError: LLM Provider NOT provided. ..."}}

After (PR head 0fdbe03c50): no configured pricing returns 200 with the resolved provider

$ curl -s -X POST http://localhost:44923/cost/estimate -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d '{"model":"nvidia/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500}'
{"model":"nvidia/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500,"num_requests_per_day":null,"num_requests_per_month":null,"cost_per_request":0.0,"input_cost_per_request":0.0,"output_cost_per_request":0.0,"margin_cost_per_request":0.0,"daily_cost":null,"daily_input_cost":null,"daily_output_cost":null,"daily_margin_cost":null,"monthly_cost":null,"monthly_input_cost":null,"monthly_output_cost":null,"monthly_margin_cost":null,"input_cost_per_token":null,"output_cost_per_token":null,"provider":"openai"}

After (PR head 0fdbe03c50): litellm_params per-token pricing is honored

$ curl -s -X POST http://localhost:44923/cost/estimate -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d '{"model":"nvidia-priced/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500,"num_requests_per_day":100}'
{"model":"nvidia-priced/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500,"num_requests_per_day":100,"num_requests_per_month":null,"cost_per_request":0.002,"input_cost_per_request":0.001,"output_cost_per_request":0.001,"margin_cost_per_request":0.0,"daily_cost":0.2,"daily_input_cost":0.1,"daily_output_cost":0.1,"daily_margin_cost":0.0,"monthly_cost":null,"monthly_input_cost":null,"monthly_output_cost":null,"monthly_margin_cost":null,"input_cost_per_token":1e-6,"output_cost_per_token":2e-6,"provider":"openai"}

After (PR head 0fdbe03c50): model_info per-token pricing (how DB / Admin UI added deployments store it) is honored too

$ curl -s -X POST http://localhost:44923/cost/estimate -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d '{"model":"nvidia-mi-priced/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500,"num_requests_per_day":100}'
{"model":"nvidia-mi-priced/zai-org/glm-5.2","input_tokens":1000,"output_tokens":500,"num_requests_per_day":100,"num_requests_per_month":null,"cost_per_request":0.005,"input_cost_per_request":0.003,"output_cost_per_request":0.002,"margin_cost_per_request":0.0,"daily_cost":0.5,"daily_input_cost":0.3,"daily_output_cost":0.2,"daily_margin_cost":0.0,"monthly_cost":null,"monthly_input_cost":null,"monthly_output_cost":null,"monthly_margin_cost":null,"input_cost_per_token":3e-6,"output_cost_per_token":4e-6,"provider":"openai"}

Public models are unaffected; gpt-4 estimates its mapped cost identically on both legs

$ curl -s -X POST http://localhost:44923/cost/estimate -H "Authorization: Bearer sk-1234" \
    -H "Content-Type: application/json" \
    -d '{"model":"gpt-4","input_tokens":1000,"output_tokens":500}'
{"model":"gpt-4","input_tokens":1000,"output_tokens":500,...,"cost_per_request":0.060000000000000005,"input_cost_per_token":0.00003,"output_cost_per_token":0.00006,"provider":"openai"}

QA observations

• Unpriced on-prem models now estimate 0.0 silently, by design
• This PR causes that; configured pricing is the remedy

Type

🐛 Bug Fix

Caveats (if any)

QA runbook

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

The PR forwards router-resolved provider metadata and deployment-specific token pricing into /cost/estimate.

• Introduces a structured result for resolved model, provider, and custom pricing.
• Applies deployment pricing from litellm_params or model_info, with litellm_params taking precedence.
• Adds regression coverage for priced and unpriced self-hosted deployments.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/proxy/management_endpoints/cost_tracking_settings.py	Resolves deployment provider and custom token pricing before invoking the existing cost calculator.
tests/test_litellm/proxy/management_endpoints/test_cost_tracking_settings.py	Adds coverage for self-hosted model estimation and deployment-pricing precedence.

_{Reviews (4): Last reviewed commit: "fix(proxy): honor model_info custom pric..." | Re-trigger Greptile}

[greptile-apps]

New explanatory comments violate guidance

The new helper and regression tests add explanatory docstrings and comments despite the repository instruction prohibiting new comments unless explicitly requested, adding convention cleanup across both changed files

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing devin_ai_fix_cost_estimate_onprem_provider_35210 (0fdbe03) with litellm_internal_staging (7e80e09)}

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 0fdbe03. Configure here.}

[mateo-berri]

LGTM