fix: suppress misleading register_model unresolved-cost warnings for entries without custom pricing - #38542
Conversation
…entries without custom pricing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
No action taken on #38542 — repo and author ( |
|
|
Greptile SummaryThe PR suppresses unresolved-cache-pricing warnings for registrations without explicit custom pricing and replaces opaque deployment IDs with readable model names.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| litellm/utils.py | Narrows the warning condition and accurately describes the zero-valued cache pricing fallback for non-tiered custom pricing. |
| litellm/router.py | Supplies the configured backend model as the warning display name when registering deployment IDs. |
| litellm/main.py | Supplies the shared provider/model key as the display name during per-request router registration. |
| tests/test_litellm/test_register_model_custom_pricing.py | Adds focused coverage for silent unpriced and tiered registrations plus readable router warning names. |
Reviews (2): Last reviewed commit: "fix: do not warn about zero cache costs ..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
Fixed in 992b74b: tiered entries no longer warn since tiered cache reads fall back to the input rate, and a regression test covers it |
…itellm_fix_register_model_unresolved_cost_warnings
TLDR
Problem this solves:
register_modelunresolved-cost warnings for deployments with no custom pricingHow it solves it:
User Flow
Before: a proxy admin with Bedrock models configured (with or without explicit pricing) sees scary unresolved-cost warnings at every boot
register_model: model=d3582bc5... not in built-in cost map ... cache cost fields will default to 0for hashed ids and for deployments that have no custom pricing at allAfter: the same boot is quiet unless a deployment genuinely has incomplete custom pricing
register_modelwarning is printed for deployments without custom pricing or whose backend model resolves in the built-in cost mapPOST http://localhost:4000/v1/chat/completionsreturns 200 with the samex-litellm-response-costas beforeRelevant issues
Fixes #32484
Linear ticket
Resolves LIT-6318
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Shared setup: live proxy on localhost:4000,
LITELLM_LOCAL_MODEL_COST_MAP=True, real AWS Bedrock credentials. Main config has a Bedrock deployment with explicit input/output/cache_read pricing plus an inference profile id, an Azure deployment with a base_model, and a plain Bedrock invoke deployment. Edge config has one Bedrock deployment with custom input/output pricing on a made-up model name (bedrock/lit6318-totally-made-up-model) so pricing is genuinely incompleteBefore (86ef1fb)
Boot with configured pricing emits opaque-hash warnings
litellm --config lit6318_config.yaml --port 4000, thengrep register_model proxy.logLive Bedrock request is billed correctly despite the warnings
curl -sD - http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"bedrock-invoke-haiku","messages":[{"role":"user","content":"Say OK"}],"max_tokens":10}'HTTP/1.1 200 OKwithx-litellm-response-cost: 3.19e-05and the completion"OK", proving the warnings were noiseGenuinely incomplete custom pricing warns with a hash
grep register_model proxy.logAfter (d321dc6)
Boot with configured pricing emits opaque-hash warnings
litellm --config lit6318_config.yaml --port 4000(byte-identical config), thengrep register_model proxy.logregister_modelwarningsLive Bedrock request is billed correctly despite the warnings
HTTP/1.1 200 OKwith the identicalx-litellm-response-cost: 3.19e-05, so per-request spend is unchangedGenuinely incomplete custom pricing warns with a hash
grep register_model proxy.logType
Bug Fix
Caveats (if any)
Low
provider/modelkey, not the hashFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/7f184ca33c3f4ac08bcbc818441b74c0
Open in Devin Desktop: https://app.devin.ai/desktop/session/7f184ca33c3f4ac08bcbc818441b74c0?variant=devin
Requested by: @yassin-berriai