fix(router): don't log 'Could not identify azure model' when the deployment name resolves from the cost map

[tin-berri]

Note

Copy of #33292 by @mihidumh, pushed to a litellm_ branch so the full CircleCI suite runs. Commit authorship is preserved; full credit to the original author. The only difference from the original branch is that 476 regenerated litellm/proxy/_experimental/out build files that were accidentally swept into the second commit have been dropped, and the base has been updated to latest litellm_internal_staging (one trivial append conflict in tests/test_litellm/test_router.py resolved by keeping both new test classes)

Relevant issues

Fixes #33172

Original PR: #33292

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

Screenshots / Proof of Fix

Production observation by the original author (litellm proxy 1.92.0 on Azure, real traffic): every request to a multi-deployment azure group without base_model logs

Could not identify azure model 'gpt-5.4'. Set azure 'base_model' for accurate max tokens, cost tracking, etc.

thousands of ERROR lines a day, even though azure/gpt-5.4 is an exact key in the shipped cost map and get_router_model_info's existing if model is None: model = _model fallback resolves it correctly. The original author verified on a live proxy that hand-setting model_info.base_model = litellm_params.model on every such row (56 rows on one env) changes nothing about resolution and only silences the log; the ERROR was spurious for exactly the deployments that need no operator action

Membership in litellm.model_cost cannot be the discriminator on its own: Router.__init__ auto-registers every deployment name into the map as a zeroed stub (register_model: model=... not in built-in cost map ... will default to 0). The check therefore requires the fallback entry to carry usable limits/costs

Unit tests (new TestAzureBaseModelFallbackLogging in tests/test_litellm/test_router.py):

Before (base): test_map_known_deployment_name_resolves_without_error_log FAILS (spurious ERROR fires); the other 2 pass

After (this branch):

$ pytest tests/test_litellm/test_router.py::TestAzureBaseModelFallbackLogging -q
3 passed

covering: (1) map-known deployment name resolves with no ERROR and model_info carries the map's max tokens/costs, (2) genuinely unmappable name still logs the ERROR and returns the zeroed stub unchanged, (3) explicit base_model still wins

Type

🐛 Bug Fix

Changes

• In get_router_model_info's azure branch: when base_model is unset, check whether the azure/<deployment model name> entry in the cost map carries usable limits/costs (guarding against Router-init's zeroed auto-registration stubs). If so, log at debug; the existing fallback resolution handles it. Otherwise keep the existing error
• No behavior change to resolution or returned model_info; logging level only

---

Note

Low Risk
Logging-only change in Azure model-info lookup; cost/token resolution behavior is unchanged.

Overview
Stops get_router_model_info from ERROR-logging “Could not identify azure model” on every request when an Azure deployment has no base_model but the name already maps to a real cost-map entry (usable max tokens or input cost). Those cases now log at debug; genuinely unmappable names still ERROR.

Resolution and returned model_info are unchanged. Router-init zeroed stubs are ignored so membership in litellm.model_cost alone is not treated as a hit. Tests cover known names, unmappable names, and explicit base_model.

^{Reviewed by Cursor Bugbot for commit 80b7d53. Bugbot is set up for automated code reviews on this repo. Configure here.}

[tin-berri]

@greptileai can you reply here with a score and why that score?

[tin-berri]

bugbot run

[greptile-apps]

Score: 4/5

What the PR does well:

• Surgical scope. Only the log level changes; model resolution, returned model_info, and all callers are untouched. Zero backwards-compatibility risk.
• Zeroed-stub guard is correct. Router init auto-registers every deployment name as a zeroed entry, so checking bare membership in litellm.model_cost would be wrong. The _fallback_resolves check (max_input_tokens > 0 or max_tokens > 0 or input_cost_per_token > 0) correctly discriminates real entries from stubs.
• azure/ prefix handling. The _model.startswith("azure/") branch avoids double-prefixing for deployment names that are already fully qualified.
• Tests are solid. TestAzureBaseModelFallbackLogging covers the three distinct cases (known name → no ERROR, unmappable name → ERROR stays, explicit base_model wins), and the monkeypatch/autouse setup is correct.

One minor concern keeping it from 5/5:

The inline comments in router.py are five lines of prose to explain ~10 lines of logic. The "zeroed stub" behavior is genuinely non-obvious (I'd keep a short comment about it), but the rest restates what the code already says. Per the repo's convention, comments should only appear where the logic isn't self-evident — trimming to just the zeroed-stub note would tighten the diff and reduce router.py noise.

+            # membership alone isn't enough: Router init auto-registers every
+            # deployment name into litellm.model_cost as a zeroed stub, so
+            # require the entry to carry usable limits/costs.
             _azure_fallback_key = _model if _model.startswith("azure/") else f"azure/{_model}"

That's a style nit, not a correctness issue. The fix itself is correct and the tests verify all paths.

[greptile-apps]

Greptile Summary

The PR suppresses an Azure model-identification error when the deployment name resolves to a cost-map entry with usable limits or pricing, while preserving the error for zeroed registration stubs

• Adds a positive-field check before choosing debug or error logging
• Adds regression coverage for known, unknown, and explicitly configured Azure models

Confidence Score: 4/5

The PR appears safe to merge after addressing two non-blocking maintainability issues in the implementation comments and test fixture

The Azure fallback remains aligned with downstream cost-map resolution, and the accepted concerns are limited to redundant comments and avoidable global-state mutation in tests

Files Needing Attention: litellm/router.py, tests/test_litellm/test_router.py

Important Files Changed

Filename	Overview
litellm/router.py	The fallback logging behavior is narrowly corrected, but the added explanatory block violates the repository's concise-comment policy
tests/test_litellm/test_router.py	The tests cover the intended logging cases, but their local fixture directly mutates the global cost map instead of reusing isolated fixture infrastructure

_{Reviews (1): Last reviewed commit: "fix(router): use consistent positive che..." | Re-trigger Greptile}

[greptile-apps]

Redundant fallback explanation

This comment restates the fallback lookup and positive-field check immediately below, adding maintenance overhead if the implementation changes

Suggested change

	# the `if model is None` fallback below resolves the deployment's
	# model name against the model cost map — when the name is a known
	# azure key (e.g. deployment model "azure/gpt-4o"), that resolution
	# gives correct max tokens / costs and there is nothing for the
	# operator to fix, so don't spam an ERROR on every request.
	# membership alone isn't enough: Router init auto-registers every
	# deployment name into litellm.model_cost as a zeroed stub, so
	# require the entry to carry usable limits/costs.

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[tin-berri]

Trimmed to just the zeroed-stub note in 419d306

[greptile-apps]

Direct global cost-map mutation

This fixture directly reassigns litellm.model_cost, adding global-state coupling and manual restoration instead of reusing the repository's isolated cost-map fixture and cache handling

Context Used: CLAUDE.md (source)

Knowledge Base Used: Roll back live Router membership between tests

[tin-berri]

Switched to the shared local_model_cost_map fixture in 419d306, which also clears the get_model_info cache

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 43f6a5f. Configure here.}

[tin-berri]

Addressed both notes in 419d306. @greptileai can you reply here with a score and why that score?

[greptile-apps]

The implementation looks correct and addresses both previous notes. Here's the breakdown:

Score: 5/5

Why:

1. Logic is sound. The zeroed-stub guard (max_input_tokens > 0 || max_tokens > 0 || input_cost_per_token > 0) correctly distinguishes a Router-init auto-registered stub (all zeros) from a real cost-map entry. Membership alone would false-positive on every deployment.

2. No behavior change. Resolution and returned model_info are unaffected — only the log level changes. The elif custom_llm_provider != "azure": model = _model path is untouched.

3. _azure_fallback_key prefix guard (if _model.startswith("azure/") else f"azure/{_model}") handles both "azure/gpt-4o" and bare "gpt-4o" forms correctly.

4. Test coverage is solid. Three cases: map-known name (no ERROR, correct values surfaced), unmappable name (ERROR preserved, zeroed stub unchanged), explicit base_model wins. The local_model_cost_map fixture pins the cost map so tests are deterministic and won't drift with future model additions.

5. No rule violations. This is a logging-only fix in the Azure branch of get_router_model_info. No new DB queries, no new Router objects, no FastAPI imports, no hardcoded model flags, no provider-specific code outside the right path, no backwards-incompatible behavior change.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_pr33292_azure_base_model_log (80b7d53) with litellm_internal_staging (0a5fa4f)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (247eaaa) during the generation of this report, so 0a5fa4f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 419d306. Configure here.}

[tin-berri]

Merged latest litellm_internal_staging to pick up the test tree lint fix. @greptileai can you reply here with a score and why that score?

[greptile-apps]

Score: 4.5/5

What the change does: When an Azure deployment has no base_model, it now checks whether the azure/<deployment_model_name> entry in litellm.model_cost has usable limits/costs (non-zero max_input_tokens, max_tokens, or input_cost_per_token). If it does, it logs at debug instead of error. The zeroed-stub guard is the key insight — without it, every Router-registered deployment would suppress the error spuriously.

Why this is correct:

• The discriminator (non-zero cost/token values) correctly distinguishes real cost-map entries from the zeroed stubs auto-registered by Router.__init__
• Resolution logic is untouched — only the log level changes, so no behavioral regression is possible
• The three test cases cover the three meaningful branches: known name (no error), unknown name (error preserved), and explicit base_model wins

Minor flag (reason it's not 5/5): The router.py change adds comments inside production code (# Router init auto-registers every deployment name...). Per the repo style guide, comments are only warranted where logic isn't self-evident. This non-obvious zeroed-stub guard is precisely that case, so the comment is arguably justified — but it's a judgment call, and a strict reading of the style guide would prefer it removed.

Everything else is tight: no behavior change, no new DB queries in the critical path, no provider-specific code outside llms/, no backwards-incompatible changes, and tests are genuine regression detectors (not weakened to hide a bug).

[tin-berri]

Keeping the stub guard note: the first review asked to retain exactly this comment, and the guard is genuinely non-obvious

[tin-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 80b7d53. Configure here.}