Skip to content

feat(azure-ai): add latest catalog model metadata - #39422

Closed
emerzon wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
emerzon:litellm_azure_ai_catalog_models
Closed

feat(azure-ai): add latest catalog model metadata#39422
emerzon wants to merge 2 commits into
BerriAI:litellm_internal_stagingfrom
emerzon:litellm_azure_ai_catalog_models

Conversation

@emerzon

@emerzon emerzon commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Eight exact Azure AI catalog IDs lack model metadata
  • Missing prices and limits weaken routing and spend tracking

How it solves it:

  • Adds modes, prices, limits, capabilities, and retirement dates
  • Keeps primary and bundled catalogs covered by focused tests

User Flow

Before: a developer cannot inspect complete built-in metadata for a newly configured Azure AI catalog deployment

  1. They configure azure_ai/FW-GLM-5.3 for an Azure AI deployment
  2. They call GET https://litellm-domain/v1/model/info
  3. The deployment has no built-in price, context, retirement, or capability metadata

After: the same developer can inspect the model's published metadata before sending traffic

  1. They configure azure_ai/FW-GLM-5.3 for an Azure AI deployment
  2. They call GET https://litellm-domain/v1/model/info
  3. The deployment includes its price, context, retirement, and capability metadata

Relevant issues

Linear ticket

Sources

Validation notes

  • Native Azure Kimi-K3 deployments provisioned successfully in two regions
  • Both regional native inference endpoints returned HTTP 404
  • The live gateway removed those routes and retained working FW-Kimi-K3
  • The Kimi-K3 entry remains published catalog metadata, not live-route validation

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review)

Screenshots / Proof of Fix

This PR adds catalog metadata only; live inference varies by model and Azure deployment

Type

New Feature
Test

Caveats (if any)

Medium

  • Fireworks DeepSeek pricing is provisional until Azure publishes its meter
  • Fireworks GLM pricing is provisional until Azure publishes its meter
  • xAI Grok pricing is provisional until Azure publishes its meter
  • Native Kimi-K3 metadata does not imply working Azure inference
    • Two regional deployments provisioned successfully
    • Both regional inference endpoints returned HTTP 404
    • Live routes were removed while FW-Kimi-K3 remained active

Low

  • MAI Image uses the live Azure catalog retirement date
  • Microsoft's public schedule currently reports 2026-10-01 instead

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@devin-ai-integration devin-ai-integration Bot added the risk:medium Moderate-risk functional change label Sep 2, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

PR #39422 labeled risk:medium (3 files, +570/-0: additive model pricing metadata + tests; exceeds low-risk size threshold, no sensitive area). No enterprise label, so Linear routing did not run.

@codspeed-hq

codspeed-hq Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing emerzon:litellm_azure_ai_catalog_models (1a8c889) with litellm_internal_staging (6c5fb0e)

Open in CodSpeed

@codecov

codecov Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@devin-ai-integration

Copy link
Copy Markdown
Contributor

#39388 now carries Codestral-2501, MAI-Thinking-1 and mistral-ocr-4-0 from here. The other five entries have no Azure Retail Prices meter yet, so they stay unabsorbed

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Rolling registry PR #39388 now carries Codestral-2501, MAI-Thinking-1 and mistral-ocr-4-0 with Azure-published fields; the other five rows still need an Azure meter

@mateo-berri

Copy link
Copy Markdown
Contributor

Closing: the verifiable rows landed in #39388; the remaining entries have no Azure meter or pricing row to check against. Reopen with an official source.

@mateo-berri mateo-berri closed this Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:medium Moderate-risk functional change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants