Skip to content

fix(cost-map): take the later azure deprecation date for gpt-4.1-nano, gpt-4o-transcribe and gpt-realtime-2.1 - #42986

Merged
litellm-stampbot merged 1 commit into
mainfrom
litellm_sync_azure_deprecation
Sep 24, 2026
Merged

litellm-stampbot merged 1 commit into
mainfrom
litellm_sync_azure_deprecation

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

How it solves it:

  • Kerry ruled to take the later date for each disputed row
  • Sets deprecation_date on the azure gpt-4.1-nano, gpt-4o-transcribe and gpt-realtime-2.1 keys
  • No prices or other fields change, no keys deleted

User Flow

Before: a developer checking when azure gpt-4.1-nano stops serving sees the earlier of two vendor dates

  1. They run python -c "import litellm; print(litellm.model_cost['azure/gpt-4.1-nano']['deprecation_date'])"
  2. It prints 2026-10-14
  3. Azure's own Models API says the same version serves until 2027-04-14, so they plan a migration six months early

After: the same lookup reports the later vendor date

  1. They run the same command
  2. It prints 2027-04-14, the later of the two Azure sources

Evidence

Models API: npm run providers -- list azure --raw in litellm-providers on 2026-09-24 (management.azure.com locations/eastus2/models, model.deprecation.inference). Page: https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/model-retirement-schedule fetched raw with a cache buster on 2026-09-24 (ms.date 2026-09-21), cells quoted as Model | Version | Lifecycle | Retirement date | Replacement. Kerry ruled on 2026-09-24 to take the later of the two dates per row

API row API inference date Page cell (quoted) Chosen Catalog keys Change
OpenAI.gpt-4.1-nano.2025-04-14 2027-04-14 gpt-4.1-nano | 2025-04-14 | Deprecated | 2026-10-14 | — 2027-04-14 azure/gpt-4.1-nano, azure/gpt-4.1-nano-2025-04-14, azure/us/gpt-4.1-nano, azure/us/gpt-4.1-nano-2025-04-14, azure/eu/gpt-4.1-nano 2026-10-14 -> 2027-04-14
OpenAI.gpt-4o.2024-05-13 2026-12-09 gpt-4o | 2024-05-13 | Deprecated | 2026-12-09 | gpt-5.6-sol 2026-12-09 azure/gpt-4o-2024-05-13, azure/us/gpt-4o-2024-05-13, azure/eu/gpt-4o-2024-05-13 none, sources now agree and catalog already carries it
OpenAI.gpt-4o-transcribe.2025-03-20 2026-12-31 gpt-4o-transcribe | 2025-03-20 | GA | 2026-10-15 | — 2026-12-31 azure/gpt-4o-transcribe 2026-10-15 -> 2026-12-31
OpenAI.gpt-realtime-2.1.2026-07-07 2027-07-31 gpt-realtime-2.1 | 2026-07-07 | GA | 2027-06-25 | — 2027-07-31 azure/gpt-realtime-2.1 2027-06-25 -> 2027-07-31
OpenAI.gpt-image-2.5-flare.2026-09-08, OpenAI.gpt-image-2.5-sunburst.2026-09-08 2027-09-08 gpt-image-2.5-flare | 2026-09-09 | GA | 2027-09-09 | — and gpt-image-2.5-sunburst | 2026-09-09 | GA | 2027-09-09 | — 2027-09-09 azure/gpt-image-2.5-flare, azure/gpt-image-2.5-sunburst none, catalog already carries the later date

The same page also has a "Deployment retirement date" table listing gpt-4.1-nano 2025-04-14 at 2027-10-14 for training retirement extensions. That table was not part of the ruling and is not used here

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests (data-only change, covered by the existing cost map schema and guard tests)
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/test_model_prices_schema.py tests/test_litellm/test_cost_map_guard.py -v
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

The proxy's /model/info does not expose deprecation_date, so the check reads the cost map the SDK loads. Before reads the published map on main, After reads the branch's local map

Before (f0e671f)

  1. for m in azure/gpt-4.1-nano azure/gpt-4o-transcribe azure/gpt-realtime-2.1; do uv run --no-sync python -c "import litellm; print('$m', litellm.model_cost['$m'].get('deprecation_date'))"; done
  2. Output:
azure/gpt-4.1-nano 2026-10-14
azure/gpt-4o-transcribe 2026-10-15
azure/gpt-realtime-2.1 2027-06-25

After (01f61cc)

  1. Same loop with LITELLM_LOCAL_MODEL_COST_MAP=True set
  2. Output:
azure/gpt-4.1-nano 2027-04-14
azure/gpt-4o-transcribe 2026-12-31
azure/gpt-realtime-2.1 2027-07-31

Type

🐛 Bug Fix

Caveats (if any)

Low

  • azure/gpt-realtime-2.1-mini has the same API vs page gap (2027-07-31 vs 2027-06-25) and was left at 2027-06-25 because it was outside the ruling
  • us-gov azure rows are untouched by design

Link to Devin session: https://app.devin.ai/sessions/390dba1c22de4ef9af7ab0c0d3a94c1a
Open in Devin Desktop: https://app.devin.ai/desktop/session/390dba1c22de4ef9af7ab0c0d3a94c1a?variant=devin
Requested by: @kerry-berri


Note

Low Risk
Data-only metadata updates in the model cost map with no changes to pricing, routing, or runtime inference behavior.

Overview
Updates deprecation_date on disputed Azure Foundry model entries in model_prices_and_context_window.json and its LiteLLM backup copy so the catalog reflects the later of Azure’s Models API vs. retirement schedule page (per #42933 follow-up).

gpt-4.1-nano keys (azure/gpt-4.1-nano, dated variant, and us/eu regional aliases) move from 2026-10-14 to 2027-04-14. azure/gpt-4o-transcribe moves 2026-10-15 → 2026-12-31. azure/gpt-realtime-2.1 moves 2027-06-25 → 2027-07-31.

No token pricing, capability flags, or model keys are added or removed—only retirement metadata for SDK/litellm.model_cost lookups.

Reviewed by Cursor Bugbot for commit 01f61cc. Bugbot is set up for automated code reviews on this repo. Configure here.

…, gpt-4o-transcribe and gpt-realtime-2.1

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@litellm-stampbot

Copy link
Copy Markdown
Contributor

@greptileai

@litellm-stampbot

Copy link
Copy Markdown
Contributor

bugbot run

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 01f61cc. Configure here.

@codspeed

codspeed Bot commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_sync_azure_deprecation (01f61cc) with main (f0e671f)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (bd55afc) during the generation of this report, so f0e671f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@greptile-apps

greptile-apps Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

The PR appears safe to merge; targeted date assertions would provide useful protection against regression.

Findings

  1. P2 Retirement dates lack test coverage ▶

Summary

The PR updates Azure deprecation metadata to the later selected dates for gpt-4.1-nano aliases, gpt-4o-transcribe, and gpt-realtime-2.1 in both cost-map copies.

  • Pricing, model keys, and inference behavior are unchanged.
  • The selected dates lack assertions that would protect them from a later cost-map update.

Reviews (1) · Last reviewed commit: "fix(cost-map): take the later azure depr..."

},
"azure/gpt-4.1-nano": {
"deprecation_date": "2026-10-14",
"deprecation_date": "2027-04-14",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Retirement dates lack test coverage The existing schema and cost-map guard tests would still pass if these seven Azure dates were changed back to the earlier, valid dates. They check date format and file synchronization, not the later-date decision. An assertion for the affected keys would help prevent a future cost-map update from silently undoing this correction.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These dates are vendor data synced from the Azure Models API and retirement schedule page, so pinning them in tests would break every legitimate sync

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s a fair point. These values are vendor-sourced metadata that can legitimately change during future catalog syncs, so exact-date assertions would turn routine upstream updates into test failures rather than catch a defect. The existing schema and synchronization checks are appropriate here; I’m withdrawing the test-coverage suggestion.

@codecov

codecov Bot commented Sep 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@litellm-stampbot litellm-stampbot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@litellm-stampbot
litellm-stampbot merged commit de314f8 into main Sep 24, 2026
100 checks passed
@litellm-stampbot
litellm-stampbot deleted the litellm_sync_azure_deprecation branch September 24, 2026 16:57
devin-ai-integration Bot added a commit that referenced this pull request Sep 24, 2026
…, gpt-4o-transcribe and gpt-realtime-2.1 (#42986)

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants