fix(model_prices): correct stale retirement dates (Azure, Vertex, Bedrock) and add deepseek-v4-flash-vision-exp - #37843
Conversation
…-flash-vision-exp Verified against Azure Foundry model retirement schedule, Vertex AI model versions, AWS Bedrock model lifecycle and DeepSeek pricing docs. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
Greptile SummaryThis PR updates stale or missing retirement metadata for selected Azure, Vertex AI, and Bedrock models and registers DeepSeek V4 Flash Vision Experimental under bare and provider-prefixed names.
Confidence Score: 5/5The PR appears safe to merge; no concrete changed-code failure was identified in the synchronized metadata updates. The retirement dates use valid registry fields, the new DeepSeek entries follow established bare and provider-prefixed lookup conventions, and both runtime and source registry copies contain matching changes.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Updates retirement metadata and adds internally consistent DeepSeek vision-model registry entries without an identified defect. |
| litellm/model_prices_and_context_window_backup.json | Mirrors the primary registry changes so packaged runtime metadata remains synchronized. |
Reviews (1): Last reviewed commit: "fix(model_prices): correct stale retirem..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Hi @mateo-berri , I just want to let you know this PR not complete for supporting deepseek-v4-flash-vision-exp. Adding Inside |
|
Superseded by rolling registry audit #37902, which reverified and absorbed the provider-documented metadata changes at field level |
TLDR
Problem this solves:
azure/gpt-4o,azure/sora-2, TTS/transcribe entries had no retirement datetext-embedding-004retirement date predated Google's current scheduleeu.anthropic.claude-opus-4-1was missing its Bedrock EOL dateHow it solves it:
deprecation_dateto the provider's published retirement datedeepseek-v4-flash-vision-exp(anddeepseek/alias)User Flow
Before: a proxy admin tracking model end-of-life sees wrong dates for Azure GPT-4.1-nano and Vertex
text-embedding-004, and no date at all forazure/gpt-4oazure/gpt-4.1-nanocomes back with"deprecation_date": "2026-10-14"— six months earlier than Microsoft's published retirement, so it looks urgent when it isn'ttext-embedding-004comes back with"deprecation_date": "2026-01-14", a date already in the past, so the model reads as retired while it still serves trafficazure/gpt-4o,azure/sora-2,azure/gpt-4o-mini-tts,azure/gpt-4o-mini-transcribeandeu.anthropic.claude-opus-4-1-20250805-v1:0are absent from the response entirely, so no advance warning fires for them"model": "deepseek/deepseek-v4-flash-vision-exp"and the response is logged at $0 spend, since the model has no pricing entryAfter: the same endpoint reports each provider's published retirement date, and the DeepSeek vision model is priced
azure/gpt-4.1-nanonow reports2027-04-14, matching Microsoft's scheduletext-embedding-004now reports2027-04-01, matching Google's scheduleazure/gpt-4o(2027-04-14),azure/sora-2(2026-09-15),azure/gpt-4o-mini-ttsandazure/gpt-4o-mini-transcribe(2027-06-15) andeu.anthropic.claude-opus-4-1-20250805-v1:0(2027-01-08) now appear, so alerts fire ahead of retirement"model": "deepseek/deepseek-v4-flash-vision-exp"is logged at real spend at $0.44/M input, $1.32/M output, and image input is advertised as supportedSources for every changed value
Azure — https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/model-retirement-schedule (Azure OpenAI table, "Retirement date" column):
azure/gpt-4.1-nanoazure/gpt-4.1-nano-2025-04-14azure/us/gpt-4.1-nano-2025-04-14azure/gpt-4oazure/gpt-4o-mini-transcribeazure/gpt-4o-mini-ttsazure/sora-2The earlier 2026-10-14 value matches no row on that page; the fine-tuned-model table on the lifecycle page (2027-04-01/2027-04-14 training, 2027-10-xx deployment) covers fine-tuned deployments only and is not the base-model retirement date.
Vertex AI — https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions ("Embeddings models" table):
text-embedding-004, released May 14 2024, retirement April 1 2027 →text-embedding-0042026-01-14 → 2027-04-01. Sibling rowstext-embedding-005,text-multilingual-embedding-002,multimodalembedding@001already carry 2027-04-01 in the registry.Bedrock — https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html (legacy/EOL table):
anthropic.claude-opus-4-1-20250805-v1:0EOL January 8 2027 → added toeu.anthropic.claude-opus-4-1-20250805-v1:0, which lacked it while the base andus.keys already had it.DeepSeek — https://api-docs.deepseek.com/quick_start/pricing:
deepseek-v4-flash-vision-expshares thedeepseek-v4-flashcolumn values (1M context, 384K max output, peak $0.014/M cache-hit input, $0.44/M cache-miss input, $1.32/M output; JSON output, tool calls, chat-prefix completion supported). New entriesdeepseek-v4-flash-vision-expanddeepseek/deepseek-v4-flash-vision-expmirror the existing flash entries withsupports_vision: true.Audited and left unchanged
Checked against the raw provider tables, no registry change needed:
claude-mythos-previewis stated as deprecated with no retirement date published, so nothing to write.gemini-3.1-flash-lite2027-05-07,gemini-2.5-flash-image2026-10-02,gemini-embedding-0012028-05-14 on the Gemini key vs 2028-05-20 on the Vertex key, imagen-4.0 2026-08-17, veo 2026-06-30, robotics-er-1.6 2026-08-31). The GA Gemini 3.x models say "no shutdown date announced" and are correctly left without one.gpt-4o-realtime-preview*,gpt-4o-mini-realtime-preview,gpt-4o-audio-preview,gpt-4o-mini-audio-previewcarry 2027-01-20 (thegpt-4o-realtime/gpt-4o-audiofamily row) while the older per-snapshot rows say 2026-05-07; the page supports both readings, so left alone rather than guessed.azure/gpt-realtime-mini-2025-10-06has two conflicting rows for the same version (2027-04-06 and 2026-09-21) so it was left unset.command-a-plus-05-2026,command-a-reasoning-08-2025,command-a-vision-07-2025,command-a-translate-08-2025,rerank-v4.0-pro/fastare missing under thecohereprovider, but Cohere's pricing page renders its per-model API rates client-side and no official price for them could be read, so they were not added. Same for xAI's Imagine image/video and voice models and Cohere's Aya/transcribe models.Rolling registry PR
This is the single open registry PR. The four other open PRs touching this file are code/behavior changes that only incidentally edit the JSON, so they were left alone and nothing was closed: #37375 and #35987 (Bedrock Mantle web-search transformation fix) and #36422 and #36416 (moving
fallback_generalizationsout of the cost map).Relevant issues
Supports #26900 (proactive model deprecation alerts /
/model/deprecations).Linear ticket
Pre-Submission checklist
Screenshots / Proof of Fix
After (36837d3)
Registry checks
python3 ci_cd/check_files_match.py→Passed! Files model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json match.uv run --no-sync python ci_cd/generate_model_prices_schema.py→ schema regenerated with no diffLITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest tests/test_litellm/test_model_prices_schema.py tests/test_litellm/test_model_cost_aliases.py tests/test_litellm/proxy/test_model_deprecations_endpoint.py tests/test_litellm/proxy/common_utils/test_model_deprecation.py tests/test_litellm/integrations/SlackAlerting/test_model_deprecation_alert.py -q→66 passedLink to Devin session: https://app.devin.ai/sessions/cee732482ae349b08b2b8c336a38e369