Skip to content

fix(model_prices): correct stale retirement dates (Azure, Vertex, Bedrock) and add deepseek-v4-flash-vision-exp - #37843

Closed
devin-ai-integration[bot] wants to merge 1 commit into
litellm_internal_stagingfrom
devin/1787318013-registry-audit
Closed

fix(model_prices): correct stale retirement dates (Azure, Vertex, Bedrock) and add deepseek-v4-flash-vision-exp#37843
devin-ai-integration[bot] wants to merge 1 commit into
litellm_internal_stagingfrom
devin/1787318013-registry-audit

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Four Azure model retirement dates were stale or wrong
  • azure/gpt-4o, azure/sora-2, TTS/transcribe entries had no retirement date
  • text-embedding-004 retirement date predated Google's current schedule
  • eu.anthropic.claude-opus-4-1 was missing its Bedrock EOL date
  • DeepSeek's vision model was missing from the registry

How it solves it:

  • Set each deprecation_date to the provider's published retirement date
  • Added deepseek-v4-flash-vision-exp (and deepseek/ alias)
  • Every value taken from the official provider table (links below)

User Flow

Before: a proxy admin tracking model end-of-life sees wrong dates for Azure GPT-4.1-nano and Vertex text-embedding-004, and no date at all for azure/gpt-4o

  1. They call GET https://litellm-domain/model/deprecations
  2. azure/gpt-4.1-nano comes back with "deprecation_date": "2026-10-14" — six months earlier than Microsoft's published retirement, so it looks urgent when it isn't
  3. text-embedding-004 comes back with "deprecation_date": "2026-01-14", a date already in the past, so the model reads as retired while it still serves traffic
  4. azure/gpt-4o, azure/sora-2, azure/gpt-4o-mini-tts, azure/gpt-4o-mini-transcribe and eu.anthropic.claude-opus-4-1-20250805-v1:0 are absent from the response entirely, so no advance warning fires for them
  5. They send POST https://litellm-domain/v1/chat/completions with "model": "deepseek/deepseek-v4-flash-vision-exp" and the response is logged at $0 spend, since the model has no pricing entry

After: the same endpoint reports each provider's published retirement date, and the DeepSeek vision model is priced

  1. They call GET https://litellm-domain/model/deprecations
  2. azure/gpt-4.1-nano now reports 2027-04-14, matching Microsoft's schedule
  3. text-embedding-004 now reports 2027-04-01, matching Google's schedule
  4. azure/gpt-4o (2027-04-14), azure/sora-2 (2026-09-15), azure/gpt-4o-mini-tts and azure/gpt-4o-mini-transcribe (2027-06-15) and eu.anthropic.claude-opus-4-1-20250805-v1:0 (2027-01-08) now appear, so alerts fire ahead of retirement
  5. The same POST with "model": "deepseek/deepseek-v4-flash-vision-exp" is logged at real spend at $0.44/M input, $1.32/M output, and image input is advertised as supported

Sources for every changed value

Azure — https://learn.microsoft.com/en-us/azure/foundry/openai/concepts/model-retirement-schedule (Azure OpenAI table, "Retirement date" column):

key before after doc row
azure/gpt-4.1-nano 2026-10-14 2027-04-14 gpt-4.1-nano 2025-04-14, Legacy, 2027-04-14
azure/gpt-4.1-nano-2025-04-14 2026-10-14 2027-04-14 same row
azure/us/gpt-4.1-nano-2025-04-14 2026-10-14 2027-04-14 same row
azure/gpt-4o (none) 2027-04-14 gpt-4o 2024-08-06 and 2024-11-20, 2027-04-14
azure/gpt-4o-mini-transcribe (none) 2027-06-15 gpt-4o-mini-transcribe 2025-12-15, GA, 2027-06-15
azure/gpt-4o-mini-tts (none) 2027-06-15 gpt-4o-mini-tts 2025-12-15, GA, 2027-06-15
azure/sora-2 (none) 2026-09-15 sora-2 2025-12-08, Preview, 2026-09-15

The earlier 2026-10-14 value matches no row on that page; the fine-tuned-model table on the lifecycle page (2027-04-01/2027-04-14 training, 2027-10-xx deployment) covers fine-tuned deployments only and is not the base-model retirement date.

Vertex AI — https://cloud.google.com/vertex-ai/generative-ai/docs/learn/model-versions ("Embeddings models" table): text-embedding-004, released May 14 2024, retirement April 1 2027 → text-embedding-004 2026-01-14 → 2027-04-01. Sibling rows text-embedding-005, text-multilingual-embedding-002, multimodalembedding@001 already carry 2027-04-01 in the registry.

Bedrock — https://docs.aws.amazon.com/bedrock/latest/userguide/model-lifecycle.html (legacy/EOL table): anthropic.claude-opus-4-1-20250805-v1:0 EOL January 8 2027 → added to eu.anthropic.claude-opus-4-1-20250805-v1:0, which lacked it while the base and us. keys already had it.

DeepSeek — https://api-docs.deepseek.com/quick_start/pricing: deepseek-v4-flash-vision-exp shares the deepseek-v4-flash column values (1M context, 384K max output, peak $0.014/M cache-hit input, $0.44/M cache-miss input, $1.32/M output; JSON output, tool calls, chat-prefix completion supported). New entries deepseek-v4-flash-vision-exp and deepseek/deepseek-v4-flash-vision-exp mirror the existing flash entries with supports_vision: true.

Audited and left unchanged

Checked against the raw provider tables, no registry change needed:

  • Anthropic https://docs.claude.com/en/docs/about-claude/model-deprecations — all 16 dated rows already match, including Fable 5, Opus 5/4.8/4.7/4.6, Sonnet 5/4.6. claude-mythos-preview is stated as deprecated with no retirement date published, so nothing to write.
  • Gemini API https://ai.google.dev/gemini-api/docs/deprecations and Vertex model-versions — every announced shutdown date already matches (gemini-3.1-flash-lite 2027-05-07, gemini-2.5-flash-image 2026-10-02, gemini-embedding-001 2028-05-14 on the Gemini key vs 2028-05-20 on the Vertex key, imagen-4.0 2026-08-17, veo 2026-06-30, robotics-er-1.6 2026-08-31). The GA Gemini 3.x models say "no shutdown date announced" and are correctly left without one.
  • OpenAI https://platform.openai.com/docs/deprecations — all shutdown rows match. Not changed: gpt-4o-realtime-preview*, gpt-4o-mini-realtime-preview, gpt-4o-audio-preview, gpt-4o-mini-audio-preview carry 2027-01-20 (the gpt-4o-realtime/gpt-4o-audio family row) while the older per-snapshot rows say 2026-05-07; the page supports both readings, so left alone rather than guessed.
  • Azure partner tables (Anthropic, DeepSeek, MoonshotAI, Fireworks, Microsoft) — all already match. azure/gpt-realtime-mini-2025-10-06 has two conflicting rows for the same version (2027-04-06 and 2026-09-21) so it was left unset.
  • Bedrock lifecycle table — all other rows already match.
  • Cohere https://docs.cohere.com/docs/models — the Sept 15 2025 deprecations and existing context windows all match. command-a-plus-05-2026, command-a-reasoning-08-2025, command-a-vision-07-2025, command-a-translate-08-2025, rerank-v4.0-pro/fast are missing under the cohere provider, but Cohere's pricing page renders its per-model API rates client-side and no official price for them could be read, so they were not added. Same for xAI's Imagine image/video and voice models and Cohere's Aya/transcribe models.

Rolling registry PR

This is the single open registry PR. The four other open PRs touching this file are code/behavior changes that only incidentally edit the JSON, so they were left alone and nothing was closed: #37375 and #35987 (Bedrock Mantle web-search transformation fix) and #36422 and #36416 (moving fallback_generalizations out of the cost map).

Relevant issues

Supports #26900 (proactive model deprecation alerts / /model/deprecations).

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests — data-only change; covered by the existing schema/deprecation suites
  • The handful of test files covering my change pass locally
  • My PR passes all required CI/CD checks
  • My PR's scope is as isolated as possible
  • I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

After (36837d3)

Registry checks

  1. python3 ci_cd/check_files_match.pyPassed! Files model_prices_and_context_window.json and litellm/model_prices_and_context_window_backup.json match.
  2. uv run --no-sync python ci_cd/generate_model_prices_schema.py → schema regenerated with no diff
  3. LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest tests/test_litellm/test_model_prices_schema.py tests/test_litellm/test_model_cost_aliases.py tests/test_litellm/proxy/test_model_deprecations_endpoint.py tests/test_litellm/proxy/common_utils/test_model_deprecation.py tests/test_litellm/integrations/SlackAlerting/test_model_deprecation_alert.py -q66 passed

Link to Devin session: https://app.devin.ai/sessions/cee732482ae349b08b2b8c336a38e369

…-flash-vision-exp

Verified against Azure Foundry model retirement schedule, Vertex AI model versions, AWS Bedrock model lifecycle and DeepSeek pricing docs.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@greptile-apps

greptile-apps Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR updates stale or missing retirement metadata for selected Azure, Vertex AI, and Bedrock models and registers DeepSeek V4 Flash Vision Experimental under bare and provider-prefixed names.

  • Corrects three Azure GPT-4.1 Nano retirement dates.
  • Adds retirement dates for Azure GPT-4o, audio models, Sora 2, and an EU Bedrock Claude model.
  • Corrects the Vertex AI text-embedding-004 retirement date.
  • Adds DeepSeek vision-model pricing, limits, capabilities, and alias metadata.
  • Keeps the primary and packaged backup registries synchronized.

Confidence Score: 5/5

The PR appears safe to merge; no concrete changed-code failure was identified in the synchronized metadata updates.

The retirement dates use valid registry fields, the new DeepSeek entries follow established bare and provider-prefixed lookup conventions, and both runtime and source registry copies contain matching changes.

Important Files Changed

Filename Overview
model_prices_and_context_window.json Updates retirement metadata and adds internally consistent DeepSeek vision-model registry entries without an identified defect.
litellm/model_prices_and_context_window_backup.json Mirrors the primary registry changes so packaged runtime metadata remains synchronized.

Reviews (1): Last reviewed commit: "fix(model_prices): correct stale retirem..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing devin/1787318013-registry-audit (36837d3) with litellm_internal_staging (ff02d5c)

Open in CodSpeed

@jasondelio

Copy link
Copy Markdown

Hi @mateo-berri , I just want to let you know this PR not complete for supporting deepseek-v4-flash-vision-exp.

Adding "supports_vision": True alone does not fix image inputs at runtime because DeepSeekChatConfig._transform_messages in litellm/llms/deepseek/chat/transformation.py unconditionally flattens content lists to strings before dispatching:

# litellm/llms/deepseek/chat/transformation.py
def _transform_messages(
    self, messages: list[AllMessageValues], model: str, is_async: bool = False
) -> list[AllMessageValues] | Coroutine[Any, Any, list[AllMessageValues]]:
    # ⚠️ This line unconditionally converts content lists to plain string,
    # stripping all `image_url` / `input_image` blocks!
    messages = handle_messages_with_content_list_to_str_conversion(messages)

Inside handle_messages_with_content_list_to_str_conversion, convert_content_list_to_str only extracts c.get("text") and silently drops any image_url objects. As a result, the outgoing request sent to DeepSeek only receives text and no images.

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Superseded by rolling registry audit #37902, which reverified and absorbed the provider-documented metadata changes at field level

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants