Skip to content

fix(model_prices): refresh deprecation_date metadata from OpenAI/Anthropic/Gemini docs - #36188

Closed
devin-ai-integration[bot] wants to merge 2 commits into
litellm_internal_stagingfrom
devin/1786108180-model-deprecation-metadata
Closed

fix(model_prices): refresh deprecation_date metadata from OpenAI/Anthropic/Gemini docs#36188
devin-ai-integration[bot] wants to merge 2 commits into
litellm_internal_stagingfrom
devin/1786108180-model-deprecation-metadata

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

How it solves it:

  • Backfills dates from OpenAI/Anthropic/Gemini deprecation pages
  • Corrects 5 stale dates against those same pages
  • No pricing/capability edits; deprecation_date only

Relevant issues

Supports #26900 (/model/deprecations endpoint + Slack deprecation alerts), which reads deprecation_date from this file.

Linear ticket

Pre-Submission checklist

  • I have added meaningful tests — data-only change; covered by the existing schema/pricing tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Data-only change to model_prices_and_context_window.json (no code paths touched), verified by the repo's own checks at commit on this branch:

LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest tests/test_litellm/test_model_prices_schema.py -q
  18 passed

LITELLM_LOCAL_MODEL_COST_MAP=True uv run --no-sync pytest tests/test_litellm/test_cost_calculator.py tests/test_litellm/litellm_core_utils -q
  1571 passed, 1 skipped

Type

🐛 Bug Fix

Changes

Only the deprecation_date key is touched — 86 entries (81 added, 5 corrected). Every value comes from the provider's own deprecation page; nothing was inferred.

Corrections (existing values that disagree with the provider)

model was now source
claude-3-opus-20240229 2026-05-01 2026-01-05 Anthropic
claude-opus-4-20250514 2026-05-14 2026-06-15 Anthropic
claude-sonnet-4-20250514 2026-05-14 2026-06-15 Anthropic
gpt-4-0613 2025-06-06 2026-10-23 OpenAI
gpt-4-1106-preview 2026-03-26 2026-10-23 OpenAI

Backfills

Anthropic (docs.claude.com) — retired models that had no date:

  • 2026-04-20: claude-3-haiku-20240307
  • 2026-06-15: claude-4-opus-20250514, claude-4-sonnet-20250514 (aliases of the retired 2025-05-14 snapshots)

OpenAI (platform.openai.com/docs/deprecations), grouped by the shutdown dates in the "Upcoming deprecations" tables:

  • 2027-01-20 — legacy audio/realtime/transcription families: gpt-realtime*, gpt-audio*, gpt-4o-audio-preview*, gpt-4o-realtime-preview*, gpt-4o-mini-audio-preview*, gpt-4o-mini-realtime-preview*, gpt-4o-mini-transcribe-2025-03-20 (family-level entries in the doc are applied to that family's snapshots, excluding separately-named successors such as gpt-audio-1.5 / gpt-realtime-2.1)
  • 2026-12-11gpt-5-2025-08-07, gpt-5-mini-2025-08-07, gpt-5-nano-2025-08-07, gpt-5-pro-2025-10-06, o3-2025-04-16, o3-pro-2025-06-10
  • 2026-12-01gpt-image-1-mini, gpt-image-1.5, chatgpt-image-latest
  • 2026-10-23gpt-3.5-turbo(-0125), gpt-4, gpt-4-turbo(-2024-04-09), gpt-4.1-nano(-2025-04-14), gpt-4o-2024-05-13, gpt-image-1, o1(-2024-12-17), o1-pro(-2025-03-19), o3-mini(-2025-01-31), o4-mini(-2025-04-16)
  • 2026-08-10gpt-5.2-chat-latest, gpt-5.3-chat-latest
  • 2026-07-23gpt-5-chat-latest, gpt-5-codex, gpt-5.1-chat-latest, gpt-5.1-codex{,-max,-mini}, gpt-5.2-codex, gpt-4o{,-mini}-search-preview-2025-03-11, gpt-4o-mini-tts-2025-03-20, gpt-audio-mini-2025-10-06, gpt-realtime-mini-2025-10-06, o3-deep-research{,-2025-06-26}, o4-mini-deep-research{,-2025-06-26}

Gemini (ai.google.dev/gemini-api/docs/deprecations) — applied to the gemini/ (Gemini API) entries only, since Vertex publishes its own schedule:

  • 2027-05-07 gemini-3.1-flash-lite; 2026-10-16 gemini-2.5-pro, gemini-2.5-flash, gemini-2.5-flash-lite; 2026-10-02 gemini-2.5-flash-image; 2026-08-17 imagen-4.0-{,ultra-,fast-}generate-001; 2026-07-14 gemini-embedding-001; 2026-06-30 veo-2.0-generate-001; 2026-06-25 gemini-3.1-flash-image-preview, gemini-3-pro-image-preview; 2026-05-25 gemini-3.1-flash-lite-preview; 2026-04-30 gemini-robotics-er-1.5-preview; 2026-03-31 gemini-2.5-flash-lite-preview-09-2025

Deliberately not changed

  • Azure, Bedrock and Vertex mirrors of these models — those platforms publish separate retirement schedules, and the existing Azure/Bedrock dates in the file already reflect them.
  • Active models whose only published date is Anthropic's "not sooner than" floor (e.g. claude-opus-4-5, claude-sonnet-4-6) — recording a floor as a deprecation date would produce false alerts in feat(proxy): proactive model deprecation alerts and /model/deprecations endpoint #26900.
  • gpt-4-0314, gpt-4-0125-preview, gpt-3.5-turbo-1106: their entries carry dates I could not confirm on the current OpenAI page (the historical "already shut down" section was not retrievable), so they were left untouched rather than guessed at.

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/6d98ff2f5b1d41da9b47eaa1e333103f

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@CLAassistant

CLAassistant commented Aug 7, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@devin-ai-integration
devin-ai-integration Bot changed the base branch from litellm_internal_staging to main August 7, 2026 13:11
@greptile-apps

greptile-apps Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR refreshes provider deprecation metadata and also changes E2E model-creation polling to wait across data-plane synchronization. The new poller can falsely time out after a transient worker miss.

  • Adds or corrects deprecation dates for OpenAI, Anthropic, and Gemini model entries
  • Adds per-request GET timeout support to the E2E transports
  • Introduces continuous model-list polling after model creation

Confidence Score: 4/5

The model metadata appears sound, but the E2E polling regression should be fixed before merging because transient worker visibility can falsely fail model creation

A late miss clears the continuous-listing state and restores an already-expired initial deadline, so split-deployment E2E setup can fail while propagation is still converging

Files Needing Attention: tests/e2e/proxy_client.py

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds and corrects provider-scoped deprecation metadata; no concrete incorrect entry was established from repository evidence
tests/e2e/proxy_client.py Adds multi-worker servability polling, but resetting visibility after the original deadline can cause false setup failures and the implementation violates repository conventions
tests/e2e/transport.py Adds an optional per-call GET timeout consistently across the protocol and both known implementations

Comments Outside Diff (2)

  1. tests/e2e/proxy_client.py, line 154-155 (link)

    P1 Reset reuses expired deadline

    When a cold worker misses after the initial deadline, resetting first_seen_at reuses that expired deadline, causing create_model to fail during convergence

  2. tests/e2e/proxy_client.py, line 78-86 (link)

    P2 Polling violates source conventions

    The added comments and repeated local reassignments violate the repository's source conventions, making this timing-sensitive state harder to maintain safely

    Context Used: CLAUDE.md (source)

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (1): Last reviewed commit: "fix(model_prices): refresh deprecation_d..." | Re-trigger Greptile

@codecov

codecov Bot commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…ropic/Gemini docs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the devin/1786108180-model-deprecation-metadata branch from 761a134 to 4e10afb Compare August 7, 2026 13:48
@devin-ai-integration
devin-ai-integration Bot changed the base branch from main to litellm_internal_staging August 7, 2026 13:48
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@codspeed-hq

codspeed-hq Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing devin/1786108180-model-deprecation-metadata (b9a3b41) with litellm_internal_staging (e1717c5)

Open in CodSpeed

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Superseded by #37658, which carries this PR's registry data (field-level, on top of current litellm_internal_staging) together with the other open cost-map PRs, so maintainers only have one registry PR to review. See the per-source-PR table in #37658 for exactly what was carried in, and reopen if anything looks dropped.

Srivatsa03 pushed a commit to Srivatsa03/litellm that referenced this pull request Aug 21, 2026
…ngeset

Combines the model-cost-map data from BerriAI#35911, BerriAI#36017, BerriAI#36080, BerriAI#36113, BerriAI#36188, BerriAI#36444, BerriAI#37029, BerriAI#37252 and BerriAI#37632 onto current litellm_internal_staging, merged per entry field so older branches no longer revert fields the base has gained since they were opened. Drops the Gemini deprecation dates from BerriAI#36188 and the text-embedding-004 date from BerriAI#36080 that the official docs contradict.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant