Skip to content

fix(model_prices): annotate retired xAI grok-2-era models and mark grok-4.20-multi-agent Responses-only - #38526

Closed
ben7am1n wants to merge 12 commits into
BerriAI:litellm_internal_stagingfrom
ben7am1n:fix/xai-retired-models
Closed

fix(model_prices): annotate retired xAI grok-2-era models and mark grok-4.20-multi-agent Responses-only#38526
ben7am1n wants to merge 12 commits into
BerriAI:litellm_internal_stagingfrom
ben7am1n:fix/xai-retired-models

Conversation

@ben7am1n

Copy link
Copy Markdown

TLDR

Problem this solves:

  • xAI retired the grok-2 generation (effective 2026-02-28) and the retired slugs now hard-fail with "Model not found — retired by xAI", but LiteLLM's model list gave no hint
  • xai/grok-4.20-multi-agent-* was listed as a chat model even though xAI documents that it does not work with the Chat Completions API

How it solves it:

  • Annotate the retired grok-2-era entries with deprecation_date so they surface via GET /v1/model/deprecations; pricing metadata is kept for historical spend data
  • Mark the two grok-4.20-multi-agent entries as Responses-API-only (mode: responses)

User Flow

Before: a stale model entry causes unexplained production failures

  1. A developer configures xai/grok-2 in their LiteLLM setup
  2. Every request fails upstream with "Model not found — retired by xAI", and GET /v1/model/deprecations lists nothing about it
  3. The admin cannot tell which of their configured models are dead upstream

After: retirement is visible before it breaks anything

  1. The same developer configures the same model on the updated gateway
  2. GET /v1/model/deprecations lists xai/grok-2 (and the other grok-2-era slugs) with its deprecation date
  3. Browsing models, xai/grok-4.20-multi-agent-0309 shows under Responses-API capabilities rather than chat

Relevant issues

Fixes #38179

Pre-Submission checklist

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Caveats

  • The May 15, 2026 retirement slugs were already annotated upstream; this PR covers the grok-2-era batch, whose official announcement (Feb 3, 2026 notice, effective 2026-02-28) matches the existing xai/grok-2-vision-1212 annotation
  • xai/grok-beta / xai/grok-vision-beta were retired earlier without a published date; they are confirmed hard-failing upstream (repro in [Bug]: xAI models can't use #38179) and are annotated with the same batch date — happy to adjust per maintainer preference
  • Provider-hosted variants (oci/xai.grok-4.20-multi-agent etc.) are untouched: the Responses-only limitation is xAI-API-specific

Screenshots / Proof of Fix

tests/test_litellm/test_xai_grok_model_metadata.py — 4 passed
- retired grok-2-era entries carry deprecation_date 2026-02-28
- grok-4.20-multi-agent entries are mode=responses
- active models (grok-4.5/4.6/4.20-0309) are NOT marked deprecated
- main and backup cost maps stay in sync

yuneng-berri and others added 11 commits August 8, 2026 13:11
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
models.litellm.ai and released litellm versions read
model_prices_and_context_window.json from main at runtime, so Kimi K3 is
missing from the hosted catalog even though the entry is in review for
litellm_internal_staging in BerriAI#37552. This copies that entry onto main so
the catalog picks it up on its next fetch.

Data only: the cost map and its backup copy, no code changes. Pricing
matches Moonshot's published rates ($3/M input, $0.30/M cache read,
$15/M output, 1,048,576-token context). The fireworks_ai and Azure
Foundry kimi-k3 variants are separate work in BerriAI#37512 and BerriAI#37658; neither
touches the native moonshot/kimi-k3 key.
…cost_map

fix: add moonshot/kimi-k3 to the cost map on main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
@ben7am1n
ben7am1n requested a review from mateo-berri as a code owner August 27, 2026 15:31
@greptile-apps

greptile-apps Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR annotates retired xAI Grok model entries with deprecation dates and marks two Grok multi-agent models as Responses-only

  • Updates the primary and backup model metadata maps consistently
  • Adds regression coverage for retirement dates, API modes, active models, and map parity
  • The metadata changes align with the existing deprecation-reporting and Responses dispatch contracts

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking request to place the regression coverage in the mapped tests and remove unused global mutation

The model metadata updates are synchronized and match supported deprecation and Responses-mode behavior; the only accepted concern is test organization and unnecessary fixture state mutation

Files Needing Attention: tests/test_litellm/test_xai_grok_model_metadata.py

Important Files Changed

Filename Overview
model_prices_and_context_window.json Adds valid ISO deprecation dates and changes two xAI multi-agent entries to the supported responses mode without an identified functional defect
litellm/model_prices_and_context_window_backup.json Mirrors the primary model metadata changes and remains synchronized for all affected entries
tests/test_litellm/test_xai_grok_model_metadata.py Adds useful metadata assertions, but creates a separate bug-fix test module and includes an unnecessary global model-cost fixture

Reviews (1): Last reviewed commit: "fix(model_prices): annotate retired xAI ..." | Re-trigger Greptile

@@ -0,0 +1,98 @@
import json

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Avoid redundant test state

This bug fix creates a separate test module whose autouse fixture replaces litellm.model_cost, although every assertion reads the JSON files directly. This bypasses the mapped-test convention and adds global-state and cache-isolation risk without contributing to the regression coverage

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing ben7am1n:fix/xai-retired-models (f0d753b) with litellm_internal_staging (8ebcb3e)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (cd63c7e) during the generation of this report, so 8ebcb3e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

…ok-4.20-multi-agent Responses-only

xAI retired the grok-2 generation effective 2026-02-28 and the retired slugs
now hard-fail with "Model not found - retired by xAI"; grok-beta and
grok-vision-beta were retired earlier without a published date and are
annotated with the same batch date.

- add deprecation_date to xai/grok-2, grok-2-1212, grok-2-latest,
  grok-2-vision, grok-2-vision-latest, grok-beta, grok-vision-beta so they
  surface via GET /v1/model/deprecations
- set mode=responses on xai/grok-4.20-multi-agent-0309 and
  xai/grok-4.20-multi-agent-beta-0309: xAI documents that the multi-agent
  model does not work with the Chat Completions API
- pricing metadata unchanged; main and backup cost maps kept in sync

Fixes BerriAI#38179
@devin-ai-integration

Copy link
Copy Markdown
Contributor

Folded into rolling PR #38693; the Feb 28 dates could not be verified on live xAI docs, other changes already landed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: xAI models can't use

3 participants