Add new vercel ai anthropic models - #20745
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile OverviewGreptile SummaryThis PR extends the model registry/pricing metadata with a set of new The change plugs into LiteLLM’s existing model-lookup mechanism: downstream routing and feature enablement (vision/tools/schema/caching) depend on these JSON entries being keyed by the exact model identifiers and having accurate capability flags. Confidence Score: 3/5
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds multiple Vercel AI Gateway Anthropic model entries with pricing/context. Notable inconsistency: claude-sonnet-4.5 lacks supports_response_schema, and new model key naming differs from existing claude-4-* Vercel Anthropic entries (verify IDs/aliases). |
| litellm/model_prices_and_context_window_backup.json | Mirrors the same new Vercel AI Gateway Anthropic model entries as the primary pricing file; inherits the same concerns about missing supports_response_schema for claude-sonnet-4.5 and potential model-id naming mismatches vs existing Vercel Anthropic keys. |
Sequence Diagram
sequenceDiagram
participant App as LiteLLM runtime
participant Prices as model_prices_and_context_window.json
participant Provider as Vercel AI Gateway
App->>Prices: Lookup config by model key
Prices-->>App: Return pricing + feature flags
App->>App: Enable/disable features based on flags
App->>Provider: Send request using selected model ID
Provider-->>App: Response
App-->>App: Apply schema/vision/tools handling
Note over Prices,App: If model key is wrong/unrecognized,
Note over Prices,App: lookup/routing may fail or use fallback.
Note over Prices,App: If supports_response_schema missing,
Note over Prices,App: schema features may be disabled for that model.
| "vercel_ai_gateway/anthropic/claude-sonnet-4.5": { | ||
| "cache_creation_input_token_cost": 3.75e-06, | ||
| "cache_read_input_token_cost": 3e-07, | ||
| "input_cost_per_token": 3e-06, |
There was a problem hiding this comment.
Missing response schema flag
vercel_ai_gateway/anthropic/claude-sonnet-4.5 omits "supports_response_schema": true even though the other newly-added vercel/anthropic models include it (and the PR description suggests these are Vercel AI Gateway Anthropic models with schema support). If LiteLLM uses this flag to gate JSON-schema / response-format features, this entry will behave inconsistently vs the rest. Add supports_response_schema here (or remove it from the others if it’s not actually supported).
| "vercel_ai_gateway/anthropic/claude-3-7-sonnet": { | ||
| "cache_creation_input_token_cost": 3.75e-06, | ||
| "cache_read_input_token_cost": 3e-07, | ||
| "input_cost_per_token": 3e-06, | ||
| "litellm_provider": "vercel_ai_gateway", |
There was a problem hiding this comment.
Possible bad model identifiers
These new keys use both claude-3-5-* / claude-3-7-* and claude-sonnet-4* naming, but the existing adjacent Vercel Anthropic entries use claude-4-opus / claude-4-sonnet (see vercel_ai_gateway/anthropic/claude-4-opus and .../claude-4-sonnet). If the runtime expects exact model IDs that match Vercel’s gateway (or has a normalization layer), these entries may be unreachable / silently misrouted. Please confirm these exact strings are valid model IDs in the gateway and align naming with the existing Vercel Anthropic conventions (or add aliases/normalization if required).
Also appears in: litellm/model_prices_and_context_window_backup.json:28606.
Add new vercel ai anthropic models
Relevant issues
https://vercel.com/ai-gateway/models/claude-sonnet-4.5
Fixes #20743
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
Changes