Repository navigation
chore(prices): sync AWS Bedrock prices: 6 models [enrichment failed: AWS Bedrock, 4 held] - #42338
Conversation
…AWS Bedrock, 4 held] minimax.minimax-m2: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_audio_input, supports_response_schema, supports_function_calling minimax.minimax-m2.1: max_tokens, supports_vision, max_output_tokens, supports_audio_input, supports_response_schema minimax.minimax-m2.5: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_audio_input, supports_response_schema moonshot.kimi-k2-thinking: max_tokens, supports_vision, max_input_tokens, max_output_tokens, supports_audio_input, supports_response_schema, supports_function_calling openai.gpt-oss-safeguard-120b: max_tokens, supports_vision, max_output_tokens, supports_audio_input, supports_response_schema, supports_function_calling openai.gpt-oss-safeguard-20b: max_tokens, supports_vision, max_output_tokens, supports_audio_input, supports_response_schema, supports_function_calling
|
|
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
* fix: repair seven regressions caught by CircleCI on main - vertex_ai: stop treating fine-tuned endpoint ids (numeric or vertex_ai/gemini/<id>) and gemma models as Gemini 3+, which injected temperature=1.0 and Gemini 3 thinking config into their requests (#42465) - cost: price Azure DALL-E 3 from its azure/<quality>/<size>/dall-e-3 rows; it only worked through the OpenAI rows that #42435 removed - bedrock: stream bedrock/invoke/moonshot through an OpenAI-shaped chunk decoder; the generic decoder dropped every chunk, which the supports_response_schema flag from #42338 un-skipped in CI - proxy: keep the public model_group on pre-routing rejections so the Usage page groups them under the model name, not the deployment (#41077) - cost map: mirror the base rows' capability flags onto Bedrock regional and cross-region copies (#42254 and later syncs) - whitelist the new regional Bedrock rows from #42543 and #42588 for the converse routing check, following the existing regional-row convention * fix(model-prices): mirror capability flags onto ap-southeast-3 bedrock rows * refactor(bedrock): tighten types on the moonshot stream decoder and its tests
TLDR
Problem this solves:
How it solves it:
User Flow
A before and after user flow does not apply to an automated price sync; the only change is updated metadata in the model catalog file
Relevant issues
Affected release
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Screenshots and live command output do not apply to an automated price sync; each applied value was read from the AWS Bedrock pricing page and the six models were verified through LiteLLM on /v1/chat/completions before being written
Type
Chore
Caveats (if any)
Medium
minimax.minimax-m2.5max_input_tokens drops from 1T to 196BLow
QA runbook
This PR does not add or change tests, so there is no manual runbook
Final Attestation
Sync report
Updates 6 models, each read from its provider's own published prices.
AWS Bedrock
Updates 6 models from the provider's pricing page (page
c21fff8ee033).Verified through LiteLLM before being added: 6 via /v1/chat/completions.
Changes
minimax.minimax-m2: max_input_tokens $128000000000/1M → $1000000000000/1M, max_output_tokens $8192000000/1M → $8000000000/1M, max_tokens $8192000000/1M → $8000000000/1M, supports_audio_input false, supports_function_calling true, supports_response_schema true, supports_vision false, source setminimax.minimax-m2.1: max_output_tokens $8192000000/1M → $8000000000/1M, max_tokens $8192000000/1M → $8000000000/1M, supports_audio_input false, supports_response_schema true, supports_vision falseminimax.minimax-m2.5: max_input_tokens $1000000000000/1M → $196000000000/1M, max_output_tokens $8192000000/1M → $8000000000/1M, max_tokens $8192000000/1M → $8000000000/1M, supports_audio_input false, supports_response_schema true, supports_vision falsemoonshot.kimi-k2-thinking: max_input_tokens $128000000000/1M → $256000000000/1M, max_output_tokens $8192000000/1M → $16000000000/1M, max_tokens $8192000000/1M → $16000000000/1M, supports_audio_input false, supports_function_calling true, supports_response_schema true, supports_vision false, source setopenai.gpt-oss-safeguard-120b: max_output_tokens $8192000000/1M → $16000000000/1M, max_tokens $8192000000/1M → $16000000000/1M, supports_audio_input false, supports_function_calling true, supports_response_schema true, supports_vision false, source setopenai.gpt-oss-safeguard-20b: max_output_tokens $8192000000/1M → $16000000000/1M, max_tokens $8192000000/1M → $16000000000/1M, supports_audio_input false, supports_function_calling true, supports_response_schema true, supports_vision false, source setHeld for review (4) — not in this diff
These values were read from the source but did not pass the gates on their own. A human needs to check them against the page and apply them by hand if right.
moonshotai.kimi-k3input_cost_per_token: $3/1M — smoke-failed: LiteLLM returned HTTP 400 (invalid_request_error): "litellm.BadRequestError: BedrockException - {"message":"Invocation of model ID moonshotai.kimi-k3 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model."}No fallback model group found for lookup_groups=bedrock/moonshotai.kim…" (https://aws.amazon.com/bedrock/pricing/ row "Global CRIS" input_cost_per_token = $3.00 per 1M tokens)moonshotai.kimi-k3output_cost_per_token: $15/1M — smoke-failed: LiteLLM returned HTTP 400 (invalid_request_error): "litellm.BadRequestError: BedrockException - {"message":"Invocation of model ID moonshotai.kimi-k3 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model."}No fallback model group found for lookup_groups=bedrock/moonshotai.kim…" (https://aws.amazon.com/bedrock/pricing/ row "Global CRIS" output_cost_per_token = $15.00 per 1M tokens)moonshotai.kimi-k3cache_read_input_token_cost: $0.3/1M — smoke-failed: LiteLLM returned HTTP 400 (invalid_request_error): "litellm.BadRequestError: BedrockException - {"message":"Invocation of model ID moonshotai.kimi-k3 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model."}No fallback model group found for lookup_groups=bedrock/moonshotai.kim…" (https://aws.amazon.com/bedrock/pricing/ row "Global CRIS" cache_read_input_token_cost = $0.30 per 1M tokens)moonshotai.kimi-k3cache_creation_input_token_cost: $3.75/1M — smoke-failed: LiteLLM returned HTTP 400 (invalid_request_error): "litellm.BadRequestError: BedrockException - {"message":"Invocation of model ID moonshotai.kimi-k3 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model."}No fallback model group found for lookup_groups=bedrock/moonshotai.kim…" (https://aws.amazon.com/bedrock/pricing/ row "Global CRIS" cache_creation_input_token_cost = $3.75 per 1M tokens)No litellm key or field (28)
Rows the page prices that litellm has no key or field for. Nothing to apply; listed so the omission is deliberate, not missed.
Claude: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude (100K): not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude 3 Haiku: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude 3 Opus: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude 3 Sonnet: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude 3.5 Haiku: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude 3.5 Sonnet: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude 3.5 Sonnet v2: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude 3.7 Sonnet: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Haiku 4.5: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Instant: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Instant (100K): not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Mythos 5: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Mythos 5.1: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Mythos Preview: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Opus 4: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Opus 4.1: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Opus 4.5: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Opus 4.6: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Sonnet 4: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameClaude Sonnet 4.5: not in the synced set: the price list states no Bedrock model id, and this model's id is not its display nameGemma 4 26B A4B: not in the synced set: the page states no Bedrock model idGemma 4 31B: not in the synced set: the page states no Bedrock model idGemma 4 E2B: not in the synced set: the page states no Bedrock model idLlama 2 Chat (13B): not in the synced set: the page states no Bedrock model idLlama 2 Chat (70B): not in the synced set: the page states no Bedrock model idVoxtral Mini 1.0: not in the synced set: the page states no Bedrock model idVoxtral Small 1.0: not in the synced set: the page states no Bedrock model idOpened by the litellm-providers price sync. Every price is read from the provider's own published source and gated before it is applied; the audit trail for each value is in the portal's sync history.
Note
Medium Risk
Catalog context and capability flags drive routing and validation; the large
minimax.minimax-m2.5input limit reduction could change which requests LiteLLM accepts for that model.Overview
Syncs six AWS Bedrock
bedrock_conversecatalog entries inmodel_prices_and_context_window.jsonand its backup from the AWS Bedrock pricing page, without changing per-token prices in this diff.Token limits are updated:
minimax.minimax-m2gains a much largermax_input_tokens(128k → 1M) with output capped at 8k;minimax.minimax-m2.5correctsmax_input_tokensdown (1M → 196k) and aligns output to 8k;moonshot.kimi-k2-thinkingdoubles input context (128k → 256k) and raises max output (8k → 16k); bothopenai.gpt-oss-safeguard-*models get 16k max output instead of 8k.Capability metadata is filled in across these models (
supports_function_calling,supports_response_schema,supports_audio_input/supports_vision, andsourceURLs where missing).moonshotai.kimi-k3pricing is intentionally not included (held for manual review per the sync report).Reviewed by Cursor Bugbot for commit b305928. Bugbot is set up for automated code reviews on this repo. Configure here.