feat: add bedrock grok 4.6 to model cost map - #37517
Conversation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
|
Greptile SummaryAdds synchronized Bedrock Grok 4.6 model metadata so LiteLLM can route supported identifiers and calculate spend.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains.
|
| Filename | Overview |
|---|---|
| model_prices_and_context_window.json | Adds the canonical pricing, routing, context-window, and capability metadata for three Bedrock Grok 4.6 identifiers. |
| litellm/model_prices_and_context_window_backup.json | Mirrors the canonical Grok 4.6 model metadata exactly in the packaged backup cost map. |
Reviews (2): Last reviewed commit: "fix: drop source url from grok 4.6 cost ..." | Re-trigger Greptile
| "supports_vision": true, | ||
| "source": "https://aws.amazon.com/bedrock/pricing/" | ||
| }, | ||
| "bedrock_mantle/xai.grok-4.6": { |
There was a problem hiding this comment.
Grok metadata lacks regression coverage
The three new mappings lack a focused test for their lookup keys, pricing, token limits, and capability differences, so valid JSON changes can silently regress routing or spend calculation
Context Used: CLAUDE.md (source)
Knowledge Base Used: Cost Tracking and Budget Enforcement
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
@greptile we do not need to do this when adding models
There was a problem hiding this comment.
Fair point — cost map additions are covered by the existing CI checks (check_files_match.py, generate_model_prices_schema.py), so dedicated per-model tests would be redundant overhead.
Shall I remember this as a rule: "Do not require dedicated tests for new model entries in model_prices_and_context_window.json; existing CI checks (check_files_match.py, generate_model_prices_schema.py) are sufficient coverage"?
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 1f6bef7. Configure here.
TLDR
Problem this solves:
bedrock/global.xai.grok-4.6slug hits Invoke, which the model rejectsbedrock/converse/workaround answers but logs $0 spendHow it solves it:
bedrock_mantle/xai.grok-4.6,us.xai.grok-4.6,global.xai.grok-4.6to the cost mapbedrock_converseprovider routes the bare CRIS slugs to ConverseUser Flow
Before: a developer calling Grok 4.6 on Bedrock through the proxy gets a 404 on the natural slug, and the workaround slug logs no spend
"model": "bedrock/global.xai.grok-4.6"and a short user messageBedrockException - Bedrock Invoke HTTPX: Unknown provider=None, model=global.xai.grok-4.6. Try calling via converse route - bedrock/converse/<model>"model": "bedrock/converse/global.xai.grok-4.6"and get 200 with the model's replyAfter: the same request on the natural slug succeeds and is priced
"model": "bedrock/global.xai.grok-4.6"and a short user messagebedrock/us.xai.grok-4.6prices at $2.20/$6.60Relevant issues
Linear ticket
Resolves LIT-5877
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Pricing source: AWS Bedrock Grok 4.6 model card (pricing table, model IDs, 500K context, capabilities), cross-checked against the AWS launch announcement. Evidence screenshots of the AWS pricing table are posted in the requesting Slack thread. Per reviewer request the entries carry no
sourcefieldBoth legs boot a real proxy from the named commit (
LITELLM_LOCAL_MODEL_COST_MAP=True, Postgres,AWS_BEARER_TOKEN_BEDROCK,aws_region_name: us-east-1, mantle in us-west-2) and hit real Bedrock; spend is read back from/spend/logs.$PORTis that leg's proxy port.Before (3d51eb3)
bare
bedrock/global.xai.grok-4.6on chat completions:bare slug on
/v1/responses:bare slug on
/v1/messages:bare
bedrock/us.xai.grok-4.6:bare slug with
tools:bedrock_mantle/xai.grok-4.6:interim
bedrock/converse/global.xai.grok-4.6works but is unpriced:After (1f6bef7)
bare
bedrock/global.xai.grok-4.6on chat completions:bare slug on
/v1/responses:bare slug on
/v1/messages:bare
bedrock/us.xai.grok-4.6:bare slug with
tools:bare slug with an inline image:
bedrock_mantle/xai.grok-4.6:interim
bedrock/converse/global.xai.grok-4.6now priced:/model/infoat the tip:Type
🆕 New Feature
Caveats (if any)
bedrock-runtimeIDs since the model card lists them unsupported therebedrock/xai.grok-4.6still routes to Invoke; AWS offers no in-Region bedrock-runtime ID for itcache_controlcheckpoints on the Converse path get a Bedrock "unsupported model or your request did not allow prompt caching" error for this model, identically before and after this PR;supports_prompt_cachingfollows the model card and the cache-read rate applies to whatever cache hits Bedrock reportsFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/46e38dde986b431593aa66f50f29b7dc
Note
Cursor Bugbot is generating a summary for commit 1f6bef7. Configure here.