Repository navigation
feat(models): region-selectable AWS Mantle GPT-5.6 - #3369
Merged
Merged
Conversation
AWS advertises a "272K" context window for GPT-5.6 Sol/Terra/Luna on Bedrock Mantle, which the catalogue read as 272,000 tokens. The real limit is 272 * 1024 = 278,528: upstream accepts prompts up to 278,527 tokens and rejects 278,528 or more with a validation_error naming the exact ceiling. Verified live against bedrock-mantle for all three models, plus a gateway request at 275,494 prompt tokens — inside the band the old value wrongly excluded from routing. Bedrock in-region inference is also priced at OpenAI's data-residency tier, a flat 10% premium over the standard first-party rates, so the mappings were undercharging by 10% on every token type. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Contributor
|
Warning Review limit reached
Next review available in: 28 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
WalkthroughAWS Mantle now supports three US regions with regional endpoint routing and shared credentials. GPT-5.6 Sol, Terra, and Luna entries use updated pricing, regional metadata, and 278,528-token context limits. API, UI, endpoint, and pricing tests cover the changes. ChangesAWS Mantle regional configuration and routing
Estimated code review effort: 3 (Moderate) | ~25 minutes Possibly related PRs
Sequence Diagram(s)sequenceDiagram
participant Client
participant ProviderKeyDialog
participant ProviderKeyAPI
participant EndpointResolver
participant AWSMantle
Client->>ProviderKeyDialog: select region or Any region
ProviderKeyDialog->>ProviderKeyAPI: submit provider-key options
ProviderKeyAPI->>AWSMantle: validate and store region
Client->>EndpointResolver: send AWS Mantle request
EndpointResolver->>AWSMantle: resolve configured or fallback region
AWSMantle-->>EndpointResolver: return regional endpoint
EndpointResolver->>AWSMantle: send request to endpoint
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Updates the GPT‑5.6 model catalogue to accurately represent AWS Bedrock Mantle’s effective context window and token pricing, and adds a regression test to prevent future drift between first‑party OpenAI mappings and Mantle’s premium tier.
Changes:
- Adjust
aws-mantleGPT‑5.6 Sol/Terra/LunacontextSizeto272 * 1024(278,528) and update all token prices to the 10% data‑residency premium. - Add a gateway unit spec asserting the 1.1× premium vs OpenAI, the cache-write ratio invariants,
contextSize === 272 * 1024, and nopricingTiersfor Mantle.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
packages/models/src/models/openai.ts |
Updates Bedrock Mantle GPT‑5.6 mappings for corrected context size and +10% pricing premium. |
apps/gateway/src/lib/openai-gpt56-pricing.spec.ts |
Adds test coverage to lock in Mantle-vs-OpenAI pricing ratios and Mantle context sizing expectations. |
Suppressed comments (2)
packages/models/src/models/openai.ts:2107
- Same as above: the comment implies the over-272K tier can’t apply due to the cap, but the cap is 278,528 > 272,000. Clarify that Bedrock pricing doesn’t have the long-context tier instead of tying it to the cap.
// AWS caps the Mantle deployment at a 272K context (vs 1.05M
// first-party), so OpenAI's over-272K pricing tier never applies.
// AWS's "272K" is 272 * 1024: upstream rejects prompts of 278528
// tokens or more.
packages/models/src/models/openai.ts:2256
- Same clarification needed here: with contextSize set to 278,528, the comment shouldn’t claim the over-272K tier can’t apply because of the cap. It’s better to state that Bedrock doesn’t offer the long-context tier for Mantle pricing.
// AWS caps the Mantle deployment at a 272K context (vs 1.05M
// first-party), so OpenAI's over-272K pricing tier never applies.
// AWS's "272K" is 272 * 1024: upstream rejects prompts of 278528
// tokens or more.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1955
to
+1958
| // AWS caps the Mantle deployment at a 272K context (vs 1.05M | ||
| // first-party), so OpenAI's over-272K pricing tier never applies. | ||
| contextSize: 272000, | ||
| // AWS's "272K" is 272 * 1024: upstream rejects prompts of 278528 | ||
| // tokens or more. |
Comment on lines
+122
to
+125
| // Bedrock Mantle in-region inference is priced at OpenAI's data-residency | ||
| // tier, a flat 10% premium over the first-party rates, and AWS caps the | ||
| // deployment at a 272 * 1024 token context (upstream rejects prompts of | ||
| // 278528 tokens or more), so the over-272K tier can never apply. |
AWS lists global cross-region pricing for Mantle as "coming soon" and the model cards mark Geo/Global as unsupported, so the usual Bedrock geo-premium/global-parity split does not apply — in-region is the only rate that exists, and AWS prices it at OpenAI's data-residency tier. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mantle serves each model from concrete AWS regions with no cross-region inference profiles, so a single env-var-pinned region hid two thirds of the deployment. Give aws-mantle a regionConfig and per-model region lists instead, following the Alibaba pattern: every entry is a real region, there is no synthetic "global", and pinDefaultRegion stays unset so the gateway routes across regions rather than to a fixed default. Regions come from the model cards and were confirmed against the live endpoint: Sol is us-east-1/us-east-2 only (us-west-2 404s "the model does not exist"), Terra and Luna add us-west-2. Pricing is identical in every region, so the entries carry no per-region overrides. Bedrock long-term API keys are IAM-global, so sharedCredentialAcrossRegions lets one key reach every region without per-region env keys, and the provider-key dialog now says so instead of claiming keys are regional. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
steebchen
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/gateway/src/lib/openai-gpt56-pricing.spec.ts`:
- Around line 245-261: Update the test “expands each region without altering
pricing” to assert that the expanded root entry has no region, and that all
subsequent entries’ region values exactly match EXPECTED_REGIONS[modelId] in
order. Keep the existing length and pricing assertions unchanged.
In `@packages/actions/src/get-provider-endpoint.ts`:
- Around line 390-392: Update the Mantle base URL resolution in
get-provider-endpoint to preserve the current variant by using envValueOrDefault
or passing undefined and variant to getProviderEnvValue. Add a regression test
covering a variant-specific Mantle base URL and verifying the resolved endpoint
uses that value.
- Around line 393-398: Update the AWS Mantle branch in getProviderEndpoint to
use the incoming validationRegion/region value as the Mantle region before
falling back to environment defaults, so provider-key options take precedence.
Add a BYOK/spec case that starts with provider-key options and verifies the
resulting Mantle endpoint uses the configured region.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: e2bf5ba9-bff3-4d20-8c60-ef7dfa4d6ed1
📒 Files selected for processing (8)
apps/api/src/routes/keys-provider.tsapps/gateway/src/lib/openai-gpt56-pricing.spec.tsapps/ui/src/components/provider-keys/create-provider-key-dialog.tsxpackages/actions/src/get-provider-endpoint.spec.tspackages/actions/src/get-provider-endpoint.tspackages/db/src/schema.tspackages/models/src/models/openai.tspackages/models/src/providers.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/models/src/models/openai.ts
Storing a region on a provider key locks routing to it, which is right for data residency but wrong as a default: the dialog pre-filled the provider's defaultRegion, so a BYOK user who never opened the dropdown was silently pinned to one region. For providers whose credential works everywhere (aws-mantle, aws-bedrock) the dropdown now defaults to "Any region" and stores nothing, leaving all regions as routing candidates. Region-scoped providers like Alibaba keep defaulting, since their keys really do belong to one region. Picking a region explicitly still locks it. Also addresses PR review: pass `variant` through Mantle's baseUrl lookup so variant-specific overrides are honoured, and reword the pricing-tier comments, which claimed the over-272K tier was unreachable when the real reason is that AWS never exposes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The endpoint spec pinned region precedence by passing the region directly, which skipped the chain the BYOK path actually uses: the region stored on a provider key resolves through regionConfig.optionsKey into the request. These drive validateProviderKey from provider-key options and assert the resulting Mantle URL, plus that a key pinned to us-west-2 does not validate against Sol, which AWS does not deploy there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
steebchen
enabled auto-merge
August 3, 2026 16:25
steebchen
added this pull request to the merge queue
Aug 3, 2026
This was referenced Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Corrects the
aws-mantleGPT-5.6 catalogue entries on three fronts — context size, pricing, and region support.1. Context size
AWS's model cards advertise a "272K token" context window, which the catalogue took literally as
272000. AWS means 272 × 1024. Probingbedrock-mantledirectly:Identical for all three models, and it applies to prompt tokens only — a 278,527-token prompt returning 5 output tokens (278,532 total) succeeds. We were understating the usable window by 6,528 tokens, and the router's
contextSize < requiredContextSizefilter was excluding Mantle from requests it can serve.2. Pricing
Bedrock in-region inference is billed at OpenAI's data-residency tier — a flat 10% premium over the standard first-party rates the mappings were copying. Every token type was undercharged by 10%.
USD per 1M tokens, already incorporating OpenAI's 2026-07-30 Terra/Luna cut.
The usual Bedrock rule — geo profiles at a ~10% premium, global at roughly base — is correct but doesn't reach Mantle, which has no global option: the model cards mark Geo and Global as Not supported, and the pricing page says "In-region inference is priced at parity with OpenAI data residency tier. Global cross-region inference pricing coming soon." OpenAI's data-residency tier is itself the +10% tier (a 10% uplift for models released on or after 2026-03-05, which covers GPT-5.6). So "parity with the data residency tier" resolves to exactly 1.1× standard.
Values were read out of the raw pricing-page HTML rather than a rendered summary (columns are
input / 30m cache write / cache read / output):Two models nothing here touches corroborate the ratio independently: our first-party GPT-5.5 is 5.00/0.50/30.00 and AWS lists 5.50/0.55/33.00; GPT-5.4 is 2.50/0.25/15.00 → 2.75/0.275/16.50. Both exactly 1.1×. Sol's
$6.88is a rounding of6.875— Luna's$0.275is printed to three decimals and is exact. No long-context tier exists on Bedrock, and the 278,528 cap means OpenAI's over-272K tier could never apply, sopricingTiersstays unset.3. Region support
aws-mantlepreviously had a single region pinned byLLM_AWS_MANTLE_REGION(defaultus-east-1), with no way for a user to choose — noregionConfig, noregions, no provider-key option. That hid two thirds of the deployment.It now follows the Alibaba pattern: every entry is a concrete AWS region, there is no synthetic
global, andpinDefaultRegionstays unset so the gateway routes across regions instead of pinning to one.Taken from the model cards and confirmed against the live endpoint across seven regions — there is no EU or APAC presence at all. Pricing is identical in every region, so the region entries carry no overrides.
Bedrock long-term API keys are IAM-global (one ABSK key authenticated against all three regional endpoints), so
sharedCredentialAcrossRegions: trueavoids requiring per-regionLLM_AWS_MANTLE_API_KEY__<REGION>env keys. The provider-key dialog's hardcoded "API keys are region-specific" line is now conditional on that flag, since it's wrong for Mantle.Also adds
aws_mantle_regiontoProviderKeyOptionsand both provider-key zod schemas, and routes the endpoint throughregionConfig.endpointMap(the env var stays as a deployment-level override).Verification
chat-basice2e — including:us-west-2for Terra/Luna and correctly no:us-west-2for Sol.openai-gpt56-pricing.spec.ts,get-provider-endpoint.spec.tsandproviders.spec.ts. New coverage asserts the 1.1× premium against the first-party mapping, the surviving 1.25× cache-write ratio,contextSize === 272 * 1024, absence ofpricingTiers, the exact per-model region lists, that no cross-region profile leaks into the config, that every declared region has an endpoint, and that expansion preserves pricing.pnpm formatandpnpm buildpass.Note on e2e flakiness
Running the full suite against all 11 Mantle mappings at the default 16-way concurrency produced intermittent 60s timeouts — but on a different mapping each run (
luna:us-east-2once,sol:us-east-1the next, whilesol:us-east-2answered in 1.2s). Individually each mapping responds in about a second. That points at account-level Mantle throughput rather than any one deployment, so nothing is markedtest: "skip". Worth knowing that this PR triples the number of Mantle mappings the e2e job exercises.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
us-east-1,us-east-2, andus-west-2.Updates