Skip to content

feat: add AWS Bedrock service tier support #9955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mrubens merged 7 commits into from
Dec 14, 2025
Merged

feat: add AWS Bedrock service tier support #9955

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
7f3481b
feat: add AWS Bedrock service tier support
roomote Dec 9, 2025
44190d1
fix: add amazon.nova-2-lite-v1:0 to service tier models and add test …
roomote Dec 9, 2025
a0b9fd7
fix: move service_tier to top-level request parameter per AWS docs
roomote Dec 9, 2025
4936295
Add translations
mrubens Dec 14, 2025
990e58c
Update webview-ui/src/i18n/locales/ca/settings.json
mrubens Dec 14, 2025
fb13fc3
Update webview-ui/src/i18n/locales/ca/settings.json
mrubens Dec 14, 2025
eb5a470
Update webview-ui/src/i18n/locales/es/settings.json
mrubens Dec 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions packages/types/src/provider-settings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,7 @@ const bedrockSchema = apiModelIdProviderModelSchema.extend({
awsBedrockEndpointEnabled: z.boolean().optional(),
awsBedrockEndpoint: z.string().optional(),
awsBedrock1MContext: z.boolean().optional(), // Enable 'context-1m-2025-08-07' beta for 1M context window.
awsBedrockServiceTier: z.enum(["STANDARD", "FLEX", "PRIORITY"]).optional(), // AWS Bedrock service tier selection
})

const vertexSchema = apiModelIdProviderModelSchema.extend({
Expand Down
27 changes: 27 additions & 0 deletions packages/types/src/providers/bedrock.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -562,3 +562,30 @@ export const BEDROCK_GLOBAL_INFERENCE_MODEL_IDS = [
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-opus-4-5-20251101-v1:0",
] as const

// Amazon Bedrock Service Tier types
export type BedrockServiceTier = "STANDARD" | "FLEX" | "PRIORITY"

// Models that support service tiers based on AWS documentation
// https://docs.aws.amazon.com/bedrock/latest/userguide/service-tiers-inference.html
export const BEDROCK_SERVICE_TIER_MODEL_IDS = [
// Amazon Nova models
"amazon.nova-lite-v1:0",
"amazon.nova-pro-v1:0",
"amazon.nova-pro-latency-optimized-v1:0",
// DeepSeek models
"deepseek.r1-v1:0",
// Qwen models
"qwen.qwen3-next-80b-a3b",
"qwen.qwen3-coder-480b-a35b-v1:0",
// OpenAI GPT-OSS models
"openai.gpt-oss-20b-1:0",
"openai.gpt-oss-120b-1:0",
] as const

// Service tier pricing multipliers
export const BEDROCK_SERVICE_TIER_PRICING = {
STANDARD: 1.0, // Base price
FLEX: 0.5, // 50% discount from standard
PRIORITY: 1.75, // 75% premium over standard
} as const
43 changes: 42 additions & 1 deletion src/api/providers/bedrock.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
type ModelInfo,
type ProviderSettings,
type BedrockModelId,
type BedrockServiceTier,
bedrockDefaultModelId,
bedrockModels,
bedrockDefaultPromptRouterModelId,
Expand All @@ -27,6 +28,8 @@ import {
AWS_INFERENCE_PROFILE_MAPPING,
BEDROCK_1M_CONTEXT_MODEL_IDS,
BEDROCK_GLOBAL_INFERENCE_MODEL_IDS,
BEDROCK_SERVICE_TIER_MODEL_IDS,
BEDROCK_SERVICE_TIER_PRICING,
} from "@roo-code/types"

import { ApiStream } from "../transform/stream"
Expand All @@ -53,13 +56,14 @@ interface BedrockInferenceConfig {
}

// Define interface for Bedrock additional model request fields
// This includes thinking configuration, 1M context beta, and other model-specific parameters
// This includes thinking configuration, 1M context beta, service tier, and other model-specific parameters
interface BedrockAdditionalModelFields {
thinking?: {
type: "enabled"
budget_tokens: number
}
anthropic_beta?: string[]
service_tier?: BedrockServiceTier
[key: string]: any // Add index signature to be compatible with DocumentType
}

Expand Down Expand Up @@ -433,6 +437,19 @@ export class AwsBedrockHandler extends BaseProvider implements SingleCompletionH
additionalModelRequestFields.anthropic_beta = anthropicBetas
}

// Add service tier if specified and model supports it
if (this.options.awsBedrockServiceTier && BEDROCK_SERVICE_TIER_MODEL_IDS.includes(baseModelId as any)) {
Copy link
Contributor Author

@roomote roomote bot Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This service tier feature should have test coverage. The existing bedrock.spec.ts has comprehensive tests for similar features (like the 1M context beta). Consider adding tests for: (1) service tier parameter being included in API requests for supported models, (2) pricing multipliers being correctly applied in getModel(), and (3) service tier NOT being applied for unsupported models.

Fix it with Roo Code or mention @roomote and request a fix.

Copy link
Contributor

@ronyblum ronyblum Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roomote amazon.nova-2-lite-v1:0 is also supported for the Service Tiers. Add to the list of models

Copy link
Contributor Author

@roomote roomote bot Dec 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixaroo Clock   See task on Roo Cloud

Fixed the reported issues. All local checks passed.

View commit | Revert commit

Copy link
Contributor

@ronyblum ronyblum Dec 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roomote
1/Error message received - Unknown Error: The model returned the following errors: Malformed input request: #: extraneous key [service_tier] is not permitted, please reformat your input and try again.
2/The code is adding service_tier to additionalModelRequestFields, but according to the AWS Bedrock documentation, service_tier should be a top-level parameter in the request, not nested inside additionalModelRequestFields. Fix accordingly
3/Documentation https://docs.aws.amazon.com/bedrock/latest/userguide/service-tiers-inference.html

https://docs.aws.amazon.com/bedrock/latest/userguide/custom-model-import-advanced-features.html

Copy link
Contributor Author

@roomote roomote bot Dec 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixaroo Clock   See task on Roo Cloud

Fixed the service_tier placement. Per AWS documentation, it must be a top-level request parameter, not nested inside additionalModelRequestFields. All tests pass.

View commit | Revert commit

if (!additionalModelRequestFields) {
additionalModelRequestFields = {} as BedrockAdditionalModelFields
}
additionalModelRequestFields.service_tier = this.options.awsBedrockServiceTier
logger.info("Service tier specified for Bedrock request", {
ctx: "bedrock",
modelId: modelConfig.id,
serviceTier: this.options.awsBedrockServiceTier,
})
}

// Build tool configuration if native tools are enabled
let toolConfig: ToolConfiguration | undefined
if (useNativeTools && metadata?.tools) {
Expand Down Expand Up @@ -1089,6 +1106,30 @@ export class AwsBedrockHandler extends BaseProvider implements SingleCompletionH
defaultTemperature: BEDROCK_DEFAULT_TEMPERATURE,
})

// Apply service tier pricing if specified and model supports it
const baseModelIdForTier = this.parseBaseModelId(modelConfig.id)
if (this.options.awsBedrockServiceTier && BEDROCK_SERVICE_TIER_MODEL_IDS.includes(baseModelIdForTier as any)) {
const pricingMultiplier = BEDROCK_SERVICE_TIER_PRICING[this.options.awsBedrockServiceTier]
if (pricingMultiplier && pricingMultiplier !== 1.0) {
// Apply pricing multiplier to all price fields
modelConfig.info = {
...modelConfig.info,
inputPrice: modelConfig.info.inputPrice
? modelConfig.info.inputPrice * pricingMultiplier
: undefined,
outputPrice: modelConfig.info.outputPrice
? modelConfig.info.outputPrice * pricingMultiplier
: undefined,
cacheWritesPrice: modelConfig.info.cacheWritesPrice
? modelConfig.info.cacheWritesPrice * pricingMultiplier
: undefined,
cacheReadsPrice: modelConfig.info.cacheReadsPrice
? modelConfig.info.cacheReadsPrice * pricingMultiplier
: undefined,
}
}
}

// Don't override maxTokens/contextWindow here; handled in getModelById (and includes user overrides)
return { ...modelConfig, ...params } as {
id: BedrockModelId | string
Expand Down
49 changes: 49 additions & 0 deletions webview-ui/src/components/settings/providers/Bedrock.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@ import { VSCodeTextField } from "@vscode/webview-ui-toolkit/react"
import {
type ProviderSettings,
type ModelInfo,
type BedrockServiceTier,
BEDROCK_REGIONS,
BEDROCK_1M_CONTEXT_MODEL_IDS,
BEDROCK_GLOBAL_INFERENCE_MODEL_IDS,
BEDROCK_SERVICE_TIER_MODEL_IDS,
} from "@roo-code/types"

import { useAppTranslation } from "@src/i18n/TranslationContext"
Expand Down Expand Up @@ -35,6 +37,10 @@ export const Bedrock = ({ apiConfiguration, setApiConfigurationField, selectedMo
!!apiConfiguration?.apiModelId &&
BEDROCK_GLOBAL_INFERENCE_MODEL_IDS.includes(apiConfiguration.apiModelId as any)

// Check if the selected model supports service tiers
const supportsServiceTiers =
!!apiConfiguration?.apiModelId && BEDROCK_SERVICE_TIER_MODEL_IDS.includes(apiConfiguration.apiModelId as any)

// Update the endpoint enabled state when the configuration changes
useEffect(() => {
setAwsEndpointSelected(!!apiConfiguration?.awsBedrockEndpointEnabled)
Expand Down Expand Up @@ -150,6 +156,49 @@ export const Bedrock = ({ apiConfiguration, setApiConfigurationField, selectedMo
</SelectContent>
</Select>
</div>
{supportsServiceTiers && (
<div>
<label className="block font-medium mb-1">{t("settings:providers.awsServiceTier")}</label>
<Select
value={apiConfiguration?.awsBedrockServiceTier || "STANDARD"}
onValueChange={(value) =>
setApiConfigurationField("awsBedrockServiceTier", value as BedrockServiceTier)
}>
<SelectTrigger className="w-full">
<SelectValue placeholder={t("settings:common.select")} />
</SelectTrigger>
<SelectContent>
<SelectItem value="STANDARD">
<div>
<div>{t("settings:providers.awsServiceTierStandard")}</div>
<div className="text-xs text-vscode-descriptionForeground">
{t("settings:providers.awsServiceTierStandardDesc")}
</div>
</div>
</SelectItem>
<SelectItem value="FLEX">
<div>
<div>{t("settings:providers.awsServiceTierFlex")}</div>
<div className="text-xs text-vscode-descriptionForeground">
{t("settings:providers.awsServiceTierFlexDesc")}
</div>
</div>
</SelectItem>
<SelectItem value="PRIORITY">
<div>
<div>{t("settings:providers.awsServiceTierPriority")}</div>
<div className="text-xs text-vscode-descriptionForeground">
{t("settings:providers.awsServiceTierPriorityDesc")}
</div>
</div>
</SelectItem>
</SelectContent>
</Select>
<div className="text-sm text-vscode-descriptionForeground mt-1">
{t("settings:providers.awsServiceTierNote")}
</div>
</div>
)}
{supportsGlobalInference && (
<Checkbox
checked={apiConfiguration?.awsUseGlobalInference || false}
Expand Down
8 changes: 8 additions & 0 deletions webview-ui/src/i18n/locales/en/settings.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,14 @@
"awsRegion": "AWS Region",
"awsCrossRegion": "Use cross-region inference",
"awsGlobalInference": "Use Global inference (auto-select optimal AWS Region)",
"awsServiceTier": "Service Tier",
"awsServiceTierStandard": "Standard",
"awsServiceTierStandardDesc": "Balanced performance and cost",
"awsServiceTierFlex": "Flex (50% discount)",
"awsServiceTierFlexDesc": "Lower cost, higher latency for non-critical tasks",
"awsServiceTierPriority": "Priority (75% premium)",
"awsServiceTierPriorityDesc": "Fastest performance for mission-critical applications",
"awsServiceTierNote": "Service tiers affect pricing and performance. Flex offers 50% discount with higher latency, Priority offers 25% better performance with 75% premium.",
"awsBedrockVpc": {
"useCustomVpcEndpoint": "Use custom VPC endpoint",
"vpcEndpointUrlPlaceholder": "Enter VPC Endpoint URL (optional)",
Expand Down
Loading