Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 23 additions & 23 deletions packages/ai/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ Other `aisdk:` packages, including Google Vertex, Azure, and Bedrock, currently
| `@ai-sdk/google-vertex/xai` | Vertex Chat / Responses | Partial / usable | Decide Chat/Responses selection for catalog models, add runner mapping and recorded coverage, and review xAI-specific request options. |
| `@ai-sdk/azure` | Azure OpenAI Chat/Responses facade | Partial | Map runner/catalog metadata to native Azure, handle resourceName/baseURL/apiVersion variants, add AAD/token auth story, and verify Chat vs Responses deployment selection. |
| `@ai-sdk/amazon-bedrock` | Bedrock Converse | Partial | Add default AWS credential chain/profile support, region/inference-profile model ID handling, provider option parity via `additionalModelRequestFields`, guardrails/performance config, and runner/catalog mapping. |
| `@ai-sdk/amazon-bedrock/mantle` | Bedrock Mantle OpenAI-compatible Chat/Responses namespace | Missing | Decide native Mantle shape, likely separate from Converse because it uses OpenAI-compatible Chat/Responses semantics over Bedrock. Add package mapping and tests. |
| `@ai-sdk/amazon-bedrock/mantle` | Bedrock Mantle OpenAI-compatible Chat/Responses namespace | Partial / usable | Add default AWS credential chain/profile support; native catalog mapping currently requires bearer auth or explicit static credentials. |

## Highest-Risk Gaps

Expand All @@ -71,30 +71,31 @@ Other `aisdk:` packages, including Google Vertex, Azure, and Bedrock, currently
5. Azure is only a provider facade, not a full runtime replacement. Native Azure exists, but the catalog runner does not select it, and token auth/resource variants need review.
6. Provider option typing is uneven. OpenAI, Anthropic, Gemini, Bedrock, and OpenRouter each expose a small typed subset plus raw HTTP overlays; this is useful but not equivalent to AI SDK provider option coverage.
7. Structured output is not provider-native yet. `LLM.generateObject` still uses a synthetic tool strategy, while the future design expects native structured output where reliable and tool fallback where needed.
8. Package/namespace boundaries for the current native loading set are explicit in docs and exports. Other exported provider facades are not catalog package entrypoints until they implement the contract. Vertex xAI still needs catalog API selection; the missing native boundary is Bedrock Mantle.
9. Recorded coverage is uneven. OpenAI, Anthropic, Gemini, Bedrock Converse, Cloudflare, OpenRouter, and several OpenAI-compatible Chat providers have cassettes. Azure, Vertex, and Mantle need first-class recorded scenarios before switching defaults.
8. Package/namespace boundaries for the current native loading set are explicit in docs and exports. Other exported provider facades are not catalog package entrypoints until they implement the contract. Vertex xAI still needs catalog API selection.
9. Recorded coverage is uneven. OpenAI, Anthropic, Gemini, Bedrock Converse, Bedrock Mantle, Cloudflare, OpenRouter, and several OpenAI-compatible Chat providers have cassettes. Azure and Vertex still need first-class recorded scenarios before switching defaults.

## Native Namespace Shape

These are implementation/API slices, not separate npm packages.

| API slice | Package-like entrypoint | Purpose |
| ----------------------------- | ------------------------------------------------------- | ---------------------------------------------------------------------------- |
| OpenAI Chat | `@opencode-ai/ai/providers/openai/chat` | OpenAI `/chat/completions` semantics. |
| OpenAI Responses | `@opencode-ai/ai/providers/openai/responses` | OpenAI `/responses` semantics with HTTP/WebSocket selected through settings. |
| OpenAI-compatible Chat | `@opencode-ai/ai/providers/openai-compatible` | Generic OpenAI-compatible `/chat/completions`. |
| Open Responses-compatible | `@opencode-ai/ai/providers/openai-compatible/responses` | Generic provider-neutral `/responses`. |
| Anthropic-compatible Messages | `@opencode-ai/ai/providers/anthropic-compatible` | Generic Anthropic-compatible `/messages`. |
| Anthropic Messages | `@opencode-ai/ai/providers/anthropic` | Anthropic Messages API. |
| Gemini Developer API | `@opencode-ai/ai/providers/google` | Google AI Studio Gemini API. |
| Vertex Gemini | `@opencode-ai/ai/providers/google-vertex/gemini` | Vertex Gemini API; `providers/google-vertex` is the default alias. |
| Vertex Chat | `@opencode-ai/ai/providers/google-vertex/chat` | Vertex OpenAI-compatible Chat Completions for MaaS models. |
| Vertex Responses | `@opencode-ai/ai/providers/google-vertex/responses` | Vertex Open Responses for Grok models. |
| Vertex Messages | `@opencode-ai/ai/providers/google-vertex/messages` | Vertex-hosted Anthropic Messages API. |
| Bedrock Converse | `@opencode-ai/ai/providers/amazon-bedrock` | AWS Bedrock Converse API. |
| Bedrock Mantle | Missing | AWS Bedrock Mantle OpenAI-compatible APIs. |
| Azure OpenAI Chat | `@opencode-ai/ai/providers/azure/chat` | Azure specialization of OpenAI Chat. |
| Azure OpenAI Responses | `@opencode-ai/ai/providers/azure/responses` | Azure specialization of OpenAI Responses. |
| API slice | Package-like entrypoint | Purpose |
| ----------------------------- | ----------------------------------------------------------- | ---------------------------------------------------------------------------- |
| OpenAI Chat | `@opencode-ai/ai/providers/openai/chat` | OpenAI `/chat/completions` semantics. |
| OpenAI Responses | `@opencode-ai/ai/providers/openai/responses` | OpenAI `/responses` semantics with HTTP/WebSocket selected through settings. |
| OpenAI-compatible Chat | `@opencode-ai/ai/providers/openai-compatible` | Generic OpenAI-compatible `/chat/completions`. |
| Open Responses-compatible | `@opencode-ai/ai/providers/openai-compatible/responses` | Generic provider-neutral `/responses`. |
| Anthropic-compatible Messages | `@opencode-ai/ai/providers/anthropic-compatible` | Generic Anthropic-compatible `/messages`. |
| Anthropic Messages | `@opencode-ai/ai/providers/anthropic` | Anthropic Messages API. |
| Gemini Developer API | `@opencode-ai/ai/providers/google` | Google AI Studio Gemini API. |
| Vertex Gemini | `@opencode-ai/ai/providers/google-vertex/gemini` | Vertex Gemini API; `providers/google-vertex` is the default alias. |
| Vertex Chat | `@opencode-ai/ai/providers/google-vertex/chat` | Vertex OpenAI-compatible Chat Completions for MaaS models. |
| Vertex Responses | `@opencode-ai/ai/providers/google-vertex/responses` | Vertex Open Responses for Grok models. |
| Vertex Messages | `@opencode-ai/ai/providers/google-vertex/messages` | Vertex-hosted Anthropic Messages API. |
| Bedrock Converse | `@opencode-ai/ai/providers/amazon-bedrock` | AWS Bedrock Converse API. |
| Bedrock Mantle Chat | `@opencode-ai/ai/providers/amazon-bedrock/mantle/chat` | AWS Bedrock Mantle OpenAI-compatible Chat API. |
| Bedrock Mantle Responses | `@opencode-ai/ai/providers/amazon-bedrock/mantle/responses` | AWS Bedrock Mantle OpenAI-compatible Responses API. |
| Azure OpenAI Chat | `@opencode-ai/ai/providers/azure/chat` | Azure specialization of OpenAI Chat. |
| Azure OpenAI Responses | `@opencode-ai/ai/providers/azure/responses` | Azure specialization of OpenAI Responses. |

## Suggested Next Work Slices

Expand All @@ -103,6 +104,5 @@ These are implementation/API slices, not separate npm packages.
3. Bring Bedrock native auth/config to AI SDK parity: region, profile, default AWS credential chain, bearer token env, endpoint override, and cross-region inference profile handling.
4. Add runner/catalog mappings and recorded scenarios for the native Vertex Gemini, Chat, Responses, and Messages entrypoints.
5. Decide Chat/Responses selection for `@ai-sdk/google-vertex/xai` catalog models.
6. Add Bedrock Mantle as a separate OpenAI-compatible Bedrock namespace after deciding whether it uses Chat, Responses, or both by model.
7. Expand typed provider options from the existing V1 lowerer knowledge in `packages/core/src/v1/config/provider-options.ts` before adding more raw overlay examples.
8. Add recorded provider tests for Azure, Vertex Gemini, Vertex Chat, Vertex Responses, Vertex Messages, Bedrock credential-chain behavior, and Mantle before making native runtime the default for those packages.
6. Expand typed provider options from the existing V1 lowerer knowledge in `packages/core/src/v1/config/provider-options.ts` before adding more raw overlay examples.
7. Add recorded provider tests for Azure, Vertex Gemini, Vertex Chat, Vertex Responses, Vertex Messages, and Bedrock credential-chain behavior before making native runtime the default for those packages.
15 changes: 11 additions & 4 deletions packages/ai/src/protocols/utils/bedrock-auth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ const signRequest = (input: {
readonly body: string
readonly headers: Headers.Headers
readonly credentials: Credentials
readonly service: string
readonly name: string
}) =>
Effect.tryPromise({
try: async () => {
Expand All @@ -34,23 +36,26 @@ const signRequest = (input: {
accessKeyId: input.credentials.accessKeyId,
secretAccessKey: input.credentials.secretAccessKey,
sessionToken: input.credentials.sessionToken,
service: "bedrock",
service: input.service,
}).sign()
return Object.fromEntries(signed.headers.entries())
},
catch: (error) =>
ProviderShared.invalidRequest(
`Bedrock Converse SigV4 signing failed: ${error instanceof Error ? error.message : String(error)}`,
`${input.name} SigV4 signing failed: ${error instanceof Error ? error.message : String(error)}`,
),
})

/** Sign the exact JSON bytes with SigV4 using credentials configured on the route. */
export const sigV4 = (credentials: Credentials | undefined) =>
export const sigV4 = (
credentials: Credentials | undefined,
options: { readonly service?: string; readonly name?: string } = {},
) =>
Auth.custom((input: AuthInput) => {
return Effect.gen(function* () {
if (!credentials) {
return yield* ProviderShared.invalidRequest(
"Bedrock Converse requires either route bearer auth or AWS credentials configured on the route",
`${options.name ?? "Bedrock Converse"} requires either route bearer auth or AWS credentials configured on the route`,
)
}
const headersForSigning = Headers.set(input.headers, "content-type", "application/json")
Expand All @@ -59,6 +64,8 @@ export const sigV4 = (credentials: Credentials | undefined) =>
body: input.body,
headers: headersForSigning,
credentials,
service: options.service ?? "bedrock",
name: options.name ?? "Bedrock Converse",
})
return Headers.setAll(headersForSigning, signed)
})
Expand Down
107 changes: 107 additions & 0 deletions packages/ai/src/providers/amazon-bedrock-mantle.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
import { Auth } from "../route/auth"
import type { Route as RouteDef, RouteDefaultsInput } from "../route/client"
import type { ProviderPackage } from "../provider-package"
import { OpenAIChat } from "../protocols/openai-chat"
import { OpenAIResponses } from "../protocols/openai-responses"
import { BedrockAuth, type Credentials } from "../protocols/utils/bedrock-auth"
import { ProviderID, type ModelID } from "../schema"
import { withOpenAIOptions, type OpenAIProviderOptionsInput } from "./openai-options"

export const id = ProviderID.make("amazon-bedrock")

export type Config = RouteDefaultsInput & {
readonly apiKey?: string
readonly baseURL?: string
readonly credentials?: Credentials
readonly region?: string
readonly providerOptions?: OpenAIProviderOptionsInput
}

export interface Settings extends ProviderPackage.Settings {
readonly apiKey?: string
readonly auth?: "bearer" | "sigv4"
readonly baseURL?: string
readonly credentials?: Credentials
readonly region?: string
readonly providerOptions?: OpenAIProviderOptionsInput
}

const responsesRoute = OpenAIResponses.route.with({
id: "bedrock-mantle-responses",
provider: id,
})

const chatRoute = OpenAIChat.route.with({
id: "bedrock-mantle-chat",
provider: id,
})

export const routes = [responsesRoute, chatRoute]

const configuredRoute = <Body, Prepared>(route: RouteDef<Body, Prepared>, input: Config) => {
const region = input.region ?? input.credentials?.region ?? "us-east-1"
const credentials = input.credentials === undefined ? undefined : { ...input.credentials, region }
return route.with({
endpoint: { baseURL: input.baseURL ?? `https://bedrock-mantle.${region}.api.aws/v1` },
auth:
input.apiKey === undefined
? BedrockAuth.sigV4(credentials, { service: "bedrock-mantle", name: "Bedrock Mantle" })
: Auth.bearer(input.apiKey),
})
}

const defaults = (input: Config) => {
const { apiKey: _, baseURL: _baseURL, credentials: _credentials, region: _region, ...rest } = input
return rest
}

export const configure = (input: Config = {}) => {
const configuredResponsesRoute = configuredRoute(responsesRoute, input)
const configuredChatRoute = configuredRoute(chatRoute, input)
const modelDefaults = defaults(input)
const responses = (modelID: string | ModelID) =>
configuredResponsesRoute
.with(withOpenAIOptions(modelID, modelDefaults))
.model<OpenAIProviderOptionsInput>({ id: modelID })
const chat = (modelID: string | ModelID) =>
configuredChatRoute
.with(withOpenAIOptions(modelID, modelDefaults))
.model<OpenAIProviderOptionsInput>({ id: modelID })

return {
id,
model: chat,
chat,
responses,
configure,
}
}

export const provider = configure()

const config = (settings: Settings): Config => {
if (settings.auth === "bearer" && settings.apiKey === undefined)
throw new Error("Amazon Bedrock Mantle bearer auth requires apiKey")
if (settings.auth === "sigv4" && settings.apiKey !== undefined)
throw new Error("Amazon Bedrock Mantle SigV4 auth does not accept apiKey")
return {
apiKey: settings.auth === "sigv4" ? undefined : settings.apiKey,
baseURL: settings.baseURL,
credentials: settings.credentials,
headers: settings.headers === undefined ? undefined : { ...settings.headers },
http: settings.body === undefined ? undefined : { body: { ...settings.body } },
limits: settings.limits,
providerOptions: settings.providerOptions,
region: settings.region,
}
}

export const chatModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).chat(modelID)
export const responsesModel: ProviderPackage.Definition<Settings, OpenAIProviderOptionsInput>["model"] = (
modelID,
settings,
) => configure(config(settings)).responses(modelID)
export const model = chatModel
2 changes: 2 additions & 0 deletions packages/ai/src/providers/amazon-bedrock/mantle.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export { chatModel as model } from "../amazon-bedrock-mantle"
export type { Settings } from "../amazon-bedrock-mantle"
2 changes: 2 additions & 0 deletions packages/ai/src/providers/amazon-bedrock/mantle/chat.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export { chatModel as model } from "../../amazon-bedrock-mantle"
export type { Settings } from "../../amazon-bedrock-mantle"
2 changes: 2 additions & 0 deletions packages/ai/src/providers/amazon-bedrock/mantle/responses.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export { responsesModel as model } from "../../amazon-bedrock-mantle"
export type { Settings } from "../../amazon-bedrock-mantle"
1 change: 1 addition & 0 deletions packages/ai/src/providers/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
export * as Anthropic from "./anthropic"
export * as AnthropicCompatible from "./anthropic-compatible"
export * as AmazonBedrock from "./amazon-bedrock"
export * as AmazonBedrockMantle from "./amazon-bedrock-mantle"
export * as Azure from "./azure"
export * as Cloudflare from "./cloudflare"
export { CloudflareAIGateway, CloudflareWorkersAI } from "./cloudflare"
Expand Down
Loading
Loading