feat: add Bedrock Mantle inference engine support for gpt-oss models via OpenAI-compatible SSE endpoint - #3489
Merged
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughSummary by CodeRabbit
WalkthroughRoutes gpt-oss (Mantle) models to Mantle handlers by adding fasthttp Mantle clients, a sign-from-key wrapper, model detection and Mantle URL builders, plus non-streaming and streaming Mantle handlers that delegate to existing OpenAI-compatible handlers. ChangesBedrock Mantle Routing
Sequence Diagram(s)sequenceDiagram
participant Client as Client
participant BedrockProvider as BedrockProvider
participant Router as isMantleModel
participant Signer as mantleSigV4Headers
participant Mantle as BedrockMantle
participant OpenAI as OpenAIHandler
Client->>BedrockProvider: ChatCompletion/Responses(request)
BedrockProvider->>Router: Check model string
alt Mantle model (gpt-oss)
Router->>Signer: Pre-sign exact request bytes + Accept
Signer->>Mantle: POST with SigV4 headers (Authorization, X-Amz-Date, x-amz-content-sha256, Accept, X-Amz-Security-Token?)
Mantle->>OpenAI: Return OpenAI-compatible JSON or SSE stream
OpenAI->>Client: Parsed response / stream chunks
else Non-Mantle
BedrockProvider->>OpenAI: Route to standard Bedrock runtime handlers
end
sequenceDiagram
participant StreamingHandler as StreamingHandler
participant Signer as mantleSigV4Headers
participant MantleSSE as MantleSSE
participant Chunker as StreamChunkPipeline
StreamingHandler->>Signer: Sign event-stream bytes with Accept: text/event-stream
Signer->>MantleSSE: Open SSE stream with SigV4 headers
loop Per SSE Line
MantleSSE->>StreamingHandler: data/event line
StreamingHandler->>Chunker: Parse and emit incremental chunk
end
StreamingHandler->>Chunker: Emit terminal chunk (finish reason + usage)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Comment |
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
BearTS
changed the title
gpt-oss models via OpenAI-compatible SSE endpoint
BearTS
marked this pull request as ready for review
May 14, 2026 08:25
Contributor
Confidence Score: 5/5The change is well-scoped and safe to merge; Mantle routing only engages for models containing gpt-oss and all entry points check operation permissions first. The core routing, SigV4 signing, and streaming delegation are all logically sound. Several issues from earlier rounds have been addressed. The remaining finding is a defensive hygiene concern rather than a current breakage. core/providers/bedrock/mantle.go — the isMantleModel predicate deserves a tighter definition to avoid accidental routing of future model names that contain gpt-oss. Important Files Changed
Reviews (9): Last reviewed commit: "feat: add support for bedrock mantle" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/providers/bedrock/mantle.go`:
- Around line 346-357: When handling readErr from sseReader.ReadDataLine() in
the mantle stream loop (and the analogous branch in responsesStreamViaMantle),
check isStreamTransportError(readErr) first and treat such errors as non-Bifrost
transport failures (i.e., set IsBifrostError=false / allow retry semantics)
instead of immediately calling providerUtils.ProcessAndSendError; only call
ProcessAndSendError for non-transport errors. Update the branch that currently
sets schemas.BifrostContextKeyStreamEndIndicator and calls
provider.logger.Warn/ProcessAndSendError to first call
isStreamTransportError(readErr) and return after marking or delegating so
executeRequestWithRetries can retry transport failures.
- Around line 22-25: The predicate in isMantleModel currently only checks
bareModel for the "gpt-oss" prefix and misses documented normalized IDs like
"openai.gpt-oss-120b"; update isMantleModel (which calls
parseBedrockRegionAndModel) to return true when bareModel starts with either
"gpt-oss" OR "openai.gpt-oss" (e.g., use strings.HasPrefix(bareModel, "gpt-oss")
|| strings.HasPrefix(bareModel, "openai.gpt-oss")) so both legacy and normalized
Mantle model IDs are routed through the Mantle path.
- Line 60: Add a new constant named bedrockMantleSigningService =
"bedrock-mantle" (in types.go) and update the two calls to signAWSRequestFromKey
in mantle.go that currently pass bedrockSigningService ("bedrock") so they
instead pass bedrockMantleSigningService; specifically, replace usages where
signAWSRequestFromKey(ctx, req, key.BedrockKeyConfig, region,
bedrockSigningService) is invoked (both occurrences) so Mantle requests are
SigV4-signed with the "bedrock-mantle" service name.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: f1dfb778-2ada-4fe3-a42c-c47c35902a74
📒 Files selected for processing (2)
core/providers/bedrock/bedrock.gocore/providers/bedrock/mantle.go
BearTS
force-pushed
the
05-14-feat_add_support_for_bedrock_mantle
branch
from
May 14, 2026 09:47
4779623 to
6f272ba
Compare
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (2)
core/providers/bedrock/mantle.go (2)
40-65:⚠️ Potential issue | 🔴 Critical | ⚡ Quick winUse dedicated
bedrockMantleSigningServiceconstant for Mantle SigV4 signing.Line 52 signs Mantle requests with
bedrockSigningService("bedrock"), but AWS Bedrock Mantle requires the service name"bedrock-mantle"in the SigV4 credential scope. Using the wrong service name will cause signature verification failures when relying on AWS credentials.Suggested fix
In
types.go, add:const bedrockSigningService = "bedrock" +const bedrockMantleSigningService = "bedrock-mantle"In
mantle.goline 52:- if bifrostErr := signAWSRequestFromKey(ctx, req, key.BedrockKeyConfig, region, bedrockSigningService); bifrostErr != nil { + if bifrostErr := signAWSRequestFromKey(ctx, req, key.BedrockKeyConfig, region, bedrockMantleSigningService); bifrostErr != nil {🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/providers/bedrock/mantle.go` around lines 40 - 65, Change Mantle SigV4 signing to use a dedicated service name: add a new constant bedrockMantleSigningService = "bedrock-mantle" (e.g., in types.go) and update the call in mantleSigV4Headers to pass bedrockMantleSigningService instead of bedrockSigningService when calling signAWSRequestFromKey; ensure any references to the Mantle signing service use the new constant so the SigV4 credential scope is "bedrock-mantle".
17-19:⚠️ Potential issue | 🟠 Major | ⚡ Quick winUse bare model extraction for precise Mantle model detection.
strings.Contains(model, "gpt-oss")will incorrectly match model IDs like"my-custom-gpt-oss-model"or"gpt-ossified". Extract the bare model first (viaparseBedrockRegionAndModel) and check prefixes to avoid false positives.Suggested fix
func isMantleModel(model string) bool { - return strings.Contains(model, "gpt-oss") + _, bareModel := parseBedrockRegionAndModel(model) + return strings.HasPrefix(bareModel, "gpt-oss") || + strings.HasPrefix(bareModel, "openai.gpt-oss") }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/providers/bedrock/mantle.go` around lines 17 - 19, Update isMantleModel to first extract the bare model via parseBedrockRegionAndModel and then check the bare model using a prefix check (e.g., strings.HasPrefix) instead of strings.Contains; specifically, call parseBedrockRegionAndModel(model) to get the stripped model identifier (use the returned bareModel variable) and return strings.HasPrefix(bareModel, "gpt-oss") so names like "my-custom-gpt-oss-model" or "gpt-ossified" no longer match incorrectly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@core/providers/bedrock/mantle.go`:
- Around line 40-65: Change Mantle SigV4 signing to use a dedicated service
name: add a new constant bedrockMantleSigningService = "bedrock-mantle" (e.g.,
in types.go) and update the call in mantleSigV4Headers to pass
bedrockMantleSigningService instead of bedrockSigningService when calling
signAWSRequestFromKey; ensure any references to the Mantle signing service use
the new constant so the SigV4 credential scope is "bedrock-mantle".
- Around line 17-19: Update isMantleModel to first extract the bare model via
parseBedrockRegionAndModel and then check the bare model using a prefix check
(e.g., strings.HasPrefix) instead of strings.Contains; specifically, call
parseBedrockRegionAndModel(model) to get the stripped model identifier (use the
returned bareModel variable) and return strings.HasPrefix(bareModel, "gpt-oss")
so names like "my-custom-gpt-oss-model" or "gpt-ossified" no longer match
incorrectly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: f64b3fb9-11b8-4462-b259-246d8a74a4d9
📒 Files selected for processing (2)
core/providers/bedrock/bedrock.gocore/providers/bedrock/mantle.go
🚧 Files skipped from review as they are similar to previous changes (1)
- core/providers/bedrock/bedrock.go
BearTS
force-pushed
the
05-14-feat_add_support_for_bedrock_mantle
branch
from
May 14, 2026 10:04
6f272ba to
f4e4400
Compare
BearTS
force-pushed
the
05-14-feat_add_support_for_bedrock_mantle
branch
from
May 14, 2026 10:23
f4e4400 to
7cc0f0e
Compare
BearTS
force-pushed
the
05-14-feat_add_support_for_bedrock_mantle
branch
from
May 14, 2026 10:51
7cc0f0e to
3234b6a
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/providers/bedrock/bedrock.go`:
- Around line 31-40: The BedrockProvider currently declares mantleClient and
mantleStreamingClient as *fasthttp.Client which violates the net/http + HTTP/2
guideline; replace those fasthttp.Client fields with *http.Client (or reuse the
provider's existing client/streamingClient) and ensure Mantle requests use the
same net/http.Transport configured with ForceAttemptHTTP2=true and
MaxConnsPerHost pulled from NetworkConfig; update any code that constructs or
uses mantleClient/mantleStreamingClient to build them from the same Transport
used for client/streamingClient so Mantle shares the net/http stack rather than
a fasthttp one.
In `@core/providers/bedrock/mantle.go`:
- Around line 86-97: The call to the shared OpenAI handler
openai.HandleOpenAIChatCompletionRequest is passing a hardcoded schemas.Bedrock
which collapses custom provider aliases; replace that argument with
provider.GetProviderKey() (call the method on the same provider instance used in
the call) so the downstream handlers receive the real provider key string. Do
the same replacement for the other delegated OpenAI handler invocations in this
file that currently pass schemas.Bedrock (e.g., the other openai.HandleOpenAI...
calls around the other request/response delegations) so all error/response
contexts reflect the configured provider key.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ccedee5d-c0ea-4063-a262-7c14c60b11e4
📒 Files selected for processing (3)
core/providers/bedrock/bedrock.gocore/providers/bedrock/mantle.gocore/providers/bedrock/types.go
🚧 Files skipped from review as they are similar to previous changes (1)
- core/providers/bedrock/types.go
BearTS
force-pushed
the
05-14-feat_add_support_for_bedrock_mantle
branch
from
May 14, 2026 11:04
3234b6a to
4ef1562
Compare
BearTS
force-pushed
the
05-14-feat_add_support_for_bedrock_mantle
branch
from
May 14, 2026 12:58
4ef1562 to
52b9150
Compare
BearTS
force-pushed
the
05-14-feat_add_support_for_bedrock_mantle
branch
from
May 14, 2026 18:26
52b9150 to
0bfa0e0
Compare
akshaydeo
dismissed
coderabbitai[bot]’s stale review
May 15, 2026 05:59
The merge-base changed after approval.
BearTS
force-pushed
the
05-14-feat_add_support_for_bedrock_mantle
branch
from
May 15, 2026 08:00
0bfa0e0 to
67cab70
Compare
akshaydeo
approved these changes
May 15, 2026
Contributor
Merge activity
|
akshaydeo
pushed a commit
that referenced
this pull request
May 15, 2026
…s via OpenAI-compatible SSE endpoint (#3489) ## Summary Adds support for routing `gpt-oss` model requests through the AWS Bedrock Mantle inference endpoint (`bedrock-mantle.{region}.api.aws`). These are OpenAI-compatible models hosted on Bedrock that require a different endpoint and request/response format than standard Bedrock models. ## Changes - Added `isMantleModel` and `mantleModelID` helpers to detect and transform `gpt-oss` model identifiers for the Mantle endpoint. - Added `signAWSRequestFromKey` as a convenience wrapper around `signAWSRequest` that reads credentials from a `BedrockKeyConfig`, falling back to the default AWS credential chain when no config is provided. - Added `completeMantleRequest` and `makeMantleStreamingRequest` to handle non-streaming and SSE streaming HTTP requests to the Mantle endpoint. Both support Bearer token auth (via `key.Value`) or SigV4 signing as a fallback. - Added `chatCompletionViaMantle`, `chatCompletionStreamViaMantle`, `responsesViaMantle`, and `responsesStreamViaMantle` to convert Bifrost requests into OpenAI-compatible payloads and route them to the Mantle endpoint. - Injected Mantle routing checks at the top of `ChatCompletion`, `ChatCompletionStream`, `Responses`, and `ResponsesStream` so that any `gpt-oss` model is transparently redirected before the standard Bedrock path executes. - Mantle streaming uses SSE (`text/event-stream`) rather than AWS EventStream binary framing, with full support for idle timeouts, cancellation, usage accumulation, and terminal chunk emission. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/bedrock/... ``` To validate end-to-end, configure a Bedrock key and send a request with a `gpt-oss` model (e.g. `gpt-oss-mini`). Verify the request is routed to `bedrock-mantle.{region}.api.aws/v1/chat/completions` and that both streaming and non-streaming responses are returned correctly. For SigV4 auth, omit `key.Value` and ensure `BedrockKeyConfig` credentials or the default AWS credential chain (env vars, IAM role, instance profile) are available. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations - SigV4 request signing is applied to Mantle requests when no Bearer token is present, using the same credential resolution path as standard Bedrock requests. - When `BedrockKeyConfig` is `nil`, signing falls back to the default AWS credential chain, which may pick up ambient credentials (env vars, instance profile). This is intentional and consistent with existing Bedrock behavior. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
18 tasks
akshaydeo
pushed a commit
that referenced
this pull request
May 20, 2026
…s via OpenAI-compatible SSE endpoint (#3489) ## Summary Adds support for routing `gpt-oss` model requests through the AWS Bedrock Mantle inference endpoint (`bedrock-mantle.{region}.api.aws`). These are OpenAI-compatible models hosted on Bedrock that require a different endpoint and request/response format than standard Bedrock models. ## Changes - Added `isMantleModel` and `mantleModelID` helpers to detect and transform `gpt-oss` model identifiers for the Mantle endpoint. - Added `signAWSRequestFromKey` as a convenience wrapper around `signAWSRequest` that reads credentials from a `BedrockKeyConfig`, falling back to the default AWS credential chain when no config is provided. - Added `completeMantleRequest` and `makeMantleStreamingRequest` to handle non-streaming and SSE streaming HTTP requests to the Mantle endpoint. Both support Bearer token auth (via `key.Value`) or SigV4 signing as a fallback. - Added `chatCompletionViaMantle`, `chatCompletionStreamViaMantle`, `responsesViaMantle`, and `responsesStreamViaMantle` to convert Bifrost requests into OpenAI-compatible payloads and route them to the Mantle endpoint. - Injected Mantle routing checks at the top of `ChatCompletion`, `ChatCompletionStream`, `Responses`, and `ResponsesStream` so that any `gpt-oss` model is transparently redirected before the standard Bedrock path executes. - Mantle streaming uses SSE (`text/event-stream`) rather than AWS EventStream binary framing, with full support for idle timeouts, cancellation, usage accumulation, and terminal chunk emission. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/bedrock/... ``` To validate end-to-end, configure a Bedrock key and send a request with a `gpt-oss` model (e.g. `gpt-oss-mini`). Verify the request is routed to `bedrock-mantle.{region}.api.aws/v1/chat/completions` and that both streaming and non-streaming responses are returned correctly. For SigV4 auth, omit `key.Value` and ensure `BedrockKeyConfig` credentials or the default AWS credential chain (env vars, IAM role, instance profile) are available. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations - SigV4 request signing is applied to Mantle requests when no Bearer token is present, using the same credential resolution path as standard Bedrock requests. - When `BedrockKeyConfig` is `nil`, signing falls back to the default AWS credential chain, which may pick up ambient credentials (env vars, instance profile). This is intentional and consistent with existing Bedrock behavior. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
akshaydeo
added a commit
that referenced
this pull request
May 20, 2026
## Summary This PR cuts the `v1.5.11` / `v1.3.11` release across core, framework, and all plugins, and introduces a new Claude skill (`release-checklist`) for pre-release migration safety auditing. ## Changes - **`release-checklist` skill** — Adds `.claude/skills/release-checklist/SKILL.md`, a read-only pre-release audit tool that scans Go-defined database migrations changed in a release for high-scale deadlock/lock-contention risks and boot-time-blocking operations. It produces a structured `PASS`/`WARN`/`FAIL` report with a concrete remediation plan per finding. The skill is designed to grow via an extensible Checks Registry. - **Version bumps** — `core` → `1.5.11`, `framework` → `1.3.11`, `transports` → `1.5.3`, `plugins/governance` → `1.5.11`, `plugins/logging` → `1.5.11`, `plugins/semanticcache` → `1.5.11`, `plugins/otel` → `1.2.11`, `plugins/maxim` → `1.6.11`, `plugins/prompts` → `1.0.11`, and remaining plugins bumped accordingly. - **Changelogs populated** — All per-package changelogs updated with the full set of features and fixes shipping in this release. Key highlights in this release: - Temporary access tokens for scoped, time-limited API access - MCP per-user OAuth flow refactor - Bedrock Mantle inference engine support - Azure Realtime provider with enriched session tracking - Direct access control (DAC) and virtual key rotation - Cluster-aware log metadata and per-node usage aggregation - Feature flag framework - Config-hash-based file value override of DB on restart - Semantic cache plugin rewrite - Numerous streaming stability, Bedrock, Anthropic, and Gemini fixes - AWS SDK and dependency security updates ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Verify version files reflect the new release cat core/version # expect 1.5.11 cat framework/version # expect 1.3.11 cat transports/version # expect 1.5.3 # Core/Transports go test ./... ``` To exercise the new `release-checklist` skill, invoke it via Claude with: ``` /release-checklist origin/dev...HEAD ``` Expected output: a structured report with `PASS`/`WARN`/`FAIL` per check and a Remediation Plan table for any findings. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues #3603, #3565, #3489, #3334, #3335, #3435, #3554, #3590, #3444, #3198, #3581, #3610, #3599, #3567, #3382, #3461 and others listed in the changelogs. ## Security considerations - AWS SDK and dependency security updates are included (#3461). - `FullyRedacted()` for proxy passwords and `MarshalForStorage()` for `ProxyConfig` prevent partial secret leakage in API responses (#3445). - The `release-checklist` skill is strictly read-only and never modifies files. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
2 tasks
Merged
2 tasks
This was referenced Jun 5, 2026
This was referenced Jun 15, 2026
This was referenced Jul 13, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for routing
gpt-ossmodel requests through the AWS Bedrock Mantle inference endpoint (bedrock-mantle.{region}.api.aws). These are OpenAI-compatible models hosted on Bedrock that require a different endpoint and request/response format than standard Bedrock models.Changes
isMantleModelandmantleModelIDhelpers to detect and transformgpt-ossmodel identifiers for the Mantle endpoint.signAWSRequestFromKeyas a convenience wrapper aroundsignAWSRequestthat reads credentials from aBedrockKeyConfig, falling back to the default AWS credential chain when no config is provided.completeMantleRequestandmakeMantleStreamingRequestto handle non-streaming and SSE streaming HTTP requests to the Mantle endpoint. Both support Bearer token auth (viakey.Value) or SigV4 signing as a fallback.chatCompletionViaMantle,chatCompletionStreamViaMantle,responsesViaMantle, andresponsesStreamViaMantleto convert Bifrost requests into OpenAI-compatible payloads and route them to the Mantle endpoint.ChatCompletion,ChatCompletionStream,Responses, andResponsesStreamso that anygpt-ossmodel is transparently redirected before the standard Bedrock path executes.text/event-stream) rather than AWS EventStream binary framing, with full support for idle timeouts, cancellation, usage accumulation, and terminal chunk emission.Type of change
Affected areas
How to test
go test ./core/providers/bedrock/...To validate end-to-end, configure a Bedrock key and send a request with a
gpt-ossmodel (e.g.gpt-oss-mini). Verify the request is routed tobedrock-mantle.{region}.api.aws/v1/chat/completionsand that both streaming and non-streaming responses are returned correctly.For SigV4 auth, omit
key.Valueand ensureBedrockKeyConfigcredentials or the default AWS credential chain (env vars, IAM role, instance profile) are available.Breaking changes
Related issues
Security considerations
BedrockKeyConfigisnil, signing falls back to the default AWS credential chain, which may pick up ambient credentials (env vars, instance profile). This is intentional and consistent with existing Bedrock behavior.Checklist
docs/contributing/README.mdand followed the guidelines