feat: add DeepSeek provider support across the gateway - #3234
Closed
etnperlong wants to merge 30 commits into
Closed
feat: add DeepSeek provider support across the gateway#3234etnperlong wants to merge 30 commits into
etnperlong wants to merge 30 commits into
Conversation
|
|
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds DeepSeek support, alias-aware routing and provider conversions, new pre-request hook orchestration, Vertex batch/GCS support, and community MCP library documentation. ChangesRouting and provider behavior
Community MCP library
Sequence Diagram(s)sequenceDiagram
participant Bifrost
participant PluginPipeline
participant Provider
Bifrost->>PluginPipeline: RunPreRequestHooks(req)
PluginPipeline-->>Bifrost: mutated request
Bifrost->>Provider: request / fallback / retry
Provider-->>Bifrost: response or error with RoutingInfo
sequenceDiagram
participant Bifrost
participant DeepSeekProvider
participant DeepSeekAPI
Bifrost->>DeepSeekProvider: ChatCompletion / ListModels / Responses
DeepSeekProvider->>DeepSeekAPI: /chat/completions or /models
DeepSeekAPI-->>DeepSeekProvider: response payload
DeepSeekProvider-->>Bifrost: unified response or unsupported error
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
Poem
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
Confidence Score: 4/5The Go implementation is structurally sound and safe to merge; the open concern is the live CI test using model names that remain unverified against the actual DeepSeek endpoint. The provider implementation faithfully mirrors the Cerebras template — correct two-client split (unary vs. streaming), proper ConfigureDialer call with AllowPrivateNetwork, streaming uses streamingClient, all unsupported operations return the standard error. Every registration point (schemas, gateway switch, config schema, UI, CI) was updated. The live integration test in deepseek_test.go hardcodes deepseek-v4-flash and deepseek-v4-pro as the chat model and fallback; these identifiers do not appear in DeepSeek's current public model documentation. If the secret is present in CI, every live scenario in the comprehensive test suite will hit an invalid-model error from the API. core/providers/deepseek/deepseek_test.go — live integration test uses model names that need verification against the DeepSeek API before the CI job runs with a live key. docs/providers/supported-providers/deepseek.mdx — migration guidance references the same unverified model names. Important Files Changed
Reviews (12): Last reviewed commit: "fix: align DeepSeek provider with upstre..." | Re-trigger Greptile |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/release-pipeline.yml:
- Line 215: The workflow sets DEEPSEEK_API_KEY but the blocked-job
allowed-endpoints lists do not include the DeepSeek API host, causing outbound
calls to be blocked; update each allowed-endpoints list used by DeepSeek-calling
jobs to include the DeepSeek API host (e.g., api.deepseek.ai) so requests using
DEEPSEEK_API_KEY can reach the provider, and make the same addition for the
other allowed-endpoints occurrences referenced in the review.
In `@core/providers/deepseek/deepseek_test.go`:
- Around line 81-86: In the httptest handler closure inside deepseek_test.go
where the test server is created (the http.HandlerFunc passed to
httptest.NewServer), replace the fatal test helpers (require.Equal) with
non-fatal assertions (assert.Equal) for the URL path and Authorization header
checks so failures don't call t.FailNow from a goroutine; update imports to
include testify/assert if needed and use assert.Equal(t, "/chat/completions",
r.URL.Path) and assert.Equal(t, "Bearer test-api-key",
r.Header.Get("Authorization")) (same for the other handler at line ~112) so
assertion failures are reported without terminating the goroutine.
In `@docs/providers/supported-providers/deepseek.mdx`:
- Around line 85-112: Replace the standalone example blocks in the DeepSeek docs
with Mintlify tabbed sections labeled "Web UI", "API", and "config.json": move
the current curl request and any UI usage into the "Web UI" and "API" tabs as
appropriate, and place the JSON config example under the "config.json" tab
(preserve the exact JSON content). Ensure the "config.json" example validates
against transports/config.schema.json and update headings or brief descriptions
inside each tab to match existing style guidelines.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a4812c2a-f98c-46c9-b9e6-9efb5654da7e
📒 Files selected for processing (18)
.github/workflows/pr-tests.yml.github/workflows/release-pipeline.ymlcore/bifrost.gocore/providers/deepseek/cachedcontents.gocore/providers/deepseek/deepseek.gocore/providers/deepseek/deepseek_test.gocore/schemas/bifrost.godocs/deployment-guides/config-json/schema-reference.mdxdocs/docs.jsondocs/openapi/openapi.jsondocs/openapi/schemas/inference/common.yamldocs/overview.mdxdocs/providers/supported-providers/deepseek.mdxdocs/providers/supported-providers/overview.mdxtransports/config.schema.jsonui/lib/constants/config.tsui/lib/constants/icons.tsxui/lib/constants/logs.ts
Contributor
Author
|
I checked the model-name concern against the current DeepSeek docs before changing anything. Right now the official docs list References used:
All three are on |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
docs/providers/supported-providers/deepseek.mdx (1)
112-130: ⚡ Quick winConsider adding a
network_configexample to theconfig.jsontab.Line 55 explicitly tells users to override
network_config.base_urlto access DeepSeek's beta endpoints (https://api.deepseek.com/beta), but theconfig.jsontab doesn't illustrate this. Users following the tab to configure DeepSeek will miss this unless they also read the prose above.📄 Suggested addition
{ "providers": { "deepseek": { "api_keys": [ { "value": "env.DEEPSEEK_API_KEY" } - ] + ], + "network_config": { + "base_url": "https://api.deepseek.com/beta" + } } } }(Remove
network_configfrom the snippet if it is not a valid schema field, or show it commented-out with a note.)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/providers/supported-providers/deepseek.mdx` around lines 112 - 130, Update the config.json example in the config.json Tab of the deepseek.mdx docs to include a network_config example that shows overriding network_config.base_url (e.g., pointing to https://api.deepseek.com/beta) so users see how to target the beta endpoint; if network_config is not part of the real schema, include it as a commented-out example or annotated snippet instead and add a brief inline note. Reference the existing config.json Tab and the network_config.base_url field so the change is made alongside the "providers.deepseek" api_keys example.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/providers/supported-providers/deepseek.mdx`:
- Line 83: The docs line claiming "deepseek-chat and deepseek-reasoner remain
legacy compatibility aliases" is incorrect and understates that those names will
be retired; update the text in docs/providers/supported-providers/deepseek.mdx
to mark `deepseek-chat` and `deepseek-reasoner` as deprecated and being retired
on July 24, 2026, and explicitly instruct users to migrate to
`deepseek-v4-flash` and `deepseek-v4-pro`; ensure the sentence replaces the
alias wording and includes the retirement date and recommended replacements.
---
Nitpick comments:
In `@docs/providers/supported-providers/deepseek.mdx`:
- Around line 112-130: Update the config.json example in the config.json Tab of
the deepseek.mdx docs to include a network_config example that shows overriding
network_config.base_url (e.g., pointing to https://api.deepseek.com/beta) so
users see how to target the beta endpoint; if network_config is not part of the
real schema, include it as a commented-out example or annotated snippet instead
and add a brief inline note. Reference the existing config.json Tab and the
network_config.base_url field so the change is made alongside the
"providers.deepseek" api_keys example.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 34c4bfa6-f95e-42ae-950e-9250ee3c10d9
📒 Files selected for processing (3)
.github/workflows/release-pipeline.ymlcore/providers/deepseek/deepseek_test.godocs/providers/supported-providers/deepseek.mdx
🚧 Files skipped from review as they are similar to previous changes (2)
- core/providers/deepseek/deepseek_test.go
- .github/workflows/release-pipeline.yml
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
etnperlong
force-pushed
the
feat/deepseek-provider
branch
from
May 7, 2026 12:02
1848aa4 to
836ccb0
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/providers/supported-providers/deepseek.mdx`:
- Around line 44-50: Add a single clarifying sentence in the DeepSeek provider
docs that explicitly distinguishes DeepSeek's upstream root-based endpoints
(e.g., "/chat/completions" and "/models") from the Bifrost gateway paths (which
use the "/v1/..." prefix such as "/v1/chat/completions"), so readers understand
the examples under "API" refer to the Bifrost gateway while the listed DeepSeek
paths are the upstream provider paths.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 9dbfe00f-28b6-404e-ad5c-e7fc1a5bf5cb
📒 Files selected for processing (18)
.github/workflows/pr-tests.yml.github/workflows/release-pipeline.ymlcore/bifrost.gocore/providers/deepseek/cachedcontents.gocore/providers/deepseek/deepseek.gocore/providers/deepseek/deepseek_test.gocore/schemas/bifrost.godocs/deployment-guides/config-json/schema-reference.mdxdocs/docs.jsondocs/openapi/openapi.jsondocs/openapi/schemas/inference/common.yamldocs/overview.mdxdocs/providers/supported-providers/deepseek.mdxdocs/providers/supported-providers/overview.mdxtransports/config.schema.jsonui/lib/constants/config.tsui/lib/constants/icons.tsxui/lib/constants/logs.ts
✅ Files skipped from review due to trivial changes (4)
- docs/docs.json
- core/providers/deepseek/cachedcontents.go
- docs/openapi/schemas/inference/common.yaml
- core/bifrost.go
🚧 Files skipped from review as they are similar to previous changes (12)
- docs/providers/supported-providers/overview.mdx
- docs/deployment-guides/config-json/schema-reference.mdx
- docs/openapi/openapi.json
- .github/workflows/pr-tests.yml
- transports/config.schema.json
- ui/lib/constants/config.ts
- docs/overview.mdx
- ui/lib/constants/logs.ts
- core/schemas/bifrost.go
- core/providers/deepseek/deepseek_test.go
- .github/workflows/release-pipeline.yml
- core/providers/deepseek/deepseek.go
etnperlong
force-pushed
the
feat/deepseek-provider
branch
from
May 8, 2026 00:49
836ccb0 to
762453d
Compare
etnperlong
force-pushed
the
feat/deepseek-provider
branch
2 times, most recently
from
May 13, 2026 18:02
79fa819 to
3b7c7d0
Compare
etnperlong
force-pushed
the
feat/deepseek-provider
branch
from
May 14, 2026 01:16
3b7c7d0 to
028fb4c
Compare
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.github/workflows/pr-tests.yml (1)
66-69:⚠️ Potential issue | 🟠 Major | ⚖️ Poor tradeoffUpgrade pr-tests.yml security hardening to match other workflows.
The pr-tests.yml workflow calls 20+ external AI provider APIs (including Deepseek, Anthropic, Gemini, OpenAI, etc.) but uses
egress-policy: auditinstead of the stricteregress-policy: blockimplemented in release-pipeline.yml and other workflows. The stack context description claiming allowlisting is inaccurate—pr-tests.yml currently logs but does not enforce endpoint restrictions.For proper security hardening, switch to
egress-policy: blockand configureallowed-endpointsto include all provider APIs being called, following the pattern already established in release-pipeline.yml (which allowlists api.deepseek.com:443 and other provider endpoints). This would prevent unintended outbound calls while allowing the provider tests to function correctly.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/pr-tests.yml around lines 66 - 69, Update the harden-runner step that uses step-security/harden-runner@fa2e9d605c4eeb9f... to switch egress-policy from "audit" to "block" and add an allowed-endpoints list that enumerates every external AI provider domain used by the pr-tests workflow (e.g., api.deepseek.com:443, api.openai.com:443, api.anthropic.com:443, etc.) following the pattern used in release-pipeline.yml; ensure the same step name ("Harden the runner (Audit all outbound calls)" or its usage block) is updated to include the explicit allowed-endpoints entries so tests can reach only those endpoints while all other outbound calls are blocked.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/workflows/pr-tests.yml:
- Around line 66-69: Update the harden-runner step that uses
step-security/harden-runner@fa2e9d605c4eeb9f... to switch egress-policy from
"audit" to "block" and add an allowed-endpoints list that enumerates every
external AI provider domain used by the pr-tests workflow (e.g.,
api.deepseek.com:443, api.openai.com:443, api.anthropic.com:443, etc.) following
the pattern used in release-pipeline.yml; ensure the same step name ("Harden the
runner (Audit all outbound calls)" or its usage block) is updated to include the
explicit allowed-endpoints entries so tests can reach only those endpoints while
all other outbound calls are blocked.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2a043888-5671-4c46-96c5-b2d2aa6d7671
📒 Files selected for processing (18)
.github/workflows/pr-tests.yml.github/workflows/release-pipeline.ymlcore/bifrost.gocore/providers/deepseek/cachedcontents.gocore/providers/deepseek/deepseek.gocore/providers/deepseek/deepseek_test.gocore/schemas/bifrost.godocs/deployment-guides/config-json/schema-reference.mdxdocs/docs.jsondocs/openapi/openapi.jsondocs/openapi/schemas/inference/common.yamldocs/overview.mdxdocs/providers/supported-providers/deepseek.mdxdocs/providers/supported-providers/overview.mdxtransports/config.schema.jsonui/lib/constants/config.tsui/lib/constants/icons.tsxui/lib/constants/logs.ts
✅ Files skipped from review due to trivial changes (5)
- docs/openapi/schemas/inference/common.yaml
- docs/docs.json
- docs/deployment-guides/config-json/schema-reference.mdx
- docs/overview.mdx
- docs/providers/supported-providers/deepseek.mdx
🚧 Files skipped from review as they are similar to previous changes (12)
- transports/config.schema.json
- core/schemas/bifrost.go
- core/bifrost.go
- docs/openapi/openapi.json
- docs/providers/supported-providers/overview.mdx
- ui/lib/constants/icons.tsx
- core/providers/deepseek/cachedcontents.go
- .github/workflows/release-pipeline.yml
- ui/lib/constants/logs.ts
- ui/lib/constants/config.ts
- core/providers/deepseek/deepseek_test.go
- core/providers/deepseek/deepseek.go
etnperlong
force-pushed
the
feat/deepseek-provider
branch
from
May 15, 2026 12:11
028fb4c to
b69a271
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/providers/supported-providers/deepseek.mdx`:
- Around line 114-123: The example JSON for providers.deepseek.keys[] is invalid
because each key object lacks the required name and weight fields; update the
key objects under "providers": {"deepseek": {"keys": [...]}} so each entry
includes a "name" (e.g., "default" or descriptive name) and a numeric "weight"
alongside the existing "value" (env.DEEPSEEK_API_KEY) to conform to
transports/config.schema.json validation; ensure the final config.json example
in docs/providers/supported-providers/deepseek.mdx passes schema validation and
remains present in the MDX Web UI / API / config.json tabs.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: a8570ee3-7179-4328-a43e-f279384dfabf
📒 Files selected for processing (18)
.github/workflows/pr-tests.yml.github/workflows/release-pipeline.ymlcore/bifrost.gocore/providers/deepseek/cachedcontents.gocore/providers/deepseek/deepseek.gocore/providers/deepseek/deepseek_test.gocore/schemas/bifrost.godocs/deployment-guides/config-json/schema-reference.mdxdocs/docs.jsondocs/openapi/openapi.jsondocs/openapi/schemas/inference/common.yamldocs/overview.mdxdocs/providers/supported-providers/deepseek.mdxdocs/providers/supported-providers/overview.mdxtransports/config.schema.jsonui/lib/constants/config.tsui/lib/constants/icons.tsxui/lib/constants/logs.ts
✅ Files skipped from review due to trivial changes (4)
- .github/workflows/pr-tests.yml
- docs/openapi/openapi.json
- docs/overview.mdx
- docs/docs.json
🚧 Files skipped from review as they are similar to previous changes (12)
- docs/deployment-guides/config-json/schema-reference.mdx
- docs/openapi/schemas/inference/common.yaml
- docs/providers/supported-providers/overview.mdx
- core/schemas/bifrost.go
- core/bifrost.go
- ui/lib/constants/config.ts
- core/providers/deepseek/deepseek_test.go
- ui/lib/constants/icons.tsx
- ui/lib/constants/logs.ts
- core/providers/deepseek/deepseek.go
- core/providers/deepseek/cachedcontents.go
- .github/workflows/release-pipeline.yml
Comment on lines
+114
to
+123
| ```json | ||
| { | ||
| "providers": { | ||
| "deepseek": { | ||
| "keys": [ | ||
| { | ||
| "value": "env.DEEPSEEK_API_KEY" | ||
| } | ||
| ] | ||
| } |
Contributor
There was a problem hiding this comment.
Config example is schema-invalid for providers.deepseek.keys[].
At Line 118-Line 121, each key object is missing required name and weight, so this example won’t validate against transports/config.schema.json.
Suggested fix
"keys": [
{
+ "name": "deepseek-primary",
"value": "env.DEEPSEEK_API_KEY"
+ ,
+ "weight": 1
}
]As per coding guidelines docs/**/*.mdx: "Mintlify MDX documentation must have Web UI / API / config.json tabs; validate config.json examples against transports/config.schema.json".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/providers/supported-providers/deepseek.mdx` around lines 114 - 123, The
example JSON for providers.deepseek.keys[] is invalid because each key object
lacks the required name and weight fields; update the key objects under
"providers": {"deepseek": {"keys": [...]}} so each entry includes a "name"
(e.g., "default" or descriptive name) and a numeric "weight" alongside the
existing "value" (env.DEEPSEEK_API_KEY) to conform to
transports/config.schema.json validation; ensure the final config.json example
in docs/providers/supported-providers/deepseek.mdx passes schema validation and
remains present in the MDX Web UI / API / config.json tabs.
akshaydeo
reviewed
May 15, 2026
| @@ -0,0 +1,33 @@ | |||
| package deepseek | |||
Contributor
There was a problem hiding this comment.
we usually split these functions into multiple files and this file only contains the main implementations
could we do the same here please
Contributor
|
Hi @etnperlong 👋 — thanks for this PR, the implementation looks nearly complete and the review feedback so far has been well addressed. Since there's been no activity for about a month and this PR is currently blocked, I'd love to help get it over the finish line — I'm using Bifrost in my own project and would really like DeepSeek support. Are you still planning to continue with it? If so, happy to step back; if you're short on time, I'm willing to take over and finish the remaining items. @akshaydeo in case there's no response from the author in a week or so, would you be open to a fresh PR superseding this one (with credit to @etnperlong)? The CLA here is unsigned, so I assume this branch can't be merged as-is either way. |
etnperlong
force-pushed
the
feat/deepseek-provider
branch
from
June 11, 2026 16:31
b69a271 to
3c56a2d
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 15
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
core/providers/anthropic/responses.go (1)
1241-1276:⚠️ Potential issue | 🟠 Major | ⚡ Quick winPersist the synthesized structured-output message before returning.
This branch returns before the generic persistence path on Lines 1325-1360, so
response.completedis emitted without the assistant output for structured-output turns. Store a cloned copy ofiteminstate.OutputItems[outputIndex]before returning.Suggested fix
item := &schemas.ResponsesMessage{ Type: schemas.Ptr(schemas.ResponsesMessageTypeMessage), Role: schemas.Ptr(schemas.ResponsesInputMessageRoleAssistant), Status: schemas.Ptr("completed"), Content: &schemas.ResponsesMessageContent{ ContentBlocks: []schemas.ResponsesMessageContentBlock{contentBlock}, }, } if itemID != "" { item.ID = &itemID } + + cloned := *item + clonedContent := *item.Content + cloned.Content = &clonedContent + state.OutputItems[outputIndex] = &cloned // Emit output_item.added for the text message responses = append(responses, &schemas.BifrostResponsesStreamResponse{🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/providers/anthropic/responses.go` around lines 1241 - 1276, The branch that emits ResponsesStreamResponseTypeOutputItemAdded/Done returns without persisting the structured-output message; before clearing buffers and returning from the function, clone the constructed item and assign it into state.OutputItems at the outputIndex (e.g., state.OutputItems[outputIndex] = clonedItem) so the synthesized structured-output is persisted; reference the local item variable, state.OutputItems, outputIndex, state.ToolArgumentBuffers, state.StructuredOutputIndex, and state.UsedStructuredOutputTool when making the change.core/providers/anthropic/text.go (1)
52-61:⚠️ Potential issue | 🟠 Major | ⚡ Quick winPreserve the Anthropic default provider when reconstructing the Bifrost request.
Line 57 drops the ctx-derived default provider. For the common bare Anthropic model IDs,
ParseModelString(req.Model, "")leavesproviderempty, so this round-trip loses the Anthropic routing hint and can send the reconstructed request down the wrong provider-selection path.Suggested fix
- provider, model := schemas.ParseModelString(req.Model, "") + provider, model := schemas.ParseModelString( + req.Model, + providerUtils.CheckAndSetDefaultProvider(ctx, schemas.Anthropic), + )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/providers/anthropic/text.go` around lines 52 - 61, In ToBifrostTextCompletionRequest, when ParseModelString(req.Model, "") yields an empty provider, preserve the Anthropic default provider from the passed ctx instead of dropping it; update the construction of the schemas.BifrostTextCompletionRequest so Provider is set to the parsed provider when non-empty, otherwise fallback to ctx's default provider (or the Anthropic-specific default available on ctx), ensuring ToBifrostTextCompletionRequest, ParseModelString, and schemas.BifrostTextCompletionRequest use the ctx-derived provider when needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/release-pipeline.yml:
- Around line 1808-1809: The checkout steps in the docker-manifest and
docker-manifest-ubi9 jobs leave the GitHub token available; modify the
actions/checkout usage in both jobs to include persist-credentials: false (i.e.,
change the checkout step for those jobs to use: actions/checkout@... with
persist-credentials: false under its with: block) so credentials are not
persisted for subsequent steps.
In `@cli/changelog.md`:
- Around line 1-12: Add a top-level Markdown heading at the start of
cli/changelog.md (e.g., "Changelog" or similar) and ensure the file ends with a
final newline to satisfy markdownlint rules MD041 and MD047; update the existing
content block (the diff listing of feat/improvement/fix lines) to follow the new
heading so the document begins with a single top-level heading and ends with a
newline.
In `@community/mcp-library/servers.json`:
- Around line 2020-2037: The "Quo" server entry uses auth_type "headers" but
omits the required_header_keys array, so update the Quo object in
community/mcp-library/servers.json to include a required_header_keys field
listing the header names Quo expects (e.g., "Authorization" or the
vendor-specific header) so framework/modelcatalog/mcp_library_sync.go can
persist RequiredHeaderKeys; consult Quo API docs for the exact header key(s) and
add them to the Quo JSON entry (keep the key name as required_header_keys and
supply an array of strings).
- Around line 1805-1821: The Similarweb catalog entry has auth_type set to
"headers" but is missing the required_header_keys array; update the Similarweb
JSON object (the entry with "name": "Similarweb" and "connection_url":
"https://mcp.similarweb.com") to include a "required_header_keys" field listing
the exact header names clients must provide (e.g., ["x-api-key"] or the
appropriate header names for Similarweb), so the downstream unmarshalling
(RequiredHeaderKeys in framework/modelcatalog/mcp_library_sync.go) can validate
header-based auth correctly.
In `@community/README.md`:
- Around line 13-25: The fenced ASCII flow-diagram block (the diagram beginning
with "┌──────────────┐ PR" and ending with "└──────────────┘") is missing a
language hint and triggers markdownlint MD040; add a language specifier (e.g.,
change the opening ``` to ```text or ```plaintext) so the block is treated as
plain text and MD040 is resolved—update the README.md fenced block accordingly.
In `@core/bifrost.go`:
- Around line 2171-2172: The guard that rejects fileless uploads checks
req.Provider directly and blocks custom providers whose BaseProviderType is
Vertex; update the logic to resolve the effective base provider (use
createBaseProvider or inspect CustomProviderConfig.BaseProviderType) before
enforcing the file-required check so that providers whose resolved base is
schemas.Vertex are exempted. Specifically, change the check around req.Provider
at the start of the upload path to compute the base provider type (via
createBaseProvider or the provider config) and compare that against
schemas.Vertex, leaving the existing BifrostError and fileless/GCS handling
intact.
- Around line 4852-4858: The successful streaming fallback path returns the raw
channel from bifrost.tryStreamRequest without setting Primary/IsFallback on
emitted chunks, so consumers lose fallback routing info; modify the success
branch that handles the chan returned by tryStreamRequest to wrap that channel
with a forwarding goroutine or helper that intercepts each emitted chunk and
calls SetFallbackRoutingInfo(provider, model) (or otherwise sets
Primary/IsFallback on RoutingInfo) before sending it onward, mirroring what the
error path does; locate the streaming success return site in bifrost.go where
tryStreamRequest is used and implement the channel wrapper so all stream chunks
carry the same fallback metadata as non-stream responses (see
SetFallbackRoutingInfo doc for intended behavior).
- Around line 7588-7591: The early returns that directly return a schemas.Key
from ctx.Value(schemas.BifrostContextKeyDirectKey) bypass validateKey and
central setup; instead, extract the key from BifrostContextKeyDirectKey then
pass it through validateKey (the same validation used by
selectKeyFromProviderForModel / getKeysForBatchAndFileOps), handle any
validation error, and return the validated/normalized key(s); apply the same
change to the other occurrences mentioned (around lines 7632-7635 and 7717-7721)
so provider-specific configuration is initialized consistently before returning.
- Around line 5452-5466: The defer early-return prevents terminal routing logs
on first-attempt completions because attempts is 0; remove the guard "if
attempts <= 0 { return }" (or change it to only skip when attempts < 0 if
negative is meaningful) so the defer block always appends the routing engine to
context and emits the final ctx.AppendRoutingEngineLog lines using bifrostError,
providerKey, model and attempts.
In `@core/providers/anthropic/chat.go`:
- Around line 1235-1240: In the AnthropicStreamDeltaTypeInputJSON case, avoid
defaulting to toolCallIdx 0 by guarding access to
state.contentBlockToToolCallIdx using the chunk.Index key: check that
chunk.Delta.PartialJSON is non-nil and that the mapping for *chunk.Index
actually exists (or that the index is within bounds) before dereferencing/using
toolCallIdx; if the mapping is missing, skip handling this input_json delta so
you don't emit arguments for a non-existent tool call (references:
AnthropicStreamDeltaTypeInputJSON case, chunk.Delta.PartialJSON, chunk.Index,
state.contentBlockToToolCallIdx, toolCallIdx).
- Around line 388-397: The code forces anthropicReq.ToolChoice when
thinkingEnabled is false, but adaptive-only models should be considered
"thinking" even when reasoning params are omitted; update the thinkingEnabled
calculation to also return true for adaptive-only models (e.g., add a check like
isAdaptiveOnlyModel(bifrostReq.Model) or use the existing model capability flag)
so that AnthropicToolChoice is not set for adaptive-only models without explicit
reasoning; modify the logic around thinkingEnabled (used where
bifrostReq.Params.Reasoning is inspected and before assigning
anthropicReq.ToolChoice) to include that adaptive-only check.
In `@core/providers/anthropic/responses.go`:
- Around line 1164-1179: The finalization paths currently build
ResponsesOutputMessageContentText with empty Annotations, dropping streamed
annotation events; modify the code that constructs the part (variable part /
ResponsesOutputMessageContentText) and the later output_item.done /
response.completed blocks (around the other section) to pull the accumulated
annotations for the given outputIndex (e.g. from a map like
annotationsByOutputIndex[outputIndex] or similar accumulator used when handling
output_text.annotation.added events) and set that slice into
ResponsesOutputMessageContentText.Annotations instead of creating an empty
slice; keep existing LogProbs handling and ensure you attach the correct
annotations when creating partDoneResponse and the final output item/response
objects.
- Around line 2214-2221: Normalize the incoming user ID
(strings.TrimSpace(*req.Metadata.UserID)) and always replace it with its SHA-256
hex digest before assigning to params.PromptCacheKey so short IDs can't collide
with preimage-looking hex strings; specifically, in the block that reads
req.Metadata.UserID compute sum := fmt.Sprintf("%x", sha256.Sum256([]byte(key)))
and set params.PromptCacheKey = &sum (instead of only hashing when len(key) >
64), preserving the normalized value-to-hash flow and nil-checking
req.Metadata.UserID as currently done.
In `@core/providers/anthropic/types.go`:
- Around line 807-812: Update the comment on the AnthropicMessage.Role field to
clarify that the "system" role (AnthropicMessageRoleSystem) refers to
mid-conversation system messages supported only by certain models, not the
top-level initial system prompt; change the comment on the AnthropicMessage
struct's Role field to explicitly state that role:"system" is model-dependent
and only emitted when midConvSystemSupported is true (see responses.go behavior)
so readers/tests understand the distinction.
In `@core/providers/anthropic/utils.go`:
- Around line 102-112: The chat/raw Anthropic tool gate is out of sync with
ValidateToolsForProvider: update anthropicToolTypePrefixToFeature (used by
ValidateChatToolsForProvider) to include the new Nova capability flags
(features.WebSearchNova and features.CodeExecNova) when mapping tool prefixes to
supported features so that web-search and code-exec prefixes are allowed when
either the original or Nova flag is set; adjust the mapping/conditional checks
in anthropicToolTypePrefixToFeature to check both features.WebSearch ||
features.WebSearchNova and features.CodeExecution || features.CodeExecNova (and
preserve existing logic for other prefixes) so the chat-path mirrors the
Responses-path behavior.
---
Outside diff comments:
In `@core/providers/anthropic/responses.go`:
- Around line 1241-1276: The branch that emits
ResponsesStreamResponseTypeOutputItemAdded/Done returns without persisting the
structured-output message; before clearing buffers and returning from the
function, clone the constructed item and assign it into state.OutputItems at the
outputIndex (e.g., state.OutputItems[outputIndex] = clonedItem) so the
synthesized structured-output is persisted; reference the local item variable,
state.OutputItems, outputIndex, state.ToolArgumentBuffers,
state.StructuredOutputIndex, and state.UsedStructuredOutputTool when making the
change.
In `@core/providers/anthropic/text.go`:
- Around line 52-61: In ToBifrostTextCompletionRequest, when
ParseModelString(req.Model, "") yields an empty provider, preserve the Anthropic
default provider from the passed ctx instead of dropping it; update the
construction of the schemas.BifrostTextCompletionRequest so Provider is set to
the parsed provider when non-empty, otherwise fallback to ctx's default provider
(or the Anthropic-specific default available on ctx), ensuring
ToBifrostTextCompletionRequest, ParseModelString, and
schemas.BifrostTextCompletionRequest use the ctx-derived provider when needed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 0fd97333-1db8-43d5-83ab-1176e7fc03fd
⛔ Files ignored due to path filters (1)
core/go.sumis excluded by!**/*.sum
📒 Files selected for processing (111)
.github/ISSUE_TEMPLATE/mcp_library_submission.yml.github/dependabot.yml.github/workflows/pr-tests.yml.github/workflows/release-pipeline.yml.github/workflows/scripts/run-migration-tests.sh.github/workflows/scripts/validate-helm-config-fields.sh.gitignoreMakefilecli/changelog.mdcommunity/README.mdcommunity/mcp-library/README.mdcommunity/mcp-library/schema.jsoncommunity/mcp-library/servers.jsoncore/bifrost.gocore/bifrost_test.gocore/go.modcore/internal/llmtests/account.gocore/internal/llmtests/batch.gocore/mcp/healthmonitor.gocore/providers/anthropic/chat.gocore/providers/anthropic/models.gocore/providers/anthropic/passthrough_usage.gocore/providers/anthropic/responses.gocore/providers/anthropic/text.gocore/providers/anthropic/types.gocore/providers/anthropic/utils.gocore/providers/anthropic/utils_test.gocore/providers/azure/azure.gocore/providers/azure/azure_passthrough_test.gocore/providers/azure/models.gocore/providers/azure/utils.gocore/providers/bedrock/bedrock.gocore/providers/bedrock/chat.gocore/providers/bedrock/embedding.gocore/providers/bedrock/invoke.gocore/providers/bedrock/mantle.gocore/providers/bedrock/models.gocore/providers/bedrock/region_test.gocore/providers/bedrock/rerank.gocore/providers/bedrock/rerank_test.gocore/providers/bedrock/responses.gocore/providers/bedrock/text.gocore/providers/bedrock/usage_headers_test.gocore/providers/bedrock/utils.gocore/providers/cohere/chat.gocore/providers/cohere/count_tokens.gocore/providers/cohere/embedding.gocore/providers/cohere/models.gocore/providers/cohere/rerank.gocore/providers/deepseek/cachedcontents.gocore/providers/deepseek/deepseek.gocore/providers/deepseek/deepseek_test.gocore/providers/elevenlabs/models.gocore/providers/gemini/batch.gocore/providers/gemini/embedding.gocore/providers/gemini/gemini.gocore/providers/gemini/gemini_test.gocore/providers/gemini/images.gocore/providers/gemini/models.gocore/providers/gemini/responses.gocore/providers/gemini/speech.gocore/providers/gemini/transcription.gocore/providers/gemini/types.gocore/providers/gemini/videos.gocore/providers/huggingface/models.gocore/providers/mistral/models.gocore/providers/openai/chat.gocore/providers/openai/chat_test.gocore/providers/openai/embedding.gocore/providers/openai/images.gocore/providers/openai/models.gocore/providers/openai/responses.gocore/providers/openai/speech.gocore/providers/openai/text.gocore/providers/openai/transcription.gocore/providers/openai/types.gocore/providers/openai/videos.gocore/providers/openrouter/openrouter.gocore/providers/replicate/models.gocore/providers/replicate/replicate.gocore/providers/replicate/replicate_test.gocore/providers/replicate/use_deployments_endpoint_test.gocore/providers/utils/models.gocore/providers/utils/utils.gocore/providers/utils/utils_test.gocore/providers/vertex/batch.gocore/providers/vertex/cachedcontents.gocore/providers/vertex/models.gocore/providers/vertex/rerank.gocore/providers/vertex/types.gocore/providers/vertex/utils.gocore/providers/vertex/utils_test.gocore/providers/vertex/vertex.gocore/providers/vertex/vertex_test.gocore/schemas/account.gocore/schemas/account_test.gocore/schemas/batch.gocore/schemas/bifrost.gocore/schemas/chatcompletions.gocore/schemas/context.gocore/schemas/files.gocore/schemas/images.gocore/schemas/models.gocore/schemas/passthrough.gocore/schemas/plugin.gocore/schemas/responses.gocore/schemas/span_filter.gocore/schemas/span_filter_test.gocore/schemas/utils.gocore/utils.godocs/architecture/core/plugins.mdx
💤 Files with no reviewable changes (85)
- core/providers/gemini/transcription.go
- core/providers/azure/models.go
- core/providers/openai/transcription.go
- core/providers/bedrock/usage_headers_test.go
- core/schemas/models.go
- core/schemas/files.go
- core/providers/cohere/models.go
- core/providers/gemini/models.go
- core/schemas/images.go
- core/providers/gemini/speech.go
- core/providers/bedrock/models.go
- core/providers/huggingface/models.go
- core/providers/mistral/models.go
- core/providers/openai/models.go
- core/providers/elevenlabs/models.go
- core/providers/replicate/use_deployments_endpoint_test.go
- core/providers/gemini/gemini.go
- core/schemas/chatcompletions.go
- core/providers/replicate/models.go
- core/providers/replicate/replicate_test.go
- core/providers/cohere/chat.go
- core/providers/gemini/embedding.go
- core/providers/cohere/embedding.go
- core/schemas/responses.go
- core/schemas/passthrough.go
- core/providers/bedrock/rerank_test.go
- core/schemas/utils.go
- core/providers/openai/speech.go
- core/schemas/batch.go
- core/schemas/plugin.go
- core/providers/openrouter/openrouter.go
- core/providers/bedrock/chat.go
- core/schemas/span_filter.go
- core/providers/vertex/models.go
- core/schemas/context.go
- core/providers/openai/embedding.go
- core/providers/openai/images.go
- core/providers/gemini/videos.go
- core/providers/azure/utils.go
- core/providers/bedrock/mantle.go
- core/providers/vertex/utils.go
- core/providers/openai/text.go
- core/providers/openai/responses.go
- core/utils.go
- core/providers/openai/chat.go
- core/providers/utils/models.go
- core/providers/bedrock/invoke.go
- docs/architecture/core/plugins.mdx
- core/providers/gemini/images.go
- core/providers/vertex/cachedcontents.go
- core/providers/cohere/rerank.go
- core/providers/cohere/count_tokens.go
- core/providers/openai/chat_test.go
- core/providers/openai/types.go
- core/providers/bedrock/embedding.go
- core/providers/azure/azure_passthrough_test.go
- core/providers/vertex/utils_test.go
- core/schemas/account_test.go
- core/providers/deepseek/cachedcontents.go
- core/schemas/span_filter_test.go
- core/providers/bedrock/rerank.go
- core/providers/gemini/types.go
- core/providers/openai/videos.go
- core/providers/anthropic/utils_test.go
- core/providers/utils/utils.go
- core/providers/gemini/gemini_test.go
- core/providers/utils/utils_test.go
- core/providers/bedrock/region_test.go
- core/providers/bedrock/text.go
- core/schemas/account.go
- core/providers/vertex/vertex_test.go
- core/providers/vertex/batch.go
- core/providers/vertex/types.go
- core/providers/gemini/batch.go
- core/providers/gemini/responses.go
- core/providers/vertex/rerank.go
- core/providers/bedrock/responses.go
- core/providers/deepseek/deepseek_test.go
- core/providers/deepseek/deepseek.go
- core/providers/bedrock/utils.go
- core/providers/replicate/replicate.go
- core/providers/bedrock/bedrock.go
- core/schemas/bifrost.go
- core/providers/vertex/vertex.go
- core/providers/azure/azure.go
…hropicResponsesRequest` and make `completeRequest` a package-level func shared by Anthropic, Azure, and Bedrock providers (maximhq#4394) ## Summary The Anthropic provider's unary request logic was duplicated across the Anthropic, Azure, Bedrock, and Vertex providers. This PR extracts the core non-streaming request execution into a package-level `completeRequest` function and introduces two exported handler functions — `HandleAnthropicChatCompletionRequest` and `HandleAnthropicResponsesRequest` — that encapsulate the full build → send → parse pipeline for chat completions and the Responses API respectively. Azure and Bedrock now delegate directly to these shared handlers for Anthropic-family models instead of reimplementing request dispatch, response parsing, and raw request/response handling inline. A secondary bug fix is included: the large-response streaming client was being activated for count-tokens requests (which should always be buffered) and skipped for all other requests — the condition was inverted. ## Changes - Extracted `completeRequest` as a package-level function accepting explicit `client`, `headers`, `extraHeaders`, `betaHeaderOverrides`, `providerName`, and `logger` arguments, removing the method receiver dependency so it can be called by other providers. - Added `anthropicRequestHeaders` as a provider method to build the `x-api-key` / `anthropic-version` header map, shared across `TextCompletion`, `ChatCompletion`, `Responses`, and `CountTokens`. - Introduced `HandleAnthropicChatCompletionRequest` and `HandleAnthropicResponsesRequest` as exported functions that perform the full unary request lifecycle (body build, HTTP send, large-response detection, response parse, raw request/response attachment). These are now called by the Anthropic, Azure, and Bedrock providers. - Removed `completeMantleRequest` from Bedrock — its logic is now covered by `completeRequest` inside the shared handlers. - Azure's `ChatCompletion` and `Responses` methods now branch early for Anthropic-family models, calling the shared handlers with Azure-specific auth headers, and fall through to the OpenAI-compatible path otherwise, eliminating the post-response model-family branch. - Fixed the inverted condition in `completeRequest` that caused the large-response streaming client to be used for count-tokens requests instead of being skipped for them. - `AnthropicRequestBuildConfig` now carries `BetaHeaderOverrides` so callers do not need to pass it separately. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` Validate that chat completions and Responses API requests succeed for Anthropic-family models routed through the Azure and Bedrock providers, and that count-tokens requests return buffered responses without triggering large-response mode. ## Breaking changes - [ ] Yes - [x] No ## Security considerations Auth headers (`x-api-key`, Bearer tokens, SigV4-signed headers) are applied last in `completeRequest`, after network-config extra headers, ensuring they cannot be overridden by user-supplied configuration. No new secrets or PII handling paths are introduced. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…atible providers from `schemas.Key` to `map[string]string` auth header param (maximhq#4425) ## Summary This PR standardizes how Bearer token authentication headers are constructed across all OpenAI-compatible providers. Previously, each call site independently built the `Authorization: Bearer <token>` header map with duplicated inline logic. A new `BearerAuthHeader(key)` helper is introduced in the OpenAI package and used uniformly everywhere. Additionally, the Azure provider's private `completeRequest` method is removed. Its non-Anthropic request paths (text completion, chat completion, responses, embedding, compaction) are now delegated directly to the shared `Handle*` functions in the OpenAI package, consistent with how other providers already work. The `Handle*` functions themselves are updated to accept a pre-built `authHeader map[string]string` instead of a raw `schemas.Key`, making them provider-agnostic and compatible with non-Bearer auth schemes (e.g., Azure API key headers, SigV4). ## Changes - Added `BearerAuthHeader(key schemas.Key) map[string]string` to the OpenAI provider package, which returns an `Authorization: Bearer <token>` header map, or an empty map when the key carries no value. - Updated all `Handle*Request` and `handleOpenAILargePayloadPassthrough` function signatures to accept `authHeader map[string]string` instead of `schemas.Key`, applying the map directly to request headers. - Replaced all inline `var authHeader map[string]string` + conditional assignment blocks across Cerebras, Fireworks, Groq, HuggingFace, Mistral, Nebius, Ollama, Opencode, OpenRouter, Parasail, Perplexity, SGL, VLLM, xAI, and Bedrock with calls to `openai.BearerAuthHeader(key)`. - Removed the Azure provider's `completeRequest` method and replaced its usage in `TextCompletion`, `ChatCompletion`, `Responses`, `Embedding`, and `Compaction` with direct calls to the corresponding shared OpenAI `Handle*` functions, passing Azure-specific auth headers and pre-resolved endpoint URLs. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` Verify that all OpenAI-compatible providers (OpenAI, Azure, Cerebras, Fireworks, Groq, HuggingFace, Mistral, Nebius, Ollama, Opencode, OpenRouter, Parasail, Perplexity, SGL, VLLM, xAI, Bedrock Mantle) continue to authenticate correctly and that requests succeed for text completion, chat completion, responses, embeddings, and compaction endpoints. ## Breaking changes - [ ] Yes - [x] No ## Security considerations The `BearerAuthHeader` helper preserves the existing behavior of omitting the `Authorization` header when the key value is empty, which is intentional for providers that supply auth via other mechanisms (e.g., extra headers or SigV4 signing). No secrets are logged or exposed. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… closure passed through request handlers (maximhq#4735) ## Summary Replaces the pre-build-and-sign approach for Bedrock Mantle SigV4 authentication with a `BodySigner` callback that is invoked after the request handler has marshaled the body. This ensures the signature always covers the exact bytes sent on the wire, eliminating the previous double-marshal pattern where the body was built once for signing and again inside the handler. ## Changes - Introduces a new `BodySigner` type (`func(jsonData []byte) (map[string]string, *schemas.BifrostError)`) in `core/providers/utils/bodysigner.go`. Handlers call it after building the request body and apply the returned headers to the outgoing request. - Adds the `signer` parameter to `HandleOpenAIChatCompletionRequest`, `HandleOpenAIChatCompletionStreaming`, `HandleOpenAIResponsesRequest`, `HandleOpenAIResponsesStreaming`, `HandleAnthropicChatCompletionRequest`, `HandleAnthropicChatCompletionStreaming`, `HandleAnthropicResponsesRequest`, and `HandleAnthropicResponsesStream`. All existing callers pass `nil`. - Rewrites Bedrock Mantle's SigV4 paths (`mantleChatCompletions`, `mantleChatCompletionsStream`, `mantleResponses`, `mantleResponsesStream`) to construct a `BodySigner` closure when no API key is present, instead of pre-building the body, signing it, and merging the signature headers into `extraHeaders`. The Bearer path no longer needs a separate early-return branch. - Removes the now-unnecessary `maps` import and the intermediate `extraHeaders` map copies in the Mantle code paths. ## Type of change - [ ] Bug fix - [x] Refactor - [ ] Feature - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` For Bedrock Mantle with SigV4 (empty key value), verify that requests to chat completions, streaming chat completions, responses, and streaming responses are signed correctly and accepted by the Bedrock endpoint. For Bearer key paths, confirm that no signing is attempted and the `Authorization` header is set as expected. ## Breaking changes - [ ] Yes - [x] No ## Security considerations The `BodySigner` callback signs the exact serialized bytes that are placed on the wire. Previously, the body was serialized twice (once for signing, once inside the handler), which could in theory produce a signature mismatch if marshaling were non-deterministic. This change closes that gap by signing after the final body is set. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ropic and OpenAI-compatible routing (maximhq#4736) ## Summary Introduces `bedrock_mantle` as a first-class, standalone provider that owns the Bedrock Mantle surface (`bedrock-mantle.{region}.api.aws`). Previously, Mantle routing was handled as an internal routing decision inside the existing `bedrock` provider. The new provider gives operators a dedicated configuration surface for Claude (native Anthropic Messages API), OpenAI-compatible models (gpt-*), and Gemma models served through Mantle, without requiring a full Bedrock setup. ## Changes - Added `schemas.BedrockMantle` (`"bedrock_mantle"`) as a new `ModelProvider` constant and registered it in `StandardProviders`, `dynamicallyConfigurableProviders`, `CanProviderKeyValueBeEmpty`, and `isKeySkippingAllowed`. - Added `BedrockMantleKeyConfig` to the `Key` struct, carrying AWS credentials and region for SigV4 auth against the `bedrock-mantle` service. The existing `BedrockKeyConfig` is unchanged. - Introduced the `core/providers/bedrockmantle` package implementing the full `Provider` interface. Chat, streaming chat, Responses, and streaming Responses dispatch by model family: Anthropic-family models use the native Anthropic Messages surface (`/anthropic/v1/messages`); all others use the OpenAI-compatible surface (`/v1` or `/openai/v1`). All other operations return unsupported-operation errors. - Refactored `signAWSRequest` in the `bedrock` package to accept a `*BedrockKeyConfig` instead of individual credential fields, eliminating the now-redundant `signAWSRequestFromKey` wrapper. All call sites updated accordingly. - Exported `SignMantleV4Headers` (previously `mantleSigV4Headers`, a method on `BedrockProvider`) so the new `bedrockmantle` package can sign requests without depending on the internal Bedrock provider struct. The function now supports both `BedrockKeyConfig` and `BedrockMantleKeyConfig` by mapping the latter into a synthetic `BedrockKeyConfig` for signing, and correctly handles GET requests (nil body) for the list-models path. - Extended the Anthropic chat and Responses request builders to convert native structured outputs to tool calls for `BedrockMantle`, matching the existing `Vertex` workaround. - Added `BedrockMantle` to the comprehensive LLM test harness (`ComprehensiveTestAccount`) with key config, provider config, and a full test file covering the supported scenarios (chat, streaming, tool calls, vision, structured outputs, prompt caching, reasoning, list models) and explicitly disabling unsupported ones. - Marked `isMantleModel` in `bedrock/mantle.go` as deprecated in favour of the new provider. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Set AWS credentials and run the new provider test: ```sh export AWS_ACCESS_KEY_ID=... export AWS_SECRET_ACCESS_KEY=... export AWS_SESSION_TOKEN=... # optional, for temporary credentials export AWS_REGION=us-east-1 go test ./core/providers/bedrockmantle/... -v -run TestBedrockMantle ``` To run the full suite (skips Bedrock Mantle automatically when credentials are absent): ```sh go test ./... ``` Configure a `bedrock_mantle` provider by supplying a `BedrockMantleKeyConfig` (or a Bearer API key in `Value`) with the desired region. The region can also be embedded as a prefix in the model ID (e.g. `us-west-2/anthropic.claude-haiku-4-5`) or set at the alias level via `AliasConfig.Region`. ## Breaking changes - [ ] Yes - [x] No The `signAWSRequest` signature change is internal to the `bedrock` package and does not affect any public API. The `isMantleModel` function is deprecated but not removed. ## Security considerations AWS credentials for `BedrockMantleKeyConfig` follow the same `SecretVar` resolution pattern used by `BedrockKeyConfig` (env-var references, never inlined literals). SigV4 signing is performed per-request on the exact body bytes that are sent, so the signature always covers what is transmitted. When a Bearer API key is present it takes precedence and no AWS credentials are required. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…on, and UI support (maximhq#4737) ## Summary Adds `bedrock_mantle` as a first-class provider, enabling Bifrost to route requests to AWS Bedrock through a Mantle proxy endpoint. The provider supports the same SigV4 credential options as the existing Bedrock provider (inherited IAM role, explicit access/secret key, session token, AssumeRole) as well as a Bearer API key authentication mode. ## Changes - Added `BedrockMantle` to the Anthropic passthrough allowlist in `clearAnthropicPassthroughForNonNativeProvider` so raw request bodies are preserved when routing through Bedrock Mantle. - Added `BedrockMantleKeyConfig` redaction logic in `clientconfig.go`, mirroring the existing Bedrock redaction pattern. - Added a new `migrationAddBedrockMantleKeyColumns` database migration that introduces seven `bedrock_mantle_*` SigV4 credential columns to the `config_keys` table. - Extended `TableKey` with the seven Bedrock Mantle credential fields, along with `BeforeSave` serialization and `AfterFind` reconstruction hooks. - Updated `mergeUpdatedKey` in the HTTP handler to correctly restore redacted Bedrock Mantle credential fields during key updates. - Fixed `isClaudeModel` in the Anthropic integration to recognize `bedrock_mantle` (previously incorrectly matched `bedrock`) as a provider that can serve Claude models. - Included `BedrockMantleKeyConfig` in the key hash inputs used by `mergeProviderKeys` and `reconcileProviderKeys` for config file/DB reconciliation. - Added Bedrock Mantle credential redaction to `GetAllKeys`. - Extended `config.schema.json` with `bedrock_mantle_key` and `provider_with_bedrock_mantle_config` definitions and registered `bedrock_mantle` as a valid provider name throughout the schema. - Added UI support: provider icon (reusing the Bedrock SVG mark with a distinct gradient ID), model placeholder text, `isKeyRequiredByProvider` entry, label, form schema (`BedrockMantleKeyConfigSchema`), type definitions (`BedrockMantleKeyConfig`, `DefaultBedrockMantleKeyConfig`), and a full authentication method tab UI (IAM Role / Explicit Credentials / API Key) matching the Bedrock provider UX. - Added `bedrock_mantle` to the Anthropic beta-headers provider family and the provider config sheet's Anthropic family list. - Stripped the internal `_auth_type` field from `bedrock_mantle_key_config` before submitting the form payload. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Configure a `bedrock_mantle` provider in `config.json` or via the UI with one of the three auth methods: - **IAM Role (Inherited):** set only `region`; leave access/secret key empty. - **Explicit Credentials:** set `access_key`, `secret_key`, and `region`; optionally set `session_token`, `role_arn`, `external_id`, and `session_name`. - **API Key:** set `region` and provide a Bearer token as the key `value`. Send a request targeting a Claude model through the `bedrock_mantle` provider and verify the response is returned correctly and that credentials are redacted in the UI and API responses. ## Screenshots/Recordings _Add before/after screenshots of the new Bedrock Mantle provider form and icon in the UI._ ## Breaking changes - [ ] Yes - [x] No ## Related issues _Link related issues and discussions._ ## Security considerations - All seven Bedrock Mantle credential fields (`access_key`, `secret_key`, `session_token`, `region`, `role_arn`, `external_id`, `role_session_name`) are stored as `SecretVar` and are redacted in API responses and the UI, consistent with the existing Bedrock provider handling. - The `_auth_type` discriminator field is stripped from the payload before it is persisted or transmitted. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
Adds documentation for the AWS Bedrock Mantle provider, a distinct AWS endpoint (`bedrock-mantle.{region}.api.aws`) that exposes Claude models via the native Anthropic Messages API and OpenAI-family/Gemma models via an OpenAI-compatible API — all addressable through a single `bedrock_mantle/<model>` prefix in Bifrost.
## Changes
- Added a new `bedrock-mantle.mdx` provider page covering model ID formats, supported operations, all three authentication modes (SigV4 with explicit credentials, IAM role/inherited credentials, and Bearer API key), IAM role assumption via `role_arn`, and usage examples.
- Added the Bedrock Mantle configuration block to the `providers.mdx` config reference, with tabs for Static Credentials, IAM Role, and API Key (Bearer) auth modes.
- Added Bedrock Mantle to the provider capability matrix in `overview.mdx`.
- Registered `bedrock-mantle` in `docs.json` so it appears in the sidebar navigation.
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [x] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [x] Docs
## How to test
Navigate to the Bedrock Mantle provider page and config reference in the rendered docs and verify:
- The sidebar entry for `bedrock-mantle` appears between `bedrock` and `cerebras`.
- All three auth tabs (Static Credentials, IAM Role, API Key) render correctly in both the provider page and the config reference.
- The capability matrix row for `bedrock_mantle/<model>` is present and accurate.
- Cross-links between the provider page and the config reference resolve correctly.
## Breaking changes
- [x] No
## Security considerations
Authentication credentials (`access_key`, `secret_key`, `session_token`, API keys) are documented using the `env.*` indirection pattern, consistent with how other providers handle secrets. No credentials are hardcoded in examples.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
…ment config (maximhq#4739) Adds E2E test configuration and capability definitions for the `bedrock_mantle` provider, enabling it to be tested through the Bifrost V1 API test suite. - Added `bedrock_mantle` to `provider-capabilities.json` with `chat_completions`, `chat_completions_with_tools`, `responses`, `responses_with_tools`, and `list_models` enabled - Added a new Postman environment file (`bifrost-v1-bedrock-mantle.postman_environment.json`) configured to use `anthropic.claude-opus-4-8` as the default model and `us-east-1` as the default region, with secret placeholders for API key, access key, secret key, and session token - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs Run the E2E test suite targeting the `bedrock_mantle` provider using the new Postman environment: ```sh newman run tests/e2e/api/bifrost-v1.postman_collection.json \ -e tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json \ --env-var "bedrock_mantle_api_key=<your_api_key>" \ --env-var "bedrock_mantle_access_key=<your_access_key>" \ --env-var "bedrock_mantle_secret_key=<your_secret_key>" ``` Expected outcome: chat completions, tool-use, responses, and model listing tests pass; all unsupported capability tests are skipped or return expected errors. N/A - [ ] Yes - [x] No N/A The Postman environment file stores API key, access key, secret key, and session token as `secret` type fields with empty default values, ensuring credentials are not committed to the repository. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…d toggle when budget is alignable (maximhq#4724) ## Summary The calendar alignment toggle in the provider governance form was previously shown whenever any budget existed. This PR restricts its visibility and submission to only when at least one budget uses a calendar-alignable reset period (day, week, month, or year). ## Changes - Introduced a `showCalendarAlignment` derived boolean that checks whether any configured budget has a reset duration supported by `supportsCalendarAlignment`. - Replaced the previous condition (`watchedBudgets.length > 0`) with `showCalendarAlignment` to control rendering of the calendar alignment toggle. - Updated the form submission payload so that `calendar_aligned` is only set to `true` when at least one budget actually supports calendar alignment — preventing the flag from being submitted for incompatible budget configurations. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Navigate to a provider's governance settings in the UI. 2. Add a budget with a reset duration that does **not** support calendar alignment (e.g., hourly). Verify the calendar alignment toggle does **not** appear. 3. Add or change a budget to use a calendar-alignable period (e.g., daily, weekly, monthly, yearly). Verify the toggle **does** appear. 4. Enable the toggle and save. Confirm `calendar_aligned: true` is included in the submitted payload. 5. Remove all calendar-alignable budgets and save. Confirm `calendar_aligned` is not set to `true` in the payload. ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` ## Screenshots/Recordings _Before:_ Calendar alignment toggle appears whenever any budget is present, regardless of reset period. _After:_ Calendar alignment toggle only appears when at least one budget uses a day/week/month/year reset period. ## Breaking changes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…hq#4725) ## Summary Fixes a bug where updating or deleting provider-level governance for a custom provider whose name contains a space (e.g. `"OpenRouter Base"`) would return a 404. The UI percent-encodes the provider name in the URL path (`OpenRouter%20Base`), but the handler was comparing the raw encoded string directly against the stored provider name, causing the lookup to fail. Closes maximhq#4689 ## Changes - `updateProviderGovernance` and `deleteProviderGovernance` now call `url.PathUnescape` on the `provider_name` path parameter before using it, matching the decoded name against what is stored in the config store. - Returns a `400` if the path parameter contains an invalid percent-encoding sequence. - Added a regression test (`TestProviderGovernance_DecodesEncodedProviderName`) that seeds a provider with a space in its name, issues a PUT and DELETE using the percent-encoded path param, and asserts both succeed and persist correctly. - Added a guard test (`TestProviderGovernance_UnknownProviderStill404`) to confirm that a genuinely unknown provider still returns 404 after the decode change. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./transports/bifrost-http/handlers/... -run TestProviderGovernance ``` Expected output: all three `TestProviderGovernance_*` tests pass. Specifically: - `TestProviderGovernance_DecodesEncodedProviderName` — PUT and DELETE with `OpenRouter%20Base` return `200`. - `TestProviderGovernance_UnknownProviderStill404` — PUT with an unknown encoded name returns `404`. ## Breaking changes - [ ] Yes - [x] No ## Related issues - Resolves maximhq#4689 ## Security considerations `url.PathUnescape` is used rather than `url.QueryUnescape` to correctly handle path-encoded characters. Invalid encoding sequences are rejected with a `400` rather than passed through, preventing malformed input from reaching the config store. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… (maximhq#4568) Signed-off-by: Akshay Deo <akshay@akshaydeo.com> Co-authored-by: Akshay Deo <akshay@akshaydeo.com>
* Fix mcp reconnect failure on startup * test: assert failed MCP client cleanup --------- Co-authored-by: Gowtham <692171+HackToHell@users.noreply.github.com>
…put items (maximhq#4121) * fix(responses): preserve codex tool_search_call/tool_search_output input items Bifrost's Responses input deserializer rejected codex's tool-search follow-up request with HTTP 400 "openai responses request input is neither a string nor an array of responses messages", which hung/failed the agent turn. The fix teaches ResponsesMessage about the two tool_search item types and round-trips them verbatim. Background, since tool_search is non-obvious: How codex's tool_search works (the path that hits this bug) ----------------------------------------------------------- codex normally sends every MCP tool inline in the request `tools[]` as `{type:"function", ...}`. But when a model's catalog has `supports_search_tool: true` AND the tool count crosses DIRECT_MCP_TOOL_EXPOSURE_THRESHOLD (= 100) — e.g. an agent wired to several MCP servers — codex stops sending them inline and "defers" them behind a discovery tool: should_defer = supports_search_tool && (ToolSearchAlwaysDeferMcpTools || n >= 100) The deferred flow is a two-request round-trip: 1. Request 1: codex hides the deferred tools and instead declares one tool: {"type":"tool_search","execution":"client","description":"...", "parameters":{query, limit}} `execution:"client"` means the model does NOT run the search — codex does. 2. The model emits a `tool_search_call` with `arguments` = {query, limit}. 3. codex runs the search CLIENT-SIDE: a BM25 index over the deferred tool metadata (codex's ToolSearchHandler, core/src/tools/handlers/tool_search.rs, using the `bm25` crate). It picks the top-N matching tools. 4. Request 2 (follow-up): codex appends two items to `input[]`: - {"type":"tool_search_call", "call_id":..., "execution":"client", "arguments":{...}} - {"type":"tool_search_output", "call_id":..., "status":"completed", "execution":"client", "tools":[ {type:"function", ...the matches} ]} and also surfaces the discovered tools in `tools[]`. The model can now call them. This repeats as the model needs more tools. Root cause ---------- ResponsesMessage (the element type of the Responses `input` array AND the response `Output` array) doesn't model `tool_search_call` / `tool_search_output`: - The call's `arguments` is a JSON OBJECT, whereas function_call's `arguments` is a JSON STRING. So it cannot decode into ResponsesToolMessage.Arguments (*string) -> sonic.Unmarshal of the whole []ResponsesMessage errors -> OpenAIResponsesRequestInput.UnmarshalJSON falls through to the "neither a string nor an array" 400. The entire request dies before reaching OpenAI. - The output's `tools` array is also unmodeled (would be dropped/mangled, which OpenAI then rejects with "Missing input[N].tools[0].type"). OpenAI's Responses API supports both items natively (verified end-to-end against the gateway: the tool_search tool spec is accepted and echoed; OpenAI validates arguments-as-object and tools[].type). So this is purely a Bifrost modelling gap, in the same family as the tool-type allowlist that already lists ResponsesToolTypeToolSearch / ResponsesToolTypeNamespace — just a different code path (input-item deserialization vs the request tools[] allowlist). Fix --- Add ResponsesMessageTypeToolSearchCall / ResponsesMessageTypeToolSearchOutput and give ResponsesMessage custom (Un)MarshalJSON that preserves these two item types verbatim (original bytes in, original bytes out), so the object `arguments` and the `tools` array survive intact. Every other item type defers to the default struct (de)coding, unchanged. One change covers both directions because request input and response Output are both []ResponsesMessage. Impact: unblocks codex tool-search deferral (multi-MCP-server / >=100-tool agents) through Bifrost. Verified with a round-trip test reproducing the exact follow-up payload, plus the existing providers/openai and schemas suites (no regressions). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(responses): reset ResponsesMessage receiver in UnmarshalJSON Clear the receiver at the top of ResponsesMessage.UnmarshalJSON so a reused instance never retains a stale rawToolSearch (or other field) from a prior decode. Without this, unmarshalling a tool_search item and then a normal message into the same value would leave the preserved bytes in place, and MarshalJSON would re-emit them. Not reachable via the array-decode path (each element starts zero), but a cheap, defensive correctness fix. Addresses CodeRabbit review on PR maximhq#4121. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(openapi): correct prompt_cache_retention enum to in_memory The chat schema declared the enum as [in-memory, 24h], but OpenAI's actual accepted values are in_memory (underscore) and 24h. The hyphenated form was a typo from when the enum was first added and never matched OpenAI, so spec-generated clients produced Literal['in-memory', '24h'] and rejected the valid value with a pydantic literal_error. The Go runtime treats prompt_cache_retention as a pass-through *string, so no behavior changes — only the spec enum, the regenerated openapi.json, and the doc comment are corrected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Akshay Deo <akshay@akshaydeo.com> Co-authored-by: Suresh Kumar Ponnusamy <suresh@atomicwork.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Akshay Deo <akshay@akshaydeo.com>
## Summary Fixes a bug in the Vertex provider where the code path for handling Gemini/Gemma model families was duplicated, with the non-streaming branch incorrectly using `ToGeminiChatCompletionRequest` (without image URL scheme support) while the streaming branch used `ToGeminiChatCompletionRequestWithImageURLSchemes`. This consolidates the logic so both paths use the image URL scheme-aware converter. ## Changes - Replaced `ToGeminiChatCompletionRequest` with `ToGeminiChatCompletionRequestWithImageURLSchemes` in the non-streaming Gemini/Gemma branch, making it consistent with the streaming branch - Removed the duplicate non-streaming Gemini/Gemma and OpenAI handler blocks that had been incorrectly separated from the streaming path, consolidating them into a single unified code path ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Send a chat completion request to the Vertex provider using a Gemini or Gemma model with image URL content. Verify that image URLs are correctly processed in both streaming and non-streaming modes. ```sh go test ./core/providers/vertex/... ``` ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
Consolidates the two separate `UnmarshalJSON` implementations on `ResponsesMessage` into a single method that handles both the verbatim `tool_search` preservation and the `arguments` normalization logic. Previously, the file contained a duplicate `UnmarshalJSON` definition — the first handled `tool_search` items and fell back to a plain `sonic.Unmarshal`, while the second (the correct one) handled argument normalization. The duplicate caused the normalization path to be unreachable for non-`tool_search` items, meaning `tool_search_call` items with object-typed `arguments` would silently fail mid-stream and hang streaming clients.
## Changes
- Removed the redundant first `UnmarshalJSON` that short-circuited to `sonic.Unmarshal` without normalizing `arguments`, leaving only the correct implementation that handles both the `rawToolSearch` early-return and the `arguments` object-to-string normalization.
- Relocated `MarshalJSON` to follow `UnmarshalJSON` for logical grouping.
- The fix ensures `tool_search_call` items whose `arguments` field is a JSON object (e.g. `{}` while in-progress, `{"query":"...","limit":10}` when completed) are correctly stringified into the `*string` field expected by `ResponsesToolMessage`, preventing decode failures that previously dropped items silently.
## Type of change
- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/schemas/...
```
Validate by sending a request that triggers `tool_search_call` streaming events and confirming that items with both `{}` (in-progress) and `{"query":"...","limit":10}` (completed) `arguments` values are decoded without error and do not hang the streaming client.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
None.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds a `server` (client ID) filter to the MCP clients list endpoint and UI, allowing users to filter the MCP clients table to a specific server. The filter state is persisted in the URL via query parameters, enabling shareable and bookmarkable filtered views. ## Changes - Added `ClientID` field to `MCPClientsQueryParams` and applied it as a `WHERE client_id = ?` clause in `GetMCPClientsPaginated` - Exposed the filter via a new `server` query parameter on `GET /api/mcp/clients` - Migrated the MCP registry page from local `useState` to `nuqs` `useQueryStates`, storing `search`, `server`, and `offset` in the URL - Added `server` prop and `onServerFilterClear` callback to `MCPClientsTable`, rendering a dismissible "Server filter" badge/button when the filter is active - Extended `GetMCPClientsParams` type and the RTK Query API call to pass the `server` parameter through to the backend - `hasActiveFilters` now accounts for both `debouncedSearch` and `server`, preventing the empty state from showing while a server filter is active ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Navigate to the MCP Registry page. 2. Confirm that `search`, `server`, and `offset` appear in the URL and survive a page refresh. 3. Set a `server` query param (e.g. `?server=<client_id>`) directly in the URL and verify the table filters to only that client. 4. Click the "Server filter" dismiss button and confirm the filter clears and the URL updates. 5. Verify that the empty state is not shown when a server filter is active but returns no results. ```sh # Core/Transports go test ./framework/configstore/... ./transports/bifrost-http/... # UI cd ui pnpm i pnpm build ``` ## Screenshots/Recordings _Add before/after screenshots showing the server filter badge and URL state._ ## Breaking changes - [ ] Yes - [x] No ## Related issues _Link related issues here._ ## Security considerations The `client_id` filter is applied as a parameterised query (`WHERE client_id = ?`), so there is no SQL injection risk. No secrets or PII are exposed through the new filter parameter. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…mhq#4770) ## Summary Error messages returned on JSON decode failures were leaking internal decoder details (e.g., field names, Go type information, and `cannot unmarshal` messages) back to API callers. This replaces all such messages with a single, generic `"Invalid request payload"` string to avoid exposing implementation internals. ## Changes - Replaced all `fmt.Sprintf("invalid/Invalid request format: %v", err)` and similar patterns across handlers (`config`, `featureflags`, `governance`, `inference`, `mcp`, `mcp_per_user_headers`, `mcpinference`, `provider_keys`, `providers`, `session`) with the static string `"Invalid request payload"`. - Removed now-unused `fmt` import from `mcpinference.go`. - Added `requestpayload_test.go` with two tests that assert the generic message is returned and that decoder internals (`cannot unmarshal`, field names, Go struct details) are not present in the response body or error string. - Updated the existing `governance_test.go` assertion for the unknown-field case to expect `"Invalid request payload"` instead of `"unknown field"`. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./transports/bifrost-http/handlers/... ``` Confirm that: - `TestSessionLoginInvalidPayloadDoesNotExposeDecoderDetails` passes and the response body contains `"Invalid request payload"` with no decoder internals. - `TestPrepareRequestInvalidPayloadDoesNotExposeDecoderDetails` passes and the returned error is exactly `"invalid request payload"`. - `TestComplexityAnalyzerConfigPutRejectsInvalidPayloads` passes with the updated `"Invalid request payload"` expectation for the unknown-field case. ## Breaking changes - [ ] Yes - [x] No ## Security considerations Decoder error messages from Go's `encoding/json` and `sonic` can expose internal struct field names, type information, and value details. Returning these verbatim in HTTP responses constitutes an information disclosure risk. This change ensures all parse-failure responses return a fixed, opaque message regardless of the underlying decode error. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Use the shared OpenAI-compatible handlers so DeepSeek models can be routed through Bifrost with the provider's root-path API surface and conservative operation support.
Expose DeepSeek consistently in config metadata, docs, UI, and CI so the new provider can be configured, discovered, and tested like the existing providers.
etnperlong
force-pushed
the
feat/deepseek-provider
branch
from
June 30, 2026 11:41
80e2338 to
a4ca271
Compare
Contributor
|
Too many files changed for review. ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds DeepSeek as a first-class provider and wires it through the core gateway, HTTP transport, docs, config schema, and UI metadata so it can be selected and used consistently across the product.
Changes
Type of change
Affected areas
How to test