Feat/dynamic worker autoscaler v2 - #4768
Closed
Prateek-Gupta001 wants to merge 25 commits into
Closed
Conversation
…dle special characters in URLs (maximhq#4714) ## Summary Array-type query parameters in the logs page (models, providers, etc.) were not using the safe URI-encoding parser, meaning values containing characters like `://` (e.g. model names such as `gpt://host/model`) could be misinterpreted as path or query delimiters by TanStack Router. This introduces `parseAsSafeArrayOf` and applies it consistently across all array filters. ## Changes - Added `parseAsSafeArrayOf` to `queryParamsParser.ts` by composing `parseAsArrayOf` with the existing `parseAsSafeString` parser, ensuring full URI-encoding for comma-separated filter values. - Replaced all usages of `parseAsArrayOf(parseAsString)` in the logs page with `parseAsSafeArrayOf` so that array filters (models, providers, aliases, status, etc.) benefit from the same encoding guarantees as string filters. - Added unit tests for both `parseAsSafeString` and `parseAsSafeArrayOf` to verify round-trip correctness with model names containing `://`. ## Type of change - [x] Bug fix ## Affected areas - [x] UI (React) ## How to test ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` To manually verify, navigate to the logs page and apply a filter using a model name containing `://` (e.g. `gpt://host/model`). Confirm the URL encodes correctly and the filter persists on page reload without routing errors. ## Screenshots/Recordings N/A ## Breaking changes - [x] No ## Related issues maximhq#4603 ## Security considerations No security implications. This change only affects URL query parameter encoding in the UI. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…4715) ## Summary Adds a reusable `TruncatedLabel` component that displays truncated text with an automatic tooltip when the content overflows its container. ## Changes - Introduces `TruncatedLabel`, a `<span>`-based component that detects when its text content is truncated via CSS overflow and conditionally renders a `Tooltip` to show the full content - Truncation detection is performed by comparing `scrollWidth` to `clientWidth`, and re-evaluated on window resize or when `children` changes - The tooltip content defaults to the `children` value if it is a string, but accepts an explicit `tooltip` prop for custom content - The tooltip is only rendered when the text is actually truncated, avoiding unnecessary DOM overhead when content fits ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` Render a `TruncatedLabel` inside a constrained-width container with a long string. Verify that hovering over the truncated text shows a tooltip with the full content, and that no tooltip appears when the text is not truncated. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations No security implications. The component renders user-provided React nodes, which is consistent with existing UI patterns. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ation logic (maximhq#4716) ## Summary Extracts the truncated label logic (truncate text with a tooltip on overflow) into a shared `TruncatedLabel` component and replaces all inline implementations with it. ## Changes - Added a reusable `TruncatedLabel` component in `ui/components/ui/truncatedLabel` that handles text truncation and conditionally renders a tooltip when the content overflows - Removed the local `TruncatedName` component from the providers page, which duplicated this logic using `useRef`, `useState`, and a resize event listener - Replaced inline `<span className="truncate ...">` elements in the logs and MCP filter sidebars with `TruncatedLabel` ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Verify that truncated labels in the providers list and filter sidebars still show a tooltip on hover when the text overflows, and no tooltip when it does not. ## Screenshots/Recordings Verify the providers sidebar and log/MCP filter sidebars visually behave the same as before — truncated text shows a tooltip, non-truncated text does not. ## Breaking changes - [ ] Yes - [x] No ## Related issues maximhq#4604 ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ximhq#4719) ## Summary Fixes the layout of the Audit Logs page to correctly fill the viewport and apply the appropriate background and border styles. ## Changes - Replaced `h-[calc(100dvh-1rem)]` with `h-[calc(100vh-16px)]` for consistent viewport height calculation - Swapped `mx-auto flex flex-col p-4` utility classes for `no-border-parent bg-background flex` to align with the layout conventions used elsewhere in the app - Added missing newline at end of file ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test Navigate to the Audit Logs page and verify: - The page fills the full viewport height without overflow or clipping - The background color and border styling match the rest of the workspace layout ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` ## Screenshots/Recordings Add before/after screenshots showing the corrected Audit Logs page layout. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds collapsible tag support to the `TagInput` component and applies it to the keyword lists on the Complexity Router page. When a keyword list exceeds a configurable limit, tags beyond that limit are hidden behind a gradient overlay with a "Show more" toggle, keeping the UI compact while still allowing full access to all tags. ## Changes - Added `collapsedTagLimit` and `expandButtonTestId` props to `TagInput`. When `collapsedTagLimit` is provided, the component renders in a collapsible layout: tags beyond the limit are hidden with a fade gradient, and "Show more" / "Show less" buttons toggle the expanded state. The collapsed state auto-resets when the tag count drops back to or below the limit. - Set `KEYWORD_COLLAPSED_LIMIT = 8` on the Complexity Router page and passed it along with a `expandButtonTestId` to each keyword `TagInput`. - Standardized border radius tokens from `rounded-lg`/`rounded-md`/`rounded-full` to `rounded-sm` across the Complexity Router page for visual consistency. - Reformatted `index.html` inline shell skeleton from a single minified line to readable, indented HTML and CSS. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` 1. Navigate to the Complexity Router page. 2. Add more than 8 keywords to any keyword list. 3. Verify that tags beyond 8 are hidden with a gradient overlay and a "Show more" button appears. 4. Click "Show more" and confirm all tags are visible with a "Show less" button. 5. Click "Show less" and confirm the list collapses again. 6. Remove tags until 8 or fewer remain and confirm the list stays expanded without the toggle controls. ## Screenshots/Recordings Before/after screenshots of the keyword lists with collapse behavior recommended. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…maximhq#4722) * fix: rebuild token_usage from denormalized columns in hybrid log list * refactor: inline hybrid token usage reconstruction * fix: preserve malformed serialized token usage state --------- Co-authored-by: gexiangdong <xiangdong.ge@pandasofcaribbean.com>
…hatRequestBody` and `AnthropicProviderRequestDefaultsMap` (maximhq#3309) ## Summary This PR consolidates Anthropic-family request building across all providers (Anthropic native, Azure, Vertex, Bedrock) into two shared builder functions — `BuildAnthropicChatRequestBody` and `BuildAnthropicResponsesRequestBody` — eliminating duplicated inline logic and provider-specific wrapper helpers that previously scattered the same field-stripping, beta-header injection, and model-field manipulation across multiple files. ## Changes - Introduced `AnthropicProviderRequestDefaults` and `AnthropicProviderRequestDefaultsMap` to encode static, per-provider request-shaping flags (e.g. `DeleteModelField`, `DeleteStreamField`, `AddAnthropicVersion`, `InjectBetaHeadersIntoBody`) in one place. Callers no longer pass these flags directly; the builder looks them up by `cfg.Provider`. - Renamed `Deployment` to `Model` in `AnthropicRequestBuildConfig` for clarity, since all providers now use the same field for model/deployment overrides. - Added `BuildAnthropicChatRequestBody` as the chat-completion analogue of `BuildAnthropicResponsesRequestBody`, covering both raw-body and typed paths, including field stripping, beta-header injection, streaming flag handling, and `fallbacks` deletion. - Bedrock now routes Anthropic models through the Anthropic Messages API format (`invoke` / `invoke-with-response-stream` endpoints) for both chat and responses, rather than the Bedrock Converse API. This includes proper response parsing via `AcquireAnthropicMessageResponse` and streaming via `AnthropicStreamState` / `AnthropicResponsesStreamState`. - Removed private wrapper functions `getRequestBodyForResponses` (Anthropic), `getRequestBodyForAnthropicResponses` (Azure, Vertex), and the inline `CheckContextAndGetRequestBody` closures for Anthropic models in Vertex and Azure, replacing all call sites with direct `BuildAnthropicChatRequestBody` / `BuildAnthropicResponsesRequestBody` calls. - Exported `AcquireAnthropicResponsesStreamState`, `ReleaseAnthropicResponsesStreamState`, `AcquireAnthropicMessageResponse`, and `ReleaseAnthropicMessageResponse` so Bedrock can reuse the Anthropic stream state pool. - Removed `DefaultVertexAnthropicVersion` constant from the Vertex package; the canonical version string now lives in `AnthropicProviderRequestDefaultsMap`. - Bedrock's `releaseBedrockChatResponse` now zeroes the struct before returning it to the pool. - `stripUnsupportedAnthropicFields` is now called inside `BuildAnthropicResponsesRequestBody` on the typed path, making field stripping symmetric across raw and typed paths and across both APIs. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/anthropic/... go test ./core/providers/azure/... go test ./core/providers/bedrock/... go test ./core/providers/vertex/... go test ./... ``` Run integration tests against Anthropic, Azure (Anthropic models), Vertex (Claude models), and Bedrock (Claude models) for chat completion, streaming, responses, responses streaming, and count-tokens endpoints. Verify that raw-body passthrough requests produce the same field stripping and beta-header injection as typed requests. ## Breaking changes - [x] Yes - [ ] No `AnthropicRequestBuildConfig` has a breaking field rename: `Deployment` → `Model`. Any external code constructing this struct directly must update the field name. The static shaping flags (`DeleteModelField`, `DeleteRegionField`, `AddAnthropicVersion`, `AnthropicVersion`, `StripCacheControlScope`, `RemapToolVersions`, `InjectBetaHeadersIntoBody`) have been removed from `AnthropicRequestBuildConfig` and are now looked up internally via `AnthropicProviderRequestDefaultsMap`; callers that set these fields must remove them. ## Related issues ## Security considerations No new auth flows, secrets handling, or PII exposure introduced. Field stripping ensures provider-unsupported fields are not forwarded to external APIs. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…hropicResponsesRequest` and make `completeRequest` a package-level func shared by Anthropic, Azure, and Bedrock providers (maximhq#4394) ## Summary The Anthropic provider's unary request logic was duplicated across the Anthropic, Azure, Bedrock, and Vertex providers. This PR extracts the core non-streaming request execution into a package-level `completeRequest` function and introduces two exported handler functions — `HandleAnthropicChatCompletionRequest` and `HandleAnthropicResponsesRequest` — that encapsulate the full build → send → parse pipeline for chat completions and the Responses API respectively. Azure and Bedrock now delegate directly to these shared handlers for Anthropic-family models instead of reimplementing request dispatch, response parsing, and raw request/response handling inline. A secondary bug fix is included: the large-response streaming client was being activated for count-tokens requests (which should always be buffered) and skipped for all other requests — the condition was inverted. ## Changes - Extracted `completeRequest` as a package-level function accepting explicit `client`, `headers`, `extraHeaders`, `betaHeaderOverrides`, `providerName`, and `logger` arguments, removing the method receiver dependency so it can be called by other providers. - Added `anthropicRequestHeaders` as a provider method to build the `x-api-key` / `anthropic-version` header map, shared across `TextCompletion`, `ChatCompletion`, `Responses`, and `CountTokens`. - Introduced `HandleAnthropicChatCompletionRequest` and `HandleAnthropicResponsesRequest` as exported functions that perform the full unary request lifecycle (body build, HTTP send, large-response detection, response parse, raw request/response attachment). These are now called by the Anthropic, Azure, and Bedrock providers. - Removed `completeMantleRequest` from Bedrock — its logic is now covered by `completeRequest` inside the shared handlers. - Azure's `ChatCompletion` and `Responses` methods now branch early for Anthropic-family models, calling the shared handlers with Azure-specific auth headers, and fall through to the OpenAI-compatible path otherwise, eliminating the post-response model-family branch. - Fixed the inverted condition in `completeRequest` that caused the large-response streaming client to be used for count-tokens requests instead of being skipped for them. - `AnthropicRequestBuildConfig` now carries `BetaHeaderOverrides` so callers do not need to pass it separately. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` Validate that chat completions and Responses API requests succeed for Anthropic-family models routed through the Azure and Bedrock providers, and that count-tokens requests return buffered responses without triggering large-response mode. ## Breaking changes - [ ] Yes - [x] No ## Security considerations Auth headers (`x-api-key`, Bearer tokens, SigV4-signed headers) are applied last in `completeRequest`, after network-config extra headers, ensuring they cannot be overridden by user-supplied configuration. No new secrets or PII handling paths are introduced. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…atible providers from `schemas.Key` to `map[string]string` auth header param (maximhq#4425) ## Summary This PR standardizes how Bearer token authentication headers are constructed across all OpenAI-compatible providers. Previously, each call site independently built the `Authorization: Bearer <token>` header map with duplicated inline logic. A new `BearerAuthHeader(key)` helper is introduced in the OpenAI package and used uniformly everywhere. Additionally, the Azure provider's private `completeRequest` method is removed. Its non-Anthropic request paths (text completion, chat completion, responses, embedding, compaction) are now delegated directly to the shared `Handle*` functions in the OpenAI package, consistent with how other providers already work. The `Handle*` functions themselves are updated to accept a pre-built `authHeader map[string]string` instead of a raw `schemas.Key`, making them provider-agnostic and compatible with non-Bearer auth schemes (e.g., Azure API key headers, SigV4). ## Changes - Added `BearerAuthHeader(key schemas.Key) map[string]string` to the OpenAI provider package, which returns an `Authorization: Bearer <token>` header map, or an empty map when the key carries no value. - Updated all `Handle*Request` and `handleOpenAILargePayloadPassthrough` function signatures to accept `authHeader map[string]string` instead of `schemas.Key`, applying the map directly to request headers. - Replaced all inline `var authHeader map[string]string` + conditional assignment blocks across Cerebras, Fireworks, Groq, HuggingFace, Mistral, Nebius, Ollama, Opencode, OpenRouter, Parasail, Perplexity, SGL, VLLM, xAI, and Bedrock with calls to `openai.BearerAuthHeader(key)`. - Removed the Azure provider's `completeRequest` method and replaced its usage in `TextCompletion`, `ChatCompletion`, `Responses`, `Embedding`, and `Compaction` with direct calls to the corresponding shared OpenAI `Handle*` functions, passing Azure-specific auth headers and pre-resolved endpoint URLs. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` Verify that all OpenAI-compatible providers (OpenAI, Azure, Cerebras, Fireworks, Groq, HuggingFace, Mistral, Nebius, Ollama, Opencode, OpenRouter, Parasail, Perplexity, SGL, VLLM, xAI, Bedrock Mantle) continue to authenticate correctly and that requests succeed for text completion, chat completion, responses, embeddings, and compaction endpoints. ## Breaking changes - [ ] Yes - [x] No ## Security considerations The `BearerAuthHeader` helper preserves the existing behavior of omitting the `Authorization` header when the key value is empty, which is intentional for providers that supply auth via other mechanisms (e.g., extra headers or SigV4 signing). No secrets are logged or exposed. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… closure passed through request handlers (maximhq#4735) ## Summary Replaces the pre-build-and-sign approach for Bedrock Mantle SigV4 authentication with a `BodySigner` callback that is invoked after the request handler has marshaled the body. This ensures the signature always covers the exact bytes sent on the wire, eliminating the previous double-marshal pattern where the body was built once for signing and again inside the handler. ## Changes - Introduces a new `BodySigner` type (`func(jsonData []byte) (map[string]string, *schemas.BifrostError)`) in `core/providers/utils/bodysigner.go`. Handlers call it after building the request body and apply the returned headers to the outgoing request. - Adds the `signer` parameter to `HandleOpenAIChatCompletionRequest`, `HandleOpenAIChatCompletionStreaming`, `HandleOpenAIResponsesRequest`, `HandleOpenAIResponsesStreaming`, `HandleAnthropicChatCompletionRequest`, `HandleAnthropicChatCompletionStreaming`, `HandleAnthropicResponsesRequest`, and `HandleAnthropicResponsesStream`. All existing callers pass `nil`. - Rewrites Bedrock Mantle's SigV4 paths (`mantleChatCompletions`, `mantleChatCompletionsStream`, `mantleResponses`, `mantleResponsesStream`) to construct a `BodySigner` closure when no API key is present, instead of pre-building the body, signing it, and merging the signature headers into `extraHeaders`. The Bearer path no longer needs a separate early-return branch. - Removes the now-unnecessary `maps` import and the intermediate `extraHeaders` map copies in the Mantle code paths. ## Type of change - [ ] Bug fix - [x] Refactor - [ ] Feature - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` For Bedrock Mantle with SigV4 (empty key value), verify that requests to chat completions, streaming chat completions, responses, and streaming responses are signed correctly and accepted by the Bedrock endpoint. For Bearer key paths, confirm that no signing is attempted and the `Authorization` header is set as expected. ## Breaking changes - [ ] Yes - [x] No ## Security considerations The `BodySigner` callback signs the exact serialized bytes that are placed on the wire. Previously, the body was serialized twice (once for signing, once inside the handler), which could in theory produce a signature mismatch if marshaling were non-deterministic. This change closes that gap by signing after the final body is set. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ropic and OpenAI-compatible routing (maximhq#4736) ## Summary Introduces `bedrock_mantle` as a first-class, standalone provider that owns the Bedrock Mantle surface (`bedrock-mantle.{region}.api.aws`). Previously, Mantle routing was handled as an internal routing decision inside the existing `bedrock` provider. The new provider gives operators a dedicated configuration surface for Claude (native Anthropic Messages API), OpenAI-compatible models (gpt-*), and Gemma models served through Mantle, without requiring a full Bedrock setup. ## Changes - Added `schemas.BedrockMantle` (`"bedrock_mantle"`) as a new `ModelProvider` constant and registered it in `StandardProviders`, `dynamicallyConfigurableProviders`, `CanProviderKeyValueBeEmpty`, and `isKeySkippingAllowed`. - Added `BedrockMantleKeyConfig` to the `Key` struct, carrying AWS credentials and region for SigV4 auth against the `bedrock-mantle` service. The existing `BedrockKeyConfig` is unchanged. - Introduced the `core/providers/bedrockmantle` package implementing the full `Provider` interface. Chat, streaming chat, Responses, and streaming Responses dispatch by model family: Anthropic-family models use the native Anthropic Messages surface (`/anthropic/v1/messages`); all others use the OpenAI-compatible surface (`/v1` or `/openai/v1`). All other operations return unsupported-operation errors. - Refactored `signAWSRequest` in the `bedrock` package to accept a `*BedrockKeyConfig` instead of individual credential fields, eliminating the now-redundant `signAWSRequestFromKey` wrapper. All call sites updated accordingly. - Exported `SignMantleV4Headers` (previously `mantleSigV4Headers`, a method on `BedrockProvider`) so the new `bedrockmantle` package can sign requests without depending on the internal Bedrock provider struct. The function now supports both `BedrockKeyConfig` and `BedrockMantleKeyConfig` by mapping the latter into a synthetic `BedrockKeyConfig` for signing, and correctly handles GET requests (nil body) for the list-models path. - Extended the Anthropic chat and Responses request builders to convert native structured outputs to tool calls for `BedrockMantle`, matching the existing `Vertex` workaround. - Added `BedrockMantle` to the comprehensive LLM test harness (`ComprehensiveTestAccount`) with key config, provider config, and a full test file covering the supported scenarios (chat, streaming, tool calls, vision, structured outputs, prompt caching, reasoning, list models) and explicitly disabling unsupported ones. - Marked `isMantleModel` in `bedrock/mantle.go` as deprecated in favour of the new provider. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Set AWS credentials and run the new provider test: ```sh export AWS_ACCESS_KEY_ID=... export AWS_SECRET_ACCESS_KEY=... export AWS_SESSION_TOKEN=... # optional, for temporary credentials export AWS_REGION=us-east-1 go test ./core/providers/bedrockmantle/... -v -run TestBedrockMantle ``` To run the full suite (skips Bedrock Mantle automatically when credentials are absent): ```sh go test ./... ``` Configure a `bedrock_mantle` provider by supplying a `BedrockMantleKeyConfig` (or a Bearer API key in `Value`) with the desired region. The region can also be embedded as a prefix in the model ID (e.g. `us-west-2/anthropic.claude-haiku-4-5`) or set at the alias level via `AliasConfig.Region`. ## Breaking changes - [ ] Yes - [x] No The `signAWSRequest` signature change is internal to the `bedrock` package and does not affect any public API. The `isMantleModel` function is deprecated but not removed. ## Security considerations AWS credentials for `BedrockMantleKeyConfig` follow the same `SecretVar` resolution pattern used by `BedrockKeyConfig` (env-var references, never inlined literals). SigV4 signing is performed per-request on the exact body bytes that are sent, so the signature always covers what is transmitted. When a Bearer API key is present it takes precedence and no AWS credentials are required. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…on, and UI support (maximhq#4737) ## Summary Adds `bedrock_mantle` as a first-class provider, enabling Bifrost to route requests to AWS Bedrock through a Mantle proxy endpoint. The provider supports the same SigV4 credential options as the existing Bedrock provider (inherited IAM role, explicit access/secret key, session token, AssumeRole) as well as a Bearer API key authentication mode. ## Changes - Added `BedrockMantle` to the Anthropic passthrough allowlist in `clearAnthropicPassthroughForNonNativeProvider` so raw request bodies are preserved when routing through Bedrock Mantle. - Added `BedrockMantleKeyConfig` redaction logic in `clientconfig.go`, mirroring the existing Bedrock redaction pattern. - Added a new `migrationAddBedrockMantleKeyColumns` database migration that introduces seven `bedrock_mantle_*` SigV4 credential columns to the `config_keys` table. - Extended `TableKey` with the seven Bedrock Mantle credential fields, along with `BeforeSave` serialization and `AfterFind` reconstruction hooks. - Updated `mergeUpdatedKey` in the HTTP handler to correctly restore redacted Bedrock Mantle credential fields during key updates. - Fixed `isClaudeModel` in the Anthropic integration to recognize `bedrock_mantle` (previously incorrectly matched `bedrock`) as a provider that can serve Claude models. - Included `BedrockMantleKeyConfig` in the key hash inputs used by `mergeProviderKeys` and `reconcileProviderKeys` for config file/DB reconciliation. - Added Bedrock Mantle credential redaction to `GetAllKeys`. - Extended `config.schema.json` with `bedrock_mantle_key` and `provider_with_bedrock_mantle_config` definitions and registered `bedrock_mantle` as a valid provider name throughout the schema. - Added UI support: provider icon (reusing the Bedrock SVG mark with a distinct gradient ID), model placeholder text, `isKeyRequiredByProvider` entry, label, form schema (`BedrockMantleKeyConfigSchema`), type definitions (`BedrockMantleKeyConfig`, `DefaultBedrockMantleKeyConfig`), and a full authentication method tab UI (IAM Role / Explicit Credentials / API Key) matching the Bedrock provider UX. - Added `bedrock_mantle` to the Anthropic beta-headers provider family and the provider config sheet's Anthropic family list. - Stripped the internal `_auth_type` field from `bedrock_mantle_key_config` before submitting the form payload. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Configure a `bedrock_mantle` provider in `config.json` or via the UI with one of the three auth methods: - **IAM Role (Inherited):** set only `region`; leave access/secret key empty. - **Explicit Credentials:** set `access_key`, `secret_key`, and `region`; optionally set `session_token`, `role_arn`, `external_id`, and `session_name`. - **API Key:** set `region` and provide a Bearer token as the key `value`. Send a request targeting a Claude model through the `bedrock_mantle` provider and verify the response is returned correctly and that credentials are redacted in the UI and API responses. ## Screenshots/Recordings _Add before/after screenshots of the new Bedrock Mantle provider form and icon in the UI._ ## Breaking changes - [ ] Yes - [x] No ## Related issues _Link related issues and discussions._ ## Security considerations - All seven Bedrock Mantle credential fields (`access_key`, `secret_key`, `session_token`, `region`, `role_arn`, `external_id`, `role_session_name`) are stored as `SecretVar` and are redacted in API responses and the UI, consistent with the existing Bedrock provider handling. - The `_auth_type` discriminator field is stripped from the payload before it is persisted or transmitted. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
Adds documentation for the AWS Bedrock Mantle provider, a distinct AWS endpoint (`bedrock-mantle.{region}.api.aws`) that exposes Claude models via the native Anthropic Messages API and OpenAI-family/Gemma models via an OpenAI-compatible API — all addressable through a single `bedrock_mantle/<model>` prefix in Bifrost.
## Changes
- Added a new `bedrock-mantle.mdx` provider page covering model ID formats, supported operations, all three authentication modes (SigV4 with explicit credentials, IAM role/inherited credentials, and Bearer API key), IAM role assumption via `role_arn`, and usage examples.
- Added the Bedrock Mantle configuration block to the `providers.mdx` config reference, with tabs for Static Credentials, IAM Role, and API Key (Bearer) auth modes.
- Added Bedrock Mantle to the provider capability matrix in `overview.mdx`.
- Registered `bedrock-mantle` in `docs.json` so it appears in the sidebar navigation.
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [x] Documentation
- [ ] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [x] Docs
## How to test
Navigate to the Bedrock Mantle provider page and config reference in the rendered docs and verify:
- The sidebar entry for `bedrock-mantle` appears between `bedrock` and `cerebras`.
- All three auth tabs (Static Credentials, IAM Role, API Key) render correctly in both the provider page and the config reference.
- The capability matrix row for `bedrock_mantle/<model>` is present and accurate.
- Cross-links between the provider page and the config reference resolve correctly.
## Breaking changes
- [x] No
## Security considerations
Authentication credentials (`access_key`, `secret_key`, `session_token`, API keys) are documented using the `env.*` indirection pattern, consistent with how other providers handle secrets. No credentials are hardcoded in examples.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
…ment config (maximhq#4739) ## Summary Adds E2E test configuration and capability definitions for the `bedrock_mantle` provider, enabling it to be tested through the Bifrost V1 API test suite. ## Changes - Added `bedrock_mantle` to `provider-capabilities.json` with `chat_completions`, `chat_completions_with_tools`, `responses`, `responses_with_tools`, and `list_models` enabled - Added a new Postman environment file (`bifrost-v1-bedrock-mantle.postman_environment.json`) configured to use `anthropic.claude-opus-4-8` as the default model and `us-east-1` as the default region, with secret placeholders for API key, access key, secret key, and session token ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Run the E2E test suite targeting the `bedrock_mantle` provider using the new Postman environment: ```sh # Ensure the Bifrost server is running locally on port 8080 # Load the environment file and run the collection against bedrock_mantle newman run tests/e2e/api/bifrost-v1.postman_collection.json \ -e tests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.json \ --env-var "bedrock_mantle_api_key=<your_api_key>" \ --env-var "bedrock_mantle_access_key=<your_access_key>" \ --env-var "bedrock_mantle_secret_key=<your_secret_key>" ``` Expected outcome: chat completions, tool-use, responses, and model listing tests pass; all unsupported capability tests are skipped or return expected errors. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations The Postman environment file stores API key, access key, secret key, and session token as `secret` type fields with empty default values, ensuring credentials are not committed to the repository. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…d toggle when budget is alignable (maximhq#4724) ## Summary The calendar alignment toggle in the provider governance form was previously shown whenever any budget existed. This PR restricts its visibility and submission to only when at least one budget uses a calendar-alignable reset period (day, week, month, or year). ## Changes - Introduced a `showCalendarAlignment` derived boolean that checks whether any configured budget has a reset duration supported by `supportsCalendarAlignment`. - Replaced the previous condition (`watchedBudgets.length > 0`) with `showCalendarAlignment` to control rendering of the calendar alignment toggle. - Updated the form submission payload so that `calendar_aligned` is only set to `true` when at least one budget actually supports calendar alignment — preventing the flag from being submitted for incompatible budget configurations. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Navigate to a provider's governance settings in the UI. 2. Add a budget with a reset duration that does **not** support calendar alignment (e.g., hourly). Verify the calendar alignment toggle does **not** appear. 3. Add or change a budget to use a calendar-alignable period (e.g., daily, weekly, monthly, yearly). Verify the toggle **does** appear. 4. Enable the toggle and save. Confirm `calendar_aligned: true` is included in the submitted payload. 5. Remove all calendar-alignable budgets and save. Confirm `calendar_aligned` is not set to `true` in the payload. ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` ## Screenshots/Recordings _Before:_ Calendar alignment toggle appears whenever any budget is present, regardless of reset period. _After:_ Calendar alignment toggle only appears when at least one budget uses a day/week/month/year reset period. ## Breaking changes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…hq#4725) ## Summary Fixes a bug where updating or deleting provider-level governance for a custom provider whose name contains a space (e.g. `"OpenRouter Base"`) would return a 404. The UI percent-encodes the provider name in the URL path (`OpenRouter%20Base`), but the handler was comparing the raw encoded string directly against the stored provider name, causing the lookup to fail. Closes maximhq#4689 ## Changes - `updateProviderGovernance` and `deleteProviderGovernance` now call `url.PathUnescape` on the `provider_name` path parameter before using it, matching the decoded name against what is stored in the config store. - Returns a `400` if the path parameter contains an invalid percent-encoding sequence. - Added a regression test (`TestProviderGovernance_DecodesEncodedProviderName`) that seeds a provider with a space in its name, issues a PUT and DELETE using the percent-encoded path param, and asserts both succeed and persist correctly. - Added a guard test (`TestProviderGovernance_UnknownProviderStill404`) to confirm that a genuinely unknown provider still returns 404 after the decode change. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./transports/bifrost-http/handlers/... -run TestProviderGovernance ``` Expected output: all three `TestProviderGovernance_*` tests pass. Specifically: - `TestProviderGovernance_DecodesEncodedProviderName` — PUT and DELETE with `OpenRouter%20Base` return `200`. - `TestProviderGovernance_UnknownProviderStill404` — PUT with an unknown encoded name returns `404`. ## Breaking changes - [ ] Yes - [x] No ## Related issues - Resolves maximhq#4689 ## Security considerations `url.PathUnescape` is used rather than `url.QueryUnescape` to correctly handle path-encoded characters. Invalid encoding sequences are rejected with a `400` rather than passed through, preventing malformed input from reaching the config store. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… (maximhq#4568) Signed-off-by: Akshay Deo <akshay@akshaydeo.com> Co-authored-by: Akshay Deo <akshay@akshaydeo.com>
* Fix mcp reconnect failure on startup * test: assert failed MCP client cleanup --------- Co-authored-by: Gowtham <692171+HackToHell@users.noreply.github.com>
…put items (maximhq#4121) * fix(responses): preserve codex tool_search_call/tool_search_output input items Bifrost's Responses input deserializer rejected codex's tool-search follow-up request with HTTP 400 "openai responses request input is neither a string nor an array of responses messages", which hung/failed the agent turn. The fix teaches ResponsesMessage about the two tool_search item types and round-trips them verbatim. Background, since tool_search is non-obvious: How codex's tool_search works (the path that hits this bug) ----------------------------------------------------------- codex normally sends every MCP tool inline in the request `tools[]` as `{type:"function", ...}`. But when a model's catalog has `supports_search_tool: true` AND the tool count crosses DIRECT_MCP_TOOL_EXPOSURE_THRESHOLD (= 100) — e.g. an agent wired to several MCP servers — codex stops sending them inline and "defers" them behind a discovery tool: should_defer = supports_search_tool && (ToolSearchAlwaysDeferMcpTools || n >= 100) The deferred flow is a two-request round-trip: 1. Request 1: codex hides the deferred tools and instead declares one tool: {"type":"tool_search","execution":"client","description":"...", "parameters":{query, limit}} `execution:"client"` means the model does NOT run the search — codex does. 2. The model emits a `tool_search_call` with `arguments` = {query, limit}. 3. codex runs the search CLIENT-SIDE: a BM25 index over the deferred tool metadata (codex's ToolSearchHandler, core/src/tools/handlers/tool_search.rs, using the `bm25` crate). It picks the top-N matching tools. 4. Request 2 (follow-up): codex appends two items to `input[]`: - {"type":"tool_search_call", "call_id":..., "execution":"client", "arguments":{...}} - {"type":"tool_search_output", "call_id":..., "status":"completed", "execution":"client", "tools":[ {type:"function", ...the matches} ]} and also surfaces the discovered tools in `tools[]`. The model can now call them. This repeats as the model needs more tools. Root cause ---------- ResponsesMessage (the element type of the Responses `input` array AND the response `Output` array) doesn't model `tool_search_call` / `tool_search_output`: - The call's `arguments` is a JSON OBJECT, whereas function_call's `arguments` is a JSON STRING. So it cannot decode into ResponsesToolMessage.Arguments (*string) -> sonic.Unmarshal of the whole []ResponsesMessage errors -> OpenAIResponsesRequestInput.UnmarshalJSON falls through to the "neither a string nor an array" 400. The entire request dies before reaching OpenAI. - The output's `tools` array is also unmodeled (would be dropped/mangled, which OpenAI then rejects with "Missing input[N].tools[0].type"). OpenAI's Responses API supports both items natively (verified end-to-end against the gateway: the tool_search tool spec is accepted and echoed; OpenAI validates arguments-as-object and tools[].type). So this is purely a Bifrost modelling gap, in the same family as the tool-type allowlist that already lists ResponsesToolTypeToolSearch / ResponsesToolTypeNamespace — just a different code path (input-item deserialization vs the request tools[] allowlist). Fix --- Add ResponsesMessageTypeToolSearchCall / ResponsesMessageTypeToolSearchOutput and give ResponsesMessage custom (Un)MarshalJSON that preserves these two item types verbatim (original bytes in, original bytes out), so the object `arguments` and the `tools` array survive intact. Every other item type defers to the default struct (de)coding, unchanged. One change covers both directions because request input and response Output are both []ResponsesMessage. Impact: unblocks codex tool-search deferral (multi-MCP-server / >=100-tool agents) through Bifrost. Verified with a round-trip test reproducing the exact follow-up payload, plus the existing providers/openai and schemas suites (no regressions). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(responses): reset ResponsesMessage receiver in UnmarshalJSON Clear the receiver at the top of ResponsesMessage.UnmarshalJSON so a reused instance never retains a stale rawToolSearch (or other field) from a prior decode. Without this, unmarshalling a tool_search item and then a normal message into the same value would leave the preserved bytes in place, and MarshalJSON would re-emit them. Not reachable via the array-decode path (each element starts zero), but a cheap, defensive correctness fix. Addresses CodeRabbit review on PR maximhq#4121. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(openapi): correct prompt_cache_retention enum to in_memory The chat schema declared the enum as [in-memory, 24h], but OpenAI's actual accepted values are in_memory (underscore) and 24h. The hyphenated form was a typo from when the enum was first added and never matched OpenAI, so spec-generated clients produced Literal['in-memory', '24h'] and rejected the valid value with a pydantic literal_error. The Go runtime treats prompt_cache_retention as a pass-through *string, so no behavior changes — only the spec enum, the regenerated openapi.json, and the doc comment are corrected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Signed-off-by: Akshay Deo <akshay@akshaydeo.com> Co-authored-by: Suresh Kumar Ponnusamy <suresh@atomicwork.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Akshay Deo <akshay@akshaydeo.com>
Contributor
|
Too many files changed for review. ( |
|
|
## Summary Fixes a bug in the Vertex provider where the code path for handling Gemini/Gemma model families was duplicated, with the non-streaming branch incorrectly using `ToGeminiChatCompletionRequest` (without image URL scheme support) while the streaming branch used `ToGeminiChatCompletionRequestWithImageURLSchemes`. This consolidates the logic so both paths use the image URL scheme-aware converter. ## Changes - Replaced `ToGeminiChatCompletionRequest` with `ToGeminiChatCompletionRequestWithImageURLSchemes` in the non-streaming Gemini/Gemma branch, making it consistent with the streaming branch - Removed the duplicate non-streaming Gemini/Gemma and OpenAI handler blocks that had been incorrectly separated from the streaming path, consolidating them into a single unified code path ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Send a chat completion request to the Vertex provider using a Gemini or Gemma model with image URL content. Verify that image URLs are correctly processed in both streaming and non-streaming modes. ```sh go test ./core/providers/vertex/... ``` ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Contributor
📝 WalkthroughSummary by CodeRabbit
WalkthroughThis PR introduces the AWS Bedrock Mantle provider end-to-end (schema, implementation, UI, persistence, docs), implements dynamic per-provider worker pool autoscaling, refactors the OpenAI/Anthropic request pipelines to centralize bearer auth via ChangesBedrockMantle Provider
Dynamic Worker Autoscaling
OpenAI Auth Refactor, Anthropic Request Builder, and BodySigner
Gemini/Vertex GCS Image URL Allowlist
Independent Bug Fixes
UI Improvements
Docs, Changelogs, and CI
Sequence Diagram(s)sequenceDiagram
participant Client
participant BedrockMantleProvider
participant mantleSigner
participant AnthropicHandler
participant OpenAIHandler
Client->>BedrockMantleProvider: ChatCompletion(ctx, key, request)
BedrockMantleProvider->>BedrockMantleProvider: resolveRegion(ctx, key, model)
BedrockMantleProvider->>BedrockMantleProvider: IsAnthropicModel(model)?
alt Claude family
BedrockMantleProvider->>mantleSigner: key.Value empty → build SigV4 signer closure
BedrockMantleProvider->>AnthropicHandler: HandleAnthropicChatCompletionRequest(cfg, signer)
AnthropicHandler->>mantleSigner: signer(jsonBody)
mantleSigner-->>AnthropicHandler: signed headers (Authorization, X-Amz-*)
AnthropicHandler-->>BedrockMantleProvider: BifrostChatResponse
else OpenAI family / Gemma
BedrockMantleProvider->>OpenAIHandler: HandleOpenAIChatCompletionRequest(authHeader, signer)
OpenAIHandler-->>BedrockMantleProvider: BifrostChatResponse
end
BedrockMantleProvider-->>Client: BifrostChatResponse
Estimated code review effort🎯 5 (Critical) | ⏱️ ~180 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
## Summary
Consolidates the two separate `UnmarshalJSON` implementations on `ResponsesMessage` into a single method that handles both the verbatim `tool_search` preservation and the `arguments` normalization logic. Previously, the file contained a duplicate `UnmarshalJSON` definition — the first handled `tool_search` items and fell back to a plain `sonic.Unmarshal`, while the second (the correct one) handled argument normalization. The duplicate caused the normalization path to be unreachable for non-`tool_search` items, meaning `tool_search_call` items with object-typed `arguments` would silently fail mid-stream and hang streaming clients.
## Changes
- Removed the redundant first `UnmarshalJSON` that short-circuited to `sonic.Unmarshal` without normalizing `arguments`, leaving only the correct implementation that handles both the `rawToolSearch` early-return and the `arguments` object-to-string normalization.
- Relocated `MarshalJSON` to follow `UnmarshalJSON` for logical grouping.
- The fix ensures `tool_search_call` items whose `arguments` field is a JSON object (e.g. `{}` while in-progress, `{"query":"...","limit":10}` when completed) are correctly stringified into the `*string` field expected by `ResponsesToolMessage`, preventing decode failures that previously dropped items silently.
## Type of change
- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/schemas/...
```
Validate by sending a request that triggers `tool_search_call` streaming events and confirming that items with both `{}` (in-progress) and `{"query":"...","limit":10}` (completed) `arguments` values are decoded without error and do not hang the streaming client.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
None.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/bifrost.go (1)
3533-3608: 🩺 Stability & Availability | 🔴 Critical | ⚡ Quick winValidate autoscaling config on
UpdateProviderbefore using it.This hot-reload path bypasses
ValidateDynamicScalingConfig, unlikeprepareProvider. A reload withScalingInterval <= 0will panic intime.NewTicker, and a negativeMaxWorkerswill panic whennewPq.quitis allocated. Apply the same validate/clamp/disable step here before touchingDynamicScaling,MaxWorkers, orScalingInterval.Suggested change
// Step 2: Create new ProviderQueue and wait group with updated settings. + providerConfig.ConcurrencyAndBufferSize.DynamicScaling = + ValidateDynamicScalingConfig(providerConfig, providerKey, bifrost.logger) + newPq := &ProviderQueue{ queue: make(chan *ChannelMessage, providerConfig.ConcurrencyAndBufferSize.BufferSize), done: make(chan struct{}), signalOnce: sync.Once{}, }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/bifrost.go` around lines 3533 - 3608, Validate and normalize the autoscaling settings in UpdateProvider before using DynamicScaling, MaxWorkers, or ScalingInterval, since this hot-reload path currently bypasses the same checks used by prepareProvider. Apply the existing ValidateDynamicScalingConfig-style clamping/disabling logic on the providerConfig.ConcurrencyAndBufferSize values before creating newPq, allocating newPq.quit, or starting DynamicWorkerScaling, so invalid reload values cannot trigger ticker or channel panics.
🧹 Nitpick comments (1)
core/schemas/provider.go (1)
230-238: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueFix comment typos and improve clarity.
- Line 230: "reponses" → "responses"
- Line 232: Rephrase to "Enable dynamic worker pool scaling for this provider"
- Line 233: "nesscary" → "necessary", "go routine" → "goroutine"
- Line 234-235: Use "Number" instead of "No." for consistency
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/schemas/provider.go` around lines 230 - 238, The struct field comments in provider schema contain typos and inconsistent wording that should be cleaned up for clarity. Update the comments on Concurrency, DynamicScaling, ScalingInterval, MinWorkers, MaxWorkers, ScaleUpThreshold, and ScaleDownThreshold to use correct spelling (responses, necessary, goroutine) and consistent phrasing, especially the DynamicScaling description and the “Number of workers” wording. Keep the meaning the same while making the comment text professional and easy to read.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/bifrost_test.go`:
- Around line 3057-3094: The scaling test reuses a single mutable ChannelMessage
across many queue submissions, which can race because requestWorker mutates
per-request state on the message and its context. Update the enqueue loop in the
test to create a fresh ChannelMessage for each send while still deriving each
one from the same cancelable parent context (reqCtx/cancelReq), and keep the
existing cleanup/shutdown logic around testDone and cancelReq.
In `@docs/docs.json`:
- Around line 488-494: The Google Workspace navigation entry is still pointing
to the legacy redirect source instead of the renamed OIDC page. Update the page
reference in docs/docs.json under the Google Workspace group to use the new
enterprise/setting-up-google-workspace/oidc slug so the nav matches the current
page layout and stays aligned with the renamed Mintlify page.
In `@docs/enterprise/setting-up-entra/oidc.mdx`:
- Around line 203-217: The manifest-based path in the OIDC setup is incomplete:
the note references a nonexistent later step and does not show how to configure
the required roles claim. Update the guidance around the Token configuration /
Manifest editor flow to either remove the shortcut or explicitly document the
manifest settings, including the `optionalClaims.idToken` `roles` entry and any
related claim/version fields, so readers can follow the manifest route without
missing the `roles` claim. Use the existing OIDC setup section and the manifest
note as the anchors for this fix.
In `@docs/enterprise/setting-up-entra/scim.mdx`:
- Line 6: The intro and related SCIM sections overstate immediate “real-time”
propagation, which conflicts with the later cycle-based timing details. Update
the affected prose in scim.mdx around the SCIM overview and the sections
referenced by the comment to describe Entra sync as incremental/cycle-based, and
make the wording consistent with the documented ~40 minute cycles and initial
sync delay. Use the SCIM overview text and the later timing explanation as the
source of truth, and keep terminology aligned with the provider behavior.
In `@docs/enterprise/setting-up-generic-oidc/scim.mdx`:
- Line 6: The SCIM overview text in the docs overstates provider behavior by
implying real-time push semantics for all compatible IdPs. Update the wording in
the SCIM guide to describe cycle-based provisioning unless the behavior is
guaranteed across all supported providers, and keep the description aligned with
the provider-specific behavior noted later in the same document. Refer to the
SCIM intro copy in the generic OIDC SCIM page and ensure it matches the
documented provisioning flow used by the relevant SCIM provider sections.
In `@docs/enterprise/setting-up-google-workspace/scim.mdx`:
- Around line 2-6: The page metadata and intro in scim.mdx are misleading
because they describe real-time SCIM provisioning, but the documented behavior
is Google Workspace reconciliation via Directory API and session refreshes.
Update the title, description, and opening copy in the SCIM page to match the
actual implementation, and reframe it around Google Workspace provisioning or
SCIM-equivalent sync rather than a true SCIM endpoint. Use the existing section
content in the page to align the wording with the implemented sync flow.
- Around line 132-135: The troubleshooting note in the Google Workspace SCIM
docs uses the wrong scope name for group membership syncing. In the “Group
memberships not syncing” guidance, replace the Microsoft Graph permission
reference with the Google Directory API Domain-Wide Delegation scope used by
this integration, and keep the wording aligned with the existing SCIM/provider
setup guidance so admins verify the correct authorization in Google Admin
Console.
In `@docs/enterprise/setting-up-okta/scim.mdx`:
- Around line 169-172: The Okta SCIM mapping example is inconsistent between the
body text and the figure caption, so update the example in the surrounding prose
and the Frame caption to use the same source field. In the section describing
the attribute mapping, align the `user.employeeNumber`/`user.employeeID`
reference with the actual example shown, or explicitly note that `employeeID` is
a custom Okta profile field. Make the wording in the SCIM mapping instructions
and the caption for the employeeID image match exactly so the flow stays
unambiguous.
In `@docs/enterprise/setting-up-zitadel/oidc.mdx`:
- Around line 68-87: The Step 2 content in the Zitadel OIDC guide is marked
optional, but it also contains the only instruction to capture the Project ID
that later setup requires unconditionally. Update the wording around the Step 2
section and the “Note the Project ID” guidance so readers are told to record it
even if they skip role-claim setup, while keeping the optional note limited to
the role-claim-specific parts of the step. Use the existing Step/Note/Frame
structure in oidc.mdx to keep the flow consistent with the later provider
configuration steps.
In `@docs/enterprise/user-provisioning.mdx`:
- Around line 46-47: The Google Workspace card is pointing to a route that does
not exist in the current docs structure. Update the Card in the user
provisioning page so its href matches the actual Google Workspace guide
entrypoint, using the existing OIDC/SCIM docs path pattern rather than the
outdated setting-up-google-workspace link.
---
Outside diff comments:
In `@core/bifrost.go`:
- Around line 3533-3608: Validate and normalize the autoscaling settings in
UpdateProvider before using DynamicScaling, MaxWorkers, or ScalingInterval,
since this hot-reload path currently bypasses the same checks used by
prepareProvider. Apply the existing ValidateDynamicScalingConfig-style
clamping/disabling logic on the providerConfig.ConcurrencyAndBufferSize values
before creating newPq, allocating newPq.quit, or starting DynamicWorkerScaling,
so invalid reload values cannot trigger ticker or channel panics.
---
Nitpick comments:
In `@core/schemas/provider.go`:
- Around line 230-238: The struct field comments in provider schema contain
typos and inconsistent wording that should be cleaned up for clarity. Update the
comments on Concurrency, DynamicScaling, ScalingInterval, MinWorkers,
MaxWorkers, ScaleUpThreshold, and ScaleDownThreshold to use correct spelling
(responses, necessary, goroutine) and consistent phrasing, especially the
DynamicScaling description and the “Number of workers” wording. Keep the meaning
the same while making the comment text professional and easy to read.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: c16e0618-49bc-4b90-a638-25e291c43d41
⛔ Files ignored due to path filters (145)
docs/media/auth0-card.svgis excluded by!**/*.svgdocs/media/auth0-icon.svgis excluded by!**/*.svgdocs/media/keycloak-card.svgis excluded by!**/*.svgdocs/media/keycloak-icon.svgis excluded by!**/*.svgdocs/media/okta-card.svgis excluded by!**/*.svgdocs/media/okta-icon.svgis excluded by!**/*.svgdocs/media/ui-provider-key-vault-ref.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/auth0-action-code.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/auth0-action-flow.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/auth0-actions-triggers.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/auth0-app-credentials.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/auth0-app-settings.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/auth0-applications.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/auth0-callback-urls.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/auth0-m2m-credentials.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/bifrost-attribute-mapping.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/bifrost-choose-provider.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/bifrost-discover-claims.pngis excluded by!**/*.pngdocs/media/user-provisioning/auth0/bifrost-review-enable.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-api-permissions.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-app-information.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-bifrost-attribute-mappings.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-bifrost-choose-provider.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-bifrost-discover-claims.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-create-app-roles.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-create-client-secret.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-enable-assignment.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-enterprise-applications-list.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-redirect-uri-config.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-register-application.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-token-configuration.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra-user-assignments.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/bifrost-attribute-setup.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/bifrost-discover-claims.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/bifrost-provider-selection.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-api-permissions.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-app-overview.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-app-roles.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-client-secret.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-enable-assignment.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-enterprise-apps-list.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-redirect-uri.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-register-app.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-token-configuration.pngis excluded by!**/*.pngdocs/media/user-provisioning/entra/entra-oidc-user-assignments.pngis excluded by!**/*.pngdocs/media/user-provisioning/generic-oidc/bifrost-attribute-setup.pngis excluded by!**/*.pngdocs/media/user-provisioning/generic-oidc/bifrost-discover-claims.pngis excluded by!**/*.pngdocs/media/user-provisioning/generic-oidc/bifrost-enable-scim.pngis excluded by!**/*.pngdocs/media/user-provisioning/generic-oidc/bifrost-provider-config.pngis excluded by!**/*.pngdocs/media/user-provisioning/generic-oidc/bifrost-provider-dashboard.pngis excluded by!**/*.pngdocs/media/user-provisioning/generic-oidc/bifrost-provider-selection.pngis excluded by!**/*.pngdocs/media/user-provisioning/generic-oidc/bifrost-review-enable.pngis excluded by!**/*.pngdocs/media/user-provisioning/generic-oidc/bifrost-scim-token-dialog.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/bifrost-keycloak-attribute-mapping.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/bifrost-keycloak-discover-claims.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/bifrost-keycloak-provider-config.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/bifrost-sync-users.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/bifrost-user-provisioning.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/bifrost-users-page.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-client-config.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-client-creation.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-client-scopes.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-configure-new-mapper.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-create-client.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-groups-mapping.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-login-settings.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-realm-role-config.pngis excluded by!**/*.pngdocs/media/user-provisioning/keycloak/keycloak-realm-role-mapping.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-api-token-created.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-app-configuration.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-app-group-claim-setup.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-app-integration-main-page.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-assign-custom-role.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-assign-users.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-authorization-server.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-claim-addition.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-create-app.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-create-groups.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-create-token-form.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-custom-attribute-creation.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-form.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-group-claim-mapping.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-group-configuration.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-groups-created.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-groups-page.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-org-server-group-in-claims.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-profile-editor-screen.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-scim-assignments.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-scim-configure-api.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-scim-configure-provisioning.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-scim-profile-editor-add-attribute.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-scim-profile-editor-mapping.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-scim-profile-editor-page.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-scim-push-groups.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-select-auth-server.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-tokens-screen.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta-user-type-creation.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-attribute-setup.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-custom-user-attribute-scim.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-discover-claims.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-enable-scim.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-okta-provider-config.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-provider-selection.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-provider-setting.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-review-oidc.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/bifrost-scim-provisioning-token-dialog.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-api-custom-claim-oauth.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-api-custom-oauth.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-app-settings-signon.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-browse-app-catalog.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-choose-scim-header-auth.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-custom-attribute-employee-id.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-custom-attribute-mapping-employeeid.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-custom-attribute-mapping.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-oidc-api-token.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-oidc-app-settings.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-oidc-applications.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-oidc-assignments.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-oidc-create-app.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-oidc-credentials.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-oidc-group-assignment.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-profile-editor-add-attribute.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-scim-assignments.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-scim-custom-attribute-mapping-button.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-scim-provisioning-setup.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-scim-provisioning-to-app.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-scim-push-groups-active.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-scim-push-groups-by-rule.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-scim-push-groups.pngis excluded by!**/*.pngdocs/media/user-provisioning/okta/okta-token-claims.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/bifrost-choose-provider.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-attribute-mapping.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-client-id.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-create-app.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-project-roles.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-provider-config.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-redirect-uris.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-return-user-roles.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-review-enable.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-role-assignments.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-service-account-key.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-service-account-role.pngis excluded by!**/*.pngdocs/media/user-provisioning/zitadel/zitadel-token-settings.pngis excluded by!**/*.pngdocs/media/zitadel-card.svgis excluded by!**/*.svgdocs/media/zitadel-icon.svgis excluded by!**/*.svg
📒 Files selected for processing (38)
.github/workflows/release-pipeline.ymlcore/bifrost.gocore/bifrost_test.gocore/schemas/provider.gocore/utils.godocs/changelogs/ent-v1.5.0.mdxdocs/changelogs/helm-v2.1.25.mdxdocs/deployment-guides/config-json.mdxdocs/deployment-guides/config-json/secret-management.mdxdocs/deployment-guides/helm.mdxdocs/deployment-guides/helm/secret-management.mdxdocs/docs.jsondocs/enterprise/clustering.mdxdocs/enterprise/secret-management.mdxdocs/enterprise/setting-up-auth0/oidc.mdxdocs/enterprise/setting-up-entra.mdxdocs/enterprise/setting-up-entra/oidc.mdxdocs/enterprise/setting-up-entra/scim.mdxdocs/enterprise/setting-up-generic-oidc/oidc.mdxdocs/enterprise/setting-up-generic-oidc/scim.mdxdocs/enterprise/setting-up-google-workspace/oidc.mdxdocs/enterprise/setting-up-google-workspace/scim.mdxdocs/enterprise/setting-up-keycloak.mdxdocs/enterprise/setting-up-keycloak/oidc.mdxdocs/enterprise/setting-up-okta.mdxdocs/enterprise/setting-up-okta/oidc.mdxdocs/enterprise/setting-up-okta/scim.mdxdocs/enterprise/setting-up-zitadel.mdxdocs/enterprise/setting-up-zitadel/oidc.mdxdocs/enterprise/user-provisioning.mdxdocs/mcp/auth/headers.mdxdocs/openapi/openapi.jsondocs/openapi/openapi.yamldocs/openapi/paths/management/vault.yamlhelm-charts/bifrost/README.mdtests/cmd/e2eseed/go.modtests/cmd/seed/go.modtests/cmd/seedvks/go.mod
💤 Files with no reviewable changes (4)
- docs/enterprise/setting-up-zitadel.mdx
- docs/enterprise/setting-up-okta.mdx
- docs/enterprise/setting-up-keycloak.mdx
- docs/enterprise/setting-up-entra.mdx
Comment on lines
+3057
to
+3094
| testDone := make(chan struct{}) | ||
| reqCtx, cancelReq := context.WithCancel(context.Background()) | ||
| dummyReq := &ChannelMessage{ | ||
| Context: schemas.NewBifrostContext(reqCtx, time.Time{}), | ||
| Err: make(chan schemas.BifrostError, 1), | ||
| Response: make(chan *schemas.BifrostResponse, 1), | ||
| } | ||
| go func() { | ||
|
|
||
| //Using the same dummyReq pauses other workers while they push their errors in the Err chan. This keeps the | ||
| //provider queue full and allows us to test our scale up feature. At the end we call cancel to cancel all the | ||
| //requests, free our workers and shutdown. | ||
| for i := 0; i < test.NumRequests; i++ { | ||
| select { | ||
| case <-testDone: | ||
| return | ||
| case pq.queue <- dummyReq: | ||
| } | ||
| } | ||
|
|
||
| }() | ||
| time.Sleep(100 * time.Millisecond) | ||
|
|
||
| assert.Eventually(t, func() bool { | ||
| return int(pq.ActiveWorkers.Load()) == test.MaxWorkers | ||
| }, 3*time.Second, 250*time.Millisecond, | ||
| "test %q: workers failed to scale up to %d within timeout. Final count: %d", | ||
| test.Name, test.MaxWorkers, pq.ActiveWorkers.Load()) | ||
| close(testDone) | ||
| cancelReq() | ||
| go func() { | ||
| for { | ||
| select { | ||
| case <-dummyReq.Err: | ||
| case <-dummyReq.Response: | ||
| } | ||
| } | ||
| }() |
Contributor
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Queue fresh requests here instead of reusing one ChannelMessage.
This enqueues the same mutable request into the worker pool dozens of times. requestWorker writes per-request state back onto the message/context, so the test can race and turn flaky under concurrency. Create a fresh ChannelMessage per enqueue and share only the cancelable parent context.
As per coding guidelines, Go review should enforce race-safe shared state and deterministic tests.
Suggested change
- testDone := make(chan struct{})
- reqCtx, cancelReq := context.WithCancel(context.Background())
- dummyReq := &ChannelMessage{
- Context: schemas.NewBifrostContext(reqCtx, time.Time{}),
- Err: make(chan schemas.BifrostError, 1),
- Response: make(chan *schemas.BifrostResponse, 1),
- }
+ testDone := make(chan struct{})
+ reqCtx, cancelReq := context.WithCancel(context.Background())
go func() {
-
- //Using the same dummyReq pauses other workers while they push their errors in the Err chan. This keeps the
- //provider queue full and allows us to test our scale up feature. At the end we call cancel to cancel all the
- //requests, free our workers and shutdown.
for i := 0; i < test.NumRequests; i++ {
+ dummyReq := &ChannelMessage{
+ Context: schemas.NewBifrostContext(reqCtx, time.Time{}),
+ Err: make(chan schemas.BifrostError),
+ Response: make(chan *schemas.BifrostResponse),
+ }
select {
case <-testDone:
return
case pq.queue <- dummyReq:
}
}
}()
@@
- go func() {
- for {
- select {
- case <-dummyReq.Err:
- case <-dummyReq.Response:
- }
- }
- }()
-
bifrost.Shutdown()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@core/bifrost_test.go` around lines 3057 - 3094, The scaling test reuses a
single mutable ChannelMessage across many queue submissions, which can race
because requestWorker mutates per-request state on the message and its context.
Update the enqueue loop in the test to create a fresh ChannelMessage for each
send while still deriving each one from the same cancelable parent context
(reqCtx/cancelReq), and keep the existing cleanup/shutdown logic around testDone
and cancelReq.
Source: Coding guidelines
Comment on lines
+488
to
+494
| { | ||
| "group": "Google Workspace", | ||
| "icon": "google", | ||
| "collapsed": true, | ||
| "pages": [ | ||
| "enterprise/setting-up-google-workspace" | ||
| ] |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Update the Google Workspace nav slug to the renamed OIDC page.
This stack adds enterprise/setting-up-google-workspace/oidc and keeps a redirect for the legacy slug, but the navigation still points at enterprise/setting-up-google-workspace. Keeping the menu on the redirect source leaves docs/docs.json out of sync with the actual page layout.
As per path instructions, "Verify new or renamed Mintlify pages are reflected in docs/docs.json where appropriate."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/docs.json` around lines 488 - 494, The Google Workspace navigation entry
is still pointing to the legacy redirect source instead of the renamed OIDC
page. Update the page reference in docs/docs.json under the Google Workspace
group to use the new enterprise/setting-up-google-workspace/oidc slug so the nav
matches the current page layout and stays aligned with the renamed Mintlify
page.
Source: Path instructions
Comment on lines
+203
to
+217
| <Note> | ||
| If you prefer to configure claims via the App Manifest JSON in Step 7, you can skip this step — the manifest overrides UI-based token configuration. | ||
| </Note> | ||
|
|
||
| In your app registration, go to **Token configuration** and click **Add groups claim**. | ||
|
|
||
| Select **Security groups** or **Groups assigned to the application**, enable **ID** and **Access** token types, and click **Add**. | ||
|
|
||
| <Frame caption="Token configuration — groups claim added for ID and Access tokens."> | ||
| <img src="/media/user-provisioning/entra/entra-oidc-token-configuration.png" alt="Token configuration page showing the groups claim configured for ID, Access, and SAML tokens" /> | ||
| </Frame> | ||
|
|
||
| <Note> | ||
| If you configure claims via the **Manifest** editor instead, also set `"requestedAccessTokenVersion": 2` (or `"accessTokenAcceptedVersion": 2` for legacy registrations) and `"groupMembershipClaims": "ApplicationGroup"` to restrict the groups claim to only app-assigned groups. | ||
| </Note> |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Document the manifest-based roles claim path or drop the manifest shortcut.
Step 6 tells readers they can skip the UI flow and configure claims via the app manifest, but it points to a nonexistent “Step 7” manifest step. Later, troubleshooting requires roles under optionalClaims.idToken, yet this page never shows that manifest entry. Following the manifest path as written can leave Entra users without a roles claim and blocked at login.
As per path instructions, "Check docs for parity with code, config.schema.json, and provider behavior."
Also applies to: 401-405
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/enterprise/setting-up-entra/oidc.mdx` around lines 203 - 217, The
manifest-based path in the OIDC setup is incomplete: the note references a
nonexistent later step and does not show how to configure the required roles
claim. Update the guidance around the Token configuration / Manifest editor flow
to either remove the shortcut or explicitly document the manifest settings,
including the `optionalClaims.idToken` `roles` entry and any related
claim/version fields, so readers can follow the manifest route without missing
the `roles` claim. Use the existing OIDC setup section and the manifest note as
the anchors for this fix.
Source: Path instructions
| description: "Enable real-time user and group provisioning from Microsoft Entra ID to Bifrost Enterprise using SCIM 2.0." | ||
| --- | ||
|
|
||
| SCIM (System for Cross-domain Identity Management) keeps Bifrost in sync with Entra in real time - new users are provisioned, deactivated users are suspended, and group memberships are updated without waiting for the next login or background sync. |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Align the sync-timing claims with the later cycle-based explanation.
This page promises “real-time” provisioning/deactivation early on, but later says Entra applies changes on incremental cycles (~40 minutes) and that the initial sync can take up to 4 hours. Please reword the earlier sections to match that cycle-based behavior so operators do not expect immediate propagation. As per path instructions, "Check docs for parity with code, config.schema.json, and provider behavior."
Also applies to: 156-156, 184-195
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/enterprise/setting-up-entra/scim.mdx` at line 6, The intro and related
SCIM sections overstate immediate “real-time” propagation, which conflicts with
the later cycle-based timing details. Update the affected prose in scim.mdx
around the SCIM overview and the sections referenced by the comment to describe
Entra sync as incremental/cycle-based, and make the wording consistent with the
documented ~40 minute cycles and initial sync delay. Use the SCIM overview text
and the later timing explanation as the source of truth, and keep terminology
aligned with the provider behavior.
Source: Path instructions
| description: "Enable real-time user and group provisioning from any SCIM 2.0-capable identity provider to Bifrost Enterprise." | ||
| --- | ||
|
|
||
| Bifrost exposes a SCIM 2.0 endpoint that any compatible identity provider can push user and group changes to in real time — new users are provisioned, deactivated users are suspended, and group memberships are updated without waiting for the next background sync cycle. |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Avoid promising real-time SCIM behavior for every IdP.
This guide is provider-agnostic, but it currently claims immediate push semantics while also noting that many IdPs apply changes on their next provisioning cycle. Please soften the wording to cycle-based behavior unless real-time sync is guaranteed across all supported SCIM providers. As per path instructions, "Check docs for parity with code, config.schema.json, and provider behavior."
Also applies to: 115-115, 156-158
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/enterprise/setting-up-generic-oidc/scim.mdx` at line 6, The SCIM
overview text in the docs overstates provider behavior by implying real-time
push semantics for all compatible IdPs. Update the wording in the SCIM guide to
describe cycle-based provisioning unless the behavior is guaranteed across all
supported providers, and keep the description aligned with the provider-specific
behavior noted later in the same document. Refer to the SCIM intro copy in the
generic OIDC SCIM page and ensure it matches the documented provisioning flow
used by the relevant SCIM provider sections.
Source: Path instructions
Comment on lines
+2
to
+6
| title: "Setup SCIM" | ||
| description: "Enable real-time user and group provisioning from Google Workspace to Bifrost Enterprise." | ||
| --- | ||
|
|
||
| SCIM keeps Bifrost in sync with Google Workspace in real time - new users are provisioned, deactivated accounts are suspended, and group memberships are updated without waiting for the next background sync cycle. |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
This page is not actually describing SCIM.
The title/description promise real-time SCIM provisioning, but the body says Google Workspace does not push SCIM and that Bifrost relies on 24-hour Directory API reconciliation plus 15-minute session refreshes. Please rename/reframe this as Google Workspace provisioning or SCIM-equivalent sync so readers do not expect a true SCIM endpoint. Based on learnings, I checked the surrounding stack first; as per path instructions, docs should match the implemented behavior.
Also applies to: 16-23
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/enterprise/setting-up-google-workspace/scim.mdx` around lines 2 - 6, The
page metadata and intro in scim.mdx are misleading because they describe
real-time SCIM provisioning, but the documented behavior is Google Workspace
reconciliation via Directory API and session refreshes. Update the title,
description, and opening copy in the SCIM page to match the actual
implementation, and reframe it around Google Workspace provisioning or
SCIM-equivalent sync rather than a true SCIM endpoint. Use the existing section
content in the page to align the wording with the implemented sync flow.
Sources: Path instructions, Learnings
Comment on lines
+132
to
+135
| **Users not appearing in Import Preview** - confirm the service account has the three Directory API scopes authorized in Domain-Wide Delegation. Check that the `adminEmail` is a Super Admin or has the **User Management Admin** role. | ||
|
|
||
| **Group memberships not syncing** - verify `GroupMember.Read.All` scope is included in Domain-Wide Delegation. Groups must be assigned to the Bifrost application in the Google Admin Console for the `groups` claim to appear in the token. | ||
|
|
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Use the Google Directory API scope name here.
GroupMember.Read.All is a Microsoft Graph permission, not a Google Workspace Domain-Wide Delegation scope. This troubleshooting step should point back to https://www.googleapis.com/auth/admin.directory.group.member.readonly, otherwise admins will search for the wrong permission and fail setup. As per path instructions, docs should stay aligned with the actual provider behavior and configuration surface.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/enterprise/setting-up-google-workspace/scim.mdx` around lines 132 - 135,
The troubleshooting note in the Google Workspace SCIM docs uses the wrong scope
name for group membership syncing. In the “Group memberships not syncing”
guidance, replace the Microsoft Graph permission reference with the Google
Directory API Domain-Wide Delegation scope used by this integration, and keep
the wording aligned with the existing SCIM/provider setup guidance so admins
verify the correct authorization in Google Admin Console.
Source: Path instructions
Comment on lines
+169
to
+172
| Scroll to find your attribute and set its source from the Okta user profile — e.g. `user.employeeNumber` → `employeeID`. Click **Save Mappings**. | ||
|
|
||
| <Frame caption="Mapping user.employeeID from the Okta user profile to the employeeID SCIM attribute."> | ||
| <img src="/media/user-provisioning/okta/okta-custom-attribute-mapping-employeeid.png" alt="Attribute mapping row showing user.employeeID mapped to the employeeID SCIM attribute" /> |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Use one source-field example here.
This step switches from user.employeeNumber in the text to user.employeeID in the caption. Please make both examples match, or explicitly call out that employeeID is a custom Okta profile field, otherwise the mapping instructions are ambiguous.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/enterprise/setting-up-okta/scim.mdx` around lines 169 - 172, The Okta
SCIM mapping example is inconsistent between the body text and the figure
caption, so update the example in the surrounding prose and the Frame caption to
use the same source field. In the section describing the attribute mapping,
align the `user.employeeNumber`/`user.employeeID` reference with the actual
example shown, or explicitly note that `employeeID` is a custom Okta profile
field. Make the wording in the SCIM mapping instructions and the caption for the
employeeID image match exactly so the flow stays unambiguous.
Comment on lines
+68
to
+87
| ## Step 2: Enable role claims on the project (optional) | ||
|
|
||
| <Note> | ||
| Skip this step if you plan to map roles using a different claim (e.g. groups or a custom attribute) rather than Zitadel project roles. | ||
| </Note> | ||
|
|
||
| <Steps> | ||
|
|
||
| <Step title="Enable Return user roles during authentication"> | ||
|
|
||
| Open **Projects → your project → General** and enable: | ||
|
|
||
| - **Return user roles during authentication** - required for role claims to appear in the token | ||
| - **Only authorized users can authenticate** - enforces that every user has at least one project role | ||
|
|
||
| <Frame caption="Project General settings - enable Return user roles during authentication to include role claims in the token."> | ||
| <img src="/media/user-provisioning/zitadel/zitadel-return-user-roles.png" alt="Zitadel project General settings showing Return user roles during authentication checkbox enabled" /> | ||
| </Frame> | ||
|
|
||
| Note the **Project ID** - you will need it for the Bifrost configuration. |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Don't hide Project ID behind an optional step.
Step 6 requires Project ID unconditionally, but Step 2 is marked optional and is currently the only place this page tells readers to capture that value. Anyone skipping the role-claim setup still has to do Step 2 to finish provider configuration.
As per path instructions, "Check docs for parity with code, config.schema.json, and provider behavior."
Also applies to: 215-219
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/enterprise/setting-up-zitadel/oidc.mdx` around lines 68 - 87, The Step 2
content in the Zitadel OIDC guide is marked optional, but it also contains the
only instruction to capture the Project ID that later setup requires
unconditionally. Update the wording around the Step 2 section and the “Note the
Project ID” guidance so readers are told to record it even if they skip
role-claim setup, while keeping the optional note limited to the
role-claim-specific parts of the step. Use the existing Step/Note/Frame
structure in oidc.mdx to keep the flow consistent with the later provider
configuration steps.
Source: Path instructions
Comment on lines
46
to
47
| <Card title="Google Workspace" icon="google" href="/enterprise/setting-up-google-workspace"> | ||
| Google Workspace domains with OAuth login plus optional Directory API sync via a service account. |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Point the Google Workspace card at the actual guide page.
href="/enterprise/setting-up-google-workspace" does not match the docs structure introduced in this stack, which exposes .../oidc and .../scim pages. As written, this card is likely to land on a missing route instead of the OIDC entrypoint. As per path instructions, docs should stay in parity with the actual docs structure.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/enterprise/user-provisioning.mdx` around lines 46 - 47, The Google
Workspace card is pointing to a route that does not exist in the current docs
structure. Update the Card in the user provisioning page so its href matches the
actual Google Workspace guide entrypoint, using the existing OIDC/SCIM docs path
pattern rather than the outdated setting-up-google-workspace link.
Source: Path instructions
Introduces autonomous scaling for provider queues to handle traffic spikes and scale down during idle periods. A dedicated background goroutine periodically evaluates queue capacity against configurable thresholds to spin up or terminate workers. - Refactored `ConcurrencyAndBufferSize` in schemas to include scaling bounds (MinWorkers, MaxWorkers), thresholds, and intervals. - Implemented custom JSON marshal/unmarshal for the config struct to safely handle `time.Duration` conversion to/from milliseconds. - Integrated scaling lifecycle logic into `prepareProvider`, `updateProvider`, and `requestWorker`. - Added initial test coverage validating configuration bounds and extreme load scenarios (0% and 100% capacity). Affected packages: - core/bifrost.go - Scaling goroutine and worker lifecycle integration - core/schemas/provider.go - Configuration structs and JSON serialization - core/utils.go - Utility support for scaling logic - core/bifrost_test.go - Boundary and configuration validation tests
Prateek-Gupta001
force-pushed
the
feat/dynamic-worker-autoscaler-v2
branch
from
June 30, 2026 07:41
73a12ad to
16f1871
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ui/app/workspace/providers/views/modelProviderKeysTableView.tsx (1)
234-241: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winInclude all Bedrock Mantle secret fields in the warning detector.
Line 239 only checks
region; unresolved secret refs foraccess_key,secret_key,session_token,role_arn,external_id, orsession_namewill still show as a generic list-model error instead of the actionable secret-resolution warning.Proposed fix
const hasSecretVarConfig = (key.azure_key_config?.endpoint?.type && key.azure_key_config.endpoint.type !== "plain_text") || (key.vertex_key_config?.project_id?.type && key.vertex_key_config.project_id.type !== "plain_text") || (key.vertex_key_config?.region?.type && key.vertex_key_config.region.type !== "plain_text") || (key.bedrock_key_config?.region?.type && key.bedrock_key_config.region.type !== "plain_text") || - (key.bedrock_mantle_key_config?.region?.type && key.bedrock_mantle_key_config.region.type !== "plain_text") || + (key.bedrock_mantle_key_config?.access_key?.type && key.bedrock_mantle_key_config.access_key.type !== "plain_text") || + (key.bedrock_mantle_key_config?.secret_key?.type && key.bedrock_mantle_key_config.secret_key.type !== "plain_text") || + (key.bedrock_mantle_key_config?.session_token?.type && key.bedrock_mantle_key_config.session_token.type !== "plain_text") || + (key.bedrock_mantle_key_config?.region?.type && key.bedrock_mantle_key_config.region.type !== "plain_text") || + (key.bedrock_mantle_key_config?.role_arn?.type && key.bedrock_mantle_key_config.role_arn.type !== "plain_text") || + (key.bedrock_mantle_key_config?.external_id?.type && key.bedrock_mantle_key_config.external_id.type !== "plain_text") || + (key.bedrock_mantle_key_config?.session_name?.type && key.bedrock_mantle_key_config.session_name.type !== "plain_text") || (key.vllm_key_config?.url?.type && key.vllm_key_config.url.type !== "plain_text") || (key.value?.type && key.value.type !== "plain_text");As per path instructions, “For
ui/**, check interactive workflows for loading, empty, error, and success states.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ui/app/workspace/providers/views/modelProviderKeysTableView.tsx` around lines 234 - 241, The warning detector in modelProviderKeysTableView.tsx only checks bedrock_mantle_key_config.region, so unresolved Bedrock Mantle secret refs in access_key, secret_key, session_token, role_arn, external_id, and session_name are missed. Update the hasSecretVarConfig logic in the relevant render path to include all Bedrock Mantle key config fields alongside the existing Azure, Vertex, Bedrock, VLLM, and value checks so these cases surface the actionable secret-resolution warning instead of the generic list-model error.Source: Path instructions
🧹 Nitpick comments (2)
core/bifrost.go (2)
143-143: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueRemove the leftover dev TODO.
// TODO: Now just write the nc.Copy things in the provider.go files!!reads like a personal note and references nothing concrete in this function. It will confuse future readers of the autoscaler.
Want me to open a tracking issue if this references unfinished work, or should it just be deleted?🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/bifrost.go` at line 143, Remove the leftover dev TODO comment from the relevant area in bifrost-related code, since it is an undocumented personal note rather than actionable guidance. If there is still unfinished work, replace it with a concrete, scoped note tied to the specific provider.go or nc.Copy logic; otherwise delete it entirely so the surrounding code in the bifrost/autoscaler path remains clean.
175-178: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueBoundary log lines will spam at Info on every tick.
When a provider sits at
MaxWorkersunder sustained load (orMinWorkerswhile idle), these branches emit an Info log on everyScalingIntervaltick indefinitely. Consider logging the "already at max/min" cases at Debug, or only when the state transitions.Also applies to: 201-204
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/bifrost.go` around lines 175 - 178, The boundary cases in bifrost scaling are logging too loudly on every tick, so adjust the logging in the worker-scaling branches around the MaxWorkers and MinWorkers checks inside bifrost’s scaling logic to avoid Info-level spam. Update the relevant messages in the Bifrost scaling path (including the max-worker and min-worker “already at limit” branches) to Debug, or gate them so they only log on state transitions, while keeping the existing scaling behavior unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/bifrost.go`:
- Around line 150-151: The provider goroutine is reloading the wait group from
bifrost.waitGroups instead of using the waitGroup argument already tied to this
pq, which can point workers at the wrong shutdown barrier if the map entry
changes. Update the logic in the provider-handling flow to use the existing
waitGroup parameter directly where currentWaitGroup is set, and remove the extra
Load from bifrost.waitGroups in that path.
In `@core/providers/anthropic/requestbuilder.go`:
- Around line 99-114: The comment above the schemas.Bedrock entry is mislabeled
and currently describes Bedrock Mantle instead of Bedrock itself. Update the
block comment in RequestBuilder’s provider-features map so it clearly documents
schemas.Bedrock behavior, and leave the separate schemas.BedrockMantle comment
to explain the Mantle-specific entry; this keeps the two adjacent
RemapToolVersions settings unambiguous.
---
Outside diff comments:
In `@ui/app/workspace/providers/views/modelProviderKeysTableView.tsx`:
- Around line 234-241: The warning detector in modelProviderKeysTableView.tsx
only checks bedrock_mantle_key_config.region, so unresolved Bedrock Mantle
secret refs in access_key, secret_key, session_token, role_arn, external_id, and
session_name are missed. Update the hasSecretVarConfig logic in the relevant
render path to include all Bedrock Mantle key config fields alongside the
existing Azure, Vertex, Bedrock, VLLM, and value checks so these cases surface
the actionable secret-resolution warning instead of the generic list-model
error.
---
Nitpick comments:
In `@core/bifrost.go`:
- Line 143: Remove the leftover dev TODO comment from the relevant area in
bifrost-related code, since it is an undocumented personal note rather than
actionable guidance. If there is still unfinished work, replace it with a
concrete, scoped note tied to the specific provider.go or nc.Copy logic;
otherwise delete it entirely so the surrounding code in the bifrost/autoscaler
path remains clean.
- Around line 175-178: The boundary cases in bifrost scaling are logging too
loudly on every tick, so adjust the logging in the worker-scaling branches
around the MaxWorkers and MinWorkers checks inside bifrost’s scaling logic to
avoid Info-level spam. Update the relevant messages in the Bifrost scaling path
(including the max-worker and min-worker “already at limit” branches) to Debug,
or gate them so they only log on state transitions, while keeping the existing
scaling behavior unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 78968ae9-76af-4c22-9bef-dc9766505005
📒 Files selected for processing (119)
.github/workflows/release-pipeline.ymlcore/bifrost.gocore/bifrost_test.gocore/changelog.mdcore/internal/llmtests/account.gocore/internal/mcptests/connect_ping_listtools_test.gocore/mcp/clientmanager.gocore/providers/anthropic/advisor_test.gocore/providers/anthropic/anthropic.gocore/providers/anthropic/chat.gocore/providers/anthropic/requestbuilder.gocore/providers/anthropic/requestbuilder_test.gocore/providers/anthropic/responses.gocore/providers/anthropic/types.gocore/providers/anthropic/utils.gocore/providers/anthropic/utils_test.gocore/providers/azure/azure.gocore/providers/azure/utils.gocore/providers/bedrock/bedrock.gocore/providers/bedrock/mantle.gocore/providers/bedrock/mantle_test.gocore/providers/bedrock/streambuffering_test.gocore/providers/bedrock/transport_test.gocore/providers/bedrockmantle/bedrockmantle.gocore/providers/bedrockmantle/bedrockmantle_test.gocore/providers/bedrockmantle/utils.gocore/providers/cerebras/cerebras.gocore/providers/fireworks/fireworks.gocore/providers/gemini/batch.gocore/providers/gemini/chat.gocore/providers/gemini/gemini_test.gocore/providers/gemini/responses.gocore/providers/gemini/utils.gocore/providers/gemini/videos.gocore/providers/groq/groq.gocore/providers/huggingface/huggingface.gocore/providers/mistral/mistral.gocore/providers/nebius/nebius.gocore/providers/ollama/ollama.gocore/providers/openai/large_payload.gocore/providers/openai/openai.gocore/providers/openai/tool_search_roundtrip_test.gocore/providers/openai/transcription.gocore/providers/openai/transcription_test.gocore/providers/opencode/opencode.gocore/providers/openrouter/openrouter.gocore/providers/parasail/parasail.gocore/providers/perplexity/perplexity.gocore/providers/sgl/sgl.gocore/providers/utils/bodysigner.gocore/providers/vertex/types.gocore/providers/vertex/utils.gocore/providers/vertex/utils_test.gocore/providers/vertex/vertex.gocore/providers/vertex/vertex_test.gocore/providers/vllm/vllm.gocore/providers/xai/xai.gocore/schemas/account.gocore/schemas/account_test.gocore/schemas/bifrost.gocore/schemas/chatcompletions.gocore/schemas/provider.gocore/schemas/responses.gocore/schemas/utils.gocore/schemas/utils_test.gocore/utils.godocs/changelogs/ent-v1.5.0.mdxdocs/changelogs/helm-v2.1.25.mdxdocs/deployment-guides/config-json/providers.mdxdocs/docs.jsondocs/openapi/openapi.jsondocs/openapi/schemas/inference/chat.yamldocs/providers/supported-providers/bedrock-mantle.mdxdocs/providers/supported-providers/overview.mdxframework/changelog.mdframework/configstore/clientconfig.goframework/configstore/migrations.goframework/configstore/tables/key.goframework/logstore/hybrid_test.goframework/logstore/tables.goframework/logstore/tables_test.gohelm-charts/bifrost/README.mdtests/cmd/e2eseed/go.modtests/cmd/seed/go.modtests/cmd/seedvks/go.modtests/e2e/api/collections/provider-harness.jsontests/e2e/api/provider-capabilities.jsontests/e2e/api/provider_config/bifrost-v1-bedrock-mantle.postman_environment.jsontests/e2e/api/runners/filter-collection.mjstests/e2e/api/runners/harness-monitor.mjstransports/bifrost-http/handlers/governance.gotransports/bifrost-http/handlers/governance_test.gotransports/bifrost-http/handlers/provider_keys.gotransports/bifrost-http/integrations/anthropic.gotransports/bifrost-http/integrations/openai.gotransports/bifrost-http/lib/config.gotransports/config.schema.jsonui/app/workspace/audit-logs/page.tsxui/app/workspace/complexity-router/page.tsxui/app/workspace/logs/page.tsxui/app/workspace/providers/dialogs/providerConfigSheet.tsxui/app/workspace/providers/fragments/apiKeysFormFragment.tsxui/app/workspace/providers/fragments/betaHeadersFormFragment.tsxui/app/workspace/providers/fragments/governanceFormFragment.tsxui/app/workspace/providers/page.tsxui/app/workspace/providers/views/modelProviderKeysTableView.tsxui/app/workspace/providers/views/providerKeyForm.tsxui/components/filters/logsFilterSidebar.tsxui/components/filters/mcpFilterSidebar.tsxui/components/ui/tagInput.tsxui/components/ui/truncatedLabel.tsxui/lib/constants/config.tsui/lib/constants/icons.tsxui/lib/constants/logs.tsui/lib/queryParamsParser.test.tsui/lib/queryParamsParser.tsui/lib/schemas/providerForm.tsui/lib/types/config.tsui/lib/types/schemas.ts
💤 Files with no reviewable changes (3)
- core/providers/vertex/types.go
- core/providers/vertex/utils.go
- core/providers/anthropic/utils.go
✅ Files skipped from review due to trivial changes (15)
- helm-charts/bifrost/README.md
- ui/app/workspace/audit-logs/page.tsx
- core/changelog.md
- framework/changelog.md
- core/providers/openai/tool_search_roundtrip_test.go
- ui/app/workspace/providers/dialogs/providerConfigSheet.tsx
- core/schemas/chatcompletions.go
- ui/lib/constants/config.ts
- ui/components/filters/mcpFilterSidebar.tsx
- docs/docs.json
- docs/providers/supported-providers/bedrock-mantle.mdx
- ui/components/filters/logsFilterSidebar.tsx
- docs/changelogs/helm-v2.1.25.mdx
- docs/deployment-guides/config-json/providers.mdx
- docs/changelogs/ent-v1.5.0.mdx
🚧 Files skipped from review as they are similar to previous changes (6)
- tests/cmd/seedvks/go.mod
- tests/cmd/e2eseed/go.mod
- tests/cmd/seed/go.mod
- core/schemas/provider.go
- .github/workflows/release-pipeline.yml
- core/bifrost_test.go
Comment on lines
+150
to
+151
| waitGroupValue, _ := bifrost.waitGroups.Load(providerName) | ||
| currentWaitGroup := waitGroupValue.(*sync.WaitGroup) |
Contributor
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
ast-grep run --lang go --pattern 'go bifrost.DynamicWorkerScaling($$$)' core/bifrost.go
rg -nP -C3 'DynamicWorkerScaling|waitGroups\.(Store|Load)' core/bifrost.goRepository: maximhq/bifrost
Length of output: 3621
🏁 Script executed:
#!/bin/bash
sed -n '144,206p' core/bifrost.go
printf '\n----\n'
sed -n '3516,3610p' core/bifrost.go
printf '\n----\n'
sed -n '4200,4248p' core/bifrost.goRepository: maximhq/bifrost
Length of output: 8230
🏁 Script executed:
#!/bin/bash
python3 - <<'PY'
from pathlib import Path
p = Path('core/bifrost.go')
text = p.read_text()
for needle in [
'func (bifrost *Bifrost) DynamicWorkerScaling',
'waitGroupValue, _ := bifrost.waitGroups.Load(providerName)',
'currentWaitGroup := waitGroupValue.(*sync.WaitGroup)',
'go bifrost.DynamicWorkerScaling(provider, providerConfig, newPq, newWaitGroup)',
'go bifrost.DynamicWorkerScaling(provider, config, pq, currentWaitGroup)',
]:
print(needle, text.find(needle))
PYRepository: maximhq/bifrost
Length of output: 490
Use the waitGroup parameter here instead of reloading from bifrost.waitGroups. A later provider update can swap the map entry before this goroutine starts, which sends workers to the wrong shutdown barrier. waitGroup is already the group tied to this pq.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@core/bifrost.go` around lines 150 - 151, The provider goroutine is reloading
the wait group from bifrost.waitGroups instead of using the waitGroup argument
already tied to this pq, which can point workers at the wrong shutdown barrier
if the map entry changes. Update the logic in the provider-handling flow to use
the existing waitGroup parameter directly where currentWaitGroup is set, and
remove the extra Load from bifrost.waitGroups in that path.
Source: Linters/SAST tools
Comment on lines
+99
to
+114
| // Bedrock Mantle native-Anthropic endpoint (/anthropic/v1/messages): the | ||
| // request is the native Anthropic Messages body, so model stays in the body | ||
| // (set to the bare Bedrock model id), the version is sent as an | ||
| // "anthropic-version" HTTP header rather than a body field, and stream is a | ||
| // body field. Tool type versions are still remapped to the canonical pair | ||
| // the hosted Claude generation expects. | ||
| schemas.Bedrock: { | ||
| RemapToolVersions: true, | ||
| }, | ||
| // Bedrock Mantle shares the Bedrock native-Anthropic request shape (model in | ||
| // body, anthropic-version HTTP header, tool versions remapped). It has its own | ||
| // entry so its feature surface in ProviderFeatures can diverge from Bedrock's | ||
| // Converse path without coupling the two. | ||
| schemas.BedrockMantle: { | ||
| RemapToolVersions: true, | ||
| }, |
Contributor
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Mislabeled comment on the schemas.Bedrock map entry.
The block comment above schemas.Bedrock opens with "Bedrock Mantle native-Anthropic endpoint…", yet there is a distinct schemas.BedrockMantle entry directly below it. This makes it ambiguous which provider the comment actually documents and obscures why both entries currently carry identical defaults (RemapToolVersions: true). Please retarget the comment to describe the Bedrock entry's behavior (and let the existing schemas.BedrockMantle comment cover Mantle).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@core/providers/anthropic/requestbuilder.go` around lines 99 - 114, The
comment above the schemas.Bedrock entry is mislabeled and currently describes
Bedrock Mantle instead of Bedrock itself. Update the block comment in
RequestBuilder’s provider-features map so it clearly documents schemas.Bedrock
behavior, and leave the separate schemas.BedrockMantle comment to explain the
Mantle-specific entry; this keeps the two adjacent RemapToolVersions settings
unambiguous.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Previously, the number of worker goroutines per provider in Bifrost was fixed at initialization time via the Concurrency field. Under heavy load, a provider with a small worker pool became a bottleneck, requests queued up faster than workers could drain them. During idle periods, pre-allocated goroutines sat doing nothing, wasting memory.
This PR implements opt-in dynamic worker pool auto-scaling per provider. A background goroutine monitors queue utilization at a configurable interval and adds or removes workers at runtime, bounded by MinWorkers and MaxWorkers. Providers that don't opt in see zero behavior change.
Closes #128
Changes:
core/bifrost.go
Design decisions: Buffer resizing is out of scope. It would require stop-the-world channel swap operation that cannot be done frequently. This was confirmed in the issue thread by @HzTTT and acknowledged by @Pratham-Mishra04.
Dynamic worker scaling achieves the same throughput benefit without touching the buffer.
core/schemas/provider.go
core/bifrost_test.go
Type of change
Affected areas
Breaking changes
All new fields are additive and zero-valued by default. Providers without DynamicScaling config are entirely unaffected.
Security considerations
No auth, secrets, or PII involved. The scaler only reads len(queue) and cap(queue) ,no request content is accessed.
How to test
New config changes:
Testing
Some tests are failing on main branch as well (unrelated to this PR).
No additional failures introduced by this change.
All tests in core (the package affected by this PR pass)
Linting
golangci-lintwhen done on core introduced existing issues.This PR does not introduce new lint errors in modified files.
##Notes
I will update the documentation once we finalise on certain small issues such as:
and other implementation based details.
Checklist
docs/contributing/README.mdand followed the guidelines