Conversation
<!-- This is an auto-generated comment: release notes by coderabbit.ai --> * **Bug Fixes** * Improved token parameter compatibility handling to preserve alternative formats when the primary option is unsupported. * **Chores** * Version updated to 2.0.0. * Enhanced load testing configuration for more reliable builds. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Bedrock was rejecting documents with "The PDF specified was not valid" because the document format was always resolved to `"pdf"` regardless of the actual file type. Standard OpenAI clients encode the MIME type inside the data URL (e.g. `data:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;base64,...`) rather than in the `file_type` field, which was the only source previously consulted. This PR fixes format resolution for both the Chat and Responses paths, unifies the mapping logic, and corrects several related data URL parsing defects. Fixes #5472 - Extracted a shared `bedrockDocumentFormat` helper in `utils.go` that maps MIME types and bare file extensions to Bedrock Converse document format strings, replacing two duplicated inline switch blocks that were missing most MIME types. - Format resolution now follows a priority chain: `file_type` → data URL media type → filename extension → `"pdf"` default. Previously only `file_type` was consulted. - `ParseDataURL` in `schemas/utils.go` is now a public function that correctly handles media type parameters (e.g. `;charset=utf-8`), uppercase media types, and payloads containing newlines. The old regex silently dropped any data URL whose header contained a parameter, causing the entire `"data:..."` string to be forwarded to Bedrock as the document payload. - Non-base64 data URLs (e.g. `data:text/plain,Hello%20World`) are now percent-decoded and their text content is populated in both `source.text` and `source.bytes` instead of being forwarded verbatim. - The Responses path (`responses.go`) previously ignored `file_url` entirely, emitting a document block with an empty source. It now fetches and inlines the bytes the same way the Chat path does, and propagates fetch errors rather than swallowing them. - `convertBifrostMessageToBedrockMessage` now returns an error instead of silently returning `nil` on conversion failure, so a missing turn is never silently dropped from the request. - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ```sh go test ./core/providers/bedrock/... ./core/schemas/... ``` Key test cases added: - `TestDocumentFormatFromDataURL` — verifies that each supported MIME type embedded in a data URL resolves to the correct Bedrock format string and that the `data:...` prefix is stripped from `source.bytes`. - `TestDocumentFormatResolutionPrecedence` — verifies the `file_type` → data URL → filename extension → default priority chain. - `TestDocumentInlineTextDataURL` — verifies that non-base64 data URLs are percent-decoded and stored in both `source.text` and `source.bytes`. - `TestToBedrockResponsesRequest_DocumentFormatFromDataURL` — same format fix verified on the Responses path. - `TestToBedrockResponsesRequest_DocumentFileURLIsFetched` — verifies that an unreachable `file_url` surfaces as an error rather than producing an empty document block. - `TestParseDataURL` — unit tests for the new public `ParseDataURL` function covering parameters, uppercase, newlines in payload, and invalid inputs. - [ ] Yes - [x] No `file_url` values are now fetched over the network on the Responses path (matching existing Chat path behaviour). The fetch is performed with the existing `providerUtils.FetchAndEncodeURL` helper, which is subject to the same controls already in place for image URL fetching. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
) Anthropic returns a 400 error with no error code when it rejects a `redacted_thinking` block containing a foreign or invalid payload. The existing `isEncryptedReasoningRejection` detection did not match this error format, causing the retry logic to miss these rejections and fail to strip the offending encrypted content before retrying. - Extended `isEncryptedReasoningRejection` to also match Anthropic's `redacted_thinking`-specific rejection message: `"Invalid \`data\` in \`redacted_thinking\` block"`. - Added a comment explaining why this additional check is needed (Anthropic omits an error code and names the offending block in the message text instead). - Added three new test cases: - Confirms the `redacted_thinking` rejection is correctly detected. - Confirms that a `thinking` block signature rejection is intentionally **not** matched (since stripping encrypted content would not fix it and would cause an infinite retry loop). - Confirms that an unrelated Anthropic 400 mentioning `thinking` (e.g., invalid `budget_tokens`) is not incorrectly matched. - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ```sh go test ./core/... ``` The three new test cases in `TestIsEncryptedReasoningRejection` cover the added detection logic and the intentional non-matches. - [ ] Yes - [x] No No auth, secrets, or PII implications. The change only affects error message pattern matching used to decide whether to strip encrypted reasoning content before retrying a request. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…6041) The encrypted-reasoning fail-soft that strips `encrypted_content` from reasoning items before retrying a rejected request only covered `ResponsesRequest`. Bifrost models `/v1/responses/compact` as a separate top-level request shape (`CompactionRequest` with its own `Input` and `RawRequestBody`), so the strip returned `false` for compaction requests, the retry never fired, and the raw `400 invalid_encrypted_content` was handed straight to the client. Codex surfaced this as "Error running remote compact task" and retried the identical body indefinitely. - Introduced `encryptedReasoningCarriers`, which resolves the `Input` and `RawRequestBody` pointers for whichever of the three Responses-shaped request types (`ResponsesRequest`, `CountTokensRequest`, `CompactionRequest`) is present on a given `BifrostRequest`. This lets a single code path in `stripResponsesEncryptedContent` and `stripRawResponsesEncryptedContent` cover all three shapes without duplicating logic. - `stripResponsesEncryptedContent` now keys off the resolved pointers rather than a hard check for `ResponsesRequest != nil`, so compaction and token-counting requests are rewritten on the same retry path. - `stripRawResponsesEncryptedContent` now accepts a `*[]byte` instead of a `*schemas.BifrostResponsesRequest`, removing the coupling to one specific request type. - Added unit tests covering the compaction strip path for both the structured and raw-body (passthrough) cases, and for the drop-on-empty-summary behaviour on compaction input. - Added two provider-harness cases under entry 45 that pin the end-to-end behaviour: one where the stripped reasoning item survives (it has a summary), and one where it is dropped entirely (empty summary) rather than forwarded as a bare id the upstream never issued. - Updated `AGENTS.md` to make explicit that any wire-visible change under `core/` must ship with a provider-harness case, not only bug fixes, and documents the structural validation workflow and the narrow exemptions. - [x] Bug fix - [x] Core (Go) ```sh go test ./core/... -run TestStripResponsesEncryptedContent node tests/e2e/api/runners/augment-provider-harness.mjs \ --source tests/e2e/api/collections/provider-harness.json \ --out tmp/harness-augmented.json node tests/e2e/api/runners/filter-collection.mjs \ --source tmp/harness-augmented.json \ --out tmp/filtered.json \ --feature "Encrypted Reasoning Fail-Soft on Compaction" ``` The two new harness cases (`openai/gpt-5-mini /openai/v1/responses/compact unverifiable encrypted reasoning degrades` and the summary-less drop variant) should pass structurally. Against a live endpoint they verify that the compaction response returns `object: "response.compaction"` with a non-empty `output` array and no `invalid_encrypted_content` in the body. N/A - [x] No N/A No auth, secrets, or PII implications. The strip removes `encrypted_content` blobs that the upstream has already refused; no key material is logged or forwarded. - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
Introduces a `BifrostContextKeySkipProviderCheck` context flag for requests that are evaluated by governance but never routed (e.g., `/inspect`). For these requests, the provider is whatever upstream the caller was already talking to rather than an operator-selected provider, so the virtual key's provider allowlist is meaningless. Without this flag, such requests would be incorrectly blocked by the VK provider gate, and then if that were bypassed, would fail again on a model allowlist that only exists inside a provider config the VK doesn't have. Also bumps `github.com/bytedance/sonic` from v1.15.1 to v1.15.2 across all modules. - Added `BifrostContextKeySkipProviderCheck` context key (`bifrost-skip-provider-check`) to `core/schemas/bifrost.go` and registered it as a reserved key in `core/schemas/context.go`. - `EvaluateVirtualKeyRequest` in `plugins/governance/resolver.go` now accepts a `skipProviderCheck bool` parameter. When set, the provider allowlist gate is skipped. Additionally, if the VK has no provider config for the requested provider (meaning it also has no model allowlist for it), the model gate is skipped as well. A provider the VK does configure retains its model allowlist regardless of the flag. - `EvaluateGovernanceRequest` in `plugins/governance/main.go` reads the new context key and passes it through to `EvaluateVirtualKeyRequest`. - `github.com/bytedance/sonic` bumped to v1.15.2 across all modules. - `google.golang.org/x/crypto`, `x/net`, `x/sync`, `x/sys`, and `x/text` bumped to latest patch versions in test seed modules. - Node engine constraint removed from `ui/package.json`. - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs Two new tests cover the behavior directly: - `TestBudgetResolver_EvaluateRequest_SkipProviderCheckAllowsUnconfiguredProvider` — verifies that a VK with no config for the requested provider allows the request when the flag is set, and does not fall through to a model block. - `TestBudgetResolver_EvaluateRequest_SkipProviderCheckKeepsModelAllowlist` — verifies that a provider the VK does configure keeps its model allowlist enforced even when the flag is set. ```sh cd plugins/governance go test ./... -run TestBudgetResolver_EvaluateRequest_SkipProviderCheck ``` - [ ] Yes - [x] No The `EvaluateVirtualKeyRequest` signature gains a new `skipProviderCheck bool` parameter. All internal call sites have been updated. External callers implementing the interface directly will need to add the parameter. The flag is intended exclusively for transport-set, read-only inspection paths where the provider is not an operator choice. It bypasses the VK provider and (for unconfigured providers) model allowlists only for those requests. VK existence, active status, expiry, rate limits, and budgets are unaffected. The flag is registered as a reserved context key so it cannot be set by arbitrary plugin or caller code without going through the transport layer. - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Add mobile responsiveness to make the dashboard usable on smaller devices. It does not have full coverage, but it includes basic responsiveness so it can be used or at the very least viewed, on mobile screens. ## Changes - Responsiveness ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [x] No If yes, describe impact and migration instructions. ## Related issues ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…creen (#6126) ## Summary Adds graceful version-skew handling so that when Bifrost is being rolled out, users see a clear "upgrading" UI instead of a broken page. Stale asset errors (failed dynamic imports, chunk load failures) are detected, classified, and surfaced through two purpose-built screens: a non-blocking banner for soft failures and a full-page upgrade screen for hard failures. An auto-reload mechanism polls `/api/version` for stability and reloads the page automatically, with a session-storage guard to prevent reload loops. ## Changes - **`versionSkew.ts`** — New utility module that classifies skew errors by matching known browser/bundler error patterns (`ChunkLoadError`, failed dynamic imports, etc.), maintains a reactive `SkewMode` store (`none | soft | hard`), installs global listeners for `vite:preloadError`, `unhandledrejection`, and asset element errors, and manages a session-storage reload budget (`MAX_AUTO_RELOADS = 2` within a 60-second window) to prevent infinite reload loops. - **`__updating.tsx`** — New `UpdatingBanner` (non-blocking overlay for soft skew) and `UpdatingScreen` (full-page replacement for hard skew) components. `UpdatingScreen` polls `/api/version` every 3 seconds, requires 3 consecutive matching responses before triggering an auto-reload, and times out after 90 seconds with a manual reload fallback. - **`__error.tsx`** — `ErrorComponent` now receives the error prop and redirects to `UpdatingScreen` when a skew error is detected, escalating to hard mode via `reportSkew("hard")`. - **`clientLayout.tsx`** — Adds a `ConfigUnreachable` component shown when the core config fetch fails, with a retry button wired to RTK Query's `refetch`. `FullPage` now receives `hasError`, `isRetrying`, and `onRetry` props to drive this state. - **`main.tsx`** — Introduces a `Root` component that subscribes to the skew store via `useSyncExternalStore`, renders `UpdatingScreen` on hard skew, overlays `UpdatingBanner` on soft skew, and clears the auto-reload guard after 30 seconds of healthy uptime. Sets `window.__bifrostBooted` to coordinate with the inline boot script. - **`index.html`** — Adds an inline script that renders a minimal native-HTML upgrading screen if assets fail to load before React boots, using the same session-storage reload guard logic to cap retries. - **`globals.css`** — Adds the `update-progress` keyframe animation used by the progress bar in `UpdatingScreen`, and fixes a nested media query indentation issue. - **`versionSkew.test.ts`** — Full test coverage for `isSkewError`, the skew store (subscribe/notify/escalation/downgrade prevention), and the auto-reload guard (budget exhaustion, window expiry, `clearAutoReloadGuard`, and `sessionStorage` unavailability). ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i pnpm test pnpm build ``` To manually verify: 1. Build the UI and serve it, then invalidate a JS asset URL (e.g., rename a chunk file) to trigger a `ChunkLoadError`. The upgrading banner or screen should appear. 2. Reload the page more than twice within 60 seconds while skew is active — the auto-reload should stop and display the manual reload fallback. 3. Simulate a failed `/api/core-config` response; the `ConfigUnreachable` card should appear with a working "Try again" button. ## Screenshots/Recordings - **Soft skew:** A fixed bottom banner reading "Bifrost is upgrading" with a manual reload button appears without disrupting the current view. - **Hard skew / boot failure:** A full-page card with an animated progress bar, status text, and "Reload now" button replaces the broken route. - **Config unreachable:** A card with a `WifiOff` icon and retry button is shown in the main content area. **Soft skew**  **Hard skew**  ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations The auto-reload guard uses `sessionStorage`, which is scoped to the tab and origin. No auth tokens or PII are stored. The inline boot script in `index.html` is self-contained and does not make authenticated requests. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Briefly explain the purpose of this PR and the problem it solves. - What was changed and why - Any notable design decisions or trade-offs - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs Describe the steps to validate this change. Include commands and expected outcomes. ```sh go version go test ./... cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. If UI changes, add before/after screenshots or short clips. - [ ] Yes - [ ] No If yes, describe impact and migration instructions. Link related issues and discussions. Example: Closes #123 Note any security implications (auth, secrets, PII, sandboxing, etc.). - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai --> * **Bug Fixes** * Improved token parameter compatibility handling to preserve alternative formats when the primary option is unsupported. * **Chores** * Version updated to 2.0.0. * Enhanced load testing configuration for more reliable builds. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Bedrock was rejecting documents with "The PDF specified was not valid" because the document format was always resolved to `"pdf"` regardless of the actual file type. Standard OpenAI clients encode the MIME type inside the data URL (e.g. `data:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;base64,...`) rather than in the `file_type` field, which was the only source previously consulted. This PR fixes format resolution for both the Chat and Responses paths, unifies the mapping logic, and corrects several related data URL parsing defects. Fixes #5472 - Extracted a shared `bedrockDocumentFormat` helper in `utils.go` that maps MIME types and bare file extensions to Bedrock Converse document format strings, replacing two duplicated inline switch blocks that were missing most MIME types. - Format resolution now follows a priority chain: `file_type` → data URL media type → filename extension → `"pdf"` default. Previously only `file_type` was consulted. - `ParseDataURL` in `schemas/utils.go` is now a public function that correctly handles media type parameters (e.g. `;charset=utf-8`), uppercase media types, and payloads containing newlines. The old regex silently dropped any data URL whose header contained a parameter, causing the entire `"data:..."` string to be forwarded to Bedrock as the document payload. - Non-base64 data URLs (e.g. `data:text/plain,Hello%20World`) are now percent-decoded and their text content is populated in both `source.text` and `source.bytes` instead of being forwarded verbatim. - The Responses path (`responses.go`) previously ignored `file_url` entirely, emitting a document block with an empty source. It now fetches and inlines the bytes the same way the Chat path does, and propagates fetch errors rather than swallowing them. - `convertBifrostMessageToBedrockMessage` now returns an error instead of silently returning `nil` on conversion failure, so a missing turn is never silently dropped from the request. - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ```sh go test ./core/providers/bedrock/... ./core/schemas/... ``` Key test cases added: - `TestDocumentFormatFromDataURL` — verifies that each supported MIME type embedded in a data URL resolves to the correct Bedrock format string and that the `data:...` prefix is stripped from `source.bytes`. - `TestDocumentFormatResolutionPrecedence` — verifies the `file_type` → data URL → filename extension → default priority chain. - `TestDocumentInlineTextDataURL` — verifies that non-base64 data URLs are percent-decoded and stored in both `source.text` and `source.bytes`. - `TestToBedrockResponsesRequest_DocumentFormatFromDataURL` — same format fix verified on the Responses path. - `TestToBedrockResponsesRequest_DocumentFileURLIsFetched` — verifies that an unreachable `file_url` surfaces as an error rather than producing an empty document block. - `TestParseDataURL` — unit tests for the new public `ParseDataURL` function covering parameters, uppercase, newlines in payload, and invalid inputs. - [ ] Yes - [x] No `file_url` values are now fetched over the network on the Responses path (matching existing Chat path behaviour). The fetch is performed with the existing `providerUtils.FetchAndEncodeURL` helper, which is subject to the same controls already in place for image URL fetching. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
) Anthropic returns a 400 error with no error code when it rejects a `redacted_thinking` block containing a foreign or invalid payload. The existing `isEncryptedReasoningRejection` detection did not match this error format, causing the retry logic to miss these rejections and fail to strip the offending encrypted content before retrying. - Extended `isEncryptedReasoningRejection` to also match Anthropic's `redacted_thinking`-specific rejection message: `"Invalid \`data\` in \`redacted_thinking\` block"`. - Added a comment explaining why this additional check is needed (Anthropic omits an error code and names the offending block in the message text instead). - Added three new test cases: - Confirms the `redacted_thinking` rejection is correctly detected. - Confirms that a `thinking` block signature rejection is intentionally **not** matched (since stripping encrypted content would not fix it and would cause an infinite retry loop). - Confirms that an unrelated Anthropic 400 mentioning `thinking` (e.g., invalid `budget_tokens`) is not incorrectly matched. - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ```sh go test ./core/... ``` The three new test cases in `TestIsEncryptedReasoningRejection` cover the added detection logic and the intentional non-matches. - [ ] Yes - [x] No No auth, secrets, or PII implications. The change only affects error message pattern matching used to decide whether to strip encrypted reasoning content before retrying a request. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…6041) The encrypted-reasoning fail-soft that strips `encrypted_content` from reasoning items before retrying a rejected request only covered `ResponsesRequest`. Bifrost models `/v1/responses/compact` as a separate top-level request shape (`CompactionRequest` with its own `Input` and `RawRequestBody`), so the strip returned `false` for compaction requests, the retry never fired, and the raw `400 invalid_encrypted_content` was handed straight to the client. Codex surfaced this as "Error running remote compact task" and retried the identical body indefinitely. - Introduced `encryptedReasoningCarriers`, which resolves the `Input` and `RawRequestBody` pointers for whichever of the three Responses-shaped request types (`ResponsesRequest`, `CountTokensRequest`, `CompactionRequest`) is present on a given `BifrostRequest`. This lets a single code path in `stripResponsesEncryptedContent` and `stripRawResponsesEncryptedContent` cover all three shapes without duplicating logic. - `stripResponsesEncryptedContent` now keys off the resolved pointers rather than a hard check for `ResponsesRequest != nil`, so compaction and token-counting requests are rewritten on the same retry path. - `stripRawResponsesEncryptedContent` now accepts a `*[]byte` instead of a `*schemas.BifrostResponsesRequest`, removing the coupling to one specific request type. - Added unit tests covering the compaction strip path for both the structured and raw-body (passthrough) cases, and for the drop-on-empty-summary behaviour on compaction input. - Added two provider-harness cases under entry 45 that pin the end-to-end behaviour: one where the stripped reasoning item survives (it has a summary), and one where it is dropped entirely (empty summary) rather than forwarded as a bare id the upstream never issued. - Updated `AGENTS.md` to make explicit that any wire-visible change under `core/` must ship with a provider-harness case, not only bug fixes, and documents the structural validation workflow and the narrow exemptions. - [x] Bug fix - [x] Core (Go) ```sh go test ./core/... -run TestStripResponsesEncryptedContent node tests/e2e/api/runners/augment-provider-harness.mjs \ --source tests/e2e/api/collections/provider-harness.json \ --out tmp/harness-augmented.json node tests/e2e/api/runners/filter-collection.mjs \ --source tmp/harness-augmented.json \ --out tmp/filtered.json \ --feature "Encrypted Reasoning Fail-Soft on Compaction" ``` The two new harness cases (`openai/gpt-5-mini /openai/v1/responses/compact unverifiable encrypted reasoning degrades` and the summary-less drop variant) should pass structurally. Against a live endpoint they verify that the compaction response returns `object: "response.compaction"` with a non-empty `output` array and no `invalid_encrypted_content` in the body. N/A - [x] No N/A No auth, secrets, or PII implications. The strip removes `encrypted_content` blobs that the upstream has already refused; no key material is logged or forwarded. - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
Introduces a `BifrostContextKeySkipProviderCheck` context flag for requests that are evaluated by governance but never routed (e.g., `/inspect`). For these requests, the provider is whatever upstream the caller was already talking to rather than an operator-selected provider, so the virtual key's provider allowlist is meaningless. Without this flag, such requests would be incorrectly blocked by the VK provider gate, and then if that were bypassed, would fail again on a model allowlist that only exists inside a provider config the VK doesn't have. Also bumps `github.com/bytedance/sonic` from v1.15.1 to v1.15.2 across all modules. - Added `BifrostContextKeySkipProviderCheck` context key (`bifrost-skip-provider-check`) to `core/schemas/bifrost.go` and registered it as a reserved key in `core/schemas/context.go`. - `EvaluateVirtualKeyRequest` in `plugins/governance/resolver.go` now accepts a `skipProviderCheck bool` parameter. When set, the provider allowlist gate is skipped. Additionally, if the VK has no provider config for the requested provider (meaning it also has no model allowlist for it), the model gate is skipped as well. A provider the VK does configure retains its model allowlist regardless of the flag. - `EvaluateGovernanceRequest` in `plugins/governance/main.go` reads the new context key and passes it through to `EvaluateVirtualKeyRequest`. - `github.com/bytedance/sonic` bumped to v1.15.2 across all modules. - `google.golang.org/x/crypto`, `x/net`, `x/sync`, `x/sys`, and `x/text` bumped to latest patch versions in test seed modules. - Node engine constraint removed from `ui/package.json`. - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs Two new tests cover the behavior directly: - `TestBudgetResolver_EvaluateRequest_SkipProviderCheckAllowsUnconfiguredProvider` — verifies that a VK with no config for the requested provider allows the request when the flag is set, and does not fall through to a model block. - `TestBudgetResolver_EvaluateRequest_SkipProviderCheckKeepsModelAllowlist` — verifies that a provider the VK does configure keeps its model allowlist enforced even when the flag is set. ```sh cd plugins/governance go test ./... -run TestBudgetResolver_EvaluateRequest_SkipProviderCheck ``` - [ ] Yes - [x] No The `EvaluateVirtualKeyRequest` signature gains a new `skipProviderCheck bool` parameter. All internal call sites have been updated. External callers implementing the interface directly will need to add the parameter. The flag is intended exclusively for transport-set, read-only inspection paths where the provider is not an operator choice. It bypasses the VK provider and (for unconfigured providers) model allowlists only for those requests. VK existence, active status, expiry, rate limits, and budgets are unaffected. The flag is registered as a reserved context key so it cannot be set by arbitrary plugin or caller code without going through the transport layer. - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
…oughtSignature round-trip fidelity (#6071) Adds support for Gemini's server-side tool invocations (`toolCall`/`toolResponse` parts) that are reported when `toolConfig.includeServerSideToolInvocations` is enabled. Previously, these parts were silently dropped. Now they are parsed, mapped onto Bifrost's neutral `web_search_call` item type (for Google Search variants), and preserved verbatim so the exact parts — including `thoughtSignature` bytes Gemini requires on replay — survive a round-trip through the OpenAI-shaped schema. - Added `ToolCall` and `ToolResponse` types to `types.go`, with `UnmarshalJSON` implementations that accept both camelCase (Google's REST wire format) and snake_case (google-genai SDK replay format). Both are wired into `Part`'s marshal/unmarshal paths. - Added `isSearchToolType` and a registry of known search tool type strings (`GOOGLE_SEARCH_WEB`, `GOOGLE_SEARCH_IMAGE`) to distinguish mappable tools from unmapped built-ins like `CODE_EXECUTION`. - In the non-streaming path (`convertGeminiCandidatesToResponsesOutput`), `toolCall` parts now produce a `web_search_call` item using Gemini's own call ID and queries. A sibling `toolResponse` part marks the item `completed`. When grounding metadata is also present, its sources and any missing queries are merged onto the existing item rather than emitting a duplicate `web_search_call`. - `thoughtSignature` bytes carried on `toolCall`/`toolResponse` parts are emitted as standalone reasoning items so Gemini can receive them back on replay. - `serverSideToolParts` stashes the raw `toolCall`/`toolResponse` parts into `ProviderExtraFields["serverSideToolParts"]`. `ToGeminiResponsesResponse` recovers them and prepends them to the candidate's parts, skipping duplicate signature-only parts for signatures already present in those preserved parts. - In the streaming path (`ToBifrostResponsesStream`), `toolCall` parts record the call ID and queries into new `GeminiResponsesStreamState` fields (`ServerSearchRounds`). At finish, `emitWebSearchFromGroundingMetadata` uses the recorded call ID as the item ID and merges grounding data on top, and now fires even when grounding metadata is absent but a server-side call was observed. Multiple search rounds each produce their own item in the order the model ran them. - `nativePartPayload` and `nativePartsFromItem` serialize server-side tool parts onto `ResponsesMessage.ProviderNativeParts` so the streaming `/genai` surface can re-emit them byte-for-byte rather than emitting a bare signature-only part. - `emitWebSearchFromGroundingMetadata` is hardened against nil `metadata` throughout so it can operate on server-side-call-only responses. - Added `serversidetools_test.go` covering non-streaming scenarios: single search round with grounding merge, unknown tool type preservation, and two search rounds interleaved with a client function call. Added `serversidetools_stream_test.go` covering the streaming path end-to-end, including a full Gemini→Bifrost→Gemini round-trip asserting each `thoughtSignature` appears exactly once. - Added e2e harness folder 49 covering the Gemini converter path, Vertex converter path (both pre-Gemini-3 and Gemini-3.6), raw passthrough, and a multi-turn replay that lets Gemini validate the `thoughtSignature` bytes server-side. - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ```sh go test ./core/providers/gemini/... ``` The new tests exercise: - A single server-side Google Search round with grounding metadata: expect exactly one `web_search_call` item carrying Gemini's own call ID, the toolCall's queries, and grounding's sources. - An unmapped tool type (`CODE_EXECUTION`): expect no `web_search_call` item and the part preserved on the native round-trip. - Two search rounds interleaved with a client `functionCall`, no grounding metadata: expect two `web_search_call` items paired by ID and the function call unaffected. - Streaming with two search rounds: expect one item per round in order, each carrying its own call ID and queries, with grounding sources merged onto the first. - Streaming GenAI round-trip: each `thoughtSignature` appears exactly once across the reconstructed parts; no duplicate signature-only parts alongside native tool parts. - [ ] Yes - [x] No None. The `toolResponse` payload (rendered search-suggestion HTML) is carried opaquely in `ProviderExtraFields` and `ProviderNativeParts` and is not interpreted or executed. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…dii/shadows (#6204) ## Summary Fixes a crash in the updating/version-skew screen caused by calling `useBranding` outside of `<ReduxProvider>`, and standardizes UI border radius styling to use `rounded-sm` instead of larger variants. ## Changes - Extracted a `getCachedBrandingAssets` function from `useBranding` that reads branding directly from the local cache without requiring Redux store access. The `UpdatingScreen` component now uses this instead of the hook, since it renders above `<ReduxProvider>` and also serves as the router's error component. - Refactored the shared asset-building logic into a `toBrandingAssets` helper to avoid duplication between `getCachedBrandingAssets` and `useBranding`. - Replaced `rounded-lg`, `rounded-md`, and `rounded-xl` with `rounded-sm` across the not-found page, updating banner, updating screen, and config-unreachable section for visual consistency. - Removed `shadow` and `shadow-xl` from several components as part of the same styling pass. ## Type of change - [ ] Bug fix - [x] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Trigger a version-skew scenario (e.g., deploy a new backend while the UI is open) and confirm the updating screen renders without errors and displays branding correctly. Also verify the not-found and config-unreachable screens render with the updated styling. ## Screenshots/Recordings Before/after screenshots of the updating screen, not-found page, and config-unreachable section showing the updated border radius and removed shadows. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
Extends Bifrost's air-gapped deployment support to cover the MCP server library catalog in addition to the existing pricing and model parameter datasheets. Previously, air-gapped hosts had no way to suppress the catalog fetch or serve it from a local file, causing unnecessary network attempts to `getbifrost.ai` on every sync tick.
## Changes
- **`mcp_library_sync_interval: 0` disables background catalog syncing** — introduces `MCPLibrarySyncDisabled` as an explicit sentinel (mirroring `LiveModelsSyncDisabled`). A zero interval skips the startup fetch and never schedules a background sync, so no requests go to `getbifrost.ai`. Force Sync Now from the UI still works. Negative values continue to be treated as corrupted config and fall back to the default cadence.
- **`file://` URLs for the MCP library catalog** — `fetchMCPLibrary` now resolves file URLs through the shared `datasheet.FilePathFromURL` helper (exported from `sync.go`) so relative forms (`file://./servers.json`, `file:servers.json`) and `file://localhost/...` work identically to how they work for the pricing datasheets.
- **No retry backoff on local file paths** — `SyncMCPLibrary` skips the exponential-backoff retry loop when the URL is a `file://` reference, since a missing local file is not a transient failure and retrying only adds boot latency.
- **Config resolution fixes** — `ResolveFrameworkPricingConfig` previously treated `0` as corrupted and backfilled the default, which would silently re-enable syncing on the next boot. It now passes `MCPLibrarySyncDisabled` through untouched in both the file-config and DB-config paths.
- **Helm chart nil-awareness** — the `mcpLibrarySyncInterval` template condition is updated from a truthiness check to `kindIs "invalid"` so that `0` is correctly written into the rendered config rather than omitted.
- **Schema updates** — both `config.schema.json` and `values.schema.json` now allow `0` as a valid value via `anyOf: [{ const: 0 }, { minimum: 3600 }]`.
- **UI updates** — the MCP Library Settings sheet accepts `file://` URLs, allows a sync interval of `0` (with updated validation message), and preserves `0` through the hours round-trip without collapsing it to the 24h default. MCP Settings page layout is tightened to `max-w-4xl` with consistent padding.
- **Documentation** — the air-gapped guide is restructured into separate Datasheets and MCP server library sections, documents both Option A (local file) and Option B (disable sync), and adds a sync-settings reference table.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [x] UI (React)
- [x] Docs
## How to test
```sh
# Core/Transports
go test ./framework/modelcatalog/... ./transports/bifrost-http/lib/...
# UI
cd ui
pnpm i
pnpm build
```
**Air-gapped datasheet path:**
1. Download `https://getbifrost.ai/mcp-library` to a local file.
2. Set `mcp_library_url: "file:///opt/bifrost/mcp-library.json"` in `config.json`.
3. Start Bifrost — the MCP Library page should populate from the local file with no outbound requests.
**Disabled sync path:**
1. Set `mcp_library_sync_interval: 0` in `config.json`.
2. Start Bifrost — confirm the log line `MCP library sync is disabled (mcp_library_sync_interval=0), skipping startup sync` appears and no requests are made to `getbifrost.ai` on subsequent ticks.
3. Confirm Force Sync Now in the UI still triggers a sync.
**Relative file URL:**
1. Place `servers.json` in the Bifrost working directory.
2. Set `mcp_library_url: "file://./servers.json"` and verify the catalog loads correctly.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
`file://` URL support is limited to paths readable by the Bifrost process user. No new network surface is introduced; the change reduces outbound connections for air-gapped deployments.
## Checklist
- [x] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [x] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
Introduces a persistent `<Topbar>` component that sits above the inset content card on every page. Page titles are hoisted into it via a lightweight context (`TopbarProvider` / `useSetTopbarTitle`), and page descriptions are portalled into a DOM slot the topbar exposes next to the title — avoiding the re-render loop that would result from storing arbitrary JSX in context state. A new `<PageTitle>` component replaces every inline `<h1>`/`<h2>` + description block across the workspace, rendering nothing inline and instead driving the topbar title and an info-icon hover card. The external links (Discord, GitHub, bug report, docs) and the user/logout controls that previously lived in the sidebar footer are moved into a topbar dropdown menu, where they are labelled and more discoverable. The sidebar footer is simplified to just the expand affordance for the collapsed rail. - **`ui/components/topbar.tsx`** — new 48px header strip. Renders the page title (from context or derived from the last path segment with acronym normalisation), a description slot anchor, the theme toggle, and a dropdown menu containing external links and the user/logout action. - **`ui/lib/contexts/topbarContext.tsx`** — new context providing `useSetTopbarTitle`, `useTopbarTitle`, `useDescriptionSlot`, and `useDescriptionSlotRef`. Title ownership is tracked with a ref so that a mounting page's `setTitle` call is not wiped by the unmounting page's cleanup. - **`ui/components/pageTitle.tsx`** — new component. Calls `useSetTopbarTitle` and portals an `<Info>` hover card into the topbar's description slot. Renders nothing in the page body. - **`ui/app/clientLayout.tsx`** — wraps the sidebar provider in `<TopbarProvider>`, inserts `<Topbar>` above the content card, removes the old mobile sticky header (title + `SidebarTrigger`), and adjusts the flex layout so the topbar takes its fixed height and the content card fills the remainder. - **`ui/components/sidebar.tsx`** — removes external links, theme toggle, user popover, and logout button from the footer. Retains only the collapsed-rail expand button and the promo card stack. - **`ui/components/themeToggle.tsx`** — extracts `<ThemeToggleItems>` (bare dropdown items with active-state checkmarks) so the items can be embedded in a larger menu. `<ThemeToggle>` now uses them internally. - **All workspace page/view components** — inline `<h1>`/`<h2>` + description `<p>` blocks replaced with `<PageTitle title="…">description</PageTitle>`. Action buttons that were paired with the heading are moved into the search/filter toolbar row, pushed to the right with `sm:ml-auto`. - **`tests/integrations/python/config.json`** — removes `env_label` field and collapses single-element JSON arrays onto one line for readability. - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ```sh cd ui pnpm i pnpm build ``` 1. Open the dashboard and navigate between pages — the topbar should display the correct page title on each route. 2. Pages with a `<PageTitle>` description should show an `ⓘ` icon beside the title; hovering it should reveal the description in a card. 3. The sidebar footer should no longer contain external links, the theme toggle, or the logout button. 4. The topbar menu (hamburger or user pill) should contain Discord, GitHub, bug report, and docs links, plus a "Sign out" entry when auth is enabled. 5. On mobile, the `SidebarTrigger` should appear in the topbar rather than in a sticky in-page header. 6. Theme switching via the topbar toggle should work as before. Before/after screenshots recommended — the topbar is a visible layout change on every page. - [ ] Yes - [x] No The logout flow and user-info display are unchanged in behaviour; only their render location moved from the sidebar to the topbar dropdown. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
Adds a persistent, role-targeted notification system that allows operators to publish dashboard notifications to all users or specific roles. Notifications are stored in the database, delivered in real-time over WebSocket, and surfaced in the UI via a new notification center in the topbar.
## Changes
- **`core/schemas/notification.go`** — Defines `Notification`, `NotificationInput`, `NotificationSeverity`, `NotificationAudience`, and a `NotificationPublisher` function type shared across the stack.
- **`framework/configstore`** — Adds `TableNotification` GORM model (with JSON-serialized `RoleIDs` to avoid a hard dependency on enterprise role tables), a `NotificationStore` interface, and `CreateNotification` / `ListNotifications` / `DeleteExpiredNotifications` implementations on `RDBConfigStore`. A new migration creates the `notifications` table.
- **`transports/bifrost-http/handlers/notifications.go`** — Introduces `NotificationService` with `Publish`, cursor-paginated `list`, and `create` HTTP handlers (`GET /api/notifications`, `POST /api/notifications`). Input validation enforces title/message length, severity enum, audience/role-ID consistency, and that `action_path` is an internal absolute path. Expired notifications are pruned on startup and hourly.
- **`transports/bifrost-http/handlers/websocket.go`** — `WebSocketClient` now carries `roleID`, `hasRole`, and `localAdmin` fields populated at connection time. `BroadcastNotification` uses these to fan out only to clients whose role matches the notification audience, avoiding unnecessary delivery.
- **`transports/bifrost-http/server/server.go`** — `NotificationService` is instantiated during `Bootstrap` and `RegisterAPIRoutes`; `Config.NotificationPublisher` is wired to `NotificationService.Publish` so other subsystems can publish notifications in-process.
- **UI** — Adds `Notification` and `NotificationListResponse` types, a `notificationsApi` RTK Query endpoint, `localStorage`-backed per-user preference storage (read/dismissed IDs, scoped by user identity), Redux slice actions (`setNotifications`, `addNotification`, `hydrateNotificationPreferences`, `markNotificationRead`, `removeNotification`, `clearAllNotifications`, `markAllNotificationsRead`) with memoized selectors, a `useNotificationSync` hook that hydrates preferences, merges API results, and subscribes to live WebSocket `notification` events, and a `NotificationCenter` popover component mounted in the topbar.
**Design decisions:**
- Read and dismissed state are intentionally local to each UI client (localStorage) rather than persisted server-side, keeping the server schema simple and avoiding per-user state in the OSS database.
- `RoleIDs` is stored as JSON text rather than a relational foreign key so the notifications table works in OSS deployments that do not have an enterprise roles table.
- The list endpoint applies role filtering in the application layer after a bounded DB scan (`maxNotificationScan = 250`) to support role-filtered pagination without complex SQL across optional enterprise tables.
- Cursor pagination encodes `createdAt` (nanosecond Unix timestamp) and `id` as a base64 opaque token.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [ ] Providers/Integrations
- [ ] Plugins
- [x] UI (React)
- [ ] Docs
## How to test
```sh
# Core/Transports
go test ./framework/configstore/... ./transports/bifrost-http/handlers/...
# UI
cd ui
pnpm i
pnpm test
pnpm build
```
1. Start the server and open the dashboard.
2. `POST /api/notifications` with a valid `NotificationInput` payload (e.g. `{"audience":"all","severity":"info","title":"Hello","message":"World"}`).
3. Verify the bell icon in the topbar shows an unread badge and the notification appears in the tray.
4. Connect a second browser session with a different role and confirm role-targeted notifications (`audience: "roles"`) are only visible to the matching role.
5. Dismiss or mark notifications as read; confirm state persists across page reloads and is isolated per user.
## Breaking changes
- [ ] Yes
- [x] No
## Security considerations
- `action_path` is validated to be an internal absolute path (no scheme, no host, must start with `/`), preventing open-redirect payloads from being stored in notifications.
- Role filtering is enforced both at WebSocket broadcast time and at HTTP list time, so users cannot read notifications targeted at other roles.
- Read/dismissed preferences are scoped by user identity (sub, id, or email) to prevent one user's dismissals from affecting another on a shared browser.
## Checklist
- [x] I added/updated tests where appropriate
- [x] I verified builds succeed (Go and UI)
… or open (#6227) ## Summary The notification center icon in the topbar was visible during initial load even when there were no notifications, causing a brief flash where the icon would appear and then disappear. This PR fixes that glitch by deferring the render of the notification center until there is actually something to show, while also keeping the popover mounted when a user dismisses the last notification so it closes gracefully rather than unmounting mid-interaction. ## Changes - Added a `useState` hook to track the open/closed state of the notification popover and pass it as controlled state to `<Popover>`. - Added an early return that hides the notification center trigger when the popover is closed and either the feed is still loading or there are no notifications. This prevents the icon from flashing in and then disappearing on deployments with no notifications. - The `open` state guard ensures the popover stays mounted while the user is actively working in it, so dismissing the last notification doesn't cause the popover to vanish from under the pointer. - A failed initial load (which results in an empty list) also benefits from this change — a broken feed hides silently rather than advertising itself, while RTK Query's remount and websocket-push refetch behavior still handles recovery. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Open a deployment with no notifications. 2. Verify the notification bell icon does not appear and then disappear in the topbar during initial load. 3. Open a deployment with existing notifications and confirm the icon appears and the popover opens correctly. 4. Mark all notifications as read or dismiss them one by one and confirm the popover closes cleanly after the last one is dismissed rather than snapping shut mid-interaction. ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` ## Screenshots/Recordings Before: The notification icon briefly flashes in the topbar on load for deployments with no notifications. After: The notification icon only appears once there are notifications to display. ## Breaking changes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… topbar portal (#6232) ## Summary Extracts the duplicated collapsed-state filter sidebar trigger button into a shared `FilterSidebarTrigger` component and improves the mobile topbar layout so the filter trigger appears inline with the notification bell rather than as a floating overlay. ## Changes - Added `ui/components/filters/filterSidebarTrigger.tsx` — a new shared component that renders the collapsed filter sidebar trigger. On desktop it renders the existing full-height sidebar rail button. On mobile it portals a compact icon button into a new `mobileFilterSlot` anchor in the topbar, placing it immediately before the notification bell. - Replaced the duplicated inline `<Button>` collapsed-state blocks in `logsFilterSidebar`, `mcpFilterSidebar`, `mcpLibraryFilterSidebar`, `mcpClientsFilterSidebar`, `mcpSessionsFilterSidebar`, and `oauthGrantsFilterSidebar` with a single `<FilterSidebarTrigger />` call. - Added `mobileFilterSlot` and `setMobileFilterSlot` to `TopbarContext` and exposed `useMobileFilterSlot` / `useMobileFilterSlotRef` hooks so filter sidebars can portal their mobile trigger into the topbar without the topbar needing to know page-specific content. - Updated `Topbar` to render the `mobileFilterSlot` anchor span between the left content area and the notification bell, and to show the brand logo on mobile in place of the page title (which is now hidden on small screens). - Collapsed the user pill on mobile to a bare icon button, hiding the display name and chevron below the `md` breakpoint. - Changed the notification badge to use explicit `bg-red-600`/`dark:bg-red-700` classes instead of `bg-destructive` to ensure consistent color regardless of theme token overrides. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i pnpm build ``` 1. Open any page that has a filter sidebar (Logs, MCP Logs, MCP Library, MCP Clients, MCP Sessions, OAuth Grants). 2. Collapse the filter sidebar and verify the trigger appears correctly on desktop (full-height rail) and mobile (icon in topbar next to the notification bell). 3. Confirm the active filter count badge renders on both breakpoints when filters are applied. 4. Verify the mobile topbar shows the brand logo and a bare user icon, with the full pill restored at the `md` breakpoint. 5. Confirm the notification badge is visually red in both light and dark themes. ## Screenshots/Recordings Before/after screenshots recommended for the mobile topbar layout and the collapsed filter trigger placement on both breakpoints. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds `service_tier` as a tracked field on log entries, enabling cost recomputation to reprice requests at the rates they were actually served at. This surfaces the billing tier (e.g., OpenAI's `"priority"`, `"flex"`, or `"default"`) in both the logs table and the log detail view. ## Changes - Added `service_tier?: string` to the `LogEntry` type, denormalized onto the log row so cost recomputation can use the correct tier rates. - Added a `service_tier` column to the logs table, rendering the tier as an uppercase badge when present and `-` when absent. - Added `"Service Tier"` to the column label map and included `"service_tier"` in the default hidden columns list so it is available but not shown by default. - Added a `Service Tier` field to the log detail view that renders conditionally when the value is present. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Make a request through a provider that returns a `service_tier` in its response (e.g., OpenAI with `service_tier: "flex"` or `"priority"`). 2. Open the Logs page and enable the **Service Tier** column via the column visibility menu. 3. Verify the tier is displayed as an uppercase badge in the table row. 4. Click into the log entry and confirm the **Service Tier** field appears in the detail view. 5. For a request without a `service_tier`, confirm the column shows `-` and the detail view field is absent. ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` ## Screenshots/Recordings _Add before/after screenshots showing the Service Tier column in the logs table and the field in the log detail view._ ## Breaking changes - [x] No ## Related issues ## Security considerations No security implications. `service_tier` is a non-sensitive billing metadata field. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…r to final chunk and log entry (#6236) ## Summary Anthropic reports `service_tier` on the `message_start` usage block during streaming. The per-event converter drops it, and `BifrostLLMUsage` has no `service_tier` field, so it had nowhere to travel. As a result, every streamed Anthropic request logged an empty `service_tier` and was repriced at standard rates instead of the actual served tier (priority/flex). This fix latches the tier across streaming events and stamps it onto the final chunk's response envelope, mirroring the existing pattern for `speed` and `inference_geo`. ## Changes - In the Anthropic chat completion and responses streaming loops, `service_tier` from `message_start` usage is now latched into a `servedServiceTier` variable and applied to the final chunk's response envelope, matching how `speed` and `inference_geo` are already handled. - `StreamAccumulatorResult` gains a `ServiceTier` field so the resolved tier survives the tracer boundary. Without this field, the tier was lost when the accumulator handed off to the tracer, causing streamed rows to reprice at standard rates. - `ProcessStreamingChunk` in the tracer now copies `ServiceTier` from the processed response into the accumulator result explicitly, since it lives on the response envelope rather than inside `BifrostLLMUsage`. - `convertToProcessedStreamResponse` in the logging plugin now forwards `ServiceTier` from `StreamAccumulatorResult` into the processed response so `applyStreamingOutputToEntry` can write it to the log entry. - Tests added to verify the final chunk carries the correct `service_tier` for both the chat completion and responses streaming paths, and that the tier survives the full accumulator-to-log-entry handoff. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/anthropic/... go test ./plugins/logging/... go test ./... ``` The new test `TestAnthropicChatStreamFinalChunkCarriesServedServiceTier` replays a synthetic Anthropic SSE stream where `service_tier: priority` appears on `message_start` and asserts the final chunk's `ServiceTier` equals `priority` alongside the existing `speed` and `inference_geo` assertions. `TestStreamingServiceTierSurvivesAccumulatorHandoff` verifies that a `StreamAccumulatorResult` carrying `priority` tier produces a log entry with `service_tier: priority` after the full conversion chain. ## Breaking changes - [ ] Yes - [x] No ## Related issues Related to the same class of mis-billing addressed in #5669 for non-streamed rows. ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Preserve the bare heartbeat default for older OpenAI Go clients while allowing GenAI streams to emit a self-contained comment block.\n\nAffected packages:\n- transports/bifrost-http/lib\n- transports/bifrost-http/integrations\n- transports/changelog.md
## Summary Mixed-modality user turns (e.g. text + image, text + file) were previously excluded entirely from complexity routing because the extraction functions returned false as soon as any non-text block was encountered. This meant the text portion of those prompts was silently dropped, preventing the complexity analyzer from acting on legitimate lexical signal. This PR changes the extraction logic to collect text blocks and ignore non-text blocks, so mixed-modality prompts still contribute their text to routing. ## Changes - `extractChatTextOnly` and `extractResponsesTextOnly` now iterate all content blocks, accumulate text from text-typed blocks, and skip non-text blocks (images, files, audio) rather than bailing out on the first non-text block. - Both functions now return false only when no usable text is found at all (empty or whitespace-only result), preserving the existing behavior for purely non-text turns. - The early-exit guard on empty `ContentBlocks` slices was removed since the loop and the post-loop empty check handle that case correctly. - The test `TestBuildComplexityInput_SkipsMixedModalityUserContent` was renamed to `TestBuildComplexityInput_ExtractsTextFromMixedModalityUserContent` and updated to assert that the extracted text matches the text block content. - A new test `TestBuildComplexityInput_SkipsUserContentWithoutText` covers the case where a turn contains only non-text blocks (image-only, file-only), confirming those still produce no complexity input. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./plugins/governance/... ``` The updated and new test cases cover both the mixed-modality extraction path and the text-absent fallback path. Confirm that `TestBuildComplexityInput_ExtractsTextFromMixedModalityUserContent` passes with the expected text values and that `TestBuildComplexityInput_SkipsUserContentWithoutText` confirms no input is produced for image-only or file-only turns. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations No auth, secrets, PII, or sandboxing implications. The change affects only how text is extracted from structured content blocks before being passed to the complexity analyzer. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
This PR fixes a cluster of provider-level bugs around URL-sourced file inputs, Bedrock rerank model identifiers, Gemini candidate assembly, and OpenAI file block marshalling, and adds the CI egress allowlist entries and harness rows needed to keep those fixes covered in the release pipeline.
- **Vertex URL source routing**: `gs://` URIs are now forwarded to Gemini/Gemma as `fileData.fileUri` (the documented Cloud Storage form, IAM-resolved, no inline cap) instead of being handed to the HTTP fetcher and dying with "unsupported URL scheme". For Claude-on-Vertex, `gs://` is fetched from Cloud Storage using the request key's own Google credentials and inlined, because Claude on Google Cloud accepts base64 sources only. A new `classifyURLSource` function encodes the per-scheme, per-family rules with citations. `http(s)` continues to be fetched for both families; forwarding it was measured and Vertex rejected every endpoint shape after ~59 s each.
- **Bedrock `s3://` sources**: `s3://` image and document references now travel to Converse as the `s3Location` union member of `ImageSource`/`DocumentSource` instead of being downloaded and re-uploaded. This skips a round trip and the 25 MiB inline cap. Format is derived from the object key extension when no `file_type` is declared, matching the existing image path. An extension-less object is rejected up front.
- **Bedrock rerank ARN synthesis**: Bedrock's Rerank API requires a full foundation-model ARN while every other Bedrock surface takes a bare model ID. Bifrost now synthesizes the ARN from the resolved region when a bare ID is passed, using the correct partition (`aws`, `aws-cn`, `aws-us-gov`) for GovCloud and China. An explicit ARN passes through untouched.
- **OpenAI file block `file_url` marshalling**: `MarshalJSON` was stripping `file_url` from file blocks, producing `{"type":"file","file":{}}` and an upstream complaint about a missing `file_id`. `file_url` is now preserved on the wire; `file_type` (a Bifrost extension) is still stripped. `ResolveChatFileURLs` skips non-`http(s)` schemes rather than attempting to fetch them, leaving the reference intact for the provider to judge.
- **Anthropic URL source inlining**: Non-`http(s)` schemes (`s3://`, `gs://`, etc.) are now passed through rather than handed to the fetcher, which would have failed. The provider's own answer is authoritative on what it accepts.
- **Gemini candidate assembly**: A thinking model that exhausts its token budget before emitting a visible token now always produces a candidate carrying the real finish reason. Previously, `Candidates` was `omitempty` and the body contained only `usageMetadata`. Payload-free parts (`{}`) are filtered at candidate assembly time. A new `buildGeminiTerminalCandidate` helper centralises finish-reason, grounding metadata, safety ratings, and `avgLogprobs` attachment so role-change flushes and the no-output branch both carry the full metadata. Preserved server-side tool parts are prepended to the first candidate rather than the last.
- **CI egress allowlist**: `www.berkshirehathaway.com` (the PDF host used by document-input harness rows, downloaded by Bifrost for providers with no URL document type) and `discoveryengine.googleapis.com` (the Vertex semantic-ranker backend, assembled in Go rather than declared in config) are added to the allowlists in all three workflow files. The `check-egress-allowlist.sh` script gains a second guard that scans the harness collection, provider config, and Go provider source for external hosts and asserts each is either allowlisted or explicitly exempted with a reason.
- **Token-parity matrix**: Vertex direct legs are now skipped when no gcloud-minted access token is available in the environment, rather than posting an unresolved `{{vertexAccessToken}}` placeholder and producing 33 hard 401 failures. An `expectedTokenParityCells` census is exported so the report renderer can distinguish "not attempted" from "passed" and surface missing cells explicitly.
- **Cache-matrix implicit rounds**: Increased from 4 to 6 after observing models that first engaged caching on round 4, making a 4-round window a coin flip. A `writeTotal` field summing writes across all rounds is added to the verdict report so the renderer can correctly identify warm-start cells (the best round is almost never round 1, where the write happens).
- **Harness collection**: Adds folder 52 covering `gs://`, `https://`, and `s3://` file sources across Gemini and Claude model families on Vertex and Bedrock. Updates `bedrockOpenaiModel` to the inference profile form required by Converse. Replaces retired `imagen-4.0-generate-001` references with `gemini-3.1-flash-image`. Marks Gemini 3.6 Vertex tool-combination rows as `[PREVIEW]`.
- [x] Bug fix
- [x] Feature
- [x] Chore/CI
- [x] Core (Go)
- [x] Providers/Integrations
```sh
go test ./core/providers/...
make run-provider-harness-test
.github/workflows/scripts/check-egress-allowlist.sh \
.github/workflows/release-pipeline.yml \
.github/workflows/run-core-tests.yml
node tests/e2e/api/runners/lib/crossprovider-cache-matrix.test.mjs
```
- [x] Yes
Gemini API: a request carrying both function declarations and Google Search without `include_server_side_tool_invocations` previously kept Google Search and dropped the function declarations. It now does the opposite — function declarations win because dropping them leaves the model unable to invoke caller-supplied tools at all. Set `include_server_side_tool_invocations: true` to send both (supported on Gemini 3 models). Vertex is unaffected; it accepts the combination natively.
The egress allowlist additions (`www.berkshirehathaway.com`, `discoveryengine.googleapis.com`) are public endpoints required by existing harness rows. The GCS fetch path for Claude-on-Vertex uses the request key's own Google credentials and does not introduce new credential scopes.
- [x] I added/updated tests where appropriate
- [x] I verified builds succeed (Go and UI)
- [x] I verified the CI pipeline passes locally if applicable
## Summary
This PR extends Runware provider support to cover two new operation types: **image upscaling** (via the image edit endpoint) and **image-to-3D generation** (via the video generation endpoint). It also relaxes the prompt-required validation in both the core and HTTP transport layers so that asset-driven operations — which carry no text prompt — are accepted.
## Changes
- **Image upscale via edit endpoint**: When `type=upscale` is set on an image edit request, the request is routed to Runware's `upscale` task type instead of `imageInference`. The input image is nested under `inputs.image`, and inference-only fields (prompt, dimensions, steps) are left unset. Upscaler-specific extra params (`upscaleFactor`, `targetMegapixels`, `settings`) are promoted to typed struct fields and removed from the passthrough map to avoid double-sending.
- **Image-to-3D via video endpoint**: A new `type` field on `VideoGenerationParameters` (e.g. `"3d"`) selects the `3dInference` task type without requiring callers to know Runware-internal task type names. The reference image is routed into the nested `inputs` object (singular or array form, depending on the model) rather than `frameImages`, which the 3D task does not accept. A per-model lookup table (`runware3DImageInputIsArray`) encodes which form each 3D model expects. `includeCost` is set automatically for 3D tasks since they have no datasheet rate.
- **`settings` extra param handling**: A shared `runwareSettings` helper coerces the `settings` extra param from either a JSON string (multipart form callers) or a plain object (JSON callers) into a typed `map[string]interface{}` and removes it from `ExtraParams` to prevent double-emission.
- **Prompt validation relaxed**: Both the core `VideoGenerationRequest` guard and the HTTP transport handler now accept requests that supply an `input_reference` or `video_uri` in place of a prompt, since asset-driven operations (upscale, image-to-3D) do not require one. The error message is updated accordingly.
- **`upscale` added to prompt-optional image edit types**: `isPromptOptionalImageEditType` now recognises the generic `"upscale"` value alongside the existing provider-specific variants.
- **`type` field registered as a known video generation param**: The HTTP transport's known-fields map is updated so `type` is not incorrectly treated as an extra param.
- **New `RunwareInputs` struct and `taskTypeUpscale` constant** added to the Runware types layer to support the new task shapes.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/providers/runware/...
go test ./core/...
go test ./transports/bifrost-http/...
```
**Image upscale** — send an image edit request with `type=upscale`, a Runware upscaler model (e.g. `topazlabs:wonder@3.5`), and optionally `upscaleFactor` or `targetMegapixels` in extra params. Verify the outgoing Runware payload uses `taskType=upscale` and nests the image under `inputs.image`.
**Image-to-3D** — send a video generation request with `type=3d`, a supported 3D model (e.g. `tencent:hunyuan-3d@3.1-rapid`), and an `input_reference` URL instead of a prompt. Verify the outgoing payload uses `taskType=3dInference`, routes the image into `inputs.image` or `inputs.images[]` per the model table, and includes `includeCost=true`.
**Video generation unchanged** — send a standard video generation request without `type`; verify `taskType=videoInference` and `frameImages` anchoring are preserved.
## Breaking changes
- [ ] Yes
- [x] No
## Related issues
## Security considerations
No new auth, secrets, or PII surface. Input image bytes are base64-encoded before transmission, consistent with existing behaviour. The `settings` JSON string is parsed with `sonic.Unmarshal` into a typed map before forwarding; no raw string passthrough reaches the wire.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
## Summary Enables chat completion, streaming, and Responses API support for the Runware provider by routing requests through Runware's OpenAI-compatible `/v1/chat/completions` endpoint. Previously, these operations returned unsupported errors. ## Changes - `ChatCompletion` now delegates to `openai.HandleOpenAIChatCompletionRequest`, using Runware's base URL with the `/chat/completions` path and bearer auth. Extra params passthrough is enabled since Runware accepts OpenAI-shaped request bodies including `reasoning_content`. - `ChatCompletionStream` delegates to `openai.HandleOpenAIChatCompletionStreaming` using a dedicated `streamingClient` (no `ReadTimeout`, idle governed by `NewIdleTimeoutReader`) to avoid premature timeout on long-running streams. - `Responses` and `ResponsesStream` are implemented by converting the Responses request to a chat request via `ToChatRequest()` and muxing through the chat completion handlers, since Runware rejects models on its native `/v1/responses` route. - A `streamingClient` is constructed via `providerUtils.BuildStreamingClient` and stored alongside the existing unary `client` on the provider struct. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./... ``` Send a chat completion request targeting a Runware AIR model identifier and verify a valid response is returned. Repeat with streaming enabled and confirm SSE chunks arrive and terminate with `[DONE]`. Send a Responses API request and confirm it is fulfilled via the chat completions fallback path. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations API keys are passed as bearer tokens via the existing `openai.BearerAuthHeader` helper, consistent with other OpenAI-compatible providers. No new secrets or PII handling introduced. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
Adds a `VideoEdit` operation that lets callers apply prompt-driven edits, upscaling, and background removal to an existing video. The source video can be supplied as raw bytes (multipart upload), a public URL, or a provider-side video ID. The response reuses the same job shape as video generation, so callers poll and download results through the existing video endpoints.
## Changes
- Added `VideoEditRequest` request type constant and `BifrostVideoEditRequest` / `BifrostVideoEditResponse` / `VideoEditInput` / `VideoEditParameters` schemas. `BifrostVideoEditResponse` is a type alias for `BifrostVideoGenerationResponse` so the polling and download flow is unchanged.
- Added `VideoEdit` to the `Provider` interface and implemented it for OpenAI (multipart upload or JSON reference via `/v1/videos/edits`) and Runware (async task dispatched as `videoInference`, `upscale`, or `removeBackground` depending on `params.type`). All other providers return an unsupported-operation error.
- Added `VideoEditRequest` to `BifrostRequest` and wired it through `handleProviderRequest`, `resetBifrostRequest`, `GetRequestFields`, `SetProvider`, `SetModel`, `SetFallbacks`, and `SetRawRequestBody`.
- `BackfillParams` on `BifrostVideoGenerationResponse` now also reads the model from a `VideoEditRequest` when the response model is empty.
- `isModellessVideoRequestType` includes `VideoEditRequest` because the model is optional when the source is an existing video ID.
- `isPromptOptionalVideoEditType` skips the prompt-required check for `upscale`, `background_removal`, and `remove_bg` task types.
- Added `POST /v1/videos/edits` to the HTTP transport, accepting both JSON and multipart bodies. Provider resolution checks the model prefix, then the `?provider=` query param, then the `x-model-provider` header, then the provider suffix on the video ID.
- Added the OpenAI integration route for `/v1/videos/edits` and `/videos/edits`, including a `parseOpenAIVideoEditRequest` parser that handles both JSON and multipart (bracketed `video[id]` field) bodies.
- Added `VideoGenerationResponseConverter` dispatch for `VideoEditRequest` in the generic integration router.
- Added `video_edit_input` log column (migration, serialization, deserialization, payload extraction, content summary) and wired the logging plugin to capture the input, dropping raw video bytes when they exceed the large-payload threshold.
- Cost computation and model-catalog normalization treat `VideoEditRequest` the same as `VideoGenerationRequest`.
- Updated provider-capabilities fixture, config JSON schema, UI type definitions, request-type labels/colors, and the custom-provider sheet to include `video_edit`.
- Added unit tests for `ToOpenAIVideoEditRequest`, `parseVideoEditFormDataBodyFromRequest`, `ToBifrostVideoEditRequest`, `ToRunwareVideoEditRequest`, `prepareVideoEditRequest`, `videoIDProviderSuffix`, and route-shape conflict detection.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [x] Providers/Integrations
- [x] Plugins
- [x] UI (React)
- [ ] Docs
## How to test
```sh
# Core/Transports
go test ./core/... ./transports/...
# Run the new provider-specific tests
go test ./core/providers/openai/... -run TestToOpenAIVideoEdit
go test ./core/providers/runware/... -run TestToRunwareVideoEdit
go test ./transports/bifrost-http/handlers/... -run TestPrepareVideoEdit
go test ./transports/bifrost-http/handlers/... -run TestVideoRouteShapesDoNotConflict
# UI
cd ui
pnpm i
pnpm build
```
To exercise the endpoint end-to-end, send a multipart request with an uploaded video:
```sh
curl -X POST http://localhost:8080/v1/videos/edits \
-H "Authorization: Bearer $API_KEY" \
-F "model=runware/runway:aleph@2.0" \
-F "prompt=shift the palette to teal" \
-F "video=@clip.mp4"
```
Or reference an existing video by ID:
```sh
curl -X POST http://localhost:8080/v1/videos/edits \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{"prompt":"shift the palette to teal","video":{"id":"video_abc123:openai"}}'
```
## Breaking changes
- [ ] Yes
- [x] No
The `Provider` interface gains a new `VideoEdit` method. Any custom provider implementations outside this repository must add a stub returning an unsupported-operation error to satisfy the interface.
## Related issues
## Security considerations
Raw video bytes uploaded via multipart are held in memory only for the duration of the request and are not persisted. When the large-payload logging threshold is set, the bytes are dropped from the log entry before it is written, keeping PII and proprietary content out of the log store.
## Checklist
- [x] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
## Summary Streaming requests were misreporting Bifrost overhead because the total-wall-clock minus upstream calculation included off-CPU relay/scheduler wait between provider chunks — time the request goroutine spends parked, not doing Bifrost work. This inflated the overhead figure and produced a misleading "core = 95% of overhead" breakdown. This PR instruments the actual Bifrost CPU phases during streaming (SSE framing, per-event JSON decode, schema conversion, outbound marshal, client write) and uses the sum of those measured buckets as the overhead for streams, folding the off-CPU remainder back into upstream so `latency = upstream + overhead` still holds. ## Changes - **SSE framing attributed to `response-parse`**: `defaultSSEDataReader.ReadDataLine` now carries a `BifrostContext` and wraps each call with a deferred timer that subtracts any upstream-latency delta accrued during the socket read, leaving only the CPU cost of scanner splitting, prefix parsing, and the buffer copy. This runs once centrally for every provider using the shared reader. - **Per-event decode timed in OpenAI handlers**: `sonic.UnmarshalString` calls in the text-completion, chat-completion, and responses streaming handlers are wrapped with `schemas.AddStreamParse` so per-chunk JSON decode lands in the Serialization bucket rather than disappearing into unmeasured overhead. - **Per-event schema conversion timed**: `ToBifrostResponsesStreamResponse` and `postResponseConverter` calls are wrapped with `schemas.AddStreamConvert` so API-translation work lands in the Conversion bucket. - **Outbound relay timed in the HTTP transport**: Per-chunk `sonic.Marshal` (transport CPU) and `reader.SendEvent` (client socket write) are accumulated into `streamTransportCPUNs` / `streamClientWriteNs` and stamped onto the root span via `bifrostCtx.StampStreamTransport` on every exit path. - **`computeOverheadBreakdown` detects streaming**: Presence of any stream-phase attribute (`AttrBifrostStreamParseMs`, `AttrBifrostStreamConvertMs`, `AttrBifrostStreamBackpressureMs`) on the root span marks the trace as streaming. For streaming traces, the `provider-internal` bucket and the `scheduling` residual are suppressed (both would double-count the off-CPU relay wait). The function now returns `(buckets, measuredMs, isStreaming)`. - **`Inject` uses `measuredMs` as overhead for streams**: When `isStreaming` is true, overhead is set to the sum of the measured buckets and upstream absorbs the remainder, keeping the latency identity without surfacing the off-CPU wait as Bifrost cost. - **`core` bucket renamed to `scheduling`**: The residual bucket (goroutine-hop latency between phases) is now labelled `scheduling` to accurately describe what it measures. All tests and UI references updated. - **UI overhead breakdown reworked**: Categories are reorganised into Serialization, Conversion, Plugins, Middleware, Key selection, Processing, Networking, Client delivery, and Scheduling. `OVERHEAD_LABELS`, `OVERHEAD_BUCKET_CATEGORY`, and `OVERHEAD_MEMBER_MERGE` replace the previous flat `OVERHEAD_CATEGORY_META` map. `key-pool` and `key.selection` are merged into a single "Key selection" member. Plugin display names are title-cased from their kebab IDs with overrides for known acronyms. ## Type of change - [ ] Bug fix - [x] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go test ./core/providers/utils/... go test ./plugins/logging/... go test ./transports/... # UI cd ui pnpm i pnpm build ``` To validate streaming overhead correctness end-to-end: 1. Send a streaming chat or responses request through the HTTP transport. 2. Open the log detail view for that request. 3. Confirm the overhead breakdown shows `response-parse`, `convertor`, `stream-client-write` buckets and no `scheduling` residual. 4. Confirm `upstream + overhead ≈ latency` and that overhead reflects only Bifrost CPU, not the full wall-clock minus upstream. For unary requests, confirm the `scheduling` bucket still appears (renamed from `core`) and the total still sums to overhead. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. Changes are limited to latency instrumentation, span attribution, and UI rendering of overhead buckets. No auth, secrets, PII, or sandboxing paths are affected. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…6511) ## Summary The log detail view previously showed only a single scalar `cost` field. This PR introduces a `cost_breakdown` field on log entries that exposes the input/output/additional cost split to the UI, with per-category detail (cached read, reasoning, guardrail, MCP, semantic cache, etc.) grafted in from the `token_usage` payload when it reconciles with the authoritative denormalized columns. ## Changes - Added a virtual `CostBreakdown *schemas.BifrostCost` field to the `Log` struct (tagged `gorm:"-"` so it is never stored). It is assembled during `DeserializeFields` by `assembleCostBreakdown`. - `assembleCostBreakdown` uses the denormalized `InputCost`/`OutputCost`/`AdditionalCost`/`Cost` columns as the authoritative top-level split. This ensures repriced rows (where `BulkUpdateCost` refreshes the columns but leaves the `token_usage` blob stale) always reflect the current pricing. Finer per-category detail objects from `token_usage.cost` are grafted in only when they reconcile with the column values within float noise, preventing stale detail from contradicting a fresh reprice. - Legacy rows that carry only the total `Cost` column (written before the split columns existed) have the total attributed to `InputCost` in the breakdown, matching the existing `SerializeFields` and `CostUpdateFromBreakdown` convention for opaque provider totals. - Added `costsReconcile` helper for float-tolerant comparison used to gate detail grafting. - Added `CostBreakdown`, `InputCostDetails`, `OutputCostDetails`, and `AdditionalCostDetails` TypeScript interfaces to `logs.ts`, mirroring `schemas.BifrostCost`. - Replaced the single "Cost" row in the log detail view with individual rows for Input Cost, Output Cost, Total Cost, Additional Cost, Guardrail Cost, Semantic Cache Cost, and MCP Cost. Each row is conditionally rendered and only appears when its value is non-zero. - Extracted `formatCostPrecise` in the detail view to render costs to 6 decimal places, since per-request costs are frequently below `$0.01` where 2–4 dp rounding would hide the value. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go test ./framework/logstore/... # UI cd ui pnpm i || npm i pnpm build || npm run build ``` Open a log detail sheet for a request that has cost data. Verify: - Input Cost, Output Cost, and Total Cost rows appear with 6 dp precision. - Additional Cost, Guardrail Cost, Semantic Cache Cost, and MCP Cost rows appear only when non-zero. - After a reprice via `BulkUpdateCost`, the displayed split reflects the updated columns, not the stale `token_usage` blob. - Rows with no cost data show no cost breakdown rows. ## Screenshots/Recordings Before: a single "Cost" row showing the total. After: individual Input Cost / Output Cost / Total Cost rows, with Additional Cost, Guardrail Cost, Semantic Cache Cost, and MCP Cost rows appearing when applicable. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations `cost_breakdown` is assembled from already-stored cost columns and the existing `token_usage` blob. No new data is persisted and no new fields are exposed beyond what the scalar `cost` field already conveyed. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…6515) ## Summary Documents the fifth breaking change introduced in v2.0.0: the restructuring of the `BifrostCost` object from a flat list of token-category cost fields into a nested per-category breakdown (`input_cost`, `output_cost`, `additional_cost`) with optional details objects. Also updates the Enterprise v2.0.0 migration guide to reflect that the OSS base is the final v2.0.0 release (not `2.0.0-prerelease3`) and adds the three previously undocumented OSS breaking changes (governance API namespace move, `HTTPTransportPreHook` phase change, and cost restructure) to the Enterprise inherited-changes table. ## Changes - Updated the Enterprise migration guide introduction to reference the OSS v2.0.0 base instead of `2.0.0-prerelease3`, and expanded the inherited breaking changes table to include OSS changes 3, 4, and 5 with migration actions. - Updated the OSS v2.0.0 migration guide introduction to mention the cost restructure as the fifth breaking change. - Added a full "Breaking Change 5" section to the OSS migration guide covering: - A JSON field mapping table from the flat v1.x shape to the nested 2.0.0 shape. - Before/after JSON examples. - Before/after Go struct access examples for `schemas.BifrostCost`. - `LogStore.BulkUpdateCost` signature change from `map[string]float64` to `map[string]CostUpdate`, with guidance on using `CostUpdateFromBreakdown`. - A note that legacy flat-shape deserialization still works for historical data. - Added two new migration checklist steps for updating cost object consumers and custom log store implementations. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered documentation pages for: - `docs/migration-guides/v2.0.0.mdx` — confirm Breaking Change 5 section renders correctly, the JSON field mapping table is complete, and the two new checklist steps appear. - `docs/enterprise/migration-guides/v2.0.0.mdx` — confirm the introduction no longer references `2.0.0-prerelease3`, and that rows 3, 4, and 5 appear in the inherited breaking changes table with correct migration actions. ## Screenshots/Recordings N/A — documentation-only change. ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations None. This is a documentation update only. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary The "Format" and "Export Timeout" fields in the OTel profile form section are now displayed side-by-side on wider screens, improving the layout density and visual consistency of the observability configuration form. ## Changes - Wrapped the `trace_type` (Format) and `export_timeout` (Export Timeout) form fields in a flex container that stacks vertically on small screens and switches to a horizontal row on `sm` and larger breakpoints. - Replaced fixed `max-w-xs` width constraints on both fields with `sm:flex-1` so they share available space equally when displayed in a row. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` Navigate to the observability settings page and open an OTel profile. Verify that the "Format" and "Export Timeout" fields appear side-by-side on wider viewports and stack vertically on narrow/mobile viewports. ## Screenshots/Recordings Before/after screenshots of the OTel profile form on both mobile and desktop viewports recommended. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
#6503) * [fix]: core/providers/opencode - route Responses requests to /v1/responses * test: cover native opencode responses routing --------- Co-authored-by: mohammadrezwankhan <3.326051e+07+mohammadrezwankhan@users.noreply.github.com>
#6506) The bundled PostgreSQL deployment had no scheduling controls, while the Bifrost pod itself is steerable via the top-level nodeSelector/tolerations/ affinity. On clusters that mix long-lived services with ephemeral autoscaled workloads the database cannot be kept off nodes that scale in, and draining the single-replica Postgres takes the whole gateway down with it. Defaults are empty, so rendering is unchanged unless set. Co-authored-by: Pratham Mishra <99235987+Pratham-Mishra04@users.noreply.github.com>
…ulation billing them, and enforce model allowlist for inline batch requests (#6522) ## Summary Batch `/results` calls that did not settle a batch were getting their own log rows treated as billable aggregate rows during cost recalculation, causing the batch to be billed once per `/results` fetch. This PR introduces an `Echo` marker on batch accounting rows to distinguish read-only display copies from the single settlement row that owns the bill, and extends governance model-allowlist enforcement to cover inline batch create requests. ## Changes - **Echo marker on batch accounting rows**: A new `Echo bool` field on `BatchAccountingDebug` marks log rows written by `/results` calls that did not settle the batch. These rows carry a snapshot of the settled price for display but must never be billed. - **`MissingCostOnly` filter exclusion**: The logstore query for missing-cost rows now excludes echo rows (`batch_debug NOT LIKE "%\"echo\":true%"`), since their NULL cost is final and no recalculation will ever fill it. - **`batchRowRoleOf` replaces `isBatchAggregateRow`**: The classification function now returns one of three roles — `None`, `Aggregate`, or `Echo`. Rows without the echo marker are classified by whether their ID matches the deterministic aggregate ID derived from `(provider, batch_id)`, providing backward compatibility for rows written before the marker existed. - **Echo rows reprice display-only**: `calculateBatchAggregateCost` accepts a `refreshSnapshotCost bool` parameter. When true, `Accounting.Cost` is updated so the displayed price stays current, but the row's `cost` column is left NULL via the new `batchDebugOnly` path in `persistRecalcOutcomes`. - **Governance allowlist applied to inline batch create**: `PreLLMHook` now iterates every distinct model named across batch item bodies/params via `BatchCreateModels`, evaluating governance for each. `IsModelCheckedWhenPresent` is extracted into a shared utility and extended to include `BatchCreateRequest`, so the model allowlist applies whenever a model is present even if it is not required. - **Anthropic batch route: mixed-model handling**: The Anthropic integration now tracks when items carry different models and sets the top-level model to nil rather than erroring, allowing mixed-model Anthropic batches while still rejecting mixed models for non-Anthropic providers. - **`batchCreate` handler model extraction**: The handler now checks both `Body` and `Params` when inferring the model from the first batch item, matching the Anthropic integration's dual-shape awareness. - **Test IDs use `AccountingLogID`**: Existing cost-fidelity tests now derive their log IDs from `batchaccounting.AccountingLogID` so the `batchRowRoleOf` classification correctly identifies them as aggregate rows. ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/schemas/... ./framework/logstore/... ./plugins/governance/... ./plugins/logging/... ./transports/bifrost-http/... ``` Key scenarios to validate: - A batch that is fetched via `/results` multiple times produces exactly one billed row; subsequent fetch rows have `NULL` cost and `echo: true` in `batch_debug`. - `MissingCostOnly` search does not return echo rows. - After `RecalculateCosts`, echo rows have an updated `Accounting.Cost` snapshot but their `cost` column remains `NULL`. - A `BatchCreateRequest` with a model on the virtual key's disallowed list is rejected by governance. - A `BatchCreateRequest` with no model passes governance without restriction. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None beyond the governance enforcement fix, which tightens model allowlist checks to cover inline batch requests that were previously bypassing them. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…earch test to avoid flakiness under parallel test runs (#6524) ## Summary Fixes a flaky Redis vector search test that occasionally failed when running under `go test ./...` because a fixed 500ms sleep was not always long enough for RediSearch to make newly written documents searchable. ## Changes - Replaced the fixed `time.Sleep(500ms)` with a polling loop that retries the search query up to 5 seconds before failing. This accounts for the slight delay between a Redis write returning and the document becoming searchable in RediSearch, which is more pronounced when many packages share a single Redis instance during parallel test runs. - A query error (e.g. from an unescaped special character causing a syntax error) still fails immediately, since that produces an error rather than an empty result set. ## Type of change - [x] Bug fix ## Affected areas - [x] Core (Go) ## How to test ```sh go test ./framework/vectorstore/... -run TestRedisStore_VectorSearch -count=5 ``` The test should pass consistently across multiple runs without intermittent failures due to indexing delays. ## Breaking changes - [x] No ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary This PR delivers the v2.0.0 stable release of Bifrost transports (first stable on the 2.0 line), core v1.8.0, framework v1.6.0, and coordinated plugin version bumps. It introduces batch job accounting and settlement, a comprehensive overhead latency measurement system, video edit operations, expanded provider support, a new routing plugin, and a dashboard notification center, alongside a large set of correctness fixes across providers, MCP, streaming, and cost accounting. ## Changes - **Batch Accounting**: `batch_jobs` table with lifecycle store API, runner-ID ownership fencing, idempotent aggregate log writes, catalog batch pricing (`computeBatchTextCost` with 0.5 default ratio), a background sweeper with capped jittered backoff, governance budget/rate-limit settlement via `ReportBatchUsage` charged exactly once per request ID, and Claude-on-Vertex batch support routing Anthropic families to `publishers/anthropic/models/...` - **Bifrost Overhead Latency**: `upstream_latency` and `overhead_latency` on every log with per-phase overhead spans (`queue-wait`, `convertor`, `request-marshal`, `response-parse`, `key-pool`, etc.), lock-free stream overhead accumulators, `overhead_breakdown` persisted and rendered as a stacked bar in the log detail view, and `bifrost_overhead_latency_microseconds` histograms exported to Prometheus and OpenTelemetry - **Input/Output/Additional Cost Split**: Denormalized `input_cost`, `output_cost`, and `additional_cost` columns on logs carried through matviews, ClickHouse, recalculation, and the quota API; semantic cache cost folded into additional cost - **Video Edits**: `POST /v1/videos/edits` with `BifrostVideoEditRequest`, `VideoEditInput`, and `VideoEditParameters` for prompt-driven edits, upscaling, and background removal on OpenAI and Runware - **Routing Plugin**: Routing rules engine and complexity router extracted from governance into a dedicated plugin running at priority 5; endpoints moved to `/api/routing/rules` and `/api/routing/complexity-analyzer-config` with deprecated `/api/governance/*` aliases; complexity routing now reads text from mixed-modality turns - **HTTP Transport Pre-Auth Hook**: New `HTTPTransportPreAuthHook` phase runs before transport authentication; `HTTPTransportPreHook` now runs after auth. Plugins injecting credentials must move that work to the new hook - **Notification Center**: Role-targeted dashboard notifications stored in the database, delivered over WebSocket, surfaced in a topbar tray - **Runware Expansion**: Chat completions, streaming, Responses, `ListModels`, image upscale, image-to-3D, async 3D generation, provider-reported per-task cost, and a raw `/runware_passthrough` route - **OpenAI Ultrafast Service Tier**: `service_tier: "ultrafast"` forwarded only to capable models and billed at dedicated rates - **Gemini 3 Thinking Levels**: Per-model `thinkingLevel` support table with `clampThinkingLevel` snapping to the nearest rung; `reasoning_effort: "none"` sets the floor level instead of zeroing `thinkingBudget` - **Datasheet-Backed Compatibility**: Anthropic, Bedrock, Cohere, and Gemini request shaping resolved through `schemas.ResolveModelCaps` instead of hardcoded model-name checks - **Structured Output Schema Order**: `response_format` JSON schemas forwarded byte-for-byte so models generate fields in the caller's declared order - **MCP Lifecycle Fixes**: `SetClientTools` replaces rather than merges the tool map, per-call shared-credential clients refresh tools synchronously on credential update, failed `EnableClient` dials park at `Disabled`, and the global `tool_sync_interval` hot-reloads and re-times running checkers - **Pricing Fields**: Megapixel-tier image rates, per-size and joint size+quality image rates, `input_cost_per_query` for rerank, ultrafast service tier rates, and `cost_per_request` flat fee - **Typed Embeddings**: `EmbeddingData.EncodingFormat` with `int8`, `uint8`, `binary`, `ubinary`, and `base64` vectors; Bedrock Titan V2 and Cohere `embedding_types` on Converse and native invoke - **Rerank Upgrades**: Structured JSON documents, `return_documents`, `next_token`, caller document IDs, Cohere-shaped errors, cross-provider response conversion, and `/genai/v1/rank` served cross-provider - **Hot-Path Performance**: Cached MCP tool serialization, direct `OrderedMap` JSON writer, bulk span attribute writes with cached span pointers, reusable delivery timers, generation-stamped `gencache` memoization, and sonic-based JSON responses - **Legacy Attribute Removal**: `gen_ai.*`-namespaced Bifrost-internal span attributes, nanosecond `time_to_first_token`, and `x-bf-prom-*` Prometheus dimensions removed from OTel and Prometheus connectors in favor of canonical `bifrost.*` keys - **Database**: 7 configstore migrations (notifications table, batch jobs table, megapixel tier pricing, rerank query cost, ultrafast rates, image size+quality rates, batch attribution columns) and 6 logstore migrations (video edit input, upstream/overhead latency, batch debug, cost breakdown columns, matview rebuild, overhead breakdown), all additive and reversible ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Run logstore and configstore migrations during a low-activity window. Five of the six logstore migrations alter the `logs` table, and the hourly matview is rebuilt against the full table on first boot after upgrading. ## Breaking changes - [x] Yes **`HTTPTransportPreHook` now runs after authentication.** Plugins that inject a credential (`x-bf-vk`, `Authorization`, `x-api-key`) from `HTTPTransportPreHook` must move that work to the new `HTTPTransportPreAuthHook`. Go plugins implementing `HTTPTransportPlugin` must add the method (`.so` plugins that predate it are skipped for that phase). **Legacy telemetry attributes removed.** Dashboards and alerts reading `gen_ai.*` Bifrost-internal attributes, `gen_ai.usage.prompt_tokens`/`completion_tokens`, the nanosecond `time_to_first_token` attribute, or `x-bf-prom-*` Prometheus dimensions must migrate to the `bifrost.*` keys and `time_to_first_chunk`. **Gemini tool preference reversed.** A Gemini API request carrying both function declarations and Google Search without `include_server_side_tool_invocations` now keeps function declarations and drops Google Search (previously the opposite). Set `include_server_side_tool_invocations: true` to send both on Gemini 3 models. Vertex is unaffected. **Governance APIs moved.** Routing rules and the complexity analyzer moved from `/api/governance/*` to `/api/routing/rules` and `/api/routing/complexity-analyzer-config`; the old paths remain as deprecated aliases. **Custom plugin SSRF protection.** A plugin `path` pointing at an http(s) URL is rejected if it resolves to a loopback, private, CGNAT, or link-local address, and every custom plugin path is re-verified on each restart. See the [v2.0.0 migration guide](https://docs.getbifrost.ai/migration-guides/v2.0.0) for full details. ## Related issues Closes #123, #2347, #3455, #4318, #4353, #4367, #4402, #4477, #4679, #4689, #4712, #4780, #4834, #4846, #4851, #4870, #4940, #4963, #5002, #5013, #5026, #5027, #5036, #5037, #5051, #5061, #5093, #5097, #5100, #5101, #5108, #5113, #5432, #5472, #5871, #5874, #5885, #5900, #5978, #6044, #6240, #6248, #6334, #6342, #6416, #6457 ## Security considerations - Custom plugin `path` values pointing at http(s) URLs are SSRF-protected: loopback, private, CGNAT, and link-local addresses are rejected and re-verified on each restart. - Custom plugin create and update require admin authentication when dashboard auth is configured. - `Authorization`, `x-api-key`, Cloudflare Access (`cf-access-*`), and AWS ALB OIDC (`x-amzn-oidc-*`) headers are redacted before export to every observability backend. - DAC-scoped virtual key reads are blocked for `from_memory` callers. - The first-time setup token gates fresh deployments so they are not open to the world before configuration. - An allowlist for private-use redirect URI schemes (RFC 8252 §7.1) hardens the OAuth2 flow. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
…lding in-window follow-up fixes into feature entries and reserving `fix:` for pre-existing bugs (#6527) ## Summary Improves the changelog-writer skill to enforce a "one entry per feature" rule, preventing duplicate or misleading entries when a release window contains follow-up fixes and additions to features introduced in the same window. ## Changes - Added a new "One Entry Per Feature (Collapse Follow-Ups)" section that explains how to group commits by feature before writing entries, fold in-window follow-up fixes and additions into the originating feature entry, and reserve `fix:` / `## 🐞 Fixed` exclusively for bugs that existed in a previously released version - Includes a `git grep` command to verify whether a symbol existed at the previous release tag, helping distinguish true bug fixes from in-window follow-ups - Extends the rule to roll-up changelogs (e.g. GA releases consolidating prereleases) - Provides a worked example showing how multiple `feat:` and `fix:` commits for a batch accounting feature collapse into a single changelog entry - Updated the quick-reference bullet in the per-module `changelog.md` format section to reference the new rule - Updated the `transports/changelog.md` grouping rule to explicitly state that `## 🐞 Fixed` is reserved for pre-existing bugs and that all contributing PRs should be cited in the folded entry ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Review the updated `SKILL.md` and verify the new section is clear, the worked example is accurate, and the cross-references in the quick-reference bullets point to the correct section heading. ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ld it rejects (#6532) Requests carrying service_tier to bedrock_mantle fail with "'priority' is not supported for 'service_tier' on this model". Mantle's OpenAI-compatible surface does not implement service_tier, but Bifrost forwards it anyway. serviceTierForModel already strips tiers a model does not support, but it asks the datasheet and falls back to keeping the tier when no record exists (ServiceTierSupported's fallback=true). Caps resolve against the verbatim Mantle id — openai.gpt-5.6-terra, not gpt-5.6-terra — which generally has no row, so the fallback fires and the field reaches AWS. Gate on the provider before consulting the datasheet: Mantle accepts no tier at all, so the answer does not depend on catalog coverage. Mirrors the existing Gemini handling, which nulls ServiceTier outright. Provider bedrock is included because it reaches these converters only through the deprecated in-provider Mantle routing in bedrock/mantle.go — every other Bedrock path uses Converse and never touches the OpenAI converters. The fix lands in serviceTierForModel, which both ToOpenAIChatRequest and ToOpenAIResponsesRequest call, so chat and Responses are covered by one change. Callers lose no functionality: the field was producing a 400, and omitting it lets the endpoint serve at its default tier.
…, and reindent nested arrays (#6531) ## Summary Removes the external "Evals" sidebar link pointing to `https://www.getmaxim.ai` and cleans up indentation inconsistencies throughout the sidebar component. ## Changes - Removed the "Evals" sidebar item that linked externally to `getmaxim.ai`, along with its associated `FlaskConical` icon import - Reformatted template literal class name expressions for `buttonClassName` and `subItemClassName` to remove unnecessary wrapping and trailing spaces - Normalized indentation in conditional sidebar item arrays (Prompt Repository, Skills Repository, Proxy, Branding, License Info) to be consistent with surrounding code ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` Verify the sidebar renders correctly and that no "Evals" link appears in the workspace navigation. ## Screenshots/Recordings Verify the sidebar no longer displays an "Evals" entry linking to an external site. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations Removes an outbound external link from the sidebar, reducing potential confusion around third-party navigation. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Bumps the Go toolchain version from `1.26.x` to `1.27.0` across the entire repository. This ensures all modules, CI workflows, Dockerfiles, Nix flake, and documentation consistently reference the same Go version. ## Changes - All `go.mod` files updated to `go 1.27.0` - All CI workflow files (`e2e-tests.yml`, `pr-tests.yml`, `helm-release.yml`, `release-pipeline.yml`, `run-core-tests.yml`, `snyk.yml`, `release-cli.yml`, `release-bifrost-migration-cli.yml`) updated to use `go-version: "1.27.0"` - Dockerfiles (`transports/Dockerfile`, `transports/Dockerfile.local`, `transports/Dockerfile.redhat`) updated to `golang:1.27.0-alpine3.24` with the corresponding new image digest - `Makefile` Docker cross-compilation targets updated to `golang:1.27.0-alpine3.24` - `flake.nix` overlay updated from `go_1_26` to `go_1_27`, pinning `1.27.0` with the correct source hash - Nix devshell and `bifrost-http.nix` package updated to reference `go_1_27` - Release prep script (`release-bifrost-http-prep.sh`) updated to normalize `go.mod` to `go 1.27.0` - Documentation (`building-dynamic-binary.mdx`, `writing-go-plugin.mdx`, `security.mdx`, `AGENTS.md`) updated to reflect the new required Go version ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [x] Docs ## How to test ```sh # Verify Go version go version # Should output: go version go1.27.0 ... # Build and test go test ./... # Verify Docker image builds docker build -f transports/Dockerfile . ``` ## Breaking changes - [x] Yes - [ ] No Go plugins must be compiled with the exact same Go version as Bifrost. Any existing plugins built with Go `1.26.x` must be recompiled with Go `1.27.0` to remain compatible. ## Related issues N/A ## Security considerations The base Docker image has been updated from `alpine3.23` to `alpine3.24`, which includes the latest security patches for the Alpine base layer. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… helm (#6226) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…e with `models`/`aliases` in values.yaml comments (#6510) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…e `rules` with expanded inline examples and `provider_config_ids` linkage (#6512) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…hema and values (#6513) ## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes #123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… and drop `keys` field from semantic cache config across Helm charts, values, and docs (#6517) ## Summary The semantic cache plugin's embedding provider API keys are now inherited from `bifrost.providers` instead of being configured directly inside the plugin's `config` block. This removes the redundant `keys` field from the semantic cache config and aligns key management with the rest of the Bifrost provider configuration pattern. ## Changes - Removed the `keys` field from `semanticCache.config` in the Helm chart schema, values, helpers, and all example overlays. The embedding provider's API key must now be configured under `bifrost.providers` and is inherited automatically by the plugin. - Updated the validation error message in `_helpers.tpl` to reflect that keys are no longer configured in the plugin block. - Added concrete `bifrost.providers.openai` blocks to all semantic cache example overlays (`sqlite-redis`, `sqlite-qdrant`, `sqlite-weaviate`, `postgres-redis`, `postgres-qdrant`, `postgres-weaviate`, `production-ha`) showing how to wire the Kubernetes secret into the provider via `env.SEMANTIC_CACHE_API_KEY`. - Added a Helm tab to the semantic caching documentation with a full working example and a note explaining the key inheritance model. - Removed the deprecated `disable_auth_on_inference` field from several example configs and moved `auth_config` into the `governance` block where it belongs in `withconfigstore/config.json`. - Renamed `credentials` to `credentials_json` in the GCS object storage example config. - Changed the default `trace_type` in the OTel example config from `otel` to `genai_extension`. - Replaced the deprecated `enforceGovernanceHeader` / `enforceSCIMAuth` fields with `enforceAuthOnInference` in the client config examples. - Removed `azure_key_config.api_version` and `azure_key_config.deployments` from Azure provider examples, replacing deployments with the `aliases` field and noting that the Azure v1 API requires no `api_version`. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [ ] UI (React) - [x] Docs ## How to test 1. Deploy any of the updated example overlays (e.g. `sqlite-redis.yaml`) and confirm the semantic cache initialises correctly using the provider key defined under `bifrost.providers`. 2. Confirm that omitting `bifrost.plugins.semanticCache.config.provider` (with `dimension != 1`) still produces the updated validation error message. 3. Confirm that passing a `keys` field inside `semanticCache.config` no longer has any effect and does not cause a schema validation error. ```sh helm template bifrost ./helm-charts/bifrost -f helm-charts/bifrost/values-examples/sqlite-redis.yaml | grep -A5 semanticCache ``` ## Breaking changes - [x] Yes - [ ] No The `keys` field inside `bifrost.plugins.semanticCache.config` is removed. Any existing values files that set `semanticCache.config.keys` must be migrated: move the API key to `bifrost.providers.<provider>.keys` and reference it via an environment variable (e.g. `env.SEMANTIC_CACHE_API_KEY`). The `secretRef` mechanism for injecting the key into the environment remains unchanged. ## Security considerations Embedding provider API keys are no longer accepted as a plain list inside the plugin config block, reducing the surface area for accidentally committing keys in values files. Keys must flow through `bifrost.providers`, which already supports `env.*` references and Kubernetes secret injection via `secretRef`. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary `SkipKeySelection` was previously allowed for any provider that wasn't Azure, Bedrock, BedrockMantle, or Vertex. This meant that if a governance routing rule rewrote the provider/model after the Claude Code OAuth transport set the flag, a non-Anthropic provider (e.g. Fireworks) would skip key selection entirely — leaving it without a configured key and causing request construction to fall back to the OpenAI schema, which breaks providers like Fireworks that require `max_tokens`. The fix tightens `isKeySkippingAllowed` to an allowlist of exactly one provider (`Anthropic`), since `SkipKeySelection` exists solely for Claude Code OAuth passthrough where the caller's token is the upstream credential and only the Anthropic provider forwards it. ## Changes - `isKeySkippingAllowed` now returns `true` only for `schemas.Anthropic`, replacing the previous denylist approach. This ensures non-Anthropic providers always receive a configured key from the pool. - The `selectKeyFromProviderForModelWithPool` call site now passes `baseProviderType` instead of `providerKey` to `isKeySkippingAllowed`, so the gate is evaluated against the resolved base provider. - `SkipKeySelection` is intentionally **not** cleared in `clearAnthropicPassthroughForNonNativeProvider` — it also drives `IsClaudeCodeMaxMode`, which suppresses `x-api-key` on the Anthropic provider. Clearing it during a non-native attempt would cause an Anthropic fallback to send the account key alongside the caller's OAuth token. The flag is gated at the read site instead. - A new test `TestSelectKeyFromProviderForModelWithPool_SkipKeySelectionGatedOnBaseProvider` verifies that Anthropic skips key selection while Fireworks (with `UseAnthropicEndpoints`) still selects its own key. - `TestClearAnthropicPassthroughForNonNativeProvider` is updated to assert that `SkipKeySelection` survives the clear operation and documents why. ## Type of change - [x] Bug fix ## Affected areas - [x] Core (Go) - [x] Providers/Integrations ## How to test ```sh go test ./core/... -run TestSelectKeyFromProviderForModelWithPool_SkipKeySelectionGatedOnBaseProvider go test ./core/... -run TestClearAnthropicPassthroughForNonNativeProvider go test ./... ``` The new test covers the regression directly: with `SkipKeySelection` set and a Fireworks provider, the selected key must be present and must have `UseAnthropicEndpoints = true`. Without this fix, the key pool would be empty and that assertion would fail. ## Breaking changes - [ ] Yes - [x] No ## Security considerations `SkipKeySelection` bypasses key injection entirely, relying on the caller's OAuth token as the upstream credential. Tightening the allowlist to Anthropic-only reduces the surface where a misconfigured or rewritten routing rule could cause a request to be sent without any credential, or with the wrong credential type for the target provider. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
…6538) ## Summary Two context values were leaking across provider boundaries during fallback and passthrough-clearing flows. A caller-supplied direct key (`x-bf-direct-key`) was surviving `clearCtxForFallback`, meaning the credential intended for provider A could be forwarded to provider B. Separately, the caller's Anthropic URL path (`/v1/messages`) was surviving `clearAnthropicPassthroughForNonNativeProvider`, meaning a converted request could attempt to land on Anthropic's endpoint instead of the fallback provider's own endpoint. ## Changes - `clearCtxForFallback` now clears `BifrostContextKeyDirectKey` so a caller-supplied key cannot ride a fallback onto a different provider - `clearCtxForFallback` now clears `BifrostContextKeyRoutingPinnedAPIKeyID` so a key pin scoped to the primary provider's pool is not carried into the fallback attempt - `clearAnthropicPassthroughForNonNativeProvider` now clears `BifrostContextKeyURLPath` so the Anthropic-specific path is not carried over when the request is converted for a non-native provider - Tests added to verify `DirectKey` and `RoutingPinnedAPIKeyID` are dropped before fallback key selection resolves to the configured provider pool, and that `URLPath` is cleared for non-native providers but preserved for native Anthropic passthrough ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/... -run TestClearAnthropicPassthroughForNonNativeProvider go test ./core/... -run TestClearCtxForFallback_DropsCallerSuppliedKey go test ./... ``` ## Breaking changes - [ ] Yes - [x] No ## Security considerations The `DirectKey` fix is security-relevant: without it, a credential supplied by the caller via `x-bf-direct-key` for one provider could be forwarded in a request to a different provider during fallback. This change ensures the caller's key is scoped strictly to the intended provider and that fallback attempts always resolve keys from the fallback provider's own configured pool. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
…l/logging semaphore controls, PostgreSQL node scheduling, and separate logs-store Postgres (#6542) ### TL;DR Releases Bifrost Helm chart v2.1.37 with Splunk HEC support, OTel/logging concurrency controls, PostgreSQL node scheduling options, and a separate logs store PostgreSQL configuration. ### What changed? - Added `bifrost.plugins.splunk` to enable the Splunk HTTP Event Collector (HEC) observability connector (Enterprise). Supports sending one flattened event per request to `events_index` and derived metrics to `metrics_index`, with TLS options (`ca_cert` / `insecure_skip_verify`), content logging toggle (`disable_content_logging`), request-header capture, and indexer acknowledgement controls (`indexer_ack`, `ack_poll_interval_ms`, `ack_timeout_ms`, `max_ack_attempts`). - Added `semaphore_size` and `inject_timeout` to `bifrost.plugins.otel.config` (defaulting to `10000` and `5` respectively) to cap concurrent in-flight trace injects and prevent a hung collector from holding a concurrency slot indefinitely. The same keys are also accepted under `bifrost.plugins.logging.config`, with `inject_timeout` expressed as a duration string (e.g. `"5s"`). - Added `postgresql.primary.nodeSelector`, `postgresql.primary.tolerations`, and `postgresql.primary.affinity` to allow the hosted PostgreSQL deployment to be scheduled independently of the Bifrost pod. All three default to empty, preserving existing rendering behavior unless explicitly set. - Added `storage.logsStore.postgres` to configure a separate external PostgreSQL instance for the logs store, distinct from the config store. Disabled by default (`enabled: false`), preserving existing behavior. Supports the same fields as `postgresql.external`, and when `existingSecret` is used, the password is injected via the `BIFROST_LOGS_POSTGRES_PASSWORD` environment variable. ### How to test? - Deploy the chart at v2.1.37 and configure `bifrost.plugins.splunk` with a valid Splunk HEC endpoint, then verify events and metrics appear in the configured indexes. - Set `semaphore_size` and `inject_timeout` under `bifrost.plugins.otel.config` and confirm trace injection respects the concurrency cap and timeout under load. - Apply `postgresql.primary.nodeSelector` or `tolerations` and verify the hosted PostgreSQL pod is scheduled to the intended nodes. - Enable `storage.logsStore.postgres` pointing to a separate PostgreSQL instance and confirm logs are written to that database while the config store remains unaffected. ### Why make this change? These additions expand observability options with Splunk HEC support, improve resilience of trace injection under collector failures, provide finer control over PostgreSQL pod placement for scaling scenarios, and allow the logs store to be separated from the config store for independent scaling and management.
Contributor
|
Important Review skippedToo many files! This PR contains 302 files, which is 2 over the limit of 300. To get a review, reduce the PR to 300 files or fewer by splitting it into smaller PRs or changing its base branch. Usage-priced reviews support at most 300 files. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (302)
You can disable this status message by setting the Warning Your free Security trial is over. An organization admin can activate billing to continue. Comment |
|
tejas ghatte seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
## Summary Bedrock's Converse API exposes an `s3Location` source union member for image and document blocks, but only some model families actually resolve it. Anthropic models have no S3 source type in their native Messages format, so Converse validates the union, drops the member during translation, and the model returns `document.source.type: Field required` — a field Converse does not have. This PR adds a model-capability gate that refuses `s3://` references for models that cannot read them, returning a 400 with an actionable error message naming the working alternative, rather than letting the request reach the model and fail with a cryptic internal error. ## Changes - Added `BedrockModelSupportsS3Location` to `core/schemas/utils.go` — an allowlist (currently Nova only) that gates `s3Location` forwarding. Allowlist rather than denylist because AWS no longer publishes a per-model S3 column; a model missing from the list is refused with an actionable error rather than silently mangled. - Added `bedrockS3LocationUnsupportedError` in `files.go` to produce a consistent refusal message that names the model, the content kind, the URI, and the working alternative (`file_data` for documents, `data:` URL for images). - Added `InvalidRequestErrorf` and `AsBifrostBadRequestError` in `core/providers/utils/utils.go` so converter errors caused by caller input promote to HTTP 400 rather than the default 500. `CheckContextAndGetRequestBody` now checks for this sentinel before wrapping in `ErrRequestBodyConversion`. - Threaded `model string` through the entire chat and responses message conversion call chain (`convertMessages`, `convertMessage`, `convertContent`, `convertContentBlock`, `convertToolMessages`, `convertImageToBedrockSource`, `ConvertBifrostMessagesToBedrockMessages`, `convertBifrostMessageToBedrockMessage`, `convertBifrostResponsesMessageContentBlocksToBedrockContentBlocks`) so the gate is reachable at every content-block conversion site. - The model gate runs ahead of format resolution: a model that cannot read the reference has no use for its format, and surfacing "cannot determine document format" would send the caller off renaming their S3 object for no gain. - Malformed `s3://` references (bucket with no object key) are still reported before the model gate, because that is the caller's more immediate problem and would not work on Nova either. - Existing tests that used an Anthropic model for S3 path coverage are switched to `novaModel` (`amazon.nova-lite-v1:0`), since the s3Location path is only reachable for a model whose Converse backend resolves it. - Added `s3locationmodelgate_test.go` with tests covering: refusal for Anthropic on both Chat and Responses surfaces, forwarding still works for Nova, malformed URI is reported before the model gate, and the refusal error survives `%w` wrapping and surfaces as a 400 (not 500). - Updated e2e harness cases 52.E1 and 52.E2 to assert the new Bifrost-level 400 refusal for Anthropic, and added 52.E3 and 52.E4 to assert that Nova still forwards `s3Location` to Converse verbatim. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./core/providers/bedrock/... -run TestS3Location go test ./core/providers/bedrock/... -run TestMalformedS3 go test ./core/providers/bedrock/... -run TestBedrockDocumentS3URI go test ./core/providers/bedrock/... -run TestBedrockImageS3URI go test ./core/providers/utils/... ``` For the e2e harness, run the provider-harness collection against a live Bifrost instance with Bedrock credentials. Cases 52.E1 and 52.E2 must return 400 with `does not support s3:// references` in the body. Cases 52.E3 and 52.E4 must show `s3Location` in the raw outbound request and a 4xx from AWS (bucket does not exist), not from Bifrost. ## Breaking changes - [x] Yes - [ ] No Callers sending `s3://` document or image references to Anthropic models on Bedrock will now receive a 400 from Bifrost instead of a cryptic model-level error. The request was never working; only the error surface changes. No change for Nova or other models confirmed to support `s3Location`. ## Security considerations None. The gate operates on the model identifier and URL scheme only; no credentials or object contents are accessed. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
akhsaul
pushed a commit
to akhsaul/bifrost
that referenced
this pull request
Aug 27, 2026
<Note> v2.0.0 is the first stable release on the 2.0 line. This changelog rolls up `2.0.0-prerelease1` (based on [v1.6.3](https://docs.getbifrost.ai/changelogs/v1.6.3)), `2.0.0-prerelease2`, `2.0.0-prerelease3` and the final release window, so it is the complete delta for a deployment upgrading from any v1.6.x release. Fixes that also shipped on the v1.6.x line after v1.6.3 are listed once here. </Note> <Warning> **Breaking changes.** Read the [v2.0.0 migration guide](https://docs.getbifrost.ai/migration-guides/v2.0.0) before upgrading. - **Custom plugin downloads are SSRF-protected** - a plugin `path` pointing at an http(s) URL is rejected if it resolves to a loopback, private, CGNAT, link-local or otherwise non-public address, and every custom plugin path is re-verified on each restart, including ones defined in `config.json`. - **Custom plugin create and update require admin authentication** - `POST /api/plugins` and `PUT /api/plugins/{name}` reject a custom `path` when the caller only got through because dashboard auth is disabled or unconfigured. - **Governance APIs moved under `/api/governance/*`** - `/api/teams`, `/api/users`, `/api/roles`, `/api/audit-logs` and other top-level governance paths moved under one namespace; Team and User lists use `limit`/`offset` pagination. Routing rules and the complexity analyzer moved from `/api/governance/*` to `/api/routing/rules` and `/api/routing/complexity-analyzer-config`; the old paths remain as deprecated aliases. - **`HTTPTransportPreHook` now runs after authentication** - the pipeline is `HTTPTransportPreAuthHook -> auth -> HTTPTransportPreHook -> handler`. Plugins that inject a credential (`x-bf-vk`, `Authorization`, `x-api-key`) must move that work to the new `HTTPTransportPreAuthHook`, and Go plugins implementing `HTTPTransportPlugin` must add the method (`.so` plugins that predate it are skipped for that phase). - **Legacy telemetry attributes removed** - the `gen_ai.*`-namespaced Bifrost-internal span attributes, `gen_ai.usage.prompt_tokens`/`completion_tokens`, the nanosecond `time_to_first_token` attribute and `x-bf-prom-*` request-header Prometheus dimensions are gone from the OTel and Prometheus connectors. Dashboards should read the `bifrost.*` keys and `time_to_first_chunk`. - **Gemini tool preference** - a Gemini API request carrying both function declarations and Google Search without `include_server_side_tool_invocations` now keeps the function declarations and drops Google Search (previously the opposite). Set `include_server_side_tool_invocations: true` to send both on Gemini 3 models. Vertex is unaffected. </Warning> ## ✨ Features - **Batch Accounting** - Provider batch jobs are tracked in a new `batch_jobs` table and settled asynchronously: results are priced per model from catalog batch rates (0.5 default ratio) on the `/results` path, one aggregate cost log is written idempotently with the creating request's identity, a background sweeper with ownership fencing re-drives jobs that timed out, settled usage is charged exactly once to the creating user's budgets and rate limits (including unscoped virtual key budgets on model-less batch-create requests), mixed-model batch rows are repriced during cost recalculation, and the log detail view shows a Batch Details block with per-state request counts and the settled cost (maximhq#5291, maximhq#5292, maximhq#5293, maximhq#5294, maximhq#5295, maximhq#5296, maximhq#6109, maximhq#6121, maximhq#6376, maximhq#6410, maximhq#6474, maximhq#6505) - **Claude-on-Vertex Batches** - Vertex batch jobs route Anthropic models to `publishers/anthropic/...`, build Claude-on-Vertex JSONL instances, round-trip `custom_id`, and preserve `tools`, `toolConfig`, `cachedContent`, `labels` and `display_name` on Gemini/Vertex batch requests (maximhq#5368) - **Input / Output Cost Split** - Every log carries `input_cost`, `output_cost` and `additional_cost` (guardrails, semantic cache, MCP) next to the total, across the RDB, ClickHouse, matviews, recalculation and the quota API; speech, transcription and OCR usages carry `BifrostCost`; the log detail view shows the split with per-category detail (maximhq#6511) - **Bifrost Overhead Latency** - `upstream_latency` and `overhead_latency` are recorded on every log, aggregated (avg, p90, p95, p99) in the dashboard's new Bifrost Overhead chart and shown in the log detail view; the overhead is decomposed by span self-time into serialization, conversion, plugins, middleware, key selection, queue wait, networking, client delivery and scheduling buckets (including streaming per-chunk parse, conversion and backpressure and the worker hand-off), persisted to `overhead_breakdown` and rendered as a stacked bar in the log detail view; a `bifrost_overhead_latency_microseconds` histogram is exported to Prometheus and OpenTelemetry and `upstream_latency_ms`/`overhead_latency_ms` tags to Maxim, while breakdown spans are kept out of observability connectors (maximhq#5533, maximhq#5534, maximhq#5535, maximhq#6345, maximhq#6388, maximhq#6389, maximhq#6433, maximhq#6470, maximhq#6495) - **Notification Center** - Role-targeted dashboard notifications stored in the database, delivered over WebSocket and surfaced in a topbar tray via `GET/POST /api/notifications` (maximhq#6207, maximhq#6227, maximhq#6324) - **Topbar and Responsive Dashboard** - Persistent topbar with page titles, theme toggle, external links, user menu and version; responsive layouts across all views with truncation and tooltips for long values and icon-only buttons; version-skew detection with an auto-reloading upgrading screen (maximhq#6196, maximhq#6105, maximhq#6126, maximhq#6204, maximhq#6232, maximhq#6330, maximhq#6370, maximhq#6476, maximhq#6485, maximhq#6493) - **Video Edits** - `POST /v1/videos/edits` applies prompt-driven edits, upscaling and background removal to an existing video supplied as bytes, a URL or a provider video ID, on OpenAI and Runware (maximhq#6270) - **Runware Chat, Catalog and Media Operations** - Chat completions, streaming and Responses via Runware's OpenAI-compatible endpoint, `ListModels` from the curated catalog, image upscale via `/v1/images/edits` (`type=upscale`), image-to-3D and async 3D generation via `/v1/videos` (`type=3d`), provider-reported per-task cost, and a raw `/runware_passthrough` route (maximhq#6260, maximhq#6372, maximhq#6208, maximhq#6075) - **JSON Image Edits** - `POST /v1/images/edits` accepts JSON bodies with URL or base64 images and typed extra params in addition to multipart (maximhq#6418) - **OpenAI Ultrafast Service Tier** - `service_tier: "ultrafast"` is forwarded only to models that support it and billed at dedicated ultrafast rates, with matching custom pricing override fields (maximhq#6396, maximhq#6399) - **Service Tier on Logs** - Logs record the tier actually served, including Anthropic's `service_tier` from `message_start` on streams, with a Service Tier column and detail field so repricing uses the served tier (maximhq#6233, maximhq#6236) - **Pricing Fields** - New per-request flat fee (`cost_per_request`), megapixel-based image tiers (4/8/16/32/64 MP), per-size and joint size+quality image rates for `gpt-image-1`-style models, and `input_cost_per_query` for rerank flow through datasheet sync, the cost engine, custom overrides, the API and the UI override form; upscale output resolution is backfilled from `target`/`factor` on Replicate so tiered rates bill the real output size (maximhq#6079, maximhq#6082, maximhq#6083, maximhq#6379, maximhq#6380) - **Model Catalog Pricing and Overrides** - Pricing data in the model catalog (thanks [@johnbrett](https://github.com/johnbrett)!), with resolved pricing overrides exposed on `/api/models/details` and on catalog rows, shown in the dashboard (maximhq#6055, maximhq#6056, maximhq#6058) - **Typed Embeddings on Bedrock** - Titan V2 `embeddingTypes` and Cohere `embedding_types` on Converse, the native invoke route and LangChain `BedrockEmbeddings` (maximhq#6381) - **Rerank Upgrades** - Structured JSON documents, `return_documents`, `next_token` pagination, caller document IDs preserved in every result, Cohere-shaped errors, cross-provider responses converted back to the caller's wire shape, and `/genai/v1/rank` served cross-provider (maximhq#6328, maximhq#6301, maximhq#6432) - **OpenRouter Speech, Transcription and Embeddings** - TTS and STT through OpenRouter's audio endpoints, and embedding models included in `ListModels` (maximhq#5734, maximhq#6264) - **Grok on Bedrock Mantle** - `xai.` models route through the `openai/v1` Mantle path (maximhq#6022) - **Gemini 3 Thinking Levels** - A per-model `thinkingLevel` support table clamps requested levels to the rungs each model implements; `reasoning_effort: "none"` sets the model's floor level instead of zeroing `thinkingBudget` (maximhq#6280) - **Datasheet-Backed Compatibility** - Anthropic, Bedrock, Cohere and Gemini request shaping (adaptive thinking, native effort, disable-reasoning, mid-conversation system turns, computer-use and text-editor tool generations, default max output tokens, tool validation) is resolved from model capabilities instead of hardcoded model-name checks (maximhq#6281, maximhq#6492) - **Reasoning Effort None** - Models that reason by default but do not support reasoning with tool calls get `reasoning.effort: "none"` when they advertise `supports_none_reasoning_effort`, instead of losing `reasoning` entirely (maximhq#6293) - **HTTP Transport Pre-Auth Hook** - New `HTTPTransportPreAuthHook` plugin phase runs before transport authentication so plugins can inject credentials such as `x-bf-vk`; a `virtual-key-from-config` native plugin example ships alongside it (maximhq#6375, maximhq#6373) - **Plugin Inject Limits** - Per-plugin `semaphore_size` and `inject_timeout` on `PluginConfig` bound observability `Inject` calls so a hung connector releases its slot (maximhq#6341) - **Harness Session Autodetection** - Claude Code, Codex CLI and OpenCode session headers populate the session ID when `x-bf-session-id` is absent (maximhq#6333) - **Auth and Model Check Skip Paths** - Context keys let trusted internal callers bypass auth resolution, and let evaluate-only requests such as `/inspect` bypass the virtual key provider and model allowlists while budgets and rate limits still apply (maximhq#6124, maximhq#6479) - **Passthrough Encoding Negotiation** - Forwarded `Accept-Encoding` is filtered to decodable codecs (gzip, deflate, brotli, zstd; gzip and identity for streams) and chained content encodings are decoded (maximhq#6360) - **Routing Plugin** - Routing rules and the complexity router live in a dedicated `routing` plugin that runs after governance so rules evaluate on the fully stamped context; endpoints moved to `/api/routing/rules` and `/api/routing/complexity-analyzer-config` with deprecated `/api/governance/*` aliases; complexity routing now reads the text of mixed text+image turns (maximhq#6144, maximhq#6145, maximhq#6146, maximhq#6147, maximhq#6253) - **Dimension Scope Ceiling** - Grouped log analytics (rankings, histograms, key pairs) are bounded to the customer, team, business unit, user and virtual key ids the caller may see (maximhq#6262) - **MCP Per-User OAuth and Token Exchange** - MCP clients can hold per-user OAuth credentials and per-user headers, configurable from `config.json` as well as the UI, with a documented shared vs per-identity token lookup contract, `oauth_config.resource` (RFC 8707), VK/Users filters on the OAuth Grants and MCP Auth Sessions sidebars and one shared create/install client form; `token_exchange` gains `use_idp_credentials` to reuse SSO login app credentials for providers such as Microsoft Entra ID (`client_id` becomes optional) and combines `offline_access` with `<audience>/.default` for Entra OBO; shared-OAuth clients show `needs_reauth` when their token row is invalidated, `Reauthorize` is limited to shared clients, the OAuth flow claim is atomic against concurrent reauth, stored scopes survive a decode failure, and credential caches propagate cancellation and version their entries (maximhq#6068, maximhq#6069, maximhq#6078, maximhq#6411, maximhq#6428, maximhq#6429, maximhq#6504) - **MCP Connection Lifecycle and Tool Discovery** - Discovered tools persist and resync uniformly across all client types through a hash-gated core callback, surviving restarts and propagating across a cluster; connections use make-before-break reconnects with ephemeral clients rebuilt across the whole connect+init retry, last-known tool maps preserved, connect attempts bound to entry identity and background reconnects deduped; `needs_session_stickiness` is pinned across `config.json` reconciliation; updating static headers on a sticky client pre-flight verifies the new credential and swaps it onto the live connection, per-call shared-credential clients refresh tools synchronously, and a failed enable parks the client at `Disabled` so it can be retried; the global `tool_sync_interval` hot-reloads and re-times running checkers; state badges render with spaces and the `disconnected` filter bucket is now `unstable` (maximhq#6409, maximhq#6430, maximhq#6431, maximhq#6483, maximhq#6502) - **Air-Gapped MCP Catalog** - `mcp_library_sync_interval: 0` disables catalog sync and `file://` URLs load the MCP server library from disk (maximhq#6195) - **MCP Log Redaction and Plugin Logs** - MCP tool logs carry redaction mappings and plugin logs (maximhq#5744, maximhq#5746) - **Splunk Connector Configuration** - `config.schema.json`, Helm values and dashboard entries for the Splunk HEC observability connector (maximhq#6296, maximhq#6091, maximhq#6099) - **Helm Broker Clustering** - `bifrost.cluster.type: broker` with broker address, port and TLS settings alongside the existing mesh transport (maximhq#6398) - **HTTP/2 Ping Interval in the UI** - Provider network configuration exposes `http2_ping_interval_in_seconds` (maximhq#6228) - **Status Code Badges** - Error and passthrough logs show the upstream HTTP status code in the log detail header (maximhq#5536) - **Server-Side Tool Calls in Logs** - `web_search_call`, `code_interpreter_call` and similar Responses items render their full payload in the log detail view (maximhq#6475) - **Gemini Server-Side Tool Calls** - Gemini `toolCall`/`toolResponse` parts surface as `web_search_call` items with their own call ID and queries, unmapped tool types are preserved on the native round-trip, and each `thoughtSignature` appears exactly once on replay (maximhq#6071) - **Bedrock VPC Endpoints** - AWS Bedrock keys can target VPC endpoints (maximhq#6064) - **W3C Trace ID Propagation** - Requests carry a W3C trace ID on the context (maximhq#5945) - **Durable Background Jobs** - New `sidekiq` background-job table, store methods, and runner with recovery and reaper; cost recalculation migrated to a durable, resumable and cancellable job with polling instead of SSE (maximhq#5800, maximhq#5801) - **Separate OTEL Metrics Pipeline** - The OTEL collector supports a metrics tab independent of traces, plus separate headers for traces and metrics (maximhq#5939, maximhq#5940) - **Grouped Logs View** - The logs table groups fallback chains under expandable roots backed by the new `roots_only` filter with child aggregates, and the model catalog persists tab, search and provider in the URL (maximhq#5522, maximhq#5737, maximhq#6059) - **User Agent and App Attribution** - Logs and MCP tool logs record user agent, app, source, decision, app key and device ID, with custom user-agent mapping and dashboard dimension rankings; MCP tool logs observed by the Bifrost Edge agent can be ingested with device, app key, decision and source attribution - **S3 Log Export Metadata** - Additional metadata is written alongside S3 log exports (maximhq#6070) - **Matview Maintenance Off Switch** - `matview_refresh_interval` accepts `"off"` to disable logstore matview maintenance entirely (thanks [@jeremym-tanium](https://github.com/jeremym-tanium)!) (maximhq#5693) - **Video Request Info in Logs UI** - Video requests surface their details in the logs UI (maximhq#5946) - **Shell Rewriter Hook** - The UI handler exposes a `ShellRewriter` hook for pre-hydration HTML rewriting (maximhq#5807) - **Custom Branding** - Logo and icon branding support with an OSS fallback stub, cached in localStorage to prevent a logo flash on load (maximhq#5806, maximhq#6096) - **User Assignment on Virtual Keys** - Users can be assigned from the virtual key sheet (maximhq#5863) - **Quarterly Budgets** - Quarterly budget windows with a configurable fiscal year start for customers and virtual key provider configs, surfaced in budget labels (maximhq#5996, maximhq#5997, maximhq#5999, maximhq#6115, maximhq#6116) - **Sarvam AI Provider** - Added Sarvam AI as a first-class provider with chat, text-to-speech, and speech-to-text support (thanks [@Purvi09](https://github.com/Purvi09)!) - **ElevenLabs Sound Effects** - Added text-to-sound generation support via `/v1/sound-generation` (thanks [@SecretSun](https://github.com/SecretSun)!) - **Bedrock Project Scoping** - Added optional `project_id` to Bedrock and Bedrock Mantle key configs with per-alias overrides for Bedrock, Bedrock Mantle, and Vertex, plus UI support - **Trace Redaction** - Phase-scoped redaction and revealing, transient redaction data field for guardrails, and trace content redaction before connector export - **Audit Log Object Storage** - S3/GCS object storage config schema for audit log archival - **Alerting Configuration** - Alerting schema in `config.schema.json` with declarative channels and CEL-based rules, Helm chart support, and enterprise fallback pages - **Canonical Model Names** - Dashboard model rankings now show canonical model names instead of inference-profile IDs (thanks [@satyamkrishna](https://github.com/satyamkrishna)!) - **OAuth2 Hardening** - Allowlist for private-use redirect URI schemes (RFC 8252 §7.1) and a `shouldSweep` gate on the OAuth2 sweep worker - **Mirrored Schema Support** - `schema_url` / `BIFROST_SCHEMA_URL` for mirrored schema locations in isolated deployments - **Vertex Single-Region Config** - Enforce single-region configuration in Vertex key config - **Helm Chart Updates** - `bifrost.alerting`, audit-log object storage, `postgresql.external.port` string support, and `bifrost.mcp.toolGroups[*].id` - **ChatGPT Passthrough** - Added a ChatGPT passthrough route on the OpenAI integration with dedicated request handling - **Edge Fallback Pages** - Added fallback pages for Bifrost Edge control views (config, devices, inventory) backed by governance resolver support - **Agent Handover View** - Added an agent handover page with seeded end-to-end data support - **First-Time Setup Token** - A setup token gates first-time setup so a fresh deployment is not open to the world, and the onboarding checklist is back, completing its dashboard auth step on SSO deployments (maximhq#5759, maximhq#5784, maximhq#6322) ## 🐞 Fixed - **Structured Output Schema Order** - `response_format` JSON schemas are forwarded byte-for-byte to OpenAI, Anthropic, Bedrock, Gemini and Cohere so the model generates fields in the caller's declared order instead of a re-sorted one (maximhq#6235) - **Thinking Block Typing on Streams** - Reasoning items carrying both an encrypted payload and a visible summary open as `thinking` blocks instead of `redacted_thinking` (maximhq#6292) - **Replayed Thinking Blocks via `bedrock/` Prefix** - Content-less `tool_result` blocks are kept, interleaved block order is preserved, `incomplete` maps to `error` on Converse, and pending reasoning is consumed by its owning item, so multi-turn tool use no longer wedges (maximhq#6346) - **Gemini 400s on Claude Code Traffic** - Trailing assistant prefills are trimmed and mid-conversation system turns are inlined for Gemini/Vertex; `extra_fields` is echoed on `/anthropic/v1/messages` (maximhq#6363) - **Bedrock Tool Use IDs** - IDs longer than 64 characters or outside Bedrock's charset (such as Gemini thought-signature IDs) are aliased deterministically on both `tool_use` and `tool_result` (maximhq#6300) - **Azure Responses Stream Errors** - Terminal `error` and `response.failed` events inside an already-open HTTP 200 SSE stream are surfaced as errors with their nested type, code and message (thanks [@dani29](https://github.com/dani29)!) (maximhq#6302) - **GenAI SSE Heartbeats** - GenAI streams delimit heartbeat comments so Google SDK clients preserve the following event, while older openai-go clients keep the bare heartbeat (thanks [@dani29](https://github.com/dani29)!) (maximhq#6252) - **OpenCode max_tokens** - `max_tokens` is preserved for OpenCode-compatible chat endpoints (thanks [@Alex-wangyang](https://github.com/Alex-wangyang)!) (maximhq#6458) - **HuggingFace Streaming Usage** - HuggingFace is no longer listed as omitting the `[DONE]` marker, and `stream_options.include_usage` defaults on its chat streaming path, so streamed calls stop reporting zero tokens and zero cost (thanks [@elliottrabac](https://github.com/elliottrabac)!) (maximhq#6478) - **Provider Key Name on Update** - A key PUT that omits `name` no longer clears it, and already-exists errors keep their constraint detail (thanks [@cpsc](https://github.com/cpsc)!) (maximhq#6417) - **Bedrock Mantle Streaming** - Bedrock Mantle is registered in `ProviderSendsDoneMarker` so streams end after `finish_reason` (maximhq#6021) - **URL-Sourced Files and Images** - `gs://` URIs go to Gemini/Gemma as `fileData.fileUri` and are read from Cloud Storage for Claude-on-Vertex, `s3://` references go to Bedrock Converse as `s3Location`, Bedrock rerank synthesizes the foundation-model ARN from a bare model ID, OpenAI file blocks keep `file_url`, non-http schemes pass through on the OpenAI and native-Anthropic paths, and Gemini always emits a candidate with its finish reason and drops payload-free parts (maximhq#6239) - **Together and Alias Pricing** - The management catalog resolves runtime provider `together` to the datasheet identity and prices configured aliases through their target model (thanks [@dani29](https://github.com/dani29)!) (maximhq#6257, maximhq#6320) - **Redis Vector Store TAG Escaping** - All RediSearch special characters are escaped in TAG query values (thanks [@AdityaPainuli](https://github.com/AdityaPainuli)!) (maximhq#5351) - **MCP Tool Sync Interval Corruption** - Toggling an MCP client's enable/disable switch no longer corrupts `tool_sync_interval`; the value is a whole number of minutes, negative values are rejected instead of silently disabling sync, and re-enabling a per-call client restarts its discovery cycle (maximhq#6409, maximhq#6502) - **MCP Tool Map Staleness** - `SetClientTools` replaces the in-memory tool map instead of merging, so tools removed upstream leave memory once the database has dropped them (maximhq#6484) - **SSE Reconnect Identity** - `OnConnectionLost` on SSE MCP clients is gated on connection identity so a stale connection cannot tear down its replacement - **Connector Header Redaction** - `Authorization`, `x-api-key`, Cloudflare Access and AWS ALB OIDC headers are redacted before export to every observability backend (maximhq#6371) - **Vertex Mixed Tools** - Vertex AI accepts function declarations and Google Search in the same request without `includeServerSideToolInvocations`, and search localization via `retrievalConfig.latLng` is preserved (maximhq#6066) - **Gemini Tool Preference** - When tool combination is disabled, function declarations win over Google Search so the model can still call the caller's tools (maximhq#6065) - **Bedrock Stop Reasons** - Bedrock `content_filter` and `guardrail_intervened` stop reasons map to `incomplete` status with a `content_filter` reason - **Encrypted Reasoning on Compaction** - The fail-soft that strips `encrypted_content` before retrying a rejected request also covers `/v1/responses/compact` and count-tokens requests, and recognizes Anthropic's `redacted_thinking` rejection (maximhq#6041, maximhq#5960) - **DAC-Scoped VK Reads** - `from_memory` virtual key reads are blocked for DAC-scoped callers - **Path Normalization Auth Bypass** - Fixed a path normalization flaw that allowed auth to be bypassed (maximhq#5763) - **Minimal Reasoning Effort on GPT-5 Models** - `reasoning_effort: "minimal"` is preserved for GPT-5-family OpenAI models instead of being downgraded to `low` (thanks [@jitokim](https://github.com/jitokim)!) (maximhq#6046) - **Gemini Truncated Response Finish Reason** - Truncated Gemini responses report `MAX_TOKENS` instead of `OTHER` (thanks [@AdityaPainuli](https://github.com/AdityaPainuli)!) (maximhq#5979) - **Null Tool-Call Function Name on Streaming** - Streaming continuation deltas no longer materialize an absent tool-call function name as `null` (thanks [@AdityaPainuli](https://github.com/AdityaPainuli)!) (maximhq#5966) - **Bedrock Document Uploads** - Fixed Bedrock file handling in inference so office and PDF documents sent as OpenAI `type: "file"` are accepted (maximhq#5947) - **xAI Usage Cost** - Fixed USD cost ticks for xAI usage (maximhq#5950) - **Governance List-Models Call** - Budgets and rate limits no longer trigger a list-models call (maximhq#6051) - **Realtime Response Create Input** - Guarded `response.create` input (maximhq#6050) - **Governance Rate-Limit Reset CPU** - Guards against invalid reset timeouts, parallelized resting-budget flows only when absolutely required, and fixed the calendar-based alignment qualifier - **Masked Key Persistence** - Never persist masked provider key previews to config storage (thanks [@eyeveil](https://github.com/eyeveil)!) - **OpenShift Arbitrary UIDs** - Build-time group-0 ownership with no runtime chown (thanks [@eyeveil](https://github.com/eyeveil)!) - **Passthrough Virtual Key Attribution** - Passthrough calls via the Azure `api-key` header now attribute to the virtual key (thanks [@eyeveil](https://github.com/eyeveil)!) - **Rerank for Custom Providers** - `/v1/rerank` now works with custom OpenAI-compatible providers (thanks [@eyeveil](https://github.com/eyeveil)!) - **Responses Stream Usage** - Persist stream usage when providers omit or reuse sequence numbers (thanks [@eyeveil](https://github.com/eyeveil)!) - **Wildcard allowed_models Repair** - Repair bare wildcard `allowed_models` rows that broke admin provider updates (thanks [@eyeveil](https://github.com/eyeveil)!) - **Streaming Error Panic** - Nil-safe tracing span lookup prevents panics on streaming errors (thanks [@eyeveil](https://github.com/eyeveil)!) - **Anthropic Tool ID Sanitization** - Sanitize `tool_use`/`tool_result` ids to Anthropic's charset (thanks [@Shaik-Sirajuddin](https://github.com/Shaik-Sirajuddin)!) - **Realtime Transcription Sessions** - Support GA transcription-type sessions in `POST /v1/realtime/client_secrets` (thanks [@Shaik-Sirajuddin](https://github.com/Shaik-Sirajuddin)!) - **Diarized Transcription** - Support `diarized_json` segments and ElevenLabs speaker passthrough (thanks [@Shaik-Sirajuddin](https://github.com/Shaik-Sirajuddin)!) - **Model Discovery** - Skip disabled keys when scheduling model-discovery fetches (thanks [@Shaik-Sirajuddin](https://github.com/Shaik-Sirajuddin)!) - **MCP Timeout Placeholder** - Show the real global default in the MCP tool execution timeout placeholder (thanks [@Shaik-Sirajuddin](https://github.com/Shaik-Sirajuddin)!) - **Redacted Thinking Round-Trip** - Round-trip Anthropic `redacted_thinking` blocks on the Responses surface (thanks [@fus3r](https://github.com/fus3r)!) - **Streaming Accumulation** - Preserve citation annotations and `finish_reason` in the accumulated streaming response (thanks [@fus3r](https://github.com/fus3r)!) - **Gemini Grounded Streaming** - Reset web-search flag when recycling pooled stream state so `web_search_call` items keep emitting (thanks [@fus3r](https://github.com/fus3r)!) - **Bedrock Truncation Signal** - Signal `max_output_tokens` truncation on the Responses API (thanks [@jeremym-tanium](https://github.com/jeremym-tanium)!) - **Bedrock Reasoning Config** - Preserve `reasoning_config` on cross-provider translation so fallbacks keep extended thinking (thanks [@Purvi09](https://github.com/Purvi09)!) - **Anthropic tool_search** - Forward and rebuild server-side `tool_search` on the Responses path (thanks [@ws4charlie](https://github.com/ws4charlie)!) - **OpenAI Responses Input** - Strip `role` from non-message input items (thanks [@nettee](https://github.com/nettee)!) and serialize compaction request `input` correctly (thanks [@mcclurmc](https://github.com/mcclurmc)!) - **additional_tools Support** - Added `additional_tools` message type support, preserving nested tool types on `/v1/responses` - **Plugin Stream Errors** - Emit structured plugin stream errors on integration routes (thanks [@jeffhos](https://github.com/jeffhos)!) - **Pooled Object Hygiene** - Zero pooled ChannelMessage references on release and sweep orphaned deferred spans in trace store TTL cleanup (thanks [@citrocat](https://github.com/citrocat)!) - **Hybrid Log Token Usage** - Rebuild token usage from denormalized columns in hybrid log list (thanks [@G-XD](https://github.com/G-XD)!) - **MCP Tool Ordering** - Deterministic MCP tool ordering for prompt cache stability - **MCP Inline-Auth Links** - Warn callers not to truncate the `#t=` temp-token fragment (thanks [@MarcusPeng](https://github.com/MarcusPeng)!) - **Gemini Fixes** - Web search options map to Google Search grounding, file upload MIME types preserved, and video reference fields map to instances (thanks [@vojthor](https://github.com/vojthor)!) - **OpenAI Parameters** - Honor service tier in chat completion and cap max reasoning effort - **Anthropic Costing** - Correct inference geo cost and cache rate for fast mode - **SecretVar Parsing** - Parse `SecretVar` JSON with `ref`/`env_var` fields even when `value` is absent - **Telemetry** - Forward request id and trace id, reduce metrics cardinality explosion risk, and send status codes on OTEL metrics - **Dashboard** - Preserve active time period when applying dimension filters, adjust bucket size thresholds for month-range durations, show user popover with `preferred_username` fallback, filter provider-level keys from the prompt manager selector (thanks [@rlex](https://github.com/rlex)!), skip password validation for redacted credentials, and improve `ModelMultiselect` empty and error states - **API Key Provider Selection** - Fixed provider selection for API keys - **Azure Auth Headers** - Pass Azure auth headers in helpers - **Stream Delta Schema** - Added `ExtraContent` to `ChatStreamResponseChoiceDelta` (thanks [@nghodkicisco](https://github.com/nghodkicisco)!) - **API Auth Bypass** - Stopped `/api/devices` bypassing auth via the `/api/dev` prefix - **Bedrock Error Types** - Surface the AWS exception type (`X-Amzn-Errortype`) on non-streaming Bedrock error responses instead of dropping it ## 🔧 Maintenance - **Hot-Path Performance** - Cached serialization for shared MCP tools, a direct `OrderedMap` JSON writer, bulk span attribute writes with cached span pointers, reusable worker delivery timers, retained span attribute maps, generation-stamped memoization of `GetProvidersForModel` and `GetModelsForProvider` via the new `gencache` package, sonic-based JSON responses, and a plugin-log existence check before draining (maximhq#6242, maximhq#6241, maximhq#5956, maximhq#5957, maximhq#5657, maximhq#6387, maximhq#5641, maximhq#6224, maximhq#6268, maximhq#6211) - **Go Toolchain** - Modules build with Go 1.26.6 and the Nix flake pins 1.26.7 (maximhq#6269, maximhq#6385) - **Dependency Upgrades** - Dependabot updates across all modules, newman 6.2.2 with pinned transitive overrides, module path fixes and `openai_config` referenced from every provider config schema (maximhq#6040, maximhq#5864, maximhq#6267, maximhq#6305, maximhq#6275) - **Test Coverage** - vLLM instances provisioned on RunPod in the release pipeline, Runware harness coverage including `/v1/images/edits` and `/v1/videos`, batch and pricing-override lifecycle harness cases, an Anthropic `message_start` usage regression test, LangChain rerank and embedding integration tests, and e2e fixes for dashboard auth, budget reset and MCP state (maximhq#5541, maximhq#6303, maximhq#6319, maximhq#6299, maximhq#6327, maximhq#6432, maximhq#6351) - **Documentation** - v2.0.0 migration guide with the governance namespace mapping and a v1.5.x downgrade guide for `prerelease3` deployments, v2.0.0 availability callouts, routing API namespace docs, Bedrock application inference profiles, Splunk connector docs, config.schema.json and Datadog env var reference fixes, and Discord badge fixes (thanks [@Swpn0neel](https://github.com/Swpn0neel)!) (maximhq#6332, maximhq#6374, maximhq#6420, maximhq#6147, maximhq#6203, maximhq#6099, maximhq#5938, maximhq#6019, maximhq#6425, maximhq#6448) - **Helm** - Chart releases v2.1.35 and v2.1.36 (maximhq#6129, maximhq#6249) - **Governance Route Families** - Editions can override governance route families (maximhq#5839) ## 🗄️ Database Migrations All migrations below are new relative to v1.6.11. Deployments on an older v1.6.x release should also review the intermediate v1.6.x changelogs. **configstore:** - **add_mcp_client_pending_oauth_config_json_column** - Adds `pending_oauth_config_json` to `config_mcp_clients`. Reversible: drops the added column. - **merge_oauth_token_tables** - Consolidates `oauth_tokens` and `oauth_user_tokens` into `mcp_oauth_tokens`. **Non-reversible**: rollback deliberately leaves `mcp_oauth_tokens` in place, because every OAuth read and write targets it from this migration onward and dropping it would destroy any token created or refreshed since, forcing every holder to re-authorize. - **create_mcp_oauth_flows_table** - Creates `mcp_oauth_flows` to track in-flight OAuth flows. Reversible: drops the new table. - **drop_oauth_config_pkce_columns** - Drops CSRF state, PKCE verifier and `expires_at` from the OAuth config table now that they live on `mcp_oauth_flows`. **Non-reversible**: forward-only, the dropped values were per-flow ephemeral and re-adding empty columns would restore nothing. - **drop_oauth_config_token_id_column** - Drops `token_id`. **Non-reversible**: forward-only, it was a pure FK shortcut now reachable via `(oauth_config_id, auth_mode)`. - **add_mcp_admin_auth_mode_indexes** - Adds admin partial unique indexes on `mcp_oauth_tokens` and `mcp_per_user_header_credentials`. Reversible: drops both indexes. - **add_mcp_client_token_exchange_json_column** - Adds `token_exchange_json` to `config_mcp_clients`. Reversible: drops the added column. - **add_needs_session_stickiness_column** - Adds `needs_session_stickiness` to `config_mcp_clients`. Reversible: drops the added column. - **add_bedrock_endpoints_columns** - Adds Bedrock VPC endpoint columns to the keys table. Reversible: drops the added columns. - **add_cost_per_request_pricing_column** - Adds `cost_per_request` to model pricing. Reversible: drops the added column. - **add_notifications_table** - Creates the `notifications` table for the dashboard notification center. Reversible: drops the table. - **add_batch_jobs_table** - Creates `batch_jobs` with a unique `(provider, batch_id)` identity index, a sweeper scan index and a runner-id index. Reversible: drops the table. - **add_image_megapixel_tier_pricing_columns** - Adds the five `output_cost_per_image_above_{4,8,16,32,64}_megapixels` columns to model pricing. Reversible: drops the added columns. - **add_input_cost_per_query_column** - Adds `input_cost_per_query` to model pricing for rerank. Reversible: drops the added column. - **add_ultrafast_pricing_columns** - Adds the four `*_ultrafast` token rate columns to model pricing. Reversible: drops the added columns. - **add_image_size_quality_pricing_columns** - Adds the 14 per-size and size+quality image output rate columns to model pricing. Reversible: drops the added columns. - **add_batch_jobs_attribution_columns** - Adds `user_id`, `team_id`, `customer_id` and `source_log_id` to `batch_jobs` plus a `user_id` index. Reversible: drops the index and the four columns. **logstore:** - **logs_add_guardrail_debug_column** - Adds `guardrail_debug` to logs. Reversible: drops the added column. - **mcp_tool_logs_add_redaction_mapping_column** - Adds the redaction mapping column to MCP tool logs. **Non-reversible**: rollback is a no-op because dropping the column would permanently destroy reveal data for already-redacted MCP logs. - **logs_add_user_agent_column** - Adds user agent and app columns, their indexes, and a `UserAgentMapping` table. Reversible: drops the indexes and the mapping table. - **mcp_tool_logs_add_user_agent_column** - Adds user agent and app columns plus indexes to MCP tool logs. Reversible: drops both indexes and the `app` column. - **logs_recreate_matviews_with_app_column** - Recreates the log materialized views to include the user agent and app columns. Rollback is a no-op because `ensureMatViews` recreates them on next startup. - **mcp_tool_logs_add_endpoint_columns** - Adds `source`, `decision`, `app_key` and `device_id` to MCP tool logs. Reversible: drops all four columns. - **mcp_tool_logs_add_plugin_logs_column** - Adds `plugin_logs` to MCP tool logs. Reversible: drops the added column. - **logs_add_video_edit_input_column** - Adds `video_edit_input` to logs. Reversible: drops the added column. - **logs_add_upstream_and_overhead_latency_columns** - Adds `upstream_latency` and `overhead_latency` to logs. Reversible: drops both columns. - **logs_add_batch_debug_column** - Adds `batch_debug` to logs. Reversible: drops the added column. - **logs_add_cost_breakdown_columns** - Adds `input_cost`, `output_cost` and `additional_cost` to logs. Reversible: drops the three columns. - **logs_recreate_matviews_with_cost_breakdown** - Marks the hourly matview for rebuild with the cost split columns; `repairMatViewShapes` drops and recreates `mv_logs_hourly` on the next startup. Rollback is a no-op because `ensureMatViews` recreates it on next startup. - **logs_add_overhead_breakdown_column** - Adds `overhead_breakdown` to logs. Reversible: drops the added column. <Warning> **High-throughput deployments: run the logstore migrations during a low-activity window.** Every logstore migration above alters `logs` or `mcp_tool_logs`, the two highest-insert tables in Bifrost, and several also build indexes on them. On a busy instance the index builds hold locks that block concurrent log inserts for the duration of the build, and the matview recreations rebuild against the full table. Schedule the upgrade for a low-traffic period, or expect elevated log-write latency and possible request-path backpressure while the migrations run. </Warning> <Warning> `merge_oauth_token_tables`, `drop_oauth_config_pkce_columns` and `drop_oauth_config_token_id_column` transform or remove existing OAuth state and cannot be rolled back. Take a database backup before upgrading, and do not roll the binary back past this release once the migration has run. </Warning> ## 🐙 Closed GitHub Issues - [maximhq#123](maximhq#123) - Files API Support - [maximhq#2347](maximhq#2347) - MCP tool ordering is non-deterministic, breaking prefix-based prompt caching - [maximhq#3455](maximhq#3455) - Segfault/nil dereference panic in Bedrock provider - [maximhq#4318](maximhq#4318) - allowed_models persisted as bare "*" string blocks subsequent provider updates - [maximhq#4353](maximhq#4353) - config.db corruption from masked-key preview in provider_configs JSON column - [maximhq#4367](maximhq#4367) - Image incompatible with OpenShift arbitrary UIDs - [maximhq#4402](maximhq#4402) - Vertex provider drops image blocks whose URL uses gs:// scheme - [maximhq#4477](maximhq#4477) - Passthrough calls using a Virtual Key log as actual key - [maximhq#4679](maximhq#4679) - Bedrock Responses API does not signal max_output_tokens truncation - [maximhq#4689](maximhq#4689) - Custom providers cannot set budget - [maximhq#4712](maximhq#4712) - ElevenLabs sound effects (/v1/sound-generation) - [maximhq#4780](maximhq#4780) - Anthropic server-side tool_search results are dropped on /v1/responses - [maximhq#4834](maximhq#4834) - /v1/rerank is not available with custom providers - [maximhq#4846](maximhq#4846) - Responses stream usage present in response.completed but not persisted in LLM Logs - [maximhq#4851](maximhq#4851) - Governance rate-limit reset causes high CPU in BumpRateLimitUsage - [maximhq#4870](maximhq#4870) - Pooled ChannelMessage retains request body, context, and undelivered response while idle - [maximhq#4940](maximhq#4940) - Show canonical model names instead of Bedrock inference-profile IDs in Model Rankings - [maximhq#4963](maximhq#4963) - Streaming finish_reason dropped from the accumulated (logged) response - [maximhq#5002](maximhq#5002) - gpt-4o-transcribe-diarize transcription fails due to string segment IDs - [maximhq#5013](maximhq#5013) - OpenAI /responses/compact input serialized as a JSON object causing 400 - [maximhq#5026](maximhq#5026) - [Bug]: Toggling an MCP client's enable/disable switch corrupts its tool_sync_interval (nanoseconds resent as minutes) - [maximhq#5027](maximhq#5027) - MCP Tool Execution Timeout placeholder shows 0 instead of real global default - [maximhq#5036](maximhq#5036) - Plugin StreamInterceptionError is flattened on integration routes - [maximhq#5037](maximhq#5037) - Disabled keys break provider model discovery - [maximhq#5051](maximhq#5051) - Add Sarvam AI provider (chat + TTS/STT) - [maximhq#5061](maximhq#5061) - Streaming responses drop citation annotations from the accumulated message - [maximhq#5093](maximhq#5093) - Streaming /v1/responses drops Anthropic redacted_thinking blocks - [maximhq#5097](maximhq#5097) - Anthropic rejects replayed tool_use/tool_result ids from non-conforming upstream providers - [maximhq#5100](maximhq#5100) - additional_tools loses nested tool types on /v1/responses - [maximhq#5101](maximhq#5101) - Chat-to-Responses tool replay sends role on function_call input items - [maximhq#5108](maximhq#5108) - Bedrock reasoning_config silently dropped on cross-provider translation - [maximhq#5113](maximhq#5113) - Gemini/Vertex streaming stops emitting web_search_call items after first grounded request - [maximhq#5432](maximhq#5432) - Add TTS and STT support for OpenRouter - [maximhq#5472](maximhq#5472) - [Bug]: Bedrock rejects office/PDF document uploads via OpenAI `type:"file"` - "The PDF specified was not valid" - [maximhq#5871](maximhq#5871) - [Bug]: AWS Bedrock Mantle streaming is broken - [maximhq#5874](maximhq#5874) - [Bug]: SSE heartbeat frame aborts streams for openai-go ssestream consumers (< v3.43.0) with "unexpected end of JSON input" - [maximhq#5885](maximhq#5885) - [Bug]: v1.6.8 omits message_start.message.usage on Bedrock-backed providers, breaking @ai-sdk/anthropic streaming - [maximhq#5900](maximhq#5900) - [Bug]: Streaming continuation chunks materialize omitted tool-call metadata as null - [maximhq#5978](maximhq#5978) - [Bug]: Gemini egress reports truncated responses as FinishReason OTHER, IncompleteDetails switch matches a string that never occurs - [maximhq#6044](maximhq#6044) - [Bug]: normalizeOpenAIReasoningEffort maps 'minimal' to 'low' for ALL OpenAI models, even ones that natively support 'minimal' - [maximhq#6240](maximhq#6240) - [Bug]: GenAI SSE heartbeat framing causes @google/genai to silently drop the following data event - [maximhq#6248](maximhq#6248) - [Bug]: OpenRouter embedding models missing from Semantic Cache dropdown - [maximhq#6334](maximhq#6334) - [Bug]: Gemini/Vertex provider fails on Claude Code assistant prefills and mid-conversation system turns (Gemini 3.6 Flash & 3.7 Flash HTTP 400) - [maximhq#6342](maximhq#6342) - [Bug]: Anthropic ingress with bedrock/ prefix restructures replayed thinking blocks, wedging multi-turn tool use on claude-opus-4-8 - [maximhq#6416](maximhq#6416) - [Bug]: Provider key update silently clears "name" when omitted, then the unique-name index 409s subsequent updates - [maximhq#6457](maximhq#6457) - [Bug]: OpenCode chat endpoints drop max completion limit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pathpointing at an http(s) URL is rejected if it resolves to a loopback, private, CGNAT, link-local or otherwise non-public address, and every custom plugin path is re-verified on each restart, including ones defined inconfig.json.POST /api/pluginsandPUT /api/plugins/{name}reject a custompathwhen the caller only got through because dashboard auth is disabled or unconfigured./api/governance/*-/api/teams,/api/users,/api/roles,/api/audit-logsand other top-level governance paths moved under one namespace; Team and User lists uselimit/offsetpagination. Routing rules and the complexity analyzer moved from/api/governance/*to/api/routing/rulesand/api/routing/complexity-analyzer-config; the old paths remain as deprecated aliases.HTTPTransportPreHooknow runs after authentication - the pipeline isHTTPTransportPreAuthHook -> auth -> HTTPTransportPreHook -> handler. Plugins that inject a credential (x-bf-vk,Authorization,x-api-key) must move that work to the newHTTPTransportPreAuthHook, and Go plugins implementingHTTPTransportPluginmust add the method (.soplugins that predate it are skipped for that phase).gen_ai.*-namespaced Bifrost-internal span attributes,gen_ai.usage.prompt_tokens/completion_tokens, the nanosecondtime_to_first_tokenattribute andx-bf-prom-*request-header Prometheus dimensions are gone from the OTel and Prometheus connectors. Dashboards should read thebifrost.*keys andtime_to_first_chunk.include_server_side_tool_invocationsnow keeps the function declarations and drops Google Search (previously the opposite). Setinclude_server_side_tool_invocations: trueto send both on Gemini 3 models. Vertex is unaffected.✨ Features
batch_jobstable and settled asynchronously: results are priced per model from catalog batch rates (0.5 default ratio) on the/resultspath, one aggregate cost log is written idempotently with the creating request's identity, a background sweeper with ownership fencing re-drives jobs that timed out, settled usage is charged exactly once to the creating user's budgets and rate limits (including unscoped virtual key budgets on model-less batch-create requests), mixed-model batch rows are repriced during cost recalculation, and the log detail view shows a Batch Details block with per-state request counts and the settled cost (feat: add batch schemas and provider plumbing #5291, feat: add batch pricing to model catalog #5292, feat: add batch jobs table to configstore and batch debug to logstore #5293, feat: add batch accounting engine and sweeper #5294, feat: report batch usage to governance #5295, feat: wire batch accounting into logging and transport #5296, feat: adds row in logs sheet to render batch pricing information #6109, fix: handle log cost recalculation for batch #6121, fix: adds batch types in filters and make batch id column take full width #6376, fix: logs in batches ui, resolve VK-scoped wildcard budgets when request has no model #6410, fix: use bifrost as user-agent name, show pricing for manual batch calls when resolved by sweeper #6474, fix: stop background cost recalculation zeroing batch aggregate rows and charge access profile budgets for model-less requests #6505)publishers/anthropic/..., build Claude-on-Vertex JSONL instances, round-tripcustom_id, and preservetools,toolConfig,cachedContent,labelsanddisplay_nameon Gemini/Vertex batch requests (fix: vertex batches anthropic compatibility #5368)input_cost,output_costandadditional_cost(guardrails, semantic cache, MCP) next to the total, across the RDB, ClickHouse, matviews, recalculation and the quota API; speech, transcription and OCR usages carryBifrostCost; the log detail view shows the split with per-category detail (feat: update log details views UI to show input / output cost split #6511)upstream_latencyandoverhead_latencyare recorded on every log, aggregated (avg, p90, p95, p99) in the dashboard's new Bifrost Overhead chart and shown in the log detail view; the overhead is decomposed by span self-time into serialization, conversion, plugins, middleware, key selection, queue wait, networking, client delivery and scheduling buckets (including streaming per-chunk parse, conversion and backpressure and the worker hand-off), persisted tooverhead_breakdownand rendered as a stacked bar in the log detail view; abifrost_overhead_latency_microsecondshistogram is exported to Prometheus and OpenTelemetry andupstream_latency_ms/overhead_latency_mstags to Maxim, while breakdown spans are kept out of observability connectors (feat: add Bifrost overhead latency to logs details view #5533, feat: adds overhead data to matviews and db #5534, feat: adds bifrost overhead chart to UI dashboards #5535, feat: start sending overhead metrics to connectors #6345, feat: per-phase overhead breakdown - instrument, compute, persist, display #6388, feat: keep breakdown spans out of observability connectors #6389, fix: update stream overhead with sub-buckets and UI fixes #6433, chore: update core with sub-buckets #6470, chore: fix streaming time calculations #6495)GET/POST /api/notifications(adds notification center #6207, fix(ui): hide notification center icon until notifications are loaded or open #6227, fix(ui): move beta badge intoPageTitle, simplify notification trigger logic, and fix virtual keys toolbar overflow #6324)FilterSidebarTriggercomponent with mobile topbar portal #6232, fix(ui): truncate long values in log detail view and improve provider button responsiveness #6330, fix(ui): unify full-height page sizing with--app-bottom-paddingCSS variable and tighten sidebar/content margins #6370, move version into the dropdown #6476, fix(ui): add open-state highlight to topbar triggers, inlinesideOffset, and removeTOPBAR_MENU_SIDE_OFFSETconstant #6485, topbar fixes #6493)POST /v1/videos/editsapplies prompt-driven edits, upscaling and background removal to an existing video supplied as bytes, a URL or a provider video ID, on OpenAI and Runware (feat: video edit request type #6270)ListModelsfrom the curated catalog, image upscale via/v1/images/edits(type=upscale), image-to-3D and async 3D generation via/v1/videos(type=3d), provider-reported per-task cost, and a raw/runware_passthroughroute (feat: chat completions for runware provider #6260, feat: runware list models api #6372, fix: runware upscale compatibility #6208, feat(runware): async 3D generation via /videos + passthrough route #6075)POST /v1/images/editsaccepts JSON bodies with URL or base64 images and typed extra params in addition to multipart (fix: accept json bodies in image edit request #6418)service_tier: "ultrafast"is forwarded only to models that support it and billed at dedicated ultrafast rates, with matching custom pricing override fields (fix: openai service tier ultrafast #6396, fix: update overrides fields #6399)service_tierfrommessage_starton streams, with a Service Tier column and detail field so repricing uses the served tier (feat(ui): add service tier column and detail view to logs #6233, fix(anthropic): propagateservice_tierthrough streaming accumulator to final chunk and log entry #6236)cost_per_request), megapixel-based image tiers (4/8/16/32/64 MP), per-size and joint size+quality image rates forgpt-image-1-style models, andinput_cost_per_queryfor rerank flow through datasheet sync, the cost engine, custom overrides, the API and the UI override form; upscale output resolution is backfilled fromtarget/factoron Replicate so tiered rates bill the real output size (feat: addcost_per_requestflat-fee pricing field across DB, cost engine, overrides, and docs #6079, feat: add megapixel-tier image pricing fields (4/8/16/32/64MP) across DB, cost engine, overrides, API, docs, and UI #6082, feat: backfill upscale output resolution fromtarget/factorparams andmetrics.resolution_targetfor accurate resolution-tiered pricing #6083, feat: add per-size and joint size+quality image pricing columns, cost computation, and overrides forgpt-image-1-style models #6379, feat: add per-size and joint size+quality image pricing fields for 1024×1536 and 1536×1024 resolutions #6380)/api/models/detailsand on catalog rows, shown in the dashboard (feat(modelcatalog): resolve pricing overrides for catalog rows #6055, feat(api): expose pricing overrides on /api/models/details #6056, feat(ui): show overridden pricing in the model catalog #6058)embeddingTypesand Cohereembedding_typeson Converse, the native invoke route and LangChainBedrockEmbeddings(fix: bedrock langchain embedding #6381)return_documents,next_tokenpagination, caller document IDs preserved in every result, Cohere-shaped errors, cross-provider responses converted back to the caller's wire shape, and/genai/v1/rankserved cross-provider (fix: rerank compatibility and integration tests #6328, fix: rerank integration #6301, tests: integration for rerank langchain #6432)ListModels(fix: adds openrouter stt / tts #5734, fix: include openrouter embedding models in list models call #6264)xai.models route through theopenai/v1Mantle path (fix: route grok models through openai/v1 path for bedrock mantle #6022)thinkingLevelsupport table clamps requested levels to the rungs each model implements;reasoning_effort: "none"sets the model's floor level instead of zeroingthinkingBudget(gemini tool call fixes #6280)reasoning.effort: "none"when they advertisesupports_none_reasoning_effort, instead of losingreasoningentirely (fix: force reasoning effort to none when reasoning with tools is unsupported #6293)HTTPTransportPreAuthHookplugin phase runs before transport authentication so plugins can inject credentials such asx-bf-vk; avirtual-key-from-confignative plugin example ships alongside it (feat: addHTTPTransportPreAuthHookphase that runs before transport authentication, movingHTTPTransportPreHookto run after #6375, feat: addvirtual-key-from-confignative Go plugin example with pre-auth hook, tests, and README #6373)semaphore_sizeandinject_timeoutonPluginConfigbound observabilityInjectcalls so a hung connector releases its slot (feat: add per-pluginsemaphore_sizeandinject_timeouttoPluginConfigwith context-boundedInjectcalls and tracer-default fallbacks #6341)x-bf-session-idis absent (autodetect harness level session ids #6333)/inspectbypass the virtual key provider and model allowlists while budgets and rate limits still apply (adds path for skipping auth #6124, add support for skip provider checks #6479)Accept-Encodingis filtered to decodable codecs (gzip, deflate, brotli, zstd; gzip and identity for streams) and chained content encodings are decoded (feat: filterAccept-Encodingto supported codecs on passthrough headers and expandCheckAndDecodeBodyto handle deflate, brotli, zstd, and chained encodings #6360)routingplugin that runs after governance so rules evaluate on the fully stamped context; endpoints moved to/api/routing/rulesand/api/routing/complexity-analyzer-configwith deprecated/api/governance/*aliases; complexity routing now reads the text of mixed text+image turns (refactor: extract routing rules and complexity router into a dedicated routing plugin #6144, refactor: movePublishRoutingAllowlistandLoadBalanceProviderfrom governance'sPreRequestHookinto the routing plugin so both run after rule evaluation on the post-rule model #6145, refactor: extract routing rules and complexity analyzer endpoints into dedicatedRoutingHandlerunder/api/routing/*with backwards-compatible/api/governance/*aliases #6146, docs: migrate routing rules and complexity analyzer endpoints from/api/governance/*to/api/routing/*with deprecated legacy aliases #6147, complexity: supports image+text input #6253)DimensionScopeceiling to bound grouping dimension values per caller #6262)config.jsonas well as the UI, with a documented shared vs per-identity token lookup contract,oauth_config.resource(RFC 8707), VK/Users filters on the OAuth Grants and MCP Auth Sessions sidebars and one shared create/install client form;token_exchangegainsuse_idp_credentialsto reuse SSO login app credentials for providers such as Microsoft Entra ID (client_idbecomes optional) and combinesoffline_accesswith<audience>/.defaultfor Entra OBO; shared-OAuth clients showneeds_reauthwhen their token row is invalidated,Reauthorizeis limited to shared clients, the OAuth flow claim is atomic against concurrent reauth, stored scopes survive a decode failure, and credential caches propagate cancellation and version their entries (feat: adduse_idp_credentialsto token-exchange config, allowing SSO login app credentials to be reused for providers like Microsoft Entra ID that require it #6068, docs: adduse_idp_credentialstotoken_exchangeconfig, makeclient_idoptional when set, and document Entra ID requirement with updated prerequisites and gotchas #6069, fix: combineoffline_accesswith<audience>/.defaultfor Entra OBO instead of replacing it, and document the.default-scope replacement behavior as a new gotcha #6078, refactor: extract shared MCP client form fields, validation, and payload builder intomcpClientFormFieldsand reuse across create and library install sheets #6411, chore: fix indentation and replaceoverflow-x-hiddenwithoverflow-hidden!inmcpClientSheet#6428, feat: projectneeds_reauthonto shared-OAuth clients when their token row is invalidated via batchGetSharedOauthTokensByConfigIDslookup #6429, fix: unifygetAuthScopeDisplaywithauthScopeOffrom form fields, usingMCPAuthTypeinstead ofstring#6504)needs_session_stickinessis pinned acrossconfig.jsonreconciliation; updating static headers on a sticky client pre-flight verifies the new credential and swaps it onto the live connection, per-call shared-credential clients refresh tools synchronously, and a failed enable parks the client atDisabledso it can be retried; the globaltool_sync_intervalhot-reloads and re-times running checkers; state badges render with spaces and thedisconnectedfilter bucket is nowunstable(fix: make tool_sync_interval to be in minutes #6409, feat: pre-flight verify and live-reconnect sticky MCP HTTP client on static header update, withmcpHeadersEqualdiff guard and tests #6430, fix: park failedEnableClientatDisabledinstead ofUnstableto prevent retry wedging, addErrMCPEnableConnectFailed, and guardisEnableableon both state and config #6431, fix: refresh tools synchronously on per-call shared-credential client credential update instead of returningErrMCPReconnectNotApplicable, with disabled-client guard and per-user sentinel preserved #6483, fix: remove "disable sync" semantic fromtool_sync_interval(negative now rejected), addUpdateMCPToolSyncIntervalfor hot-reload, restart per-call checker onEnableClient/sticky→per-call flip, and re-time running checkers on global or per-client in #6502)mcp_library_sync_interval: 0disables catalog sync andfile://URLs load the MCP server library from disk (adds airgapped flow for mcp catalog #6195)config.schema.json, Helm values and dashboard entries for the Splunk HEC observability connector (feat: add config json and helm charts support for Splunk #6296, feat: add fallback UI and logos for Splunk connector #6091, docs: Add Splunk connector docs #6099)bifrost.cluster.type: brokerwith broker address, port and TLS settings alongside the existing mesh transport (feat: add broker clustering support to Helm chart #6398)http2_ping_interval_in_seconds(fix: default http2 timeout interval in ui #6228)web_search_call,code_interpreter_calland similar Responses items render their full payload in the log detail view (fix: update server tool calls in log details ui #6475)toolCall/toolResponseparts surface asweb_search_callitems with their own call ID and queries, unmapped tool types are preserved on the native round-trip, and eachthoughtSignatureappears exactly once on replay (feat(gemini): support server-side toolCall/toolResponse parts with thoughtSignature round-trip fidelity #6071)sidekiqbackground-job table, store methods, and runner with recovery and reaper; cost recalculation migrated to a durable, resumable and cancellable job with polling instead of SSE (feat: make log recalculation task cancellable #5800, feat: make log recalculation task cancellable backend #5801)roots_onlyfilter with child aggregates, and the model catalog persists tab, search and provider in the URL (feat: add grouped view to logs table with expandable fallback chains #5522, feat: addroots_onlyfilter to collapse fallback chains with child aggregates #5737, feat(ui): persist model catalog tab, search and provider in the URL #6059)matview_refresh_intervalaccepts"off"to disable logstore matview maintenance entirely (thanks @jeremym-tanium!) (feat: support matview_refresh_interval "off" to disable logstore matview maintenance #5693)ShellRewriterhook for pre-hydration HTML rewriting (feat: addShellRewriterhook to UI handler for pre-hydration HTML rewriting #5807)fiscalQuarterNotehelper and surface fiscal year start in budget UI labels #6115, feat: redesignQuarterStartSelectlayout to horizontal label/preview with right-aligned select #6116)/v1/sound-generation(thanks @SecretSun!)project_idto Bedrock and Bedrock Mantle key configs with per-alias overrides for Bedrock, Bedrock Mantle, and Vertex, plus UI supportconfig.schema.jsonwith declarative channels and CEL-based rules, Helm chart support, and enterprise fallback pagesshouldSweepgate on the OAuth2 sweep workerschema_url/BIFROST_SCHEMA_URLfor mirrored schema locations in isolated deploymentsbifrost.alerting, audit-log object storage,postgresql.external.portstring support, andbifrost.mcp.toolGroups[*].id🐞 Fixed
response_formatJSON schemas are forwarded byte-for-byte to OpenAI, Anthropic, Bedrock, Gemini and Cohere so the model generates fields in the caller's declared order instead of a re-sorted one (fixes params sequence in chat completions #6235)thinkingblocks instead ofredacted_thinking(reasoning fix #6292)bedrock/Prefix - Content-lesstool_resultblocks are kept, interleaved block order is preserved,incompletemaps toerroron Converse, and pending reasoning is consumed by its owning item, so multi-turn tool use no longer wedges (fix: restructuring of replayed thinking blocks #6346)extra_fieldsis echoed on/anthropic/v1/messages(gemini inmessage system role bug fix #6363)tool_useandtool_result(fix: bedrock tool use ids #6300)errorandresponse.failedevents inside an already-open HTTP 200 SSE stream are surfaced as errors with their nested type, code and message (thanks @dani29!) (Preserve Azure Responses stream errors after HTTP 200 #6302)max_tokensis preserved for OpenCode-compatible chat endpoints (thanks @Alex-wangyang!) ([fix]: core/providers/openai - preserve max_tokens for OpenCode endpoints #6458)[DONE]marker, andstream_options.include_usagedefaults on its chat streaming path, so streamed calls stop reporting zero tokens and zero cost (thanks @elliottrabac!) ([fix]: HuggingFace provider - default stream_options.include_usage on chat streaming #6478)nameno longer clears it, and already-exists errors keep their constraint detail (thanks @cpsc!) (fix: preserve provider key name on update, keep constraint detail in already-exists errors #6417)ProviderSendsDoneMarkerso streams end afterfinish_reason(fix: register bedrock mantle in ProviderSendsDoneMarker #6021)gs://URIs go to Gemini/Gemma asfileData.fileUriand are read from Cloud Storage for Claude-on-Vertex,s3://references go to Bedrock Converse ass3Location, Bedrock rerank synthesizes the foundation-model ARN from a bare model ID, OpenAI file blocks keepfile_url, non-http schemes pass through on the OpenAI and native-Anthropic paths, and Gemini always emits a candidate with its finish reason and drops payload-free parts (file/image embedding flow fixes #6239)togetherto the datasheet identity and prices configured aliases through their target model (thanks @dani29!) (fix: resolve Together and alias pricing in management catalog #6257, fix: usestrings.Containsfor "together" provider normalization to handle substrings #6320)tool_sync_interval; the value is a whole number of minutes, negative values are rejected instead of silently disabling sync, and re-enabling a per-call client restarts its discovery cycle (fix: make tool_sync_interval to be in minutes #6409, fix: remove "disable sync" semantic fromtool_sync_interval(negative now rejected), addUpdateMCPToolSyncIntervalfor hot-reload, restart per-call checker onEnableClient/sticky→per-call flip, and re-time running checkers on global or per-client in #6502)SetClientToolsreplaces the in-memory tool map instead of merging, so tools removed upstream leave memory once the database has dropped them (fix: replacemaps.Copymerge with wholesale replacement inSetClientToolsandUpdateClientCredentialsto prevent removed tools from persisting in memory #6484)OnConnectionLoston SSE MCP clients is gated on connection identity so a stale connection cannot tear down its replacementAuthorization,x-api-key, Cloudflare Access and AWS ALB OIDC headers are redacted before export to every observability backend (fix: redact identity-aware-proxy headers before connector export #6371)includeServerSideToolInvocations, and search localization viaretrievalConfig.latLngis preserved (fix(gemini): allow Vertex AI to send mixed tools without the server-side invocations flag #6066)content_filterandguardrail_intervenedstop reasons map toincompletestatus with acontent_filterreasonencrypted_contentbefore retrying a rejected request also covers/v1/responses/compactand count-tokens requests, and recognizes Anthropic'sredacted_thinkingrejection (encrypted content patch for encrytped content could not be verified #6041, fix: add anthropic error branch on stripping on encrypted content #5960)from_memoryvirtual key reads are blocked for DAC-scoped callersreasoning_effort: "minimal"is preserved for GPT-5-family OpenAI models instead of being downgraded tolow(thanks @jitokim!) (fix: preserve minimal reasoning effort for GPT-5-family OpenAI models #6046)MAX_TOKENSinstead ofOTHER(thanks @AdityaPainuli!) ([fix]: Gemini - truncated responses report finishReason MAX_TOKENS instead of OTHER #5979)null(thanks @AdityaPainuli!) ([fix]: Chat streaming - continuation deltas no longer materialize absent function name as null #5966)type: "file"are accepted (fix: bedrock files handling in inference #5947)response.createinput (fix: guard response.create input #6050)api-keyheader now attribute to the virtual key (thanks @eyeveil!)/v1/reranknow works with custom OpenAI-compatible providers (thanks @eyeveil!)allowed_modelsrows that broke admin provider updates (thanks @eyeveil!)tool_use/tool_resultids to Anthropic's charset (thanks @Shaik-Sirajuddin!)POST /v1/realtime/client_secrets(thanks @Shaik-Sirajuddin!)diarized_jsonsegments and ElevenLabs speaker passthrough (thanks @Shaik-Sirajuddin!)redacted_thinkingblocks on the Responses surface (thanks @fus3r!)finish_reasonin the accumulated streaming response (thanks @fus3r!)web_search_callitems keep emitting (thanks @fus3r!)max_output_tokenstruncation on the Responses API (thanks @jeremym-tanium!)reasoning_configon cross-provider translation so fallbacks keep extended thinking (thanks @Purvi09!)tool_searchon the Responses path (thanks @ws4charlie!)rolefrom non-message input items (thanks @nettee!) and serialize compaction requestinputcorrectly (thanks @mcclurmc!)additional_toolsmessage type support, preserving nested tool types on/v1/responses#t=temp-token fragment (thanks @MarcusPeng!)SecretVarJSON withref/env_varfields even whenvalueis absentpreferred_usernamefallback, filter provider-level keys from the prompt manager selector (thanks @rlex!), skip password validation for redacted credentials, and improveModelMultiselectempty and error statesExtraContenttoChatStreamResponseChoiceDelta(thanks @nghodkicisco!)/api/devicesbypassing auth via the/api/devprefixX-Amzn-Errortype) on non-streaming Bedrock error responses instead of dropping it🔧 Maintenance
OrderedMapJSON writer, bulk span attribute writes with cached span pointers, reusable worker delivery timers, retained span attribute maps, generation-stamped memoization ofGetProvidersForModelandGetModelsForProvidervia the newgencachepackage, sonic-based JSON responses, and a plugin-log existence check before draining (perf: cache serialized json for shared mcp tools #6242, perf: optimize ordered map's marshalJson #6241, perf: rewrites individual span writes to bulk and reuses delivery timer #5956, perf: alloc reductions, span attribute map reuse, and logging context read on final chunk #5957, refactor: fixes some minor unnecessary allocations majorly in core #5657, perf: carry the *Span pointer on span handles #6387, refactor: updates modelcatalog to memoise provider configs recomputing for allowed models / providers #5641, refactor: add gencache package and wire in GetModelsForProvider to gencache #6224, perf: update json encode usage with sonic marshal #6268, fix: adds plugin logs existence check before draining #6211)openai_configreferenced from every provider config schema (dependabot fixes #6040, mod fix #5864, deendabot fixes #6267, dependabot fixes #6305, chore: addopenai_configref to all transport provider config schemas #6275)/v1/images/editsand/v1/videos, batch and pricing-override lifecycle harness cases, an Anthropicmessage_startusage regression test, LangChain rerank and embedding integration tests, and e2e fixes for dashboard auth, budget reset and MCP state (fix: adds gh workflow to spin up vllm instances for testing vllm #5541, tests: harness for runware #6303, tests: fixes runware harness tests #6319, chore: adds missing batch / batch pricing tests in harness #6299, tests: anthropic message start harness test #6327, tests: integration for rerank langchain #6432, fix: close leaked postgres pools and stale test date in logstore tests #6351)prerelease3deployments, v2.0.0 availability callouts, routing API namespace docs, Bedrock application inference profiles, Splunk connector docs, config.schema.json and Datadog env var reference fixes, and Discord badge fixes (thanks @Swpn0neel!) (docs(migration): consolidate governance API namespace guide into v2.0.0 migration page #6332, docs: add v1.5.x downgrade guide for deployments that ranv2.0.0-prerelease3withent_split_oidc_session_auth_token_columnmigration #6374, docs: add v2.0.0 availability callouts across features, providers, and API references #6420, docs: migrate routing rules and complexity analyzer endpoints from/api/governance/*to/api/routing/*with deprecated legacy aliases #6147, docs: bedrock application inference profiles #6203, docs: Add Splunk connector docs #6099, chore: doc fixes for config.schema.json #5938, chore: update helm-charts doc for env var reference fixes for Datadog #6019, docs: fix Discord badge rendering on Glama #6425, fix: use Glama-compatible Discord badge #6448)claimsSyncModefor all SCIM providers and optional OktaapiToken#6249)🗄️ Database Migrations
All migrations below are new relative to v1.6.11. Deployments on an older v1.6.x release should also review the intermediate v1.6.x changelogs.
configstore:
pending_oauth_config_jsontoconfig_mcp_clients. Reversible: drops the added column.oauth_tokensandoauth_user_tokensintomcp_oauth_tokens. Non-reversible: rollback deliberately leavesmcp_oauth_tokensin place, because every OAuth read and write targets it from this migration onward and dropping it would destroy any token created or refreshed since, forcing every holder to re-authorize.mcp_oauth_flowsto track in-flight OAuth flows. Reversible: drops the new table.expires_atfrom the OAuth config table now that they live onmcp_oauth_flows. Non-reversible: forward-only, the dropped values were per-flow ephemeral and re-adding empty columns would restore nothing.token_id. Non-reversible: forward-only, it was a pure FK shortcut now reachable via(oauth_config_id, auth_mode).mcp_oauth_tokensandmcp_per_user_header_credentials. Reversible: drops both indexes.token_exchange_jsontoconfig_mcp_clients. Reversible: drops the added column.needs_session_stickinesstoconfig_mcp_clients. Reversible: drops the added column.cost_per_requestto model pricing. Reversible: drops the added column.notificationstable for the dashboard notification center. Reversible: drops the table.batch_jobswith a unique(provider, batch_id)identity index, a sweeper scan index and a runner-id index. Reversible: drops the table.output_cost_per_image_above_{4,8,16,32,64}_megapixelscolumns to model pricing. Reversible: drops the added columns.input_cost_per_queryto model pricing for rerank. Reversible: drops the added column.*_ultrafasttoken rate columns to model pricing. Reversible: drops the added columns.user_id,team_id,customer_idandsource_log_idtobatch_jobsplus auser_idindex. Reversible: drops the index and the four columns.logstore:
- logs_add_guardrail_debug_column - Adds
- mcp_tool_logs_add_redaction_mapping_column - Adds the redaction mapping column to MCP tool logs. Non-reversible: rollback is a no-op because dropping the column would permanently destroy reveal data for already-redacted MCP logs.
- logs_add_user_agent_column - Adds user agent and app columns, their indexes, and a
- mcp_tool_logs_add_user_agent_column - Adds user agent and app columns plus indexes to MCP tool logs. Reversible: drops both indexes and the
- logs_recreate_matviews_with_app_column - Recreates the log materialized views to include the user agent and app columns. Rollback is a no-op because
- mcp_tool_logs_add_endpoint_columns - Adds
- mcp_tool_logs_add_plugin_logs_column - Adds
- logs_add_video_edit_input_column - Adds
- logs_add_upstream_and_overhead_latency_columns - Adds
- logs_add_batch_debug_column - Adds
- logs_add_cost_breakdown_columns - Adds
- logs_recreate_matviews_with_cost_breakdown - Marks the hourly matview for rebuild with the cost split columns;
- logs_add_overhead_breakdown_column - Adds
**High-throughput deployments: run the logstore migrations during a low-activity window.**guardrail_debugto logs. Reversible: drops the added column.UserAgentMappingtable. Reversible: drops the indexes and the mapping table.appcolumn.ensureMatViewsrecreates them on next startup.source,decision,app_keyanddevice_idto MCP tool logs. Reversible: drops all four columns.plugin_logsto MCP tool logs. Reversible: drops the added column.video_edit_inputto logs. Reversible: drops the added column.upstream_latencyandoverhead_latencyto logs. Reversible: drops both columns.batch_debugto logs. Reversible: drops the added column.input_cost,output_costandadditional_costto logs. Reversible: drops the three columns.repairMatViewShapesdrops and recreatesmv_logs_hourlyon the next startup. Rollback is a no-op becauseensureMatViewsrecreates it on next startup.overhead_breakdownto logs. Reversible: drops the added column.Every logstore migration above alters
`merge_oauth_token_tables`, `drop_oauth_config_pkce_columns` and `drop_oauth_config_token_id_column` transform or remove existing OAuth state and cannot be rolled back. Take a database backup before upgrading, and do not roll the binary back past this release once the migration has run.logsormcp_tool_logs, the two highest-insert tables in Bifrost, and several also build indexes on them. On a busy instance the index builds hold locks that block concurrent log inserts for the duration of the build, and the matview recreations rebuild against the full table. Schedule the upgrade for a low-traffic period, or expect elevated log-write latency and possible request-path backpressure while the migrations run.🐙 Closed GitHub Issues
type:"file"- "The PDF specified was not valid"