[fix]: fill Azure GPT-5.6 cache-creation rates at 1.25x input - #6179
Open
Edward-Upton wants to merge 311 commits into
Open
Edward-Upton wants to merge 311 commits into
Edward-Upton wants to merge 311 commits into
Conversation
…ser auth types and the selected Type filter
…d-OAuth reconnect/verify errors
…ient types via a hash-gated core callback
…n verify-headers/verify-exchange to close a stale-read race
… connection-state naming throughout
…re-completion 409 on complete-oauth
* fix(ui): skip password validation for redacted credential * fix(ui): validate newly entered redaction sentinels
…aggregates (maximhq#5737) ## Summary Adds a `roots_only` filter to the log search API that collapses fallback chains into a single root row. When enabled, any log whose `parent_request_id` points at an actual log row is hidden from the list view, leaving only the chain's root visible. Each root is annotated with child aggregates (`child_count`, `children_cost`, `children_tokens`) so the UI can render an expandable row summarizing the full chain without additional queries. ## Changes - Added `RootsOnly bool` to `SearchFilters` and wired it to the `roots_only` query parameter in the HTTP handler via `strconv.ParseBool`. - In `applyFilters`, when `RootsOnly` is set and no `ParentRequestID` filter is active, a subquery filters out rows whose `parent_request_id` matches an existing log ID. ClickHouse uses an uncorrelated `NOT IN` subquery (correlated subqueries are unsupported); all other dialects use `NOT EXISTS`. - After a `roots_only` search returns a page, `attachChildAggregates` runs a single grouped query over the page's IDs to populate `ChildCount`, `ChildrenCost`, and `ChildrenTokens` on each root. These fields are transient (`gorm:"-"`) and never stored. - `ParentRequestID` takes precedence over `RootsOnly` — when a parent filter is active the full child list is returned, matching the expand-on-click behaviour. - `canUseMatViewFilters` now returns `false` when `RootsOnly` is set, since the per-row existence predicate cannot be expressed in the hourly materialized view count path. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./framework/logstore/... -run TestSearchLogsRootsOnly go test ./framework/logstore/... -run TestCanUseMatViewFiltersExcludesRootsOnly go test ./... ``` **HTTP:** ```sh GET /logs?roots_only=true ``` Expected: only root rows returned, each carrying `child_count`, `children_cost`, and `children_tokens` where children exist. ```sh GET /logs?roots_only=true&parent_request_id=<id> ``` Expected: `roots_only` is ignored; all children of the given parent are returned. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations The `roots_only` subquery operates only on the `logs` table within the tenant-scoped DB connection. No new data is exposed; child rows remain accessible via `GetSessionLogs` using the root's ID. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
* docs: add Bedrock runbooks for Claude Code and Codex * docs: use Bedrock deployment mappings in runbooks * remove unecessary warning * replace static json with UI image * docs: add Edge setup paths to Bedrock runbooks * docs: clarify Claude Code model validation
## Summary Improves the documentation for Datadog integration configuration fields to clarify that `service_name`, `ml_app`, `env`, and `version` all support the `env.VAR_NAME` prefix for environment variable substitution at runtime. ## Changes - Added descriptions to previously undocumented `service_name`, `env`, and `version` fields in the Helm chart schema, explicitly noting `env.VAR_NAME` substitution support with examples - Updated `ml_app` description in the Helm chart schema to mention `env.VAR_NAME` support - Updated `service_name`, `ml_app`, `env`, and `version` descriptions in the transport config schema to note the `env.` prefix capability - Added inline comments in `values.yaml` for `service_name`, `env`, `version`, and `ml_app` to surface the `env.VAR_NAME` support directly in the default config ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test No behavioral changes. Validate that the schema descriptions render correctly by inspecting the JSON schema files and confirming the Helm chart lints cleanly. ```sh helm lint helm-charts/bifrost ``` ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. These are documentation-only changes to schema descriptions and YAML comments. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds end-to-end observability for guardrail judge calls — the internal LLM invocations made by the enterprise guardrails plugin to evaluate rules. Previously, these calls were invisible: their token spend was untracked, their outcomes were not logged, and their cost was not reflected in billing. This PR surfaces that data through a new `guardrail_debug` field on responses, log entries, and the UI. ## Changes - Introduced `BifrostGuardrailDebug` and `BifrostGuardrailJudgeCall` schema types in a new `guardraildebug.go` file, with typed context helpers (`GuardrailDebugFromContext`, `SetGuardrailDebugOnContext`, `AppendGuardrailJudgeCallOnContext`) that enforce copy-on-read isolation so callers cannot mutate context state. - Added `BifrostContextKeyGuardrailDebug` context key and `GuardrailDebug *BifrostGuardrailDebug` to `BifrostResponseExtraFields`, propagated through all response conversion paths (`ToTextCompletionResponse`, `ToBifrostTextCompletionResponse`) and all streaming accumulators and chunk types. - Extended `StreamAccumulatorResult` and `AccumulatedData` with `GuardrailDebug` so streaming pipelines carry the field through to the final assembled response. - Added `CalculateGuardrailCost` to the model catalog datasheet and exposed it via `ModelCatalog`. `CalculateCost` now adds judge-call cost on top of the main request cost (including cache-hit paths). Judge cost is attributed to the judge's own provider/model, preserving virtual-key attribution. - Added a `guardrail_debug` column to the logstore `Log` table via a new migration, with full serialize/deserialize, payload extraction, merge, and clear support. - Updated the logging plugin's `PostLLMHook` to read guardrail debug from context (covering input-block cases where no provider response exists) and from the response, write it to the log entry, and apply guardrail cost to `entry.Cost` — including for error paths and streaming. - Updated `calculateCostForLog` to treat a non-nil `guardrailDebug` as sufficient to proceed with cost calculation, so input-blocked requests are billed correctly. - Added `GuardrailDebug` and `GuardrailJudgeCall` TypeScript types and rendered a "Guardrail Details" section in the log detail view showing rule, phase, action (Blocked/Allowed badge), guardrail name and provider, judge provider and model, token counts, and reason. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go version go test ./core/schemas/... ./framework/logstore/... ./framework/modelcatalog/... ./framework/streaming/... ./plugins/logging/... # UI cd ui pnpm i || npm i pnpm build || npm run build ``` To validate end-to-end: 1. Send a request through a guardrail rule that triggers a judge call. 2. Confirm the response `extra_fields.guardrail_debug.judge_calls` is populated with provider, model, and token counts. 3. Open the log detail view and verify the "Guardrail Details" section appears with correct phase, action badge, and token counts. 4. Confirm `cost` on the log entry reflects both the main request and the judge call spend. 5. For an input-blocked request (no provider response), confirm `guardrail_debug` and cost are still written to the log. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations `guardrail_debug` is written to the log store and returned in API responses. It does not contain prompt content — only metadata (rule name, provider, model, token counts, action, reason). The `reason` field may contain guardrail-generated explanations; ensure content logging policies are applied consistently if reason strings are considered sensitive. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Clarifies that passthrough endpoints are not credential proxies — Bifrost always selects and injects its own provider key, and any provider credentials supplied by the caller are stripped before the request is forwarded upstream. ## Changes - Added a `Warning` callout making it explicit that callers must authenticate with a Bifrost virtual key, not a provider API key, and that provider keys in the request are never forwarded. - Added a `Note` callout explaining that Claude Code OAuth tokens (`sk-ant-oat…`) are handled on the regular `/anthropic` route, not via passthrough. - Updated the "How it works" numbered steps to explicitly describe Bifrost's key selection and credential-stripping behavior. - Updated curl examples for Anthropic, GenAI (Gemini), and Vertex passthrough to use `<YOUR-BIFROST-VIRTUAL-KEY>` instead of raw provider API key placeholders. - Replaced the Azure-specific auth note in the Notes section with a provider-agnostic statement covering all passthrough endpoints (`authorization`, `api-key`, `x-api-key`, `x-goog-api-key`). - Added a note about the `direct API keys` exception, requiring both `allow_direct_keys` server-side and `x-bf-direct-key: true` per-request. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the updated passthrough documentation and verify: - The `Warning` and `Note` callouts render correctly. - curl examples reference `<YOUR-BIFROST-VIRTUAL-KEY>` consistently across Anthropic, GenAI, and Vertex sections. - The Notes section accurately reflects the behavior for all passthrough endpoints, not just Azure. ## Breaking changes - [ ] Yes - [x] No ## Security considerations This change reinforces that provider API keys should never be sent by callers on passthrough requests — Bifrost strips them regardless. The documentation now makes this behavior explicit, reducing the risk of users inadvertently exposing provider credentials or expecting them to be forwarded upstream. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes maximhq#123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…OTEL (maximhq#5939) ## Summary Adds a `traces_enabled` flag to OTel profiles, allowing a profile to operate in a metrics-only mode without requiring a `collector_url`. Previously, every enabled profile required a collector URL because traces were always on. This change decouples trace and metrics export so each can be independently toggled. ## Changes - Added `traces_enabled` boolean field to `Profile` with a default of `true` so existing configs continue exporting spans without modification. - `collector_url` is now only required when `traces_enabled` is `true`; a metrics-only profile (`traces_enabled: false`, `metrics_enabled: true`) no longer needs one. - The trace client is only built when `traces_enabled` is `true`; `Inject` already skips a nil client. - Protocol validation is skipped entirely when both traces and metrics are disabled (no-op profile). - The JSON schema's `collector_url` requirement condition was updated to account for `traces_enabled: false`, and `protocol` was added to the `metrics_enabled` requirement. - The `profileForStorage` struct and `MarshalForStorage` now persist `traces_enabled` so the flag survives storage round-trips. - The OTel profile form in the UI was reorganized into **Traces** and **Metrics** tabs. Trace-specific fields (collector URL, format, export timeout, request headers, content logging toggles) are nested under the Traces tab and hidden when `traces_enabled` is off. The Protocol selector was promoted to a shared connection setting above the tabs since both exporters use it. - Tab headers show a destructive badge when the tab contains a validation error, and the profile header shows a "Metrics only" badge when traces are disabled but metrics are enabled. - The E2E helper for enabling metrics export now clicks the Metrics tab before interacting with the toggle, since it is no longer the default active tab. - Added unit tests covering: default `TracesEnabled` behavior, metrics-only profile initialization, traces-enabled profile requiring `collector_url`, both-disabled no-op profile, and storage round-trip fidelity. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Plugin unit tests go test ./plugins/otel/... # UI cd ui pnpm i pnpm build ``` **Metrics-only profile config example:** ```json { "profiles": [ { "traces_enabled": false, "protocol": "http", "metrics_enabled": true, "metrics_endpoint": "otel-collector:4318" } ] } ``` Expected: profile initializes without error, no trace client is built, metrics exporter is active. **Existing traces-only config (no `traces_enabled` field):** should continue to work unchanged, defaulting `traces_enabled` to `true`. ## Breaking changes - [ ] Yes - [x] No Existing configs omitting `traces_enabled` default to `true` and behave identically to before. ## Security considerations No new secrets or auth surfaces introduced. The `collector_url` secret var handling is unchanged; it is simply no longer required when traces are disabled. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…lector (maximhq#5940) ## Summary Adds support for per-signal HTTP headers in OTel profiles, allowing separate headers to be sent exclusively to the trace endpoint or the metrics endpoint, in addition to the existing shared `headers` field. This is particularly useful when a metrics collector requires a signal-specific header (e.g. a Databricks table name) that should not be forwarded to the trace endpoint. ## Changes - Added `trace_headers` and `metrics_headers` fields to the `Profile` struct and `profileForStorage` struct, alongside the existing `headers` field. - `headers` continues to apply to both endpoints. `trace_headers` and `metrics_headers` are overlaid on top of the common headers at build time, with per-signal keys winning on collision. - Introduced `mergedResolvedHeaders` to merge common and per-signal header maps and resolve `env.VAR_NAME` references without mutating the inputs. - Extracted `redactHeaderMap` to eliminate duplicated redaction logic and applied it to all three header maps in `Redacted()`. - Updated the JSON schema (`config.schema.json`) with descriptions for all three header fields. - Updated the UI form to render separate `HeadersTable` inputs for common, trace-only, and metrics-only headers, each with descriptive labels and `FormDescription` text. - Updated the Zod schema and form serialization to include `trace_headers` and `metrics_headers`. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go test ./plugins/otel/... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Configure an OTel profile with all three header fields: ```json { "headers": { "Authorization": "env.OTEL_TOKEN" }, "trace_headers": { "X-Trace-Only": "trace-value" }, "metrics_headers": { "X-Databricks-Table": "my_table" } } ``` Verify that: - Trace requests include `Authorization` and `X-Trace-Only` but not `X-Databricks-Table`. - Metrics requests include `Authorization` and `X-Databricks-Table` but not `X-Trace-Only`. - `env.OTEL_TOKEN` is resolved from the environment on both endpoints. - Redacted config masks literal header values and preserves `env.` references across all three maps. ## Breaking changes - [ ] Yes - [x] No ## Security considerations All three header maps (`headers`, `trace_headers`, `metrics_headers`) are subject to the same redaction logic in `Redacted()`. Literal header values are masked and `env.` references are preserved as-is, consistent with prior behavior. No new secret storage paths are introduced. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…aximhq#5941) Adds `traces_enabled`, `trace_headers`, and `metrics_headers` fields to the OTEL plugin configuration (both single-profile and multi-profile shapes), enabling metrics-only OTEL profiles and per-signal header overrides. - Added `traces_enabled` boolean to OTEL config. When set to `false`, trace export is skipped and `collector_url` / `trace_type` are no longer required, allowing a metrics-only profile to be configured without a trace collector. - Added `trace_headers` and `metrics_headers` maps to OTEL config. The existing `headers` field continues to apply to both endpoints; `trace_headers` and `metrics_headers` are overlaid on top per-signal, with the more specific key winning on conflict. This supports cases where a collector requires a signal-specific header (e.g. a Databricks table name on the metrics endpoint only). - Updated validation logic in `_helpers.tpl` so that `collector_url` and `trace_type` are only required when `traces_enabled` is `true`, and `protocol` is only required when at least one of traces or metrics is enabled. - Updated `values.schema.json` conditional validation (`allOf`/`if`/`then`) to reflect the same rules: `collector_url`, `trace_type`, and `protocol` are gated on both `enabled` and `traces_enabled` not being `false`; `metrics_endpoint` and `protocol` are required together when `metrics_enabled` is `true`. - Updated `values.yaml` comments and `README.md` changelog to document the new fields. - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs Deploy the Helm chart with a metrics-only OTEL profile and verify that no trace collector URL is required: ```yaml bifrost: plugins: otel: enabled: true config: traces_enabled: false metrics_enabled: true metrics_endpoint: "http://otel-collector:4318/v1/metrics" protocol: "http" metrics_headers: x-databricks-table: "my_table" ``` ```sh helm template . -f values.yaml | grep -A 30 "otel" helm lint . ``` Verify that omitting `collector_url` with `traces_enabled: false` passes linting, and that omitting it with `traces_enabled: true` (default) still fails with the appropriate error message. N/A - [ ] Yes - [x] No N/A `trace_headers` and `metrics_headers` support the `env.VAR_NAME` prefix for injecting secrets from environment variables, consistent with the existing `headers` field. No new secret handling mechanisms are introduced. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…cs list (maximhq#5942) ## Summary Documents two new OTel plugin capabilities: per-signal headers (`trace_headers` and `metrics_headers`) and a `traces_enabled` flag that enables a metrics-only mode where `collector_url` is not required. ## Changes - Added `traces_enabled` field documentation — when set to `false`, the trace client is never built and `collector_url`/`trace_type` become optional, enabling metrics-only profiles - Added `trace_headers` and `metrics_headers` fields — these are overlaid on top of the shared `headers` field for their respective endpoints, with per-signal values winning on key collision - Clarified that `headers` is sent to both trace and metrics endpoints, and that `protocol` is shared between both signals - Added a "Per-signal headers" section with a worked example showing `Authorization` shared via `headers` and `X-Databricks-Table` scoped to the metrics endpoint via `metrics_headers` - Added a "Metrics-only mode" section with a full JSON configuration example - Expanded the pushed metrics table to include `bifrost_cache_read_input_tokens_total`, `bifrost_cache_write_input_tokens_total`, `bifrost_cache_write_input_tokens_5m_total`, `bifrost_cache_write_input_tokens_1h_total`, `bifrost_request_retries`, and `mcp.client.operation.duration` - Added a note clarifying that an unreachable metrics endpoint never blocks Bifrost startup - Updated env-var substitution docs to include `trace_headers` and `metrics_headers` - Applied the same changes to both the `config-json` and Helm plugin reference pages ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Review the rendered documentation for the OTel plugin pages: - `docs/features/observability/otel.mdx` - `docs/deployment-guides/config-json/plugins.mdx` - `docs/deployment-guides/helm/plugins.mdx` Verify that: 1. The `traces_enabled: false` example produces a valid metrics-only config with no `collector_url` 2. The per-signal headers example correctly shows `Authorization` on both endpoints and `X-Databricks-Table` only on the metrics endpoint 3. All new metrics in the pushed metrics table are accurately described ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations `trace_headers` and `metrics_headers` support the `env.` prefix for environment variable substitution, consistent with the existing `headers` field. No new secrets are stored in configuration. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Extends video logging and the log detail UI to fully support delete, list, download, and generation/remix/retrieve response types. Previously, delete responses were not routed to any log column, and the video detail view lacked support for delete output, base64-encoded video, and several generation metadata fields. ## Changes - In `applyNonStreamingOutputToEntry`, added routing for `VideoGenerationResponse`, `VideoDownloadResponse`, `VideoListResponse`, and `VideoDeleteResponse` into their respective log entry fields. `VideoGenerationResponse` is shared by generation, remix, and retrieve operations, so the request type is used as the discriminator to separate retrieve into its own column. - Added `video_delete_output` to the `videoOutput` expression in `logDetailView.tsx` so delete responses trigger the video detail panel. - Updated `VideoView` to handle `BifrostVideoDeleteOutput` as a distinct output type, rendering the video ID and deleted status. - Replaced the ad-hoc `requestType.toLowerCase().includes(...)` label logic with a lookup against `RequestTypeLabels`. - Added `getVideoSrc` to resolve a video source from either a URL or a base64 payload, and updated the video rendering loop to support multiple videos and base64-encoded content. - Added display of additional generation metadata fields: duration (`seconds`), size, and `remixed_from_video_id`. - Added `CopyableId` to video ID fields in the download and generation output sections. - Added the `ContentFilterInfo` type and `content_filter` field to `BifrostVideoGenerationOutput`. - Changed `seconds` from `number` to `string` on both `VideoObject` and `BifrostVideoGenerationOutput` to match the API shape. - Added tests covering all video response types (generation, remix, retrieve, download, list, delete) and verifying that content logging disabled suppresses video output. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Core/Transports go test ./plugins/logging/... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` Trigger video generation, remix, retrieve, download, list, and delete requests and verify each response appears in the correct log column in the UI. Confirm that with content logging disabled, no video output fields are populated. ## Breaking changes - [ ] Yes - [x] No ## Security considerations No new auth, secrets, or PII surface area introduced. Video content is explicitly noted as not stored in logs for download responses. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Bedrock was rejecting documents with "The PDF specified was not valid" because the document format was always resolved to `"pdf"` regardless of the actual file type. Standard OpenAI clients encode the MIME type inside the data URL (e.g. `data:application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;base64,...`) rather than in the `file_type` field, which was the only source previously consulted. This PR fixes format resolution for both the Chat and Responses paths, unifies the mapping logic, and corrects several related data URL parsing defects. Fixes maximhq#5472 - Extracted a shared `bedrockDocumentFormat` helper in `utils.go` that maps MIME types and bare file extensions to Bedrock Converse document format strings, replacing two duplicated inline switch blocks that were missing most MIME types. - Format resolution now follows a priority chain: `file_type` → data URL media type → filename extension → `"pdf"` default. Previously only `file_type` was consulted. - `ParseDataURL` in `schemas/utils.go` is now a public function that correctly handles media type parameters (e.g. `;charset=utf-8`), uppercase media types, and payloads containing newlines. The old regex silently dropped any data URL whose header contained a parameter, causing the entire `"data:..."` string to be forwarded to Bedrock as the document payload. - Non-base64 data URLs (e.g. `data:text/plain,Hello%20World`) are now percent-decoded and their text content is populated in both `source.text` and `source.bytes` instead of being forwarded verbatim. - The Responses path (`responses.go`) previously ignored `file_url` entirely, emitting a document block with an empty source. It now fetches and inlines the bytes the same way the Chat path does, and propagates fetch errors rather than swallowing them. - `convertBifrostMessageToBedrockMessage` now returns an error instead of silently returning `nil` on conversion failure, so a missing turn is never silently dropped from the request. - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [x] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ```sh go test ./core/providers/bedrock/... ./core/schemas/... ``` Key test cases added: - `TestDocumentFormatFromDataURL` — verifies that each supported MIME type embedded in a data URL resolves to the correct Bedrock format string and that the `data:...` prefix is stripped from `source.bytes`. - `TestDocumentFormatResolutionPrecedence` — verifies the `file_type` → data URL → filename extension → default priority chain. - `TestDocumentInlineTextDataURL` — verifies that non-base64 data URLs are percent-decoded and stored in both `source.text` and `source.bytes`. - `TestToBedrockResponsesRequest_DocumentFormatFromDataURL` — same format fix verified on the Responses path. - `TestToBedrockResponsesRequest_DocumentFileURLIsFetched` — verifies that an unreachable `file_url` surfaces as an error rather than producing an empty document block. - `TestParseDataURL` — unit tests for the new public `ParseDataURL` function covering parameters, uppercase, newlines in payload, and invalid inputs. - [ ] Yes - [x] No `file_url` values are now fetched over the network on the Responses path (matching existing Chat path behaviour). The fetch is performed with the existing `providerUtils.FetchAndEncodeURL` helper, which is subject to the same controls already in place for image URL fetching. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Adds regression coverage for maximhq#5472, where Bedrock's document format converter defaulted every uploaded document to `format:"pdf"` regardless of the actual file type, causing AWS to reject non-PDF documents with `ValidationException`. This PR adds 14 end-to-end test cases to the provider harness collection covering the fixed behavior across both `/v1/chat/completions` and `/v1/responses`. - Added folder **42. Bedrock Document Uploads via OpenAI type:"file" (maximhq#5472)** to the provider harness collection with 14 test cases: - Cases 1–11 exercise `/v1/chat/completions` with XLSX, DOCX, CSV, PDF, TXT, and `file_url` inputs, covering format resolution by data URL media type, filename extension, explicit `file_type`, charset-parameterized data URLs, non-base64 percent-encoded data URLs, opaque media types, and streaming - Cases 12–14 pin the same invariants on `/v1/responses` `input_file` blocks (XLSX data URL, CSV data URL, `file_url`) - Every fixture embeds the token `BIFROST7788` so assertions confirm the document was actually parsed by Claude, not merely accepted - Updated `HARNESS_COVERAGE_BACKLOG.md` to mark the **Document input** item as partially covered (`[~]`), noting that the OpenAI `type:"file"` / Responses `input_file` path is now covered by folder 42, while a native Converse-shaped `document` block posted directly at `/bedrock/model/{id}/converse` remains uncovered - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [x] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs Import `tests/e2e/api/collections/provider-harness.json` into Postman and run folder **42. Bedrock Document Uploads via OpenAI type:"file" (maximhq#5472)** against a running Bifrost instance with Bedrock credentials configured. Each test asserts: - The response does not contain `"The PDF specified was not valid"`, `"could not be parsed as the specified format"`, or `"The document source bytes"` (the AWS rejection messages from the bug) - The response status is below 400 - For document-content cases, the model's reply includes `BIFROST7788`, confirming the document was read Before the fix, cases 1–3, 5–8, and 12–14 all returned a 400 `ValidationException`. - [x] No Closes maximhq#5472 None. Test fixtures contain only synthetic document content with no real credentials or PII. - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
xAI's `grok-imagine` image generation API returns a `cost_in_usd_ticks` field in its usage object instead of token counts. Without this field on `ImageUsage`, the value was silently dropped during unmarshalling, causing the response to surface an empty `"usage":{}`.
Fixes maximhq#5498
## Changes
- Added `CostInUsdTicks *int64` to `ImageUsage` with `omitempty` so it is only serialized when present, leaving existing provider responses (OpenAI, Gemini, etc.) unaffected.
- Extended `DeepCopy` to allocate a new pointer for `CostInUsdTicks`, preserving the no-shared-pointers contract relied on by cost calculation logic.
- Added tests covering round-trip marshal/unmarshal of `cost_in_usd_ticks`, omission of the field when absent, and pointer independence after `DeepCopy`.
## Type of change
- [x] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [ ] Transports (HTTP)
- [x] Providers/Integrations
- [ ] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./core/schemas/...
```
Expected: all three new tests pass — `TestImageUsage_CostInUsdTicksRoundTrip`, `TestImageUsage_CostInUsdTicksOmittedWhenAbsent`, and `TestImageUsage_DeepCopyCostInUsdTicks`.
## Breaking changes
- [ ] Yes
- [x] No
## Security considerations
No security implications. The new field is a cost/billing value returned by xAI and is passed through as-is.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
Briefly explain the purpose of this PR and the problem it solves. - What was changed and why - Any notable design decisions or trade-offs - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs Describe the steps to validate this change. Include commands and expected outcomes. ```sh go version go test ./... cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. If UI changes, add before/after screenshots or short clips. - [ ] Yes - [ ] No If yes, describe impact and migration instructions. Link related issues and discussions. Example: Closes maximhq#123 Note any security implications (auth, secrets, PII, sandboxing, etc.). - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
akshaydeo
dismissed
coderabbitai[bot]’s stale review
August 14, 2026 20:05
The merge-base changed after approval.
Edward-Upton
force-pushed
the
ed/azure-gpt56-cache-creation-cost
branch
from
August 14, 2026 20:55
07a0324 to
6c9a963
Compare
Contributor
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes maximhq#123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds the `ent-v1.5.11` changelog entry to the documentation navigation and adds a trailing newline to `docs.json`. ## Changes - Added `changelogs/ent-v1.5.11` as the first entry in the Enterprise changelogs section of `docs.json`, placing it at the top of the list in the correct chronological order - Added a trailing newline at the end of `docs.json` ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [x] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [x] Docs ## How to test Navigate to the Enterprise changelogs section of the documentation and verify that the `ent-v1.5.11` changelog appears at the top of the list and links correctly to its page. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
The hosted datasheet copies cache-read onto Azure GPT-5.6 and omits cache-write. Azure bills those writes at 1.25× the matching input rate. Affected packages: - framework/modelcatalog/datasheet/ Co-authored-by: Cursor <cursoragent@cursor.com>
Edward-Upton
force-pushed
the
ed/azure-gpt56-cache-creation-cost
branch
from
August 17, 2026 00:05
6c9a963 to
9afcfaa
Compare
akshaydeo
dismissed
coderabbitai[bot]’s stale review
August 17, 2026 00:06
The base branch was changed.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Azure GPT-5.6 datasheet rows (
azure/gpt-5.6-terra,-sol,-luna, andazure/us|eu/...variants) currently havecache_read_input_token_costbut nocache_creation_input_token_cost. Cache-write tokens then fall back to the uncached input rate (1.0×) instead of Azure's published 1.25× write SKU.This fills missing cache-creation fields on Azure GPT-5.6 rows at 1.25× the matching input rate when the datasheet is loaded. Existing values are left alone (OpenAI-native rows, Azure Claude, and any future datasheet that already includes the SKU).
Changes
$2.50/1M), US data-zone ($2.75/1M), and leave OpenAI / Azure GPT-4o / Claude untouched.Type of change
Affected areas
How to test
Breaking changes
Bifrost
logs.costfor Azure GPT-5.6 cache writes will move from 1.0× input to 1.25× input. That matches Azure's published write meter. Microsoft starts billing that SKU on or after 21 Aug 2026; until then Azure still invoices writes as ordinary input.Related issues
LiteLLM: BerriAI/litellm#35125
Checklist
Made with Cursor