Conversation
…filter inaccessible sidebar items (maximhq#3295) This PR improves RBAC granularity in the sidebar by introducing dedicated resource types for `APIKeys`, `Inference`, and `Metrics`, and fixes sidebar visibility logic so that items and groups are hidden when the user lacks access rather than relying on broader, less specific permissions. - Added three new `RbacResource` enum values: `APIKeys`, `Inference`, and `Metrics` to the fallback RBAC context. - The API Keys sidebar item now gates access via the new `hasAPIKeyAccess` (`RbacResource.APIKeys`) check instead of the generic `hasSettingsAccess`. - The MCP Logs sidebar item now correctly gates access via `hasMCPGatewayAccess` instead of the unrelated `hasLogsAccess`. - Introduced an `accessibleItems` memoized computation that filters out sidebar items and entire groups whose sub-items are all inaccessible, ensuring users never see empty navigation sections. Previously, access filtering only happened during search. - Removed unused imports (`PanelLeft`, `PanelRight`, `cn`). - [ ] Bug fix - [x] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs 1. Log in as a user with restricted RBAC permissions that exclude `APIKeys` and/or `Settings`. 2. Verify the API Keys entry under the Config section is hidden for users without `APIKeys` view permission. 3. Verify the MCP Logs entry is hidden for users without `MCPGateway` view permission. 4. Verify that sidebar groups with no accessible sub-items are hidden entirely rather than showing an empty group. 5. Verify that users with full access see no change in sidebar behavior. ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` _Add before/after screenshots showing sidebar items hidden for restricted users._ - [ ] Yes - [x] No _Link related issues here._ Access control checks for API Keys management are now scoped to a dedicated `APIKeys` RBAC resource rather than the broader `Settings` resource, reducing the risk of unintended access to key management for users who have settings visibility but should not manage API keys. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…elete access (maximhq#3314) The delete button in log tables was always rendered (just disabled) for users without delete access. This PR hides the actions column entirely when the user lacks delete permissions, and fixes the RBAC resource check for MCP logs to use the correct `MCPGateway` resource instead of `Logs`. - The actions column in both the workspace logs and MCP logs tables is now conditionally included in the column definitions only when `hasDeleteAccess` is `true`, rather than always rendering a disabled button. - The delete button styling was updated to use more visible destructive colors (`text-destructive/60 border-destructive/60`) instead of the previous muted secondary foreground styles. - The RBAC resource used to gate delete access on the MCP logs page was corrected from `RbacResource.Logs` to `RbacResource.MCPGateway`. - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs 1. Log in as a user **without** delete access on Logs or MCPGateway resources. 2. Navigate to the workspace logs page and the MCP logs page. 3. Verify the delete button/column is not visible. 4. Log in as a user **with** delete access. 5. Verify the delete button appears and is functional. ```sh cd ui pnpm i pnpm test pnpm build ``` Before: Delete button rendered but disabled for users without access. After: Delete column is hidden entirely for users without delete access. - [ ] Yes - [x] No The RBAC fix ensures MCP log deletion is gated on the correct `MCPGateway` resource permission, preventing users with only `Logs` delete access from incorrectly being granted delete access to MCP logs. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…ogs route and sidebar (maximhq#3316) Introduces a dedicated `MCPLogs` RBAC resource, decoupling MCP log access control from the `MCPGateway` resource. This allows permissions for viewing and deleting MCP logs to be managed independently from gateway-level permissions. - Added `MCPLogs` as a new `RbacResource` enum value in the fallback RBAC context. - The MCP Logs route now checks `MCPLogs` view permission and renders a `NoPermissionView` when access is denied, rather than rendering the page unconditionally. - Delete access on the MCP Logs page now checks `RbacResource.MCPLogs` instead of `RbacResource.MCPGateway`. - The sidebar MCP Logs entry now uses `hasMCPLogsAccess` (derived from `RbacResource.MCPLogs`) to control visibility, rather than reusing `hasMCPGatewayAccess`. - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs 1. Configure a role that has `MCPGateway` access but **no** `MCPLogs` access. 2. Log in as a user with that role and navigate to the MCP Logs page — the `NoPermissionView` should be displayed and the sidebar entry should be hidden. 3. Grant the role `MCPLogs` view access and confirm the page and sidebar entry become accessible. 4. Verify that delete functionality on the MCP Logs page is gated by `MCPLogs` delete permission independently of `MCPGateway` delete permission. ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` N/A - [x] Yes - [ ] No Any role configuration that previously relied on `MCPGateway` permissions to grant access to MCP Logs will need to be updated to explicitly grant `MCPLogs` permissions. N/A Access to MCP log data (which may contain sensitive tool execution details) is now enforced by a dedicated RBAC resource, reducing the risk of unintended access through overly broad `MCPGateway` permissions. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…SQL helper to prevent malformed JSON from aborting list queries (maximhq#3407) ## Summary The `/api/logs` list query was aborting entirely when a single row contained malformed JSON in `input_history` or `responses_input_history`. The previous inline guard only checked the first character before casting to `jsonb`, so rows that appeared array-shaped but contained malformed JSON (unterminated structures, trailing commas, unpaired UTF-16 surrogates, `\u0000` escapes, etc.) would trigger a `22P02`/`22P05` error and kill the entire response. This PR fixes that by introducing a PL/pgSQL helper function (`bifrost_safe_jsonb`) that wraps the cast in an `EXCEPTION` block and falls back to returning the raw text on any parse failure. ## Changes - Added a new migration `migrationAddSafeJsonbFunction` that installs the `bifrost_safe_jsonb(text)` PL/pgSQL function on Postgres. The function validates the input, attempts the `jsonb` cast inside an `EXCEPTION` block, and returns the last array element on success or the raw text on any failure. - Replaced the multi-condition inline `CASE` guards in `listSelectColumns` for Postgres with calls to `bifrost_safe_jsonb`, simplifying the SQL and correctly handling all malformed-JSON edge cases that the previous character-check approach missed. - For SQLite, added `json_valid()`, `json_type()`, and `json_array_length()` guards to the `CASE` expressions to prevent extraction attempts on invalid or empty arrays. - Added `safe_jsonb_test.go` covering both the SQLite and Postgres dialect branches of `listSelectColumns`, as well as direct invocation of `bifrost_safe_jsonb` across all relevant edge cases (malformed structures, surrogate pairs, `\u0000` escapes, non-array values, SQL `NULL`). ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh cd framework && docker compose up -d postgres go test ./framework/logstore/ -run 'MalformedInputHistory|BifrostSafeJsonb' -count=1 -v ``` Insert a row into the logs table with a malformed JSON value in `input_history` (e.g., `[{"key": "val"` — unterminated) and verify that a call to the list endpoint returns successfully without a 500 error, with the malformed row's `input_history` returned as raw text rather than aborting the query. ## Test Coverage ### `TestSearchLogs_MalformedInputHistory_{SQLite,Postgres}` — end-to-end list query | # | Case | Column | Payload shape | Pre-fix behavior | Path exercised | | --- | --- | --- | --- | --- | --- | | 1 | `unterminated_object_in_array` | `input_history` | `[{"role":"user","content":"hi"` | 22P02 aborts query | EXCEPTION fallback | | 2 | `garbage_after_bracket` | `input_history` | `[abc, not json]` | 22P02 aborts query | EXCEPTION fallback | | 3 | `trailing_comma` | `input_history` | `[{"role":"user","content":"hi"},]` | 22P02 aborts query | EXCEPTION fallback | | 4 | `unclosed_array_only` | `input_history` | `[` | 22P02 aborts query | EXCEPTION fallback | | 5 | `open_bracket_then_brace_unclosed` | `input_history` | `[{` | 22P02 aborts query | EXCEPTION fallback | | 6 | `nan_value_not_valid_json` | `input_history` | `[NaN]` | 22P02 aborts query | EXCEPTION fallback | | 7 | `infinity_value_not_valid_json` | `input_history` | `[Infinity]` | 22P02 aborts query | EXCEPTION fallback | | 8 | `unpaired_high_surrogate` | `input_history` | `[{"...":"bad \uD800 surrogate"}]` | 22P05 aborts query | EXCEPTION fallback | | 9 | `unpaired_low_surrogate` | `input_history` | `[{"...":"bad \uDC00 low"}]` | 22P05 aborts query | EXCEPTION fallback | | 10 | `bad_surrogate_pair_high_then_ascii` | `input_history` | `[{"c":"\uD800A"}]` | 22P05 aborts query | EXCEPTION fallback | | 11 | `u0000_escape_inside_string` | `input_history` | `[{"...":"null byte � here"}]` | 22P05 aborts query | EXCEPTION fallback | | 12 | `literal_backslash_u0000_valid_jsonb` | `input_history` | `[{"...":"... \\u0000 literal"}]` | OK (degraded to raw by old guard) | Fast path, last-element extraction | | 13 | `single_element_array` | `input_history` | `[{"role":"user","content":"only one"}]` | OK | Fast path | | 14 | `array_of_primitives` | `input_history` | `[1,2,3]` | OK | Fast path | | 15 | `array_with_null_last_element` | `input_history` | `[{...}, null]` | OK | Fast path | | 16 | `deeply_nested_valid` | `input_history` | `[{"role":"user","content":{"nested":{"deep":{"value":42}}}}]` | OK | Fast path | | 17 | `unicode_emoji_content` | `input_history` | `[{"...":"hello 🎉 world ✨"}]` | OK | Fast path | | 18 | `large_valid_array` | `input_history` | 1001-element array | OK | Fast path at scale | | 19 | `leading_whitespace_then_array` | `input_history` | ` [\t{...}]` | OK | `btrim` + fast path | | 20 | `top_level_object_not_array` | `input_history` | `{"not":"an array"}` | OK | Non-array fall-through | | 21 | `null_literal` | `input_history` | `null` | OK | Non-array fall-through | | 22 | `whitespace_only` | `input_history` | `" \t "` | OK | Empty-after-btrim fall-through | | 23 | `realtime_turn_malformed_passthrough` | `input_history` (object_type=`realtime.turn`) | `[{"role":"user"` | OK (outer CASE bypassed safe fn) | Realtime-turn bypass branch | | 24 | `malformed_responses_input_history` | `responses_input_history` | `[{"role":"user"` | 22P02 aborts query | Mirror column, EXCEPTION fallback | | 25 | `valid_responses_input_history` | `responses_input_history` | `[{...},{...}]` | OK | Mirror column, fast path | ## Breaking changes - [ ] Yes - [x] No ## Related issues [https://github.com/maximhq/bifrost/issues/3255](https://github.com/maximhq/bifrost/issues/3255#issuecomment-4427506449) ## Security considerations None. The function is `IMMUTABLE` and operates only on text values already stored in the database. No new inputs are exposed. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…aximhq#3412) ## Summary Adds support for server-configured required request headers in the prompt playground. When the server specifies `required_headers` in its client config, users can now provide values for those headers directly in the settings panel, and they are forwarded with every chat completion request. ## Changes - Added `customHeaders` state and `requiredHeaders` derived from the core config's `client_config.required_headers` to the `PromptContext`, keeping header keys in sync with the server config while preserving user-entered values. - Exposed a "Required Headers" section in the settings panel that renders an input field for each required header name when any are configured. - Extended `ExecutionConfig` in the executor to accept `customHeaders`, which are merged into the fetch request headers (skipping any entries with empty names or values). - Passed `customHeaders` through both `handleSubmit` and `handleSubmitToolResult` execution paths and included it in their respective `useCallback` dependency arrays. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test 1. Configure `required_headers` in the server's client config (e.g., `["X-My-Custom-Header"]`). 2. Open the prompt playground and navigate to the settings panel. 3. Verify a "Required Headers" section appears with an input for each configured header name. 4. Enter a value for each header and send a chat completion request. 5. Confirm the header is present in the outgoing request. 6. Remove a header from the server config and verify it disappears from the UI without affecting other header values. ```sh cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` ## Screenshots/Recordings _Add before/after screenshots of the settings panel showing the new Required Headers section._ ## Breaking changes - [x] No ## Related issues ## Security considerations Header values are entered by the user and sent only to the configured backend endpoint. Empty header names or values are explicitly skipped before being added to the request, preventing accidental forwarding of blank headers. Users should be cautious not to enter sensitive credentials unless the connection to the server is secured. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…3416) ## Summary When exporting virtual keys, pagination clamping was being applied unnecessarily, which could interfere with retrieving the full dataset. This PR skips the pagination limit/offset clamping when the export flag is set, while still ensuring the offset is non-negative. ## Changes - Pagination clamping via `ClampPaginationParams` is now bypassed when `params.Export` is `true`, allowing exports to retrieve data without artificially constrained limits - A minimal guard ensures `params.Offset` is still set to `0` if negative during an export request ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Trigger a virtual keys export request and verify that all keys are returned without being truncated by pagination limits. Compare the export result count against the total number of virtual keys in the system. ```sh go test ./... ``` ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues maximhq#3414 ## Security considerations No additional security implications. Export access is still gated by existing authentication and authorization checks. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Bumps the `@maximhq/bifrost` NPX package version to `1.6.3` to align the `package.json` and `package-lock.json` version fields, which were previously out of sync. ## Changes - Updated `package.json` version from `1.6.2` to `1.6.3` - Corrected `package-lock.json` to reflect `1.6.3` consistently across both the lockfile root and the package entry (previously mismatched at `1.0.6` and `1.0.4`) ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh cd npx/bifrost npm install npm pack --dry-run # Verify the reported version is 1.6.3 ``` ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations No security implications. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… bar click suppression (maximhq#3431) Adds a log volume histogram chart to the MCP Logs page, matching the existing chart behavior on the main Logs page. Also fixes a bug where clicking a bar immediately after a drag-select would overwrite the dragged time range with a single-bucket zoom. - Added `useGetMCPHistogramQuery` to the MCP Logs page to fetch histogram data with optional polling, and rendered the `LogsVolumeChart` component in the MCP Logs view. - Added `handleTimeRangeChange`, `handleResetZoom`, and `isZoomed` logic to the MCP Logs page, mirroring the behavior already present on the main Logs page. - Fixed `isZoomed` on the main Logs page to return `false` when a named `period` (e.g. `"1h"`) is active, so resetting zoom correctly clears the zoomed state. - When resetting zoom, `period: "1h"` and `polling: true` are now explicitly set in URL state to ensure the page returns to a live-polling relative range. - Fixed a race condition in `LogsVolumeChart` where Recharts fires a Bar `onClick` event immediately after a drag-select `mouseUp`, which was overwriting the dragged range with a single-bucket zoom. A `suppressNextBarClickRef` ref is set during drag completion and cleared on the next bar click to suppress the spurious event. - [x] Bug fix - [x] Feature - [x] UI (React) 1. Navigate to the MCP Logs page and confirm the log volume histogram chart renders and updates with polling. 2. Click a bar in the histogram and confirm the time range zooms into that bucket. 3. Drag-select a range on the histogram and confirm the time range updates to the dragged selection without immediately snapping to a single bucket. 4. Click "Reset Zoom" and confirm the chart returns to the default 1-hour live-polling view. ```sh cd ui pnpm i pnpm build ``` Before: MCP Logs page had no histogram chart. After: MCP Logs page displays the log volume histogram with zoom, drag-select, and reset zoom functionality identical to the main Logs page. - [x] No None. - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
This PR refactors the semantic cache plugin to simplify its internal state management, improves cache lookup correctness, and adds a new `cache_hit_types` filter to the logs API and UI. The direct cache lookup path is now a single deterministic point-fetch by a UUIDv5 `directCacheID` (replacing the previous dual-path of chunk lookup + legacy metadata scan), and several context keys are consolidated. The UI gains a "Local Caching" filter sidebar section and cache hit type badges in the log detail view.
## Changes
- **Semantic cache plugin refactor:**
- Replaced the dual direct-search path (`performDirectChunkLookup` + `performLegacyDirectSearch`) with a single `performDirectSearch` that does an O(1) `GetChunk` by deterministic `directCacheID` (UUIDv5 derived from provider, model, cacheKey, requestHash, paramsHash).
- `generateDirectCacheID` now returns an error instead of silently falling back to a string concatenation, making failures explicit.
- `request_hash` is no longer stored as a top-level metadata field; it is encoded into the `directCacheID` instead.
- Reduced context keys from ~10 to 4 (`directCacheIDKey`, `paramsHashKey`, `embeddingsKey`, `embeddingsInputTokensKey`), removing stale keys like `requestIDKey`, `requestHashKey`, `isCacheHitKey`, and `cacheHitTypeKey`.
- `shouldSkipCaching` is extracted into its own method; cache-hit detection now reads `CacheDebug.CacheHit` from the response rather than a context flag.
- `buildUnifiedMetadata` no longer accepts `requestHash` as a parameter.
- `addSingleResponse` renamed to `addNonStreamingResponse`.
- `StreamAccumulator` fields `HasError`, `FinalTimestamp`, and `FinishReason` on `StreamChunk` are removed; error streams are handled by early return in `PostLLMHook`.
- Streaming replay goroutine now guards every send with `ctx.Done()` to prevent goroutine leaks on dropped consumers.
- A background `runStreamCleanupLoop` goroutine (started by `Init`, stopped by `Cleanup` via `stopCh`) replaces the one-shot cleanup call, periodically reaping stale stream accumulators.
- `buildResponseFromResult` now accepts `threshold`, `similarity`, and `inputTokens` as pointers, and `attachCacheDebug` is extracted as a shared helper for both streaming and non-streaming paths.
- `isExpiredEntry` is extracted as a standalone function.
- `chunkSortKey` replaces the large inline sort comparator in `processAccumulatedStream`.
- Tools, stop sequences, modalities, include lists, and other order-insensitive set fields are now hashed with `hashSortedSet` / `sortedStringSet` to prevent MCP's randomized map iteration from perturbing the request hash.
- `extractAttachmentsForCaching` is extracted so attachment URLs are included in the cache key metadata rather than the embedding text.
- `extractTextForEmbedding` no longer returns a `paramsHash`; callers compute it once via `buildRequestMetadataForCaching` + `hashMap`.
- `generateEmbedding` moved from `utils.go` to `search.go`.
- `generateRequestHash` now accepts prebuilt metadata to avoid recomputing it.
- `removeField` no longer mutates the input slice's backing array.
- Added `PronunciationDictionaryLocators`, `TimestampGranularities`, `Include`, `AdditionalFormats`, and `InputImages` to their respective parameter metadata extractors.
- Public context key names changed from `semantic_cache_*` to `semantic_cache-*` (underscore → hyphen separator after the plugin prefix).
- `SelectFields` no longer includes `request_hash`.
- `VectorStoreProperties` no longer includes a `request_hash` entry.
- `CacheByModel` and `CacheByProvider` default-value log messages added.
- **Log filtering — `cache_hit_types`:**
- Added `CacheHitTypes []string` to `SearchFilters` in `framework/logstore/tables.go`.
- `applyFilters` in `rdb.go` applies a JSON path filter on `cache_debug` for both SQLite (`json_extract`) and PostgreSQL (`substring` regex) dialects, restricted to the allowlist `["direct", "semantic"]`.
- `canUseMatViewFilters` excludes queries with `CacheHitTypes` set from the materialized-view fast path.
- HTTP handlers (`getLogs`, `getLogsStats`, `parseHistogramFilters`) parse a `cache_hit_types` comma-separated query parameter.
- **UI:**
- Added a "Local Caching" filter section to `LogsFilterSidebar` with checkboxes for "Direct cache" and "Semantic cache".
- `cache_hit_types` is added to URL state, filter state, and the `buildFilterParams` API helper.
- Log detail view shows "Direct Cache" (indigo) and "Semantic Cache" (rose) badges based on `cache_debug.hit_type`.
- Plugins form now filters the provider dropdown to embedding-capable providers only (`EmbeddingSupportedProviders` for built-ins; `custom_provider_config.allowed_requests.embedding` for custom providers), shows an error message when no embedding provider is configured, and disables the toggle accordingly.
- Embedding model input replaced with `ModelMultiselect` (single-select mode) scoped to the selected provider.
- Provider dropdown clears the embedding model when the provider changes.
- Provider icons rendered in the provider dropdown.
- `EmbeddingSupportedProviders` constant added to `ui/lib/constants/logs.ts`.
- **Misc:**
- HTTP request logging in `CorsMiddleware` and an auth debug log are commented out.
- `transports/bifrost-http/v1.5.x` added to `.gitignore`.
- Minor formatting fixes in `core/schemas/bifrost.go` and `framework/modelcatalog/sync.go`.
- Missing newline at end of `sync.go` added.
## Type of change
- [ ] Bug fix
- [x] Feature
- [x] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [x] UI (React)
- [ ] Docs
## How to test
```sh
# Core/Transports
go test ./plugins/semanticcache/...
go test ./framework/logstore/...
go test ./transports/bifrost-http/...
# UI
cd ui
pnpm i
pnpm build
```
- Configure the semantic cache plugin with a direct and/or semantic cache type and verify that cache hits are recorded with the correct `hit_type` in `cache_debug`.
- Query `/logs?cache_hit_types=direct` and `/logs?cache_hit_types=semantic` and confirm only matching entries are returned.
- In the UI, open the logs filter sidebar and verify the "Local Caching" section appears with "Direct cache" and "Semantic cache" checkboxes that correctly filter the log list.
- Open a log detail for a cache hit and confirm the appropriate badge ("Direct Cache" or "Semantic Cache") is displayed.
- In the plugins form, verify that only embedding-capable providers appear in the provider dropdown and that the embedding model field uses the model multiselect.
## Breaking changes
- [x] Yes
The public semantic cache context key names have changed from `semantic_cache_*` to `semantic_cache-*`. Any caller setting `CacheKey`, `CacheTTLKey`, `CacheThresholdKey`, `CacheTypeKey`, or `CacheNoStoreKey` via the old string values will no longer be recognized by the plugin. Update all call sites to use the exported constants from the plugin package rather than raw string literals.
`request_hash` is no longer stored as a top-level metadata field in the vector store. Existing cache entries written by prior versions will not be found by the new direct-search path (they will be treated as misses and re-populated).
`ClearCacheForRequestID` is documented as currently broken for entries written by the new direct-search path; callers should not rely on it until the TODO is resolved.
## Related issues
N/A
## Security considerations
The `CacheHitTypes` filter allowlists values to `"direct"` and `"semantic"` before interpolating them into SQL, preventing arbitrary input from reaching the JSON path expression.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
…aximhq#3330) ## Summary Removes the `cleanup_on_shutdown` option from the semantic cache plugin. Cache data now always persists between Bifrost restarts. The previous behavior of deleting all cache entries and the vector store namespace on shutdown is no longer supported. ## Changes - Removed `CleanUpOnShutdown` field from `Config` struct in `plugins/semanticcache/main.go` and stripped the corresponding shutdown deletion logic from `Cleanup()` - Removed `cleanup_on_shutdown` from the JSON config schema (`transports/config.schema.json`), Helm values schema (`helm-charts/bifrost/values.schema.json`), Helm template helper (`_helpers.tpl`), and default `values.yaml` - Removed `cleanup_on_shutdown` from all example Kubernetes values files and documentation code samples - Added migration guide entry (Breaking Change 16) in `docs/migration-guides/v1.5.0.mdx` describing the removal, how to clear cache data using the existing invalidation endpoints, and how to handle dimension/provider/model rotation without the old escape hatch - Updated the semantic caching feature docs to remove references to `cleanup_on_shutdown` and the associated warning block - Removed `TestCleanup_DeletesEntriesAndNamespaceWhenEnabled` test and simplified `newTestPlugin` helper to drop the `cleanupOnShutdown` parameter across all test files ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [x] Docs ## How to test ```sh go test ./plugins/semanticcache/... ``` Verify that passing `cleanup_on_shutdown` in a semantic cache plugin config is rejected by schema validation. Confirm that restarting Bifrost with a semantic cache configured leaves existing vector store entries intact. ## Breaking changes - [x] Yes - [ ] No The `cleanup_on_shutdown` field is removed from the semantic cache plugin config schema and will be rejected by validation. Remove it from `config.json`, Helm values, and any `PUT /api/config` payloads. To clear cache data, use `DELETE /api/cache/clear/{cacheId}`, `DELETE /api/cache/clear-by-key/{cacheKey}`, or rotate `vector_store_namespace` to a fresh name. ## Related issues See Breaking Change 16 in the v1.5.0 migration guide. ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Replaces the separate `PluginsForm` component with a fully self-contained `CachingView` that introduces a first-class **Direct / Direct + Semantic** mode toggle for the local cache plugin. Previously, the UI only exposed provider-backed semantic cache settings and had no concept of direct-only (hash-based) caching as a distinct, supported mode. This rewrite makes direct-only mode the default and gates semantic configuration behind an explicit mode selection. ## Changes - Deleted `pluginsForm.tsx` and consolidated all local cache configuration logic directly into `cachingView.tsx`. - Introduced a `CacheMode` type (`"direct"` | `"semantic"`) with a tab-based picker. Direct-only mode requires no embedding provider; semantic mode adds vector similarity on top and requires a provider, model, and dimension. - The enable/disable toggle now immediately calls `updatePlugin` or `createPlugin` (for first-time setup) rather than deferring the enabled-state change to the Save button, decoupling the plugin lifecycle from config edits. - Added `inferMode` to derive the active mode from a saved config, `isEmptyConfig` to detect zero-value configs from the API, `buildPayload` to strip semantic-only fields when persisting a direct-only config, and `validateForSave` for inline validation surfaced before the user clicks Save. - Structural change warnings (provider/model/dimension drift vs. server state) are now shown only when the user has actually modified those fields, rather than permanently in semantic mode. - Removed the Zod `cacheConfigSchema` validation path in favor of the new `validateForSave` function. - Removed the effect that auto-seeded a default provider/model/dimension on first load, since direct-only mode no longer requires those fields. - Per-request override documentation expanded to include `x-bf-cache-key` and `x-bf-cache-no-store` with clearer descriptions. ## Type of change - [ ] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh cd ui pnpm i || npm i pnpm build || npm run build ``` 1. Navigate to the Workspace → Config → Caching view. 2. Verify the page loads with **Direct only** selected by default and no provider/model/dimension fields visible. 3. Switch to **Direct + Semantic** and confirm provider, model, and dimension fields appear with inline validation. 4. Toggle caching on without a vector store configured and confirm the toggle is disabled. 5. Save a direct-only config and confirm the plugin is created/updated with `dimension: 1` and no provider fields. 6. Save a semantic config with a valid provider, model, and dimension and confirm the full payload is persisted. 7. Reload the page and confirm the saved mode and config are correctly hydrated. ## Screenshots/Recordings Before/after screenshots recommended showing the mode tab picker, the conditional semantic fields, and the structural change warning banner. ## Breaking changes - [x] Yes - [ ] No The `PluginsForm` component is removed. Any code importing it directly will need to be updated. The enable/disable toggle now persists immediately rather than requiring a Save click, which changes the interaction model for existing users. ## Related issues N/A ## Security considerations No new auth, secrets, or PII handling introduced. API keys for embedding providers continue to be inherited from the provider's existing configuration and are not re-entered or stored in the cache config. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…and plugin reloads (maximhq#3423) ## Summary The `CacheHandler` previously captured a reference to the `semantic_cache` plugin at boot time. This caused two bugs: (1) if the plugin was not present in `config.json` at startup, cache-clear routes were never registered, resulting in HTTP 405 for the entire process lifetime; (2) if the plugin was loaded or reloaded via `/api/plugins` after boot, the handler held a stale (or nil) pointer and would silently misbehave. Additionally, `GET /api/plugins/:name` was returning the raw plugin config without runtime status, causing the UI to see an empty status when refetching a single plugin. ## Changes - `CacheHandler` now accepts a `CacheClearerResolver` function instead of a concrete plugin pointer. The resolver is called on every cache-clear request, so plugin lifecycle changes via `/api/plugins` are always honored. - `CacheClearer` and `CacheClearerResolver` are exported so server wiring can supply the resolver without importing the plugin's concrete type. - Cache routes are registered unconditionally at startup. When no plugin is loaded, requests return HTTP 400 with a descriptive message instead of HTTP 405. - The server wiring in `RegisterAPIRoutes` uses a closure over `lib.FindPluginAs` to resolve the plugin per request, replacing the boot-time capture. - `getPlugin` now returns the same response shape as list/create/update (with runtime status merged in), fixing the empty status seen by `useGetPluginQuery` in the UI. - Tests cover the new "plugin not loaded" path for both `clearCache` and `clearCacheByKey`, and existing tests are updated to use the resolver-based constructor. ## Type of change - [x] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./transports/bifrost-http/handlers/... go test ./transports/bifrost-http/... ``` 1. Start the server **without** `semantic_cache` in `config.json`. Issue `DELETE /api/cache/clear/{cacheId}` — expect HTTP 400 with `"semantic_cache plugin is not loaded"` (previously HTTP 405). 2. Load the `semantic_cache` plugin via `POST /api/plugins`. Repeat the request — expect the cache-clear to succeed. 3. Reload or remove the plugin via `PUT`/`DELETE /api/plugins`. Verify the handler reflects the new state on the next request without a server restart. 4. Issue `GET /api/plugins/{name}` for a loaded plugin and confirm the response includes runtime status fields, matching the shape returned by the list endpoint. ## Breaking changes - [x] Yes - [ ] No `NewCacheHandler` now accepts a `CacheClearerResolver` function instead of a `schemas.LLMPlugin`. Any caller constructing a `CacheHandler` directly must be updated to pass a resolver. ## Related issues ## Security considerations None. The change does not affect authentication, secrets, or PII handling. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
…rch paths in semantic cache (maximhq#3424) ## Summary Fixes several correctness issues in the semantic cache plugin's `PostLLMHook` and related helpers: cache telemetry (`cache_debug`) was previously invisible to callers using `no-store`, cache-hit replay detection was fragile, non-positive per-request TTL overrides could silently kill cache writes, and requests with a `cache_type` header narrowed to a path the plugin cannot serve would still produce orphan cache entries. ## Changes - **Early exit for unsupported search paths in `PreLLMHook`**: When `resolveCacheTypes` resolves to a path the plugin cannot actually serve (e.g. `x-bf-cache-type=semantic` against a direct-only plugin, or an unknown header value), the hook now clears cache state and returns early instead of proceeding to generate an embedding or write an orphan entry under a random request UUID that no future read can match. - **Separated cache-hit replay handling from write-skip logic**: The `shouldSkipCaching` method (which conflated cache-hit detection with write-skip conditions) is replaced by `shouldSkipCacheWrite`. Cache-hit replay is now handled as a dedicated early return in `PostLLMHook` before any telemetry stamping, while `shouldSkipCacheWrite` gates only the write decision after telemetry is already stamped. This ensures `cache_debug` is always populated for callers using `no-store` or large-payload modes. - **Telemetry stamped before write decision**: `stampCacheDebugForMiss` is now called before `shouldSkipCacheWrite` is consulted, so observability is not conditional on whether the entry is ultimately written. - **Non-positive TTL overrides fall back to plugin default**: `resolveTTL` now treats a zero or negative per-request TTL override as "use default" rather than applying it, which would have set `expires_at=now` and silently discarded the cache write. - **Cleaned up stale comments**: Removed an outdated ordering constraint comment in `PostLLMHook` that no longer applies after the restructuring. - **Tests updated**: Test cases for `shouldSkipCaching` are renamed and updated to reflect the new `shouldSkipCacheWrite` contract. The cache-hit replay test case is removed from this suite (it is now an early return in `PostLLMHook`, not a condition inside the helper). A new default-is-false test is added. ## Type of change - [x] Bug fix - [ ] Feature - [x] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh go test ./plugins/semanticcache/... ``` Validate the following scenarios: - A request with `x-bf-cache-type=semantic` against a plugin configured with `Provider=""` or `Dimension=1` should log a warning and skip caching entirely — no orphan entry should appear in the store. - A request with `Cache-Control: no-store` should still produce a populated `cache_debug` field in the response with `cache_hit=false`. - A per-request TTL override of `0s` should fall back to the plugin's configured default TTL and not silently discard the cache write. ## Breaking changes - [x] No ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds a standalone end-to-end test suite for the `semantic_cache` plugin under `tests/semanticcache`. The suite validates the full caching lifecycle against a live Bifrost instance — plugin creation/teardown, cache miss/hit assertions, cross-provider behavior, streaming, and log cross-checking — without provisioning any infrastructure itself. ## Changes - **`e2e_test.go`** — `TestMain` entry point: loads config, initializes the report directory, checks Bifrost reachability, enforces plugin-absent precondition (with `RUN_FORCE=1` auto-delete), runs all phases, and performs best-effort teardown on exit. - **`preconditions_test.go`** — Phase 0 checks: Bifrost reachable, OpenAI configured, optional providers (Gemini, Anthropic) present with warnings if absent. - **`http_test.go`** — HTTP helpers for all request types: chat completions (streaming and non-streaming), text completions, embeddings, image generation, and the Responses API. Each helper dumps full request/response bodies to the report directory for forensics. - **`plugin_test.go`** — Plugin lifecycle helpers (`pluginCreate`, `pluginUpdate`, `pluginDelete`, `pluginGet`) mirroring the exact wire format the UI sends to `/api/plugins`. - **`assert_test.go`** — Assertion helpers (`assertMiss`, `assertHit`, `assertNoCacheDebug`, `assertSameCacheID`, `assertDifferentCacheID`) plus a configurable async write-settle wait (`SC_WRITE_SETTLE_MS`) to account for the plugin's async PostLLMHook store write. - **`cache_test.go`** — Cache management helpers (`clearByCacheID`, `clearByCacheKey`) wrapping the `/api/cache/clear/*` endpoints. - **`logs_crosscheck_test.go`** — Cross-checks the persisted log row's `cache_debug` against the in-flight response stamp, with polling to handle Bifrost's async logging pipeline and float epsilon tolerance for JSON encoder differences. - **`fixtures_test.go`** — Hand-curated paraphrase pairs for Phase 2 semantic cases, designed to land well above (canonical→paraphrase) or well below (canonical→unrelated) the default 0.8 similarity threshold. - **`log_test.go`** — Structured per-run logging to `reports/<UTC-timestamp>/run.log` with optional `TRAIL_SESSION_ID` stamping for trail integration. - **`go.mod`** — Standalone module (`github.com/maximhq/bifrost/tests/semanticcache`), consistent with the `tests/governance` pattern, excluded from the repo's `go.work`. - **`README.md`** — Documents prerequisites, env vars, run commands, trail integration, and report output format. - **`.gitignore`** — Excludes `reports/` and `*.log` from version control. Notable design decisions: the suite is intentionally verify-only (no infrastructure provisioning), uses a dedicated vector store namespace (`BifrostSemanticCachePluginE2E`) to isolate test data, and writes full wire-level request/response artifacts per step to support post-mortem debugging without re-running. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test Requires a running Bifrost instance with Weaviate configured, OpenAI (required), and optionally Gemini and Anthropic providers. ```sh cd tests/semanticcache # All phases GOWORK=off go test -v ./... # Single phase GOWORK=off go test -v -run TestPhase1_DirectOnly ./... # Auto-delete any pre-existing plugin row before run RUN_FORCE=1 GOWORK=off go test -v ./... # Keep plugin after run for post-mortem inspection RUN_KEEP_PLUGIN=1 GOWORK=off go test -v ./... ``` Environment variables: | Variable | Default | Purpose | |---|---|---| | `BIFROST_URL` | `http://localhost:8080` | Bifrost base URL | | `SC_CHAT_MODEL_OPENAI` | `openai/gpt-4o-mini` | OpenAI chat model | | `SC_CHAT_MODEL_OPENAI_ALT` | `openai/gpt-4o` | Alternate OpenAI model for cache-by-model cases | | `SC_EMBED_MODEL_OPENAI` | `text-embedding-3-small` | Embedding model for Phase 2 | | `SC_CHAT_MODEL_GEMINI` | `gemini/gemini-2.5-flash` | Gemini chat model | | `SC_CHAT_MODEL_ANTHROPIC` | `anthropic/claude-haiku-4-5` | Anthropic chat model | | `SC_NAMESPACE` | `BifrostSemanticCachePluginE2E` | Vector store namespace | | `SC_WRITE_SETTLE_MS` | `500` | Async write settle wait in ms | | `RUN_FORCE` | unset | `1` to delete pre-existing plugin before run | | `RUN_KEEP_PLUGIN` | unset | `1` to skip teardown on exit | | `TRAIL_SESSION_ID` | unset | Stamped onto every log line for trail integration | ## Screenshots/Recordings N/A ## Breaking changes - [x] No ## Related issues N/A ## Security considerations No secrets are stored in the test suite. API keys are consumed from the existing Bifrost provider configuration and never passed directly through the test harness. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds a comprehensive end-to-end test suite (`TestDirect`) for the semantic cache plugin operating in direct-only mode. The suite covers 55 test cases (plan §1.1–1.55) validating cache hit/miss behavior, key isolation, TTL handling, config flag mutations, normalization, streaming, multi-endpoint support, parameter hashing, tool definitions, and cache management operations. ## Changes - Introduces `tests/semanticcache/direct_test.go` with `TestDirect`, covering: - **Basic hit/miss and key isolation** (1.1, 1.2, 1.3, 1.4) - **`cache_by_model` and `cache_by_provider` flag behavior** (1.5–1.8), including serial config-mutation cases that restore baseline via `t.Cleanup` - **`exclude_system_prompt` flag** (1.9, 1.10) - **Conversation threshold boundary conditions** (1.11, 1.12) - **TTL expiry, per-request TTL override, invalid TTL fallback, and zero/negative TTL fallback** (1.13, 1.14, 1.15, 1.54) - **`no-store` header semantics**, including case-sensitivity and explicit `false` value (1.16, 1.17, 1.45, 1.46) - **`cache-type` header behavior** in direct-only mode, including the `semantic` header bug case (1.18, 1.19) - **Streaming SSE**: hit/miss, chunk replay order, and non-final chunk cache_debug absence (1.24, 1.25, 1.47) - **Multi-endpoint coverage**: text completions, responses API, embeddings, and image generation (1.20–1.23) - **Input normalization**: case folding, whitespace trimming, Unicode, and large prompts (1.26–1.29) - **Image attachment hashing**: same URL hits, different URL misses (1.30, 1.31) - **Edge cases**: nil content messages, empty messages array, unknown cache ID deletion (1.42, 1.43, 1.40) - **Parameter hash isolation**: temperature, top_p, seed, max_tokens, top_logprobs, tools (order-independent and name-change), prompt_cache_key, service_tier, store flag (1.32–1.37, 1.48–1.52) - **Cache management**: clear by cache ID, clear by key (1.38, 1.39) - **Plugin status round-trip** via GET (1.44) - **`/api/logs` cross-check**: verifies persisted `cache_debug` matches in-flight response stamp (1.55) - **`responses` API `previous_response_id` isolation** (1.53) - **Threshold header no-op in direct-only mode** (1.41) - Adds helper functions: `simpleChat`, `chatWithSystem`, `chatWithImage`, `restoreDirectBaseline`, `assertHitAndReturnCacheDebug` - Establishes a parallelism contract: cases that mutate plugin config run serially (no `t.Parallel()`); all others run concurrently with unique cache keys to prevent collisions ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Run the full direct-mode suite go test ./tests/semanticcache/... -run TestDirect -v -timeout 300s # Skip the expensive image generation case SC_SKIP_IMAGE_GEN=1 go test ./tests/semanticcache/... -run TestDirect -v -timeout 300s ``` Required environment variables (same as the broader semantic cache e2e suite): - `OPENAI_MODEL` — primary OpenAI-compatible model (e.g. `openai/gpt-4o-mini`) - `OPENAI_MODEL_ALT` — secondary model for cross-model isolation cases - `OPENAI_EMBED` — embedding model name (e.g. `text-embedding-3-small`) - `ANTHRO_MODEL` — (optional) Anthropic model; cases 1.7 and 1.8 skip if unset - `SC_SKIP_IMAGE_GEN=1` — (optional) skip case 1.23 to avoid DALL-E costs ## Screenshots/Recordings N/A — test-only change. ## Breaking changes - [x] No ## Related issues ## Security considerations No new auth, secrets, or PII surface. Test prompts are benign and do not contain sensitive data. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Adds a comprehensive integration test suite for the semantic cache mode (Phase 2), covering the full lifecycle of semantic similarity-based cache hits and misses using Weaviate as the vector store and OpenAI's `text-embedding-3-small` as the embedding model. This suite validates that the semantic cache behaves correctly across a wide range of real-world scenarios, complementing the existing direct-mode (Phase 1) tests. ## Changes - Added `TestParaphraseFixtures` to pre-flight all paraphrase pairs against the live embedding model, asserting cosine similarity thresholds before any semantic cache cases run. This prevents flaky downstream failures caused by borderline fixture pairs. - Added `TestSemantic` containing 44 sub-cases (2.1–2.44) covering: - Semantic hit on paraphrase, miss on unrelated content - Per-request threshold overrides (relax, tighten, clamp above/below valid range) - `x-bf-cache-type` header forcing direct-only or semantic-only lookup paths - Cache key and model/provider isolation in semantic mode - `cache_by_model=false` and `cache_by_provider=false` cross-model/cross-provider hits - Streaming replay of semantic hits, including tool call preservation - TTL expiry, per-request TTL, TTL=0 fallback, and `no-store` header semantics - Namespace isolation and dimension-change silent miss behavior - Embedding endpoint bypass (semantic search skipped for `/v1/embeddings`) - Image generation and Responses API semantic hits - Text completion semantic hits - Gemini provider with OpenAI embedding provider - `params_hash` isolation (temperature, service tier, store flag, prompt cache key, previous response ID) - `exclude_system_prompt` flag effect on semantic matching - Conversation message threshold skipping semantic search - Attachment URL changes causing misses - `cache_debug` field presence and correctness on hits and misses, including log endpoint cross-check - Streaming chunk-level `cache_debug` placement (final chunk only) - Serial (non-parallel) cases that mutate plugin config restore baseline via `t.Cleanup` to avoid test pollution. - A dedicated Weaviate namespace (`cfg.Namespace + "Semantic"`) is used to avoid dimension conflicts with the Phase 1 direct-mode namespace. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Run fixture pre-flight (requires OpenAI embedding access) go test ./tests/semanticcache/... -run TestParaphraseFixtures -v # Run full semantic suite go test ./tests/semanticcache/... -run TestSemantic -v -timeout 10m # Skip fixture verification if embedding access is unavailable SC_SKIP_FIXTURE_VERIFY=1 go test ./tests/semanticcache/... -run TestSemantic -v -timeout 10m # Skip image generation cases if DALL-E is unavailable SC_SKIP_IMAGE_GEN=1 go test ./tests/semanticcache/... -run TestSemantic -v -timeout 10m ``` Required environment/config: - `cfg.OpenAIEmbed` — embedding model name (e.g. `text-embedding-3-small`) - `cfg.OpenAIModel` / `cfg.OpenAIModelAlt` — chat models for isolation tests - `cfg.AnthroModel` — optional; skipped if empty (case 2.13) - `cfg.GeminiModel` — optional; skipped if empty (case 2.28) - `cfg.Namespace` — base Weaviate namespace; suite appends `Semantic` suffix - `SC_SKIP_FIXTURE_VERIFY=1` — skip embedding pre-flight - `SC_SKIP_IMAGE_GEN=1` — skip DALL-E case ## Screenshots/Recordings N/A ## Breaking changes - [x] No ## Related issues N/A ## Security considerations No new auth, secrets, or PII handling introduced. Tests call live external APIs (OpenAI, optionally Anthropic/Gemini) and require valid credentials in the test environment; no credentials are hardcoded. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [ ] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary
Adds an end-to-end lifecycle test for the semantic cache plugin, covering the full disable → re-enable → delete → recreate flow and asserting that namespace data persists across each state transition.
## Changes
- Introduces `TestLifecycle` in `tests/semanticcache/lifecycle_test.go`, which runs 10 serial subtests (3.1–3.10) exercising the plugin's lifecycle state machine:
- **3.1** – Disabling the plugin via PUT sets `enabled=false` and `status=disabled`
- **3.2** – Requests while disabled bypass the cache pipeline entirely (no `cache_debug` header)
- **3.3 / 3.4** – Cache-clear endpoints (`/api/cache/clear/{id}` and `/api/cache/clear-by-key/{k}`) return HTTP 400 when the plugin is not loaded
- **3.5** – Re-enabling via PUT restores `enabled=true` and `status=active`
- **3.6** – Entries written before disable are still queryable after re-enable
- **3.7** – DELETE removes both the DB row and the in-memory plugin instance
- **3.8** – Requests after delete bypass the cache pipeline (no `cache_debug` header)
- **3.9** – Recreating the plugin with the same config succeeds and surfaces `status=active`
- **3.10** – Entries written before delete are still queryable after recreate, validating the namespace-persistence contract introduced by the removal of `CleanUpOnShutdown`
- Tests are intentionally serial (no `t.Parallel()`) because each subtest mutates globally shared plugin lifecycle state
- A `t.Cleanup` handler performs best-effort key clearing regardless of which lifecycle state the plugin is left in at teardown
## Type of change
- [ ] Bug fix
- [ ] Feature
- [ ] Refactor
- [ ] Documentation
- [x] Chore/CI
## Affected areas
- [ ] Core (Go)
- [ ] Transports (HTTP)
- [ ] Providers/Integrations
- [x] Plugins
- [ ] UI (React)
- [ ] Docs
## How to test
```sh
go test ./tests/semanticcache/... -run TestLifecycle -v
```
Expected outcome: all 10 subtests (3.1–3.10) pass, with structured log output at each step confirming correct status transitions and cache hit/miss behaviour.
## Breaking changes
- [x] No
## Related issues
## Security considerations
None. Tests run against a local Bifrost instance and do not introduce new auth paths, secrets handling, or PII exposure.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [x] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [x] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
…kefile targets (maximhq#3429) ## Summary Adds Makefile targets for running `semantic_cache` plugin unit tests and end-to-end tests, with optional integration of the `trail` CLI for capture-based debugging sessions. ## Changes - Added `test-semantic-cache` target that runs e2e tests from `tests/semanticcache`, supporting a `CACHE_TYPE` variable (`direct` or `semantic`) to filter which test phases are executed. Automatically wraps the run in `trail run` if the `trail` binary is available on `PATH`. - Added `test-semantic-cache-complete` target that runs both the plugin unit tests (`plugins/semanticcache`) and the e2e tests in sequence, optionally wrapping the entire session in a single `trail run` invocation. - Added `_test-semantic-cache-complete-inner` as an internal helper target that performs the actual sequential execution of unit and e2e tests with formatted output banners. - Registered all three new targets in the `.PHONY` declaration. ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Run all semantic_cache e2e tests make test-semantic-cache # Run only direct cache tests CACHE_TYPE=direct make test-semantic-cache # Run only semantic cache tests CACHE_TYPE=semantic make test-semantic-cache # Run both unit and e2e tests together make test-semantic-cache-complete # Force e2e run regardless of preconditions RUN_FORCE=1 make test-semantic-cache-complete ``` If `trail` is installed and on `PATH`, all commands will automatically wrap execution in a `trail run` session for capture-based debugging. ## Breaking changes - [ ] Yes - [x] No ## Related issues ## Security considerations None. ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
## Summary Briefly explain the purpose of this PR and the problem it solves. ## Changes - What was changed and why - Any notable design decisions or trade-offs ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [ ] Core (Go) - [ ] Transports (HTTP) - [ ] Providers/Integrations - [ ] Plugins - [ ] UI (React) - [ ] Docs ## How to test Describe the steps to validate this change. Include commands and expected outcomes. ```sh # Core/Transports go version go test ./... # UI cd ui pnpm i || npm i pnpm test || npm test pnpm build || npm run build ``` If adding new configs or environment variables, document them here. ## Screenshots/Recordings If UI changes, add before/after screenshots or short clips. ## Breaking changes - [ ] Yes - [ ] No If yes, describe impact and migration instructions. ## Related issues Link related issues and discussions. Example: Closes maximhq#123 ## Security considerations Note any security implications (auth, secrets, PII, sandboxing, etc.). ## Checklist - [ ] I read `docs/contributing/README.md` and followed the guidelines - [ ] I added/updated tests where appropriate - [ ] I updated documentation where needed - [ ] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
… responses Detects when an OpenAI custom provider has Responses disabled but Chat Completions enabled, and routes the Responses request through the Chat Completions endpoint instead. Changes: - core/providers/openai/openai.go - Add fallback to chat completions Update changelogs: 1. core/changelog.md: [feat]: use chat completions for openai custom providers that disable responses [@kevinpdev](https://github.com/kevinpdev)
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (104)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThis PR refactors semantic cache invalidation from request-ID to deterministic cache-ID based clearing, removes ChangesCore Semantic Cache Plugin Refactor
Cache ID–Based Invalidation API
Logging Filter and Telemetry Updates
Configuration and Schema Updates
Semantic Cache Plugin Unit Tests
End-to-End Test Suite
UI and Frontend Layer
Documentation and Migration Guides
Provider and Core Logic Updates
Makefile and Build Targets
Test Runner CLI Enhancements
Sequence Diagram(s)sequenceDiagram
participant Client
participant Plugin as Cache Plugin
participant VS as Vector Store
participant Logger as Logging
Client->>Plugin: PreLLMHook(request)
activate Plugin
Plugin->>Plugin: Resolve cache key & type
Plugin->>Plugin: Build params hash
alt Direct Cache Enabled
Plugin->>Plugin: Compute deterministic UUID (params + provider/model)
Plugin->>VS: GetChunk(directCacheID)
VS-->>Plugin: cached response (or nil)
end
alt Semantic Cache Enabled
Plugin->>Plugin: Generate embedding from input
Plugin->>VS: GetNearest(embedding, filters)
VS-->>Plugin: similarity score & cached response
end
Plugin-->>Client: ShortCircuit (if cache hit)
deactivate Plugin
Client->>Plugin: PostLLMHook(response)
activate Plugin
Plugin->>Plugin: Stamp cache_debug (hit/miss info)
alt Should Cache Response
Plugin->>Plugin: Resolve storage ID & embedding
Plugin->>VS: Add(metadata, response, embedding, TTL)
VS-->>Plugin: ack
end
Plugin-->>Client: response with cache_debug
deactivate Plugin
Client->>Logger: Query logs
Logger->>Logger: Filter by cache_hit_types
Logger-->>Client: logs with cache_debug included
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Suggested reviewers
✨ Finishing Touches🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The OpenAI provider in Bifrost can be reused as a base for OpenAI compatible custom providers.
Some providers of openai compatible OSS models do not support the the responses api, only chat completions. Before this PR, calling responses against such a provider would error out.
One example of this is open source models in Azure AI Foundry like kimi k2, minimax, glm, etc...
The request flow that was initially broken that this PR now fixes is one like the following
Changes
When a custom provider has responses disabled but chat_completions allowed in its config, translate the Responses request into a Chat Completions call and convert the result back into the Responses shape, so a client calling Bifrost's Responses API still gets a valid Responses-shaped reply, even when the upstream provider only supports chat completions.
Type of change
Affected areas
How to test
Setup a custom provider that points to an openai compatible model that only supports chat completions:
Then run claude code and point it at that model with that provider
Before this PR we see these logs:
After this PR we do not see those errors and we are succesful
Breaking changes
If yes, describe impact and migration instructions.
Related issues
Related to #2826
Security considerations
No security considerations
Checklist
docs/contributing/README.mdand followed the guidelines