feat: openai compaction support - #4053
Merged
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a new "compaction" request/response type and wires it through schemas, the Bifrost pipeline, providers (OpenAI/Azure/XAI implementations + unsupported stubs), HTTP routing, pricing, logging, UI, and comprehensive external tests. ChangesCompaction Request Feature
Sequence Diagram(s) sequenceDiagram
participant Bifrost
participant OpenAIProvider
participant OpenAIResponsesAPI
Bifrost->>OpenAIProvider: Compaction(ctx, key, BifrostCompactionRequest)
OpenAIProvider->>OpenAIProvider: HandleOpenAICompactionRequest(ToOpenAICompactionRequest)
OpenAIProvider->>OpenAIResponsesAPI: POST /v1/responses/compact (JSON)
OpenAIResponsesAPI-->>OpenAIProvider: response.compaction
OpenAIProvider-->>Bifrost: BifrostCompactionResponse + ExtraFields
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
|
Collaborator
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
TejasGhatte
marked this pull request as ready for review
June 4, 2026 11:36
Contributor
Confidence Score: 5/5
Important Files Changed
Reviews (8): Last reviewed commit: "feat: openai compaction support" | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
core/schemas/provider.go (1)
326-400:⚠️ Potential issue | 🟠 Major | ⚡ Quick winAdd the compaction schema allow-lists in the same PR.
The Go-side gate is wired here, but the transport config schema is still missing
compaction. That meanscustom_provider_config.allowed_requests.compactioncannot be configured through validated/persisted config, and pricing overrides still cannot target compaction requests, so the feature remains partially unreachable outside in-memory config.As per coding guidelines,
transports/config.schema.jsonis the source of truth for config fields, and the referenced schema contents say$defs.custom_provider_config.allowed_requestsplus$defs.pricing_override_request_typestill lackcompaction.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/schemas/provider.go` around lines 326 - 400, The AllowedRequests Go struct and IsOperationAllowed already handle Compaction, but transports/config.schema.json is missing the compaction field and enum value; update the JSON schema by adding "compaction": { "type": "boolean" } to $defs.custom_provider_config.allowed_requests and add "compaction" to $defs.pricing_override_request_type (or equivalent enum for RequestType) so custom_provider_config.allowed_requests.compaction is validated/persisted and compaction can be targeted in pricing overrides; ensure the schema description and examples (if any) are consistent with other boolean flags.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/schemas/bifrost.go`:
- Line 157: The schema changes add a new RequestType constant CompactionRequest
but the JSON transport schema is missing "compaction" in its allow-lists; update
transports/config.schema.json by adding "compaction" to the enum under
$defs.pricing_override_request_type and to the allowed values array/object under
$defs.custom_provider_config.properties.allowed_requests.properties so that the
new CompactionRequest is recognized by transport-side
governance/pricing/custom-provider checks.
In `@core/schemas/responses.go`:
- Around line 69-78: Add a WithDefaults() method on BifrostCompactionResponse
that returns a normalized copy (or nil if receiver is nil); ensure it sets
Object = "response.compaction", preserves ID/Usage/ExtraFields, sets CreatedAt
to time.Now().Unix() if zero, and ensures Output is a non-nil []ResponsesMessage
(copy resp.Output if present, otherwise empty slice). Name the method
WithDefaults on type BifrostCompactionResponse to match other response types so
the pipeline can call it consistently.
In `@framework/modelcatalog/pricing.go`:
- Around line 339-340: Passthrough requests for the compaction endpoint aren’t
being classified as schemas.CompactionRequest, so /v1/responses/compact can fall
through to chat and be billed via computeTextCost incorrectly; update the
passthrough/type-inference code that maps request paths (the function that
infers passthrough types) to return schemas.CompactionRequest for the
"/v1/responses/compact" route, ensure the billing switch that calls
computeTextCost(...) already handles schemas.CompactionRequest, and add a unit
test verifying that a passthrough request to "/v1/responses/compact" is
classified as schemas.CompactionRequest (not chat) so it uses the compaction
billing path.
In `@transports/bifrost-http/handlers/inference.go`:
- Around line 699-700: Add the missing "compaction" request type to the
transport config schema: update the enum at
$defs.pricing_override_request_type.enum to include "compaction" and add a
corresponding property named "compaction" under
$defs.custom_provider_config.allowed_requests.properties so POST
"/v1/responses/compact" is explicitly allowed (preserve existing keys like
max_request_body_size_mb and large_payload_optimization when adding the new
entry).
- Around line 1536-1556: In prepareCompactionRequest, ensure we validate that
the compaction payload contains at least an input or a PreviousResponseID before
returning the BifrostCompactionRequest: after you normalize
ResponsesRequestInputArray from ResponsesRequestInputStr (the input variable),
if input is nil/empty and req.PreviousResponseID (or PreviousResponseID on the
returned BifrostCompactionRequest) is also empty, return a validation error
(HTTP 400 / appropriate handler error) instead of forwarding the request; update
the function to perform this check and return early with an error when both are
missing.
In `@transports/bifrost-http/integrations/openai.go`:
- Around line 842-852: The pre-hook openAILargePayloadPreHook is used for
compaction requests but hydrateOpenAIRequestFromLargePayloadMetadata doesn't
handle *openai.OpenAICompactionRequest, so compaction requests can end up with
an empty Model; update hydrateOpenAIRequestFromLargePayloadMetadata to detect
and populate openai.OpenAICompactionRequest (setting the Model and any other
required fields from the large-payload metadata or hydrated payload), ensure
GetRequestModel (the closure returning r.Model) sees the filled value, and add a
fallback/error when Model remains empty to avoid broken compaction routing.
---
Outside diff comments:
In `@core/schemas/provider.go`:
- Around line 326-400: The AllowedRequests Go struct and IsOperationAllowed
already handle Compaction, but transports/config.schema.json is missing the
compaction field and enum value; update the JSON schema by adding "compaction":
{ "type": "boolean" } to $defs.custom_provider_config.allowed_requests and add
"compaction" to $defs.pricing_override_request_type (or equivalent enum for
RequestType) so custom_provider_config.allowed_requests.compaction is
validated/persisted and compaction can be targeted in pricing overrides; ensure
the schema description and examples (if any) are consistent with other boolean
flags.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3429ea98-2e1f-4e6a-94dc-62eabb87823c
📒 Files selected for processing (37)
core/bifrost.gocore/providers/anthropic/anthropic.gocore/providers/azure/azure.gocore/providers/bedrock/bedrock.gocore/providers/cerebras/cerebras.gocore/providers/cohere/cohere.gocore/providers/elevenlabs/elevenlabs.gocore/providers/fireworks/fireworks.gocore/providers/gemini/gemini.gocore/providers/groq/groq.gocore/providers/huggingface/huggingface.gocore/providers/mistral/mistral.gocore/providers/nebius/nebius.gocore/providers/ollama/ollama.gocore/providers/openai/openai.gocore/providers/openai/responses.gocore/providers/openrouter/openrouter.gocore/providers/parasail/parasail.gocore/providers/perplexity/perplexity.gocore/providers/replicate/replicate.gocore/providers/runway/runway.gocore/providers/sgl/sgl.gocore/providers/vertex/vertex.gocore/providers/vllm/vllm.gocore/providers/xai/xai.gocore/schemas/bifrost.gocore/schemas/provider.gocore/schemas/responses.goframework/modelcatalog/pricing.goframework/modelcatalog/utils.goplugins/logging/operations.goplugins/logging/utils.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/integrations/openai.gotransports/bifrost-http/integrations/router.goui/app/workspace/logs/sheets/logDetailView.tsxui/lib/constants/logs.ts
TejasGhatte
force-pushed
the
06-04-feat_openai_compaction_support
branch
from
June 4, 2026 12:33
c8cdf9c to
7c72bca
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (1)
transports/bifrost-http/handlers/inference.go (1)
1536-1538:⚠️ Potential issue | 🟠 Major | ⚡ Quick winAllow compaction requests to pass through without mandatory
input.This guard rejects valid compaction payloads for this endpoint and breaks the intended passthrough validation boundary.
💡 Proposed fix
- if len(req.Input.ResponsesRequestInputArray) == 0 && req.Input.ResponsesRequestInputStr == nil { - return nil, nil, fmt.Errorf("input is required for compaction") - }Based on learnings: In maximhq/bifrost,
POST /v1/responses/compactintentionally does NOT requireinputorprevious_response_idat handler level; validation is delegated to the upstream provider.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@transports/bifrost-http/handlers/inference.go` around lines 1536 - 1538, Remove the mandatory input guard that rejects requests when both req.Input.ResponsesRequestInputArray is empty and req.Input.ResponsesRequestInputStr is nil for the compaction endpoint; specifically, in the handler that serves POST /v1/responses/compact (the code around inference.go using req.Input.ResponsesRequestInputArray and req.Input.ResponsesRequestInputStr), either delete this if-block or wrap it so it only runs for non-compaction paths, allowing compaction requests to pass through and letting upstream providers validate presence of input/previous_response_id.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/providers/openai/openai.go`:
- Around line 4151-4154: The code currently calls logger.Debug with the full
upstream response body (logger.Debug("error from %s provider: %s", providerName,
string(resp.Body()))) which can leak sensitive data; remove or replace that call
with a safe, non-sensitive log (e.g., log providerName, resp.StatusCode(), and
optionally a redacted/truncated indicator or body length). In the block around
providerUtils.MaterializeStreamErrorBody(ctx, resp) and the return that calls
providerUtils.EnrichError(ctx, ParseOpenAIError(resp), jsonData, nil,
sendBackRawRequest, sendBackRawResponse), delete the raw resp.Body() logging and
instead log only non-sensitive metadata (providerName and status code or
"response body omitted/redacted") or a truncated/hashed summary if you must
record something for debugging; keep MaterializeStreamErrorBody and EnrichError
behavior unchanged.
- Around line 4061-4078: HandleOpenAICompactionRequest is currently logging the
full upstream error body (logger.Debug("error from %s provider: %s", ...,
string(resp.Body()))) for non-200 responses when calling POST
/v1/responses/compact; change this to avoid leaking sensitive content by either
truncating the body to a small safe length, or preferably only logging the
parsed ParseOpenAIError fields (e.g., error.message, error.type, error.param,
error.code) and a short hex/sha256 of the raw body if needed for correlation;
update the logging call in HandleOpenAICompactionRequest to use those sanitized
fields and remove any direct string(resp.Body()) usage, and ignore the reviewer
note about disableStore parity since BifrostCompactionRequest /
OpenAICompactionRequest and ToOpenAICompactionRequest do not send a store field.
In `@core/providers/openai/responses.go`:
- Around line 450-466: The conversion in ToBifrostCompactionRequest currently
only sets Input to r.Input.OpenAIResponsesRequestInputArray, dropping
string-form inputs; update ToBifrostCompactionRequest to preserve the string
form by setting Input to r.Input.OpenAIResponsesRequestInputArray when
non-empty, otherwise use the string input (e.g.,
r.Input.OpenAIResponsesRequestInputString or r.Input.String) so that
OpenAICompactionRequest -> BifrostCompactionRequest retains either the array or
the original string input; keep all other fields (Provider, Model, Instructions,
PreviousResponseID, PromptCacheKey, PromptCacheRetention, ServiceTier)
unchanged.
In `@transports/config.schema.json`:
- Around line 4629-4630: The schema update added "compaction" to
$defs.pricing_override_request_type but
provider_pricing_override.request_types.items still allows any string so the
enum isn't enforced; update provider_pricing_override.request_types.items to
reference the shared definition ($defs.pricing_override_request_type) or to use
the same enum values so validation will reject unknown request types —
specifically change the items schema under
provider_pricing_override.request_types to $ref:
"`#/`$defs/pricing_override_request_type" (or mirror that enum) to make the new
"compaction" value effective.
---
Duplicate comments:
In `@transports/bifrost-http/handlers/inference.go`:
- Around line 1536-1538: Remove the mandatory input guard that rejects requests
when both req.Input.ResponsesRequestInputArray is empty and
req.Input.ResponsesRequestInputStr is nil for the compaction endpoint;
specifically, in the handler that serves POST /v1/responses/compact (the code
around inference.go using req.Input.ResponsesRequestInputArray and
req.Input.ResponsesRequestInputStr), either delete this if-block or wrap it so
it only runs for non-compaction paths, allowing compaction requests to pass
through and letting upstream providers validate presence of
input/previous_response_id.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 839df775-86d7-4786-9417-5d1b3e927740
📒 Files selected for processing (42)
core/bifrost.gocore/internal/llmtests/account.gocore/internal/llmtests/compaction.gocore/internal/llmtests/tests.gocore/providers/anthropic/anthropic.gocore/providers/azure/azure.gocore/providers/bedrock/bedrock.gocore/providers/cerebras/cerebras.gocore/providers/cohere/cohere.gocore/providers/elevenlabs/elevenlabs.gocore/providers/fireworks/fireworks.gocore/providers/gemini/gemini.gocore/providers/groq/groq.gocore/providers/huggingface/huggingface.gocore/providers/mistral/mistral.gocore/providers/nebius/nebius.gocore/providers/ollama/ollama.gocore/providers/openai/openai.gocore/providers/openai/openai_test.gocore/providers/openai/responses.gocore/providers/openrouter/openrouter.gocore/providers/parasail/parasail.gocore/providers/perplexity/perplexity.gocore/providers/replicate/replicate.gocore/providers/runway/runway.gocore/providers/sgl/sgl.gocore/providers/vertex/vertex.gocore/providers/vllm/vllm.gocore/providers/xai/xai.gocore/schemas/bifrost.gocore/schemas/provider.gocore/schemas/responses.goframework/modelcatalog/pricing.goframework/modelcatalog/utils.goplugins/logging/operations.goplugins/logging/utils.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/integrations/openai.gotransports/bifrost-http/integrations/router.gotransports/config.schema.jsonui/app/workspace/logs/sheets/logDetailView.tsxui/lib/constants/logs.ts
TejasGhatte
force-pushed
the
06-04-feat_openai_compaction_support
branch
from
June 4, 2026 13:13
7c72bca to
49a9c29
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/bifrost.go`:
- Around line 924-925: The exported doc comment for CompactionRequest is
outdated and should be updated to accurately describe which providers support
compaction; edit the top-of-declaration comment for the exported
CompactionRequest type/function so it begins with "CompactionRequest" (Go doc
convention) and clearly lists the current supported providers and behavior for
others (e.g., "supported by OpenAI and <OtherProviderName>; other providers
return an unsupported-operation error"), and include any relevant details about
the /v1/responses/compact endpoint and expected semantics so SDK users and
maintainers see the true provider surface.
- Around line 6229-6234: Before calling provider.Compaction in the
schemas.CompactionRequest branch, check the provider's allowed_requests
compaction flag and refuse dispatch if compaction is not allowed; specifically,
inspect the provider's custom config (e.g.,
provider.CustomProviderConfig.AllowedRequests.Compaction or
provider.AllowedRequests.Compaction) and return an appropriate bifrost error
when false instead of invoking provider.Compaction, otherwise proceed to call
provider.Compaction and set response.CompactionResponse as before.
In `@core/providers/azure/azure.go`:
- Around line 2694-2706: completeRequest may set large-response streaming mode
which leaves responseBody nil; before calling
providerUtils.HandleProviderResponse, check the context flag used by Azure
handlers (ctx.Value(schemas.BifrostContextKeyLargeResponseMode)) and
short-circuit when it's set (i.e., return immediately the same way other Azure
unary handlers do so the streaming path handles the response) instead of
attempting to unmarshal a nil responseBody; add this check right after
provider.completeRequest returns and before calling
providerUtils.HandleProviderResponse in the function that contains
provider.completeRequest/response handling.
In `@core/providers/openai/responses.go`:
- Around line 146-149: The loop that early-continues for non-reasoning models is
still skipping compaction items before the later guard preserves
encrypted_content; update the earlier skip condition to exempt compaction items
by checking message.Type against schemas.ResponsesMessageTypeCompaction (i.e.,
treat isCompactionMessage as a pass-through), so compaction messages with
content blocks reach the branch that preserves
ResponsesReasoning.EncryptedContent; adjust the condition that uses isReasoning
and the continue to include a check like "and not isCompactionMessage" (using
the existing message.Type / schemas.ResponsesMessageTypeCompaction symbols) so
compaction items are not dropped.
- Around line 431-447: To avoid bypassing OpenAI-specific message normalization,
update ToOpenAICompactionRequest so it does not assign req.Input verbatim;
instead reuse the same conversion/normalization used by ToOpenAIResponsesRequest
(extract or call the shared helper that shapes inputs into
OpenAIResponsesRequestInput) to produce the OpenAIResponsesRequestInput shape
(preserving role cleanup and compaction-content conversion) and assign that to
r.Input; reference the ToOpenAICompactionRequest and ToOpenAIResponsesRequest
conversion path (or create a small shared helper like
NormalizeOpenAIResponsesInput used by both) and replace the direct
OpenAIResponsesRequestInput{...: req.Input} assignment with a call to that
normalization function.
In `@transports/bifrost-http/handlers/inference.go`:
- Around line 1536-1538: Remove the hard input requirement that rejects
compaction payloads by deleting or disabling the conditional that returns an
error when both req.Input.ResponsesRequestInputArray and
req.Input.ResponsesRequestInputStr are empty/nil in the compaction handler in
inference.go; allow the compaction request to proceed with nil/empty input (so
previous_response_id-only requests are accepted) and rely on upstream validation
instead of returning fmt.Errorf("input is required for compaction") from that
check.
In `@transports/bifrost-http/integrations/openai.go`:
- Around line 846-847: The compaction route currently assigns PreCallback:
openAILargePayloadPreHook but that hook doesn't set the
BifrostContextKeyIsAzureUserAgent flag, so Azure SDK requests can be misrouted;
update the compaction route to ensure Azure-compatible routing by either (a)
modifying openAILargePayloadPreHook to set BifrostContextKeyIsAzureUserAgent
when the request matches Azure user-agent/headers, or (b) wrap/replace the
PreCallback for the compaction handler so it calls the existing Azure-aware
prehook used by /responses (preserving current behavior) before delegating to
openAILargePayloadPreHook; target the PreCallback assignment and ensure
BifrostContextKeyIsAzureUserAgent is set consistently with the /responses flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: fca686f6-afc3-4954-a14c-8c7a0df1da2a
📒 Files selected for processing (44)
core/bifrost.gocore/internal/llmtests/account.gocore/internal/llmtests/compaction.gocore/internal/llmtests/tests.gocore/providers/anthropic/anthropic.gocore/providers/azure/azure.gocore/providers/azure/azure_test.gocore/providers/bedrock/bedrock.gocore/providers/cerebras/cerebras.gocore/providers/cohere/cohere.gocore/providers/elevenlabs/elevenlabs.gocore/providers/fireworks/fireworks.gocore/providers/gemini/gemini.gocore/providers/groq/groq.gocore/providers/huggingface/huggingface.gocore/providers/mistral/mistral.gocore/providers/nebius/nebius.gocore/providers/ollama/ollama.gocore/providers/openai/openai.gocore/providers/openai/openai_test.gocore/providers/openai/responses.gocore/providers/openrouter/openrouter.gocore/providers/parasail/parasail.gocore/providers/perplexity/perplexity.gocore/providers/replicate/replicate.gocore/providers/runway/runway.gocore/providers/sgl/sgl.gocore/providers/vertex/vertex.gocore/providers/vllm/vllm.gocore/providers/xai/xai.gocore/providers/xai/xai_test.gocore/schemas/bifrost.gocore/schemas/provider.gocore/schemas/responses.goframework/modelcatalog/pricing.goframework/modelcatalog/utils.goplugins/logging/operations.goplugins/logging/utils.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/integrations/openai.gotransports/bifrost-http/integrations/router.gotransports/config.schema.jsonui/app/workspace/logs/sheets/logDetailView.tsxui/lib/constants/logs.ts
TejasGhatte
force-pushed
the
06-04-feat_openai_compaction_support
branch
from
June 4, 2026 14:33
49a9c29 to
e20e7d2
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
ui/app/workspace/logs/sheets/logDetailView.tsx (1)
2244-2285:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winInclude canonical Responses object aliases in copy-request detection.
Line 2244 adds
compaction, butisResponsesstill misses common response object values likeresponses/responses_stream, so copy-request can still be incorrectly blocked while Line 2284 says responses are supported.Suggested fix
- const isResponses = log.object === "response" || log.object === "response.completion.chunk" || log.object === "compaction"; + const isResponses = + log.object === "response" || + log.object === "response.completion.chunk" || + log.object === "responses" || + log.object === "responses_stream" || + log.object === "websocket_responses" || + log.object === "compaction";🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@ui/app/workspace/logs/sheets/logDetailView.tsx` around lines 2244 - 2285, The copy-request detection misses canonical Responses aliases: update the isResponses expression to include the additional object values "responses" and "responses_stream" (alongside the existing "response", "response.completion.chunk", and "compaction") so that isResponses (and thus isSupportedType) correctly treats those logs as supported for copy-request; locate the isResponses constant in logDetailView.tsx and add those two string checks to its condition.
♻️ Duplicate comments (1)
transports/config.schema.json (1)
4629-4630:⚠️ Potential issue | 🟠 Major | ⚡ Quick win
compactionenum addition is not enforced in pricing overrides
$defs.pricing_override_request_typenow includescompaction, butprovider_pricing_override.request_types.items(Line 4588) is still"type": "string", so invalid request types still pass schema validation.Suggested fix
"request_types": { "type": "array", "description": "Request types this override applies to. At least one value is required.", "minItems": 1, "items": { - "type": "string" + "$ref": "`#/`$defs/pricing_override_request_type" } },As per coding guidelines,
transports/config.schema.jsonis the source of truth for config fields.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@transports/config.schema.json` around lines 4629 - 4630, The schema added "compaction" to $defs.pricing_override_request_type but did not enforce it in provider_pricing_override.request_types.items; update provider_pricing_override.request_types.items to reference the canonical enum ($ref to $defs.pricing_override_request_type) or replace its "type": "string" with an "enum" that matches $defs.pricing_override_request_type so only valid request types (including "compaction") pass validation; target the provider_pricing_override.request_types.items node to make this change.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/bifrost.go`:
- Around line 927-949: In Bifrost.CompactionRequest, add an early guard that
checks req.Input (the input field on the incoming
*schemas.BifrostCompactionRequest) and return a *schemas.BifrostError with
IsBifrostError:false, Error: &schemas.ErrorField{Message: "compaction input is
nil"} and ExtraFields.RequestType set to schemas.CompactionRequest when Input is
nil/empty; keep this check before creating bifrostReq or calling
bifrost.handleRequest so the exported API enforces the same contract as the HTTP
endpoint.
In `@transports/bifrost-http/handlers/inference.go`:
- Around line 1531-1560: The compaction path drops provider-specific extra
params: prepareRequest fills base.ExtraParams but the returned
schemas.BifrostCompactionRequest does not include them; update the code that
constructs the BifrostCompactionRequest in the compaction handler to copy/assign
base.ExtraParams (or a properly typed clone) into the request (e.g., set the
ExtraParams/Extra fields on the returned BifrostCompactionRequest to
base.ExtraParams) so unknown/sdk-specific fields are preserved when dispatching;
keep the existing Input conversion logic and ensure the field name matches the
schemas.BifrostCompactionRequest definition.
---
Outside diff comments:
In `@ui/app/workspace/logs/sheets/logDetailView.tsx`:
- Around line 2244-2285: The copy-request detection misses canonical Responses
aliases: update the isResponses expression to include the additional object
values "responses" and "responses_stream" (alongside the existing "response",
"response.completion.chunk", and "compaction") so that isResponses (and thus
isSupportedType) correctly treats those logs as supported for copy-request;
locate the isResponses constant in logDetailView.tsx and add those two string
checks to its condition.
---
Duplicate comments:
In `@transports/config.schema.json`:
- Around line 4629-4630: The schema added "compaction" to
$defs.pricing_override_request_type but did not enforce it in
provider_pricing_override.request_types.items; update
provider_pricing_override.request_types.items to reference the canonical enum
($ref to $defs.pricing_override_request_type) or replace its "type": "string"
with an "enum" that matches $defs.pricing_override_request_type so only valid
request types (including "compaction") pass validation; target the
provider_pricing_override.request_types.items node to make this change.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 599eb010-bc76-45e3-a559-5bfe81b237a9
📒 Files selected for processing (44)
core/bifrost.gocore/internal/llmtests/account.gocore/internal/llmtests/compaction.gocore/internal/llmtests/tests.gocore/providers/anthropic/anthropic.gocore/providers/azure/azure.gocore/providers/azure/azure_test.gocore/providers/bedrock/bedrock.gocore/providers/cerebras/cerebras.gocore/providers/cohere/cohere.gocore/providers/elevenlabs/elevenlabs.gocore/providers/fireworks/fireworks.gocore/providers/gemini/gemini.gocore/providers/groq/groq.gocore/providers/huggingface/huggingface.gocore/providers/mistral/mistral.gocore/providers/nebius/nebius.gocore/providers/ollama/ollama.gocore/providers/openai/openai.gocore/providers/openai/openai_test.gocore/providers/openai/responses.gocore/providers/openrouter/openrouter.gocore/providers/parasail/parasail.gocore/providers/perplexity/perplexity.gocore/providers/replicate/replicate.gocore/providers/runway/runway.gocore/providers/sgl/sgl.gocore/providers/vertex/vertex.gocore/providers/vllm/vllm.gocore/providers/xai/xai.gocore/providers/xai/xai_test.gocore/schemas/bifrost.gocore/schemas/provider.gocore/schemas/responses.goframework/modelcatalog/pricing.goframework/modelcatalog/utils.goplugins/logging/operations.goplugins/logging/utils.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/integrations/openai.gotransports/bifrost-http/integrations/router.gotransports/config.schema.jsonui/app/workspace/logs/sheets/logDetailView.tsxui/lib/constants/logs.ts
TejasGhatte
force-pushed
the
06-04-feat_openai_compaction_support
branch
from
June 4, 2026 16:49
e20e7d2 to
54f6513
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
core/bifrost.go (1)
940-952:⚠️ Potential issue | 🟠 Major | ⚡ Quick winEnforce mandatory
inputinCompactionRequestfor contract parity.Line 940 currently permits empty
inputwhenPreviousResponseIDis set, which makes the Go API behavior diverge from the compaction endpoint contract.🛠️ Suggested fix
- if len(req.Input) == 0 && req.PreviousResponseID == nil && !isLargePayloadPassthrough(ctx) { + if len(req.Input) == 0 && !isLargePayloadPassthrough(ctx) { return nil, &schemas.BifrostError{ IsBifrostError: false, Error: &schemas.ErrorField{ Message: "input not provided for compaction request", },Based on learnings, the compaction endpoint intentionally requires
inputto be present and returns an error when it is absent.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@core/bifrost.go` around lines 940 - 952, The compaction handler currently allows empty req.Input when req.PreviousResponseID is set, diverging from the compaction endpoint contract; change the validation in the compaction request path so that if req.Input is empty (len(req.Input) == 0) it always returns the schemas.BifrostError (same structure shown) regardless of req.PreviousResponseID or isLargePayloadPassthrough(ctx); locate the check around the compaction request handling (references: req.Input, req.PreviousResponseID, isLargePayloadPassthrough(ctx), schemas.BifrostError, RequestType: schemas.CompactionRequest) and remove the conditional exemption that permits empty input when PreviousResponseID is present so the error is consistently returned.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/providers/openai/responses.go`:
- Around line 485-494: The returned BifrostCompactionRequest from
(*OpenAICompactionRequest).ToBifrostCompactionRequest is missing the ExtraParams
field, so inbound req.ExtraParams are discarded; update
ToBifrostCompactionRequest to set ExtraParams: r.ExtraParams on the returned
schemas.BifrostCompactionRequest (referencing the ToBifrostCompactionRequest
method and schemas.BifrostCompactionRequest/ExtraParams field) so the map is
preserved when converting.
---
Duplicate comments:
In `@core/bifrost.go`:
- Around line 940-952: The compaction handler currently allows empty req.Input
when req.PreviousResponseID is set, diverging from the compaction endpoint
contract; change the validation in the compaction request path so that if
req.Input is empty (len(req.Input) == 0) it always returns the
schemas.BifrostError (same structure shown) regardless of req.PreviousResponseID
or isLargePayloadPassthrough(ctx); locate the check around the compaction
request handling (references: req.Input, req.PreviousResponseID,
isLargePayloadPassthrough(ctx), schemas.BifrostError, RequestType:
schemas.CompactionRequest) and remove the conditional exemption that permits
empty input when PreviousResponseID is present so the error is consistently
returned.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 8ad75d80-6b7f-4c06-94a4-c9e1e23527ff
📒 Files selected for processing (44)
core/bifrost.gocore/internal/llmtests/account.gocore/internal/llmtests/compaction.gocore/internal/llmtests/tests.gocore/providers/anthropic/anthropic.gocore/providers/azure/azure.gocore/providers/azure/azure_test.gocore/providers/bedrock/bedrock.gocore/providers/cerebras/cerebras.gocore/providers/cohere/cohere.gocore/providers/elevenlabs/elevenlabs.gocore/providers/fireworks/fireworks.gocore/providers/gemini/gemini.gocore/providers/groq/groq.gocore/providers/huggingface/huggingface.gocore/providers/mistral/mistral.gocore/providers/nebius/nebius.gocore/providers/ollama/ollama.gocore/providers/openai/openai.gocore/providers/openai/openai_test.gocore/providers/openai/responses.gocore/providers/openrouter/openrouter.gocore/providers/parasail/parasail.gocore/providers/perplexity/perplexity.gocore/providers/replicate/replicate.gocore/providers/runway/runway.gocore/providers/sgl/sgl.gocore/providers/vertex/vertex.gocore/providers/vllm/vllm.gocore/providers/xai/xai.gocore/providers/xai/xai_test.gocore/schemas/bifrost.gocore/schemas/provider.gocore/schemas/responses.goframework/modelcatalog/pricing.goframework/modelcatalog/utils.goplugins/logging/operations.goplugins/logging/utils.gotransports/bifrost-http/handlers/inference.gotransports/bifrost-http/integrations/openai.gotransports/bifrost-http/integrations/router.gotransports/config.schema.jsonui/app/workspace/logs/sheets/logDetailView.tsxui/lib/constants/logs.ts
TejasGhatte
force-pushed
the
06-04-feat_openai_compaction_support
branch
2 times, most recently
from
June 5, 2026 10:39
bb9a927 to
01b9003
Compare
TejasGhatte
force-pushed
the
06-04-feat_openai_compaction_support
branch
from
June 5, 2026 11:05
01b9003 to
c6586a0
Compare
18 tasks
Collaborator
Merge activity
|
Pratham-Mishra04
force-pushed
the
06-04-feat_openai_compaction_support
branch
from
June 5, 2026 13:36
c6586a0 to
21d3113
Compare
Pratham-Mishra04
force-pushed
the
06-04-feat_openai_compaction_support
branch
from
June 5, 2026 13:38
21d3113 to
b4b1697
Compare
18 tasks
akshaydeo
added a commit
that referenced
this pull request
Jun 6, 2026
## Summary This PR bumps the Go toolchain version from `1.26.3` to `1.26.4` across all modules and CI workflows, and cuts a new release (`core` v1.5.17, `framework` v1.3.17, `transports` v1.5.9, `plugins/compat` v0.1.16, `plugins/governance` v1.5.17, and associated plugin versions) incorporating a large batch of features and fixes accumulated since the previous release. ## Changes - **Go 1.26.4** — Updated `go-version` in all GitHub Actions workflows (`e2e-tests`, `helm-release`, `pr-tests`, `release-cli`, `release-pipeline`, `snyk`) and all `go.mod` files (core, framework, transports, cli, all plugins, examples, and test modules). - **Core (v1.5.17)** — OpenAI compaction support, multi-customer logs and usage tracking, multiple team/business unit support, `request_headers` wildcard pattern capture for OTel and Maxim plugins, xAI `x_search` tool, fetch URL validation with SSRF hardening, `file://` pricing URL scheme, virtual key provider fan-out filtering, and a broad set of fixes including Anthropic prompt cache key, empty thinking block stripping, OpenAI stream usage event cleanup, Gemini numeric schema constraints, stale connection retries, Azure Claude diagnostic strip, and passthrough budget handling. - **Framework (v1.3.17)** — Scope-aware budgets and limits wired from model configs, provider-level governance, multiple customer budget support with `calendar_aligned` windows, paginated virtual key fetch, `config.json` source-of-truth flow, FTS index cap reduction, sync worker drift fix, cascade deletes for model configs, and high-scale virtual key flow improvements. - **Transports (v1.5.9)** — Full changelog covering all of the above plus UI improvements (log navigation, customer detail sheet, `BudgetDisplay` component, inline loading shell, materialized view alias), SCIM provisioning fields, Helm/config schema additions (`roles`, `per_user_oauth`), client IP resolution from forwarded headers, and dependency upgrades (`recharts` to 3.8.1, `golang.org/x` CVE remediation). - **Plugins** — `governance` v1.5.17 adds team budget/rate-limit exporters, ghost node reconciliation fix, and VK double usage counting fix; `logging` v1.5.17 adds wildcard header capture and file attachment rendering; `otel` v1.2.17 adds `disable_content_logging` and multiple collectors support; `maxim` v1.6.17 adds `request_headers` wildcard capture; `compat` v0.1.16 fixes `max_tokens` preservation during param filtering. ## Type of change - [ ] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Verify Go version go version # should report go1.26.4 # Run core tests cd core && go test ./... # Run framework tests cd framework && go test ./... # Run transports tests cd transports && go test ./... # Run plugin tests cd plugins/governance && go test ./... cd plugins/logging && go test ./... cd plugins/otel && go test ./... # UI cd ui pnpm i pnpm build pnpm test ``` ## Breaking changes - [ ] Yes - [x] No ## Related issues #4053, #4066, #4041, #4012, #3976, #3947, #3991, #4045, #3957, #3938, #3937, #3939, #3981, #3998, #3997, #4092, #4091, #4079, #4080, #4086, #3929, #3994, #4028, #3970, #3919, #3861, #3664, #3999, #4088, #4070, #4051, #4043, #4057, #4023, #3941, #3955, #4024, #3956, #3967, #3925, #3992, #3900 ## Security considerations - Fetch URL validation hardened against SSRF by tightening IP checks for private networks and link-local addresses (#4092, #3947, #3991). - Transitive `golang.org/x` dependencies (crypto, net, sys, text) bumped to address Docker Scout CVEs (#3900). ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * OpenAI compaction, multi-customer/team logstore support, request-header wildcard capture, enhanced governance (provider-level & scope-aware limits), disable-content-logging option, support for multiple OpenTelemetry collectors, SSRF hardening and URL validation. * **Chores** * Bumped Go toolchain across modules and updated component/plugin version releases. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Merged
akshaydeo
pushed a commit
that referenced
this pull request
Jun 7, 2026
## Summary
Adds support for OpenAI's context compaction endpoint (`POST /v1/responses/compact`), which compacts a conversation context window to reduce token usage. All other providers return an unsupported-operation error for this request type.
## Changes
- Introduced `CompactionRequest` as a new `RequestType` constant and added `BifrostCompactionRequest` / `BifrostCompactionResponse` schema types as a strict subset of the responses API (no tools, sampling params, or streaming).
- Added `Compaction` to the `Provider` interface and implemented it for OpenAI via `HandleOpenAICompactionRequest`, routing to `/v1/responses/compact`. All other providers return `NewUnsupportedOperationError`.
- Wired `CompactionRequest` through `BifrostRequest` / `BifrostResponse` field accessors (`GetRequestFields`, `SetProvider`, `SetModel`, `SetFallbacks`, `SetRawRequestBody`, `GetExtraFields`, `PopulateExtraFields`) and through the fallback preparation and request dispatch logic in `bifrost.go`.
- Registered `POST /v1/responses/compact` in the HTTP transport with a dedicated `compaction` handler, `CompactionHTTPRequest` type, and `compactionParamsKnownFields` validation map.
- Added `CompactionResponseConverter` to `RouteConfig` and handled `CompactionRequest` in the generic router's non-streaming dispatch path.
- Added the OpenAI integration route config for `/v1/responses/compact` (and aliased paths), including `OpenAICompactionRequest` wire type with `ToOpenAICompactionRequest` / `ToBifrostCompactionRequest` converters.
- Extended model catalog pricing to treat `CompactionRequest` the same as `ResponsesRequest` for cost computation, usage extraction, and fallback pricing lookups, normalizing it to the `"responses"` base type.
- Extended the logging plugin to extract usage and output from `CompactionResponse` when building log entries.
- Added `"compaction"` to the UI request type list, labels, and color map (indigo), and updated the log detail view to recognize `"compaction"` as a responses-family object for copy-request-body support.
## Type of change
- [ ] Bug fix
- [x] Feature
- [ ] Refactor
- [ ] Documentation
- [ ] Chore/CI
## Affected areas
- [x] Core (Go)
- [x] Transports (HTTP)
- [x] Providers/Integrations
- [x] Plugins
- [x] UI (React)
- [ ] Docs
## How to test
```sh
# Core/Transports
go test ./...
# Send a compaction request via the HTTP transport
curl -X POST http://localhost:8080/v1/responses/compact \
-H "Content-Type: application/json" \
-d '{
"model": "openai/gpt-4o",
"input": [
{"role": "user", "content": "Hello, summarize this conversation."}
]
}'
# Expected: 200 with a BifrostCompactionResponse (object: "response.compaction")
# Verify unsupported providers return an error
curl -X POST http://localhost:8080/v1/responses/compact \
-H "Content-Type: application/json" \
-d '{"model": "anthropic/claude-opus-4-5", "input": []}'
# Expected: error indicating unsupported operation for the Anthropic provider
# UI
cd ui
pnpm i
pnpm build
```
## Breaking changes
- [x] Yes
- [ ] No
The `Provider` interface gains a new required method `Compaction`. Any custom provider implementations outside this repository must add a `Compaction` method returning `NewUnsupportedOperationError` (or a real implementation) to satisfy the interface.
## Related issues
## Security considerations
No new auth mechanisms are introduced. The compaction endpoint forwards the caller's API key as a Bearer token to OpenAI, consistent with all other provider requests. No PII handling changes.
## Checklist
- [ ] I read `docs/contributing/README.md` and followed the guidelines
- [ ] I added/updated tests where appropriate
- [ ] I updated documentation where needed
- [ ] I verified builds succeed (Go and UI)
- [ ] I verified the CI pipeline passes locally if applicable
<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit
* **New Features**
* Added a compaction operation with a public POST /v1/responses/compact endpoint; OpenAI and Azure compaction supported.
* **Integration**
* Pricing, usage tracking, logging, routing, and passthrough flows updated to surface compaction results, provider headers, and latency/metadata.
* **UI**
* Logs/UI now include a "Compaction" request type and copy/export behavior for compaction entries.
* **Tests**
* New toggleable end-to-end compaction tests validating response contract and follow-up usability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
akshaydeo
added a commit
that referenced
this pull request
Jun 7, 2026
## ✨ Features - **OpenAI Compaction** — Added OpenAI conversation compaction support across core, framework, logging, and the API surface (#4053) - **Multi-Customer & Org Hierarchy** — Logs and usage tracking now support multiple customers, teams, and business units, including business unit CRUD, team assignment, and governance endpoints in the OpenAPI spec (#4066, #4041, #4082) - **Provider-Level Governance** — Budgets & limits are now scope-aware and can be applied at the virtual-key top level and per provider, wired from the model configs table, with UI filters for scope and providers (#3938, #3937, #3939, #3981, #3962) - **Customer Budgets** — Customers support multiple budgets and `calendar_aligned` budget windows (#3998, #3997) - **Virtual Key Attribution & Controls** — Added a `created_by` user attribution column and a `blacklisted_models` column for virtual key provider configs (#3672, #3653) - **Request Header Capture** — OTel and Maxim observability plugins capture `request_headers` by pattern, with wildcard support (e.g. `x-custom-*`); logging gained the same wildcard header capture (#4012, #3958) - **OTel Content Controls & Collectors** — New `disable_content_logging` option drops message/tool content from exported spans, plus support for multiple OTel collectors (#4064, #3894) - **xAI x_search** — Added xAI `x_search` tool support (#3976) - **URL Validation** — Added fetch URL validation with private-network configuration and link-local blocking (#3947, #3991) - **File Scheme Pricing URLs** — Pricing source URLs now accept the `file://` scheme for air-gapped and self-hosted deployments (#4045) - **Paginated Virtual Keys** — Virtual key fetching is paginated to handle deployments with very large numbers of keys (#3957) - **Client IP Resolution** — Resolve client IP from `X-Forwarded-For`/`X-Real-IP` headers - **SCIM Provisioning** — Added `attributeType`/`attributeValue` SCIM provisioning fields - **Helm/Config Schema** — Added `roles` RBAC governance config and `per_user_oauth` MCP auth to the Helm chart and config schema (#4004, #4009) - **Log Navigation UI** — Added a "View logs" menu item to customer, team, and virtual key tables, clickable links in log detail views, a customer detail sheet, and a reusable `BudgetDisplay` component (#4073, #4054, #4026, #4055) - **Faster First Paint** — Added an inline loading shell to `#root` before React mounts (#4063) - **Materialized View Alias** — Added an `alias` column to the materialized view with filter support (#4078) ## 🐞 Fixed - **Fetch URL IP Checks** — Hardened fetch URL IP checks against SSRF (#4092) - **Mantle Model Matching** — Broadened Mantle model matching to all `gpt` variants (#4091) - **Empty Thinking Blocks** — Strip thinking blocks when the signature is empty (#4079) - **OpenAI Stream Usage** — Removed usage from the `responses.created` event in the OpenAI stream (#4080) - **Prompt Cache Key** — Set the prompt cache key from the Anthropic integration (#4086) - **Upstream Failure Status** — Map upstream connection failures to 502 instead of 400 (#3929) (thanks [@chris-colinsky](https://github.com/chris-colinsky)!) - **Gemini Schema Constraints** — Accept numeric schema integer constraints for Gemini (#3994) (thanks [@yanhao98](https://github.com/yanhao98)!) - **Files Provider Param** — Accept the `?provider=` query param on `GET /v1/files` (#3971) (thanks [@alexef](https://github.com/alexef)!) - **Optional Batch Model** — Made the `model` field optional on `POST /v1/batches` (#3973) (thanks [@alexef](https://github.com/alexef)!) - **Helm Azure Config** — Added missing `azure_key_config` fields to the Helm schema (#3996) (thanks [@axelray-dev](https://github.com/axelray-dev)!) - **Text Completion Chunk Model** — Added the missing `Model` field to `TextCompletionChunkResponse` (#3970) (thanks [@kuishou68](https://github.com/kuishou68)!) - **MCP Inline stdio Env** — MCP stdio server configs accept inline environment variable assignments (#3861) (thanks [@Shushmitaaaa](https://github.com/Shushmitaaaa)!) - **Orphaned Tool Results** — Orphaned tool results in the OpenAI to Anthropic conversion flow are no longer rejected by the Anthropic API (#3919) - **Node Usage Reconciliation** — Added a monotonic `inc_number` log cursor so node usage reconciliation does not skip late async log writes (#3664) - **Bedrock Output Assessments** — Corrected the type of `outputAssessments` in Bedrock responses (#4028) - **Model Pool Pricing Reloads** — Preserve non-pricing model pool entries across pricing reloads (#3999) - **Ghost Node Reconciliation** — Replicate the VK hierarchy flow for ghost node reconciliation (#4088) - **VK Double Usage Counting** — Fixed double usage counting when creating a virtual key (#4070) - **Model Config Lifecycle** — Cascade deletes for model configs and removal of stale in-memory model configs (#4051, #4043) - **FTS Index Cap** — Reduced the FTS index `left()` cap from 800k to 250k chars to stay within the tsvector limit (#4057) - **Sync Worker Drift** — Reduced the sync worker ticker period to 5m to prevent threshold drift (#4023) - **Passthrough** — Fixed passthrough budgets, gated passthrough models per VK, model extraction for Azure passthrough, and restricted fallbacks/provider selection to the VK boundary (#3941, #3988, #3983, #3924) - **Provider Response Headers** — Strip provider response headers and add a content-type filter (#3955, #4024) - **Stream Handling** — Drain non-SSE stream readers and retry stale connections (#3956, #3967) - **Azure Claude** — Strip Azure diagnostic property for Claude models (#3925) - **Compat max_tokens** — Preserve chat `max_tokens` during param filtering (#3992) - **Raw Request Flag** — Removed the raw request flag from providers that don't support it (#4058) - **UI Fixes** — Standardized page container layout, virtual key model configs UI, and dashboard chart tooltips (#4046, #4052, #4044) ## 🔧 Maintenance - **Dependency Upgrades** — Bumped transitive `golang.org/x` dependencies (crypto, net, sys, text) for Docker Scout CVE remediation and `recharts` to 3.8.1; cascaded version bumps across all modules (#3900, #4003)
11 tasks
18 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for OpenAI's context compaction endpoint (
POST /v1/responses/compact), which compacts a conversation context window to reduce token usage. All other providers return an unsupported-operation error for this request type.Changes
CompactionRequestas a newRequestTypeconstant and addedBifrostCompactionRequest/BifrostCompactionResponseschema types as a strict subset of the responses API (no tools, sampling params, or streaming).Compactionto theProviderinterface and implemented it for OpenAI viaHandleOpenAICompactionRequest, routing to/v1/responses/compact. All other providers returnNewUnsupportedOperationError.CompactionRequestthroughBifrostRequest/BifrostResponsefield accessors (GetRequestFields,SetProvider,SetModel,SetFallbacks,SetRawRequestBody,GetExtraFields,PopulateExtraFields) and through the fallback preparation and request dispatch logic inbifrost.go.POST /v1/responses/compactin the HTTP transport with a dedicatedcompactionhandler,CompactionHTTPRequesttype, andcompactionParamsKnownFieldsvalidation map.CompactionResponseConvertertoRouteConfigand handledCompactionRequestin the generic router's non-streaming dispatch path./v1/responses/compact(and aliased paths), includingOpenAICompactionRequestwire type withToOpenAICompactionRequest/ToBifrostCompactionRequestconverters.CompactionRequestthe same asResponsesRequestfor cost computation, usage extraction, and fallback pricing lookups, normalizing it to the"responses"base type.CompactionResponsewhen building log entries."compaction"to the UI request type list, labels, and color map (indigo), and updated the log detail view to recognize"compaction"as a responses-family object for copy-request-body support.Type of change
Affected areas
How to test
Breaking changes
The
Providerinterface gains a new required methodCompaction. Any custom provider implementations outside this repository must add aCompactionmethod returningNewUnsupportedOperationError(or a real implementation) to satisfy the interface.Related issues
Security considerations
No new auth mechanisms are introduced. The compaction endpoint forwards the caller's API key as a Bearer token to OpenAI, consistent with all other provider requests. No PII handling changes.
Checklist
docs/contributing/README.mdand followed the guidelinesSummary by CodeRabbit
New Features
Integration
UI
Tests