Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
d36435e
feat(providers): add Minimax provider
octo-patch Jul 5, 2026
26bb5ff
upgrades clickhouse library version
akshaydeo Jul 6, 2026
c17bf16
upgrades clickhouse library version (#4977)
akshaydeo Jul 6, 2026
35945b1
allow v2.0.0 to publish prerelease cuts
akshaydeo Jul 6, 2026
67a2bf2
allow v2.0.0 to publish prerelease cuts (#4978)
akshaydeo Jul 6, 2026
d307d24
updates clickhouse main library
akshaydeo Jul 6, 2026
24ac12e
updates clickhouse main library (#4979)
akshaydeo Jul 6, 2026
f793a81
feat(logging): add transient redaction data field and context key for…
Madhuvod Jul 7, 2026
54eba48
feat(logging): redact trace content before connector export (#4417)
Madhuvod Jul 7, 2026
d9577ba
docs for redaction (#4565)
Madhuvod Jul 7, 2026
941461c
[fix]: preserve streaming finish_reason in the accumulated response w…
fus3r Jul 7, 2026
a632065
fix: pass azure auth headers in helpers (#4999)
TejasGhatte Jul 7, 2026
5210275
fix(redaction): phase scoped redaction and revealing (#5007)
Madhuvod Jul 7, 2026
0f69096
fix(openai): serialize compaction request `input` correctly (#5014)
mcclurmc Jul 7, 2026
14fa8fa
fix(schemas): add ExtraContent to ChatStreamResponseChoiceDelta (#4569)
nghodkicisco Jul 8, 2026
f5629b0
added enterprise fallback pages for alerting (#4685)
SahilChoudhary22 Jul 8, 2026
c20f383
feat(ui): add Microsoft Teams icon and alert API tags (#4826)
SahilChoudhary22 Jul 8, 2026
1af7722
fix(transcription): support diarized_json segments, fix ElevenLabs sp…
Shaik-Sirajuddin Jul 8, 2026
fe42c74
fix: pass container block from anthropic api (#5024)
TejasGhatte Jul 9, 2026
fae994f
feat: force single region config in vertex key config (#5035)
TejasGhatte Jul 9, 2026
dee8ffb
fix: skip disabled keys when scheduling model-discovery fetches (#5046)
Shaik-Sirajuddin Jul 9, 2026
76c731a
fix: fixes race conditions in tracer related to span locks (#5023)
roroghost17 Jul 9, 2026
eae2840
fix: fixes telemetry plugin cardinality explosion risk (#5041)
roroghost17 Jul 9, 2026
efa59f5
fix: fixes OTEL metrics not sending status code (#5043)
roroghost17 Jul 9, 2026
fe69407
feat: add allowlist for private-use redirect URI schemes (RFC 8252 §7…
Pratham-Mishra04 Jul 9, 2026
838d5a2
feat: add `shouldSweep` gate to OAuth2 sweep worker and expose `Start…
Pratham-Mishra04 Jul 9, 2026
08dc5aa
Forward ScopedDB from HybridLogStore (#5052)
SahilChoudhary22 Jul 9, 2026
9ec346d
changed alerting icon from bell to a siren (#5054)
SahilChoudhary22 Jul 9, 2026
00b0533
docs: Bigquery integration docs (#5055)
roroghost17 Jul 9, 2026
2f5e052
chore: fixes OTEL tests and strengthens harness (#5056)
roroghost17 Jul 9, 2026
8567ee8
chore: adds metrics vs logs sync check and tests for telemetry plugin…
roroghost17 Jul 9, 2026
c91adf3
feat: add durable background-job `sidekiq` table, store methods, and …
BearTS Jul 9, 2026
30ebc95
feat: show canonical model names in dashboard model rankings (#4941)
satyamkrishna Jul 9, 2026
db0c813
add model catalog pricing (#5033)
johnbrett Jul 9, 2026
24221ec
fix: forwards request id and trace id through telemetry (#5058)
roroghost17 Jul 9, 2026
a372102
feat: add `GetInFlightSidekiqJobByKind` to config store interface (#5…
impoiler Jul 9, 2026
e1621bd
feat: migrate cost recalculation to durable background sidekiq job wi…
impoiler Jul 9, 2026
384dfe3
feat: add connector test cases and missing attributes (#5059)
roroghost17 Jul 9, 2026
ee63540
feat: replace SSE streaming with background job polling for cost reca…
impoiler Jul 9, 2026
13e304a
docs: add PubSub connector docs (#5060)
roroghost17 Jul 9, 2026
1f662f8
fix: match model filter on canonical_model_name and restore routing i…
impoiler Jul 9, 2026
b3853f1
fix: preserve active time period when applying dimension filters (#5030)
impoiler Jul 9, 2026
d62abd1
feat: add `schema_url`/`BIFROST_SCHEMA_URL` support for mirrored sche…
impoiler Jul 9, 2026
e29d968
feat: add S3/GCS object storage config schema for audit log archival …
impoiler Jul 9, 2026
c1dc9bf
feat: add object storage archival docs for audit logs (#4992)
impoiler Jul 9, 2026
49d6e21
feat: add created by user id to sidekiq (#5066)
BearTS Jul 9, 2026
ee815d0
fix: UI package upgrades (#5067)
impoiler Jul 9, 2026
2e0ef07
fix: send mime type when specified (#5048)
TejasGhatte Jul 9, 2026
2164080
feat: pass `createdBy` user ID to sidekiq enqueue and add runner ID t…
impoiler Jul 9, 2026
61820b7
fix: preserve citation annotations in the accumulated streaming respo…
fus3r Jul 9, 2026
2bd7ffe
fix: annthropic cache rate for fast mode (#5063)
TejasGhatte Jul 9, 2026
c911b56
fix: inference geo cost on anthropic (#5072)
TejasGhatte Jul 9, 2026
3743a0c
fix: service tier in openai chat completion (#5073)
TejasGhatte Jul 9, 2026
3fbd9a4
fix: api key provider selection (#5076)
TejasGhatte Jul 9, 2026
e4bf522
adds alerting schema to config.schema.json (#5078)
SahilChoudhary22 Jul 10, 2026
abed892
feat: add audit log object storage (S3/GCS), `calendar_aligned` field…
BearTS Jul 10, 2026
53d47e7
feat: add `bifrost.alerting` to Helm chart with declarative channels …
BearTS Jul 10, 2026
87beb17
chore: release Bifrost Helm chart v2.1.27 (#5080)
BearTS Jul 10, 2026
e0d2ac1
redaction: added ClearPausedStreamBuffer method in core to support re…
Madhuvod Jul 10, 2026
65f33df
fix(transports): emit structured plugin stream errors on integration …
jeffhos Jul 10, 2026
ea04627
fix(providers): complete MiniMax integration
octo-patch Jul 12, 2026
ef1a8b5
chore: merge latest dev
octo-patch Jul 12, 2026
5d5a1e8
docs(providers): clarify MiniMax Anthropic endpoint
octo-patch Jul 12, 2026
90679c5
fix: reset web search flag when recycling pooled gemini responses str…
fus3r Jul 12, 2026
3c77641
fix(bedrock): preserve reasoning_config on cross-provider translation…
Purvi09 Jul 12, 2026
a3fb95e
feat: adds addition_tools message type support (#5103)
Pratham-Mishra04 Jul 12, 2026
095c69d
fix(anthropic): sanitize tool_use/tool_result ids to Anthropic's char…
Shaik-Sirajuddin Jul 12, 2026
8147100
[fix]: strip role from non-message OpenAI Responses input items (#5102)
nettee Jul 12, 2026
c38dcd2
fix: round-trip anthropic redacted_thinking blocks on the responses s…
fus3r Jul 12, 2026
322a734
fix(ui): show real global default in MCP tool execution timeout place…
Shaik-Sirajuddin Jul 12, 2026
23e61ec
[fix]: zero pooled ChannelMessage references on release (#4871)
citrocat Jul 12, 2026
8caf4dd
fix: persist Responses stream usage when providers omit or reuse sequ…
eyeveil Jul 12, 2026
0109d4f
fix: enable rerank for custom OpenAI-compatible providers (#4897)
eyeveil Jul 12, 2026
49433d5
fix(providers/anthropic): forward and rebuild server-side tool_search…
ws4charlie Jul 12, 2026
cc68973
feat: add ElevenLabs sound effects (text-to-sound) support (#4726)
SecretSun Jul 12, 2026
bbd693b
fix: attribute passthrough virtual keys sent via Azure api-key header…
eyeveil Jul 12, 2026
6da4987
fix: support OpenShift arbitrary UIDs — build-time group-0 ownership,…
eyeveil Jul 12, 2026
1bacabf
fix: synthesize per-query rerank usage for Bedrock and Vertex (#4322)
kypkk Jul 12, 2026
ec1d90a
fix: less strict unmarshalling for secret var (#4723)
BearTS Jul 12, 2026
b9d2aaf
[fix]: configstore - repair bare wildcard allowed_models rows that br…
eyeveil Jul 12, 2026
81d5723
Revert "fix: synthesize per-query rerank usage for Bedrock and Vertex…
akshaydeo Jul 12, 2026
1d4fb54
fix: make tracing span lookup nil-safe to prevent panic on streaming …
eyeveil Jul 12, 2026
3ae255c
Add Trendshift badge to README (#5124)
akshaydeo Jul 12, 2026
dbe8ab9
dds new harness skill and updates based on merged PRs (#5123)
akshaydeo Jul 12, 2026
f4f7716
dds new harness skill and updates based on merged PRs (#5126)
akshaydeo Jul 13, 2026
57dfffb
[fix]: preserve MiniMax multimodal content after merging dev
octo-patch Jul 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
174 changes: 174 additions & 0 deletions .claude/skills/harness-test-writer/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
---
name: harness-test-writer
description: Add regression test cases to the Bifrost provider harness (the Postman collection run via `make run-provider-harness-test`) based on a merged PR or a GitHub issue. Fetches the PR/issue, traces the affected wire path in the codebase, checks existing harness coverage, designs cases following harness conventions, inserts them into tests/e2e/api/collections/provider-harness.json without reformatting the file, and validates via the augment and filter scripts. Invoked with /harness-test-writer <PR# | issue# | URL> or /harness-test-writer (prompts for a reference).
allowed-tools: Read, Grep, Glob, Bash, WebFetch, Task, AskUserQuestion, TodoWrite, Edit, Write
---

# Provider Harness Test Writer

Turn a PR or GitHub issue into regression coverage in the provider harness: the Postman
collection at `tests/e2e/api/collections/provider-harness.json`, executed by newman via
`make run-provider-harness-test`.

This is NOT the Go test harness in `core/internal/llmtests/` (that one is run via
`make test-core PROVIDER=...`). If the user seems to want Go-level scenario tests,
confirm before proceeding.

## Step 1 - Resolve the reference

The argument may be a PR number, an issue number, or a GitHub URL.

- URL containing `/pull/` => PR. URL containing `/issues/` => issue.
- A bare number is ambiguous: try `gh pr view <N> --repo maximhq/bifrost` first; if it
404s, try `gh issue view <N>`. If BOTH exist and refer to different things, ask the
user which one they mean with AskUserQuestion.
- No argument at all => ask the user for the PR/issue reference.

Fetch full context:

```bash
gh pr view <N> --repo maximhq/bifrost --json title,body,state,files,baseRefName
gh pr diff <N> --repo maximhq/bifrost
# or
gh issue view <N> --repo maximhq/bifrost --json title,body,state,labels,comments
```

For a PR, also note any `Closes #X` issue and fetch that issue too - the issue usually
contains the client-visible reproduction (exact request shapes, error bodies, status
codes) that the harness case must mirror.

## Step 2 - Understand the wire-level behavior to pin

The harness tests Bifrost from the outside: HTTP requests against gateway routes, with
assertions on status codes and response/SSE bodies. Translate the PR/issue into that
frame:

1. Which route(s)? (`/v1/chat/completions`, `/v1/responses`, drop-ins like
`/openai`, `/anthropic`, `/bedrock`, `/genai`, passthrough routes, ...)
2. Which provider(s) and model(s)?
3. What request shape triggers the bug/feature? Reconstruct it from the issue repro or
from the code path in the diff (read the changed functions and their callers).
4. What is the observable failure signature before the fix (status code, error body
substring) and the expected behavior after?

Trace the chain in code with Grep/Read until you can state it in one sentence, e.g.:
"`x-bf-compat` header -> compat plugin marks `ChangeRequestType=ResponsesRequest` ->
`ToResponsesRequest()` -> `ToOpenAIResponsesRequest` strips role -> OpenAI 200".

Useful switches the harness relies on:
- `x-bf-compat` header: `true` enables all compat features; a JSON array like
`["convert_chat_to_responses"]` enables only specific ones (parsed in
`transports/bifrost-http/lib/ctx.go`, consumed by `plugins/compat/main.go`). Prefer
the targeted array form in regression cases so only the feature under test is active.
Note: when the suite runs with `COMPAT=on`, a collection-level prerequest script
upserts `x-bf-compat: true` over any per-request value - the targeted form matters
for `COMPAT=off` runs, which is exactly when a forced-conversion case needs it.
- `x-bf-passthrough-extra-params: true` for passthrough extra-param cases.

## Step 3 - Check existing coverage

Search the collection for the feature's keywords before writing anything:

```bash
cd tests/e2e/api
grep -c '<keyword>' collections/provider-harness.json
node -e "
const c = require('./collections/provider-harness.json');
function walk(items, path) { for (const it of items) {
if (it.item) walk(it.item, path + '/' + it.name);
else if (JSON.stringify(it).includes('<keyword>')) console.log(path + ' :: ' + it.name);
} }
walk(c.item, '');"
```

Also skim `HARNESS_COVERAGE_BACKLOG.md` - if the gap is listed there as `[ ]`, flip it
to `[x]` as part of the change (and if you find adjacent gaps worth noting, leave them,
do not scope-creep).

If coverage already exists, report where and stop - do not add duplicates.

## Step 4 - Design the cases

Conventions (match the existing collection exactly):

- **Folder**: issue-pinned regressions get their own top-level folder named
`<N>. <Short Title> (#<issue> / PR #<pr>)` where `<N>` is the next unused top-level
number (folders 17, 18, 19 are prior examples). Give the folder a `description`
explaining the bug, the production path, and what each case pins.
- **Case names** must contain the provider keyword and model so
`runners/filter-collection.mjs` PROVIDER filtering catches them - check
`PROVIDER_KEYWORDS` in that file (e.g. openai matches "openai", "gpt-"). Suffix each
name with ` - #<issue>`.
- **Coverage shape**: typically 2-3 cases - the real production route (non-streaming),
a streaming variant if the route streams, and where applicable a second route that
pins the same invariant independently (e.g. native `/v1/responses` alongside the
converted `/v1/chat/completions` path).
- **Test scripts** (Postman `event[].script.exec`, plain ES5 JavaScript):
- Start with an infra guard so auth/rate/server noise skips instead of false-failing:
`if ([401, 403, 429, 500, 502, 503, 504].indexOf(pm.response.code) !== -1) { return; }`
Do NOT guard on 400 when a 400 IS the regression signature - that must fail loudly.
- Assert the specific failure signature is absent (error substring, param name) AND
that the happy path succeeded (status below 400, expected fields present).
- Include the response text in failure messages:
`pm.expect(pm.response.code, 'failed: ' + pm.response.text()).to.be.below(400);`
- **Variables**: use `{{baseUrl}}` for the gateway. Inline `provider/model` strings
(e.g. `openai/gpt-4o-mini`) like the cross-cut folders do; only use variables such as
`{{bedrockModel}}`, `{{genaiModel}}`, `{{vertexModel}}` where existing folders do.
- Keep request bodies minimal and cheap (small `max_tokens`/`max_output_tokens`,
gpt-4o-mini-class models) - the harness runs as a paid live sweep.

Present the designed cases (names, route, body, assertions) to the user for approval
before editing the collection.

## Step 5 - Insert without reformatting

CRITICAL: `provider-harness.json` is ~1.5MB and is NOT byte-stable under
`JSON.stringify(JSON.parse(raw), null, 2)` (escape differences). Never rewrite the
whole file - the diff must contain only your added lines.

Use a Node script that appends the new folder textually before the closing `]` of the
top-level `item` array:

```js
const raw = fs.readFileSync(PATH, 'utf8');
const before = JSON.parse(raw);
// abort if the folder already exists (idempotence)
const indented = JSON.stringify(folder, null, 2).split('\n').map(l => ' ' + l).join('\n');
const tail = '\n ]\n}';
if (!raw.endsWith(tail)) throw new Error('unexpected file tail');
const out = raw.slice(0, -tail.length) + ',\n' + indented + tail;
const after = JSON.parse(out); // must parse; item count must be before + 1
fs.writeFileSync(PATH, out);
```

Adding cases INSIDE an existing folder is harder to do textually; if that is truly the
right placement, locate the folder's closing bracket precisely and verify the diff is
additions-only afterward. Default to a new top-level folder for issue regressions.

## Step 6 - Validate

All from `tests/e2e/api/`:

```bash
git diff --stat # must show only additions in provider-harness.json (+ backlog md if touched)
node runners/augment-provider-harness.mjs --source collections/provider-harness.json --out /tmp/aug.json
node runners/filter-collection.mjs --source /tmp/aug.json --out /tmp/filtered.json --provider <provider>
node -e "const c=require('/tmp/filtered.json'); const f=c.item.find(i=>i.name.startsWith('<N>.')); console.log('kept:', !!f, f && f.item.length);"
```

The augment script only regenerates its own "(generated)" folders, so a new top-level
folder passes through untouched - but run it anyway to catch parse breakage.

Do NOT auto-run the live suite: it starts a gateway and makes paid provider calls.
Report the run command and offer to execute it:

```bash
make run-provider-harness-test PROVIDER=<provider> FEATURE="<distinctive keyword from your case names>"
```

## Step 7 - Report

Summarize: what the PR/issue changed, the coverage gap found (cite evidence, e.g.
"tool_call_id appeared zero times in the collection"), each added case and what it
pins, validation results, and the run command. Leave the change unstaged - never
commit.
2 changes: 1 addition & 1 deletion .github/workflows/release-pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Release Pipeline
# Triggers automatically on push to main when any version file changes
on:
push:
branches: ["main"]
branches: ["main", "v2.0.0"]
workflow_dispatch:
inputs:
skip_tests:
Expand Down
54 changes: 48 additions & 6 deletions .github/workflows/scripts/validate-helm-templates.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ test_template() {

# 1. Storage Combinations (9 tests)
echo ""
echo -e "${CYAN}📦 1/6 - Testing Storage Combinations (9 tests)...${NC}"
echo -e "${CYAN}📦 1/7 - Testing Storage Combinations (9 tests)...${NC}"
echo "---------------------------------------------------"

# config=no, logs=no
Expand Down Expand Up @@ -126,7 +126,7 @@ test_template "config=postgres, logs=postgres" \

# 2. Vector Store Combinations (6 tests)
echo ""
echo -e "${CYAN}🗄️ 2/6 - Testing Vector Store Combinations (6 tests)...${NC}"
echo -e "${CYAN}🗄️ 2/7 - Testing Vector Store Combinations (6 tests)...${NC}"
echo "--------------------------------------------------------"

# Weaviate
Expand Down Expand Up @@ -175,7 +175,7 @@ test_template "sqlite + qdrant" \

# 3. Special Configurations (7 tests)
echo ""
echo -e "${CYAN}⚙️ 3/6 - Testing Special Configurations (7 tests)...${NC}"
echo -e "${CYAN}⚙️ 3/7 - Testing Special Configurations (7 tests)...${NC}"
echo "-----------------------------------------------------"

# semantic cache: direct mode (dimension: 1, no provider/keys)
Expand Down Expand Up @@ -251,7 +251,7 @@ test_template "production-like config" \

# 4. New Property Rendering (Gap 1-8 tests)
echo ""
echo -e "${CYAN}🆕 4/6 - Testing New Property Rendering (Gap 1-8)...${NC}"
echo -e "${CYAN}🆕 4/7 - Testing New Property Rendering (Gap 1-8)...${NC}"
echo "-----------------------------------------------------"

# Gap 1+2: Client new properties
Expand Down Expand Up @@ -337,7 +337,7 @@ test_template "combined: all new Gap 1-9 fields" \

# 5. Plugin Name Validation
echo ""
echo -e "${CYAN}🔌 5/6 - Validating Plugin Names Match Go Registry...${NC}"
echo -e "${CYAN}🔌 5/7 - Validating Plugin Names Match Go Registry...${NC}"
echo "------------------------------------------------------"

# Verify semantic cache plugin renders with correct name ("semantic_cache", not "semantic_cache")
Expand Down Expand Up @@ -366,7 +366,7 @@ fi

# 6. Custom Plugin Placement and Order Rendering
echo ""
echo -e "${CYAN}🔧 6/6 - Validating Custom Plugin placement and order Rendering...${NC}"
echo -e "${CYAN}🔧 6/7 - Validating Custom Plugin placement and order Rendering...${NC}"
echo "-------------------------------------------------------------------"

# Test custom plugin renders successfully with placement and order
Expand Down Expand Up @@ -421,6 +421,48 @@ else
head -10 /tmp/helm-template-output.yaml | sed 's/^/ /'
fi

# 7. Security Context Rendering
echo ""
echo -e "${CYAN}🔒 7/7 - Validating OpenShift-compatible Security Contexts...${NC}"
echo "----------------------------------------------------------------"

test_name="default Bifrost pod does not set runAsUser (SCC assigns UID)"
if helm template bifrost ./helm-charts/bifrost \
--set image.tag=v1.0.0 \
> /tmp/helm-template-output.yaml 2>&1; then
if grep -Eq '^[[:space:]]*runAsUser:' /tmp/helm-template-output.yaml; then
report_result "$test_name" 1
echo -e "${YELLOW} runAsUser found in default render (must stay unset so OpenShift can assign a UID)${NC}"
else
report_result "$test_name" 0
fi
else
report_result "$test_name" 1
echo -e "${YELLOW} Error output:${NC}"
head -10 /tmp/helm-template-output.yaml | sed 's/^/ /'
fi

# Postgres mode renders a Deployment (not the sqlite StatefulSet); assert the
# pinned-UID regression can't sneak in on that code path either.
test_name="postgres-mode Bifrost pod does not set runAsUser (SCC assigns UID)"
if helm template bifrost ./helm-charts/bifrost \
--set image.tag=v1.0.0 \
--set storage.mode=postgres \
--set postgresql.enabled=true \
--set postgresql.auth.password=testpass \
> /tmp/helm-template-output.yaml 2>&1; then
if grep -Eq '^[[:space:]]*runAsUser:' /tmp/helm-template-output.yaml; then
report_result "$test_name" 1
echo -e "${YELLOW} runAsUser found in postgres render (must stay unset so OpenShift can assign a UID)${NC}"
else
report_result "$test_name" 0
fi
else
report_result "$test_name" 1
echo -e "${YELLOW} Error output:${NC}"
head -10 /tmp/helm-template-output.yaml | sed 's/^/ /'
fi

# Cleanup
rm -f /tmp/helm-template-output.yaml

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Bifrost AI Gateway

[![Go Report Card](https://goreportcard.com/badge/github.com/maximhq/bifrost/core)](https://goreportcard.com/report/github.com/maximhq/bifrost/core)
<a href="https://trendshift.io/repositories/14529?utm_source=repository-badge&amp;utm_medium=badge&amp;utm_campaign=badge-repository-14529" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/repositories/14529" alt="maximhq%2Fbifrost | Trendshift" width="250" height="55"/></a>

[![Discord badge](https://dcbadge.limes.pink/api/server/https://discord.gg/exN5KAydbU?style=flat)](https://discord.gg/exN5KAydbU)
[![codecov](https://codecov.io/gh/maximhq/bifrost/branch/main/graph/badge.svg)](https://codecov.io/gh/maximhq/bifrost)
![Docker Pulls](https://img.shields.io/docker/pulls/maximhq/bifrost)
Expand Down
27 changes: 25 additions & 2 deletions core/bifrost.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import (
"github.com/maximhq/bifrost/core/providers/gemini"
"github.com/maximhq/bifrost/core/providers/groq"
"github.com/maximhq/bifrost/core/providers/huggingface"
"github.com/maximhq/bifrost/core/providers/minimax"
"github.com/maximhq/bifrost/core/providers/mistral"
"github.com/maximhq/bifrost/core/providers/nebius"
"github.com/maximhq/bifrost/core/providers/ollama"
Expand Down Expand Up @@ -4267,6 +4268,8 @@ func (bifrost *Bifrost) createBaseProvider(providerKey schemas.ModelProvider, co
return perplexity.NewPerplexityProvider(config, bifrost.logger)
case schemas.Cerebras:
return cerebras.NewCerebrasProvider(config, bifrost.logger)
case schemas.Minimax:
return minimax.NewMinimaxProvider(config, bifrost.logger)
case schemas.DeepSeek:
return deepseek.NewDeepSeekProvider(config, bifrost.logger)
case schemas.Gemini:
Expand Down Expand Up @@ -7951,6 +7954,18 @@ func (bifrost *Bifrost) drainQueueWithErrors(pq *ProviderQueue) {

// releaseChannelMessage returns a ChannelMessage and its channels to their respective pools.
func (bifrost *Bifrost) releaseChannelMessage(msg *ChannelMessage) {
// Drain any undelivered values before pooling so an idle pooled channel
// doesn't pin a full response/error until its next reuse. getChannelMessage
// drains again on acquire as defense in depth.
select {
case <-msg.Response:
default:
}
select {
case <-msg.Err:
default:
}

// Put channels back in pools
bifrost.responseChannelPool.Put(msg.Response)
bifrost.errorChannelPool.Put(msg.Err)
Expand All @@ -7965,9 +7980,17 @@ func (bifrost *Bifrost) releaseChannelMessage(msg *ChannelMessage) {
bifrost.responseStreamPool.Put(msg.ResponseStream)
}

// Release of Bifrost Request is handled in handle methods as they are required for fallbacks
// Release of the pooled *BifrostRequest object is handled in handle methods
// as it is required for fallbacks; msg.BifrostRequest is a value copy, so
// zeroing it here only drops this message's references.

// Clear references and return to pool
// Clear all references before returning to the pool. The embedded
// BifrostRequest holds pointers to the fully parsed request body and
// Context holds per-request user values; leaving them set pins those
// allocations for as long as the message sits idle in the pool.
// getChannelMessage overwrites both on acquire.
msg.BifrostRequest = schemas.BifrostRequest{}
msg.Context = nil
msg.Response = nil
msg.ResponseStream = nil
msg.Err = nil
Expand Down
Loading