feat: enrich realtime routing, logging, cost, and session tracking - #3335
Merged
akshaydeo merged 3 commits intoMay 14, 2026
Merged
Conversation
Collaborator
Author
|
|
This was referenced May 8, 2026
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (14)
✅ Files skipped from review due to trivial changes (2)
🚧 Files skipped from review as they are similar to previous changes (11)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThis PR extends realtime request infrastructure across WebRTC and WebSocket transports, adding model-catalog auto-resolution, session tool/voice tracking, trace/raw-store preservation, per-provider raw payload storage computation with per-request overrides, and audio token billing. ChangesRealtime Request Support with Audio Billing and Transport Integration
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 golangci-lint (2.12.2)level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies" Warning Review ran into problems🔥 ProblemsStopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Comment |
Contributor
Confidence Score: 5/5Safe to merge. All changes are additive observability enrichments; no existing call paths are altered in a way that would change behavior for non-realtime requests. The core context-propagation logic (snapshot before WebSocket upgrade, apply after fresh context creation) is carefully ordered and protected against stale overwrites. Audio cost computation uses a mathematically correct delta adjustment to avoid double-counting text-rate tokens. Session state mutations are properly guarded by the existing RWMutex and closed-session checks. The dedup strategy for function-call outputs is well-keyed and exercised by the new merge test. The governance realtime path follows the same nil-VK conventions as the existing body-based path. No files require special attention. The only outstanding nit — the unreachable errRealtimeDeploymentFormat variable in wsrealtime.go — was flagged in a prior review round and does not affect correctness. Important Files Changed
Reviews (7): Last reviewed commit: "feat: enrich realtime routing, logging, ..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
transports/bifrost-http/handlers/realtime_client_secrets.go (1)
162-167:⚠️ Potential issue | 🔴 Critical | ⚡ Quick winGuard nil upstream responses before dereferencing
resp.At Line 162,
resp.StatusCode(and at Line 166,resp.Body) is read before any nil check. If a provider returns(nil, nil), this panics instead of returning the intended structured error response.Proposed fix
resp, bifrostErr := sessionProvider.CreateRealtimeClientSecret(bifrostCtx, key, route.EndpointType, normalizedBody) if bifrostErr != nil { logger.Error("[realtime-client-secrets] upstream error: provider=%s model=%s error=%s", providerKey, model, bifrostErr.Error) SendBifrostError(ctx, bifrostErr) return } + if resp == nil { + writeRealtimeClientSecretResponse(ctx, resp) + return + } logger.Info("[realtime-client-secrets] upstream success: provider=%s model=%s status=%d", providerKey, model, resp.StatusCode) cacheRealtimeEphemeralKeyMapping( h.handlerStore.GetKVStore(),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@transports/bifrost-http/handlers/realtime_client_secrets.go` around lines 162 - 167, The code dereferences resp (resp.StatusCode and resp.Body) before checking for nil which can panic if the upstream returns (nil, nil); update the handler (the block that logs "[realtime-client-secrets] upstream success" and calls cacheRealtimeEphemeralKeyMapping) to first guard that resp != nil, and if nil return the intended structured error response (rather than proceeding), otherwise read resp.StatusCode and resp.Body and then call cacheRealtimeEphemeralKeyMapping(h.handlerStore.GetKVStore(), resp.Body, key.ID); ensure the nil-check happens immediately after the upstream call that produces resp.plugins/governance/main.go (1)
679-688:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftReplace raw
"model"/"modelId"context keys with typed identifiers.The new realtime query-param flow now depends on these rewrites, but they still travel through
ctx.Value("model"),ctx.Value("modelId"), and matchingctx.SetValue(...)calls with raw strings. That makes the routing contract collision-prone and violates the repo-wide context-key rule. Please move these to dedicated typed keys and update both the read and write sites together.As per coding guidelines, "Rule: No raw context keys in Go" and "The key passed to WithValue/Value MUST be that typed identifier."
Also applies to: 842-845, 999-1007
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@plugins/governance/main.go` around lines 679 - 688, Replace raw string context keys "model" and "modelId" with dedicated typed context key identifiers and update all read/write sites. Define a typed key type (e.g., type contextKey string) and package-level vars like modelCtxKey and modelIDCtxKey, then change reads from ctx.Value("model") and ctx.Value("modelId") to ctx.Value(modelCtxKey) / ctx.Value(modelIDCtxKey) and change all corresponding ctx.SetValue(...) calls to use those typed keys. Update every occurrence mentioned (the branch using modelValue, the bedrock branch with isBedrockPath, and the other sites around the ranges you noted) so both the WithValue/SetValue and Value calls use the same typed identifiers and leave other logic (e.g., falling back to req.CaseInsensitivePathParamLookup or modelId path extraction) intact.
🧹 Nitpick comments (1)
transports/bifrost-http/handlers/realtime_client_secrets_test.go (1)
68-70: ⚡ Quick winAdd one positive test for model-catalog auto-resolution behavior.
These updated call sites always pass
&lib.Config{}with no catalog data, so the new auto-resolution branch is still untested. Please add a case that configures providers for a bare model and asserts both resolved provider selection andFastHTTPUserValueModelCatalogResolutionbeing set on the request context.Also applies to: 122-124
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@transports/bifrost-http/handlers/realtime_client_secrets_test.go` around lines 68 - 70, Add a positive table-driven test case in realtime_client_secrets_test.go that exercises resolveRealtimeClientSecretTarget's model-catalog auto-resolution branch by supplying a lib.Config with a minimal ModelCatalog entry mapping the bare model name to providers; call resolveRealtimeClientSecretTarget(&ctx, &cfg, tt.route, tt.body) (use the same gotProvider, gotModel, err variables) and assert no error, that gotProvider equals the expected provider for the bare model, that gotModel reflects the resolved catalog model, and that ctx.UserValue(FastHTTPUserValueModelCatalogResolution) is set (non-nil/true) to verify the FastHTTPUserValueModelCatalogResolution side-effect.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@core/bifrost.go`:
- Around line 4143-4152: ComputeRawStorageForProvider calls ctx.Value without
guarding against a nil ctx which can panic; update the function
(ComputeRawStorageForProvider on type Bifrost) to first ensure ctx is non-nil by
defaulting to bifrost.ctx when ctx == nil, then proceed to read ctx.Value(...)
for BifrostContextKeyAllowPerRequestStorageOverride and
BifrostContextKeyStoreRawRequestResponse so the exported helper is safe for
callers that pass a nil context.
- Around line 4155-4164: The current block in core/bifrost.go sets
effectiveStore = true whenever send-back raw flags (ctx values
BifrostContextKeySendBackRawRequest or BifrostContextKeySendBackRawResponse) are
present, which wrongly couples send-back with persistent log storage; remove the
branches that set effectiveStore based on those send-back keys and ensure
effectiveStore is only controlled by the explicit storage-related flags (e.g.,
the existing BifrostContextKeyStoreRawRequest/BifrostContextKeyStoreRawResponse
logic or other storage-specific checks), keeping the allowRawOverride and
send-back checks only to gate send-back behavior and not mutate effectiveStore.
In `@framework/modelcatalog/pricing.go`:
- Around line 565-591: The audio token counts
(inputAudioTokens/outputAudioTokens) must be clamped to the actual token totals
before applying the audio-rate delta to avoid over/negative billing: when
computing audioCost, replace the raw usage.PromptTokensDetails.AudioTokens and
usage.CompletionTokensDetails.AudioTokens with min(audioTokens,
usage.PromptTokensDetails.PromptTokens) and min(audioTokens,
usage.CompletionTokensDetails.CompletionTokens) respectively (and treat nil
totals as 0), then proceed with the existing checks against
pricing.InputCostPerAudioToken and pricing.OutputCostPerAudioToken and compute
audioCost using these clamped values.
In `@plugins/governance/main.go`:
- Around line 358-365: The branch that handles bodyless realtime requests only
checks req.Query["model"] and calls p.governRealtimeQueryParam, so requests that
specify a deployment via req.Query["deployment"] are not processed; modify the
logic where len(req.Body) == 0 (and the similar block around lines 586-645) to
detect either req.Query["model"] or req.Query["deployment"], call
p.governRealtimeQueryParam for both cases, and ensure that any rewrite performed
by governRealtimeQueryParam is written back to the same query key that was
present (i.e., if the incoming param was "deployment" update
req.Query["deployment"], if it was "model" update req.Query["model"]) so routing
rules and virtual key load-balancing apply uniformly.
In `@transports/bifrost-http/handlers/realtime_turn_pipeline.go`:
- Around line 154-158: The synthetic pre-request builder drops explicit "clear
tools" semantics because buildRealtimeTurnPreRequest only sets Params.Tools when
len(tools) > 0; change it to set Params.Tools whenever tools != nil (allow empty
slice) so an explicit [] is preserved, matching updateRealtimeSessionFromEvent
which calls session.SetRealtimeSessionTools when event.Session.Tools != nil;
apply the same nil-vs-empty check in the other occurrence noted (the similar
block around the lines indicated) so turn hooks/logging receive [] for clears
instead of nil.
In `@transports/bifrost-http/handlers/webrtc_realtime.go`:
- Around line 168-179: The code currently pins bare-model /v1/realtime requests
to providers[0] during catalog auto-resolution (see providerKey,
config.GetProvidersForModel and lib.FastHTTPUserValueModelCatalogResolution /
ModelCatalogResolution), which bypasses later governance/key selection; remove
the eager assignment of providerKey = providers[0] and instead only store the
ModelCatalogResolution with Model, AllProviders (and optionally ResolvedProvider
left empty or nil) so downstream functions (handleLegacyRequest,
resolveRealtimeSDPTarget) and the governance/key selection can choose the
correct provider; in short, keep the provider list in the user value but do not
set providerKey to providers[0].
- Around line 1117-1127: The relay context created by newRealtimeRelayContext is
missing propagation of per-request storage/logging controls: add
schemas.BifrostContextKeyAllowPerRequestStorageOverride and
schemas.BifrostContextKeyDisableContentLogging to the keys you copy from
requestCtx into relayCtx (the same place you already copy
BifrostContextKeyShouldStoreRawInLogs and set
BifrostContextKeyRealtimeTransport) so downstream realtime logging respects
HTTP-level content-logging overrides.
In `@transports/bifrost-http/handlers/wsrealtime.go`:
- Around line 292-293: The code currently calls
updateRealtimeSessionFromEvent(session, event) before
provider.ToProviderRealtimeEvent(event), causing session metadata (tools/voice)
to be written even if provider validation fails; change the flow so any
normalization/extraction logic needed for translation is done without mutating
session (e.g., add a pure helper like normalizeSessionFromEvent or have
updateRealtimeSessionFromEvent return the normalized metadata), then call
provider.ToProviderRealtimeEvent(event) and only if that call succeeds apply the
session-state write (updateRealtimeSessionFromEvent or a
session.ApplyNormalizedMetadata method) to persist tools/voice into the
websocket session; ensure provider.ToProviderRealtimeEvent is the gatekeeper for
committing session changes.
---
Outside diff comments:
In `@plugins/governance/main.go`:
- Around line 679-688: Replace raw string context keys "model" and "modelId"
with dedicated typed context key identifiers and update all read/write sites.
Define a typed key type (e.g., type contextKey string) and package-level vars
like modelCtxKey and modelIDCtxKey, then change reads from ctx.Value("model")
and ctx.Value("modelId") to ctx.Value(modelCtxKey) / ctx.Value(modelIDCtxKey)
and change all corresponding ctx.SetValue(...) calls to use those typed keys.
Update every occurrence mentioned (the branch using modelValue, the bedrock
branch with isBedrockPath, and the other sites around the ranges you noted) so
both the WithValue/SetValue and Value calls use the same typed identifiers and
leave other logic (e.g., falling back to req.CaseInsensitivePathParamLookup or
modelId path extraction) intact.
In `@transports/bifrost-http/handlers/realtime_client_secrets.go`:
- Around line 162-167: The code dereferences resp (resp.StatusCode and
resp.Body) before checking for nil which can panic if the upstream returns (nil,
nil); update the handler (the block that logs "[realtime-client-secrets]
upstream success" and calls cacheRealtimeEphemeralKeyMapping) to first guard
that resp != nil, and if nil return the intended structured error response
(rather than proceeding), otherwise read resp.StatusCode and resp.Body and then
call cacheRealtimeEphemeralKeyMapping(h.handlerStore.GetKVStore(), resp.Body,
key.ID); ensure the nil-check happens immediately after the upstream call that
produces resp.
---
Nitpick comments:
In `@transports/bifrost-http/handlers/realtime_client_secrets_test.go`:
- Around line 68-70: Add a positive table-driven test case in
realtime_client_secrets_test.go that exercises
resolveRealtimeClientSecretTarget's model-catalog auto-resolution branch by
supplying a lib.Config with a minimal ModelCatalog entry mapping the bare model
name to providers; call resolveRealtimeClientSecretTarget(&ctx, &cfg, tt.route,
tt.body) (use the same gotProvider, gotModel, err variables) and assert no
error, that gotProvider equals the expected provider for the bare model, that
gotModel reflects the resolved catalog model, and that
ctx.UserValue(FastHTTPUserValueModelCatalogResolution) is set (non-nil/true) to
verify the FastHTTPUserValueModelCatalogResolution side-effect.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b92cbc48-9758-4327-933f-c94488200b23
📒 Files selected for processing (14)
core/bifrost.gocore/schemas/bifrost.goframework/modelcatalog/pricing.goplugins/governance/main.goplugins/logging/main.goplugins/logging/utils.gotransports/bifrost-http/handlers/realtime_client_secrets.gotransports/bifrost-http/handlers/realtime_client_secrets_test.gotransports/bifrost-http/handlers/realtime_logging_test.gotransports/bifrost-http/handlers/realtime_turn_pipeline.gotransports/bifrost-http/handlers/webrtc_realtime.gotransports/bifrost-http/handlers/webrtc_realtime_test.gotransports/bifrost-http/handlers/wsrealtime.gotransports/bifrost-http/websocket/session.go
danpiths
force-pushed
the
feat/05-08-feat_enrich_realtime_routing_logging_cost_and_session_tracking
branch
from
May 13, 2026 06:46
df8eea6 to
e7fe056
Compare
danpiths
force-pushed
the
feat/05-08-feat_add_azure_realtime_provider_and_nested_model_normalization
branch
from
May 13, 2026 06:46
ddc3e56 to
a1a3812
Compare
danpiths
force-pushed
the
feat/05-08-feat_enrich_realtime_routing_logging_cost_and_session_tracking
branch
from
May 13, 2026 07:35
e7fe056 to
357cbf4
Compare
danpiths
force-pushed
the
feat/05-08-feat_add_azure_realtime_provider_and_nested_model_normalization
branch
from
May 13, 2026 07:35
a1a3812 to
b5b4f19
Compare
danpiths
force-pushed
the
feat/05-08-feat_enrich_realtime_routing_logging_cost_and_session_tracking
branch
2 times, most recently
from
May 13, 2026 10:20
15d4b2d to
e6d5cd1
Compare
danpiths
force-pushed
the
feat/05-08-feat_add_azure_realtime_provider_and_nested_model_normalization
branch
from
May 13, 2026 10:20
b5b4f19 to
d256aa3
Compare
danpiths
force-pushed
the
feat/05-08-feat_enrich_realtime_routing_logging_cost_and_session_tracking
branch
from
May 14, 2026 10:38
e6d5cd1 to
6e910cd
Compare
danpiths
force-pushed
the
feat/05-08-feat_add_azure_realtime_provider_and_nested_model_normalization
branch
from
May 14, 2026 10:38
d256aa3 to
34a893c
Compare
akshaydeo
previously approved these changes
May 14, 2026
danpiths
force-pushed
the
feat/05-08-feat_add_azure_realtime_provider_and_nested_model_normalization
branch
from
May 14, 2026 13:34
34a893c to
de6e290
Compare
danpiths
force-pushed
the
feat/05-08-feat_enrich_realtime_routing_logging_cost_and_session_tracking
branch
from
May 14, 2026 13:34
6e910cd to
5e00d3f
Compare
Contributor
Merge activity
|
akshaydeo
changed the base branch from
feat/05-08-feat_add_azure_realtime_provider_and_nested_model_normalization
to
graphite-base/3335
May 14, 2026 14:00
akshaydeo
dismissed stale reviews from coderabbitai[bot] and themself
May 14, 2026 14:00
The base branch was changed.
akshaydeo
deleted the
feat/05-08-feat_enrich_realtime_routing_logging_cost_and_session_tracking
branch
May 14, 2026 14:01
18 tasks
akshaydeo
pushed a commit
that referenced
this pull request
May 15, 2026
…3335) ## Summary Closes multiple gaps in realtime observability so that realtime turns have the same logging richness as normal LLM calls: routing engine logs, tool definitions, voice/transport metadata, cost calculation, and proper context propagation through the turn lifecycle. ## Changes - **Governance routing for realtime** (`plugins/governance/main.go`): Added `governRealtimeQueryParam` so realtime WebSocket requests participate in routing rules and virtual key governance via query param model resolution - **WebSocket middleware context propagation** (`handlers/wsrealtime.go`): Added `snapshotRealtimeMiddlewareValues` / `applyRealtimeMiddlewareValues` to capture governance context values from the HTTP middleware and propagate them to the WebSocket session context. Processes model catalog resolution to generate routing engine log entries. Computes raw storage flag via `ComputeRawStorageForProvider` - **WebRTC middleware context propagation** (`handlers/webrtc_realtime.go`): Same propagation for WebRTC relay. Added nested model prefix stripping for legacy `/v1/realtime` POST path. Added model catalog auto-resolution for bare models - **Client secrets operational logging** (`handlers/realtime_client_secrets.go`): Added model catalog auto-resolution and operational logs (Info for requests/success, Error for failures) - **Session tool and voice tracking** (`websocket/session.go`): Added `realtimeSessionTools` and `realtimeVoice` fields with getters/setters. Replaced `strings.Contains` dedup with replace-with-latest strategy for tool output events. Added close guards on mutating methods. `Close()` now nils all accumulated data - **Turn pipeline enrichment** (`handlers/realtime_turn_pipeline.go`): Added `updateRealtimeSessionFromEvent` to track tools, voice, and strip nested model prefixes from session events. Updated `buildRealtimeTurnPreRequest` to attach session tools. Added `restoreRealtimeTurnTraceContext` and `applyRealtimeRawStorageContext` for context propagation. Rewrote `buildRealtimeTurnOutputMessages` to merge provider extractor output with raw `response.done` payload so both text and tool calls appear - **Logging plugin fixes** (`plugins/logging/main.go`): Added nil check for `ResponsesRequest.Params` before iterating tools (fixed panic). Moved realtime-specific PostLLMHook enrichment to run after `routingEngineLogs` extraction so routing engine decision logs are populated - **Voice/transport metadata** (`plugins/logging/utils.go`): `mergeRealtimeMetadata` now writes `realtime_voice` and `realtime_transport` - **Cost calculation** (`framework/modelcatalog/pricing.go`): Added `RealtimeRequest` to `calculateBaseCost`. `computeTextCost` now subtracts audio tokens from text counts and prices them at `InputCostPerAudioToken`/`OutputCostPerAudioToken` rates. Added `AudioTokens` mapping for output token details - **Core exports** (`core/bifrost.go`, `core/schemas/bifrost.go`): Added `ComputeRawStorageForProvider` method and `BifrostContextKeyRealtimeVoice`/`BifrostContextKeyRealtimeTransport` context keys - **Tests**: Added dedup behavior tests, output message merging tests, updated existing test expectations ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Handler tests (covers turn pipeline, dedup, output merging) go test ./transports/bifrost-http/handlers/ -count=1 -v # Logging plugin tests go test ./plugins/logging/ -count=1 -v # Cost calculation tests go test ./framework/modelcatalog/ -count=1 -v # Full build make build LOCAL=1 ``` Manual verification: 1. Send a bare model like `gpt-4o-realtime-preview-2025-06-03` to `/v1/realtime` — verify routing engine logs show model catalog resolution in the Logs UI 2. Use a virtual key — verify virtual key info appears in the log entry 3. Have the model make a tool call alongside a text response — verify both appear in the log output 4. Check the cost column shows non-zero values for realtime turns 5. Verify tool definitions from `session.update` appear in the log's "Tools" field ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations `ComputeRawStorageForProvider` is a read-only method that checks provider config. No new auth surfaces or secret handling. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
18 tasks
akshaydeo
pushed a commit
that referenced
this pull request
May 15, 2026
…3335) ## Summary Closes multiple gaps in realtime observability so that realtime turns have the same logging richness as normal LLM calls: routing engine logs, tool definitions, voice/transport metadata, cost calculation, and proper context propagation through the turn lifecycle. ## Changes - **Governance routing for realtime** (`plugins/governance/main.go`): Added `governRealtimeQueryParam` so realtime WebSocket requests participate in routing rules and virtual key governance via query param model resolution - **WebSocket middleware context propagation** (`handlers/wsrealtime.go`): Added `snapshotRealtimeMiddlewareValues` / `applyRealtimeMiddlewareValues` to capture governance context values from the HTTP middleware and propagate them to the WebSocket session context. Processes model catalog resolution to generate routing engine log entries. Computes raw storage flag via `ComputeRawStorageForProvider` - **WebRTC middleware context propagation** (`handlers/webrtc_realtime.go`): Same propagation for WebRTC relay. Added nested model prefix stripping for legacy `/v1/realtime` POST path. Added model catalog auto-resolution for bare models - **Client secrets operational logging** (`handlers/realtime_client_secrets.go`): Added model catalog auto-resolution and operational logs (Info for requests/success, Error for failures) - **Session tool and voice tracking** (`websocket/session.go`): Added `realtimeSessionTools` and `realtimeVoice` fields with getters/setters. Replaced `strings.Contains` dedup with replace-with-latest strategy for tool output events. Added close guards on mutating methods. `Close()` now nils all accumulated data - **Turn pipeline enrichment** (`handlers/realtime_turn_pipeline.go`): Added `updateRealtimeSessionFromEvent` to track tools, voice, and strip nested model prefixes from session events. Updated `buildRealtimeTurnPreRequest` to attach session tools. Added `restoreRealtimeTurnTraceContext` and `applyRealtimeRawStorageContext` for context propagation. Rewrote `buildRealtimeTurnOutputMessages` to merge provider extractor output with raw `response.done` payload so both text and tool calls appear - **Logging plugin fixes** (`plugins/logging/main.go`): Added nil check for `ResponsesRequest.Params` before iterating tools (fixed panic). Moved realtime-specific PostLLMHook enrichment to run after `routingEngineLogs` extraction so routing engine decision logs are populated - **Voice/transport metadata** (`plugins/logging/utils.go`): `mergeRealtimeMetadata` now writes `realtime_voice` and `realtime_transport` - **Cost calculation** (`framework/modelcatalog/pricing.go`): Added `RealtimeRequest` to `calculateBaseCost`. `computeTextCost` now subtracts audio tokens from text counts and prices them at `InputCostPerAudioToken`/`OutputCostPerAudioToken` rates. Added `AudioTokens` mapping for output token details - **Core exports** (`core/bifrost.go`, `core/schemas/bifrost.go`): Added `ComputeRawStorageForProvider` method and `BifrostContextKeyRealtimeVoice`/`BifrostContextKeyRealtimeTransport` context keys - **Tests**: Added dedup behavior tests, output message merging tests, updated existing test expectations ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Handler tests (covers turn pipeline, dedup, output merging) go test ./transports/bifrost-http/handlers/ -count=1 -v # Logging plugin tests go test ./plugins/logging/ -count=1 -v # Cost calculation tests go test ./framework/modelcatalog/ -count=1 -v # Full build make build LOCAL=1 ``` Manual verification: 1. Send a bare model like `gpt-4o-realtime-preview-2025-06-03` to `/v1/realtime` — verify routing engine logs show model catalog resolution in the Logs UI 2. Use a virtual key — verify virtual key info appears in the log entry 3. Have the model make a tool call alongside a text response — verify both appear in the log output 4. Check the cost column shows non-zero values for realtime turns 5. Verify tool definitions from `session.update` appear in the log's "Tools" field ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations `ComputeRawStorageForProvider` is a read-only method that checks provider config. No new auth surfaces or secret handling. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
akshaydeo
pushed a commit
that referenced
this pull request
May 20, 2026
…3335) ## Summary Closes multiple gaps in realtime observability so that realtime turns have the same logging richness as normal LLM calls: routing engine logs, tool definitions, voice/transport metadata, cost calculation, and proper context propagation through the turn lifecycle. ## Changes - **Governance routing for realtime** (`plugins/governance/main.go`): Added `governRealtimeQueryParam` so realtime WebSocket requests participate in routing rules and virtual key governance via query param model resolution - **WebSocket middleware context propagation** (`handlers/wsrealtime.go`): Added `snapshotRealtimeMiddlewareValues` / `applyRealtimeMiddlewareValues` to capture governance context values from the HTTP middleware and propagate them to the WebSocket session context. Processes model catalog resolution to generate routing engine log entries. Computes raw storage flag via `ComputeRawStorageForProvider` - **WebRTC middleware context propagation** (`handlers/webrtc_realtime.go`): Same propagation for WebRTC relay. Added nested model prefix stripping for legacy `/v1/realtime` POST path. Added model catalog auto-resolution for bare models - **Client secrets operational logging** (`handlers/realtime_client_secrets.go`): Added model catalog auto-resolution and operational logs (Info for requests/success, Error for failures) - **Session tool and voice tracking** (`websocket/session.go`): Added `realtimeSessionTools` and `realtimeVoice` fields with getters/setters. Replaced `strings.Contains` dedup with replace-with-latest strategy for tool output events. Added close guards on mutating methods. `Close()` now nils all accumulated data - **Turn pipeline enrichment** (`handlers/realtime_turn_pipeline.go`): Added `updateRealtimeSessionFromEvent` to track tools, voice, and strip nested model prefixes from session events. Updated `buildRealtimeTurnPreRequest` to attach session tools. Added `restoreRealtimeTurnTraceContext` and `applyRealtimeRawStorageContext` for context propagation. Rewrote `buildRealtimeTurnOutputMessages` to merge provider extractor output with raw `response.done` payload so both text and tool calls appear - **Logging plugin fixes** (`plugins/logging/main.go`): Added nil check for `ResponsesRequest.Params` before iterating tools (fixed panic). Moved realtime-specific PostLLMHook enrichment to run after `routingEngineLogs` extraction so routing engine decision logs are populated - **Voice/transport metadata** (`plugins/logging/utils.go`): `mergeRealtimeMetadata` now writes `realtime_voice` and `realtime_transport` - **Cost calculation** (`framework/modelcatalog/pricing.go`): Added `RealtimeRequest` to `calculateBaseCost`. `computeTextCost` now subtracts audio tokens from text counts and prices them at `InputCostPerAudioToken`/`OutputCostPerAudioToken` rates. Added `AudioTokens` mapping for output token details - **Core exports** (`core/bifrost.go`, `core/schemas/bifrost.go`): Added `ComputeRawStorageForProvider` method and `BifrostContextKeyRealtimeVoice`/`BifrostContextKeyRealtimeTransport` context keys - **Tests**: Added dedup behavior tests, output message merging tests, updated existing test expectations ## Type of change - [x] Bug fix - [x] Feature - [ ] Refactor - [ ] Documentation - [ ] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [ ] Providers/Integrations - [x] Plugins - [ ] UI (React) - [ ] Docs ## How to test ```sh # Handler tests (covers turn pipeline, dedup, output merging) go test ./transports/bifrost-http/handlers/ -count=1 -v # Logging plugin tests go test ./plugins/logging/ -count=1 -v # Cost calculation tests go test ./framework/modelcatalog/ -count=1 -v # Full build make build LOCAL=1 ``` Manual verification: 1. Send a bare model like `gpt-4o-realtime-preview-2025-06-03` to `/v1/realtime` — verify routing engine logs show model catalog resolution in the Logs UI 2. Use a virtual key — verify virtual key info appears in the log entry 3. Have the model make a tool call alongside a text response — verify both appear in the log output 4. Check the cost column shows non-zero values for realtime turns 5. Verify tool definitions from `session.update` appear in the log's "Tools" field ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues N/A ## Security considerations `ComputeRawStorageForProvider` is a read-only method that checks provider config. No new auth surfaces or secret handling. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [x] I verified the CI pipeline passes locally if applicable
akshaydeo
added a commit
that referenced
this pull request
May 20, 2026
## Summary This PR cuts the `v1.5.11` / `v1.3.11` release across core, framework, and all plugins, and introduces a new Claude skill (`release-checklist`) for pre-release migration safety auditing. ## Changes - **`release-checklist` skill** — Adds `.claude/skills/release-checklist/SKILL.md`, a read-only pre-release audit tool that scans Go-defined database migrations changed in a release for high-scale deadlock/lock-contention risks and boot-time-blocking operations. It produces a structured `PASS`/`WARN`/`FAIL` report with a concrete remediation plan per finding. The skill is designed to grow via an extensible Checks Registry. - **Version bumps** — `core` → `1.5.11`, `framework` → `1.3.11`, `transports` → `1.5.3`, `plugins/governance` → `1.5.11`, `plugins/logging` → `1.5.11`, `plugins/semanticcache` → `1.5.11`, `plugins/otel` → `1.2.11`, `plugins/maxim` → `1.6.11`, `plugins/prompts` → `1.0.11`, and remaining plugins bumped accordingly. - **Changelogs populated** — All per-package changelogs updated with the full set of features and fixes shipping in this release. Key highlights in this release: - Temporary access tokens for scoped, time-limited API access - MCP per-user OAuth flow refactor - Bedrock Mantle inference engine support - Azure Realtime provider with enriched session tracking - Direct access control (DAC) and virtual key rotation - Cluster-aware log metadata and per-node usage aggregation - Feature flag framework - Config-hash-based file value override of DB on restart - Semantic cache plugin rewrite - Numerous streaming stability, Bedrock, Anthropic, and Gemini fixes - AWS SDK and dependency security updates ## Type of change - [ ] Bug fix - [ ] Feature - [ ] Refactor - [ ] Documentation - [x] Chore/CI ## Affected areas - [x] Core (Go) - [x] Transports (HTTP) - [x] Providers/Integrations - [x] Plugins - [x] UI (React) - [ ] Docs ## How to test ```sh # Verify version files reflect the new release cat core/version # expect 1.5.11 cat framework/version # expect 1.3.11 cat transports/version # expect 1.5.3 # Core/Transports go test ./... ``` To exercise the new `release-checklist` skill, invoke it via Claude with: ``` /release-checklist origin/dev...HEAD ``` Expected output: a structured report with `PASS`/`WARN`/`FAIL` per check and a Remediation Plan table for any findings. ## Screenshots/Recordings N/A ## Breaking changes - [ ] Yes - [x] No ## Related issues #3603, #3565, #3489, #3334, #3335, #3435, #3554, #3590, #3444, #3198, #3581, #3610, #3599, #3567, #3382, #3461 and others listed in the changelogs. ## Security considerations - AWS SDK and dependency security updates are included (#3461). - `FullyRedacted()` for proxy passwords and `MarshalForStorage()` for `ProxyConfig` prevent partial secret leakage in API responses (#3445). - The `release-checklist` skill is strictly read-only and never modifies files. ## Checklist - [x] I read `docs/contributing/README.md` and followed the guidelines - [x] I added/updated tests where appropriate - [x] I updated documentation where needed - [x] I verified builds succeed (Go and UI) - [ ] I verified the CI pipeline passes locally if applicable
Merged
This was referenced May 20, 2026
18 tasks
This was referenced Jul 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes multiple gaps in realtime observability so that realtime turns have the
same logging richness as normal LLM calls: routing engine logs, tool
definitions, voice/transport metadata, cost calculation, and proper context
propagation through the turn lifecycle.
Changes
plugins/governance/main.go): AddedgovernRealtimeQueryParamso realtime WebSocket requests participate inrouting rules and virtual key governance via query param model resolution
handlers/wsrealtime.go): AddedsnapshotRealtimeMiddlewareValues/applyRealtimeMiddlewareValuestocapture governance context values from the HTTP middleware and propagate them
to the WebSocket session context. Processes model catalog resolution to
generate routing engine log entries. Computes raw storage flag via
ComputeRawStorageForProviderhandlers/webrtc_realtime.go):Same propagation for WebRTC relay. Added nested model prefix stripping for
legacy
/v1/realtimePOST path. Added model catalog auto-resolution for baremodels
(
handlers/realtime_client_secrets.go): Added model catalog auto-resolutionand operational logs (Info for requests/success, Error for failures)
websocket/session.go): AddedrealtimeSessionToolsandrealtimeVoicefields with getters/setters.Replaced
strings.Containsdedup with replace-with-latest strategy for tooloutput events. Added close guards on mutating methods.
Close()now nils allaccumulated data
handlers/realtime_turn_pipeline.go): AddedupdateRealtimeSessionFromEventto track tools, voice, and strip nested modelprefixes from session events. Updated
buildRealtimeTurnPreRequestto attachsession tools. Added
restoreRealtimeTurnTraceContextandapplyRealtimeRawStorageContextfor context propagation. RewrotebuildRealtimeTurnOutputMessagesto merge provider extractor output with rawresponse.donepayload so both text and tool calls appearplugins/logging/main.go): Added nil check forResponsesRequest.Paramsbefore iterating tools (fixed panic). Movedrealtime-specific PostLLMHook enrichment to run after
routingEngineLogsextraction so routing engine decision logs are populated
plugins/logging/utils.go):mergeRealtimeMetadatanow writesrealtime_voiceandrealtime_transportframework/modelcatalog/pricing.go): AddedRealtimeRequesttocalculateBaseCost.computeTextCostnow subtractsaudio tokens from text counts and prices them at
InputCostPerAudioToken/OutputCostPerAudioTokenrates. AddedAudioTokensmapping for output token details
core/bifrost.go,core/schemas/bifrost.go): AddedComputeRawStorageForProvidermethod andBifrostContextKeyRealtimeVoice/BifrostContextKeyRealtimeTransportcontextkeys
existing test expectations
Type of change
Affected areas
How to test
Manual verification:
gpt-4o-realtime-preview-2025-06-03to/v1/realtime— verify routing engine logs show model catalog resolution in the Logs UI
appear in the log output
session.updateappear in the log's "Tools"field
Screenshots/Recordings
N/A
Breaking changes
Related issues
N/A
Security considerations
ComputeRawStorageForProvideris a read-only method that checks providerconfig. No new auth surfaces or secret handling.
Checklist
docs/contributing/README.mdand followed the guidelines