Repository navigation
feat(gateway): add OpenAI realtime API support - #3231
Merged
Merged
Conversation
Add a dedicated /v1/realtime WebSocket proxy service: - Catalogue: realtime capability flag; gpt-realtime-2.1 and gpt-realtime-2.1-mini models (legacy gpt-realtime family added as deprecated); modality-specific audio prices exposed via /v1/models - DB: realtime_session table; log columns for exact decimal billingCost, audio output tokens/cost, and an idempotent (session, usage key) partial unique index - Realtime service: preflight auth/provider resolution, model pinning, gated VAD response mediation, per-response exact usage billing with discounts (persisted before forwarding response.done), credit/api-key/member-budget/rate-limit gates, atomic Redis concurrency leases, backpressure, drain on disconnect, kill switch - Worker: debit exact billingCost over float cost; decimal strings for api-key/session/wallet usage updates - Infra: dedicated Helm deployment/service/PDB and /v1/realtime ingress routing with long termination grace Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extends the /v1/realtime proxy with input-audio transcription billing and ephemeral client secrets, adds the playground voice page, and lets the gateway host realtime inline (REALTIME_INLINE) for local dev. Transcription is metered per completed event against an ASR mapping that is pinned for the session, so a model swap cannot reprice work already committed, and disabling transcription keeps the pinned mapping so pending items stay billable instead of killing the session. Authorization and lifecycle fixes: - ASR model selection now clears the same per-key IAM rules as the realtime model, enforced at mint, upgrade and session.update. - Generation gates re-check IAM and the compliance policy, so a revoked model, IP range or provider takes effect within one turn. - Stored prompt references are rejected: a prompt can carry tool definitions and slip hosted tools past the function-tools-only check. - Treat a handleUpgrade that never invokes its callback as failure; ws aborts without throwing when a client vanishes mid-preflight, which orphaned the upstream socket, the lease and the session row. - Attach the upgrade socket's error handler first, before any await, so a client reset during preflight cannot become an uncaughtException. - Auto-response no longer starts while draining a disconnect, and transcription spend re-checks limits so an exhausted account can overrun by at most one turn. - Resolve the client IP via the shared X-Forwarded-For helper, dropping a socket-address fallback that would match a load balancer against customer CIDR allowlists. Also corrects gpt-realtime-2.1 metadata (128k context, 32k max output, configurable reasoning), indexes the per-response unsettled-spend query, exposes audio token prices so the playground shows what voice sessions actually cost, and ships the AudioWorklet that .gitignore excluded. Co-Authored-By: Claude <noreply@anthropic.com>
audioOutputCost was written to the log table by the realtime billing path but existed on none of the 7 stats tables and was never summed by the aggregators. Session totals were correct, but every analytics, activity and admin breakdown silently omitted the audio-output line — the dominant realtime cost at $64/M. A billed response totalling 0.0492325 reported components summing to only 0.0236325. Plumb the field through the full pipeline: the 7 stats tables (plus migration), getCommonAggregationFields (covers all 7 write paths), the global aggregator's AGGREGATE_KEYS conflict-update set, the activity route, UI types and seed data. Breakdowns now reconcile against the total. Also add audioInputCost/audioOutputCost to the API test harness copy of getCommonAggregationFields, which had drifted from the worker and would have let an audio-cost stats test pass with zeros. Alongside this, correct two realtime billing comments and close a pricing gap: - The "fail closed on discount" comments were wrong. getEffectiveDiscount swallows its own database errors and returns a 0% discount, so a lookup outage bills a discounted customer at full price rather than reaching the catch. Same on the HTTP path; the comments now say so instead of claiming the opposite. - Document that realtime deliberately does not honour BILL_CANCELLED_REQUESTS: a cancelled response still reports audio already generated and the provider bills us for it. - Fail closed when a realtime or ASR mapping declares a non-zero flat requestPrice. Realtime pricing is token-only, so such a mapping would under-bill silently. All six current mappings declare "0", so this is a guard against future drift rather than a behaviour change. Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis change adds a realtime voice-call platform spanning model metadata, gateway WebSocket proxying, session billing, browser audio capture/playback, playground UI and history, database persistence, activity aggregation, and optional dedicated deployment infrastructure. ChangesRealtime voice platform
Estimated code review effort: 5 (Critical) | ~120 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ca2cd591d6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 11
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/playground/src/components/model-selector.tsx (1)
2072-2147: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win"Output" label isn't updated to "Text Out" for realtime mode, unlike its "Text In" sibling.
Line 2072-2074 relabels "Input" to "Text In" when
mode === "realtime", but the "Output" cell just below (unchanged, line ~2104) stays generic "Output" even though a separate "Audio Out" cell is now rendered right next to it in the same grid — leaving the text/audio distinction asymmetric and mildly confusing.🏷️ Proposed fix
<div className="space-y-1"> <span className="text-[11px] font-medium text-muted-foreground uppercase tracking-wide"> - Output + {mode === "realtime" ? "Text Out" : "Output"} </span>🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/playground/src/components/model-selector.tsx` around lines 2072 - 2147, Update the output label in the pricing grid to display “Text Out” when mode === "realtime", matching the existing conditional “Text In” label; retain “Output” for all other modes and leave the separate audio pricing cells unchanged.
🧹 Nitpick comments (16)
packages/db/migrations/meta/1784973019_snapshot.json (1)
17279-17341: 🗄️ Data Integrity & Integration | 🔵 Trivial | 💤 Low value
bytes_in/bytes_outuse 32-bitinteger; considerbigintfor long-lived sessions.
realtime_session.bytes_inandbytes_outareinteger(max ~2.147 GB). For long-running bidirectional audio WebSocket sessions with continuous PCM streaming, cumulative byte counters could theoretically approach or exceed this limit, causing overflow. Since this snapshot is machine-generated, any fix belongs inpackages/db/src/schema.ts(change the column type tobigint) followed by regenerating migrations — not in this file directly.Based on path instructions, snapshot/journal files must never be manually edited and changes should originate from
packages/db/src/schema.tsfollowed bypnpm migrations.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/db/migrations/meta/1784973019_snapshot.json` around lines 17279 - 17341, Update the realtime_session schema definition in packages/db/src/schema.ts so bytes_in and bytes_out use bigint instead of 32-bit integer, preserving their existing non-null and zero-default behavior. Then regenerate the migration artifacts with pnpm migrations; do not edit the generated snapshot directly.apps/gateway/src/realtime/transcription.ts (1)
24-37: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winExtract the duplicated
readCount/priceComponenthelpers into a shared module.Both helpers are byte-identical to the ones in
pricing.ts(Lines 31-44 and 242-254 there). Since these encode billing invariants (integer/non-negative validation, zero-token short-circuit), divergence between the two copies would silently change billing behavior for one path only.As per coding guidelines, "Apply DRY principles for code reuse".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/realtime/transcription.ts` around lines 24 - 37, Move the duplicated readCount and priceComponent helpers into a shared module, then update both transcription.ts and pricing.ts to import and reuse those shared implementations. Preserve their existing validation and zero-token behavior, and remove the local duplicate definitions from both files.Source: Coding guidelines
apps/gateway/src/realtime/pricing.spec.ts (1)
128-264: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winThe non-zero
requestPricefail-closed guard is untested in both pricing suites.priceRealtimeUsage(pricing.tsLines 267-275) andpriceTranscriptionUsage(transcription.tsLines 169-177) both reject mappings carrying a flat per-request price so it can't be silently dropped, but neither spec exercises that branch — a regression there would under-bill without any test failing.
apps/gateway/src/realtime/pricing.spec.ts#L128-L264: add a case assertingpriceRealtimeUsagereturnsok: falsefor a snapshot built from a mapping with a non-zerorequestPrice, plus a case coveringapplyRealtimeDiscount(currently untested, unlike its transcription analogue).apps/gateway/src/realtime/transcription.spec.ts#L121-L189: add the equivalent case assertingpriceTranscriptionUsagereturnsok: falsefor a non-zerorequestPricesnapshot.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/realtime/pricing.spec.ts` around lines 128 - 264, The realtime pricing tests lack coverage for non-zero requestPrice rejection and applyRealtimeDiscount. In apps/gateway/src/realtime/pricing.spec.ts lines 128-264, add a priceRealtimeUsage case using a snapshot with non-zero requestPrice and assert ok is false, plus a case covering applyRealtimeDiscount. In apps/gateway/src/realtime/transcription.spec.ts lines 121-189, add the equivalent priceTranscriptionUsage case asserting ok is false for a snapshot with non-zero requestPrice.apps/gateway/src/realtime/session.spec.ts (1)
154-154: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winType the preflight overrides so a mistyped key can't silently no-op.
preflightOverridesisRecord<string, unknown>andbuildPreflight()is cast throughas unknown, so an override key that doesn't match a realRealtimePreflightResultfield (e.g. theallowedTranscriptionModelIdsused by the authorization test at Lines 263-265) would be accepted by the compiler and the test would silently assert against the default fixture instead.♻️ Suggested change
-function createSession(preflightOverrides: Record<string, unknown> = {}) { +function createSession(preflightOverrides: Partial<RealtimePreflightResult> = {}) {Also applies to: 163-172
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/realtime/session.spec.ts` at line 154, Type the preflightOverrides fixture in buildPreflight as a Partial<RealtimePreflightResult> (or the repository’s equivalent typed override), removing the broad Record<string, unknown> and unknown cast. Ensure keys such as allowedTranscriptionModelIds are validated against RealtimePreflightResult so mistyped overrides fail compilation while existing fixture defaults remain intact.apps/worker/src/worker.ts (1)
1036-1046: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winInclude
billing_cost/effectiveCostin thelog-processstructured log.The log record at Line 1006 still emits only
row.cost, so for decimal-billed realtime rows (wherecostmay be null or lossy) the audit trail won't show the amount actually debited.♻️ Suggested addition
cost: row.cost, + billingCost: row.billing_cost, inputCost: row.input_cost,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/worker/src/worker.ts` around lines 1036 - 1046, Update the structured `log-process` record near the existing `row.cost` field to include the exact billing amount from `effectiveCost` (or the underlying `billing_cost` value), while preserving the fallback behavior for legacy costs. Ensure decimal-billed realtime rows record the amount actually debited rather than only `row.cost`.apps/playground/src/components/playground/realtime-page-client.tsx (2)
181-182: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value
showAuthDialogrestatesisAuthenticated.
!isAuthenticatedalready covers!isUserLoading && !user; simplify to!isUserLoading && !user.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/playground/src/components/playground/realtime-page-client.tsx` around lines 181 - 182, Update the showAuthDialog assignment in the realtime page client to use the direct condition !isUserLoading && !user, removing the redundant !isAuthenticated check while preserving the existing behavior.
297-315: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAnnounce call status changes to screen readers.
The status text and timer update purely visually; add
aria-live="polite"on the status span so state transitions (Connecting → Live → Ending) are conveyed non-visually.♿ Proposed change
<div className="ml-auto flex items-center gap-3 text-sm"> <span + aria-live="polite" className={🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/playground/src/components/playground/realtime-page-client.tsx` around lines 297 - 315, Add aria-live="polite" to the status span containing STATUS_LABELS[status] in the realtime status display, ensuring Connecting, Live, and Ending transitions are announced without changing the existing visual behavior or timer rendering.apps/playground/src/app/api/realtime/session/route.ts (1)
105-117: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDerive the gateway base URL once.
resolveRealtimeWsUrland thegatewayBaseUrlblock duplicate the sameGATEWAY_URLnormalization and dev/prod fallbacks; the two can drift.♻️ Extract a shared helper
+function resolveGatewayBaseUrl(): string { + return ( + process.env.GATEWAY_URL?.replace(/\/v1$/, "") ?? + (process.env.NODE_ENV === "development" + ? "http://localhost:4001" + : "https://api.llmgateway.io") + ); +} + function resolveRealtimeWsUrl(): string { const override = process.env.REALTIME_WS_URL; if (override) { return override; } - const gatewayUrl = process.env.GATEWAY_URL?.replace(/\/v1$/, ""); - if (gatewayUrl) { - return `${gatewayUrl.replace(/^http/, "ws")}/v1/realtime`; - } - return process.env.NODE_ENV === "development" - ? "ws://localhost:4001/v1/realtime" - : "wss://api.llmgateway.io/v1/realtime"; + return `${resolveGatewayBaseUrl().replace(/^http/, "ws")}/v1/realtime`; }Also applies to: 174-178
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/playground/src/app/api/realtime/session/route.ts` around lines 105 - 117, Extract the shared gateway base URL resolution, including GATEWAY_URL normalization and development/production fallbacks, into a single helper. Update resolveRealtimeWsUrl and the gatewayBaseUrl block to reuse that helper, preserving the existing WebSocket protocol conversion and endpoint suffix.apps/playground/src/hooks/use-realtime-call.ts (1)
121-124: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueMutating refs during render.
voiceRef.current = voice/onCallErrorRef.current = onCallErrorrun on every render, which is unsafe under concurrent rendering (a discarded render still writes). React 19.2'suseEffectEvent(or auseEffectassignment) is the sanctioned way to keep a latest-value handle.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/playground/src/hooks/use-realtime-call.ts` around lines 121 - 124, Update the ref synchronization in the realtime call hook around voiceRef and onCallErrorRef to avoid mutating ref.current during render. Use React 19.2’s useEffectEvent, or synchronize the latest values inside an effect, while preserving access to the current voice and onCallError values for call callbacks.apps/playground/src/components/playground/realtime-sidebar.tsx (1)
155-265: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy liftSidebar nav is now copied across four playground sidebars.
RealtimeSidebarduplicates the header/nav/footer ofvideo-sidebar.tsxand friends almost verbatim; each new mode requires editing every file (as this PR does for four of them). Extracting the shared nav list + user dropdown into one component would make future modes a single-line change.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/playground/src/components/playground/realtime-sidebar.tsx` around lines 155 - 265, Extract the duplicated SidebarHeader navigation and shared user dropdown from RealtimeSidebar and the other playground sidebar components into a reusable shared component. Parameterize only the mode-specific route or active-state details, then replace each sidebar’s duplicated markup with the shared component so adding a new mode requires updating one navigation definition while preserving existing links, active states, and layout.apps/playground/src/components/model-selector.tsx (1)
2656-2722: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winModel Details dialog is missing the new Audio In/Audio Out pricing for realtime mode.
The preview panel now shows "Audio In"/"Audio Out" via
MappingPriceCellwhenmode === "realtime"(lines 2134-2147), but the "Model Details" dialog's own Pricing & Limits grid has no equivalent branch, so clicking through to full details on a realtime model shows no audio pricing at all.♻️ Proposed fix — mirror the preview-panel addition inside the dialog
</div> </> )} + {mode === "realtime" && ( + <> + <MappingPriceCell + label="Audio In" + mapping={selectedDetails.mapping} + field="inputAudio" + /> + <MappingPriceCell + label="Audio Out" + mapping={selectedDetails.mapping} + field="outputAudio" + /> + </> + )}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/playground/src/components/model-selector.tsx` around lines 2656 - 2722, Update the Pricing & Limits grid in the Model Details dialog to render Audio In and Audio Out pricing for realtime models, mirroring the preview panel’s MappingPriceCell behavior and existing price formatting. Add this realtime branch alongside the current Input/Output rendering, using the selected mapping and the established pricing helpers.apps/gateway/src/realtime/client-secrets-route.ts (1)
146-170: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueConsider pinning the canonical
match.modelIdinstead of the raw requested string.
transcriptionModelstores the caller-supplied name (alias or upstreamexternalId), so the upgrade path must re-resolve it. Persistingmatch.modelIdmakes the pin unambiguous for billing.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/realtime/client-secrets-route.ts` around lines 146 - 170, Update the transcription handling in the requestedTranscription block to assign the canonical match.modelId to transcriptionModel after validation, rather than retaining requestedTranscription.model. Preserve the existing mapping and allowlist checks.apps/gateway/src/realtime/preflight.ts (1)
239-251: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick winSequential IAM validation per transcription mapping adds connect latency.
Each candidate triggers an awaited validation in series on the connection path. Run them concurrently.
♻️ Suggested refactor
- const allowedTranscriptionModelIds: string[] = []; - for (const candidate of listRealtimeTranscriptionMappings(providerId)) { - const validation = await validateRequestModelAccess( - apiKey, - candidate.modelId, - providerId, - candidate.modelDef, - input.clientIp, - ); - if (validation.allowed) { - allowedTranscriptionModelIds.push(candidate.modelId); - } - } + const transcriptionCandidates = listRealtimeTranscriptionMappings(providerId); + const validations = await Promise.all( + transcriptionCandidates.map((candidate) => + validateRequestModelAccess( + apiKey, + candidate.modelId, + providerId, + candidate.modelDef, + input.clientIp, + ), + ), + ); + const allowedTranscriptionModelIds = transcriptionCandidates + .filter((_, i) => validations[i].allowed) + .map((candidate) => candidate.modelId);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/realtime/preflight.ts` around lines 239 - 251, Update the transcription mapping validation loop in the preflight flow to validate all candidates concurrently rather than awaiting each validateRequestModelAccess call serially. Use Promise.all or equivalent, then collect only allowed candidate.modelId values into allowedTranscriptionModelIds while preserving the existing validation arguments and filtering behavior.infra/helm/llmgateway/values.yaml (1)
159-173: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winConcurrency caps are per-org/per-key only — no per-pod ceiling behind these resource limits.
With 512Mi/500m limits and audio buffering per session, aggregate load across many orgs on one replica can OOM or saturate the pod while every individual lease stays within its cap. Consider a per-instance max-sessions setting (and HPA on session count, as the header comment suggests) so the pod sheds load before hitting limits.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@infra/helm/llmgateway/values.yaml` around lines 159 - 173, Add a per-instance maximum sessions setting to the llmgateway config alongside maxSessionsPerOrg and maxSessionsPerKey, and enforce it in the session admission/lease path so aggregate sessions on one pod cannot exceed the configured ceiling. Wire session-count metrics into the HPA so scaling responds to active sessions, preserving the existing per-org and per-key caps.apps/gateway/src/realtime/catalog.ts (1)
148-173: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDuplicate traversal: derive the default from
listRealtimeTranscriptionMappings.♻️ Suggested simplification
export function findDefaultRealtimeTranscriptionMapping( providerId: string, now: Date = new Date(), ): RealtimeMappingMatch | null { - for (const rawModel of modelDefinitions) { - const model = rawModel as ModelDefinition; - for (const mapping of model.providers) { - const candidate = mapping as ProviderModelMapping; - if (candidate.realtimeTranscription !== true) { - continue; - } - if (candidate.providerId !== providerId) { - continue; - } - if (candidate.deactivatedAt && now > candidate.deactivatedAt) { - continue; - } - return { - modelDef: model, - mapping: candidate, - modelId: model.id, - }; - } - } - return null; + return listRealtimeTranscriptionMappings(providerId, now)[0] ?? null; }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/gateway/src/realtime/catalog.ts` around lines 148 - 173, Update findDefaultRealtimeTranscriptionMapping to obtain candidates through listRealtimeTranscriptionMappings instead of independently traversing modelDefinitions and provider mappings. Reuse that helper’s filtering and mapping behavior, then select and return the first matching mapping for providerId while preserving the now-based deactivation handling and null result when no match exists.infra/helm/llmgateway/templates/ingress.yaml (1)
66-77: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winAdd default proxy/read timeout annotations for the WebSocket path.
The comment correctly notes the controller's idle timeout must exceed
realtime.config.maxSessionSeconds(3600s default), but nothing in the chart sets it — typical nginx defaults (60s) will sever sessions unless operators addnginx.ingress.kubernetes.io/proxy-read-timeout/proxy-send-timeoutthemselves. Consider shipping controller-appropriate defaults or documenting this invalues.yaml.Also note that conformant ingress controllers select paths by specificity (Exact before Prefix), not declaration order, so the ordering rationale in the comment above is stricter than required.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@infra/helm/llmgateway/templates/ingress.yaml` around lines 66 - 77, Add nginx ingress proxy-read-timeout and proxy-send-timeout annotations for the /v1/realtime WebSocket route, using defaults greater than realtime.config.maxSessionSeconds (including the 3600-second default). Place the settings in the chart’s ingress annotation configuration and update the nearby comment to avoid claiming declaration order controls path selection.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/gateway/src/realtime/billing.ts`:
- Around line 226-232: Update the insert using onConflictDoNothing near the
billing log flow to target only the idempotency unique index on
(realtimeSessionId, realtimeUsageKey), so unrelated constraint violations
propagate instead of being treated as duplicates. Apply the same targeted
conflict configuration to the transcription insert path.
In `@apps/gateway/src/realtime/client-secrets.ts`:
- Around line 93-111: The RealtimeClientSecretRecord currently persists
input.token in plaintext; change the client-secret storage flow around
RealtimeClientSecretRecord and redisClient.set to store only the API key
identifier or fingerprint, then resolve the credential server-side during the
upgrade flow. If re-resolution is unavailable, encrypt the token with the
established service key before JSON serialization, and ensure consumers decrypt
or resolve it without changing the ephemeral-secret behavior.
In `@apps/gateway/src/realtime/preflight.ts`:
- Around line 220-226: Update the denied-authorization branch around
throwIamException so it explicitly rejects after invoking the helper, even if
throwIamException returns normally. Preserve the existing toConnectError
conversion for thrown errors, and ensure the !iamValidation.allowed path cannot
continue into credential resolution or session creation.
In `@apps/gateway/src/realtime/serve.ts`:
- Around line 89-101: Update gracefulShutdown to accept and use an exit-code
parameter, keeping SIGTERM and SIGINT as clean exits while passing a nonzero
code from the uncaughtException and unhandledRejection handlers. In the
unhandledRejection handler, convert reason with toError before calling
logger.fatal to match the logger error contract.
In `@apps/gateway/src/realtime/server.ts`:
- Around line 527-581: Gate the outer catch’s writeHttpError fallback on the
upgraded state tracked in the upgrade handler. When RealtimeProxySession
construction or post-upgrade setup fails after upgraded becomes true, log the
error but do not write an HTTP 500; only send the HTTP error when the WebSocket
upgrade was not completed.
In `@apps/gateway/src/realtime/session.ts`:
- Around line 316-339: Serialize per-connection client-message processing so
asynchronous handling in handleClientMessage, especially the response.create
gate chain, cannot be overtaken by later frames. Add a promise chain or
equivalent queue at the session connection level and route each void
this.handleClientMessage(...) dispatch through it, preserving arrival order for
response.create, conversation.item.create, and default forwarding while
retaining existing error handling.
In `@apps/gateway/src/serve.ts`:
- Around line 135-147: Update the realtime shutdown flow around
realtime.closeAll() in the inline server entrypoint to wait for the same
2-second billing/session flush window used by the dedicated realtime entrypoint
before closeDatabase() can run. Preserve the existing force-close behavior and
ensure the delay occurs only when realtime is enabled.
In `@apps/playground/src/lib/realtime-audio.spec.ts`:
- Around line 22-27: Update the test named “interpolates between neighboring
samples” to use resampling parameters and input data that produce a fractional
source index, then assert the expected blended sample value from the neighboring
inputs. Replace the current 48k-to-24k [0, 1] case, which only samples index 0,
while preserving coverage of resampleLinear’s interpolation behavior.
In `@apps/playground/src/lib/realtime-audio.ts`:
- Around line 226-246: Update the realtime audio stop function to stop all
tracks from the associated MediaStream when stopping, in addition to
disconnecting the audio nodes. Use the existing stream reference and invoke
track.stop() for each track so microphone capture and the browser’s mic-in-use
indicator are released.
In `@infra/helm/llmgateway/templates/realtime-pdb.yaml`:
- Line 9: Update the pod disruption budget template’s realtime maxUnavailable
configuration to allow at least one voluntary eviction instead of defaulting to
zero. Preserve the existing Values-based override while changing the fallback
used by realtime.podDisruptionBudget.maxUnavailable, and ensure the realtime
deployment continues using its graceful termination path during eviction.
In `@packages/db/migrations/1784649163_redundant_eternals.sql`:
- Line 32: Change the index statement in
packages/db/migrations/1784649163_redundant_eternals.sql at lines 32-32 to use
CREATE UNIQUE INDEX CONCURRENTLY, and wrap this build in its own transaction
boundary. Make the corresponding CREATE INDEX CONCURRENTLY and
transaction-boundary change in
packages/db/migrations/1784973019_fair_stardust.sql at lines 1-1; modify only
these generated SQL migrations and leave snapshot/journal files untouched.
---
Outside diff comments:
In `@apps/playground/src/components/model-selector.tsx`:
- Around line 2072-2147: Update the output label in the pricing grid to display
“Text Out” when mode === "realtime", matching the existing conditional “Text In”
label; retain “Output” for all other modes and leave the separate audio pricing
cells unchanged.
---
Nitpick comments:
In `@apps/gateway/src/realtime/catalog.ts`:
- Around line 148-173: Update findDefaultRealtimeTranscriptionMapping to obtain
candidates through listRealtimeTranscriptionMappings instead of independently
traversing modelDefinitions and provider mappings. Reuse that helper’s filtering
and mapping behavior, then select and return the first matching mapping for
providerId while preserving the now-based deactivation handling and null result
when no match exists.
In `@apps/gateway/src/realtime/client-secrets-route.ts`:
- Around line 146-170: Update the transcription handling in the
requestedTranscription block to assign the canonical match.modelId to
transcriptionModel after validation, rather than retaining
requestedTranscription.model. Preserve the existing mapping and allowlist
checks.
In `@apps/gateway/src/realtime/preflight.ts`:
- Around line 239-251: Update the transcription mapping validation loop in the
preflight flow to validate all candidates concurrently rather than awaiting each
validateRequestModelAccess call serially. Use Promise.all or equivalent, then
collect only allowed candidate.modelId values into allowedTranscriptionModelIds
while preserving the existing validation arguments and filtering behavior.
In `@apps/gateway/src/realtime/pricing.spec.ts`:
- Around line 128-264: The realtime pricing tests lack coverage for non-zero
requestPrice rejection and applyRealtimeDiscount. In
apps/gateway/src/realtime/pricing.spec.ts lines 128-264, add a
priceRealtimeUsage case using a snapshot with non-zero requestPrice and assert
ok is false, plus a case covering applyRealtimeDiscount. In
apps/gateway/src/realtime/transcription.spec.ts lines 121-189, add the
equivalent priceTranscriptionUsage case asserting ok is false for a snapshot
with non-zero requestPrice.
In `@apps/gateway/src/realtime/session.spec.ts`:
- Line 154: Type the preflightOverrides fixture in buildPreflight as a
Partial<RealtimePreflightResult> (or the repository’s equivalent typed
override), removing the broad Record<string, unknown> and unknown cast. Ensure
keys such as allowedTranscriptionModelIds are validated against
RealtimePreflightResult so mistyped overrides fail compilation while existing
fixture defaults remain intact.
In `@apps/gateway/src/realtime/transcription.ts`:
- Around line 24-37: Move the duplicated readCount and priceComponent helpers
into a shared module, then update both transcription.ts and pricing.ts to import
and reuse those shared implementations. Preserve their existing validation and
zero-token behavior, and remove the local duplicate definitions from both files.
In `@apps/playground/src/app/api/realtime/session/route.ts`:
- Around line 105-117: Extract the shared gateway base URL resolution, including
GATEWAY_URL normalization and development/production fallbacks, into a single
helper. Update resolveRealtimeWsUrl and the gatewayBaseUrl block to reuse that
helper, preserving the existing WebSocket protocol conversion and endpoint
suffix.
In `@apps/playground/src/components/model-selector.tsx`:
- Around line 2656-2722: Update the Pricing & Limits grid in the Model Details
dialog to render Audio In and Audio Out pricing for realtime models, mirroring
the preview panel’s MappingPriceCell behavior and existing price formatting. Add
this realtime branch alongside the current Input/Output rendering, using the
selected mapping and the established pricing helpers.
In `@apps/playground/src/components/playground/realtime-page-client.tsx`:
- Around line 181-182: Update the showAuthDialog assignment in the realtime page
client to use the direct condition !isUserLoading && !user, removing the
redundant !isAuthenticated check while preserving the existing behavior.
- Around line 297-315: Add aria-live="polite" to the status span containing
STATUS_LABELS[status] in the realtime status display, ensuring Connecting, Live,
and Ending transitions are announced without changing the existing visual
behavior or timer rendering.
In `@apps/playground/src/components/playground/realtime-sidebar.tsx`:
- Around line 155-265: Extract the duplicated SidebarHeader navigation and
shared user dropdown from RealtimeSidebar and the other playground sidebar
components into a reusable shared component. Parameterize only the mode-specific
route or active-state details, then replace each sidebar’s duplicated markup
with the shared component so adding a new mode requires updating one navigation
definition while preserving existing links, active states, and layout.
In `@apps/playground/src/hooks/use-realtime-call.ts`:
- Around line 121-124: Update the ref synchronization in the realtime call hook
around voiceRef and onCallErrorRef to avoid mutating ref.current during render.
Use React 19.2’s useEffectEvent, or synchronize the latest values inside an
effect, while preserving access to the current voice and onCallError values for
call callbacks.
In `@apps/worker/src/worker.ts`:
- Around line 1036-1046: Update the structured `log-process` record near the
existing `row.cost` field to include the exact billing amount from
`effectiveCost` (or the underlying `billing_cost` value), while preserving the
fallback behavior for legacy costs. Ensure decimal-billed realtime rows record
the amount actually debited rather than only `row.cost`.
In `@infra/helm/llmgateway/templates/ingress.yaml`:
- Around line 66-77: Add nginx ingress proxy-read-timeout and proxy-send-timeout
annotations for the /v1/realtime WebSocket route, using defaults greater than
realtime.config.maxSessionSeconds (including the 3600-second default). Place the
settings in the chart’s ingress annotation configuration and update the nearby
comment to avoid claiming declaration order controls path selection.
In `@infra/helm/llmgateway/values.yaml`:
- Around line 159-173: Add a per-instance maximum sessions setting to the
llmgateway config alongside maxSessionsPerOrg and maxSessionsPerKey, and enforce
it in the session admission/lease path so aggregate sessions on one pod cannot
exceed the configured ceiling. Wire session-count metrics into the HPA so
scaling responds to active sessions, preserving the existing per-org and per-key
caps.
In `@packages/db/migrations/meta/1784973019_snapshot.json`:
- Around line 17279-17341: Update the realtime_session schema definition in
packages/db/src/schema.ts so bytes_in and bytes_out use bigint instead of 32-bit
integer, preserving their existing non-null and zero-default behavior. Then
regenerate the migration artifacts with pnpm migrations; do not edit the
generated snapshot directly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 859577af-602f-4db3-848f-f383fc2f96d1
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (70)
.env.example.gitignoreapps/api/src/routes/activity.tsapps/api/src/routes/internal-models.tsapps/api/src/testing.tsapps/gateway/package.jsonapps/gateway/src/app.tsapps/gateway/src/chat/chat.tsapps/gateway/src/lib/client-ip.tsapps/gateway/src/models/models.tsapps/gateway/src/realtime/billing.tsapps/gateway/src/realtime/catalog.spec.tsapps/gateway/src/realtime/catalog.tsapps/gateway/src/realtime/client-secrets-route.tsapps/gateway/src/realtime/client-secrets.spec.tsapps/gateway/src/realtime/client-secrets.tsapps/gateway/src/realtime/errors.tsapps/gateway/src/realtime/leases.tsapps/gateway/src/realtime/preflight.tsapps/gateway/src/realtime/pricing.spec.tsapps/gateway/src/realtime/pricing.tsapps/gateway/src/realtime/serve.tsapps/gateway/src/realtime/server.tsapps/gateway/src/realtime/session.spec.tsapps/gateway/src/realtime/session.tsapps/gateway/src/realtime/transcription.spec.tsapps/gateway/src/realtime/transcription.tsapps/gateway/src/serve.tsapps/playground/public/worklets/pcm-recorder.jsapps/playground/src/app/api/realtime/session/route.tsapps/playground/src/app/realtime/page.tsxapps/playground/src/components/model-selector.tsxapps/playground/src/components/playground/audio-sidebar.tsxapps/playground/src/components/playground/canvas-sidebar.tsxapps/playground/src/components/playground/chat-sidebar.tsxapps/playground/src/components/playground/image-sidebar.tsxapps/playground/src/components/playground/org-sidebar.tsxapps/playground/src/components/playground/projects-sidebar.tsxapps/playground/src/components/playground/realtime-page-client.tsxapps/playground/src/components/playground/realtime-sidebar.tsxapps/playground/src/components/playground/skills-sidebar.tsxapps/playground/src/components/playground/video-sidebar.tsxapps/playground/src/hooks/use-realtime-call.tsapps/playground/src/lib/fetch-models.tsapps/playground/src/lib/model-preferences.tsapps/playground/src/lib/realtime-audio.spec.tsapps/playground/src/lib/realtime-audio.tsapps/playground/src/lib/video-gen.spec.tsapps/ui/src/types/activity.tsapps/worker/src/services/global-stats-aggregator.tsapps/worker/src/services/project-stats-aggregator.tsapps/worker/src/worker.tsinfra/helm/llmgateway/templates/configmap.yamlinfra/helm/llmgateway/templates/ingress.yamlinfra/helm/llmgateway/templates/realtime-deployment.yamlinfra/helm/llmgateway/templates/realtime-pdb.yamlinfra/helm/llmgateway/templates/realtime-service.yamlinfra/helm/llmgateway/values.yamlpackages/db/migrations/1784649163_redundant_eternals.sqlpackages/db/migrations/1784973019_fair_stardust.sqlpackages/db/migrations/1784980032_chief_obadiah_stane.sqlpackages/db/migrations/meta/1784649163_snapshot.jsonpackages/db/migrations/meta/1784973019_snapshot.jsonpackages/db/migrations/meta/1784980032_snapshot.jsonpackages/db/migrations/meta/_journal.jsonpackages/db/src/schema.tspackages/db/src/seed.tspackages/models/src/models.tspackages/models/src/models/openai.tspackages/models/src/realtime-models.spec.ts
Resolve conflicts from merging upstream/main: - packages/db/migrations: reset to upstream/main and regenerated a single migration (1784984339_wealthy_swordsman), replacing the three branch migrations that tripped the "at most one new migration" check. - apps/gateway/src/app.ts: keep both /audio/transcriptions (main) and /realtime (branch) routes. - packages/models/src/models.ts: keep both the realtime / realtimeTranscription flags (branch) and transcriptions (main). - pnpm-lock.yaml: regenerated via pnpm install. Adapt the realtime call sites to main's validateRequestModelAccess refactor, which switched to a single object param and now requires organizationId. Co-Authored-By: Claude <noreply@anthropic.com>
Surface who has the floor during a live call: a mic level meter driven by RMS of the captured chunks, the server VAD speech_started/stopped signal for "listening", and an analyser on the playback chain for the assistant's own output level. Both meters are sampled on a single 80ms timer with an epsilon on the state writes, so a live call costs a bounded number of renders per second instead of one per microphone chunk. Co-Authored-By: Claude <noreply@anthropic.com>
RATCHAW
marked this pull request as draft
July 25, 2026 14:48
Persist the transcript of playground voice calls so past calls can be reopened from the sidebar. The gateway deliberately does not store realtime conversation content, so this saves what the browser already assembled, following the same per-studio history pattern as the image, audio and video playgrounds. Adds playground_realtime_history, five CRUD routes under /playground, and a date-grouped call list in the realtime sidebar. The list endpoint counts turns in SQL so transcripts are only fetched when a call is opened. The earlier realtime migration is regenerated into a single file to satisfy the one-new-migration CI check; the SQL is unchanged apart from the added table. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
RATCHAW
marked this pull request as ready for review
July 25, 2026 15:40
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc2d461319
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/playground/src/components/playground/call-history-list.tsx`:
- Around line 352-358: Update handleDeleteItem so onItemDeleted is invoked only
after deleteItem succeeds, using the mutation’s success callback rather than
calling it immediately after mutate. Preserve passing the same id and avoid
notifying the parent when the deletion request fails.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 4245ea92-6e7b-454f-81db-6233cebed8e2
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (17)
apps/api/src/routes/playground.tsapps/gateway/package.jsonapps/gateway/src/chat/chat.tsapps/gateway/src/serve.tsapps/playground/src/components/playground/call-history-list.tsxapps/playground/src/components/playground/realtime-page-client.tsxapps/playground/src/components/playground/realtime-sidebar.tsxapps/playground/src/hooks/use-realtime-call.tsapps/playground/src/hooks/usePlaygroundHistory.tsapps/playground/src/lib/call-history.spec.tsapps/playground/src/lib/call-history.tsapps/playground/src/lib/realtime-audio.spec.tsapps/playground/src/lib/realtime-audio.tspackages/db/migrations/1784991061_graceful_cyclops.sqlpackages/db/migrations/meta/1784991061_snapshot.jsonpackages/db/migrations/meta/_journal.jsonpackages/db/src/schema.ts
💤 Files with no reviewable changes (2)
- apps/playground/src/lib/realtime-audio.spec.ts
- apps/playground/src/lib/realtime-audio.ts
🚧 Files skipped from review as they are similar to previous changes (4)
- apps/gateway/package.json
- apps/gateway/src/serve.ts
- apps/playground/src/components/playground/realtime-page-client.tsx
- apps/gateway/src/chat/chat.ts
Address review findings on the realtime session proxy: - Re-run the full account authorization gates after each billed transcription. A transcription-only session that never sends response.create previously only re-checked the session spend cap and (in credits mode) the org balance, so a revoked or budget-exhausted key could keep producing billable transcriptions. The checks are now shared with the per-response gate via authorizeAccount(). - Scope the unsettled realtime spend the credit gate subtracts to the organization instead of the current session, so concurrent sessions cannot each authorize work against the same undebited balance. - Commit session.update transcription state only after every validation passes, so an update rejected for a hosted tool or stored prompt no longer pins or enables transcription the provider never applied. - Reject realtimeTranscription mappings from /v1/chat/completions: they declare text output but are only served over /v1/realtime. - Notify the playground call-history parent from the delete mutation's onSuccess rather than unconditionally after mutate(). Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/db/migrations/1784996550_mysterious_kinsey_walden.sql`:
- Line 54: Update the CREATE INDEX statement for
log_realtime_unsettled_organization_id_idx to use concurrent index creation,
matching the existing concurrently built indexes in this migration. Modify only
this generated SQL statement and preserve its partial-index predicate.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: f3ced99d-8c8c-4254-b7fe-748d67c3f2df
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (12)
.env.exampleapps/gateway/src/chat/chat.tsapps/gateway/src/realtime/billing.tsapps/gateway/src/realtime/session.spec.tsapps/gateway/src/realtime/session.tsapps/playground/src/components/playground/call-history-list.tsxinfra/helm/llmgateway/values.yamlpackages/db/migrations/1784996550_mysterious_kinsey_walden.sqlpackages/db/migrations/meta/1784996550_snapshot.jsonpackages/db/migrations/meta/_journal.jsonpackages/db/src/schema.tspackages/models/src/models/openai.ts
🚧 Files skipped from review as they are similar to previous changes (8)
- apps/gateway/src/chat/chat.ts
- packages/models/src/models/openai.ts
- packages/db/migrations/meta/_journal.json
- .env.example
- apps/playground/src/components/playground/call-history-list.tsx
- infra/helm/llmgateway/values.yaml
- packages/db/src/schema.ts
- apps/gateway/src/realtime/session.ts
Shutdown force-closed every active realtime session, cutting calls off mid-conversation on each rollout. Stop accepting new sessions, then wait for live ones to hang up before force-closing stragglers past the deadline. Host realtime in the gateway process rather than a dedicated deployment: a separate deployment needs the same drain-before-shutdown behaviour anyway, so it buys isolation at the cost of another service to operate. Drop the realtime deployment, service and PDB along with the standalone entrypoint, and route /v1/realtime through the gateway's existing ingress catch-all. The gateway's terminationGracePeriodSeconds was hardcoded to 120, which would have SIGKILLed the pod long before the drain finished. Derive it from the session cap when realtime is enabled, overridable via gateway.terminationGracePeriodSeconds. Co-Authored-By: Claude <noreply@anthropic.com>
…to RATCHAW/plan-openai-realtime
Handle ws unexpected-response with its real (req, res) signature: destroy the request and absorb late socket errors so a rejected upstream handshake cannot leak the connection or crash the pod via an unhandled error event, and report the actual status code. Re-check `accepting` right before handleUpgrade so a session cannot establish after the shutdown drain loop saw zero sessions and moved on to closing the database. Flush writeHttpError responses with end() + destroy-on-finish so rejected upgrades see the crafted error instead of a reset. Claude-Session: https://claude.ai/code/session_019nTUJXdPQwXc6JYqW92DBM
The transcript save only ran on the in-component idle transition, so navigating to another page during a live call unmounted the component and silently discarded the transcript (the browser is the only place it exists). Persist from the unmount cleanup via a latest-ref; the mutation and its cache updates outlive the component. Claude-Session: https://claude.ai/code/session_019nTUJXdPQwXc6JYqW92DBM
The realtime sidebar headers were copied from a pre-rebrand sidebar: Logo + "LLM Gateway" text and a "Chat" badge in the signed-in state. Use the Lounge Wordmark like every other sidebar, with the Voice badge on the loading/signed-out states. Claude-Session: https://claude.ai/code/session_019nTUJXdPQwXc6JYqW92DBM
Realtime log rows carry audioOutputTokens/audioOutputCost — the dominant cost component of a voice response — but the logs response schema only exposed the input-side audio columns, so the dashboard breakdown could not account for most of the row's cost. Claude-Session: https://claude.ai/code/session_019nTUJXdPQwXc6JYqW92DBM
!apps/*/public/**/*.js surfaced every stray generated .js under any app's public dir as untracked (e.g. a local theme-init.js), inviting accidental commits. Un-ignore only the playground worklets dir, matching the existing targeted exceptions. Claude-Session: https://claude.ai/code/session_019nTUJXdPQwXc6JYqW92DBM
Member
|
Reviewed the full diff (gateway realtime service, billing path, playground voice UI, DB schema/migration, worker aggregation, models catalogue, helm). Overall this is a very solid PR — the billing path fails closed everywhere it should, the idempotency arbiter matches the partial unique index exactly, the preflight/lease/drain lifecycle is careful, and test coverage on the money path is thorough. Pushed 5 fix commits for the issues found: Fixed in follow-up commits
Verified clean (highlights)
🤖 Generated with Claude Code |
…ealtime # Conflicts: # apps/playground/src/components/playground/audio-sidebar.tsx # apps/playground/src/components/playground/canvas-sidebar.tsx # apps/playground/src/components/playground/chat-sidebar.tsx # apps/playground/src/components/playground/image-sidebar.tsx # apps/playground/src/components/playground/org-sidebar.tsx # apps/playground/src/components/playground/projects-sidebar.tsx # apps/playground/src/components/playground/skills-sidebar.tsx # apps/playground/src/components/playground/video-sidebar.tsx # packages/db/src/schema.ts
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…ts' into RATCHAW/plan-openai-realtime # Conflicts: # packages/db/migrations/1785100568_swift_whistler.sql # packages/db/migrations/meta/1785100568_snapshot.json # packages/db/migrations/meta/_journal.json
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
steebchen
added a commit
that referenced
this pull request
Jul 26, 2026
This reverts commit 8368470.
steebchen
added a commit
that referenced
this pull request
Jul 27, 2026
Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: smakosh <ismai23l@hotmail.com> Co-authored-by: Luca Steeb <contact@luca-steeb.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a dedicated
/v1/realtimeWebSocket proxy service that fronts OpenAI's realtime API: preflight auth/provider resolution, session model pinning, gated VAD response mediation, per-response exact-decimal billing with discounts, credit/API-key/member-budget/rate-limit gates, atomic Redis concurrency leases, backpressure, drain-on-disconnect, ephemeral client secrets for browser clients, and a kill switch. Input-audio transcription is metered per completed event against an ASR mapping pinned for the session, so a model swap cannot reprice work already committed.The catalogue gains a
realtimecapability flag with thegpt-realtime-2.1family and modality-specific audio prices exposed via/v1/models, while the DB gains arealtime_sessiontable, exactbillingCostand audio-token log columns, an idempotent (session, usage key) partial unique index, andaudioInputCost/audioOutputCostplumbed through all seven stats tables and the aggregators so analytics breakdowns reconcile against session totals. The playground ships a voice page at/realtimewith an AudioWorklet PCM recorder, and infra adds a dedicated Helm deployment/service/PDB plus/v1/realtimeingress routing with a long termination grace (the gateway can also host realtime inline viaREALTIME_INLINEfor local dev).🤖 Generated with Claude Code
Summary by CodeRabbit