[codex] add responses websocket relay

[AAirLin]

Adds OpenAI Responses WebSocket relay support so the gateway can accept client WS responses traffic and relay it to upstream channels using the gateway's own upstream credentials.

Validated with targeted Go tests for middleware, service, relay, and channel packages.

Summary by CodeRabbit

• New Features

  • Added WebSocket streaming support for responses with event-based communication
  • Enhanced token usage tracking with detailed output token metrics

• Improvements

  • Optimized rate limiting for WebSocket connections with pre-check validation
  • Improved error handling and retry logic for relay operations
  • Refined WebSocket authentication with subprotocol-based authorization

• Tests

  • Added comprehensive WebSocket event handling test coverage
  • Added authentication and rate limiting validation tests

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a5e737b7-f063-4991-815a-f7c7cda33ca7

📥 Commits

Reviewing files that changed from the base of the PR and between 74985fa and ef2537d.

📒 Files selected for processing (15)

• common/json.go
• common/rate-limit.go
• controller/relay.go
• dto/openai_response.go
• middleware/auth.go
• middleware/auth_test.go
• middleware/model-rate-limit.go
• relay/channel/openai/relay_responses.go
• relay/responses_websocket.go
• relay/responses_websocket_test.go
• router/relay-router.go
• service/relay_error.go
• service/relay_error_test.go
• service/responses_usage.go
• service/responses_usage_test.go

---

Walkthrough

This PR introduces complete OpenAI "responses" WebSocket streaming support by adding authentication, rate limiting, a WebSocket relay handler, token tracking, error handling, and HTTP integration across the relay stack.

Changes

OpenAI Responses WebSocket Streaming

Layer / File(s)	Summary
Type aliases and rate-limit check common/json.go, common/rate-limit.go	RawMessage type alias wraps json.RawMessage for cleaner function signatures; InMemoryRateLimiter.Check evaluates allowance without recording state, enabling pre-flight validation.
WebSocket subprotocol authentication middleware/auth.go, middleware/auth_test.go	New helpers extract API key from Sec-WebSocket-Protocol: openai-insecure-api-key.<key> marker and apply Bearer token authorization; tests verify marker parsing, rejection of invalid forms, and header override behavior.
Model rate-limit pre-check middleware middleware/model-rate-limit.go	Refactored rate limiting into reusable CheckModelRequestRateLimit function returning a deferred success-recording callback; supports Redis and in-memory backends, group-based overrides, and bypasses rate limiting for /v1/responses WebSocket handshakes.
Response usage struct dto/openai_response.go	Extends Usage struct with optional OutputTokensDetails field to carry granular output token breakdowns in API responses.
Token usage merging utility service/responses_usage.go, service/responses_usage_test.go	Implements ApplyResponsesUsage to safely merge token counts and detailed breakdowns from source to destination, with fallback handling for reasoning tokens; tests verify multi-field copying and completion-token detail fallback.
Relay error handling service/relay_error.go, service/relay_error_test.go	Introduces ShouldRetryRelayError (retry gating via error classification and context-aware rules) and ProcessChannelError (logging, async disabling, error-log recording); test validates specific-channel override behavior.
WebSocket relay core implementation relay/responses_websocket.go, relay/responses_websocket_test.go	Complete ResponsesWebSocketHelper handler: reads client events, normalizes response.create, validates model/access, enforces single in-flight request, selects upstream channel with pinned/auto modes, dials upstream, streams messages bidirectionally while tracking usage/deltas/tool calls, finalizes tokens (including output-text-based fallback), posts quotas, and commits/rolls back rate limits on completion; comprehensive tests cover event normalization (wrapped/flat), payload building, error handling, transport-field removal, URL conversion, and session lifecycle.
Controller and HTTP integration controller/relay.go, router/relay-router.go, relay/channel/openai/relay_responses.go	Adds ResponsesWebSocket controller handler to upgrade HTTP and forward to relay helper; updates WebSocket subprotocol advertisement; simplifies relay logic by delegating error/retry decisions to new service helpers; registers /v1/responses endpoint; updates OpenAI response handlers to use unified ApplyResponsesUsage.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• QuantumNous/new-api#3400: Main PR's service.ApplyResponsesUsage merges UsageSemantic/UsageSource and token details into dto.Usage, which depends on the retrieved PR's dto/openai_response.go Usage struct additions.
• QuantumNous/new-api#1775: Both PRs modify controller/relay.go error/quota flow with processChannelError and channel disabling, so the relay error refactor overlaps with the retrieved PR's changes.
• QuantumNous/new-api#4409: Main PR's service.ApplyResponsesUsage generically merges dto.Usage token-detail structs, complementing the retrieved PR's addition of OutputTokenDetails.image_tokens.

Suggested reviewers

• seefs001
• creamlike1024

Poem

🐰 A WebSocket whisper through the streams so bright,
Responses flowing smooth from left to right,
With tokens tallied, channels chosen true,
Rate limits guarding all the way through—
A relay born to bridge the realtime sky! ✨

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}